Method and device for operating a windscreen display system with cover detection, and windscreen display system

Description

Technical field

[0001] The invention relates to windshield display systems, in particular reflective display systems, such as PHUD, for motor vehicles. In particular, the present invention relates to measures for detecting a foreign object located in the optical beam path of the windshield display system. Technical background

[0002] Reflective display systems, such as PHUD, comprise a display unit mounted on top of the instrument panel. The display is reflected on the inside of the windshield, allowing the reflected image to be perceived by a vehicle occupant. The display unit is positioned slightly recessed on top of the instrument panel, preventing direct view of the display surface and thus preventing glare from light directly hitting the user's eyes.

[0003] However, due to the placement of the display unit in such a recess on the top of the instrument panel, foreign objects resting on the display unit are not readily visible from the normal visual position of the vehicle occupants. However, these foreign objects lie in the optical path of the display image and can thus obscure parts of the display, so that legally relevant display symbols, such as the speedometer, chamber lights, remaining range, and the like, are not visible. The absence of these display symbols is also not noticeable, as they are not permanently displayed.

[0004] The use of a cover glass, as used, for example, in classic head-up displays, is disadvantageous for such reflective display devices, as it can result in new reflections of ambient light onto the windshield, which can significantly impair the perceptibility of the reflected image in high ambient brightness.

[0005] US 2014 / 333647 A1 discloses a head-up display system for a vehicle, the display system comprising a transparent display disposed between a driver's seat and a vehicle windshield; a display mode setting unit configured to set a display mode of the transparent display as at least one of a first mode and a second mode based at least in part on an angle formed between the transparent display and a vehicle body; and an image setting unit for adjusting the image displayed on the transparent display according to the display mode.

[0006] US 2019 / 271905 A1 discloses a projection display device having a projection display unit including a light modulation unit that spatially modulates light emitted from a light source based on input image information and projects image light obtained as a result of the spatial modulation onto a projection surface of a vehicle to display an image based on the image light. A cover covers an opening portion through which the image light exits. Using an object removal mechanism, an object adhering to a front surface of the cover is removed by scratching the front surface of the cover.An object removal control unit is provided which activates an operation of the object removal mechanism when a predetermined display content is included in a part of the image light, the part being blocked due to the object attachment area detected by the area detection unit.

[0007] Document WO 2020 / 189646 A1 discloses a head-up display that displays a predetermined image toward the vehicle's occupants. The housing has an opening and a transparent cover covering the opening of the housing. An image generation unit that emits light to generate the predetermined image is provided within a receiving portion formed by the transparent cover, and the light emitted by the image generation unit is directed toward the windshield.

[0008] US 2019 / 212550 A1 discloses a projection display device having a light modulation unit that spatially modulates the light emitted from a light source in accordance with input image data, and a projection optical system that projects the spatially modulated light onto a projection surface of a vehicle through an opening portion of a housing. A cover closes the opening portion, and a movable partition member divides a front surface of the cover into a plurality of regions. The partition member is moved to a position on the cover so that the cover is separated into a plurality of regions when an object is detected.

[0009] It is an object of the present invention to provide an improved method for detecting occlusions of a display image in a reflective display system. Disclosure of the invention

[0010] This object is achieved by a reflection display system according to claim 1 and by a method for a detection device and a display system according to the independent claims.

[0011] Further embodiments are specified in the dependent claims.

[0012] According to a first aspect, a reflection display system for displaying a display image to a vehicle occupant of a motor vehicle by reflecting the display image on a windscreen is provided, comprising: a display unit designed to display the display image of the display surface; a detection sensor system for detecting the displayed display image on the display surface; and a control unit designed to ∘ control the display unit for display; ∘ determine detection information depending on the detected displayed display image, ∘ determine the presence of a foreign object on the display surface of the display unit depending on the displayed display image and depending on the detection information; and ∘ signal a fault in the display if the presence of a foreign object on the display surface of the display unit is detected.

[0013] Foreign objects placed on the instrument panel can completely or partially obscure components used to display information on the windshield. However, if the reflective display system is used to display safety-relevant information, it is necessary to be able to distinguish between non-display of information and obstruction of the information display by a foreign object.

[0014] Foreign objects within the meaning of this invention can be any objects that hinder the perception of a display image, such as items of clothing, papers and the like.

[0015] However, particularly when the display unit is located in a recess on top of the instrument panel, foreign objects resting on the display unit are not readily visible from the normal visual position of the vehicle occupants. However, these foreign objects lie in the optical path of the display image and can thus obscure parts of the display, making safety-relevant and legally required display symbols, such as speedometer, chamber lights, remaining range, and the like, invisible. However, the absence of these display symbols may not be readily noticeable, as they are not permanently displayed.

[0016] Using the above reflective display system, it is possible to detect whether the display surface of the display unit is free of foreign objects. This is achieved by obtaining detection information based on the detected displayed display image and determining the presence of the foreign object on the display surface of the display unit by comparing the displayed display image and the detection information.

[0017] This makes it possible to detect whether the beam path of the reflective display system is disrupted by a foreign object on the display surface. Detecting such an interruption of the display of the reflective display system is essential for signaling a warning or other countermeasures. The above reflective display system thus enables automated detection of a foreign object on the display surface of the display unit that is disrupting the display image.

[0018] Furthermore, the display unit can be arranged in the recess in such a way that the display image is not directly visible in the eye area.

[0019] Furthermore, the control unit can be designed to control the display unit to output a test image with a display property, to capture a capture image of the test image as capture information and to check the presence of the display property based on the capture image in order to signal a fault in the display if the display property is not present with respect to the capture image.

[0020] In particular, the test image may correspond to a display with different display information than the display image and / or to an image with a different display property than the display image, in particular the overall brightness of the image.

[0021] One idea behind the above-mentioned reflective display system is to use the detection sensors to evaluate certain properties of the display image. This display property is altered by a foreign object obscuring the display image, so that the presence of a foreign object can be inferred from a deviation from the expected detection of the detection sensors.

[0022] According to one embodiment, the detection sensor system may comprise an image capture sensor system configured to detect the display of the display surface directly or indirectly via one or more reflections, wherein the display of the display surface is detected indirectly via a further reflection region on the windscreen or via a reflection in the eye of a vehicle occupant.

[0023] In particular, the display property may comprise a periodic display of a test image for an imperceptible period of time on the display surface, wherein the control unit is configured to determine the display of the test image as the detection image and to determine the presence of the foreign object on the display surface by comparing the detection image with the test image.

[0024] It can be provided that the display property comprises a display of a test image in a non-visible light, wherein the control unit is designed to determine the display of the test image in the non-visible light as the detection image and to determine the presence of the foreign object on the display surface by comparing the detection image with the test image.

[0025] Furthermore, the detection sensor system may comprise a brightness sensor which is designed to detect an overall brightness of the display image directly or indirectly via one or more reflections as detection brightness, wherein the control unit is designed to determine the presence of the foreign object on the display surface depending on the detection brightness and a reference brightness.

[0026] In particular, the reference brightness can be determined as the total brightness of the display image to be displayed.

[0027] According to a further aspect, a motor vehicle is provided, comprising: an instrument panel between a windshield of the motor vehicle and a steering column; the above reflective display system.

[0028] According to a further aspect, a method for detecting a foreign object on a display unit of a reflective display system, which causes a reflection of a display image on a windscreen into an eye area of ​​a vehicle occupant, comprises the following steps: Displaying the display image on the display surface; detecting the displayed display image on the display surface by a detection sensor; determining detection information depending on the detected displayed display image, determining the presence of a foreign object on the display surface of the display unit depending on the displayed display image and depending on the detection information; and signaling a display fault if the presence of a foreign object on the display surface of the display unit is detected. Short description of the drawing

[0029] Embodiments are explained in more detail below with reference to the attached drawings. They show: Figure 1 shows a schematic cross-sectional view of a reflection display system in a motor vehicle with a light barrier as a detection sensor; Figure 2 shows a cross-sectional view of a reflection display system with a detection sensor directed onto the display surface via a further reflection area for evaluating a display property of the displayed image; Figure 3 shows a flowchart illustrating a method for operating the reflection display system. Description of embodiments

[0030] Figures 1a and 1bshow a schematic cross-sectional view through a motor vehicle with a reflective display system 1 and a view from the perspective of the driver of the motor vehicle. The reflective display system is arranged on the upper side of an instrument panel 4 and has a display unit 2 comprising a display surface 3 for displaying a display image. The display unit 2 is arranged on the upper side of the instrument panel 4 below a windshield 5.

[0031] The display surface 3 is aligned with respect to the windscreen 5 such that a display image displayed on the display surface 3 of the display unit 2 is reflected at a lower region of the inside of the windscreen 5 and can be perceived by a vehicle occupant in an eye region B. As a result, the alignment of the display unit 2 or its display surface can preferably be substantially parallel to the vehicle's longitudinal and transverse axes or deviate from these by an angle of, for example, no more than 0-20°.

[0032] The display unit 2 can be controlled by a control unit 9 to output a desired display image to be perceived by the vehicle occupant.

[0033] The display unit 2 can be mounted in a recess 6 of the instrument panel so that the vehicle occupant cannot directly view the display surface 3 of the display unit 2 and dazzling of the vehicle occupants by direct light from the display surface 3 into the eye area B is avoided. Furthermore, the display surface 3 of the display unit 2 is aligned such that a displayed image is reflected on the inside of the windscreen 5 and can be perceived by a vehicle occupant in an eye area B as a reflection image in a reflection area R on the windscreen 5.

[0034] The display unit 2 can be designed as a light-emitting display unit 2 and can preferably be designed as an LCD display unit with a high-brightness backlight or as a micro-LED display unit in order to provide a bright display image so that the generated reflection image can be perceived over the corresponding area of ​​the front screen 5 even in high ambient brightness.

[0035] Since the display unit 2 is arranged in the recess 6 on the upper side of the instrument panel 4, a foreign object 7 can also enter this recess and thus come to rest on the display surface 3 of the display unit 2. A display image displayed on the display unit 2 can then not be perceived or can only be partially perceived as a reflected image by a vehicle occupant.

[0036] To detect whether the foreign object 7 is resting on the display surface 3 or not, a detection sensor 10 is provided, which is integrated in the Figures 1a and 1bis initially shown schematically. The detection sensor system 10 can comprise one or more camera systems or one or more brightness sensors and, if necessary, combine these to detect a foreign object 7 resting on the display surface 3.

[0037] The detection sensor system 10 communicates with the control unit 9, which, upon detection of the foreign object 7, signals the blocking of the display by the foreign object 7 to the vehicle occupants, in particular the driver of the motor vehicle, in a suitable manner, for example by means of an acoustic or visual warning signal. The visual warning signal can be output via a separate output unit or via the display unit 2. In the latter case, for example, the warning signal can be output across the entire display surface so that the vehicle occupants can perceive the warning signal despite the foreign object 7 being in contact (partially obscuring the display image). For example, the warning signal can be displayed as a flashing, flat display on the display surface 3.

[0038] In Figure 21 shows a cross-sectional view of an exemplary embodiment in which the detection sensor system 10 comprises an image capture sensor system 10a. The image capture sensor system 10a can, for example, have one or more cameras which are directed directly or indirectly, i.e., for example, via a corresponding reflection area R' of the windscreen 5, which can be different from the reflection area R, at the display surface 3 of the display unit 2. The image capture sensor system 10a should capture the entire area of ​​the display image. The image capture sensor system 10a is arranged such that it is not in the field of vision of the driver and / or the other vehicle occupants and is in the field of vision of the reflection image in the reflection area R, so that an unobstructed view of the driver and / or the other vehicle occupants is possible.

[0039] In particular, it does not interrupt the beam path between the display surface 3, the reflection area on the front screen 5 and the eye area B.

[0040] To detect whether a foreign object 7 is resting on the display surface 3, the control unit 9 can control the display unit 2 to impart an additional characteristic as a display property to the display image to be displayed. The control unit 9 can obtain a detection image by evaluating a display image captured by the image detection sensor system 10a and determine whether the display image is partially obscured by checking the detection image for the display property. The detection image captured by the image detection sensor system 10a is checked to determine whether it exhibits the display property of the imprinted additional characteristic.

[0041] The control unit 9 can, for example, control the display unit 2 to periodically replace or overlay the display image to be displayed with a test image that is displayed for such a short time that it is imperceptible to the human eye. For example, the test image can be displayed for periods between 5 ms and 40 ms with a repetition frequency of, for example, 1 Hz. Accordingly, the captured image of the image capture sensor system 10a can be evaluated to detect the periodically displayed test image and provide it as a captured image. A comparison of the captured image with the displayed test image makes it easy to determine the presence of a foreign object 7 on the display surface 3. In particular, if it is determined that the captured image does not completely correspond to the displayed test image, a malfunction of the display of the reflective display system 1 can be assumed.

[0042] Alternatively, the test image can also be displayed using invisible light, particularly infrared light, so that it is superimposed on the display image. The periodic intermittent display of the test image is then unnecessary. For this purpose, the image capture sensor system 10a must be designed to be particularly sensitive to the wavelengths of invisible light.

[0043] Alternatively, the detection sensor system 10 can also have a brightness sensor 10b, which is directed directly or indirectly at the display image of the display surface 3 and detects an overall detection brightness. By evaluating the overall detection brightness with an expected reference brightness, the obscuration of the display image by a foreign object 7 can be detected. If it is determined that the overall detection brightness is lower than the expected reference brightness, a display fault is detected by the display unit 2 and signaled accordingly.

[0044] The detection brightness can continuously detect the brightness of the entire display image and evaluate it against a reference brightness determined from the display image. Alternatively, a test image can be periodically output, as described above, and the detection brightness can be determined solely with respect to the display of the test image. The corresponding reference brightness is then derived from the overall brightness of the test image.

[0045] The reference brightness results from the display of the displayed image, in particular as the sum of the brightnesses of the individual pixels.

[0046] In order to compensate for the influence of ambient brightness, e.g., due to sunlight incident on the display surface, two test images with different overall brightnesses can be displayed one after the other in the case of periodic display of the test image and brightness evaluation only with respect to the displayed test image. By evaluating the difference between the detection brightnesses detected by the brightness sensor 10b in relation to a difference between the reference brightnesses determined from the test images or assigned to them, the influence of ambient brightness can be eliminated. Since, assuming that the ambient brightness does not change during the detection of the two test images, the brightness difference between the two test display images must remain constant with the display surface uncovered, regardless of the ambient brightness.

[0047] In an alternative embodiment, the detection sensor system 10 can be directed at the eye area B, in particular at the eyes of the vehicle occupants. In this case, the display image is evaluated according to the methods described above, indirectly by reflection on the windshield 5 and in the eyes of the vehicle occupant. Pixels of the test image that are not reflected in the eye of the vehicle occupant by the image capture sensor system are recognized as obscured, and the foreign object 7 is thereby detected.

[0048] To determine whether the foreign object 7 is resting on the display surface 3, the control unit 9 can control the display unit 2 to impose an additional characteristic as a display property on the display image to be displayed. The control unit 9 can determine whether the display image is partially obscured by evaluating an image captured by the image capture sensor 10a on the display surface 3.

[0049] The display property of the display image can be a periodic display of a test image sequence as a sequence of multiple test images, each of which addresses different pixels of the display area 3. For example, a periodic, successive activation of individual pixels or groups of pixels (e.g., pixel segments) on the display area 3 can be carried out. For example, display segments can be activated sequentially in a grid-like manner, so that each of the pixels or each of the regions of the display area 3 is activated once within the test image sequence.

[0050] This makes it possible to design the detection sensor 10 as a simple brightness sensor 10b. This is operated to generate a signal sequence by appropriately detecting the test image sequence, i.e. when detected at the corresponding display times, which indicates the brightness of the test images in the test image sequence one after the other. By detecting a signal gap with reduced brightness, a hidden area on the display surface 3 can be determined. If, therefore, at a predetermined display time of one of the test images in the test image sequence, a brightness corresponding to the test image cannot be detected, it can be concluded that the corresponding surface area is hidden. The surface area can be localized in particular depending on the temporal position of the signal gap. In particular, the localization of the surface area can be used to determine the area on the display surface that is hidden by the foreign object 7.Accordingly, a display malfunction can be signaled if at least a certain predefined part of the display image to be displayed is impaired by the masking.

[0051] In the flowchart of the Fig. 3 a method for detecting a foreign object 7 is shown, which is carried out in the control unit 9 in conjunction with the detection sensor system 10 and the display unit 2.

[0052] In step S1, a test image can be displayed according to a specific display property. The display property can correspond to a periodic display of the test image for a display period that is preferably sufficiently short so that it cannot be perceived by a vehicle occupant. The display property can also provide a specific overall brightness of the display image or an overall brightness in a specific wavelength range. Alternatively, the test image can also comprise a display in the non-visible wavelength range.

[0053] In step S2, the display image can now be captured using the detection sensor system 10. The capture can provide image capture, for example, using the image capture sensor system 10a, or brightness capture using the brightness sensor 10b. The capture can be provided by directly directing the detection sensor system 10 at the display surface 3, by indirectly directing the brightness sensor at the display surface 3, for example, via the further reflection area R' on the windshield 5, or by directing the detection sensor system 10 at an eye of a vehicle occupant.

[0054] In step S3, the directly or indirectly acquired display image is evaluated for display properties to obtain a detection image. For example, in the case of a periodic display of a test image, the detection image is assumed to be the display image acquired at the time the test image was output. The detection image may also correspond to a display image acquired in a non-visible wavelength range.

[0055] Alternatively, a brightness of the display image can be recorded and compared with a reference brightness resulting from the display image for evaluation.

[0056] In step S4, the control unit 9 uses the captured image or the captured brightness to check whether the display property imprinted on the display unit's display can be fully confirmed by the captured image or the captured brightness. If this is the case (alternative: yes), the display surface 3 is not obscured, and the method continues with step S1. Otherwise (alternative: no), the method continues with step S5.

[0057] In step S5, a malfunction of the display or an obscuration of the display surface can be signaled accordingly. The signaling can be done using a visual or acoustic signal or in another way that alerts the vehicle occupants that the display is at least partially obscured.

[0058] In particular, in step S4, it can be checked whether the acquired image or a sequence of acquired images corresponds to a corresponding test image (with the same display and acquisition times), test image, or a sequence of test images (with the same display and acquisition times). If it is determined that the test image or the sequence of test images corresponds to the acquired image or the sequence of acquired images, no obscuration of the display image is detected. Alternatively, obscuration of the display image is detected, and a corresponding display disturbance is signaled by the reflective display system 1.

[0059] Alternatively, in step S4, it can be checked whether the detection brightness of a display image at a display time corresponds to a reference brightness of the display image at that display time. If the detection brightness deviates from the reference brightness, it can be concluded that the display surface 3 is at least partially obscured.

Claims

1. Reflection display system (1) for displaying a display image for a vehicle occupant of a motor vehicle by reflection of the display image on a windshield (5), comprising: - a display unit (2), which is designed to display the display image on a display surface (3) of the display unit (2); - a detection sensor system (10, 10a, 10b) for acquiring the displayed display image on the display surface (3) of the display unit (2), and - a control unit (9), which is designed ∘ to activate the display unit (2) for display; ∘ to ascertain acquisition information depending on the acquired displayed display image, ∘ to establish the presence of a foreign object (7) on the display surface (3) of the display unit (2) depending on the displayed display image and depending on the acquisition information; and ∘ to signal a disturbance of the display if the presence of a foreign object (7) on the display surface (3) of the display unit (2) is established.

2. Reflection display system (1) according to Claim 1, wherein the display unit (2) is arranged in a depression (6) such that the display image is not perceptible directly in an eye region (B).

3. Reflection display system (1) according to Claim 1 or 2, wherein the control unit (9) is designed to activate the display unit (2) to output a test image having a display property, to acquire an acquisition image of the test image as acquisition information, and to check the presence of the display property on the basis of the acquisition image in order to signal a disturbance of the display if the display property is not present with respect to the acquisition image.

4. Reflection display system (1) according to Claim 3, wherein the test image corresponds to a display having different display information than the display image and / or an image having a different display property than the display image, in particular the overall brightness of the image.

5. Reflection display system (1) according to Claim 3 or 4, wherein the detection sensor system (10, 10a, 10b) includes an image acquisition sensor system (10a), which is designed to acquire the display of the display surface (3) directly or indirectly via one or more reflections, wherein the display of the display surface (3) is acquired indirectly via a further reflection region (R') on the windshield or via a reflection in the eye of a vehicle occupant.

6. Reflection display system (1) according to Claim 5, wherein the display property comprises a periodic display of a test image for an imperceptible period of time on the display surface, wherein the control unit (9) is designed to ascertain the display of the test image as the acquisition image, and to determine the presence of the foreign object (7) on the display surface (3) by comparing the acquisition image to the test image.

7. Reflection display system (1) according to one of Claims 3 to 4, wherein the display property comprises a display of a test image in nonvisible light, wherein the control unit (9) is designed to ascertain the display of the test image in the nonvisible light as the acquisition image and to determine the presence of the foreign object (7) on the display surface (3) by comparing the acquisition image to the test image.

8. Reflection display system (1) according to one of Claims 1 to 4, wherein the detection sensor system (10, 10a, 10b) comprises a brightness sensor (10b), which is designed to acquire an overall brightness of the display image directly or indirectly via one or more reflections as the acquisition brightness, wherein the control unit (9) is designed to ascertain the presence of the foreign object (7) on the display surface (3) depending on the acquisition brightness and a reference brightness.

9. Reflection display system (1) according to Claim 8, wherein the reference brightness is ascertained as the overall brightness of the display image to be displayed.

10. Motor vehicle comprising: - a dashboard (7) between a windshield (5) of the motor vehicle and a steering column; - the reflection display system (1) according to any one of Claims 1-9.

11. Method for detecting a foreign object (7) on a display unit (2) of a reflection display system (1), which causes a reflection of a display image on a windshield (5) in an eye region (B) of a vehicle occupant, having the following steps: - displaying (S1) the display image on the display surface (3); - acquiring (S2) the displayed display image on the display surface (3) by way of a detection sensor system (10, 10a, 10b); - ascertaining (S3) acquisition information depending on the acquired displayed display image, - establishing (S4) the presence of a foreign object (7) on the display surface (3) of the display unit (2) depending on the displayed display image and depending on the acquisition information; and - signaling (S5) a disturbance of the display if the presence of a foreign object (7) on the display surface (3) of the display unit (2) is established.