Method for controlling the brightness of an image projected by an imaging device of a head-up display located in a motor vehicle

Description

[0001] The invention relates to a method for controlling the projection brightness of an image projected by an imaging device of a head-up display located in a motor vehicle according to the features of the preamble of patent claim 1.

[0002] Such a method is known from DE 10 2012 204 303 A1. In this known method, a brightness curve is first determined for the background area located behind the virtual image from the driver's perspective. After determining the brightness curve at the pixel level, it is dynamically divided into areas of similar brightness, and the segments or pixels of similar brightness are assigned the same brightness value. This forms the basis for creating a brightness control matrix. For the purpose of individually controlling the brightness of a segment using the brightness control matrix, a value from the brightness control matrix is ​​assigned to each segment or pixel of the transmitted image whose brightness is individually controlled. The brightness of the segments of the transmitted image is thus controlled depending on the brightness control matrix.

[0003] The disadvantage here is that the processing at pixel level and especially the dynamic subdivision of the brightness profile into areas of similar brightness is very complex and therefore requires high computing capacity and speed, which can usually only be provided in the upper price segment of motor vehicles.

[0004] DE 10 2009 041 205 A1 also discloses a method for controlling a head-up display in a vehicle. A camera is used to capture the road surface in front of the vehicle. The camera signals (brightness and / or color level of the road surface) are evaluated to control the brightness and / or color of the head-up display's projection signal. The measurement range for determining the brightness can be adjusted via the camera's exposure time.

[0005] US 2018 / 120572 A1 describes a vehicle display device that has a head-up display for generating a virtual image in front of a windshield in the driver's field of vision. Furthermore, an interior camera and an outward-facing camera (exterior camera) are present. The interior camera is used to record the driver's line of sight or eye movements, while the exterior camera records the area in front of the vehicle. To improve the visibility of the virtual image, the brightness of the virtual image to be generated is adjusted to the image recorded by the exterior camera. This is done by first selecting a specific image section within the recorded image. The selected image section is corrected if necessary depending on an eye area or depending on eye movements of the driver's eye points.The brightness of the image section is then calculated based on various possible algorithms and the brightness of the virtual image to be generated is adjusted to the brightness of the image section.

[0006] It should also be noted that this method is intended for use in conjunction with so-called augmented reality head-up displays (AR-HUDs). With AR-HUDs, the image to be projected is not just projected a few centimeters in front of the windshield within the driver's field of vision, but rather a few meters in front of the vehicle onto a very large background area, usually the road. The brightness profile of the background area is therefore extremely important for a needs-based projection of the image to be projected.

[0007] The invention is therefore based on the object of providing a method for controlling the projection brightness of an image projected by an imaging device of a head-up display located in a motor vehicle, which method requires less computing power and can therefore be implemented cost-effectively.

[0008] This object is achieved by a method according to the features of patent claim 1. Advantageous embodiments or further developments of the method can be found in the dependent claims.

[0009] The invention is based on a method for controlling the brightness of an image projected by an imaging device of a head-up display in a motor vehicle. The projected image is reflected into the driver's field of vision, causing the driver to perceive a virtual image. A brightness profile is determined for a background area located behind the virtual image from the driver's perspective. A brightness control matrix is ​​then determined from the brightness profile, which is used to control the projection brightness of the image projected by the imaging device.

[0010] The following steps are carried out over a specific period of time: Recording multiple background images of the background area using one camera

[0011] According to the invention, the following further steps are carried out: Division of each recorded background image into a predefined grid of sectors Determination of a local average brightness value across all pixels of each sector of a recorded background image Derivation of a temporal average brightness value of each sector across all recorded background images, valid for the time period Determination of the brightness control matrix, consisting of the temporal average brightness values ​​assigned to the sectors Individual sector control of the projection brightness for sectors of the image projected by the imaging device depending on the brightness control matrix, whereby the projected image is divided into a grid of sectors which corresponds to the grid of the recorded background image. The number of sectors of the imaging device of the head-up display, their arrangement and size is therefore determined by the number of sectors of the recording camera orof the captured background image, although this does not necessarily require that the number of pixels of both be identical.

[0012] It should also be mentioned that the method according to the invention can of course also be carried out using DMD technology (DMD=Digital Micromirror Device), in which individual image components are generated by movable micromirrors.

[0013] These features and the sequence of process steps make it possible to adjust the image projected by a head-up display as needed, depending on the background onto which it is to be projected, so that the driver can clearly see the displayed information without being dazzled. In particular, dividing each recorded background image into a predefined grid of sectors saves considerable computational effort.

[0014] According to a further development of the method, it is proposed that the predefined grid of sectors divide the recorded background image and the projected image into equally sized, square areas. This allows for the desired alignment of the projected image with the background of the projection to be optimized.

[0015] According to another development of the invention, in the individual sector control of the projection brightness, each pixel of the image projected by the imaging device within a sector is assigned the same brightness value. This development contributes to a further possible reduction in computational effort.

[0016] According to another refinement, it is also proposed that the temporal brightness averages of the sectors be transmitted as matrix brightness values ​​of the brightness control matrix, one after the other and in a predetermined order, to a control device for sector-specific control of the projection brightness of the projected image. This refinement also contributes to simplifying the method. Thus, when transmitting the matrix brightness values ​​in certain data packets, information about the respective position of a transmitted matrix brightness value within the brightness control matrix can be omitted, since this information can be defined in a control logic. The matrix brightness values ​​can be transmitted, for example, via a data bus (e.g., CAN bus).

[0017] It is further advantageous if projection brightness values ​​of the projected image are derived from at least some of the matrix brightness values ​​of the brightness control matrix by forming a respective projection brightness value from the product of the respective matrix brightness value with a predetermined empirical factor. The empirical factors are thus determined empirically. This requires only a certain initial effort, which, however, also contributes to reducing the computational effort when the method is actually implemented.

[0018] Furthermore, it can help reduce the necessary computational effort if the imaging device generates the symbols, pictograms, and / or alphanumeric characters to be projected based on vector graphics. This makes it possible to scale projected symbols, pictograms, or alphanumeric characters as required with minimal computational effort. Pixel-precise storage of such graphics can reduce the computational effort even further, but at the expense of display flexibility.

[0019] Finally, according to a very advantageous development of the method, it is proposed that the position of the driver's eyes be detected and the position of the image to be projected be derived from this. For example, it is conceivable that the driver briefly leans slightly to the right or left from their normal seating position. This can be detected, for example, with the help of an interior camera. This also changes the position of their eyes, which in turn influences the location or position of the image to be projected by the head-up display on the background surface. If the driver leans to the right, the image is projected correspondingly more to the right; if they lean to the left, the opposite is true.

[0020] A preferred embodiment of the invention is illustrated in the figures and will be explained in more detail in the following description with reference to the figures. This also makes further advantages of the invention clear. Identical reference symbols, even in different figures, refer to identical, comparable or functionally identical components. Corresponding or comparable properties and advantages are achieved even if a repeated description or reference to them is not made. The figures are not, or at least not always, to scale. In some figures, proportions or distances may be exaggerated in order to emphasize features of an embodiment more clearly.

[0021] They show, schematically Fig. 1 shows a motor vehicle with a head-up display which operates according to the method according to the invention, Fig. 2 shows the division of a recorded background image into a grid of sectors, Fig. 3 shows the calculation of a temporal average from several local averages, Fig. 4 shows a brightness control matrix, Fig. 5 shows the generation of projection brightnesses from matrix brightness values ​​and Fig. 6 shows a view of a driver from a windscreen of the motor vehicle.

[0022] First, the Fig. 1 Reference is made to this. A motor vehicle K can be seen in the area of ​​its windshield 10.

[0023] The motor vehicle K is equipped with a so-called head-up display 20, which is installed in the area of ​​an instrument panel 30 in the primary field of vision of a driver F.

[0024] The head-up display 20 has an imaging device 21 as one of its optical components.

[0025] The imaging device 21 can project an image 22 onto a mirror of unspecified number (for example, a mirror using DMD technology), which image 22 is reflected into the eyes of the driver F. The driver F perceives a virtual image 40 in front of the windscreen 10 or in front of the motor vehicle K.

[0026] Furthermore, it can be seen that the imaging device 21 is provided with a control device 23 in terms of signal technology and this in turn is provided with a storage device 24 in terms of signal technology.

[0027] A camera 50, which is also connected to the control device 23 by means of signals, serves to record an image of a background in front of the motor vehicle K, which is intended to make the virtual image 40 more easily recognizable for the driver F and yet appear glare-free. This is explained in more detail below: Fig. 2 a recorded background image 51 of a background surface A in front of the motor vehicle K (compare Fig. 6 ). According to the method, several background images 51 are captured within a specific time span of, for example, a few milliseconds. Their values ​​are stored in the memory device 24.

[0028] Each of the background images 51 is then divided into a fixed grid R of sectors S by an evaluation logic stored in the control device 23. The grid R is preferably selected such that all formed sectors S are of equal size and, particularly preferably, have a square outline. For example, the size of each individual sector S can be 20 by 20 pixels P.

[0029] Deviating from the exemplary embodiment, it is also conceivable that the sectors S have different sizes and are not square.

[0030] Each of the sectors S is assigned a specific position within the background image 51. Thus, the sector S in the first column and first row from the left is assigned the position 1-1. The sector S in the second column and first row is assigned the position 2-1, the sector S in the first column and fifth row is assigned the position 1-5, and so on.

[0031] After dividing the background image 51 into sectors S, each of the sectors S is assigned a local brightness mean HM1 formed from the brightness values ​​of the individual pixels P. This is performed for all sectors S of a background image 51 and for all background images 51 within a specific time period. As an example, the local brightness mean HM1 (4-4) of sector S is listed for the position in the fourth column and the fourth row in the background image 51. A temporal brightness mean HM2 is then formed from the local brightness mean values ​​HM1 of a sector S within a time period.

[0032] Based on Fig. 3This is explained in more detail. This figure plots the local mean values ​​HM1 over time t. Furthermore, a time period T is visible within which, at five times t1 to t5, a temporal brightness mean value HM2 was calculated from the respectively measured local brightness mean values ​​HM1 (t1) to HM1 (t5) for a specific sector S. This is performed for all sectors S of the background image 51.

[0033] All temporal brightness averages HM2 calculated in this way are then transferred by the control device 24 into a brightness control matrix 52, which Fig. 4 is shown. In the brightness control matrix 52, a specific temporal brightness average HM2 is assigned to each sector S. The numbers in parentheses refer to the position of a sector within the matrix.

[0034] For example, the temporal brightness mean value HM2 (1-1) is assigned to sector S in the first column and first row. The temporal brightness mean value HM2 (15-5) is assigned to sector S in the fifteenth column and fifth row. This is done analogously for all brightness mean values ​​HM2 of the brightness control matrix 52.

[0035] The Fig. 5 now describes how the temporal brightness mean values ​​HM2 of the brightness control matrix 52 are used to control a projection brightness PH of the image 22 or 40 to be projected by the imaging device 21.

[0036] Thus, the temporal brightness mean values ​​HM2 of the sectors S are transmitted as matrix brightness values ​​of the brightness control matrix 52, one after the other and in a predetermined sequence, to the control device 23 for sector-specific control of the projection brightness PH of the projected image 22 or 40. For example, a control table 53 can store that the first transmitted matrix brightness value HM2 is always assigned to a sector S' of the projected image 22 or 40 in the position (1-1)', i.e., also in the first column and in the first row. Analogously, the temporal brightness mean value HM2 read in at the seventy-sixth position is always assigned a sector S' with the position (16-1)' on the projected image 22 or 40, and so on.

[0037] Each read-in temporal brightness mean value HM2 is multiplied in the control device 23 by an empirically determined experience factor E. The experience factor E and / or the control table 53 can be stored, for example, in the memory device 24.

[0038] Thus, from the temporal brightness mean values ​​HM2, projection brightnesses PH are calculated for all Fig. 6 visible sectors S' of the projected image 40 visible to the driver F. It should be noted that an additional camera 60 (cf. Fig. 1) may be present. The camera 60 can be used to capture the interior or the position of the eyes of the driver F and to transmit the captured position data to the control device 23. Based on a specific algorithm stored in the control device 23, a changed position of the image 22 or 40 to be projected within the background image 51 is then calculated from this information. The assignment of the projection brightnesses PH thus also changes depending on the position of the driver F.

[0039] In the previously described sector-specific control of the projection brightness PH, the same projection brightness PH is assigned to each pixel P of the image 40 projected by the imaging device 21 or to the partial images 41 and 42 within a sector S'.

[0040] The figure shows the view of the driver F through the windscreen 10. It can be seen that the background area A for the virtual image 40 to be projected, from whose partial area the already described background image 51 was created, is a roadway.

[0041] The virtual image 40 is now intended to project a virtual partial image 41 in the form of a display of the driving speed in the left image area and a virtual partial image 42 in the form of a turning arrow in the right image area.

[0042] Furthermore, it can be seen that the projected virtual image 40 corresponds to the size and position of the recorded background image 51. The selection of the grid R' of the projected image 40 is also selected according to the grid R. Sectors S' of the projected image 40 thus formed are thus approximately the same size as the sectors S of the recorded background image 51.

[0043] It is also evident that the partial image 41 to be projected lies in sectors S' whose positions lie on a brightly lit road section, as can be seen, for example, in positions (2-1)' and (5-2)'. Due to the previously calculated temporal brightness averages HM2, the virtual partial image 41 is therefore projected particularly brightly in order to be clearly recognizable by driver F despite the brightness of the background.

[0044] The virtual partial image 42, on the other hand, is located in an image area where the brightness of the sectors S of the background image 51 is significantly lower due to shadows. This can be seen, for example, in the sector S with the position 11-1. The temporal brightness averages HM2 previously formed here are correspondingly lower, so that the projection brightnesses PH for the turn arrow to be displayed are also lower here. This prevents dazzling of the driver F.

[0045] It should also be noted that the representations to be projected in partial images 41 and 42 are stored as so-called vector graphics in the memory device 24. These vector graphics can be scaled as required by the control device 23 without significant computational effort. Deviating from the exemplary embodiment, pixel-precise storage of graphics is also conceivable, which can further reduce the computational effort. List of reference symbols

[0046] 10Windscreen 20Head-up display 21Imaging device 22Projected image 23Control device 24Storage device 30Instrument panel 40Projected virtual image 41Virtual partial image 42Virtual partial image 50Camera 51Recorded background image 52Brightness control matrix 53Control table 60Camera 1-1 to 16-5Position of the sectors on the recorded background image (1-1)' to (16-5)'Position of the sectors on the projected image ABackground area EExperience factor FDriver HM1Local mean brightness HM2Temporal mean brightness, matrix brightness value KMotor vehicle PPixel PHProjection brightness RGrid of the recorded background image R'Grid of the projected image SSectors of the recorded background image S'Sectors of the projected image tTime t1-t5Time points TTime span

Claims

1. Method for controlling the projection brightness (PH) of an image (22) projected by an imaging device (21) of a head-up display (20) located in a motor vehicle (K), which image is reflected into the field of vision of a driver (F), whereby the driver (F) perceives a virtual image (40), a brightness profile being determined for a background surface (A) located behind the virtual image (40) from the perspective of the driver (F), and a brightness control matrix (52) being determined from the brightness profile, with the aid of which a projection brightness (PH) of the image (22) projected by the imaging device (21) is controlled, the following successive steps being carried out for a specific period of time (T): • recording several background images (51) of the background surface (A) using a camera (50) characterized by the following further successive steps: • dividing each recorded background image (51) into a predefined grid (R) of sectors (S), the size of each sector (S) comprising a plurality of pixels (P) • determining a local brightness mean value (HM1) over all pixels (P) of each sector (S) of a recorded background image (51) • deriving a temporal brightness mean value (HM2) of each sector (S) for the period of time (T) over all recorded background images (51) • determining the brightness control matrix (52), consisting of the temporal brightness mean values (HM2) assigned to the sectors (S) • sector-individual control of the projection brightness (PH) for sectors (S') of the image (22) projected by the imaging device (21) depending on the brightness control matrix (52), the projected image (22) being divided into a grid (R') of sectors (S') which corresponds to the grid (R) of the recorded background image (51).

2. Method according to claim 1, characterized in that by the predefined grid (R, R') of sectors (S, S'), the recorded background image (51) and the projected image (22, 40) are divided into equally sized, square regions.

3. Method according to claim 1 or 2, characterized in that during sector-individual control of the projection brightness (PH), each pixel (P) of the image (22, 41) projected by the imaging device (21) within a sector (S') is assigned the same brightness value.

4. Method according to any of the preceding claims, characterized in that the temporal brightness mean values (HM2) of the sectors (S) are transmitted as matrix brightness values (HM2) of the brightness control matrix (52) one after the other and in a fixed predetermined order to a control device (23) for sector-individual control of the projection brightness (PH) of the projected image (22, 40).

5. Method according to any of the preceding claims, characterized in that projection brightness values (PH) of the projected image (22,40) are derived from at least some of the matrix brightness values (HM2) of the brightness control matrix (52) by forming a projection brightness value (PH) from the product of the relevant matrix brightness value (HM2) with a predetermined experience factor (E).

6. Method according to claim 5, characterized in that symbols, pictograms and / or alphanumeric characters to be projected by the imaging device (21) are generated on the basis of vector graphics.

7. Method according to any of the preceding claims, characterized in that a position of the driver's (F) eyes is detected and a position of the image (22) to be projected is derived therefrom.