Head-up display with reduced ghosting artifacts

Abstract

The present application relates to a method for providing image data for projection onto a windshield, the method comprising the following steps at an image processing unit: determining an inverse filter function u to be used to remove ghosting artifacts occurring when the image data is projected onto the windshield; determining a first image f to be projected on the windshield; and determining a ghosting function, the ghosting function representing the ghosting artifact by which the first image to be projected is distorted when projected on the windshield. The inverse filter is applied to the first image f to generate a filtered image, and the filtered image is provided to a projection device configured to project the filtered image onto the windshield.

Description

Technical Field

[0001] This application relates to a method for providing image data for projection onto a windshield, and an image processing unit configured to provide image data for projection. Furthermore, a display system is provided, including a projection device, the image processing unit, and a computer program including program code. Background Technology

[0002] A head-up display (HUD) is a device that allows virtual content to be rendered within an observer's field of vision. In a car setting, the observer is the driver; in a fighter jet or any other aircraft, the observer is the pilot. The main idea behind a HUD is to have the display reflect light from a windshield / windshield, thus creating a virtual image at a distance. Light from the display is reflected by the inside and outside of the windshield. The former produces the main image, while the latter produces a ghosted image.

[0003] To eliminate ghosting, a wedge is added to the windshield, which directs the external reflection to the same position as the primary reflection. However, this wedge significantly increases the cost of the windshield. Without such a wedge, the head-up display (HUD) suffers from severe ghosting artifacts. Therefore, there is a need to provide a way to reduce or eliminate ghosting artifacts in HUDs, especially when no additional wedge is used on the windshield. Summary of the Invention

[0004] The features of the independent claims of this application satisfy this requirement. Other aspects are described in the dependent claims.

[0005] According to a first aspect, a method is provided for providing image data for projection onto a windshield, wherein an image processing unit determines an inverse filter function u to be used to remove ghosting artifacts that occur when the image data is projected onto the windshield. Furthermore, a first image f to be projected onto the windshield is determined, and a ghosting function representing the ghosting artifacts by which the first image to be projected is distorted when projected onto the windshield. The inverse filter is applied to the first image f to generate a filtered image v, and the filtered image v is provided to a projection device configured to project the filtered image onto the windshield.

[0006] According to one perspective, the inverse filter is applied even before artifacts appear during display. Therefore, the inverse filter can be applied before sending the filtered image for display, and the ghosting can be applied during projection so that the two effects cancel each other out and a clear image without ghosting is obtained. This is mainly due to the fact that the mathematical operations occurring in different processing steps are linear, thus the order of filter application is irrelevant.

[0007] In addition, a corresponding image processing unit is provided, including a memory and at least one processor, wherein the memory contains instructions executable by the at least one processor, and the image processing unit is configured to operate as discussed above or in further detail below.

[0008] In addition, a display system is provided, which is configured to project image data onto a windshield, wherein the display system includes a projection device configured to project a filtered image onto the windshield, and the display system further includes an image processing unit as mentioned above or discussed in further detail below.

[0009] In addition, a computer program including program code is provided, which is executed by at least one image processing unit, wherein the execution of the program code causes at least one image processing unit to perform the methods mentioned above or discussed in further detail below.

[0010] It should be understood that, without departing from the scope of the invention, the above features, as well as those to be explained below, can be used not only in the indicated combinations, but also in other combinations or individually. Unless otherwise expressly stated, the features of the above aspects and the embodiments described below can be combined with each other in other embodiments. Attached Figure Description

[0011] The foregoing and additional features and effects of this application will become apparent from the following detailed description when read in conjunction with the accompanying drawings, in which the same reference numerals denote the same elements.

[0012] Figure 1 It is a schematic structural view of a display system including projection equipment and image processing unit, which can reduce ghosting artifacts when projecting image data onto the windshield.

[0013] Figure 2 It shows including Figure 1 This is a schematic diagram of the input image for the content that the system needs to display.

[0014] Figure 3a and Figure 3b It shows Figure 1 The diagram shows an example image displayed on a display system that does not apply the features of the present invention, and the synthesized image includes ghosting artifacts.

[0015] Figure 4 A diagram is provided showing how to correct image pixels in a view of ghosting artifacts.

[0016] Figure 5 It shows the Figure 4 The image shown is a schematic diagram of the application of an inverse filter to an image with ghosting.

[0017] Figure 6 A schematic diagram of an image processing step performed using inverse filtering is shown in one embodiment.

[0018] Figure 7 A further schematic diagram of the image processing steps using inverse filtering according to another embodiment is shown.

[0019] Figure 8 It is used to avoid Figure 1 An example flowchart of a method for dealing with ghosting artifacts in image data projection in a system.

[0020] Figure 9 Further example views of images processed by the image processing unit when combining different filter options are shown.

[0021] Figure 10 The configuration for the application is shown. Figure 1 A schematic diagram of an example image processing unit in the system shown. Detailed Implementation

[0022] In the following description, embodiments of the invention will be illustrated in detail with reference to the accompanying drawings. It should be understood that the following description of the embodiments should not be considered limiting. The scope of the invention is not intended to be limited to the embodiments or drawings described below, which are for illustrative purposes only.

[0023] The accompanying drawings should be considered schematic representations, and the elements shown are not necessarily shown to scale. Rather, the representation of the individual elements is intended to make their general function and purpose obvious to those skilled in the art. Any connection or coupling between functional blocks, devices, or components of the physical or functional units shown in the drawings and described below may also be achieved through indirect connection or coupling. Coupling between components may be established through wired or wireless connections. Functional blocks may be implemented through hardware, software, firmware, or a combination thereof. Ghosting is described mathematically below, followed by a more detailed description of inverse filtering methods for eliminating ghosting. Later, several methods for modifying the input image so that any intermediate images appearing during processing do not have negative light values ​​were proposed.

[0024] The ghosting effect is modeled mathematically, based on an input image containing the content to be displayed on the windshield. Figure 2 This shows a possible example of such input image 30.

[0025] Ghosting can be modeled using simple convolutions, where the convolution kernel for ghosting is given by the following function:

[0026]

[0027] Where tx , t y is the offset of the ghost image relative to the input image, while m is the amplitude of the ghost image relative to the total reflected light.

[0028] Figure 1 This is a schematic diagram of a display system such as a head-up display (HUD) that can present information for display to users such as drivers, where the information is projected onto a transparent screen such as a windshield 10. The display system includes an image processing unit 100 that performs image processing to remove any ghosting artifacts that occur when an image is projected and reflected by the windshield 10. System 200 includes a projection device 210 and an image processor 100. The projection device 210 uses a mirror 20 to project image data onto the windshield 10, and because the windshield includes an inner and outer surface, it generates both the actual image and the ghosting image, which the user 5 can see through the windshield, where the user's eyes... Figure 1 It is shown schematically in the middle.

[0029] exist Figure 1 In the architectural view, the different processing steps performed in the image processing unit 100 are schematically shown as functional entities, and it should be understood that... Figure 1 The different functional entities shown do not need to be implemented as depicted in the figure, but can be achieved through a combination of... Figure 10 The processing unit discussed implements this through program code. In vehicles, systems such as head units can be provided that determine the information to be displayed to the user via a head-up display, such as... Figure 1 As shown, this allows the design unit 101 to design the content to be displayed and the input image to be displayed. Examples of such input images may include the current speed, speed limit, driving direction, or any other information of the vehicle using the system. Figure 2 The image 30 shown can be generated by the image generation unit 102 that generates the input image. Furthermore, ghosting artifacts are determined in the inverse filter calculation unit 103, and a contrast adjustment unit 104 can be provided for contrast adjustment, as will be discussed below. The output of the contrast-adjusted image is the first image to be projected onto the windshield. The inverse filtering unit 105 applies an inverse filter to eliminate ghosting artifacts that will appear during projection but have not yet appeared. The filtered image is fed to the projection device 210, which is then projected onto the windshield screen 10.

[0030] As can be seen from Figure 3A, severe ghosting is introduced when no countermeasures are taken, and this can be seen in the displayed image 31. Generally, effective countermeasures require a thorough understanding of the problem, specifically how the ghosting is generated. Figure 3B now shows the situation using the example setting t. x =1、t y=6, m=1 / 5. Applying the ghosting indicated by Equation 1 to the input image yields the result of image 32. Figure 4 The ghosting filter is also shown, indicating that the filter (the convolutional filter model for ghosting) has only two non-zero entries. The contribution of the main image 41 at the origin and the contribution of the ghost 45 at the corresponding pixel location are shown. The grid indicates the pixels.

[0031] By comparing the synthesized image shown in Figure 3B with the image generated by the head-up display shown in Figure 3A, it can be inferred that the model indicated by Equation 1 correctly reproduces the ghosting, that is, the pixel brightness values ​​and even the aliasing of the fonts are matched.

[0032] Inverse filtering is a known image enhancement technique. By understanding the linear filters that distort images, one can eliminate the introduced artifacts. Let f be the original image, also known as the first image to be displayed. Let g be the convolution kernel that convolves with the signal. The resulting image is b = f\circ g, where \circ is the convolution operator. The reconstructed original image is Where u can be calculated as:

[0033]

[0034] in This represents the Fourier transform. In this example, the Fast Fourier Transform (FFT) is used to calculate... The results are then normalized, so the sum of all coefficients is 1. ,in

[0035] yes The non-zero coefficients, and the index It increases based on the distance from the origin.

[0036] It has the following form:

[0037]

[0038] in, It is a positive number, including 0.

[0039] The application of g can be undone using inverse filtering. The result is an image where ghosting has been eliminated. Inverse filter like Figure 5 As shown. Table 1 lists The maximum weight is 15. In principle, There are an infinite number of non-zero entries. For any practical purpose, the computation needs to be limited. Note that... The amplitude decreases rapidly with distance from the origin. Therefore, only the first k weights can be used. The value of k can be determined based on checking the results of the inverse filter. If ghosting artifacts still exist, the weights should be increased. In the example use case, it was used .

[0040]

[0041] Because convolution is a linear operation, the order in which filters are applied doesn't matter. Therefore, it's possible to eliminate artifacts or ghosting even before they occur or appear. Applying this to the current situation, it means that before sending the image to a head-up display for display, an inverse filter u can be applied to the image data, while the head-up display applies the ghosting g, so that the two cancel each other out and a sharp image is obtained.

[0042] Therefore, the procedure used to remove ghosting artifacts is generally as follows:

[0043] In the first step, the ghosting function is determined, such as the function shown in Equation 1, where the ghosting function can be determined by evaluating an image including ghosting artifacts. In the second step, the inverse filter u is determined, for example, as shown in Equation 3 or... Figure 5 As shown. In Figure 5 In this context, the inverse filter u theoretically has an infinite number of non-zero entries, where in Figure 5 In the first image f, horizontal shading entries 51 and 52 have positive values, while vertical shading areas (such as entry 53) have negative values, and non-shading areas have zero values. In a further step, an inverse filter is applied to the first image f, where the result is called the filtered image v. In the next step, this filtered image is sent to a projection device (such as...). Figure 1 The device 210 in the image is used, and in step 5, a ghosting artifact g is introduced during projection. Therefore, the user or driver sees a filtered image without ghosting artifacts.

[0044] Below, we explain how negative light might be handled in the current context.

[0045] Unfortunately, the original image convolved with the inverse filter can result in negative pixel values. A single transition in image intensity from white (pixel value 1) to black (pixel value 0) can be considered in two cases. In the first case, the transition is from black to white, while in the second case, a transition is considered from white to black.

[0046] In the first case, the output of the inverse filter will be the value. Because all other values ​​are multiplied by the black value of zero, note that... In the second case, it would be ,Right now Therefore, in the first case, more light is needed to output from the system, while in the second case, negative light is required. It's impossible to output more light than the system is designed for, nor is it possible to physically implement negative light using an LCD. Therefore, it's necessary to ensure that negative values ​​do not occur. This can be achieved by adding a certain brightness to the input image and slightly reducing the contrast. The theoretical background for this method is given by [SW]. To calculate the contrast reduction, let... Become the value mapped to black, and let This becomes a value that can be mapped to white. Therefore, we can reformulate the constraint as:

[0047]

[0048]

[0049] What is needed is to make the property

[0050] Maximize the nontrivial solution. This is such a solution to the above equation:

[0051]

[0052]

[0053] In our example =1.250000097453239, therefore =0.8333332900207883 and =0.1666667099792117. Using the above calculated value, ghosting caused by a single transition can be eliminated. However, if there are two transitions, from black to white and back to black or vice versa, the contrast reduction is insufficient to completely eliminate ghosting. The following focuses on two types of transitions, again with two scenarios: one where a white stripe covers... And all other coefficients are covered in black. And the inverse input. This leads to these two equations:

[0054]

[0055]

[0056] In this example

[0057] and

[0058] .

[0059] The filter u is applied to the modified input image. This image is then sent to the HUD for display. Note that the image does contain very dark pixels, but these pixels are relatively localized. The HUD output is a perfect reconstruction of the modified input.

[0060] Another approach, known as the expansion method, will be discussed below.

[0061] The method presented in the previous section reduced contrast overall. In this section, we present a method that only locally increases the brightness of the input image, thus allowing for completely dark areas on the HUD again, given that they are sufficiently far from bright pixels. A key observation led to the dilation method. If the goal is to completely eliminate ghosting, then it cannot be darker than the ghosting itself, which remains the objective. Therefore, to eliminate hard ghosting edges, more light is added to adjacent pixels. There is one degree of freedom: how quickly this added light disappears. This is the parameter of the algorithm, called s. The algorithm has only one linear operation: dilation, hence the current name. The steps are as follows, as discussed below. Figure 6 As shown in the image.

[0062] Combination Figure 6 The dilation method is discussed, in which an input image to be displayed, such as image 61, is provided in the first step. In a further step, the input image 61 is convolved only with the ghosting contribution. It is simply based on... Figure 6 Image 62 shows ghosting contributions that shift pixels and make them darker. In the third step, an expansion operation of size s is applied to the result, where the output is called the expansion. Figure 6 The examples show two values. In image 63A, seven pixels are used for the dilation operation, while in image 63B, three pixels are used for the dilation operation. The next step applies a blur filter of size s to produce blurred images, as shown in images 64A and 64B. In step 5, the maximum value operation of the input image and the blurred image is calculated, resulting in maximum value images 65A and 65B. In step 6, an inverse filter is applied to obtain images 66A and 66B. This image is fed to a projection device where ghosting is applied, and the user or driver sees a portion of the image without ghosting, similar to the maximum value images, as shown in images 67A and 67B.

[0063] Another approach, referred to below as the local approach, is discussed below. In the previous section on the dilation method, the brightness values ​​of individual pixels were considered to avoid requiring negative light. However, this method did not account for situations where the required light exceeds the light the system can produce. This deficiency is addressed in the local approach below. This method uses a similar operation to the method described above, but works differently. An inverse filter is applied to the input image to obtain an image with negative values ​​as well as values ​​greater than 1. Further, it is assumed that the inverse filter is a function h that maps the range from [0,1] to a new range [a, b], and that h is a linear function that depends only on the pixel values ​​themselves. Therefore, a search function is performed that maps the range from [0,1] to a smaller range [c,d], where a <= 0, b >= 1, c >= 0, and d <= 1. Thus, if the inverse filter h is applied again here, a range between 0 and 1 is obtained.

[0064] The following text will also combine Figure 7 The processing steps are discussed. In the first step, input image 71 is provided. In the second step, a maximum value operation is performed, where an inverse filter is applied to the negative part of the input image to generate a minimum value image 72A, which is also called out-of-range processing. 低 .

[0065] The operation is max(0 - apply inverse filtering to the input image, 0).

[0066] Furthermore, a maximum value operation is performed on the image, where an inverse filter is applied to the input image and the result of the application is inverted to generate the maximum value image 72B, which is also known as out-of-range operation. 高 .

[0067] The operation is max(apply inverse filter -1, 0 to the input image).

[0068] The maximum and minimum value images are out of range. 低 and beyond the scope 高 The result is an expansion operation of size s, where the outputs are called expansions. 低 and expansion 高 In the example, s=7 pixels was used. The result is the corresponding images 73A and 73B. In this next step, a blur filter of size s is applied, which results in images 74A and 74B, also known as blurred images. 低 and blur 高 In the following steps, a range image of 75 is generated, calculated as follows: 1 / (blur) 低 +1+ Blur 高 This is called the range. In the following steps, an image is calculated using the input image and the blurred minimum image, called the range offset, and this image is calculated as follows: (Input image + blurred minimum image)低 *The range and this is the modified input to the inverse filter.

[0069] In the next step, an inverse filter u is applied, which yields image 77, and this image is then applied to a projection device so that the user sees an image without ghosting (such as image 78), but otherwise similar to image 77.

[0070] A further approach, known as the redesign approach, will be discussed below. Figure 7 The local approach discussed, combined with inverse filtering, can eliminate ghosting. However, this comes at the cost of altering the design of the content using the applied algorithm. A further option is to design the content within hardware limitations and, in the first place, avoid abrupt transitions from black to white. The algorithm will still perform all necessary adjustments, but these adjustments will introduce noticeable but less noticeable changes. This redesign can be as simple as cropping the corresponding region from the global approach image. Figure 9 Example image 81 is shown, thus identifying two parts 82 and 83 in the image containing content. Within these parts 82 and 83, local methods can be applied only to these parts, and ghosting can be eliminated by inverse filtering, but the design is only slightly modified.

[0071] Figure 8 Some of the main steps performed in the different embodiments described above are shown. In step S91, a ghosting function is determined, which represents ghosting artifacts that cause the first image to be projected to be distorted when projected onto the windshield. Figure 4 The image containing ghosting is shown, including contribution 41 to the image and contribution 45 to the ghosting. Furthermore, in step S92, a first image to be projected onto the windshield is determined. (Reference) Figure 1 The first image is an image that has been modulated, in which some brightness values ​​are corrected relative to the input image showing the content to be displayed. Furthermore, an inverse filter is determined in step S93. Figure 5 An example of an inverse filter is shown. In step S94, the inverse filter is then applied to the first image to generate a filtered image, and the filtered image is provided to a projection device 210, which projects the filtered image onto the windshield. During step S95, ghosting is introduced, but since the ghosting artifacts have been removed beforehand, the user or driver can see a substantially ghost-free image.

[0072] Figure 10 It shows Figure 1 The schematic architecture view of the image processing device 100 shown in the figure is as follows, while Figure 1The image processing device is illustrated, having corresponding functional entities that perform different image processing steps. The image processing entities include an interface 110, which is provided for transmitting user data, images, or control messages to other entities (such as projection devices), and for receiving user data, control information, images, or information to be displayed (such as information about ghosting) from other entities. The image processing device further includes a processor 120 responsible for the operation of the image processing unit 100. The processor 120 may include one or more processing elements and is capable of executing instructions stored in a memory 130, which may include read-only memory, random access memory, mass storage devices, hard disks, etc. The memory 130 may further include appropriate program code executed by the processor 120 to implement the aforementioned functionality of the processing unit 100.

[0073] Based on the above, some general conclusions can be drawn:

[0074] According to one aspect of the image processing method, the inverse filter is applied even before the filtered image is provided to the projection device. This means that ghosting artifacts are removed even before they appear during the projection step.

[0075] Furthermore, the first image f may be a modulated image, wherein the input image (such as...) Figure 2 The brightness values ​​of at least some of the pixels in the image 30 shown are adjusted relative to the input image, which includes the content to be displayed on the windshield.

[0076] The modulated image can have lower contrast compared to the input image, between the brightest and lowest possible pixel values. This may involve reducing the contrast of all pixels in the input image.

[0077] In a further embodiment, the brightness values ​​of the modulated image may be increased only for a subset of pixels in the input image, rather than for all pixels. The location of the subset of pixels in the modulated image can be determined based on the location of ghost artifacts. One option here is an expansion method, in which ghost regions in the input image may be identified, where ghost artifacts alter pixel values ​​in the input image. Furthermore, the ghost regions are expanded by neighboring pixels located within a defined distance from pixels located in the ghost regions, to generate an expanded region that includes the ghost regions and neighboring pixels. Furthermore, blurring is applied to the expanded regions to generate the blurred regions discussed above in conjunction with images 64A and 64B. Additionally, for each pixel in the blurred region, a maximum value is determined from the input image and the blurred region to generate a maximum value image, and an inverse filter is applied to this maximum value image.

[0078] Furthermore, as discussed in the local method, it is possible that, in order to determine a subset of pixels, an inverse filter is used to transform the range of possible pixel values ​​projectable by the projection device into new values ​​larger than the range of possible pixel values. Additionally, a range function is determined, which is used to map the new value range to an adapted range of pixel values ​​within the possible value range. This was discussed above as part of the local method for determining a subset of pixels, where the first ghost of the input image is determined by applying the inverse filter to the input image and inverting the result of the application of the inverse filter to generate a maximum value image such as image 72b. Furthermore, the second ghost of the input image is determined by applying the inverse filter to the negative portion of the input image to generate a minimum value image as shown in 72a. Furthermore, the first and second ghosts are amplified by neighboring pixels located within a pixel-defined distance from the first and second ghosts to generate an expanded region including the first and second ghosts and neighboring pixels, as discussed above in conjunction with images 73a and 73a. The first and second ghosts are then amplified by neighboring pixels located within a pixel-defined distance to generate an expanded region including the first and second ghosts and their neighboring pixels.

[0079] Furthermore, it is possible to generate at least one out-of-range image based on the input image and an inverse filter, the out-of-range image having an image intensity value greater than or less than the possible intensity values ​​projected by the projection device. Then, a range image can be determined based on the at least one out-of-range image having an image intensity value within a smaller range of image intensity values ​​than the maximum range defined by the possible intensity values ​​projected by the projection device. Furthermore, an inverse filter is then applied to an offset image generated based on the range image and including image intensity values ​​within a smaller range than the possible intensity range, in order to determine the modulated image, wherein the application of the inverse filter increases the intensity values ​​of the offset image to the maximum range. This has already been discussed above in conjunction with local methods.

[0080] The generation of at least one out-of-range image may include applying an inverse filter to an input image and determining a first out-of-range image with intensity values ​​greater than possible intensity values ​​and a second out-of-range image with intensity values ​​less than possible intensity values, wherein the range is determined based on the first and second out-of-range images. Then, an expansion operation may be applied to pixels in the first out-of-range image with intensity values ​​greater than possible intensity values ​​to generate a first expanded out-of-range image, and an expansion operation may be applied to pixels in the second out-of-range image with intensity values ​​less than possible intensity values ​​to generate a second expanded out-of-range image, wherein a blurring filter is applied to the first and second out-of-range images to generate a blurred first and blurred second out-of-range images, and a range image is determined based on the first and second blurred out-of-range images.

[0081] The offset image can be determined based on the first blurred image, the second blurred image, and the range image.

Claims

1. A method for providing image data for projection onto a windshield, the method comprising, at an image processing unit (100): - Determine the inverse filter function u to use to remove ghosting artifacts that appear when the image data is projected onto the windshield. - Determine the first image f to be projected onto the windshield. - Determine the ghosting function, which represents the ghosting artifact, causing the first image to be projected to be distorted when projected onto the windshield. - Apply the inverse filter to the first image f to generate a filtered image. - Provide the filtered image to a projection device, the projection device being configured to project the filtered image onto the windshield.

2. The method of claim 1, wherein the inverse filter is applied before the filtered image is provided to the projection device.

3. The method of any of the preceding claims, wherein the first image f is a modulated image, wherein the brightness values ​​of at least some of the pixels of the input image are adjusted relative to the input image, the input image including content to be displayed on the windshield.

4. The method of claim 3, wherein the modulated image has a lower contrast compared to the input image between the brightest possible pixel value and the lowest possible pixel value.

5. The method of claim 3 or 4, wherein the contrast is reduced for all pixels of the input image.

6. The method of claim 3 or 4, wherein the brightness value of the modulated image is increased only for a subset of the pixels of the input image rather than for all pixels, wherein the location of the subset of pixels in the modulated image is determined based on the location of the ghost artifact.

7. The method of claim 6, wherein determining the subset of pixels comprises: - Identify ghosting regions in the input image, wherein the ghosting artifacts alter the pixel values ​​of the input image. - The ghosting region is increased by using neighboring pixels located within a defined distance from pixels in the ghosting region, so as to generate an expanded region including the ghosting region and the neighboring pixels. - Apply blurring to the expanded region to generate a blurred region. - For each pixel in the blurred region, a maximum value is determined from the input image and the blurred region to generate a maximum value image, wherein the inverse filter is applied to the maximum value image.

8. The method of claim 6, wherein determining the subset of pixels comprises using the inverse filter to convert the range of possible pixel values ​​that the projection device can project into a new range of values ​​greater than the range of possible pixel values, and determining a range function to map the new range of values ​​to an adaptation range having pixel values ​​within the range of possible values.

9. The method of claim 8, wherein determining the subset of pixels comprises: - The first ghosting of the input image is determined by applying the inverse filter to the input image and inverting the result of the application of the inverse filter to generate a maximum value image. - The second ghosting of the input image is determined by applying the inverse filter to the negative portion of the input image to generate a minimum image. - The first and second ghosts are increased by adjacent pixels located within the pixel definition distance of the first and second ghosts, so as to generate an expanded region including the first and second ghosts and the adjacent pixels.

10. The method of any of the preceding claims, further comprising: - Generate at least one out-of-range image based on the input image and the inverse filter, wherein the at least one out-of-range image has an image intensity value that is greater than or less than the possible intensity value that the projection device can project. - Determine the range image based on at least one out-of-range image having image intensity values ​​within a smaller range than the maximum range defined by the possible intensity values. - The inverse filter is applied to an offset image generated based on the range image and including image intensity values ​​in the smaller range, in order to determine the modulated image, wherein the application of the inverse filter increases the intensity values ​​of the offset image to the maximum value range.

11. The method of claim 10, wherein generating the at least one out-of-range image comprises applying the inverse filter to the input image and determining a first out-of-range image with an intensity value greater than the possible intensity value and a second out-of-range image with an intensity value less than the possible intensity value, wherein the out-of-range image is determined based on the first out-of-range image and the second out-of-range image.

12. The method of claim 11, wherein an expansion operation is applied to pixels of the first out-of-range image with intensity values ​​greater than the possible value to generate a first expanded out-of-range image, and an expansion operation is applied to pixels of the second out-of-range image with intensity values ​​less than the possible intensity value to generate a second expanded out-of-range image, wherein a blurring filter is applied to the first out-of-range image and the second out-of-range image to generate a blurred first out-of-range image and a blurred second out-of-range image, wherein the range image is determined based on the first blurred first out-of-range image and the second blurred second out-of-range image.

13. The method of claims 10 to 12, wherein the offset image is determined based on the first blurred image, the second blurred image, and the range image.

14. An image processing unit, comprising a memory and at least one processor, the memory containing instructions executable by the at least one processor, wherein the image processing unit is configured to... - Determine the inverse filter function u to use to remove ghosting artifacts that appear when the image data is projected onto the windshield. - Determine the first image f to be projected onto the windshield. - Determine the ghosting function, which represents the ghosting artifact, causing the first image to be projected to be distorted when projected onto the windshield. - Determine the inverse filter function u to use to remove the ghosting artifacts. - Apply the inverse filter to the first image f to generate a filtered image. - Provide the filtered image to a projection device, the projection device being configured to project the filtered image onto the windshield.

15. The image processing unit of claim 14, further configured to apply the inverse filter before providing the filtered image to the projection device.

16. The image processing unit of claim 14 or 15, further configured to provide the first image f as a modulated image, wherein the brightness values ​​of at least some of the pixels of the input image are adjusted relative to the input image, the input image including content to be displayed on the windshield.

17. The image processing unit of claim 16, wherein the modulated image has a lower contrast compared to the input image between the brightest possible pixel value and the lowest possible pixel value.

18. The image processing unit of claim 16 or 17, further configured to reduce the contrast in the modulated image such that the contrast is reduced relative to the input image for all pixels.

19. The image processing unit of claim 16 or 17, further configured to increase the brightness value of the modulated image for only a subset of the pixels of the input image rather than for all pixels of the input image, wherein the location of the subset of pixels in the modulated image is determined based on the location of the ghost artifact.

20. The image processing unit of claim 19, further configured to determine the subset of pixels: - Identify ghosting regions in the input image, wherein the ghosting artifacts alter the pixel values ​​of the input image. - The ghosting region is increased by using neighboring pixels located within a defined distance from pixels in the ghosting region, so as to generate an expanded region including the ghosting region and the neighboring pixels. - Apply blurring to the expanded region to generate a blurred region. - For each pixel in the blurred region, a maximum value is determined from the input image and the blurred region to generate a maximum value image, wherein the inverse filter is applied to the maximum value image.

21. The image processing unit of claim 19, further configured to, in order to determine the subset of pixels, use the inverse filter to convert the range of possible pixel values ​​that the projection device can project into a new range of values ​​greater than the range of possible pixel values, and determine a range function to map the new range of values ​​to an adaptation range having pixel values ​​within the range of possible values.

22. The image processing unit of claim 21, further configured to determine the subset of pixels: - The first ghosting of the input image is determined by applying the inverse filter to the input image and inverting the result of the application of the inverse filter to generate a maximum value image. - The second ghosting of the input image is determined by applying the inverse filter to the negative portion of the input image to generate a minimum image. - The first and second ghosts are increased by adjacent pixels located within the pixel definition distance of the first and second ghosts, so as to generate an expanded region including the first and second ghosts and the adjacent pixels.

23. A display system configured to project image data onto a vehicle's windshield, comprising: A projection device is configured to project a filtered image onto the windshield; And the image processing unit as described in any one of claims 14 to 22.

24. A computer program comprising program code executed by at least one image processing unit, wherein execution of the program code causes the at least one image processing unit to perform the method as claimed in any one of claims 1 to 13.

Images