System and method for displaying an image on a windshield

Abstract

The method for displaying an image on a windshield of a vehicle comprises the following steps: acquiring at least one physical property of ambient light incident on the windshield of the vehicle by a light sensor unit; generating an image to be displayed on the windshield of the vehicle by a processor unit; based on the determination of the at least one physical property of the ambient light incident on the windshield of the vehicle, individually adjusting the intensity of each image pixel according to the at least one physical property of the ambient light by the processor unit; and displaying the image by a transparent display module integrated in the windshield of the vehicle.

Description

Technical Field

[0001] The present invention relates to systems and methods for displaying images on a windshield of a vehicle. In particular, the present invention relates to systems and methods for displaying images on a windshield of a vehicle by using a transparent display module integrated into the windshield of the vehicle. Background Art

[0002] Currently, road vehicles are often equipped with a head-up display (HUD) to assist the driver under various driving conditions by displaying specific information related to the vehicle or the vehicle's environment (e.g., road signs, other vehicles or objects in the vehicle's environment, etc.).

[0003] A head-up display is a transparent graphic display that displays visual information to a user without the user having to look away from his or her usual viewpoint. Today, head-up displays are commonly used in vehicles, in which a transparent surface is arranged in front of the driver and the information to be displayed is projected from a projection device to the transparent surface by an optical system. Current HUDs can be divided into two categories: windshield HUDs and combination HUDs. Windshield HUDs use the windshield of the vehicle as a projection surface, while combination HUDs use an additional transparent screen, the so-called combination screen, as a projection surface. Although combination screen HUDs are more common in road vehicles due to their lower overall cost, they have the particular disadvantage of a smaller screen size of the projection surface. The smaller projection surface of the combination screen allows the display of smaller-sized texts and symbols, or less information.

[0004] A disadvantage of a windshield HUD is that it may produce artifacts because, on the one hand, the projected image appears on the inside of the windshield, and, on the other hand, due to the thickness of the windshield, the projected image also appears at a slightly offset position on the outside of the windshield. Figure 1 This phenomenon is schematically illustrated in FIG. 1 . When the projection device 10 projects an image 12 onto the inner surface of the windshield 20, most of the emitted light is reflected from the inner surface of the windshield 20, thereby generating a primary (real) image 12 of the image on the windshield. However, a small portion of the emitted light enters the windshield and is reflected from the outer surface of the windshield 20, which generates a secondary image 13 of the projected image, called an artifact. The artifact may disturb the driver of the vehicle and make the driver feel tired.

[0005] Another disadvantage of windshield HUD is that during the day, especially when the sun is shining directly on the windshield, the projected image is not very obvious or even not visible at all to the driver. Figure 1This phenomenon is also schematically illustrated in . When incident light 30 from a high intensity external lighting source (such as sunlight) reaches the outer surface of the windshield 20, only a small portion of the incident light 30 is reflected from the surface as reflected light 31, while most of the incident light 30 passes through the windshield 20, making the inside of the windshield very bright. Because the windshield is too bright, the relative intensity of the projected image is significantly reduced, which may make it difficult for the driver to see the image. Although the brightness of the windshield can be reduced by using a blackout film on the inside of the windshield, the permanent use of such a blackout film is unfavorable (and dangerous) for night driving.

[0006] A large number of documents disclose head-up display systems for vehicles. For example, document US 2005 / 0154505 A1 describes a head-up display that uses the entire windshield as a display area. The head-up display system includes a video projector, a screen, a windshield, and a Fresnel mirror. The Fresnel mirror magnifies the image from the screen so that the driver can visually recognize the image through his or her eyes.

[0007] In view of the above-mentioned problems with currently used HUDs, there is a need for an improved display system and method for displaying images on a windshield that eliminates the adverse effects of external light during daytime operation and eliminates the undesirable optical effects of traditional vehicle windshield display systems, such as artifacts. Summary of the invention

[0008] It is therefore an object of the present invention to provide a display system and method for displaying an image on a vehicle windshield that individually adjusts image pixel intensity based on one or more physical properties of ambient light incident on the vehicle windshield.

[0009] Another object of the present invention is to provide a vehicle display system and method, which uses a transparent display module integrated into the windshield of the vehicle, wherein the transparent display module includes a plurality of transparent pixel units and a plurality of transparent inter-pixel regions.

[0010] These and other objects are achieved by providing a method for displaying an image on a windshield of a vehicle, the method comprising:

[0011] - acquiring, by means of a light sensor unit, at least one physical property of ambient light incident on a windshield of the vehicle;

[0012] - generating, by means of a processor unit, an image to be displayed on a windshield of the vehicle;

[0013] - based on the determination of the at least one physical property of the ambient light incident on the windshield of the vehicle, individually adjusting, by the processor unit, the intensity of each image pixel according to the at least one physical property of the ambient light; and

[0014] -Displays images through a transparent display module integrated into the vehicle's windshield.

[0015] The above object is also achieved by providing a method for displaying an image on a windshield of a vehicle, the method comprising:

[0016] - acquiring, by means of a light sensor unit, at least one physical property of ambient light incident on a windshield of the vehicle;

[0017] - capturing a first image of the environment in front of the vehicle by a first camera device;

[0018] - capturing a second image of the eyes of a user, in particular a vehicle driver, by means of a second camera device;

[0019] - determining, by the processor unit based on the second image, a viewing point of the passenger with respect to the windshield;

[0020] - mapping the first image onto at least a portion of the surface of the windshield by the processor unit, thereby generating a base image;

[0021] - analyzing, by a processor unit, the content of the first image;

[0022] - generating an image object based on the content of the first image;

[0023] - fitting the image object to the base image by a processor unit;

[0024] - generating, by the processor unit, a third image to be displayed based on the fitted image object;

[0025] - individually adjusting, by the processor unit, the intensity of each pixel of the image according to the at least one physical property of the ambient light incident on the windshield; and

[0026] - Displaying the third image via a transparent display module integrated into the windshield of the vehicle.

[0027] Finally, the above object is also achieved by providing a system for displaying an image on a windshield of a vehicle, the system comprising:

[0028] - a light sensor unit for acquiring at least one physical property of ambient light incident on a windshield of the vehicle and for providing ambient light information;

[0029] - a processor unit configured to generate an image to be displayed and further configured to individually adjust the intensity of each image pixel according to ambient light information;

[0030] - A transparent windshield display unit, used for displaying the image on the windshield of the vehicle, the display unit comprising two transparent protective layers and a transparent display module located between the two transparent protective layers.

[0031] The present invention also provides a computer program product, comprising instructions, which, when executed by a processor device, cause the processor device to perform the steps of the above method. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The present invention will now be described in more detail with reference to the accompanying drawings, in which:

[0033] Figure 1 Disadvantageous optical phenomena of a prior art windshield head-up display are schematically illustrated.

[0034] Figure 2 FIG. 4 is a schematic block diagram showing several main components of a display system according to the present invention.

[0035] Figure 3A The light transmission of an LCD-type windshield display used in the system and method according to the present invention is schematically shown.

[0036] Figure 3B The light transmission of an LED-type windshield display used in the system and method according to the present invention is schematically shown.

[0037] Figure 4 FIG. 1 is a flow chart showing the main steps of the first method of the present invention.

[0038] Figure 5 Several major components of a particular embodiment of a display system according to the present invention are shown, which display system can be used to present augmented reality.

[0039] Figure 6 FIG. 4 is a flow chart showing the main steps of the second method according to the present invention. DETAILED DESCRIPTION

[0040] Figure 2The main components of the display system 200 according to the present invention are shown in the schematic block diagram of . Accordingly, the display system 200 includes a light sensor unit 210, which is used to obtain at least one physical property of ambient light incident on the vehicle windshield. In one embodiment of the system, the ambient light sensor unit may include one or more light sensors, and the one or more light sensors are suitable for measuring the average ambient light intensity. In another embodiment of the system, the light sensor unit may include a camera, which is suitable for capturing the environment in front of the vehicle; the light sensor unit may also include a processor device, which is used to process the captured image and provide processing information about at least one physical property of the ambient light based on the captured image. The physical properties of the ambient light may include the average ambient light intensity, the light intensity distribution within the captured image of the environment in front of the vehicle, the color distribution within the captured image of the environment in front of the vehicle, etc.

[0041] The system 200 also includes a processor unit 220. The processor unit 220 is configured to process the ambient light information received from the light sensor unit 210 and to generate an image to be displayed by the transparent windshield display unit 230. The image generated by the processor unit 220 may include one or more image objects, a background, or a real view of at least a portion of the environment in front of the vehicle.

[0042] An image object is a graphical representation of any one or more of the following: a plurality of operating features of a vehicle, a plurality of traffic-related features of a vehicle, a plurality of visible or invisible objects in front of the vehicle, etc. For example, an image object may represent the speed of a vehicle, the speed of a vehicle's engine, a display on a vehicle dashboard, a traffic sign, a traffic light, any type of navigation object, a view of another vehicle in the vehicle's environment, any type of stationary or moving object (partial or full view, or an outline thereof) in the vehicle's environment, a visual effect, a darkened or blanked area, any type of object (partial or full view, or an outline thereof) that is not visible to a user (driver or passenger) of the vehicle, etc.

[0043] The background of the image is defined as the area of ​​the displayed image containing image pixels other than image pixels of the at least one image object.Preferably, the background of the image consists of one or more background objects.

[0044] The processor unit 220 is used to individually adjust the intensity of each image pixel according to the ambient light information received from the light sensor unit 210, which will be described in detail later.

[0045] The image generated by the processor unit 220 is displayed on the transparent windshield display unit 230. The transparent windshield display unit 230 includes a transparent display module, which is integrated into the windshield of the vehicle. The transparent display module includes a plurality of transparent pixel units and a plurality of transparent inter-pixel regions. The transparent display module can be an LCD display module or an LED display module. In the LCD display module, the intensity of the corresponding pixel unit is individually adjusted by controlling the amount of backlight passing through each pixel unit. In the LED display module, the intensity of the corresponding pixel unit is individually adjusted by controlling the intensity of the LED belonging to each pixel unit.

[0046] Processor unit 220 determines the intensity and color of the plurality of image pixels forming the image itself based on the content of the image to be displayed and one or more physical properties of the ambient light incident on the windshield.

[0047] Figure 3A Schematically shows the light transmission of a transparent windshield display unit of LCD type. This type of windshield display unit 230 may include two transparent protective layers 232 and an LCD display module 234 integrated between the two transparent protective layers 232. Preferably, the two protective layers 232 are made of glass.

[0048] The LCD display module 234 includes a plurality of color pixel units 236 and a plurality of inter-pixel regions 238 between the pixel units 236. In the context of the present invention, the term "pixel unit" is used as a pixel of the display module (i.e., in terms of hardware), and the term "image pixel" or "pixel" is used as a pixel of an image to be displayed (i.e., in terms of image processing).

[0049] like Figure 3A As shown, the inter-pixel region 238 allows incident light to pass through the windshield display unit 230 without substantial loss (eg, reflection or absorption), so the intensity I of the light passing through the windshield display unit 230 is IN (a) The intensity I of the incident light irradiating the windshield display unit 230 from the outside of the vehicle EX Basically the same. In this example, the light intensity can be defined as the average light intensity along the entire outer surface or the entire inner surface of the windshield. In this example, the backlight of the LCD display module 234 can also use ambient light.

[0050] The intensities may be determined to be different for the pixels of the plurality of image objects and the pixels of the image background so that a desired level of intensity contrast (also referred to as “contrast”) between the plurality of image objects and other portions of the image displayed by the windshield display unit 230 may be achieved at any time, regardless of the physical properties of the ambient light incident on the windshield (e.g., average light intensity, light distribution, color distribution, etc.). For example, during the day, the light (denoted as I ) allowed to pass through the windshield display unit 230 at the plurality of pixel units displaying the plurality of pixels of an image object may be different from the light allowed to pass through the windshield display unit 230 at the plurality of pixel units displaying the plurality of pixels of an image object. IN (b)), more light than that allowed to pass through the windshield display unit 230 at a plurality of pixel units for displaying a plurality of pixels of the image background (denoted as I IN (c)). The light intensity of the pixels of these image objects is IN (b) and the light intensity I of the pixels of the background objects IN (c) can be determined by the processor unit 220 according to the ambient light intensity I EX The pixel intensity level may also be determined such that the image object pixels are darker than the background object pixels, for example, when the view of very bright objects in the vehicle environment is dimmed or completely blanked out through the windshield display unit 230, the pixel intensity level may be determined such that the image object pixels are darker than the background object pixels.

[0051] Figure 3B Schematically shows the light transmission of an LED type transparent windshield display unit. This type of windshield display unit 230 may include two transparent protective layers 232 and an LED display module 334 integrated between the two transparent protective layers 232. Preferably, the two protective layers 232 are made of glass. The LED display module 334 includes a plurality of color pixel units 336 and a plurality of inter-pixel regions 338 between the pixel units 336.

[0052] like Figure 3B As shown, the inter-pixel region 338 allows incident light to pass through the windshield display unit 230 without substantial loss (eg, reflection or absorption), so the intensity I of the light passing through the windshield display unit 230 is IN (a) The intensity of the ambient light incident on the windshield I EX In this example, the light intensity can be defined as the average light intensity along the entire outer surface or the entire inner surface of the windshield. Due to the presence of the LED display module 334, most of the ambient light will pass through these transparent inter-pixel areas 338.

[0053] The intensities of the pixels of the image objects and the pixels of the background objects may be determined to be different so that a desired level of contrast between the image objects and other parts of the image displayed by the windshield display unit 230 can be achieved at any time, regardless of the physical properties of the ambient light incident on the windshield (e.g., average light intensity, light distribution, color distribution, etc.). For example, during the day, the intensity of the LED of the pixel unit displaying the image object (denoted as I IN (b)), set to be greater than the intensity of the pixel unit showing the background object (denoted as I IN (c)) is higher. The light intensity of the image object pixel I IN (b) and the light intensity I of the background object pixel IN (c) are all processed by the processor unit 220 according to the incident ambient light intensity I EX The pixel intensity levels may also be determined such that the image object pixels are darker than the background object pixels, for example, when the view of very bright objects in the vehicle environment is dimmed or completely blanked out through the windshield display unit 230, the pixel intensity levels may be determined such that the image object pixels are darker than the background object pixels.

[0054] It is noted that the transparency of the inter-pixel region is usually much higher than the transparency of the pixel unit of the transparent display module. It is particularly preferred that the transparency of the inter-pixel region is approximately (or completely) the same as the transparency of the protective glass layer of the windshield. Since electronic components are involved in the pixel units of the transparent display module, the transparency of the pixel units of the display module is lower, or sometimes significantly lower, than the transparency of the inter-pixel region. In addition, assuming that the display module is of LCD type, the transparency of the pixel unit can even be changed (e.g., increased or decreased) by electronically adjusting the light intensity of the pixel unit according to the image to be displayed. Unlike LCD-type transparent display modules, the transparency of the pixel unit of LED-type display modules is approximately constant, and the pixel intensity is adjusted based on the image to be displayed by the intensity of the built-in light source (i.e., LED) of the pixel unit. However, in either case, the overall transparency of the windshield display unit depends on the transparency of the inter-pixel region of the transparent display module and the (stable or adjustable) transparency of the pixel unit. For road vehicles such as cars, trucks, etc., the minimum transparency of the windshield is set to about 60-70% due to traffic safety regulations, which means that the transparent display module used in the display system of the present invention should also meet this requirement.

[0055] Now refer to Figure 4 A method of displaying an image by the display system introduced above is described in detail.

[0056] As a first step 400 of the first method according to the present invention, at least one physical property of ambient light incident on a windshield of a vehicle is acquired by a light sensor unit, such as a light detector, a radar detector, an ultrasonic detector, or a camera device.

[0057] As described above, the physical properties of the incident ambient light may include the average ambient light intensity in the vehicle environment, the light intensity distribution within an image captured by a camera, the color distribution within an image captured by a camera, etc.

[0058] Next, in step 410, the processor unit generates an image to be displayed on the vehicle windshield. The content of the image may include one or more image objects and one or more background objects. An image object is a visual (or graphical) representation of any one or more of the following: multiple operating features of the vehicle, multiple traffic-related features of the vehicle, multiple visible or invisible objects in front of the vehicle, etc. Several examples of image objects have been mentioned above. A background object is defined as any type of image content that cannot be considered as an image object, that is, image content that does not carry information related to the vehicle or the vehicle environment. Background objects include areas of the displayed image that contain image pixels other than image pixels of at least one image object. Background objects typically may include solid color areas, color transition areas, texture areas, etc.

[0059] Based on the determination of the at least one physical property of the ambient light incident on the vehicle windshield, in step 420, the intensity of the pixel within the image is individually adjusted for each image pixel by the processor unit according to the at least one physical property of the ambient light incident on the windshield.

[0060] The processor unit processes the information obtained by at least one sensor device and / or measuring device to generate an image to be displayed on the windshield. As described above, the generated image may include one or more image objects, a graphic (or real-view) representation of a portion of the environment in front of the vehicle, and one or more background objects, the one or more image objects representing the detected or measured information in a graphical form, and the one or more background objects are used to adjust the contrast of the image object under various ambient light conditions.

[0061] In one embodiment of the method, the intensity of the image object pixels and the intensity of the background object pixels are determined so that the average intensity of the background object and the inter-pixel area surrounding the image object provides the image object with a desired level of intensity contrast C relative to the background object and the inter-pixel area. Accordingly, the contrast C of the image object can be expressed by the following equation (using Figure 3A and Figure 3B Symbols):

[0062]

[0063] Among them, I IN (b) is the average pixel intensity of the pixels of the image object, I IN (c) is the average pixel intensity of one or more background objects surrounding the image object, I IN (a) is the average light intensity of multiple pixel areas around the image object.

[0064] Preferably, the image to be displayed on the windshield of the vehicle is generated so that the contrast C of any image object within the image is at least 500: 1, preferably at least 1000: 1. Within these ranges of contrast C, a viewer such as a driver of the vehicle can clearly see the image objects on the windshield.

[0065] It should be noted that the light intensity of the inter-pixel area perceived by a user (e.g., a driver) may depend on the time of day (e.g., daytime, nighttime, dusk, dawn), on the weather (e.g., sunny, cloudy, rainy, foggy, etc.), and on the specific optical properties of the environment in which the vehicle is currently moving or staying (e.g., reflective or bright objects, shade, tunnel, etc.).

[0066] In order to maintain a desired level of contrast for an image object, various adjustments may be made to the pixel intensity of the image, for example, as shown in Table 1 below.

[0067]

[0068] Table 1

[0069] In cases 1-3, the contrast of the image object is adjusted so that the viewer obtains useful information on the windshield at the most appropriate intensity, which means that the image object is displayed in a clearly recognizable form while the display of the entire image on the windshield enables the driver to drive the vehicle safely.

[0070] In the fourth case, when the vehicle is traveling on a county road with no or almost no oncoming vehicles, the displayed image is adjusted to adapt to night conditions. In this case, the image object is displayed only at a low intensity, thereby providing a so-called "looming" display mode. In this case, for an LCD-type transparent display module, it may be necessary to use an additional light source, such as a lighting unit integrated into the periphery of the LCD-type display module, to produce the backlight required to display the image object. Obviously, for an LED-type transparent display module, such additional lighting is not required.

[0071] In a fifth case, a darkened image object may be displayed on the windshield of a vehicle to darken (or completely blank out) the area of ​​the windshield where the user (driver or passenger) may perceive interference from high intensity light from another oncoming vehicle. The image pixels of this darkened image object are set to have zero intensity within the displayed image.

[0072] Finally, in step 430, the image is displayed on the windshield of the vehicle through a transparent display module integrated in the windshield of the vehicle, wherein the transparent display module includes a plurality of transparent pixel units and a plurality of transparent inter-pixel regions.

[0073] In one embodiment of the method, one or more sensor devices and / or measuring devices may be used to obtain information about at least one physical characteristic of the vehicle and / or the vehicle's external environment. The sensor device may include, for example, a light detector, a radar detector, an ultrasonic detector, a camera device, a GPS position sensor, etc. The measuring device may be, for example, a rate meter, a tachometer, a thermometer, etc. In this embodiment, the processor unit also uses the sensor data and / or measurement data of the at least one sensor device or measuring device to generate an image to be displayed on the windshield of the vehicle through the integrated transparent display module.

[0074] In one embodiment of the method, an LCD type display module is used as the transparent display module. In this embodiment, ambient light can be used as the backlight of the display module, or a light source (e.g., an LED unit) integrated in the periphery of the display module can be used to generate backlight for the display module, or ambient light and an integrated light source can be used simultaneously to provide backlight for the LCD type display module.

[0075] Alternatively, in another embodiment of the method, an LED type display module is used as the transparent display module. In this embodiment, since the LED light source of the display pixel unit itself generates light for displaying an image, a backlight is not required to control the display.

[0076] Preferably, the image is displayed as a moving picture. In particular, one or more image objects may be displayed as a moving picture, i.e. the content of the image object may change over time. For example, the image object may appear as a graphical representation of a speedometer indicating the speed of the vehicle, or may display a navigation screen showing a road map and other navigation information.

[0077] When the image object is presented as a moving picture, it is particularly preferred that the image object is displayed in real time in response to changes in the sensed or measured physical characteristics or environmental information related to the vehicle. For example, an image object showing a speedometer may be displayed to show the instantaneous speed of the vehicle, or a navigation screen that updates a road map and other navigation information in real time may be displayed.

[0078] In a preferred embodiment of the system and method of the present invention, an augmented reality (AR) image may be displayed on the windshield of a vehicle. A specific embodiment of the display system according to the present invention may be used to generate an augmented reality image, which will be described below with reference to Figure 5 Describe it. Figure 5 The main components of this embodiment of the display system are depicted.

[0079] like Figure 5 As shown, the system 500 includes a first camera device 510 for capturing images of the environment in front of the vehicle. The viewing angle and other optical parameters (e.g., focal length, resolution, etc.) of the first camera device 510 may depend on the position of the first camera device 510 relative to the vehicle. The image quality of the first camera device 510, for example, may depend on the multiple filters used in front of the lens (e.g., UV filters, anti-glare coatings, etc.), the quality of the materials of the glass and these coatings, etc. The first camera device 510 may include a conventional video camera, an infrared camera, an ultraviolet (UV) camera, or a radar camera, etc.

[0080] The system 500 also includes a second camera device 520 for capturing images of the eyes of a user (e.g., a driver or a passenger) in the vehicle. The first camera device 510 and the second camera device 520 are also coupled to the processor unit 220 (e.g., Figure 2 As shown), the processor unit 220 is also configured to process the image of the second camera device 520 to determine the user's eyes in the predetermined reference coordinate system CS b A three-dimensional position in .

[0081] In system 500, the coordinate system CS b and the three-dimensional surface of the windshield to fix the position and orientation of the second camera device 520, more specifically, to map the integrated display module to the reference coordinate system CS b , so that the processor unit 220 can determine the relative position of the user's eyes with respect to the windshield based on the image of the second camera device 520. In addition, the processor unit 220 can also determine the direction of the user's line of sight based on the image captured by the second camera device. Based on these parameters, the processor unit 220 is able to reconstruct the field of view that the user can observe through the vehicle windshield. Once the field of view is determined, the processor unit 220 can map the objects in the field of view onto the three-dimensional surface of the windshield, or onto at least a portion of the windshield in front of the user (driver or passenger). Therefore, the mapped view of the environment that the user (driver or passenger) can see through the windshield will serve as a base image in order to fit any image objects to a specific portion of the base image.

[0082] In this embodiment, the processor unit 220 is also configured to fit the image object to the base image so that the user (driver or passenger) can see the relevant image object on the windshield as if the image object is part of the vehicle environment. The technology for generating such AR image objects is well known in the art, so its calculation details are omitted here. It should also be noted that in this embodiment of the system, since the base image (i.e., the view of the environment in front of the vehicle) can be seen through the transparent windshield of the vehicle, the base image itself does not need to be displayed. However, it is also feasible that the windshield display unit also displays at least a portion of the base image based on the image captured by the first camera device, preferably displaying the at least a portion at a lower intensity than the AR image object.

[0083] The optical sensor unit 210 and the transparent windshield display unit 230 of the system 500 are similar to those described above. Figure 2 Same description done.

[0084] Preferably, the AR images are presented in real time in response to changes in the position of the vehicle relative to its environment as it travels.

[0085] The following will refer to Figure 6 A second method according to the present invention using the above-mentioned display system is introduced.

[0086] As a first step 600 of the method, at least one physical property of the ambient light incident on the vehicle windshield is acquired by a light sensor unit. This step corresponds to the above reference Figure 4 Step 400 of the method described.

[0087] In step 602, a first image of an environment in front of a vehicle is captured by a first camera device.

[0088] Next, in step 604, a second image of the eyes of a user (e.g., a driver or a passenger) is captured by a second camera device. In step 606, a viewing point of the user is determined based on the captured second image by a processor unit, and optionally, a sight line direction of the user with respect to the windshield is also determined based on the captured second image by the processor unit.

[0089] Then, in step 608, the first image of the environment captured by the first camera device is mapped to the three-dimensional surface of the windshield, or to at least a portion of the windshield close to the user (driver or passenger). The surface of the windshield can be defined in a predetermined reference coordinate system of the display system. The first image of the environment mapped to the surface of the windshield will serve as a base image for subsequent image processing.

[0090] In the next step 610, the first image of the environment captured by the first camera device is processed to analyze the content of the first image. Then, in step 612, at least one image object is generated based on the content of the first image of the environment, wherein the at least one image object is to be added to the base image to generate an augmented reality image on the windshield, i.e., the third image. For example, the image object may include a lane in front of the car, any traffic objects around the vehicle, such as pedestrians, another car, intersections, etc.

[0091] Then, in step 614, the processor unit fits the at least one image object associated with the base image to the base image so that the user (driver or passenger) can see the at least one image object on the windshield as if the at least one image object is part of the environment in front of the vehicle. To fit the at least one image object to the base image, the processor unit uses the position of the user's eyes previously determined in step 606.

[0092] Then, in step 616, the processor unit generates a third image to be displayed based on the at least one image object fitted into the base image. The generated third image contains one or more fitted image objects and one or more background objects.

[0093] In step 620, the intensity of the plurality of image pixels is individually determined by the processor unit based on the at least one physical property of the ambient light incident on the windshield. This step is performed in the same manner as the above step 420. The physical property of the incident ambient light may include at least one of an average ambient light intensity in the vehicle environment, a light intensity distribution within an image captured by a camera, and a color distribution within an image captured by a camera.

[0094] Finally, in step 630, the third image is displayed on the windshield by the transparent display module without displaying the base image. Together with the at least one image object, at least one background object may also be generated and displayed on the transparent display module in a manner that the contrast of the at least one image object is within a desired range. By displaying the at least one image object on the windshield in a position suitable for a user (e.g., a driver or passenger) to see a view of the environment, an augmented reality may be presented through the transparent windshield display unit of the vehicle.

[0095] In a particularly preferred embodiment of the display method, the third image containing the fitted one or more image objects and the one or more background objects is a moving picture, in which the graphic representation of the fitted one or more image objects always follows the changes of the environment in the user's field of view in real time when the vehicle is driving.

[0096] The image object of the second method can be a graphical representation of at least one of the following: a plurality of operating features of the vehicle, a plurality of traffic-related features of the vehicle, a plurality of visible or invisible objects in front of the vehicle. The background object can include at least one of a solid color area, a color transition area, or a texture area.

[0097] In a second method, the intensity of the image object pixels and the intensity of the background object pixels are set so that the average intensity of the background object and the inter-pixel area surrounding the image object provides a preset level of contrast for the image object relative to the background object and the inter-pixel area of ​​the transparent display unit. Preferably, the contrast is set to at least 500:1, more preferably at least 1000:1.

[0098] In one embodiment of the display system according to the present invention, the transparent display module can be integrated into a substantial portion of a vehicle windshield, so that the driver and other passengers of the vehicle can use the windshield display unit at the same time. In this case, at least a portion of the transparent windshield display unit can be curved in one or two directions. Accordingly, the shape of the integrated transparent display module can be adapted to the shape of the three-dimensional surface of the windshield.

[0099] When the transparent display module has a curved portion that is curved in one or two directions, those parts of the image to be presented on the curved portion of the display module should be appropriately distorted so that the user (driver or passenger) can see the image appearing on the curved portion of the windshield without distortion. The required image transformation is performed by the processor unit, which is well known in the art, so its details are omitted here.

[0100] In one embodiment of the display method according to the present invention, the processor unit may optimize at least one of the following display properties of the transparent display module: transparency, saturation, contrast, color balance, etc.

[0101] The present invention also provides a computer program product, comprising instructions, which, when executed by a processor device, cause the processor device to perform the steps of the above method.

Claims

1. A method for displaying an image on a windshield of a vehicle, the method comprising: - acquiring (600) by means of a light sensor unit at least one physical property of ambient light incident on a windshield of the vehicle; - capturing (602) a first image of the environment in front of the vehicle by a first camera device; - capturing (604) a second image of the user's eye by a second camera device; - determining (606) by a processor unit a viewing point of a passenger with respect to the windshield based on the second image; - mapping (608) the first image by the processor unit onto at least a portion of the surface of the windshield, thereby generating a base image; - analyzing (610) by the processor unit the content of the first image; - generating (612) at least one image object based on the content of said first image; - fitting (614) the at least one image object to the base image by the processor unit; - generating (616) a third image to be displayed by the processor unit based on the fitted at least one image object; - individually adjusting, by the processor unit, the intensity of each pixel of the third image according to the at least one physical property of the ambient light incident on the windshield (620); - displaying (630) the third image via a transparent display module integrated into a windshield of the vehicle, the transparent display module comprising a plurality of transparent pixel units and a plurality of transparent inter-pixel regions; wherein the third image includes one or more background objects, and wherein, during use of the vehicle, the pixel intensity of the at least one image object and the pixel intensity of the one or more background objects are set so that the sum of the average pixel intensity of the one or more background objects and the average intensity of the inter-pixel area surrounding the at least one image object of the transparent display module provides a preset level of contrast for the at least one image object relative to the one or more background objects and the inter-pixel area of ​​the transparent display module.

2. The method according to claim 1, wherein: The optical sensor unit includes at least one of a light detector, a radar detector, an ultrasonic detector, and a camera device.

3. The method according to claim 1 or 2, wherein: The physical property of the incident ambient light includes at least one of an average ambient light intensity in the environment of the vehicle, a light intensity distribution within an image captured by a camera, and a color distribution within an image captured by a camera.

4. A method according to any one of the preceding claims, wherein: An image object is a graphical representation of at least one of: a plurality of operating features of the vehicle, a plurality of traffic-related features of the vehicle, a plurality of visible or invisible objects in front of the vehicle.

5. The method according to claim 1, wherein: The background object includes at least one of a solid color area, a color transition area, or a texture area.

6. The method according to claim 1, wherein: The contrast ratio is set to at least 500:

1.

7. The method according to claim 6, wherein: The contrast ratio is set to at least 1000:

1.

8. The method according to any one of claims 1 to 7, wherein: - the transparent display module comprises at least one curved portion curved in one or two directions, and - The step of displaying an image includes displaying a portion of the image in a distorted form on the curved portion of the transparent display module so that a user can perceive the portion of the image without distortion.

9. A method for displaying an image on a windshield of a vehicle, the method comprising: - acquiring (400) by means of a light sensor unit at least one physical property of ambient light incident on a windshield of the vehicle; - generating (410) by a processor unit an image to be displayed on a windshield of the vehicle; - based on the determination of the at least one physical property of the ambient light incident on the windshield of the vehicle, individually adjusting, by the processor unit, the intensity of each image pixel according to the at least one physical property of the ambient light (420); - displaying (430) an image via a transparent display module integrated into a windshield of the vehicle, the transparent display module comprising a plurality of transparent pixel units and a plurality of transparent inter-pixel regions; wherein the image comprises at least one image object and one or more background objects, and wherein, during use of the vehicle, the pixel intensity of the at least one image object and the pixel intensity of the one or more background objects are set so that the sum of the average pixel intensity of the one or more background objects and the average intensity of the inter-pixel area surrounding the at least one image object of the transparent display module provides a preset level of contrast for the at least one image object relative to the one or more background objects and the inter-pixel area of ​​the transparent display module.

10. The method according to claim 9, wherein: The optical sensor unit includes at least one of a light detector, a radar detector, an ultrasonic detector, and a camera device.

11. The method according to claim 9 or 10, wherein: The physical property of the incident ambient light includes at least one of an average ambient light intensity in the environment of the vehicle, a light intensity distribution within an image captured by a camera, and a color distribution within an image captured by a camera.

12. The method according to any one of claims 9 to 11, wherein: The image object is a graphical representation of at least one of: a plurality of operating features of the vehicle, a plurality of traffic-related features of the vehicle, a plurality of visible or invisible objects in front of the vehicle.

13. The method according to claim 11, wherein: The background object includes at least one of a solid color area, a color transition area, or a texture area.

14. The method according to claim 11, wherein: The contrast ratio is set to at least 500:

1.

15. The method according to claim 14, wherein: The contrast ratio is set to at least 1000:

1.

16. The method according to any one of claims 9 to 15, wherein: - the transparent display module comprises at least one curved portion curved in one or two directions, and - The step of displaying an image includes displaying a portion of the image in a distorted form on the curved portion of the transparent display module so that a user can perceive the portion of the image without distortion.

17. A system for displaying an image on a windshield of a vehicle, for executing the method according to any one of claims 1 to 16, the system comprising: - a light sensor unit (210) for acquiring at least one physical property of ambient light incident on a windshield of the vehicle and for providing ambient light information; - a processor unit (220) configured to generate an image to be displayed, and further configured to individually adjust the intensity of each image pixel according to the ambient light information; - A transparent windshield display unit (230), used for displaying the image on the windshield of the vehicle, the display unit comprising two transparent protective layers and a transparent display module located between the two transparent protective layers.

18. The system of claim 17, wherein: The transparent display module is one of an LCD type display module (234) and an LED type display module (334).

19. The system of claim 18, wherein: A light source is integrated on the periphery of the LCD display module (234).

20. A system according to any one of claims 17 to 19, wherein: The transparent windshield display unit (230) is curved in one or two directions.

21. A computer program product comprising instructions which, when executed by a processor device, cause the processor device to perform the steps of the method according to any one of claims 1 to 16.

Images