Dual image plane HUD with automatic illumination settings for AR graphics displayed in a distant virtual image plane

Abstract

The present invention relates to a dual image plane HUD with automatic illumination setting for AR graphics displayed in a distant virtual image plane. A projection system includes a projection device and a control module. The projection device is configured to (i) project a first image in a first virtual image plane, and (ii) project a second image in a second virtual image plane, wherein the second virtual image plane is farther from an eye box than the first virtual image plane. The control module is configured to: determine an ambient light brightness level; determine a first brightness level of the first virtual image plane based on the ambient light brightness level; determine a second brightness level of the second virtual image plane based on the first brightness level of the first virtual image plane; and control at least one projection device to (i) display the first image in a first virtual image plane having a first brightness level, and (ii) display the second image in a second virtual image plane having a second brightness level.

Description

Technical Field

[0001] The information provided in this section is for the purpose of generally presenting the context of the present disclosure. The work of the presently named inventors within the scope described in this section and aspects of this specification that may not have been prior art at the time of filing are neither explicitly nor implicitly admitted to be prior art to the present disclosure.

[0002] The present disclosure relates to a dual-plane projection system for a head-up display for a vehicle. Background Art

[0003] Display devices are used in a variety of applications. Some exemplary display devices are flat panel displays, projection displays, and heads-up displays. Display devices can be transmissive or reflective.

[0004] Traditionally, the driver of a vehicle views the surroundings of the vehicle through the windows, windshield, and other glass of the vehicle. The driver can control the acceleration, deceleration, and steering of the vehicle based on the driver's visual observation of the surroundings of the vehicle. The vehicle may include one or more displays that display various information to the driver. For example, some vehicles include an infotainment system that includes a display that displays various infotainment and other vehicle information. The vehicle may also include a head-up display (HUD) that displays information by forming a virtual image at a certain distance with the help of reflections from the windshield. For example, the HUD can display vehicle speed and other vehicle information (such as warnings such as lane departure warnings and collision avoidance warnings). Summary of the invention

[0005] A projection system is provided, and includes at least one projection device and a control module. The at least one projection device is configured to (i) project a first image in a first virtual image plane, and (ii) project a second image in a second virtual image plane, wherein the second virtual image plane is farther from an eye box than the first virtual image plane. The control module is configured to: determine an ambient light brightness level; determine a first brightness level of the first virtual image plane based on the ambient light brightness level; determine a second brightness level of the second virtual image plane based on the first brightness level of the first virtual image plane; and control the at least one projection device to (i) display the first image in the first virtual image plane having the first brightness level, and (ii) display the second image in the second virtual image plane having the second brightness level.

[0006] In other features, the projection system of technical solution 1 further includes an ambient light sensor configured to generate an output signal indicating the brightness level of the ambient light. The control module is configured to determine the brightness level of the ambient light based on the output signal.

[0007] In other features, the control module is configured to: determine a brightness threshold for cognitive capture based on an ambient light brightness level; and determine a second brightness level based on the brightness threshold.

[0008] In other features, the control module is configured to set the second brightness level of the second virtual image plane such that a contrast ratio between the second virtual image plane and the road surface is between 1.2:1 and 10:1.

[0009] In other features, the control module is configured to: determine whether the ambient light brightness level is less than a predetermined level associated with nighttime; and in response to detecting that it is nighttime, set a second brightness level of the second virtual image plane so that the contrast between the second virtual image plane and the road surface is between 1.2:1 and 10:1.

[0010] In other features, the control module is configured to: determine that the augmented reality application of the second virtual image plane is a navigation application; and based on the augmented reality application being a navigation application, set the second brightness level of the second virtual image plane to be less than or equal to at least one of: (i) a detected brightness level of the lane marking, or (ii) a predetermined brightness level of the lane marking.

[0011] In other features, the control module is configured to: determine an augmented reality application for the second virtual image plane; and based on the augmented reality application, perform at least one of the following: (i) set a second brightness level for the second virtual image plane, or (ii) select a grayscale from a plurality of available grayscales for the second virtual image plane.

[0012] In other features, the control module is configured to: determine whether the ambient light brightness level is greater than a predetermined level associated with daytime; and in response to detecting that it is daytime, limit the second brightness level of the second virtual image plane to less than or equal to 10,000 cd / m 2 .

[0013] In other features, the control module is configured to: determine whether the ambient light brightness level is greater than a predetermined level associated with daytime; and in response to detecting that it is daytime, control the second brightness level of the second virtual image plane to perform at least one of the following: (i) provide a nominal daytime brightness level of the second virtual image plane between 2400 cd / m 2 or (ii) making the contrast between the brightness level of the augmented reality graphic displayed in the second virtual image plane and the brightness level of the road surface be 1.2:1 to 10:1.

[0014] In other features, the control module is configured to: determine that the augmented reality application of the second virtual image plane is a navigation application; and based on the augmented reality application being the navigation application, control the second brightness level of the second virtual image plane to perform at least one of the following: (i) provide a nominal daytime brightness level of the second virtual image plane at 2400 cd / m 2 or (ii) making the contrast between the brightness level of the augmented reality graphic displayed in the second virtual image plane and the brightness level of the road surface be 1.2:1 to 10:1.

[0015] In other features, the control module is configured to set a second brightness level of the second virtual image plane so that a first ratio between the second brightness level of the second virtual image plane and the first brightness level of the first virtual image plane exists within a first brightness level range of the first virtual image plane and is greater than the first ratio within the second brightness level range of the first virtual image plane.

[0016] In other features, the control module is configured to permit a user to manually adjust a second brightness level of the second virtual image plane independent of a first brightness level of the first virtual image plane.

[0017] In other features, the control module is configured to prevent a user from manually adjusting a second brightness level of the second virtual image plane independent of a first brightness level of the first virtual image plane.

[0018] In other features, a vehicle system is provided and includes the projection system and an input device. The projection system is configured to project the first image and the second image at a window of the vehicle. The input device is configured to manually adjust a first brightness level of a first virtual image plane. The control module is configured to adjust the first brightness level of the first virtual image plane based on an output of the input device.

[0019] In other features, a method of operating a projection system is provided. The method includes: determining an ambient light brightness level; determining a first brightness level of a first virtual image plane based on the ambient light brightness level; determining a second brightness level of a second virtual image plane based on the first brightness level of the first virtual image plane, wherein the second virtual image plane is farther from an eye box than the first virtual image plane; and controlling the projection device to (i) project a first image in the first virtual image plane and (ii) project a second image in the second virtual image plane, wherein the first image is displayed in the first virtual image plane having the first brightness level, and wherein the second image is displayed in the second virtual image plane having the second brightness level.

[0020] In other features, the method further includes setting a second brightness level of the second virtual image plane to be less than or equal to at least one of: (i) a detected brightness level of the lane marking, or (ii) a predetermined brightness level of the lane marking.

[0021] In other features, the method further includes: determining whether the ambient light brightness level is less than a predetermined level associated with nighttime; determining that the augmented reality application of the second virtual image plane is a navigation application; and in response to detecting that it is nighttime and based on the augmented reality application being a navigation application, setting the second brightness level of the second virtual image plane to be less than or equal to at least one of: (i) a detected brightness level of the lane marking, or (ii) a predetermined brightness level of the lane marking.

[0022] In other features, the method further includes: determining an augmented reality application for the second virtual image plane; and based on the augmented reality application, performing at least one of the following: (i) setting a second brightness level for the second virtual image plane, or (ii) selecting a grayscale from a plurality of available grayscales for the second virtual image plane.

[0023] In other features, the method further comprises: determining whether the ambient light brightness level is greater than a predetermined level associated with daytime; and in response to detecting that it is daytime, limiting the second brightness level of the second virtual image plane to less than or equal to 5000 cd / m 2 .

[0024] In other features, the method further includes setting a second brightness level of the second virtual image plane so that a ratio between the second brightness level of the second virtual image plane and the first brightness level of the first virtual image plane is 2:1 within the first brightness level range of the first virtual image plane and is greater than 2:1 within the second brightness level range of the first virtual image plane.

[0025] The present invention also provides the following scheme:

[0026] Solution 1. A projection system comprising:

[0027] at least one projection device configured to (i) project a first image in a first virtual image plane and (ii) project a second image in a second virtual image plane, wherein the second virtual image plane is further from the eye box than the first virtual image plane; and

[0028] A control module, wherein the control module is configured to:

[0029] Determine the ambient light level,

[0030] determining a first brightness level of the first virtual image plane based on the ambient light brightness level,

[0031] determining a second brightness level of the second virtual image plane based on the first brightness level of the first virtual image plane, and

[0032] The at least one projection device is controlled to (i) display the first image in the first virtual image plane having the first brightness level, and (ii) display the second image in the second virtual image plane having the second brightness level.

[0033] Solution 2. The projection system according to Solution 1, further comprising an ambient light sensor configured to generate an output signal indicating the brightness level of the ambient light,

[0034] The control module is configured to determine the ambient light brightness level based on the output signal.

[0035] Solution 3. The projection system according to Solution 1, wherein the control module is configured to:

[0036] Determining a brightness threshold for cognitive capture based on the ambient light brightness level; and

[0037] The second brightness level is determined based on the brightness threshold.

[0038] Option 4. The projection system according to Option 1, wherein the control module is configured to set the second brightness level of the second virtual image plane so that a contrast ratio between the second virtual image plane and a road surface is between 1.2:1 and 10:1.

[0039] Solution 5. The projection system according to Solution 1, wherein the control module is configured to:

[0040] determining whether the ambient light brightness level is less than a predetermined level associated with nighttime; and

[0041] In response to detecting that it is nighttime, the second brightness level of the second virtual image plane is set so that the contrast between the second virtual image plane and the road surface is between 1.2:1 and 10:1.

[0042] Solution 6. The projection system according to Solution 5, wherein the control module is configured to:

[0043] determining that the augmented reality application of the second virtual image plane is a navigation application; and

[0044] Based on the augmented reality application being the navigation application, the second brightness level of the second virtual image plane is set to be less than or equal to at least one of: (i) a detected brightness level of a lane marking, or (ii) a predetermined brightness level of a lane marking.

[0045] Solution 7. The projection system according to Solution 1, wherein the control module is configured to:

[0046] determining an augmented reality application of the second virtual image plane; and

[0047] Based on the augmented reality application, at least one of the following is performed: (i) setting the second brightness level of the second virtual image plane, or (ii) selecting a grayscale from a plurality of available grayscales of the second virtual image plane.

[0048] Solution 8. The projection system according to Solution 1, wherein the control module is configured to:

[0049] determining whether the ambient light brightness level is greater than a predetermined level associated with daytime; and

[0050] In response to detecting that it is daytime, limiting the second brightness level of the second virtual image plane to less than or equal to 10000 cd / m 2 .

[0051] Solution 9. The projection system according to Solution 1, wherein the control module is configured to:

[0052] determining whether the ambient light brightness level is greater than a predetermined level associated with daytime; and

[0053] In response to detecting that it is daytime, controlling the second brightness level of the second virtual image plane to perform at least one of the following: (i) providing a nominal daytime brightness level of the second virtual image plane at 2400 cd / m 2 or (ii) making the contrast between the brightness level of the augmented reality graphic displayed in the second virtual image plane and the brightness level of the road surface be 1.2:1 to 10:1.

[0054] Embodiment 10. The projection system according to embodiment 9, wherein the control module is configured to:

[0055] determining that the augmented reality application of the second virtual image plane is a navigation application; and

[0056] Based on the augmented reality application being the navigation application, controlling the second brightness level of the second virtual image plane to perform at least one of the following: (i) providing the nominal daytime brightness level of the second virtual image plane to be between 2400 cd / m 2 or (ii) making the contrast between the brightness level of the augmented reality graphic displayed in the second virtual image plane and the brightness level of the surface of the road 1.2:1 to 10:1.

[0057] Option 11. A projection system according to Option 1, wherein the control module is configured to set the second brightness level of the second virtual image plane so that a first ratio between the second brightness level of the second virtual image plane and the first brightness level of the first virtual image plane exists within a first brightness level range of the first virtual image plane and is greater than the first ratio within a second brightness level range of the first virtual image plane.

[0058] Embodiment 12. The projection system of Embodiment 1, wherein the control module is configured to permit a user to manually adjust the second brightness level of the second virtual image plane independently of the first brightness level of the first virtual image plane.

[0059] Embodiment 13. The projection system of Embodiment 1, wherein the control module is configured to prevent a user from manually adjusting the second brightness level of the second virtual image plane independently of the first brightness level of the first virtual image plane.

[0060] Solution 14. A vehicle system comprising:

[0061] A projection system according to claim 1 configured to project a first image and a second image at a window of a vehicle; and

[0062] an input device configured to manually adjust a first brightness level of the first virtual image plane,

[0063] Wherein the control module is configured to adjust the first brightness level of the first virtual image plane based on an output of the input device.

[0064] Solution 15. A method of operating a projection system, the method comprising:

[0065] Determine the ambient light brightness level;

[0066] determining a first brightness level of a first virtual image plane based on the ambient light brightness level;

[0067] determining a second brightness level of a second virtual image plane based on the first brightness level of the first virtual image plane, wherein the second virtual image plane is farther from an eye box than the first virtual image plane; and

[0068] The projection device is controlled to (i) project a first image in the first virtual image plane, and (ii) project a second image in the second virtual image plane, wherein the first image is displayed in the first virtual image plane with the first brightness level, and wherein the second image is displayed in the second virtual image plane with the second brightness level.

[0069] Option 16. The method according to Option 15 further includes setting the second brightness level of the second virtual image plane to be less than or equal to at least one of: (i) a detected brightness level of the lane marking, or (ii) a predetermined brightness level of the lane marking.

[0070] Scheme 17. The method according to Scheme 16, further comprising:

[0071] determining whether the ambient light brightness level is less than a predetermined level associated with nighttime;

[0072] determining that the augmented reality application of the second virtual image plane is a navigation application; and

[0073] In response to detecting that it is night time and based on the augmented reality application being the navigation application, the second brightness level of the second virtual image plane is set to be less than or equal to at least one of: (i) a detected brightness level of a lane marking, or (ii) a predetermined brightness level of a lane marking.

[0074] Scheme 18. The method according to Scheme 15, further comprising:

[0075] determining an augmented reality application of the second virtual image plane; and

[0076] Based on the augmented reality application, at least one of the following is performed: (i) setting the second brightness level of the second virtual image plane, or (ii) selecting a grayscale from a plurality of available grayscales of the second virtual image plane.

[0077] Scheme 19. The method according to Scheme 15, further comprising:

[0078] determining whether the ambient light brightness level is greater than a predetermined level associated with daytime; and

[0079] In response to detecting that it is daytime, limiting the second brightness level of the second virtual image plane to less than or equal to 5000 cd / m 2 .

[0080] Option 20. The method according to Option 15 further includes setting the second brightness level of the second virtual image plane so that the ratio between the second brightness level of the second virtual image plane and the first brightness level of the first virtual image plane is 2:1 within the first brightness level range of the first virtual image plane, and is greater than 2:1 within the second brightness level range of the first virtual image plane.

[0081] Further areas of applicability of the present disclosure will become apparent from the detailed description, claims and drawings.The detailed description and specific examples are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] The present disclosure will become more fully understood based on the detailed description and accompanying drawings, in which:

[0083] Figure 1 is a virtual image plane diagram illustrating near and far virtual image planes of a holographic head-up display (HUD);

[0084] Figure 2 is a functional block diagram of an exemplary holographic projection system including a control module implementing an augmented reality graphics method according to the present disclosure;

[0085] Figure 3 Yes Figure 2 a perspective view of an exemplary interior of a vehicle of a holographic projection system;

[0086] Figure 4 An augmented reality graphics method including grayscale dimming to control the brightness range of a distant image in a distant virtual image plane according to the present disclosure is illustrated;

[0087] Figure 5 is an exemplary bar graph of near and far virtual image plane brightness values ​​versus brightness range with grayscale dimming in accordance with the present disclosure; and

[0088] Figure 6 is an exemplary graph of near view and far view virtual image plane brightness curve values ​​versus brightness control voltage percentage of the near view virtual image plane according to the present disclosure.

[0089] Among the drawings, reference numerals may be repeated to identify similar and / or identical elements. DETAILED DESCRIPTION

[0090] A dual-plane projection system, such as provided in a dual-plane holographic HUD, may be used to display images at two different virtual image planes. For example, a near-view virtual image plane (hereinafter referred to as a near-view image plane (NIP)) may be used to display vehicle information, such as vehicle status information, vehicle speed, radio stations, indoor and / or outdoor temperature, warning indications, and the like. For augmented reality purposes, a far-view virtual image plane (hereinafter referred to as a far-view image plane (FIP)) may be used to display various other information. For example, a dual-plane projection system may be used to display advertisements, information about objects in the surrounding environment, advanced driver assistance (ADAS) information, such as collision warning information, point of interest (POI) information, navigation information, and the like. Figure 1 An exemplary vehicle 100 is shown including a holographic HUD that displays images in the NIP and in the FIP. The NIP is represented by dashed line 102, and the FIP is represented by dashed line 104. The NIP may be 2.3-2.5 meters in front of the eye box (or eyes) of the vehicle driver. The FIP may be 7 meters or more in front of the eye box.

[0091] An exemplary view 106 of the area in front of the vehicle 100 is shown. The dashed box 110 refers to a distant view image that is displayed in the FIP and with the environment to provide an augmented view (or augmented reality). In the example shown, lane markings and direction indicators are displayed on the road ahead. The dashed box 112 refers to a near view image that is displayed in the NIP and with the environment to provide vehicle information such as vehicle status information.

[0092] Compared to the FIP, the NIP has a small field of view (FOV). The near-view image is displayed at a high overlooking angle relative to a horizontal plane extending at eye-height level and / or passing through the middle of the corresponding eye frame. The FIP includes augmented reality (AR) information displayed on a wide FOV and at a overlooking angle lower than that of the near-view image. Since the AR information is close to the driver's horizontal line of sight, the display of the image in the FIP at a high brightness level may distract the driver. In addition, physiological cues for distance perception (such as accommodation, convergence, and parallax) all become invalid beyond 6 meters. AR HUDs typically position wide FOV virtual images at a distance of more than 7 meters, which enables AR graphics to become "fused" with the real world.

[0093] The example set forth herein includes a projection system that implements an augmented reality graphics method, which includes grayscale dimming to control the brightness range of a distant image in a FIP. The brightness level of the AR graphics in the FIP is controlled and limited to prevent distraction of the driver. The brightness level of the AR graphics and the FIP is not user-selectable, and is prevented from being directly and / or independently adjusted by the driver. This prevents the driver from "turning up" the brightness level of the FIP, and causes the distracting image in the FIP to be presented to the driver. The projection system displays the AR graphics in the FIP and adjusts the brightness of the AR graphics to increase significance while avoiding cognitive capture. The increased significance may be a highlighted object. Object highlighting may refer to displaying one or more AR graphics to identify and / or point out objects in the environment. In the context of the present application, cognitive capture (also referred to as cognitive tunneling) refers to the phenomenon of unintentional blindness in which the driver is too focused on the displayed AR graphics and is not sufficiently focused on and / or aware of other aspects of the current environment.

[0094] The AR graphics method uses human factors and perceptual principles to determine the brightness level of the AR graphics of the FIP, which increases the prominence of the highlighted object while limiting the brightness of the AR graphics to below the brightness threshold of cognitive capture. The brightness of the AR graphics is set to be lower relative to the ambient light to provide low contrast to avoid cognitive capture, while setting it high enough to enhance the prominence of the highlighted object. The brightness of the AR graphics is adjusted based on changes in ambient brightness to maintain low contrast and enhance the prominence of the AR graphics. The brightness threshold of cognitive capture changes and is set based on the detected ambient brightness level. Since the brightness threshold of cognitive capture changes with ambient brightness, the method automatically adjusts the brightness of the AR graphics to continue to prevent the occurrence of cognitive capture.

[0095] For safety reasons, the brightness of the AR graphics is not user-adjustable, but is determined based on the output of the ambient light sensor. The brightness of the NIP image is also adjusted based on the output of the ambient light sensor.

[0096] Figure 2A projection system 200 is shown. The projection system 200 is provided as an example of a dual-plane projection system. Although the projection system 200 is shown as a holographic projection system, the examples disclosed herein are also applicable to other types of dual-plane projection systems. The projection system 200 may include one or more light (or laser) sources 202, a beam expander 204, a spatial light modulator (SLM) 206, a diffuser 208, a first magnifying optical device 210, a second magnifying optical device 212, a windshield (or reflector) 214, a control module 216, and an ambient light sensor 218. The light source 202 generates a light beam 220, which is received at the beam expander 204. The beam expander 204 expands the width of the light beam 220 to provide an expanded beam 222. The SLM 206 receives the expanded beam 222 and encodes it with one or more graphic holograms via the control module 216, and provides a modulated beam 224 including multiple images. The plurality of images include a first (or near view) image for display in the NIP and a second (or distant view) image for display in the FIP. In one embodiment, the beam expander 204 is not included, and the output of the light source 202 is received at the SLM 206. The control module 216 may include one or more display drivers 230 for controlling the state of the SLM 206.

[0097] The output of the SLM 206 is received at the diffuser 208. Two optical paths are provided at the output of the diffuser 208. Two images (e.g., a distant view image and a near view image) are received by the diffuser 208 and projected to the windshield 214 via respective optical paths. The diffuser 208 receives images including a first (or distant view) image provided to the first optical path and a second (or near view) image provided to the second optical path. The optical paths include a first magnifying optic 210 and a second magnifying optic 212, respectively. The magnifying optics 210, 212 may include respective mirrors, lenses, and / or other optical components having respective curvatures. The first and second images are provided to the eye frame in respective projected beams 232, 234. The SLM 206 is encoded with one or more graphic holograms via the control module 216 and provides two projected beams, which are viewed by the retina 232 of the eye 234 of the viewer. The projected beams 232, 234 are reflected by the windshield 214 and projected to the eye frame. The front of the eye box is represented by focal plane 236. The eye box refers to the area where a person's eye or eyes are located.

[0098] In another embodiment in which the projection system 200 is not a holographic projection system, the light source 202, the beam expander 204, the SLM 206, and the diffuser 208 are replaced by a thin film transistor (TFT) liquid crystal display (LCD) controlled by a control module 216. The control module 216 controls the TFT LCD to generate two projected beams. The light source 202, the beam expander 204, the SLM 206, the diffuser 208, and the TFT LCD can be referred to as a projection device and / or a projection circuit.

[0099] The control module 216 executes an AR graphics method (or algorithm) for automatically adjusting the brightness level of the AR graphics in the FIP. Figure 4 An example of an AR graphics method is described. As a few examples, the brightness level of the AR graphics can be adjusted by adjusting the current level of the light source 202 and / or by adjusting the manner in which the AR graphics are generated (e.g., adjusting the modulation and / or operation of the SLM 206). The adjustment of the brightness of the AR graphics is based on the brightness level of the graphics displayed in the NIP.

[0100] The brightness level of the NIP can be automatically adjusted based on the ambient brightness level detected via the ambient light sensor 218 and manually adjusted via the input device 240. The ambient light sensor 218 can be embedded in the corresponding HUD and / or projection system 200. The input device 240 may include one or more buttons, dials, touch screens, one or more switches, microphones, etc., for adjusting the brightness and brightness of the near-field image displayed in the NIP. In one embodiment, the control module 216 assigns a unique brightness range to each of the multiple AR applications and the corresponding AR graphics set displayed in the FIP. The control module 216 implements grayscale dimming to control the brightness range of the AR graphics displayed in the FIP. The projection system 200 may also include a camera for detecting the brightness level of objects such as lane markings.

[0101] The control module 216 prevents the user from directly and / or independently adjusting the brightness level of the FIP, and thus adjusting the brightness level of the distant image and the associated AR graphics. In one embodiment, the projection system 200 may not include an input device, circuit, and / or control logic for allowing the brightness level of the FIP to be directly and / or independently adjusted. In another embodiment, the projection system 200 does include an input device, circuit, and / or control logic that allows a technician to directly and / or independently adjust the brightness level of the FIP while, for example, testing the projection system 200. In this embodiment, the projection system 200 prevents the driver and / or other unauthorized users from directly and / or independently adjusting the brightness level of the FIP. The technician may be required to provide a specific username, password, code, etc., via, for example, a diagnostic tool, in order to implement the adjustment.

[0102] Figure 3An exemplary perspective view from the driver's seat of vehicle 300 is shown. Vehicle 300 includes a windshield 304 located in the front opening of vehicle 300. Passengers in passenger compartment 308 of vehicle 300 can look forward through windshield 404 to the front of vehicle 300. Although land-based examples are described, the present application is also applicable to air-based vehicles (e.g., airplanes, helicopters, etc.) and water-based vehicles (e.g., boats, etc.). Moreover, although some examples are disclosed herein with respect to vehicle embodiments, the examples are also applicable to non-vehicle embodiments.

[0103] 3, windshield 304 is visually located above dashboard 306 of vehicle 300. Vehicle 300 may include steering wheel 310. Vehicle 300 may be an autonomous vehicle, a semi-autonomous vehicle, or a non-autonomous vehicle.

[0104] Including references such as above Figure 2 The HUD of the holographic projection system depicted projects a near-view image (first near-view hologram) 312 onto a portion of the windshield 304 through an aperture 316 in the instrument panel 306. The near-view image is shown on a screen such as Figure 1 . When the vehicle is in use, the near-view image 312 includes various vehicle information, such as the current speed of the vehicle 300, the current gear of the transmission of the vehicle 300, the engine speed, the directional heading of the vehicle 300, the current infotainment system settings, and / or other vehicle information. The near-view image 312 presents data to the driver of the vehicle without the driver having to look away from the objects in front of the vehicle. The HUD also projects a distant image, which may include a first portion 320 and a second portion 322, as shown. The distant image is displayed on a display such as Figure 1 Portions 320, 322 and / or other portions may include various information, indicators, markings, etc. to provide augmented reality and enhance the driving experience. Figure 2 The input device 240 may be included in the instrument panel 306 .

[0105] Figure 4 An augmented reality graphics method including grayscale dimming to control the brightness range of a distant image in a FIP is shown. The method is a human factor-based method (or algorithm) for controlling the brightness of AR graphics while increasing object highlighting significance and avoiding a brightness threshold for cognitive capture. The algorithm adjusts the brightness of the AR graphics based on ambient brightness, and includes grayscale dimming to improve / trim the brightness of the AR graphics set for various different AR applications. Grayscale dimming refers to applying a grayscale selected from a plurality of different grayscales. Each of the grayscales has a lower brightness range than the grayscale set for the NIP. Although mainly referenced Figure 2The following operations are described for the embodiment of the present disclosure, but the operations can be easily modified to be applied to other embodiments of the present disclosure. The operations of the method can be implemented iteratively.

[0106] The method may begin at 400. At 402, the control module 216 displays a near view image in the NIP. At 404, the control module 216 receives an input for adjusting the brightness (or intensity) level of the near view image. The input may be in the form of a brightness control voltage V At 406, the control module 216 receives the output of the ambient light sensor 218. The output of the ambient light sensor 218 indicates the ambient light conditions corresponding to the exterior of the vehicle. The output can be converted into a brightness value of the environment outside and / or in front of the vehicle.

[0107] At 408, the control module 216 may (i) determine a brightness level of the environment based on the output of the ambient light sensor, and (ii) determine a brightness level of the NIP if the brightness level of the NIP is not already known and / or determined. At 410, the control module 216 adjusts the brightness of the near view image based on (i) the brightness level of the environment, (ii) the determined and / or known brightness level of the NIP, and (iii) the input to adjust the brightness level. By adjusting the brightness level of the near view image, the control module 216 adjusts the brightness of the near view image. At 411, the control module 216 may determine a brightness threshold for cognitive capture based on the brightness of the ambient light, such as the brightness level of the environment.

[0108] At 412, the control module 216 determines one or more applications of the one or more perspective images to be displayed in the FIP. One or more of the various applications may be selected based on the current situation and / or condition that the vehicle is experiencing. For example, the control module 216 may select a forward collision alert (FCA) application, a point of interest (POI) application, a navigation application, and / or other applications. For example, when collision warning information is to be displayed due to detection of an impending collision, the FCA application may be selected. For example, when information about a POI (such as a building, monument, store, park, fire station, police station, hospital, school, etc.) is displayed, the POI application may be selected. For example, when navigation information is displayed to guide the driver to a selected geographic location, a navigation application may be selected.

[0109] At 414, the control module 216 can select a unique and / or specified grayscale for the selected application. The grayscale can have a unique and / or specified maximum brightness level. Figure 5A bar graph showing exemplary near and far virtual image plane brightness values ​​versus brightness ranges with grayscale dimming. The first bar 500 is shown with a grayscale set for the NIP based on a brightness control voltage received from the input device 240, the output of the ambient light sensor 218, and / or the brightness level of the environment outside the vehicle. Figure 5 , GS refers to a specific grayscale with a peak brightness value. GS255 refers to a grayscale with a peak brightness value of 255. GS150 and GS100 refer to other grayscales with peak brightness values ​​of 150 and 100, respectively. Peak brightness values ​​255, 150, 100 have corresponding peak brightness values. Bars 502, 504, 506 are exemplary grayscale ranges for different AR graphics applications, specifically FCA, POI, and navigation (NAV). The unit of measurement for brightness is nits or candela per square meter (cd / m 2 ).

[0110] Each application is assigned a specific brightness range. In one embodiment, all applications have different grayscale brightness ranges. For the same application, different brightness ranges can be provided for NIP and FIP. For example, Figure 5 In the example, the maximum brightness of NIP is L max For GS255: 10,000 cd / m 2 , and the corresponding maximum brightness of FIP is L max 5,000 cd / m for GS255 2 . FCA applications may be specified GS255 for FIP. GS255 may be too bright for other applications. Therefore, the grayscale for other applications may be lower. The grayscale is selected to avoid distracting the driver by displaying AR graphics in the FIP. GS150 and GS100 may be specified for POI and NAV applications. Therefore, not all 256 grayscale settings may be used for all applications of distant view images displayed in the FIP.

[0111] At 416, the control module 216 determines whether it is nighttime. Nighttime refers to when the brightness level of the ambient light outside the vehicle is below a predetermined level. This determination may be based on the output of the ambient light sensor 218 and / or the determined brightness level of the environment. Brightness is equal to the product of the reflectance (or reflectivity) ρ and the illuminance E divided by π (or ρE / π). The unit of measurement for illuminance E is lumens per square meter (lm / m 2 ). A light source during the day may be the sun, a headlight, a street lamp, etc. A light source at night may be a headlight, a street lamp, etc. Light may also be reflected by objects such as the moon, clouds, reflective signs, etc., and thus be a light source.

[0112] At 418, the control module 216 determines the brightness level of the lane marking, which can be used as a brightness reference level for night driving, and the brightness level of the AR graphics can be determined based on the brightness reference level. The brightness level of the lane marking can be detected via a sensor (such as the camera 242) or can be a pre-determined stored estimated level. As an example, the lane marking can be white and have a brightness of 130 cd / m when illuminated by the vehicle's headlights. 2 For night driving, the brightness of the FIP navigation graphics may be limited to be no brighter than the lane markings. The contrast ratio between the white lane markings and the portion of the surface of the corresponding road that does not include the lane markings may be 3:1. For night driving, the brightness of the FIP is limited to less than or equal to much less than 130 cd / m 2 For example, the limit value may be less than half the brightness of the lane marking and / or less than 130 / 2 cd / m 2 (or 65 cd / m 2 ).

[0113] At 420, the control module 216 sets the brightness level of the FIP based on the brightness level of the NIP, the cognitively captured brightness threshold, and / or the selected grayscale determination. This is done while limiting the brightness level of the FIP to a brightness level that is less than the brightness level of the lane marking and / or a predetermined maximum brightness level. The brightness level of the lane marking may be equal to the cognitively captured brightness threshold, in which case operation 411 may not be performed. A mathematical relationship, a lookup table, and / or other means may be used to relate the brightness level of the NIP to the brightness level of the FIP. Some examples of this are described below.

[0114] Table 1 is an exemplary table for determining the maximum FIP brightness level based on the NIP:FIP ratio and the maximum NIP brightness.

[0115]

[0116] Table 1 - NIP and FIP brightness relationship.

[0117] NIP brightness control voltage V The brightness control voltage may be generated by the input device 240 or by the control module 216 in response to the output of the input device 240. The brightness control voltage may have a voltage range corresponding to 0-100% brightness, where 0% is dark and 100% is the peak brightness setting. For the example shown, when the selected percentage is 0, the minimum brightness L min For NIP it is equal to 2 cd / m 2 When the selected percentage is 100%, the maximum brightness L max For NIP equal to 10000 cd / m2 The relationship between the brightness control voltage and the commanded brightness can be expressed by Equation 1, where is the commanded brightness level, is a constant, gamma ( ) is equal to 2.2, V is the brightness control voltage, and L min is the minimum brightness. = 2.2 is the V The linear change of brightness provides a linear brightness response setting. = 2.2, V = 100 and L max =10000 cd / m 2 When, constant Equal to 0.4.

[0118] (1).

[0119] In Table 1, ten different brightness control voltage percentages are shown. A different number of brightness control voltage percentages may be included and used.

[0120] FIP L max The selected contrast ratio of the FIP to the road surface and the nominal road brightness value may be determined based on the selected contrast ratio of the FIP to the road surface and the nominal road brightness value. As an example, the selected contrast ratio may be 6.25 and the nominal road brightness value may be 800 cd / m 2 , as shown in Table 1. In one embodiment, the NIP to FIP ratio is between 1:1 and 5:1. In another embodiment, a first NIP to FIP ratio is set for a first portion of the brightness dimming curve (or a first brightness level range of the NIP), and a second NIP to FIP ratio is set for a second portion of the brightness dimming curve (or a second brightness level range of the NIP). In yet another embodiment, a 2:1 NIP to FIP ratio is maintained over most or most of the brightness dimming curve, as represented by the NIP:FIP ratios in Table 1 for brightness control voltage percentages equal to and between 30% and 100%. The NIP:FIP ratio is not 2:1 at the low end of the brightness dimming curve to avoid distracting the driver, as represented by the NIP:FIP ratios of 2.25, 2.60, 3.15, and 4.00 in Table 1 for brightness control voltage percentages of 20%, 10%, 5%, and 0%, respectively. The values ​​in Table 1 are examples. The values ​​are adjustable and may be different than shown. In one embodiment, 10000 cd / m 2 and 2 cd / m 2The maximum and minimum brightness values ​​of are fixed, and the other values ​​in Table 1 are adjustable. The values ​​may not change by a large amount. For brightness control voltage percentages less than or equal to a predetermined percentage (e.g., 30%), the values ​​that change the most may be at the low end.

[0121] At 422 , the control module 216 determines a brightness level of a road surface that is not marked with, for example, lane markings, such as the surface between lane markings.

[0122] At 424, the control module 216 determines to set the brightness level of the FIP based on the brightness level of the NIP, the cognitively captured brightness threshold, the selected grayscale, and the brightness level of the road surface. This can be done while limiting the brightness level of the FIP to a predetermined maximum brightness level. Table 1 shows an exemplary table for determining the maximum FIP brightness level based on the NIP:FIP ratio and the maximum NIP brightness. In one embodiment, the brightness level of the FIP is selected so that the contrast between the FIP and the road surface is between 1.2:1 and 10:1.

[0123] In one embodiment, a separate brightness curve is used to relate the brightness level in the NIP to the brightness level in the FIP. The curve may be used to determine the brightness level in the FIP based on the brightness level in the NIP. Figure 6 An exemplary graph of near view and far view virtual image plane brightness curve values ​​versus brightness control voltage percentage for the near view virtual image plane is shown. The brightness levels of the NIP and FIP can be determined based on the brightness control voltage percentage of the NIP. The brightness curves 600, 602 are shown for both the NIP and the FIP and can follow a power law with gamma (ɣ) equal to, for example, 2.2, so that brightness varies linearly. The shape of the near view and far view virtual image plane brightness curves 600, 602 (similar to the shape of an exponential curve) is related to the relationship between brightness and human perception of brightness changes. The near view and far view virtual image plane brightness curves 600, 602 are selected so that the projection system 200 operates to provide a linear brightness change perceived by the driver with changes in the NIP control voltage. Brightness is logarithmically related to brightness perception.

[0124] For daytime driving, the FIP navigation graphics may have a 3:1 contrast ratio relative to the surface of the road (i.e., the road surface without lane markings and / or other markings). This results in a 2400 cd / m 2 The control module 216 may control the brightness level of the AR graphics in the FIP to provide the nominal daytime brightness level of the AR graphics to be between 2400 cd / m 2 A partly cloudy day might have 800 cd / m 2To accommodate driving on very bright days, the maximum brightness level of the FIP can be set to 5000 cd / m 2 In one embodiment, the maximum brightness level of the FIP is set at 10,000 cd / m 2 .

[0125] The brightness level of the FIP determined at 420 and / or 424 may be set so that the contrast ratio (CR) of Equation 2 is satisfied, where Refers to the brightness level of the AR graphics displayed in front of the environment. is the brightness level of the environment, and is the brightness level of the AR graphics when displayed in a dark room.

[0126] (2).

[0127] Brightness level of the environment The brightness level of the ambient light may be detected using the ambient light sensor 218 and is equivalent to the brightness level of the ambient light.

[0128] At 426, the control module 216 displays the perspective image including the AR graphics in the FIP at the determined brightness level of the FIP. After operation 426, the method can end at 428, or can return to one of the previous operations, such as one of operations 402, 404, 406 or 412.

[0129] Since each set of AR graphics displayed follows a corresponding unique graphics brightness table and / or grayscale, brightness adjustment is performed via grayscale dimming based on one or more corresponding parameters. The parameters include ambient light brightness level, cognitive capture brightness threshold, NIP brightness level, NIP brightness control voltage, selected NIP to FIP ratio, lane marking (or other object) brightness level, etc. The selected NIP to FIP ratio can be determined based on the selected grayscale and / or lookup table, as described above.

[0130] The above operations are intended to be illustrative examples. Depending on the application, the operations may be performed sequentially, synchronously, simultaneously, continuously, or in a different order during overlapping time periods. Moreover, depending on the implementation and / or sequence of events, any of the operations may not be performed or skipped.

[0131] The above example mitigates the risk of presenting distracting AR graphics to the driver by using grayscale dimming with a maximum brightness that is unique to each AR application and adjusted based on ambient brightness.

[0132] The above description is essentially illustrative only and is in no way intended to limit the present disclosure, its application or use. The broad teachings of the present disclosure can be implemented in various forms. Therefore, although the present disclosure includes specific examples, the true scope of the present disclosure should not be limited thereto, because once the drawings, descriptions and appended claims are studied, other modifications will become apparent. It should be understood that one or more steps within the method can be performed in different orders (or simultaneously) without changing the principles of the present disclosure. Further, although each of the embodiments is described above as having certain features, any one or more of these features described in any embodiment of the present disclosure can be implemented in any of the other embodiments and / or implemented in combination with the features of any of the other embodiments, even if the combination is not explicitly described. In other words, the described embodiments are not mutually exclusive, and the replacement of one or more embodiments with each other is still within the scope of the present disclosure.

[0133] Various terms are used, including "connected," "engaged," "coupled," "adjacent," "next to," "on top of," "above," "below," and "positioned," to describe spatial and functional relationships between elements (e.g., between modules, circuit elements, semiconductor layers, etc.). Unless explicitly described as "directly," when describing a relationship between a first and a second element in the above disclosure, the relationship can be a direct relationship with no other intervening elements between the first and second elements, but can also be an indirect relationship with one or more intervening elements (spatially or functionally) between the first and second elements. As used herein, the phrase at least one of A, B, and C should be construed to mean a logical (A or B or C), using a non-exclusive logical OR, and should not be construed to mean "at least one of A, at least one of B, and at least one of C."

[0134] In the drawings, the direction of the arrows, as indicated by the arrow heads, generally shows the flow of information (such as data or instructions) of interest to the illustration. For example, when component A and component B exchange various information, but the information transmitted from component A to component B is relevant to the illustration, the arrow may point from component A to component B. This unidirectional arrow does not imply that no other information is transmitted from component B to component A. Further, for information sent from component A to component B, component B may send a request for the information or a receipt confirmation to component A.

[0135] In this application, including the following definitions, the term "circuit" may be used to replace the term "module" or the term "controller". The term "module" may refer to a portion of or include an application specific integrated circuit (ASIC); a digital, analog, or mixed analog / digital discrete circuit; a digital, analog, or mixed analog / digital integrated circuit; a combinational logic circuit; a field programmable gate array (FPGA); a processor circuit (shared, dedicated, or grouped) that executes code; a memory circuit (shared, dedicated, or grouped) that stores code executed by the processor circuit; other suitable hardware components that provide the described functionality; or a combination of some or all of the above, such as in a system on a chip.

[0136] The module may include one or more interface circuits. In some examples, the interface circuit may include a wired or wireless interface connected to a local area network (LAN), the Internet, a wide area network (WAN), or a combination thereof. The functionality of any given module of the present disclosure may be distributed among multiple modules connected via interface circuits. For example, multiple modules may allow for load balancing. In another example, a server (also known as a remote or cloud) module may perform some functions on behalf of a client module.

[0137] The term code used above may include software, firmware and / or microcode, and may refer to programs, routines, functions, classes, data structures and / or objects. The term shared processor circuit includes a single processor circuit that executes some or all code from multiple modules. The term grouped processor circuit includes a processor circuit that is combined with additional processor circuits to execute some or all code from one or more modules. Reference to multiple processor circuits includes multiple processor circuits on discrete die, multiple processor circuits on a single die, multiple cores of a single processor circuit, multiple threads of a single processor circuit, or a combination of the above. The term shared memory circuit includes a single memory circuit that stores some or all code from multiple modules. The term grouped memory circuit includes a memory circuit that is combined with additional memory to store some or all code from one or more modules.

[0138] The term memory circuit is a subset of the term computer-readable medium. The term computer-readable medium as used herein does not include transient electrical or electromagnetic signals propagated through a medium (such as on a carrier wave); the term computer-readable medium may therefore be considered tangible and non-transient. Non-limiting examples of non-transient tangible computer-readable media are non-volatile memory circuits (such as flash memory circuits, erasable programmable read-only memory circuits, or mask read-only memory circuits), volatile memory circuits (such as static random access memory circuits or dynamic random access memory circuits), magnetic storage media (such as analog or digital tapes or hard drives), and optical storage media (such as CDs, DVDs, or Blu-ray discs).

[0139] The apparatus and methods described in this application may be implemented in part or in whole by a special-purpose computer, which may be generated by configuring a general-purpose computer to perform one or more specific functions implemented in a computer program. The above-mentioned function blocks, flow chart components and other elements serve as software specifications, which can be converted into a computer program through routine work of a technician or programmer.

[0140] The computer program includes processor executable instructions stored on at least one non-transitory tangible computer readable medium. The computer program may also include or rely on stored data. The computer program may include a basic input / output system (BIOS) that interacts with the hardware of the special-purpose computer, device drivers that interact with specific devices of the special-purpose computer, one or more operating systems, user applications, background services, background applications, etc.

[0141] A computer program may include: (i) declarative text to be parsed, such as HTML (Hypertext Markup Language), XML (Extensible Markup Language), or JSON (JavaScript Object Notation), (ii) assembly code, (iii) object code generated by a compiler from source code, (iv) source code for execution by a translator, (v) source code for compilation and execution by a just-in-time compiler, etc. By way of example only, the source code may be written using syntax from languages ​​including C, C++, C#, Objective-C, Swift, Haskell, Go, SQL, R, Lisp, Java®, Fortran, Perl, Pascal, Curl, OCaml, Javascript®, HTML5 (Hypertext Markup Language 5th Revision), Ada, ASP (Active Server Pages), PHP (PHP: Hypertext Preprocessor), Scala, Eiffel, Smalltalk, Erlang, Ruby, Flash®, Visual Basic®, Lua, MATLAB, SIMULINK, and Python®.

Claims

1. A projection system, comprising: at least one projection device configured to (i) project a first image in a first virtual image plane and (ii) project a second image in a second virtual image plane, wherein the second virtual image plane is further from the eye box than the first virtual image plane; as well as A control module, wherein the control module is configured to: Determine the ambient light level, determining a first brightness level of the first virtual image plane based on the ambient light brightness level, determining a second brightness level of the second virtual image plane based on the first brightness level of the first virtual image plane, and The at least one projection device is controlled to (i) display the first image in the first virtual image plane having the first brightness level, and (ii) display the second image in the second virtual image plane having the second brightness level.

2. The projection system of claim 1 , further comprising an ambient light sensor configured to generate an output signal indicative of the ambient light brightness level, The control module is configured to determine the ambient light brightness level based on the output signal.

3. The projection system according to claim 1, wherein the control module is configured to: Determining a brightness threshold for cognitive capture based on the ambient light brightness level; and The second brightness level is determined based on the brightness threshold.

4. The projection system of claim 1, wherein the control module is configured to set the second brightness level of the second virtual image plane so that a contrast ratio between the second virtual image plane and a road surface is between 1.2:1 and 10:

1.

5. The projection system according to claim 1, wherein the control module is configured to: determining whether the ambient light brightness level is less than a predetermined level associated with nighttime; and In response to detecting that it is nighttime, the second brightness level of the second virtual image plane is set so that the contrast between the second virtual image plane and the road surface is between 1.2:1 and 10:

1.

6. The projection system according to claim 5, wherein the control module is configured to: determining that the augmented reality application of the second virtual image plane is a navigation application; and Based on the augmented reality application being the navigation application, the second brightness level of the second virtual image plane is set to be less than or equal to at least one of: (i) a detected brightness level of a lane marking, or (ii) a predetermined brightness level of a lane marking.

7. The projection system according to claim 1, wherein the control module is configured to: determining an augmented reality application of the second virtual image plane; and Based on the augmented reality application, at least one of the following is performed: (i) setting the second brightness level of the second virtual image plane, or (ii) selecting a grayscale from a plurality of available grayscales of the second virtual image plane.

8. The projection system according to claim 1, wherein the control module is configured to: determining whether the ambient light brightness level is greater than a predetermined level associated with daytime; and In response to detecting that it is daytime, limiting the second brightness level of the second virtual image plane to less than or equal to 10000 cd / m 2 .

9. The projection system according to claim 1, wherein the control module is configured to: determining whether the ambient light brightness level is greater than a predetermined level associated with daytime; and In response to detecting that it is daytime, controlling the second brightness level of the second virtual image plane to perform at least one of the following: (i) providing a nominal daytime brightness level of the second virtual image plane at 2400 cd / m 2 or (ii) making the contrast between the brightness level of the augmented reality graphics displayed in the second virtual image plane and the brightness level of the road surface be 1.2:1 to 10:

1.

10. The projection system of claim 9, wherein the control module is configured to: determining that the augmented reality application of the second virtual image plane is a navigation application; and Based on the augmented reality application being the navigation application, controlling the second brightness level of the second virtual image plane to perform at least one of the following: (i) providing the nominal daytime brightness level of the second virtual image plane to be between 2400 cd / m 2 , or (ii) making the contrast between the brightness level of the augmented reality graphics displayed in the second virtual image plane and the brightness level of the surface of the road 1.2:1 to 10:

1.

11. The projection system of claim 1 , wherein the control module is configured to set the second brightness level of the second virtual image plane so that a first ratio between the second brightness level of the second virtual image plane and the first brightness level of the first virtual image plane exists within a first brightness level range of the first virtual image plane and is greater than the first ratio within a second brightness level range of the first virtual image plane.

12. The projection system of claim 1, wherein the control module is configured to permit a user to manually adjust the second brightness level of the second virtual image plane independently of the first brightness level of the first virtual image plane.

13. The projection system of claim 1, wherein the control module is configured to prevent a user from manually adjusting the second brightness level of the second virtual image plane independently of the first brightness level of the first virtual image plane.

14. A vehicle system comprising: The projection system according to claim 1 configured to project the first image and the second image at a window of a vehicle; as well as an input device configured to manually adjust a first brightness level of the first virtual image plane, Wherein the control module is configured to adjust the first brightness level of the first virtual image plane based on an output of the input device.

15. A method of operating a projection system, the method comprising: Determine the ambient light brightness level; determining a first brightness level of a first virtual image plane based on the ambient light brightness level; determining a second brightness level of a second virtual image plane based on the first brightness level of the first virtual image plane, wherein the second virtual image plane is farther from the eye box than the first virtual image plane; as well as Controlling the projection system includes a projection device that (i) projects a first image in the first virtual image plane, and (ii) projects a second image in the second virtual image plane, wherein the first image is displayed in the first virtual image plane with the first brightness level, and wherein the second image is displayed in the second virtual image plane with the second brightness level.

16. The method of claim 15, further comprising setting the second brightness level of the second virtual image plane to be less than or equal to at least one of: (i) a detected brightness level of a lane marking, or (ii) a predetermined brightness level of a lane marking.

17. The method according to claim 16, further comprising: determining whether the ambient light brightness level is less than a predetermined level associated with nighttime; determining that the augmented reality application of the second virtual image plane is a navigation application; as well as In response to detecting that it is night time and based on the augmented reality application being the navigation application, the second brightness level of the second virtual image plane is set to be less than or equal to at least one of: (i) a detected brightness level of the lane marking, or (ii) a predetermined brightness level of the lane marking.

18. The method of claim 15, further comprising: determining an augmented reality application of the second virtual image plane; as well as Based on the augmented reality application, at least one of the following is performed: (i) setting the second brightness level of the second virtual image plane, or (ii) selecting a grayscale from a plurality of available grayscales of the second virtual image plane.

19. The method of claim 15, further comprising: determining whether the ambient light brightness level is greater than a predetermined level associated with daytime; as well as In response to detecting that it is daytime, limiting the second brightness level of the second virtual image plane to less than or equal to 5000 cd / m 2 .

20. The method according to claim 15 further includes setting the second brightness level of the second virtual image plane so that a ratio between the second brightness level of the second virtual image plane and the first brightness level of the first virtual image plane is 2:1 within a first brightness level range of the first virtual image plane and is greater than 2:1 within a second brightness level range of the first virtual image plane.

Images