Vehicle display image enhancement

Abstract

To provide an image enhancement system in a vehicle.SOLUTION: An image enhancement system in a vehicle includes a forward looking light sensor, an ambient light sensor, and an electronic control unit. The forward looking light sensor is configured to sense a forward looking light seen by a driver to generate a forward luminance value. The ambient light sensor is configured to sense an ambient light seen by the driver to generate an ambient luminance value. The electronic control unit is configured to calculate a display luminance control value on the basis of the forward luminance value and the ambient luminance value, analyze a video image in a video input signal to determine a plurality of shades of gray within the video image, adjust the shades of gray in the video image on the basis of the forward luminance value, the ambient luminance value, and the display luminance control value to generate the enhanced video image, and present the enhanced video image to a display.SELECTED DRAWING: Figure 11

Description

[Technical field]

[0001] [CROSS REFERENCE TO RELATED APPLICATIONS] This application claims the benefit of U.S. Provisional Application No. 63 / 288,747, filed December 13, 2021, which is incorporated by reference in its entirety.

[0002] [Technical field] FIELD OF THE DISCLOSURE This disclosure relates generally to systems and methods for vehicle display image enhancement. [Background technology]

[0003] In automotive display applications, light sensors are used to automatically control display brightness as a function of the ambient lighting environment. As the ambient lighting environment increases, the display brightness increases to maintain image legibility. Automatic brightness control methods maintain a comfortable level of visual brightness and reduce display power consumption as ambient illumination decreases. Automatic brightness control methods maintain the legibility of symbols in gray shades of peak white, but may reduce the legibility of lower gray shades. Recently, image enhancement methods have become available that help make all gray shades more visible to the user. However, such image enhancement methods do not improve image visibility in response to ambient lighting levels. Summary of the Invention

[0004] Provided herein is an image enhancement system in a vehicle, the enhancement system including a forward light sensor, an ambient light sensor, and an electronic control unit, the forward light sensor configured to sense light ahead of the vehicle as viewed by a driver while the driver is viewing the display, to generate a forward luminance value, and the ambient light sensor configured to sense ambient light ahead of the vehicle as viewed by the driver while the driver is viewing the display, to generate an ambient luminance value.

[0005] The electronic control unit is configured to calculate a display brightness control value based on the forward luminance value and the ambient luminance value to maintain visibility of the enhanced video image in a variety of lighting conditions, analyze a video image in the video input signal to determine a number of shades of gray in the video image, adjust the number of shades of gray in the video image based on the forward luminance value, the ambient luminance value and the display brightness control value to generate an enhanced video image, and present the enhanced video image on the display.

[0006] In one or more embodiments of the image enhancement system, the forward light sensor is a dedicated light sensor that directly measures forward light entering through the vehicle's windshield.

[0007] In one or more embodiments of the image enhancement system, the forward light sensor is a vehicle dashboard daylight sensor.

[0008] In one or more embodiments of the image enhancement system, the forward light sensor is a biometric sensor configured to measure a pupil diameter of the driver while the driver is looking at the display, and the electronic control unit is further configured to calculate a forward luminance value as a function of the pupil diameter.

[0009] In one or more embodiments of the image enhancement system, a forward light sensor is located on another vehicle and the forward brightness value is received wirelessly at the electronic control unit.

[0010] In one or more embodiments of the image enhancement system, the forward light sensor is a camera configured to determine a forward luminance value.

[0011] In one or more embodiments of the image enhancement system, the ambient light sensor is a dedicated light sensor configured to measure light illumination levels on the surface of the display that faces the driver.

[0012] In one or more embodiments of the image enhancement system, the ambient light sensor is a vehicle dashboard daylight sensor.

[0013] In one or more embodiments of the image enhancement system, the ambient light sensor is a biometric sensor configured to measure a pupil diameter of the driver while the driver is looking at the display, and the electronic control unit is configured to calculate an ambient luminance value based on the pupil diameter.

[0014] In one or more embodiments of the image enhancement system, an ambient light sensor is located on a separate vehicle and the ambient brightness values ​​are received wirelessly by the electronic control unit.

[0015] In one or more embodiments of the image enhancement system, the ambient light sensor is a camera configured to measure ambient luminance values.

[0016] In one or more embodiments of the image enhancement system, the electronic control unit uses a display brightness control value to adjust and maintain visibility of dark shades of gray in the enhanced video image.

[0017] In one or more embodiments of the image enhancement system, the electronic control unit uses the forward luminance value and the surrounding luminance value to adjust and maintain the visibility of multiple dark shades of gray in the enhanced video image.

[0018] In one or more embodiments of the image enhancement system, adjusting multiple shades of gray in a video image enhances one or more portions of the video image based on the grayscale content in the one or more portions.

[0019] In one or more embodiments of the image enhancement system, the electronic control unit dynamically adjusts intermediate shades of gray based on a display brightness control value, a front brightness value, an ambient brightness value, and grayscale content.

[0020] In one or more embodiments of the image enhancement system, the forward luminance value and the surrounding luminance value are estimated from radio weather data around the vehicle, satellite-based vehicle position, satellite-based vehicle heading, and time of day.

[0021] Provided herein is a method for enhancing a displayed image in a vehicle, the method including sensing light ahead as viewed by a driver of the vehicle while the driver is viewing a display using a forward light sensor to generate a forward luminance value, sensing ambient light as viewed by the driver while the driver is viewing the display using an ambient light sensor to generate an ambient luminance value, and calculating a display brightness control value based on the forward luminance value and the ambient luminance value to maintain visibility of the enhanced video image on the display in a variety of lighting conditions.

[0022] The method further includes analyzing a video image in the video input signal to determine a plurality of shades of gray within the video image, adjusting the plurality of shades of gray in the video image based on a forward luminance value, an ambient luminance value and a display luminance control value to generate an enhanced video image, and presenting the enhanced video image on a display to the driver.

[0023] In one or more embodiments of the method, adjusting the multiple shades of gray in the video image is a dynamic image enhancement that lightens the multiple dark shades of gray and the multiple medium shades of gray in the enhanced video image as a function of ambient lighting conditions.

[0024] In one or more embodiments of the method, generating the display brightness control is an automatic brightness control that dynamically adjusts the brightness level of the display as a function of ambient lighting conditions.

[0025] Provided herein is an instrument panel including a display, a forward light sensor, an ambient light sensor, and an electronic control unit. The display is configured to present an enhanced video image to a driver. The forward light sensor is configured to sense light ahead of the vehicle driver while the driver is viewing the display to generate a forward luminance value. The ambient light sensor is configured to sense ambient light ahead of the vehicle driver while the driver is viewing the display to generate an ambient luminance value.

[0026] The electronic control unit is configured to calculate a display brightness control value based on the forward luminance value and the ambient luminance value to maintain visibility of the enhanced video image on the display in a variety of lighting conditions, analyze a video image in the video input signal to determine a number of shades of gray in the video image, and adjust the number of shades of gray in the video image based on the forward luminance value, the ambient luminance value, and the display brightness control value to generate the enhanced video image.

[0027] The foregoing and other features and advantages of the present teachings are readily apparent from the following detailed description of the best modes for carrying out the teachings when taken in conjunction with the accompanying drawings. [Brief description of the drawings]

[0028] [Figure 1] FIG. 2 is a diagram illustrating an example of a vehicle situation.

[0029] [Diagram 2] 1 is a schematic side view of a driver relative to a display in accordance with one or more exemplary embodiments.

[0030] [Diagram 3] FIG. 1 is a schematic diagram of an electronic control unit according to one or more exemplary embodiments.

[0031] [Figure 4]1 is a graph of display luminance as a function of input gray shade in accordance with one or more exemplary embodiments.

[0032] [Diagram 5] 1 is a graph of gray shade enhancement in accordance with one or more exemplary embodiments.

[0033] [Figure 6] FIG. 1 is a schematic diagram of an optical sensor in accordance with one or more exemplary embodiments.

[0034] [Figure 7] 1 is a schematic diagram of another optical sensor in accordance with one or more exemplary embodiments.

[0035] [Figure 8] 1 is a schematic diagram of yet another optical sensor in accordance with one or more exemplary embodiments.

[0036] [Figure 9] FIG. 1 is a schematic diagram of an optical sensor in accordance with one or more exemplary embodiments.

[0037] [Figure 10] FIG. 1 is a schematic diagram for optical sensing according to an example embodiment.

[0038] [Figure 11] 1 is an image shown on a display in accordance with one or more exemplary embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0039] The present disclosure may be susceptible to various modifications and alternative forms, and several exemplary embodiments are shown by way of example in the drawings and will be described in detail herein. The novel aspects of the present disclosure are not limited to the specific forms illustrated in the above-listed drawings. Rather, the present disclosure covers modifications, equivalents, and combinations that are within the scope of the present disclosure as encompassed by the appended claims.

[0040] Generally, embodiments of the present disclosure provide a display enhancement system in a vehicle that provides the benefits of both automatic brightness control coupled with image enhancement adjustment as a function of ambient illumination. The combination provides a new product class that provides overall image visibility under a variety of ambient lighting conditions while simultaneously reducing the overall power consumption of the display system.

[0041] FIG. 1 illustrates the context of a vehicle 90. The vehicle 90 typically includes a body 92, an electronic control unit 94, and an instrument panel 96 having one or more displays 100a-100c. The body 92 may form the interior body of the vehicle 90. The vehicle 90 may include a moving vehicle, such as a car, a truck, a motorcycle, a boat, a train, and / or an aircraft. In some embodiments, the body 92 may be part of a stationary object. The stationary object may include, but is not limited to, a billboard, a kiosk, and / or a marquee. Other types of vehicles 90 may be implemented to meet the design criteria of a particular application.

[0042] The electronic control unit 94 may implement one or more display driver circuits. The electronic control unit 94 is typically operable to generate control signals that drive the displays 100a-100c. In various embodiments, the control signals may be configured to provide instrumentation (e.g., speed, tachometer, fuel, temperature, etc.) to at least one of the displays 100a-100c (e.g., 100a). In some embodiments, the control signals may be configured to provide video (e.g., rearview camera video, forward view camera video, on-board DVD player, etc.) to the displays 100a-100c. In other embodiments, the control signals may be further configured to provide alphanumeric information for display on one or more of the displays 100a-100c.

[0043] In various embodiments, the electronic control unit 94 typically includes at least one microcontroller. The at least one microcontroller may include one or more processors, each embodied as a separate processor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or a dedicated electronic control unit. The at least one microcontroller may be any type of electronic processor (implemented in hardware, software running on hardware, or a combination of both). The at least one microcontroller may also include tangible non-transitory memory (e.g., read-only memory in the form of optical, magnetic, and / or flash memory). For example, the at least one microcontroller may include an amount of random access memory, read-only memory, flash memory, and other types of electrically erasable programmable read-only memory appropriate for the application, as well as associated hardware in the form of high-speed clocks or timers, analog / digital and digital / analog circuits, input / output circuits and devices, and appropriate signal processing and buffering circuits.

[0044] Computer readable and executable instructions embodying the method may be stored in memory and executed as described herein. The executable instructions may be a series of instructions used to run an application (either in the foreground or background) on the at least one microcontroller. The at least one microcontroller may receive commands and information in the form of one or more input signals from various controls or components in the vehicle 90 and communicate instructions to the displays 100a-100c through one or more control signals for controlling the displays 100a-100c.

[0045] Instrument panel 96 implements a structure (or instrument cluster) that supports displays 100a-100c. As illustrated, display 100a may be a cluster display positioned for use by the driver. Display 100b may be a console display positioned for use by the driver and a passenger. Display 100c may be a passenger display positioned for use by a passenger.

[0046] Displays 100a-100c are typically mounted on instrument panel 96. In various embodiments, one or more of displays 100a-100c may be located inside vehicle 90. In other embodiments, one or more of displays 100a-100c may be located outside vehicle 90. One or more of displays 100a-100c may implement enhanced vehicle displays that are easy for the driver to view under various lighting conditions. Control signals used to generate images on displays 100a-100c may be received as electrical communications from electronic control unit 94.

[0047] 2 illustrates a side schematic view of an exemplary driver 98 relative to a display 100x, according to one or more exemplary embodiments. Display 100x may represent displays 100a-100c (e.g., 100a). Driver 98 is shown seated in the driver's seat of vehicle 90 behind display 100a. In other embodiments, driver 98 may be a passenger seated in a different seat and / or a passenger located behind separate displays 100b and / or 100c. Display 100x typically has a face (or front) 112 visible to driver 98. Vehicle 90 includes electronic control unit 94, windshield 102, forward light sensor 104, and ambient light sensor 108.

[0048] The sun 120 may provide forward light 122 that is received by forward light sensor 104 through windshield 102. While driver 98 is looking up at display 100x and out windshield 102, driver 98 also sees forward light 122.

[0049] Ambient light 124 may be visible to driver 98 from directions other than from sun 120. Ambient light 124 may result from reflections of light from the sun 120, other lights around vehicle 90 (e.g., street lights), lights within vehicle 90 (e.g., interior lights), other vehicle headlights, etc. While driver 98 looks down on face 112 of display 100x and / or instrument panel 96, driver 98 sees ambient light 124 directly while forward light 122 is out of direct line of sight.

[0050] The electronic control unit 94 is in electrical communication with the forward light sensor 104, the ambient light sensor 108, and the display 100x. The electronic control unit 94 receives a forward brightness value 106 from the forward light sensor 104. The forward brightness value 106 is proportional to the intensity of forward light 122 sensed by the forward light sensor 104. The electronic control unit 94 also receives an ambient brightness value 110 from the ambient light sensor 108. The ambient brightness value 110 is proportional to the intensity of ambient light 124 sensed by the ambient light sensor 108.

[0051] The electronic control unit 94 is operable to use the forward brightness value 106 and / or the ambient brightness value 110 to dynamically adjust the display brightness of the display 100x via the display brightness control value 114. Under bright conditions with the driver's 98's eyes narrowed, the electronic control unit 94 increases the overall brightness of the display 100x (e.g., increases the projected light source within the display) to prevent the image on the display 100x from being washed out. Thus, the driver 98 may comfortably view the brightened image on the display 100x. Under dark conditions with the driver's 98's eyes dilated, the electronic control unit 94 decreases the overall brightness of the display 100x (e.g., decreases the projected light source) to prevent the image on the display 100x from being distracting. Reducing the brightness of the display 100x also helps to reduce the power consumption of the display 100x.

[0052] 3 illustrates a schematic diagram of an example implementation of electronic control unit 94 in accordance with one or more example embodiments. Electronic control unit 94 communicates with a forward light sensor 104 to receive a forward luminance value 106 and with an ambient light sensor 108 to receive an ambient luminance value 110. A display luminance control value 114 may be generated by electronic control unit 94 and transmitted to display 100x.

[0053] A video input signal 130 is received at an input node of electronic control unit 94. Video input signal 130 typically carries a series of video images intended for presentation to driver 98. A video output signal 132 is generated by electronic control unit 94 and presented on display 100x. Video output signal 132 carries an enhanced version of the video images received in video input signal 130.

[0054] In some embodiments, the electronic control unit 94 includes an image analysis block 134, a dynamic image enhancement block 136, a video modification block 138, a background (or ambient) brightness determination block 140, a front brightness determination block 142, and an automatic brightness determination block 144. The blocks 134-144 may be implemented in hardware and / or software running on the hardware. A video input signal 130 is received by the image analysis block 134 and the video modification block 138. A video output signal 132 is generated by the video modification block 138. A front brightness value 106 is received by the front brightness determination block 142. A surrounding brightness value 110 is received by the background brightness determination block 140. A display brightness control value 114 is generated by the automatic brightness determination block 144.

[0055] The video input signal 130 is first analyzed frame by frame by the image analysis block 134 to evaluate the image content by the gray shade level per pixel. In various embodiments, the frame by frame analysis is performed at a nominal frame rate (e.g., 16 ms / frame) to ensure that there is no data latency noticeable to the driver 98. Based on the ambient brightness information from the automatic brightness determination block 144, the dynamic image enhancement block 136 determines and sends gray shade modification information to the video modification block 138. The video input signal 130 is then dynamically modified in real time by the video modification block 138 to enhance the visibility of gray shades in the image. The video modification block 138 generates the enhanced image in the video output signal 132.

[0056] The ambient light sensor 108 measures the ambient illumination illuminating / reflected from the display 100x and communicates the illumination lux level to the background brightness determination block 140. The background brightness determination block 140 determines background brightness (LBG) information of the ambient light 124 observed by the driver 98 on the front 112 of the display 100x. The background brightness LBG information is relayed to the automatic brightness determination block 144. The automatic brightness determination block 144 uses the background brightness LBG information received from the background brightness determination block 140 to determine a desired display brightness control value 114. The display brightness control value 114 is used by the display 100x to control the brightness level of the display 100x. The display brightness (LDisplay) information (e.g., the display brightness control value 114 inside the electronic control unit 94) can be transmitted to the dynamic image enhancement block 136.

[0057] In parallel with the ambient (background) luminance processing, the forward light sensor 104 measures a forward luminance value 106 as seen by the driver 98 looking through the windshield 102. In addition to controlling the display luminance as a function of the ambient luminance value 110 measured by the ambient light sensor 106, the forward light sensor 104 may be utilized to improve display visibility performance to compensate for transient accommodation mismatch or eye accommodation mismatch conditions. When the driver 98 is viewing a bright scene (such as a sunrise or sunset), the display luminance may be increased because the driver's 98 pupils are narrowed and therefore more display luminance may be required to obtain image visibility. Under a sunrise / sunset scenario, the ambient light sensor 108 may be in shadow, so the display luminance may be reduced without the forward light sensor 104. Implementation of the forward light sensor 104 may appropriately control the display luminance to increase it when appropriate.

[0058] The illuminance lux level of the front light sensor 104 is transmitted to a front brightness determination block 142, which determines front brightness (LFL) information for the light 122 ahead as seen by the driver 98. The front brightness LFL information is transmitted to an automatic brightness determination block 144, which uses this information along with background brightness level LBG information to determine the appropriate display brightness control value 114.

[0059] 4 illustrates an exemplary graph 150 of display luminance as a function of input gray shades in accordance with one or more exemplary embodiments. The graph 150 has a first axis 152 and a second axis 154. The first axis 152 generally illustrates the input gray shades available in the input video signal 130. The second axis 154 illustrates the display luminance in units of nits, which may be a measure of candela per square meter (cd / m2).

[0060] Curve 156 shows the display gamma at 1000 nits. For light (or high) shades of gray (e.g., > about 200 out of a maximum of 255 for an 8-bit image), the display luminance may range from about 600 nits at shade 200 to about 1000 nits at the maximum shade of gray 255. For dark (or low) shades of gray (e.g., < about 50), the display luminance changes very little with the change in shade of gray and remains near zero nits.

[0061] Curve 158 shows the display gamma at 500 nits. For light (or high) shades of gray (e.g., > about 200 out of a maximum of 255), the display luminance ranges from about 300 nits at shade 200 to about 500 nits at shade 255. For dark (or low) shades of gray (e.g., < about 50), the display luminance changes little with the change in shade of gray and remains near zero nits. Other ranges of shades of gray and / or other subdivisions of the range into light, medium and dark may be implemented to meet the design criteria of a particular application.

[0062] A unique problem related to automatic brightness control is that dark gray shades are difficult to see under all ambient lighting conditions. Simply increasing the display brightness has little effect on the visibility of dark gray shades. This is mainly due to the nature of the gamma function (γ) used in automotive displays 100a - 100c according to Equation 1 below.

[0063] L GrayShade =L max (GS / GS max ) γ (1)

[0064] Here, GS is a specific gray shade, GSmax is the maximum gray shade of the image received by the display, L max is the maximum brightness of the display, γ is the gamma function of display 100x, and L GrayShade is the display brightness at a specific gray shade GS.

[0065] Many automotive images have most pixels in dark gray shades (e.g., GS < about 50) and medium gray shades (e.g., about 51 < GS < about 200), with a few pixels having higher (lighter) gray shade content (e.g., GS > about 200). Therefore, to have good visibility for dark gray and medium gray shades, the dark gray and medium gray levels are dynamically adjusted upward as a function of ambient lighting conditions to higher gray levels. This dynamic adjustment is called dynamic image enhancement (DIA). The dynamic image enhancement can be achieved by measuring the current lighting conditions and dynamically adjusting the image content for image visibility. When the dynamic image enhancement is combined with automatic brightness control, the display image can continue to be visible under all lighting conditions, and the peak white brightness can be adjusted to obtain comfortable viewing. Limiting the peak white brightness results in the benefit of reduced display power consumption.

[0066] 5 illustrates a graph 160 of an example enhancement of gray shades in accordance with one or more example embodiments. The graph 160 has a first axis 162 and a second axis 164. The first axis 162 generally indicates the input gray shades available in the input video signal 130. The input gray shades in this example have a range from 0 to 255. The second axis 164 indicates the output gray shades available in the output video signal 132. The output gray shades in this example also have a range from 0 to 255. Other ranges of gray shades (e.g., 0 to 1023 for a 10-bit image) may be implemented to meet the design criteria of a particular application.

[0067] Curve 166 (e.g., a straight line) illustrates the unenhanced transfer of gray shades from input video signal 130 to output video signal 132. Each gray shade in output video signal 132 matches a corresponding gray shade in input video signal 130. In a typical automotive application, a small percentage (e.g., <5 percent) of the images in input video signal 130 have gray shade content to the right (e.g., lighter) of line 170 in light shade region 178. The majority of the gray shade content is typically in a dark shade region 174 between the zero gray shade and line 172, and in an intermediate shade region 176 between line 170 and line 172.

[0068] Curve 168 illustrates the transfer of gray shade enhancement from input video signal 130 to output video signal 132. The enhancement may increase the output gray shade in dark shade region 174 and mid-shade region 176, while keeping light shade region 178 with little or no change. As a result, the dark and mid-shade gray shades are brighter on display 100x and therefore more easily viewed by driver 98 under brighter lighting conditions.

[0069] 6 illustrates a schematic diagram of an example implementation of light sensors according to one or more example embodiments. Other light sensors (e.g., daylight sensors 180a and 180b) on the dashboard and / or in an e-mirror (not shown) may be used for the ambient light sensor 108 and / or the forward light sensor 104 to estimate the forward luminance value 106 and the ambient luminance value 110. The daylight sensor 180a may be positioned and oriented to sense the forward light 122 received through the windshield 102. The daylight sensor 180b may be positioned and oriented to sense the ambient light 124.

[0070] 7 illustrates a schematic diagram of another exemplary implementation of a light sensor according to one or more exemplary embodiments. A biometric sensor 182 may be used to measure the pupil diameter of one or both eyes of the driver 98 to determine the forward light 122 and / or the ambient light 124. While the driver 98 is looking through the windshield 102, the biometric sensor 182 may measure a pupil diameter 184a looking forward. From that measurement, the biometric sensor 182 may determine an approximate intensity of the forward light 122 and communicate an estimated forward luminance value 106 to the electronic control unit 94. While the driver 98 is looking at the display 100x, the biometric sensor 182 may measure a pupil diameter 184b looking around. From that measurement, the biometric sensor 182 may determine an approximate intensity of the ambient light 124 and communicate an estimated ambient luminance value 110 to the electronic control unit 94.

[0071] 8 illustrates a schematic diagram of yet another exemplary implementation of a light sensor according to one or more exemplary embodiments. In various embodiments, vehicle-to-vehicle communication may be used to estimate the forward light 122 and the ambient light 124 in a vehicle 90 and then communicate that information to another vehicle 90a. While the vehicle 90 and the other vehicle 90a are traveling in the same direction with a typical interval between them, the forward light 122 entering the vehicle 90 may be similar or the same as the forward light 122 entering the other vehicle 90a. Similarly, the ambient light 124 experienced inside the vehicle 90 may be similar or the same as the ambient light 124 experienced inside the other vehicle 90a. Thus, the forward luminance value 106 and the ambient luminance value 110 determined in the vehicle 90 may be suitable for controlling a display in the other vehicle 90a.

[0072] 9 illustrates a schematic diagram of an example implementation of a light sensor according to one or more example embodiments. A forward-facing camera 188a may be used as the forward light sensor 104 to capture forward light 122 received through the windshield 102. An inward-facing camera 188b may be used as the ambient light sensor 108 to capture ambient light 124. In various embodiments, other interior cockpit cameras and / or other cameras may be implemented to meet the design criteria of a particular application.

[0073] FIG. 10 shows a schematic diagram of an example implementation for light sensing according to an exemplary embodiment. The vehicle 90 may include a wireless connection 190 to the Internet and a satellite-based positioning receiver 192. In various embodiments, the satellite-based positioning receiver may be a Global Positioning System (GPS) receiver. The satellite-based positioning receiver 192 may be used to determine the time, location, orientation, and heading of the vehicle 90. The wireless connection 190 may receive weather conditions, time, and sunlight reports for the location of the vehicle 90. The weather data, time, sunlight data, location, orientation, and heading may be used to estimate the sunlight illumination conditions (e.g., light ahead 122) as may be experienced by the driver 98 based on the heading, weather conditions, and time of day. Ambient light 124 may be determined by an on-board light sensor.

[0074] The display luminance LDisplay information, background luminance LBG information, and front luminance LFL information are communicated to the dynamic image enhancement block 136, which uses the information to dynamically determine the correct image enhancement correction for perfect display visibility. The dynamic image enhancement block 136 may be implemented in a variety of ways and may employ artificial intelligence and / or deterministic methods.

[0075] One aspect of the integrated enhancement approach is to send reflected background LBG information, front luminance LFL information, and automatically determined display luminance signal LDisplay information to dynamic image enhancement 138. If display luminance signal LDisplay information is available, dynamic image enhancement 138 uses the commanded display luminance to determine appropriate correction values ​​for the dark shade(s) of gray to maintain visibility. Thus, even if the display luminance may be reduced, the visibility of the dark shade(s) of gray is independently maintained.

[0076] In addition to maintaining visibility of darker gray shades, visibility of mid-gray shades is also maintained by using the commanded display luminance in determining the correction factor. Thus, when the display luminance is automatically reduced, the enhancement technique independently determines the correction factor for mid-gray shades to maintain visibility. Visibility of higher (lighter) gray shades is maintained by the automatic luminance determination block 144, which adjusts the display luminance to maintain comfortable visibility of higher gray shades (e.g., peak white symbols).

[0077] 11 illustrates an exemplary image 200 shown on a display according to one or more exemplary embodiments. Image 200 illustrates the benefits of both automatic brightness control combined with image enhancement adjustment as a function of ambient lighting. The grayscale of the illustrated image 200 has been inverted for reproducibility. Thus, in an actual display, the dark symbols are actually white, the white symbols are actually black, and the gray symbols are gray.

[0078] Image 200 includes a portion 202 of darker shades of gray (e.g., shades of gray less than about 50 on a range of 0-255), a portion 204 of medium shades of gray (e.g., shades of gray between about 51 and about 200), and portions 206a-b of lighter shades of gray (e.g., shades of gray greater than about 200). The portion 202 of darker shades of gray may include the entire background of image 200 and therefore appears as black to driver 98. The portion 204 of medium shades of gray includes a plurality of lane markings and a predicted course, which appear as gray symbols on a black background. The portions 206a-b of lighter shades of gray include driving information (e.g., speed, tachometer, fuel, etc.) and typically consume a small amount of space within image 200.

[0079] As will be appreciated by those skilled in the art, terms such as "upper," "lower," "front," "rear," "upper," "lower," "top," "bottom," and the like, may be used descriptively herein without indicating a limitation on the scope of the present disclosure. Further, the present teachings may be described in terms of functional and / or logical block components and / or various processing steps. Such block components may be made up of various hardware components, software components executing on the hardware, and / or firmware components executing on the hardware.

[0080] The foregoing detailed description and drawings support and describe the present disclosure, the scope of which is defined solely by the claims. As will be appreciated by those skilled in the art, there may be various alternative designs and embodiments for carrying out the present disclosure as defined in the appended claims.

Claims

1. 1. An image enhancement system in a vehicle, comprising: a forward light sensor configured to sense forward light viewed by a driver of the vehicle while the driver is viewing the display to generate a forward luminance value; an ambient light sensor configured to sense ambient light viewed by the driver while the driver is viewing the display to generate an ambient luminance value; An electronic control unit; Equipped with the display includes a first input for receiving a display brightness control value; the display brightness control value controls a projection light source of the display; The electronic control unit includes: calculating the display brightness control value based on the front brightness value and the surrounding brightness value; presenting the display brightness control value to the first input of the display; Analyzing a video image in the video input signal to determine a plurality of shades of gray within the video image; Adjusting the multiple shades of gray in the video image based on the forward luminance value, the ambient luminance value, and the display luminance control value to generate an enhanced video image such that visibility of the enhanced video image when displayed is maintained under a variety of lighting conditions. It is structured as follows: the display further includes a second input for receiving the enhanced video image; The electronic control unit further comprises: presenting the enhanced video image to the second input of the display. It is configured as follows:

1. An image enhancement system comprising:

2. The forward light sensor is a dedicated light sensor that directly measures the forward light entering through the vehicle's windshield.

2. The image enhancement system of claim 1.

3. The forward light sensor is a vehicle dashboard daylight sensor.

2. The image enhancement system of claim 1.

4. the forward light sensor is a biometric sensor configured to measure a pupil diameter of the driver while the driver is viewing the display; The electronic control unit is further configured to calculate the forward luminance value as a function of the pupil diameter.

2. The image enhancement system of claim 1.

5. the forward light sensor is located on another vehicle traveling in the same direction at a typical distance from the vehicle; The forward brightness value is received wirelessly at the electronic control unit.

2. The image enhancement system of claim 1.

6. The forward light sensor is a camera configured to determine the forward luminance value.

2. The image enhancement system of claim 1.

7. The ambient light sensor is a dedicated light sensor configured to measure light illumination levels on a surface of the display that faces the driver.

2. The image enhancement system of claim 1.

8. The ambient light sensor is a vehicle dashboard daylight sensor.

2. The image enhancement system of claim 1.

9. the ambient light sensor is a biometric sensor configured to measure a pupil diameter of the driver while the driver is viewing the display; The electronic control unit is configured to calculate the ambient luminance value based on the pupil diameter.

2. The image enhancement system of claim 1.

10. the ambient light sensor is located on another vehicle traveling in the same direction at a typical distance from the vehicle; The ambient brightness value is received wirelessly by the electronic control unit.

2. The image enhancement system of claim 1.

11. The ambient light sensor is a camera configured to measure the ambient luminance value.

2. The image enhancement system of claim 1.

12. The electronic control unit uses the display brightness control value to adjust and maintain visibility of multiple dark shades of gray in the enhanced video image.

2. The image enhancement system of claim 1.

13. The electronic control unit uses the forward luminance value and the surrounding luminance value to adjust and maintain visibility of multiple dark shades of gray in the enhanced video image.

2. The image enhancement system of claim 1.

14. The adjusting of multiple shades of gray in the video image enhances one or more portions of the video image based on the grayscale content in the one or more portions.

2. The image enhancement system of claim 1.

15. The electronic control unit dynamically adjusts a plurality of intermediate shades of gray based on the display brightness control value, the front brightness value, the surrounding brightness value, and the grayscale content.

15. The image enhancement system of claim 14.

16. The forward luminance value and the surrounding luminance value are estimated from wireless weather data around the vehicle, a satellite-based position of the vehicle, a satellite-based heading of the vehicle, and a time of day.

2. The image enhancement system of claim 1.

17. 1. A method for display image enhancement in a vehicle, comprising: sensing, with a forward light sensor, light viewed by a driver of the vehicle while the driver is viewing the display to generate a forward luminance value; Including, the display includes a first input for receiving a display brightness control value; the display brightness control value controls a projection light source of the display; The method further comprises: sensing ambient light viewed by the driver while the driver is viewing the display with an ambient light sensor to generate an ambient luminance value; calculating the display brightness control value based on the front brightness value and the surrounding brightness value; presenting said display brightness control value to said first input of said display; analyzing a video image in a video input signal to determine a number of shades of grey within the video image; adjusting the plurality of shades of grey in the video image based on the front luminance value, the ambient luminance value, and the display luminance control value to generate an enhanced video image, whereby visibility of the enhanced video image is maintained when displayed under a variety of lighting conditions; Including, the display further includes a second input for receiving the enhanced video image; The method further comprises: presenting the enhanced video image to the second input of the display; The method according to claim 1, further comprising:

18. Adjusting the plurality of shades of gray in the video image is a dynamic image enhancement that lightens the plurality of dark shades of gray and the plurality of medium shades of gray in the enhanced video image as a function of ambient lighting conditions.

20. The method of claim 17 .

19. The step of calculating the display brightness control value is an automatic brightness control that dynamically adjusts the brightness level of the display as a function of ambient lighting conditions.

20. The method of claim 18 .

20. An instrument panel, a display configured to present the enhanced video image to a driver; a forward light sensor configured to sense light ahead of the vehicle as viewed by the driver while the driver is viewing the display to generate a forward luminance value; an ambient light sensor configured to sense ambient light viewed by the driver while the driver is viewing the display and generate an ambient luminance value; An electronic control unit; Equipped with the display includes a first input for receiving a display brightness control value; the display brightness control value controls a projection light source of the display; The electronic control unit includes: calculating the display brightness control value based on the front brightness value and the surrounding brightness value; presenting the display brightness control value to the first input of the display; Analyzing a video image in the video input signal to determine a plurality of shades of gray within the video image; Adjusting the multiple shades of gray in the video image based on the forward luminance value, the ambient luminance value, and the display luminance control value to generate an enhanced video image such that visibility of the enhanced video image when displayed is maintained under a variety of lighting conditions. It is structured as follows: the display further includes a second input for receiving the enhanced video image; The electronic control unit further comprises: presenting the enhanced video image to the second input of the display. It is configured as follows: An instrument panel comprising:

Images