Image enhancement system

Abstract

An image enhancement system for a vehicle includes an imager configured to capture image data and a display operably coupled to the imager. The display includes display data. The image enhancement system also includes an electronic control unit (ECU) communicatively coupled to the imager and the display. The ECU includes data processing hardware configured to execute an image enhancement application. The image enhancement application is configured to receive image data from the imager and to display data from the display. The image enhancement application is configured to perform image enhancement and edge enhancement modifications of the image data based on the display data.

Description

Technical Field

[0001] The present disclosure generally relates to an image enhancement system. Background Art

[0002] The information provided in this section is for the purpose of presenting the background of the present disclosure generally. To the extent described in this section, the work of the presently named inventors, as well as aspects of the description that may not qualify as prior art at the time of filing, are neither expressly nor implicitly admitted as prior art against the present disclosure.

[0003] Modern vehicles may include a monitoring system equipped with cameras to detect various activities in the vehicle's main driver's cab. The cameras are typically within the line of sight of the passengers inside the vehicle to maximize image capture and minimize interference from surrounding displays to the pixels. In some cases, the cameras may be disabled by the passengers or otherwise blocked. During operation, the monitoring system typically utilizes machine learning techniques. However, machine learning requires a large amount of training and higher computing power, which reduces the efficiency of the system. Therefore, there is a need for an improved imaging system that can effectively capture and analyze cab activities with minimal interference to the passengers. Summary of the Invention

[0004] In some aspects, an image enhancement system for a vehicle includes an imager configured to capture image data and a display operably coupled to the imager. The display includes display data that includes an operating state of the display. The image enhancement system further includes an electronic control unit (ECU) communicatively coupled to the imager and the display. The ECU includes data processing hardware configured to execute an image enhancement application. The image enhancement application is configured to receive the image data from the imager and the display data from the display. The image enhancement application is configured to perform image enhancement and edge enhancement modifications of the image data based on the display data. The image enhancement application is further configured to render the image in response to the image enhancement and edge enhancement modifications.

[0005] In some examples, the image enhancement application may include image signal processing configured to receive the display data from the display. The image signal processing may be configured to render the image data based on the display data received from the display. In some cases, the image enhancement may include at least one of bit-plane splicing, tone mapping recovery, contrast recovery, brightness recovery, and chromaticity adjustment. The display data may include at least one of a backlight ratio, a contrast ratio, and a display density. The data processing hardware may be configured to perform image enhancement based on at least one of the operating state, the backlight ratio, the contrast ratio, and the display density. Optionally, the image enhancement application may be configured to compare the pixel density from the image data with the display density of the display data. In some configurations, the edge enhancement modification may include image synthesis, and the data processing hardware may be configured to perform image synthesis in response to the image enhancement.

[0006] In other aspects, an image enhancement system includes a display including display data and an imager operably coupled to the display and disposed behind the display. The imager is configured to capture image data. The image enhancement system further includes an electronic control unit (ECU) communicatively coupled to the imager and the display. The ECU includes data processing hardware configured to execute an image enhancement application. The image enhancement application is configured to receive image data from the imager and display data from the display, and perform image enhancement and edge enhancement modification of the image data based on the display data.

[0007] In some examples, the image enhancement application includes image signal processing configured to receive display data from the display. The image signal processing may be configured to render the image data based on the display data received from the display. In some cases, the image enhancement may include at least one of bitplane stitching, tone mapping restoration, contrast restoration, brightness restoration, and chromaticity adjustment. The display data may include at least one of an operating state, a backlight ratio, a contrast ratio, and a display density. The data processing hardware may be configured to perform image enhancement based on at least one of the operating state, the backlight ratio, the contrast ratio, and the display density. Optionally, the image enhancement application may be configured to compare the pixel density from the image data with the display density of the display data. In some configurations, the edge enhancement modification may include image composition, and the data processing hardware is configured to perform image composition in response to the image enhancement.

[0008] In a further aspect, a computer-implemented method causes data processing hardware to perform operations when executed by the data processing hardware. The operations include obtaining display data from a display including an operating state, monitoring the display data of the display via a look-up table value, obtaining image data from the imager, and rendering the obtained image data via an image enhancement application in response to the operating state of the display. The operations further include performing image enhancement of the rendered image data via the image enhancement application based on the display data, performing edge enhancement modification of the image data via the image enhancement application, and synthesizing the enhanced image data.

[0009] In some examples, performing image enhancement may include performing one or more of bitplane stitching of the image data, tone mapping restoration of the image data, contrast restoration of the image data, brightness restoration of the image data, and chromaticity adjustment of the image data. Performing bitplane stitching may include removing image artifacts of the image data. Optionally, the operations may include constructing the imager behind the display. In some cases, obtaining the display data may include obtaining a backlight ratio, a contrast ratio, and a display density. In some cases, performing image enhancement and performing edge enhancement modification may include referring to the display data. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings described herein are for illustrative purposes only of selected configurations and are not intended to limit the scope of the present disclosure.

[0011] Figure 1 is a partial perspective view of the interior compartment of a vehicle equipped with an image enhancement system in accordance with the present disclosure;

[0012] Figure 2 is a block diagram of an image enhancement system in accordance with the present disclosure;

[0013] Figure 3 is Figure 2 another block diagram of the image enhancement system of

[0014] Figure 4 is an example flowchart of an image enhancement system in accordance with the present disclosure; and

[0015] Figure 5 is Figure 4 another flowchart of the image enhancement system of

[0016] In all of the drawings, corresponding reference numerals represent corresponding parts. Detailed Description

[0017] Example configurations will now be described more fully with reference to the accompanying drawings. Example configurations are provided so that this disclosure will be thorough and will fully convey the scope of the disclosure to those of ordinary skill in the art. Specific details, such as examples of specific components, devices, and methods, are set forth to provide a thorough understanding of the configurations of the present disclosure. It will be apparent to those of ordinary skill in the art that the example configurations may be implemented in many different forms and that specific details and example configurations should not be construed as limiting the scope of the present disclosure.

[0018] The terminology used herein is for the purpose of describing particular example configurations only and is not intended to be limiting. As used herein, the singular forms "a", "an", and "the" may also be intended to include the plural forms, unless the context clearly indicates otherwise. The terms "comprising", "including", "containing", and "having" are inclusive and therefore specify the presence of stated features, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof. The method steps, processes, and operations described herein should not be construed as necessarily requiring them to be performed in the particular order discussed or illustrated, unless specifically identified as an order of performance. Additional or alternative steps may be utilized.

[0019] When an element or layer is referred to as "on", "engaged to", "connected to", "attached to", or "coupled to" another element or layer, it can be directly on, engaged, connected, attached, or coupled to the other element or layer, or intervening elements or layers may be present. In contrast, when an element is referred to as "directly on", "directly engaged to", "directly connected to", "directly attached to", or "directly coupled to" another element or layer, intervening elements or layers may not be present. Other words used to describe the relationship between elements should be interpreted in a similar manner (e.g., "between" versus "directly between", "adjacent" versus "directly adjacent", etc.). As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0020] The terms first, second, third, etc. may be used herein to describe various elements, components, regions, layers, and / or sections. These elements, components, regions, layers, and / or sections should not be limited by these terms. These terms are only used to distinguish one element, component, region, layer, or section from another. Terms such as "first", "second", and other numerical terms do not imply an order or sequence unless the context clearly indicates otherwise. Thus, a first element, component, region, layer, or section discussed below may be referred to as a second element, component, region, layer, or section without departing from the teachings of the exemplary configuration.

[0021] In this application, including the definitions below, the term module may be replaced by the term circuit. The term "module" may refer to, or include, an application specific integrated circuit (ASIC); digital, analog, or mixed analog / digital discrete circuits; digital, analog, or mixed analog / digital integrated circuits; combinational logic circuits; a field programmable gate array (FPGA); a processor (shared, dedicated, or group) that executes code; a memory (shared, dedicated, or group) that stores code executed by the processor; other suitable hardware components that provide the recited functionality; or some or all of the foregoing combinations, such as in a system-on-chip.

[0022] The term code used above may include software, firmware, and / or microcode, and may refer to programs, routines, functions, classes, and / or objects. The term shared processor includes a single processor that executes portions or all of the code from multiple modules. The term group processor includes a processor that, in combination with additional processors, executes some or all of the code from one or more modules. The term shared memory includes a single memory that stores some or all of the code from multiple modules. The term group memory includes a memory that, in combination with additional memories, stores some or all of the code from one or more modules. The term memory may be a subset of the term computer-readable medium. The term computer-readable medium does not include transient electrical and electromagnetic signals propagated through a medium and may thus be considered tangible non-transitory memory. Non-limiting examples of non-transitory memory include tangible computer-readable media, including non-volatile memory, magnetic memory, and optical memory.

[0023] The apparatus and methods described in this application may be implemented in part or in whole by one or more computer programs executed by one or more processors. The computer programs include processor-executable instructions stored on at least one non-transitory tangible computer-readable medium. The computer programs may also include and / or rely on stored data.

[0024] A software application (i.e., software resource) may refer to computer software that causes a computing device to perform tasks. In some examples, a software application may be referred to as an "application program", "app", or "program". Example applications include, but are not limited to, system diagnostic applications, system management applications, system maintenance applications, word processing applications, spreadsheet applications, messaging applications, media streaming applications, social networking applications, and gaming applications.

[0025] Non-transitory memory may be a physical device for temporarily or permanently storing programs (e.g., sequences of instructions) or data (e.g., program state information) for use by a computing device. Non-transitory memory may be volatile and / or non-volatile addressable semiconductor memory. Examples of non-volatile memory include, but are not limited to, flash memory and read-only memory (ROM) / programmable read-only memory (PROM) / erasable programmable read-only memory (EPROM) / electrically erasable programmable read-only memory (EEPROM) (e.g., commonly used for firmware, such as a boot program). Examples of volatile memory include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), phase change memory (PCM), and magnetic disks or tapes.

[0026] These computer programs (also known as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented in high-level procedural and / or object-oriented programming languages and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, non-transitory computer-readable medium, apparatus, and / or device (e.g., a magnetic disk, an optical disk, a memory, a programmable logic device (PLD)) that provides machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal that provides machine instructions and / or data to a programmable processor.

[0027] The various implementations of the systems and techniques described herein can be implemented in digital electronic and / or optical circuits, integrated circuits, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These different implementations can include implementations in one or more computer programs executable and / or interpretable on a programmable system including at least one programmable processor, at least one input device, and at least one output device, the programmable processor being either special-purpose or general-purpose and being coupled to receive data and instructions from, and to send data and instructions to, a storage system.

[0028] The processes and logical flows described in this specification can be performed by one or more programmable processors, also known as data processing hardware, executing one or more computer programs to perform functions by operating on input data and generating output. These processes and logical flows can also be performed by special-purpose logic circuitry, such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). By way of example, processors suitable for executing computer programs include both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. In general, a processor will receive instructions and data from a read only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more storage devices for storing instructions and data. In general, a computer will also include, or be operatively coupled to, one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, to receive data from or transfer data to, or both, the mass storage device(s). However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and storage devices, including by way of example semiconductor storage devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special-purpose logic circuitry.

[0029] To provide for interaction with a user, one or more aspects of the present disclosure can be implemented on a computer having a display device, such as a CRT (Cathode Ray Tube), LCD (Liquid Crystal Display) monitor, or touch screen, for displaying information to the user and, optionally, a keyboard and a pointing device, such as a mouse or a trackball, by which the user can provide input to the computer. Other types of devices can also be used to provide for interaction with the user; for example, feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input received from the user can be in any form, including acoustic, speech, or tactile input. Additionally, a computer can interact with the user by sending documents to and receiving documents from the device(s) the user uses; for example, by sending a web page to a web browser on a client device of the user in response to a request received from the web browser.

[0030] Reference Figures 1 - 3, the image enhancement system 10 includes an electronic control unit (ECU) 12 and an imager 100. The image enhancement system 10 is configured for a vehicle 200 such that the ECU 12 can be the central computing system of the vehicle 200. The ECU 12 is configured with an image enhancement application 14, which is configured to process image data 102 received from the imager 100. The image enhancement system 10 also includes a display 202 of the vehicle 200, which provides display data 204 to the ECU 12, as described below. When executing the image enhancement application 14, the ECU 12 receives the display data 204 of the display 202 and the acquired image data 102, which will be described in more detail below.

[0031] The imager 100 is configured to monitor the passengers of the vehicle 200 and can also provide an image 102a on the display 202. The image 102a is provided to the ECU 12 as part of the image data 102 for enhancement by the image enhancement application 14. For example, independent of enhancement, the image 102a can have low visibility such that the display 202 can project a flickering and / or dim image 102a. Thus, the image enhancement application 14 advantageously helps to render a clear image to be displayed by the display 202. The imager 100 is operatively coupled to the display 202 at different positions within the vehicle 200 and is disposed behind the display 202. The imager 100 can thus include a plurality of imagers 100 commensurate with the number of displays 202 equipped within the vehicle 200. In some examples, the imager 100 can be disposed behind the circuitry 206 of the display 202, which may affect the quality of the original image data 102. Thus, the display data 204 can directly affect the image data 102. The image enhancement application 14 is configured to advantageously enhance and render the image data 102, as described in more detail below.

[0032] The ECU 12 is communicatively coupled to the imager 100 and the display 202 and includes data processing hardware 16 configured to execute the image enhancement application 14. The ECU 12 also includes memory hardware 18, as described below, which is communicatively coupled to the data processing hardware 16. The image enhancement application 14 is configured to receive the image data 102 from the imager 100 and the display data 204 from the display 202. As part of the rendering process of the image data 102, the image enhancement application 14 includes image enhancement 20, image signal processing 22, and image synthesis 24, which are described separately herein.

[0033] In response to receiving the image data 102, the data processing hardware 16 performs each of image enhancement 20, image signal processing 22, and image composition 24. Thus, the image enhancement application 14 performs the image enhancement 20 of the image data 102 based on the display data 204 of the display 202, which will be described in further detail below. For example, the image enhancement application 14 obtains the image data 102 and performs the image signal processing 22. The image signal processing 22 uses the display data 204 as the input for evaluating the image data 102.

[0034] With further reference Figures 1 - 3 , the imager 100 captures the image data 102, which may include the image 102a to be displayed on the display 202. As described herein, the image 102a may be rendered by the image enhancement application 14. The image data 102 also includes image artifacts 104, which may be the result of positioning the imager 100 behind the display 202, and the image enhancement 20 is used to correct the image artifacts 104 to generate the rendered image 26. For example, the image enhancement 20 eliminates the image artifacts by using the display data 204 as the input to the image enhancement application 14. In addition, the image enhancement application 14 may utilize the pixel density 106 of the image artifacts 104 during the process of image enhancement 20, as described below.

[0035] The display data 204 may be analyzed by the image signal processing 22 of the image enhancement application 14. For example, the display data 204 includes the operating state 208 of the display 202, which includes an active state 208a and an inactive state 208b. When the display 202 is in the inactive state 208b, the image signal processing 22 is activated by the image enhancement application 14 without having to perform additional enhancement of the image enhancement application 14. Thus, the image enhancement application 14 may directly synthesize and render the image data 102 using the image signal processing 22. As described herein, when the image data 102 is rendered in response to the active state 208a of the display 202, the image enhancement application 14 performs additional enhancement. Thus, the display data 204 is used as the input to the image enhancement application 14 in determining which image enhancement 20 (if any) to perform.

[0036] Still referring Figures 1 - 3 , the display data 204 includes additional data related to the display 202, which may jointly define the look-up table values 210 of the display 202. The image enhancement system 10 utilizes the look-up table values 210 to monitor the display data 204 of the display 202 to determine which image enhancement 20 to perform. The look-up table values 210 include, but are not limited to, the backlight ratio 212, the contrast 214, and the display density 216. The data processing hardware 16 is configured to perform the image enhancement 20 based on at least one of the operating state 208, the backlight ratio 212, the contrast ratio 214, and the display density 216.

[0037] The backlight ratio 212 may include a power value 212a. The contrast ratio 214 provides the ECU 12 with contrast data of the image data 102 relative to the display 202. For example, the display 202 may project an image 102a related to the image data 102, and the contrast ratio 214 may reflect the contrast of the displayed image 102a. In addition, the ECU 12 may receive a display density 216, which is generally related to the pixel density 106 of the image artifacts 104. For example, the image enhancement application 14 compares the display density 216 with the pixel density 106.

[0038] Further reference Figures 1 - 3 , in response to receiving the image data 102, the image enhancement application 14 obtains display data 204 including each of the backlight ratio 212, the contrast ratio 214, and the display density 216 to perform image enhancement 20. For example, as described above, the image enhancement application 14 obtains the display data 204 and initially renders the image data 102 based on the operating state 208. Generally, the image enhancement application 14 is configured to compare the pixel density 106 of the image data 102 with the display density 216 of the display data 204. The image enhancement 20 performs processing based on the display data 204 to enhance the image data 102 and finally renders the rendered image 26. The rendered image 26 may be stored on the memory hardware 18. In some examples, the rendered image 26 may be displayed on the display 202. As part of the image enhancement 20, the image enhancement 20 performs at least one of bitplane stitching 30, tone mapping recovery 32, contrast recovery 34, brightness recovery 36, and chromaticity adjustment 38.

[0039] For example, the image enhancement application 14 performs bitplane stitching to detect a display modulation 218 from the display 202. The display modulation 218 occurs when there is flicker and / or banding on the display 202 due to unrendered image data 102. The display modulation 218 may be detected by the ECU 12 based on the lookup table value 210 received from the display 202. The image enhancement application 14 performs bitplane stitching 30 in response to the detected display modulation 218. The ECU 12 performs bitplane stitching 30 by dividing the pixel density 106 of the image data 102. For example, the image data 102 may be converted into a binary form and then divided into corresponding bitplanes by the image enhancement application 14. Thus, the bitplane stitching 30 modifies the pixel arrangement of the image data 102 to minimize and / or eliminate the flicker or banding of the image data 102. For example, the bitplane stitching 30 performed by the image enhancement application 14 removes the image artifacts 104 of the image data 102 that may correspond to the flicker and / or banding.

[0040] In parallel with bitplane splicing 30, image enhancement application 14 performs contrast restoration 34 and tone mapping restoration 32. Tone mapping restoration 32 removes flicker and / or banding detected by image enhancement application 14. Additionally, tone mapping restoration 32 may reduce the tone value of image data 102. Generally, tone mapping restoration 32 can be used to extend the brightness range of image data 102 by mapping the colors of the image data relative to the display density 216 of display 202 to approximate the appearance of display 202 with image data 102.

[0041] Contrast restoration 34 utilizes the contrast ratio 214 received from display 202 to adjust the contrast of image data 102. For example, the contrast of image data 102 may not be consistent with the contrast ratio 214 of display 202. The inconsistency between the contrast of image data 102 and display 202 is adjusted by image enhancement application 14 by performing contrast restoration 34. Contrast restoration 34 is performed at the pixel level of image data 102. For example, image enhancement application 14 can clarify image artifacts 104 by adjusting pixel density 106. The adjusted pixel density 106 can change the contrast of image data 102 relative to display 202 to ultimately enhance or improve the quality of the rendered image 26.

[0042] Further reference Figures 1 - 3 Referring further, image enhancement application 14 can perform brightness restoration 36 in parallel with performing contrast restoration 34. Brightness restoration 36 is configured to restore the brightness of image data 102. As described above, imager 100 is disposed behind display 202. It is contemplated that display 202 may include a plurality of displays 202 throughout the interior cabin 220 of vehicle 200. Similarly, a plurality of imagers 100 may be located behind the plurality of displays 202 such that each display 202 may have a corresponding imager 100 disposed behind the respective display 202. In some cases, display 202 may be exposed to high levels of internal and external lighting. In other cases, display 202 and imager 100 may be located in areas of interior cabin 220 that are exposed to minimal lighting, such that the overall brightness of image data 102 may be skewed.

[0043] For example, the imager 100 and the display 202 can be located behind the steering wheel 222 of the vehicle 200. Accordingly, the backlight ratio 212 of the display 202 can be increased and can affect the brightness of the image data 102. The image enhancement application 14 is configured to correct the differential brightness of the image data 102 by performing a brightness restoration 36 of the image enhancement 20. In some examples, the brightness restoration 36 can be performed to correct for overexposure of the image data 102 due to an increased level of external light received by the imager 100 and the display 202. Additionally, the image enhancement application 14 is configured to simultaneously adjust the image data 102 received from multiple imagers 100 such that the brightness restoration 36 performed for one set of image data 102 can be different from the brightness restoration 36 performed for another set of image data 102.

[0044] Still referring Figures 1 - 3 , the image enhancement application 14 can perform a chromaticity adjustment 38 when each of the bit-plane stitching 30, tone-mapping restoration 32, contrast restoration 34, and brightness restoration 36 is completed. The chromaticity adjustment 38 is configured to adjust the color intensity or saturation of the image data 102. As described above, the positioning of the imager 100 relative to the display 202 can change the saturation of the image data 102. For example, positioning the imager 100 behind the circuitry 206 of the display 202 can change the saturation of the image data 102 because the imager 100 can capture reflections from the circuitry 206 of the display 202.

[0045] Once the chromaticity adjustment 38 is completed, the image enhancement application 14 performs an edge enhancement modification 40 of the image data 102. The edge enhancement modification 40 adjusts the sharpness of the image data 102. The image enhancement application 14 performs the edge enhancement modification 40 as the last step before image composition 24. Accordingly, after each image enhancement 20 is completed, the edge enhancement modification 40 is performed by the image enhancement application 14. The edge enhancement modification 40 identifies the edge boundaries 42 of the image data 102 and increases the contrast of the image data 102 around the edge boundaries 42 to sharpen the image data 102.

[0046] Further referring Figures 1 - 3 , the image enhancement application 14 performs the image composition 24 in response to the completion of the edge enhancement modification 40. In some examples, the edge enhancement modification 40 can include the image composition 24. The data processing hardware 16 can be configured to perform the image composition 24 in response to the image enhancement 20. The image composition 24 edits the pixels of the image data 102 that have undergone the image enhancement 20 and the edge enhancement modification 40. Accordingly, the image composition 24 produces a rendered image 26 that is ultimately stored in the memory hardware 18 and / or displayed on the display 202.

[0047] Now referring Figure 4 and 5, which shows an example flowchart of the image enhancement system 10. In the initial step 500, the image data 102 is acquired by the ECU 12. At 502, the ECU 12 performs image signal processing 22. At 504, the ECU 12 also acquires the display data 204 from the display 202 and determines at 506 whether the display data 204 is in the active state 208a. If not, the ECU 12 continues to perform image signal processing 22 at 502 and performs image composition 24 at 508. The image enhancement application 14 then generates a rendered image 26 at 510.

[0048] If the display data 204 is in the active state 208a, the image enhancement application 14 performs bitplane stitching 30 at 520 and detects display modulation 218 at 522. The image enhancement application 14 performs contrast recovery 34 at 524 and, in parallel, performs brightness recovery 36 at 526. At 528, the image enhancement application 14 also performs tone mapping recovery 32 in parallel. Next, at 530, the image enhancement application 14 performs chromaticity adjustment 38. At 532, edge enhancement modification 40 is subsequently performed. Based on the edge enhancement modification 40, the image enhancement application 14 performs image composition 24 at 534 and generates a rendered image 26 at 536.

[0049] Referring again to Figures 1 - 5 , the image enhancement system 10 advantageously renders the image data 102, where the imager 100 is disposed behind the display 202. The positioning of the imager 100 behind the display 202 minimizes the overall package complexity while keeping the imager 100 relatively invisible to the driver. In addition, the execution of the image enhancement application 14 by the ECU 12 minimizes the total computing power required to render the image data 102. Thus, the image enhancement system 10 improves efficiency due to the reduced computing power required by the image enhancement application 14.

[0050] Numerous embodiments have been described. However, it should be understood that various modifications can be made without departing from the spirit and scope of the present disclosure. Accordingly, other embodiments are also within the scope of the following claims.

[0051] For purposes of illustration and description, the foregoing description has been provided. It is not intended to be exhaustive or to limit the present disclosure. A single element or feature of a particular configuration is generally not limited to that particular configuration, but is interchangeable where applicable and can be used in a selected configuration. This can also vary in many ways. Such variations should not be regarded as a departure from the present disclosure, and all such modifications are intended to be included within the scope of the present disclosure.

Claims

1. An image enhancement system includes: A display, which includes display data; An imager, which is operably coupled to the display and disposed behind the display, and the imager is configured to capture image data; And An electronic control unit (ECU), which is communicatively coupled to the imager and the display, and includes data processing hardware configured to execute an image enhancement application. The image enhancement application is configured to receive image data from the imager and display data from the display, and perform image enhancement and edge enhancement modification of the image data based on the display data.

2. The image enhancement system according to claim 1, wherein, The image enhancement application includes image signal processing, which is configured to receive display data from the display.

3. The image enhancement system according to claim 2, wherein, The image signal processing is configured to render the image data based on the display data received from the display.

4. The image enhancement system according to claim 1, wherein, The image enhancement includes at least one of bitplane splicing, tone mapping restoration, contrast restoration, brightness restoration, and chromaticity adjustment.

5. The image enhancement system according to claim 4, wherein, The display data includes at least one of an operating state, a backlight ratio, a contrast ratio, and a display density.

6. The image enhancement system according to claim 5, wherein, The data processing hardware is configured to perform the image enhancement based on at least one of the operating state, the backlight ratio, the contrast ratio, and the display density.

7. The image enhancement system according to claim 6, wherein, The image enhancement application is configured to compare the pixel density of the image data with the display density of the display data.

8. The image enhancement system according to claim 1, wherein, The edge enhancement modification includes image synthesis.

9. The image enhancement system according to claim 8, wherein, The data processing hardware is configured to perform the image synthesis in response to the image enhancement.

10. A vehicle configured with the image enhancement system according to claim 1.

Images