Image processing apparatus and image processing method and application processor
Patent Information
- Application Number
- CN202110796230.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2020-08-31
- Filing Date
- 2021-07-14
- Publication Date
- 2026-08-21
- Estimated Expiration
- 2041-07-14
Smart Images
Figure CN114125322B_ABST
Abstract
Description
[0001] Cross-references to related applications
[0002] This application is based on and claims priority to Korean Patent Application No. 10-2020-0110041, filed on August 31, 2020, with the Korean Intellectual Property Office (KIPO), the disclosure of which is incorporated herein by reference in its entirety. Technical Field
[0003] The exemplary embodiments of this disclosure relate to semiconductor integrated circuits, and more specifically, to image processing apparatus and image processing methods for high-resolution displays, and to application processors including the image processing apparatus. Background Technology
[0004] With the continuous development of information technology, display devices play a crucial role in providing information to users. Various display devices, such as liquid crystal displays (LCDs), plasma displays, and electroluminescent displays, have become widespread. Among these display devices, electroluminescent displays typically feature fast response times and low power consumption, and utilize light-emitting diodes (LEDs) or organic light-emitting diodes (OLEDs) that emit light through the recombination of electrons and holes.
[0005] Recently, with the increase in display device resolution and pixel-per-inch (PPI), there has been a demand for display device quality enhancement or improvement. For example, multi-window display of multiple applications on a single screen has become popular, and there is a need for display quality enhancement for each type of image. Therefore, various schemes for display quality enhancement have been investigated. Summary of the Invention
[0006] An image processing apparatus and image processing method are provided that can apply display quality enhancement algorithms to suit the actual screen displayed on a high-resolution display device.
[0007] Additionally, an application processor including the image processing device is provided.
[0008] According to an exemplary embodiment, an image processing apparatus is provided, comprising: at least one processor configured to implement: a mixer configured to: receive a plurality of layer data; generate first image data by mixing the plurality of layer data, the first image data including a plurality of pixel values corresponding to a screen in a display device; and generate pixel mapping data based on the plurality of layer data including a plurality of pixel identities (IDs), the plurality of layer data representing a plurality of images to be displayed on a screen of the display device, the plurality of pixel IDs indicating one or more display quality enhancement algorithms to be applied to the plurality of pixel values; and a display quality enhancer configured to generate second image data including a plurality of display quality enhanced pixel values by applying the one or more display quality enhancement algorithms to the plurality of pixel values based on the first image data and the pixel mapping data.
[0009] According to an exemplary embodiment, an image processing method is provided. The method includes: receiving multiple layer data, the multiple layer data representing multiple images to be displayed on a screen of a display device; generating first image data by mixing the multiple layer data, the first image data including multiple pixel values corresponding to the screen; generating pixel mapping data including multiple pixel identities (IDs) based on the multiple layer data, the multiple pixel IDs indicating one or more display quality enhancement algorithms to be applied to the multiple pixel values; and generating second image data including multiple display quality enhanced pixel values based on the first image data and the pixel mapping data by applying the one or more display quality enhancement algorithms to the multiple pixel values.
[0010] According to an exemplary embodiment, an application processor is provided, the application processor including: at least one processor; and a display controller configured to interoperate with the at least one processor. The display controller includes: a high dynamic range (HDR) unit configured to receive multiple layer data from the at least one processor and perform HDR processing on the multiple layer data based on a first control signal, the multiple layer data representing multiple images to be displayed on a screen of a display device; a mixer configured to generate first image data by mixing the multiple layer data based on the output of the HDR unit and a second control signal, and to generate pixel mapping data including multiple pixel identity IDs based on the output of the HDR unit and the second control signal, the first image data including multiple pixel values corresponding to the screen, the multiple pixel IDs indicating one or more display quality enhancement algorithms to be applied to the multiple pixel values; a display quality enhancer configured to generate second image data including multiple display quality enhanced pixel values by applying the one or more display quality enhancement algorithms to the multiple pixel values based on the first image data and the pixel mapping data; a register configured to receive at least one metadata corresponding to at least one of the multiple layer data from the at least one processor, and to generate a first control signal, a second control signal, and a third control signal based on the at least one metadata; and a frame rate control unit configured to control the frame rate of the display device based on the third control signal.
[0011] According to an exemplary embodiment, an image processing apparatus is provided, comprising: at least one processor configured to implement: a mixer configured to: receive a plurality of layer data; generate first image data by mixing the plurality of layer data, the first image data including a plurality of pixel values corresponding to a screen in a display device; and generate block mapping data including a plurality of block identities (IDs) based on the plurality of layer data, the plurality of layer data representing a plurality of images to be displayed on a screen of the display device, the plurality of block IDs indicating one or more display quality enhancement algorithms to be applied to the plurality of pixel values; and a display quality enhancer configured to generate second image data including a plurality of display quality enhanced pixel values by applying the one or more display quality enhancement algorithms to the plurality of blocks based on the first image data and the block mapping data, wherein the display device includes a plurality of pixels, and each of the plurality of pixel values corresponds to a corresponding one of the plurality of pixels, and wherein two or more of the plurality of pixels are combined to form each of the plurality of blocks, and each of the plurality of block IDs corresponds to a corresponding one of the plurality of blocks. Attached Figure Description
[0012] The above and other aspects, features and advantages of this embodiment will become clear from the following description taken in conjunction with the accompanying drawings, in which:
[0013] Figure 1 This is a block diagram illustrating an image processing apparatus according to an exemplary embodiment;
[0014] Figure 2 This is a detailed illustration of an exemplary embodiment. Figure 1 A block diagram of an image processing device;
[0015] Figure 3 , Figure 4 , Figure 5A , Figure 5B , Figure 5C , Figure 5D , Figure 5E , Figure 5F , Figure 5G , Figure 6 , Figure 7A , Figure 7B , Figure 7C , Figure 8 , Figure 9 , Figure 10A and Figure 10B It is a diagram used to describe the operation of an image processing apparatus according to one or more exemplary embodiments;
[0016] Figure 11 This is a block diagram illustrating a display controller including an image processing apparatus according to an exemplary embodiment;
[0017] Figure 12 and Figure 13 This is a block diagram illustrating an application processor including an image processing apparatus according to an exemplary embodiment;
[0018] Figure 14 This is a block diagram illustrating an electronic device including an application processor according to an exemplary embodiment;
[0019] Figure 15 This is a block diagram illustrating an image processing apparatus according to an exemplary embodiment;
[0020] Figure 16 This is a diagram used to describe the operation of an image processing apparatus according to an exemplary embodiment;
[0021] Figure 17 This is a flowchart illustrating an image processing method according to an exemplary embodiment;
[0022] Figure 18 and Figure 19 It is shown Figure 17 The flowchart shows an example of generating second image data.
[0023] Figure 20 , Figure 21 and Figure 22 This is a flowchart illustrating an image processing method according to an exemplary embodiment; and
[0024] Figure 23 This is a block diagram illustrating an electronic system including an application processor according to an exemplary embodiment. Detailed Implementation
[0025] Various exemplary embodiments will be described in more detail with reference to the accompanying drawings, in which one or more embodiments are illustrated. However, this disclosure may be implemented in many different forms and should not be construed as limited to the embodiments set forth herein. The same reference numerals refer to the same elements throughout this disclosure.
[0026] It should be understood that when a component or layer is referred to as being "above" another component or layer, "on top of" another component or layer, "on" another component or layer, "below" another component or layer, "under" another component or layer, "connected to" or "bonded to" another component or layer, the component or layer may be directly above, directly above, directly on, directly below, directly below, directly connected to, or directly bonded to the other component or layer, or there may be intermediate components or layers. Conversely, when a component is referred to as being "directly above," "directly above," "directly above," "directly above," "directly below," "directly below," "directly below," "directly connected to," or "directly bonded to" another component or layer, there is no intermediate component or layer. The same label always refers to the same component.
[0027] The expression "at least one of a, b, and c" should be understood to include only a, only b, only c, both a and b, both a and c, both b and c, or all of a, b, and c. Terms such as "first," "second," etc., can be used to modify various elements regardless of order and / or importance, and can be used to simply distinguish one element from another.
[0028] The terms used in one or more embodiments of this disclosure (such as “unit” or “module”) refer to a unit for performing at least one function or operation, and may be implemented in hardware, software or a combination of hardware and software.
[0029] The term "unit" or "module" can be implemented by a program that is stored in an addressable storage medium and is executable by a processor.
[0030] For example, the term "unit" or "module" may include software components, object-oriented software components, class components and task components, processes, functions, attributes, procedures, subroutines, program code segments, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays and / or variables.
[0031] Figure 1 This is a block diagram illustrating an image processing apparatus according to an exemplary embodiment.
[0032] Reference Figure 1 The image processing apparatus 100 includes a mixer 110 and a display quality enhancer 120.
[0033] Mixer 110 receives multiple layer data LDATs. The multiple layer data LDATs represent multiple images to be displayed on a screen of a display device (or display panel). For example, the multiple images may be displayed as partially and / or completely overlapping on the screen of the display device, and each of the multiple layer data LDATs may correspond to a specific one of the multiple images. Each of the multiple images may be referred to as a layer, a layer image, and / or a partial image. References will follow below. Figure 4 Provides a more detailed description of LDAT for multiple layers of data.
[0034] Mixer 110 generates first image data IDAT by mixing multiple layer data LDATs. The first image data IDAT includes multiple pixel values corresponding to the screen. For example, a display device may include multiple pixels, and each of the multiple pixels may have a corresponding one of the multiple pixel values. For example, each of the multiple pixel values may include a grayscale value, illuminance value, and / or luminance value of a corresponding one of the multiple pixels. Similarly, each of the multiple layer data LDATs may include a pixel value corresponding to a corresponding one of the multiple images. (Refer to...) Figure 3 and Figure 4 A more detailed description is provided for LDAT used to mix data from multiple layers.
[0035] Blending refers to the operation of calculating the actual pixel values displayed across multiple layers (e.g., images) that make up a screen. When blending is performed, the pixel values actually displayed on each pixel can be obtained. For example, when only one layer is set, arranged, or placed on a pixel, the pixel values included in that layer can be obtained as is. When two or more layers are set on a pixel, the pixel values included in one of the two or more layers can be obtained, or new pixel values can be obtained based on the pixel values included in the two or more layers. Blending can be referred to as combining and / or merging.
[0036] Mixer 110 generates pixel mapping data PMDAT based on multiple layer data LDAT. The pixel mapping data PMDAT includes multiple pixel identities (IDs) representing display quality enhancement algorithms (or image quality improvement algorithms) to be applied to the multiple pixel values. For example, as will be described below with reference to FIG7, each of the multiple pixels may have a corresponding one of the multiple pixel IDs. For example, each pixel ID for each pixel may be set based on mixing information (e.g., what pixel value is actually obtained for each pixel through mixing). Each of the multiple pixel IDs may be referred to as a display quality enhancement algorithm setting ID, etc.
[0037] Display quality enhancer 120 generates second image data EDAT based on first image data IDAT and pixel mapping data PMDAT by applying different display quality enhancement algorithms to the plurality of pixel values. The second image data EDAT includes a plurality of display quality enhanced pixel values. For example, as with the plurality of pixel values, each of the plurality of pixels may have a corresponding one of the plurality of display quality enhanced pixel values. The plurality of pixels may emit light based on the plurality of display quality enhanced pixel values to display an image corresponding to the screen.
[0038] In the image processing apparatus 100 according to the example embodiment, blending can be performed on multiple layers constituting a screen, pixel IDs can be generated, and pixel mapping data PMDAT as a set of pixel IDs can be generated. Each pixel ID can represent an optimized (or optimized) display quality enhancement algorithm that will be applied to each pixel value obtained as a result of blending. Furthermore, the optimized display quality enhancement algorithm can be applied to the multiple pixels based on the pixel mapping data PMDAT, and different display quality enhancement algorithms can be applied on a pixel-by-pixel basis. Therefore, display quality optimization can be achieved for each pixel, and display quality can be enhanced or improved.
[0039] Figure 2 This is a detailed illustration of an exemplary embodiment. Figure 1 A block diagram of an image processing device. (The referenced portion will be omitted.) Figure 1 The provided description.
[0040] Reference Figure 2 The image processing apparatus 100a includes a mixer 110a and a display quality enhancer 120a.
[0041] exist Figure 2 In the example, the multiple layer data LDAT may include first layer data to Kth layer data LDAT1, LDAT2, ..., LDATK, and the multiple images may include first image to Kth image, where K is a natural number greater than or equal to two. Additionally, the multiple pixel values included in the first image data IDAT may include first pixel values to Nth pixel values PV1, PV2, ..., PVN; the multiple pixel IDs included in the pixel mapping data PMDAT may include first pixel IDs to Nth pixel IDs PID1, PID2, ..., PIDN; and the multiple display quality enhancement pixel values included in the second image data EDAT may include first display quality enhancement pixel values to Nth display quality enhancement pixel values EPV1, EPV2, ..., EPVN, where N is a natural number greater than or equal to two. Furthermore, the multiple display quality enhancement algorithms applicable to each pixel value may include first display quality enhancement algorithms to Mth display quality enhancement algorithms, where M is a natural number greater than or equal to two.
[0042] Mixer 110a may include mixing block 112 and pixel map generator 114.
[0043] The blending block 112 can synthesize first to Kth images based on first to Kth layer data LDAT1, LDAT2, ..., LDATK to generate first to Nth pixel values PV1, PV2, ..., PVN corresponding to a composite image (or combined image) to be actually displayed on the screen. For example, the blending block 112 can determine the arrangement of these layers. That is, the blending block 112 can determine which layer is placed on top and which layer is placed below. For example, the blending block 112 can determine the display scheme of these layers, such as displaying only the topmost layer or displaying the layer placed on top as semi-transparent to partially display the layer placed below. For example, the blending block 112 can obtain or acquire the first to Nth pixel values PV1, PV2, ..., PVN constituting a composite image based on the above determination.
[0044] The pixel mapping generator 114 can generate first pixel IDs to Nth pixel IDs PID1, PID2, ..., PIDN based on the first layer data to the Kth layer data LDAT1, LDAT2, ..., LDATK, for the first pixel values to the Nth pixel values PV1, PV2, ..., PVN corresponding to the composite image. For example, the pixel mapping generator 114 can set each pixel ID based on each pixel value included in each layer. For example, the first pixel ID PID1 can correspond to the first pixel value PV1, the second pixel ID PID2 can correspond to the second pixel value PV2, and the Nth pixel ID PIDN can correspond to the Nth pixel value PVN.
[0045] In some example embodiments, when two or more images overlap and are set on a first pixel corresponding to a first pixel value PV1 (e.g., when the first pixel corresponds to two or more layer data), the pixel mapping generator 114 may generate a first pixel ID PID1 corresponding to the first pixel value PV1 based on one of the two or more layer data.
[0046] In some example embodiments, pixel IDs corresponding to pixel values included in the same layer may have the same value. In other words, the same pixel ID can be set for each layer. However, the example embodiments are not limited to this. In other example embodiments, some pixel IDs corresponding to pixel values included in the same layer may have different values, or some pixel IDs corresponding to pixel values included in different layers may have the same value.
[0047] The display quality enhancer 120a may include multiple registers (REG1, REG2, ..., REGM) 122a, 122b, ..., 122m, a multiplexer 124, and an enhancement block 126.
[0048] Multiple registers 122a, 122b, ..., 122m can store multiple display quality enhancement parameters for the first to the Mth display quality enhancement algorithms. For example, the first register 122a can store at least one display quality enhancement parameter for the first display quality enhancement algorithm, the second register 122b can store at least one display quality enhancement parameter for the second display quality enhancement algorithm, and the Mth register 122m can store at least one display quality enhancement parameter for the Mth display quality enhancement algorithm.
[0049] In some example embodiments, each of the plurality of registers 122a, 122b, ..., 122m may be a configuration register and may include, for example, a special function register (SFR). The plurality of display quality enhancement parameters may be stored in the plurality of registers 122a, 122b, ..., 122m in the form of a lookup table (LUT), or may be stored in various ways representing a display quality enhancement algorithm.
[0050] In some example embodiments, the plurality of display quality enhancement algorithms may include detail enhancement (DE), scaling (or scalers), adaptive tone mapping control (ATC), hue and saturation control (HSC), gamma and degamma, Android Open Source Project (AOSP), color gamut control (CGC), dithering (or slicing), rounded corner display (RCD), subpixel rendering (SPR), etc. DE can represent an algorithm for sharpening image contours. Scaling can represent an algorithm for changing image size. ATC can represent an algorithm for improving outdoor visibility. HSC can represent an algorithm for improving the hue and saturation of colors. Gamma can represent an algorithm for gamma correction or compensation. AOSP can represent an algorithm for processing image transformation matrices defined by the Android OS (e.g., colorblind mode or night mode). CGC can represent an algorithm for matching the color coordinates of the display panel. Dithering can represent an algorithm for using a finite number of colors to represent high-bit color effects. RCD can represent an algorithm for processing the rounded corners of the display panel. SPR can represent an algorithm for improving resolution. However, the example embodiments are not limited thereto, and the plurality of display quality enhancement algorithms may also include a variety of other algorithms.
[0051] Multiplexer 124 can select at least one of a first to an Mth display quality enhancement algorithm based on a first pixel ID to an Nth pixel ID PID1, PID2, ..., PIDN. For example, multiplexer 124 can select at least one display quality enhancement algorithm for a first pixel value PV1 based on a first pixel ID PID1. Multiplexer 124 can select at least one display quality enhancement algorithm for a second pixel value PV2 based on a second pixel ID PID2. Multiplexer 124 can select at least one display quality enhancement algorithm for a Nth pixel value PVN based on an Nth pixel ID PIDN.
[0052] In some example embodiments, only one of the first to Mth display quality enhancement algorithms may be selected for a pixel based on a pixel ID. In other example embodiments, two or more of the first to Mth display quality enhancement algorithms may be selected for a pixel based on a pixel ID.
[0053] In some example embodiments, each of the first pixel ID to the Nth pixel ID PID1, PID2, ..., PIDN may include M bits, and at least one of the first to Mth display quality enhancement algorithms can be selected based on the respective bit values indicated in the Nth pixel ID. For example, the first pixel ID PID1 may include the first to Mth bits, and at least one display quality enhancement algorithm corresponding to the bit value of "1" in the first to Mth bits can be selected. For example, the first to Mth bits may correspond to the first to Mth display quality enhancement algorithms, respectively. When only the first bit is "1", the first display quality enhancement algorithm can be selected only for the first pixel value PV1. When both the first and second bits are "1", both the first and second display quality enhancement algorithms can be selected for the first pixel value PV1.
[0054] Enhancement block 126 can generate first display quality enhanced pixel values to Nth display quality enhanced pixel values EPV1, EPV2, ..., EPVN based on first pixel values to Nth pixel values PV1, PV2, ..., PVN and the output of multiplexer 124. For example, enhancement block 126 can generate a first display quality enhanced pixel value EPV1 by applying at least one display quality enhancement algorithm selected based on a first pixel ID PID1 to the first pixel value PV1. Enhancement block 126 can generate a second display quality enhanced pixel value EPV2 by applying at least one display quality enhancement algorithm selected based on a second pixel ID PID2 to the second pixel value PV2. Enhancement block 126 can generate an Nth display quality enhanced pixel value EPVN by applying at least one display quality enhancement algorithm selected based on an Nth pixel ID PIDN to the Nth pixel value PVN.
[0055] In some example embodiments, the operations of selecting a display quality enhancement algorithm and generating a display quality enhancement pixel value can be performed sequentially for each of the first pixel value to the Nth pixel value PV1, PV2, ..., PVN. For example, the operations of selecting a display quality enhancement algorithm for the first pixel value PV1 based on the first pixel ID PID1 and generating a first display quality enhancement pixel value EPV1 based on the first pixel value PV1 can be performed sequentially. Next, the operations of selecting a display quality enhancement algorithm for the second pixel value PV2 based on the second pixel ID PID2 and generating a second display quality enhancement pixel value EPV2 based on the second pixel value PV2 can be performed sequentially. Afterward, the operations of selecting a display quality enhancement algorithm for the Nth pixel value PVN based on the Nth pixel ID PIDN and generating an Nth display quality enhancement pixel value EPVN based on the Nth pixel value PVN can be performed sequentially.
[0056] In other example embodiments, the operation of selecting a display quality enhancement algorithm can be performed sequentially for all pixel values from the first pixel value to the Nth pixel value PV1, PV2, ..., PVN. Then, the operation of generating a display quality enhancement pixel value can be performed sequentially for all pixel values from the first pixel value to the Nth pixel value PV1, PV2, ..., PVN. For example, the operation of selecting a display quality enhancement algorithm for the first pixel value PV1 based on the first pixel ID PID1 can be performed, then the operation of selecting a display quality enhancement algorithm for the second pixel value PV2 based on the second pixel ID PID2 can be performed, and then the operation of selecting a display quality enhancement algorithm for the Nth pixel value PVN based on the Nth pixel ID PIDN can be performed. Afterwards, the operation of generating a first display quality enhancement pixel value EPV1 based on the first pixel value PV1 can be performed, then the operation of generating a second display quality enhancement pixel value EPV2 based on the second pixel value PV2 can be performed, and then the operation of generating an Nth display quality enhancement pixel value EPVN based on the Nth pixel value PVN can be performed.
[0057] Figure 3 , Figure 4 , Figure 5A , Figure 5B , Figure 5C , Figure 5D , Figure 5E , Figure 5F , Figure 5G , Figure 6 , Figure 7A , Figure 7B , Figure 7C , Figure 8 , Figure 9 , Figure 10A and Figure 10B This is a diagram used to describe the operation of an image processing apparatus according to one or more exemplary embodiments.
[0058] Reference Figure 3The display device 200, which displays an image based on the second image data EDAT output from the image processing device 100, may include a plurality of pixels P11, P12, P13, P14, P15, P16, P17, P18, P21, P22, P23, P24, P25, P26, P27, P28, P31, P32, P33, P34, P35, P36, P37, P38, P41, P42, P43, P44, P45, P46, P47, P48, P51, P52, P53, P54, P55, P56, P57, P58, P61, P62, P 63, P64, P65, P66, P67, P68, P71, P72, P73, P74, P75, P76, P77, P78, P81, P82, P83, P84, P85, P86, P87, P88, P91, P92, P93, P94, P95, P96, P97, P98, PA1, PA2, PA3, PA4, PA5, PA6, PA7, PA8, PB1, PB2, PB3, PB4, PB5, PB6, PB7, PB8, PC1, PC2, PC3, PC4, PC5, PC6, PC7, and PC8.
[0059] Here, each pixel may include a light-emitting element (e.g., an organic light-emitting diode (OLED)) and at least one transistor for driving the light-emitting element.
[0060] Although Figure 3 The display device 200 shown includes 12*8 pixels, but the example embodiment is not limited thereto.
[0061] Reference Figure 4 The following is shown: Figure 3 A composite image CIMG is displayed on the display device 200. A composite image CIMG can represent an image fully displayed on one screen of the display device 200, and can represent an image synthesized from a reference image. Figures 5A to 5G An image obtained by describing multiple layers (e.g., multiple images).
[0062] The composite image CIMG may include multiple pixel values: PC_11, PC_12, PC_13, PC_14, PC_15, PC_16, PC_17, PC_18, PB_21, PB_22, PB_23, PB_24, PB_25, PB_26, PB_27, PB_28, PB_31, PB_32, PG_33, PB_34, PE_35, PE_36, PB_37, PB_38, PB_41, PF_42, PG_43, PF_44, PF_45, PF_46, PF_47, PB_48, PB_51, PF_52, PG_53, PF_54, PF_55, PF_56, PF_57, PB_58, PB_61, PB_62, PG_63, PB_64, PE_65, PE_66, PB_6 7. PB_68, PB_71, PB_72, PB_73, PB_74, PB_75, PB_76, PB_77, PB_78, PB_81, PB_82, PB_83, PB_84, PB_85, PB_86, PB_87, PB_88, PB_91, PD_92, PD_93, PD_94, PD_95, PD_96, PD_97, PB _98, PB_A1, PD_A2, PD_A3, PD_A4, PD_A5, PD_A6, PD_A7, PB_A8, PB_B1, PB_B2, PB_B3, PB_B 4. PB_B5, PB_B6, PB_B7, PB_B8, PA_C1, PA_C2, PA_C3, PA_C4, PA_C5, PA_C6, PA_C7 and PA_C8.
[0063] exist Figure 4 In the following diagrams, each pixel value can correspond to each pixel at the same location or position. For example, Figure 4 The pixel value PC_11 in the image can correspond to Figure 3 Pixel P11 in the image, and pixel P11 can have a pixel value PC_11, and can emit light based on the pixel value PC_11.
[0064] Reference Figure 5A , Figure 5B , Figure 5C , Figure 5D , Figure 5E , Figure 5F and Figure 5G , Figure 4 The composite image CIMG can be an image obtained by synthesizing layers LYA, LYB, LYC, LYD, LYE, LYF and LYG from the first to the seventh layer.
[0065] Figure 5ALYA cross-section (PA_11, PA_12, P A_13、PA_14、PA_15、PA_16、PA_17、PA_18、PA_ 21、PA_22、PA_23、PA_24、PA_25、PA_26、PA_27、PA_28、PA_31、PA_32、PA_33、PA_34、PA_35、 PA_36, PA_37, PA_38, PA_41, PA_42, PA_43, PA_44, PA_45, PA_46, PA_47, PA_48, PA_51, PA_ 52、PA_53、PA_54、PA_55、PA_56、PA_57、PA_58、PA_61、PA_62、PA_63、PA_64、PA_65、PA_66、 PA_67, PA_68, PA_71, PA_72, PA_73, PA_74, PA_75, PA_76, PA_77, PA_78, PA_81, PA_82, PA_ 83、PA_84、PA_85、PA_86、PA_87、PA_88、PA_91、PA_92、PA_93、PA_94、PA_95、PA_96、PA_97、 PA_98, PA_A1, PA_A2, PA_A3, PA_A4, PA_A5, PA_A6, PA_A7, PA_A8, PA_B1, PA_B2, PA_B3, PA_ B4, PA_B5, PA_B6, PA_B7, PA_B8, PA_C1, PA_C2, PA_C3, PA_C4, PA_C5, PA_C6, PA_C7 and PA_C8.
[0066] Figure 5BThe second layer (e.g., the second image) LYB may include multiple pixel values PB_11, PB_12, PB_13, PB_14, PB_15, PB_16, PB_17, PB_18, PB_21, PB_22, PB_23, PB_24, PB_25, PB_26, PB_27, PB_28, PB_31, PB_32, PB_33, PB_34, PB_35, PB_36, PB_37, PB_38, PB_41, PB_42, PB_43, PB_44, PB_45, PB_46, PB_47, PB_48, PB_51, PB_52, PB_53, PB_54, PB_55, PB_56, PB_57, PB_58, PB_61, PB_62. PB_63, PB_64, PB_65, PB_66, PB_67, PB_68, PB_71, PB_72, PB_73, PB_74, PB_75, PB_76, PB_77, PB_78, PB_81, PB_82, PB_83, PB_84, PB_85, PB_86, PB_87, PB_88, PB_91, PB_92, PB_93, PB_94, PB_95, PB_96, PB_97, PB_98, PB_A1, PB_A2, PB_A3, PB_A4, PB_A5, PB_A6, PB_A7, PB_A8, PB_B1, PB_B2, PB_B3, PB_B4, PB_B5, PB_B6, PB_B7 and PB_B8.
[0067] Figure 5C The third layer (e.g., the third image) LYC may include multiple pixel values PC_11, PC_12, PC_13, PC_14, PC_15, PC_16, PC_17, and PC_18.
[0068] Figure 5D The fourth layer (e.g., the fourth image) LYD may include multiple pixel values PD_92, PD_93, PD_94, PD_95, PD_96, PD_97, PD_A2, PD_A3, PD_A4, PD_A5, PD_A6, and PD_A7.
[0069] Figure 5E The fifth layer (e.g., the fifth image) LYE may include multiple pixel values PE_35, PE_36, PE_45, PE_46, PE_55, PE_56, PE_65, and PE_66.
[0070] Figure 5FThe sixth layer (e.g., the sixth image) LYF may include multiple pixel values PF_42, PF_43, PF_44, PF_45, PF_46, PF_47, PF_52, PF_53, PF_54, PF_55, PF_56 and PF_57.
[0071] Figure 5G The seventh layer (e.g., the seventh image) LYG may include multiple pixel values PG_33, PG_43, PG_53 and PG_63.
[0072] In some example embodiments, each of the first layer LYA to the seventh layer LYG may represent an application executed by and displayed thereon by an electronic device (or electronic system) including display device 200. However, the example embodiments are not limited thereto.
[0073] exist Figures 5A to 5G In this context, a portion indicated by blank space (e.g., a portion that does not include or describe pixel values) may be an area without pixel values (e.g., an area where there is no corresponding image).
[0074] exist Figure 4 In the composite image CIMG, Figure 5A The first layer LYA to Figure 5G The seventh layer of LYG can be overlapped and set sequentially. For example, Figure 5A The first layer of LYA can be set at the bottom. Figure 5G The seventh layer, LYG, can be set at the top, and only the topmost layer on each pixel can be displayed. Therefore, the composite image CIMG can include only the pixel values PA_C1, PA_C2, PA_C3, PA_C4, PA_C5, PA_C6, PA_C7, and PA_C8 from the first layer, LYA. Figure 5A(The last row of pixel values shown). Similarly, the composite image CIMG may only include the pixel values PB_21, PB_22, PB_23, PB_24, PB_25, PB_26, PB_27, PB_28, PB_31, PB_32, PB_34, PB_37, PB_38, PB_41, PB_48, PB_51, PB_58, PB_61, PB_62, PB_64, PB_67, PB_68, PB_7 from the second layer LYB. 1. PB_72, PB_73, PB_74, PB_75, PB_76, PB_77, PB_78, PB_81, PB_82, PB_83, PB_84, PB_85, PB_86, PB_87, PB_88, PB_91, PB_98, PB_A1, PB_A8, PB_B1, PB_B2, PB_B3, PB_B4, PB_B5, PB_B6, PB_B7, and PB_B8. The composite image CIMG may only include pixel values PC_11, PC_12, PC_13, PC_14, PC_15, PC_16, PC_17, and PC_18 from the third layer LYC. The composite image CIMG may consist only of pixel values PD_92, PD_93, PD_94, PD_95, PD_96, PD_97, PD_A2, PD_A3, PD_A4, PD_A5, PD_A6, and PD_A7 from the fourth layer LYD. The composite image CIMG may consist only of pixel values PE_35, PE_36, PE_65, and PE_66 from the fifth layer LYE. The composite image CIMG may consist only of pixel values PF_42, PF_44, PF_45, PF_46, PF_47, PF_52, PF_54, PF_55, PF_56, and PF_57 from the sixth layer LYF. The composite image CIMG may consist only of pixel values PG_33, PG_43, PG_53, and PG_63 from the seventh layer LYG.
[0075] Reference Figure 6 This shows when displayed on the display device 200 Figure 4 CIMG composite images Figure 3 The arrangement of layers on the pixels P45 of the display device 200.
[0076] For example, the first layer LYA, the second layer LYB, the fifth layer LYE, and the sixth layer LYF can be overlapped and set on pixel P45. At least one display quality enhancement algorithm for pixel P45 can be selected or determined based on one of the first layer LYA, the second layer LYB, the fifth layer LYE, and the sixth layer LYF.
[0077] In some example embodiments, the sixth layer LYF can be displayed as the topmost layer among the first layer LYA, the second layer LYB, the fifth layer LYE, and the sixth layer LYF, and pixel P45 can have the pixel value PF_45 included in the sixth layer LYF. The at least one display quality enhancement algorithm for pixel P45 can be selected based on the sixth layer LYF (e.g., based on layer data corresponding to the sixth layer LYF), and a pixel ID corresponding to the selected display quality enhancement algorithm can be generated.
[0078] In some example embodiments, the sixth layer LYF can be displayed as semi-transparent, and the fifth layer LYE, which is set below the sixth layer LYF, can be partially displayed.
[0079] Reference Figure 7A , showing the corresponding Figure 4 The pixel map PMAP generated from the composite image CIMG.
[0080] Pixel Mapping (PMAP) can include multiple pixel IDs. ID_11, ID_12, ID_13, ID_14, ID_15, ID_16, ID_17, ID_18, ID_21, ID_22, ID_23, ID_24, ID_25, ID_26, ID_27, ID_28, ID_31, ID_32, ID_33, ID_34, ID_35, ID_36, ID_37, ID_38, ID_41, ID_42, ID_43, ID_44, ID_45, ID_46, ID_47, ID_48, ID_51, ID_52, ID_53, ID_54, ID_55, ID_56, ID_57, ID_58, ID_61, ID_62, ID_63, ID_64, ID_65, ID_66, ID_67, ID_68, ID_71, ID_72, ID_73, ID_74, ID_75, ID_76, ID_77, ID_78, ID_81, ID_82, ID_83, ID_84, ID_85, ID_86, ID_87, ID_88, ID_91, ID_92, ID_93, ID_94, ID_95, ID_96, ID_97, ID_98, ID_A1, ID_A2, ID_A3, ID_A4, ID_A5, ID_A6, ID_A7, ID_A8, ID_B1, ID_B2, ID_B3, ID_B4, ID_B5, ID_B6, ID_B7, ID_B8, ID_C1, ID_C2, ID_C3, ID_C4, ID_C5, ID_C6, ID_C7 and ID_C8.
[0081] exist Figure 7AIn the following diagrams, each pixel ID can correspond to each pixel and each pixel value at the same location. For example, Figure 7A The pixel ID ID_11 in the image can correspond to Figure 3 Pixel P11 and Figure 4 The pixel value PC_11.
[0082] Reference Figure 7B and Figure 7C , showed Figure 7A A specific example of a pixel-mapped image (PMAP).
[0083] exist Figure 7BIn the pixel mapping PMAP1, all pixel IDs corresponding to pixel values included in the same layer can have the same label or pixel ID. For example, the pixel IDs ID_C1, ID_C2, ID_C3, PA_C4, PA_C5, PA_C6, PA_C7, and ID_C8 corresponding to pixel values PA_C1, PA_C2, PA_C3, PA_C4, ID_C5, ID_C6, ID_C7, and ID_C8 included in the first layer LYA can have the label "a". These correspond to the pixel values PB_21, PB_22, PB_23, PB_24, PB_25, PB_26, PB_27, PB_28, PB_31, PB_32, PB_34, PB_37, PB_38, PB_41, PB_48, PB_51, PB_58, PB_61, PB_62, PB_64, PB_67, PB_68, PB_71, and PB_72 included in the second LYB layer. PB_73, PB_74, PB_75, PB_76, PB_77, PB_78, PB_81, PB_82, PB_83, PB_84, PB_85, PB_86, PB_87, PB Pixel IDs for _88, PB_91, PB_98, PB_A1, PB_A8, PB_B1, PB_B2, PB_B3, PB_B4, PB_B5, PB_B6, PB_B7, and PB_B8 ID_21, ID_22, ID_23, ID_24, ID_25, ID_26, ID_27, ID_28, ID_31, ID_32, ID_34, ID_37, ID_38 , ID_41, ID_48, ID_51, ID_58, ID_61, ID_62, ID_64, ID_67, ID_68, ID_71, ID_72, ID_73, ID_7 4. ID_75, ID_76, ID_77, ID_78, ID_81, ID_82, ID_83, ID_84, ID_85, ID_86, ID_87, ID_88, ID_91, ID_98, ID_A1, ID_A8, ID_B1, ID_B2, ID_B3, ID_B4, ID_B5, ID_B6, ID_B7, and ID_B8 can have the label "b". The pixel IDs corresponding to the pixel values PC_11, PC_12, PC_13, PC_14, PC_15, PC_16, PC_17, and PC_18 included in the third layer LYC, ID_11, ID_12, ID_13, ID_14, ID_15, ID_16, ID_17, and ID_18, can have the label "c".The pixel IDs ID_92, ID_93, ID_94, PD_95, PD_96, PD_97, PD_A2, PD_A3, PD_A4, PD_A5, PD_A6, and PD_A7 corresponding to the pixel values PD_92, PD_93, ID_94, ID_95, ID_96, ID_97, ID_A2, ID_A3, ID_A4, ID_A5, ID_A6, and ID_A7 included in the fourth layer LYD can have the label "d". The pixel IDs ID_35, ID_36, ID_65, and ID_66 corresponding to the pixel values PE_35, PE_36, PE_65, and PE_66 included in the fifth layer LYE can have the label "e". The pixel IDs ID_42, ID_44, ID_45, ID_46, PF_47, PF_52, PF_54, PF_55, PF_56, and PF_57 corresponding to the pixel values PF_42, PF_44, PF_45, PF_46, PF_47, ID_52, ID_54, ID_55, ID_56, and ID_57 included in the sixth layer LYF can have the label "f". The pixel IDs ID_33, ID_43, ID_53, and ID_63 corresponding to the pixel values PG_33, PG_43, PG_53, and PG_63 included in the seventh layer LYG can have the label "g".
[0084] exist Figure 7C In PMAP2 pixel mapping, some pixel IDs corresponding to pixel values included in the same layer can have different labels or pixel IDs. In other words, a layer is not limited to one pixel ID value. (The above references will be omitted.) Figure 7BThe provided description is as follows. For example, among the pixel values included in the fourth layer LYD, the pixel IDs ID_92, ID_93, ID_94, PD_A2, PD_A3, and ID_A4 corresponding to pixel values PD_92, PD_93, PD_94, PD_A2, PD_A3, and ID_A4 can have the label "d1", and the pixel IDs ID_95, ID_96, ID_97, PD_A5, PD_A6, and ID_A7 corresponding to pixel values PD_95, PD_96, PD_97, PD_A5, PD_A6, and ID_A7 can have the label "d2". Among the pixel values included in the fifth layer LYE, the pixel IDs ID_35 and ID_36 corresponding to pixel values PE_35 and PE_36 can have the label "e1", and the pixel IDs ID_65 and ID_66 corresponding to pixel values PE_65 and PE_66 can have the label "e2". In the sixth layer LYF, the pixel values corresponding to pixel values PF_42, PF_44, PF_52, and PF_54 can have the label "f1", and the pixel IDs corresponding to pixel values PF_45, PF_46, PF_47, PF_55, PF_56, and PF_57 can have the label "f2". In the seventh layer LYG, the pixel values corresponding to pixel values PG_33 and PG_43 can have the label "g1", and the pixel IDs corresponding to pixel values PG_53 and PG_63 can have the label "g2".
[0085] Reference Figure 8 This shows that based on Figure 7B PMAP1 (Pixel Mapping Mapping 1) applies different optimized display quality enhancement algorithms on a pixel-by-pixel basis. Figure 4 The composite image CIMG' is generated from the composite image CIMG. In other words, Figure 8 The composite image CIMG' can be an image actually displayed on the display device 200 based on the second image data EDAT output from the image processing device 100, and can be an image in which display quality enhancement is performed on a pixel-by-pixel basis.
[0086] exist Figure 8In the composite image CIMG', pixel values PA_C1a, PA_C2a, PA_C3a, PA_C4a, PA_C5a, PA_C6a, PA_C7a, and PA_C8a can be generated by applying at least one display quality enhancement algorithm based on pixel IDs labeled "a". Pixel values PB_21b, PB_22b, PB_23b, PB_24b, PB_25b, PB_26b, PB_27b, PB_28b, PB_31b, PB_32b, PB_34b, PB_37b, PB_38b, PB_41b, PB_48b, PB_51b, PB_58b, PB_61b, PB_62b, PB_64b, PB_67b, PB_68b, and PA_C8a can be generated by applying at least one display quality enhancement algorithm based on pixel IDs labeled "b". PB_71b, PB_72b, PB_73b, PB_74b, PB_75b, PB_76b, PB_77b, PB_78b, PB_81b, PB_82b, PB_83b, PB_84b, PB_85b, PB_86b, PB_87b, PB_88b, PB_91b, PB_98b, PB_A1b, PB_A8b, PB_B1b, PB_B2b, PB_B3b, PB_B4b, PB_B5b, PB_B6b, PB_B7b, and PB_B8b. Pixel values PC_11c, PC_12c, PC_13c, PC_14c, PC_15c, PC_16c, PC_17c, and PC_18c can be generated by applying at least one display quality enhancement algorithm based on pixel IDs labeled "c". Pixel values PD_92d, PD_93d, PD_94d, PD_95d, PD_96d, PD_97d, PD_A2d, PD_A3d, PD_A4d, PD_A5d, PD_A6d, and PD_A7d can be generated by applying at least one display quality enhancement algorithm based on pixel IDs labeled "d". Pixel values PE_35e, PE_36e, PE_65e, and PE_66e can be generated by applying at least one display quality enhancement algorithm based on pixel IDs labeled "e". Pixel values PF_42f, PF_44f, PF_45f, PF_46f, PF_47f, PF_52f, PF_54f, PF_55f, PF_56f, and PF_57f can be generated by applying at least one display quality enhancement algorithm based on pixel IDs labeled "f". Pixel values PG_33g, PG_43g, PG_53g, and PG_63g can be generated by applying at least one display quality enhancement algorithm based on the pixel ID selected by the label "g".
[0087] Reference Figure 9 , showing the corresponding Figure 4 The pixel map PMAP generated from the composite image CIMG. Figure 9 An example is shown where pixel IDs are generated only for some pixel values.
[0088] The pixel map PMAP' can include multiple pixel IDs: ID_33, ID_35, ID_36, ID_42, ID_43, ID_44, ID_45, ID_46, ID_47, ID_52, ID_53, ID_54, ID_55, ID_56, ID_57, ID_63, ID_65, ID_66, ID_92, ID_93, ID_94, ID_95, ID_96, ID_97, ID_A2, ID_A3, ID_A4, ID_A5, ID_A6, and ID_A7. Figure 9 In the blank space, a portion of it (e.g., a portion where pixel IDs are not included or described) may be an area where pixel IDs are not generated.
[0089] In some example embodiments, the display quality enhancement algorithm may be applied only to the pixel values for which a pixel ID has been generated. For example, when based on Figure 9 The pixel mapping PMAP' will show the quality enhancement algorithm applied to Figure 4When synthesizing CIMG images, different optimized display quality enhancement algorithms can be applied to the pixel values PG_33, PE_35, PE_36, PF_42, PG_43, PF_44, PF_45, PF_46, PF_47, PF_52, PG_53, PF_54, PF_55, PF_56, PF_57, PG_63, PE_65, PE_66, PD_92, PD_93, PD_94, PD_95, PD_96, PD_97, PD_A2, PD_A3, PD_A4, PD_A5, PD_A6, and PD_A7. The display quality enhancement algorithm may not be applied to pixel values PC_11, PC_12, PC_13, PC_14, PC_15, PC_16, PC_17, PC_18, PB_21, PB_22, PB_23, PB_24, PB_25, PB_26, PB_27, PB_28, PB_31, PB_32, PB_34, PB_37, PB_38, PB_41, PB_48, PB_51, PB_58, PB_61, PB_62, PB_64, PB_67, PB_68, PB_71, PB_ 72. PB_73, PB_74, PB_75, PB_76, PB_77, PB_78, PB_81, PB_82, PB_83, PB_84, PB_85, PB_86, PB_87, PB_88, PB_91, PB_98, PB_A 1. PB_A8, PB_B1, PB_B2, PB_B3, PB_B4, PB_B5, PB_B6, PB_B7, PB_B8, PA_C1, PA_C2, PA_C3, PA_C4, PA_C5, PA_C6, PA_C7 and PA_C8.
[0090] In other example embodiments, for pixel values for which a pixel ID has been generated, a display quality enhancement algorithm can be applied based on the current pixel ID. For pixel values for which no pixel ID has been generated, a display quality enhancement algorithm based on a previous pixel ID stored in memory can be applied. For example, when based on... Figure 9 The pixel mapping PMAP' will show the quality enhancement algorithm applied to Figure 4When synthesizing a CIMG image, different optimized display quality enhancement algorithms can be applied pixel by pixel to the pixel values PG_33, PE_35, PE_36, PF_42, PG_43, PF_44, PF_45, PF_46, PF_47, PF_52, PG_53, PF_54, PF_55, PF_56, PF_57, PG_63, PE_65, PE_66, PD_92, PD_93, PD_94, PD_95, PD_96, PD_97, PD_A2, PD_A3, PD_A4, PD_A5, PD_A6, and PD_A7 based on the pixel IDs included in the pixel map PMAP'. Alternatively, algorithms can be applied based on previously generated pixel maps (e.g., Figure 7A The pixel IDs included in the pixel map (PMAP) are pixel-based. Different optimized display quality enhancement algorithms are applied to the pixel values in the composite image CIMG: PC_11, PC_12, PC_13, PC_14, PC_15, PC_16, PC_17, PC_18, PB_21, PB_22, PB_23, PB_24, PB_25, PB_26, PB_27, PB_28, PB_31, PB_32, PB_34, PB_37, PB_38, PB_41, PB_48, PB_51, PB_58, PB_61, PB_62, PB_64, PB_ 67. PB_68, PB_71, PB_72, PB_73, PB_74, PB_75, PB_76, PB_77, PB_78, PB_81, PB_82, PB_83, PB_84, PB_85, PB_86, PB_87, PB_88, PB_91, P B_98, PB_A1, PB_A8, PB_B1, PB_B2, PB_B3, PB_B4, PB_B5, PB_B6, PB_B7, PB_B8, PA_C1, PA_C2, PA_C3, PA_C4, PA_C5, PA_C6, PA_C7 and PA_C8.
[0091] Figure 10A and Figure 10B An example is shown in which multiple frame images displayed on the display device 200 are generated in sequence based on first image data IDAT, pixel mapping data PMDAT, and second image data EDAT.
[0092] exist Figure 10A and Figure 10BIn this context, each of the frame images F1, F2, F3, F4, F5, F6, F7, F8, F9, and F10 can correspond to a composite image displayed based on the first image data IDAT, each of the pixel maps PM1, PM2, PM3, PM4, PM5, PM6, PM7, PM8, PM9, and PM10 can correspond to pixel map data PMDAT, and each of the display quality enhanced frame images EF1, EF2, EF2', EF3, EF4, EF4', EF5, EF6, EF6', EF7, EF8, EF8', EF9, EF10, and EF10' can correspond to a composite image with enhanced display quality and displayed based on the second image data EDAT.
[0093] exist Figure 10A In the example, first image data IDAT and pixel mapping data PMDAT can be generated for each of multiple frames, and second image data EDAT can be generated for each frame based on the first image data IDAT and pixel mapping data PMDAT. For example, in the first frame, a first frame image F1 and a first pixel mapping PM1 can be generated, and a first display quality enhancement frame image EF1 can be generated based on the first frame image F1 and the first pixel mapping PM1. In the second frame following the first frame, a second frame image F2 and a second pixel mapping PM2 can be generated, and a second display quality enhancement frame image EF2 can be generated based on the second frame image F2 and the second pixel mapping PM2.
[0094] exist Figure 10B In the example, first image data IDAT can be generated for each frame, pixel mapping data PMDAT can be generated for X frames (or every X frames), where X is a natural number greater than or equal to two, and second image data EDAT can be generated for each frame based on the first image data IDAT and the pixel mapping data PMDAT. Figure 10B An example where X = 2 is shown. For example, in the first frame, a first frame image F1 and a first pixel map PM1 can be generated, and a first display quality enhanced frame image EF1 can be generated based on the first frame image F1 and the first pixel map PM1. In the second frame after the first frame, a second frame image F2 can be generated, a second pixel map PM2 can be not generated, and a second display quality enhanced frame image EF2' can be generated based on the currently generated second frame image F2 and the previously generated first pixel map PM1.
[0095] Although Figure 10B An example is shown in which pixel mapping is generated uniformly for each odd-numbered frame; however, the example embodiment is not limited to this, and pixel mapping can be generated uniformly or irregularly for frames of arbitrary intervals. Furthermore, as shown in the reference... Figure 7AThe aforementioned method can generate a pixel map (PMAP) including pixel IDs corresponding to all pixel values for some frames (e.g., for odd-numbered frames), and as referred to Figure 9 The aforementioned method can generate a pixel map (PMAP) that includes only pixel IDs corresponding to some pixel values for other frames (e.g., for even-numbered frames).
[0096] While exemplary embodiments are described based on a specific number of pixels, layers, pixel values, pixel IDs, and frames, the one or more embodiments are not limited thereto.
[0097] Figure 11 This is a block diagram illustrating a display controller including an image processing apparatus according to an exemplary embodiment. The above-mentioned references will not be repeated. Figure 1 The description provided.
[0098] Reference Figure 11 The display controller 300 includes a mixer 320 and a display quality enhancer 330. The display controller 300 may also include a high dynamic range (HDR) unit 310, a register 340, and a frame rate control unit 350. The display controller 300 may be referred to as a display processing unit (DPU).
[0099] HDR unit 310 receives multiple layer data LDATs and performs HDR processing on the multiple layer data LDATs based on the first control signal CONT1 from register 340. The multiple layer data LDATs can be compared with a reference... Figure 1 The described multiple layer data LDATs are substantially the same. Compared with the multiple images corresponding to the multiple layer data LDATs, the multiple images corresponding to the multiple layer data LDAT' output from HDR unit 310 and to which HDR processing has been performed may include HDR images with extended dynamic range. As will be described later, a first control signal CONT1 can be generated based on at least one metadata MDAT input to register 340, so HDR unit 310 can generate multiple layer data LDAT' to which HDR processing has been performed based on at least one metadata MDAT.
[0100] Mixer 320 generates first image data IDAT and pixel mapping data PMDAT based on a second control signal CONT2 from register 340 and multiple layer data LDAT' output from HDR unit 310 and to which HDR processing has been performed. Display quality enhancer 330 generates second image data EDAT based on the first image data IDAT and pixel mapping data PMDAT output from mixer 320. As will be described later, the second control signal CONT2 can be generated by register 340 based on at least one metadata MDAT; therefore, mixer 320 can generate first image data IDAT and pixel mapping data PMDAT based on at least one metadata MDAT.
[0101] The mixer 320 and the display quality enhancer 330 can be respectively connected with Figure 1 The mixer 110 and display quality enhancer 120 are essentially the same. For example, the mixer 320 and display quality enhancer 330 may have... Figure 2 The configuration shown can be executed as referenced. Figures 3 to 10B The described operation.
[0102] In some example embodiments, the display controller 300, which includes a mixer 320 and a display quality enhancer 330, can be described as including the image processing apparatus 100 according to the example embodiment, and / or the display controller 300, which includes an HDR unit 310, a mixer 320, a display quality enhancer 330, a register 340, and a frame rate control unit 350, can be described as the image processing apparatus according to the example embodiment.
[0103] Register 340 receives at least one metadata MDAT corresponding to at least one of a plurality of layer data LDATs, and generates a first control signal CONT1, a second control signal CONT2, and a third control signal CONT3 based on the at least one metadata MDAT. For example, register 340 may include at least one setting register.
[0104] The frame rate control unit 350 generates a frame rate control signal FRC for controlling the frame rate of the display device based on a third control signal CONT3 generated based on at least one metadata MDAT. For example, the frame rate control unit 350 may generate a frame rate control signal FRC for adjusting the frame rate of the display device.
[0105] Figure 12 and Figure 13 This is a block diagram illustrating an application processor including an image processing apparatus according to an exemplary embodiment. The above references will not be repeated. Figure 1 and Figure 11 The description provided.
[0106] Reference Figure 12The application processor 500 includes a processor (e.g., intellectual property (IP) or graphics processing unit) 510, a frame buffer 512, a metadata buffer (MB) 514, an HDR unit 310, a mixer 320, a display quality enhancer 330, a register 340, and a frame rate control unit 350.
[0107] Processor 510 provides layer data LDAT11 and LDAT21, as well as metadata MDAT11 corresponding to layer data LDAT11 and LDAT21. For example, processor 510 may include a graphics processing unit (GPU).
[0108] Frame buffer 512 stores and outputs layer data LDAT11 and LDAT21, and metadata buffer 514 stores and outputs metadata MDAT11. For example, each of frame buffer 512 and metadata buffer 514 may correspond to a portion of a memory device.
[0109] exist Figure 12 In the example, layer data LDAT11 and LDAT21 can be provided from a processor (or a data processing device) 510.
[0110] The HDR unit 310, mixer 320, display quality enhancer 330, register 340, and frame rate control unit 350 can be respectively connected to Figure 11 The HDR unit 310, mixer 320, display quality enhancer 330, register 340, and frame rate control unit 350 are basically the same. The HDR unit 310 performs HDR processing on the layer data LDAT11 and LDAT21, and the register 340 generates control signals CONT1, CONT2, and CONT3 based on the metadata MDAT11.
[0111] Reference Figure 13 The application processor 600 includes multiple processors (or IPs) 610, 620, 630 and 640, frame buffers 612, 622, 632 and 642, metadata buffers 624, 634 and 644, a post-processing unit 650, an HDR unit 310, a mixer 320, a display quality enhancer 330, a register 340 and a frame rate control unit 350.
[0112] Processor 610 provides layer data LDAT12, processor 620 provides layer data LDAT22 and corresponding metadata MDAT22, processor 630 provides layer data LDAT32 and corresponding metadata MDAT32, and processor 640 provides layer data LDAT42 and corresponding metadata MDAT42. For example, processor 610 may include third-party IP, processor 620 may include an image signal processor (ISP) and / or a graphics display controller (GDC), processor 630 may include a multi-format codec (MFC), and processor 640 may include a GPU. For example, third-party IP may not provide metadata.
[0113] Frame buffer 612 stores and outputs layer data LDAT12, frame buffer 622 stores and outputs layer data LDAT22, frame buffer 632 stores and outputs layer data LDAT32, and frame buffer 642 stores and outputs layer data LDAT42. Metadata buffer 624 stores and outputs metadata MDAT22, metadata buffer 634 stores and outputs metadata MDAT32, and metadata buffer 644 stores and outputs metadata MDAT42.
[0114] The post-processing unit 650 can perform post-processing on the layer data LDAT12 and provide the post-processed layer data LDAT12. For example, the post-processing unit 650 may include at least one of a GPU, a central processing unit (CPU), a digital signal processor (DSP), and a neural processing unit (NPU), and / or may include various other data processing devices. When the layer data LDAT12 is post-processed, the post-processing unit 650 may provide the post-processed layer data LDAT12 together with the corresponding metadata.
[0115] exist Figure 13 In the example, layer data LDAT12, LDAT22, LDAT32 and LDAT42 can be provided from two or more processors (or two or more data processing devices) 610, 620, 630 and 640.
[0116] The HDR unit 310, mixer 320, display quality enhancer 330, register 340, and frame rate control unit 350 can be respectively connected to Figure 11The HDR unit 310, mixer 320, display quality enhancer 330, register 340, and frame rate control unit 350 are basically the same. The HDR unit 310 performs HDR processing on the layer data LDAT12, LDAT22, LDAT32, and LDAT42, and the register 340 generates control signals CONT1, CONT2, and CONT3 based on the metadata MDAT22, MDAT32, and MDAT42.
[0117] Figure 14 This is a block diagram illustrating an electronic device including an application processor according to an exemplary embodiment.
[0118] Reference Figure 14 The electronic device 700 includes an application processor 701 and a display device.
[0119] Application processor 701 includes display controller 702. Application processor 701 can be... Figure 12 Application processor 500 and Figure 13 One of the application processors 600, the display controller 702 can be Figure 11 The display controller 300.
[0120] The display device includes a display panel 710 and a display driver integrated circuit. The display driver integrated circuit may include a data driver 720, a scan driver 730, a power supply 740, and a timing controller 750.
[0121] The display panel 710 operates based on image data (e.g., displaying an image). The display panel 710 can be connected to the data driver 720 via multiple data lines D1, D2, ..., DM, and to the scan driver 730 via multiple scan lines S1, S2, ..., SN. The multiple data lines D1, D2, ..., DM can extend in a first direction, and the multiple scan lines S1, S2, ..., SN can extend in a second direction that intersects the first direction (e.g., is substantially perpendicular to the first direction).
[0122] The display panel 710 may include a plurality of pixels PX arranged in a matrix having multiple rows and multiple columns. Each of the plurality of pixels PX may include a light-emitting element and a driving transistor for driving the light-emitting element. Each of the plurality of pixels PX may be electrically connected to a corresponding one of a plurality of data lines D1, D2, ..., DM and a corresponding one of a plurality of scan lines S1, S2, ..., SN.
[0123] In some example embodiments, the display panel 710 may be a self-emitting display panel that emits light without using a backlight unit. For example, the display panel 710 may be an organic light-emitting diode (OLED) display panel that includes an OLED as the light-emitting element.
[0124] In some example embodiments, each of the plurality of pixels PX included in the display panel 710 may have various configurations depending on the driving scheme of the display device. For example, the display device may be driven by either an analog or digital driving scheme. While an analog driving scheme utilizes available voltage levels corresponding to the input data to generate grayscale, a digital driving scheme utilizes variable time periods in which LEDs emit light to generate grayscale. Analog driving schemes are difficult to implement because they require the fabrication of complex driver integrated circuits (ICs) if the display is large and has a high resolution. On the other hand, digital driving schemes can easily achieve the required high resolution with a simpler IC structure.
[0125] The timing controller 750 controls the overall operation of the display device. For example, the timing controller 750 can receive input control signals ICS from the application processor 701, and can provide predetermined control signals CS1, CS2, and CS3 to the data driver 720, scan driver 730, and power supply 740 based on the input control signals ICS to control the operation of the display device. For example, the input control signals ICS may include a master clock signal, a data enable signal, a horizontal synchronization signal, a vertical synchronization signal, etc. For example, the input control signals ICS may also include a reference... Figure 11 The frame rate control signal FRC is described.
[0126] The timing controller 750 receives multiple input image data IDS from the application processor 701 and generates multiple output image data ODS for image display based on the multiple input image data IDS. For example, the multiple input image data IDS may include references Figure 1 The second image data EDAT is described. For example, the input image data IDS may include red image data, green image data, and blue image data. Alternatively, the input image data IDS may include white image data. Alternatively, the input image data IDS may include magenta image data, yellow image data, cyan image data, etc. Each of the multiple input image data IDS and each of the multiple output image data ODS may correspond to one frame of image.
[0127] The data driver 720 can generate multiple data voltages based on the control signal CS1 from the timing controller 750 and multiple output image data ODS, and can apply the multiple data voltages to the display panel 710 through multiple data lines D1, D2, ..., DM. For example, the data driver 720 may include a digital-to-analog converter (DAC) that converts the multiple output image data ODS in digital form into the multiple data voltages in analog form.
[0128] The scan driver 730 can generate multiple scan signals based on the control signal CS2 from the timing controller 750, and can apply the multiple scan signals to the display panel 710 through multiple scan lines S1, S2, ..., SN. The multiple scan lines S1, S2, ..., SN can be activated sequentially based on the multiple scan signals.
[0129] In some example embodiments, the data driver 720, scan driver 730, and timing controller 750 may be implemented as a single integrated circuit (IC). In other example embodiments, the data driver 720, scan driver 730, and timing controller 750 may be implemented as two or more integrated circuits. A driver module that includes at least the timing controller 750 and the data driver 720 may be referred to as a timing controller embedded data driver (TED).
[0130] Power supply 740 can apply a first power supply voltage ELVDD and a second power supply voltage ELVSS to display panel 710 based on control signal CS3 from timing controller 750. For example, the first power supply voltage ELVDD can be a high power supply voltage, and the second power supply voltage ELVSS can be a low power supply voltage.
[0131] In some example embodiments, at least some components included in the display driver integrated circuit may be disposed (e.g., directly mounted) on the display panel 710, or may be connected to the display panel 710 in a tape-on-a-carrier (TCP) package. Alternatively, at least some components included in the display driver integrated circuit may be integrated on the display panel 710. In some example embodiments, the components included in the display driver integrated circuit may be implemented separately using separate circuits / modules / chips. In other example embodiments, based on function, some components included in the display driver integrated circuit may be combined into a single circuit / module / chip, or may be separated into multiple circuits / modules / chips.
[0132] Figure 15 This is a block diagram illustrating an image processing apparatus according to an exemplary embodiment. (The following will not be repeated.) Figure 1 The description provided.
[0133] Reference Figure 15 The image processing apparatus 800 includes a mixer 810 and a display quality enhancer 820.
[0134] Mixer 810 receives multiple layer data LDATs, generates first image data IDAT by mixing the multiple layer data LDATs, and generates block mapping data BMDAT based on the multiple layer data LDATs. Besides generating block mapping data BMDAT instead of pixel mapping data PMDAT, mixer 810 can be used with... Figure 1The mixer 110 is basically the same.
[0135] The block mapping data BMDAT includes multiple block IDs representing display quality enhancement algorithms that will be applied to the multiple pixel values included in the first image data IDAT. For example, as referenced... Figure 16 The description describes a display device comprising two or more pixels grouped to form multiple blocks, and each of the multiple block IDs corresponds to a corresponding pixel within the multiple blocks. Besides the block ID corresponding to a block rather than a pixel, the block ID can be related to... Figure 1 The pixel IDs are basically the same.
[0136] The display quality enhancer 820 generates second image data EDAT' based on first image data IDAT and block mapping data BMDAT by applying different display quality enhancement algorithms to at least some of the plurality of pixel values. (As shown in...) Figure 1 Similar to the second image data EDAT in the previous example, the second image data EDAT' includes multiple display quality enhancement pixel values. Figure 1 The second image data EDAT is different, and the same display quality enhancement algorithm can be applied to the pixel values corresponding to the same block based on the same block ID.
[0137] The image processing device 800 may have the same characteristics as... Figure 2 The configuration shown is similar to the one described above. For example, mixer 810 may include a mixing block and a block map generator, and display quality enhancer 820 may include multiple registers, a multiplexer, and an enhancement block. Additionally, image processing device 800 may be similar to the one described above. Figures 3 to 10B The described operations are similar to those described, and can be included in the reference. Figures 11 to 14 In the described display controller, application processor, and / or electronic device.
[0138] Figure 16 This is a diagram used to describe the operation of an image processing apparatus according to an exemplary embodiment.
[0139] Reference Figure 16 The corresponding display shows Figure 3 A composite image (e.g., on the display device 200) Figure 4 The block map BMAP generated from the composite image CIMG.
[0140] exist Figure 16In the example, two pixels can form a block, and the block map (BMAP) can include multiple block IDs corresponding to multiple blocks: BID_11, BID_12, BID_13, BID_14, BID_15, BID_16, BID_17, BID_18, BID_21, BID_22, BID_23, BID_24, BID_25, BID_26, BID_27, BID_28, BID_31, BID_32, BID_33, BID_34, BID_35, BID_36, BID_37, BID_38. BID_41, BID_42, BID_43, BID_44, BID_45, BID_46, BID_47, BID_48, BID_51, BID_52, BID_53, BID_54, BID_55, BID_56, BID_57, BID_58, BID_61, BID_62, BID_63, BID_64, BID_65, BID_66, BID_67 and BID_68.
[0141] When block-mapped image processing (BMAP) is applied to synthetic images on a block-by-block basis, different optimized display quality enhancement algorithms are applied (e.g.,...). Figure 4 When generating a composite image with improved display quality using a composite image (CIMG), for example, pixel values for pixels P11 and P21 can be generated by applying at least one display quality enhancement algorithm selected based on block ID BID_11, and pixel values for pixels P12 and P22 can be generated by applying at least one display quality enhancement algorithm selected based on block ID BID_12. The display quality enhancement algorithms applied to the pixel values of pixels P11 and P21 and the display quality enhancement algorithms applied to the pixel values of pixels P12 and P22 can be the same or different from each other. That is, at least one display quality enhancement algorithm configured to process each block can cover a wider range of pixels, thereby increasing the speed of processing each pixel.
[0142] Although the example embodiments are described based on a specific number of pixels, blocks, and block IDs, the example embodiments are not limited thereto.
[0143] Figure 17 This is a flowchart illustrating an image processing method according to an exemplary embodiment.
[0144] Reference Figure 1 and Figure 17In the image processing method according to the example embodiment, multiple layer data LDATs are received (step S100). The multiple layer data LDATs represent multiple images to be displayed on a screen in a display device. First image data IDAT is generated by mixing the multiple layer data LDATs (step S200). The first image data IDAT includes multiple pixel values corresponding to the screen. Pixel mapping data PMDAT is generated based on the multiple layer data LDATs (step S300). The pixel mapping data PMDAT includes multiple pixel identities (IDs) indicating a display quality enhancement algorithm to be applied to the multiple pixel values. Here, it can be achieved through... Figure 1 The mixer 110 shown or other mixers according to one or more embodiments of the above perform steps S100, S200 and S300.
[0145] Based on the first image data IDAT and pixel mapping data PMDAT, a second image data EDAT is generated by applying different display quality enhancement algorithms to the plurality of pixel values (step S400). The second image data EDAT includes a plurality of display quality enhancement pixel values. This can be achieved through... Figure 1 The display quality enhancer 120 or other display quality enhancers according to one or more embodiments of the above shall perform step S400.
[0146] Figure 18 and Figure 19 It is shown Figure 17 The flowchart shows an example of generating second image data.
[0147] Reference Figure 2 , Figure 17 and Figure 18 When the second image data EDAT is generated (step S400), at least one display quality enhancement algorithm for the first pixel value PV1 can be selected based on the first pixel ID PID1 (step S510). A first display quality enhanced pixel value EPV1 can be generated by applying the at least one display quality enhancement algorithm selected in step S510 to the first pixel value PV1 (step S610). Then, at least one display quality enhancement algorithm for the Nth pixel value PVN can be selected based on the Nth pixel ID PIDN (step S520), and an Nth display quality enhanced pixel value EPVN can be generated by applying the at least one display quality enhancement algorithm selected in step S520 to the Nth pixel value PVN (step S620). In other words, Figure 18 The example illustrates the operation of selecting the display quality enhancement algorithm sequentially for each pixel value and the operation of generating display quality enhanced pixel values.
[0148] Reference Figure 2 , Figure 17 and Figure 19 When the second image data EDAT is generated (step S400), at least one display quality enhancement algorithm for the first pixel value PV1 can be selected based on the first pixel ID PID1 (step S510), and at least one display quality enhancement algorithm for the Nth pixel value PVN can be selected based on the Nth pixel ID PIDN (step S520). Next, a first display quality enhanced pixel value EPV1 can be generated by applying the at least one display quality enhancement algorithm selected in step S510 to the first pixel value PV1 (step S610), and an Nth display quality enhanced pixel value EPVN can be generated by applying the at least one display quality enhancement algorithm selected in step S520 to the Nth pixel value PVN (step S620). Figure 19 The example shows the operation of selecting the display quality enhancement algorithm for all pixel values in sequence, and then performing the operation of generating display quality enhanced pixel values for all pixel values in sequence.
[0149] Figure 20 , Figure 21 and Figure 22 This is a flowchart illustrating an image processing method according to an exemplary embodiment. The above references will not be repeated. Figure 17 The description provided.
[0150] Reference Figure 10A and Figure 20 In the image processing method according to the example embodiment, a first frame image F1, a first pixel map PM1, and a first display quality enhancement frame image EF1 are generated in the first frame (step S1100). A second frame image F2, a second pixel map PM2, and a second display quality enhancement frame image EF2 are generated in the second frame following the first frame (step S1200). This can be based on... Figure 17 Steps S100, S200, S300 and S400 in the process are performed by each of steps S1100 and S1200. Figure 20 The example shows the generation of image data and pixel mapping data for each frame.
[0151] Reference Figure 10B and Figure 21 In the image processing method according to the example embodiment, step S1100 can be combined with... Figure 20 The steps in step S1100 are essentially the same. A second frame image F2 and a second display quality enhanced frame image EF2' can be generated in the second frame following the first frame (step S1300). This can be based on... Figure 17Step S1300 is executed from steps S100, S200, and S400. In the second frame, the second pixel map PM2 may not be generated, and the second display quality enhancement frame image EF2' may be generated based on the currently generated second frame image F2 and the previously generated first pixel map PM1. Figure 21 The example shows the generation of image data for each frame and pixel mapping data for X frames.
[0152] Reference Figure 15 and Figure 22 In the image processing method according to the example embodiment, steps S100 and S200 can be respectively connected with... Figure 17 Steps S100 and S200 are basically the same. (Refer to...) Figure 17 The described embodiments are the opposite of those described in the example. Figure 22 In step S350, block mapping data BMDAT is generated based on multiple layer data LDAT. The block mapping data BMDAT includes multiple block IDs, which indicate the display quality enhancement algorithm to be applied to the multiple blocks, including one or more pixel values. Steps S100, S200, and S350 can be executed via mixer 810.
[0153] A second image data EDAT' is generated based on the first image data IDAT and the block mapping data BMDAT by applying different display quality enhancement algorithms to at least some of the plurality of pixel values (step S450). The second image data EDAT' includes a plurality of display quality enhanced pixel values. Step S450 can be performed by the display quality enhancer 820. The same display quality enhancement algorithm can be applied to pixel values corresponding to the same block based on the same block ID.
[0154] Those skilled in the art will understand that the inventive concept can be implemented as a system, method, computer program product, and / or a computer program product embodied in one or more computer-readable media having computer-readable program code stored thereon. The computer-readable program code can be accessed by a computer's processor or other programmable data processing device. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be any tangible medium that may contain or store a program used by or in conjunction with an instruction execution system, device, or apparatus. For example, the computer-readable medium can be a non-transitory computer-readable medium.
[0155] Figure 23 This is a block diagram illustrating an electronic system including an application processor according to an exemplary embodiment.
[0156] Reference Figure 23The electronic system 1000 can be implemented as a data processing device using or supporting a Mobile Industrial Processor Interface (MIPI) interface. The electronic system 1000 may include an application processor 1110, an image sensor 1140, a display device 1150, etc. The electronic system 1000 may also include a radio frequency (RF) chip 1160, a global positioning system (GPS) 1120, a storage unit 1170, a microphone (MIC) 1180, dynamic random access memory (DRAM) 1185, and a speaker 1190. Furthermore, the electronic system 1000 can perform communication using ultra-wideband (UWB) 1210, wireless local area network (WLAN) 1220, and WiMAX 1230, etc.
[0157] Application processor 1110 may be a controller or processor that controls the operation of image sensor 1140 and display device 1150.
[0158] Application processor 1110 may include a Display Serial Interface (DSI) host 1111 that performs serial communication with the DSI device 1151 of display device 1150, a Camera Serial Interface (CSI) host 1112 that performs serial communication with the CSI device 1141 of image sensor 1140, a PHY 1113 that performs data communication with the physical layer (PHY) 1161 of RF chip 1160 based on MIPI DigRF, and a DigRF master device 1114 that controls the data communication of physical layer 1161. The DigRF master device 1114 can control the DigRF slave device 1162 of RF chip 1160.
[0159] In some example embodiments, DSI host 1111 may include a serializer (SER), and DSI device 1151 may include a deserializer (DES). In some example embodiments, CSI host 1112 may include a deserializer (DES), and CSI device 1141 may include a serializer (SER).
[0160] The application processor 1110 and the DSI host 1111 may be an application processor and a display controller according to an exemplary embodiment, and may include an image processing device according to an exemplary embodiment.
[0161] One or more inventive concepts can be applied to a variety of devices and systems, including image processing devices and display devices. For example, the inventive concepts can be applied to systems such as personal computers (PCs), server computers, data centers, workstations, mobile phones, smartphones, tablet computers, laptop computers, personal digital assistants (PDAs), portable multimedia players (PMPs), digital cameras, portable game consoles, music players, camcorders, video players, navigation devices, wearable devices, Internet of Things (IoT) devices, Internet of Things (IoE) devices, e-book readers, virtual reality (VR) devices, augmented reality (AR) devices, robotic devices, drones, and the like.
[0162] The foregoing description is an illustration of exemplary embodiments and should not be construed as limiting the scope of one or more embodiments of the present invention. Although some exemplary embodiments have been described, those skilled in the art will readily understand that many modifications, substitutions, and improvements can be made to the exemplary embodiments without substantially departing from the novel teachings and advantages of the exemplary embodiments. Therefore, all such modifications, substitutions, and improvements should be construed as falling within the scope of the exemplary embodiments as defined in the appended claims.
Claims
1. An image processing apparatus comprising at least one processor, said at least one processor being configured to perform: The mixer is configured as follows: Receives data from multiple layers; First image data is generated by mixing the multiple layers of data, the first image data including multiple pixel values corresponding to a screen in a display device; as well as Pixel mapping data including multiple pixel IDs is generated based on the multiple layer data, the multiple layer data representing multiple images to be displayed on the one screen in the display device, and the multiple pixel IDs indicating multiple display quality enhancement algorithms to be applied to the multiple pixel values; as well as A display quality enhancer is configured to generate second image data comprising multiple display quality enhanced pixel values by applying the multiple display quality enhancement algorithms to the multiple pixel values based on the first image data and the pixel mapping data. The mixer includes: A mixing block, configured to generate, based on the multiple layer data, multiple images corresponding to a composite image to be displayed on the one screen by synthesizing the multiple images; and A pixel mapping generator is configured to generate the plurality of pixel IDs based on the plurality of layer data for the plurality of pixel values corresponding to the composite image.
2. The image processing apparatus according to claim 1, wherein, The plurality of pixel values include the first pixel value to the Nth pixel value, where N is a natural number greater than or equal to two; The plurality of pixel IDs includes the first pixel ID to the Nth pixel ID; The plurality of display quality enhancement pixel values include a first display quality enhancement pixel value to an Nth display quality enhancement pixel value; and The display quality enhancer is also configured to: The first display quality enhancement algorithm is selected from the plurality of display quality enhancement algorithms based on the first pixel ID; as well as The first display quality enhanced pixel value is generated by applying the first display quality enhancement algorithm to the first pixel value among the plurality of pixel values.
3. The image processing apparatus according to claim 2, wherein, The display quality enhancer is also configured to: Based on the second pixel ID, select a second display quality enhancement algorithm that is different from the first display quality enhancement algorithm among the plurality of display quality enhancement algorithms; as well as The second display quality enhanced pixel value is generated by applying the second display quality enhancement algorithm to the second pixel value among the plurality of pixel values.
4. The image processing apparatus according to claim 2, wherein, The display quality enhancer is also configured to: Based on the first pixel ID, a second display quality enhancement algorithm that is different from the first display quality enhancement algorithm is selected from the plurality of display quality enhancement algorithms; as well as The first display quality enhanced pixel value is generated by applying the first display quality enhancement algorithm and the second display quality enhancement algorithm to the first pixel value.
5. The image processing apparatus according to claim 2, wherein, The plurality of images includes the first image to the Kth image, where K is a natural number greater than or equal to two; The multiple layers of data include data from the first layer to the Kth layer; and Based on a first image and a second image that are overlapped and set on a first area including a first pixel on the screen, the mixer is further configured to generate the first pixel ID based on one of a first layer of data representing the first image and a second layer of data representing the second image.
6. The image processing apparatus according to claim 5, wherein, Based on setting the first image to be displayed on the first area, the mixer is also configured to generate the first pixel ID based on the first layer data.
7. The image processing apparatus according to claim 1, wherein, One or more of the pixel IDs corresponding to the layer data in the plurality of layer data have the same pixel ID.
8. The image processing apparatus according to claim 1, wherein, One or more of the pixel IDs corresponding to the layer data in the plurality of layer data have pixel IDs that are different from each other.
9. The image processing apparatus according to claim 1, wherein, The display quality enhancer includes: Multiple registers are configured to store multiple display quality enhancement parameters for multiple display quality enhancement algorithms; A multiplexer configured to select at least one of the plurality of display quality enhancement algorithms based on the plurality of pixel IDs; and An enhancement block is configured to generate the plurality of display quality enhanced pixel values based on the plurality of pixel values and at least one of the plurality of display quality enhancement algorithms.
10. The image processing apparatus according to claim 1, wherein, The image processing apparatus is further configured to receive at least one metadata corresponding to at least one layer of data among the plurality of layer data; and The mixer is also configured to generate the pixel mapping data based on the plurality of layer data and the at least one metadata.
11. The image processing apparatus of claim 10, further comprising a frame rate control unit configured to control the frame rate of the image processing apparatus based on the at least one metadata.
12. The image processing apparatus according to claim 1, wherein, The multiple layers of data are provided from one or more external data processing devices.
13. The image processing apparatus according to claim 1, wherein, The mixer is also configured to selectively generate pixel IDs for a subset of the pixel values; and The display quality enhancer is further configured to generate a portion of the plurality of display quality enhanced pixel values by applying the plurality of display quality enhancement algorithms to a portion of the plurality of pixel values.
14. The image processing apparatus according to claim 1, wherein, The mixer is also configured to generate the first image data and the pixel mapping data for each frame.
15. The image processing apparatus according to claim 1, wherein, The mixer is also configured to generate the first image data for each frame and the pixel mapping data for X frames, where X is a natural number greater than or equal to two.
16. An image processing method, comprising: Receive multiple layers of data, which represent multiple images to be displayed on a screen in a display device; First image data is generated by mixing the multiple layers of data, the first image data including multiple pixel values corresponding to the one screen; Pixel mapping data including multiple pixel IDs is generated based on the multiple layer data, and the multiple pixel IDs indicate multiple display quality enhancement algorithms that will be applied to the multiple pixel values; as well as Based on the first image data and the pixel mapping data, a second image data comprising multiple display quality enhanced pixel values is generated by applying the multiple display quality enhancement algorithms to the multiple pixel values. The generation of the first image data includes: Based on the multiple layers of data, the multiple images are synthesized to generate multiple pixel values corresponding to a synthesized image that will be displayed on the screen. The generation of the pixel mapping data includes: The plurality of pixel IDs are generated based on the plurality of layer data for the plurality of pixel values corresponding to the composite image.
17. The image processing method according to claim 16, wherein, The plurality of pixel values include the first pixel value to the Nth pixel value, where N is a natural number greater than or equal to two; The plurality of pixel IDs includes the first pixel ID to the Nth pixel ID; The plurality of display quality enhancement pixel values include a first display quality enhancement pixel value to an Nth display quality enhancement pixel value; and The step of generating the second image data, which includes the plurality of display quality enhancement pixel values, includes: Based on the first pixel ID, a first display quality enhancement algorithm is selected from the plurality of display quality enhancement algorithms for the first pixel value, and a first display quality enhancement pixel value is generated based on the first pixel value; and Based on the Nth pixel ID, an Nth display quality enhancement algorithm is selected for the Nth pixel value, and the Nth display quality enhancement pixel value is generated based on the Nth pixel value.
18. The image processing method according to claim 16, wherein, The plurality of pixel values include the first pixel value to the Nth pixel value, where N is a natural number greater than or equal to two; The plurality of pixel IDs includes the first pixel ID to the Nth pixel ID; The plurality of display quality enhancement pixel values include a first display quality enhancement pixel value to an Nth display quality enhancement pixel value; and The step of generating the second image data, which includes the plurality of display quality enhancement pixel values, includes: Based on the first pixel ID to the Nth pixel ID, multiple display quality enhancement algorithms are selected for the first pixel value to the Nth pixel value; and The first display quality enhancement pixel value to the Nth display quality enhancement pixel value are generated based on the first pixel value to the Nth pixel value.
19. An application processor, comprising: At least one processor; as well as A display controller configured to interoperate with the at least one processor. The display controller includes: A high dynamic range unit is configured to receive multiple layers of data from the at least one processor and perform high dynamic range processing on the multiple layers of data based on a first control signal, the multiple layers of data representing multiple images to be displayed on a screen of a display device; A mixer is configured to generate first image data by mixing the multiple layers of data based on the output of the high dynamic range unit and a second control signal, and to generate pixel mapping data including multiple pixel IDs based on the output of the high dynamic range unit and the second control signal. The first image data includes multiple pixel values corresponding to the screen, and the multiple pixel IDs indicate multiple display quality enhancement algorithms to be applied to the multiple pixel values. A display quality enhancer is configured to generate second image data comprising a plurality of display quality enhanced pixel values by applying the plurality of display quality enhancement algorithms to the plurality of pixel values based on the first image data and the pixel mapping data; A register configured to receive at least one metadata corresponding to at least one layer of the plurality of layer data from the at least one processor, and to generate the first control signal, the second control signal, and the third control signal based on the at least one metadata; and A frame rate control unit is configured to control the frame rate of the display device based on the third control signal.
Citation Information
Patent Citations
Film for display panel
KR1020200110041A
Display apparatus and method of driving the same
KR1020150041459A
Liquid crystal display device and image processing method for same
US20180270400A1
Real time tone mapping of high dynamic range image data at time of playback on a lower dynamic range display
US20200134792A1