Pixel modification to reduce energy consumption of display devices

Abstract

A method and device for reducing power consumption of a display device propose reducing the total amount of light emitted in a perceptually indistinguishable manner based on human visual sensitivity at the location of a pixel. While the visibility of processing can be maintained below a threshold, other techniques may not provide such proof. This is achieved by determining the minimum detectable modulation or the corresponding minimum distinguishable difference based on frequency intensity information for a pixel indicating how many sets of frequencies exist in that pixel, and a contrast sensitivity function representing a model of human vision that predicts which contrasts of which frequencies are visible to the human eye. Based on this minimum detectable modulation or minimum distinguishable difference, the pixel value can be reduced. The contrast sensitivity function can be given by Barten's model, and the frequency intensity information can be based on a layer map constructed using a discrete wavelet transform or a continuous wavelet transform.

Description

Technology Field

[0001] The present invention relates generally to image processing and energy consumption. More specifically, the present invention relates to a reduction in the energy requirements of a display device. Background Technology

[0002] The media and entertainment industry consumes a significant amount of energy to produce, distribute, and deliver video content to consumers. A substantial portion of this energy is used by display devices, particularly the more than 1.9 billion displays installed in consumer homes worldwide. The generation of light within displays is the primary cause of this energy consumption across all technologies currently used in TVs. Therefore, reducing energy consumption in display devices globally by mobilizing all possible means is a major concern. Given the sheer number of displays currently in use, even a slight reduction in energy consumption per display can save a massive amount of energy globally. Furthermore, as display resolutions have increased from SD to HD, 4K, and soon beyond to 8K, and with the introduction of High Dynamic Range (HDR) imaging, the energy requirements of display devices have further increased. This fails to meet the global demand to reduce energy consumption.

[0003] This is particularly relevant to display devices based on OLED technology, especially mobile display devices where battery life is a critical factor. In these OLED-based display devices, the relationship between the light emitted from a pixel and the energy used to generate this light is nearly linear, with brighter pixels consuming more energy. The efficiency of OLED panels is primarily determined by the driving circuit design, LED quantum efficiency, and optical system efficiency (see reference [Huang et al, "Mini-LED, Micro-LED and OLED displays: present status and future perspectives", Light: Science and Applications 9:105, 2020]). This is also a problem for backlit LCD displays.

[0004] The embodiments described below were designed with the foregoing in mind.

[0005] The present disclosure proposes a new and novel solution for reducing energy consumption of a display device. It proposes reducing pixel values ​​so that a display device (e.g., an OLED or LED display) uses less electricity to generate an image.

[0006] These problems have been addressed from various angles, such as blurring areas of the picture displayed outside the detection area of ​​interest (for example, based on saliency or visual attention models) or based on user interaction with the device.

[0007] The generation of light for display devices (e.g., TVs, smartphones, tablets, laptops, cameras) is costly. It is desirable to reduce the amount of light generated because it helps reduce the amount of energy required to operate the display. The advantages of this are twofold: less pressure on the climate and longer battery life for mobile devices. For the same reason, compared to other methods aimed at reducing pixel values, at least one embodiment, by its configuration, ensures that processing remains below a visible threshold. To achieve this, the amount of change for each pixel can be less than one just noticeable difference (JND). The key point here is that the corresponding JND may differ for each pixel. The JND per pixel can be calculated for each pixel using the steps described below. To determine how much contrast is available for a given pixel in a given frame, a layer map can be constructed. This is typically done using a wavelet transform. While a simple discrete wavelet transform using Haar wavelets is a logical choice for many graphics applications, the trade-off between spatial localization and frequency analysis is not optimal. This is used in the first method described below. In this regard, another option is the Continuous Wavelet Transform (CWT), which is used in the second method described below. Since the sensitivity of the human visual system does not depend significantly on direction, the Continuous Wavelet Transform using anisotropic Mexican hat wavelet can achieve excellent results without consuming more computational cycles than necessary.

[0008] Although the principles of the present disclosure are described in the context of display devices based on OLED technology, they are not limited to this context and apply to other types of displays such as local dimming LED displays, mini LED displays, and micro LED displays. The principles of the present disclosure also apply to MEMs-based display technologies.

[0009] A first aspect of the present disclosure relates to a method for determining a minimum detectable modulation value for a pixel (x) of an image located at a position within an input image, based on human visual sensitivity at the location of the pixel, and scaling the luminance of the pixel by an amount based on the minimum detectable modulation value. In a variation of the first aspect, the scaled luminance value is determined as Lreduced(x) = L(x)(1 - fm(x)) / (1 + fm(x)), wherein L(x) is the luminance level at the location of the pixel (x), m(x) is the minimum detectable modulation determined for the pixel (x), and f is the modulation coefficient. In a variation of the first aspect, human visual sensitivity at the location of the pixel is based on the luminance of the pixel, frequency intensity information for the pixel indicating how many sets of frequencies exist in the pixel, and a contrast sensitivity function representing a model of human vision that predicts what contrast of which frequencies is seen by the human eye. In an additional variation of the first embodiment, the contrast sensitivity function is given by Barten's model, and the frequency intensity information is based on a layer map constructed using a discrete wavelet transform or a continuous wavelet transform.

[0010] A second aspect of the present disclosure relates to a device comprising a processor configured to determine a minimum detectable modulation value for a pixel (x) of an image located at a position within an input image, based on human visual sensitivity at the location of the pixel, and to scale the luminance of the pixel by an amount based on the minimum detectable modulation value. In a variation of the second aspect, the scaled luminance value is determined as Lreduced(x) = L(x)(1 - fm(x)) / (1 + fm(x)), wherein L(x) is the luminance level at the location of the pixel (x), m(x) is the minimum detectable modulation determined for the pixel (x), and f is the modulation coefficient. In a variation of the second aspect, human visual sensitivity at the location of the pixel is based on the luminance of the pixel, frequency intensity information for the pixel indicating how many sets of frequencies exist in the pixel, and a contrast sensitivity function representing a model of human vision that predicts what contrast of which frequencies is visible to the human eye. In an additional variation of the second embodiment, the contrast sensitivity function is given by Barton's model, and the frequency intensity information is based on a layer map constructed using a discrete wavelet transform or a continuous wavelet transform. In a variation of the second embodiment, the device is selected from the group comprising a set-top box, a video receiver, and a video player. In another variation of the second embodiment, the device further comprises a screen, and the processor is further configured to display a determined image with reduced brightness, and the screen is based on an LCD display technology having a uniform or non-uniform backlight based on OLED display technology, micro-LED technology, mini-LED technology, MEMs technology, CCFL, LED, mini-LED, or micro-LED technology, and the device is selected from the group comprising a TV set, a smartphone, a laptop, a camera, and a tablet.

[0011] A third embodiment relates to a computer program product comprising program code instructions, wherein when the program is executed on a computer or processor, a method according to the first embodiment or any modified embodiment of the first embodiment is implemented.

[0012] A fourth embodiment relates to a non-transient computer-readable storage medium storing program code instructions for implementing a method according to the first embodiment or any modified embodiment of the first embodiment. Brief explanation of the drawing

[0013] The present invention can be better understood by referring to the following description and drawings, the drawings being provided by way of example and not limiting the scope of protection: Figure 1 illustrates a chart showing the visibility of contrast at various frequencies. FIG. 2 illustrates a flowchart of a method for reducing the brightness of an image according to at least one embodiment. FIG. 3 illustrates a display device that implements a method for reducing energy consumption according to at least one embodiment. Specific details for implementing the invention

[0014] Exemplary embodiments may be varied in many ways and in alternative forms, but such embodiments are illustrated in the drawings in an exemplary manner and will be described in detail herein. However, there is no intention to limit exemplary embodiments to the specific forms disclosed; rather, exemplary embodiments should be understood to encompass all variations, equivalents, and alternatives falling within the scope of the claims. Throughout the drawings, the same reference numerals indicate similar components.

[0015] Before discussing exemplary embodiments in more detail, note that some exemplary embodiments are described as processes or methods illustrated in flowcharts. Although flowcharts represent operations as a sequential process, many operations may be performed in parallel, together, or simultaneously. Additionally, the order of operations may be rearranged. This process may terminate when the work is completed, but there may be additional steps not included in the drawings. This process may correspond to a method, function, procedure, subroutine, subprogram, etc.

[0016] The methods discussed below (some of which are illustrated in flowcharts) may be implemented by hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented by software, firmware, middleware, or microcode, program code or code segments for performing the necessary tasks may be stored on a machine or computer-readable medium, such as a storage medium. Processor(s) may also perform the necessary tasks. The specific structural and functional details disclosed herein are for the purposes only of describing exemplary embodiments of the invention. However, the invention may be embodied in many alternative forms and should not be interpreted as being limited only to the embodiments described herein.

[0017] The embodiments described herein use both the terms “Minimum Discriminative Difference (JND)” and “Minimum Detectable Modulation (MDM).” Although expressed differently, these terms correspond to the same concept, representing a threshold value for variations in luminance for a pixel; variations smaller than this value are not perceived by a general observer, whereas variations larger than this value can be perceived by an observer. The relationship between the two terms can be expressed as follows:

[0018]

[0019] In this equation, JND is the minimum distinguishable difference, m(x) is the minimum detectable modulation for pixel (x), and L is the luminance of this pixel. The two terms are used interchangeably.

[0020] At least one embodiment proposes reducing the total amount of light emitted in a perceptually indistinguishable manner by directly using a Contrast Sensitivity Function (CSF). The visibility of this processing is maintained below a minimum detectable modulation threshold, whereas other techniques may not provide such proof. This is achieved by using a CSF model capable of calculating a Minimum Differential Difference (JND) and reducing pixel values ​​based on this JND. In at least one variant embodiment, reducing pixel values ​​by up to 1 JND is proposed. In at least one variant embodiment, reducing pixel values ​​by more than 1 JND is proposed to provide better energy consumption, although provability is lost.

[0021] 도 1Figure 1 illustrates a chart showing the visibility of contrast at various frequencies. The contrast sensitivity function (CSF) is a model of human vision that predicts which contrast at which frequency is visible to the human eye. The visibility of contrast depends on the frequency and magnitude of the contrast, as illustrated in Figure 1. In this figure, the chart (100) is a Campbell-Robson chart in which luminance is modulated according to a sine function in which the frequency increases linearly from left to right and the magnitude increases linearly from top to bottom. The curve (101) indicates the limit where the contrast is barely visible. Thus, the lower part (102) represents the area where the contrast is visible, and the upper part (103) represents the area where the contrast is not visible. Humans are most sensitive to contrast of about 1 to 2 cycles per angle. Therefore, if image modification is performed within the area (103), it may not be visible to the eye.

[0022] There are many models available for modeling contrast sensitivity. Barton's model (Barten, Peter GJ. Contrast sensitivity of the human eye and its effects on image quality. SPIE press, 1999.) tends to be considered the most complete model, and other models are often verified by comparing them to this model.

[0023] Barton's CSF model can be summarized as follows: Luminance L overfrequency u Contrast sensitivity as a function of S(L,u) is calculated as Equation 1:

[0024] (Equation 1)

[0025] In this equation

[0026]

[0027] The following water supply is designated.

[0028]

[0029]

[0030]

[0031]

[0032] This model is independent of the image content and can be calculated on the device or predetermined and loaded from memory.

[0033] The input image is typically specified as an 8-bit SDR image, or as a 10-bit or 12-bit HDR image. Nominally, the codeword values ​​of an SDR image are 0 and 100 cd / m 2 Encodes the luminance range between. The codeword values ​​of HDR images are in a larger range, for example, 0 and 10,000 cd / m 2 You can encode between them. Since Barton's model is specified by absolute luminance values, the input RGB image is first converted to linear XYZ, and a luminance-only image is obtained from the Y channel of the RGB image. L This is derived. In the case of an SDR image, this luminance image is scaled between 0 and 100. If the input image is an HDR image, the luminance image is scaled according to the assumed peak luminance.

[0034] The principle of the embodiments disclosed herein is to reduce pixel values, e.g., luminance values, so that the display uses less electricity to generate an image. In at least one embodiment, the amount of change for each pixel will be less than 1 JND (Minimum Identifiable Difference). The key point here is that the corresponding JND may differ for each pixel. The JND per pixel can be calculated using the following steps, wherein the location within the image x The pixel located at is a pixel ( x It is displayed as ).

[0035] To determine how much contrast is available at a given pixel of a given frame, a first method is proposed. This first method constructs a hierarchical map using, for example, the Discrete Wavelet Transform (DWT), based on, for example, Haar wavelets, or based on, for example, wavelet families: Daubechies, coiflets, symlets, Fejer-Korovkin, Discrete Meyer, or (inverse) biorthogonal. The hierarchical map is at a given pixel location x At each frequency u i It provides information on how much there is. Each pixel ( x For a given frequency u i The amount of contrast in DWT(x,u i ) Assume that it is given as. pixel x The luminance of ) L It is given as. This map and Barton's CSF S(L,u i ) Using, pixel location x At some frequency u i Weighted CSF for DWT(x,u t )S(L,u i ) You can determine whether it generates a maximum value:

[0036] (Equation 2a)

[0037] Wavelet level i Since there are 10, the output of this process u It has only one of 10 different values. Since this process is performed individually for every pixel, the frequency u The pixel map contains only 10 distinct values ​​that are logarithmically separated from each other. Therefore, an additional filtering step is inserted, thereby making the smoothing kernel a frequency map u It is passed through. The description below is based on the Gaussian kernel, but other types of smoothing such as box filters, tent filters, cubic filters, sink filters, and two-sided filters can also be used. Standard deviation of the kernel σ is determined empirically as follows:

[0038] (Equation 2b)

[0039] In this equation N x and N y is the horizontal and vertical resolution of the input image. Gaussian smoothing kernel G is calculated as follows:

[0040] (Equation 2c)

[0041] The smoothed frequency map is calculated as follows:

[0042] (Equation 2d)

[0043] The smoothing effect has undesirable side effects, which is u' The value range of u It means that it decreases compared to the value range of. To solve this problem, a filtered frequency map u' is scaled as follows:

[0044] (Equation 2e)

[0045] This scaling of the filtered frequency map represents an appropriate solution for still images. However, for video content u and u' Scaling by the ratio of the maximum value of may result in transient artifacts, particularly flickering. These transient artifacts can be resolved by applying these maximum values ​​to a process known as leaky integration, which is achieved as follows: subscript t represents the frame number, and the frame t-Scaling for 1 is s t-1 It is assumed that it is specified as. The scaling of the current frame is s t It is given as.

[0046] (Equation 2f)

[0047] Afterwards, frame t The scaled frequency map for is It is given as. For video content, this is a suitable solution for scaling frequency maps. α The value of is within the range [0; 1]. An exemplary value is α = is given as 0.8.

[0048] Using Barton's contrast sensitivity function, now set the pixel position by setting the following. x human visual sensitivity S(x) Can determine:

[0049] S(x) = S(L,u) (Equation 3a)

[0050] For reference, Equations 2a through 3a are mostly expressed as per-pixel operations, with the exception of the filtering step; however, since the filtering step is inevitably a spatially varying operation, it cannot be expressed as a per-pixel operation. Nevertheless, the procedure of Equations 2a through 3a can be summarized as a single vector value expression as follows:

[0051] (Equation 3b)

[0052] In this equation S img is a pixel map of contrast sensitivity equivalent to the per-pixel formula of Equation 9a, and L is a pixel map for the luminance values ​​of the input image, and k ε is a constant representing the scaling of Equation 8a or Equation 8b. Sensitivity S(x)From this, the minimum detectable modulation m(x) can be determined (reference [Miller, Scott, Mandi Nezamabadi, and Scott Daly. "Perceptual signal coding for more efficient usage of bit codes." SMPTE Motion Imaging Journal 122, no. 4 (2013): 52-59.])

[0053] (Equation 4)

[0054] Two luminance values L high and L low The contrast between them can be determined by the known Michelson contrast.

[0055]

[0056] luminance L given pixel ( x For ), the minimum detectable modulation can be equal to the Michelson contrast, and L second L high It can be replaced with:

[0057]

[0058] The only unknown thing in this equation L low is determined as follows.

[0059] (Equation 5a)

[0060] This is pixel luminance L and from the related contrast sensitivity, reduced low pixel brightness L low can calculate. Carefully, m For example, modulation coefficients f = It can be scaled down by using 0.9.

[0061] (Equation 5b)

[0062] Using the above equation, each new pixel has a value less than 1 JND lower than the previous one. Given that Barton's model was used, high-luminance pixels are reduced more than low-luminance pixels. Frequency sensitivity is constructed using DWT (which may be replaced by other frequency determination methods).

[0063] The principle explained above is f As long as is selected as less than 1, it ensures that the visibility of the processing remains below the threshold.

[0064] This method can be applied at various locations in the imaging pipeline. For example, it can be used before encoding to ensure that visually equivalent images / videos are transmitted. This method can also be used after decoding on user devices, such as set-top boxes or Blu-ray players. In both cases, the display generates less light, thus consuming less energy while maintaining the visual quality of the image / video.

[0065] The effect of this reduction is the modulation coefficient of Equation 5b f By adjusting it, it can be tailored to various needs. In fact, f If = 1, the reduction in light is perceptually indistinguishable. For some specific uses (e.g., in critical viewing applications occurring during post-processing), the modulation coefficient of Equation 5b f By making it smaller than 1.0, a margin is introduced, resulting in less light attenuation and less reduction in energy consumption. For example, the modulation coefficient f = 0.9 or f =0.5 can be used. Conversely, if energy reduction is very important, the modulation coefficient of Equation 5b greater than 1.0 can be used. f ...may also be used. For example, modulation coefficients f = 1.5 or f=2.0 can be used. In this case, pixel correction may be visible, but energy reduction is more important.

[0066] To determine the frequency of contrast present in the vicinity of each pixel, the first method described in detail above is based on DWT and uses, for example, Haar wavelets. For the analysis of image frequency components, DWT is computationally very fast, but it is not the most accurate choice because an optimal trade-off between spatial localization and frequency analysis is impossible. A slower but more accurate choice is to use the Continuous Wavelet Transform (CWT), which provides frequency analysis along with better spatial localization.

[0067] A large number of wavelet modal functions are available in terms of 2D CWT, and in this application, Mexican hat wavelets (also called Ricker wavelets), Morlet wavelets, halo and arc wavelets, Cauchy wavelets, Poisson wavelets, etc., may be used. Since the focus is on image analysis and frequency occurrences often coincide with the presence of edges, a wavelet function suitable for edge detection is a reasonable choice. Furthermore, CWT can be performed in any desired direction. Anisotropic wavelet functions may be selected to allow frequencies in various directions to be preferred. Human vision is also known to be anisotropic in the sense that it is more sensitive to horizontal and vertical edges. However, the difference in sensitivity to horizontal / vertical edges and diagonal edges is not significant enough to guarantee direction-selective frequency analysis. Therefore, an isotropic wavelet can be selected because the computational cost of wavelet analysis is significantly low. Considering these constraints, it is appropriate to select the Mexican Hat wavelet. Furthermore, this isotropic wavelet generates real-value outputs.

[0068] Therefore, the second method is 10 algebraically separated scales i We propose performing frequency analysis by continuous wavelet transform (CWT) using Mexican hat wavelets having.

[0069] (Equation 6)

[0070] Wavelet level i Since there are 10, the output of this process u It has only one of 10 different values. Since this process is performed individually for every pixel, the frequency u The pixel map contains only 10 distinct values ​​that are logarithmically separated from each other. Therefore, an additional filtering step is inserted, thereby making the Gaussian smoothing kernel a frequency map u It is passed through. The kernel's standard deviation σ is determined empirically as follows:

[0071] (Equation 7a)

[0072] In this equation N x and N y is the horizontal and vertical resolution of the input image. The Gaussian smoothing kernel G is calculated as follows:

[0073] (Equation 7b)

[0074] The smoothed frequency map is calculated as follows:

[0075] (Equation 7c)

[0076] The smoothing effect has undesirable side effects, which is u' The value range of u It means that it decreases compared to the value range of. To solve this problem, a filtered frequency map u' is scaled as follows:

[0077] (Equation 8a)

[0078] This scaling of the filtered frequency map represents an appropriate solution for still images. However, for video content u and u' Scaling by the ratio of the maximum value of may cause transient artifacts, particularly flickering. These transient artifacts can be resolved by applying these maximum values ​​to a process known as leakage integration, which is achieved as follows: subscript t represents the frame number, and the frame t- Scaling for 1 is s t-1 It is assumed that it is specified as. The scaling of the current frame is s t It is given as.

[0079] (Equation 8b)

[0080] Afterwards, frame t The scaled frequency map for is It is given as. For video content, this is a suitable solution for scaling frequency maps. α The value of is within the range [0; 1]. An exemplary value is α = is given as 0.8.

[0081] As in the first method, using Barton's contrast sensitivity function, now set the pixel position by setting the following. x human visual sensitivity S(x) Can determine:

[0082] S(x) = S( Read ") (Equation 9a)

[0083] For reference, Equations 6 through 9 are mostly expressed as per-pixel operations, with the exception of the filtering step; however, the filtering step is inevitably a spatially varying operation and cannot be expressed as a per-pixel operation. Nevertheless, the procedure of Equations 6 through 9 can be summarized as a single vector value expression as follows:

[0084] (Equation 9b)

[0085] In this equation S img is a pixel map of contrast sensitivity equivalent to the per-pixel formula of Equation 9a, and L is a pixel map for the luminance values ​​of the input image, and k is a constant representing the scaling of Equation 8a or Equation 8b.

[0086] As before, the minimum detectable modulation is determined.

[0087] (Equation 10)

[0088] Note that ITU-R Report BT.2246-7 (Current State of Ultra High Definition TV) mentions an alternative minimum detectable modulation. However, This involves supercritical value adjustment, meaning it results in a noticeably different outcome from the input. Therefore, this method is not recommended.

[0089] Using the minimum detectable modulation, the following multiplier r(x) is induced.

[0090] (Equation 11)

[0091] In this equation, the scalar By using, adjustments of less than 1 JND can be applied. In this application, the processed result is f = 1 It was possible to see that the input for the value of could not be visually distinguished.

[0092] The first method described above applies a scale based on this multiplier to each of the three color values ​​of a pixel in RGB space. However, this can lead to subtle changes in hue and chromaticity errors. A better approach is this multiplier in the LCh color space, which allows for the individual adjustment of luminance, chromaticity, and hue. r(x) It applies a scaling factor based on this. In this color space, luminance is modified as follows.

[0093] L'(x) = L(x) r(x) (Equation 12a)

[0094] Since the perception of luminance and chromaticity is linked, color difference is also somewhat reduced.

[0095] C'(x) = C (x)(k + (1 - k)r (x)) (Equation 12b)

[0096] In this equation k is a constant that determines the amount of adjustment. k The range of values ​​must be limited to the interval [0,1]. Appropriate adjustment is k = is obtained for a value of 0.5. This specific k For the value, the above equation can be simplified as follows.

[0097] (Equation 12c)

[0098] After this adjustment, the image is converted to the desired output color space (e.g., RGB color space).

[0099] 도 2 represents a flowchart of a process for reducing the brightness of an image according to at least one embodiment. The process (200) is implemented, for example, by a processor (301) of a display device (300).

[0100] In step 210, the image to be modified is obtained, for example, by receiving it through a communication interface (303) or by loading it from memory (302).

[0101] In step 220, a contrast sensitivity function (CSF) is obtained. The contrast sensitivity function (CSF) may be predetermined. In the latter case, it may be received via a communication interface (303) or loaded from memory (302). In at least one embodiment, Barton's CSF model is used to determine the CSF.

[0102] In step 230, the frequency hierarchy map for the image is determined based on the pixels of the image. At pixel location x, the hierarchy map is the frequency It indicates how much contrast can be used. In at least one embodiment based on the first method, this is performed using a discrete wavelet transform. In at least one embodiment based on the second method, this is performed using a continuous wavelet transform.

[0103] Loop 240 of steps 250 through 280 is repeated for a pixel (x) of the image.

[0104] In step 250, the local contrast for a pixel (x) for a set of frequencies is determined by reading the frequency hierarchy map.

[0105] In step 260, human sensitivity to pixel (x) is determined for a set of frequencies.

[0106] In step 270, the frequency at which the product of local contrast and human sensitivity is highest is determined, and the minimum detectable modulation is determined by evaluating the inverse of the contrast sensitivity function obtained in step 220 for both the luminance associated with pixel (x) and the frequency determined in step 270.

[0107] In step 280, the luminance of pixel (x) is reduced by a coefficient corresponding to the minimum detectable modulation. As described above, the initial pixel luminance L and new low pixel luminance from contrast sensitivity related to pixels L low ...is determined. In other words, the pixel luminance is scaled (i.e., reduced) by an amount based on the minimum distinguishable difference value or the minimum detectable modulation (identical to that described above). This scaling can be used interchangeably with the scaling described in relation to the first method or the second method.

[0108] In step 290, an image with reduced brightness is provided. The image may then be displayed or transmitted to a displayable device.

[0109] The process (200) applies to both of the two methods described above. For the first method based on the discrete wavelet transform, the correspondence between the steps of the process (200) and the equations is as follows: Equation 1 for step 220, Equation 2a for step 230, Equations 2b, 2c, and 2d for step 233, Equations 2e and 2f for step 236, Equations 3a and 3b for step 260, Equation 4 for step 270, and Equations 5a and 5b for step 280. For the second method based on a continuous wavelet transform, the correspondence between the steps of the process (200) and the equations is as follows: Equation 1 for step 220, Equation 6 for step 230, Equations 7a, 7b, 7c for step 233, Equations 8a, 8b for step 236, Equations 3a, 3b for step 260, Equations 10, 11 for step 270, and Equations 12a, 12b, 12c for step 280.

[0110] The second method was tested on hundreds of SDR images. The test results showed that the brightness reduction averaged 8.86%. Since the relationship between the energy consumed and the light generated in OLED displays is nearly linear, energy consumption can be reduced.

[0111] Also, for SDR images, the peak luminance is approximately 2.88 on average. cd / m 2 It decreases (peak brightness is 100 cd / m 2 (assuming that). This means that in the case of LCD displays, the backlight can be reduced by about 2.88%, and since the backlight is the main cause of energy consumption, energy consumption can be reduced equally.

[0112] Peak brightness is 1000 cd / m 2 For the HDR image dataset (ITU-R Report BT.2245), the average reduction is 10.5% and the maximum luminance is an average of 121 cd / m 2 It decreases. Therefore, the energy savings possible with this method are significant, especially in the case of HDR images.

[0113] 도 3 The invention illustrates a display device that implements a method for reducing energy consumption according to at least one embodiment. The display device (300) includes a processor (301), memory (302), a communication interface (303), and a display panel (304).

[0114] The processor (301) is configured to acquire an image to be displayed via a communication interface (303). The image may be stored in memory (302) to perform necessary calculations before being provided to a display panel (304).

[0115] The display device (300) includes a processor (301) configured to execute instructions loaded therein to implement, for example, various embodiments described in this document. The processor may be a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an Application Specific Integrated Circuit (ASIC), a field programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, etc. The processor may include built-in memory, an input / output interface, and various other circuit parts as known in the art.

[0116] The display device (300) includes memory (302) which may include non-volatile memory and / or volatile memory, including but not limited to electrically eraseable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash, magnetic disk drive and / or optical disk drive. As a non-limiting example, the memory may include an internal storage device, an attached storage device (including a removable storage device and a non-removable storage device), and / or a network-accessible storage device.

[0117] A display device (300) includes a communication interface (303) that provides input and / or output to the device (300). Such input includes, but is not limited to, (i) an RF portion that receives a radio frequency (RF) signal transmitted over the air by, for example, a broadcaster, (ii) a component (COMP) input terminal (or a set of COMP input terminals), (iii) a Universal Serial Bus (USB) input terminal, (iv) a High Definition Multimedia Interface (HDMI) input terminal, and / or (v) composite video.

[0118] In various embodiments, the communication interface (303) has associated input processing elements known in the art. For example, the RF portion may be associated with elements suitable for (i) selecting a desired frequency (also referred to as selecting a signal, or band-limiting the signal to a frequency band), (ii) downconverting the selected signal, (iii) band-limiting it again to a narrower band of frequency to select a signal frequency band that may be referred to as a channel in a given embodiment, (iv) demodulating the downconverted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired stream of data packets. The RF portion of various embodiments includes one or more elements for performing these functions, such as a frequency selector, a signal selector, a band-limiter, a channel selector, a filter, a downconverter, a demodulator, an error corrector, and a demultiplexer. The RF section may include a tuner that performs various such functions, for example, including down-converting the received signal to a lower frequency (e.g., an intermediate frequency or a frequency near the baseband) or to the baseband. In one embodiment, the RF section and its associated input processing element receive an RF signal transmitted over a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and re-filtering it to a desired frequency band. Various embodiments rearrange the order of the aforementioned (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions. Adding elements may include inserting elements between existing elements, such as inserting an amplifier and an analog-to-digital converter. In various embodiments, the RF section includes an antenna.Additionally, the communication interface (303) may include USB and / or HDMI terminals that may each include an interface processor for connecting the device (300) to another electronic device via a USB and / or HDMI connector. The mode of USB or HDMI interface processing may be implemented within a separate interface IC or within the processor (301) as needed.

[0119] The communication interface (303) may include, but is not limited to, a transceiver configured to transmit and receive data through a communication channel. The communication interface (303) may include, but is not limited to, a modem or a network card and may be implemented, for example, within a wired and / or wireless medium.

[0120] In various embodiments, data is streamed to or otherwise provided to a display device (300) using a wireless network, such as a Wi-Fi network, for example, IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). Wi-Fi signals in these embodiments are received through a communication interface (303) configured for Wi-Fi communication. The communication channel in these embodiments is typically connected to an access point or router that provides access to an external network, including the Internet, which enables streaming applications and other over-the-top communication. Another embodiment provides the streamed data to the display device (300) using a set-top box that transmits the data through the HDMI connection of the communication interface (303). Yet another embodiment provides the streamed data to the display device (300) using the RF connection of the communication interface (303). As described above, various embodiments provide data in a non-streaming manner. Additionally, various embodiments use a wireless network other than Wi-Fi, for example, a cellular network or a Bluetooth network.

[0121] In at least one embodiment, the display panel (304) is based on an OLED display panel. In a variant embodiment, the display panel (304) is based on an LED display panel, or alternatively, on a mini LED display panel or a micro LED display panel. In another variant embodiment, the display panel includes a light-generating backlight (not shown) coupled to a light-transmitting panel that filters light to generate an image. The backlight may be based on an LED, mini LED, micro LED, OLED, QD-LED (Quantum Dot-Light Emitting Diode), or CCFL (Cold-Cathode Fluorescent Lamps). The light-transmitting panel may be a Liquid Crystal Display (LCD) panel.

[0122] Various elements of a display device (300) may be provided within an integrated housing. Within the integrated housing, various elements may be interconnected and data may be transmitted between them using a suitable connection arrangement, such as an internal bus (305) as known in the art, which includes an inter-IC (I2C) bus, wiring, and a printed circuit board.

[0123] In at least one embodiment, the device (300) is not a display device and does not include a display panel (304) located in a second device coupled to the device (300) via a communication interface (303), and the second device may include a display panel. Examples of such devices include a set-top box, a media player, a Blu-ray or DVD player, or a more general content receiver and decoder. In this embodiment, the result of the method is that the image contains less light and therefore requires less energy when displayed.

[0124] Although some embodiments of the present invention are illustrated in the accompanying drawings and described in the foregoing description, it should be understood that the present invention is not limited to the disclosed embodiments and that numerous rearrangements, modifications, and substitutions are possible within the scope of the invention as presented and defined by the following claims without departing from the invention.

Claims

Claim 1 A method comprising: a step of determining a scaling value based on human visual sensitivity at the location of an image pixel x located at a position within an input image; and a step of scaling the luminance of the pixel by an amount based on the scaling value, wherein the human visual sensitivity at the location of the pixel x is: Determined by, and the above CWT is a continuous wavelet transform, and is a pixel map of the luminance values ​​of the input image above, and G is a smoothing kernel, and is a frequency defined by level i of the above CWT, and is a constant representing the second scaling value, and is a method that is a contrast sensitivity function. Claim 2 In claim 1, the method wherein the amount of scaling is the same as the scaling value. Claim 3 A method according to claim 1, wherein the amount of scaling is smaller than the scaling value. Claim 4 A method according to claim 1, wherein the amount of scaling is greater than the scaling value. Claim 5 In claim 1, the method wherein the contrast sensitivity function is given by Barten's model. Claim 6 In claim 1, the method wherein the continuous wavelet transform is isotropic. Claim 7 A device comprising at least one processor, wherein the at least one processor is configured to determine a scaling value based on human visual sensitivity at the location of an image pixel x located at a location within an input image, and to scale the luminance of the pixel by an amount based on the scaling value, wherein the human visual sensitivity at the location of the pixel x is: Determined by, and the above CWT is a continuous wavelet transform, and is a pixel map of the luminance values ​​of the input image above, and G is a smoothing kernel, and is a frequency defined by level i of the above CWT, and is a constant representing the second scaling value, and is a device that is a contrast sensitivity function. Claim 8 In claim 7, the device further comprises a screen, and the processor is also configured to display an image having scaled brightness. Claim 9 In paragraph 8, a device selected from the group including TV sets, smartphones, laptops, cameras, and tablets. Claim 10 In claim 8, the screen is a device based on LCD display technology having a uniform or non-uniform backlight based on OLED display technology, micro LED technology, mini LED technology, MEMs technology, CCFL, LED, mini LED, or micro LED technology. Claim 11 A computer program product stored on a recording medium that includes instructions for executing a step of a method according to any one of claims 1 to 6 when the program is executed on a computer. Claim 12 A non-transient processor-readable recording medium comprising program code instructions for executing a step of a method according to any one of claims 1 to 6 when the program code instructions are executed on a computer. Claim 13 delete Claim 14 delete Claim 15 delete Claim 16 delete Claim 17 delete Claim 18 delete Claim 19 delete Claim 20 delete Claim 21 delete Claim 22 delete Claim 23 delete Claim 24 delete Claim 25 delete Claim 26 delete Claim 27 delete Claim 28 delete Claim 29 delete Claim 30 delete Claim 31 delete Claim 32 delete Claim 33 delete Claim 34 delete Claim 35 delete Claim 36 delete Claim 37 delete Claim 38 delete

Images