Method and device for reducing display energy by using temporally alternating complementary colors

Abstract

The method and device reduce the energy consumption required to render an image by replacing pixels of the image with temporally consecutive pixels of alternating complementary colors, which require less energy to display. This solution exploits the flicker fusion properties of the human visual system to display temporally consecutive alternating complementary colors that visually have the same perceptual properties as corresponding monochrome colors. The alternating complementary colors are selected to be more economical than monochrome colors in terms of the energy consumption required to render the colors. This combination doubles the dimension of the search space for energy reduction from three to six. The temporal modulation used to replace pixels with temporally consecutive pixels of alternating complementary colors is performed by frame doubling, frame skipping, or frame averaging. The association between colors and corresponding alternating complementary colors can be stored in a lookup table. These principles can be used for images or videos consisting of image sequences.

Description

[Technical Field]

[0001] At least one of the present embodiments relates generally to reducing energy consumption in display devices, and more particularly to methods and devices for reducing the energy required to render an image by replacing pixels of the image with temporally consecutive pixels of alternating complementary colors that require less energy to display. [Background technology]

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to European Patent Application No. 22306995.6, filed December 22, 2022, which is incorporated herein by reference in its entirety.

[0003] Reducing the energy consumption of electronic devices has become a requirement not only for electronic device manufacturers, but also for those seeking to limit their environmental impact as much as possible and contribute to the emergence of a sustainable display industry. The increase in display resolution from SD to HD, then 4K, and potentially 8K and beyond in the near future, coupled with the introduction of high dynamic range imaging, has correspondingly increased the energy demands of display devices. This is inconsistent with the global need to reduce energy consumption, given the vast number of devices (i.e., televisions, mobile phones, tablets, etc.) that contain displays. In fact, displays are the most significant energy consumer for consumer electronic devices, whether they are battery-powered (e.g., smartphones, tablets, head-mounted displays, in-vehicle display screens) or non-battery-powered (e.g., television sets, advertising display panels).

[0004] Various display technologies have been developed in recent years. Although modern displays consume energy in a more controllable and efficient manner than older displays, they still represent the most significant energy consumer in the video chain.

[0005] Organic light-emitting diodes (OLEDs) are an example of a display technology that has become increasingly popular due to many advantages over previous technologies such as thin-film transistor liquid crystal displays (TFT-LCDs). Rather than using a uniform backlight, OLED displays consist of individual LEDs as image pixels. As such, OLED energy consumption is highly correlated with the image content, and the energy consumption of a given input image can be estimated by considering the values ​​of the image pixels being displayed.

[0006] Although OLED displays consume energy in a more controllable and efficient manner, they still represent the most significant energy consumer in the video chain. Various techniques have been developed to reduce the energy required to display an image on a display device. Until now, most solutions to the problem of reducing the energy required to display a single image pulse have focused on slightly modifying the color of each frame of the pulse in terms of luminance and / or color. As a result, these solutions limit the number of dimensions searched to find a more energy-saving image: they propose alternative pixel colors for each pixel color, limiting the dimensionality of the search space to three color channels. [Prior art documents] [Non-patent literature]

[0007] [Non-Patent Document 1] De Lange, 1958 [Non-patent document 2] “Perfecting the color reproduction of RGBW OLED” proc.30th International Congress of Imaging Science Summary of the Invention

[0008] The embodiments described below are designed with the above in mind and describe a method and device for reducing the energy (i.e., power consumption) required to render an image by replacing pixels of the image with temporally consecutive pixels of alternating complementary colors, which require less energy to display. This solution exploits the flicker fusion characteristic of the human visual system to display temporally consecutive alternating complementary colors that visually have the same perceptual properties as corresponding monochrome colors. The alternating complementary colors are selected to be more economical than monochrome colors in terms of the energy consumption required to render the colors. This combination doubles the dimension of the search space for energy reduction from three to six. The temporal modulation used to replace pixels with temporally consecutive pixels of alternating complementary colors is performed by frame doubling, frame skipping, or frame averaging. The concept of consecutive pixels is temporal. In other words, when frame doubling is used, one pixel is replaced with two temporally consecutive pixels of alternating complementary colors, and the duration of the replaced pixels is halved. When using frame skipping or averaging, two temporally consecutive pixels are replaced by two temporally consecutive pixels of alternating complementary colors, with the replaced pixels being of the same duration.

[0009] The association between colors and their corresponding alternating complementary colors can be stored in a look-up table. These principles can be used for videos made up of images or sequences of images.

[0010] A first aspect of at least one embodiment is directed to a method including, for an input color of a pixel, determining alternating complementary color pairs based on the input color of the pixel, wherein the average color of the alternating complementary color pairs is identical or perceptually similar to the input color, and the total energy consumed by displaying the pixel pairs, each pixel having a respective one of the alternating complementary colors, is less than twice the energy consumed by displaying the pixel with the input color. A variation of the first aspect further includes iterating multiple times to generate a set of alternating complementary color pairs, and selecting an alternating complementary color pair from the set of alternating complementary color pairs that consumes the least amount of energy when displayed. A further variation of the first aspect includes iterating over a set of input colors that includes the colors of all pixels of the input image or a subset of all pixels of the input image, or over all possible color values ​​of a selected color space or a subset of all possible color values ​​of the selected color space. A further variation of the first aspect includes storing an association between the set of input colors and the corresponding determined alternating complementary color pairs.

[0011] A second aspect of at least one embodiment is directed to a method including obtaining a first pixel of a first image of a video; obtaining an alternating complementary color pair according to a first aspect based on a color of the first pixel of the first image of the video; inserting a second image into the video that is temporally consecutive to the first image; and setting the color of the first pixel of the first image to a first color of the alternating complementary color pair and setting the color of the second pixel of the second image to a second color of the alternating complementary color pair, wherein the first pixel and the second pixel are at the same location in each image.

[0012] A third aspect of at least one embodiment is directed to a method including obtaining pairs of temporally consecutive pixels of a video; obtaining alternating complementary color pairs according to the first aspect based on a color of a first pixel of the temporally consecutive pair of pixels; and setting the color of the first pixel of the temporally consecutive pair of pixels to the first color of the alternating complementary color pair and setting the color of the second pixel of the temporally consecutive pair of pixels to the second color of the alternating complementary color pair.

[0013] A fourth aspect of at least one embodiment is directed to a method including obtaining pairs of temporally consecutive pixels of a video; obtaining alternating complementary color pairs according to the first aspect based on an average color of the colors of the temporally consecutive pixel pairs; and setting a color of a first pixel of the temporally consecutive pixel pairs to a first color of the alternating complementary color pair and setting a color of a second pixel of the temporally consecutive pixel pairs to a second color of the alternating complementary color pair.

[0014] A fifth aspect of at least one embodiment is directed to a device comprising one or more processors configured to perform, for an input color of a pixel, determining alternating complementary color pairs based on the input color of the pixel, where the average color of the alternating complementary color pairs is identical or perceptually similar to the input color, and the total energy consumed by displaying the pixel pairs, each pixel having a respective one of the alternating complementary colors, is less than twice the energy consumed by displaying the pixel having the input color. A variation of the fifth aspect further includes generating the set of alternating complementary color pairs multiple times and selecting the alternating complementary color pair from the set of alternating complementary color pairs that consumes the least amount of energy when displayed. A further variation of the fifth aspect includes iterating over a set of input colors that includes the colors of all pixels of the input image or a subset of all pixels of the input image, or over all possible color values ​​of a selected color space or a subset of all possible color values ​​of the selected color space. A further variation of the fifth aspect includes storing an association between the set of input colors and the corresponding determined alternating complementary color pairs.

[0015] A sixth aspect of at least one embodiment is directed to a device comprising one or more processors configured to: obtain a first pixel of a first image of a video; obtain an alternating complementary color pair according to the fourth aspect based on a color of the first pixel of the first image of the video; insert into the video a second image that is temporally consecutive to the first image; and set the color of the first pixel of the first image to a first color of the alternating complementary color pair and set the color of the second pixel of the second image to a second color of the alternating complementary color pair, wherein the first pixel and the second pixel are at the same location within the respective images.

[0016] A seventh aspect of at least one embodiment is directed to a device comprising one or more processors configured to obtain pairs of temporally consecutive pixels of a video; obtain alternating complementary color pairs according to the fourth aspect based on a color of a first pixel of the temporally consecutive pair of pixels; and set the color of the first pixel of the temporally consecutive pair of pixels to the first color of the alternating complementary color pair and set the color of the second pixel of the temporally consecutive pair of pixels to the second color of the alternating complementary color pair.

[0017] An eighth aspect of at least one embodiment is directed to a device comprising one or more processors configured to: obtain pairs of temporally consecutive pixels of a video; obtain alternating complementary color pairs according to the fourth aspect based on an average color of the colors of the pairs of temporally consecutive pixels; and set a color of a first pixel of the pairs of temporally consecutive pixels to a first color of the alternating complementary color pair and set a color of a second pixel of the pairs of temporally consecutive pixels to a second color of the alternating complementary color pair.

[0018] A ninth aspect of at least one embodiment is directed to a computer program comprising program code instructions executable by a processor, the computer program performing at least the steps of a method according to the first, second or third aspect, or any variation of these aspects.

[0019] A tenth aspect of at least one embodiment is directed to a non-transitory computer-readable medium comprising program code instructions executable by a processor, the computer program product performing at least the steps of a method according to the first, second, or third aspect, or any variation of these aspects. [Brief explanation of the drawings]

[0020] [Figure 1]FIG. 1 is a block diagram of an example of a display device in which various aspects and embodiments may be implemented. [Figure 2A] FIG. 1 shows the normalized response spectrum of human cones. [Figure 2B] FIG. 1 illustrates the temporal contrast sensitivity function at different adaptation fields. [Figure 2C] FIG. 10 illustrates modulation sensitivity as a function of frequency for luminance and color flicker. [Figure 3] FIG. 10 illustrates an example of separating a color into alternating complementary colors, according to an embodiment. [Figure 4] FIG. 1 illustrates an example process for reducing energy consumption of pixels in an image using alternating complementary colors, according to an embodiment. [Figure 5] 3A-3C illustrate an example of a process for establishing candidate pairs of alternating complementary colors according to a first embodiment. [Figure 6] 10A-10C illustrate an example process for establishing candidate pairs of alternating complementary colors according to a second embodiment in a color space that provides color transformations and inverse color transformations. [Figure 7] FIG. 10 illustrates an example of pixel replacement with temporally consecutive pixels of alternating complementary colors according to an embodiment based on frame doubling. [Figure 8] FIG. 10 illustrates an example of pixel replacement with temporally consecutive pixels of alternating complementary colors according to an embodiment based on frame skipping. [Figure 9] FIG. 10 illustrates an example of pixel replacement with temporally consecutive pixels of alternating complementary colors according to an embodiment based on frame averaging. [Figure 10] 1A-1C illustrate an example process for generating a lookup table of alternating complementary colors according to an embodiment, and an example process for modifying an image using the lookup table. [Figure 11] 1A-1C illustrate two examples of alternate complementary color process development according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0021] 1 illustrates a block diagram of an example of a display device in which various aspects and embodiments may be implemented. In the illustrated environment, a user interacts with a display device 100 connected to a data provider 180 via a communications network 150.

[0022] Display device 100 includes processor 101. Processor 101 may be a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors in conjunction with a DSP core, a controller, a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, etc. The processor may perform data processing such as process 400 of FIG. 4, process 500 of FIG. 5, or process 600 of FIG. 6, and related embodiments operating in a uniform color space.

[0023] The processor 101 may be coupled to an input unit 102 configured to communicate user interactions. For this purpose, multiple types of input and modalities may be used. A physical keypad or a touch-sensitive surface are typical examples of input adapted for this application, although voice control may also be used. Furthermore, the input unit may comprise a digital camera capable of capturing two-dimensional still images or video, or a more complex sensor capable of determining depth information in addition to the photo or video, and thus capturing a full 3D representation.

[0024] The processor 101 may be coupled to a display unit 103 configured to output visual data to be displayed on a screen. Several types of displays may be used for that purpose, such as an organic light emitting diode (OLED) display unit. The processor 101 may also be coupled to an audio unit 104 configured to render audio data that is converted into audio waves via an adapted transducer, for example a speaker.

[0025] The processor 101 may be coupled to a communication interface 105 configured to exchange data with external devices. The exchange via the communication interface 105 preferably uses a wireless communication standard such as cellular (e.g., LTE) communication, Wi-Fi communication, etc. to provide mobility of the display device.

[0026] Processor 101 may access information from and store data in memory 106, which may comprise multiple types of memory including random access memory (RAM), read-only memory (ROM), a hard disk, a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, or any other type of memory storage device. In an embodiment, processor 101 may access information from and store data in memory that is not physically located on the device, such as on a server, a home computer, or another device.

[0027] The processor 101 may receive energy from an energy source 108 and may be configured to distribute and / or control the energy to other components within the device 100. The energy source may be any device suitable for powering the device. By way of example, the energy source may include one or more dry cell batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel-metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, etc.

[0028] While the figure depicts the processor 101 and the other elements 102-108 as separate components, it will be understood that these elements can be integrated into an electronic package or chip. It will be understood that the display device 100 can include any subcombination of the elements described herein while remaining consistent with the embodiments described below. The processor 101 can be further coupled to other peripheral devices or units not shown in FIG. 1 , which can include one or more software and / or hardware modules that provide additional features, functionality, and / or wired or wireless connectivity. For example, peripheral devices can include a universal serial bus (USB) port, a vibration device, a television receiver / transmitter, a hands-free headset, a Bluetooth module, a frequency modulation (FM) radio unit, a digital music player, a media player, a video game player module, an internet browser, etc. For example, the processor 101 can be coupled to a localization unit configured to locate the display device within its environment. The positioning unit may integrate a GPS chipset that provides longitude and latitude position relative to the current location of the display device, but may also integrate other motion sensors such as an accelerometer and / or electronic compass that provide positioning services.

[0029] Typical examples of display device 100 include smartphones, tablets, laptops, external monitors, head-mounted displays, television sets, video projectors, computer screens, vehicles (e.g., control and / or entertainment systems for automobiles, airplanes, ships, etc.), advertising display panels, medical monitors, etc. However, any device or combination of devices providing similar functionality while conforming to the principles of the present disclosure can be used as display device 100. In at least one embodiment, the device does not include a display unit but prepares data for display so that another device, such as a screen, can perform the display. Examples of such devices are set-top boxes, media players, desktop computers, encoders, decoders, servers, computing grids, cloud computers, etc.

[0030] The following embodiments describe methods for modifying image light pulses of color pixels while reducing the energy required to display the modified image pulses on a display device, while striving to maintain as much visual similarity and quality of experience as possible with the original light pulses. More generally, the embodiments are based on using temporal modulation to determine a color pair to replace an input color, where the average color of the color pair is identical or perceptually similar to the input color, and the energy of the color pair is less than twice the energy of the input color. The term "color energy" should be understood here as the energy required to render a color pixel. Various types of temporal modulation are described, such as frequency doubling, frame skipping, and frame averaging.

[0031] Compared to the state-of-the-art techniques, which involve selecting a different color with lower energy consumption in a single frame, using color pairs has the advantage of being more versatile, where the color pair is selected based on two parameters: energy consumption and visual match to the first color. This method expands the search space of lower-energy colors, thereby increasing the probability of finding a combination that results in even greater energy savings.

[0032] Figure 2A shows the normalized response spectrum (i.e., spectral sensitivity function) of human cones. Electromagnetic radiation is characterized by its wavelength (or frequency) and its intensity. The range of wavelengths perceptible to humans is approximately 380 nm to 780 nm. Wavelengths within this range are known as "visible light." Color perception is based on the differential sensitivity of different cells in the retina (color receptors: cones and rods) to different wavelengths of light. Human observers have three types of color receptors known as cone cells. This confers trichromatic color vision, and cones are typically labeled short (S), intermediate (M), and long (L) according to the wavelength of their peak spectral sensitivity, as shown in Figure 2A, or simply blue, green, or red according to the primary color centered around those peaks.

[0033] Trichromatic theory states that the color of a light spectrum perceived by a human observer can be characterized by three individual scalar values. From a mathematical perspective, this first step in human vision can be likened to that of a triple-kernel energy computation process. If si(λ) is the wavelength response of a given light spectrum, and l(λ), m(λ), and s(λ) are the spectral sensitivity functions of the L, M, and S cones, respectively, then Equation 1 defines Li, Mi, and Si. These are the three scalar values ​​that characterize the color of spectrum si(λ) as seen by a human observer.

[0034]

number

[0035] Although the spectrum of light reaching the eye from a given direction determines the color sensation in that direction, there are many more possible spectral combinations that result in the same color sensation. In colorimetry, the term metamerism refers to the association of the same apparent color of a light signal with different spectral power distributions. Such matching color spectra are called metameric spectra. Based on equation (1), the mathematical definition of metamerism is as follows:

[0036]

number

[0037] ∃λ∈[380,780],s1(λ)≠s2(λ) (2a)

[0038]

number

[0039]

number

[0040]

number

[0041] where s1(λ) and s2(λ) are the spectral responses of the two metameric (but separate) spectra.

[0042] Figure 2B shows the temporal contrast sensitivity function (TCF) for various adaptation fields. In the spatial domain, spatial vision can be characterized by the contrast sensitivity function (CSF). The TCSF or DeLange function can be plotted (see non-patent document 1). The TSF is a plot showing how flicker varies with contrast and vice versa. In this figure, the area above the curve represents the area where flicker is not perceived by a human observer, and the area below the curve represents the area where flicker is perceived. The eye appears to be most sensitive to flicker frequencies between 15 and 20 Hz at high luminance (photopic vision). At photopic light levels, the contrast required to detect a stimulus is less than 1%, and the high-temporal frequency cutoff is around 60 Hz. At low light levels, the maximum contrast is about 20%, and the high-temporal frequency cutoff is approximately 15 Hz. Maximum contrast is required to detect high-frequency flicker. Temporal resolution is less efficient at low luminance (scotopic vision).

[0043] Figure 2C shows modulation sensitivity as a function of frequency for luminance and chromatic flicker. In this figure, luminance levels are measured in trolands (td), which characterize retinal illuminance. This figure was obtained in a typical application of heterochromatic flicker photometry (HFP) in psychovisual studies. Participants viewed stimuli in which two lights of different colors rapidly alternated over time. They were then asked to adjust the intensity (i.e., the spectral amplitude) of one of the lights to minimize the perceived flicker caused by the alternating lights. The left figure is for luminance flicker, and the right figure is for chromatic flicker. HFP has long been the standard psychophysical method for finding isoluminant colors.

[0044] The principles shown in Figures 2A, 2B, and 2C are used to determine pairs of colors that, when combined in time, are perceived by a human observer as distinct (single, stable) colors. The technical effect used herein relies on temporal psychovisual modulation and the existence of a maximum cutoff frequency in the human eye's flicker sensitivity. Therefore, the high-level principle of the present invention can be thought of as adding a dimension to an image signal by temporally transforming each pixel into two visually complementary, temporally consecutive pixels, and using this added dimension to minimize pixel-equivalent energy consumption. This principle is referred to herein as alternating complementary color (ACC). Two temporally consecutive pixels will be perceived by a user as a single pixel if their alternation is faster than the flicker fusion frequency.

[0045] The typical flicker fusion frequency is about 50 Hz to 60 Hz, depending on retinal illuminance. However, sensitivity to flicker in isoluminant situations is less (20 Hz to 30 Hz) than in situations where the luminance changes between the two images of a pair. The additional specific condition of a small luminance difference between the two colors minimizes flicker caused by alternating two colors. This isoluminant condition, combined with the alternating display of basic colors, can limit the visibility of flicker.

[0046] FIG. 3 illustrates an example of decomposing a color into alternating complementary colors, according to an embodiment. In this diagram, line 300 represents a sequence of pixels 301 through 306. The three numbers within each block correspond to the color of the corresponding pixel, expressed as RGB values ​​expressed using 8-bit depth. For example, the first pixel is defined by the following values ​​of the pixel's color components: red 147, green 107, and blue 0. This results in a brown pixel. The colors of the other pixels are: second pixel 302 is medium gray, third pixel 303 is navy blue, fourth pixel 304 is dark magenta, fifth pixel 305 is reddish-brown, and sixth pixel 306 is bright green, respectively.

[0047] Line 310 shows a set of temporally consecutive pixel pairs (301A, 301B) through (306A, 306B). These temporally consecutive pixel pairs correspond to alternating complementary colors that can be used to replace the original pixels 301 through 306. As with line 300, the values ​​within the blocks represent the colors of the temporally consecutive pixels. In at least one embodiment, the temporally consecutive pixels are half the duration of the original pixels. In other words, a first image frequency (e.g., 60 Hz) is doubled to a second image frequency (e.g., 120 Hz), and the input image is decomposed into an output image pair displayed at the second image frequency. For example, pixel 301, displayed in the input image at a frequency of 60 Hz, can be replaced by a sequence of pixels 301A (a green pixel) and 301B (a red pixel), displayed at a global frequency of 120 Hz. A succession of green and red pixels is perceived by a human observer as a brown pixel due to heterochromatic flicker fusion. A complete example is described below in connection with FIG. 7. In other embodiments, other techniques such as frame skipping or frame averaging are used, for example, when doubling the display frequency is not possible. Examples of such methods are described below in connection with FIG. 8 and FIG. 9.

[0048] 4 illustrates an example process for reducing energy consumption of pixels of an image using alternating complementary colors, according to an embodiment. Process 400 may be performed, for example, by processor 101 of device 100 of FIG. 1. In at least one embodiment, process 400 is iterated over a set of colors consisting of the colors of all pixels of the input image. In another embodiment, process 400 is iterated over a set of colors consisting of all possible color values ​​according to a selected color space. In other embodiments, iteration occurs over a subset of pixels or a subset of color spaces.

[0049] In step 410, the processor calculates the color c of pixel p. IN Get.

[0050] In step 420, the processor IN Alternating complementary color pairs c corresponding to A , c B The color pair is selected based on two constraints. The first constraint is related to the quality of experience, and the color c A and c B Temporally consecutive pixels p A and p B The combination of color c IN The second constraint is related to reducing the energy required for display. In embodiments using frame doubling, this is achieved by A and c B Two time-modulated half-periods of temporally consecutive pixels p A and p B The energy required to display the color c IN In other embodiments using frame skipping or frame averaging, the second constraint is verified in a different manner, as described further below. Step 420 selects the first color c according to certain criteria, as described in further embodiments. A Then, select a suitable second color c according to the similarity and energy reduction constraints.B This involves determining the input color c IN Step 420 is repeated 415 multiple times to determine a set of candidate pairs of alternating complementary colors {C AB} is determined.

[0051] In step 430, the processor IN A candidate pair C is a pair of temporally consecutive pixel colors to replace the pixel A , C B , for example, select the candidate pair with the lowest energy consumption.

[0052] In step 440, the processor modulates the color c according to one of the temporal modulation techniques presented herein. IN Pixel p of color c A and c B Two temporally consecutive pixels p A and p B Replace with.

[0053] Regarding color similarity, the first constraint is verified by A , c B The average of color c IN The color pair average is calculated in a display color space, a standard color space, or a color space that represents human color vision. Examples of display color spaces are sRGB and AdobeRGB. Examples of standard color spaces (also known as measurement color spaces) are CIEXYZ and CIELUV. Examples of color spaces that represent human color vision (also known as uniform color spaces) are CIELab, IPT, OKLab, and OSA-UCS. Therefore, the resulting colors of the processed image displayed for human perception will be the desired colors similar to the source image, but may vary depending on the display color power model (c A , c B ) Energy consumption is reduced due to the proper selection of color pairs.

[0054] Regarding energy consumption, in embodiments using frame doubling, the verification of the second constraint is Aand c B Each of the two temporally consecutive pixels p A and p B It is based on comparing the energy of pixel p of color c with the energy of pixel p of successive pixels in time. A and p B The duration of display of pixel p is half the duration of pixel p. Energies mentioned throughout this document are based on a display color power model. A simple example of such a model is based on the sum of RGB pixel values ​​raised to the power of gamma, where gamma is between 1.8 and 2.3, e.g., 2.2. For example, pixel 302 in Figure 3, with RGB values ​​of 127, 141, 141, is 2.2 +141 2.2 +141 2.2 The first pixel 302A in the temporal sequence with RGB values ​​of 147, 147, 0 is represented by a color power value of 1 / 2×(147 2.2 +147 2.2 +0 2.2 )=29420. The second of the temporally consecutive pixels, 302B, is represented by a color power value of 103042. As shown in Table 1, an energy comparison leads to the conclusion that it would be more efficient to replace pixel 302 (with a color power value of 149480) with the combination of the pair of temporally consecutive pixels 302A and 302B (which has a lower combined color power value of 132462).

[0055] [Table 1]

[0056] In another embodiment using frame skipping, the second constraint related to energy consumption is verified by comparing the color power values ​​of the original pixel and the skipped pixel with the determined pair of replacement pixels (302A and 302B above).

[0057] In another embodiment using frame averaging, the second constraint related to energy consumption is verified by comparing the color power values ​​of the two averaged pixels with the pair of replacement pixels (302A and 302B above) determined for the averaged pixels.

[0058] The color presented to a human observer is determined by the display response to an RGB triplet, such as in sRGB or BT-709 or other display color space implemented in a display instance or model with given parameter adjustments (brightness, contrast, color temperature, etc.). A color space {C} is chosen in which the color operations are realized. A display whose colors are represented by {C} has a color gamut {G} that represents the complete subset of colors the display can render. For an image or video, each pixel color pulse C IN is C A and C B is replaced by a subpulse of C A and C B The temporal combination of C IN The color perception of C A and C B The energy consumption of IN will be lower than the original consumption.

[0059] Repeating step 420 generates a set of candidate pairs of complementary colors alternating in time {C AB} is created. From this set, a preferred candidate pair (e.g., one with the lowest energy consumption) can be chosen for a given input, thus creating an association between the input color and pairs of temporally alternating complementary colors. In at least one embodiment, this association is stored in a lookup table. This allows for faster implementation by eliminating the need for the display device to perform all iterations of step 420 again for each image. Thus, at least one embodiment includes repeating steps 410, 420, and 430, e.g., across all possible colors in the color gamut, to create the lookup table, while other embodiments include repeating only steps 410, 430, and 440 across all pairs of pixels in the input image, resulting in a modified image that requires less energy to display. Exemplary embodiments are shown in FIGS. 10 and 11.

[0060] Figure 5 shows an example of a process for establishing candidate pairs of alternating complementary colors according to the first embodiment. Process 500 may be performed, for example, by processor 101 of device 100 of Figure 1 and corresponds to step 420 of Figure 4. This process begins with an input color triplet R IN G IN B IN In step 510, the processor calculates the energy consumption P of the input color triplet according to the selected color power consumption model. IN The power consumption model is display dependent. For OLED displays, the power consumption model can follow a color model, including the RGBW case where a white LED supports an RGB LED in each physical pixel. A color model for RGBW displays is given by Murdoch et al. in "Color Modeling for RGBW Displays," IEEE Transactions on Color Science, Vol. 1, No. 1, pp. 111-114, 2003. This model can be extended with appropriate parameters to represent the power per pixel. In step 520, the processor determines the power consumption of a pixel within the color gamut {G} of the color space {C}. IN G IN B IN Color point C corresponding to the triplet INThe color space for this process is selected from among the display color space, the standard color space, or the human visual color space. In step 530, the processor samples the color space {C} within the color gamut {G} to determine a set of candidate colors for the first pixel of temporally consecutive pixels {C A Various criteria can be used to determine the sampling space. In at least one embodiment, a maximum color distance criterion is used to determine the sampling space C IN In various embodiments, the set of candidates is limited to saturated colors, or grayscale colors (i.e., part of the gray ramp), or C IN or a combination of these colors. In at least one embodiment, the entire color gamut space is searched, so the set of candidates is the complete set of possible values. In at least another embodiment, a subset of the color gamut space is chosen, for example, using a smaller color resolution. In another embodiment, several randomly selected candidates are used.

[0061] The process is then repeated through steps 540 through 590 to determine the set of candidate colors for the first pixel of the temporally consecutive pixels {C A}, a set of candidate color pairs is constructed. In step 540, the processor A , the second color C of the second pixel of the temporally consecutive pixels B of, C B =2.C IN -C A , In other words, 2.C IN =(C A +C B ) It is determined so that

[0062] This results in color C A and C B The combination of temporally consecutive pixels of C IN is the color C in color space {C} Aand C B Since it is the average of at least color C IN In step 550, the processor B Verify that the color is contained in the gamut {G} of the color space {C}. In fact, if a color is outside the gamut, it cannot be displayed, so C A and C B Subpulse combination is color C IN If the color is out of gamut, it will not result in the correct color pair for temporally consecutive pixels, so C A The iteration over the selected values ​​of C stops. In this case, if there are any left in the set, the process A The iteration resumes at step 540 for the next value of .

[0063] In step 560, the processor A , C B for the corresponding triplet R A G A B A , R B G B B B Determine.

[0064] In step 570, the processor A , C B The energy consumption of the pixel combination P AB Since the power consumption model of the display device is not necessarily known, the energy consumption to display a color can be approximated simply by the sum of its RGB values ​​to the gamma power, as shown in Table 1. In embodiments using frame doubling, this is evaluated over a half period using the selected color power consumption model as follows:

[0065]

number

[0066] In step 580, the processor AB <P IN In fact, a candidate pair of temporally consecutive pixel colors is considered only if it results in some energy reduction. If not, the candidate pair is discarded, and if there are any remaining in the set, the process A The iteration resumes at step 540 for the next value of P. A +P B <2.P IN It can also be formulated as, where P A , P B , P IN are color C A , C B , C IN represents the energy of the pixel in the input color. In other words, in embodiments using frame doubling, a candidate pair of alternating complementary colors is considered if the sum of the energy of the pair of alternating complementary colors is lower than twice the energy of the input color. In step 590, the processor A , C B Let us denote the set of candidate pairs {C AB}.

[0067] In step 430, the processor IN A candidate pair C is a pair of temporally consecutive pixel colors to replace the pixel in A , C B In at least one variation, the processor selects one of a list of candidate pairs {C AB}, identify the position of the candidate pair with the lowest energy consumption.

[0068]

number

[0069] As a result, the processor

[0070]

number

[0071] C IN Determine the color C as the best replacement for IN Pixels of color C Axmin and C Bxmin For example, if two successive pixels of color C are replaced by two successive pixels of color C, each for half a period, the successive pixels appear to at least most human observers as color C. IN This reduces the energy consumption required to display the pixel while maintaining a high quality of experience, as the pixel is perceived as a single pixel.

[0072] In at least one embodiment, color C IN and its best color pair permutation C Axmin , C Bxmin The correspondence between the input color and the color pair is stored in the lookup table, allowing subsequent modifications to be performed very efficiently. By providing an input color to the lookup table, the corresponding color pair can be quickly obtained without having to perform process 400 of FIG. 4 and process 500 of FIG. 5 again.

[0073] In at least one embodiment, a mathematical minimization method, such as least-squares minimization, is used to replace steps 520 through 590 to find color C. IN and its best color pair permutation C Axmin , C Bxmin Find the correspondence between

[0074] In at least one embodiment, the triplet R IN G IN B INAn additional step is added, for example between steps 520 and 530, to check that the triplet R spatially belongs to the subset of the image we want to process, e.g., belongs to a region in the image that has the best ability to mask artifacts. Such a region or mask can be given, for example, by a spatiotemporal just noticeable difference (JND) map, a motion field, a saliency map, etc. IN G IN B IN If does not belong to this region or mask, then no color pair replacement is considered for this color.

[0075] In at least one embodiment, the triplet {R IN G IN B IN} are ordered according to their energy consumption, so that the triplets are processed in descending order, from most energy-consuming to least energy-consuming. In such a case, a threshold can be defined corresponding to the overall energy reduction to be achieved. If this threshold is reached when a given number of triplets have been processed, the overall process is stopped. Other ordering criteria can also be defined, such as determining the RGB combination that can be replaced on the display with the greatest energy consumption. Replacement power ratio

[0076]

number

[0077] A map in color space can be constructed that stores the set of colors in R before replacement is performed on the image pixels. p Other examples of ordering would be to make it depend on the distance between the input color and the replacement color, or to select RGB values ​​on the border of the gamut first, or in descending order of saturation value.

[0078] 6 shows an example of a process for establishing candidate pairs of alternating complementary colors according to the second embodiment in a color space that provides a color transform and an inverse color transform. Process 600 is performed, for example, by processor 101 of device 100 of FIG. 1 and corresponds to step 420 of FIG. 4. The difference from the first embodiment is that the operations are performed in a color space that provides a color transform and an inverse color transform. Examples of such color spaces are the CIE XYZ color space or the OKLab color space used herein.

[0079] When using CIE XYZ, a color is defined by a triplet of coordinates in XYZ space, where Y is luminance, Z is approximately equal to B, and X is a blend of the three RGB curves. A compression / expansion transformation is conventionally applied (also called "applying gamma" or "companding"), i.e., raising each RGB value to the power of a gamma value; this is display-dependent and not shown in the diagram. A forward color transform FT(RGB) allows the XYZ coordinates to be calculated from the RGB values, and an inverse color transform IT(XYZ) applies the inverse operation, i.e., allows the RGB values ​​to be calculated from the XYZ coordinates, followed by an inverse companding operation.

[0080] This process uses the input color triplet R IN G IN B IN In step 610, the processor calculates the energy consumption P of the input color triplet according to the selected color power consumption model. IN In step 620, the processor determines R in the color gamut {G} of the color space {C}. IN G IN B IN Color point C corresponding to the triplet IN In step 630, the processor performs a forward color transformation from RGB to XYZ as R IN G IN B IN Applying it to a triplet, we can calculate the X in XYZ space. IN Y IN Z INThe forward color transform is display dependent. An example of a color transform is the sRGBtoXYZ matrix, where the white point corresponds to the CIE standard illuminant D65. In step 640, the processor samples the color space {C} within the color gamut {G} to obtain a set of candidate colors for temporally consecutive pixels, {C A}. Determine the coordinate X of the candidate color. A Y A Z A can verify certain conditions. In the first variant, X A =Y A =Z A And the chosen C A The color will be part of the gray ramp (i.e., grayscale color). In the second variant, X A or Y A or Z A One of the coordinates is set to zero and the selected C A The color becomes saturated. In the third variant, Y A =Y IN And the chosen C A The color approaches isoluminance because there is no or minimal change in luminance compared to the input color. A =α.Y IN where α∈[0.8,1.2], which minimizes the change in luminance compared to the input color (quasi-isoluminance).

[0081] The process is then repeated through steps 650 to 690 to generate a set of candidate colors for temporally consecutive pixels {C A}, a set of candidate color pairs is constructed. In step 650, the processor A If the color is C IN Regarding C A The coordinate X is symmetric to B Y B Z B Second color C B In the selected color space, this is done simply using the following calculation:

[0082]

number

[0083] In step 655, the processor B Verify that the color is contained in the gamut {G} of the color space {C}. In fact, if a color is outside the gamut, it cannot be displayed, so C A and C B Subpulse combination is color C IN If the color is out of gamut, it will not result in the correct color pair for temporally consecutive pixels, so C A The iteration over the selected values ​​of C stops. In this case, if there are any left in the set, the process A The iteration resumes at step 650 for the next value of .

[0084] In step 660, the processor calculates the inverse transform R A G A B A =IT(X A Y A Z A ) and R B G B B B =IT(X B Y B Z B ) to find the color pair C A , C B The corresponding triplet R A G A B A , R B G B B B In step 670, the processor determines color C A , C B In embodiments using frame doubling, this is evaluated using the selected color power consumption model for each half period as follows:

[0085]

number

[0086] In step 680, the processor AB <P IN In fact, a candidate pair of temporally consecutive pixel colors is considered only if it results in some energy reduction. If not, the candidate pair is discarded, and if there are any remaining in the set, the process A The iteration resumes at step 650 for the next value of P. In embodiments using frame doubling, the test in step 680 is A +P B <2.P IN It can also be formulated as, where P A , P B , P IN are color C A , C B , C IN In other words, in embodiments using frame doubling, a candidate pair of alternating complementary colors is considered if the sum of the energy of the pair is lower than twice the energy of the input color.

[0087] In step 690, the processor selects the candidate pair C A , C B Let us denote the set of candidate pairs {C AB}.

[0088] In step 430, the processor IN A candidate pair C is a pair of temporally consecutive pixel colors to replace the pixel in A , C B The variations and embodiments for selecting one pair from a set of candidate pairs described in relation to Figure 5 can also be applied here.

[0089] The third embodiment is based on the same process as described in FIG. 6, with the difference that the color space selected is a uniform color space (e.g., CIELab, IPT, OKLab, or other). A uniform color space is constructed so that the same geometric distance (2-distance) anywhere in the color space reflects the same amount of perceived color difference. In the following description, the CIELab color space is selected. In this color space, color is expressed by three values: perceptual lightness L, and a and b, which relate to the four inherent opponent colors of human vision: red, green, blue, and yellow. In this embodiment, modifications of some steps in process 600 make it possible to accommodate a uniform color space. In step 630, the processor performs a forward color transformation from RGB to CIELab using R IN G IN B IN Applying it to triplets, L in CIELab color space IN a IN b IN The forward color transformation conventionally involves a gamma operation. In step 640, the processor samples the color space {C} within the color gamut {G} to obtain a set of candidate colors for temporally consecutive pixels {C A} is determined. The coordinates of the candidate color L A a A b A can verify certain conditions. A =b A And the chosen C A The color becomes part of the gray ramp (i.e., a grayscale color). In a second variant, L A a A b A is its cylindrical version L A C A h A But, L IN a IN b IN In the third variant, L is set to have the same luminance and hue as L, but with maximum saturation. A =L IN And the chosen C AThe color approaches isoluminance because there is no or minimal change in luminance compared to the input color. A =α.L IN where α∈[0.8,1.2], which minimizes the change in luminance compared to the input color (quasi-isoluminant). In step 650, the processor A Regarding color C IN Regarding C A The coordinate L is symmetric to B a B b B Second color C B In a uniform color space, this is done simply using the following calculation:

[0090]

number

[0091] In step 660, the processor calculates the inverse transform R A G A B A =IT(X A Y A Z A ) and R B G B B B =IT(X B Y B Z B ) to find the color pair C A , C B The corresponding triplet R A G A B A , R B G B B B The other steps of process 600 are identical. The variations and embodiments for selecting a pair from a set of candidate pairs described in relation to FIG. 5 also apply to the third embodiment.

[0092] 7 shows an example of pixel replacement by temporally consecutive pixels of alternating complementary colors according to an embodiment based on frame doubling. In this embodiment, the display frequency is doubled compared to the original image frequency. For example, if the sequence of images was intended to be displayed at 50 Hz, the display frequency is doubled and the modified sequence of images is displayed at 100 Hz, allowing the original pixels to be replaced by pixels of alternating complementary colors, thereby reducing the energy consumption of the display while maintaining the quality of experience.

[0093] In this figure, line 700 represents a temporal sequence of original (i.e., unmodified) images to be displayed, here including three images 701, 702, and 703. These images are displayed during periods t1, t2, and t3, respectively. In the example of a 50 Hz display frequency, these periods are 20 milliseconds long. To simplify the drawing, images 701, 702, and 703 are composed of two rows of three pixels each. The pixels are represented here as numbered blocks. The numbers identify pixel colors with reference to the colors introduced in FIG. 3. For example, the first line of image 701 is composed of pixels 301, 302, and 303. Thus, the first pixel 301 of this line is brown with RGB values ​​of 147, 107, 0; the second pixel 302 is navy blue with RGB values ​​of 127, 141, 141; and the third pixel 303 is dark magenta with RGB values ​​of 82, 108, 160. In the second line, the three pixels are navy blue, brown, and dark magenta, respectively.

[0094] Line 710 represents the temporal sequence of modified images to be displayed, including images 711, 712, 713, 714, and 715. Each of these images is displayed for half the duration compared to line 700, corresponding to the frequency doubling. Thus, the initial 50 Hz display frequency of line 700 is doubled to 100 Hz in line 710, resulting in periods t1A, t1B, t2A, t2B, and t3A of 10 milliseconds in length. Compared to line 700, twice as many images are displayed in line 710 (the final image, referenced 716, is not shown). This allows for intermediate images to be inserted to introduce pixels of alternating complementary colors, thereby reducing energy consumption when displaying the images.

[0095] For each pixel in original image 701, a color pair is determined as described above in connection with FIG. 5 or 6. This color pair is used to define a first pixel of the first color of the color pair in image 711 and a second pixel of the second color in image 712, and these two pixels are displayed consecutively at twice the intended display frequency of pixel 301. For example, brown pixel 301 in image 701 is replaced with green pixel 301A in image 711 and red pixel 301B in image 712. These replaced red and green pixels are displayed for half the time as the original brown pixel. As described above, thanks to the human visual system, these pixels are perceived by a human viewer as having brown pixel 301, and less energy is required to display them.

[0096] The frame doubling mechanism for pixel replacement with pixels of alternating complementary colors as applied to an image sequence, in other words, video, is presented in Figure 7. However, the same principles apply when displaying a single still image (e.g., a text editing application on a computer screen content, a settings screen on a tablet, an email application on a smartphone, a still image on an advertising screen, etc.). In this case, Figure 7 would be limited to elements related to image 701 (the single image being displayed) and images 711 and 712. Instead of displaying image 701 at a given frequency as in the past, images 711 and 712 are displayed alternately at twice the frequency.

[0097] FIG. 8 shows an example of pixel replacement with temporally consecutive pixels of alternating complementary colors according to a frame-skipping-based embodiment. This method can be used, for example, when the display frequency cannot be doubled. It is based on skipping one of two images in an image sequence, deriving a color pair from the colors of the remaining images, and replacing the original image sequence with an image sequence containing, in succession, an image containing a pixel of the first color of the color pair and an image containing a pixel of the second color of the color pair. Another way to describe this embodiment is to replace a pair of temporally consecutive pixels at the same location in two consecutive images with another pair of temporally consecutive pixels at the same location in two consecutive images, where the second pair is identical or perceptually similar to the first pair but requires less energy to display. The color of the new pair is chosen based on the color of the first pixel of the first pair.

[0098] This is illustrated in the figure, where line 800 shows the temporal sequence of original images 801, 802, 803, 804, and 805. Line 810 shows the temporal succession of displayed energy-reduced images 811, 812, 813, 814, and 815. Images 811 and 812 are obtained from original image 801 by determining color pairs that are identical or perceptually similar to the colors of original image 801, but that require reduced energy for display, as described above. For example, color 301 of first pixel p1 of first line in image 801 is processed as described above to determine color pair 301A, 301B. These colors are used in first image 811 and second image 812 of the sequence, respectively. Images 802 and 804 are discarded, so the original pixels of these images (e.g., first pixel p2 of the first line) are not considered at all in the displayed image.

[0099] In this embodiment, the constraint on reducing the energy required for display is now the color c A and c B Temporally consecutive pixels p A and p B The energy required to display the color c IN In fact, in such an embodiment, the energy required to display the original pixel p and the skipped pixels in A and p B is less than half the size compared to the original pixel and replaces two original pixels of the same duration: original pixel p1 and skipped pixel p2.

[0100] FIG. 9 illustrates an example of pixel replacement with temporally consecutive pixels of alternating complementary colors according to an embodiment based on frame averaging. This method can be used, for example, when the display frequency cannot be doubled. It is based on averaging the pixel colors of two consecutive frames and replacing these frames with two frames composed of alternating complementary colors determined based on the averaged colors. Another way to describe this embodiment is to replace a pair of temporally consecutive pixels at the same location in two consecutive images with another pair of temporally consecutive pixels at the same location in two consecutive images, where the second pair is identical or perceptually similar to the first pair but requires less energy to display. The colors of the new pair are chosen based on the average of the colors of the first pair. Compared to the frame skipping technique described in connection with FIG. 8, this allows all pixels of all images in the original image sequence to be taken into account. As a result, modified images 901 and 902 depend on images 801 and 802 of FIG. 8.

[0101] Line 900 shows a temporal sequence of energy-reduced images 901, 902, 903, 904, and 905 obtained by processing images 801, 802, 803, 804, and 805 of FIG. 8. In this method, the processor first averages the colors of the pixels in images 801 and 802 of FIG. 8. For example, a first pixel p1 (of color 301) in a first line of a first image is averaged with a first pixel p2 (of color 302) in a first line of a second image. In a preferred embodiment, the averaging is performed by converting the pixel color values ​​RGB to a uniform color space such as CIELab, calculating the average of the two pixels, and converting the result back to RGB color space. This results in an average color value 312 for the first pixel in the first line. Using the same method as above, the processor then averages the average color value 312 based on the average color value 312, calculating the color of each of the alternating complementary colors c of values ​​312A and 312B. A and c B These colors are determined for the first pixel p of the first line of the first image 901. Aand the first pixel p of the first line of the second image 902 B and is used.

[0102] In this embodiment, the constraint on reducing the energy required for display is now the color c A and c B Temporally consecutive pixels p A and p B is verified by determining that the energy required to display the pixel pair p1 and p2 is lower than the energy required to display the original pixel pair p1 and p2. Indeed, in such an embodiment, A and p B replaces two original pixels of equal duration and is less than half the size of the original.

[0103] Both frame skipping and frame averaging methods may lose some of the original signal, which may affect the spatial or temporal resolution and therefore the visual quality of the modified image. To improve the quality of the signal, spatial or temporal filtering can be used.

[0104] One refinement would be to apply the alternating complementary color process only to uniform regions, leaving the image unchanged where there is high spatial frequency (i.e., edges). In at least one embodiment, for each pair of consecutive images, the processor detects edges using a contour filter (Canny edge detector, difference of Gaussians, etc.), optionally dilates these contours, and saves the contour zones as mask (M1, M2). Two parameters are required: the dilation size and the threshold at which the mask is binarized. Mask (M1, M2) is then inverted to remove non-contour zones.

[0105]

number

[0106] and common non-contour zones

[0107]

number

[0108] If there are non-contour zones in both images of the pair,

[0109]

number

[0110] , the processor divides the two zones into colors

[0111]

number

[0112] where C1 and C2 are the colors of the two zones. The process then duplicates the average image to form a pair, and processes the colors of this pair as above to produce a single image of C 12 Alternating complementary color pairs based on (C A ,C B ) and calculate the saved contours of each image in the first image pair by (C A ,C B ) and the resulting colors C3 and C4 of the two images are defined as follows:

[0113]

number

[0114] As a result, ACC is only applied to more homogeneous regions. Based on high spatial frequencies, temporal information from (C1, C2) is retained and transferred directly to the (C3, C4) subframe.

[0115] The above enhancement techniques can be applied in three temporal modes: frequency doubling, frame skipping, and frame averaging.

[0116] As shown in Table 2, prediction of regions of interest (e.g., using eye tracking, attention modeling, meta-databases, etc.) can also be used to determine which regions should be kept intact and which should undergo ACC processing.

[0117] [Table 2]

[0118] 10 illustrates an example process for generating a lookup table of alternating complementary colors and an example process for modifying an image using the lookup table according to an embodiment, which are implemented based on the embodiments described above.

[0119] Process 1000 aims to generate a lookup table of alternating complementary colors. Step 1010 is repeated multiple times. In at least one embodiment, the iteration is performed over all possible colors in the color gamut. In at least one other embodiment, the iteration is performed only over a subsampled color space. In at least one other embodiment, the iteration is performed over all colors in a given image set. In other embodiments, other color subsets can be used. It comprises steps 1020 through 1050. In step 1020, an input color is obtained. In step 1030, a set of pairs of alternating complementary colors corresponding to the input color is determined, for example, using one of the embodiments described above. In step 1040, one of the pairs in the set is selected, for example, the pair that provides the best energy performance. In step 1050, the associations between the input color and the alternating complementary color pairs are stored in a lookup table. At the end of process 1000, the lookup table contains a set of associations between the input color and the alternating complementary color pairs.

[0120] Process 1001 aims to modify an image using a lookup table. In step 1060, an input image is obtained. In step 1070, a pixel is selected, and in step 1080, a pair of temporally alternating complementary colors corresponding to the pixel's color is obtained from the lookup table. In step 1090, the colors of pairs of temporally consecutive pixels are set to the alternating complementary color pair. Steps 1070, 1080, and 1090 are then repeated for the next pair of adjacent pixels, if one exists.

[0121] FIG. 11 illustrates two examples of deployment of the alternating complementary color process according to embodiments. In at least one embodiment, device 1101 is a display device such as that described in FIG. 1. In this case, processor 101 of device 1101 is configured to acquire an input image or video 1100, process it through the ACC process as described above, and then display it on display unit 103 of FIG. 1. In other words, processor 101 of device 1101 is configured to acquire the input image or video 1100 and determine a modified image or video to be displayed using the spatial alternating complementary colors acquired by using a lookup table (LUT), resulting in an image that provides reduced energy consumption of the display device compared to displaying the original input image. The lookup table can be acquired from a data provider via a communications network and / or from the device's internal memory. The image or video 1100, stored, for example after being captured by the input unit, can be acquired from a data provider via a communications network or from the device's internal memory. Typical examples of device 1101 are smartphones, tablets, laptops, external monitors, head-mounted displays, television sets, video projectors, computer screens, vehicles (e.g., control and / or entertainment systems of automobiles, airplanes, ships, etc.), advertising display panels, medical monitors, etc. However, any device or combination of devices providing similar functionality may be used as display device 1101 while conforming to the principles of the present disclosure.

[0122] In at least one embodiment, the device 1102 does not include a display unit, but rather prepares data for display so that another device 1103, such as a screen, can perform the display. In this case, a processor in the device performs the ACC process described herein to generate a new image or video 1110 that is identical or perceptually similar to the original video but requires less energy when displayed. This modified video is then provided to the display device 1103 for presentation to a human viewer. Examples of such devices 1102 are set-top boxes, media players, desktop computers, encoders, decoders, servers, computing grids, cloud computers, etc.

[0123] Light generation in display devices, including mobile phones and televisions, is costly. Reducing the amount of light generated is desirable, as it helps reduce the amount of energy needed to operate the display. The benefits of this are twofold: reducing the burden on the climate and extending the battery life of mobile devices. Compared to other methods that aim to reduce energy consumption for the same reasons, the proposed method, by its structure, ensures that the light emitted from each pixel is generated by a combination of two light pulses, minimizing energy consumption on average. This is intended to be more versatile and therefore more efficient than operating according to a single light pulse for a single pixel.

[0124] Although various embodiments have been described separately, the embodiments can be combined in any manner while respecting the principles of the present disclosure.

[0125] References to "one embodiment," or "one embodiment," or "one implementation," or "one implementation," as well as other variations thereof, mean that a particular feature, structure, characteristic, etc. described in connection with an embodiment is included in at least one embodiment. Thus, the appearances of the phrases "in one embodiment," or "in one embodiment," or "in one implementation," or "in one implementation" in various places throughout this specification, as well as any other variations thereof, are not necessarily all referring to the same embodiment.

[0126] Additionally, this application or its claims may refer to "determining" various information. Determining information may include, for example, one or more of estimating information, calculating information, predicting information, or retrieving information from memory.

[0127] Additionally, the application or its claims may refer to "obtaining" various information. Obtaining, like "accessing," is intended to be a broad term. Obtaining information can include, for example, one or more of receiving information, accessing information, or retrieving information (e.g., from memory or optical media storage). Furthermore, "obtaining" is typically involved in some way in operations such as storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0128] Furthermore, the terms "image" and "frame" are used interchangeably herein and are both used to refer to a set of pixels arranged, for example, in a two-dimensional array. A sequence of images or frames arranged in a temporal order is conventionally referred to as a "video."

[0129] The terms power and energy are also used interchangeably in this document and typically refer to the quantity of electricity required to display an image or a pixel of an image.

[0130] It should be noted that the use of any of " / ," "and / or," and "at least one of," for example, in the cases of "A / B," "A and / or B," and "at least one of A and B," is intended to encompass selection of only the first listed option (A), selection of only the second listed option (B), or selection of both options (A and B). As a further example, in the cases of "A, B, and / or C" and "at least one of A, B, and C," such language is intended to encompass selection of only the first listed option (A), or selection of only the second listed option (B), or selection of only the third listed option (C), or selection of only the first and second listed options (A and B), or selection of only the first and third listed options (A and C), or selection of only the second and third listed options (B and C), or selection of all three options (A and B and C). This can be expanded as many times as the number of items listed, as would be readily apparent to one of ordinary skill in this and related arts.

Claims

1. For the input color of a pixel, determining pairs of alternating complementary colors based on the input colors of the pixels; Equipped with the average color of the alternating complementary color pairs is identical to or perceptually similar to the input color; the sum of the energy consumed by displaying a pair of pixels, each pixel having a respective one of said alternating complementary colors, is less than twice the energy consumed by displaying a pixel having said input color; method.

2. selecting a color value in a color space for a first color of the pair of alternating complementary colors according to a selection criterion; determining a second color of the alternating complementary color pair based on the selected first color value and the input color; The method of claim 1 further comprising:

3. The method of claim 2 , wherein the selection criterion is based on a maximum color distance from the input color.

4. The method of claim 2 , wherein the selection criterion is that the first color is a saturated color.

5. The method of claim 2 , wherein the selection criterion is that the first color is a grayscale color.

6. The method of claim 2 , wherein the selection criterion is that the first color has the same luminance as the input color.

7. 7. The method of claim 1, wherein the color space is an XYZ color space and the second color of the alternating complementary pair is selected to be symmetrical to the first color of the alternating complementary pair with respect to the input color.

8. 7. The method of claim 1, wherein the color space is a uniform color space and the second color of the alternating complementary pair is selected to be symmetrical to the first color of the alternating complementary pair with respect to the input color.

9. repeating said step of determining said color pairs a plurality of times to generate a set of alternating complementary color pairs; selecting the pair of alternating complementary colors from the set of pairs of alternating complementary colors that consumes the least amount of energy when displayed; The method of any one of claims 1 to 8, further comprising:

10. 10. The method of claim 9, further iterating over a set of input colors comprising colors of all pixels of an input image or a subset of all pixels of the input image.

11. 10. The method of claim 9, further iterating over a set of input colors comprising all possible color values ​​in the selected color space or a subset of all possible color values ​​in the selected color space.

12. storing associations between the set of input colors and the corresponding determined pairs of alternating complementary colors; The method of claim 10 or 11, further comprising:

13. The method of claim 12 , wherein the associations are stored in a lookup table that uses the input colors as an index.

14. obtaining a first pixel of a first image of a video; - obtaining pairs of alternating complementary colors according to any one of claims 1 to 13 based on the color of the first pixel of the first image of the video; inserting a second image into the video that is temporally consecutive to the first image; setting the color of the first pixel of the first image to a first color of the pair of alternating complementary colors and setting the color of a second pixel of the second image to a second color of the pair of alternating complementary colors, the first pixel and the second pixel being at the same location within each image; A method for providing the above.

15. obtaining pairs of temporally consecutive pixels of the video; - obtaining pairs of alternating complementary colors according to any one of claims 1 to 13 based on the color of a first pixel of said pairs of temporally consecutive pixels; setting the color of the first pixel of the pair of temporally consecutive pixels to a first color of the pair of alternating complementary colors and setting the color of the second pixel of the pair of temporally consecutive pixels to a second color of the pair of alternating complementary colors; A method for providing the above.

16. obtaining pairs of temporally consecutive pixels of the video; - obtaining pairs of alternating complementary colors according to any one of claims 1 to 13 based on an average color of the colors of the pairs of temporally consecutive pixels; setting the color of a first pixel of said pair of temporally consecutive pixels to a first color of said pair of alternating complementary colors and setting the color of a second pixel of said pair of temporally consecutive pixels to a second color of said pair of alternating complementary colors; A method for providing the above.

17. 17. The method of any one of claims 14 to 16, further comprising displaying the video with modified pixels.

18. 17. The method of any one of claims 14 to 16, further comprising providing the video with modified pixels.

19. acquiring pixels of an image or video; Obtaining pairs of alternating complementary colours for the pixels of the image or video according to any one of claims 1 to 13; replacing the pixel of the image of the video with a pair of temporally consecutive pixels if the pixel is spatially contained within a region of interest selected according to a criterion, wherein a first temporally consecutive pixel has a first color of the alternating complementary color pair and a second temporally consecutive pixel has a second color of the alternating complementary color pair; A method for providing the above.

20. 20. The method of claim 19, wherein the selection criteria is based on a spatiotemporal just noticeable difference map.

21. The method of claim 19, wherein the selection criteria is based on a motion field.

22. The method of claim 19 , wherein the selection criteria is based on a saliency map.

23. The method of claim 19 , wherein the selection criteria is based on eye tracking.

24. 20. The method of claim 19, wherein the selection criteria is based on attention modeling.

25. 20. The method of claim 19, wherein the selection criteria is based on metadata.

26. For the input color of a pixel, one or more processors configured to determine pairs of alternating complementary colors based on the input colors of the pixels; the average color of said alternating complementary color pairs is the same as or perceptually similar to said input color; the sum of the energy consumed by displaying a pair of pixels, each pixel having a respective one of said alternating complementary colors, is less than twice the energy consumed by displaying a pixel having said input color; device.

27. 27. The device of claim 26, further comprising: selecting a color value in a color space for a first color of the alternating complementary color pair according to a selection criterion; and determining a second color of the alternating complementary color pair based on the selected first color value and the input color.

28. 28. The device of claim 27, wherein the selection criteria is based on a maximum color distance from the input color.

29. 28. The device of claim 27, wherein the selection criterion is that the first color be a saturated color.

30. 28. The device of claim 27, wherein the selection criterion is that the first color is a grayscale color.

31. 28. The device of claim 27, wherein the selection criterion is that the first color has the same luminance as the input color.

32. 32. A device according to any one of claims 26 to 31, wherein the colour space is the XYZ colour space and the second colour of the alternating complementary pair is selected to be symmetrical to the first colour of the alternating complementary pair with respect to the input colour.

33. 32. A device according to any one of claims 26 to 31, wherein the colour space is a uniform colour space and the second colour of the alternating complementary pair is selected to be symmetrical to the first colour of the alternating complementary pair with respect to the input colour.

34. 34. The device of claim 26, further comprising: repeating the determining of the color pairs a plurality of times to generate a set of alternating complementary color pairs; and selecting the alternating complementary color pair from the set of alternating complementary color pairs that consumes the least amount of energy when displayed.

35. 35. The device of claim 34, further iterating over a set of input colors that includes colors of all pixels of the input image or a subset of all pixels of the input image.

36. 35. The device of claim 34, further iterating over a set of input colors comprising all possible color values ​​in a selected color space or a subset of all possible color values ​​in a selected color space.

37. 37. A device according to claim 35 or 36, further comprising storing an association between the set of input colours and the corresponding determined pairs of alternating complementary colours.

38. 38. The device of claim 37, wherein the associations are stored in a lookup table that uses the input color as an index.

39. Obtaining a first pixel of a first image of the video; Obtaining pairs of alternating complementary colors according to any one of claims 26 to 38 based on the color of the first pixel of the first image of the video; inserting a second image into the video that is temporally consecutive to the first image; setting the color of the first pixel of the first image to a first color of the pair of alternating complementary colors and setting the color of a second pixel of the second image to a second color of the pair of alternating complementary colors, the first pixel and the second pixel being at the same location within each image; One or more processors configured to A device with.

40. Obtain pairs of temporally consecutive pixels of the video, Obtaining pairs of alternating complementary colors according to any one of claims 26 to 38 based on the color of a first pixel of the pairs of temporally consecutive pixels; setting the color of the first pixel of the pair of temporally consecutive pixels to the first color of the pair of alternating complementary colors and setting the color of the second pixel of the pair of temporally consecutive pixels to the second color of the pair of alternating complementary colors. One or more processors configured to A device with.

41. Take pairs of temporally consecutive pixels of the video, Obtaining pairs of alternating complementary colors according to any one of claims 26 to 38 based on an average color of the colors of the pairs of temporally consecutive pixels; setting the color of a first pixel of the pair of temporally consecutive pixels to the first color of the pair of alternating complementary colors and setting the color of a second pixel of the pair of temporally consecutive pixels to the second color of the pair of alternating complementary colors. One or more processors configured to A device with.

42. 42. A device according to any one of claims 39 to 41, further comprising displaying the video with modified pixels.

43. 42. A device according to any one of claims 39 to 41, further comprising providing the video with modified pixels.

44. Obtaining pixels of a first image of the video; Obtaining pairs of alternating complementary colors according to any one of claims 26 to 38 based on the colors of the pixels of the first image of the video; replacing the pixel of the image of the video with a pair of temporally consecutive pixels if the pixel is spatially contained within a region of interest selected according to a criterion, a first temporally consecutive pixel having a first color of the alternating complementary color pair and a second temporally consecutive pixel having a second color of the alternating complementary color pair; One or more processors configured to A device with.

45. 45. The device of claim 44, wherein the selection criteria is based on a spatiotemporal just noticeable difference map.

46. 45. The device of claim 44, wherein the selection criteria is based on a motion field.

47. 45. The device of claim 44, wherein the selection criteria is based on a saliency map.

48. 45. The device of claim 44, wherein the selection criteria is based on eye tracking.

49. 45. The device of claim 44, wherein the selection criteria is based on attention modeling.

50. 45. The device of claim 44, wherein the selection criteria is based on metadata.

51. 51. The device of any one of claims 26 to 50, wherein the device is selected from the set including a smartphone, a tablet, a laptop, an external monitor, a head mounted display, a television set, a video projector, a computer screen, a vehicle control system, a vehicle entertainment system, an advertising display panel, a medical monitor.

52. A computer program comprising program code instructions for performing the method of any one of claims 1 to 25 when executed by a processor.

53. A non-transitory computer readable medium comprising program code instructions for performing the method of any one of claims 1 to 25 when executed by a processor.

Images