Method and device for pixel color replacement in video based on complementary colors

Abstract

A method and device propose to modify colors of pixels of a video for reducing the energy required for displaying it. This is achieved by determining, for a pair of colocalized pixels of successive frames of a video, a pair of replacement complementary colors and replacing the colors of the pair of colocalized pixels by the pair of replacement complementary colors. The pair of replacement complementary colors is selected to provide energy reduction while ensuring perceptual similarity in order to provide a satisfying quality of experience. Different embodiments propose different solutions to apply the replacement process on different arrangement of frames and pixels.

Description

[0001]METHOD AND DEVICE FOR PIXEL COLOR REPLACEMENT IN VIDEO BASED ON COMPLEMENTARY COLORS CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the priority to European Application N°23306641.4 filed 29thof September 2023, which is incorporated herein by reference in its entirety. TECHNICAL FIELD The disclosure is in the field of video content distribution, and at least one embodiment relates more specifically to reduce energy consumption for displaying a video by replacing colors of a pair of colocalized pixels in successive frames of a video by complementary colors. BACKGROUND ART Reducing energy consumption of electronic devices has become a requirement not only for manufacturers of electronic devices but also to limit, as much as possible, the environmental impact and to contribute to the emergence of a sustainable display industry. The increase in display resolution from SD to HD, then to 4K and soon to 8K and beyond, as well as the introduction of high dynamic range imaging, has brought about a corresponding increase in energy requirements of display devices. This is not consistent with the global need to reduce energy consumption knowing that a huge number of devices has a display (i.e., TV, Mobile phones, tablets, etc.). Indeed, displays are the most important source of energy consumption, for consumer electronic devices, either battery-powered (e.g., smartphones, tablets, head- mounted displays, car display screens) or not (e.g., television sets, advertisement display panels). Different display technologies have been developed in the recent years. Although modern displays consume energy in a more controllable and efficient manner than older displays, they remain the most important source of energy consumption in a video chain. As far as backlight displays are concerned, their energy consumption is largely determined by the intensity of the backlight. Organic Light Emitting Diode (OLED) is one example of display technology that is finding increasingly widespread use because of numerous advantages compared to former technologies such as Thin-Film Transistor Liquid Crystal Displays (TFT-LCDs). Rather than using a uniform backlight, OLED displays, as well as mini LEDS, are composed of individual directly emissive image pixels. OLEDs power consumption is therefore highly correlated to the image content and the power consumption for a given input image can be estimated by considering the values of the displayed image pixels. It is therefore interesting to elaborate energy-aware images or videos, i.e., images or videos that will need less energy when displayed, notably on consumer electronics OLED displays. Techniques to reduce the energy consumption comprise using, for example, spatially alternating complementary colors (SACC) as described in the international patent application WO2024132682A1 or, for example, temporally alternating complementary colors (TACC) as described in the international patent application WO2024132680A1. What is common within the techniques described in both applications is that they relate to the reduction of the energy requirements of display devices with the goal to preserve the quality of experience (QoE) based on contrast, luminance, temporal smoothness, or color levels for instance by exploiting the principles of alternating complementary colors. SUMMARY Embodiments described hereafter have been designed with the foregoing in mind and introduce a process for modifying a video to lower its energy consumption while preserving the perceptual similarity. The process may be applied onto a pair of colocalized pixels in successive frames of a video and is based on replacing the original colors of the pair of colocalized pixels by a pair of replacement colors that are symmetrical with regards to an average color of the colors of the pair of pixels. The pair of replacement colors may be chosen so that the sum of the energies consumed by displaying a pair of pixels of replacement colors is lower than the sum of the energies consumed by displaying a pair of pixels of original colors. The pair of replacement colors may be chosen to be perceptually individually similar or close to the original pair of colors. This process is named Color Replacement in Videos for Energy Reduction, hereafter abbreviated CRV. A first aspect is directed to a method comprising obtaining a first pair of colors of a pair of pixels, wherein a first pixel of the pair of pixels is comprised in a first image of a video and a second pixel of the pair of pixels is comprised in a second image of the video, and wherein the first pixel and the second pixel are colocalized in the images, determining an average color between colors of the first pair of colors, determining a second pair of colors, wherein the colors of the second pair of colors are symmetrical with regards to the average color and for which a sum of energies consumed by displaying a pair of pixels having the second pair of colors is lower than a sum of energies consumed by displaying a pair of pixels having the first pair of colors, and replacing the first pair of colors of the pair of pixels by the second pair of replacement colors. A second aspect is directed to a method comprising obtaining a visual content, iterating over successive frames of the visual content, iterating over pairs of pixels colocalized in the successive frames, and for an iterated pair of pixels: determining a second pair of colors for the iterated pair of pixels according to the first aspect, and modifying the visual content by replacing the colors of the iterated pair of pixels by the second pair of colors, and providing the modified visual content. A third aspect is directed to an apparatus comprising a processor configured to obtain a first pair of colors of a pair of pixels, wherein a first pixel of the pair of pixels is comprised in a first image of a video and a second pixel of the pair of pixels is comprised in a second image of the video, and wherein the first pixel and the second pixel are colocalized in the images, to determine an average color between colors of the first pair of colors, to determine a second pair of colors, wherein the colors of the second pair of colors are symmetrical with regards to the average color and for which a sum of energies consumed by displaying a pair of pixels having the second pair of colors is lower than a sum of energies consumed by displaying a pair of pixels having the first pair of colors, and to replace the first pair of colors of the pair of pixels by the second pair of replacement colors. A fourth aspect is directed to an apparatus comprising a processor configured to obtain a visual content, to iterate over successive frames of the visual content, to iterate over pairs of pixels colocalized in the successive frames, and for an iterated pair of pixels: to determine a second pair of colors for the iterated pair of pixels according to the first aspect, and to modify the visual content by replacing the colors of the iterated pair of pixels by the second pair of colors, and to provide the modified visual content. A fifth aspect is directed to a computer program including instructions, which, when executed by a computer, cause the computer to carry out the method according to the first or second aspect. A sixth aspect is directed to a non-transitory computer readable medium storing executable program instructions to cause a computer executing the instructions to perform a method according to the first or second aspect. The above presents a simplified summary of the subject matter to provide a basic understanding of some aspects of the present disclosure. This summary is not an extensive overview of the subject matter. It is not intended to identify key / critical elements of the embodiments or to delineate the scope of the subject matter. Its sole purpose is to present some concepts of the subject matter in a simplified form as a prelude to the more detailed description provided below. BRIEF SUMMARY OF THE DRAWINGS The present disclosure may be better understood by consideration of the detailed description below in conjunction with the accompanying figures in which: Figure 1 illustrates a block diagram of an example of architecture comprising a device in which various aspects and embodiments are implemented. Figure 2 illustrates two examples of deployment for a CRV process according to embodiments. Figure 3 illustrates a first example of process for modifying a video to lower its energy consumption according to embodiments. Figure 4 illustrates numerical examples of replacement complementary colors for a pair of pixels using a CRV process according to embodiments. Figure 5 illustrates a second example of CRV process according to embodiments. Figure 6 illustrates an example of replacement complementary colors according to embodiments. Figure 7 illustrates a third example of CRV process according to embodiments. Figure 8 illustrates an example of replacement complementary colors selection according to embodiments based on the third example of CRV process. Figure 9 illustrates a first example of application of the CRV process on successive images according to embodiments. Figure 10 illustrates a second example of application of the CRV process on successive images according to embodiments. Figure 11 illustrates a third example of application of the CRV process on successive images according to embodiments. Figure 12 illustrates an example of replacement complementary colors selection according to embodiments for which the energy gain is not maximized. Figure 13 illustrates an example of process for modifying a video using the CRV process according to embodiments. Figure 14 illustrates an example of process for generating a correspondence table for replacement complementary colors according to embodiments. It should be understood that the drawings are for purposes of illustrating examples of various aspects, features, and embodiments in accordance with the present disclosure and are not necessarily the only possible configurations. Throughout the various figures, like reference designators refer to the same or similar features. DETAILED DESCRIPTION The SACC and TACC techniques can be applied in a frame-by-frame manner on successive images of a video. However, such a frame-by-frame process will modify one pixel color by two other complementary colors. Thus, it does not consider the fact that, in a video, at each instant of time and at each pixel location, a different color might be displayed, contrary to what happens for static images. Indeed, when displaying video, the frame-by-frame images may be subject to camera or subject to movement that results in displaying successively (i.e., at a same pixel location in two successive frames) pixels of very different colors. Furthermore, the techniques described in the applications mentioned above aim at a maximal gain without any constraint on the alternating replacement complementary colors, except that they reduce the overall energy consumption and that, once fused by the viewer’s visual system, they convey the exact same perceptual color. As a result, the replacement of one pixel color by two other colors may lead to choices of colors that can be perceptually very different from the original color. Embodiments described herein propose to replace colors of a pair of pixels in successive frames of a video sequence and located at the same spatial position in the frames by a replacement pair of colors that are perceptually individually similar to the original pair of colors (for example as close as possible, in terms of distance in the color space of the colors), and which will consume less energy when being displayed on screens. In this context, perceptual similarity may be defined as the similarity in the low-level physical properties of a stimuli (contrast, luminance) using an objective measure (for example a set of XYZ measures performed using a chromameter or the set of corresponding Lab values) that parallels the response profile of neurons in the visual cortex. Figure 1 illustrates a block diagram of an example of architecture comprising a device in which various aspects and embodiments are implemented. In the depicted environment, a user interacts with the device 100, for example a television, that is connected to a server 180 for example operated by a content provider. The server 180 delivers multimedia content 190 such as video streams based on images. In a video distribution system, multiple devices 100, 1x0 are interacting with multiple content providers and corresponding servers 180, 18x delivering multiple multimedia content 190, 19x. A single content provider may use a plurality of servers. The devices exchange data through a communication network 150. The communication network 150 preferably uses a communication standard to provide interoperability between content provider and devices. Such communication standard may be wireless, such as cellular (e.g., LTE) communications, Wi-Fi communications, and the like, to ensure the mobility of the device. Cable, satellite, or terrestrial digital television broadcast communication may also be used for the communication network 150 as well as broadband television communications. Such digital television standards may on based on well-established standards like DVB, ATSC, or the like. General purpose network standards may also be used, for example based on Ethernet. The device 100 comprises a processor 101. The processor 101 may be a general- purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific Integrated Circuits (ASICs), Field Programmable Gate Array (FPGAs) circuits, any other type of integrated circuit (IC), a state machine, and the like. The processor may perform data processing such as the CRV process 212, 222 of figure 2, the process 300 of figure 3, the process 500 of figure 5, the process 700 of figure 7, the process 1300 of figure 13 or the process 1400 of figure 14. The processor 101 may be coupled to an input unit 102 configured to convey user interactions. Multiple types of inputs and modalities can be used for that purpose. Physical keypad or a touch sensitive surface are typical examples of input adapted to this usage although voice control could also be used. In addition, the input unit may also comprise a digital camera able to capture still pictures or video in two dimensions or a more complex sensor able to determine the depth information in addition to the picture or video and thus able to capture a complete 3D representation. The processor 101 may be coupled to a display unit 103 configured to output visual data to be displayed on a screen. Multiple types of displays can be used for that purpose such as a liquid crystal display (LCD) or organic light-emitting diode (OLED) display unit. The processor 101 may also be coupled to an audio unit 104 configured to render sound data to be converted into audio waves through an adapted transducer such as a loudspeaker for example. The processor 101 may be coupled to a communication interface 105 configured to exchange data with external devices. The communication preferably uses a wireless communication standard to provide mobility of the device, such as cellular (e.g., LTE) communications, Wi-Fi communications, and the like. The processor 101 may access information from, and store data in, the memory 106, that may comprise multiple types of memory including random access memory (RAM), read- only memory (ROM), a hard disk, a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, any other type of memory storage device. In embodiments, the processor 101 may access information from, and store data in, memory that is not physically located on the device, such as on a server, a home computer, or another device. The processor 101 is configured to execute an image energy reduction algorithm that modifies an input image into an image that requires less energy when being used, for example displayed, in comparison to using the input image. Different techniques have been disclosed to provide such feature. The processor 101 may receive power from the power source 108 and may be configured to distribute and / or control the power to the other components in the device 100. The power source may be any suitable device for powering the device. As examples, the power source may include one or more dry cell batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li-ion), and the like), solar cells, fuel cells, and the like. While the figure depicts the processor 101 and the other elements 102 to 108 as separate components, it will be appreciated that these elements may be integrated together in an electronic package or chip. It will be appreciated that the device 100 may include any sub- combination of the elements described herein while remaining consistent with the embodiments described hereafter. The processor 101 may further be coupled to other peripherals or units not depicted in figure 1 which may include one or more software and / or hardware modules that provide additional features, functionality and / or wired or wireless connectivity. For example, the peripherals may include a universal serial bus (USB) port, a vibration device, a television transceiver, a hands-free headset, a Bluetooth® module, a frequency modulated (FM) radio unit, a digital music player, a media player, a video game player module, an Internet browser, and the like. For example, the processor 101 may be coupled to a localization unit configured to localize the device within its environment. The localization unit may integrate a GPS chipset providing longitude and latitude position regarding the current location of the device but also other motion sensors such as an accelerometer and / or an e-compass that provide localization services. At least one example of an embodiment can involve a device including an apparatus as described herein and at least one of (i) an antenna configured to receive a signal, the signal including data representative of the image information, (ii) a band limiter configured to limit the received signal to a band of frequencies that includes the data representative of the image information, and (iii) a display configured to display an image from the image information. At least one example of an embodiment can involve a device as described herein, wherein the device comprises one of a television, a television signal receiver, a set-top box, a gateway device, a mobile device, a cell phone, a tablet, a computer, a laptop, or other electronic device. Figure 2 illustrates two examples of deployment for a CRV process according to embodiments. In at least one embodiment, the device 100 of figure 1 is a display device 210. In other words, the display device 210 comprises a display unit and a processor configured to implement the CRV process 212 according to embodiments as described herein to generate a modified video 214 from an input video 201, the modified video being displayed on the display unit of the display device. The input video 201 may be obtained from a data provider through a communication network, from an internal memory of the device, stored for example after being captured by an input unit. Typical examples of devices 210 are smartphones, tablets, laptops, monitors, head-mounted displays, television sets, video projectors, computer screens, vehicles (e.g., control and / or entertainment systems for cars, planes, boats, etc.), advertisement display panels, medical monitors, etc. However, any device or composition of devices that provides similar functionalities can be used as display device 210 while still conforming with the principles of the disclosure. In at least one embodiment, the principles described herein are implemented in a device 220 that does not include a display unit but prepares data for being displayed by another device 230 that includes a display unit. In this case, the processor of the device 220 implements the CRV process 222 according to embodiments as described herein to generate a modified video 224 from an input video 201 and provides the modified video to the display device 230. Example of such devices 220 are set top boxes, media players, desktop computers, encoders, decoders, servers, computing grids, cloud computers, etc. In both embodiments described in relation with figure 2, the CRV process may use a correspondence table 216, or 226 that may be obtained from a data provider through a communication network and / or from an internal memory of the device. Figure 3 illustrates a first example of process for modifying a video to lower its energy consumption according to embodiments. The CRV process 300 is for example implemented by a processor 101 of the device 100 of figure 1. In step 310, the processor obtains a pair of pixels of a video to be modified. The first pixel is obtained from a first frame and the second pixel is obtain from a second frame. The two pixels are colocalized in the sense that these pixels occupy the same location within the two frames. The two frames may be successive frames. In step 320, the processor determines a pair of replacement colors symmetrical with regards to an average color between the color of the first pixel and the color of the second pixel. The pair of replacement colors is determined based on a distance with the colors of the two pixels so that to lower (for example: to minimize) the sum of the energies consumed by displaying the two pixels while ensuring that the replacement complementary color of each pixel is similar to the original color of each pixel respectively. In step 330, the processor replaces, in the video, the colors of the pair of pixels by the determined pair of replacement colors. The CRV process 300 may be iterated on all pairs of pixels of all pairs of frames of the video. For some pair of colors, there is no replacement pair that lowers the sum of the energies so that the pair of colors is not replaced (for example: a pair of black pixels) and in this case the step 330 is not executed. Figure 4 illustrates numerical examples of replacement complementary colors for a pair of pixels using a CRV process according to embodiments. This example uses very simplified images of sizes four by one, in other words, a single line of four pixels. The image 400 shows the 4 pixels (401, 402, 403, 404) of a first frame and the image 410 shows the 4 pixels (411, 412, 413, 414) of a second frame. The second frame is successive to the first frame. In at least one embodiment, a successive frame refers to the frame directly consecutive to the first frame. In at least one embodiment, more than two successive frames may be used. In each pixel, R, G, and B values are shown vertically. Pixels 401 and 411 are colocalized. The CRV process, when applied to the couple of images 400, 410 may modify the colors of the pixels as described in embodiments herein, which results in the couple of images 420, 430. For example, the first pixel 401 of the first image (RGB values = 111, 109, 72) looks like an ‘olive green’ color and the first pixel 411 of the second image (RGB values = 99, 127, 171) looks like a ‘petrol blue’ color. When modified according to the CRV process, these pixels are replaced respectively with pixels 421 and 431. The first pixel 421 of the first image (RGB values = 88, 108, 85) looks like a greener version of the ‘olive green’ color and the first pixel 431 of the second image (RGB values = 110, 122, 170) looks like more purple than the ‘petrol blue’ color. Figure 5 illustrates a second example of CRV process according to embodiments. The CRV process 500 is for example implemented by a processor 101 of the device 100 of figure 1 and corresponds for example to the step 430 of the process 400 of figure 4. The process is operated on a couple of RGB triplets ^^^^1^^^^1^^^^1and ^^^^2^^^^2^^^^2, ^^^^1^^^^1^^^^1being the color of a first pixel in a first image ^^^^ ^^^^1and ^^^^2^^^^2^^^^2being the color of a second pixel in a second image ^^^^ ^^^^2of a video, ^^^^ ^^^^2and ^^^^ ^^^^2being images of successive frames (or intermediate images previously generated) and the first pixel and the second pixel being colocalized (i.e., at the same position in the images). A color space{∁}is selected for the operations. This color space is a color space with color transforms and inverse color transforms, for example the XYZ color space as used in the description of the process. In such color space, forward color transforms ℱ ^^^^( ^^^^ ^^^^ ^^^^) act on RGB to compute (CIE)XYZ, RGB ^XYZ and inverse color transforms ℐ ^^^^( ^^^^ ^^^^ ^^^^) apply inverse operations XYZ ^RGB. XYZ is chosen as it reduces the computation load to find the replacement complementary colors. A uniform color space such as CIELab, IPT or OKLab can be also chosen. In such case, the operation described below has to be adapted to the selected color space. Uniform color spaces (UCSs) are built such that the same geometrical distance anywhere in the color space reflects the same amount of perceived color difference. In this case, most often forward color transforms ℱ ^^^^( ^^^^ ^^^^ ^^^^) act on RGB to compute (CIE)XYZ and then Lab or equivalent visual coordinates RGB ^XYZ ^Lab. Lab can also be computed directly from RGB: RGB ^Lab. Inverse transforms act similarly: RGB = ℐ ^^^^( ^^^^ ^^^^ ^^^^). In step 510, the processor determines the energy consumption ^^^^( ^^^^1^^^^1^^^^1) and ^^^^( ^^^^2^^^^2^^^^2) for the input color triplets of respectively the first pixel in the first image ^^^^ ^^^^1and the colocalized pixel in the second image ^^^^ ^^^^2of the video according to a selected color power consumption model. The power consumption model is display dependent. For OLED displays, the power consumption model can follow the color model, including the RGBW case where a white LED supports the RGB LEDs for each physical pixel. A color model for a RGBW display is given by Murdoch et al. in “Perfecting the color reproduction of RGBW OLED” proc.30th International Congress of Imaging Science. This model can be extended with adequate parameters to represent per pixel power. In step 515, the processor determines the two color points ^^^^1and ^^^^2in { ^^^^} being the response to ^^^^1^^^^1^^^^1and ^^^^2^^^^2^^^^2triplets, within the color space {∁} and apply color transform to obtain the coordinates ^^^^1^^^1^ ^^^^1of ^^^^1and ^^^^2^^^2^ ^^^^2of ^^^^2as follows: ^^^^1^^^1^ ^^^^1= ℱ ^^^^( ^^^^1^^^^1^^^^1) ^^^^2^^^2^ ^^^^2= ℱ ^^^^( ^^^^2^^^^2^^^^2) In step 520, the processor samples the color space{∁}within the gamut{^^^^}todetermine a set of couples of candidate colors{^^^^3, ^^^^4}such that the ^^^^3^^^3^ ^^^^3coordinates of ^^^^3and ^^^^4^^^4^ ^^^^4coordinates of ^^^^4verify the following equations: In other words, ^^^^4is symmetrical to ^^^^3with regards to the middle point between and ^^^^2(average The operations of remaining steps 525 to 530 are then iterated on the set of couples of candidate colors { ^^^^3, ^^^^4}. In step 525, the processor applies the inverse transform to ^^^^3and ^^^^4to obtain the corresponding RGB values: ^^^^3^^^^3^^^^3= ℐ ^^^^( ^^^^3^^^3^ ^^^^3) ^^^^4^^^^4^^^^4= ℐ ^^^^( ^^^^4^^^4^ ^^^^4) In step 530, the processor evaluates the energy consumption for each ^^^^ ^^^^ ^^^^ triplet of the set of couples of candidate colors { ^^^^3, ^^^^4} to obtain the list of corresponding energy consumptions { ^^^^( ^^^^3^^^^3^^^^3) + ^^^^( ^^^^4^^^^4^^^^4)}. In step 535, the processor identifies the minimum power ^^^^ ^^^^ ^^^^ ^^^^ in the set{ ^^^^( ^^^^3 ^^^^3 ^^^^3) + ^^^^( ^^^^4 ^^^^4 ^^^^4)}: and keeps the corresponding color pairs{^^^^3, ^^^^4}^^^^ ^^^^ ^^^^. This corresponds to identifyingthe locus{ℒ}of the minimum power in the series, i.e., the set of color pairs{^^^^3, ^^^^4}^^^^ ^^^^ ^^^^ thathave a power of ^^^^ ^^^^ ^^^^ ^^^^:{^^^^3, ^^^^4 } ^^^^ ^^^^ ^^^^ = ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ { ^^^^( ^^^^3 ^^^^3 ^^^^3) + ^^^^( ^^^^4 ^^^^4 ^^^^4)} In step 540, the processor compares the energy consumption of ^^^^^^^^ ^^^^ ^^^^to the energy consumption of the couple of input color triplets + ^^^^( ^^^^2^^^^2^^^^2). If ^^^^( ^^^^1^^^^1^^^^1) + ^^^^( ^^^^2^^^^2^^^^2) ≤ ^^^^^^^^ ^^^^ ^^^^, then replacing the couple of colors ^^^^1, ^^^^2by the couple of colors ^^^^3, ^^^^4with the minimal energy consumption does not bring any gain in terms of energy consumption. In this case, in step 545, the couple of colors ^^^^1, ^^^^2is not replaced. If ^^^^( ^^^^1^^^^1^^^^1) + ^^^^( ^^^^2^^^^2^^^^2) > ^^^^^^^^ ^^^^ ^^^^, then replacing the couple of colors ^^^^1, ^^^^2by the couple of colors ^^^^3, ^^^^4with the minimal energy consumption brings some gain in terms of energy consumption. In this case, in step 550, the couple of colors ^^^^3, ^^^^4that minimizes thedistance with ^^^^1, ^^^^2 is selected:(^^^^3, ^^^^4) ^^^^ ^^^^ ^^^^ ^^^^ = ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^( ^^^^ ^^^^ ^^^^ ^^^^( ^^^^1, ^^^^3))As ( ^^^^3, ^^^^4) are symmetric with regard to ( ^^^^1+ ^^^^2) / 2 , minimizing the distance between the two pairs is equivalent to minimizing the distance between one color of the first pair and its corresponding color in the second pair. In the case where multiple pairs of colors ^^^^3, ^^^^4are found, the first pair of a list of multiple pairs can be chosen. In another embodiment, a random pair is chosen. In another embodiment, a pair is chosen according to another criteria, for example, ^^^^3has the maximum luminance. In step 555, the processor replaces the couple of colors ( ^^^^1, ^^^^2) by the selected couple of colors ( ^^^^3, ^^^^4)^^^^ ^^^^ ^^^^ ^^^^. In the above embodiment, if the choice of CRV colors is done in the XYZ color space and if the power is linear with the LEDs intensity and also if the targeted display is an RGBW display, an optional criterion and step can be added before step 510 to filter the triplets ^^^^1^^^^1^^^^1and ^^^^2^^^^2^^^^2for which no CRV processing is possible. In this optional step, it will be determined whether ^^^^1^^^^1^^^^1and ^^^^2^^^^2^^^^2belong to different triangles defined by three out of the four ( ^^^^, ^^^^, ^^^^, ^^^^) primaries in the gamut as shown in the figure 6. Indeed, if ^^^^1^^^^1^^^^1and ^^^^2^^^^2^^^^2belong to the same triangle and if the power is linear with the LEDs intensity, no power gain is possible. Adding such a criterion will save the processing load for colors for which it is impossible to find replacement complementary colors. However, when the search for CRV colors is done in a uniform color space such as Lab, it is possible to find replacement complementary colors consuming less power even if ^^^^1^^^^1^^^^1 and ^^^^2^^^^2^^^^2are located in the same sub triangle in the gamut { ^^^^}. In this case, this optional step of choosing ^^^^1^^^^1^^^^1and ^^^^2^^^^2^^^^2in different triangles is not necessary. Figure 6 illustrates an example of replacement complementary colors according to embodiments. It shows the color space of an RGBW display. In addition to the conventional red, green, blue color LEDs, such displays also comprise white LEDs. This allows finer adjustments of the colors and lowers energy consumption. The figure shows the three different sub-triangles in gamut{^^^^}, and, for an example pair of input colors ( ^^^^1, ^^^^2), the average color determined based on the pair of input colors ( ^^^^1, ^^^^2), the locus of minimum power for the color pair ( ^^^^1, ^^^^2) and the couple of colors ^^^^3, ^^^^4that minimizes the distance with ^^^^1, ^^^^2and have the minimum power, i.e., are belonging to the locus of minimum power. Figure 7 illustrates a third example of CRV process according to embodiments. The CRV process 700 is for example implemented by a processor 101 of the device 100 of figure 1 and corresponds for example to the step 430 of the process 400 of figure 4. The process is operated on a couple of RGB triplets ^^^^1^^^^1^^^^1and ^^^^2^^^^2^^^^2, ^^^^1^^^^1^^^^1being the color of a first pixel in a first image ^^^^ ^^^^1and ^^^^2^^^^2^^^^2being the color of a second pixel in a second image ^^^^ ^^^^2of a video, ^^^^ ^^^^1and ^^^^ ^^^^2being images of successive frames and the first pixel and the second pixel being colocalized (i.e., at the same position in the images). This third example of CRV process is a fast version that provides the video replacement complementary colors with a reduced set of operations. It is based on a geometric method to identify replacement complementary colors that uses the XYZ color space. In other color spaces (e.g., OKLab), projections are much more complex. The operations of the process are illustrated in figure 8. In step 710, the processor determines the two color points ^^^^1and ^^^^2in { ^^^^} being the response to ^^^^1^^^^1^^^^1and ^^^^2^^^^2^^^^2triplets, within the color space {∁} and apply color transform to obtain the coordinates ^^^^1^^^1^ ^^^^1of ^^^^1and ^^^^2^^^2^ ^^^^2of ^^^^2as follows: ^^^^1^^^1^ ^^^^1= ℱ ^^^^( ^^^^1^^^^1^^^^1) ^^^^2^^^2^ ^^^^2= ℱ ^^^^( ^^^^2^^^^2^^^^2) In step 720, the midpoint ^^^^ is computed from ^^^^1, ^^^^2: In step 730, the sub-triangle where ^^^^ lies, is identified. In figure 8, this corresponds to the BGW sub-triangle. In step 740, a ( ^^^^, ^^^^, ^^^^, ^^^^) parallelogram is defined in the sub-triangle where ^^^^ lies. This parallelogram is defined as follows: - Q is set to be the White point Q= ^^^^ - ^^^^ is the symmetric point of Q with regard to ^^^^: ^^^^ = 2. ^^^^ − ^^^^ - the two primary lines ^^^^1 and ^^^^2 of the identified sub-triangle are identified (on Figure 5-9 the Green primary line ^^^^ ^^^^ and the Blue primary line ^^^^ ^^^^), - ^^^^ is built as the projection of ^^^^ parallel to ^^^^1 on ^^^^2 - ^^^^ is built as the projection of ^^^^ parallel to ^^^^2 on ^^^^1 The resulting parallelogram ( ^^^^, ^^^^, ^^^^, ^^^^) is the locus of minimum power. In step 750, ^^^^3, ^^^^4are determined as being the orthogonal projections of ^^^^1, ^^^^2respectively on the closest ( ^^^^, ^^^^, ^^^^, ^^^^) parallelogram borders. The obtained pair of colors ^^^^3, ^^^^4are considered as the best compromise between energy reduction and perceptual quality since these colors are the closest to ^^^^1, ^^^^2with minimum power consumption. Figure 8 illustrates an example of replacement complementary colors selection according to embodiments based on the third example of CRV process. It allows to illustrate the steps of the process 700 of figure 7. It shows the ^^^^1, ^^^^2colors mapped onto the BGW sub- triangle, the O midpoint between these colors, the ( ^^^^, ^^^^, ^^^^, ^^^^) parallelogram, the two primary lines ^^^^1 and ^^^^2 of the BGW sub-triangle, and the ^^^^3, ^^^^4colors being the orthogonal projections of ^^^^1, ^^^^2respectively on the closest ( ^^^^, ^^^^, ^^^^, ^^^^) parallelogram borders. The CRV process may be repeated to replace colors of a complete video. This may be done by applying the process on a plurality of successive images according to at least three different variant embodiments described below, illustrated in figures 9 to 11. In these figures, the circle represents one of the processes 300, 500 or 700 of respectively figures 3, 5, or 7 that determines a pair of replacement complementary colors from a pair of input colors. Figure 9 illustrates a first example of application of the CRV process on successive images according to embodiments. The vertical direction represents the temporal dimension. The rectangles in the column on the left side represent the successive images of the video to be modified, noted Im1, Im2, Im3, Im4, Im5, Im6. For example, Im1 is the first image of the video, Im2 is the second image of the video, and so on. The rectangles in the column on the right side represent the successive images as modified by the CRV process and are noted Im1’, Im2’, Im3’, Im4’, Im5’, Im6’. In this first example, the successive images are handled two by two. A first pair Im1 and Im2 is used as input for the CRV process, which results into the modified images Im1’ and Im2’. This simple variant embodiment uses only half the number of processes with regards to the number of images. Figure 10 illustrates a second example of application of the CRV process on successive images according to embodiments. The vertical direction represents the temporal dimension. The rectangles in the column on the left side represent the successive images of the video to be modified, noted Im1, Im2, Im3, Im4, Im5, Im6. The rectangles in the column on the right side represent the successive images as modified by the CRV process and are noted Im1’, Im2’, Im3’, Im4’, Im5’, Im6’. A first pair Im1 and Im2 is used as input for the CRV process, which results into the modified images Im1’ and Im2”. Im1’ is used as the final modified image Im1’. Im2” is used as an input for the CRV process, together with the next image Im3 which results into the modified images Im2’ used as the final modified image, and Im3” used as an input for the CRV process, together with the next image, and so on. Therefore, with this method, a modified image will depend on the previous image, the current image and the next image. This allows to better tie the images together. Furthermore, since the color replacement process is applied for each image, this method provides an increased energy reduction in comparison with the first method. Figure 11 illustrates a third example of application of the CRV process on successive images according to embodiments. The vertical direction represents the temporal dimension. The rectangles in the column on the left side represent the successive images of the video to be modified, noted Im1, Im2, Im3, Im4, Im5, Im6. The rectangles in the column on the right side represent the successive images as modified by the CRV process and are noted Im1”, Im2”, Im3”, Im4”, Im5”, Im6”. The intermediate column represents intermediate images noted Im1’, Im2’, Im3’, Im4’, Im5’, Im6’ that are used to generate the modified images. A first pair Im1 and Im2 is used as input for the CRV process, which results into the intermediate images Im1’ and Im2’. A second pair Im3 and Im4 is used as input for the CRV process, which results into the intermediate images Im3’ and Im4’. In a further step, the intermediate images Im2’ and Im3’ are used as input for the CRV process which results into the modified images Im2’’ and Im3’. Im1” (which equals Im1’), Im2’’ and Im3’’ are used as the final modified images to replace respectively Im1, Im2 and Im3. The replacement of Im4 will use an intermediate result from the next pair of input images Im5 and Im6, and so on. This third example allows to better tie the images together. Furthermore, since the color replacement process is applied for each image, this method provides an increased energy reduction in comparison with the first method. Each embodiment described in figures 9, 10 or 11 can be cascaded to further improve the energy reduction and the color similarity by processing the resulting images Imi’ one or more times. Figure 12 illustrates an example of replacement complementary colors selection according to embodiments for which the energy gain is not maximized. In this example, the choice of ( ^^^^3, ^^^^4) is done to further emphasize a quality constraint defined by a distance between ^^^^1and ^^^^3at the expense of the energy gain. In such example, the colors are not projected onto the lines of the parallelogram but are chosen closer to the original input color. In at least one embodiment, this distance may be adjusted in order to introduce a tuning mechanism in order to select a desired compromise between quality and energy reduction. For example, the adjustment may be under control of the viewer through graphical elements allowing to select a strength of the modification performed on the input video. The adjustment may also be done by the device, for example according to battery charge level in the case of mobile devices. Figure 13 illustrates an example of process for modifying a video using the CRV process according to embodiments. The process 1300 is for example implemented by a processor 101 of the device 100 of figure 1. In step 1310, the processor obtains the visual content, or a part of the visual content, for example a video or a sequence of frames (i.e., images comprising a set of pixels). In step 1330, the CRV process is applied to the visual content. The step 1330 is iterated over a plurality of pixels for successive pairs of frames of the video. In step 1332, the processor obtains a pixel p1 of color ^^^^1and a pixel p2 of color ^^^^2. In step 1334, the processor determines a pair of replacement complementary colors ^^^^3, ^^^^4with reduced energy consumption according to embodiments described above for the colors ^^^^1, ^^^^2. In step 1336, the processor replaces the colors of pixels p1 and p2 by the replacement complementary colors ^^^^3, ^^^^4as determined in step 1334, using one of the processes 300, 500, or 700 described above. In step 1350, the modified visual content is provided, for example displayed on a screen when the process 1300 is executed by a device 210 of figure 2 or transmitted on a communication network when the process 1300 is executed by a device 220 of figure 2. In at least an embodiment, the step 1334 is based on the process 300 of figure 3. In at least an embodiment, the step 1334 is based on the process 500 of figure 5. In at least an embodiment, the step 1334 is based on the process 700 of figure 7. In at least an embodiment, the step 1334 is based on a correspondence table generated by the process 1400 of figure 14 described below. Such correspondence table stores an association between a pair of input colors and a pair of replacement complementary colors. The correspondence table may use the pair of input colors as an index, so that accessing the correspondence table with a pair of input colors will provide the pair of replacement complementary colors. In at least one embodiment, the correspondence table provides several pairs of replacement complementary color corresponding to different levels of color similarity distance. In at least one embodiment, only a subset of the whole image is processed instead of processing the whole image. For that purpose, an additional step is added before the step 932 to check that the pixel ^^^^ spatially belongs to a subset of the image that is to be processed, for example belonging to a region in the image having the highest ability to mask artefacts. Such a region or mask can be given by for example a spatio-temporal just noticeable difference (JND) map, a motion field, a saliency map, etc. If the pair of pixels ^^^^1, ^^^^2 does not belong to this region or mask, no color pair replacement will be considered for this pixel and thus the process steps 1332 to 1336 will not be performed for this pair of pixels. Other considerations could trigger the exclusion of subsets of the pixels, for example the overlay of graphics over the image. Figure 14 illustrates an example of process for generating a correspondence table for replacement complementary colors according to embodiments. The process 1400 is for example implemented by a processor 101 of the device 100 of figure 1. The step 1410 may be iterated multiple times. In at least one embodiment, the iterations are done over all possible colors in the color gamut. In at least one other embodiment, the iterations are done over a sub- sampled color space only. In at least one other embodiment, the iterations are done over all unique colors of a predetermined set of videos or a video or a predetermined set of images or an image or regions of an image. Each iteration comprises the steps 1411, 1412 and 1413. In step 1411, the processor obtains a pair of input colors ^^^^1, ^^^^2. In step 1412, a pair of replacement complementary colors ^^^^3, ^^^^4corresponding to the pair of input colors is determined, for example using the process 300 of figure 3 or the process 500 of figure 5 or the process 700 of figure 7. In step 1013, the association between the pair of input colors ^^^^1, ^^^^2and the selected pair of replacement complementary colors ^^^^3, ^^^^4is stored in a correspondence table. At the end of process 1400, the correspondence table comprises a set of associations between pairs of input colors and pairs of replacement complementary colors. In figure 2, the elements 216 and 226 are correspondence tables that may be obtained according to the process 1400. The proposed CRV processes leverage the use of two colors at a time to search for two other replacement complementary colors. This allows to find potentially higher gain in terms of power. It also allows to work in a linear color space such as XYZ which considerably lower the processing load of the search of replacement complementary colors. No minimization is required; a direct computation of the locus of minimal power and of the color pair that is closer to the original colors is possible. CRV is more efficient than TACC or SACC in the sense that it provides higher energy gains on moving zones of pictures. On statics zones, TACC and SACC provide the same results than CRV. It also allows to find replacement complementary colors that are close to the original colors, thus avoiding replacing frames in video by frames with completely different colors. Compared to TACC which inherently introduce flicker, CRV does not introduce flicker, which also means that the post processing operations required to reduce flicker in TACC (equiluminance, swap of pixels) are no longer required. Although some parts of the description refer to video, the embodiments are not restricted to video and apply to any type of visual media content such as stereoscopic (3D) videos, 360° immersive videos, based on the same principles as described above. In general, one or more other examples of embodiments can also provide a computer readable storage medium, e.g., a non-volatile computer readable storage medium, having stored thereon instructions for encoding or decoding picture information such as video data according to the methods or the apparatus described herein. One or more embodiments can also provide a computer readable storage medium having stored thereon a bitstream generated according to methods or apparatus described herein. One or more embodiments can also provide methods and apparatus for transmitting or receiving a bitstream or signal generated according to methods or apparatus described herein. Many of the examples of embodiments described herein are described with specificity and, at least to show the individual characteristics, are often described in a manner that may sound limiting. However, this is for purposes of clarity in description, and does not limit the application or scope of those aspects. Indeed, all the different aspects can be combined and interchanged to provide further aspects. Moreover, the embodiments, features, etc. can be combined and interchanged with others described in earlier filings as well. Note that the syntax elements as used herein are descriptive terms. As such, they do not preclude the use of other syntax element names. When a figure is presented as a flow diagram, it also provides a block diagram of a corresponding apparatus. Similarly, when a figure is presented as a block diagram, it also provides a flow diagram of a corresponding method / process. In general, the examples of embodiments, implementations, features, etc., described herein can be implemented in, for example, a method or a process, an apparatus, a software program, a data stream, or a signal. Even if only discussed in the context of a single form of implementation (for example, discussed only as a method), the implementation of features discussed can also be implemented in other forms (for example, an apparatus or program). An apparatus can be implemented in, for example, appropriate hardware, software, and firmware. One or more examples of methods can be implemented in, for example, a processor, which refers to processing devices in general, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, such as, for example, computers, cell phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate communication of information between end-users. Also, use of the term "processor" herein is intended to broadly encompass various configurations of one processor or more than one processor. Reference to “one embodiment” or “an embodiment” or “one implementation” or “an implementation”, as well as other variations thereof, means that a particular feature, structure, characteristic, and so forth described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of the phrase “in one embodiment” or “in an embodiment” or “in one implementation” or “in an implementation”, as well any other variations, appearing in various places throughout this application are not necessarily all referring to the same embodiment. Additionally, this application may refer to “determining” various pieces of information. Determining the information can include one or more of, for example, estimating the information, calculating the information, predicting the information, or retrieving the information from memory. Further, this application may refer to “accessing” various pieces of information. Accessing the information can include one or more of, for example, receiving the information, retrieving the information (for example, from memory), storing the information, moving the information, copying the information, calculating the information, determining the information, predicting the information, or estimating the information. Additionally, this application may refer to “receiving” various pieces of information. Receiving is, as with “accessing”, intended to be a broad term. Receiving the information can include one or more of, for example, accessing the information, or retrieving the information (for example, from memory). Further, “receiving” is typically involved, in one way or another, during operations such as, for example, storing the information, processing the information, transmitting the information, moving the information, copying the information, erasing the information, calculating the information, determining the information, predicting the information, or estimating the information. It is to be appreciated that the use of any of the following “ / ”, “and / or”, and “at least one of”, for example, in the cases of “A / B”, “A and / or B” and “at least one of A and B”, is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of both options (A and B). As a further example, in the cases of “A, B, and / or C” and “at least one of A, B, and C”, such phrasing is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of the third listed option (C) only, or the selection of the first and the second listed options (A and B) only, or the selection of the first and third listed options (A and C) only, or the selection of the second and third listed options (B and C) only, or the selection of all three options (A and B and C). This may be extended, as is clear to one of ordinary skill in this and related arts, for as many items as are listed. As will be evident to one of ordinary skill in the art, implementations can produce a variety of signals formatted to carry information that can be, for example, stored or transmitted. The information can include, for example, instructions for performing a method, or data produced by one of the described implementations. For example, a signal can be formatted to carry the bitstream of a described embodiment. Such a signal can be formatted, for example, as an electromagnetic wave (for example, using a radio frequency portion of spectrum) or as a baseband signal. The formatting can include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information that the signal carries can be, for example, analog or digital information. The signal can be transmitted over a variety of different wired or wireless links, as is known. The signal can be stored on a processor-readable medium. Various embodiments are described herein. Features of these embodiments can be provided alone or in any combination, across various claim categories and types.

Claims

CLAIMS 1. A method comprising: obtaining a first pair of colors of a pair of pixels, wherein a first pixel of the pair of pixels is comprised in a first image of a video and a second pixel of the pair of pixels is comprised in a second image of the video, and wherein the first pixel and the second pixel are colocalized in the images; determining an average color between colors of the first pair of colors; determining a second pair of colors, wherein the colors of the second pair of colors are symmetrical with regards to the average color and for which a sum of energies consumed by displaying a pair of pixels having the second pair of colors is lower than a sum of energies consumed by displaying a pair of pixels having the first pair of colors; and replacing the first pair of colors of the pair of pixels by the second pair of replacement colors.

2. The method of claim 1, wherein a first color of the second pair of colors is determined based on a distance with a first color of the first pair of colors.

3. The method of any of claims 1 or 2, wherein a first color of the first pair of color is perceptually identical to a first color of the second pair of colors and a second color of the first pair of color is perceptually identical to a second color of the second pair of colors.

4. The method of any of claims 1 to 3, further comprising determining a set of pair of colors of minimal energies and selecting the second pair of colors in the set of pair of colors of minimal energies.

5. The method of any of claims 1 to 4, wherein the second image is directly consecutive to the first image.

6. The method of any of claims 1 to 5, further comprising iterating the method for a plurality of first pairs of colors and storing an association between a first pair of colors and a corresponding second pair of colors.

7. The method of claim 6, wherein the association is stored in a correspondence table using a pair of colors as index.

8. A method comprising: obtaining a visual content; iterating over successive frames of the visual content, iterating over pairs of pixels colocalized in the successive frames, and for an iterated pair of pixels: determining a second pair of colors for the iterated pair of pixels according to the method of any of claims 1 to 7; and modifying the visual content by replacing the colors of the iterated pair of pixels by the second pair of colors; and providing the modified visual content.

9. An apparatus comprising a processor configured to: obtain a first pair of colors of a pair of pixels, wherein a first pixel of the pair of pixels is comprised in a first image of a video and a second pixel of the pair of pixels is comprised in a second image of the video, and wherein the first pixel and the second pixel are colocalized in the images; determine an average color between colors of the first pair of colors; determine a second pair of colors, wherein the colors of the second pair of colors are symmetrical with regards to the average color and for which a sum of energies consumed by displaying a pair of pixels having the second pair of colors is lower than a sum of energies consumed by displaying a pair of pixels having the first pair of colors; and replace the first pair of colors of the pair of pixels by the second pair of replacement colors.

10. The apparatus of claim 9, wherein a first color of the first pair of color is perceptually identical to a first color of the second pair of colors and a second color of the first pair of color is perceptually identical to a second color of the second pair of colors.

11. The apparatus of any of claims 9 or 10, wherein the processor further comprises iterating the method for a plurality of first pairs of colors and storing an association between a first pair of colors and a corresponding second pair of colors.

12. The apparatus of claim 11, wherein the association is stored in a correspondence table using a pair of colors as index.

13. An apparatus comprising a processor configured to: obtain a visual content; iterate over successive frames of the visual content, iterating over pairs of pixels colocalized in the successive frames, and for an iterated pair of pixels: determine a second pair of colors for the iterated pair of pixels according to the method of any of claims 9 to 12; and modify the visual content by replacing the colors of the iterated pair of pixels by the second pair of colors; and provide the modified visual content.

14. A computer program including instructions, which, when executed by a computer, cause the computer to carry out the method according to any of claims 1 to 8.

15. A non-transitory computer readable medium storing executable program instructions to cause a computer executing the instructions to perform a method according to any of claims 1 to 8.

Images