Method and device for energy reduction control of visual content

Abstract

The method and device introduce the concept of quality / energy trade-off control for visual content. This concept provides the user with fine and continuous control over the intensity of any algorithm applied to the visual content by the device. This control allows the user to balance the quality of the experience with the energy consumed for using, consuming, transmitting, and / or displaying the visual content. In fact, the user may be willing to accept lower quality content, provided that they benefit from greater energy savings.

Description

Technical Field

[0001] Cross - reference to Related Applications This application claims priority to European Patent Application Publication No. 23305480.8, filed on April 3, 2023, which is hereby incorporated by reference in its entirety.

[0002] Technical Field This disclosure relates to the field of multimedia content delivery, and at least one embodiment relates more particularly to controlling energy consumption reduction in a system for handling visual content such as images or videos.

Background Art

[0003] Background Art Reducing the energy consumption of electronic devices is not only a requirement for manufacturers of electronic devices, but also a requirement for limiting the impact on the environment as much as possible and contributing to the emergence of a sustainable display industry. As display resolutions increase from SD to HD, then to 4K, and soon to 8K or more, and with the introduction of high - dynamic - range imaging, the energy requirements of display devices have increased correspondingly. This runs counter to the global need to reduce energy consumption, given that many devices (i.e., TVs, mobile phones, tablets, etc.) have displays. In fact, displays are the most important energy - consuming source in consumer - oriented electronic devices, whether battery - powered (e.g., smartphones, tablets, head - mounted displays, in - vehicle display screens) or not (e.g., TV sets, advertising display panels).

[0004] In recent years, various display technologies have been developed. The latest displays consume energy in a more controllable and efficient way than older displays, but still remain the most important energy - consuming source in the video chain.

[0005] As far as backlit displays are concerned, their energy consumption is primarily determined by the intensity of the backlight.

[0006] Organic light-emitting diodes (OLEDs) are an example of a display technology that is becoming increasingly popular due to their many advantages over previous technologies such as thin-film transistor liquid crystal displays (TFT-LCDs). OLED displays and miniLEDs consist of individual, directly emitting image pixels rather than using a uniform backlight. Therefore, the power consumption of an OLED is highly correlated with the image content, and the power consumption for a given input image can be estimated by considering the values ​​of the displayed image pixels.

[0007] Therefore, there is interest in devising energy-efficient images or videos, i.e., images or videos that require less energy when displayed, especially when displayed on consumer electronics OLED displays. Techniques have been disclosed for generating energy-efficient images from an original image by using so-called attenuation maps, which reduce the pixel values ​​of an image to reduce the energy required to display them. Such attenuation maps can be designed to ensure good properties such as smoothness and scalability. Another example of reducing energy consumption is replacing pixels in an image with temporally consecutive pixels or spatially adjacent pixels of alternating complementary colors that require less energy to display.

[0008] What these techniques have in common is that they are all based on algorithms designed to reduce the energy requirements of display devices, with the goal of maintaining quality of experience (QoE) based on factors such as contrast, brightness, temporal smoothness, or color levels. [Overview of the project]

[0009] overview Image / video processing algorithms dedicated to energy reduction can achieve this task with varying degrees of experience quality. Such algorithms can prevent the introduction of defects, or more precisely, visible defects. Such algorithms can also introduce visible defects of varying degrees or that are disruptive to the end user. If there are no defects or no visible defects are introduced (hereinafter referred to as the perfect case), the quality of the experience remains unchanged, and therefore there is no need to balance energy reduction with experience quality. This applies to video broadcasting, where there is a constraint that the image displayed on the user's screen should show as little degradation as possible relative to a given transmission and display environment. While the requirement of absolute high quality is natural for programs of high artistic value, such as feature films, it can be argued that absolute crisp quality is not necessary for many other program types, such as weather forecasts, talk shows, video games, animation, sitcoms, and advertisements. At least some users may be willing to tolerate a decrease in the quality of their chosen program type in exchange for some reduction in energy consumption.

[0010] Another perspective is that when defects are introduced, there is interest in finding a balance between energy savings and quality of economy (QoE). Such defects, even severe ones, may be tolerable to end-users in some cases, and their existence may be even more acceptable if, for example, users benefit from energy savings.

[0011] Some algorithms may have some flaws, but they may also be more efficient at reducing energy consumption than others.

[0012] In all of the above cases, it is necessary to consider the relationship with current energy efficiency and the balance between increased energy consumption in addressing climate change and perfect quality.

[0013] The embodiments described below are designed with the above in mind and introduce the concept of quality / energy trade-off control for visual content. This concept provides the user with fine and continuous (or nearly continuous or partially continuous) control over the intensity of any algorithms applied to the visual content by the device. This control allows the user to balance the quality of the experience with the energy consumed for using, consuming, transmitting, and / or displaying the visual content. In fact, the user may be willing to accept a lower quality of visualization if they benefit from a greater improvement in energy saving.

[0014] The objective is to offer users the possibility of further reducing energy consumption by enabling a stronger degradation of content quality (which is particularly acceptable to users) compared to the average or to values ​​externally determined by, for example, content creators, content providers, or device manufacturers. Such adjustments, which are specific to the user and their choices, can result in greater energy savings than the typical balance between energy reduction and QoE that is conventionally proposed by content providers. On the other hand, in certain cases, users may want to improve visual quality even at the cost of additional energy consumption.

[0015] In any case, it involves giving the user control, allowing them to define presets of algorithms to be implemented for energy reduction, while simultaneously giving the user control over fine-tuning and continuous adjustments to the energy reduction / quality balance for a given algorithm. In fact, the presets may be more conservative than those the user is prepared to accept as image changes. The presence of defects may be even more tolerable, for example, if the user benefits from energy savings.

[0016] The first aspect relates to a method comprising acquiring visual content, determining the algorithmic strength of an algorithm for reducing the energy consumption required for the visual content, determining modified visual content by applying the algorithm to the visual content using the determined algorithmic strength, and providing the modified visual content, wherein the modified visual content requires less energy than the original visual content.

[0017] The second aspect relates to a device including a processor configured to acquire visual content, determine the algorithmic strength of an algorithm that reduces the energy consumption required for the visual content, determine modified visual content by applying the algorithm to the visual content using the determined algorithmic strength, and provide the modified visual content, wherein the modified visual content requires less energy than the original visual content.

[0018] In at least one embodiment of the first and second embodiments, the algorithm intensity is obtained through a user interface that provides means for adjusting a value representing the algorithm intensity. The at least one embodiment of the first and second embodiments further includes obtaining an attenuation map related to an energy reduction rate corresponding to the determined algorithm intensity, and the algorithm includes combining the attenuation map with the visual content by adding the attenuation map value to the pixel value, subtracting the attenuation map value from the pixel value, or multiplying the attenuation map value by the pixel value for each pixel of the image of the visual content.

[0019] The third aspect relates to a non-temporary computer-readable medium containing instructions that, when the program is executed by the computer, cause the computer to carry out the described embodiments relating to the first aspect.

[0020] The fourth aspect is directed to a computer program that, when executed by a computer, includes instructions for causing the computer to perform any of the described embodiments related to the first aspect or a variant thereof.

[0021] The above presents a simplified overview of the subject matter to provide a basic understanding of some aspects of the present disclosure. This overview is not an extensive overview of the subject matter. It is not intended to identify key / important elements of the embodiments or to delineate the scope of the subject matter. Its sole purpose is to present some concepts of the subject matter in a simplified form as a prelude to the more detailed description provided below.

[0022] Brief Description of the Drawings The present disclosure may be better understood by considering the following detailed description in conjunction with the accompanying drawings.

Brief Description of the Drawings

[0023] [Figure 1] A block diagram of an example of a display device in which various aspects and embodiments are implemented is shown. [Figure 2] An example of an architecture for quality / energy trade-off control according to one embodiment is shown. [Figure 3] An exemplary process for controlling reduction of energy consumption of a device for displaying visual content according to an embodiment is shown. [Figure 4] Different examples of user interfaces for selecting algorithm strength according to an embodiment are shown.

Modes for Carrying Out the Invention

[0024] It should be understood that the drawings are for purposes of illustrating examples of various aspects, features, and embodiments according to the present disclosure and are not necessarily the only possible configurations. Throughout the various figures, like reference numerals refer to the same or similar features.

[0025] Detailed explanation Figure 1 shows a block diagram of an example of a display device in which various embodiments and designs are implemented. In the illustrated environment, a user interacts with a display device 100, such as a television, connected to a server 180 operated by a content provider, for example. The server 180 distributes multimedia content 190, such as video streams based on images. In the video distribution system, multiple devices 100, 1xx interact with multiple content providers and corresponding servers 180, 18x that distribute multiple multimedia content 190, 19x. A single content provider may also use multiple servers. The devices exchange data via a communication network 150.

[0026] The communication network 150 preferably uses a communication standard to provide interoperability between content providers and display devices. Such a communication standard may be wireless, such as cellular (e.g., LTE) communication or Wi-Fi communication, to ensure the mobility of display devices. Cable, satellite, or terrestrial digital television broadcasting communications may also be used in the communication network 150 and broadband television communications. Such digital television standards may be based on well-established standards such as DVB, ATSC, etc. General-purpose network standards based on Ethernet may also be used, for example.

[0027] The display device 100 includes a processor 101. The processor 101 may be a general-purpose processor, a dedicated processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors working with a DSP core, a controller, a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, etc. The processor may perform data processing such as process 300 for controlling the reduction of energy consumption of the device that displays the visual content shown in Figure 3.

[0028] The processor 101 may be coupled to an input unit 102 configured to transmit user interaction. For this purpose, multiple types of input and modalities can be used. Physical keypads or touch-sensitive surfaces are typical examples of inputs suited to this application, but voice control can also be used. In addition, the input unit may include a digital camera that can capture still pictures or videos in two dimensions, or a more complex sensor that can determine depth information in addition to pictures or videos, and thus capture a full 3D representation.

[0029] The processor 101 may be coupled to a display unit 103 configured to output visual data to be displayed on a screen. For this purpose, several types of displays can be used, such as liquid crystal displays (LCDs) or organic light-emitting diode (OLED) display units. The processor 101 may also be coupled to an audio unit 104 configured to convert audio data into sound waves through a suitable transducer, such as a speaker.

[0030] The processor 101 may be coupled to a communication interface 105 configured to exchange data with an external device. The communication preferably uses a wireless communication standard that provides mobility for the display device, such as cellular (e.g., LTE) communication or Wi-Fi communication.

[0031] The processor 101 may access information from or store data in the memory 106. The memory 106 may include several types of memory, including random access memory (RAM), read-only memory (ROM), hard disk, subscriber identification module (SIM) card, memory stick, secure digital (SD) memory card, and any other type of memory storage device. In embodiments, the processor 101 may access information from or store data in memory not physically located on this device, such as on a server, home computer, or another device.

[0032] The processor 101 is configured to execute an image energy reduction algorithm that modifies the input image to an image that requires less energy when in use, for example, when displayed, compared to when the input image is used. Various techniques for providing such features are disclosed.

[0033] The processor 101 may receive power from the power supply 108 and may be configured to supply and / or control power to other components within the device 100. The power supply may be any suitable device for supplying power to the device. Examples of power supplies include one or more dry cell batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium ion (Li ion), etc.), solar cells, fuel cells, etc.

[0034] Although the drawings show the processor 101 and other elements 102-108 as separate components, it should be understood that these elements can be integrated together into an electronic package or chip. It should be understood that the display device 100 may include any partial combination of the elements described herein while maintaining consistency with the embodiments described below. The processor 101 may be further coupled to other peripherals or units not shown in Figure 1, which may include one or more software modules and / or hardware modules that provide additional features, functions and / or wired or wireless connectivity. Examples of peripherals include a Universal Serial Bus (USB) port, a vibration device, a television transceiver, a hands-free headset, a Bluetooth® module, a frequency modulation (FM) radio unit, a digital music player, a media player, a video game player module, and an internet browser. For example, the processor 101 may be coupled to a location unit configured to locate the display device within its environment. The location unit may integrate not only a GPS chipset that provides the longitude and latitude position of the display device's current location, but also other motion sensors such as accelerometers and / or e-compasses that provide location services.

[0035] In at least one embodiment, the processor 101 of the display device 100 is configured to display an image on the display unit 103 according to an embodiment further described below. In a first variant embodiment, the image 190 is obtained from a content provider server 180 via a communication network 150. In a second variant embodiment, the image is obtained from a memory 106 that is stored after being captured by an input unit 102 or transferred from the server.

[0036] Typical examples of device 100 include smartphones, tablets, laptops, monitors, head-mounted displays, television sets, video projectors, computer screens, control and / or entertainment systems for vehicles (e.g., automobiles, airplanes, boats, etc.), advertising display panels, and medical monitors. However, any device or component of a device that provides similar functionality may also be used as display device 100 in accordance with the principles of this disclosure.

[0037] In at least one embodiment, the device does not include a display unit but prepares data representing energy-reduced visual content so that another device can utilize the energy-reduced visual content for further processing. In at least one embodiment, such a device prepares data that is displayed by another device, such as a screen. Examples of such devices include set-top boxes, media players, desktop computers, encoders, decoders, servers, computing grids, and cloud computers.

[0038] At least one example of one embodiment may include an apparatus as described herein and a device comprising (i) an antenna configured to receive a signal containing data representing image information, (ii) a band limiter configured to limit the received signal to a frequency band containing data representing image information, and (iii) at least one display configured to display an image from the image information.

[0039] At least one example of one embodiment is a device described herein, which may include a television, a television signal receiver, a set-top box, a gateway device, a mobile device, a cell phone, a tablet, a computer, a laptop, or any other electronic device.

[0040] Generally, the goal of various energy reduction systems for visual content is to minimize the energy consumption of presenting or displaying visual media, primarily on television sets or mobile displays. This reduction in energy consumption is typically achieved while maintaining / acquiring the Quality of Experience (QoE) of the visual media presentation.

[0041] Many algorithms allow for the definition of a parameter, such as algorithm intensity, that acts on the level of energy reduction associated with a quality level. For a given energy reduction corresponding to a given quality level, the algorithm intensity can be determined analytically or through user testing. In this specification, "algorithm intensity" is actually a parameter in an algorithm that acts on changes to a video image, depending on this algorithm intensity, to reach different sequential energy reduction levels and obtain the corresponding QoE. In other words, increasing the algorithm intensity increases energy reduction, and decreasing the algorithm intensity decreases energy reduction.

[0042] Limiting algorithmic intensity to the perfect case (i.e., where no visual degradation is perceptible to the viewer) is common practice when the criterion is the content creator's intent or the broadcaster's choice of the highest visual quality. Therefore, intensity is often adjusted to meet an invisible degradation target. In this case, during user testing, a difference of at most 1 JND (just perceptible difference) is sought, where 1 JND is defined as the amount of change in some value for the difference to be discriminable or detectable for at least half the time.

[0043] Starting from a conservative strength state with no visible defaults on average, the algorithm can evolve in a more powerful direction for energy reduction, while introducing acceptable flaws. In this case, the algorithm strength increases. The algorithm strength can be increased to a level where some flaws become slightly visible or become more visible as the algorithm strength increases. However, even strong flaws may be acceptable to the end user in some cases, and their presence may be even more acceptable if the user benefits from, for example, energy savings.

[0044] Based on this, at least one embodiment proposes adjusting the quality / energy trade-off by adjusting the level of algorithmic intensity. In such a system, the user may choose to sacrifice some of their visual satisfaction for the benefit of consuming less energy.

[0045] In some embodiments, the quality of the experience is perceived by the user through visual feedback that may indicate artifacts that are not measured but may be considered acceptable or unacceptable by the user. In other embodiments, the quality of the experience is measured using conventional image quality metrics (i.e., PSNR, SSIM, VMAF).

[0046] Figure 2 shows an example of an architecture for quality / energy trade-off control according to one embodiment. This architecture is implemented by a device such as device 100 in Figure 1. The device includes an image energy reduction algorithm 201 that converts an input image 210 into a modified image 211 that requires less energy when used. This algorithm is based on the principle of applying a dimming map (also known as an attenuation map) as described in, for example, international application PCT / EP2023 / 082360, or the principle of using temporally alternating complementary colors as described in international application PCT / EP2023 / 085301, or the principle of using spatially alternating complementary colors as described in international application PCT / EP2023 / 085303.

[0047] The device includes a quality / energy tradeoff control module 200 that handles the tradeoff 203 between the quality of the experience when using visual content and the reduction of energy consumption. This control is achieved by setting an appropriate value for algorithm intensity 202, which affects the image energy reduction algorithm. The tradeoff is driven by several inputs, including, for example, user settings 204, user profiles 205, or content provider rules 206. Information from algorithms 201 or images 210, 211 may also be taken into consideration. In at least one embodiment, the algorithm intensity value is 1 minus the value of the energy reduction rate.

[0048] This approach may be used for less critical viewing applications, or when viewers wish to reduce their device's energy consumption and are willing to tolerate a certain degree of visual quality regarding the type of content they are viewing. One example of such a user-adjustable approach is in streaming scenarios where the user can choose which content should be delivered to them.

[0049] Since some defects are easily visible or difficult to see, the quality / energy trade-off control module provides the user with some control over the degree of defects through algorithmic intensity, allowing them to choose between visual comfort or reduced consumption.

[0050] In the first embodiment, control is performed directly by the user, in other words, directly by user settings 204 such as actions on user interface elements when displaying visual content. Direct control may be provided, for example, by user input via sliders, checkboxes, buttons, or any other user interface elements that allow the user to directly or indirectly increase or decrease the algorithm intensity value. Such actions affect the amount of energy saved in correlation with the level of quality of the experience.

[0051] In the second embodiment, control is performed indirectly, for example, by the user, through several settings or parameters in the user profile 205. These settings can be entered using the same user interface elements as in the first embodiment, except that these settings apply to all visual content. In at least one embodiment, the user profile 205 can be replaced by a device configuration. In practice, a user profile depends on the user (or group of users) using the device, and therefore user identification is required to benefit from the user profile. A device configuration applies to all users of the device and therefore does not require such identification. Both offer the same features, allowing several parameters to be set up in the configuration phase and these parameters to be applied during normal use of the device to adapt the behavior to the user's or device's preferences.

[0052] In a third embodiment, control is performed in accordance with Provider Rule 206, in other words, the quality / energy trade-off is determined by the content provider and implemented by the device. For example, a provider of a blockbuster movie would want the audience to enjoy the original image, thereby preventing unintended processing of the image and ensuring that the producer's artistic intent is perceived. Conversely, a producer of a talk show on environmental issues might want to implement maximum energy reduction in a way that does not contradict the talk show's topic.

[0053] In the fourth embodiment, all of the above settings are combined together. To prevent conflicting settings, a priority mechanism is defined, for example, by giving user settings a higher priority than other settings. Another option is to assign weights to each type of setting and implement a weighted sum.

[0054] In modified embodiments applicable to all embodiments described herein, the user's selection is further displayed for informational purposes and as feedback of the optional choice, either through a new amount of energy required to display the content or a new quality level corresponding to the user's selection, or both. This amount of energy and / or quality may be displayed through any other user interface widget that displays a number, color, histogram bar, or quantity of a characteristic. As far as energy is concerned, the amount of energy consumed or the amount of energy saved may be displayed. In the latter case, the feedback to the user may be translated into a more descriptive and specific representation, such as an icon indicating carbon dioxide emissions and / or, for example, "the health of the planet."

[0055] Figure 3 shows an exemplary process for controlling the energy consumption reduction of a device that displays visual content, according to an embodiment. This process 300 is performed, for example, by the processor 101 of device 100 in Figure 1. In step 310, the processor retrieves the visual content or a portion of the visual content, for example, an image from a series of images. In step 320, the processor determines the algorithm strength. As mentioned above, this can be based on multiple inputs, such as user settings, user profile, system status, or rules from a content provider. In step 330, the energy reduction algorithm is applied to the visual content using the determined algorithm strength. A low strength value has only a slight effect on the result. In at least one embodiment, if the algorithm strength value is zero or below a threshold, the algorithm is not applied (dotted line in the figure). In practice, if the energy reduction is insufficient, the energy required to compute the modified content may be greater than the energy savings provided by the modified content. Exemplary values ​​for such a threshold are 0.1 or 0.05.

[0056] The application of energy reduction algorithms may be carried out using attenuation (or dimming) maps. In this case, pixels of the image in the visual content are combined with attenuation that reduces the level of the pixels, for example, while ensuring a satisfactory quality of experience. In at least one embodiment, the combination is carried out via a scaling operation, in other words, by multiplying the value of the image pixel by the value of the corresponding pixel in the attenuation map (i.e., the pixel at the same location in the image). In this case, the pixels in the attenuation map are floating values ​​selected in the range of 0.0 to 1.0. In at least one embodiment, the combination is carried out by subtraction, by subtracting the value of the pixel in the attenuation map from the value of the pixel in the image. In this case, the pixels of attenuation are positive values ​​selected in the range between 0 and the maximum pixel range value (e.g., 255 if represented in 8 bits). In at least one embodiment, the combination is carried out by addition, by adding the value of the pixel in the attenuation map to the value of the pixel in the image. In this case, the pixels of attenuation are negative values ​​selected in the range of "-maximum pixel range value-1" (e.g., -254 if represented in 8 bits).

[0057] In step 340, the processor provides the modified (or unmodified) content. In one embodiment, the content is then displayed directly on the screen. In other embodiments, the content is provided to another device for further use, such as processing or display on another device.

[0058] The calculation of the algorithm strength in step 320 actually depends on the algorithm itself. In the example of international application PCT / EP2023 / 082360, an attenuation map corresponding to a given energy reduction rate is determined with respect to the input image. This attenuation map also corresponds to a given level of experience quality. The proposed technique makes it possible to infer another attenuation map from a first attenuation map by linearly scaling the original attenuation map to obtain a smaller energy reduction rate.

[0059] Applying such linear scaling, a lower energy reduction rate

number

number

number

number

[0060] In at least one embodiment, an energy-efficiency-considered image corresponding to the attenuation map DM(i│R) is constructed by subtracting the attenuation map from the original image.

number

[0061] In another embodiment, the user requests to achieve a reduction greater than the reduction provided by the attenuation map.

number

[0062] In either case,

number

[0063] In at least one embodiment, the attenuation map DM(I R The energy-efficient image corresponding to this is constructed by multiplying the original image by an attenuation map. I R =I×DM(I R )

[0064] In this case, the values ​​of the attenuation map should be in the range [0,1]. Starting with one single attenuation map corresponding to the energy reduction rate R, the other attenuation map corresponding to the other energy reduction rate is calculated as follows:

number

number

number

[0065] In this case as well,

number

[0066] International application PCT / EP2023 / 083362 proposes transmitting multiple attenuation maps corresponding to one of several energy reduction rates and associated quality of experience for a given input image, and in addition to conventional techniques, interpolating pairs of attenuation maps to enable inference of another attenuation map from a pair of attenuation maps via bilinear interpolation of the pairs of attenuation maps.

[0067] In this case as well,

number

[0068] In a global system where the user influences the construction of the attenuation map (as opposed to a system that only receives content and at least one associated attenuation map), the construction of the attenuation map may be driven by some interaction with the end user who selects the algorithmic intensity (1-R). This intensity directly determines the target energy reduction rate R, the expected image quality considering the resulting energy efficiency, and thus the construction of the attenuation map. In addition, the construction of the first attenuation map may depend on a target energy profile by some end user or end device. The selection of the algorithmic intensity may also be guided by the end device itself, based on other parameters such as battery status or expected video duration.

[0069] In a global system where users influence the delivery of attenuation maps (e.g., a DASH-based streaming environment), users can select one or more attenuation maps, categorized by ordered algorithmic intensity based on their associated energy reduction rates and expected quality of experience.

[0070] Figure 4 shows different examples of user interfaces for selecting algorithm intensity according to the embodiment. These examples only show the graphic elements of the user interface that enable intensity control; they do not include the complete user interface.

[0071] The first example of a graphic element for controlling algorithm intensity 401 is based on a slider that directly drives the algorithm intensity, for example, represented as a percentage of the maximum intensity. Moving the cursor to the right increases the algorithm intensity. Moving the cursor to the left decreases the algorithm intensity. The cursor can be moved using conventional techniques, for example, using left and right arrows or by direct control via a touchscreen. A value equal to zero (i.e., the cursor at the far left) disables energy reduction and bypasses image changes. Optionally, the user can also directly input a numerical value, for example, using number keys or voice input. Optionally, a color range that changes between green on the right and red on the left can be associated with the slider, with more green on the right resulting in greater energy reduction.

[0072] A second example of a graphic element for controlling algorithm intensity 402 is based on a slider that drives the energy reduction rate provided by the algorithm, expressed as a percentage of reduction. Optionally, the user can also directly input a numerical value or use a color range.

[0073] A third example of a graphic element for controlling algorithm strength 403 is based on a pair of interconnected sliders that illustrate the relationship between energy reduction and quality of experience. In practice, the user increases the energy reduction rate using the upper slider while simultaneously decreasing the value of the second slider, and vice versa. Optionally, the user can also directly input numerical values ​​or use a color range.

[0074] A fourth example of a graphic element for controlling algorithm strength 404 is based on a user interface panel that allows the user to select the algorithm strength from checkboxes labeled "No energy reduction," "Small reduction," "Medium reduction," "Large reduction," and "Maximum reduction."

[0075] In addition, the user interface can also display indicators of the quality of the experience, such as numerical values ​​(i.e., similar percentages).

[0076] Those skilled in the art will likely conceive of many other conventional techniques for inputting algorithm strength (such as voice control and gestures).

[0077] While some parts of this specification refer to images, embodiments are not limited to images and apply to any type of visual media content, such as conventional (2D) video, stereoscopic (3D) images or videos, or 360° immersive images or videos, which are repeated in time and / or space, but are based on the same principles as described above.

[0078] In general, one or more other examples of the embodiments may also provide a computer-readable storage medium, such as a non-volatile computer-readable storage medium, that stores instructions for encoding or decoding picture information, such as video data, according to the methods or apparatus described herein. One or more embodiments may also provide a computer-readable storage medium that stores bitstreams generated according to the methods or apparatus described herein. One or more embodiments may also provide methods and apparatus for transmitting or receiving bitstreams or signals generated according to the methods or apparatus described herein.

[0079] Many of the embodiments described herein are described with specificity and are often described in a way that may seem restrictive, at least in order to illustrate individual features. However, this is for the purpose of clarity and not to limit the application or scope of those embodiments. In fact, all of the different embodiments can be combined and substituted to provide further embodiments. Furthermore, embodiments, features, etc., can be combined and substituted with other embodiments, features, etc., described in prior applications.

[0080] Various implementations involve decoding. As used in this application, “decoding” may encompass all or part of the processes performed on a received encoded sequence to produce, for example, a final output suitable for display. In various embodiments, such processes include one or more processes typically performed by a decoder (e.g., entropy decoding, inverse quantization, inverse transform, and differential decoding). In various embodiments, such processes may additionally or alternatively include processes performed by the decoder in the various implementations described herein.

[0081] As further examples, in one embodiment, “decoding” refers only to entropy decoding; in another embodiment, “decoding” refers only to differential decoding; and in yet another embodiment, “decoding” refers to a combination of entropy decoding and differential decoding. Whether the phrase “decoding process” is intended to refer specifically to a subset of operations or to a broader decoding process in general will become clear from the context of the specific description and will be well understood by those skilled in the art.

[0082] Various implementations involve encoding. As discussed above with respect to "decoding," as used in this application, "encoding" may encompass all or part of the process performed on an input video sequence to generate an encoded bitstream. In various embodiments, such a process typically includes one or more processes performed by an encoder (e.g., partitioning, differential coding, transformation, quantization, and entropy coding).

[0083] As further examples, in one embodiment, “encoding” refers only to entropy coding; in another embodiment, “encoding” refers only to differential coding; and in yet another embodiment, “encoding” refers to a combination of differential coding and entropy coding. Whether the phrase “encoding process” is intended to refer specifically to a subset of operations or to a broader encoding process in general will become clear from the context of the specific description and will be well understood by those skilled in the art.

[0084] Please note that the syntactic elements used in this specification are terms for illustrative purposes only. Therefore, this does not preclude the use of other syntactic element names.

[0085] When a diagram is presented as a flow chart, it also provides a block diagram of the corresponding device. Similarly, when a diagram is presented as a block diagram, it also provides a flow chart of the corresponding method / process.

[0086] In general, examples of embodiments, implementations, and features described herein may be implemented, for example, in methods or processes, apparatus, software programs, data streams, or signals. Even when discussed only in relation to a single implementation (e.g., discussed only as a method), the implementation of the features discussed may also be implemented in other forms (e.g., apparatus or programs). Apparatus may be implemented, for example, with appropriate hardware, software, and firmware. One or more examples of methods may be implemented, for example, in a processor, where a processor generally refers to a processing device including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices such as, for example, computers, mobile phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate the communication of information between end users. The use of the term "processor" herein is also intended to broadly encompass one processor or various configurations of two or more processors.

[0087] References to “one embodiment” or “one implementation” or “one implementation” and other variations mean that the specific features, structures, characteristics, etc. described in relation to that embodiment are included in at least one embodiment. Therefore, the appearance of the phrases “in one embodiment” or “in one embodiment” or “in one implementation” and any other variations appearing in various places throughout this application do not necessarily all refer to the same embodiment.

[0088] In addition, this application may refer to "determining" various types of information. Determining information may include, for example, one or more of the following: estimating information, calculating information, predicting information, or retrieving information from memory.

[0089] Furthermore, this application may refer to “accessing” various types of information. Accessing information may include, for example, one or more of the following: receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, computing information, determining information, predicting information, or estimating information.

[0090] In addition, this application may refer to "receiving" various types of information. Receiving is intended to be a broad term, similar to "accessing." Receiving information may include, for example, one or more of the following: accessing information or retrieving information (for example, from memory). Furthermore, "receiving" typically involves in some way during operations such as, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0091] It should be understood that the use of any of the following " / ", "and / or", and "at least one of ~" is intended to include, for example, in the cases of "A / B", "A and / or B", and "at least one of A and B", the selection of only the first enumerated option (A), or only the second enumerated option (B), or the selection of both options (A and B). As a further example, in the cases of "A, B and / or C" and "at least one of A, B and C", such a phrase is intended to include the selection of only the first enumerated option (A), or only the second enumerated option (B), or only the third enumerated option (C), or only the first and second enumerated options (A and B), or only the first and third enumerated options (A and C), or only the second and third enumerated options (B and C), or the selection of all three options (A, B, and C). This can be extended as many times as there are enumerated items, as will be apparent to those skilled in the art.

[0092] As will be apparent to those skilled in the art, implementations can generate various signals, for example, that are formatted to carry information that can be stored or transmitted. This information may include, for example, instructions for performing a method or data generated by one of the described implementations. For example, a signal may be formatted to carry a bitstream of one of the described embodiments. Such a signal may be formatted, for example, as an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or as a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. The signal may be transmitted, as is well known, over various different wired or wireless links. The signal may be stored on a processor-readable medium.

[0093] Various embodiments are described herein. Features of these embodiments may be provided individually or in any combination across various categories and types of claims.

Claims

1. - Acquiring visual content, - To determine the algorithmic strength of the algorithm for reducing the energy consumption required for the aforementioned visual content, - The modified visual content is determined by applying the algorithm to the visual content using the determined algorithm strength, - To provide the aforementioned modified visual content A method comprising increasing the algorithm intensity, which increases the reduction in energy consumption, and decreasing the algorithm intensity, which increases the reduction in energy consumption, such that the modified visual content consumes less energy than the visual content when displayed.

2. The method according to claim 1, wherein the algorithm strength is obtained through a user interface that provides means for adjusting a value representing the algorithm strength.

3. The method according to claim 1, wherein the algorithm strength is a configuration setting obtained from memory.

4. The method according to claim 3, wherein the memory includes a list of algorithmic intensities associated with classification-based categories relating to visual content, and the method further includes selecting the algorithmic intensities associated with the categories of the acquired visual content.

5. The method according to any one of claims 1 to 4, further comprising obtaining an attenuation map relating to an energy reduction rate corresponding to the determined algorithm intensity, wherein the algorithm includes combining the attenuation map with the visual content by adding the values ​​of the attenuation map to the values ​​of the pixels of the image of the visual content.

6. The method according to any one of claims 1 to 4, further comprising obtaining an attenuation map relating to an energy reduction rate corresponding to the determined algorithm intensity, wherein the algorithm combines the attenuation map with the visual content by subtracting the values ​​of the attenuation map from the values ​​of the pixels for each pixel of the image of the visual content.

7. The method according to any one of claims 1 to 4, further comprising obtaining an attenuation map relating to an energy reduction rate corresponding to the determined algorithm intensity, wherein the algorithm combines the attenuation map with the visual content by multiplying the values ​​of the attenuation map by the values ​​of the pixels in the image of the visual content.

8. If none of the energy reduction rates associated with the multiple attenuation maps are equal to the energy reduction rate corresponding to the determined algorithm intensity, - Obtain a first attenuation map in which the associated energy reduction rate is different from the energy reduction rate corresponding to the determined algorithm intensity, - A second attenuation map is generated by scaling the first attenuation map according to a ratio determined based on the energy reduction rate associated with the first attenuation map and the energy reduction rate corresponding to the determined algorithm intensity. - The second attenuation map is combined with the visual content by adding, subtracting, or multiplying the value of the second attenuation map to the value of the pixel of the image of the visual content. The method according to any one of claims 1 to 4, further comprising:

9. If none of the energy reduction rates associated with the multiple attenuation maps are equal to the energy reduction rate corresponding to the determined algorithm intensity, - Obtain a first attenuation map in which the associated energy reduction rate is greater than the energy reduction rate corresponding to the determined algorithm intensity, - Obtain a second attenuation map in which the associated energy reduction rate is smaller than the energy reduction rate corresponding to the determined algorithm intensity, - A third attenuation map is generated by performing bilinear interpolation between the first attenuation map and the second attenuation map based on the energy reduction rates associated with the first and second attenuation maps and the energy reduction rate corresponding to the determined algorithm intensity. - The third attenuation map is combined with the visual content by adding, subtracting, or multiplying the values ​​of the third attenuation map to the values ​​of the pixels in the image of the visual content. The method according to any one of claims 1 to 4, further comprising:

10. - Acquiring visual content, - To determine the algorithmic strength of the algorithm for reducing the energy consumption required for the aforementioned visual content, - The modified visual content is determined by applying the algorithm to the visual content using the determined algorithm strength, - To provide the aforementioned modified visual content A device including a processor configured to perform the following: increasing the algorithm intensity increases the reduction in energy consumption, and decreasing the algorithm intensity increases the reduction in energy consumption, such that the modified visual content consumes less energy than the visual content when displayed.

11. The apparatus according to claim 10, wherein the algorithm strength is obtained through a user interface that provides means for adjusting a value representing the algorithm strength.

12. The apparatus according to claim 10, wherein the algorithm strength is a configuration setting obtained from memory.

13. The apparatus according to claim 12, wherein the memory includes a list of algorithmic intensities related to classification-based categories relating to visual content, and the apparatus further includes selecting the algorithmic intensities related to the categories of the acquired visual content.

14. The apparatus according to any one of claims 10 to 13, further comprising obtaining an attenuation map relating to an energy reduction rate corresponding to the determined algorithm intensity, wherein the algorithm comprises combining the attenuation map with the visual content by adding the values ​​of the attenuation map to the values ​​of the pixels of the image of the visual content.

15. The apparatus according to any one of claims 10 to 13, further comprising obtaining an attenuation map relating to an energy reduction rate corresponding to the determined algorithm intensity, wherein the algorithm comprises combining the attenuation map with the visual content by adding the values ​​of the attenuation map to the values ​​of the pixels of the image of the visual content.

16. The apparatus according to any one of claims 10 to 13, further comprising obtaining an attenuation map relating to an energy reduction rate corresponding to the determined algorithm intensity, wherein the algorithm comprises combining the attenuation map with the visual content by multiplying the values ​​of the attenuation map by the values ​​of the pixels in the image of the visual content.

17. If none of the energy reduction rates associated with the multiple attenuation maps are equal to the energy reduction rate corresponding to the determined algorithm intensity, - Obtain a first attenuation map in which the associated energy reduction rate is different from the energy reduction rate corresponding to the determined algorithm intensity, - A second attenuation map is generated by scaling the first attenuation map according to a ratio determined based on the energy reduction rate associated with the first attenuation map and the energy reduction rate corresponding to the determined algorithm intensity. - The second attenuation map is combined with the visual content by adding, subtracting, or multiplying the value of the second attenuation map to the value of the pixel of the image of the visual content. The apparatus according to any one of claims 10 to 13, further comprising:

18. If none of the energy reduction rates associated with the multiple attenuation maps are equal to the energy reduction rate corresponding to the determined algorithm intensity, - Obtain a first attenuation map in which the associated energy reduction rate is greater than the energy reduction rate corresponding to the determined algorithm intensity, - Obtain a second attenuation map in which the associated energy reduction rate is smaller than the energy reduction rate corresponding to the determined algorithm intensity, - A third attenuation map is generated by performing bilinear interpolation between the first attenuation map and the second attenuation map based on the energy reduction rates associated with the first and second attenuation maps and the energy reduction rate corresponding to the determined algorithm intensity. - The third attenuation map is combined with the visual content by adding, subtracting, or multiplying the values ​​of the third attenuation map to the values ​​of the pixels in the image of the visual content. The apparatus according to any one of claims 10 to 13, further comprising:

19. The apparatus according to any one of claims 10 to 18, selected from a set including a smartphone, tablet, laptop, monitor, head-mounted display, television set, video projector, computer screen, control and / or entertainment system for a vehicle, advertising display panel, and medical monitor.

20. A computer program that, when executed by a computer, includes instructions causing the computer to perform the method described in any one of claims 1 to 9.

21. A non-temporary computer-readable medium that stores executable program instructions for causing a computer executing the instructions to perform the method described in any one of claims 1 to 9.

Images