Device and method for processing image data

Abstract

The invention relates to a device for processing image data (1), in particular video data, which can be displayed as an image, in particular an image sequence, on a terminal (30) or using a display device of a terminal (30), wherein a display of the pixels of the image, in particular of the image sequence, can be completely or partially changed by or using at least one program-controlled overlay means (21), characterized in that the transparency, the colour, the colour model, the brightness, the contrast, the shape and / or the size of the at least one overlay means (21) depend(s) at least on a property of the image data (1) for pixels in the terminal (30) and / or in the display device and / or at least one state variable (15) in the environment of the terminal (30) and / or the display device and / or at least one state variable (15) in the terminal (30) and / or in the display device itself. The invention also relates to a method for processing image data (1) and to a computer program product.

Description

[0001] Device and method for processing image data

[0002] The invention comprises an apparatus for processing image data having the features of claim 1 and a method for processing image data having the features of claim 20.

[0003] Image data, especially video content, is typically transmitted in the highest possible visual quality and displayed on display devices such as screens or projected onto light-reflecting surfaces using projectors.

[0004] The display devices (e.g., screens or projectors) can constitute a terminal device themselves or be connected to another terminal device (e.g., a set-top box or a streaming device). In the latter case, the screens or projectors, where a distinction is necessary, are referred to as display devices.

[0005] The majority of energy consumption when playing back image data, especially video data, is determined by the brightness of the display device used (e.g. a screen or projector). End devices in which these screens or projectors are installed sometimes have functions that are intended to save energy compared to end devices without these functions. Examples are so-called "Eco Modes", i.e. energy-saving modes of conventional televisions or digital screens, which usually have to be activated manually by the viewer or user. Control of this function by a content provider or service provider is difficult. To reduce energy consumption, innovative and technically more applicable energy-saving solutions are necessary.

[0006] This is addressed by the subject matter of claim 1.

[0007] The device according to claim 1 is configured to display image data, ie, still images and / or moving images (video data), on or with a terminal device. A typical terminal device for displaying the image data could be, for example, a television, but also an outdoor (out-of-home) screen. A terminal device connected to a display device could be, for example, a set-top box.

[0008] The display of the pixels of the image, in particular of the image sequence, can be changed in whole or in part by or with at least one program-controlled overlay means. The transparency, color, color model, brightness, contrast, shape and / or size of the at least one overlay means depend on at least one property of the image data, of pixels in the terminal device and / or the display device. For example, the brightness of the at least one overlay means can depend on parameters of the image data itself. If, for example, an area of ​​an image is particularly bright, this area can be darkened with a correspondingly dark overlay means.

[0009] Additionally or alternatively, the said parameters of the overlay means may also depend on at least one state variable in the environment of the terminal device and / or the display device and / or at least one state variable in the terminal device itself and / or in the display device itself.

[0010] In particular, state variables in the environment of a display device can be important, such as recorded brightness values ​​in the environment of a screen or projector. User behavior on the end device or on a display device can also represent a state variable that influences the properties of at least one overlay device.

[0011] The overlay can then be digitally placed over the actual image data to darken it, in particular without permanently altering it. The properties of the at least one overlay, such as brightness, depend on the state variable(s) determined, for example, on the display and / or end device or in the environment of the display and / or end device.

[0012] In any case, the energy consumption of the terminal device and / or the display device can be specifically reduced by means of the at least one overlay means, since the brightness of the display device in particular significantly determines the energy consumption.

[0013] For example, an overlay manipulator can be used to control the transparency, color, color model, brightness, contrast, shape and / or size of at least one overlay means.

[0014] The control is carried out depending on at least one property of the image data, of pixels in the end device and / or in the display device. Additionally or alternatively, this control can also be carried out depending on at least one state variable in the environment of the end device and / or the display device and / or at least one state variable in the end device and / or in the display device. The overlay manipulator can be designed as software that specifically generates, for example, a grayscale mask that can be arranged over the image data. However, the overlay manipulator can also be implemented as an integrated circuit that, for example, accesses image data and encodes the at least one overlay means into it. The overlay manipulator can, for example, also be used by a streaming service that provides image data with overlay means before it is received by a end device and / or a display device.

[0015] The properties of the image data or pixels that can influence the at least one overlay medium are brightness, contrast, color, and / or information about the design of the end device and / or display device. All of these parameters influence energy consumption.

[0016] In one embodiment, the at least one overlay means can be formed as a homogeneous surface that is achromatic (grayscale, black and white) or colored. Homogeneous here means that the properties of the overlay means, such as brightness or hue, are the same within the surface.

[0017] Alternatively, the at least one overlay means can be configured as an inhomogeneous surface that is achromatic or colored. In this embodiment, at least one property of the overlay means, such as the brightness, varies within the surface.

[0018] In particular, the energy consumption of the terminal device and / or display device can be reduced by the use of at least one overlay means.

[0019] In one embodiment, the terminal device comprises or is configured as a television, a screen, a large-screen indoor device, an out-of-home screen, a projector, and / or a mobile device, in particular a computer, a tablet computer, or a smartphone. In these cases, the terminal device itself comprises or is configured as a display device. Reducing energy consumption by means of overlays is particularly important for very large screens in the entertainment sector or for out-of-home screens, which are generally permanently switched on.

[0020] It is also possible for the terminal device to have a local device for encoding and / or decoding image data, in particular a set-top box or a streaming device, or for the terminal device to be designed as such a device. A streaming device can in particular be a device (e.g., in the form of an HDMI stick) that is connected to the Internet and runs an operating system. The operating system enables the execution of an application for displaying the image data.

[0021] The at least one overlay means reduces the energy consumption when displaying the image data on the display device which is coupled to such a terminal device - also via a non-displaying terminal device.

[0022] The at least one overlay means can be generated and / or modified using the overlay manipulator, with the overlay manipulator being functionally separate from a player for the image data. This separation allows for the continued use of conventional players.

[0023] Furthermore, in one embodiment, the brightness values ​​of video content as image data can be adjusted by the at least one overlay means. In particular, the displayed brightness of the image data can be completely or partially lower than without the overlay means. By reducing the display brightness, the energy consumption of the screens or projectors can be reduced.

[0024] Furthermore, in one embodiment, received image data, in particular video data, are not irreversibly altered by the use of the at least one overlay means. For example, if the overlay means is not used, the image data remains illuminated with undiminished brightness.

[0025] The at least one overlay means can also be used for the entire duration of a video unit, in particular an image data stream, e.g., globally in terms of time, or locally in terms of time only for one or more temporal sub-regions of the video data. Additionally or alternatively, the at least one overlay means for an image data stream can be used globally in terms of area for each entire image or locally in terms of area only for one or more sub-regions of each image.

[0026] The overlay devices used can also be changed over time and / or capture different parts of the image data. This allows, in particular, the brightness of the image data to be flexibly adjusted and, above all, adaptively adapted to the parameters or state variables captured on the end device (and / or display device) or in the environment of the end device (and / or display device).

[0027] Furthermore, the at least one overlay means can be created and adapted based on automatically recorded data and / or on automatic analyses of a usage situation on the terminal device and / or the display device, in particular using artificial intelligence. For example, if the activities of a user on the display device or the environment of the display device (movements, setting parameters on the display device, etc.) are recorded, the brightness—and thus the energy consumption—can be controlled accordingly. For example, if a user leaves the room in which the display device is located, the display device can be automatically dimmed using the overlay means.

[0028] In particular, the overlay manipulator can be configured to receive data on the energy consumption, in particular the power consumption, of at least one terminal device and / or display device, wherein this data is based on measurements or estimates and the at least one overlay means can be generated depending on this data on the energy consumption. If, for example, no data can be recorded on-site at the terminal device, it is possible by means of artificial intelligence to predict expected patterns in the use of the terminal devices over a certain period of time in order to adjust the brightness of the display of the terminal device accordingly using the overlay means. It is also possible, for example, to evaluate statistical data on user behavior or data on the user environment in order to adjust the energy consumption accordingly using the at least one overlay means. Furthermore, for exampleArtificial intelligence models can be created that take into account typical brightness changes (e.g., due to changes in brightness throughout the day) and / or changes in weather conditions during live sports broadcasts during the transmission of video data. Overlay devices can thus be generated and / or controlled based on such a model.

[0029] Additionally, the at least one overlay device can also be designed to be changeable through manual input. This allows, for example, a user to correct an automatically configured setting of the overlay device. Such manual settings can also be recorded by the overlay manipulator and taken into account as part of a model of user behavior.

[0030] The at least one overlay means can in particular be encoded with the image data in a stream. In this case, the overlay means can already be integrated / encoded into the image data stream, so that this then does not occur in the terminal device and / or the display device, but before the distribution of the image data, e.g. by a streaming provider. The overlay manipulator can evaluate the brightness values ​​in the image data and then create an adapted overlay means for the image data, which is then encoded with the image data. In this case, the overlay means and the image data cannot easily be changed separately from one another later. In particular, this encoding can be carried out image by image. This creates another, more energy-efficient version of the image data, in particular the video data.

[0031] The presented image data can be configured using the at least one overlay means to compensate for a visual impairment, in particular a color recognition impairment, wherein the compensation can also be processed adaptively using the terminal device and / or the display device. A user of a display device can, for example, set the overlay means so that certain properties of the image data are adapted to their personal characteristics or preferences. If a user regularly makes such adjustments to their display device, the overlay manipulator can, for example, use artificial intelligence to recognize a pattern in the settings, so that the system learns to automatically adapt to the user.

[0032] With the at least one pixel-precise overlay means, the transparency, color, color model, brightness, contrast, shape, and / or size of each individual displayed image can also be completely or partially changed independently of previously or subsequently displayed images. The subject matter of claim 20 also addresses the aforementioned problem.In this case, a) at least one property and / or one setting parameter of the image data or pixels, of the terminal device and / or of the display device and / or of the environment of the terminal device and / or of the display device is determined by a control unit, and b) with the control unit, in particular with an overlay manipulator, at least one program-controlled overlay means is generated which completely or partially covers the pixels of the image, wherein the transparency, the color, the color model, the brightness, the contrast, the shape and / or the size of the at least one overlay means depends on at least one property of the image data or pixels, a property of the terminal device and / or of the display device and / or at least one state variable in the environment of the terminal device and / or of the display device and / or at least one state variable on the terminal device itself and / or on the display device.

[0033] A computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method or the steps of the method, also addresses the problem.

[0034] A device for processing image data that can be displayed as an image on a terminal device and on a display device, wherein a display of the pixels of the image can be changed entirely or partially by at least one program-controlled means, also addresses this problem. The transparency, color, color model, brightness, contrast, shape and / or size of the display of the pixels of the image can be changed, in particular across the board, depending on a manually entered and / or automatically determined specification for reducing energy consumption. In particular, the at least one means can be encoded with the image data, so that a further, more energy-efficient version of the image data, in particular video data, is created.

[0035] Embodiments are described by way of example in connection with the figures.

[0036] Fig. 1 shows a comparison between an original image and two images with areal global overlay means;

[0037] Fig. 2 shows a comparison between an original image and an image with areal local overlay means:

[0038] Fig. 3 is a schematic representation of an embodiment of the transmission of image data to a terminal, the detection of state variables at the terminal and the use of overlay means for influencing the brightness of a display device of the terminal;

[0039] Fig. 4 shows an embodiment of a generation of overlay means for image data, in particular video data;

[0040] Fig. 5 shows an embodiment in which overlay means are encoded into an image data stream;

[0041] Fig. 6 shows an embodiment in which overlay means are encoded in an image data stream using data from a terminal device;

[0042] Fig. 7 shows an embodiment with a generation of areally local overlay means for image data;

[0043] Fig. 8 shows an embodiment with a generation of a global area overlay means for image data;

[0044] Fig. 9 shows a comparison between an original image and an image with a global area overlay; Fig. W shows the relationship between the performance of a device and the brightness of a display.

[0045] The devices and methods described below by way of example for adapting image data 1 displayed on or with a terminal 30 overlay or modify, dynamically or manually controlled, possible screen contents in such a way that energy consumption savings are achieved, in particular due to the reduction of brightness values ​​(luminance) of the screen data to be displayed on the display. The image data 1 is also referred to below as video data when it is a moving image.

[0046] For this purpose, any number of overlay means 21 (see Figures 1 , 2), which are initially barely or not visually perceptible, are generated, which then form overlays for the actual video data (masking of the video data 1 ).

[0047] These overlay means 21 are then partially or completely modified in suitable gradations such that a darkening of the video data 1 to be displayed is created entirely or partially. In one embodiment, the original video content 1 can remain unchanged, but the result to be displayed can be changed by the created or superimposed overlay means 21 with regard to the resulting brightness to be displayed. In this case, the video data is not irreversibly changed by the overlay.

[0048] Figure 1 schematically illustrates how a areal overlay means 21 is superimposed globally, ie, completely, on an image 1 (image data 23', 23" with overlay means). Figure 1 schematically illustrates that the overlay means 21 can have different brightnesses, which lead to different darkening of the image 23', 23".

[0049] In Figure 1, an original image 1 is shown on the left. In the middle, an image 23' is shown in which the entire image has been reduced in brightness (or transparency) by 10% using a global overlay means 21. On the right, the same image 23" is shown with a darkening (or reduced transparency) of 20%. The overlay means 21 can also be designed to be variable over time, for example if the brightness values ​​in the image change. Other parameters, such as the environment of the screen, can also play a role in adjusting the transparency of the overlay means 21.

[0050] However, such a global overlay is not mandatory. Figure 2 shows an embodiment in which a areal overlay means 21 only partially covers image 1; a areal local overlay is present. This is illustrated in further embodiments below.

[0051] Figure 2 thus shows a similar embodiment to Figure 1. However, the brightness of the original image 1 is not changed as a whole—i.e., globally across the entire area. It is also possible to darken individual areas of the image in a customized manner. This is achieved by the local overlay means 21A, 21B, which—as in the embodiment of Figure 1—also each have a different brightness / transparency.

[0052] The function of the overlay means 21 is not limited to darkening. In principle, other properties of the image data 1 can also be specifically changed, such as color values ​​or contrast values. For example, in the case of poor eyesight, in particular color recognition impairment, the displayable image data 1 can be changed so that it is more easily recognizable or readable for a person. The change can in particular also be carried out adaptively using the terminal 30 or a display device, so that, for example, a series of test images is shown using a television or a smartphone, which are examined by a viewer. Overlay means 21 are then adaptively calculated from the responses. This will be described in more detail below, e.g. in Figure 3.

[0053] One possible implementation of an overlay means 21 uses a JavaScript library (hereinafter also referred to as GreenView), which uses, for example, an HTTP REST API to achieve energy savings for playing web-based video data 1 on the terminal device 30. Such a library can also be written in other languages ​​(e.g., Java, Swift), depending on the operating system (e.g., Android, iOS) used.

[0054] The library can support 22 HbbTV or a native or web-based player (e.g. HTML 5) as platforms or players, but is not limited to these platforms.

[0055] One way to achieve energy savings is, in particular, to reduce the brightness on the display of the terminal device 30 using the overlay means 21 in a manner that is barely perceptible to the viewer of the video data 1. Additionally, a user can also set an eco mode on the terminal device 30.

[0056] When the above-mentioned library is used, a layer (i.e., the overlay means 21) is placed over the display of the video data 1 to reduce the brightness of the display 30. This overlay means 21 controls the opacity to adjust the brightness. The HTML 5 command can be designed, for example, as follows:

[0057] Div {opacity: 0.5;}

[0058] This command can be used to reduce the opacity (transparency) to 50%, resulting in a reduction in brightness. In the case of Java code for an Android system, the opacity can be set as follows:

[0059] PlayerView playerView = findViewByld(R. id. playerView);

[0060] View overlayView 0 findViewByld(R.id.overlayView); overlayView.setAlpha(0.5); which would cause a change in opacity—and thus brightness.

[0061] If the library is to be used in HTML code, it can be included as follows:

[0062] 1<script type="text / javascript" src=" / v1 / greenview.js">< / script>

[0063] In the application code of the player 22 (see Figure 3), a global overlay agent 21 can then be called as follows:

[0064] 1 var green View = new Green View(player, meta); / / player and meta are optional This initializes the library and uses the overlay tool 1 (GreenView). The argument player can be used to specify the player 22 used. This could, for example, be a reference to a <video>object or a reference to a JavaScript object, e.g. for a web-based browser.

[0065] This library can be used by players 22 and apps, using APIs implemented by the app to control (on, off, auto, etc.) GreenView (i.e., the overlay device) 21. Thus, the external video players 22 remain functionally unchanged.

[0066] A local overlay means 21 A, 21 B can be realized in a similar way, for example by specifying the shape of the local overlay means 21 A, 21 B (e.g. rectangle, circle, geometric shape, etc.), which is then placed in image 1 by means of an x,y coordinate.

[0067] Figure 3 schematically illustrates the transmission of video data 1 to a terminal 30, the acquisition of parameters 10, 12 of the terminal 30, the acquisition of state variables 15 around the terminal 15 and the use of overlay means 21 for influencing the brightness of a display of the terminal 30 using an example.

[0068] The video data 1 is provided by a provider 2, such as a broadcaster or a streaming service, and the video data 1 is transmitted to an overlay manipulator 20. This can, for example, analyze the content of the video data 1 and generate an overlay means 21.

[0069] In this embodiment, the overlay manipulator 20 also receives parameters 10, 12 from the terminal device 30 (here, a television), which are used to generate the overlay means 21. Additionally or alternatively, parameters 10, 12 from a device coupled to the terminal device 30 and / or state variables 15 from the environment of the terminal device 30 can also be received and processed, which will be described below.

[0070] The parameters include, for example, metadata 10, which is recorded or already known regarding the end device 30 or the video data 1. The metadata 10 can, for example, describe or classify the program and thus the video content 1. The metadata 10 can also include, for example, information about the end device 30 (e.g., user agent with, if applicable, information about the device size, model number, etc.) and thus also about the individual display technology, which can be used to link to created model libraries of "measured" or analyzed devices.

[0071] Further metadata 10 may include, for example, information from the video codec used, the video format, the transmission medium, the usage context and / or the program metadata.

[0072] User data can include metrics and / or user interactions 12 collected in the video player / on the end device 30, for example, providing data on playback behavior and allowing conclusions to be drawn about the usage situation (content awareness). The user data can also include movement patterns of one or more users in the vicinity of the end device 30.

[0073] A machine learning device 11, which can be coupled to the overlay manipulator 21 (see also Figure 4), can analyze the metadata 10, the user data 12, and / or the recorded state variables 15, for example, to infer certain usage patterns of the device user. For example, a device user will have different brightness requirements in a brightly lit room than in a dark room.

[0074] The state variables 15 include, for example, the temperature, the brightness in the environment of the terminal device 30, movement patterns in the environment of the terminal device 30, and / or the local time of the terminal device 30, which are recorded via sensors or timers on-site at the terminal device 30. AI-supported analysis processes can also be used here, using usage data and / or content-related data of the respective offer or service. It is also possible to perform camera-supported / sensor-based recording of the screen brightness.

[0075] In particular, in this embodiment, energy data of the terminal device 30 and / or the environment of the terminal device 30 are also evaluated by the overlay manipulator 20. For example, it is possible to determine how much energy is required to display the video data 1 on the terminal device 30. For example, power consumption measurements and the metadata 10 collected during the measurement can be used to create models that help determine or estimate the power consumption even for terminal devices 30 that cannot be measured directly. This is done, for example, based on the content of the video data 1 to be displayed, the environmental situation (if data is available) of the terminal device 30, and the type of terminal device 30 (e.g., via the user agent of the terminal device 30). The creation and evaluation of the models can be carried out with the aid of artificial intelligence 11 in the overlay manipulator 20.

[0076] Since different display technologies behave differently, special models may be necessary.

[0077] The provider 2 can use the information on the energy consumption of the end devices 30 / display devices, for example, in the context of a CSRD reporting 3, e.g. in the context of an emissions reporting (CSRD Scope 3 - emissions from the use of products).

[0078] The overlay manipulator 20 uses these input data 10, 12 and, in particular, the state variables 15 generated or acquired on and / or in the terminal device 30 / display device to generate overlay means 21. These overlay means 21 can, as explained above, be local and / or global overlay means 21.

[0079] These overlay means 21 are then transmitted to a streaming app 22 (player), which displays the overlay means 21 together with the image data 1 on the terminal device 30.

[0080] This separation of Player 22 and Overlay Manipulator 20 also ensures that existing Player 22s do not need to be modified.

[0081] In principle, in this embodiment, as in the embodiments presented below, the terminal 30 can be a display device, such as a screen or a projector. However, it can also be the case that the terminal 30 itself does not have a display device, such as a set-top box or a streaming device. Such a terminal 30 is then coupled to a display device for displaying the image data 1. The overlay means 21 are always displayed on the display device in order to reduce the brightness there. The generation of the overlay means 21 by the overlay manipulator 20, on the other hand, can, for example, basically take place in a terminal 30, a display device and / or during the generation of the image data 1. Some variations of this topology are presented below, in which the video data 1 are linked to the overlay means 21 in different ways.

[0082] Figure 4 shows an overlay manipulator 20 which processes different input data 10, 12 to calculate overlay means 21, as already described in connection with Figure 3, so that reference can be made to the above description.

[0083] A machine learning device 11, which is designed separately from the overlay manipulator 20 here, can analyze the metadata 10, the user data 12 and further analytical data 15 (e.g. the state variables 15 determined on the terminal device 30 / display device) in order to, for example, draw conclusions about certain usage patterns of the terminal device user. For example, a terminal user in a brightly lit room will have different brightness requirements than in a dark room. The state variables 15 are preferably recorded in the area of ​​the display devices, since these are the devices with which users primarily interact. This can be a screen that functions as the terminal device 30. However, this can also be a screen that is supplied with image data 1 from a streaming device.

[0084] The overlay manipulator 20 uses these input data 10, 12, including the state variables 15, determined on and / or in the terminal 30 / display device, and generates an overlay means 21 therefrom in the embodiment according to Figure 4. Unlike in the embodiment according to Figure 3, no direct feedback is provided here between the terminal 30 and the overlay manipulator 20. The input data 10, 12, 15 are determined separately here and made available to the overlay manipulator 20.

[0085] The overlay means 21 is then processed in the player 22 (application program) together with image data, i.e., the video data 1. This means that images 23 of the video data 1 are provided with the overlay means 21 (here, superimposed).

[0086] Here, it is initially assumed that each image of the video data 1 is provided with a global, areal darkening overlay 21, so that video images 23 are obtained with the overlay 21. Thus, in this embodiment, the overlay 21 does not directly modify the video data 1, so that after the overlay 21 is removed, the video data 1 would return to its unchanged state. The degree of possible reduction in brightness and thus the design of the brightness-reducing overlay 21 can be controlled and influenced by the various technical influencing factors mentioned. The embodiment provides a device and a method in which the overlay 21 is calculated only as a function of the input data 10, 12, 15 and then applied to normal image data 1. However, it is also possible to encode the overlay 21 directly into an image data stream 1 (see embodiments according to Figures 5 and 6).

[0087] The achieved reduction in brightness leads to a reduction in energy consumption of the terminal device 30, in particular the screen or projector. At the same time, the degree of the desired modification of the resulting brightness is defined by the various parameters 10, 12, and 15: The degree of visually perceptible dimming, the respective usage situation, and the so-called quality of experience (perceived service quality) are influencing factors.

[0088] In the embodiment according to Figure 5, the overlay manipulator 20 receives video data 1 and evaluates its brightness values ​​and then generates areal local and / or global overlay means 21, which then together result in video data 23 with overlay means 21, which can be displayed on a terminal 30 or with a terminal 30 and its display device.

[0089] The embodiment according to Figure 6 represents a variant of this procedure, wherein the overlay manipulator 20 additionally receives input data 10, 12, 15 that were acquired independently of the image data 1, as described in connection with Figure 3 or 4. Unlike in the embodiment according to Figure 4, the overlay means 21 are calculated as a function of the video data 1 and additionally the input data 10, 12, 15.

[0090] Figures 3, 4, 5, and 6 describe areal overlay means 21, meaning that the overlay means 21 have a specific global or local shape to reduce brightness. However, it is also possible to vary the overlay means 21 over time.

[0091] Thus, both image data 1 of the entire video and image data 1 in partial areas of the video can be modified in their displayed brightness (see Figure 7: Brightness Manipulator with local modifications and Figure 8: Brightness Manipulator with global modifications). The overlay means 21 can be used for each individual image 1 (as shown in Figures 7 and 8) or extended over time for a group of images 1.

[0092] As a result, the resulting video content 1 can also be optimized for energy-saving playback, or additional versions of the content can be created, for example, in the form of additional renditions for adaptive bitrate streaming formats such as HLS or MPEG DASH. These "green" or energy-saving representations of the content are transmitted jointly, in addition to, or exclusively to the actual content, to end devices 30 or stored on the server 2 and can be selected automatically, for example, by the respective player 22, or manually by, for example, the user or content provider, for energy-saving playback.

[0093] The embodiments described so far have in common that the display of the images 1 is changed completely or partially by at least one program-controlled overlay means 21, wherein, for example, the brightness depends on at least one property of the image data 1 (see embodiments according to Figures 5 and 6) and / or on input data 10, 12 and state variables 15 in the environment of the terminal 30 / display device and / or in the terminal 30 / display device itself (see, for example, embodiment according to Figures 3, 4).

[0094] In addition to or alternatively to the brightness, the transparency, the color, the color model, the contrast, the shape and / or the size of the at least one overlay means 21 can depend on at least one property of the image data 1, of pixels on the terminal device 30 / display device and / or of state variables 15 in the environment of the terminal device 30 / display device and / or of state variables 15 in the terminal device 30 / display device itself.

[0095] In addition to the automatic adjustment of the resulting brightness to be displayed, for example, controlled by the listed parameters and mechanisms in the embodiment according to Figures 3, 4, 5, and 6, in other embodiments, a manual adjustment of the resulting brightness to be displayed, initiated, for example, by the user or service provider 2, is also possible through the application of the described invention. This is described in the embodiments according to Figures 7 and 8.

[0096] Figure 7 shows a variant in which image data 1 is provided with one or more overlay means 21 A, 21 B, 21 C, 21 D via an overlay manipulator 20 with a brightness manipulator. In this case, for example, the shapes and sizes of the overlay means 21 A, 21 B, 21 C, 21 D can be adapted to the area to be darkened. If an area of ​​an image is particularly bright, a darker overlay means 21 B can be used, for example. The shapes of the overlay means 21 A, 21 C can be designed as rectangles, which simplifies their data processing. The overlay means 21 D can also have an amorphous, complex shape that corresponds to the shape of the area to be darkened in the image 1. Since the brightness values ​​in a video stream 1 change from frame to frame, the shape, size and / or number of the local overlay means 21 A, 21 B, 21 C, 21 D will change, possibly from frame to frame.

[0097] In Figure 7, the overlay means 21 A, 21 B, 21 C, 21 D are shown as being homogeneous in terms of area. However, the overlay means 21 A, 21 B, 21 C, 21 D can also be inhomogeneous in terms of area, for example if the darkening by the overlay means 21 A, 21 B, 21 C, 21 D varies within the area of ​​the overlay means 21 A, 21 B, 21 C, 21 D. For example, there can be a gray gradation in the overlay means 21 A, 21 B, 21 C, 21 D, so that lighter areas can be darkened more than darker areas. It is not necessary for the overlay means 21 A, 21 B, 21 C, 21 D to be achromatic, i.e. black / white or in grayscale. It is also possible for the overlay medium 21 A, 21 B, 21 C, 21 D to feature colors, color patterns, or color gradients. This can, for example, prevent sharp transitions between image areas with and without overlay medium 21 A, 21 B, 21 C, 21 D and / or mask unnecessarily bright areas, e.g., those caused by a point light source.

[0098] In Figure 8, the global brightness of an image of the video data 1 is adjusted by the user using an overlay means 21.

[0099] However, the desired technical effect of reducing the resulting brightness of the video content to be displayed with the embodiments presented here can already be achieved during encoding of the video content by modifying the encoding parameters, such as brightness. This is illustrated, for example, in Figures 5 and 6.

[0100] Figure 9 shows an original image on the left, and the same image with a global overlay (21) on the right. The darkening of approximately 10% has a noticeable impact on energy consumption. Despite the darkening caused by the overlay (21), the image content is still clearly visible, which is sufficient for many applications.

[0101] The energy consumption (or power consumption) of a terminal device 30 depends in particular on the brightness with which the images 1 are displayed on the display device. Figure 10 shows the dependence of power [W] on brightness for different applications. This shows that reducing the brightness using an overlay means 21 has a significant effect on energy consumption. However, brightness is only one parameter for influencing energy. The color, color model, contrast, shape, and / or size of the at least one overlay means also influence energy consumption.

[0102] In the embodiments presented so far, the focus was on determining overlay means 21 for a specific image within the image data 1 during the entire image data stream. However, it is possible to use the overlay means 21 for one or more temporal sub-ranges of the image data 1 (referred to as local in time). This can be useful, for example, if a sensor detects the lighting conditions at a terminal device 30 and, for example, detects a sudden increase in brightness in the room of the terminal device 30. It can then be useful to adaptively change one or more properties of the overlay means 21 depending on the change in brightness. If the terminal device 30 is located outdoors, for example, properties of the displayed image data can be automatically adapted to the daylight by overlay means 21. It is also possible to use the overlay means 21 for the entire period of the image data (globally in time).

[0103] For example, a laser projector can be used as the terminal device 30, with the laser projector acting as a display device. This display device can then, for example, experience changes in brightness and / or changes in the movement patterns of users outdoors. For the projection of the image data 1, it makes a difference, for example, whether the projection takes place for many users in a dark environment or for a few users in a bright environment. The overlay means 21 can be formed by controlling the lasers in the projector, so that certain areas of the projection are specifically darkened, according to the recorded parameters 10, 12 and / or the state variables 15.

[0104] Possible technical applications:

[0105] • Any type of presentation of moving image content on displays

[0106] • For example, video streaming (live, VoD) on SmartTV, tablet, smartphone, PC etc., linear TV / broadcast, hybrid TVs (such as HbbTV).

[0107] Reference symbol

[0108] 1 Image data, video data

[0109] 2 streaming service / broadcaster

[0110] 3 CSDR reporting

[0111] 10 Metadata

[0112] 1 1 Machine learning for end device data

[0113] 12 User data

[0114] 15 State variables in the environment of the terminal device

[0115] 20 Overlay manipulator

[0116] 21 Overlay agents

[0117] 21 A local overlay agent

[0118] 21 B local overlay agent

[0119] 21 C local overlay agent

[0120] 21 D local overlay agent

[0121] 22 Player / Application Program

[0122] 23 Video data with overlay medium

[0123] 30 terminal device, display device, display< / video>

Claims

Claims 1. Device for processing image data (1), in particular video data, which can be displayed as an image, in particular an image sequence, on a terminal device (30) or with a display device of a terminal device (30), wherein a display of the pixels of the image, in particular of the image sequence, can be changed in whole or in part by or with at least one program-controlled overlay means (21), characterized in that the transparency, the color, the color model, the brightness, the contrast, the shape and / or the size of the at least one overlay means (21) depends at least on one property of the image data (1) of pixels in the terminal device (30) and / or in the display device and / or at least one state variable (15) in the environment of the terminal device (30) and / or the display device and / or at least one state variable (15) in the terminal device (30) and / or in the display device itself.

2. Device according to claim 1, characterized by an overlay manipulator (20) with which the transparency, the color, the color model, the brightness, the contrast, the shape and / or the size of the at least one overlay means (21) can be controlled, wherein the control is dependent on at least one property of the image data (1), of pixels in the terminal (30) and / or in the display device, and / or at least one state variable (15) in the environment of the terminal (30) and / or the display device, and I or at least one state variable (15) in the terminal (30) and / or display devices themselves.

3. Device according to claim 1 or 2, characterized in that the property of the image data (1) or the pixels is a brightness, a contrast, a color and / or information about the design of the terminal device and / or the display device.

4. Device according to at least one of the preceding claims, characterized in that the at least one overlay means (21) is designed as a homogeneous surface which is achromatic or colored.

5. Device according to at least one of claims 1 to 3, characterized in that the at least one overlay means (21) is designed as an inhomogeneous surface which is achromatic or colored.

6. Device according to at least one of the preceding claims, characterized in that the energy consumption of the terminal (30) can be reduced by the use of the at least one overlay means (21).

7. Device according to at least one of the preceding claims, characterized in that the terminal (30) comprises or is designed as a television, a screen, a large screen in the interior, an out-of-home screen, a projector and / or a mobile terminal, in particular a computer, a tablet computer or a smartphone.

8. Device according to at least one of the preceding claims, characterized in that the terminal (30) has a local device for encoding and / or decoding image data (1), in particular a set-top box or a streaming device.

9. Device according to at least one of the preceding claims, characterized in that the at least one overlay means (21) can be generated and / or changed with the overlay manipulator (20), wherein the overlay manipulator (20) is designed to be functionally separate from a player (22) for the image data (1).

10. Device according to at least one of the preceding claims, characterized in that brightness values of the image data (1) can be adjusted by the at least one overlay means (21), in particular the displayed brightness of video content (1) is completely or partially lower than without the at least one overlay means (21). 1 1 . Device according to at least one of the preceding claims, characterized in that the at least one overlay means (21) does not irreversibly change received image data (1), in particular video data.

12. Device according to at least one of the preceding claims, characterized in that the at least one overlay means (21) can be used globally in time for the entire duration of a video unit, in particular of an image data stream (1), or locally in time only for one or more temporal sub-areas of the video unit.

13. Device according to at least one of the preceding claims, characterized in that the at least one overlay means (21) can be used for an image data stream (1) and / or globally for the entire image (1) or locally for only one or more partial areas of the image (1).

14. Device according to at least one of the preceding claims, characterized in that the at least one overlay means (21) can be created based on automatically recorded data and / or on automatic analyses (10, 11, 12, 15) of a usage situation on the terminal device (30) and / or on the display device, in particular using artificial intelligence (11).

15. Device according to claim 14, characterized in that the overlay manipulator (20) is configured to receive data on the energy consumption, in particular the power consumption of at least one terminal device (30) and / or display device, wherein these data are based on measurements or estimates and the at least one overlay means (21) can be generated as a function of these data on the energy consumption.

16. Device according to at least one of the preceding claims, characterized characterized in that the at least one overlay means (21) can also be changed with manual inputs.

17. Device according to at least one of the preceding claims, characterized in that the at least one overlay means (21) is encoded with the image data (1), so that a further, more energy-saving version of the image data (1), in particular video data, is created.

18. Device according to at least one of the preceding claims, characterized in that the imaged image data (1) compensates for a visual impairment, in particular a color recognition impairment, by means of the at least one overlay means (21), wherein the compensation can also be processed adaptively by means of the terminal device (30) and / or a display device.

19. Device according to at least one of the preceding claims, characterized in that the transparency, the color, the color model, the brightness, the contrast, the shape and / or the size of the displayed image can be changed completely or partially by the at least one pixel-precise overlay means (21).

20. Method for processing image data (1) displayed as an image on a terminal (30) and / or in a display device, wherein a) at least one property and / or one setting parameter of the image data (1) or pixels for the terminal (30) and / or the display device and / or the environment of the terminal (30) and / or the display device is determined by a control unit, and b) the control unit, in particular with an overlay manipulator (20), generates at least one program-controlled overlay means (21) that completely or partially covers the pixels of the image, wherein c) the transparency, the color, the color model, the brightness, the contrast, the shape and / or the size of the at least one overlay means (21) depends on at least one property of the image data or pixels, a property of the terminal device (30) and / or the display device and / or at least one state variable (15) in the environment of the terminal device (30) and / or the display device and / or at least one state variable (15) on the terminal device (30) and / or on the display device itself.

21. A computer program product comprising instructions which, when executed by a computer, cause the computer to carry out the method / steps of method claim 20.

22. Device for processing image data (1) which can be displayed as an image on a terminal (30) and / or a display device, wherein a display of the pixels of the image can be changed in whole or in part by at least one program-controlled overlay means, characterized in that the transparency, the colour, the colour model, the brightness, the contrast, the shape and / or the size of the display of the pixels of the image can be changed, in particular in a general manner, depending on a manually entered and / or automatically determined specification for reducing energy consumption.

23. Device according to claim 22, characterized in that the at least one means is encoded with the image data (1 ) so that a further, more energy-efficient version of the image data (1 ), in particular video data, is created.

Images