Image transformation apparatus, default demapping function data providing server, method of obtaining a first image (hdr_pred) of lumas for a first dynamic luminance range from the second lumas image (ldr_cont) for a second dynamic luminance range

Abstract

: - an input (204) in a data distribution system (205) comprising the low dynamic range image; - a data-protected input (206) in at least one predefined mapping algorithm data (gam); and - a control unit (202) arranged to manage access to the predefined mapping algorithm data under the control of an artistic content owner in the low dynamic range image, allowing for better control of the authorized regularity of the content when it is to be rendered, which is possible due to the special HDR image encoding strategy. 1 / 1

Description

1 / 38 IMAGE TRANSFORMATION DEVICE, PREDEFINED MAPPING FUNCTION DATA SUPPLY SERVER, METHOD FOR OBTAINING A FIRST IMAGE (HDR_PRED) OF LUMAS FOR A FIRST LUMINANCE DYNAMIC RANGE FROM THE SECOND IMAGE (LDR_CONT) OF LUMAS FOR A SECOND LUMINANCE DYNAMIC RANGE FIELD OF THE INVENTION

[001] The invention relates to apparatus and methods and resulting products such as products for storing or transmitting encoded data or signals, to enable improved secure distribution of high dynamic range images or video. HISTORY OF THE INVENTION

[002] Recently, image capture, display, and in particular image encoding have improved from so-called low dynamic range (LDR) image formation (such as classic systems like PAL or MPEG2) to so-called high dynamic range (HDR) image formation. Illuminances in nature can vary from 100,000 lux in sunlight, across typical office or room lighting around 500 lux, to, for example, 0.05 lx under a quarter of the moon's brightness. Luminances (L) in the world vary from 1 billion nits of the solar disk, to 10,000 nits for lamps, to a couple of ten thousand nits for objects in sunlight (such as a building, cloud rims, or a blank sheet of paper), to Petition 870220086234, dated 09 / 21 / 2022, page 11 / 22 2 / 38 hundredths or 10s of a nit for objects under a (heavily) cloudy sky or indoors, to 0.1 nit for a blank sheet of paper under moonlight, etc. This does not necessarily mean that one must render these luminances on a display in exactly the same way; instead, the image should look artistically good, which means at least that there should be roughly similar appearance differences for the regional luminances of objects when rendered on the display screen. It should be understood that tone mapping for rendering on a particular display is, with the many displays that exist today, decoupled from any capture or encoding, leading to three linked representations.In general, a requirement for HDR image formation—to be able to render, for example, a bright white wall differently from an adjacent luminous lamp on a display—is that their respective pixels are also encoded with different luma (Y) values. Sensors or cameras are becoming more powerful in this regard; in fact, they can accurately capture most of these many different luminances in the world (whether with larger images in the background, differently exposed, etc.), and for simplicity, their native color representation will be considered to be a linear luminance encoding within [Lmin, Lmax] + chromatic information. One can then use an entire and arbitrarily specified definition (according to the desired course requirements such as, for example, the ability to process the encoded information later, such as local brightening, or data compression concerns, etc.).) for the transmission codec. Petition 870220036367, dated April 28, 2022, page 24 / 115 3 / 38 Finally, this encoded data (Y_C1C2, or similar) can then be converted again in many ways to a rendering-side representation, which for simplicity can be equated to the driving values ​​for, for example, LCD pixel colors. Displays are getting more renderable dynamic range so that they can first render brighter regions, and secondly, simultaneously or successively, darker regions. This allows placing all these various luminance objects along the renderable range with optimal rendered output colors.

[003] A considerable problem for content creators is that, despite their efforts to protect their content so that it can be properly sold instead of being illegitimately copied, all sorts of counterfeiters distribute the content, sometimes in large numbers. When designing a new image encoding standard, in addition to looking at how to represent pixel colors from, for example, a data compression point of view, one can simultaneously have as an object the engineer in new requirements, such as the protection of the standard. They came up with a way to encode HDR images in such a way that they are actually represented as LDR images, as well as a set of transformation strategies to recover all the HDR effect information encompassed in that document, back into the original HDR image. One can then lessen the restriction on content protection somewhat.The image data (texture / pixel) itself can be hacked, but then the forger has a poor image, not the original maximum HDR rating. A. Petition 870220036367, dated April 28, 2022, p. 25 / 115 4 / 38 transformation strategy – which in structure is simultaneously a maximum HDR graded image cocoding and an older usable LDR variant – can then be strongly protected, for example, this other half of the HDR information in the new codec can be securely transmitted only during viewing (and, for example, never leaves the hardware of an image processing IC if it also moves along the decrypted data connections, and is deleted from within the IC's temporary memory as soon as possible). This would allow applications better control when the content creator wants to sell HDR movies, for example, they can actually sell only the HDR recovery functions or algorithms. SUMMARY OF THE INVENTION

[004] The objects are realized through an image transformation device (201) arranged to originate a first image comprising lumas that are encoded to produce on a display of a first dynamic range of the desired correct luminances (e.g., not too bright facial colors, or boosted lights, etc.), the first image, for example, is a high dynamic range image (HDR_PRED) for a display of 4000 nit, or, for example, a peak white reference image of 2000 nit when starting from an input image set relative to a higher peak white reference, the peak white reference defines what these lumas mean, where the rendered luminances should be correct if displayed on such a display in reality that has a physical peak white reference of said 6000 nit. Petition 870220036367, dated 04 / 28 / 2022, page 26 / 115 5 / 38 corresponds to the white reference peak that corresponds to that of the input image data was defined, from a second dynamic range image (LDR_CONT), for example, being a lower dynamic range image, for example, 100 nit in the case of up-conversion of luminance from a lower dynamic range input image, but this could also be an input image defined by higher dynamic range, for example, a maximum graded signal with a reference peak brightness of 6000 nit, in which case the mapping is a down-mapping to an ideal image for, for example, a physical display with a white peak of 2000 nit, wherein the achievement comprises the mapping of pixel lumas in the low dynamic range image to pixel lumas in the high dynamic range image through the application of at least one predefined mapping algorithm (gam), the image transformation apparatus comprising: - an input (204) in a data distribution system (205) comprising the low dynamic range image; - a secure data entry (206) for at least one given predefined mapping algorithm (gam); and - a control unit (202) arranged to manage access to predefined mapping algorithm data under the control of an artistic content owner in the low dynamic range image.

[005] You may want the existence of the predefined data mapping (gam) algorithm to be as rigorous and intermittent as possible, so that it is difficult to access, and even when copied, the realizations are Petition 870220036367, dated 04 / 28 / 2022, page 27 / 115 6 / 38 will make it more difficult in reality to use this data after rendering, since the second phase is still present. There may be several scenarios where the second image is (although it may contain the same information as the pixel texture) of lower quality due to the impact on brightness rendering. For example, one might have a situation where, using mapping parameters, a lower quality LDR image (e.g., according to any standard that refers luma codes to a 100 nit white peak) is converted into a high-quality HDR image, where, for example, laser beams might have high brightness, reflections on water might optimally shine when rendered on higher dynamic range displays, etc.But also when the second image is already a higher quality image, for example, a color encoding referring to a maximum white peak of 6000 nits (that is, theoretically ideal for rendering on a real display with a white peak of 6000 nits, but not much higher or lower), then the mapping parameters for mapping can be used to map a display of, for example, 2000 nits or around that white peak, and, for example, the second mapping parameters can be used to obtain a (more) realistic rendering image on displays in a range of 2500-3500 nits of white peak, that is, around 3000 nits. The principles below will be described non-limitingly with an example of LDR to HDR conversion; the technician will clearly understand how the same technology then works in the other direction of color / luma mapping. That is, various implementations can always verify that everything is normal and authorized after each use of the mapping data.That is, this system does it. Petition 870220036367, dated 04 / 28 / 2022, p. 28 / 115 7 / 38 regular use is not so difficult, unless of course one builds entire systems, but users normally buy a normal image transformation device, or a display, which can then be made to behave strictly according to the authorized method. Therefore it can even attenuate the system, in which, in some embodiments, the predefined mapping algorithm data (gam) may be relatively and freely available, but the regularity of the situation is still checked by the image transformation device (201) during or before each rendering.Owner control can be achieved in various ways (in fact, the owner doesn't necessarily have to be the copyright holder themselves, as control may have been delegated, but someone or some device working for someone's benefit has to provide the progression of this mapping data so that HDR can be released), but the control unit has to verify it.For example, the owner may indicate that this content is free (this could be, for example, information that has been previously transmitted and stored in the image transformation device's memory), in which case the mapping data can be retrieved from any server or source (in such scenarios, you may only need to connect to the film production company's standard server to see that this film title is free (and any additional information relating to it can be obtained free of charge, without any further interference, after this verification). Of course, more complicated specifications of what rights one has regarding the image content (e.g., which servers...) Petition 870220036367, dated 04 / 28 / 2022, page 29 / 115 8 / 38 or entities authorized to provide the mapping data) can also be specified by the server, or, for example, such wishes of the owner when pre-recorded in the image transformation device's memory from a previous communication session. For example, a set of handling rules can be specified to copy the pre-loaded mapping data to another user's device in the same domain (e.g., when connected for immediate verification of aspects with the image transformation device), for example, that this second device should also perform some checks with the server, whether it is synchronized with the image transformation device's communication or not, etc.

[006] Advantageously, an image transformation device (201) has the control unit (202) arranged to obtain at least the predefined mapping algorithm data through secure data input (206) within a time interval around the rendering time on a high dynamic range (HDR_PRED) image display. By removing all traces of the mapping algorithm, even from hidden memory within the hardware, it can increase security for some scenarios. For example, one cannot reverse engineer any device.

[007] Advantageously, an image transformation device (201) has the control unit (202) arranged to form a secure and private connection, via a network, such as the internet, with a server (207) of the owner of the artistic content. Although many systems do not need such rigorous security, communication measures can be implemented that prevent eavesdropping, even if Petition 870220036367, dated 04 / 28 / 2022, p. 30 / 115 9 / 38 other measures make the stolen mapping data smaller or unusable, for example, because it corresponds to a particular display or image transformation device.

[008] Advantageously, an image transformation device (201) has the control unit (202) arranged to send content identification data (id_rend) that identifies at least aspects of the low dynamic range image (LDR_CONT), such as, for example, a version number or where it was purchased, and optionally, in addition to the identification data of the image transformation device (201) or an HDR display (290) or an owner of such devices. By identifying the rendering situation at each time, the server 207 increased the opportunity to assess the situation. For example, if suddenly the user, and the content and / or devices no longer match, perhaps, somehow, the content has been copied, and is being used irregularly.Of course, if the user regularly resells their content to a new owner, they may have registered it that way on the server (but then, for example, the device specifications can be checked on the first run of that content, and replaced so that the situation is not deemed inappropriate the next time).

[009] Advantageously, an image transformation device (201) has the control unit (202) arranged to store at least one piece of data from the predefined mapping algorithm in a memory (270) in an encrypted manner, which memory may be fixed or removable. Some users or scenarios may not want to check the Petition 870220036367, dated 04 / 28 / 2022, page 31 / 115 10 / 38 regularity of each rendering, but for example, once, in a so-called initialization of the content. This can be arranged, for example, by storing the mapping data, again in a way that is as secure as possible or desirable.

[010] Advantageously, an image transformation device (201) has the control unit (202) arranged to obtain characterization information of a user from the high dynamic range image (HDR_PRED), such as, for example, a personal code or biometric information. To make the system more secure, for example, by allowing the server 207 to perform more in-depth investigations, some realizations can obtain, and even only function if they obtain, various additional data regarding the rendering situation.

[011] Furthermore, the predefined mapping algorithm data supply server (207) (gam, gam_enc) is willing to receive a data connection opening from any variant of the image transformation device (201), and is willing to perform a test based on the transmitted content identification data (id_rend) from the image transformation device (201) that tests whether the low dynamic range image (LDR_CONT) is regularly purchased or obtained, and is willing to then supply the predefined mapping algorithm data (gam, gam_enc) to the image transformation device (201). The server will perform a simple or more complex test before releasing it, for single use or permanently, for this hardware rendering situation of the mapping data. Typically, at least, it wants to perform some tests. Petition 870220036367, dated 04 / 28 / 2022, page 32 / 115 11 / 38 Simple ways to determine how the content was obtained, for example, it can check if it contains an identifier stating that this content was massively provided for free, or was considered no longer subject to copyright. But the content may also have been distributed (as a teaser, for example) so that later people buy the mapping data for the HDR experience. Or the content may have been previously streamed by the 207 server itself, or a known server, etc. In more complex variants, the server may perform further tests to assess the situation, and not provide the mapping parameters if it considers the situation irregular, or initiate a process in which it offers the user to regularize the situation, for example, by providing more information making the server confident that they are a regular user and / or made a regular purchase, for example, based on a predefined questionnaire shown on the display.

[012] Advantageously, a predefined mapping algorithm data supply server (207) is willing to perform additional tests, specifically, whether the image transformation device (201) is a normal device or behaves improperly. This can be done in several ways. For example, the server can test whether it is releasing mapping data to a device manipulated by the hacker, which mimics a normal image transformation device (201) from a consumer electronics manufacturer. A simple way to do this is to establish secret identification codes, for example, that the device was built by Philips, which the code is then Petition 870220036367, dated 04 / 28 / 2022, page 33 / 115 12 / 38 sent by the control unit. Several additional tests can be performed, allowing for an estimate of whether it's a recently soldered PCB emulator, or some emulation software, etc. For example, the server can check if its communications have inserted a particular IC, etc.

[013] Advantageously, a data provision server (207) of predefined mapping algorithm (gam, gam_enc) is willing to maintain precise temporal communication with the image transformation apparatus (201) of partial information of the predefined mapping algorithm data (gam, gam_enc). If the mapping data is sent at intervals, this not only makes copying more difficult, but can also allow additional testing. For example, a copier may require that the data be sent more quickly, either if it is in a hurry not to be discovered and blocked, or if it does not have sufficient details about the compliant temporal order of data provision (e.g., send the data for the first three scenes, and then nothing until the moment before the fourth scene, or at the latest until 10 seconds before the fourth scene).Whether this is kept secret or not, not all counterfeiters may have the difficulty of building a system that emulates all these possible aspects, which means that they may have had HDR experience with the film in the past, allowing them to create good advertising teasers.

[014] Advantageously, a predefined mapping algorithm (gam, gam_enc) data provider server (207) is willing to analyze a situation on the rendering side based on the information in Petition 870220036367, dated 04 / 28 / 2022, page 34 / 115 13 / 38 at least one aspect {the low dynamic range image (LDR_CONT), the image transformation device (201), a connected HDR display (290) for rendering the high dynamic range image (HDR_PRED), the optical characteristics of an HDR display environment (290), and a user of the image transformation device (201)}, and from there determine which, if any, predefined mapping algorithm data (gam, gam_enc) will be sent to the image transformation device (201). Depending on all the desired aspects, some embodiments could send multiple data, at least for monitoring and recording how regular and irregular content is distributed, even if the provision of mapping data is simple.

[015] Aspects of the invention can also be carried out by a method of obtaining a high dynamic range (HDR_PRED) image from a low dynamic range (LDR_CONT) image, wherein obtaining comprises mapping the tone of lumas of pixels in the low dynamic range image onto lumas of pixels in the high dynamic range image, by applying at least one predefined mapping algorithm (gam), the method comprising:

[016] - obtaining a data distribution system (205) from the low dynamic range image (205); and

[017] - managing access to predefined mapping algorithm data obtained through secure data entry (206), under the control of an artistic content owner in the low dynamic range image, and a method for providing algorithm data Petition 870220036367, dated 04 / 28 / 2022, page 35 / 115 14 / 38 of preset mapping (gam, gam_enc) to transform a low dynamic range image (LDR_CONT) ideal only for LDR display rendering into a high dynamic range image (HDR_PRED) ideal for HDR display rendering, by a source of such preset mapping algorithm data (gam, gam_enc) connectable to an image transformation device (201) having a control unit (202) arranged to manage access to the preset mapping algorithm data under the control of an owner of the artistic content in the low dynamic range image, the method performed by the source testing whether the image transformation device (201) is using an authorized low dynamic range image (LDR_CONT), and under positive verification transmitting to the image transformation device (201) the preset mapping algorithm data (gam, gam_enc).

[018] Advantageously, the method of obtaining a high dynamic range image (HDR_PRED) makes a secure and private connection over a network to a server (207) in a time interval around the rendering time on a display (290) of the high dynamic range image. Especially if the mapping data become immediately deep within (tamper-protected) ICs, and are destroyed as soon as they are no longer needed, this can make the systems very secure. But one can, of course, vary in the different components and protection measures, to make the management systems less rigid for the mapping data and the resulting ways in which one can or cannot create the HDR image(s).

[019] Advantageously, the method of obtaining Petition 870220036367, dated 04 / 28 / 2022, page 36 / 115 15 / 38 a high dynamic range image (HDR_PRED) has a control unit (202) at the high dynamic range image (HDR_PRED) rendering location that sends content identification data (id_rend) that identifies the low dynamic range image (LDR_CONT) to the server (207). The more data is communicated about which specifications are on the rendering side, the greater inference the server from the content owner can make, such as allowing certain information under certain conditions, providing other more suitable mapping data, etc.

[020] Advantageously, the method of providing predefined mapping algorithm data (gam or gam_enc) to transform a low dynamic range image (LDR_CONT) ideal only for LDR display rendering into a high dynamic range image (HDR_PRED) analyzes aspects of the rendering situation as identified from information about that rendering situation received via a control unit (202) on that rendering side. As stated, the server can infer various things based on the types of information it receives, for example, trying to track how a user obtained, for example, certain pirated data (e.g., if a batch of the same pirated movies appear, the server can try, if it can, to download them - for example, if the image transformation device has (temporally) stored the site from where the movie download from it can communicate it to the server).Of course, other variants of the device may not wish to store or reveal so much information. The information transmitted can be largely determined by... Petition 870220036367, dated 04 / 28 / 2022, page 37 / 115 16 / 38 initiative of the image transformation device, or upon requests from the server.

[021] The communicated mapping algorithm data can be distributed (and, where applicable, sold) based on various specification signals or encodings, stored on various physical communication media, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[022] These and other aspects of the method and apparatus according to the invention will be evident from and elucidated with reference to the implementations and embodiments described herein below, and with reference to the accompanying drawings, which serve merely as specific non-limiting illustrations exemplifying the more general concept, and in which dashes are used to indicate that a component is an optional component, not necessarily essential. The dashes may also be used to indicate that elements, which are explained as essential, are hidden within an object, or by intangible things, such as, for example, selections of objects / regions (and how they may be shown on a display).

[023] In the drawings: Figure 1 schematically illustrates how an HDR image is encoded; in reality, an LDR image is encoded, which is usable in an LDR, but not with sufficient visual quality on an HDR display, unless the mapping algorithm data of the encoding method is obtained; and Figure 2 schematically illustrates one of several image transformation devices arranged for this purpose. Petition 870220036367, dated 04 / 28 / 2022, page 38 / 115 17 / 38 communication, preferably around the time of HDR image rendering, with typically a server from the content owner distributing the required mapping algorithm data under appropriate security conditions. DETAILED DESCRIPTION OF THE DRAWINGS

[024] Fig. 2 schematically shows a rendering system that allows the rendering of HDR image(s) under the improved control of the owner of the artistic content. It illustrates the image transformation device 201 of the claimed invention as, for example, a decoder (“set-top box”) or any consumer device that has image processing capability. Evidently, the reader will understand that, depending mainly on the degree of security desired for any particular application, this configuration can be incorporated in different ways.For example, in an extremely secure system, everything (from the communication handling unit that communicates with the memory storing the mapping algorithm via the secure data input 206, including the current tone mapping unit 280, and even possibly signal shapers to calculate the current display drive signals) can be included in the display itself, even in a single IC. The data input can then, for example, be a pin or a bus of this IC, for example, connected to an antenna. If a hacker were to, for example, tap the display drive signals coming out of some of the IC pins, this information would be largely unusable to him in view of the dependency on the... Petition 870220036367, dated 04 / 28 / 2022, page 39 / 115 18 / 38 display. But of course, when less stringent security is sufficient, the HDR image can be sent from the HDR_PRED itself, for example, in a standardized encoding, either through an internal display bus, or, as in the exemplary illustration of Fig. 1, a generic video communication link 291 such as, for example, an HDMI connection. In such a case, the HDR image can, of course, be encrypted using an algorithm agreed upon between the decoder and the display. In fact, the image transformation device 201 can even be a professional device, which can, for example, be used in a store that offers a film purchase service, or in a part of the server of an image / video distribution system, etc.

[025] First, Fig. 1 will explain how the HDR / video image encoding method works. The content creator, for example a Hollywood film studio, has made an original HDR signal of maximum degree HDR_ORIG, which can be encoded, for example, with a codec that has a 20-bit linear luma representation. This image is HDR because it is graded so that it will appear ideal on displays that have at least higher luminance (typically above the 1000 nit white peak), and generally also deeper blacks, greater color rendering accuracy, etc. (it will focus on explaining simplicity in the luma / luminance component of the image pixels, but of course typically a system will also do color mapping to achieve ideally rendered colors). Such a signal is simply not usable on an old LDR display. Firstly, because it is inadequately encoded. In the approach, typically images are encoded, so too Petition 870220036367, dated April 28, 2022, p. 40 / 115 19 / 38 HDR images in a backward-compatible way, as well as with 10-bit or even 8-bit luma. Ignoring the actual pixel colors, such an image can then be compressed and manipulated by a classic image transmission system, for example, by MPEG2, AVC, VP8, JPEG, etc. encoding. But, the actual pixel colors are very important, and that's where the encoding method adds the second phase, otherwise it would go wrong. The HDR image is encoded in such a way that it is directly rendered on an older display, in other words, it must appear of sufficient visual quality on such a display (the colors of the objects in the reasonably approximated image should appear the same as in an original scene, or at least as best as an LDR display could render them, given however an important constraint of not losing the information of the HDR image effect in that signal).Therefore, the transformations that are in principle reversible are applied (that is, the mapping from HDR_ORIG to LDR_CONT can be undone to obtain HDR_PRED from LDR_ORIG by applying the inverse luma / color mapping strategy), or at least, such that from the encoding obtained from LDR_CONT, one can perfectly (or at least with minimal error) recover, as an estimated HDR image mapped HDR_PRED, the original maximum degree HDR_ORIG. This means that the algorithms that perform the luma (and color) mapping must be such as not to destroy the HDR information. To emphasize this important point more precisely: this HDR information, although not perfectly rendered (for example, some darker parts of the image may be lumped together so that they do not...). Petition 870220036367, dated 04 / 28 / 2022, page 41 / 115 20 / 38 are discriminated for the eye from now on in a low peak brightness LDR display under typical ambient conditions), are still recoverable through the application of mapping algorithms (lumas 10, 11, 12, 15 can, for example, be mapped to HDR lumas 25, 30, 48, 100, especially if not many artifacts such as banding are thus visible in the estimation / reconstruction through HDR image mapping. Thus, although one can say that LDR_CONT is an LDR classification / image, it is special insofar as it still contains - because it used appropriately restricted mapping to link HDR_ORIG and LDR_CONT (at least approximately) to all HDR information of the maximum degree HDR_ORIG.

[026] Failure to apply such correct luma mapping to even an 8-bit HDR_ORIG encoding will result in unusable images for legacy devices, as they will appear colorimetrically distorted. For example, one might have a dark basement scene with bright lights. Since a high peak luminance HDR display can render lower luma pixel codes with relatively high output luminance, one could allocate low pixel lumas to all these pixels (e.g., 0.10), and then have no pixels with intermediate lumas, and values ​​(250-255) for the bright lights (if one has to encode an HDR degree in an 8-bit representation). When displaying this signal on an LDR display, however, binarizes it. All dark values ​​are typically seen as the same black. Therefore, you need to apply an F_TM1 luma mapping that pre-lightens the darkest lumas (for example, 0.5 makes 10.20, Petition 870220036367, dated April 28, 2022, p. 42 / 115 21 / 38 with additive and multiplicative mapping), so that the dark room is still visible on the LDR display when this HDR encoding is directly rendered as if it were an LDR image. In this way, the HDR image is encoded as if it were an LDR image, or in other words, the HDR and LDR images are encoded with the same image representation. However, this LDR_CONT image is not directly usable to render the correct maximum degree of HDR_ORIG on an HDR display. Once, for example, the dark parts of the room have been illuminated so that they appear discriminated on an LDR display, they will appear too bright on an HDR display, losing all the spooky mood intended by the content creator. The solution to correct this is in the FL2H inverse luma mapping algorithm.

[027] This gamma algorithm can, in some scenarios, be a simple application of a single gamma function (e.g., HDR luma Y_HDR=a*Y_LDRAg), even for an entire scene or film, or it can be more sophisticated, also taking into account local color optimization since the visual system sees color appearances relatively. For example, a coarse segmentation strategy can define some boundaries before the block sets in an image. In the zigzag scan before the (X,Y) block, a first luma mapping function is used, then before the (X,Y) block, two LDR luma boundaries g_l and g_h are specified, indicating that regions in positions from (X,Y) that have pixel lumas between these boundaries should be treated differently, for example, with a second strategy / algorithm. Petition 870220036367, dated April 28, 2022, p. 43 / 115 22 / 38 luma mapping. For example, if an LDR luma Y_LDR equal to 128 was mapped to Y_HDR= 2099 (in some combined representation, e.g., 20-bit; for simplicity in the Figure, the range of reconstructed HDR lumas has been made a floating-point range (float) [0,1]) through the first luma mapping function, it can now, through the second mapping algorithm, be mapped, for example, to Y_HDR=200. For example, it can process the whites of a shirt so that they don't shine too brightly. Later, in that same image after the (X+K, Y+L) block, it can have the same range of Y_LDR values ​​in the LDR image representation LDR_CONT, but this can have a bright lamp. It can be processed differently to produce very bright Y_HDR lumas by means of a third local mapping algorithm.

[028] In any case, all the information from the mapping algorithms (functional parameters, image numbers such as, for example, display times, location format definition information, etc.), is key to obtaining the original maximum HDR_ORIG degree again, or at least an HDR_PRED degree that appears (very) close to it. This degree will appear as intended, ideal on a reference HDR display, or any bright display that has physical characteristics close to the reference HDR display, and not, for example, so bright in certain positions, or generally incorrectly colored.

[029] The system now allows the content creator to specify which mapping algorithms are to be used to achieve the best appearance, in fact their Petition 870220036367, dated April 28, 2022, page 44 / 115 23 / 38 maximum degree. They could, for example, specify different mapping algorithms for different HDR displays, as you can imagine, one could use a different mapping to map a bright sunny exterior through a window on a 1000 nit display versus a 25,000 nit display. In this way, the rendering can be adapted by the content owner (this already makes piracy easier, since the forger must accumulate all versions of the mapping, because otherwise, if they send something wrong to a new display, the final result may still be far from ideal, not giving the full quality of the experience intended by the content creator). And this even allows the creator to specify some different appearances for some reasons (for example, they can provide a cheaper appearance, which is a kind of mid-range HDR).But beyond that, having all this visual quality in this mapping algorithm(s) allows the creator to implement content protection far beyond existing measures, and therefore obtain a fair return on their effort. Of course, the user can opt for viewing a low-quality version by putting any pirated LDR version (for example, an LDR_CONT extracted somewhere from a normal system) on their HDR display, and will indeed do so if excessive prices are demanded for the HDR experience. But if they want the ideal experience, with the present system they will comply with the content owner's requirements, which have several variants that balance security with ease of use.

[030] Now returning to Fig. 2, it will elucidate Petition 870220036367, dated 04 / 28 / 2022, page 45 / 115 24 / 38 some of the possible embodiments. Thus, the basic image transformation apparatus 201 inherent in all embodiments first has access along an input 204 in a data distribution system 205 comprising the LDR_CONT image(s). As an example, a purchased Blu-ray disc has been shown. Interestingly, even though the user has purchased this disc, in some embodiments it still does not have all the information for HDR rendering, although the disc states that it is an HDR disc (of course, it can always be watched in old mode, for example on an LDR display). The control unit takes care of this, for example after each disc insertion. The experienced reader understands that the data distribution system can take many other forms, for example it can be a network connection to a server that provides a video.It identifies that the disc is a disc (regularly purchased, with permission), and can, for example, extract a unique identification code. Optionally, a 260 (mobile) device (connected or connectable) with data input capabilities can be used to enter, for example, a unique code as already present in the software. This device may even have enhanced protection measures, for example, communicating through the content owner's 207 server to recognize that it is a regular user's device (and blocking devices that are, for example, uniquely allocated to the display in case of misuse, such as piracy attempts). The control unit communicates with the content owner's 207 server and transmits this information. This server can then determine if the user wants to render a product. Petition 870220036367, dated 04 / 28 / 2022, page 46 / 115 25 / 38 duly purchased (for example, from another server of the content owner, or some organization allows distributing copies, such as for example the LDR versions on YouTube), or obtained from a pirated video source 210. In fact, some achievements of the device should also allow legitimizing inappropriate content. Even if the user has obtained the image(s) from the pirated video source 210 (for example, through another image data entry 222), the server may offer to pay for the mapping algorithm (gam) before viewing, for example, through the mobile device which may be a mobile phone or laptop, after which the mapping algorithms are sent anyway.Some implementations of the control unit and / or server can still identify, through checking which LDR_CONT the user has, whether they should send a better quality version (for example, if the content comes from an analog camera capture of some rendering, the color behavior will be far from the desired values). But if the HDR image(s) is / are not properly paid, the device will not provide the mapping algorithms, and may even take some additional actions, such as investigating the situation and the accumulation of system data, notifying the user of their inappropriate behavior, etc.

[031] After receiving the mapping algorithm data, whether in its natural state (gam) or encrypted (gam_enc), according to some agreed algorithm, the tone mapping unit 280 can transform the LDR image LDR_CONT into the HDR image HDR_PRED, and start rendering on the display 290. The connection 217 can be of various forms. Petition 870220036367, dated April 28, 2022, p. 47 / 115 26 / 38 depending on the requirements. An interesting variant is where the device (for example, incorporated into a display, such as a television) requests the mapping algorithm more or less at the desired viewing time, that is, within a time interval around the rendering time. This time interval can be specified by the system, for example, confirmed in the request from control unit 202 by server 207, and for example, be two days before. But in a strict security system, the control unit will be informed that the interval time is, for example, 10 minutes before, and that in case of non-use, it has to delete the algorithm data within 2 hours. Thus, in some embodiments, each time you want to render, you must contact server 207, for example, with a television connection via the internet.Such connections can be made with varying levels of security; for example, one can even make an https (Hypertext Transfer Protocol Secure) connection or a similar secure communication mechanism, again reducing the opportunities for hacking anywhere. The mapping algorithm parameters can also, instead of all being pre-loaded before starting the movie, be downloaded to low-latency networks 207 shortly before they are needed, for example, two seconds or minutes before a scene changes, requiring another mapping strategy. Some embodiments of the image transformation device 201 can be arranged so that they not only require this mapping data in a correctly timed manner, but even through some external connection (possibly even on the network predetermined for the server), making hacking very difficult (once). Petition 870220036367, dated 04 / 28 / 2022, page 48 / 115 27 / 38 which not only needs different mapping algorithms, but also needs to provide them through an added pirated device that emulates the correct acquisition; and since these may need to be connected to a pin(s) of an IC internally in the display instead of fixing it, for example, on a USB bus, such highly secure realizations may be too sophisticated for the average user to perform in massive quantities). The technician understands the various ways in which correct timing synchronization of LDR images and mapping can be implemented, for example, with the storage of image and / or signal data (e.g., BD disk, memory card signal or remote video server, etc.It may include timing codes that identify and request "just one new" or specific mapping algorithms, with the server typically knowing (from, for example, transmitted presentation timestamps) whose mapping data is needed, or the server may manage all this autonomy, and actively instruct the control unit and / or provide it with new mapping algorithms. Thus, numerous things can happen through the control unit. Someone might, for example, request a new type of mapping strategy after the user has dimmed their view of the room light, or, on the other end of the spectrum, it might simply display a message that HDR viewing is not possible or allowed, and typically suggest steps that can be taken to end up with an HDR-rendered image.Thus, the content owner's control can be as simple as a simple "go ahead," typically done as... Petition 870220036367, dated 04 / 28 / 2022, page 49 / 115 28 / 38 some of many possible strategies for supplying mapping algorithm data to the control unit, or it may entail a complex determination situation, in which server 207 analyzes particularities of the user's viewing situation (such as, for example, the lighting in their viewing room), and then proposes the optimized mapping algorithm(s) in real time for ideal rendering on their HDR display. Furthermore, in various embodiments, the control unit may send (on its own, but typically at the intention of server 207), depending also on what is agreed upon based on considerations such as privacy, numerous data values ​​to server 207.For example, the control unit can quickly analyze the video and check if the statistical distribution of lumas in LDR_CONT is as it should be (in one or one of several variants that exist of the image content), or it can analyze a watermark in the data to obtain more information about the source of the content (for example, it was copied from a distribution to a particular cinema). It can read administrative information stored or entered into the system, such as the vendor's address, etc.When desired, various methods can be used to identify the purchase with the present rendering, for example, the control unit can ask the user if the purchase of the HDR movie (or actually LDR_CONT) was made by credit card (or just remind them that it was), and ask for the last four digits, something difficult for a hacker to do, unless they also monitor this purchase, or are the ones who made the purchase (but at least it prevents the theft of a copy from another person whose...). Petition 870220036367, dated April 28, 2022, page 50 / 115 29 / 38 viewing can be disabled for the regular user on some systems). The achievements, while restricting the use of movie rendering (e.g., no regular outside signal can be present, and device 201 may stop working if its casing is opened or modified), make it difficult to do anything but auto-view even if a counterfeiter poses as a regular user and regularly pays for a copy. All sorts of additional information can be stored identifying how this content (to be rendered only) was obtained, which can lead to strange results if it has been copied, for example, by monitoring some signal somewhere. For example, it can be recorded that a normally purchased LDR_CONT should reside with a particular display (or a number of the same to allow, for example, viewing in place of a known and registered / identified friend), and not on another site.In many scenarios, for example, it will not be necessary for the user to provide any specifics of the purchase (such as credit card digits), since the system will be configured to be clear enough that the content was normally purchased, but the realizations may be useful in other scenarios.

[032] But secondly, the server 207 can obtain information (usually with the user's explicit consent) from the user himself, for example, his name stored in a memory in the HDR display image transformation device 290, or a password to start the system, or the user inserting a chip card, etc. More modern devices 201 can at least establish the identity of Petition 870220036367, dated 04 / 28 / 2022, page 51 / 115 30 / 38 user through, for example, the use of a fingerprint sensor, which can already be incorporated. The control unit does not actually need to send all the characteristics of this fingerprint to server 207, but only a summary of the data such as the_usual_viewer_is_using_the_system. But more advanced systems can monitor that if someone is viewing a movie more than, for example, 20 times, they may be a forger instead of someone who really likes that movie, especially if that same version (if the fine grain identified by the user or group of users) is being watched in other physical locations, and even many times as well. That is, server 207 can implement some monitoring strategies in some implementations in addition to providing more rigorously secure data distribution, and requests the necessary data from device 201.Thirdly, the server can obtain all sorts of information regarding specifics of the rendering system (e.g., peak brightness function or electro-optical transfer of the display, regarding lighting characteristics), etc. The first data is typically for registering and / or enabling HDR rendering (or monitoring aspects that are not in accordance with regular system use), while the third type of parameters can help optimize the HDR rendering experience by transmitting the most appropriate mapping algorithm parameters, if several variants exist (which it can even calculate immediately). Fourthly, all sorts of information can be communicated whether the server is 207 or... Petition 870220036367, dated April 28, 2022, page 52 / 115 31 / 38 devices 201 are normal, for example, to prevent a pirated video provider from emulating the entire content owner's validation service, and this can, for example, be done by embedding a number of internet addresses in devices 201.

[033] Other less secure implementations may offer users a little more freedom. Although many devices, such as televisions, may have access to at least some network, such as an internet connection, some users (for example, less sophisticated ones) may only want some final content, for example, on a Blu-ray disc they may also run (in HDR) a display system that does not have a network connection (for example, in the back of their car). Therefore, some implementations may allow storing the mapping algorithms in some memory 270 (hidden in the device 201 or display or in a detachable, portable memory), for example, a writable part of the Blu-ray disc or secure memory card, typically of course in a rigorously encrypted manner. For example, keys can be used based on passwords, or even on the loading times of the algorithm data.A typical system might connect, for example, a portable display (whether actually present via a connected cable or through subsequent data exchange via a memory card) to the image transformation device 201, and then define an encryption strategy for the disk and the portable display at that time. For example, the mapping algorithm data on the disk might be encrypted based on a generated key that is only present in the hardware of that portable display. Petition 870220036367, dated April 28, 2022, page 53 / 115 32 / 38 Having a single key for a particular display (and means of managing and generating it (e.g., the key can be partially generated based on an identifier on the display)) again makes hacking very difficult, since even if you copy the mapping data, it may be unusable.Depending on how intense a data protection the content owner (usually in compliance with the device manufacturer, and preferably some representation of the users) might want, the system might even only work if the correct user password is provided (otherwise a compliant HDR display or image transformation device wouldn't even read it, so it couldn't use the mapping algorithm data), but on the other end of the spectrum, one might also consider the system sufficiently protected with minimal encryption of the mapping algorithm data (or for some hardware configurations, even no encryption for regular users, for example, if the parameters are written directly to the portable display's internal memory at a single boot time, typically with an identifier of the image(s) / movie(s) they apply).Thus, this act of regularization can happen once, for example, when a user buys a disc, they immediately regularize it at home, where they have all the necessary technical components (and write down the passwords or everything else) available. Some users, however, may not want, for example, when they buy content on a holiday that they can immediately watch it, or when they have an inefficient system (for example, they don't have internet at home), etc. For these users... Petition 870220036367, dated April 28, 2022, page 54 / 115 33 / 38 A realization of the image transformation device can be placed in a store. During the purchase, for example, a "blank" BD is installed (that is, one that only contains the LDR encoding LDR_CONT, but not the mapping algorithm data), which can be done by the store clerk using their device that connects to the server 207, and stores the algorithm data, for example, in a writable sector of the portable memory containing the film. Such a realization of the device 201 can, for example, be connected / connectable to a keyboard to enter a private user code, etc. Some variants may request some identification of the user's hardware, for example, the television writes its encrypted code to a memory card, which is attached to the device 201 in the store.Advantageously, if the store immediately sends some purchase information to the 207 server, this can help with regularization at a later time (preferably at the user's home and not again at the store), for example, if the user has forgotten their password.

[034] For simplicity, the algorithm is called gam, since there is no confusion with the data specifying it (for example, at least one coefficient of low energy consumption, or software code that prescribes some color transformation, etc.). The experienced reader will also understand that the control unit 202 can comprise (or be connected to) several other units, such as a communication unit designed to apply the necessary communication protocol, perform data formatting and protection, etc., or a user interface unit that allows communication with the user, and, for example, Petition 870220036367, dated April 28, 2022, page 55 / 115 Figure 34 / 38 shows a window for the secure entry of financial information into your portable display to correct an irregular LDR_CONT, etc. It should also be clear how image processing (color mapping, (de)compression (DCT), etc.) can be done, and there's no need to go into details about display activation, etc. Note that it is clear that device 201 can connect to multiple servers to allow various owners to coordinate or independently specify mapping data for sub-parts of, for example, a film or program. For example, it can find material where material from another provider is interspersed, or even, although some Hollywood producer may own the film, the mapping for the graphics in some scenes may be distributed directly from a server at the special effects studio, while other mapping data may, for example, come from a centralized server at Disney or Paramount.It should also be clear what is technically meant by lumas when mapping a first color representation to a second. A luma is a technical code (e.g., Y=[0,255]) that has an association through a tone-defining curve to a final luminance, whether, for example, when captured by a camera or a display rendering. Several alternative technical realizations may exist, for example, in a linear representation of this third color coordinate it could be the luminance itself, but a sufficiently technically experienced reader should perfectly understand what it is (for simplicity, luma ranges are intended to be floating points (except for LDR_CONT which assumes the classic 8-bit with gamma 2.2, etc.), but it is also clear whether...). Petition 870220036367, dated April 28, 2022, p. 56 / 115 35 / 38 can, for example, project from some 10-bit luma definition to some 15-bit luma definition.

[035] The algorithmic components described in this text may (wholly or in part) be implemented in practice as hardware (e.g., parts of an application-specific IC) or as software that runs a special digital signal processor, or a generic processor, etc. They may be semi-automatic in the sense that at least some user input may be / have a state (e.g., factory, or consumer input, or other human input) present.

[036] It should be understood by the presenter that components may be optional improvements and may be implemented in combination with other components, and as (optional) steps of methods corresponding to the respective apparatus means, and vice versa. The fact that some components are described in the invention in a particular relationship (e.g., in a single figure in a particular configuration) does not mean that other configurations are not possible as embodiments under the same inventive thinking as described for patent purposes in this document. Also, the fact that for pragmatic reasons only a limited spectrum of examples has been described does not mean that other variants cannot fall within the scope of the claims.In fact, the components of the invention can be incorporated in different variants along any chain of use, for example, all variants of a part of the invention such as an encoder may be similar to or correspond to the corresponding devices in a part of the consumption of a decomposed system, for example, a. Petition 870220036367, dated April 28, 2022, page 57 / 115 36 / 38 decoder and vice versa. Various components of the implementations can be encoded as specific signal data into a signal for transmission, or even used as coordination, in any transmission technology between the encoder and the decoder, etc. The word apparatus in this application is used in its broadest sense, namely a group of means that allow the achievement of a particular objective, and may therefore, for example, be (a small part of) an IC, or a dedicated apparatus (such as an apparatus with a display), or part of a network system, etc. The arrangement or system is also intended to be used in the broadest sense, thus it may comprise inter alia a single physical apparatus purchased, a part of an apparatus, a collection of (parts of) cooperating apparatus, etc.

[037] The denotation of computer program product should be understood to encompass any physical realization of a collection of commands that enable a general-purpose or special-purpose processor, after a series of loading steps (which may include intermediate conversion steps, such as translation to an intermediate language and a final processor language) to input the commands into the processor, to execute any of the characteristic functions of an invention. In particular, the computer program product may be realized as data on a medium such as, for example, a disk or tape, data present in a memory, data moving through a wired or wireless network connection, or program code on paper. In addition to program code, the characteristic data required for the program may also be Petition 870220036367, dated 04 / 28 / 2022, page 58 / 115 37 / 38 incorporated as a computer program product. Such data may be (partially) provided in any way.

[038] The invention or any usable data according to any philosophy of the present embodiments, such as video data, may also be incorporated as signals on data carriers, which may be removable memories such as optical discs, flash memories, removable hard drives, portable devices recordable via wireless means, etc.

[039] Some of the steps necessary for the operation of any method presented may already be present in the functionality of the processor or any embodiments of the apparatus of the invention rather than described in the computer program product or any unit, apparatus or method described in this document (with specifics of the embodiments of the invention), such as data input and output steps, well known typically incorporated in processing steps such as standard display actuation, etc. Also, the desired protection for the resulting products and similar results, such as the specific new signals involved in any step of the methods or in any subpart of the apparatus, as well as any new uses of such signals, or any related methods.

[040] It should be noted that the aforementioned embodiments illustrate rather than limit the invention. Where the skilled person can easily perform a mapping of the examples presented in other regions of the claims, it is necessary for brevity not to mention all these options in depth. Petition 870220036367, dated April 28, 2022, p. 59 / 115 38 / 38 In addition to combinations of elements of the invention when combined in the claims, other combinations of the elements are possible. Any combination of the elements can be realized in a single dedicated element.

[041] Any parenthetical reference in the claim is not intended to limit the claim, nor is it any particular symbol in the drawings. The word comprising does not exclude the presence of elements or aspects not listed in a claim. The word a or an preceding an element does not exclude the presence of a plurality of such elements. Petition 870220036367, dated April 28, 2022, pp. 60-115

Claims

1 / 5 CLAIMS 1. IMAGE TRANSFORMATION APPARATUS (201) characterized by being arranged to originate a first image (HDR_PRED) with lumas for a first dynamic range of luminance from a second image (LDR_CONT) with lumas for a second dynamic range of luminance, one of the dynamic ranges of luminance being at least twice as wide as the other, wherein the obtaining comprises the mapping of lumas of pixels in the second image to lumas of pixels of the first image by applying at least one predefined mapping function (gam), the image transformation apparatus comprising: - an input (204) in a data distribution system (205) comprising the second image; - a secure data input (206) to receive at least one predefined mapping function (gam) data;and - a control unit (202) arranged to manage access to predefined mapping function data under the control of a server (207) via a network.; 2. IMAGE TRANSFORMATION APPARATUS (201), according to claim 1, characterized in that the control unit (202) is arranged to obtain at least one predefined mapping function data through secure data input (206) at a predefined time interval before the rendering time on a display of the first image (HDR_PRED).

3. IMAGE TRANSFORMATION APPARATUS (201), according to claim 2, characterized by the interval of Petition 870220086234, dated 09 / 21 / 2022, page 12 / 22 2 / 5 predefined time being two seconds.

4. IMAGE TRANSFORMATION APPARATUS (201), according to any one of claims 1 to 3, characterized in that the control unit (202) is arranged to send to the server (207) the content identification data (id_rend) that identifies at least aspects of the second image (LDR_CONT).

5. IMAGE TRANSFORMATION APPARATUS (201), according to claim 4, characterized in that the control unit is further arranged to send to the server (207) the identification data of the image transformation apparatus (201) or of an HDR display (290).

6. IMAGE TRANSFORMATION APPARATUS (201), according to any one of claims 1 to 5, characterized in that the control unit (202) is arranged to store at least one predefined mapping function data in a memory (270) in an encrypted manner.

7. IMAGE TRANSFORMATION APPARATUS (201), according to any one of claims 1 to 6, characterized in that the control unit (202) is arranged to obtain characteristic information from a user of the first image (HDR PRED).

8. IMAGE TRANSFORMATION APPARATUS (201), according to claim 7, characterized in that the characteristic information includes at least one personal code or biometric data.

9. PREDEFINED MAPPING FUNCTION DATA SUPPLY SERVER (207) characterized by being arranged to receive a connection opening of Petition 870220086234, dated 09 / 21 / 2022, page 13 / 22 3 / 5 data from an image transformation device (201), as defined in any of claims 1 to 8, and arranged to perform a test based on the transmitted content identification data (id_rend) from the image transformation device (201) that tests whether the second image (LDR_CONT) is authorized, and arranged thereafter to supply the predefined mapping function data (gam, gam_enc) to the image transformation device (image (201).

10. PREDEFINED MAPPING FUNCTION DATA SUPPLY SERVER (207) (gam, gam_enc), according to claim 9, characterized in that it is arranged to perform additional tests, specifically if the image transformation apparatus (201) is a normal authorized apparatus.

11. PREDEFINED MAPPING FUNCTION DATA SUPPLY SERVER (207) (gam, gam_enc), according to any one of claims 9 or 10, characterized by being arranged to maintain communication with the image transformation apparatus (201) of partial information of the predefined mapping algorithm data (gam, gam_enc).

12. PREDEFINED MAPPING FUNCTION DATA SUPPLY SERVER (207) (gam, gam_enc), according to any one of claims 9 or 11, characterized in that it is arranged to receive from the image transformation device (201) information on at least one aspect of - the second image (LDR CONT), - the image transformation device (201), - a connected HDR display (290) to render the first image (HDR PRED), Petition 870220086234, dated 21 / 09 / 2022, page 14 / 22 4 / 5 - optical characteristics of an HDR display environment (290), and - a user of the image transformation device (201), and thereafter determine which, if any, preset mapping function data (gam, gam_enc) will be sent to the image transformation device (201).

13. METHOD FOR OBTAINING A FIRST IMAGE (HDR_PRED) OF LUMAS FOR A FIRST LUMINANCE DYNAMIC RANGE FROM A SECOND IMAGE (LDR_CONT) OF LUMAS FOR A SECOND LUMINANCE DYNAMIC RANGE, characterized in that one of the luminance dynamic ranges is at least twice as wide as the other, wherein the obtaining comprises mapping the tone of lumas of pixels in the second image to lumas of pixels of the first image by applying at least one predefined mapping function (gam), the method comprising: - obtaining a data distribution system (205) of the second image; and - managing access to the predefined mapping function data obtained by means of a secure data entry (206) under the control of a server (207) via the network.

14. METHOD FOR OBTAINING A FIRST IMAGE (HDR PRED), according to claim 13, characterized by making a secure and private connection over the network to a server (207) at a predefined time interval before the rendering time on a display (290) of the first image. Petition 870220086234, dated 09 / 21 / 2022, page 15 / 22 5 / 5 15. METHOD FOR OBTAINING A FIRST IMAGE (HDR_PRED), according to claim 14, characterized by a control unit (202) at the rendering location of the first image (HDR_PRED) sending the content identification data (id_rend) that identifies the second image (LDR_CONT) to the server (207). Petition 870220086234, dated 09 / 21 / 2022, pp. 16 / 22

Images