Content-optimized ambient light HDR video adaptation

Abstract

To enable high quality optimization of the luminance of an image, not only optimized for a particular maximum displayable luminance of a display, but also optimized for a particular amount of light in a viewing environment in which the display is being viewed, a method has been invented for processing an input image to obtain an output image, the input image having pixels with an input luminance that is within a first luminance dynamic range DR_1, the first luminance dynamic range having a first maximum luminance PL_V_HDR, a reference luminance mapping function F_L is received as metadata associated with the input image, the reference luminance mapping function specifying a relationship between the luminance of the first reference image and the luminance of a second reference image, the first reference image having a first reference maximum luminance and the second reference image having a second reference maximum luminance, the input image being equal to one of the first and second reference images, the output image being equal to neither the first nor the second reference image, the processing comprising applying an adapted luminance mapping function FL_DA to the input pixel luminance to obtain an output luminance, the adapted luminance mapping function F_L being received as metadata associated with the input image, the reference luminance mapping function specifying a relationship between the luminance of the first reference image and the luminance of a second reference image, the first reference image having a first reference maximum luminance and the second reference image having a second reference maximum luminance, the input image being equal to one of the first and second reference images, the output image being equal to neither the first nor the second reference image, The function FL_DA is calculated based on a reference luminance mapping function F_L and a maximum luminance value PLA, said calculation including finding a position on a metric corresponding to the maximum luminance value PLA, a first end point of the metric corresponding to a first maximum luminance PL_V_HDR and a second end point of the metric corresponding to a maximum luminance of one of the first and second reference images that is not equal to the input image, characterized in that the maximum luminance value PLA is calculated based on a maximum luminance PL_D of a display to which the output image is supplied and a black level value b of the display, said calculation including applying an inverse function of an electro-optical transfer function to the maximum luminance PL_D, subtracting the black level value b from the resulting value and applying the electro-optical transfer function to said subtraction to obtain the maximum luminance value PLA.

Description

[Technical field]

[0001] The present invention relates to a method and apparatus for adapting image pixel brightness of high dynamic range video to provide a desirable appearance for the situation in which the HDR video is displayed under the ambient lighting conditions of a particular viewing location. [Background technology]

[0002] A few years ago, novel approaches to High Dynamic Range (HDR) video coding were introduced, inter alia, by the present applicant (see, for example, WO2017157977).

[0003] Video coding is generally concerned primarily with color codes (e.g., luma and two chromas per pixel) to represent an image, or only with creating or, more precisely, defining color codes. This is somehow distinct from knowing how to optimally display an HDR image (e.g., in the simplest case, one can simply use a highly nonlinear optical-electrical transfer function, OETF, to convert the desired luminance into, say, a 10-bit luma code, and vice versa, using an inverse form of the electro-optical transfer function, EOTF, to convert these video pixel luma codes into display luminance, mapping the 10-bit electrical luma codes to display optical pixel luminance, but more complex systems can deviate in several directions, particularly by decoupling the coding of an image from the specific use of the coded image).

[0004] Coding and processing HDR video stands in stark contrast to the way traditional video techniques were used, by which all video was encoded until recently, and which is now called standard dynamic range (SDR) video coding (also known as low dynamic range video coding, LDR). This SDR began in the analog era as PAL or NTSC, and transitioned to Rec. 709-based coding, such as MPEG2 compression, in the digital video era.

[0005] While a sufficient technology for communicating moving images in the 20th century, advances in display technology that exceeded the physical limitations of the electron beam of 20th century CRTs, namely global TL backlit LCDs, made it possible to display images with significantly brighter (and potentially dimmer) pixels than conventional displays, thus necessitating the ability to code and create such HDR images.

[0006] In fact, for various reasons, it was first necessary to start with much brighter and, in some cases, darker image objects that could not be coded by the SDR standard (8-bit Rec.709) in order to make it technically possible to represent these increased luminance range colors. From there, one by one, all the rules of video technology had to be rethought and often reinvented.

[0007] The Rec.709 SDR luma code definition can only encode a luminance dynamic range of about 1000:1 (at 8-bit or 10-bit luma) due to its approximately square root OETF luma shape: Y_code=power(2,N) * sqrt(L_norm), where N is the number of bits in the luma channel and L_norm is a normalized version of physical luma between 0 and 1.

[0008] Furthermore, in the SDR era, since absolute display luminance was not defined, in practice we mapped the maximum relative luminance L_norm_max=100% or 1 to the maximum normalized luma code Yn=1 (e.g., equivalent to Y_code_max=255) via the square root OETF. This has some technical differences compared to creating absolute HDR images (i.e., an image pixel coded to be displayed as 200 nits would ideally (i.e., where possible) be displayed as 200 nits on all displays and not as significantly different display luminances). In the relative paradigm, a coded pixel luminance of 200 nits would be displayed as 300 nits on a brighter display, i.e., a display with a brighter maximum displayable luminance PL_D (a.k.a. display maximum luminance), and as 100 nits on a less capable display, for example. Note that absolute encoding can also work with normalized luminance representations and normalized 3D gamuts, in which case 1.0 uniquely means, for example, 1000 nits.

[0009] On displays, such relative images have typically been displayed somewhat heuristically by mapping the brightest luminance of the video to the brightest displayable pixel luminance (this is done automatically, without further luminance mapping, via the electronic driving of the display panel with the maximum luma Y_code_max). So if you buy a 200nit PL_D display, whites will look twice as bright as they do on a 100nit PL_D display, allowing for factors such as eye adaptation that were previously thought to be of little importance, except to give a brighter, more pleasing to look at, and somewhat more beautiful version of the same SDR video image.

[0010] Conventionally, today, when speaking of SDR video images (in an absolute framework), there is usually (e.g., agreed upon according to standards) a video peak luminance of PL_V=100 nits. Therefore, in this application, we consider the maximum luminance (or SDR grading) of an SDR image to be exactly that, or generalized at approximately that value.

[0011] Grading in this application is intended to mean either an activity or a resulting image where pixels are given brightness as needed, for example, by a human color grader or an automaton. For example, when viewing an image (e.g., designing an image), there are multiple image objects, and ideally, one would like to give the pixels of those objects a brightness spread around an average brightness that is optimal for the object and the scene, also taking into account the overall image. For example, if the image capabilities are such that the brightest codable pixel of the image is 1000 nit (the maximum brightness of the image or video, PL_V), one grader might choose to give the pixel an explosion brightness value of 800-1000 nit to make the explosion look very powerful, while another filmmaker might choose an explosion with a brightness of, for example, 500 nit or less, so that it does not take up too much of the rest of the image at that time (of course, technically both situations are possible).

[0012] The maximum brightness of an HDR image or video can vary widely and is typically communicated along with the image data as metadata about the HDR video or image (common values ​​are e.g. 1000nit, 4000nit, 10,000nit (non-limiting), typically an HDR image is said to have a PL_V of at least 600nit). If a video creator chooses to define their image as PL_V=4000nit, they can of course choose to create brighter bursts, but relatively they will not reach the 100% level of PL_V, but may be up to e.g. 50% for such a high PL_V definition of the scene.

[0013] An HDR display may have a maximum capability, i.e., a highest displayable pixel luminance, such as 600 nits, 1000 nits, or Nx1000 nits (starting with lower-end HDR displays). The maximum or peak luminance of that display, PL_D, is separate from the maximum luminance of the video, PL_V, and the two should not be confused. Video creators are typically unable to create optimal videos for every possible end-user display (i.e., the capabilities of the end-user display are optimally used by the video, and (ideally) the maximum luminance of the video does not exceed the maximum luminance of the display, but is not lower either; i.e., some part of the video image should have at least some pixels where the pixel luminance is L_p=PL_V=PL_D).

[0014] The creator makes several decisions (e.g., what content to capture, how to capture) and typically creates a video with a high PL_V to at least accommodate the highest PL_D display of the intended audience today, and possibly in the future when higher PL_D displays emerge.

[0015] Then, a secondary question emerges: how to best display an image with a peak luminance PL_V on a display with a (sometimes much) lower display peak luminance PL_D, called display adaptation. In the future, there will be displays that require images with a lower dynamic range than, say, a 2000 nit PL_V image to be created and received over the communication medium. In theory, a display can always be regraded; that is, the luminance of image pixels can be mapped to make them displayable by internal heuristics, but if the video creator takes sufficient care in determining the luminance of the pixels, they can also show how to display the image to a lower PL_D value, and ideally, it may also be beneficial for the display to largely follow these technical desideratums.

[0016] With regard to the darkest displayable pixel luminance BL_D, the situation is more complicated. Some of it may be fixed physical characteristics of the display, e.g. LCD cell leak light, but even with the best displays, what the viewer can ultimately discern as a distinct darkest black also depends on the lighting of the viewing room, which is not a well-defined value. This lighting is characterized as an average illuminance level, e.g. in lux, but for video display purposes it is more elegantly characterized as a minimum pixel luminance. This also involves the human eye in a stronger manner than the appearance of bright or medium luminance, since the human eye is usually seeing a lot of pixels of high brightness, which may reduce the relevance of darker pixels, especially their absolute luminance. However, it can be considered that the eye is not a limiting factor, for example, when looking at an image of a scene that is mostly dark, but is masked by ambient light in front of the display screen. If we assume that humans can only see a noticeable difference of 2%, there is a darkest drive level (or luma) b above which the next darker luma level can be seen (i.e., displaying another 2% more luminance level, X% higher in displayed luminance).

[0017] In the LDR era, we didn't care about the darkest pixels at all. We cared mostly about the average luminance, which was about 1 / 4 of the maximum PL_V=100nit. If an image was exposed around this value, everything in the scene looked bright and colorful, except for clipping of bright parts of the scene above the 100% maximum. For the darkest parts of the scene, if they were important enough, images were created that were captured with a sufficient amount of base lighting in a recording studio or filming environment. If some part of the scene looked bad (for example because it was buried in the code Y=0), it was considered normal.

[0018] Therefore, if nothing more is specified, it can be assumed that the blackest black is zero, or in practice something like 0.1 or 0.01 nits. In such situations, engineers give more weight to the brighter pixels above the average in the coded and / or displayed HDR image.

[0019] In terms of coding, the difference between HDR and SDR is not only physical (more different pixel luminances can be displayed on a larger dynamic range capable display), but also technical, potentially involving further technical concepts such as different luma code allocation functions (using OETF or in an absolute way the inverse of EOTF) and additional dynamically (per image or per set of temporally consecutive images) changing metadata that specifies how to re-grade the pixel luminances of various image objects to obtain an image with a secondary dynamic range different from the starting image dynamic range (usually the two luminance ranges end with peak luminances that differ by a factor of at least 1.5).

[0020] Simple HDR codecs have been introduced to the market. The HDR10 codec is used, for example, to create the recently released Black Jewel Box HDR Blu-ray. This HDR10 video codec uses a logarithmic rather than square root shaped function as its OETF (inverse EOTF), i.e. the so-called Perceptual Quantizer (PQ) function standardized in SMPTE 2084. Instead of being limited to 1000:1 as in the Rec.709 OETF, this PQ OETF allows defining many more (ideally visible) luminances that are sufficient for practical production of HDR video, i.e. lumas between 1 / 10,000nit and 10,000nit.

[0021] Note that the reader should not simplistically confuse HDR with a large number of bits in the luma codeword. This applies to linear systems like the number of bits in an analog-to-digital converter, and indeed the number of bits is the base 2 logarithm of the dynamic range. However, the code allocation function can have a highly non-linear shape, so in theory you could define HDR images with only 10-bit luma (8-bit per color component HDR image) however you wish. This gives you the advantage of reusability of systems already in place (e.g. ICs have a certain bit depth, or video cables, etc.).

[0022] After luma calculation, there is a 10 bit plane of pixel luma Y_code, to which two chrominance components Cb and Cr per pixel are added as chrominance pixel planes. This image can be treated further down the line as if it were a conventionally mathematically compressed (e.g. MPEG-HEVC compressed) SDR image. The compressor does not need to consider pixel color or luminance. However, the receiving device, e.g. a display (or indeed its decoder), usually needs to make a correct color interpretation of the {Y,Cb,Cr} pixel colors and display a correct looking image, e.g. not an image with bleached colors.

[0023] This is typically handled by communicating together the three pixelated color component planes along with further image definition metadata that defines the image coding, such as an indication of the EOTF used (assuming without limitation that the PQ EOTF (or OETF) was used), the value of PL_V, etc.

[0024] More advanced codecs may include further image definition metadata, e.g., processing metadata, e.g., a function (explained in more detail in FIG. 2) that specifies how to map a normalized version of the luminance of the first image up to PL_V=1000 nit to the normalized luminance of a secondary reference image, e.g., an SDR reference image with PL_V=100 nit.

[0025] For clarity, the reference image is not a single pre-fixed image. For each image of a scene (e.g., consecutive images in a video), one or more reference images can be created. They are defined with a certain dynamic range (i.e., for the aforementioned DR, especially its highest luminance PL_V, the various intermediate object luminances have range-dependent values. Hence, the reference image represents the way in which an image is graded for various DRs, and at least two reference images show the way in which one of these two images is luminance re-graded for the DR of the other image). Equivalence of images usually means equivalence of their pixel colors. Various images of different dynamic ranges may correspond to each other in that the pixel luminance of the first image shifts to different luminances of the corresponding image of different DR.

[0026] To improve the understanding of readers who are not very knowledgeable about HDR, we will briefly elucidate some interesting aspects in Figure 1. Figure 1 shows some typical examples of many possible HDR scenes that a future HDR system (e.g., connected to a display with a 1000 nit PL_D) needs to be able to process correctly. Although the actual technical processing of pixel colors can be done in different ways and with different color space definitions, the desiderata of regrading are shown as absolute luminance mappings between luminance axes spanning different dynamic ranges.

[0027] For example, ImSCN1 is a sunny outdoor image from a Western movie with mostly bright areas. The first thing to be clear about is that the pixel brightness of any image is not usually a brightness that can actually be measured in the real world.

[0028] Even if there is no further human involvement in the creation of the output HDR image (which serves as a starter image and is called the master HDR grading or image), no matter how simple it may be by adjusting one parameter, the camera will at least always measure the relative luminance in the image sensor for that iris, so there will always be some step involved to ensure that at least the brightest image pixel falls within the available coding luminance range of the master HDR image.

[0029] For example, the specular reflection of the sun in the sheriff's star is measured to be over 100,000 nits in the real world, which is not visible on a typical near-future display, nor is it comfortable for a viewer watching a movie image, for example, in a dimly lit room at night. Instead, the video creator can decide that 5000 nits is bright enough for the star's pixels, and if this is the brightest pixel in the movie, the video creator can decide to create a video with PL_V=5000 nits. The camera is a relative pixel brightness measurement device only for the raw version of the master HDR grade, but it should also have a high enough native dynamic range (full pixels well above the noise floor) to create a good image. The pixels of the graded 5000 nit image are usually derived non-linearly from the raw image captured by the camera. For example, a color grader takes such aspects into account as a typical viewing situation. This viewing situation is not the same as the actual shooting location, i.e., standing in a hot desert. The best (highest PL_V) image selected to create for this scene ImSCN1, i.e. the 5000nit image in this example, is the Master HDR grade. This is the minimum amount of HDR data required to create and communicate, but it is not the only data communicated for all codecs, some codecs don't even communicate an image at all.

[0030] The availability of such a high encodable luminance range DR_1 (e.g. 0.001nit - 5000nit) allows content creators to provide the viewer with a better experience of bright appearance, but also dim the night scenes (if well graded throughout the movie), provided of course that the viewer has a corresponding high-end PL_D = 5000nit display. A good HDR movie balances the luminance of different image objects not only within a single image, but also over the time course of the story of the movie or the video material produced in general (e.g. a well-composed HDR soccer show).

[0031] The leftmost vertical axis of Figure 1 shows some of the (average) object luminances one would like to see in a 5000 nit PL_V master HDR grading, ideally targeted at a 5000 nit PL_D display. For example, a movie might want to show a cowboy in bright sunshine with a pixel luminance of about 500 nits (i.e. 10 times brighter than a typical LDR, although another creator might want a slightly less HDR impression, such as 300 nits). This constitutes the best manner to display this Western image, which according to the creator, gives the best possible appearance to the end consumer.

[0032] The need for a higher dynamic range of luminance is more easily understood when considering an image that has fairly dark regions, such as the shadowed corners of the cave image ImSCN3, but also relatively large areas of very bright pixels, such as the sunlit outside world seen through the cave entrance, in the same image. This creates a different visual experience than, for example, the nighttime image of ImSCN2, where only street lights contain areas of high brightness pixels.

[0033] Now, the problem is that at present, many consumers still have LDR displays, and even in the future there are good reasons to create two gradings of a movie instead of the typical single HDR image per coding, so it is necessary to define a PL_V_SDR=100nit SDR image that best corresponds to the master HDR image. This is a technical decision that is separate from the technical choice regarding the coding itself, proving that, for example, if you know how to create this secondary image (losslessly) from one of the master HDR images, you can choose to code and communicate either one of the pair (effectively communicating two images for the price of one image, i.e. communicating pixel color component planes of only one image per video instant).

[0034] Such a reduced dynamic range image, of course, cannot define an object with a pixel brightness of 5000 nits, such as a truly bright sun. The minimum pixel brightness, or deepest black, is also as high as 0.1 nit, instead of the more preferred 0.001 nit.

[0035] Therefore, the corresponding SDR image needs to be created in a reduced luminance dynamic range DR_2.

[0036] This can be done by an automatic algorithm in the receiving display, which may for example use a fixed luminance mapping function, or it may be conditioned by simple metadata such as the PL_V_HDR value and potentially one or more other luminance values.

[0037] However, in general more complex luminance mapping algorithms can be used. However, in this application, without loss of generality, we assume that the mapping is defined by some global luminance mapping function F_L (e.g. one function per image). This defines, for at least one image, how to map all possible luminances occurring in the first image (i.e. e.g. 0.0001 to 5000) to the corresponding luminances in the second output image (e.g. 0.1 to 100 nit for an SDR output image). The normalization function is obtained by dividing the luminances along both axes by their respective maximum values. In this context, global means that the same function is used for all pixels of the image, regardless of further conditions, e.g. their position in the image (e.g. more general algorithms that use several functions for pixels that can be classified according to several criteria).

[0038] Ideally, the video creator should decide how to redistribute all the luminance along the available range of the secondary image (SDR image), since the video creator knows best how to sub-optimize for the reduced dynamic range, given the limitations, so that the SDR image still looks at least as good as suitable as the intended master HDR image. The reader can understand that actually defining (locating) such object luminance corresponds to defining the shape of the luminance mapping function F_L, the details of which are outside the scope of this application.

[0039] Ideally, the shape of the function should change for different scenes, i.e. the cave scene versus the sunny western scene a little later in the movie, or generally from image to image in time - we call this dynamic metadata (call it F_L(t), where t represents the image instant).

[0040] Ideally then, the content creator would create an optimal image for each situation, i.e., for each potential end-user display, e.g., a PL_D_MDR=800 nit display that would require a corresponding PL_V_MDR=800 nit image, but this would usually be too much effort for the content creator, even for the most expensive offline video production.

[0041] However, it has been demonstrated by the applicant so far that it is sufficient to create (only) two different dynamic range reference gradings of the scene (usually at the two extremes, e.g. 5000 nits is the highest required PL_V and 100 nits is usually sufficient as the lowest required PL_V). This is because from these two reference gradings (HDR and SDR) all other gradings can be derived automatically, e.g. via a (usually fixed, e.g. standardized) display adaptation algorithm applied to the end-user display receiving the information of the two gradings. In general, the calculations can be done in any video receiver, such as a set-top box, a TV, a computer, movie production equipment, etc. The communication channel of the HDR image can also be any communication technology, such as terrestrial or cable broadcast, a physical medium such as a Blu-ray disc, the Internet, a communication channel to a portable device, video communication between professional sites, etc.

[0042] This display adaptation usually applies a luminance mapping function to the pixel luminance of, for example, the master HDR image. However, the display adaptation algorithm needs to determine a luminance mapping function different from F_L_5000to100 (the reference luminance mapping function that connects the luminances of the two reference gradings), i.e. the display adaptation luminance mapping function FL_DA, which is not necessarily trivially related to the original mapping function F_L between the two reference gradings (there are several variants in the display adaptation algorithm). The luminance mapping function between the master luminance defined in the PL_V dynamic range of 5000 nits and the intermediate dynamic range of 800 nits is written as F_L_5000to800 in this text.

[0043] Here, we have symbolically indicated the display adaptation (only for one of the average object pixel luminances) with an arrow that does not map the F_L_5000to100 function to a "simple" expected location above the 800 nit MDR image luminance range, but to a slightly higher location (i.e., in such an image, the cowboy should be slightly brighter, at least according to the selected display adaptation algorithm). So, while a more complex display adaptation algorithm may place the cowboy at a higher location than indicated, some customers may be satisfied with the simpler location where the connection between the 500 nit HDR cowboy and the 18 nit SDR cowboy exceeds the 800 nit PL_V luminance range.

[0044] Typically, a display adaptation algorithm calculates the shape of the display adaptation luminance mapping function FL_DA based on the shape of the original luminance mapping function F_L (or a reference luminance mapping function, also known as a reference regrading function).

[0045] This explanation based on Fig. 1 constitutes a technical desiderata of an HDR video coding and / or processing system, and Fig. 2 shows some exemplary technical systems and their components for realizing the desiderata (non-limiting) according to the applicant's codec approach. Those skilled in the art should understand that these components can be embodied in various devices, etc. Those skilled in the art should understand that this example is presented as a pars prototo for various HDR codec frameworks, only to get a background understanding of some principles of operation, and is not intended to specifically limit any of the embodiments of the innovative contributions presented below.

[0046] Although possible, the technical communication of two actual different images at one time (HDR and SDR grading each being communicated as three respective color planes) is expensive, especially with regard to the amount of data required.

[0047] Also, this is not necessary, since if one knows that all corresponding secondary image pixel luminances can be calculated based on the luminances of the primary image and the function F_L, one can decide at one moment to communicate only the primary image and the function F_L as metadata (and one can also choose to communicate either the master HDR image or the SDR image as representative of both). The receiver can determine the FL_DA function at its end based on this data, since it knows its (usually fixed) display adaptation algorithm (further metadata can be communicated to control or guide the display adaptation, but is not currently implemented).

[0048] There are two modes to communicate one image per instant and the function F_L.

[0049] In a first backwards compatible mode, an SDR image is communicated ("SDR communication mode"). This SDR image can be displayed directly (without needing further luminance mapping) on ​​a conventional SDR display, whereas an HDR display needs to apply the F_L or FL_DA function to obtain an HDR image from the SDR image (or vice versa, depending on the communicated function variant, i.e. upgrading or downgrading variant). The interested reader is referred to the full details of the applicant's standardized exemplary first mode approach below: ETSI TS 103 433-1 V1.2.1(2017-08):High-Performance Single Layer High Dynamic Range System for use in Consumer Electronics devices;Part 1:Directly Standard Dynamic Range(SDR) Compatible HDR System(SL-HDR1).

[0050] In another mode, the master HDR image itself is communicated ("HDR communication mode"), i.e., for example, a 5000 nit image and a function F_L that allows to compute therefrom a 100 nit SDR image (or other low dynamic range image via display adaptation). The master HDR communicated image itself can be encoded, for example, using the PQ EOTF.

[0051] Figure 2 also shows the overall video communication system. On the sender side, we start with the image source 201. This could be anything from a hard disk to a cable output from a TV studio etc, depending on whether we have an offline created video from an internet distribution company for example, or an actual broadcast.

[0052] This results in a master HDR video (MAST_HDR) that is, for example, color graded by a human color grader, or a shaded version of the camera capture, or an automatic brightness redistribution algorithm, etc.

[0053] In addition to the grading of the master HDR image, a set of often reversible color transformation functions F_ct is defined. Without intending loss of generality, we assume that this includes at least one luminance mapping function F_L (although there may be further functions and data specifying, for example, how the saturation of a pixel changes from the HDR to the SDR grading).

[0054] This brightness mapping function, as mentioned above, defines the mapping between the HDR and SDR reference gradings (the latter being, in FIG. 2, the SDR image Im_SDR that is communicated to the receiver, whether or not the data is compressed via, e.g., MPEG or other video compression algorithm).

[0055] The color mapping of the color converter 220 should not be confused with the one applied to the raw camera feed to obtain the master HDR video, which is assumed here as already input, since this color conversion is to obtain the communicated image and at the same time the regrading desiderata, which is technically formulated in the luminance mapping function F_L.

[0056] For an exemplary SDR communication type (i.e., SDR communication mode), the master HDR image is input to the color converter 202. The color converter 202 is configured to apply the F_L luminance mapping to the luminance of the master HDR image (MAST_HDR) to obtain all the corresponding luminances written to the output image Im_SDR. For the sake of explanation, we assume that the shape of this function is adjusted for each shot of the image of a similar scene in a movie by a human color grader using color grading software. The applied function F_ct (i.e., at least F_L) is written into the (dynamic, processing) metadata communicated with the image, for example MPEG Supplemental Enhancement Information data SEI (F_ct), or similar metadata mechanisms in other standardized or non-standardized communication methods.

[0057] Once the communicated HDR images have been properly redefined as corresponding SDR images Im_SDR, these images are likely to be compressed (at least for example, for broadcast to end users) using existing video compression techniques (e.g., MPEG HEVC, VVC, AV1, etc.). This is performed in a video compressor 203 forming part of a video encoder 221 (which may be included within various types of video production devices or systems).

[0058] The compressed image Im_COD is transmitted to at least one receiver via an image communication medium 205 (e.g., satellite, cable, or Internet transmission, e.g., compliant with ATSC3.0 or DVB, etc.) (although the HDR video signal can also be communicated, e.g., via a cable between two video processing devices).

[0059] Further conversion is typically performed prior to communication by a transmit formatter 204, which applies techniques such as packetization, modulation, and transmission protocol control depending on the system, typically an integrated circuit.

[0060] At any receiving site, a corresponding video signal unformatter 206 applies the necessary unformatting methods, e.g., modulation, to re-obtain the compressed video as a set of compressed HEVC images (i.e., HEVC image data).

[0061] The video decompressor 207 performs, for example, HEVC decompression to obtain a stream of pixelated, uncompressed images Im_USDR. In this example, the image Im_USDR is an SDR image, but in other modes it is an HDR image. The video decompressor also unpacks, for example from the SEI message, the necessary luminance mapping function F_L, or in general, a color conversion function F_ct. The images and functions are input to a (decoder) color converter 208, which is arranged to convert the SDR image into an image of any non-SDR dynamic range (i.e. an image with a PL_V above 100 nits, typically at least several times higher, for example 5 times higher).

[0062] For example, a 5000 nit reconstructed HDR image Im_RHDR is reconstructed as a close approximation of the master HDR image (MAST_HDR) by applying an inverse color transform IF_ct of the color transform F_ct used at the encoding side to create Im_LDR from MAST_HDR. This image is then sent, for example, to a display 210 for further display adaptation. However, a display adapted image Im_DA_MDR can be created in one go during decoding by using a FL_DA function (e.g., determined in firmware in an offline loop) in the color converter instead of the F_L function. Thus, the color converter may include a display adaptation unit 209 to derive the FL_DA function.

[0063] The optimized, e.g. 800 nit, display adapted image Im_DA_MDR is sent to, e.g., the display 210 if the video decoder 220 is included in, e.g., a set-top box or a computer, or to a display panel if the decoder is in, e.g., a mobile phone, or to a projector in a cinema if the decoder is in, e.g., a server connected to the Internet.

[0064] FIG. 3 shows a useful variant of the internal processing of a color converter 300 (i.e., corresponding to 208 in FIG. 2) of an HDR decoder (or an encoder that typically has roughly the same topology, but uses an inverse function, but typically does not include display adaptation). The luminance of a pixel (in this example a pixel of an SDR image) is input as the corresponding luma Y'SDR. The chrominance, also known as chroma components Cb and Cr, are input to the lower processing path of the color converter 300.

[0065] The luma Y'SDR is mapped to the required output luminance L'_HDR (e.g., master HDR reconstruction luminance, or other HDR image luminance) by the luminance mapping circuit 310. The luminance mapping circuit 310 applies an appropriate function for the particular image and maximum display luminance PL_D, e.g., a display adapted luminance mapping function FL_DA(t), obtained from a display adaptation function calculator 350 that uses as input a reference luminance mapping function F_L(t) communicated with the metadata. The display adaptation function calculator 350 can also determine an appropriate function for processing chrominance. Here, we simply assume that a set of multiplication coefficients mC[Y] for each possible input image pixel luma Y is stored, e.g., in the color LUT 301.

[0066] Indexing the color LUT 301 with the luma value Y of the currently color transformed (luminance mapped) pixel yields the required multiplication coefficient mC as the LUT output. This multiplication coefficient mC is used by multiplier 302 to multiply the two chrominance values ​​of the current pixel, i.e., yielding the color transformed output chrominance. Cbo=mC * Cb, Cro=mC * Cr

[0067] Via a fixed color matrixing processor 303 that applies standard colorimetric calculations, the chrominance can be transformed into lightness-free normalized non-linear R'G'B' coordinates R' / L', G' / L', and B' / L'.

[0068] The R'G'B' coordinates that give the appropriate brightness in the output image are obtained by multiplier 311, which calculates: R'_HDR=(R' / L') * L'_HDR, G'_HDR=(G' / L') * L'_HDR, B'_HDR=(B' / L') * L'_HDR, This can be summarised in the colour triplet R'G'B'_HDR.

[0069] Finally, there may be further mapping to the format required by the display by the display mapping circuit 320. This results in the display driving color D_C, which is not only formulated in the colorimetry desired for the display (e.g. also in HLG OEFT format), but this display mapping circuit 320 may in some variants be arranged to perform specific color processing for the display. That is, the display mapping circuit 320 may for example further remap some of the pixel luminances.

[0070] Some examples illustrating suitable display adaptation algorithms for deriving a FL_DA function corresponding to any possible F_L function that may have been determined by the producing grader are taught in WO2016 / 091406 or ETSI TS 103 433-2 V1.1.1(2018-01).

[0071] However, these algorithms do not take into account the minimum black that is displayable on the end user's display.

[0072] In fact, one might say that the minimum luminance BL_D is made to appear so small that it is zero. Display adaptation therefore primarily takes into account differences in the maximum luminance PL_D of various displays compared to the maximum luminance PL_V of the video.

[0073] As can be seen from Figure 18 of the prior application WO2016 / 091406, any input function (in the illustrated example, a simple function formed from two linear segments) is usually scaled toward the diagonal based on a metric arranged along an angle of 135 degrees starting from the horizontal axis of input luminance in the plot normalized to an input and output luminance of 1.0. It should be understood that this is only one example of display adaptation of a whole class of display adaptation algorithms, and is not mentioned in a manner intended to limit the applicability of the novel display adaptation concept. For example, specifically, the angle of the metric direction may be other values.

[0074] However, this metric and its action on the reshaped F_L function, i.e. the determined FL_DA function, depends only on the maximum luminance PL_V and the maximum luminance PL_D of the display to which the optimally regraded mid-dynamic range image is provided. For example, the 5000 nit position corresponds to the zero metric point on the diagonal (any position on the diagonal corresponds to a possible pixel luminance of the input image), and the 100 nit position (marked PBE) is a point of the original F_L function.

[0075] Display adaptation as a useful variant of this method is summarized in Figure 4 by showing its action on a plot of possible normalized input luminance Ln_in versus normalized output luminance Ln_out (which are converted to actual luminance by multiplication by the maximum luminance (i.e., PL_V value) of the display associated with the normalized luminance).

[0076] For example, a video creator has designed a luminance mapping strategy between two reference gradings as described in Fig. 1. Therefore, for any possible normalized luminance of a pixel of an input image Ln_in, e.g., a master HDR image, this normalized input luminance must be mapped to a normalized output luminance Ln_out of a second reference grading, which is the output image. This re-grading of all luminances corresponds to a function F_L that can have many different shapes determined by a human grader or a grading automaton, and the shape of this function is communicated together as dynamic metadata.

[0077] The question is what shape the derived quadratic version of the F_L function should have in this simple display adaptation protocol to map to an MDR image (rather than a reference SDR image) for a mid-dynamic range display (again assuming a mapping starting from an HDR reference graded image as the input image). For example, the metric can calculate that for an 800 nit display, for example, there should be a 50% grading effect. 100% of full is a regrading from the master HDR image to a 100 nit PL_V SDR image. In general, the metric can be used to determine, for any possible normalized input luminance of a pixel (Ln_in_pix), denoted as the display adapted luminance L_P_n, any point between where there is no regrading and where there is a full regrading to the second reference image. The location of the display adapted luminance L_P_n naturally depends on the input normalized luminance, but also on the value of the maximum luminance associated with the output image (PL_V_out).

[0078] The corresponding display adapted luminance mapping FL_DA can be determined as follows (see FIG. 4a): Choose any one of all input luminances (e.g. Ln_in_pix), which corresponds to a starting position on a diagonal (shown as a square) with equal angles to the input and output axes of normalized luminance. At each point on the diagonal, place a scaled version of the metric (scaled metric SM) such that it is orthogonal to the diagonal (or 135 degrees counterclockwise from the input axis), starts at the diagonal, and ends at its 100% level at a point on the F_L curve, i.e. the intersection of the F_L curve with the orthogonal scaled metric SM (shown as a pentagon). Place a point at the 50% level of the metric, i.e. in the middle (note that in this case the PL_V value of the output image is set to the PL_D value of the display that needs to deliver a display optimized image) (for this PL_D value of the display for which the image needs to be calculated in this example). By doing this for all points on the diagonal corresponding to all Ln_in values, we obtain a FL_DA curve that is shaped similarly to the original curve, i.e., with the same regrading, but with the maximum luminance rescaled / adjusted. This function is then ready to be applied to calculate the required corresponding optimally regraded / display adapted 800 nit PL_V pixel luminance, given any input HDR luminance value of Ln_in. This function FL_DA is applied by the luminance mapping circuit 310.

[0079] In general, the properties of this display adaptation are as follows (without intending to be more specific and limiting): The orientation of the metric is fixed in advance as technically desirable. Figure 4b shows another scaled metric, namely the scaled metric SMV oriented vertically (i.e., orthogonal to the axis of the normalized input luminance Ln_in). Again, 0% and 100% (or 1.0) correspond to no regrading (i.e., identity transformation to the input image luminance) and regrading to the second of the two reference graded images (related in this example by a luminance mapping function F_L2 of a different shape), respectively.

[0080] The location of the measurement point on the metric, i.e. where the 10%, 20%, etc. values ​​lie, also varies, to be precise, but is typically non-linear.

[0081] This is pre-designed in technologies such as television displays. For example, the functions described in WO2015007505 can be used. * The logarithmic function can also be designed such that (log(PL_V)+b) is equal to 1.0 for the PL_V_HDR value (e.g., 5000 nits) and the 0.0 point corresponds to the PL_V_SDR reference level of 100 nits and vice versa. The location of any PL_V_MDR where image luminance needs to be calculated is obtained from the designed mathematical processing of the metric.

[0082] Figure 5 summarizes the behavior of such metrics.

[0083] For example, a display adaptation circuit 510 in a television or set-top box includes a configuration processor 511, which sets values ​​for image processing before performing pixel color of the incoming image to be processed. For example, the maximum luminance value of the display-optimized output image PL_V_out is set once in the set-top box by being polled from the connected display (i.e., the display communicates its maximum displayable luminance PL_D to the set-top box), or if the circuit is in a television, by the manufacturer, etc.

[0084] In some embodiments, the luminance mapping function F_L varies for each input image (in other variations it is fixed for many images) and is input from a metadata source 512 (e.g. broadcast as an SEI message or read from a sector of memory such as a Blu-ray disc). This data establishes the normalized height of the normalized metric (Sm1, Sm2, etc.) on which the desired location of the PL_D value can be found from the mathematical equation of the metric.

[0085] As image 513 is input, successive pixel intensities (e.g., Ln_in_pix_33, Ln_in_pix_34, etc.) are passed through a color processing pipeline that applies display adaptation, resulting in corresponding output intensities such as Ln_out_pix_33.

[0086] Note that in this approach there is nothing specifically corresponding to minimum black luminance.

[0087] This is because the usual approach is that black levels depend heavily on the actual viewing situation and vary even more than the display characteristics (i.e. the top-level PL_D). All sorts of effects occur, ranging from physical lighting aspects to the optimal configuration of the light-sensitive molecules in the human eye.

[0088] So you create a better image "for display" and that's it (i.e. how much better the intended HDR display is in terms of brightness than a regular SDR display). You can then later correct it for the viewing situation if necessary - this is an (undefined) ad-hoc task left to the display.

[0089] Therefore, we usually assume that the display can display all the required pixel brightnesses coded in the image (here we assume that it is an MDR image already optimized for the PL_D value, i.e. an image that usually has at least some pixel regions in the image that go up to PL_D) up to its variable high brightness capability, i.e. PL_D. This is because we typically do not want to suffer the severe consequences of white clipping, but as mentioned before the blacks in the image are often not very interesting.

[0090] Either way, black is "mostly" visible, so it doesn't matter if some of it looks a little worse - at least it allows the potentially very bright pixel luminances of the master HDR grading to be optimally put into the limited upper range of the display, say above 200 nits, e.g. 200~PL_D=600 nit (for master HDR luminance up to 5000 nits).

[0091] This is akin to assuming that black is always zero nits for all images and (at least nearly) all displays: white clipping is a much more visually unpleasant property than losing some of the black (in which case something is still visible, albeit less pleasant).

[0092] However, this approach can be insufficient because a significant subrange of the darkest luminances may be hidden or at least lack visibility under significant ambient lighting in a viewing room (e.g., a living room with a large window, during the day, and / or with a consumer television audience), which differs from the ambient lighting conditions in a video editing room where the video is created, which may be dim or even dark.

[0093] It is therefore necessary, for example, to increase the brightness of these pixels using a control button (a so-called brightness button) which is normally present on the display.

[0094] Take for example the electronic behavior model of a TV as found in Rec.ITU-R BT.814-4(07 / 2018): a TV in an HDR scenario takes the luma+chroma pixel colors (which are actually driving the display) and converts these (according to standard colorimetric calculations) into non-linear R',G',B' drive values ​​to drive its panel. The display then processes these R',G',B' non-linear drive values ​​with a PQ EOTF to figure out the front screen pixel brightness to display (i.e. how to drive e.g. a PLED panel pixel or an LCD pixel; there is usually still some internal processing that describes the electro-optical physical behavior of the LCD material, but that aspect is irrelevant to this discussion).

[0095] For example, a control knob on the front of the display can give a luma offset value b (where 2% above the smallest black patch of a PLUGE or other test pattern becomes visible, while -2% black is not).

[0096] The original unmodified display behavior is, for example, as follows: LR_D=EOTF[max(0,R')]=PQ[max(0,R')] LG_D=EOTF[max(0,G')]=PQ[max(0,G')] LB_D=EOTF[max(0,B')]=PQ[max(0,B')] [Formula 1]

[0097] In this equation, LR_D is the (linear) amount of red contribution that is displayed to create a particular pixel color with a particular luminance ((fractional) nits), and R' is the non-linear luma code value (e.g., 419 out of 1023 values ​​for 10-bit coding).

[0098] The same happens for the blue and green components: if you need to make a particular achromatic color 1 nit (the total luminance of that color to the eye), you need, say, 1 unit of blue, and similarly for red and green. If you need to make 100 nits of that same color, you can say LR_D=100nit (the visual weight of the red is taken out of the equation).

[0099] Now, when we control this display driving model via the luma offset knob, the general equation becomes: LR_D_c=EOTF[max(0,a * R'+b), where a=1-b / inv_EOTF[PL_D], etc. [Equation 2] inv_EOTF[ ] is the mathematical inverse equation of EOTF[ ].

[0100] In this approach, rather than displaying the zero black of the image somewhere hidden in the invisible display black, the black is raised to a level where it is sufficiently distinguishable. (Note that in consumer displays, mechanisms other than PLUGE can be used, for example viewer preference could be used, potentially resulting in a different, possibly sub-optimal, luma offset value b that the viewer would prefer to use.)

[0101] This is a display post-processing step after creating an optimally regraded image: first an optimal theoretically regraded image (e.g. an initial mapping to the reconstructed master HDR image) is calculated by the decoder, then a luminance remapping to an MDR image with e.g. PL_V of 550 nits is calculated, i.e. taking into account the display's lightness capability PL_D.

[0102] This optimal image is then determined according to the filmmaker's ideal vision, and is then further mapped by the display, taking into account the expected visibility of black in the image.

[0103] According to the inventor, the problem is that this is a rather crude manipulation of what could have been a carefully crafted grading, and therefore there may be an opportunity to invent a better alternative that gives a better looking image.

[0104] US Patent Application Publication No. 2019 / 0304379 is another technical approach to creating display-optimized luminance regrading, which also takes into account the amount of ambient light.

[0105] The system is based on determining the appropriate amount of average backlight setting (which may vary with local dimming) in combination with the natively achievable dynamic range of the LCD display panel (where closed pixels transmit an amount of dim leak light equivalent to, for example, 1 / 1000th of the light of an open pixel).

[0106] Display mapping is based on the design of an optimal S-curve, which is defined by one or more, usually three, metadata values ​​that describe the input image, and together with the metadata values ​​the optimal S-curve is determined by the receiver of the image. This S-curve approach worked well in the wet photography era when making LDR prints from larger dynamic range negative captures, and has been found to work well in the digital era.

[0107] For example, metadata specifying the average luminance (or luma, if the method is applied in the luma domain), called the midpoint luminance value, can determine the shape and location of the linear slope of the S-curve. The reader can see that if the input image is a fairly dark image (e.g., at night when buildings are not very well lit), the low luminance of the HDR input range should be mapped to the middle of the output range (e.g., 20-60 nits for LDR) to look nice on a lower dynamic range display. Similarly, the maximum luminance metadata of the input image helps determine where the S-curve should start plateauing at the top end. If the maximum is much higher than the average (which should eventually display up to 60 nits), leaving only 40 nits available for highlights, then soft clipping of the upper part of the S-curve should start early in the input range of luminance (this works particularly well when there are not many very bright pixels, e.g., there are some shiny patches reflecting scene lighting on metal objects).

[0108] A similar type of tapering is performed for the lower available range, i.e., output luminance below 20 nits, by determining a gradual clipping of the lower end of the S-curve of the luminance mapping (sometimes called the foot or toe of the S-curve).

[0109] Thus, this display adaptation method (or display management method as it is called in U.S. '379) works well with any kind of input image, but assumes that the image is downgraded for a display with a smaller dynamic range in the same viewing environment in which the input image was created (e.g., in the ambient / ambient light of a grading booth, i.e., dimmed with controlled low lighting, and graded on a 4000 nit grading display).

[0110] The idea to generalize to any viewing environment is to design a strategy based on modifying the graded master input HDR image as it should have been graded if the grading booth was lit differently (e.g. with the amount of light in a train during the day). This redefinition of the graded master image is called the virtual image.

[0111] Again, since this is just an image, the exact same display adaptations can be used.

[0112] That master / input image is brightened (all luminances are increased, from dimmest to maximum) for the brighter environment. Hence, if the ambient light level remains at the reference dim light level after the standard S-curve display management, it will appear too bright on the end display with a smaller dynamic range. But both will look good at the new light levels. Don't forget to calculate new metadata (min, midpoint, max) of the redefined brightened (virtual) input image of your approach.

[0113] This approach differs from systems using a single control value, precisely because it allows positioning along a metric that can be placed along various points of a reference luminance mapping function, which can be specified in the metadata of almost any shape if necessary, so that at these points of the metric one can find points of a new (adjusted) curve, redefined by the black value, but not by a control value based on maximum luminance.

[0114] The ITU Radiocommunication Research Committee's document 6 / 115-E of 27 March 2017 entitled "High Dynamic range television for production and international program exchange" (Question ITU-R142-2 / 6) (Revision of Working Group 6C's Report BT.2390-1) teaches:

[0115] The image of the reference viewing environment displayed on the reference display is created by varying the reference OOTF with artistic adjustments (creative intent). The receiving image is display adjusted for a non-reference display in a non-reference viewing environment. The reference OOTF optimizes the direct camera capture, which is only a relative luminance measurement of the scene, but not optimal for the appearance of a good image for a typical display. This approach can be applied to perceptual quantizer HDR coding systems, i.e. artistically optimized pixel luminance can be encoded as PQ luma before HDR image ( / video) communication.

[0116] In addition to the camera-side OOTF, which performs a rough average grading of the camera image, an EETF is defined that is applied to adapt images created in high dynamic range to displays with a smaller dynamic range (as a luma-to-luma mapping to calculate the luminance mapping). Since the codable PQ range allows to code images with pixel luminances as dark as 1 / 10000 nit and potentially as bright as 10,000 nit, and typical HDR displays currently display from 0.01 to 1000 nit, a fixed remapping curve is presented. However, while 0.01 nit characterizes the black of such displays (e.g., a purchasable consumer TV), under different ambient light levels and in front of screen reflections, the minimum achievable black may be, for example, 0.1 nit. Thus, the same S-shaped curve can be calculated for scenarios starting from different minimum black levels. Mapping a particular curve to the available display range is, precisely speaking, very different from modifying the control parameters of the metric-based function shape that follows the display adaptation algorithm. Summary of the Invention

[0117] Images that look visually better for different ambient lighting levels can be obtained by computing them in a way that processes an input image to obtain an output image. the input image has pixels having an input luminance within a first luminance dynamic range (DR_1), the first luminance dynamic range having a first maximum luminance (PL_V_HDR); A reference luminance mapping function (F_L) is received as metadata associated with the input image; the reference luminance mapping function specifies a relationship between the luminance of the first reference image and the luminance of the second reference image; The first reference image has a first reference maximum luminance, and the second reference image has a second reference maximum luminance; the input image is equal to one of the first reference image and the second reference image; the output image is not equal to either the first reference image or the second reference image; The process includes applying an adapted luminance mapping function (FL_DA) to input pixel luminances to obtain output luminances; an adapted luminance mapping function (FL_DA) is calculated based on a reference luminance mapping function (F_L) and a maximum luminance value (PLA) as a control parameter specifying the degree to which the adapted luminance mapping function deviates from the reference luminance mapping function; The above calculation of the adapted luminance mapping function involves finding the position on the metric that corresponds to the maximum luminance value (PLA), a first endpoint of the metric corresponds to a first maximum luminance (PL_V_HDR) and a second endpoint of the metric corresponds to a maximum luminance of one of the first and second reference images that is not equal to the input image; A maximum luminance value (PLA) is calculated based on a maximum luminance (PL_D) of a display to which the output image is provided and a black level value (b) of the display; The above calculation is characterized in that it includes applying the inverse of the electro-optical transfer function to the maximum luminance (PL_D), subtracting the black level value (b) from the resulting value, and applying the electro-optical transfer function to the subtraction to obtain the maximum luminance value (PLA) to be used in the above calculation of the adapted luminance function.

[0118] The main characterization of a dynamic range of pixel luminance is the maximum luminance, i.e. all pixels of an image created (encoded) according to such a dynamic range must not be brighter than this maximum luminance. Note that a commonly used consensus is that the component luminances are pre-scaled by their visual weights for simplicity, so that 1 nit of red, green, and blue gives a neutral 1 nit gray pixel color (in this case the same OETF or EOTF equations can be used). If the image is an image of a sequence of images (aka video), not all images necessarily contain pixels with pixel luminance equal to the maximum luminance, but the sequence is defined by this maximum value.

[0119] In prior art display adaptation, the maximum luminance (PL_V) is the maximum luminance (PL_D) obtained from the connected or connected display (e.g., a 500 nit display). The content itself is optimized for the display's typically lower maximum displayable luminance compared to the content represented by the bright pixel luminance of the input image. This approach finds great applicability, giving video creators a good measure of control over how the images they create (usually a master image of the highest quality suitable for any display, but specified to be optimal for a particular associated target display, for which the image is specifically color graded) should ultimately appear on any available end-point (e.g., consumer viewer) display.

[0120] Typically we want to use the PL_V control value that is relevant to the end display on which the image will be viewed, but we can redefine this concept.

[0121] An adjusted value of maximum luminance is then determined, which takes into account in particular the black level of the display (i.e., visible on the display). This allows a different level of control over the regrading of displays with a different, usually smaller, dynamic range than the target display of the received image, which, as the inventors have found, is suitable for viewing with ambient lightness adaptive regrading.

[0122] Any HDR electro-optical transfer function can be chosen for this method (usually pre-fixed by the designer of the technology). The electro-optical transfer function (EOTF) is a function that defines the display pixel luminance corresponding to so-called luma codes. These luma codes are for example 8-bit words, 10-bit words or 12-bit words, where each word (luma level) specifies a certain luminance.

[0123] Although not a limiting factor of this innovation, the EOTF that works well is the known perceptual quantizer defined in the SMPTE ST.2084 standard, which works perceptually well as other mostly perceptually uniform HDR EOTFs.

[0124] The black level can be determined with a standard test pattern using a dark black test patch such as PLUGE as mentioned above, but in general it is any selectable value b, which represents exactly or approximately the darkest black that can still be discerned on the display, given any viewing room constraints (e.g., ambient lit lamps). That is, it is a black that appears slightly brighter than a complete black that buries all darker pixels. It is represented, without any intention of limitation in the elucidation teachings of this patent application, in the domain of the perceptual quantizer luma (what one skilled in the art understands is the set of lumas that can represent the corresponding luminance, which can also be represented in the domain of luminance itself, switching between domains when technically advantageous).

[0125] Typically, the reference image differs from both the input image and the output image, but the input image is assumed to be (identical to) one of the reference images (e.g., the HDR image of a reference image of an HDR / SDR pair). The output image is not identical to one of the reference images, and is typically an image of the mid-dynamic range of the particular display (e.g., having a mid-maximum brightness of, e.g., 900 nits, lower than the incoming HDR image of 2000 nits). For this particular display, the incoming image is optimized.

[0126] So the method (or apparatus) typically looks like this: 1. A method for processing an input image to obtain an output image, comprising the steps of: the input image has pixels having an input luminance within a first luminance dynamic range (DR_1), the first luminance dynamic range having a first maximum luminance (PL_V_HDR); A reference luminance mapping function (F_L) is received as metadata associated with the input image; the reference luminance mapping function specifies how to map the luminance of the input image to the luminance of a second image having a different dynamic range than the input image; The second image has a reference maximum brightness; the output image is not equal to the input image or the reference image and has an output maximum luminance different from the first maximum luminance and the reference maximum luminance (e.g., between these values, below the lower of these two values, or above the higher of these two values); The process includes applying an adapted luminance mapping function (FL_DA) to input pixel luminances to obtain output luminances; an adapted luminance mapping function (FL_DA) is calculated based on a reference luminance mapping function (F_L) and a control maximum luminance value (PLA) as a control parameter specifying the degree to which the adapted luminance mapping function deviates from the reference luminance mapping function; The above calculation of the adapted luminance mapping function involves finding the position on the metric that corresponds to the maximum luminance value (PLA), a first endpoint of the metric corresponds to a first maximum luminance (PL_V_HDR) and a second endpoint of the metric corresponds to a reference maximum luminance; The maximum luminance value (PLA) is calculated based on the maximum luminance (PL_D) of a display to which the output image is supplied and a black level value (b) of the display; The above calculation involves applying the inverse of the electro-optical transfer function to the maximum luminance (PL_D), subtracting the black level value (b) from the resulting value, and applying the electro-optical transfer function to the subtraction to obtain the maximum luminance value (PLA) which is used to calculate the adapted luminance mapping function.

[0127] The control maximum luminance should not be confused with the output maximum luminance of the output image, which functions to have an optimal regrading behavior and includes taking into account the black level. The output maximum luminance, on the other hand, is usually the maximum pixel luminance occurring in the generated output image (or, in a video of temporally consecutive images, usually the maximum luminance of at least one or several of the pixels of at least one of the video images). The PLA determines the location of the metric between the diagonal representing the input luminance and the locus of the curve of the criterion function, and all the positions on the located metric define the adjusted function required to obtain the output luminance.

[0128] In many cases, the input image is a high dynamic range image (e.g., a master image color-graded according to the best predictable dynamic range quality, such as a PL_V image at 5000 nit). In many cases, the reference image for which the reference luminance mapping function is defined is an LDR image, i.e., typically an image with PL_V_LDR=100 nit (but other variations are possible, e.g., the input image is an LDR image or a mid-dynamic range image). The target displays come in all capacities, ranging from traditional LDR displays (with a PL_D close to 100 nit, i.e., a PL_D that can be assumed to be approximating / 100 nit for all practical purposes), through low-end HDR displays, such as 700 nit displays, to high-quality HDR displays with a PL_D of 1000 nit or more, to high-end HDR displays, e.g., even higher-end HDR displays, e.g., above 3000 nit, to displays above 5000 nit. Viewing conditions vary from a very dark living room (e.g., no lights on except the TV) to dimly lit, daytime indoors, outdoors.

[0129] When "images with luma" are mentioned, the reader should understand that pixel luma is typically encoded as luma (e.g., 10-bit), but luma can be uniquely converted to a corresponding luma via an EOTF that defines an absolute luminance (e.g., known by the receiver by prior agreement in a single EOTF coding system, or required to be notified via metadata if variable). Similarly, luma processing may in fact be performed as luma processing in the luma domain, but is ultimately luma processing, at least since the end display displays a certain luminance (e.g., up to its maximum display capability of 750 nits).

[0130] This regrading style single control parameter approach should not be confused with, for example, methods that simply base the transformation on minimum black (ie, without going through a specifically redefined maximum luminance).

[0131] The key is not just to reuse the display adaptation approach (algorithm), but to make it closely follow the shape of the reference luminance mapping function, which has been carefully designed by the video creator and which indicates the specific luminance re-grading needs of the luminance of various objects in any particular HDR scene image.

[0132] Note that this approach can be used for any non-decreasing shape of the reference luminance mapping function rather than relying on a fixed display adaptation strategy. Furthermore, the display adaptation approach is characterized by being automatically adapted to any display in the mathematical process, which optimally takes into account the dynamic range of the output display up to its PL_D.

[0133] This mathematical process of display adaptation can be used to optimize for a lower display maximum luminance, just like existing display adaptations; however, in this approach, the maximum luminance used in this display adaptation, i.e. the maximum luminance that determines the position on the metric where the points of the display adapted luminance mapping function (FL_DA) are located, is defined quite differently - it is always the maximum luminance PL_D of the display to which the display-optimized image of the input image is fed.

[0134] Preferably, after display adaptation is performed with an adjusted maximum luminance PLA not equal to PL_D, a black level value (b) is added to the output luminances obtained by the display adaptation (preferably these output luminances are expressed as lumas in the luma domain of the perceptual quantizer).

[0135] Advantageously, the present invention may be embodied as a color conversion circuit (600), the color conversion circuit (600) comprising: An input connector (601) for receiving an input image; an output connector (699) for outputting an output image calculated by the color transformation circuit based on the input image; A metadata input unit (602) for receiving a reference luminance mapping function (F_L) which is metadata associated with an input image; Equipped with the input image has pixels having an input luminance within a first luminance dynamic range (DR_1), the first luminance dynamic range having a first maximum luminance (PL_V_HDR); the reference luminance mapping function specifies a relationship between the luminance of the first reference image and the luminance of the second reference image; The first reference image has a first reference maximum luminance, and the second reference image has a second reference maximum luminance; the input image is equal to one of the first reference image and the second reference image; the output image is not equal to either the first reference image or the second reference image; The color transformation circuit (600) comprises a luminance mapping circuit (310) for applying an adapted luminance mapping function (FL_DA) to input pixel luminances to obtain output luminances; The color conversion circuit (600) comprises a luminance mapping function calculation circuit (611) for calculating an adapted luminance mapping function (FL_DA) based on a reference luminance mapping function (F_L) and a maximum luminance value (PLA) as a control parameter specifying the degree to which the adapted luminance mapping function deviates from the reference luminance mapping function; The above calculation involves finding the position on the metric that corresponds to the maximum luminance value (PLA), a first endpoint of the metric corresponds to a first maximum luminance (PL_V_HDR) and a second endpoint of the metric corresponds to a maximum luminance of one of the first and second reference images that is not equal to the input image; The luminance mapping function calculation circuit (611) is characterized in that it receives a maximum luminance (PL_D) of a display to which an output image is provided, and a black level value (b) of the display; The luminance mapping function calculation circuit (611) calculates the maximum luminance value (PLA) by applying the inverse of the electro-optical transfer function to the maximum luminance (PL_D), subtracting the black level value (b) from the resulting value, and applying the electro-optical transfer function to the subtraction to obtain the maximum luminance value (PLA).

[0136] Advantageously, the color transformation circuit (600) comprises a luminance mapping function calculation circuit (611) which applies a perceptual quantizer function as the electro-optical transfer function.

[0137] Advantageously, the color conversion circuit (600) comprises a display mapping circuit (320) which adds a black level value (b) in the luma domain of the perceptual quantizer to the luma value of the output luminance expressed in the luma domain of the perceptual quantizer. [Brief description of the drawings]

[0138] These and other aspects of the method and apparatus according to the invention will be elucidated with reference to the implementations and embodiments described below and with reference to the accompanying drawings, which serve only as non-limiting specific examples illustrating more general concepts and in which dashed lines are used to indicate that components are optional. Components not shown with dashed lines are not necessarily required. Dashed lines are also used to indicate that an element is described as required but is hidden inside an object or for intangible things such as object / area selection.

[0139] [Figure 1] Schematically illustrates some typical color transformations that occur when optimally mapping a high dynamic range image to a corresponding image of a lower dynamic range (e.g., a standard dynamic range image with a maximum brightness of 100 nits) that is optimally color graded and looks similar (as similar as desired and possible given the difference between the first dynamic range DR_1 and the second dynamic range DR_2). In the case of losslessness, this also corresponds to the mapping of a received SDR image, which actually encodes an HDR scene, to a reconstructed HDR image of that scene. Luminance is shown as a position on a vertical axis from darkest black to maximum brightness PL_V. The luminance mapping function is symbolically illustrated by an arrow that maps the average object luminance from the luminance of the first dynamic range to the second dynamic range (those skilled in the art will understand how to equivalently draw this as a conventional function (e.g., axes normalized to 1, normalized by dividing by the respective maximum luminance). [Diagram 2]1 shows an example high-level diagram of Applicant's recently developed technique for encoding high dynamic range images (i.e., images that can typically have a brightness of at least 600 nits or more (typically 1000 nits or more)) that can actually communicate an HDR image either by itself or a corresponding brightness regraded SDR image plus metadata encoding a color transformation function that includes at least a properly determined brightness mapping function (F_L) to pixel colors that is used by a decoder to convert the received SDR image to an HDR image. [Diagram 3] As a (non-limiting) preferred embodiment, we provide details of the internals of an image decoder, in particular the pixel color processing engine. [Figure 4] Comprised of partial images Fig. 4a and Fig. 4b, it shows two possible variants of display adaptation to obtain a final display-adapted luminance mapping function FL_DA that is used to calculate an optimally display-adapted version of an input image for a specific display capability (PL_D) from a reference luminance mapping function F_L that codifies the needs of luminance regrading between two reference images. [Diagram 5] The principles of display adaptation are summarized more generally to facilitate better understanding as components of the formulation and claims of the present embodiment. [Figure 6] 2 illustrates a receiver (particularly a color processing circuit) according to one embodiment of the present invention; [Figure 7] Further embodiments are disclosed which in particular elucidate the distribution of processing between the linear domain (which may be advantageous and which may, if desired, be further mapped to a psychovisually uniform domain within the linear processing) and the perceptual quantizer domain, and which also elucidate in particular advantageous aspects of adding a black level b to the display. [Figure 8]The sub-figures 8a, 8b and 8c show some effects that result from applying the process. Fig. 8a shows an exemplary image with dark pixel regions such as the tires under a police car and the interior of a house seen through a window such as a circular attic window, pixel regions with mid-range average brightness such as the body of the police car and people, and bright HDR pixel regions such as the flashing lights on the police car. Fig. 8b shows these objects and their average brightness spread around the exemplary HDR image brightness range, and also shows the MDR image brightness range that is twice the same maximum brightness (maximum brightness of the output image ML_V_out) but obtained by brightness mapping with two different methods. Fig. 8c shows the same effect of the two methods using brightness histograms. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0140] 6 shows an ambient light level adaptation process embodied in a color conversion circuit 600. Such a circuit is typically found in an HDR video decoder of a television signal receiver (e.g., broadcast, video distribution over the Internet, etc.). Physically, it may be in a television display, a mobile phone, a receiving device (e.g., a set-top box), etc. Image pixel colors are assumed to appear one by one and encoded in the PQ domain with luma Y' and chroma Cb, Cr (i.e., luma is calculated from nonlinear red, green, and blue components by applying the inverse EOTF of a perceptual quantizer, and chroma is calculated according to the usual colorimetric definition of video colors).

[0141] The luma component Y' (assumed to be the luma of the HDR image as on the left-most axis in FIG. 1) is processed by a luma mapping circuit (310).

[0142] As metadata input we take a reference luminance mapping function F_L, which is the function to apply if a secondary reference image needs to be generated and if there is no correction for the ambient light level, i.e. the smallest distinguishable black on the display.

[0143] The purpose of the luminance mapping function calculation circuit (611) is to determine an appropriate display and environment adaptive mapping function (FL_DA(t)), where time t can typically change over time, not because the display or viewing environment changes, but because the content of the video image changes in terms of luminance distribution (which typically requires a different luminance regrading function for successive images).

[0144] This function FL_DA(t), if specified in the correct domain (e.g. as an equivalent function on the PQ luma), can be applied directly to the input luma Y' by the luma mapping circuit (310) to generate the required output luma, i.e. luma L'_HDR, i.e. communicated from the luma mapping function calculation circuit (611) to the luma mapping circuit (310) and stored in the memory of the latter, usually at the beginning of a new image to be color processed, before which the new function is stored.

[0145] The luminance mapping function calculation circuit (611) needs the following data to perform its calculations: a reference luminance mapping function F_L, the maximum luminance of the connected display (PL_D), and a black level value (b). The black level value (b) is communicated either as a PQ luma code value or as a luminance in nits that can be converted to PQ luma. The latter two come from the metadata memory 602, e.g., directly from the connected display. The rest of the color processing is as described above in this already described and only elucidated embodiment (e.g., there could be different color processing tracks, since the innovation is mainly concerned with the luminance part).

[0146] Before the luminance mapping function calculation circuit (611) executes the display mapping algorithm configured to derive the FL_DA function based on the F_L function, it needs to calculate a new (adjusted) maximum luminance value (PLA). To be clear, this is not the actual maximum value of the display (i.e., PL_D), but simply a value that acts as if it were the PL_D value in an otherwise unchanged display adaptation algorithm (as described in Figures 4 and 5).

[0147] The luminance mapping function calculation circuit (611) calculates: PLA=EOTF(inverse_EOTF(PL_D)-b) [Formula 3]

[0148] As mentioned above, preferably the EOTF applied is the perceptual quantizer EOTF (EOTF_PQ), but other EOTFs will work as well, especially if they are reasonably psychovisually uniform, i.e. if a step of k luma codes approximately exhibits some visual difference regardless of the starting luma level l at which this step is applied.

[0149] Then, in a preferred embodiment, further processing is performed, namely repositioning to black level value b. To achieve this, the display mapping circuit 320 also receives the display black level value b, namely for this viewing environment situation. The display mapping circuit 320 may have several processing functions to prepare the pixel color output signal (i.e., the display driving color D_C) generally required for a particular display. A useful aspect for realization is to work in the PQ domain, i.e., the display wants to obtain, for example, R'G'B' values ​​(or Y'CbCr values ​​derived therefrom) of PQ type. Therefore, this component also works internally in the PQ domain. The R'G'B'_HDR values ​​may be in another domain (e.g., they may be linear), in which case the display mapping circuit 320 performs an internal conversion to PQ (which, as mentioned above, is done in a standardized manner specified by SMPTE).

[0150] Addition is made to the three PQ domain color components: R'_out=a * R'+b; G'_out=a * G'+b; B'_out=a * B'+b; [Formula 4]

[0151] Again, the constant a can be a scaling factor to make the white of the signal a displayable white, but other approaches such as clipping or non-linear equations can also be used (note that Y', in the standard agreed definition, is a linear combination of the R'G'B' color components with three weights that sum to 1, so we also have Y'_out=Y'+b).

[0152] These output signals then constitute the D_C output. For example: D_C={R'_out,G'_out,B'_out}.

[0153] This approach of adding b has the advantage that the darkest colors start at the visibility threshold, but this method is also useful when applying only Equation 3 (i.e., only the different white setting adjustments), since this already specifically increases the contrast of the image.

[0154] FIG. 7 further helps elucidate the different color regions in the process.

[0155] Typically, if the color is expressed in the PQ domain, the first part of the processing is in the PQ domain (note: colorization processing, i.e., not luminance, but the appropriate color type (e.g., saturated yellow), is an independently configurable processing path, but it can also be done in the PQ domain). In FIG. 7, we have explicitly included a color space conversion circuit 711, which converts PQ luma to linear luminance Ylin. This could also be readily performed in the function of the luminance mapping circuit 310, since typically other color space conversions may be done internally anyway (applicant finds it useful to specify the regrading function definition in the display's optimal psychovisually uniformed color space, and all other processing follows), but that aspect is not essential to the innovation, and therefore should not be confused by delving into this further. Assume that processing starts in PQ in subcircuit (or processing unit) 701, is linear end-to-end between Ylin and L'_HDR for R'G'B'_HDR in subcircuit 702, and is converted back to the PQ domain in subcircuit 703 for the display mapping circuit 320. Here the required output format is generated, and according to the present innovation, an additional correction of the black level is made in the PQ domain (those skilled in the art will understand that the principles can be applied in domains other than this preferred embodiment).

[0156] Finally, there is an output connector 699 for communicating the optimized image and its pixel colors. Similar to the image input connector 601, the output connector physically spans the various technologies available for communicating the image, e.g., HDMI® cable, connection to a wireless channel antenna, etc. The metadata input may be a separate input, but advantageously this metadata is communicated together with the image color data, i.e., SEI messages embedded in the MPEG video, via an Internet connection, or via a separate data package via an HDMI® cable, etc.

[0157] FIG. 8 sheds light on how this method of luminance mapping works and its advantages over existing methods.

[0158] On the leftmost luminance axis of Fig. 8b, the average luminance of various image objects when represented in a 5000 nit HDR image is shown. The geometric structure of the image is shown in Fig. 8a. This is the master grading image, i.e. the highest quality image of the scene, which can usually be doubled as the first reference image, and which can be assumed to be the input image (the input image is, for example, communicated by a broadcaster and received at a TV display which performs a color transformation to a 600 nit MDR image optimized for a display with a PL_D of 600 nit, and the output image is output for a display with a PL_D of 600 nit).

[0159] An image (or video / movie) creator may decide to make the flashlight of a police car very bright, say 4000 nits. An unlit interior part of a house visible through a window (e.g. in a night scene) may be quite dim, being lit only by street lights outside. When creating a master image, the master image is made for the best viewing environment, such as a consumer's dimly lit living room, where there is only a small lamp. Under these circumstances, the viewer can see many different dark colors, so ideally the master image should contain the information of all these indoor pixels. Thus, the indoor object structure is encoded in the master image, for example, around an average value of 2 nits (which may be down to 0.01 nits in some cases). The problem then is that a neighbor watches the same movie, i.e. the same communicated 5000 master HDR image containing the same dark pixels, in a very bright room, possibly during the day. This neighbor may not be able to see the darkest pixels, but in some cases, he or she can see nothing darker than maybe 7 nits.

[0160] The right side of Fig. 8b shows how the received reference image is adapted to an MDR image that not only has a lower maximum luminance, but more importantly, in the present teachings, has a higher black level value b, e.g., 7 nit (even though the MDR image has the same maximum value of 5000 nit).

[0161] Method METH1 is the method described above using Equation 2, and is the way one skilled in the art would normally solve the problem: identify that there are some dark pixels that are not visible, and therefore put them all in the visible range above the black level b.

[0162] How this works is shown in the 20 nit window and further in the first histogram 810 of FIG. 8c. The shift to 20 nit is rather large, resulting in suboptimal visual contrast in the resulting MDR image. Furthermore, this correction method spreads the offset correction of dark levels quite far, in fact over the entire range, so that, for example, mid-luminances of people are also shifted significantly to brighter values ​​(which may also have a suboptimal effect on these regions, i.e., overall image contrast). This can be seen from the shift of the histogram lobes of dark and mid-luminance objects in the histogram of the conventional method relative to the second histogram 820. The second histogram 820, with its luminance axis labeled METH2, represents the new present approach.

[0163] Not only will the image generally look better with more realistic contrast, but applying this method to the display adaptation process will allow the movie creator's desiderata to be followed more realistically, i.e., specifically as expressed in the reference regrading luminance mapping function F_L. For example, the grader can specify this function so that the darkest luminance subranges, e.g., indoor pixels, are boosted significantly when mapped to a lower maximum luminance. Meanwhile, the local slope of the mapping in the HDR luminance subrange where the car's luminance falls can remain low. The grader can indicate that in this story, the interior of the house is important (compared to a scenario where the interior is not important and is simply buried in invisible black). So, each time the luminance mapping is done to a gradually lower maximum luminance image, a gradually larger normalized subrange of available luminance is used for the darkest indoor pixels, which will look relatively good on a dim display (in this case, they will look relatively good on a brighter display, but with less visibility of the dark pixels). The present approach allows these technical desiderata to be followed as accurately as possible. It does not map to the ideal available situation, i.e. does not redistribute from 0 nit to PL_Dnit, but does redistribute from the actual available brightness b nit to PL_Dnit, and then uses this range communicated in the F_L function optimally as best as possible as intended by the content creator, which is usually satisfactory for the viewer. This is at least an elegant and non-difficult practical way to do it.

[0164] A numerical example is given to illustrate the nonlinear aspects of EOTF_PQ. The lowest luma corresponds to a very dim luminance, up to fairly high values. Thus, brighter dark levels usually correspond to higher luma values. Thus, the PLA values ​​are significantly lower compared to the PL_D values, but are still optimally used in the present method. For example, PL_D=1000 nit, EOTF_PQ(b)=0.3 nit. In this case, b as the PQ luma is 0.097 *1023=99. Inverse_EOTF(1000) is 0.75 * 1023=767. The difference between these two is 668.

[0165] The question now is, to what white level does this luma level correspond to, which is adjusted to control the display adaptation.

[0166] EOTF_PQ(668 / 1023)=EOTF_PQ(0.65)=408 nits. Therefore, the PLA value of brightness (quantified as minimum visibility on the display screen) for such a display and viewing environment is 408 nits. This value is input into any embodiment of the display adaptation used by the luminance mapping function calculation circuit (611) to generate the correct display-adapted luminance mapping function FL_DA. This function is not only optimized for the display, but also for the viewing environment.

[0167] The algorithmic components disclosed in this text may actually be implemented (in whole or in part) in hardware (e.g., as part of an application specific IC) or as software running on a specialized digital signal processor or a general purpose processor, etc.

[0168] A person skilled in the art should be able to understand from the proposed description which components are optional improvements and can be realized in combination with other components, and how the (optional) steps of the method correspond to the respective means of the apparatus, or vice versa as well. The word "apparatus" in this application is used in the broadest sense, i.e. it is a group of means making it possible to realize a particular purpose, and thus may be, for example, (a small circuit part of) an IC or a dedicated device (e.g. a device with a display), or part of a networked system, etc. An "arrangement" is also intended to be used in the broadest sense, and therefore may in particular include a single device, a part of a device, a collection of (parts of) cooperating devices, etc.

[0169] The explicit meaning of a computer program product should be understood to encompass any physical embodiment of a set of commands that, after a series of loading steps (which may include intermediate conversion steps such as translation into an intermediate language and a final processor language), allows a general-purpose or special-purpose processor to input the commands to the processor and execute any of the characteristic functions of the invention. In particular, a computer program product is realized as, for example, data on a carrier such as a disk or tape, data present in a memory, data via a network connection (wired or wireless), or as program code written on paper. Apart from the program code, characteristic data necessary for the program may also be embodied as a computer program product.

[0170] Some of the steps required for the operation of the method may not be written in the computer program product but may already be present in the functionality of the processor, such as data input steps and data output steps.

[0171] It should be noted that the above embodiments are illustrative rather than limiting the present invention. For the sake of brevity, not all of these options are mentioned in detail, if a person skilled in the art can easily realize the mapping of the presented examples to other areas of the claims. Other combinations of elements are possible other than the combinations of the elements of the present invention combined in the claims. Any combination of elements can be realized in a single dedicated element.

[0172] Any reference signs between parentheses in the claims are not intended to limit the claim. The word "comprising" does not exclude the presence of elements or aspects not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements.

Claims

1. 1. A method for processing an input image to obtain an output image, comprising the steps of: the input image has pixels having an input luminance within a first luminance dynamic range, the first luminance dynamic range having a first maximum luminance; A reference luminance mapping function is received as metadata associated with the input image; the reference luminance mapping function specifies a relationship between a luminance of a first reference image and a luminance of a second reference image; the first reference image has a first reference maximum luminance, and the second reference image has a second reference maximum luminance; the input image is equal to one of the first reference image and the second reference image; the output image is not equal to either the first reference image or the second reference image; The process includes applying an adapted luminance mapping function to input pixel luminances to obtain output luminances; the adapted luminance mapping function is calculated based on the reference luminance mapping function and a maximum luminance value as a control parameter specifying the degree to which the adapted luminance mapping function deviates from the reference luminance mapping function; said calculating said adapted luminance mapping function includes finding a position on a metric that corresponds to said maximum luminance value; a first endpoint of the metric corresponds to the first maximum luminance and a second endpoint of the metric corresponds to a maximum luminance of one of the first and second reference images that is not equal to the input image; The maximum luminance value is calculated based on a maximum luminance of a display to which the output image is supplied and a black level value of the display; The method, wherein the calculation includes applying an inverse of an electro-optical transfer function to the maximum luminance, subtracting the black level value from the resulting value, and applying the electro-optical transfer function to the subtraction to obtain the maximum luminance value used in the calculation of the adapted luminance mapping function.

2. 2. The method according to claim 1, wherein a perceptual quantizer function is used as the electro-optical transfer function.

3. The method of claim 1 or 2, wherein the black level value is added to the output luminance expressed as luma in the luma domain of a perceptual quantizer.

4. an input connector for receiving an input image; an output connector for outputting an output image calculated by a color transformation circuit based on said input image; a metadata input unit for receiving a reference luminance mapping function, the reference luminance mapping function being metadata associated with the input image; The color conversion circuit comprises: the input image has pixels having an input luminance within a first luminance dynamic range, the first luminance dynamic range having a first maximum luminance; the reference luminance mapping function specifies a relationship between a luminance of a first reference image and a luminance of a second reference image; the first reference image has a first reference maximum luminance, and the second reference image has a second reference maximum luminance; the input image is equal to one of the first reference image and the second reference image; the output image is not equal to either the first reference image or the second reference image; 1. A color conversion circuit comprising a luminance mapping circuit for applying an adapted luminance mapping function to input pixel luminances to obtain output luminances, the circuit comprising: the color transformation circuit comprises a luminance mapping function calculation circuit for calculating the adapted luminance mapping function based on the reference luminance mapping function and a maximum luminance value as a control parameter specifying the degree to which the adapted luminance mapping function deviates from the reference luminance mapping function; said calculating includes finding a position on a metric that corresponds to said maximum luminance value; a first endpoint of the metric corresponds to the first maximum luminance and a second endpoint of the metric corresponds to a maximum luminance of one of the first and second reference images that is not equal to the input image; the luminance mapping function calculation circuit receives a maximum luminance of a display to which the output image is provided and a black level value of the display; a color conversion circuit, characterized in that the luminance mapping function calculation circuit calculates the maximum luminance value by applying an inverse of an electro-optical transfer function to the maximum luminance, subtracting the black level value from the resulting value, and applying the electro-optical transfer function to the subtraction to obtain the maximum luminance value.

5. 5. The color conversion circuit of claim 4, wherein the luminance mapping function calculation circuit applies a perceptual quantizer function as the electro-optical transfer function.

6. A color conversion circuit as described in claim 4 or 5, comprising a display mapping circuit that adds the black level value in the luma domain of a perceptual quantizer to the luma value of the output luminance represented in the luma domain of the perceptual quantizer.

Images