DEVICE AND METHOD FOR TONE AND COLOR GAUGE ASSIGNMENT

Description

Technical Field

[0001] This invention relates to image processing, image display, and image reproduction. Embodiments of the invention provide methods and apparatus which process image data to transform colors and / or tones for display or reproduction locally or at a downstream device. Some example embodiments are displays configured to receive and process image data and to display processed image data for viewing.Background

[0002] Commercial video production typically involves acquiring and editing video sequences. Before being distributed, colors in the video production may be adjusted in a color grading step. Color grading typically involves displaying the video production on a reference display under ideal viewing conditions and adjusting colors and tones in the video production to achieve a desired artistic intent. For example, a person operating color grading apparatus (a >color grader=) may decide to make the mood of a scene gloomy by reducing the brightness of the image. Artistic goals may also be met by making other adjustments, for example, changing the colors of certain displayed objects, changing the overall contrast in the scene, changing the overall brightness of the scene, and the like. The result, when displayed on the reference display, matches the artistic intent of the color grader.

[0003] A wide number of alternative display technologies are available commercially. These range from digital cinema displays to television sets which may be based on cathode ray tubes, plasma displays, LED displays, LCD panel displays in which the LCD panel is backlit by various types of backlight systems, and the like. Different displays have different capabilities. For example, some displays may have a broader color gamut than other displays. Some displays may be capable of reproducing a broader range of brightnesses and / or a higher maximum brightness than other displays. The inventors have recognized a need for methods and apparatus capable of transforming image data in a manner which can take advantage of the capabilities of a particular display without compromising the artistic intent represented by the images in the image data.

[0004] This need also extends to reproductions of still images by various types of printing, engraving, and other image reproduction technology in which the reproduction technology may have capabilities for reproducing colors and / or tones that are different from those assumed by the image data.

[0005] There remains a need for flexible and cost-effective tone and / or gamut mapping apparatus and methods. There is a particular need for such apparatus and methods that may be applied to produce enjoyable images on high quality displays.

[0006] US 2003 / 123722 A1 discloses a data storage format for storing images with extended color gamut that is compatible with limited color gamut imaging devices. The method for storing the images includes identifying regions within the color image that contain extended gamut color values, using a compact notation to store an identification of each extended color gamut region in a second section, gamut-mapping the image data pertaining to the extended gamut regions to a limited gamut color space, storing the gamut-mapped image data together with all image data not in an extended gamut region in a first section of the data storage format, and storing image data for the extended gamut regions in an extended gamut color space in a third section of the data storage format such that the limited color gamut image stored in the first section can be combined with the extended gamut data stored in the third section to produce an extended color gamut digital image.

[0007] US 2008 / 123164 Al discloses a colour processing apparatus used for colour processing and converting colour data between colour gamuts. The apparatus comprises an inverse collector that performs inverse correction to mapped colour data based on correction region of output gamut. The apparatus further comprises an input image collector that determines whether a lightness value of a pixel of interest falls inside the lightness range corresponding to the correction region of the input gamut. When the lightness value of the pixel of interest falls inside the lightness range, the input image collector corrects the colorimetric value of the pixel of interest.

[0008] US 7 280 705 B1 discloses a digital imaging system wherein tone correction is applied using a blending mask to select an appropriate tone correction curve so as a high contrast and high dynamic range image with areas lit by multiple light sources can be rendered properly on a display.Summary

[0009] The invention is defined by the independent claims. The dependent claims concern optional features of some embodiments of the invention.

[0010] A range of aspects of the invention and example embodiments of the invention are described below and illustrated in the accompanying drawings.Brief Description of Drawings

[0011] The appended drawings illustrate non-limiting example embodiments of the invention. Figure 1A is a block diagram of components of apparatus according to one embodiment that may be located at the display end. Figure 1B is a block diagram of components of apparatus according to one example useful for understanding the invention and that may be located at the color grading tool. Figure 2A is a flow chart illustrating a method according to an exampleuseful for understanding the invention. Figure 3 is a block diagram illustrating apparatus according to another example useful for understanding the invention. Figure 4 is a block diagram illustrating apparatus according to another exampleuseful for understanding the invention. Figure 5A is a graph of tone curves for standard video data and for an area of an image where there is a light source. Figure 5B is a graph of tone curves for standard video data and for areas of an image where there are emissive and reflective light sources. Figure 6 is an example representation of metadata which has been pre-defined to describe tone / gamut parameters. Figure 7 is a flow chart illustrating the delivery of video data and metadata through a portion of a video delivery pipeline. Figure 8 schematically illustrates a color grading process according to one example useful for understanding the invention. Description

[0012] Throughout the following description specific details are set forth in order to provide a more thorough understanding to persons skilled in the art. However, well known elements may not have been shown or described in detail to avoid unnecessarily obscuring the disclosure. Accordingly, the description and drawings are to be regarded in an illustrative, rather than a restrictive, sense.

[0013] Figures 1A is a high level block diagram illustrating a tone and gamut mapping apparatus 10 according to an embodiment of the present disclosure. Components of apparatus 10 (e.g. such as those shown in Figure 1A) are located at the display end. Certain other components of apparatus 10 (e.g. such as those shown in Figure 1B) may be located at a color grading tool, upstream of the display, and may operate with a color grader toolset (e.g. a set of color grading functions provided by the color grading tool) to drive the metadata creation. A color grading tool may comprise, for example, software and / or hardware-based systems operable by a color grader to receive input image data and to adjust colors and / or luminance of pixel data in areas of the input image data to achieve a desired artistic effect for the image.

[0014] At the display end, apparatus 10 has an input 12 at which image data 14 is received. Image data 14 may comprise video data, data specifying one or more still images or the like. Input 12 may comprise a physical input, such as a serial or parallel data bus, a wireless data input, a memory location from which image data 14 may be accessed, an interface which receives image data 14 from some data communication channel, hardware for retrieving image data 14 from a medium, such as a DVD, CD-ROM, flash memory, or other data record, or the like.

[0015] Image data 14 comprises pixel data 15 for pixels in an image to be displayed. Each set of pixel data 15 comprises a number of color coordinates, in this example C1, C2, and C3, which indicate the color and tone for a pixel. Pixel data 15 also includes either explicitly or implicitly location information, in this example identified by the coordinates X and Y, for a pixel represented by pixel data 15. Location information X, Y may be represented by an offset or other relative location within image data 14 of a particular pixel data 15. In the example embodiment shown in Figure 1A, location information X and Y are shown explicitly.

[0016] Apparatus 10 optionally includes a coordinate transform block 16 which transforms coordinates C1, C2, and C3 into a color space different from a color space of image data 14. In some embodiments, it is convenient for apparatus 10 to work internally in a different color space. The different color space may be, for example, a linear color space (i.e. a color space in which a physical value such as brightness (e.g., lightness or intensity or luminance) is represented linearly). In such embodiments, coordinate transform block 16 may perform a translation into a linear color space. In certain embodiments, coordinate transform block 16 may perform a translation into IPT, LAB or RGB color space or another color space which is more suitable for tone or gamut mapping than the color space of image data 14.

[0017] Block 18 receives pixel data 15 and performs a mapping of the pixel data 15 to mapped pixel data 15=. The mapping may be performed through use of lookup tables, logic circuits which execute mapping algorithms, or the like. The particular mapping performed is specified by mapping parameters 20 which influence the operation of mapping logic 19.

[0018] Mapped pixel data 15= is output as mapped image data 14=. Output of mapped image data 14= may comprise, for example, placing mapped pixel data 15= into a file or buffer, onto a serial or parallel data bus, transmitting the mapped pixel data 15= supplying the mapped pixel data 15= to a downstream device or circuit or the like.

[0019] In the illustrated embodiment, mapping parameters 20 include parameters 22 set by a configuration mechanism 26. Configuration mechanism 26 sets mapping parameters 22 according to metadata received from a metadata extraction / reception unit 24. In some embodiments, metadata is embedded in a signal, data structure, or other representation of image data 14 and is extracted by metadata extraction / reception unit 24 from the image data 14. In other embodiments, metadata is received on a separate channel, in a separate data structure, or the like. In any event, the operation of mapping logic 19 is controlled by parameters 22 which may be set or selected, at least in part, in response to metadata.

[0020] In the illustrated embodiment, parameters 22 include general parameters 22A, color parameters 22B, and location parameters 22C. In this embodiment, the mapping performed by mapping logic 19 may vary depending upon the color coordinates represented in a pixel data 15 being mapped and / or the location information corresponding to the pixel 15 being mapped. Example applications of these capabilities are described below.

[0021] In an example embodiment, metadata 21 supplied to configuration unit 26 includes general metadata to be applied to all of or selected portions of video data 14. For example, the general metadata may be specified for selected scenes or frames within video data 14. Configuration unit 26 may be configured to establish (for example by selection or calculation) a set of mapping parameters 22 to use for mapping the image data 14 based upon that general configuration metadata. The general configuration metadata may be specific to a particular display or image reproduction technology within which apparatus 10 is embedded or which is downstream from apparatus 10.

[0022] Metadata presented to configuration unit 26 also includes metadata which specifies particular mapping behaviour for particular colors. According to the present disclosure, the metadata specifies one or more reserved colors which should not be affected by the transformation of block 18. Such reserved colors may be specified in parameters 22B. Mapping logic 19 is configured to make transformations in such a manner that reserved colors identified in parameters 22B are not changed by the transformation.

[0023] As another example, in some embodiments, the mapping specified by mapping parameters 22 may have the capability of mapping at least some colors to transformed colors outside of a gamut represented in image data 14. In some embodiments, colors within a range of colors lying close to the boundary of gamut 14 may be subjected to a transformation which is different from a transformation applied to colors that are at color coordinates far from the boundary of the gamut in which image data 14 is represented.

[0024] As another example, one or more selected colors from image data 14 may be associated with corresponding target colors outside of the gamut of image data 14. In such cases, mapping logic 19 may be configured to map the selected colors toward the corresponding target colors to a color that is within the display capabilities of the display or other target device.

[0025] Mapping logic 19 is also configured to perform different transformations depending upon the location information corresponding to pixel data (e.g. on the coordinates X, Y in the illustrated example). For example, metadata received by configuration unit 26 may specify the locations of light sources within a current image. Different mappings may be applied depending upon whether or not the location information corresponding to pixel data being processed indicates that the pixel data corresponds to a location which is in a light source or outside of a light source. Different mappings may be applied by adjusting the tone curves. For example, Figure 5A illustrates a tone curve for standard video data (without any adjustments for light sources) and a tone curve adjusted for an area identified as a light source (the Aadjusted tone curve@). To ensure seamless adjustment, the adjusted tone curve may be matched to the tone curve for standard video data at the boundary between light source and non-light source areas. For example, as seen in Figure 5A, toward the lower input pixel drive values (e.g. corresponding to areas outside of the area identified as the light source), the adjusted tone curve may be the same as the tone curve of the standard video data. At some value between the lower input pixel drive values and the upper input pixel drive values, the adjusted tone curve diverges from the tone curve of the standard video data. The adjusted tone curve may differ from the tone curve of the standard video data for the upper input pixel drive values, which may correspond to the area identified as the light source (e.g. upper input pixel drive values may be mapped to output values higher than for standard video data, as shown in Figure 5A). At the upper input drive values, the adjusted tone curve defines the intensity of the pixels in the area identified as the light source. The adjusted tone curve of Figure 5A shows how the intensity of pixels in the light source area may be adjusted, for one example embodiment. In other embodiments, the adjusted tone curve may be a different curve than shown in Figure 5A (e.g. the adjusted tone curve may have a particular inflection point which may be positioned up or down the curve, for various embodiments). Depending on the desired effect, a particular adjusted tone curve may be applied to an image to affect how the intensity of the light source area of the image is adjusted, relative to other areas of the image.

[0026] In some embodiments, different mappings are applied depending upon the type of light source. For example, different mappings may be applied depending upon whether the light source is identified in the metadata as being a reflective light source or an emissive light source. In some embodiments, separate mapping algorithms and / or mapping parameters are provided for each of a plurality of different types of light sources. Figure 5B illustrates example tone curves to be applied for areas identified as having emissive and reflective light sources and a tone curve for standard video data (without any adjustments for the light sources).

[0027] Figure 1B represents an example useful for understanding the invention and illustrates components of apparatus 10 that may be located at the color grading tool. At the color grading tool, apparatus 10 receives input image data 14A at input 12A and outputs color graded image data 14B (which may include metadata 21). Image data 14A, 14B may comprise video data, data specifying one or more still images or the like. Image data 14A may comprise pixel data 15A for pixels in an image to be displayed. The color grading tool may comprise a color grading processor 11, which receives pixel data 15A and executes one or more color grading functions in response to user input so as to result in color graded pixel data 15B. Color grading functions may be implemented by software and / or hardware. Operation of color grading functions by processor 11 may be controlled by a professional color grader. For example, the color grader may decide to intensify certain colors, or alter luminance, saturation and / or hue of pixels in an image, and may provide appropriate user inputs to the color grading tool to cause selected color grading functions of the color grading tool to be applied to the pixel data 15A, so as to achieve the desired artistic effect.

[0028] In the illustrated embodiment, color grading processor 11 is communicatively coupled to a metadata generator 17. Metadata generator 17 may have access to, or may generate, a set of metadata definitions 13 which may describe tone / gamut parameters. Metadata generator 17 may have access to color graded pixel data 15B. Metadata generator 17 may generate metadata 21 for transmission with the color graded pixel data 15B. Metadata generation may be based on image characteristics and adjustments to the pixel data by color grading processor 11 as communicated by color grading processor 11 and / or determined from color graded pixel data 15B. Metadata generator 17 may identify parameters of the color graded pixel data to drive metadata creation.

[0029] In the illustrated embodiment, metadata generator 17 includes a program memory 23 for storing software functions, which when executed by a processor 27 of metadata generator 17, may identify parameters of the color graded pixel data such as, for example, locations of light sources in the image, colors which are intentionally color graded out of gamut, reserved colors, etc. Metadata generator 17 may use information about such parameters to generate metadata 21. Metadata 21 generated by metadata generator 17 may be combined with or transmitted with color graded pixel data 15B and output as image data 14B to a downstream device. In certain embodiments, components of apparatus 10 as shown in Figure 1A may receive image data 14B (Figure 1B) as image data 14.

[0030] Figure 8 illustrates a color grading process according to an example useful for understanding the invention. A professional color grader 35 starts the process by controlling a reference display to display video data from an external video content source; such video data may have been created through camera capture, CGI (computer generated imagery), etc. In the color grading process, while viewing the displayed video data on the reference display, the color grader 35 uses a color grading tool 88 to adjust or modify a video frame 78A (or a sequence of frames (i.e. scene)) according to the creator=s artistic intent. Color grading processor 11 adjusts video frame 78A according to inputs from color grader 35 to produce color graded video frame 47.

[0031] In the color grading process, portions of each frame may be modified differently for: particular image areas (e.g. a particular tone mapping may be applied to an image area corresponding to a light source location); special colors (e.g. certain colors or tones / gamut may be mapped according to a color-specific mapping); and / or reserved colors (e.g. adjustment of skin tones, grass tones, and other tones designated as reserved colors in the color grading process may be inhibited during tone / gamut mapping). Within a frame or scene, localized tone or gamut flags may be set on a metadata grid 41 defining the location of particular pixels for which the tone / gamut has been adjusted in the color grading process; and / or the location of particular pixels for which tone / gamut has deliberately not been adjusted in the color grading process (e.g. as may be the case for reserved colors). The localized tone or gamut flag may identify information about the tone / gamut adjustment made to the pixel. Color graded video content, localized tone or gamut flags and pixel locations corresponding to each localized tone or gamut flag may be saved to a storage or buffer device 49. Localized tone or gamut flags and the information indicated by such flags may be saved into a frame metadata container 48 as accumulated localized metadata.

[0032] A codec 82 receives the metadata stored in metadata container 48, and color graded content in video frame 47 and local tone / gamut adjustments (as stored in storage device 49 ), and encodes the metadata and color graded content for distribution. Methods for encoding by codec 82 may include, for example: Encoding accumulated localized metadata and placing into metadata at the start of the video picture or scene. The video and frame metadata may be encoded and may follow the accumulated metadata. The frame metadata may be formatted to index the accumulated localized metadata to minimize the amount of data required to transmit the metadata.

[0033] $ Ignoring the accumulated metadata and encoding the frame metadata along with the color graded video (this may increase overall bandwidth).

[0034] Figure 7 illustrates a portion of a video delivery pipeline which begins with color grading stage 80 and ends with display tone / gamut mapping stage 84. Other stages preceding or succeeding either of stages 80, 84 of the video delivery pipeline are not shown. Components of apparatus 10 shown in Figure 1B and / or color grading tool shown in Figure 8 may be used at the color grading stage 80 to generate metadata for tone / gamut mapping purposes at downstream devices. The metadata may be delivered along with video data to a codec stage 82 for encoding and / or decoding the video data and metadata. In some embodiments, the metadata may be delivered by a separate communication channel. The video data and metadata are delivered to a downstream device for tone / gamut mapping at stage 84. Components of apparatus 10 shown in Figure 1A may be used at tone / gamut mapping stage 84 to implement the tone / gamut mapping methods described herein.

[0035] Figure 2A illustrates a method 25 which may be applied to mapping image data 14 to mapped image data 14=. In block 28, metadata is obtained. The metadata may be extracted from a video stream, a file containing video data, a separate data structure, received on a separate data stream, or the like. Metadata may be obtained prior to and / or during obtaining image data. Block 28 may be performed continuously during processing of image data in some embodiments.

[0036] In block 29, mapping logic is configured in response to received metadata. The configuration may include, for example, writing values to lookup tables used for mapping color values and / or tone values; recording reserved colors, identifying special color ranges having separate mappings, or modified mappings; identifying special locations having specific mappings or modified mappings; writing parameter values to registers accessed by mapping algorithms; configuring configurable logic implement mapping algorithms or the like. Block 29 may be repeated each time updated metadata is received.

[0037] Metadata may include metadata that affects the manner in which color / luminance values for pixel data are shown on a display or the manner in which mapping is performed from the transmitted color / luminance values to the mapped color / luminance values (in either the same or a different color space). Metadata may also include metadata that assists in abbreviating metadata or reducing the amount of data required to transmit metadata. For example, in some embodiments, the metadata may define tone / gamut maps for later use in tone / gamut mapping. Subsequent metadata may indicate that for certain pixels, one of the predefined gamut maps is to be used. The predefined maps may be stored or buffered at the display end of a video delivery pipeline in a data structure that may be referenced by the mapping apparatus. The data structure may be, for example, a one-dimensional or two-dimensional data structure.

[0038] According to the present disclosure, the metadata defines metadata flags providing information about tone or gamut adjustments made in the color grading process. Information about tone or gamut adjustments identify: colors that have not been adjusted because they are reserved colors; optionally: colors that have been deliberately color graded out-of-gamut; special colors to be mapped according to a color-specific map, etc. The metadata flags and corresponding tone or gamut adjustment information may be stored in a data structure (e.g. a look-up table) that may be stored or buffered at the display end and later referenced by the mapping apparatus (e.g. see Figure 6 showing localized tone or gamut flags set at pixels on a metadata grid corresponding to a frame of image data, and a predefined gamut metadata structure). Subsequent metadata transmitted to the display end may indicate that particular localized tone or gamut flags correspond to certain pixels in the pixel data. Such information may in turn be used by the mapping apparatus to look up in the stored metadata structure the tone or gamut adjustments made to those pixels, and to select and apply a tone / gamut mapping method based on such tone or gamut adjustments.

[0039] Method 25 includes a data processing sequence 25B which performs mappings of pixel data. Sequence 25B may be repeated for each pixel data 15 in image data 14. In block 30, pixel data is received. Block 30 may, for example, comprise retrieving a next pixel data from a buffer; retrieving pixel data from a memory; receiving pixel data on an input line such as a parallel or serial bus; or the like. In block 32 the color coordinates of the pixel data are compared to a set of zero or more reserved colors. If there are no reserved colors then block 32 is optional and need not be performed. If the color coordinates of the pixel data being processed correspond to a reserved color, then either the pixel data is passed through unmapped as indicated by path 33, or, the pixel data is passed through block 34 which performs mapping of the tone of the pixel data only (for example, block 34 may expand a dynamic range of the pixel data or otherwise change a luminance value of the pixel data to suit a downstream application). In certain embodiments, the tone is preserved for pixels that represent particular objects or areas of the image (e.g. pixels representing skin, grass, sky, etc.). Accordingly, reserved colors may correspond to those used to depict such objects. Reserved colors may be set by a color grader at the color grading stage (e.g. the color grader may designate certain tones as skin tones). During mapping of the pixel data, tones that are within a predetermined margin of the reserved colors may be preserved.

[0040] In block 36, location information corresponding to the pixel data is compared to a set of zero or more special locations. If no special locations have been specified by metadata, then block 36 is optional and need not be performed. Where special locations have been identified then block 36 determines whether the location information for the pixel data corresponds to a special location and, if so, makes active a location-specific mapping for the pixel data in block 38.

[0041] According to the present disclosure, special locations are be associated with different types of mapping algorithms. For example, different mapping algorithms (or different variations of a mapping algorithm) may be provided for different types of light source. For example, different mapping algorithms may be provided for light emissive objects and for reflective objects. In some embodiments, pixel data 15 is mapped according to: a default algorithm; an algorithm for emissive objects; or an algorithm for reflective objects depending upon position-specific metadata for the pixel to which the pixel data 15 relates. In such embodiments, block 38 may comprise selection from among a plurality of algorithms.

[0042] In block 39 the location specific mapping is applied to the pixel data to obtain mapped pixel data.

[0043] In block 40 the color coordinates of the pixel data (which may include a tone or luminance value) are compared to a set of zero or more special color regions. The special color regions may consist of single points in color space or regions in the color space. If no special colors have been specified, then block 40 is optional and may be omitted. If the color coordinates of the pixel data correspond to a special region of the color space then in block 42 a color-specific map is selected. In block 43 the color-specific map is applied to map the pixel data to mapped pixel data.

[0044] In block 44, pixel data which does not correspond to a reserved color, a special location, or a special color is mapped according to a general mapping. In block 45, mapped pixel data is output. Block 45 may comprise, for example, placing mapped pixel data onto a serial or parallel data bus, sending mapped pixel data across a wired or wireless communication channel, applying mapped pixel data to further image processing displaying or reproduction hardware, writing mapped pixel data to a memory, or the like.

[0045] In some embodiments, the metadata includes an image histogram or other image statistics. A mapping algorithm may use this information, for example an average luminance of a current image to control tone and / or gamut mapping. In some embodiments the image statistics are available to data processing sequence 25B before pixel data for the current image is mapped. This can permit mapping of pixel data to be based on statistics for an image even where the mapping is commenced before all data for the image is received. In some embodiments mapping is local (i.e. each set of pixel data 15 is mapped without regard to the values of pixel data 15 for other pixels in the image).

[0046] The image histogram that is provided by metadata for a frame of image data may identify, for example, the number of pixels for each color channel having a luminance value within a luminance range associated with each one of the histogram buckets (e.g. 256 24-bit histogram buckets may be provided for each color channel). The image histogram(s) and / or image statistics may be used to analyze the image data. Tone / gamut mapping may be performed based on such image data analysis. For example, in some embodiments, a special tone / gamut mapping algorithm may be performed based on an assessment of brightness or dimness of the overall image, or the significance of bright or dim areas relative to average luminance, or other criteria as extracted from the histogram or image statistics. In some embodiments, tone / gamut mapping may comprise, for example, achieving a balance between bright and dim image areas by: adjusting the tone curve to be applied so that less emphasis is placed on objects in the bright image areas, or adjusting the tone curve to be applied so that more emphasis is placed on objects in dim image areas.

[0047] In some embodiments, general metadata includes information specifying a selected mode of operation for a tone / gamut mapping apparatus. This feature may be exploited, for example, to allow a director to exert control over the tone / gamut mapping algorithm(s) applied at a television or other downstream device, if the implementation of the mapping algorithm(s) are standardized for the upstream and downstream devices. In some embodiments the selected mode of operation is identified by a code indicating generally the nature of the mode. For example, some televisions which incorporate tone / gamut mapping apparatus as described herein may support selection among Avivid@, Acinema@, Astandard@ and Apro@ modes. Different televisions may implement supported modes in their own ways.

[0048] Figure 3 shows apparatus 50 according to another example useful for understanding the invention. Apparatus 50 comprises a metadata extractor 52 that reads metadata 21 from image data 14 or some other source. A control 54 controls a switch 55 in response to the detected metadata. For example, control 54 may set the position of switch 55 in response to a desired mode indicated by metadata 21.

[0049] Switch 55 may comprise a physical switch or a logical switch. For each position of switch 55 a different set 56 of mapping algorithms is made available to mapping logic 57. The illustrated embodiment provides sets 56A, 56B and 56C (collectively sets 56). Each set 56 comprises, for example, a general algorithm 57A, a plurality of algorithms for different specific types of light sources 57B-1, 57B-2, and a plurality of algorithms for specific colors or color regions 57C-1, 57C-2.

[0050] In some embodiments, televisions or other downstream devices may incorporate tone / gamut mapping apparatus configured to receive specification for a mapping algorithm to be applied to video data 14. The specification may be of certain mapping parameters such as look-up table values, parameters which control some aspect of operation of tone and / or gamut mapping algorithm(s) implemented by the apparatus, or specification of one or more special algorithms to be executed by the apparatus in mapping tone and / or gamut of image data 14. The look-up tables can be set upon initialization, and metadata may be used to index through the look-up tables to select the desired tone / gamut mapping. Initialization may be performed with a fixed set of data (which may include data that is not built in at the receiving end but is supplied by the user) or through the transmission of metadata. Table 1 provides an illustration of some examples of position -specific metadata codes that may be used in the context of this invention. The codes shown in Table 1 are examples only.TABLE I - EXAMPLE POSITION-SPECIFIC METADATA CODESCODEMEANINGNOTE0000default mapping0001reflective object0010emissive object1101reflective object of a reserved color1110emissive object of a reserved color Table II provides an illustration of some examples of color -specific metadata codes that may be used in the context of this invention. The codes shown in Table II are examples only. TABLE II - EXAMPLE COLOR-SPECIFIC METADATA CODESCODEMEANINGNOTE0011color graded out of gamutindicates color is intentionally out of gamut of a reference display used in color grading0100reserved color #1the reserved colors may be specified, for example, as 60-bit representations of the CIE XYZ coordinates in a CIE LAB color space (20 bits per coordinate).0101reserved color #20110reserved color #30111reserved color #4 Table III provides an illustration of some examples of general metadata information that may be used in the context of this invention. The information items shown in Table III are examples only. TABLE III - EXAMPLE GENERAL METADATA INFORMATIONCODEMEANINGNOTEImage HistogramHistogram illustrates the number of pixels for each color channel coming within ranges associated with histogram buckets.For example, 256 24-bit histogram buckets may be provided for each color channel.Director Selected ModePermits control of tone / gamut mapping modes of a downstream display device (e.g. selection among Avivid@, Acinema@, Astandard@ and Apro@ modes)Content sourceValues specify, e.g. whether the image data is camera direct or color graded.

[0051] The order of operations in method 25 may be altered. For example, determination of whether a piece of pixel data being processed corresponds to a special location may be performed after determining whether the pixel data corresponds to a special region in color space. In a further example embodiment, shown in Figure 2A, if a pixel data is identified as corresponding to a special location, then a block 40A is performed to determine whether the pixel data also corresponds to a special point or region in color space. If so, then a map that is both location-specific and color-specific is selected in block 42A and this map is then applied to map the color coordinates for the pixel in block 43A.

[0052] In an example application, a video signal has been color graded to be displayed on a "standard color television". However, the video signal will be displayed on a television having a gamut that is broader than that of the standard television and also has a greater dynamic range than the standard television. In this application, it may be desirable to slightly boost the dynamic range of the input video data and then slightly expand the gamut of the input video data in order to take advantage of the capabilities of the display. A default mapping which provides suitable expansion of dynamic range and gamut is provided, for example, in the form of a set of lookup tables. To facilitate this, the image data may be presented or transformed into a color space which includes hue and lightness coordinates.

[0053] During color grading, certain colors, such as, for example, a range of colors that appear in flesh of people depicted in the video, are identified as being reserved colors. The metadata identifies these reserved colors so that the mapping will not alter the flesh tones. Additionally, either in the preparation of the video content or at a subsequent location upstream from a mapping apparatus, the video images are processed to locate light sources within the images. Metadata identifying locations of those light sources (and, in some cases, types of the light sources) is generated.

[0054] Pixel location information is compared to the locations of light sources. If a pixel data being processed corresponds to the location of a light source, then a different mapping may be applied. In some embodiments, the selected mapping for locations corresponding to light sources expands dynamic range more aggressively than mappings applied to areas that are not identified as corresponding to light sources.

[0055] Figure 4 illustrates mapping logic 60 according to an example useful for understanding the invention. Pixel data 15 being mapped is received in a buffer or memory 62. A comparison unit 64 compares color coordinates of pixel data 15 to a set of reserved color coordinates 65 in a buffer or memory 66. A second comparison unit 68 compares location information of the pixel data to a set of special locations 69 in a buffer or memory 70. Algorithm selection logic 72 selects an appropriate mapping algorithm for the pixel data based on outputs of comparison units 64 and 68. The illustrated embodiment has several parallel mapping pipelines 75 each configured to execute a different mapping algorithm. Algorithm selection logic 72 determines which one of pipelines 75 will be used to process the pixel data 15. Mapped pixel data 15= is written to a memory or buffer 77.

[0056] Selection among pipelines 75 may be achieved in any suitable manner. Some examples are: $selectively enabling one of pipelines 75; $applying pixel data 15 only to a selected pipeline 75; $selecting the output of one pipeline 75 from which to write mapped pixel data 15=.

[0057] As a further alternative, algorithm selection unit 72 may be configured to reconfigure a programmable or otherwise configurable logic pipeline on the fly to perform a mapping algorithm corresponding to the outputs of comparison units 64 and 68. Comparison units 64 and 68 may have but are not limited to having two-state outputs. In some embodiments, outputs of one or both comparison units 64 and 68 can have three or more states.

[0058] Some specific ways in which the methods and apparatus according to this invention may be made available include: $image processing chips comprising apparatus according to an embodiment of the invention;$a set-top box, television, computer monitor, projector and / or other display incorporating tone and / or gamut mapping apparatus as described herein;$an image data source such as a DVD player (e.g. Blu-Ray player), video player, camera, mobile phone or the like incorporating tone and / or gamut mapping apparatus as described herein;$physical or non-transitory computer-readable media comprising firmware or other computer-readable instructions which, when executed, configure a programmable processor and / or configurable logic apparatus to perform a method according to the invention;$one or more central processing units (CPUs), one or more microprocessors, one or more field programmable gate arrays (FPGAs), one or more application-specific integrated circuits (ASICs), one or more graphics processing units (GPUs), or any combination thereof, or any other suitable processing unit(s) comprising hardware and / or software capable of performing a method according to the invention; and,$color grading tools incorporating tone and / or gamut mapping apparatus as described herein.

Claims

1. A method implemented in an image processing apparatus located at a display end for mapping tone, gamut or both tone and gamut of pixel data in image data, the method comprising: receiving the pixel data for an image; receiving metadata, the metadata comprising: information specifying a set of one or more special locations, wherein a special location refers to a particular image area in the pixel data that has been identified in a color grading process to be tone, gamut or both tone and gamut mapped according to mapping parameters associated with the particular image area, information specifying a set of one or more special color regions, wherein a special color region refers to a particular color region in the pixel data that has been identified in the color grading process to be tone, gamut or both tone and gamut mapped according to mapping parameters associated with the particular color region, and information specifying a set of one or more reserved colors, wherein a reserved color refers to a particular color in the pixel data that has been identified in the color grading process to be unmapped or tone mapped according to mapping parameters associated with the particular color; mapping the pixel data to mapped pixel data according to an algorithm based at least in part on the mapping parameters determined from the metadata, wherein mapping the pixel data comprises comparing color coordinates for the pixel data to a set of one or more reserved colors, and in response to the color coordinates matching one of the set of reserved colors, inhibiting gamut mapping of the pixel data, wherein mapping the pixel data comprises comparing location information corresponding to the pixel data to a set of the one or more special locations, and in response to the location information matching one of the set of one or more special locations, performing one or both of tone and gamut mapping the pixel data, and wherein mapping the pixel data comprises comparing color coordinates of the pixel data to a set of one or more special color regions, and in response to the color coordinates corresponding to one of the set of special color regions, selecting a color-specific map for tone, gamut or both tone and gamut mapping the pixel data, and mapping the pixel data according to the selected color-specific map.

2. A method according to claim 1, comprising: generating the metadata based at least in part on tone or gamut parameters for the pixel data as adjusted in the color grading process; and encoding the metadata in the image data for transmission to the mapping apparatus.

3. A method according to any one of claims 1 and 2 wherein receiving the metadata comprises extracting the metadata from a signal or file containing the pixel data.

4. A method according to any one of claims 1-3 wherein the set of one or more special locations comprise one or more particular image areas associated with a particular type of image areas, and wherein the corresponding mapping parameters are associated with the particular type.

5. A method according to claim 4 wherein the particular type of the image areas comprises one of: emissive light source and reflective light source.

6. A method according to any one of claims 1-5, the method comprising: receiving statistical information regarding luminance or chrominance values of the pixel data; wherein mapping the pixel data is based in part on the statistical information, and wherein the statistical information comprises a histogram of the number of pixels for each color channel having a luminance value within a luminance range associated with one of a plurality of predefined histogram buckets.

7. Image processing apparatus for mapping tone, gamut or both tone and gamut of received pixel data in image data at a display end, the apparatus comprising: a plurality of parallel pipelines each configured to map the received pixel data to mapped pixel data according to a corresponding mapping algorithm; and algorithm selection logic configured to select one of the plurality of pipelines for mapping specific pixel data based at least in part on location information corresponding to the specific pixel data; wherein the apparatus is configured to perform the method of any one of claims 1-6.

8. Image processing apparatus according to claim 7, further comprising a buffer or memory containing a set of one or more special locations and a first comparison unit configured to compare the location information of the pixel data to locations in the set of one or more special locations, wherein a special location refers to a particular image area in the pixel data that has been identified in a color grading process to be tone, gamut or both tone and gamut mapped according to mapping parameters associated with the particular image area, and wherein the algorithm selection logic is configured to select the one of the plurality of pipelines based at least in part on an output of the first comparison unit.

9. Image processing apparatus according to claim 8, further comprising a buffer or memory containing information specifying one or more special color regions and a second comparison unit configured to compare color coordinates of the pixel data to the set of one or more special color regions, wherein a special color region refers to a particular color region in the pixel data that has been identified in the color grading process to be tone, gamut or both tone and gamut mapped according to mapping parameters associated with the particular color region, and wherein the algorithm selection logic is configured to select the one of the plurality of pipelines based at least in part on an output of the second comparison unit.