Multi-Intent Composite Image Encoding and Rendering

Abstract

Techniques for multi-intent composite image encoding and rendering are disclosed. The techniques can include obtaining a set of component images for a composite image, determining a common rendering intent to be applied to the set of component images, adjusting one or more of the component images in the set of component images according to the common rendering intent to obtain an adjusted set of component images, generating a composite image based on the adjusted set of component images, generating metadata characterizing the common rendering intent, and encoding the composite image and the metadata to generate an encoded multi-intent composite image.

Description

Technical Field

[0001] [Related Applications] This application claims priority to U.S. Provisional Application No. 63 / 368,766, filed Jul. 18, 2022, and European Application No. 22213582.4, filed Dec. 14, 2022, the entire disclosures of which are incorporated herein by reference.

Background Art

[0002] This application generally relates to systems and methods for image encoding and decoding.

Summary of the Invention

[0003] At least some aspects of the present disclosure may be implemented via a method. Some methods can include obtaining a set of constituent images for a composite image, determining a common rendering intent to be applied to the set of constituent images, adjusting one or more of the constituent images of the set of constituent images according to the common rendering intent to obtain an adjusted set of constituent images, generating a composite image based on the adjusted set of constituent images, generating metadata characterizing the common rendering intent, and encoding the composite image and the metadata to generate an encoded multi-intent composite image.

[0004] In some examples, the step of adjusting one or more of the constituent images of the set of constituent images according to the common rendering intent can include converting the constituent images in the set of constituent images from a current rendering intent to the common rendering intent.

[0005] In some examples, the step of converting the constituent images in the set of constituent images from a current rendering intent to the common rendering intent includes reversing one or more source adjustments of the constituent image, and applying one or more common spatial adjustments to the constituent image. and can include.

[0006] In some examples, the step of applying one or more common spatial adjustments to the compositional image can include the step of converting sensor values to color values.

[0007] In some examples, the step of applying one or more common spatial adjustments to the compositional image can include the step of estimating the ambient luminance and white point of the capture environment and applying white point correction based on the estimated ambient luminance and white point of the capture environment.

[0008] In some examples, the step of applying one or more common spatial adjustments to the compositional image can include the step of estimating the ambient luminance of the capture environment and applying an optical - optical transfer function (OOTF) based at least in part on the estimated ambient luminance of the capture environment to prepare an image for rendering on a reference display device.

[0009] In some examples, the step of applying one or more common spatial adjustments to the compositional image is as follows: Saturation enhancement, Contrast enhancement, Individual color saturation adjustments, Slope offset power Tmid enhancement, Tone curve trimming, and can include the step of applying one or more of the above.

[0010] In some examples, the step of adjusting one or more compositional images of a set of compositional images according to a common rendering intent can include the step of converting the compositional images in the set of compositional images from a pre - processing state to the common rendering intent.

[0011] In some examples, the step of adjusting one or more compositional images of a set of compositional images according to a pre - common rendering intent is the step of converting a first compositional image of the set of compositional images from a first current rendering intent to the common rendering intent, and the step of converting a second compositional image of the set of compositional images from a second current rendering intent to the common rendering intent, and can include.

[0012] In some examples, the step of determining a common rendering intention as the common rendering intention can include the step of selecting the current rendering intention of a constituent image among the set of constituent images.

[0013] In some examples, the current rendering intention of a constituent image is selected as the common rendering intention based on the importance of the constituent image relative to other constituent images in the set of constituent images, and the importance of the constituent image is as follows: the relative size of the constituent image in the rendering space of the composite image, the resolution of the constituent image, the centrality of the position of the constituent image in the rendering space of the composite image, the viewer's focus in the rendering space of the composite image, and is determined based on one or more of the above.

[0014] In some examples, the step of determining a common rendering intention can include the step of identifying a preferred rendering intention as the common rendering intention.

[0015] In some examples, the preferred rendering intention can be identified based on one or both of the input received via the user interface and the information in the configuration file.

[0016] Some methods include receiving an encoded multi-intention composite image, decoding the encoded multi-intention composite image to obtain the composite image and metadata describing one or more common spatial adjustments applied when generating the composite image, identifying one or more common spatial adjustments based on the metadata, and adjusting the composite image to reverse the one or more common spatial adjustments to generate a realistic rendered composite image. Adjusting the real-render composite image according to the target rendering intention to generate a target-adjusted composite image; Displaying the target-adjusted composite image; It can include.

[0017] In some examples, the step of adjusting the real-render composite image according to the target rendering intention can include the step of applying one or more target adjustments to the real-render composite image.

[0018] In some examples, the step of applying one or more target adjustments to the real-render composite image can include the step of converting sensor values to color values.

[0019] In some examples, the step of applying one or more target adjustments to the real-render composite image can include the step of estimating the ambient luminance and white point of the capture environment and applying white point correction based on the estimated ambient luminance and white point of the capture environment.

[0020] In some examples, the step of applying one or more target adjustments to the real-render composite image can include the step of estimating the ambient luminance of the capture environment and applying an optical-optical transfer function (OOTF) based at least in part on the estimated ambient luminance of the capture environment to prepare an image for rendering on a reference display device.

[0021] In some examples, the step of applying one or more target adjustments to the real-render composite image is as follows: Saturation enhancement, Contrast enhancement, Individual color saturation adjustment, Slope offset power Tmid enhancement, Tone curve trimming, It can include the step of applying one or more of the above.

[0022] In some examples, a target rendering intent can be identified based on one or both of input received via a user interface and information within a configuration file.

[0023] Some or all of the operations, functions, and / or methods described herein may be performed by one or more devices according to instructions (e.g., software) stored on one or more non-transitory media. Such non-transitory media may include memory devices as described herein, including but not limited to random access memory (RAM), read-only memory (ROM), etc. Accordingly, various novel aspects of the subject matter described in this disclosure may be implemented via one or more non-transitory media storing software.

[0024] At least some aspects of the present disclosure may be implemented via an apparatus. For example, one or more devices may be capable of at least partially performing the methods disclosed herein. In some implementations, the apparatus is or includes an audio processing system having an interface system and a control system. The control system may include at least one of a general-purpose single or multi-chip processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic element, discrete gate or transistor logic, discrete hardware component, or combinations thereof.

[0025] One or more details of implementations of the subject matter described herein are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages will be apparent from the description, the drawings, and the claims. The relative dimensions in the following drawings may not be drawn to scale. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] These and other more detailed and specific features of the various embodiments are more fully disclosed in the following description and with reference to the accompanying drawings.

[0027]

Figure 1

[0028]

Figure 2

[0029]

Figure 3

[0030]

Figure 4

[0031]

Figure 5

[0032]

Figure 6

[0033]

Figure 7

[0034]

Figure 8

DETAILED DESCRIPTION OF THE INVENTION

[0035] The present disclosure and its aspects can be embodied in various forms, including hardware, apparatus, or circuitry controlled by a computer-implemented method, a computer program product, a computer system and network, a user interface, and an application programming interface, as well as a method implemented by hardware, a signal processing circuit, a memory array, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), etc. The above is merely intended to give an overall idea of the various aspects of the present disclosure and does not limit the scope of the present disclosure in any way.

[0036] In the following description, numerous details such as optical device configurations, timings, operations, etc. are described to provide an understanding of one or more aspects of the present disclosure. It will be immediately understood by those skilled in the art that these specific details are merely examples and are not intended to limit the scope of the present application.

[0037] FIG. 1 shows an exemplary image delivery pipeline 100 that illustrates various stages from video capture to image content display. Image 102 may, according to some implementations, be one video frame of a video frame sequence and is captured or generated using image generation block 105. Image 102 may be captured digitally (e.g., by a digital camera) or generated by a computer (e.g., using computer animation) to provide image data 107. Alternatively, image 102 may be captured on film by a film camera. The film is converted to a digital format to provide image data 107. In production stage 110, image data 107 is edited to provide image production stream 112.

[0038] The image data of production stream 112 is then provided to a processor (or one or more processors such as a central processing unit (CPU)) at block 115 for post-production editing. The post-production editing at block 115 may include adjusting or changing the color or brightness of specific regions of the image to improve the image quality or achieve a particular appearance according to the creative intent of the image maker. This is sometimes referred to as "color timing" or "color grading". The methods described herein may be executed by a processor at block 115. Other edits (such as scene selection and ordering, image cropping, addition of computer-generated visual effects, etc.) may be executed at block 115 to produce a final version 117 of the production for distribution. During post-production editing 115, the image (in some implementations, as part of a sequence of video images) is displayed on a reference display 125. The reference display 125 may be a consumer-level display or projector, as needed.

[0039] Following post-production 115, the image data 117 of the final production may be delivered to an encoding block 120 for downstream delivery to a decoding and playback device such as a computer monitor, television set, set-top box, movie theater, etc. In some embodiments, the encoding block 120 may include audio and video encoders as defined by ATSC, DVB, DVD, Blu-Ray®, and other delivery formats to generate a coded bitstream 122. At the receiver, the coded bitstream 122 is decoded by a decoding unit 130 to generate a decoded signal 132 that represents the same as or a very close approximation of the signal 117. The receiver may be attached to a target display 140 that may have characteristics quite different from the reference display 125. In that case, a display management block 135 may be used to map the dynamic range of the decoded signal 132 to the characteristics of the target display 140 by generating a display-mapped signal 137. Additional methods described herein may be performed by the decoding unit 130 or the display management block 135. Both the decoding unit 130 and the display management block 135 may include their own processors or may be integrated into a single processing unit. Although the present disclosure refers to the target display 140, it will be understood that this is merely an example. It will be further understood that the target display 140 may include any device configured to display or project light. For example, computer displays, televisions, OLED displays, LCD displays, quantum dot displays, movie theaters, consumer, and other commercial projection systems, head-up displays, virtual reality displays, etc.

[0040] When using a digital device to capture a scene, realistic scene reference radiometric measurements are rarely transferred directly to generate an image. Instead, the device's partner brand manufacturer (original electronic manufacturer (OEM)) or software application designer typically adjusts the image by adapting it for viewing in a reference viewing environment, such as a dim surround and D65 illumination, and applying artistic adjustments such as enhanced contrast and saturation. These and other adjustments produce a preferred rendering of reality that is considered favorable to the consumer.

[0041] Currently, these operations are irreversible in two ways. First, the parameters used to apply the operations are not transmitted, and second, pixel operations may be irreversible due to non-linear clipping and quantization, non-invertible operations, unknown algorithms, or an unknown order of operations.

[0042] Instead, it may be desirable to send the original captured / pre-processed image representing the "reality" captured by the imaging sensor and be able to apply these operations during playback. This can enable multiple rendering intents as follows. During playback, the device can present the original captured "reality" image, or alternatively, the device can generate a "comfortable" image modified from the original captured "reality" image.

[0043] It may also be desirable to be able to send such content in a backward-compatible manner. In this approach, corrections for generating a "comfortable" image can be applied during capture, and appropriate parameters can be sent to the playback device to enable the playback device to reverse the corrections and restore the original captured "reality" image.

[0044] Figure 2 shows an exemplary process 200. Process 200 enables encoding and decoding of images with multiple intents using metadata. Process 200 may be performed, for example, by a processor as part of block 115 and / or part of block 120 for encoding, and as part of block 130 and / or part of block 135 for decoding.

[0045] In step 202, an image is captured. In a digital capture device, the exposed scene is transferred to sensor values in a single-channel representation. Through a process known as demosaicing, the single-channel image representation is extended to a three-channel (RGB, etc.) image representation of red, green, and blue. There are numerous approaches to demosaicing, and any of them is sufficient in the embodiments disclosed herein.

[0046] Step 202 can include, as a non-limiting example, reading single-channel values from a sensor, applying a demosaicing processing scheme to generate a three-color channel (such as RGB) image, and optionally applying a 3×3 transform to adapt the image sensitivity to the sensitivity of the desired primary colors (such as RGB). Step 202 may also include measuring the capture ambient luminance (e.g., the level of ambient light in the capture environment).

[0047] To fully capture the colorimetry of a scene, the spectral sensitivity of the capture device needs to match that of the viewer. In practice, these often do not match exactly, so instead a 3x3 matrix transform is used to convert the sensor's sensitivity to some desired set of RGB primary colors. The approximation of the sensor sensitivity causes loss. To avoid contributing to this loss, transmit the camera's spectral sensitivity along with the content, similar to the applied 3x3 matrix transform, so that the playback device can apply or invert the conversion from the sensor output to the specified RGB primary colors.

[0048] Once the desired RGB makeup of the image is determined, the values can be adapted to the specified reference white point. The image can be adapted to any of the standardized white points (such as D50, D65, etc.) through, for example, a Von Kries adaptation transformation. This process includes (a) estimating the ambient luminance and white point of the capture environment, and (b) applying corrections to the image to achieve color matching for the observer in the specified reference viewing environment (e.g., an environment with a known white point and ambient luminance). Methods available for adjusting the image to match the adaptation state of the observer in a colored surrounding environment are outlined in PCT Application No. PCT / US2021 / 027826, filed on April 16, 2021, and PCT Application No. PCT / US2021 / 029476, filed on April 27, 2021, each of which is hereby incorporated by reference in its entirety and for all purposes.

[0049] In step 204, one or more source appearance adjustments, including but not limited to white balance adjustment, color correction adjustment, and optical-optical transfer function (OOTF) adjustment, can be applied to the captured image. Step 204 can include calculating a non-linear optical-optical transfer function (OOTF) for mapping from the measured capture ambient luminance to the reference review environment. The order of the white point adjustment and the 3x3 matrix may be different. The calculation and application of the optical-optical transfer function (OOTF) may establish the rendering intent of the image on a standard display device. In practice, the OOTF is applied to map the image from the display environment at capture to the display in the reference display environment. The application of the OOTF can be a non-reversible operation and may make it difficult to invert the OOTF during playback. Similar to the white point adjustment, in the first step (a), the ambient luminance of the capture environment can be estimated, and in the second step (b), the image can be corrected to match the observer within the reference environment.

[0050] In step 206, one or more source preference adjustments can be applied to the captured image, including contrast adjustment, saturation adjustment of the overall color and / or saturation adjustment of individual colors, tone curve slope offset power Tmid adjustment, and trimming and adjustment of other tone curves, but not limited to these. As used herein, "mid" refers to the average of the maxRGB values of the image in a perceptually quantized (PQ) encoded image, and each pixel has its own maxRGB value equal to the maximum color component value (R, G, or B) of that pixel. In other words, which color component of the pixel has the maximum value is the maxRGB value of that pixel, and the average of the maxRGB values across the entire PQ encoded image is the "mid" of the image. "T-mid" can refer to the "target mid", which can be the "mid" value desired by the user or content creator in the final image. In some embodiments, the individual color saturation adjustment may include saturation adjustments for six different colors, which may be referred to as "6-vector adjustment".

[0051] The correction of the captured image in steps 204 and 206 results in a source-adjusted image 207. The source rendering intent 203 can specify the source appearance adjustments applied in 204 and / or the source preference adjustments applied in 206 to generate the source-adjusted image 207. The source rendering intent 203 can be indicated by a selection received from the user, and the selection specifies which source appearance adjustments and source preference adjustments are to be made, the coefficients of such adjustments, which parts of the image the adjustments are to be applied to, and so on. Alternatively, the source rendering intent 203 can constitute a default rendering intent, or represent a combination of both to reflect both user-specified adjustments and default adjustments.

[0052] In an OEM or software application, it is common to apply source preference adjustments to the captured image. These changes are purely aesthetic and are usually arranged to render an image with a higher level of contrast and color saturation. In various embodiments of the present disclosure, these preference changes determined by the OEM are transmitted as metadata along with the content and applied at playback in the same manner as the source appearance metadata. In each case, there is a first step (a) of calculating or specifying the desired amount of correction to apply, and a step (b) of applying the correction using a parameterized function. Both (a) and (b) are transmitted as metadata, enabling the playback device to have complete flexibility in rendering either a "comfortable" image or a "realistic" image, and enabling the capture device to have complete flexibility in transmitting either a "comfortable" image or a "realistic" image.

[0053] As described herein, one advantage of the various embodiments disclosed herein is that all adjustments to the three-channel image can be encoded as metadata and transmitted to the playback device along with the content for application. In one embodiment, the OEM or encoding device can determine not to apply adjustments to both appearance and preference in order to generate a "realistic" image.

[0054] In step 208, the source-adjusted image may be encoded. Step 208 may include encoding the source-adjusted image 207 for downstream distribution to a decoding and playback device such as a computer monitor, television set, set-top box, movie theater, etc. In some embodiments, the encoding step 208 may include audio and video encoders as defined by ATSC, DVB, DVD, Blu-Ray®, and other distribution formats to generate a coded bitstream.

[0055] In addition to encoding the source-adjusted image 207, step 208 can include the step of generating and / or encoding metadata that characterizes the source appearance adjustment applied in step 204 and (if present) the source preference adjustment applied in step 206. The metadata can include metadata associated with the source appearance adjustment such as the white point of the scene specified in x, y coordinates (or other system), the ambient luminance of the scene specified in lux (or other system) (e.g., information regarding the estimated capture environment), the coefficients of the applied white point adjustment matrix, the coefficients of the applied 3x3 color matrix, the coefficients of the applied parameterized OOTF, the spectral sensitivity of the sensor used in the calculation of the 3x3 matrix, and coefficients or other information for other enhancements applied in step 204. Further, the metadata can include metadata associated with the source preference adjustment such as coefficients for contrast enhancement such as slope-offset-power-Tmid contrast adjustment, coefficients for saturation enhancement, coefficients for individual color saturation adjustments, coefficients for tone curve trimming, and coefficients for other enhancements applied in step 206.

[0056] When the metadata indicates the source appearance and / or preference adjustment applied in step 204 and / or 206, the metadata can include information that facilitates the inversion (or approximate inversion) of the functions applied in relation to such adjustments. The metadata can include information indicating the order in which the source appearance and / or preference adjustments were applied in step 204 and / or 206. If no adjustment is made in step 204 and / or 206, the metadata can indicate that no source appearance adjustment and / or preference adjustment has been made.

[0057] In step 210, the encoded source-adjusted image 207 and the metadata can be decoded. The source rendering intention 203 associated with the adjustments applied in steps 204 and 206 can be compared with the target rendering intention 216. The target rendering intention 216 can specify target appearance adjustments and / or preference adjustments. The target rendering intention 216 can be indicated by a selection received from the user, and the selection specifies which target appearance adjustments and target preference adjustments are to be made, the coefficients of such adjustments, which parts of the image the adjustments are to be applied to, etc. Alternatively, the target rendering intention 216 can constitute a default rendering intention or a desired rendering intention indicated by the metadata. In some implementations, the target rendering intention 216 can represent a combination of both the adjustments specified by the user and the adjustments associated with the default rendering intention and / or the desired rendering intention indicated by the metadata. In one example, the target rendering intention 216 can represent a combination of the appearance adjustments specified by the user and the default preference adjustments.

[0058] If the target rendering intention 216 is different from the source rendering intention 203, that is, if the target appearance adjustments and / or preference adjustments are different from the source appearance adjustments and / or preference adjustments, a series of steps can be executed to convert the source-adjusted image 207 from the source rendering intention 203 to the target rendering intention 216. These steps can generally include converting the source-adjusted image 207 to a realistic render image 215 and then converting the realistic render image 215 to a target-adjusted image 219.

[0059] The source-adjusted image 207 can be converted to the actual rendered image 215 via the operations performed in steps 212 and / or 214 to undo the adjustments performed in steps 206 and / or 204 (in relation to the application of the source rendering intent 203). The actual rendered image 215 can correspond to the image captured in step 202 and can represent the "reality" captured by the imaging sensor. In step 212, using the metadata obtained in step 210, an inverted source preference adjustment can be calculated. When applied to the source-adjusted image 207, the inverted source preference adjustment of step 212 can undo some or all of the source preference adjustment of step 206. In step 214, using the metadata obtained in step 210, an inverted source appearance adjustment can be calculated. When applied to the source-adjusted image 207, the inverted source appearance adjustment of step 214 can undo some or all of the source appearance adjustment of step 204. Based on the metadata obtained in step 210, the order in which the source appearance and / or preference adjustments were applied in steps 204 and / or 206 can be determined. This order can be reversed in relation to the application of the inverted source preference and / or appearance adjustments in steps 212 and / or 214.

[0060] The real-rendered image 215 can be converted into the target-adjusted image 219 via operations performed in step 217 and / or 218. In step 217, a target appearance adjustment can be calculated and applied. The target appearance adjustment can include, as non-limiting examples, measuring the display ambient luminance (e.g., the level of ambient light in the display environment), and then calculating and applying a non-linear light-optical transfer function (OOTF) to map to the display ambient luminance measured from a reference viewing environment (e.g., the actual viewing environment). In step 218, a target preference adjustment can be calculated and applied. The target preference adjustment can include, as non-limiting examples, contrast adjustment, chroma adjustment, slope offset power Tmid adjustment, individual chroma adjustment, and tone curve trimming. In some implementations, the inversion of the source adjustment and the application of the target adjustment can be combined into a single processing step, and the adjustment can be calculated accordingly. In other words, some or all of steps 212, 214, 217, and 218 can be combined.

[0061] In some implementations, the source rendering intent 203 can correspond to real rendering and can essentially bypass steps 204 and 206. In such implementations, the real-rendered image (the image captured in step 202) can function as the source-adjusted image 207 encoded in step 208, and the metadata encoded in step 208 can indicate that no source appearance adjustment and source preference adjustment were performed. Based on the metadata, the decoding entity can recognize that the source-adjusted image 207 is a real-rendered image and can essentially bypass steps 212 and 214. Similarly, in some implementations, the target rendering intent 216 can correspond to real rendering and can essentially bypass steps 217 and 218 to enable the real-rendered image to function as the target-adjusted image 219.

[0062] The target-adjusted image 219 can be rendered in step 220. As an example, the target-adjusted image 219 can be utilized by projection, display, storage in a storage device, transmission to another device, or other means.

[0063] Figure 3 shows an exemplary process 300. According to process 300, a composite image can be generated based on a set of configured images adjusted to apply different respective rendering intents. In the example shown in Figure 3, the set of configured images includes adjusted images 301-1, 301-2, 301-3, and 301-4. It should be understood that process 300 may include generating a composite image based on a smaller number or a larger number of configured images in various implementations.

[0064] The adjusted images 301-1, 301-2, 301-3, and 301-4 each represent captured images A, B, C, and D adjusted to implement rendering intents 1, 2, 3, and 4 (e.g., via appearance adjustment and / or preference adjustment as in steps 204 and 206 of Figure 2). In steps 302-1, 302-2, 302-3, and 302-4, an inversion adjustment can be applied to the adjusted images 301-1, 301-2, 301-3, and 301-4 respectively to cancel the previously applied adjustments associated with their current rendering intents (rendering intents 1, 2, 3, 4). These inversion adjustments can include an inversion preference adjustment to cancel the previously applied preference adjustment, an inversion appearance adjustment to cancel the previously applied appearance adjustment, or both. If both types of inversion adjustments are performed, the inversion preference adjustment can be performed first, and then the inversion source adjustment can be performed.

[0065] The inversion adjustments performed on adjusted images 301-1, 301-2, 301-3, and 301-4 with 302-1, 302-2, 302-3, and 302-4 can generate respective real render images 303-1, 303-2, 303-3, and 303-4. The real render images 303-1, 303-2, 303-3, and 303-4 can be converted to a common rendering intention 304. In some implementations, the applied common rendering intention 304 can be specified by the user or can be a default rendering intention. In some implementations, the applied common rendering intention 304 can be selected from among a plurality of candidate rendering intentions. In some implementations, such candidate rendering intentions can be statically pre-specified. In other embodiments, such candidate rendering intentions can be dynamically identified during the execution of process 300.

[0066] In some embodiments, the common rendering intention 304 can be made to match the rendering intention of one or more of the constituent images of a set of constituent images (adjusted images 301-1, 301-2, 301-3, 301-4). In some embodiments, for example, the common rendering intention 304 can be selected from among the rendering intentions of the set of constituent images based on the relative importance of each constituent image. In one example, adjusted image 301-1 can be identified as the most important in the set of constituent images, and thus, rendering intention 1 can be selected as the common rendering intention 304.

[0067] The relative importance of the various constituent images within the set can be determined based on one or more factors. In some implementations, the factors can include the relative size of the constituent images in the rendered space of the generated composite image. For example, the largest constituent image can be identified as the most important among the constituent images. In some embodiments, the factors can include the resolution of each of the constituent images. For example, the highest resolution constituent image can be identified as the most important among the constituent images.

[0068] In some implementations, the factors can include characteristics related to relatively prominent portions of the compositional images in the rendered space. For example, the central compositional image can be identified as the most important among the compositional images. In another example, eye gaze detection can be used to identify the viewer's focus within the rendered space, and the compositional image where that point exists can be identified as the most important compositional image among the compositional images. In some implementations, any of the foregoing factors can be considered in relation to determining the relative importance of the compositional images. In some implementations, in relation to determining the relative importance of the compositional images, any or all of the foregoing factors can be considered in addition to, or instead of, other factors.

[0069] In some implementations, the rendering intent indicated by a plurality of compositional images can be selected as the common rendering intent 304. For example, if a plurality, majority, or all of the compositional images are of the same rendering intent, that rendering intent can be selected as the common rendering intent 304.

[0070] Through the common space adjustment performed in step 305 and / or 306, the common rendering intent 304 can be applied to the actual render images 303-1, 303-2, 303-3, and 303-4 to obtain respective adjusted images 307-1, 307-2, 307-3, and 307-4. In step 305, a common space appearance adjustment can be calculated and applied. The common space appearance adjustment can include, as non-limiting examples, applying a non-linear optical-optical transfer function (OOTF) to map from one reference display environment to another reference display environment or to an actual display environment. In step 306, a common space preference adjustment can be calculated and applied. The common space preference adjustment can include, as non-limiting examples, contrast adjustment, chroma adjustment, slope offset power Tmid adjustment, individual chroma adjustment, and tone curve trimming.

[0071] In step 308, a composite image 309 can be generated based on the adjusted images 307-1, 307-2, 307-3, and 307-4 using a composite procedure. In step 310, metadata characterizing the composite image 309 and the common rendering intent 304 can be encoded to generate an encoded multi-intent composite image 312.

[0072] FIG. 4 shows an exemplary process 400. According to process 400, an encoded multi-intent composite image 401 (which can be generated, for example, via process 300 of FIG. 3) can be decoded in step 402 to obtain a composite image 403 and metadata indicating the common rendering intent of the image set, where the composite image 403 is generated based on the image set. The metadata can describe one or more common spatial adjustments applied to one or more of those images. The common rendering intent can be compared to a target rendering intent 408 that can specify target appearance adjustments and / or preference adjustments. The target rendering intent 408 can be indicated by a selection received from the user, and the selection can specify which target appearance adjustments and target preference adjustments are to be made, the coefficients of such adjustments, which parts of the image the adjustments are to be applied to, and the like. Alternatively, the target rendering intent 408 can constitute a default rendering intent or a desired rendering intent indicated by the metadata. In some implementations, the target rendering intent 408 can represent a combination of both the adjustments specified by the user and the adjustments associated with the default rendering intent and / or the desired rendering intent indicated by the metadata. In one example, the target rendering intent 408 can represent a combination of the appearance adjustments specified by the user and the default preference adjustments.

[0073] If the target rendering intent 408 is different from the common rendering intent, a series of steps can be executed to convert the composite image 403 from the common rendering intent to the target rendering intent 408. These steps can generally include converting the composite image 403 to a realistic render composite image 407, and then converting the realistic render image 407 to a target adjusted composite image 411.

[0074] The composite image 403 can be converted to a realistic render composite image 417 through the operations executed in steps 404 and / or 406. In step 404, using the metadata obtained in step 402, an inverted common space preference adjustment can be calculated. The inverted common space preference adjustment in step 404 can cancel one or more common space preference adjustments applied in relation to the generation of the composite image 403 (e.g., in step 305 of FIG. 3). In step 406, using the metadata obtained in step 402, an inverted common space appearance adjustment can be calculated. The inverted common space appearance adjustment in step 406 can cancel one or more common space appearance adjustments applied in relation to the generation of the composite image 403 (e.g., in step 306 of FIG. 3).

[0075] The real-rendered composite image 407 can be converted into a target-adjusted composite image 411 via operations performed in step 409 and / or 410. In step 409, a target appearance adjustment can be calculated and applied. The target appearance adjustment can include, as non-limiting examples, measuring the display ambient luminance (e.g., the level of ambient light in the display environment), and then calculating and applying a non-linear optical-optical transfer function (OOTF) to map to the display ambient luminance measured from a reference viewing environment (e.g., the actual viewing environment). In step 410, a target preference adjustment can be calculated and applied. The target preference adjustment can include, as non-limiting examples, contrast adjustment, chroma adjustment, slope offset power Tmid adjustment, individual chroma adjustment, and tone curve trimming. In some implementations, the inversion of the common space adjustment and the application of the target adjustment can be combined into a single processing step, and the adjustment can be calculated accordingly. In other words, some or all of steps 404, 406, 409, and 410 can be combined.

[0076] The target-adjusted composite image 411 can be rendered in step 412. As an example, the target-adjusted composite image 411 can be used by projection, display, storage in a storage device, transmission to another device, or other means.

[0077] FIG. 5 shows an exemplary operating environment 500 in which the encoding and rendering of a plurality of intent composite images can be implemented according to the disclosed aspects. In the operating environment 500, device 501A can generate an encoded plurality of intent composite images 510 (e.g., using a process such as process 300 of FIG. 3) and distribute it to device 501B, and device 501B can process the encoded plurality of intent composite images 510 (e.g., using a process such as process 400 of FIG. 4) to generate a target-adjusted composite image 514.

[0078] Device 501A can obtain a set of configuration images 502 (e.g., the adjusted images 301-1, 301-2, 301-3, and 301-4 in FIG. 3), and can determine a common rendering intention applied to the set of configuration images 502. In some implementations, the step of determining the common rendering intention can include the step of identifying a preferred rendering intention as the common rendering intention. In some implementations, the preferred rendering intention can be identified based on the input received via the user interface or the information in the configuration file. In some implementations, the preferred rendering intention can be identified based on a combination of both. For example, an input for identifying a configuration image can be received, the rendering intention of the configuration image constitutes a preferred rendering intention, and the configuration file can be accessed to identify the rendering intention of the identified configuration image. In some implementations, device 501A can select a common rendering intention from among a plurality of candidate rendering intentions. In some implementations, such candidate rendering intentions can be statically pre-specified. In other implementations, such candidate rendering intentions can be dynamically identified.

[0079] In some implementations, device 501A can select a common rendering intention that matches one or more rendering intentions of the set of configuration images 502. In some implementations, for example, device 501A can select the current rendering intention of the configuration image 502 (e.g., based on the relative importance of the configuration image 502) as the common rendering intention. In some implementations, device 501A can select the current rendering intention shared by a plurality, a majority, or all of the configuration images in the set of configuration images 502.

[0080] Device 501A can adjust one or more of the component images in the component image set 502 according to the common rendering intention to obtain the adjusted component image set 504. This can include converting one or more of the component images in the set of component images 502 from the current rendering intention to the common rendering intention. The conversion from the current rendering intention to the common rendering intention includes reversing one or more source adjustments to obtain the real render images (e.g., the real render images 303-1, 303-2, 303-3, and 303-4 in FIG. 3), and applying one or more common space adjustments to the real render images to obtain the adjusted component image set 504 (e.g., the adjusted images 307-1, 307-2, 307-3, and 307-4 in FIG. 3).

[0081] In some implementations, applying one or more common space adjustments can include one or more of: converting sensor values to color values, estimating the ambient luminance and white point of the capture environment, and applying white point correction based on the estimated ambient luminance and white point of the capture environment, estimating the ambient luminance of the capture environment, and applying an optical - optical transfer function (OOTF) to prepare the image for rendering on the reference display device based at least in part on the estimated ambient luminance of the capture environment. In some implementations, the step of applying one or more common space adjustments can include the step of applying one or more of the following: saturation enhancement, contrast enhancement, individual chroma adjustment, slope offset power Tmid enhancement, tone curve trimming. In some implementations, applying one or more common space adjustments can include converting the component image 502 from the pre - processed state to the common rendering intention.

[0082] Device 501A can generate a composite image 506 (e.g., composite image 309 in FIG. 3) based on the adjusted configuration image set 504 using a composite procedure. Next, device 501A can encode the composite image 506 using metadata 508 to generate an encoded multi-intent composite image 510. The metadata 508 can indicate a common rendering intent applied to the configuration image set 502 in relation to the generation of the composite image 506.

[0083] Device 501B can receive the encoded multi-intent composite image 510 from device 501A, decode the encoded multi-intent composite image 510 to obtain the composite image 506 and the metadata 508. These metadata can describe one or more common spatial adjustments applied during the generation of the composite image 506. Device 501B can adjust the composite image 506 to reverse one or more common spatial adjustments to obtain a real-rendered composite image 512 (e.g., real-rendered composite image 407 in FIG. 4). Next, device 501B can adjust the real-rendered composite image 512 according to the target rendering intent to obtain a target-adjusted composite image 514 (e.g., target-adjusted composite image 411 in FIG. 4), and can display, project, (store in a storage device), (send to another device), or otherwise utilize the target-adjusted composite image 514. In some implementations, the target rendering intent can be identified based on one or both of the input received via the user interface and the information in the configuration file.

[0084] In some implementations, the step of adjusting the real - rendered composite image 512 according to the target rendering intent can include the step of applying one or more target adjustments to the real - rendered composite image 512. In some implementations, the step of applying one or more target adjustments can include the step of estimating the ambient luminance and white point of the capture environment and applying white - point correction based on the estimated ambient luminance and white point of the capture environment. In some implementations, the step of applying one or more target adjustments can include the step of estimating the ambient luminance of the capture environment and applying an optical - optical transfer function (OOTF) based at least in part on the estimated ambient luminance of the capture environment to prepare an image for rendering on a reference display device. In some implementations, the step of applying one or more target adjustments can include the step of applying one or more of the following: saturation enhancement, contrast enhancement, individual chroma adjustment, slope offset power Tmid enhancement, tone - curve trimming.

[0085] Figure 6 shows an exemplary method 600 for encoding a composite image according to an aspect of the present disclosure. Method 600 can represent operations that can be performed by device 501A in the operating environment 500 of FIG. 5 and / or by the exemplary device 800 described below, according to some implementations.

[0086] According to method 600, at 602, a set of constituent images can be obtained for use in generating a composite image. For example, in the operating environment 500 of FIG. 5, device 501A can obtain constituent image 502. At 604, a common rendering intent to be applied to the set of constituent images can be determined. For example, in the operating environment 500 of FIG. 5, device 501A can determine the common rendering intent to be applied to constituent image 502. At 606, one or more of the constituent images of the set of constituent images are adjusted according to the common rendering intent, resulting in an adjusted set of constituent images. For example, in the operating environment 500 of FIG. 5, device 501A can adjust one or more of the constituent images of constituent image 502 to obtain adjusted constituent image 504.

[0087] At 608, the composite image can be generated based on the adjusted configuration image set. For example, in the operating environment 500 of FIG. 5, the device 501A can generate a composite image 506 based on the adjusted configuration image 504. At 610, metadata characterizing a common rendering intention can be generated. For example, in the operating environment 500 of FIG. 5, the device 501A can generate metadata 508 characterizing the common rendering intention applied to the configuration image 502. At 612, the composite image and the metadata can be encoded to generate an encoded multiple-intention composite image. For example, in the operating environment 500 of FIG. 5, the device 501A can encode the composite image 506 and the metadata 508 to generate the encoded multiple-intention composite image 510.

[0088] FIG. 7 shows an exemplary method 700 for rendering a composite image according to an aspect of the present disclosure. The method 700 can represent operations that can be performed by the device 501B in the operating environment 500 of FIG. 5 and / or by the exemplary device 800 described below, according to some implementations.

[0089] According to the method 700, at 702, an encoded multiple-intention composite image can be received. For example, in the operating environment 500 of FIG. 5, the device 501B can receive the encoded multiple-intention composite image 510 from the device 501A. At 704, the encoded multiple-intention composite image can be decoded to obtain a composite image and metadata describing one or more common spatial adjustments applied during generation of the composite image. For example, in the operating environment 500 of FIG. 5, the device 501B can decode the encoded multiple-intention composite image 510 to obtain the composite image 506 and the metadata 508, and the metadata 508 can describe one or more common spatial adjustments applied during generation of the composite image 506. At 706, one or more common spatial adjustments can be identified based on the metadata. For example, in the operating environment 500 of FIG. 5, the device 501B can identify one or more common spatial adjustments applied during generation of the composite image 506 based on the metadata 508.

[0090] At 708, the composite image can be adjusted to reverse one or more common spatial adjustments identified at 706, resulting in a real-rendered composite image. For example, in the operating environment 500 of FIG. 5, the device 501B can adjust the composite image 506 to reverse one or more common spatial adjustments identified based on the metadata 508 to obtain the real-rendered composite image 512. At 710, the real-rendered composite image can be adjusted according to the target rendering intention, resulting in a target-adjusted composite image. For example, in the operating environment 500 of FIG. 5, the device 501B can adjust the real-rendered composite image 512 based on the target rendering intention to obtain the target-adjusted composite image 514. At 712, the target-adjusted composite image can be displayed. For example, in the operating environment 500 of FIG. 5, the device 501B can render the target-adjusted composite image on a display.

[0091] FIG. 8 is a block diagram showing an example of components of a device 800 capable of implementing various aspects of the present disclosure. According to some aspects of the disclosure, the device 800 can perform operations executed by one or both of the devices 501A and 501B in the operating environment 500 of FIG. 5. According to some aspects of the disclosure, the device 800 can perform operations related to one or both of the method 600 of FIG. 6 and the method 700 of FIG. 7. Similar to the other figures provided in this specification, the number and type of elements shown in FIG. 8 are merely examples. Other implementations may include more, fewer, and / or different types and numbers of elements. According to some examples, the device 800 may be configured to execute at least some of the methods disclosed in this specification. In some embodiments, the device 800 can be or include one or more components of a television, audio, video, or multimedia system, a mobile device (such as a mobile phone), a laptop computer, a tablet device, a smart speaker, or another type of device.

[0092] In some implementations, device 800 may be or include a server. In some such implementations, device 800 may be or include an encoder. In some implementations, device 800 may be a device configured to be used within an audio, video, or multimedia environment, such as a home audio, video, or multimedia environment, and in other examples, device 800 may be a device configured to be used within a "cloud" such as a server.

[0093] In this example, device 800 includes interface system 805 and control system 810. Interface system 805 may be configured to communicate with one or more other devices within an audio, video, or multimedia environment in some implementations. The audio, video, or multimedia environment may be, in some examples, a home audio, video, or multimedia environment. In other examples, the audio, video, or multimedia environment may be another type of environment, such as an office environment, an automotive environment, a train environment, a street or sidewalk environment, a park environment, etc. Interface system 805 may be configured to exchange control information and related data with audio, video, or multimedia devices within an audio, video, or multimedia environment in some implementations. The control information and related data may, in some examples, be related to one or more software applications being executed by device 800.

[0094] Interface system 805 may be configured to receive or provide a content stream in some implementations. The content stream may include audio, video, or multimedia data. The audio, video, or multimedia data may include, but is not limited to, audio, video, or multimedia signals. In some cases, the content stream may include audio data including spatial data, such as channel data and / or spatial metadata. In some examples, the content stream may include video data and audio data corresponding to the video data.

[0095] The interface system 805 may include one or more network interfaces and / or one or more external device interfaces (e.g., one or more USB (universal serial bus) interfaces). According to some implementations, the interface system 805 may include one or more wireless interfaces. The interface system 805 may include one or more devices implementing a user interface such as one or more microphones, one or more speakers, a display system, a touch sensor system, and / or a gesture sensor system. In some examples, the interface system 805 may include one or more interfaces between the control system 810 and a memory system, such as the optional memory system 815 shown in FIG. 8. However, in some cases, the control system 810 may include a memory system. The interface system 805 may be configured to receive input from one or more microphones in the environment in some implementations.

[0096] The control system 810 may include, for example, a general-purpose single or multi-chip processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic elements, discrete gates or transistor logic, and / or discrete hardware components.

[0097] In some implementations, the functions of control system 810 may be present in more than one device. For example, a portion of control system 810 may be present in a device within one of the environments shown herein, and another portion of control system 810 may be present in a device external to the environment, such as a server, a mobile device (e.g., a smartphone, or a tablet computer), etc. In other examples, a part of control system 810 may be present in a device within one environment, and another part of control system 810 may be present in one or more other devices of the environment. For example, a part of control system 810 may be present in a device implementing a cloud-based service such as a server, and another part of control system 810 may also be present in another device implementing a cloud-based service such as another server or a memory device. Interface system 805 may also, in some examples, be present in multiple devices.

[0098] In some implementations, control system 810 may be configured to at least partially execute the methods disclosed herein. According to some examples, control system 810 may be configured to implement methods such as a method of utilizing a positivity control parameter when training a machine learning model, a method of utilizing a positivity control parameter in post-processing, etc.

[0099] Some or all of the methods described in this specification may be performed by one or more devices according to instructions (e.g., software) stored on one or more non-transitory media. Such non-transitory media may include memory devices as described herein, including but not limited to RAM (random access memory) devices, ROM (read-only memory) devices, and the like. The one or more non-transitory media may be present, for example, within an optional memory system 815 as shown in FIG. 8 and / or within a control system 810. Accordingly, various novel aspects of the subject matter described in this disclosure may be implemented on one or more non-transitory media storing software. The software may include, for example, instructions for utilizing the aggressiveness control parameter when training a machine learning model, instructions for utilizing the aggressiveness control parameter in post-processing, and the like. The software may be executable by one or more components of a control system such as, for example, control system 810 of FIG. 8.

[0100] In some examples, device 800 may include an optional microphone system 820 as shown in FIG. 8. The optional microphone system 820 may include one or more microphones. In some implementations, one or more microphones may be part of or associated with another device such as a speaker of a speaker system, a smart audio, video, or multimedia device. In some examples, device 800 may not include microphone system 820. However, in some such implementations, nevertheless, device 800 may be configured to receive microphone data from one or more microphones within an audio, video, or multimedia environment via interface system 810. In some such implementations, a cloud-based implementation of device 800 may be configured to receive microphone data or a noise metric at least partially corresponding to the microphone data from one or more microphones within an audio, video, or multimedia environment via interface system 810.

[0101] According to some implementations, device 800 may include an optional speaker system 825 as shown in FIG. 8. The optional speaker system 825 may include one or more speakers, which may also be referred to herein as "speakers" or more generally "audio playback transducers". In some examples (e.g., cloud-based implementations), device 800 may not include speaker system 825. In some implementations, device 800 may include headphones. The headphones can be connected or coupled to device 800 via a headphone jack or a wireless connection (e.g., BLUETOOTH®).

[0102] According to some embodiments, device 800 may include or be coupled to a display 830. The display 830 can include a display device that can display visual information, content, and / or effects such as, for example, the target-adjusted composite image 411 of FIG. 4. Examples of the display 830 can include, but are not limited to, a television, a monitor, a laptop computer screen, a projector, and a touch screen.

[0103] Although processes, systems, methods, heuristics, etc. are described herein, it should be understood that steps of such processes etc. are described as occurring in accordance with a particular ordered sequence, but such processes can be implemented with the described steps in an order different from that described herein. It should be further understood that certain steps can be executed simultaneously, other steps can be added, or certain steps described herein can be omitted. In other words, the description of the processes herein is provided for the purpose of describing particular embodiments and should not be considered as limiting the claims.

[0104] Accordingly, the foregoing description is intended to be illustrative and not limiting. Upon reading the foregoing description, many embodiments and applications other than the provided examples will be apparent. The scope should be determined with reference to the appended claims, rather than the foregoing description, and together with the full equivalent scope of the granted claims. Future developments may occur in the technology discussed herein, and it is expected and intended that the disclosed systems and methods will be incorporated into such future embodiments. In summary, it should be understood that this application is capable of modification and variation.

[0105] All terms used in the claims are intended to be given their broadest reasonable construction and their ordinary meaning as would be understood by one of ordinary skill in the art to which the technology described herein pertains. In particular, the use of singular articles such as "a," "the," "said," etc. should be read to state one or more of the indicated elements unless the claim expressly states a contrary limitation.

[0106] The summary of the present disclosure is provided to enable the reader to quickly evaluate the characteristics of the technical disclosure. It is understood that it is not used to interpret or limit the scope or meaning of the claims. Further, in the foregoing detailed description, it can be seen that various features are grouped together in various embodiments for the purpose of streamlining the present disclosure. This method of the present disclosure should not be construed as reflecting an intention that the claimed embodiments incorporate more features than those expressly recited in each claim. Rather, as the following claims reflect, the subject matter of the present invention lies in less features than all of a single disclosed embodiment. Accordingly, the following claims are incorporated herein by reference, and each claim stands on its own as a separately claimed subject matter.

[0107] The above-described system and method can provide for encoding and rendering a plurality of intended composite images.

[0108] Various aspects of the present invention may be apparent from the following enumerated example embodiments (EEE).

[0109] (EEE1) A method for encoding a synthetic image, the method comprising: obtaining a set of constituent images for the synthetic image; determining a common rendering intention applied to the set of constituent images; adjusting one or more of the constituent images in the set of constituent images according to the common rendering intention to obtain an adjusted set of constituent images; generating a synthetic image based on the adjusted set of constituent images; generating metadata characterizing the common rendering intention; and encoding the synthetic image and the metadata to generate an encoded multi-intention synthetic image.

[0110] (EEE2) The method according to EEE1, wherein the step of adjusting one or more of the constituent images in the set of constituent images according to the common rendering intention includes converting the constituent images in the set of constituent images from a current rendering intention to the common rendering intention.

[0111] (EEE3) The step of converting the constituent images in the set of constituent images from a current rendering intention to the common rendering intention includes: inverting one or more source adjustments of the constituent images; applying one or more common spatial adjustments to the constituent images; The method according to EEE2.

[0112] (EEE4) The method according to EEE3, wherein the step of applying the one or more common spatial adjustments to the constituent images includes converting sensor values to color values.

[0113] (EEE5) The step of applying the one or more common spatial adjustments to the compositional image includes estimating the ambient luminance and white point of the capture environment, and applying white point correction based on the estimated ambient luminance and white point of the capture environment, the method according to EEE3 or 4.

[0114] (EEE6) The step of applying the one or more common spatial adjustments to the compositional image includes estimating the ambient luminance of the capture environment, and applying an optical-optical transfer function (OOTF) based at least in part on the estimated ambient luminance of the capture environment to prepare the image for rendering on a reference display device, the method according to any one of EEE3 to 5.

[0115] (EEE7) The step of applying the one or more common spatial adjustments to the compositional image is as follows: Saturation enhancement, Contrast enhancement, Individual chroma adjustment, Slope offset power Tmid enhancement, Tone curve trimming, including the step of applying one or more of the above, the method according to any one of EEE3 to 6.

[0116] (EEE8) The step of adjusting one or more compositional images of the compositional image set according to the common rendering intention includes the step of converting the compositional images in the compositional image set from a pre-processed state to the common rendering intention, the method according to any one of EEE1 to 7.

[0117] (EEE9) The step of adjusting one or more compositional images of the compositional image set according to the common rendering intention is the step of converting a first compositional image of the compositional image set from a first current rendering intention to the common rendering intention, and the step of converting a second compositional image of the compositional image set from a second current rendering intention to the common rendering intention, including, the method according to any one of EEE1 to 8.

[0118] (EEE10) The step of determining the common rendering intention includes The method according to any one of EEE1 to 9, including the step of selecting the current rendering intention of the constituent image among the constituent image sets as the common rendering intention.

[0119] (EEE11) Further including the step of selecting the current rendering intention of the constituent image as the common rendering intention based on the importance of the constituent image with respect to other constituent images in the constituent image set, The importance of the constituent image is as follows: The relative size of the constituent image in the rendering space of the composite image, The resolution of the constituent image, The centrality of the position of the constituent image in the rendering space of the composite image, The focus of the viewer in the rendering space of the composite image, The method according to EEE10, determined based on one or more of the above.

[0120] (EEE12) The step of determining the common rendering intention includes the step of identifying a preferred rendering intention as the common rendering intention, the method according to any one of EEE1 to 9.

[0121] (EEE13) Further including the step of identifying the preferred rendering intention based on one or both of the input received via the user interface and the information in the configuration file, the method according to EEE12.

[0122] (EEE14) An apparatus configured to execute the method according to any one of EEE1 to 13.

[0123] (EEE15) A system including the apparatus according to EEE14 and a display.

[0124] A computer-readable medium storing an (EEE16) command, which, when executed by one or more processors, causes the one or more processors to execute the method according to any one of EEE1 to EEE13.

[0125] (EEE17) A method for rendering a composite image, the method comprising: Receiving an encoded multi-intent composite image; Decoding the encoded multi-intent composite image to obtain the composite image and metadata describing one or more common spatial adjustments applied when generating the composite image; Identifying the one or more common spatial adjustments based on the metadata; Adjusting the composite image to reverse the one or more common spatial adjustments to generate a real-rendered composite image; Adjusting the real-rendered composite image according to a target rendering intent to generate a target-adjusted composite image; Displaying the target-adjusted composite image. A method comprising the above steps.

[0126] (EEE18) The method according to EEE17, wherein the step of adjusting the real-rendered composite image according to the target rendering intent includes applying one or more target adjustments to the real-rendered composite image.

[0127] (EEE19) The method according to EEE18, wherein the step of applying the one or more target adjustments to the real-rendered composite image includes converting sensor values to color values.

[0128] (EEE20) The method according to EEE18 or EEE19, wherein the step of applying the one or more target adjustments to the real-rendered composite image includes estimating the ambient luminance and white point of the capture environment and applying white point correction based on the estimated ambient luminance and white point of the capture environment.

[0129] (EEE21) The step of applying the one or more target adjustments to the real-render composite image includes estimating the ambient luminance of the capture environment and applying an optical-optical transfer function (OOTF) based at least in part on the estimated ambient luminance of the capture environment to prepare the image for rendering on a reference display device, the method according to any one of EEE18 - 20.

[0130] (EEE22) The step of applying the one or more target adjustments to the real-render composite image is as follows: Saturation enhancement, Contrast enhancement, Individual chroma adjustment, Slope offset power Tmid enhancement, Tone curve trimming, including the step of applying one or more of the above, the method according to any one of EEE18 - 21.

[0131] (EEE23) Identifying the target rendering intention based on one or both of the input received via the user interface and the information in the configuration file, the method according to any one of EEE17 - 22.

[0132] (EEE24) An apparatus configured to execute the method according to any one of EEE17 - 23.

[0133] (EEE25) A system including the apparatus according to EEE24 and a display.

[0134] (EEE26) A computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to execute the method according to any one of EEE17 - 23.

[0135] The various features and advantages of the embodiments described herein are set forth in the following claims.

Claims

Claim 1 A method for encoding a composite image, the method comprising: obtaining a set of constituent images for the composite image, the set of constituent images being adjusted to apply respective different rendering intents; applying an inversion adjustment to the set of constituent images to cancel previously applied adjustments and obtain respective actual rendered images; determining a common rendering intent to be applied to the set of actual rendered images; adjusting one or more of the actual rendered images of the set of actual rendered images according to the common rendering intent to generate an adjusted set of constituent images; generating the composite image based on the adjusted set of constituent images; generating metadata characterizing the common rendering intent; encoding the composite image and the metadata to create an encoded multi-intent composite image; A method comprising the above steps. Claim 2 The step of adjusting one or more of the actual rendered images of the set of actual rendered images according to the common rendering intent includes converting a constituent image in the set of constituent images from a current rendering intent to the common rendering intent, according to Claim 1. Claim 3 The step of converting a constituent image in the set of constituent images from a current rendering intent to the common rendering intent includes: inverting one or more source adjustments of the constituent image; applying one or more common spatial adjustments to the constituent image; According to Claim 2. Claim 4 The step of applying one or more common spatial adjustments to the constituent image includes converting sensor values to color values, according to Claim 3. Claim 5 The step of applying one or more common spatial adjustments to the constituent image includes estimating a capture environment ambient luminance and a white point, and applying a white point correction based on the estimated capture environment ambient luminance and the white point, according to Claim 3 or 4. Claim 6 The step of applying one or more common spatial adjustments to the constituent image includes estimating a capture environment ambient luminance and, based at least in part on the estimated capture environment ambient luminance, applying an optical transfer function (OOTF) to prepare an image for rendering on a reference display device, according to any one of Claims 3 to 5. Claim 7 The step of applying the one or more common space adjustments to the compositional image is as follows: Saturation enhancement, Contrast enhancement, Individual color saturation adjustment, Slope offset power Tmid enhancement, Tone curve trimming, The method according to any one of claims 3 to 6, comprising the step of applying one or more of the above.

8. The step of adjusting one or more compositional images of the compositional image set according to the common rendering intention includes the step of converting the compositional images in the compositional image set from a preprocessed state to the common rendering intention. The method according to any one of claims 1 to 7.

9. The step of adjusting one or more compositional images of the compositional image set according to the common rendering intention is: The step of converting a first compositional image of the compositional image set from a first current rendering intention to the common rendering intention; and The step of converting a second compositional image of the compositional image set from a second current rendering intention to the common rendering intention. The method according to any one of claims 1 to 8.

10. The step of determining the common rendering intention includes: The method according to any one of claims 1 to 9, comprising the step of selecting the current rendering intention of the compositional image in the compositional image set as the common rendering intention.

11. The method further includes the step of selecting the current rendering intention of the compositional image as the common rendering intention based on the importance of the compositional image with respect to other compositional images of the compositional image set. The importance of the compositional image is as follows: The relative size of the compositional image in the rendering space of the composite image, The resolution of the compositional image, The centrality of the position of the compositional image in the rendering space of the composite image, The focus of the viewer in the rendering space of the composite image. The method according to claim 10, determined based on one or more of the above.

12. The step of determining the common rendering intention includes the step of identifying a preferred rendering intention as the common rendering intention. The method according to any one of claims 1 to 9.

13. The method according to claim 12, further comprising the step of identifying the preferred rendering intention based on one or both of the input received via the user interface and the information in the configuration file.

14. ​ An apparatus configured to execute the method according to any one of claims 1 to 13.

15. A system including the apparatus according to claim 14 and a display.

16. A computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to execute the method according to any one of claims 1 to 13.

17. A method for rendering a composite image, the method comprising: Receiving an encoded multi-intent composite image; Decoding the encoded multi-intent composite image to obtain the composite image and metadata describing one or more common spatial adjustments applied when generating the composite image; Identifying the one or more common spatial adjustments based on the metadata; Adjusting the composite image to reverse the one or more common spatial adjustments to generate a real-rendered composite image; Adjusting the real-rendered composite image according to a target rendering intent to generate a target-adjusted composite image; Displaying the target-adjusted composite image. The method includes.

18. The step of adjusting the real-rendered composite image according to the target rendering intent includes applying one or more target adjustments to the real-rendered composite image, according to the method of claim 17.

19. The step of applying the one or more target adjustments to the real-rendered composite image includes converting sensor values to color values, according to the method of claim 18.

20. The step of applying the one or more target adjustments to the real-rendered composite image includes estimating the ambient luminance and white point of the capture environment and applying white point correction based on the estimated ambient luminance and white point of the capture environment, according to the method of claim 18 or 19.

21. The step of applying the one or more target adjustments to the real-rendered composite image includes estimating the ambient luminance of the capture environment and applying an optical-optical transfer function (OOTF) based at least in part on the estimated ambient luminance of the capture environment to prepare an image for rendering on a reference display device, according to the method of any one of claims 18 to 20.

22. The step of applying the one or more target adjustments to the real-rendered composite image includes the following: Saturation enhancement, Contrast enhancement, Individual color saturation adjustments, Slope offset power Tmid emphasis, Tone curve trim, The method according to any one of claims 18 to 21, comprising the step of applying one or more of the above.

23. The method according to any one of claims 17 to 22, comprising the step of identifying the target rendering intention based on one or both of the input received via the user interface and the information in the configuration file.

24. An apparatus configured to execute the method according to any one of claims 17 to 23.

25. A system comprising the apparatus according to claim 24 and a display.

26. A computer-readable medium storing instructions which, when executed by one or more processors, cause the one or more processors to execute the method according to any one of claims 17 to 23.

Images