Dynamic metadata calculation for editing HDR content
The system generates metadata using fixed and average luminance values with spatial smoothing and windowed FIR filtering to enhance video quality and stability during HDR content editing and display management, addressing inefficiencies and temporal instability in existing methods.
Patent Information
- Application Number
- JP2022574122
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2020-06-03
- Filing Date
- 2021-06-02
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2041-06-02
AI Technical Summary
Existing methods for generating metadata for HDR content are inefficient, computationally intensive, and result in temporal instability during video editing and display management, particularly when transitioning between different dynamic ranges.
A system and method for generating metadata that includes calculating initial metadata values based on fixed and average luminance values, applying spatial smoothing, and using windowed FIR filtering to ensure temporal stability and reduced computational requirements.
Improves video quality and stability during editing and display management by reducing computational load and latency, allowing for efficient conversion between dynamic ranges without significant temporal flickering.
Smart Images

Figure 0007577762000005 
Figure 0007577762000006 
Figure 0007577762000007
Abstract
Description
[Technical field]
[0001] [Related Applications] This application claims priority to European Patent Application No. 20178044.2 and U.S. Provisional Application No. 63 / 034,006, both filed on June 3, 2020, which are hereby incorporated by reference in their entireties. [Background technology]
[0002] 1. Related fields This application relates generally to images, and more particularly, to generating metadata for display management associated with video content.
[0003] 2. Background technology As used herein, the term "dynamic range (DR)" may relate to the ability of the human visual system (HVS) to perceive a range of intensities (e.g., luminance, luma) in an image, e.g., from darkest blacks to brightest whites (highlights). In this scene, DR relates to "scene-referred" intensity. DR may also relate to the ability of a display device to properly or approximately render an intensity range of a particular width. In this scene, DR relates to "display-referred" intensity. At any point in the description herein, unless it is explicitly specified that a particular scene has a particular importance, it should be presumed that the terms may be used in either scene, e.g., synonymously.
[0004] As used herein, the term "high dynamic range (HDR)" refers to a DR width that spans 14-15 times the magnitude of the HVS or more. In fact, the DR where humans can simultaneously perceive a wide range in the intensity range may be omitted in some way in relation to HDR. As used herein, the terms "enhanced dynamic range (EDR)" or "visual dynamic range (VDR)" refer individually or synonymously to the DR perceivable in a scene or image by the HVS, including eye movements, allowing any light adaptation to change across the scene or image. As used herein, EDR may refer to a DR that spans 5-6 orders of magnitude. Thus, although perhaps somewhat narrower than the actual scene referred to as HDR, EDR nevertheless represents a wide DR width and may also be referred to as HDR.
[0005] In practice, an image contains one or more color components of a color space (e.g., RGB, luma Y, and chroma Cb and Cr), and in a quantized digital system, each color component is represented with n bits of precision per pixel (e.g., n=8). Bit depths n≦8 (e.g., color 24-bit JPEG images) can be used for standard dynamic range (SDR) images, while bit depths n>8 can be considered for extended dynamic range (EDR) images to avoid contouring and stair-stepping artifacts. In addition to integer data types, EDR and high dynamic range (HDR) images may be stored and distributed using high-definition (e.g., 16-bit) floating-point formats, such as the OpenEXR file format developed by Industrial Light and Magic.
[0006] A reference electro-optical transfer function (EOTF) for a given display characterizes the relationship between the color values (e.g., luminance) of the input video signal and the output screen color values (e.g., screen luminance) produced by the display. For example, ITU Rec. ITU-R BT1886, "Reference electro-optical transfer function for flat panel displays used in HDTV studio production" (March 2011), which is incorporated herein by reference in its entirety, defines a reference EOTF for flat panel displays based on the measured characteristics of a Cathode Ray Tube (CRT). Given a video stream, information about its EOTF may be embedded in the bitstream as metadata.
[0007] Many consumer desktop displays render non-HDR content at a maximum brightness of 200 to 300 cd / m2 ("nits"), while consumer high definition and ultra-high definition televisions ("HDTV" and "UHD TV") render at 300 to 500 nits. Such display outputs are typical of low dynamic range (LDR), also referred to as SDR in the context of HDR or EDR. As the availability of HDR or EDR content increases due to advances in both capture devices (e.g., cameras) and HDR displays (e.g., Dolby Laboratories' PRM-4200 Professional Reference Monitor or Sony Trimaster HX31" 4K HDR Master Monitor), HDR content may be color graded and displayed on HDR displays that support a higher dynamic range (e.g., 700 nits to 5000 nits, or higher). Such displays may be defined using an alternative EOTF that supports high brightness capabilities (e.g., 0 to 10000 nits). An example of such an EOTF is defined in SMPTE ST 2084:2014, "High Dynamic Range EOTF of Mastering Reference Displays," the entire contents of which are incorporated herein by reference.
[0008] Display management, as used herein, refers to the processing (e.g., tone and gamut mapping) required to map an input video signal of a first dynamic range (e.g., 1000 nits) to a display of a second dynamic range (e.g., 500 nits). An example of a display management process is described in PCT Application No. PCT / US2014 / 016304 (the '304 application), filed February 13, 2014, "Display management for high dynamic range video," R. Atkins et al., which is incorporated herein by reference in its entirety. Display management may be aided or controlled by input metadata generated by the source of the incoming content and multiplexed into the coding bitstream.
[0009] The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Thus, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section. Similarly, problems identified with one or more of the approaches should not be assumed to have been recognized in any prior art under this section, unless otherwise indicated. Summary of the Invention
[0010] Various aspects of the present disclosure relate to apparatus, systems, and methods for generating metadata that may be used to provide improved video quality, improved temporal stability, low computational requirements, and reduced latency.
[0011] According to one aspect of the present disclosure, there is provided a system and method for editing video content, comprising: receiving input video data; converting the input video data into a predetermined format, thereby generating converted video data; generating a plurality of initial metadata values for a frame of the transformed video data, the plurality of initial metadata values including: a first metadata value corresponding to a first fixed value not calculated from content comprising the frame, a second metadata value corresponding to an average luminance value of the frame, and a third metadata value corresponding to a second fixed value not calculated from the content; Including, Systems and methods are provided, wherein the first metadata value, the second metadata value, and the third metadata value comprise information used by a decoder to render a decoded image on a display.
[0012] According to another aspect of the present disclosure, there is provided a system for editing video content, comprising: Memory, A processor; The processor includes: Receives input video data; Converting the input video data into a predetermined format, thereby generating converted video data; generating a plurality of initial metadata values for a frame of the transformed video data, the plurality of initial metadata values including: a first metadata value corresponding to a first fixed value not calculated from content comprising the frame; a second metadata value corresponding to an average luminance value of the frame; and a third metadata value corresponding to a second fixed value not calculated from the content; A system is provided.
[0013] According to another aspect of the present disclosure, there is provided a method for editing video content, comprising: receiving input video data; converting the input video data into a predetermined format, thereby generating converted video data; generating a first metadata value for a frame of the transformed video data, the first metadata value corresponding to an average luminance value of the frame; determining a time window that includes the frame, the time window having a predetermined length of time; calculating a maximum metadata range and a minimum metadata range over the time window, thereby generating a first filtered metadata value and a second filtered metadata value; smoothing the filtered metadata values and the second filtered metadata values, thereby generating first smoothed and filtered metadata values and second smoothed and filtered metadata values; A method is provided that includes:
[0014] Thus, various aspects of the present disclosure are effective to improve video capture, editing, and signal processing, as well as distribution of related fields of video and other content. [Brief description of the drawings]
[0015] These and other more detailed and specific features of the various embodiments are more fully disclosed in the following description, which refers to the accompanying drawings.
[0016] [Figure 1] 1 illustrates another example video delivery pipeline in accordance with various aspects of the present disclosure.
[0017] [Diagram 2] 1 illustrates a process flow of an exemplary metadata generation method according to various aspects of the present disclosure.
[0018] [Diagram 3] 3 illustrates exemplary metadata generated by the method of FIG. 2.
[0019] [Figure 4] 1 illustrates a process flow of another exemplary metadata generation method according to various aspects of the present disclosure.
[0020] [Diagram 5] 3 illustrates exemplary metadata generated by the method of FIG. 2.
[0021] [Figure 6] 1 illustrates a process flow of another exemplary metadata generation method according to various aspects of the present disclosure.
[0022] [Figure 7A] 7A-7D each show exemplary metadata generated by the method of FIG. [Figure 7B] 7A-7D each show exemplary metadata generated by the method of FIG. [Figure 7C] 7A-7D each show exemplary metadata generated by the method of FIG.
[0023] [Figure 8] 1 illustrates another exemplary system according to various aspects of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0024] The present disclosure and aspects thereof may be embodied in a variety of forms, including hardware, devices, or circuits controlled by computer-implemented methods, computer program products, computer systems and networks, user interfaces, and application programming interfaces, as well as hardware-implemented methods, signal processing circuits, memory arrays, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), etc. The foregoing summary is intended merely to provide a general idea of various aspects of the disclosure and is not intended to limit the scope of the disclosure in any way.
[0025] In the following description, numerous details are set forth, such as optical device configurations, timing, operation, etc., in order to provide an understanding of one or more aspects of the present disclosure. Those skilled in the art will readily appreciate that these specific details are merely examples and are not intended to limit the scope of the present application.
[0026] Additionally, while this disclosure primarily focuses on examples where the various circuits are used in content generation, it will be understood that this is merely one example of implementation. It will be further understood that the disclosed systems and methods can be used in any device that needs to generate, modify or display image data and / or associated metadata, such as cinema, consumer and other commercial projection systems, smartphones and other consumer electronic devices, heads-up displays, virtual reality displays, etc.
[0027] Metadata
[0028] As used herein, the term "metadata" refers to any auxiliary information transmitted as part of an encoded bitstream that aids a decoder in rendering a decoded image. Such metadata includes, but is not limited to, the following: description These may include color space or gamut information, reference display parameters, and auxiliary signal parameters, as described above, collectively referred to as "imaging metadata." and You may be called.
[0029] Such metadata may be split or categorized into different levels of metadata. In one particular example, the present disclosure is implemented with a Dolby Vision architecture and the mapping metadata may include L0 metadata, L1 metadata, L2 / L8 metadata, L3 metadata, L4 metadata, L11 metadata, etc. In some examples, the L0 metadata provides information about or describes the dynamic range of the video data across the entire video content (i.e., across multiple shots and / or scenes). For example, the L0 metadata may include L0min and L0max, which represent the minimum and maximum luminance of the video content, respectively. For example, the L0 metadata may include L0min and L0max, which represent the minimum and maximum luminance of the video content, respectively.
[0030] In some examples, the L1 metadata provides or describes information regarding the distribution of luminance values of a source image, source scene, source group of pictures (GOP), etc., as represented in the video data. The distribution of luminance values may be represented by a histogram derived based on the image content (e.g., pixel values, luma values, chroma values, Y values, Cb / Cr values, RGB values, etc.) of an image, scene, GOP, etc. The L1 metadata may include parameters L1min, L1mid, L1max, which respectively represent the minimum luminance ("crush"), midtone luminance ("mid"), and maximum luminance ("clip") of one or more images, scenes, GOPS, etc., within the video content.
[0031] Other levels of metadata may also be defined. For example, L2 provides information about or describes video characteristic adjustments that originate from or trace back to adjustments made by a director, color grader, video professional, etc. in the production studio and / or in post-production. L8 metadata is similar to L2 metadata and in some cases may be equivalent to it (e.g., by their respective tone curves). L2 and L8 metadata may be referred to as "trim" parameters and may indicate or relate to gain / offset / power of image data. L2 metadata may correspond to a first reference display with a first reference dynamic range.
[0032] For example, the L3 metadata provides information about or describes video characteristic adjustments that result from or trace back to adjustments made by a director, color grader, video professional, etc. In comparison to the L2 metadata, the L3 metadata may correspond to a second reference display that has a reference dynamic range that is different from the reference dynamic range of the first reference display. The L3 metadata may include offsets or adjustments from the L1 metadata, including, for example, offsets or adjustments to crash, intermediate, and clip luminance values.
[0033] The L4 metadata provides or describes information regarding global dimming operations. The L4 metadata may be calculated by the encoder during pre-processing and may be calculated using RGB primaries. In one example, the L4 metadata may include data indicating the global backlight brightness level of the display panel on a frame-by-frame basis. Other generated metadata, such as L11 metadata, may provide or describe information used to identify a source of video data, such as movie content, computer game content, sports content, etc. Such metadata may further provide or describe intended picture settings, such as intended white point, sharpness, etc.
[0034] Taken together, the image formation metadata can include transformation data for transforming a first dynamic range to a second dynamic range different from the first dynamic range. In some aspects of the present disclosure, the first dynamic range can be higher than the second dynamic range (e.g., transforming from HDR to SDR). In other aspects of the present disclosure, the second dynamic range can be higher than the first dynamic range (e.g., transforming from SDR to HDR).
[0035] Comparing methods of generating image formation metadata include calculating per-frame metadata (e.g., per-frame L1 metadata) and applying IIR filtering (e.g., for live broadcasts and games) or calculating scene min, mean, and max to combine metadata into the scene (e.g., for studios and professional offline content). However, these methods do not allow for immediate preview of the content while editing. For example, a user may want to make an HDR picture brighter and preview the result on an SDR monitor as a reference display. To do this, the following steps are performed: adjustments are applied to the HDR picture; L1 metadata is recalculated for the adjusted HDR picture; tone mapping operation is modified; and the adjusted tone mapping is applied to the adjusted HDR picture to generate an SDR preview picture.
[0036] This sequence of operations may be satisfactory when applied to a single image, since it is reasonable to store one image in memory, and so the operations can be applied in quick succession. However, the L1 metadata (and resulting tone curve) may vary significantly from frame to frame, causing the mapped image to flicker. This is sometimes called "temporal instability". To ensure the temporal stability of the tone curve, some filtering may be performed. However, when the metadata recalculation operation depends on multiple images, the adjustment operations and metadata recalculation operations would need to be applied to many images, which may be impractical or impossible at fast video rates.
[0037] Thus, comparison methods that generate image formation metadata can slow down to iterative cycles of adjusting, updating, and previewing the results. Additionally, IIR filtering comparison methods may require that a large number of frames be analyzed prior to the current frame, since the current frame requires all previous frames for the IIR filter. Studio or professional comparison methods require every frame of the scene, which may have durations on the order of minutes or hours.
[0038] In this regard, various aspects of the present disclosure provide methods for generating metadata for efficient editing, i.e., editing that results in high image quality and high temporal stability. Various aspects of the present disclosure can achieve such editing with low computational requirements and low or no latency. In some aspects, the methods for generating metadata are incorporated into or used in a video distribution pipeline.
[0039] Video Delivery Pipeline
[0040] FIG. 1 illustrates an exemplary video delivery pipeline 100, showing various stages from video capture to video content display. It is noted that the following description is provided in terms of video (i.e., moving images), but the disclosure is not limited thereto. In some examples, the image content may be a still image, or a combination of video and still images. The image content may be represented by raster (or pixel) graphics, by vector graphics, or by a combination of raster and vector graphics. FIG. 1 illustrates an image generation block 101, a production block 102, a post-production block 103, an encoding block 104, a decoding block 105, and a display management block 106. The various blocks illustrated in FIG. 1 may be implemented as or via hardware, software, firmware, or combinations thereof. Furthermore, various groups of the illustrated blocks may combine their respective functions and / or may execute on different devices and / or at different times. Each one or group of the illustrated blocks may be implemented by circuitry including, but not limited to, a central processing unit (CPU), a graphics processing unit (GPU), an ASIC, an FPGA, and combinations thereof. The operations performed by one or more blocks may be processed locally, remotely (e.g., cloud-based), or a combination of locally and remotely.
[0041] 1, the video delivery pipeline further includes a reference display 111 and a target display 112, which are provided to assist or monitor the operations performed in the post-production block 103. For purposes of discussion, the image generation block 101, the production block 102, the post-production block 103, and the encoding block 104 may be referred to as "upstream" blocks or components, and the decoding block 105 and the display management block 106 may be referred to as "downstream" blocks or components.
[0042] In the example of FIG. 1, video content in the form of a series of video frames 121 is captured or generated in the image generation block 101. The video frames 121 may be digitally captured (e.g., by a digital camera) or computer generated (e.g., using computer animation) to generate video data 122. Alternatively, the video frames 121 may be filmed by a film camera and converted to a digital format to provide the video data 122. In either case, the video data 122 is provided to the production block 102, where it is edited to provide the production stream 123. The video frames 121 may be grouped into sequences, such as shots or scenes. A shot is, for example, a set of frames that are temporally connected. Shots may be delimited (e.g., when the entire content of the image changes, not just a portion of it). A scene is, for example, a series of shots that illustrate a storytelling segment of a larger content. In one particular example where the video content is an action movie, the video content may include a chase scene including a sequence of shots (e.g., a shot of the driver of the chasing vehicle, a shot of the driver of the chased vehicle, a shot of the street along which the chase takes place, etc.).
[0043] The video data in the production stream 123 is then provided to a processor in the post-production block 103 for post-production editing. Editing performed in the post-production block 103 may include adjusting or changing the color or brightness of certain areas of the image to improve image quality or achieve a particular look according to the video creator's (or editor's) creative intent. This may also be referred to as "color timing" or "color grading". Other editing (e.g., scene selection and sequencing, cropping images, adding computer-generated visual special effects or overlays, etc.) may also be performed in the post-production block 103 to generate the delivery stream 124. In some examples, the post-production block 103 may provide the intermediate stream 125 to the reference display 111 so that the images can be viewed on its screen, for example to aid in the editing process. One, two, or all of the production block 102, the post-production block 103, and the encoding block 104 may further include processing to add metadata to the video data. This additional processing may include, but is not limited to, statistical analysis of the properties of the content. Subsequent processing may be performed locally or remotely (e.g., cloud-based processing).
[0044] Following post-production operations, the distribution stream 124 may be delivered to an encoding block 104 for downstream distribution to decoding and playback devices such as television sets, set-top boxes, movie theaters, laptop computers, tablet computers, etc. In some examples, the encoding block 104 may include audio and video encoders, such as those defined by the Advanced Television Systems Committee (ATSC), Digital Video Broadcasting (DVB), Digital Versatile Disc (DVD), Blu-Ray, and other distribution formats, to generate a coded bitstream 126. At the receiver, the coded bitstream 126 is decoded by a decoding unit 105 to generate a decoded signal 127 that represents the same or a close approximation of the distribution stream 124. The receiver may be attached to a target display 112, which may have different characteristics than the reference display 111. If the reference display 111 and the target display 112 have different characteristics, the display management block 106 can be used to map the dynamic range or other characteristics of the decoded signal 127 to the characteristics of the target display 112 by generating a display mapping signal 128. The display management block 106 can additionally or alternatively be used to provide power management for the target display 112.
[0045] The target display 112 generates an image using an array of pixels. The particular array structure depends on the display architecture and resolution. For example, if the target display 112 operates with an LCD architecture, it may include a relatively low resolution backlight array (e.g., an array of LEDs or other light emitting devices) and a relatively high resolution liquid crystal array and color filter array for selectively attenuating the white light from the backlight array and providing colored light (often referred to as dual modulation display technology). If the target display 112 operates with an OLED architecture, it may include a high resolution array of self-emitting color pixels.
[0046] The link between the upstream block and the downstream block (i.e., the path along which the coded bitstream 126 is provided) may be implemented by broadcasting over the air using electromagnetic waves or by live or real-time transfer over content distribution lines such as optical fiber, twisted pair (Ethernet), and / or coaxial cable. In other examples, the link may be implemented by a time-independent transfer, such as recording the coded bitstream on a physical medium (e.g., DVD or hard disk) for physical delivery to an end-user device (e.g., DVD player). The decoding block 105 and the display management block 106 may be incorporated into a device associated with the target display 112, for example in the form of a smart television that includes decoding, display management, power management, and display functions. In some examples, the decoding block 105 and / or the display management block 106 may be incorporated into a device separate from the target display 112, for example in the form of a set-top box or media player.
[0047] The decoding block 105 and / or the display management block 106 may be configured to receive, analyze, and act in response to metadata included or added by upstream blocks. Such metadata may then be used to provide additional control or management of the target display 112. The metadata may include the image formation metadata discussed above (e.g., DolbyVision metadata). The metadata may be generated in the post-production block 103 using the reference display 111 and the production stream 123 and / or intermediate stream 125.
[0048] Semi-dynamic metadata generation
[0049] Imaging metadata can be generated using a "semi-dynamic" method, an example of which is shown in Figure 2. The operations shown in Figure 2 can be performed automatically using hardware, software, firmware, or a combination thereof associated with post-production block 103 of Figure 1.
[0050] As shown in FIG. 2, the semi-dynamic metadata generation process may begin with operation 201 where the video signal is converted to a predefined format (e.g., a particular EOTF). For example, the video signal may be converted from Hybrid Log-Gamma (HLG) or SLOG-3 HDR to RGB Perceptual Quantizer (PQ) HDR as described in Rec. ITU-RBT.2100-1 (06 / 2017). In some implementations, spatial smoothing and / or subsampling may be applied in operation 202. Operation 202 may be implemented using a two-dimensional smoothing kernel (e.g.,
[0011] / 2 or [1 3 3 1] / 8), which reduces sensitivity to individual pixels. In some examples, the image may be decimated to a smaller size (e.g., half size) after filtering. In other implementations, operation 202 may be omitted and the semi-dynamic metadata generation process may proceed directly from operation 201 to operation 203. In operation 203, an initial value for L1mid is calculated (e.g., frame by frame). For example, L1mid may represent the average of the maximum RGB luminance values. In other words, L1mid may represent the average of the maximum of the color component values R, G, B for each pixel. The values of L1min and L1max are not calculated from the content, instead in operation 204 fixed values are used that correspond to the minimum and maximum allowed range of the image content, as expressed in the L0 metadata (e.g., Level0.Smin, Level0.Smax). For example, when using a YCbCr representation, L1mid may represent the average luminance value of the Y component values.
[0051] As one example of the operations of FIG. 2, the following pseudocode is presented using a MATLAB®-like format.
number
[0052] FIG. 3 shows an example output of the above operation on metadata corresponding to the crash, mean, and clip values. Each graph shows normalized luminance as a function of frame number. Line 310 min, 310 mid , and 310 max Line 320 indicates the minimum, average, and maximum luminance values for a given frame, respectively. min , 320 mid , and 320 max respectively denote the values L1min, L1mid, and L1max for a given frame.
[0053] Because L1min and L1max are set to the values of L0min and L0max, respectively, the metadata values for the crash and clip do not exactly match the actual luminance values of every frame. In the particular example shown in FIG. 3, the minimum luminance value per frame (line 310 min ) varies between approximately 0.01 and 0.48, and the value L1min (line 320 min ) is set to approximately 0. The maximum luminance value per frame (line 310 max ) varies between approximately 0.53 and 0.82, and the value L1max (line 320 max ) is set to about 0.82. However, since L1mid is determined by a dynamic value, the mid metadata value closely matches the actual luminance value of every frame. As shown in Figure 3, the average luminance value per frame (line 310 mid ) and value L1mid (line 320 mid ) are highly correlated.
[0054] With semi-dynamic generation of metadata, only one frame needs to be processed. This method results in high efficiency. However, as can be seen in Figure 3, using static values for crash and clip metadata values results in slightly lower mapping image contrast for images that do not use the full dynamic range of the container.
[0055] Dynamic Metadata Generation
[0056] Imaging metadata can be generated using a "dynamic" method, an example of which is shown in Figure 4. The operations shown in Figure 4 can be performed automatically using hardware, software, firmware, or a combination thereof associated with post-production block 103 of Figure 1.
[0057] As shown in FIG. 4, the dynamic metadata generation process may begin at operation 401 where a video signal is converted to a predetermined format (eg, a particular EOTF). For example, the video signal may be converted from HLG or SLOG-3 HDR to RGB PQ HDR as described in Rec. ITU-RBT.2100-1 (06 / 2017). In some implementations, spatial smoothing and / or subsampling may be applied in operation 402. Operation 402 may be implemented using a two-dimensional smoothing kernel (e.g.,
[0011] / 2 or [1 3 3 1] / 8), which reduces sensitivity to individual pixels. In some examples, the image may be decimated to a smaller size (e.g., half size) after filtering. In other implementations, operation 402 may be omitted and the semi-dynamic metadata generation process may proceed directly from operation 401 to operation 403. In operation 403, initial values for L1min, L1mid, and L1max are calculated (e.g., on a frame-by-frame basis). For example, L1min may represent the minimum RGB luminance value, L1mid may represent the average of the maximum RGB luminance values, and L1max may represent the maximum RGB luminance value.
[0058] Thereafter, in operation 404, the L1min and L1max values are recalculated using the dynamic values for the given shot or scene. For example, each frame of the shot or scene may be analyzed to determine minimum and maximum RGB luminance values for the entire given shot or scene. In operation 405, the L1min and L1max values are recalculated using values corresponding to the entire clip minimum and maximum values determined in operation 404. Compared to the semi-dynamic scheme described above, the dynamic scheme may correspond more closely to the image data, since L1min and L1max are determined with respect to the actual minimum and maximum RGB luminance of the content, rather than the minimum and maximum RGB luminance that is allowed (i.e., possible) for the content.
[0059] As one specific example of the operations of FIG. 4, the following pseudocode is presented using a MATLAB®-like format.
number
[0060] FIG. 5 shows an example output of the above operation on metadata corresponding to the crash, mean, and clip values. Each graph shows normalized luminance as a function of frame number. Line 510 min , 510 mid , and 510 max Line 520 indicates the minimum, average, and maximum luminance values for a given frame, respectively. min , 520 mid , and 520 max respectively denote the values L1min, L1mid, and L1max for a given frame.
[0061] Because L1min and L1max are set to minimum values across the frames of the clip, the crash and clip metadata values do not match exactly with the actual luminance values of all frames. However, compared to the semi-dynamic approach described above, the crash and clip metadata values match more closely with the actual luminance values. In the particular example shown in Figure 5, the minimum luminance value per frame (line 510) min) varies between approximately 0.02 and 0.48, with the value L1min (line 510 min ) is set to about 0.02. Meanwhile, in the semi-dynamic method shown in FIG. min ) was set to 0. The maximum luminance value per frame (line 510 max ) varies between approximately 0.53 and 0.82, and the value L1max (line 520 max ) is set to about 0.82. As mentioned above, L1mid is determined by a dynamic value, so the mid metadata value closely matches the actual luminance value of every frame. As shown in Figure 5, the average luminance value per frame (line 510 mid ) and value L1mid (line 520 mid ) are highly correlated.
[0062] Dynamic generation of metadata may have higher computational requirements compared to semi-dynamic generation of metadata, since it requires processing of all frames in a given shot or scene. However, as can be seen in Figure 5, using dynamic values for crash and clip metadata values means that images that do not use the full dynamic range of the container do not suffer from the same type of contrast reduction of mapped images.
[0063] Windowed FIR Metadata Generation
[0064] The imaging metadata may be generated using a "windowing" method, an example of which is shown in Figure 6. The operations shown in Figure 6 may be performed automatically using hardware, software, firmware, or a combination thereof associated with post-production block 103 of Figure 1.
[0065] As shown in FIG. 6, the windowed metadata generation process may begin with operation 601 where a video signal is converted to a predefined format (e.g., a particular EOTF). For example, the video signal may be converted from HLG or SLOG-3 HDR to RGB PQ HDR as described in Rec. ITU-RBT.2100-1 (06 / 2017). Then, in operation 602, spatial smoothing and / or subsampling is applied. Operation 402 may be implemented using a two-dimensional smoothing kernel (e.g.,
[0011] / 2 or [1 3 3 1] / 8), which reduces sensitivity to individual pixels. In some examples, the image may be decimated to a smaller size (e.g., half size) after filtering. In operation 603, initial values for L1min, L1mid, and L1max are calculated (e.g., in pixels). For example, L1min may represent the minimum value of the minimum RGB luminance, L1mid may represent the average value of the maximum RGB luminance, and L1max may represent the maximum value of the maximum RGB luminance.
[0066] Next, in operation 604, a list of frames within a moving window (k) is determined, which may be specified as the maximum length of time T multiplied by the frame rate. The time window T (also referred to as the "temporal" window) is configurable by an operator. In some examples, the time window T may be set between 0.4 seconds and 8 seconds. In one particular example, the time window T is 4 seconds. As will be described below, a particular value of the time window T is selected to balance temporal stability and image quality. For example, a shorter window may introduce some flicker, while a longer window may result in a more static process. However, the window itself is configured not to span a scene cut, and is therefore bounded by the first and last frames of the current scene.
[0067] Operation 605 is the first pass, where the minimum and maximum values of the window are calculated. For example, for each frame f in the moving window k, operation 605 may smooth the metadata using a 1D smoothing kernel such as
[0011] / 2,
[0121] / 4, [1 3 3 1] / 8, or [1 3 8 3 1] / 16, thereby obtaining smoothed metadata L1minS(f) and L1maxS(f). This reduces the sensitivity to individual frames. Operation 605 may then determine the maximum metadata range over the moving window k. For example, it may calculate L1minM(f)=min(L1minS(k)) (i.e., the minimum smoothed minimum metadata value) and L1maxM(f)=max(L1max(S(k)) (i.e., the maximum smoothed maximum metadata value).
[0068] Operation 606 is a second pass where the minimum and maximum values of the window are smoothed. For example, operation 606 may calculate a smoothing filter F as a Gaussian shape with L taps and standard deviation s=L / 6, and then, for each frame f in the moving window k, smooth the filtered metadata calculated in operation 605 with the smoothing filter F to obtain smoothed and filtered metadata values L1minF and L1maxF. Operation 606 may also set L1min and L1max to the smoothed and filtered metadata values and L1mid to the initial value determined in operation 603 for each frame f.
[0069] Since operations 605 and 606 are applied to a moving window of frames, operations 604, 605, and 606 can be repeated so that all possible windows in a given scene are processed.
[0070] As one specific example of the operations of FIG. 6, the following pseudocode is presented using a MATLAB®-like format.
number
number
[0071] 7A-7C show example outputs of the above operations on metadata corresponding to the crush, mean, and clip values for different window lengths. Each graph shows normalized luminance as a function of frame number. In each of FIGS. 7A-7C, line 710 min , 710 mid , and 710 max In FIG. 7A, line 721 indicates the minimum, average, and maximum luminance values for a given frame, respectively. min , 721 mid , and 721 max indicates, for a given frame, the values L1min, L1mid, and L1max calculated according to the method described above with a window length of 1 second. min , 722 mid , and 722 max shows, for a given frame, the values L1min, L1mid, and L1max calculated according to the method described above with a window length of 4 seconds. min , 723 mid , and 723 max denotes for a given frame the values L1min, L1mid, L1max calculated according to the method described above with a window length of 1 second.
[0072] 7A-7C, while decreasing the window length improves image quality (e.g., line 721 max Line 710 max (more closely follows line 723), while increasing the window length improves temporal stability (e.g., line 723 max It can be seen that, in one particular implementation of the present disclosure, the window length is preferably 4 seconds, as shown in FIG. 7B.
[0073] Windowed metadata generation generates metadata that is a moving window clash and the outer envelope of the clip. Compared to semi-dynamic generation of metadata, it requires more frames to compute, but unlike the above comparison method, the number of frames required to compute the metadata for any single frame is finite. By applying the filter twice in succession, the resulting filter is essentially three times as long as the specified window. Successive operations can be combined into a single operation or flow, for example using the pseudocode above.
[0074] Exemplary Computer System
[0075] Various aspects of the disclosure may be implemented by a computer system, a system configured in electronic circuits and components, an integrated circuit (IC) device such as a microcontroller, an FPGA, or another configurable or programmable logic device (PLD), a discrete time or digital signal processor (DSP), an ASIC, and / or an apparatus including one or more of such systems, devices, or components. The computer and / or IC may execute, control, or perform instructions related to the processes and operations described above. The computer and / or IC may calculate any of the various parameters or values related to the processes and operations described above.
[0076] 8 illustrates an example of a computer system 800 in which one or more aspects of the present disclosure may be implemented. As shown, computer system 800 includes a bus 502 or other communication mechanism for communicating information, and a hardware processor 802 coupled to bus 801 for processing information. Processor 802 may be, for example, a general-purpose microprocessor.
[0077] Computer system 800 also includes a main memory 803, such as a random access memory (RAM) or other dynamic storage device, coupled to bus 801 for storing information and instructions to be executed by processor 802. Main memory 803 may also be used for storing time variables or other intermediate information during execution of instructions to be executed by processor 802. Such instructions, when stored in a non-transitory storage medium accessible by processor 802, make computer system 800 a special-purpose machine customized to perform the operations specified in the instructions.
[0078] Computer system 800 further includes a read only memory (ROM) 804 or other static storage device, coupled to bus 801, for storing static information and instructions for processor 802. A storage device 805, such as a magnetic or optical disk, may be provided and coupled to bus 801 for storing information and instructions. Computer system 800 may be coupled via bus 801 to a display 811, such as a liquid crystal display, for displaying information to a computer user. An input device 812, including alphanumeric and other keys, is coupled to bus 801 for communicating information and command selections to processor 802. Computer system 800 may further be coupled via bus 801 to a cursor control 813, such as a mouse, trackball, or cursor direction keys, for communicating directional information and command selections to processor 802 and for controlling movement of a cursor on display 811.
[0079] Computer system 800 may implement the techniques described herein using customized hardwired logic, one or more ASICs or FPGAs, firmware and / or program logic that couple with the computer system to render or program computer system 800 into a special purpose machine. According to one embodiment, the techniques described herein are performed by computer system 800 in response to processor 802 executing one or more sequences of one or more instructions contained in main memory 803. Such instructions may be read into main memory 803 from another storage medium, such as storage device 805. Execution of the sequences of instructions contained in main memory 803 causes processor 802 to perform the process steps described herein. In alternative embodiments, hardwired circuitry may be used in place of or in combination with software instructions.
[0080] The term "storage medium" as used herein refers to any non-transitory medium that stores data and / or instructions that cause a machine to operate in a specific manner. Such storage media may include non-volatile media and / or volatile media. Non-volatile media include, for example, optical or magnetic disks, such as storage device 805. Volatile media include dynamic memory, such as main memory 803. Common forms of storage media include, for example, floppy disks, flexible disks, hard disks, solid state drives, magnetic tape, or any other magnetic data storage medium, CD-ROMs, any other optical data storage medium, any physical medium with a pattern of holes, RAM, PROM, EPROM, FLASH-EPROM, NVRAM, any other memory chip or cartridge.
[0081] Storage media is distinct from, but may be used in conjunction with, transmission media, which is involved in transferring information between storage media. For example, transmission media include coaxial cables, copper wire and fiber optics, including the wires that comprise bus 801. Transmission media can also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.
[0082] Various types of media may be involved in carrying one or more sequences of one or more instructions to the processor 802 for execution. For example, the instructions may initially be carried on a magnetic disk or solid state drive of a remote computer. The remote computer may load the instructions into its dynamic memory and send the instructions over a communications link, such as fiber optic, coaxial cable, or the like. A communications interface 806 local to the computer system 800 can receive the data on a communications line and appropriate circuitry can place the data on the bus 801. The bus 801 carries the data to the main memory 803, from which the processor 802 reads and executes the instructions. The instructions received by the main memory 803 may optionally be stored on storage device 805 either before or after execution by the processor 802.
[0083] The communication interface 806 provides a two-way data communication coupling to a network link 821 that is connected to a local network 822. For example, the communication interface 806 may be an integrated services digital network (ISDN) card, cable modem, satellite modem, or a modem that provides a data communication connection to a corresponding type of telephone line. As another example, the communication interface 806 may be a local area network (LAN) card to provide a data communication connection to a compatible LAN. Wireless links may also be implemented. In any such implementation, the communication interface 806 sends and receives electrical, electromagnetic or optical signals that carry digital data streams representing various types of information.
[0084] Network link 821 typically provides data communication through one or more networks to other data devices. For example, network link 821 may provide a connection through a local network 822 to a host computer 823 or to data equipment operated by an ISP (Internet Service Provider) 824. ISP 824 also provides data communication services through the world wide packet data communication network now commonly referred to as the "Internet" 825. Local network 822 and Internet 825 use both electrical, electromagnetic or optical signals that carry digital data streams. The signals through the various networks and the signals on network link 821 and through communication interface 806 that carry the digital data to and from computer system 800 are exemplary forms of transmission media.
[0085] Computer system 800 can send messages and receive data, including program code, through the network(s), network link 821 and communication interface 806. In the Internet example, a server 826 might transmit a necessary code for an application program through Internet 825, ISP 824, local network 822 and communication interface 806. The received code may be executed by processor 504 once received by processor 802, and / or stored in storage device 805, or other non-volatile storage for later execution.
[0086] Applications and Effects
[0087] The above-described aspects of the present disclosure may be provided for generating metadata that may be used to provide improved video quality, improved temporal stability, low computational requirements, and reduced latency.
[0088] Systems, methods and devices according to the present disclosure may incorporate one or more of the following features.
[0089] 1. A method for editing video content, comprising: receiving input video data; converting the input video data into a predetermined format, thereby generating converted video data; generating a plurality of initial metadata values for a frame of the transformed video data, the plurality of initial metadata values including: a first metadata value corresponding to a first fixed value not calculated from content comprising the frame, a second metadata value corresponding to an average luminance value of the frame, and a third metadata value corresponding to a second fixed value not calculated from the content; Including, A system and method are provided, wherein the first metadata value, the second metadata value, and the third metadata value comprise information used by a decoder to render a decoded image on a display.
[0090] (2) The method of (1), further comprising the step of applying a spatial smoothing operation to the transformed video data before generating the plurality of initial metadata values.
[0091] (3) The method of (2), wherein the spatial smoothing operation includes subsampling the transformed video data.
[0092] (4) The method according to (2) or (3), wherein applying the spatial smoothing operation includes decimating the transformed video data to a smaller size.
[0093] (5) A method according to any one of (1) to (4), further comprising the step of outputting distribution video data including a portion of video data corresponding to the frame, the first metadata value, the second metadata value, and the third metadata value.
[0094] (6) The method according to any one of (1) to (5), wherein the first fixed value corresponds to a minimum value of allowable luminance of the content, and the second fixed value corresponds to a maximum value of allowable luminance of the content.
[0095] (7) The method according to any one of (1) to (6), wherein the predetermined format is a perceptually quantized high dynamic range format.
[0096] (8) A non-transitory computer-readable medium storing instructions that, when executed by a processor of a computer system, cause the computer system to perform operations including the method described in any one of (1) to (6).
[0097] (9) A system for editing video content, comprising: Memory, A processor; The processor includes: Receives input video data; Converting the input video data into a predetermined format, thereby generating converted video data; generating a plurality of initial metadata values for a frame of the transformed video data, the plurality of initial metadata values including: a first metadata value corresponding to a first fixed value not calculated from content including the frame, a second metadata value corresponding to an average luminance value of the frame, and a third metadata value corresponding to a second fixed value not calculated from the content.
[0098] (10) The processor, in the video editing system, 10. The video editing system of claim 9, further configured to apply a spatial smoothing operation to the transformed video data before generating the plurality of initial metadata values.
[0099] (11) The video editing system of (10), wherein the spatial smoothing operation includes subsampling the transformed video data.
[0100] (12) The video editing system of (10) or (11), wherein the spatial smoothing operation includes decimating the transformed video data to a smaller size.
[0101] (13) The processor, in the video editing system, The video editing system according to any one of (9) to (12), further configured to output distribution video data including a portion of the video data corresponding to the frame, the first metadata value, the second metadata value, and the third metadata value.
[0102] (14) The video editing system according to any one of (9) to (13), wherein the first fixed value corresponds to a minimum value of allowable luminance of the content, and the second fixed value corresponds to a maximum value of allowable luminance of the content.
[0103] (15) The video editing system according to any one of (9) to (14), wherein the predetermined format is a perceptually quantized high dynamic range format.
[0104] (16) A method for editing video content, comprising: receiving input video data; converting the input video data into a predetermined format, thereby generating converted video data; generating a first metadata value for a frame of the transformed video data, the first metadata value corresponding to an average luminance value of the frame; determining a time window that includes the frame, the time window having a predetermined length of time; calculating a maximum metadata range and a minimum metadata range over the time window, thereby generating a first filtered metadata value and a second filtered metadata value; smoothing the filtered metadata values and the second filtered metadata values, thereby generating first smoothed and filtered metadata values and second smoothed and filtered metadata values; The method includes:
[0105] (17) The method according to (16), wherein the predetermined period of time is 4 seconds.
[0106] (18) The method of (16) or (17), wherein the window is configured to not span scene cuts in the input video data.
[0107] (19) The method according to any one of (16) to (18), further comprising the step of applying a spatial smoothing operation to the transformed video data before generating the first metadata value.
[0108] (20) A non-transitory computer-readable medium storing instructions which, when executed by a processor of a computer system, cause the computer system to perform operations including the method according to any one of (16) to (19).
[0109] Although described herein with respect to processes, systems, methods, heuristics, and the like, it should be understood that although steps of such processes, etc. have been described as occurring according to a particular ordered sequence, such processes may be practiced with the described steps being performed in an order different from that described herein. It should be further understood that certain steps may be performed simultaneously, other steps may be added, or certain steps described herein may be omitted. In other words, the description of processes herein is provided for the purpose of describing particular embodiments and should not be considered as limiting the claims.
[0110] Therefore, it should be understood that the above description is intended to be illustrative and not restrictive. Many embodiments and applications other than the examples provided will become apparent upon reading the above description. The scope should be determined without reference to the above description, but instead with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled. It is anticipated and intended that future developments will occur in the technology discussed herein, and that the disclosed systems and methods will be incorporated into such future embodiments. In summary, it should be understood that this application is capable of modification and alteration.
[0111] All terms used in the claims are intended to be given their broadest reasonable construction and ordinary meaning as understood by those skilled in the art to which they pertain. In particular, the use of singular articles such as "a," "the," "said," etc. shall be read to describe one or more of the indicated elements, unless the claim expressly states a limitation to the contrary.
[0112] The Summary of the Disclosure is provided to enable the reader to quickly assess the nature of the technical disclosure. It is understood that it is not used to interpret or limit the scope or meaning of the claims. Moreover, in the foregoing Detailed Description, it will be seen that various features are grouped together in various embodiments for the purpose of streamlining the disclosure. This method of disclosure should not be interpreted as reflecting an intention that the claimed embodiments incorporate more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Thus, the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as separately claimed subject matter.
[0113] Various aspects of the invention may be apparent from the enumerated example embodiments (EEE) described below.
[0114] (EEE1) A method for editing video content, comprising: receiving input video data; converting the input video data into a predetermined format, thereby generating converted video data; generating a plurality of initial metadata values for a frame of the transformed video data, the plurality of initial metadata values including: a first metadata value corresponding to a first fixed value not calculated from content comprising the frame, a second metadata value corresponding to an average luminance value of the frame, and a third metadata value corresponding to a second fixed value not calculated from the content; Including, A system and method are provided, wherein the first metadata value, the second metadata value, and the third metadata value comprise information used by a decoder to render a decoded image on a display.
[0115] (EEE2) The method of (EEE1), further comprising the step of applying a spatial smoothing operation to the transformed video data prior to generating the plurality of initial metadata values.
[0116] (EEE3) The method of EEE2, wherein the spatial smoothing operation includes subsampling the transformed video data.
[0117] (EEE4) A method according to EEE2 or 3, wherein applying the spatial smoothing operation includes decimating the transformed video data to a smaller size.
[0118] (EEE5) A method according to any one of EEE1 to 4, further comprising a step of outputting distribution video data including a portion of video data corresponding to the frame, the first metadata value, the second metadata value, and the third metadata value.
[0119] (EEE6) The method according to any one of EEE1 to EEE5, wherein the first fixed value corresponds to a minimum value of allowable luminance of the content, and the second fixed value corresponds to a maximum value of allowable luminance of the content.
[0120] (EEE7) The method according to any one of EEE1 to 6, wherein the predetermined format is a perceptually quantized high dynamic range format.
[0121] (EEE8) A non-transitory computer-readable storage medium storing instructions which, when executed by a processor of a computer system, cause the computer system to perform operations including a method described in any one of EEE1-7.
[0122] (EEE9) A system for editing video content, comprising: Memory, A processor; The processor includes: Receives input video data; Converting the input video data into a predetermined format, thereby generating converted video data; generating a plurality of initial metadata values for a frame of the transformed video data, the plurality of initial metadata values including: a first metadata value corresponding to a first fixed value not calculated from content including the frame, a second metadata value corresponding to an average luminance value of the frame, and a third metadata value corresponding to a second fixed value not calculated from the content.
[0123] (EEE10) The processor may include a controller for controlling the video editing system. 9. The video editing system of claim 8, further comprising: a processor configured to generate the plurality of initial metadata values from the transformed video data;
[0124] (EEE11) The video editing system of EEE10, wherein the spatial smoothing operation includes subsampling the transformed video data.
[0125] (EEE12) The video editing system of EEE10 or 11, wherein the spatial smoothing operation includes decimating the transformed video data to a smaller size.
[0126] (EEE13) The processor may include a controller for controlling the video editing system. The video editing system of any one of claims 8 to 12, further configured to output distribution video data including a portion of the video data corresponding to the frame, the first metadata value, the second metadata value, and the third metadata value.
[0127] (EEE14) The video editing system according to any one of EEE9 to 13, wherein the first fixed value corresponds to a minimum allowable luminance value of the content, and the second fixed value corresponds to a maximum allowable luminance value of the content.
[0128] (EEE15) The video editing system according to any one of EEE9 to 14, wherein the predetermined format is a perceptually quantized high dynamic range format.
Claims
1. 1. A method for editing video content, comprising: receiving input video data; converting the input video data into a predetermined format that differs in dynamic range from a format of the input video data, thereby generating converted video data; generating a plurality of initial metadata values for a frame of the transformed video data, the plurality of initial metadata values including: a first metadata value corresponding to a first fixed value not calculated from content comprising the frame, the first fixed value corresponding to a minimum allowed luminance value of the content; a second metadata value corresponding to an average luminance value of the frame; and a third metadata value corresponding to a second fixed value not calculated from the content, the second fixed value corresponding to a maximum allowed luminance value of the content, the first metadata value, the second metadata value, and the third metadata value comprising information used by a decoder to render a decoded image on a display; determining a time window that includes the frame, the time window having a predetermined length of time; calculating a maximum metadata range and a minimum metadata range over the time window, thereby generating a first filtered metadata value and a second filtered metadata value; smoothing the first filtered metadata value and the second filtered metadata value, thereby generating first smoothed and filtered metadata values and second smoothed and filtered metadata values; The method includes:
2. 2. The method of claim 1, wherein the predetermined format is a standard dynamic range format and the input video data is a high dynamic range format.
3. The method of claim 1 or 2, further comprising the step of applying a spatial smoothing operation to the transformed video data before generating the plurality of initial metadata values.
4. The method of claim 3 , wherein applying the spatial smoothing operation comprises subsampling the transformed video data.
5. The method of claim 3 or 4, wherein applying the spatial smoothing operation comprises decimating the transformed video data to a smaller size.
6. 6. The method of claim 1, further comprising the step of outputting distributed video data comprising a portion of video data corresponding to the frame, the first metadata value, the second metadata value, and the third metadata value.
7. The method of claim 1 , wherein the predetermined format is a perceptually quantized high dynamic range format.
8. The method of any one of claims 1 to 7, wherein the predetermined length of time is 4 seconds.
9. A method according to any preceding claim, wherein the time windows are arranged not to span scene cuts in the input video data.
10. The method according to any one of claims 1 to 9, further comprising the step of applying a spatial smoothing operation to the transformed video data before generating the first metadata value.
11. A non-transitory computer readable medium storing instructions which, when executed by a processor of a computer system, cause the computer system to perform operations including the method of any one of claims 1 to 10.
12. 1. A video editing system comprising: Memory, A processor, comprising: Receive the input video data: converting the input video data into a predetermined format that differs in dynamic range from a format of the input video data, thereby generating converted video data; generating a plurality of initial metadata values for a frame of the transformed video data, the plurality of initial metadata values comprising: a first metadata value corresponding to a first fixed value not calculated from content comprising the frame, the first fixed value corresponding to a minimum allowable luminance value of the content; a second metadata value corresponding to an average luminance value of the frame; and a third metadata value corresponding to a second fixed value not calculated from the content, the second fixed value corresponding to a maximum allowable luminance value of the content; determining a time window that includes the frame, the time window having a predetermined length of time; calculating a maximum metadata range and a minimum metadata range over the time window, thereby generating a first filtered metadata value and a second filtered metadata value; a processor for smoothing the first filtered metadata value and the second filtered metadata value, thereby generating first smoothed and filtered metadata values and second smoothed and filtered metadata values; Video editing system including.
13. 13. The video editing system of claim 12, wherein the predetermined format is a standard dynamic range format and the input video data is a high dynamic range format.
14. The processor, in the video editing system, 14. A video editing system according to claim 12 or 13, arranged to apply a spatial smoothing operation to the transformed video data before generating the plurality of initial metadata values.
15. The video editing system of claim 14 , wherein the spatial smoothing operation includes subsampling the transformed video data.
16. 16. The video editing system of claim 14 or 15, wherein the spatial smoothing operation comprises decimating the transformed video data to a smaller size.
17. The processor, in the video editing system, 17. The video editing system of claim 12, further configured to output delivery video data comprising a portion of video data corresponding to the frame, the first metadata value, the second metadata value, and the third metadata value.
18. 13. The video editing system of claim 12, wherein the predetermined format is a perceptually quantized high dynamic range format.
Citation Information
Patent Citations
Display management for high dynamic range video
JP2016514395A
Workflow for content generation and assisted display management of edr video
JP2016538736A
Source color volume information messaging
JP2019537320A
Video signal processing method and apparatus
JP2021529339A
Adaptive Display Management Using 3D Look-Up Table Interpolation
US20170124983A1
