Video content editing method, system, medium and computer program product

Abstract

A video content editing method, system, medium and computer program product are provided. The system and method for editing video content include: receiving input video data; converting the input video data into a predetermined format; generating a plurality of initial metadata values ​​for a frame of the converted video data, the plurality of initial metadata values ​​including: a first metadata value, the first metadata value corresponding to a first fixed value that is not calculated based on the content including the frame; a second metadata value, the second metadata value corresponding to an average brightness value of the frame; and a third metadata value, the third metadata value corresponding to a second fixed value that is not calculated based on the content, wherein the first metadata value, the second metadata value and the third metadata value include information used by a decoder to present a decoded image on a display.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to European Patent Application No. 20178044.2 and U.S. Provisional Application No. 63 / 034,006, both filed on June 3, 2020, each of which is incorporated herein by reference in its entirety. Technical Field

[0003] The present application relates generally to images; more particularly, the present application relates to generating metadata for display management in conjunction with video content. Background Art

[0004] As used herein, the term "dynamic range (DR)" may relate to the ability of the human visual system (HVS) to perceive a range of intensities (e.g., brightness, luminance) in an image, e.g., from the darkest black (dark colors) to the brightest white (highlights). In this sense, DR is related to "scene-referred" intensities. DR may also relate to the ability of a display device to adequately or approximately render a range of intensities of a particular breadth. In this sense, DR is related to "display-referred" intensities. Unless a particular sense is explicitly specified at any point in the description herein to have a particular meaning, it should be inferred that the term may be used in either sense, e.g., interchangeably.

[0005] As used herein, the term "high dynamic range (HDR)" refers to a DR width of approximately 14 to 15 or more orders of magnitude across the HVS. In practice, the DR over a wide range of intensities that humans can simultaneously perceive may be slightly truncated relative to HDR. As used herein, the terms "enhanced dynamic range" (EDR) or "visual dynamic range" (VDR) may be individually or interchangeably related to such a DR: the DR that can be perceived by the HVS including eye movement within a scene or image, allowing for some light-adaptive changes in the scene or image. As used herein, EDR may refer to a DR spanning 5 to 6 orders of magnitude. Therefore, while HDR may be slightly narrower relative to a real scene reference, EDR may represent a wide DR width and may also be referred to as HDR.

[0006] In practice, an image includes one or more color components (e.g., RGB, luma Y, and chroma Cb and Cr), where each color component is represented by n bits of precision per pixel (e.g., n=8) in a quantized digital system. Bit depths of n≤8 (e.g., color 24-bit JPEG images) can be used for standard dynamic range (SDR) images, while bit depths of n>8 can be considered for enhanced dynamic range (EDR) images to avoid contour artifacts and staircase artifacts. In addition to integer data types, high-precision (e.g., 16-bit) floating-point formats such as the OpenEXR file format developed by Industrial Light and Magic can also be used to store and distribute EDR and high dynamic range (HDR) images.

[0007] The reference electro-optical transfer function (EOTF) of a given display characterizes the relationship between the color values ​​(e.g., luminance) of an input video signal and the output screen color values ​​(e.g., screen luminance) produced by the display. For example, ITU Rec. ITU-R BT. 1886, "Reference electro-optical transfer function for flat panel displays used in HDTV studio production" (March 2011), defines a reference EOTF for flat panel displays based on measured characteristics of cathode ray tubes (CRTs), the contents of which are incorporated herein by reference in their entirety. Given a video stream, information about its EOTF is typically embedded in the bitstream as metadata.

[0008] Many consumer desktop monitors have a 200 cd / m 2 Up to 300cd / m 2("nits") for rendering non-HDR content, while consumer high-definition and ultra-high-definition televisions ("HDTV" and "UHD TV") have a maximum brightness of 300 nits to 500 nits. Therefore, this display output represents the low dynamic range (LDR) associated with HDR or EDR, also known as SDR. As the availability of HDR or EDR content increases due to the development of both capture devices (e.g., cameras) and HDR displays (e.g., Dolby Laboratories' PRM-4200 professional reference monitor or Sony's Trimaster HX 31" 4K HDR mastering monitor), HDR content can be color graded and displayed on HDR displays that support a higher dynamic range (e.g., from 700 nits to 5000 nits or higher). Such displays may be defined using an alternative EOTF that supports high brightness capabilities (e.g., 0 nits to 10,000 nits). An example of such an EOTF is defined in SMPTE ST 2084:2014 "High Dynamic Range EOTF of Mastering Reference Displays", the contents of which are incorporated herein by reference in their entirety.

[0009] As used herein, the term "display management" refers to the processing (e.g., tone and gamut mapping) required to map an input video signal of a first dynamic range (e.g., 1000 nits) to a display of a second dynamic range (e.g., 500 nits). Examples of display management processes are described in PCT Application Serial No. PCT / US2014 / 016304, filed February 13, 2014 by R. Atkins et al. (the '304 Application) ("Display management for high dynamic range video"), which is incorporated herein by reference in its entirety. Display management may be assisted or controlled by input metadata that is generated by the source of the incoming content and multiplexed into the encoded bitstream.

[0010] The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any approach described in this section qualifies as prior art simply by virtue of its inclusion in this section. Similarly, unless otherwise indicated, issues identified with respect to one or more approaches should not be assumed to qualify as prior art based on this section. Summary of the invention

[0011] Various aspects of the present disclosure relate to devices, systems, and methods for generating metadata that can be used to provide improved video quality, improved temporal stability, low computational requirements, and reduced latency.

[0012] In one aspect of the present disclosure, a method for editing video content is provided, the method comprising: receiving input video data; converting the input video data into a predetermined format, thereby generating converted video data; generating a plurality of initial metadata values ​​for frames of the converted video data, the plurality of initial metadata values ​​comprising: a first metadata value, the first metadata value corresponding to a first fixed value that is not calculated based on content including the frame; a second metadata value, the second metadata value corresponding to an average brightness value of the frame; and a third metadata value, the third metadata value corresponding to a second fixed value that is not calculated based on the content, wherein the first metadata value, the second metadata value, and the third metadata value include information used by a decoder to present a decoded image on a display.

[0013] In another aspect of the present disclosure, a video editing system is provided, the video editing system comprising: a memory; and a processor, the processor being configured to cause the video editing system to perform the following operations: receiving input video data; converting the input video data into a predetermined format, thereby generating converted video data; generating a plurality of initial metadata values ​​for frames of the converted video data, the plurality of initial metadata values ​​comprising: a first metadata value, the first metadata value corresponding to a first fixed value that is not calculated based on content including the frame; a second metadata value, the second metadata value corresponding to an average brightness value of the frame; and a third metadata value, the third metadata value corresponding to a second fixed value that is not calculated based on the content.

[0014] In another aspect of the present disclosure, a method for editing video content is provided, the method comprising: receiving input video data; converting the input video data into a predetermined format, thereby generating converted video data; generating a first metadata value for a frame of the converted video data, the first metadata value corresponding to an average brightness value of the frame; determining a time window including the frame, the time window having a predetermined time length; calculating a maximum metadata range and a minimum metadata range over the time window, thereby generating a first filtered metadata value and a second filtered metadata value; and smoothing the first filtered metadata value and the second filtered metadata value, thereby generating a first smoothed and filtered metadata value and a second smoothed and filtered metadata value.

[0015] In this manner, various aspects of the present disclosure achieve improvements in at least the technical areas of video capture, editing, and signal processing, and related areas of video and other content delivery. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] These and other more detailed and specific features of various embodiments are more fully disclosed in the following description with reference to the accompanying drawings, in which:

[0017] Figure 1 illustrates an exemplary video delivery pipeline according to various aspects of the present disclosure;

[0018] Figure 2 illustrates a process flow of an exemplary metadata generation method according to various aspects of the present disclosure;

[0019] Figure 3 The diagram shows Figure 2 Example metadata generated by the method;

[0020] Figure 4 illustrates a process flow of another exemplary metadata generation method according to various aspects of the present disclosure;

[0021] Figure 5 The diagram shows Figure 2 Example metadata generated by the method;

[0022] Figure 6 illustrates a process flow of another exemplary metadata generation method according to various aspects of the present disclosure;

[0023] FIG. 7A to FIG. 7C The diagrams are respectively shown Figure 6 Example metadata generated by the method; and

[0024] Figure 8 Exemplary systems according to various aspects of the present disclosure are illustrated. DETAILED DESCRIPTION

[0025] The present disclosure and various aspects may be implemented in various forms, including: hardware, devices or circuits controlled by computer-implemented methods, computer program products, computer systems and networks, user interfaces and application programming interfaces; as well as hardware-implemented methods, signal processing circuits, memory arrays, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), etc. The foregoing summary is intended only to give a general idea of ​​the various aspects of the present disclosure and is not intended to limit the scope of the present disclosure in any way.

[0026] In the following description, many details such as optical device configuration, timing, operation, etc. are set forth to provide an understanding of one or more aspects of the present disclosure. It is obvious to those skilled in the art that these specific details are only exemplary and are not intended to limit the scope of the present application.

[0027] In addition, although the present disclosure focuses primarily on examples of using various circuits in content generation, it should be understood that this is merely one example of an implementation. It should be further understood that the disclosed systems and methods can be used in any device that needs to generate, modify, or display image data and / or related metadata; for example, theaters, consumer and other commercial projection systems, smartphones and other consumer electronic devices, head-up displays, virtual reality displays, etc.

[0028] Metadata

[0029] As used herein, the term "metadata" refers to any auxiliary information transmitted as part of the encoded bitstream and assisting a decoder in rendering a decoded image. Such metadata may include, but is not limited to, color space or gamut information, reference display parameters, and auxiliary signal parameters as described herein, and may be collectively referred to as "image formation metadata."

[0030] Such metadata may be divided or classified into different levels of metadata. In a specific example in which the present disclosure is implemented in the Dolby Vision architecture, mapping metadata may include L0 metadata, L1 metadata, L2 / L8 metadata, L3 parameters, L4 metadata, L11 metadata, and the like. In some examples, L0 metadata provides or describes information about the dynamic range of video data across the entire video content (i.e., across multiple shots and / or scenes). For example, L0 metadata may include L0min and L0max, which represent the minimum brightness and maximum brightness of the video content, respectively. In a specific example in which the video content is HDR content, L0min and L0max may correspond to 0.01 nits and 1000 nits, respectively.

[0031] In some examples, L1 metadata provides or describes information about the distribution of luminance values ​​in a source image, source scene, source group of pictures (GOP), etc. represented in the video data. The luminance value distribution may be represented by a histogram derived based on the image content (e.g., pixel values, luminance values, chrominance values, Y values, Cb / Cr values, RGB values, etc.) of the image, scene, GOP, etc. The L1 metadata may include parameters L1min, L1mid, and L1max, which represent the minimum luminance (“crush”), mid-tone luminance (“middle”), and maximum luminance (“clip”), respectively, of one or more images, scenes, GOPs, etc. in the video content.

[0032] Other levels of metadata may also be defined. For example, L2 provides or describes information about adjustments to video characteristics that originate from or trace back to adjustments made by supervisors, color graders, video professionals, etc. in a production studio and / or during post-production. L8 metadata is similar to L2 metadata, and in some cases may be equivalent to L2 metadata (e.g., depending on the corresponding tone curves). L2 and L8 metadata may be referred to as "trimming" parameters, and may indicate or relate to the gain / offset / power of the image data. The L2 metadata may correspond to a first reference display having a first reference dynamic range.

[0033] The L3 metadata provides or describes information about adjustments to video characteristics that originate from or trace back to adjustments made by a supervisor, color grader, video professional, etc. Compared to the L2 metadata, the L3 metadata may correspond to a second reference display having a reference dynamic range that is different from the reference dynamic range of the first reference display. The L3 metadata may include, for example, offsets or adjustments relative to the L1 metadata, including offsets or adjustments to squeeze, midrange, and / or clip luma values.

[0034] L4 metadata provides or describes information about global dimming operations. L4 metadata can be calculated by the encoder during preprocessing and can be calculated using RGB primaries. In one example, L4 metadata may include data indicating the global backlight brightness level of the display panel on a frame-by-frame basis. Other generated metadata, such as L11 metadata, may provide or describe information to be used to identify the source of the video data, such as movie content, computer game content, sports content, etc. Such metadata may further provide or describe expected picture settings, such as expected white point, sharpness, etc.

[0035] In summary, the image formation metadata may include conversion data for converting from a first dynamic range to a second dynamic range different from the first dynamic range. In some aspects of the present disclosure, the first dynamic range may be higher than the second dynamic range (e.g., conversion from HDR to SDR). In other aspects of the present disclosure, the second dynamic range may be higher than the first dynamic range (e.g., conversion from SDR to HDR).

[0036] Comparative methods for generating image formation metadata can include calculating per-frame metadata (e.g., per-frame L1 metadata) and applying IIR filtering (e.g., for live broadcasts or games) or combining metadata into scenes by calculating the minimum, median, and maximum values ​​of the scene (e.g., for studio or professional offline content). However, none of these methods allow for instant preview of content during editing. For example, a user may want to make an HDR picture brighter and preview the result on an SDR monitor as a reference display. To do this, the following steps are performed: apply adjustments to the HDR picture; recalculate L1 metadata for the adjusted HDR picture; modify the tone mapping operation; and apply the adjusted tone mapping to the adjusted HDR picture to produce an SDR preview picture.

[0037] This set of operations may be satisfactory if applied to a single image, since a single image can be reasonably stored in memory, so the operations can be applied in rapid succession. However, the L1 metadata (and the resulting tone curve) may vary significantly between frames, causing the mapped image to flicker. This may be referred to as "temporal instability". To ensure temporal stability of the tone curve, some kind of filtering may be performed. However, if the metadata recalculation operations depend on multiple images, then the adjustment operations and metadata recalculation operations must be applied to many images, which may be impractical or impossible at fast video rates.

[0038] Therefore, comparison methods that generate image formation metadata may result in slower iteration cycles of adjusting, updating, and previewing results. In addition, IIR filtered comparison methods may need to analyze a very large number of frames before the current frame, because the current frame requires all previous frames of the IIR filter. Studio or professional comparison methods require all frames of a scene, which in some cases may be minutes or even hours long.

[0039] In view of this, various aspects of the present disclosure provide methods for generating metadata for efficient editing (i.e., editing that produces high image quality and high temporal stability). Various aspects of the present disclosure can accomplish such editing with low computational requirements and low or no latency. In some aspects, the methods for generating metadata are incorporated into or used with a video delivery pipeline.

[0040] Video delivery pipeline

[0041] Figure 1An exemplary video delivery pipeline is illustrated, and the various stages from video capture to video content display are shown. In addition, although the following description is provided with respect to video (i.e., moving images), the present disclosure is not limited thereto. In some examples, the image content may be a still image or a combination of video and still images. The image content may be represented by raster (or pixel) graphics, vector graphics, or a combination of raster graphics and vector graphics. Figure 1 An image generation block 101 , a production block 102 , a post-production block 103 , an encoding block 104 , a decoding block 105 , and a display management block 106 are illustrated. Figure 1 The various blocks illustrated in the figure may be implemented as hardware, software, firmware, or a combination thereof or implemented via hardware, software, firmware, or a combination thereof. In addition, the various groups of the illustrated blocks may combine their respective functions, and / or may be executed in different devices and / or at different times. The various blocks or groups of the illustrated blocks may be implemented via circuits including, but not limited to, a central processing unit (CPU), a graphics processing unit (GPU), an ASIC, an FPGA, and a combination thereof. The operations performed by one or more of these blocks may be locally processed, remotely processed (e.g., cloud-based), or a combination of local and remote processing.

[0042] like Figure 1 , the video delivery pipeline further includes a reference display 111 and a target display 112, which may be provided to assist or monitor operations performed at the post-production block 103. For purposes of explanation, the image generation block 101, the production block 102, the post-production block 103, and the encoding block 104 may be referred to as “upstream” blocks or components, while the decoding block 105 and the display management block 106 may be referred to as “downstream” blocks or components.

[0043] exist Figure 1In the illustrated example, video content in the form of a sequence of video frames 121 is captured or generated at the image generation block 101. The video frames 121 may be captured digitally (e.g., by a digital camera) or generated by a computer (e.g., using computer animation) to generate video data 122. Alternatively, the video frames 121 may be captured on film by a film camera, and then converted to a digital format to provide video data 122. In either case, the video data 122 is provided to the production block 102, where the video data is edited to provide a production stream 123. The video frames 121 may be grouped into sequences, such as shots and scenes. For example, a shot is a group of frames connected in time. Shots may be separated by "shot clips" (e.g., points in time at which the entire content of an image changes, rather than only a portion of the content of the image). For example, a scene is a series of shots that describe a story segment of a larger content. In a specific example where the video content is an action movie, the video content may include (among other things) a chase scene, which in turn includes a series of shots (e.g., shots of the driver of the pursuing vehicle, shots of the driver of the vehicle being pursued, shots of the street where the chase occurs, etc.).

[0044] The video data in the video production stream 123 is then provided to one or more processors at the post-production block 103 for post-production editing. The editing performed at the post-production block 103 may include adjusting or modifying the color or brightness in a specific area of ​​the image to enhance the image quality or achieve a specific appearance of the image according to the creative intent of the video creator (or editor). This may be referred to as "color timing" or "color grading". Other editing (e.g., scene selection and sequencing, image cropping, adding computer-generated visual effects or overlays, etc.) may be performed at the post-production block 103 to produce a distribution stream 124. In some examples, the post-production block 103 may provide the intermediate stream 125 to a reference display 111 to allow the image to be viewed on the screen of the reference display, for example, to assist the editing process. One, two, or all of the production block 102, the post-production block 103, and the encoding block 104 may further include processing to add metadata to the video data. This further processing may include, but is not limited to, statistical analysis of content properties. This further processing may be performed locally or remotely (e.g., cloud-based processing).

[0045] After post-production operations, the distribution stream 124 may be transmitted to the encoding block 104 for downstream transmission to decoding and playback devices such as televisions, set-top boxes, movie theaters, laptops, tablet computers, etc. In some examples, the encoding block 104 may include an audio encoder and a video encoder as defined by the Advanced Television Systems Committee (ATSC), Digital Video Broadcasting (DVB), Digital Versatile Disc (DVD), Blu-ray, and other transmission formats to generate an encoded bitstream 126. In the receiver, the encoded bitstream 126 is decoded by the decoding unit 105 to generate a decoded signal 127 representing the same or closely similar content of the distribution stream 124. The receiver may be attached to a target display 112, which may have different characteristics than the reference display 111. In the case where the reference display 111 and the target display 112 have different characteristics, the display management block 106 may be used to map the dynamic range or other characteristics of the decoded signal 127 to the characteristics of the target display 112 by generating a display mapped signal 128. Additionally or alternatively, the display management block 106 may be used to provide power management of the target display 112 .

[0046] The target display 112 uses an array of pixels to generate an image. The specific array structure depends on the architecture and resolution of the display. For example, if the target display 112 operates on an LCD architecture, the target display may include a relatively low-resolution backlight array (e.g., an array of LEDs or other light-emitting elements) and a relatively high-resolution liquid crystal array and color filter array to selectively attenuate white light from the backlight array and provide colored light (commonly referred to as dual-modulation display technology). If the target display 112 operates on an OLED architecture, the target display may include a high-resolution self-luminous color pixel array.

[0047] The link between the upstream block and the downstream block (i.e., the path that provides the encoded bit stream 126) can be embodied by a live or real-time transmission such as over-the-air broadcasting using electromagnetic waves, or via a content delivery line such as optical fiber, twisted pair (Ethernet), and / or coaxial cable. In other examples, the link can be embodied by a time-independent transmission, such as recording the encoded bit stream onto a physical medium (e.g., a DVD or hard disk) for physical delivery to an end-user device (e.g., a DVD player). The decoder block 105 and the display management block 106 can be incorporated into a device associated with the target display 112; for example, in the form of a smart TV that includes decoding, display management, power management, and display functions. In some examples, the decoder block 105 and / or the display management block 106 can be incorporated into a device separate from the target display 112; for example, in the form of a set-top box or a media player.

[0048] Decoder block 105 and / or display management block 106 may be configured to receive, analyze, and act in response to metadata included or added at an upstream block. Thus, such metadata may be used to provide additional control or management of target display 112. Metadata may include the image formation metadata described above (e.g., Dolby Vision metadata). Metadata may be generated at post-production block 103 using reference display 111 and production stream 123 and / or intermediate stream 125.

[0049] Semi-dynamic metadata generation

[0050] Image formation metadata can be generated using a "semi-dynamic" approach, Figure 2 An example of this method is illustrated in FIG. Figure 2 The illustrated operation can be performed by using Figure 1 The post-production block 103 is automatically executed by hardware, software, firmware or a combination thereof associated with the post-production block 103.

[0051] like Figure 2 As illustrated, the semi-dynamic metadata generation process may begin at operation 201, where a video signal is converted into a predetermined format (e.g., a particular EOTF). For example, the video signal may be converted from hybrid log-gamma (HLG) or SLOG-3 HDR to RGB perceptual quantizer (PQ) HDR, for example, as described in Rec. ITU-R BT.2100-1 (06 / 2017). In some embodiments, at operation 202, spatial smoothing and / or subsampling may be applied. Operation 202 may be implemented using a 2D smoothing kernel (e.g., [1 1] / 2 or [1 3 3 1] / 8). This reduces the sensitivity to individual pixels. In some examples, after filtering, the image may be reduced to a smaller size (e.g., half the size). In other embodiments, operation 202 may be omitted, and the semi-dynamic metadata generation process may proceed directly from operation 201 to operation 203. At operation 203, an initial value of L1mid is calculated (e.g., frame by frame). For example, L1mid may represent the average of the maximum RGB luminance. In other words, L1mid may represent the average of the maximum values ​​of the color component values ​​R, G, B for each pixel. The values ​​of L1min and L1max are not calculated based on the content, but fixed values ​​corresponding to the minimum and maximum allowable ranges of the image content are used at operation 204, as represented in the L0 metadata (e.g., Level0.Smin and Level0.Smax). For example, when using YCbCr representation, L1mid may represent the average brightness value of the Y component value.

[0052] As Figure 2 A specific example of the operation of , presented in pseudocode using a MATLAB-like format:

[0053]

[0054] Figure 3 310 illustrates an exemplary output of the above operations for metadata corresponding to squeeze, median, and clip values. Each graph illustrates normalized luminance as a function of frame number. Line 310 min ,310 mid and 310 max The minimum, average and maximum luminance values ​​for a given frame are illustrated respectively. Line 320 min ,320 mid and 320 max The values ​​L1min, L1mid and L1max for a given frame are illustrated.

[0055] Because L1min and L1max are set to the values ​​of L0min and L0max, respectively, the squeeze and clip metadata values ​​do not strictly match the actual luminance values ​​for all frames. Figure 3 In the particular example shown, the minimum per-frame brightness value (line 310 min ) varies between about 0.01 and 0.48, while the value L1min (line 320 min ) is set to approximately 0. The maximum per-frame brightness value (line 310 max ) varies between about 0.53 and 0.82, while the value L1max (line 320 max ) is set to approximately 0.82. However, because L1mid is determined by dynamic values, the intermediate metadata values ​​more closely match the actual brightness values ​​for all frames. Figure 3 As shown, the average brightness value per frame (line 310 mid ) and value L1mid (line 320 mid ) has high correspondence.

[0056] Semi-dynamic generation of metadata requires only a single frame to be processed. This approach is highly efficient; however, Figure 3 As can be seen in Figure 2, for images that do not use the full dynamic range of the container, using static values ​​for the squeezing and clipping metadata values ​​results in a slightly lower mapped image contrast.

[0057] Dynamic metadata generation

[0058] Image formation metadata can be generated using a "dynamic" method, Figure 4 An example of this method is illustrated in FIG. Figure 4 The illustrated operation can be performed by using Figure 1 The post-production block 103 is automatically executed by hardware, software, firmware or a combination thereof associated with the post-production block 103.

[0059] like Figure 4As illustrated, the dynamic metadata generation process may begin at operation 401, where a video signal is converted into a predetermined format (e.g., a particular EOTF). For example, the video signal may be converted from HLG or SLOG-3 HDR to RGB PQ HDR, for example, as described in Rec. ITU-R BT.2100-1 (06 / 2017). In some embodiments, at operation 402, spatial smoothing and / or subsampling may be applied. Operation 402 may be implemented using a 2D smoothing kernel (e.g., [1 1] / 2 or [1 3 31] / 8). This reduces the sensitivity to individual pixels. In some examples, after filtering, the image may be reduced to a smaller size (e.g., half the size). In other embodiments, operation 402 may be omitted, and the semi-dynamic metadata generation process may proceed directly from operation 401 to operation 403. At operation 403, initial values ​​of L1min, L1mid, and L1max are calculated (e.g., frame by frame). For example, L1min may represent the minimum value of RGB brightness, L1mid may represent the average value of the maximum RGB brightness, and L1max may represent the maximum value of the RGB brightness.

[0060] Subsequently, at operation 404, the values ​​of L1min and L1max are recalculated using the dynamic values ​​for the given shot or scene. For example, each frame of the shot or scene may be analyzed to determine the minimum and maximum RGB luminances across the given shot or scene. At operation 405, the values ​​of L1min and L1max are recalculated using values ​​corresponding to the minimum and maximum values ​​of the clipping range determined in operation 404. The dynamic method may correspond more closely to the image data than the semi-dynamic method described above because L1min and L1max are determined with respect to the actual minimum and maximum RGB luminances of the content, rather than with respect to the allowed (i.e., possible) minimum and maximum RGB luminances of the content.

[0061] As Figure 4 A specific example of the operation of , presented in pseudocode using a MATLAB-like format:

[0062]

[0063]

[0064] Figure 5 510 illustrates an exemplary output of the above operations with metadata corresponding to squeeze, median, and clip values. Each graph illustrates normalized luminance as a function of frame number. Line 510 min , 510 mid and 510 max The minimum, average, and maximum luminance values ​​for a given frame are illustrated respectively. Line 520min ,520 mid and 520 max The values ​​L1min, L1mid and L1max for a given frame are illustrated.

[0065] Because L1min and L1max are set to the minimum values ​​across all frames of the clip, the squeeze and clip metadata values ​​do not strictly match the actual luminance values ​​for all frames. However, compared to the semi-dynamic method described above, the squeeze and clip metadata values ​​are more closely matched to the actual luminance values. Figure 5 In the particular example shown, the minimum per-frame brightness value (line 510 min ) varies between about 0.02 and 0.48, while the value L1min (line 510 min ) is set to about 0.02. In contrast, Figure 3 In the illustrated semi-dynamic method, the value L1min (line 310 min ) is set to 0. The maximum brightness value per frame (line 510 max ) varies between about 0.53 and 0.82, while the value L1max (line 520 max ) is set to about 0.82. As above, because L1mid is determined by dynamic values, the intermediate metadata values ​​are more closely matched to the actual brightness values ​​of all frames. Figure 5 As shown, the average brightness value per frame (line 510 mid ) and value L1mid (line 520 mid ) has high correspondence.

[0066] Dynamic generation of metadata requires processing all frames in a given shot or scene, and thus may result in higher computational requirements than semi-dynamic generation of metadata. Figure 5 As can be seen in Figure 3, using dynamic values ​​of the squeeze and clip metadata values ​​does not suffer the same type of mapped image contrast reduction as for images that do not use the full dynamic range of the container.

[0067] Windowed FIR metadata generation

[0068] Image formation metadata can be generated using a "windowing" approach, Figure 6 An example of this method is illustrated in FIG. Figure 6 The illustrated operation can be performed by using Figure 1 The post-production block 103 is automatically executed by hardware, software, firmware or a combination thereof associated with the post-production block 103.

[0069] like Figure 6As illustrated, the windowed metadata generation process may begin at operation 601, where the av video signal is converted into a predetermined format (e.g., a specific EOTF). For example, the video signal may be converted from HLG or SLOG-3HDR to RGB PQHDR, for example, as described in Rec.ITU-R BT.2100-1 (06 / 2017). Thereafter, at operation 602, spatial smoothing and / or subsampling is applied. Operation 402 may be implemented using a 2D smoothing kernel (e.g., [1 1] / 2 or [1 3 3 1] / 8). This reduces the sensitivity to a single pixel. In some examples, after filtering, the image may be reduced to a smaller size (e.g., half the size). At operation 603, initial values ​​of L1min, L1mid, and L1max are calculated (e.g., pixel by pixel). For example, L1min may represent the minimum value of the minimum RGB brightness, L1mid may represent the average value of the maximum RGB brightness, and L1max may represent the maximum value of the maximum RGB brightness.

[0070] Subsequently, at operation 604, a list of frames in the moving window (k) is determined. This can be specified as a maximum time length T multiplied by the frame rate. The time window T (also referred to as a "time" window) can be configured by the operator. In some examples, the time window T can be configured between 0.4 seconds and 8 seconds. In a specific example, the time window T is 4 seconds. As will be described in more detail below, a specific value of the time window T is selected to balance temporal stability and picture quality; for example, a shorter window may introduce a certain degree of flicker, while a longer window may produce a more static process. However, the window itself is configured so that it does not cross scene cuts, and is therefore defined by the first and last frames of the current scene.

[0071] Operation 605 is the first pass of calculating the minimum and maximum values ​​of the window. For example, for each frame f in the moving window k, operation 605 can use a 1D smoothing kernel such as [1 1] / 2, [1 2 1] / 4, [1 3 3 1] / 8, or [1 3 8 3 1] / 16 to smooth the metadata, thereby obtaining smoothed metadata L1minS(f) and L1maxS(f). This reduces the sensitivity to a single frame. Thereafter, operation 605 can determine the maximum metadata range over the moving window k; for example, by calculating L1minM(f)=min(L1minS(k)) (i.e., the minimum metadata value for minimum smoothing) and L1maxM(f)=max(L1max(S(k)) (i.e., the maximum metadata value for maximum smoothing).

[0072] Operation 606 is a second stage of smoothing the minimum and maximum values ​​of the window. For example, operation 606 may calculate a smoothing filter F as a Gaussian shape with L taps and a standard deviation s=L / 6, and then for each frame f in the moving window k, the filtered metadata calculated in operation 605 may be smoothed with the smoothing filter F to obtain smoothed and filtered metadata values ​​L1minF and L1maxF. Operation 606 may also set L1min and L1max to the smoothed and filtered metadata values ​​for each frame f, and set L1mid to its initial value determined in operation 603.

[0073] Because operations 605 and 606 are applied to a moving window of frames, operations 604, 605, and 606 may be repeated so that all possible windows within a given scene are processed.

[0074] As Figure 6 A specific example of the operation of , presented in pseudocode using a MATLAB-like format:

[0075]

[0076]

[0077] FIG. 7A to FIG. 7C An exemplary output of the above operation of metadata corresponding to squeeze, median, and clip values ​​for different window lengths is illustrated. Each figure illustrates normalized brightness as a function of frame number. FIG. 7A to FIG. 7C In each of the lines 710 min ,710 mid and 710 max The minimum, average, and maximum luminance values ​​for a given frame are plotted separately. Fig. 7A China, Line 721 min ,721 mid and 721 max The values ​​L1min, L1mid, L1max of a given frame calculated according to the above method with a window length T of 1 second are shown. Figure 7B Middle, Line 722 min ,722 mid and 722 max The values ​​L1min, L1mid, L1max of a given frame calculated according to the above method with a window length T of 4 seconds are shown. Figure 7C Middle, Line 723 min ,723 mid and 723 max The values ​​L1min, L1mid, L1max of a given frame calculated according to the above method with a window length T of 8 seconds are shown.

[0078] By FIG. 7A to FIG. 7C Compared with each other, it can be seen that the shorter window length improves the image quality (for example, line 721 max Tracing line 710 more closely max ); however, longer window lengths result in improved temporal stability (e.g., line 723 max In a specific embodiment of the present disclosure, the window length is preferably 4 seconds. Figure 7B As shown.

[0079] The windowed generation of metadata produces metadata as the outer envelope of a squeeze and clip of a moving window. It requires more frames to be computed than the semi-dynamic generation of metadata, but the number of frames required to compute the metadata for any single frame is limited, unlike the above comparison methods. By applying the filter twice in succession, the resulting filter is essentially three times the specified window length. Successive operations can be combined into a single operation or flow, for example using the above pseudo code.

[0080] Exemplary Computer System Implementation

[0081] Various aspects of the present disclosure may be implemented using a computer system, a system configured with electronic circuits and components, an integrated circuit (IC) device (such as a microcontroller, an FPGA or another configurable or programmable logic device (PLD), a discrete time or digital signal processor (DSP), an ASIC), and / or an apparatus including one or more of such systems, devices, and components. The computer and / or IC may execute, control, or implement instructions associated with the processes or operations described above. The computer and / or IC may calculate any of the various parameters or values ​​associated with the processes or operations described above.

[0082] Figure 8 One example of a computer system 800 in which one or more aspects of the present disclosure may be implemented is illustrated. As illustrated, computer system 800 includes a bus 801 or other communication mechanism for communicating information, and a hardware processor 802 coupled to bus 801 for processing information. Processor 802 may be, for example, a general purpose microprocessor.

[0083] The computer system 800 also includes a main memory 803, such as a random access memory (RAM) or other dynamic storage device, coupled to the bus 801 for storing information and instructions to be executed by the processor 802. The main memory 803 may also be used to store temporary variables or other intermediate information during the execution of instructions to be executed by the processor 802. When stored in a non-transitory storage medium accessible to the processor 802, such instructions turn the computer system 800 into a special-purpose machine customized to perform the operations specified in the instructions.

[0084] The computer system 800 further includes a read-only memory (ROM) 804 or other static storage device coupled to the bus 801 for storing static information and instructions for the processor 802. A storage device 805, such as a magnetic disk or optical disk, may be provided and coupled to the bus 801 for storing information and instructions. The computer system 800 may be coupled to a display 811, such as a liquid crystal display, via the bus 801 for displaying information to a computer user. An input device 812, including alphanumeric and other keys, is coupled to the bus 801 for communicating information and command selections to the processor 802. The computer system 800 may further be coupled to a cursor control 813, such as a mouse, trackball, or cursor direction keys, via the bus 801 for communicating direction information and command selections to the processor 802 and for controlling cursor movement on the display 811.

[0085] The computer system 800 may implement the techniques described herein using custom hardwired logic, one or more ASICs or FPGAs, firmware, and / or program logic that is combined with the computer system to make the computer system 800 a special purpose machine or to program it as a special purpose machine. According to one embodiment, the computer system 800 performs the techniques described herein in response to the processor 802 executing one or more sequences of one or more instructions contained in the main memory 803. Such instructions may be read into the main memory 803 from another storage medium (such as the storage device 805). Execution of the sequences of instructions contained in the main memory 803 causes the processor 802 to perform the process steps described herein. In alternative embodiments, hardwired circuits may be used in place of or in combination with software instructions.

[0086] The term "storage medium" as used herein refers to any non-transitory medium that stores data and / or instructions that cause a machine to operate in a particular manner. Such storage media may include non-volatile media and / or volatile media. Non-volatile media include, for example, optical or magnetic disks, such as storage device 805. Volatile media include dynamic memory, such as main memory 803. Common forms of storage media include, for example, floppy disks, diskettes, hard disks, solid-state drives, magnetic tapes or any other magnetic data storage medium, CD-ROMs, any other optical data storage medium, any physical medium with a pattern of holes, RAM, PROM and EPROM, Flash EPROM, NVRAM, any other memory chip or storage cartridge.

[0087] Storage media are distinct from but may be used in conjunction with transmission media. Transmission media participate in the transfer of information between storage media. For example, transmission media include coaxial cables, copper wires, and optical fibers, including the wires that comprise bus 801. Transmission media may also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.

[0088] Various forms of media may be involved in carrying one or more sequences of one or more instructions to the processor 802 for execution. For example, the instructions may initially be carried on a disk or solid-state drive of a remote computer. The remote computer may load the instructions into its dynamic memory and send these instructions through a communication link such as a coaxial cable, an optical fiber, etc. The communication interface 806 local to the computer system 800 may receive data on the communication line, and appropriate circuits may place the data on the bus 801. The bus 801 carries the data to the main memory 803, from which the processor 802 obtains and executes the instructions. The instructions received by the main memory 803 may optionally be stored on the storage device 805 before or after being executed by the processor 802.

[0089] The communication interface 806 provides a two-way data communication coupled to a network link 821, which is connected to a local network 822. For example, the communication interface 806 can be an integrated services digital network (ISDN) card, a cable modem, a satellite modem, or a modem for providing a data communication connection to a telephone line of a corresponding type. As another example, the communication interface 806 can be a local area network (LAN) card for providing a data communication connection with a compatible LAN. A wireless link can also be implemented. In any such implementation, the communication interface 806 sends and receives an electrical signal, an electromagnetic signal, or an optical signal carrying a digital data stream representing various types of information.

[0090] The network link 821 typically provides data communications to other data devices through one or more networks. For example, the network link 821 can provide a connection to a host computer 823 or to data devices operated by an Internet Service Provider (ISP) 824 through a local network 822. The ISP 824 in turn provides data communication services through a global packet data communication network now commonly referred to as the "Internet" 825. Both the local network 822 and the Internet 825 use electrical, electromagnetic, or optical signals that carry digital data streams. The signals through the various networks and the signals on the network link 821 and through the communication interface 806 (which carry the digital data to and from the computer system 800) are example forms of transmission media.

[0091] Computer system 800 may send messages and receive data, including program code, through network(s), network link 821, and communication interface 806. In the Internet example, server 826 may transmit the requested code for an application program through Internet 825, ISP 824, local network 822, and communication interface 806. The received code may be executed by processor 802 as it is received and / or stored in storage device 805, or other non-volatile storage device, for later execution.

[0092] Application and Effects

[0093] The above aspects of the present disclosure may provide for the generation of metadata that may be used to provide improved video quality, improved temporal stability, low computational requirements, and reduced latency.

[0094] Systems, methods, and devices according to the present disclosure may adopt any one or more of the following configurations.

[0095] (1) A method for editing video content, the method comprising: receiving input video data; converting the input video data into a predetermined format, thereby generating converted video data; generating a plurality of initial metadata values ​​for a frame of the converted video data, the plurality of initial metadata values ​​comprising: a first metadata value, the first metadata value corresponding to a first fixed value that is not calculated based on content including the frame; a second metadata value, the second metadata value corresponding to an average brightness value of the frame; and a third metadata value, the third metadata value corresponding to a second fixed value that is not calculated based on the content, wherein the first metadata value, the second metadata value, and the third metadata value comprise information used by a decoder to present a decoded image on a display.

[0096] (2) The method of (1), further comprising: applying a spatial smoothing operation to the converted video data before generating the plurality of initial metadata values.

[0097] (3) The method of (2), wherein applying the spatial smoothing operation comprises subsampling the converted video data.

[0098] (4) The method of (2) or (3), wherein applying the spatial smoothing operation comprises downscaling the converted video data to a smaller size.

[0099] (5) The method according to any one of (1) to (4), further comprising: outputting distribution video data, wherein the distribution video data includes a video data portion corresponding to the frame, the first metadata value, the second metadata value, and the third metadata value.

[0100] (6) The method according to any one of (1) to (5), wherein the first fixed value corresponds to a minimum allowed brightness value of the content, and the second fixed value corresponds to a maximum allowed brightness value of the content.

[0101] (7) The method according to any one of (1) to (6), wherein the predetermined format is a perceptual quantizer high dynamic range format.

[0102] (8) A non-transitory computer-readable medium storing instructions that, when executed by a processor of a computer system, cause the computer system to perform operations including the method according to any one of (1) to (7).

[0103] (9) A video editing system, comprising: a memory; and a processor, wherein the processor is configured to cause the video editing system to perform the following operations: receive input video data; convert the input video data into a predetermined format, thereby generating converted video data; generate multiple initial metadata values ​​for frames of the converted video data, the multiple initial metadata values ​​comprising: a first metadata value, the first metadata value corresponding to a first fixed value that is not calculated based on content including the frame; a second metadata value, the second metadata value corresponding to an average brightness value of the frame; and a third metadata value, the third metadata value corresponding to a second fixed value that is not calculated based on the content.

[0104] (10) The video editing system of (9), wherein the processor is configured to cause the video editing system to perform the following operations: applying a spatial smoothing operation to the converted video data before generating the plurality of initial metadata values.

[0105] (11) The video editing system of (10), wherein the spatial smoothing operation comprises subsampling the converted video data.

[0106] (12) The video editing system of (10) or (11), wherein the spatial smoothing operation includes reducing the converted video data to a smaller size.

[0107] (13) A video editing system according to any one of (9) to (12), wherein the processor is configured to further cause the video editing system to perform the following operations: output distribution video data, the distribution video data including a video data portion corresponding to the frame, the first metadata value, the second metadata value, and the third metadata value.

[0108] (14) The video editing system according to any one of (9) to (13), wherein the first fixed value corresponds to a minimum allowable brightness value of the content, and the second fixed value corresponds to a maximum allowable brightness value of the content.

[0109] (15) The video editing system according to any one of (9) to (14), wherein the predetermined format is a perceptual quantizer high dynamic range format.

[0110] (16) A method for editing video content, the method comprising: receiving input video data; converting the input video data into a predetermined format, thereby generating converted video data; generating a first metadata value for a frame of the converted video data, the first metadata value corresponding to an average brightness value of the frame; determining a time window including the frame, the time window having a predetermined time length; calculating a maximum metadata range and a minimum metadata range over the time window, thereby generating a first filtered metadata value and a second filtered metadata value; and smoothing the first filtered metadata value and the second filtered metadata value, thereby generating a first smoothed and filtered metadata value and a second smoothed and filtered metadata value.

[0111] (17) The method according to (16), wherein the predetermined time length is four seconds.

[0112] (18) The method according to (16) or (17), wherein the window is configured not to intersect with scene cuts in the input video data.

[0113] (19) The method of any one of (16) to (18), further comprising applying a spatial smoothing operation to the converted video data before generating the first metadata value.

[0114] (20) A non-transitory computer-readable medium storing instructions that, when executed by a processor of a computer system, cause the computer system to perform operations including the method according to any one of (16) to (19).

[0115] With respect to the processes, systems, methods, heuristics, etc. described herein, it should be understood that although the steps of such processes, etc. have been described as being performed in a particular ordered order, such processes may be practiced with the described steps performed in an order different from that described herein. It should further be understood that certain steps may be performed simultaneously, other steps may be added, or certain steps described herein may be omitted. In other words, the process descriptions herein are provided for the purpose of illustrating certain embodiments and should in no way be construed as limiting the claims.

[0116] Therefore, it should be understood that the above description is intended to be illustrative and not restrictive. By reading the above description, many embodiments and applications other than the examples provided will be apparent. The scope should not be determined with reference to the above description, but should be determined with reference to the full range of equivalents to which the appended claims and these claims are eligible for consideration. It is expected and hoped that the technology discussed herein will develop in the future, and the disclosed systems and methods will be incorporated into such future embodiments. In a word, it should be understood that the present application can be modified and changed.

[0117] All terms used in the claims are intended to be given their broadest reasonable interpretations and ordinary meanings as understood by those skilled in the art described herein, unless an explicit indication to the contrary appears herein. In particular, use of singular articles such as "a," "the," "said," etc. should be understood to recite one or more of the indicated elements, unless a claim recites an explicit limitation to the contrary.

[0118] An abstract of the present disclosure is provided to allow the reader to quickly ascertain the nature of the technical disclosure. This abstract is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing detailed description, it can be seen that various features are combined together into various embodiments for the purpose of connecting the present disclosure as a whole. The method of the present disclosure should not be interpreted as reflecting the intention that the claimed embodiments incorporate more features than those explicitly recited in each claim. On the contrary, as reflected in the claims, the inventive subject matter lies in less than all the features of a single disclosed embodiment. Therefore, the appended claims are hereby incorporated into the detailed description, with each claim acting alone as a separately claimed subject matter.

[0119] Various aspects of the present invention may be understood from the following enumerated example embodiments (EEE):

[0120] 1. A method for editing video content, the method comprising:

[0121] receiving input video data;

[0122] converting the input video data into a predetermined format, thereby generating converted video data;

[0123] generating a plurality of initial metadata values ​​for the frame of the converted video data, the plurality of initial metadata values ​​comprising: a first metadata value corresponding to a first fixed value that is not calculated based on content including the frame; a second metadata value corresponding to an average luminance value of the frame; and a third metadata value corresponding to a second fixed value that is not calculated based on the content,

[0124] The first metadata value, the second metadata value, and the third metadata value include information used by a decoder to present a decoded image on a display.

[0125] 2. The method according to EEE 1, further comprising: applying a spatial smoothing operation to the converted video data before generating the plurality of initial metadata values.

[0126] 3. The method according to EEE 2, wherein applying the spatial smoothing operation includes subsampling the converted video data.

[0127] 4. The method according to EEE 2 or EEE 3, wherein applying the spatial smoothing operation includes downscaling the converted video data to a smaller size.

[0128] 5. The method according to any one of EEE 1 to 4, further comprising:

[0129] Distribution video data is output, the distribution video data including a video data portion corresponding to the frame, the first metadata value, the second metadata value, and the third metadata value.

[0130] 6. The method according to any one of EEEs 1 to 5, wherein the first fixed value corresponds to a minimum allowed brightness value of the content, and the second fixed value corresponds to a maximum allowed brightness value of the content.

[0131] 7. The method according to any one of EEE 1 to 6, wherein the predetermined format is a perceptual quantizer high dynamic range format.

[0132] 8. A non-transitory computer-readable medium storing instructions which, when executed by a processor of a computer system, cause the computer system to perform operations including the method according to any one of EEEs 1 to 7.

[0133] 9. A video editing system, comprising:

[0134] Memory; and

[0135] A processor, wherein the processor is configured to enable the video editing system to perform the following operations:

[0136] Receive input video data,

[0137] converting the input video data into a predetermined format, thereby generating converted video data,

[0138] A plurality of initial metadata values ​​are generated for the frame of the converted video data, the plurality of initial metadata values ​​comprising: a first metadata value corresponding to a first fixed value that is not calculated based on content including the frame; a second metadata value corresponding to an average brightness value of the frame; and a third metadata value corresponding to a second fixed value that is not calculated based on the content.

[0139] 10. The video editing system according to EEE 9, wherein the processor is configured to cause the video editing system to perform the following operations:

[0140] Prior to generating the plurality of initial metadata values, a spatial smoothing operation is applied to the converted video data.

[0141] 11. The video editing system according to EEE 10, wherein the spatial smoothing operation comprises subsampling the converted video data.

[0142] 12. The video editing system according to EEE 10 or EEE 11, wherein the spatial smoothing operation comprises downscaling the converted video data to a smaller size.

[0143] 13. The video editing system according to any one of EEE 9 to 12, wherein the processor is configured to further cause the video editing system to perform the following operations:

[0144] Distribution video data is output, the distribution video data including a video data portion corresponding to the frame, the first metadata value, the second metadata value, and the third metadata value.

[0145] 14. The video editing system according to any one of EEEs 9 to 13, wherein the first fixed value corresponds to a minimum allowed brightness value of the content, and the second fixed value corresponds to a maximum allowed brightness value of the content.

[0146] 15. The video editing system according to any one of EEEs 9 to 14, wherein the predetermined format is a perceptual quantizer high dynamic range format.

Claims

1. A method for editing video content, comprising: receiving input video data in a first format; converting the input video data into a predetermined format, the predetermined format being different from the first format of the input video data in terms of dynamic range, thereby generating converted video data; as well as generating a plurality of initial metadata values ​​for the frame of the converted video data, the plurality of initial metadata values ​​comprising: a first metadata value, the first metadata value being equal to a first fixed value, wherein the first fixed value is a minimum allowable brightness value of the first format; a second metadata value, the second metadata value being equal to a calculated average brightness value of the frame; and a third metadata value, the third metadata value being equal to a second fixed value, wherein the second fixed value is a maximum allowable brightness value of the first format, The first metadata value, the second metadata value, and the third metadata value comprise information used by a decoder to present a decoded image on a display.

2. The method according to claim 1, wherein: The predetermined format is a standard dynamic range format, and the first format adopts a high dynamic range format.

3. The method according to claim 1 or 2, further comprising: Prior to generating the plurality of initial metadata values, a spatial smoothing operation is applied to the converted video data.

4. The method according to claim 3, wherein: Applying the spatial smoothing operation includes subsampling the converted video data.

5. The method according to claim 3, wherein: Applying the spatial smoothing operation includes downscaling the converted video data to a smaller size.

6. The method according to claim 1 or 2, further comprising: Distribution video data is output, the distribution video data including a video data portion corresponding to the frame, the first metadata value, the second metadata value, and the third metadata value.

7. The method according to claim 1 or 2, wherein: The predetermined format is a perceptual quantizer high dynamic range format.

8. A non-transitory computer-readable medium storing instructions which, when executed by a processor of a computer system, cause the computer system to perform operations including the method of any one of claims 1 to 7.

9. A video editing system, comprising: Memory; as well as A processor, wherein the processor is configured to enable the video editing system to perform the following operations: receiving input video data in a first format, converting the input video data into a predetermined format, the predetermined format being different from the first format of the input video data in terms of dynamic range, thereby generating converted video data, and Generating a plurality of initial metadata values ​​for the frames of the converted video data, the plurality of initial metadata values ​​comprising: a first metadata value, the first metadata value being equal to a first fixed value, wherein the first fixed value is a minimum allowed brightness value of the first format, a second metadata value, the second metadata value being equal to a calculated average luminance value of the frame; and A third metadata value, the third metadata value is equal to a second fixed value, wherein the second fixed value is a maximum allowed brightness value of the first format.

10. The video editing system according to claim 9, wherein: The predetermined format is a standard dynamic range format, and the first format adopts a high dynamic range format.

11. The video editing system according to claim 9 or 10, wherein: The processor is configured to enable the video editing system to perform the following operations: A spatial smoothing operation is applied to the converted video data prior to generating the plurality of initial metadata values.

12. The video editing system according to claim 11, wherein: The spatial smoothing operation includes subsampling the converted video data.

13. The video editing system according to claim 11, wherein: The spatial smoothing operation includes downscaling the converted video data to a smaller size.

14. The video editing system according to claim 9 or 10, wherein: The processor is configured to further cause the video editing system to perform the following operations: Distribution video data is output, the distribution video data including a video data portion corresponding to the frame, the first metadata value, the second metadata value, and the third metadata value.

15. The video editing system according to claim 9 or 10, wherein: The predetermined format is a perceptual quantizer high dynamic range format.

16. A computer program product, the computer program product comprising a computer program, wherein: The computer program implements the method according to any one of claims 1 to 7 when executed by a processor.

Images