Explanation and constraints of matrix coefficients in CICP

Abstract

A mechanism of processing video data is disclosed. The mechanism includes determining that a video or image should be in a 4:4:4 chroma format when using a luma green chroma orange chroma representation (YCgCo-R) even bit depth (YCgCo-Re) or YCgCo-R odd bit depth (YCgCo-Ro) in a codec independent color point (CICP). Conversion between visual media data and a bitstream is performed based on the CICP.

Description

[0001] Cross-references to related applications

[0002] This application claims priority and rights to U.S. Provisional Patent Application No. 63 / 564,606, filed March 13, 2024; U.S. Provisional Patent Application No. 63 / 631,721, filed April 9, 2024; U.S. Provisional Patent Application No. 63 / 636,770, filed April 20, 2024; U.S. Provisional Patent Application No. 63 / 637,213, filed April 22, 2024; U.S. Provisional Patent Application No. 63 / 637,744, filed April 23, 2024; and U.S. Provisional Patent Application No. 63 / 637,815, filed April 23, 2024. The entire contents of all these applications are incorporated herein by reference. Technical Field

[0003] This disclosure relates to the generation, storage, and use of digital audio and video media information in file formats. Background Technology

[0004] Digital video accounts for the largest share of bandwidth used on the internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth demand for digital video is likely to continue to grow. Summary of the Invention

[0005] The first aspect relates to a method for processing video data, comprising: determining that a video or image should be in a 4:4:4 chroma format when using a luminance-green chroma-orange chroma representation (YCgCo-R) with an even bit depth (YCgCo-Re) or an odd bit depth (YCgCo-Ro) in code-independent code points (CICP); and performing a conversion between visual media data and a bitstream based on the CICP.

[0006] The second aspect relates to an apparatus for processing video data, comprising: a processor; and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform any of the aspects described above.

[0007] The third aspect relates to a non-transitory computer-readable medium comprising a computer program product for use by a video codec device, the computer program product including computer-executable instructions stored on the non-transitory computer-readable medium, such that when executed by a processor, the video codec device performs the methods of any of the preceding aspects.

[0008] The fourth aspect relates to a non-transitory computer-readable recording medium storing a bitstream of video generated by a method performed by a video processing apparatus, wherein the method includes: determining that a video or image should be in a 4:4:4 chroma format when using a luminance-green chroma-orange chroma representation (YCgCo-R) with an even bit depth (YCgCo-Re) or an odd bit depth (YCgCo-R) in code-independent code points (CICP); and generating the bitstream based on the determination.

[0009] The fifth aspect relates to a method for storing a bitstream of video, comprising: determining that a video or image should be in a 4:4:4 chroma format when using luminance-green chroma-orange chroma representation (YCgCo-R) with an even bit depth (YCgCo-Re) or an odd bit depth (YCgCo-R) in codepoints independent of code (CICP); generating a bitstream based on the determination; and storing the bitstream in a non-transitory computer-readable recording medium.

[0010] The sixth aspect relates to the methods, apparatus, or systems described in this disclosure.

[0011] For clarity, any of the embodiments described above may be combined with one or more other embodiments described above to create new embodiments within the scope of this disclosure.

[0012] These and other features will become clearer through the following detailed description of the embodiments with reference to the accompanying drawings and claims. Attached Figure Description

[0013] To gain a more complete understanding of this disclosure, reference is now made to the following brief description, taken in conjunction with the accompanying drawings and detailed description, wherein like reference numerals denote like parts.

[0014] Figure 1 The diagram shows the chroma sample locations for the top and bottom fields when ChromaFormatIdc equals 1, as a function of vui_chroma_sample_loc_type_top_field and vui_chroma_sample_loc_type_bottom_field in the range of 0 to 5 (inclusive).

[0015] Figure 2 The nominal vertical and horizontal positions of the 4:2:2 luminance and chrominance samples in the image are shown.

[0016] Figure 3 The nominal vertical and horizontal positions of the 4:4:4 luminance and chrominance samples in the image are shown.

[0017] Figure 4 The image from left to right shows the position of the top left chroma sample point when ChromaFormatIdc equals 1 and Chroma420LocType equals 0 to 5 (including boundary values).

[0018] Figure 5 This shows the position of the upper left chroma sample point when Chroma420LocType equals 1 and ChromaFormatIdc equals 1.

[0019] Figure 6 The diagram shows how the four luminance channels (right side) are derived from the luminance component (left side) when nnpfc_inp_order_idc equals 3.

[0020] Figure 7 This is a block diagram illustrating an example video processing system.

[0021] Figure 8 This is a block diagram of an example video processing device.

[0022] Figure 9 This is a flowchart of an example method for video processing.

[0023] Figure 10 This is a block diagram illustrating an example video codec system.

[0024] Figure 11 This is a block diagram showing an example encoder.

[0025] Figure 12 This is a block diagram showing an example decoder.

[0026] Figure 13 This is a schematic diagram of an example encoder. Detailed Implementation

[0027] First, it should be understood that although illustrative implementations of one or more embodiments are provided below, the disclosed systems and / or methods can be implemented using any number of techniques, whether currently known or yet to be developed. This disclosure should not be limited in any way to the illustrative implementations, drawings, and techniques shown below, including the exemplary designs and implementations shown and described herein, but modifications can be made within the scope of the appended claims and all their equivalents.

[0028] 1. Preliminary Discussion

[0029] This disclosure relates to image / video coding and decoding technologies. Specifically, this disclosure relates to constraints during the period described in MatrixCoefficients as described in H.264[2] / H.265[3] / H.266[4] Reference Recommendation ITU-T H.273 | ISO / IEC 23091-2 [1] (or CICP for short). These ideas can be applied individually or in various combinations to image / video bitstreams encoded and decoded by any codec, such as the Multi-Function Video Coding (VVC) standard[4] and / or the Multi-Function Supplemental Enhancement Information (SEI) Message (VSEI) standard[5] for encoding and decoding video bitstreams.

[0030] 2. Abbreviation

[0031] The following abbreviations are used in this document: Adaptive Parameter Set (APS), Access Unit (AU), Code-independent Code Point (CICP), Code-in-Channel Video Sequence (CLVS), Code-in-Channel Video Sequence Start (CLVSS), Cyclic Redundancy Check (CRC), Code-in-Channel Video Sequence (CVS), Finite Impulse Response (FIR), Intra-Frame Random Access Point (IRAP), Network Abstraction Layer (NAL), Neural Network Post-Processing Filter (NNPF), Neural Network Post-Processing Filter Activation (NNPFA), Neural Network Post-Processing Filter Characteristics (NNPFC), Picture Parameter Set (PPS), Picture Unit (PU), Random Access Skip Prefix (RASL), Society of Motion Picture and Television Engineers (SMPTE), Supplemental Enhancement Information (SEI), Stepwise Temporal Sublayer Access (STSA), Uniform Resource Identifier (URI), Video Code-in-Channel (VCL), Multifunctional Supplemental Enhancement Information (Recommendation ITU-T H.274 | ISO / IEC 23002-7) (VSEI), Video Availability Information (VUI), Multifunctional Video Code-in-Channel (Recommendation ITU-T H.266 | ISO / IEC 23090-3) (VVC).

[0032] 3. Further discussion

[0033] 3.1 Video Coding and Decoding Standards

[0034] Video coding standards have evolved primarily through the development of standards by the International Telecommunication Union Telecommunication Standardization Sector (ITU-T) and the International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC). ITU-T developed the H.261 and H.263 standards, ISO / IEC developed the Moving Picture Experts Group (MPEG)-1 and MPEG-4 Vision, and the two organizations jointly developed the H.262 / MPEG-2 video standard and the H.264 / MPEG-4 Advanced Video Coding (AVC) [2] and H.265 / HEVC [3] standards. Starting with H.262, video coding standards are based on a hybrid video coding architecture, which utilizes temporal prediction plus transform coding. In order to explore future video coding technologies beyond HEVC, the Joint Video Exploration Team (JVET) was jointly established by the Video Coding Experts Group (VCEG) and MPEG. When the Multifunctional Video Coding (VVC) project was officially launched, JVET was later renamed the Joint Video Experts Group (JVET). VVC[4] is a codec standard that aims to reduce the bit rate by 50% compared to HEVC and is managed by JVET.

[0035] The Multi-Functional Video Coding (VVC) standard (ITU-T H.266 | ISO / IEC 23090-3) [4] and the associated Multi-Functional Supplemental Enhancement Information (VSEI) standard (ITU-T H.274 | ISO / IEC 23002-7) [5] for codec video bitstreams have been designed for the widest range of applications, including simple uses such as television broadcasting, video conferencing or playback from storage media, as well as more advanced use cases such as adaptive bitrate streaming, video region extraction, synthesis and merging of content from multiple codec video bitstreams, multi-view video, scalable layer codecs and viewport adaptive 360° immersive media.

[0036] The Basic Video Codec (EVC) standard (ISO / IEC 23094-1) is another video codec standard developed by MPEG.

[0037] 3.2 Code points independent of encoding / decoding

[0038] Recommendation ITU-T H.273 | ISO / IEC 23091-2[1] specifies different code points that are independent of the encoding / decoding of video signal type identification, in order to identify certain characteristics of video (or still image) media content that are logically independent of the compression format. These characteristics may include, for example, aspects related to the source, presentation, or function of the video (or still image) media component. These characteristics are recorded in a field that takes a coded value or item selected from an enumerated list, referred to herein as a code point.

[0039] Recommendation ITU-T H.273 | ISO / IEC 23091-2[1] provides a central definition of such code points for video and image applications to address these issues. This standard can be used to provide a general description to assist in the interpretation of decoded video and image signals, or to describe the nature of these signals before they are encoded.

[0040] 3.3 Video Availability Information

[0041] In [5], the Video Availability Information (VUI) uses the following syntax table.

[0042]

[0043] 3.3.1. VUI Parameter Semantics

[0044] VUI parameters apply to one or more CLVS.

[0045] Note 1. The interpretation of each syntax element of the VUI parameters is specified by referring to the code-decoding independent code points as defined in Recommendation ITU-T H.273 | ISO / IEC 23091-2. More information on the use of such code points can be found in ITU-T H-Suppl. 19 | ISO / IEC TR 23091-4.

[0046] Using VUI parameters requires defining the following variables:

[0047] — A chroma format indicator, denoted here as ChromaFormatIdc, such that a value of 0 indicates that the image has only a luminance component, and other values ​​indicate that the image has three color components consisting of a luminance component and two associated chroma components, such that the width and height of each chroma component are the width and height of the luminance component divided by SubWidthC and SubHeightC, respectively, where SubWidthC and SubHeightC are derived from ChromaFormatIdc, as specified in Table 1.

[0048] — The bit depth of the sample points for the luminance component, referred to here as BitDepth. Y This represents the bit depth of the samples of the two associated chroma components when ChromaFormatIdc is not equal to 0, denoted here by BitDepth. C express.

[0049] Table 1 – SubWidthC and SubHeightC values ​​derived from ChromaFormatIdc

[0050]

[0051] vui_progressive_source_flag and vui_interlaced_source_flag are explained as follows:

[0052] — If vui_progressive_source_flag equals 1 and vui_interlaced_source_flag equals 0, then the source scan type of the image should be interpreted as scanned only line by line.

[0053] — Otherwise, if vui_progressive_source_flag equals 0 and vui_interlaced_source_flag equals 1, the source scan type of the image should be interpreted as interlaced scan only.

[0054] — Otherwise, if vui_progressive_source_flag equals 0 and vui_interlaced_source_flag equals 0, the source scan type of the image should be interpreted as unknown or unspecified, or specified by external means not specified in this document.

[0055] — Otherwise (vui_progressive_source_flag equals 1 and vui_interlaced_source_flag equals 1), the source scan type for each image is indicated at the image level using the syntax element ffi_source_scan_type in the SEI message of the frame field information.

[0056] A `vui_non_packed_constraint_flag` value of 1 specifies that no frame encapsulation arrangement (SEI) messages applied to CLVS should exist in the bitstream. A `vui_non_packed_constraint_flag` value of 0 does not impose such a constraint.

[0057] A value of 1 for `vui_non_projected_constraint_flag` specifies that no equirectangular projection SEI messages or generalized cubemap projection SEI messages should exist in the bitstream. A value of 0 for `vui_non_projected_constraint_flag` does not impose such a constraint.

[0058] A value of 1 for `vui_aspect_ratio_info_present_flag` indicates the existence of `vui_aspect_ratio_idc`. A value of 0 for `vui_aspect_ratio_info_present_flag` indicates the non-existence of `vui_aspect_ratio_idc`.

[0059] A `vui_aspect_ratio_constant_flag` value of 1 indicates that the values ​​of `vui_aspect_ratio_idc`, `SarWidth`, and `SarHeight` apply to all images in the CLVS, and that Sample Aspect Ratio Information (SARI) SEI messages are not present in the CLVS. A `vui_aspect_ratio_constant_flag` value of 0 indicates that the values ​​of `vui_aspect_ratio_idc`, `SarWidth`, and `SarHeight` may or may not apply to all images in the CLVS, and that SARI SEI messages may exist in the CLVS, indicating the different sample aspect ratios that can be applied to the images associated with the SARI SEI messages. When the `vui_aspect_ratio_constant_flag` syntax element is absent, the value of `vui_aspect_ratio_constant_flag` is presumed to be 0.

[0060] `vui_aspect_ratio_idc` indicates the luminance sample aspect ratio (SAR) of the decoded image in CLVS when it is not equal to 255, unless otherwise indicated by the associated SARI SEI message when `vui_aspect_ratio_constant_flag` is equal to 0. Its semantics are the same as those specified for the `SampleAspectRatio` parameter in Recommendation ITU-T H.273 | ISO / IEC 23091-2. When the `vui_aspect_ratio_idc` syntax element is absent, the value of `vui_aspect_ratio_idc` is presumed to be equal to 0. Values ​​of `vui_aspect_ratio_idc` reserved for future use as specified in Recommendation ITU-T H.273 | ISO / IEC 23091-2 should not exist in bitstreams conforming to this version of the document. The decoder must interpret the value of vui_aspect_ratio_idc, which is reserved in Recommendation ITU-T H.273 | ISO / IEC 23091-2 for future use, as equivalent to the value 0.

[0061] When present, `vui_sar_width` indicates the horizontal size (in arbitrary units) of the SAR of the luminance samples in the decoded image in CLVS, unless otherwise indicated by the associated SARI SEI message when `vui_aspect_ratio_constant_flag` is equal to 0.

[0062] When present, `vui_sar_height` indicates the vertical dimension of the SAR of the luminance sample in the decoded image in CLVS (in any unit the same as `vui_sar_width`), unless otherwise indicated by the associated SARI SEI message when `vui_aspect_ratio_constant_flag` is equal to 0.

[0063] When present, vui_sar_width and vui_sar_height must be coprime or equal to 0. When vui_aspect_ratio_idc equals 0, or vui_sar_width equals 0, or vui_sar_height equals 0, the SAR is unknown or unspecified in this document, or can be determined by other means, such as the SARI SEI message.

[0064] A value of 1 for `vui_overscan_info_present_flag` indicates the presence of `vui_overscan_appropriate_flag`. When `vui_overscan_info_present_flag` is 0 or does not exist, the preferred display method for the video signal is unknown or unspecified, or specified by external means.

[0065] A `vui_overscan_appropriate_flag` of 1 indicates that the cropped decoded image output is suitable for display using overscan. A `vui_overscan_appropriate_flag` of 0 indicates that the cropped decoded image output includes visually important information across the entire area up to the edge of the consistent cropping window of the image, such that the cropped decoded image output should not be displayed using overscan. Instead, they should be displayed using an exact match between the display area and the consistent cropping window, or using underscan. As used in this paragraph, the term "overscan" refers to a display process where some portions near the boundary of the cropped decoded image are not visible in the display area. The term "underscan" describes a display process where the entire cropped decoded image is visible in the display area but does not cover the entire display area. For display processes that use neither overscan nor underscan, the display area perfectly matches the area of ​​the cropped decoded image.

[0066] Note 2. For example, a value of 1 for vui_overscan_appropriate_flag can be used for a live view of people in an entertainment TV show or video conference, while a value of 0 for vui_overscan_appropriate_flag can be used for computer screen capture or security camera content.

[0067] A value of 1 for `vui_colour_description_present_flag` indicates the existence of `vui_colour_primaries`, `vui_transfer_characteristics`, and `vui_matrix_coeffs`. A value of 0 for `vui_colour_description_present_flag` indicates the non-existence of `vui_colour_primaries`, `vui_transfer_characteristics`, and `vui_matrix_coeffs`.

[0068] `vui_colour_primaries` indicates the chromaticity coordinates of the source color primary colors. Its semantics are identical to those specified for the `ColourPrimaries` parameter in Recommendation ITU-T H.273 | ISO / IEC 23091-2. When the `vui_colour_primaries` syntax element is absent, the value of `vui_colour_primaries` is presumed to be equal to 2 (if the chromaticity is unknown, unspecified, or determined by other means not specified in this document). Values ​​of `vui_colour_primaries` marked as reserved for future use in Recommendation ITU-T H.273 | ISO / IEC 23091-2 should not exist in bitstreams conforming to this version of the document. The decoder must interpret reserved values ​​of `vui_colour_primaries` as equivalent to the value 2.

[0069] `vui_transfer_characteristics` indicates the transmission characteristic function of color representation. Its semantics are identical to those specified for the `TransferCharacteristics` parameter in Recommendation ITU-T H.273 | ISO / IEC 23091-2. When the `vui_transfer_characteristics` syntax element is absent, the value of `vui_transfer_characteristics` is presumed to be equal to 2 (if the transmission characteristic is unknown, unspecified, or determined by other means not specified in this document). Values ​​of `vui_transfer_characteristics` marked as reserved for future use in Recommendation ITU-T H.273 | ISO / IEC 23091-2 should not exist in bitstreams conforming to this version of the document. Decoders must interpret reserved values ​​of `vui_transfer_characteristics` as equivalent to the value 2.

[0070] vui_matrix_coeffs describes the equations used to derive luminance and chrominance signals from green, blue, and red, or from the Y, Z, and X primary colors. Its semantics are the same as those specified for MatrixCoefficients in Recommendation ITU-T H.273 | ISO / IEC 23091-2.

[0071] vui_matrix_coeffs should not be equal to 0 unless both of the following conditions are true:

[0072] — BitDepth C Equal to BitDepth Y ,and

[0073] — ChromaFormatIdc equals 3 (4:4:4 chroma format).

[0074] The provision that vui_matrix_coeffs equals 0 under all other conditions is reserved for future use by ITU-T | ISO / IEC.

[0075] vui_matrix_coeffs should not be equal to 8 unless one of the following conditions is true:

[0076] — BitDepthC equals BitDepthY, and

[0077] — BitDepthC equals BitDepthY + 1, and ChromaFormatIdc equals 3 (4:4:4 chroma format).

[0078] The provision that vui_matrix_coeffs equals 8 under all other conditions is reserved for future use by ITU-T | ISO / IEC.

[0079] When the vui_matrix_coeffs syntax element is not present, the value of vui_matrix_coeffs is presumed to be equal to 2 (unknown or unspecified or determined by other means not specified in this document).

[0080] `vui_full_range_flag` indicates the scaling and offset values ​​applied in association with the matrix coefficients. Its semantics are identical to those specified for the `VideoFullRangeFlag` parameter in Recommendation ITU-T H.273 | ISO / IEC 23091-2. When it does not exist, the value of `vui_full_range_flag` is presumed to be 0.

[0081] A value of 1 for `vui_chroma_loc_info_present_flag` indicates the existence of `vui_chroma_sample_loc_type_frame`, or the existence of both `vui_chroma_sample_loc_type_top_field` and `vui_chroma_sample_loc_type_bottom_field`. A value of 0 for `vui_chroma_loc_info_present_flag` indicates the absence of `vui_chroma_sample_loc_type_frame`, `vui_chroma_sample_loc_type_top_field`, and `vui_chroma_sample_loc_type_bottom_field`.

[0082] When ChromaFormatIdc is not equal to 1, vui_chroma_loc_info_present_flag should be equal to 0.

[0083] When vui_chroma_sample_loc_type_frame, vui_chroma_sample_loc_type_top_field, and vui_chroma_sample_loc_type_bottom_field exist, they specify the positions of the chroma samples as follows:

[0084] — If GeneralProgressiveSourceFlag equals 1, GeneralInterlacedSourceFlag equals 0, and ChromaFormatIdc equals 1 (4:2:0 chroma format), then vui_chroma_sample_loc_type_frame specifies the positions of the chroma samples of the two fields in each frame of CLVS, such as... Figure 1 As shown.

[0085] — Otherwise, if ChromaFormatIdc equals 1 (4:2:0 chroma format), then vui_chroma_sample_loc_type_top_field and vui_chroma_sample_loc_type_bottom_field specify the positions of the chroma samples for each top and bottom field of CLVS, respectively. Figure 1 As shown.

[0086] — Otherwise (ChromaFormatIdc is not equal to 1), the values ​​of the syntax elements chroma_sample_loc_type, vui_chroma_sample_loc_type_top_field, and vui_chroma_sample_loc_type_bottom_field must be ignored.

[0087] Figure 1 The diagram shows the chroma sample locations for the top and bottom fields for a ChromaFormatIdc equal to 1 (4:2:0 chroma format), as a functional relationship between vui_chroma_sample_loc_type_top_field and vui_chroma_sample_loc_type_bottom_field in the range of 0 to 5 (inclusive).

[0088] When ChromaFormatIdc equals 2 (4:2:2 chroma format), the nominal position of the chroma sample point co-located with the corresponding luminance sample point, and the nominal position in the image is as follows: Figure 2 As shown.

[0089] Figure 2 The nominal vertical and horizontal positions of the 4:2:2 luminance and chrominance samples in the image are shown.

[0090] When ChromaFormatIdc equals 3 (4:4:4 chroma format), the nominal position of the chroma sample ensures that all array samples co-occur in all image cases, and the nominal position in the image is as follows: Figure 3 As shown.

[0091] Figure 3 The nominal vertical and horizontal positions of the 4:4:4 luminance and chrominance samples in the image are shown.

[0092] When ChromaFormatIdc equals 0, there is no chroma sample array.

[0093] When present, the values ​​of vui_chroma_sample_loc_type_frame, vui_chroma_sample_loc_type_top_field, and vui_chroma_sample_loc_type_bottom_field must be in the range of 0 to 6 (inclusive).

[0094] When ChromaFormatIdc equals 1 and vui_chroma_loc_info_present_flag equals 0, vui_chroma_sample_loc_type_frame does not exist and is presumed to be equal to 6, indicating that the position of the chroma sample point is unknown, unspecified, or specified by other means not specified in this document. When vui_chroma_sample_loc_type_top_field and vui_chroma_sample_loc_type_bottom_field do not exist, the values ​​of vui_chroma_sample_loc_type_top_field and vui_chroma_sample_loc_type_bottom_field are presumed to be equal to vui_chroma_sample_loc_type_frame.

[0095] Note 3. In Recommendation ITU-T H.266 | ISO / IEC 23090-3 and Recommendation ITU-T H.265 | ISO / IEC 23008-2, ChromaFormatIdc equal to 1 identifies the nominal chroma sampling type corresponding to vui_chroma_sample_loc_type_frame, vui_chroma_sample_loc_type_top_field, and vui_chroma_sample_loc_type_bottom_field equal to 0.

[0096] Figure 4This shows the relative position of the top-left chroma sample pointer when ChromaFormatIdc equals 1 (i.e., 4:2:0 chroma format) and either vui_chroma_sample_loc_type_top_field or vui_chroma_sample_loc_type_bottom_field equals the value of the variable Chroma420LocType. The area represented by the top-left luminance sample pointer (depicted as a small black square with a small black dot in its center) is shown relative to the area represented by the top-left 4:2:0 chroma sample pointer (depicted as a large solid gray square with a large gray dot in its center). The area represented by the adjacent luminance sample pointer is depicted as a small dashed gray square with a small gray dot in its center.

[0097] Figure 4 The image from left to right shows the position of the top left chroma sample point when ChromaFormatIdc equals 1 (4:2:0 chroma format) and Chroma420LocType equals 0 to 5 (including boundary values).

[0098] like Figure 5 As shown, the relative spatial positioning of chroma samples can be expressed by defining two variables, HorizontalOffsetC and VerticalOffsetC, as functions of ChromaFormatIdc and Chroma420LocType, as given in Table 2. HorizontalOffsetC is the horizontal distance between the center of the upper left chroma sample and the center of the upper left luminance sample. Figure 5 The x) position in the figure is in units of luminance samples, and VerticalOffsetC is the vertical distance between the center of the upper left chrominance sample and the center of the upper left luminance sample. Figure 5 The y) position is in units of brightness samples.

[0099] In a typical finite impulse response (FIR) filter design, when ChromaFormatIdc is equal to 1 (4:2:0 chroma format) or 2 (4:2:2 chroma format), HorizontalOffsetC and VerticalOffsetC are used as phase offsets for the horizontal and vertical filter operations, respectively, for separable downsampling from the 4:4:4 chroma format to the chroma format indicated by ChromaFormatIdc.

[0100] Figure 5 This shows the position of the upper left chroma sample point when Chroma420LocType equals 1 and ChromaFormatIdc equals 1 (4:2:0 chroma format).

[0101] Table 2 – Definitions of HorizontalOffsetC and VerticalOffsetC as functions of ChromaFormatIdc and Chroma420LocType

[0102]

[0103] When ChromaFormatIdc equals 1 (4:2:0 chroma format) and the decoded video content is intended to be interpreted in accordance with Recommendation ITU-R BT.2020 or Recommendation ITU-R BT.2100, vui_chroma_loc_info_present_flag must be equal to 1, and vui_chroma_sample_loc_type_frame, vui_chroma_sample_loc_type_top_field, and vui_chroma_sample_loc_type_bottom_field (where applicable) must be equal to 2.

[0104] VUI parameters are applied to one or more CLVS.

[0105] Note 1. The interpretation of each syntax element of the VUI parameters is specified by referring to the code-decoding independent code points as defined in Recommendation ITU-T H.273 | ISO / IEC 23091-2. More information on the use of such code points can be found in ITU-T H-Suppl.19 | ISO / IEC TR 23091-4.

[0106] 3.4 Refer to the SEI message of MatrixCoefficients

[0107] In VSEI [5], in addition to VUI, there are some SEI messages that reference MatrixCoefficients in CICP. Specifically, these include color transformation information SEI messages, film grain property SEI messages, and neural network post-processing filter property SEI messages.

[0108] 3.4.1. SEI Message for Film Grain Characteristics

[0109] 3.4.1.1 SEI Message Syntax for Film Grain Characteristics

[0110]

[0111] 3.4.1.2. Semantics of SEI Messages for Film Grain Characteristics

[0112] This SEI message provides the decoder with a parameterized model for film grain composition. The film grain composition process should be applied to the decoded image before the decoded image is displayed.

[0113] Note 1. For example, an encoder can use the Film Grain Feature (SEI) message to characterize film grain present in the original source video material and removed by preprocessing filtering techniques. Synthesizing simulated film grain on the input image (which can be a decoded image or converted from a decoded image) for the display process is optional and does not necessarily have to strictly adhere to the semantics specified in the Film Grain Feature (SEI) message. When synthesizing simulated film grain on the input image for the display process, it is not required that the synthesis method used be the same as the parameterized model for film grain provided in the Film Grain Feature (SEI) message.

[0114] Note 2. The display process is not specified in this document.

[0115] Note 3. SMPTE RDD 5 (2006) specifies a film grain simulator based on information provided in the SEI message for film grain characteristics.

[0116] To use this SEI message, the following variables need to be defined:

[0117] — The image width and image height, expressed in units of brightness samples, are represented here as PicWidthInLumaSamples and PicHeightInLumaSamples, respectively.

[0118] — When the syntax element fg_separate_colour_description_present_flag of the SEI message for film grain characteristics is equal to 0, the following additional variables are applied:

[0119] — Chroma format indicator, referred to here as ChromaFormatIdc, as described in sub-entry 7.3.

[0120] — The bit depth of the sample points for the luminance component, referred to here as BitDepth. Y This represents the bit depth of the samples of the two associated chroma components when ChromaFormatIdc is not equal to 0, denoted here by BitDepth. C express.

[0121] The film grain model, as defined in the SEI message, is applied to decoded images with a 4:4:4 color format, where the luminance and chrominance bit depths correspond to those of the film grain model, and it uses the same color gamut as the identified film grain model. When the decoded video's color format is not 4:4:4, or when the decoded video uses a different luminance or chrominance bit depth or a different color gamut than the identified film grain model, it is expected that an undefined conversion process will be applied to transform the decoded image into a form that uses the film grain model.

[0122] Note 4. Because no specific method is required to perform the film grain generation function used in the display process, the decoder can downconvert the model information for chroma to simulate film grain for other chroma formats (4:2:0 or 4:2:2) if needed, instead of upconverting the decoded video before performing film grain generation (using methods not specified in this document).

[0123] `fg_characteristics_cancel_flag` equal to 1 indicates that the SEI message cancels the persistence of any previous film grain characteristics SEI messages applied to the current layer in the order of output. `fg_characteristics_cancel_flag` equal to 0 indicates that film grain modeling information follows.

[0124] The `fg_model_id` identifier identifies the film grain simulation model, as specified in Table 5. The value of `fg_model_id` must be in the range of 0 to 1 (boundary values). Values ​​2 and 3 of `fg_model_id` are reserved for future use by ITU-T | ISO / IEC and should not exist in the bitstream conforming to this version of the document. The decoder must ignore film grain characteristic SEI messages where `fg_model_id` is equal to 2 or 3.

[0125] Table 5 – fg_model_id values

[0126]

[0127] `fg_separate_colour_description_present_flag` equal to 1 indicates that for the film grain characteristics specified in the SEI message, there are different combinations of luma bit depth, chroma bit depth, video full range flag, primary color, transmission characteristics, and matrix coefficients in the SEI message syntax. `fg_separate_colour_description_present_flag` equal to 0 indicates that for the film grain characteristics specified in the SEI message, the combination of luma bit depth, chroma bit depth, video full range flag, primary color, transmission characteristics, and matrix coefficients is the same as the combination indicated in the CLVS VUI parameters.

[0128] Note 5. When fg_separate_colour_description_present_flag equals 1, any of the luminance bit depth, chrominance bit depth, video full range flag, primary color, transmission characteristics, and matrix coefficients specified in the SEI message can be different from the corresponding items in the CLVS image.

[0129] When there is no VUI parameter for CLVS or the value of vui_colour_description_present_flag is equal to 0, and when the equivalent information transmitted when vui_colour_description_present_flag is equal to 1 is not transmitted through external means, fg_separate_colour_description_present_flag must be equal to 1.

[0130] The input image Î (used in the equation of this sub-entry) can be a decoded image or an input image converted from a decoded image, and it resides in the same color gamut as the analog film grain signal. Therefore, when any of these parameters differs from the corresponding value for the image in CLVS, the input image Î used in the equation of this sub-entry will reside in a different color gamut than the image in CLVS. For example, when the value of fg_bit_depth_luma_minus8 + 8 is greater than BitDepthY (i.e., the bit depth of the luminance component of the image in CLVS), the bit depth of the input image Î used in the equation of this sub-entry will also be greater than BitDepthY. In this case, the input image Î corresponding to the actual decoded image will be generated by converting the actual decoded image to a color gamut that is in the same color gamut as the analog film grain signal. The process for converting an actual decoded image to a 4:4:4 color format with the same color gamut as the analog film grain signal is not specified in this document.

[0131] The increment of 8 in `fg_bit_depth_luma_minus8` specifies the bit depth of the luminance component used for the film grain characteristics specified in the SEI message. When `fg_bit_depth_luma_minus8` is not present in the film grain characteristic SEI message, the value of `fg_bit_depth_luma_minus8` is presumed to be equal to `BitDepthY - 8`.

[0132] The value of fgBitDepth[0] is derived as follows:

[0133] fgBitDepth[ 0 ] = fg_bit_depth_luma_minus8 + 8 (19)

[0134] fg_bit_depth_chroma_minus8 plus 8 specifies the bit depth of the Cb and Cr components used for the film grain characteristics specified in the SEI message. When fg_bit_depth_chroma_minus8 is not present in the film grain characteristic SEI message, the value of fg_bit_depth_chroma_minus8 is presumed to be equal to BitDepthC - 8.

[0135] For c equal to 1 and 2, the value of fgBitDepth[c] is derived as follows:

[0136] fgBitDepth[c] = fg_bit_depth_chroma_minus8 + 8, where c = 1, 2 (20)

[0137] fg_full_range_flag has the same semantics as the vui_full_range_flag syntax element specified in sub-entry 7.3, except as follows:

[0138] — fg_full_range_flag specifies the full range of video for the film grain characteristics defined in the SEI message, rather than the full range of video for CLVS.

[0139] — When fg_full_range_flag is not present in the film grain feature SEI message, the value of fg_full_range_flag is presumed to be equal to vui_full_range_flag.

[0140] fg_colour_primaries has the same semantics as the vui_colour_primaries syntax element specified in sub-entry 7.3, except as follows:

[0141] — fg_colour_primaries specifies the primary color for the film grain characteristics defined in the SEI message, rather than the primary color used for CLVS.

[0142] — When fg_colour_primaries does not exist in the film grain property SEI message, the value of fg_colour_primaries is presumed to be equal to vui_colour_primaries.

[0143] fg_transfer_characteristics has the same semantics as the vui_transfer_characteristics syntax element specified in sub-entry 7.3, except as follows:

[0144] — fg_transfer_characteristics specifies the transmission characteristics of the film grain characteristics as defined in the SEI message, rather than the transmission characteristics used for CLVS.

[0145] — When fg_transfer_characteristics is not present in the film grain characteristics SEI message, the value of fg_transfer_characteristics is presumed to be equal to vui_transfer_characteristics.

[0146] fg_matrix_coeffs has the same semantics as the vui_matrix_coeffs syntax element specified in sub-entry 7.3, except as follows:

[0147] — fg_matrix_coeffs specifies the matrix coefficients for the film grain characteristics defined in the SEI message, rather than the matrix coefficients used for CLVS.

[0148] — When fg_matrix_coeffs does not exist in the film grain property SEI message, the value of fg_matrix_coeffs is presumed to be equal to vui_matrix_coeffs.

[0149] — For the semantics of VUI parameters, the allowed values ​​of fg_matrix_coeffs are not constrained by the chroma format of the decoded video image, which is indicated by the value of ChromaFormatIdc.

[0150] The `fg_blending_mode_id` identifier specifies the blending mode used to blend analog film grain with the input image, as defined in Table 6. `fg_blending_mode_id` must be in the range of 0 to 1 (inclusive). Values ​​2 and 3 of `fg_blending_mode_id` are reserved for future use by ITU-T | ISO / IEC and should not exist in the bitstream conforming to this version of the document. The decoder must ignore film grain characteristic SEI messages where `fg_blending_mode_id` is equal to 2 or 3.

[0151] Table 6 – fg_blending_mode_id values

[0152]

[0153] Based on the value of fg_blending_mode_id, the blending mode is defined as follows:

[0154]

[0155] Where Î[c][x][y] represents the sample value at coordinates x and y of the color component c of the input image Î, G[c][x][y] is the simulated film grain value at the same position and color component, and fgBitDepth[c] is used for array I grain The number of bits per sample in the fixed-length unsigned binary representation of [c][x][y], Î[c][x][y], and G[c][x][y], where c = 0..2, x = 0..PicWidthInLumaSamples - 1, and y = 0..PicHeightInLumaSamples - 1.

[0156] fg_log2_scale_factor specifies the scaling factor used in the equation representing film grain.

[0157] The value of fg_comp_model_present_flag[c] equal to 0 indicates that film grain is not modeled for the c-th color component, where c equals 0 to reference the luminance component, c equals 1 to reference the Cb component, and c equals 2 to reference the Cr component. The value of fg_comp_model_present_flag[c] equal to 1 indicates that the syntax element specifying the modeling of film grain for color component c exists in the SEI message.

[0158] When fg_separate_colour_description_present_flag equals 0 and ChromaFormatIdc equals 0, the values ​​of fg_comp_model_present_flag[1] and fg_comp_model_present_flag[2] must be equal to 0.

[0159] The increment of 1 in fg_num_intensity_intervals_minus1[c] indicates that the number of intensity intervals for a particular set of model values ​​has been estimated.

[0160] Note 6. Intensity ranges can overlap to simulate grain from multiple generations of film.

[0161] The increment of 1 in fg_num_model_values_minus1[c] specifies the number of model values ​​present for each intensity range in which film grain has been modeled. The value of fg_num_model_values_minus1[c] must be in the range of 0 to 5 (inclusive of boundary values).

[0162] fg_intensity_interval_lower_bound[c][i] specifies the lower bound of the i-th intensity interval of the applied model value set.

[0163] fg_intensity_interval_upper_bound[c][i] specifies the upper bound of the i-th intensity interval of the applied model value set.

[0164] The variable intensityIntervalIdx[c][x][y][j] represents the j-th index of the list of intensity intervals selected by the sample value Î[c][x][y], where c = 0..2, x = 0..PicWidthInLumaSamples-1, y = 0..PicHeightInLumaSamples-1, and j = 0..numApplicableIntensityIntervals[c][x][y]-1, where numApplicableIntensityIntervals[c][x][y] is derived as follows.

[0165] Based on the value of fg_model_id, for the sample value Î[ c ][ x ][ y ], the selection of one or more intensity intervals is specified as follows:

[0166] The variable numApplicableIntensityIntervals[c][x][y] is initially set to 0.

[0167] If fg_model_id equals 0, then the following applies:

[0168] — The top-left sample position (xB, yB) of the current 8x8 block b containing sample values ​​Î[c][x][y] is derived as (xB, yB) = (x / 8, y / 8).

[0169] — The average value of the current 8x8 block b avg The derivation is as follows:

[0170] sum8x8 = 0

[0171] for( i = 0; i < 8; i++ )

[0172] for( j = 0; j < 8; j++ )

[0173] sum8x8 += Î[ c ][ xB 8 + i ][yB 8 + j]

[0174] b avg = Clip3( 0, 255, ( sum8x8 + ( 1 << ( fgBitDepth[ c ] - 3 ) ) )>> ( fgBitDepth[ c ] - 2 ) )

[0175] The value of intensityIntervalIdx[c][x][y][j] is derived as follows:

[0176] for( i = 0, j = 0; i <= fg_num_intensity_intervals_minus1[ c ]; i++)

[0177] if( b avg >= fg_intensity_interval_lower_bound[ c ][ i ] &&

[0178] b avg <= fg_intensity_interval_upper_bound[ c ][ i ] )

[0179] {

[0180] intensityIntervalIdx[ c ][ x ][ y ][ j ] = i

[0181] j++

[0182] }

[0183] numApplicableIntensityIntervals[ c ][ x ][ y ] = j

[0184] Otherwise (fg_model_id equals 1), the value of intensityIntervalIdx[c][x][y][j] is derived as follows:

[0185] I8[ c ][ x ][ y ] = ( fgBitDepth[ c ] == 8 ) ?

[0186] ( Î[ c ][ x ][ y ] : Clip3( 0, 255, ( Î[ c ][ x ][ y ] + ( 1 << (fgBitDepth[ c ] - 9 ) ) ) >> ( fgBitDepth[ c ] - 8 ))

[0187] for( i = 0, j = 0; i <= fg_num_intensity_intervals_minus1[ c ]; i++)

[0188] if( I8[ c ][ x ][ y ] >= fg_intensity_interval_lower_bound[ c ][ i] &&

[0189] I8[ c ][ x ][ y ] <= fg_intensity_interval_upper_bound[ c ][ i ] )

[0190] {

[0191] intensityIntervalIdx[ c ][ x ][ y ][ j ] = i

[0192] j++

[0193] }

[0194] numApplicableIntensityIntervals[ c ][ x ][ y ] = j

[0195] Samples that do not fall within any defined interval (i.e., those samples where the value of numApplicableIntensityIntervals[c][x][y] is equal to 0) are not modified by the particle generation function. Samples that fall within more than one interval (i.e., those samples where the value of numApplicableIntensityIntervals[c][x][y] is greater than 1) will generate multi-generational particles. Multi-generational particles are generated by adding particles that are computed independently for each intensity interval of the application.

[0196] In the equations of the remainder of this sub-entry, the variable sj in each instance of the list fg_comp_model_value[c][sj] is the value of intensityIntervalIdx[c][x][y][j] derived for the sample value Î[c][x][y].

[0197] `fg_comp_model_value[c][i][j]` specifies the `j`-th model value for color component `c` and the `i`-th intensity interval. The set of model values ​​has different meanings depending on the value of `fg_model_id`.

[0198] The value of fg_comp_model_value[c][i][j] is constrained as follows, and may be additionally constrained as specified elsewhere in this sub-entry:

[0199] — If fg_model_id equals 0, then fg_comp_model_value[c][i][j] must be between 0 and 2. fgBitDepth[ c ] - Within the range of 1 (including boundary values).

[0200] — Otherwise (fg_model_id equals 1), fg_comp_model_value[c][i][j] must be in the range of -2. ( fgBitDepth[ c ] - 1 ) to 2 ( fgBitDepth[ c ] - 1 ) - Within the range of 1 (including boundary values).

[0201] Based on the value of fg_model_id, the synthesis of film grain is modeled as follows:

[0202] — If fg_model_id equals 0, the frequency filtering model enables the simulation of the original film grain for c = 0..2, x = 0..PicWidthInLumaSamples - 1, and y = 0..PicHeightInLumaSamples - 1, as specified:

[0203] G[ c ][ x ][ y ] = ( fg_comp_model_value[ c ][ s j [0] Q[ c ][ x][ y ] + fg_comp_model_value[ c ][ s j [5]

[0204] G[ c - 1 ][ x ][ y ] ) >> fg_log2_scale_factor

[0205] Where Q[c] is generated by sampling the random-valued elements gaussRv using a normalized Gaussian distribution (samples of independent and identically distributed Gaussian random variables with zero mean and unit variance). ij The two-dimensional random process is generated by filtering a 16x16 block of GaussRv, and the value of the element G[c-1][x][y] used on the right side of the equation is presumed to be equal to 0 when c-1 is less than 0.

[0206] Note 7. A normalized Gaussian random variable can be generated using two independent uniformly distributed random values ​​(denoted as uRv0 and uRv1) from the interval 0 to 1 (and not equal to 0) by the Box-Muller transformation as specified below:

[0207] gaussRv i,j = Sqrt( -2 Ln( uRv0 ) ) Cos( 2 π uRv1 ) (27)

[0208] Where π is Archimedes' constant 3.141592653589793....

[0209] The bandpass filtering of block gaussRv can be performed in the discrete cosine transform (DCT) domain as follows:

[0210]

[0211] Where IDCT16x16(z) refers to the inverse unitary discrete cosine transform (IDCT) of the 16x16 matrix parameter z, as defined below:

[0212] IDCT16x16(z) = r z r T (29)

[0213] The superscript T indicates the matrix transpose, and r is a 16x16 matrix, where the elements r ij The regulations are as follows:

[0214] (30)

[0215] Where π is Archimedes' constant 3.141592653589793....

[0216] Q[c] is formed by the frequency-filtered block of filteredRv.

[0217] Note 8. The codec model values ​​are based on 16x16 blocks, but the decoder implementation can use other block sizes. For example, a decoder implementing IDCT on 8x8 blocks can down-convert the set of codec model values ​​fg_comp_model_value[c][sj][i] by a factor of 2, where i is equal to 1..4.

[0218] Note 9. To reduce the visibility of blocks generated by splicing frequency-filtered blocks (filteredRv), the decoder can apply a low-pass filter to the boundaries between frequency-filtered blocks.

[0219] — Otherwise (fg_model_id equals 1), the autoregressive model enables the simulation of the original film grain for c = 0..2, x = 0..PicWidthInLumaSamples - 1 and y = 0..PicHeightInLumaSamples - 1, as specified below:

[0220]

[0221] Where n[c][x][y] are random values ​​with a normalized Gaussian distribution (samples of independent and identically distributed Gaussian random variables with zero mean and unit variance for each value of c, x, and y), and the value of the element G[c][x][y] used on the right-hand side of the equation is presumed to be equal to 0 if any of the following conditions are true:

[0222] — c is less than 0,

[0223] — x is less than 0,

[0224] — y is less than 0.

[0225] fg_comp_model_value[c][i][0] provides the first model value of the model as specified by fg_model_id. fg_comp_model_value[c][i][0] corresponds to the standard deviation of the Gaussian noise term in the generating function specified in Equations 26 to 31.

[0226] fg_comp_model_value[c][i][1] provides a second model value for the model as specified by fg_model_id. When fg_model_id is equal to 0, fg_comp_model_value[c][i][1] must be greater than or equal to 0 and less than 16.

[0227] When not present in the SEI message for film grain characteristics, fg_comp_model_value[c][i][1] is presumed as follows:

[0228] — If fg_model_id equals 0, then fg_comp_model_value[c][i][1] is presumed to be equal to 8.

[0229] — Otherwise (fg_model_id equals 1), fg_comp_model_value[c][i][1] is presumed to be equal to 0.

[0230] fg_comp_model_value[c][i][1] is interpreted as follows:

[0231] — If fg_model_id equals 0, then fg_comp_model_value[c][i][1] indicates the high cutoff frequency for filtering the DCT of blocks of 16x16 random values.

[0232] — Otherwise (fg_model_id equals 1), fg_comp_model_value[c][i][1] indicates the first-order spatial correlation of neighboring samples (x-1, y) and (x, y-1).

[0233] fg_comp_model_value[c][i][2] provides the third model value of the model as specified by fg_model_id. When fg_model_id is equal to 0, fg_comp_model_value[c][i][2] must be greater than or equal to 0 and less than 16.

[0234] When not present in the SEI message for film grain characteristics, fg_comp_model_value[c][i][2] is presumed as follows:

[0235] — If fg_model_id equals 0, then fg_comp_model_value[c][i][2] is presumed to be equal to fg_comp_model_value[c][i][1].

[0236] — Otherwise (fg_model_id equals 1), fg_comp_model_value[c][i][2] is presumed to be equal to 0.

[0237] fg_comp_model_value[c][i][2] is interpreted as follows:

[0238] — If fg_model_id equals 0, then fg_comp_model_value[c][i][2] indicates the vertical high cutoff frequency for filtering the DCT of a block of 16x16 random values.

[0239] — Otherwise (fg_model_id equals 1), fg_comp_model_value[c][i][2] indicates the color correlation between consecutive color components.

[0240] fg_comp_model_value[c][i][3] provides the fourth model value of the model as specified by fg_model_id. When fg_model_id is equal to 0, fg_comp_model_value[c][i][3] must be greater than or equal to 0 and less than or equal to fg_comp_model_value[c][i][1].

[0241] When it does not exist in the film grain property SEI message, fg_comp_model_value[c][i][3] is presumed to be equal to 0.

[0242] fg_comp_model_value[c][i][3] is interpreted as follows:

[0243] — If fg_model_id equals 0, then fg_comp_model_value[c][i][3] indicates a low cutoff frequency for filtering the DCT of blocks of 16x16 random values.

[0244] — Otherwise (fg_model_id equals 1), fg_comp_model_value[c][i][3] indicates the first-order spatial correlation of neighboring samples (x-1, y-1) and (x+1, y-1).

[0245] fg_comp_model_value[c][i][4] provides the fifth model value of the model as specified by fg_model_id. When fg_model_id is equal to 0, fg_comp_model_value[c][i][4] must be greater than or equal to 0 and less than or equal to fg_comp_model_value[c][i][2].

[0246] When it does not exist in the film grain property SEI message, fg_comp_model_value[c][i][4] is presumed to be equal to fg_model_id.

[0247] fg_comp_model_value[c][i][4] is interpreted as follows:

[0248] — If fg_model_id equals 0, then fg_comp_model_value[c][i][4] indicates the vertical low cutoff frequency for filtering the DCT of a block of 16x16 random values.

[0249] — Otherwise (fg_model_id equals 1), fg_comp_model_value[c][i][4] indicates the aspect ratio of the modeled particle.

[0250] fg_comp_model_value[c][i][5] provides the sixth model value of the model specified by fg_model_id.

[0251] When it does not exist in the film grain property SEI message, fg_comp_model_value[c][i][5] is presumed to be equal to 0.

[0252] fg_comp_model_value[c][i][5] is interpreted as follows:

[0253] — If fg_model_id equals 0, then fg_comp_model_value[c][i][5] indicates the color correlation between consecutive color components.

[0254] — Otherwise (fg_model_id equals 1), fg_comp_model_value[c][i][5] indicates the second-order spatial correlation of neighboring samples (x, y - 2) and (x - 2, y).

[0255] fg_characteristics_persistence_flag specifies the persistence of the SEI message for the film grain characteristics of the current layer.

[0256] A value of 0 for fg_characteristics_persistence_flag indicates that the SEI message for film grain characteristics applies only to the currently decoded image.

[0257] A value of 1 for `fg_characteristics_persistence_flag` specifies that the film grain characteristic SEI message is applied to the currently decoded image and persists for all subsequent images of the current layer in output order until one or more of the following conditions are true:

[0258] — A new CLVS begins for the current layer.

[0259] — End of bitstream.

[0260] — The image in the current layer of the AU associated with the film grain property SEI message is output, following the current image in the output order.

[0261] 3.4.2 Color Transformation Information SEI Message

[0262] 3.4.2.1 Color Transformation Information SEI Message Syntax

[0263]

[0264] 3.4.2.2 Color Transformation Information (SEI) Message Semantics

[0265] To use this SEI message, the following variables need to be defined:

[0266] — Chroma format indicator, referred to here as ChromaFormatIdc, as described in sub-entry 7.3.

[0267] The Color Transform Information (CTI) SEI message provides information to enable the remapping of reconstructed color samples from the output image for purposes such as converting the output image into a representation more suitable for alternative displays. The color transformation model used in the CTI SEI message consists of a first piecewise linear function applied to the first color component. Depending on the values ​​of the syntax elements `colour_transform_cross_component_flag`, `colour_transform_cross_comp_inferred_flag`, and `colour_transform_lut2_present_flag`, one or two additional piecewise linear functions can be signaled for the second and third color components.

[0268] When ChromaFormatIdc equals 0 (monochrome), CTI SEI messages should not exist, although the decoder must still allow such messages to exist and ignore any such CTI SEI messages when they exist.

[0269] `colour_transform_id` contains an identifier that can be used to identify the purpose of CTI. The value of `colour_transform_id` can be used (in a manner not specified in this document) to indicate that the input to the remapping process is the output of some transformation process not specified in this document, such as converting an image to some alternative color representation (e.g., from YCbCr color representation to GBR color representation). When more than one CTI SEI message with the same value of `colour_transform_id` exists, the contents of these CTI SEI messages must be identical. When more than one CTI SEI message with a value of `colour_transform_id` exists, this can indicate that the remapping processes indicated by different values ​​of `colour_transform_id` are alternatives provided for different purposes, or indicate that the cascading of remapping processes will be applied sequentially (in an order not specified in this document). The value of `colour_transform_id` must be between 0 and 2. 32 - Within the range of 2 (including boundary values).

[0270] The value of colour_transform_id ranges from 0 to 255 and from 512 to 255. 31 - 1 can be used depending on the application. The value of colour_transform_id ranges from 256 to 511 (including boundary values) and from 2...31 to 2 32 - 2 (including boundary values) is reserved for future use by ITU-T | ISO / IEC. The decoder must ignore values ​​in the range of 256 to 511 (including boundary values) or 2. 31 to 2 32 - CTI SEI messages with values ​​of colour_transform_id within the range of 2 (including boundary values), and bitstreams conforming to this version of the document should not contain colour_transform_id with these values.

[0271] Note: colour_transform_id can be used to support different remapping processes for different display scenarios. For example, different values ​​of colour_transform_id can correspond to different remapping color spaces supported by the display.

[0272] A color_transform_cancel_flag value of 1 indicates that a CTI SEI message cancels the persistence of any previous CTI SEI messages applied to the current layer in output order. A color_transform_cancel_flag value of 0 indicates that CTI follows.

[0273] colour_transform_persistence_flag specifies the persistence of CTI SEI messages in the current layer.

[0274] A color_transform_persistence_flag value of 0 indicates that the CTI SEI message applies only to the currently decoded image.

[0275] A color_transform_persistence_flag value of 1 indicates that the CTI SEI message applies to the currently decoded image and persists for all subsequent images of the current layer in output order until one or more of the following conditions are true:

[0276] — A new CLVS begins for the current layer.

[0277] — End of bitstream.

[0278] — The image in the current layer of the AU associated with the CTI SEI message is output, following the current image in the output order.

[0279] A value of 1 for `colour_transform_video_signal_info_present_flag` indicates the existence of the syntax elements `colour_transform_full_range_flag`, `colour_transform_primaries`, `colour_transform_transfer_function`, and `colour_transform_matrix_coefficients`. A value of 0 for `colour_transform_video_signal_info_present_flag` indicates the absence of these syntax elements.

[0280] The `colour_transform_full_range_flag` has the same semantics as the `vui_full_range_flag` syntax element specified in sub-entry 7.3, except that `colour_transform_full_range_flag` identifies the color space of the remapped reconstructed image, rather than the color space used for CLVS. When it does not exist, the value of `colour_transform_full_range_flag` is presumed to be equal to the value of `vui_full_range_flag`.

[0281] `colour_transform_primaries` has the same semantics as the `vui_colour_primaries` syntax element specified in sub-entry 7.3, except that `colour_transform_primaries` identifies the color space of the remapped reconstructed image, rather than the color space used for CLVS. When it does not exist, the value of `colour_transform_primaries` is presumed to be equal to the value of `vui_colour_primaries`.

[0282] The `colour_transform_transfer_function` has the same semantics as the `vui_transfer_characteristics` syntax element specified in sub-entry 7.3, except that `colour_transform_transfer_function` identifies the color space of the remapped reconstructed image, instead of the color space used for CLVS. When it does not exist, the value of `colour_transform_transfer_function` is presumed to be equal to the value of `vui_transfer_characteristics`.

[0283] `colour_transform_matrix_coefficients` has the same semantics as the `vui_matrix_coeffs` syntax element specified in sub-entry 7.3, except that `colour_transform_matrix_coefficients` identifies the color space of the remapped reconstructed image, rather than the color space used for CLVS. When it does not exist, the value of `colour_transform_matrix_coefficients` is presumed to be equal to the value of `vui_matrix_coeffs`.

[0284] `colour_transform_bit_depth_minus8 + 8` specifies the bit depth of the color components of the associated image, used for interpreting CTI SEI messages. When any CTI SEI message exists with a value of `colour_transform_bit_depth + 8` that is not equal to the bit depth of the decoded color components, the SEI message refers to the assumed result of a transformation operation performed to convert the decoded color component samples to a bit depth equal to `colour_transform_input_bit_depth + 8`.

[0285] The value of colour_transform_bit_depth plus 8 must be in the range of 8 to 16 (inclusive). Values ​​of colour_transform_bit_depth in the range of 17 to 23 (inclusive) are reserved for future use by ITU-T | ISO / IEC. The decoder must ignore CTI SEI messages containing colour_transform_bit_depth values ​​in the range of 17 to 23 (inclusive), and bitstreams conforming to this version of the document should not contain colour_transform_bit_depth values ​​with these values.

[0286] bitDepth is set to equal to (color_transform_bit_depth + 8).

[0287] colour_transform_log2_number_of_points_per_lut_minus1 specifies the number of pivot points in the piecewise linear remapping function as log2 minus 1.

[0288] log2numLutPoints is set to equal to (color_transform_log2_number_of_points_per_lut_minus1 + 1).

[0289] numLutPoints is set to equal to (1 << log2numLutPoints).

[0290] colourTransformSize is set to equal to (numLutPoints + 1).

[0291] log2distX is set to equal to ( bitDepth - log2numLutPoints ).

[0292] A color_transform_cross_component_flag value of 1 indicates that the remapping of the second and third color components is performed based on the first color component as a cross-component remapping. A color_transform_cross_component_flag value of 0 indicates that intra-component remapping is applied to the second and third color components.

[0293] maxIntraComp is set to equal to (2) ( 1 - colour_transform_cross_component_flag ) ).

[0294] A color_transform_cross_comp_inferred_flag value of 1 indicates that the remapping piecewise linear functions for the second and third color components are derived from the remapping piecewise linear function for the first color component. A color_transform_cross_comp_inferred_flag value of 0 indicates that the remapping piecewise linear functions for the second and third color components are transmitted via signaling. When it does not exist, the value of color_transform_cross_comp_inferred_flag is assumed to be 0.

[0295] `colour_transf_lut[c][i]` specifies the piecewise linear remapping function for the color component at index `c`. When `colour_transf_lut[1][i]` exists and `colour_transf_lut[2][i]` does not exist, the value of `colour_transf_lut[2][i]` is presumed to be equal to `colour_transf_lut[1][i]`. The length of `colour_transf_lut[c][i]` is 2 + bitDepth - log2numLutPoints bits.

[0296] A color_transform_lut2_present_flag value of 1 indicates that color_transf_lut[2][i] exists in the CTI SEI message. A color_transform_lut2_present_flag value of 0 indicates that color_transf_lut[2][i] does not exist in the CTI SEI message. When it does not exist, the value of color_transform_lut2_present_flag is presumed to be equal to 0.

[0297] `colour_transform_chroma_offset` specifies the CTI chroma offset. When it does not exist, `colour_transform_chroma_offset` is presumed to be equal to 0. The length of `colour_transform_chroma_offset` is 2 + bitDepth - log2numLutPoints bits.

[0298] For c=0..2, the remapping process from the input image component rec[c] (where the width and height are equal to picWidth[c] and picHeight[c] respectively) to the output remapped image component map[c] is performed as follows.

[0299] The array pivotPointX is derived as follows.

[0300] — For j = 0 ..( numLutPoints - 1 ), pivotPointX[ j ] is set to equal to ( j < <log2distX )。

[0301] For c = 0..maxIntraComp, the arrays pivotPointY[c] and slope[c] are derived as follows:

[0302] — pivotPointY[c][0] is set to equal colour_transf_lut[c][0]

[0303] — For j=1..( numLutPoints - 1 ), pivotPointY[ c ][ j ] is derived as follows:

[0304] pivotPointY[ c ][ j ] = pivotPointY[ c ][ j - 1 ] + colour_transf_lut[ c ][ j ] (68)

[0305] — For j=0..( numLutPoints - 1 ), slope[ c ][ j ] is derived as follows:

[0306] slope[ c ][ j ] = ( ( colour_transf_lut[ c ][ j + 1 ] << 11 ) + ( 1 << ( log2distX - 1 ) ) ) >> log2distX (69)

[0307] When colour_transform_cross_component_flag equals 1, for c=1..2, the arrays ccPivotPointY[c] and ccSlope[c] are derived as follows:

[0308] — If colour_transform_cross_comp_inferred_flag equals 0, then ccPivotPointY[c] is derived as follows:

[0309] — For j=0..numLutPoints, ccPivotPointY[c][j] is set to equal to (color_transf_lut[c][j] << (11 - log2distX)).

[0310] — Otherwise (colour_transform_cross_comp_inferred_flag equals 1), ccPivotPointY[c] is derived as follows:

[0311]

[0312] For c=0..maxIntraComp, the intra-component remapping process from the input sample image rec[c] to the remapped sample image map[c] is performed as follows.

[0313]

[0314] When colour_transform_cross_component_flag equals 1, for c=1..2, the cross-component remapping process from the input sample image rec[c] to the remapped sample image map[c] is performed as follows:

[0315] — The offset is set to equal to (1 << (bitDepth - 1)).

[0316] — subWc and subHc are set to equal to (picWidth[0] / picWidth[c]) and (picHeight[0] / picHeight[c]), respectively.

[0317]

[0318] 3.4.3. Characteristics of Neural Network Post-Processing Filters (SEI Messages)

[0319] 3.4.3.1. Characteristics of Neural Network Post-processing Filters and SEI Message Syntax

[0320]

[0321]

[0322]

[0323] 3.4.3.2. Characteristics of Neural Network Post-Processing Filters and Semantic Semantics of SEI Messages

[0324] The Neural Network Post-Processing Filter Characteristics (NNPFC) SEI message specifies the neural networks that can be used as post-processing filters. The Neural Network Post-Processing Filter Activation (NNPFA) SEI message is used to instruct a specific image to use a specified Neural Network Post-Processing Filter (NNPF).

[0325] To use this SEI message, the following variables need to be defined:

[0326] — The width and height of the input image, expressed in units of brightness samples, are represented here as CroppedWidth and CroppedHeight, respectively.

[0327] — The luminance sample array CroppedYPic[idx] and chrominance sample arrays CroppedCbPic[idx] and CroppedCrPic[idx] (if present) of the input image, with index idx in the range of 0 to numInputPics - 1 (inclusive).

[0328] — BitDepthY, used for the brightness sample array of the input image.

[0329] — BitDepthC, used for the chroma sample array (if any) of the input image.

[0330] — Chroma format indicator, referred to here as ChromaFormatIdc, as described in sub-entry 7.3.

[0331] — When nnpfc_auxiliary_inp_idc equals 1, the filter strength control value array StrengthControlVal[idx] must contain real numbers in the range of 0 to 1 (inclusive) for the input image whose index idx is in the range of 0 to numInputPics - 1.

[0332] The input image at index 0 corresponds to the image for which the NNPF defined by the NNPFC SEI message is activated via the NNPFA SEI message. Input images with indices i in the range of 1 to numInputPics - 1 (inclusive) are output before the input image at index i - 1.

[0333] The variables SubWidthC and SubHeightC are derived from ChromaFormatIdc, as specified in Table 2.

[0334] Note 1. More than one NNPFC SEI message can exist for the same image. When more than one NNPFC SEI message with different values ​​of nnpfc_id exists or is activated for the same image, they can have the same or different values ​​of nnpfc_purpose and nnpfc_mode_idc.

[0335] `nnpfc_purpose` indicates the purpose of the NNPF, as specified in Table 20, where `(nnpfc_purpose & bitMask)` not equal to 0 indicates that the NNPF has a purpose associated with the `bitMask` value in Table 20. When `nnpfc_purpose` is greater than 0 and `(nnpfc_purpose & bitMask)` equals 0, the purpose associated with the `bitMask` value is not applicable to the NNPF. When `nnpfc_purpose` equals 0, the NNPF can be used based on the determination of the application.

[0336] All NNPFC SEI messages within CLVS that have a specific value of nnpfc_id must have the same value of nnpfc_purpose.

[0337] In bitstreams conforming to this version of the document, the value of nnpfc_purpose must be in the range of 0 to 63 (inclusive). Values ​​of nnpfc_purpose in the range of 64 to 65,535 (inclusive) are reserved for future use by ITU-T | ISO / IEC and should not exist in bitstreams conforming to this version of the document. Decoders conforming to this version of the document must ignore NNPFC SEI messages with nnpfc_purpose in the range of 64 to 65,535 (inclusive).

[0338] Table 20 — Definition of nnpfc_purpose

[0339]

[0340] The variables ChromaUpsamplingFlag, ResolutionResamplingFlag, PictureRateUpsamplingFlag, BitDepthUpsamplingFlag, and ColourizationFlag specify whether nnpfc_purpose indicates that the purpose of NNPF includes chroma upsampling, resolution resampling, picture rate upsampling, bit depth upsampling, and colorization, respectively. These variables are derived as follows:

[0341] ChromaUpsamplingFlag = ( ( nnpfc_purpose & 0x02 ) > 0 ) ? 1 : 0

[0342] ResolutionResamplingFlag = ( ( nnpfc_purpose & 0x04 ) > 0 ) ? 1 : 0

[0343] PictureRateUpsamplingFlag = ( ( nnpfc_purpose & 0x08 ) > 0 ) ? 1 : 0(75)

[0344] BitDepthUpsamplingFlag = ( ( nnpfc_purpose & 0x10 ) > 0 ) ? 1 : 0

[0345] ColourizationFlag = ( ( nnpfc_purpose & 0x20 ) > 0 ) ? 1 : 0

[0346] Note 2. When the reserved value of nnpfc_purpose is used by ITU-T | ISO / IEC in the future, the syntax of this SEI message may be extended using the following syntax elements, which exist only if nnpfc_purpose is equal to that value or equal to any one of a set of values ​​including that value.

[0347] When ChromaFormatIdc equals 3, ChromaUpsamplingFlag must equal 0.

[0348] When ChromaUpsamplingFlag equals 1, ColourizationFlag must equal 0.

[0349] When an input image with PictureRateUpsamplingFlag equal to 1 and index 0 is associated with a Frame Encapsulation Arrangement SEI message with fp_arrangement_type equal to 5, all input images are associated with a Frame Encapsulation Arrangement SEI message with the same values ​​of fp_arrangement_type and fp_current_frame_is_frame0_flag equal to 5.

[0350] The nnpfc_id contains an identifier that can be used to identify NNPF. The value of nnpfc_id must be in the range of 0 to 2^32 - 2 (inclusive). Values ​​of nnpfc_id from 256 to 511 (inclusive) and from 2^31 to 2^32 - 2 (inclusive) are reserved for future use by ITU-T | ISO / IEC. Decoders conforming to this version of this document must ignore NNPFC SEI messages when they encounter nnpfc_id values ​​in the range of 256 to 511 (inclusive) or 2^31 to 2^32 - 2 (inclusive).

[0351] The following applies when the NNPFC SEI message is the first NNPFC SEI message in decoding order with a specific nnpfc_id value within the current CLVS:

[0352] — This SEI message specifies the underlying NNPF.

[0353] — This SEI message applies to the currently decoded image and all subsequent decoded images of the current layer in output order, until the end of the current CLVS.

[0354] An nnpfc_base_flag value of 1 indicates that the SEI message specifies the underlying NNPF. An nnpfc_base_flag value of 0 indicates that the SEI message specifies an update relative to the underlying NNPF.

[0355] The following constraints apply to the value of nnpfc_base_flag:

[0356] — When the NNPFC SEI message is the first NNPFC SEI message in decoding order with a specific nnpfc_id value within the current CLVS, the value of nnpfc_base_flag must be equal to 1.

[0357] — All NNPFC SEI messages in CLVS with a specific nnpfc_id value and an nnpfc_base_flag equal to 1 must have the same SEI payload content.

[0358] When nnpfc_base_flag equals 0, the following applies:

[0359] — This SEI message defines an update to the base NNPF with the same nnpfc_id value, in decoding order. Updates are not cumulative; rather, each update is applied to the base NNPF, which is defined by the first NNPFC SEI message with a specific nnpfc_id value within the current CLVS, in decoding order. The NNPF defined by this SEI message is obtained by applying the update defined by this SEI message relative to the base NNPF with the same nnpfc_id value.

[0360] — This SEI message applies to the currently decoded image and all subsequent decoded images in the current layer in output order, until the end of the current CLVS, or until, but not including, the decoded images in the current CLVS that are after the currently decoded image in output order and are associated with subsequent NNPFC SEI messages in decoding order having an nnpfc_base_flag of equal to 0 and that particular nnpfc_id value in the current CLVS, whichever is earlier.

[0361] When nnpfc_mode_idc equals 0, it indicates that neural network information is included in the NNPFC SEI message, and that the neural network information is in the format of an ISO / IEC 15938-17 bitstream. When nnpfc_mode_idc equals 1, it indicates that the neural network information is identified by a URI, which is indicated by an nnpfc_uri in the format of a tag URI nnpfc_tag_uri.

[0362] The value of nnpfc_mode_idc must be in the range of 0 to 255 (inclusive). Values ​​of nnpfc_mode_idc in the range of 2 to 255 (inclusive) are reserved for future use by ITU-T | ISO / IEC and should not exist in bitstreams conforming to this version of the document. Decoders conforming to this version of the document must ignore NNPFC SEI messages with nnpfc_mode_idc in the range of 2 to 255 (inclusive).

[0363] nnpfc_alignment_zero_bit_a must be equal to 0.

[0364] The nnpfc_tag_uri contains a tag URI with syntax and semantics as specified in IETF RFC 4151, which identifies the neural network used as the underlying NNPF or the updated format and associated information relative to the underlying NNPF with the same nnpfc_id value as specified by the nnpfc_uri.

[0365] Note 3. The nnpfc_tag_uri can uniquely identify the format of neural network data specified by the nnrpf_uri, without requiring a central registry.

[0366] The nnpfc_tag_uri equal to “tag:iso.org,2023:15938-17” indicates that the neural network data identified by the nnpfc_uri conforms to ISO / IEC 15938-17.

[0367] nnpfc_uri contains a URI with syntax and semantics as specified in IETF Internet Standard 66, which identifies a neural network used as the underlying NNPF or an update relative to the underlying NNPF with the same nnpfc_id value.

[0368] A value of 1 for nnpfc_property_present_flag indicates the existence of syntax elements related to filter properties (including purpose, input format, output format, and complexity). A value of 0 for nnpfc_property_present_flag indicates the absence of syntax elements related to filter properties.

[0369] When nnpfc_base_flag is equal to 1, nnpfc_property_present_flag must be equal to 1.

[0370] When nnpfc_property_present_flag equals 0, the values ​​of all syntax elements that can only exist if nnpfc_property_present_flag equals 1 are presumed to be equal to their corresponding syntax elements in the NNPFC SEI message, which contains the underlying NNPF and provides updates to that underlying NNPF.

[0371] When the NNPFC SEI message nnpfcCurr is not the first NNPFC SEI message in decoding order with a specific nnpfc_id value within the current CLVS, is not a duplicate of the first NNPFC SEI message with that specific nnpfc_id value (in which case the value of nnpfc_base_flag is equal to 0), and the value of nnpfc_property_present_flag is equal to 1, the following constraints apply:

[0372] — The value of the syntax element in the NNPFC SEI message that follows nnpfc_property_present_flag and precedes nnpfc_complexity_info_present_flag in the decoding order must be the same as the value of the corresponding syntax element in the first NNPFC SEI message in the decoding order that has that particular nnpfc_id value in the current CLVS.

[0373] — nnpfc_complexity_info_present_flag must be equal to 0, or nnpfc_complexity_info_present_flag must be equal to 1 in the first NNPFC SEI message (hereinafter referred to as nnpfcBase) in the decoding order of that particular nnpfc_id value within the current CLVS, and all of the following constraints apply:

[0374] — The nnpfc_parameter_type_idc in nnpfcCurr must be equal to the nnpfc_parameter_type_idc in nnpfcBase.

[0375] — nnpfc_log2_parameter_bit_length_minus3 in nnpfcCurr (if it exists) must be less than or equal to nnpfc_log2_parameter_bit_length_minus3 in nnpfcBase.

[0376] — If nnpfc_num_parameters_idc in nnpfcBase is equal to 0, then nnpfc_num_parameters_idc in nnpfcCurr must be equal to 0.

[0377] — Otherwise (nnpfc_num_parameters_idc in nnpfcBase is greater than 0), nnpfc_num_parameters_idc in nnpfcCurr must be greater than 0 and less than or equal to nnpfc_num_parameters_idc in nnpfcBase.

[0378] — If nnpfc_num_kmac_operations_idc in nnpfcBase is equal to 0, then nnpfc_num_kmac_operations_idc in nnpfcCurr must be equal to 0.

[0379] — Otherwise (nnpfc_num_kmac_operations_idc in nnpfcBase is greater than 0), nnpfc_num_kmac_operations_idc in nnpfcCurr must be greater than 0 and less than or equal to nnpfc_num_kmac_operations_idc in nnpfcBase.

[0380] — If nnpfc_total_kilobyte_size in nnpfcBase is equal to 0, then nnpfc_total_kilobyte_size in nnpfcCurr must be equal to 0.

[0381] — Otherwise (nnpfc_total_kilobyte_size in nnpfcBase is greater than 0), nnpfc_total_kilobyte_size in nnpfcCurr must be greater than 0 and less than or equal to nnpfc_total_kilobyte_size in nnpfcBase.

[0382] The increment of 1 in `nnpfc_num_input_pics_minus1` specifies the number of images used as input to NNPF. The value of `nnpfc_num_input_pics_minus1` must be in the range of 0 to 63 (inclusive). When `PictureRateUpsamplingFlag` is equal to 1, the value of `nnpfc_num_input_pics_minus1` must be greater than 0.

[0383] The variable numInputPics, which specifies the number of images used as input to NNPF, is derived as follows:

[0384] numInputPics = nnpfc_num_input_pics_minus1 + 1 (76)

[0385] `nnpfc_input_pic_filtering_flag[i]` equal to 1 indicates that NNPF generates the corresponding output image for the i-th input image. `nnpfc_input_pic_filtering_flag[i]` equal to 0 indicates that NNPF does not generate the corresponding output image for the i-th input image. The image generated by each NNPF is stored in the output tensor of NNPF. When `nnpfc_num_input_pics_minus1` equals 0, `nnpfc_input_pic_filtering_flag[0]` is presumed to be equal to 1. When `PictureRateUpsamplingFlag` equals 0 and `nnpfc_num_input_pics_minus1` is greater than 0, `nnpfc_input_pic_filtering_flag[i]` must be equal to 1 for at least one value of `i` in the range of 0 to `nnpfc_num_input_pics_minus1` (inclusive).

[0386] `nnpfc_absent_input_pic_zero_flag` equal to 1 indicates that NNPF expects input images that do not exist in the bitstream to be represented by an array of samples with sample values ​​equal to 0. `nnpfc_absent_input_pic_zero_flag` equal to 0 indicates that NNPF expects input image `inputPicA` that does not exist in the bitstream to be represented by input image `inputPicB`, which is closest to `inputPicA` in the output order and exists in the bitstream.

[0387] `nnpfc_out_sub_c_flag` specifies the values ​​of variables `outSubWidthC` and `outSubHeightC` when `ChromaUpsamplingFlag` equals 1. `nnpfc_out_sub_c_flag` equal to 1 specifies that both `outSubWidthC` and `outSubHeightC` equal to 1. `nnpfc_out_sub_c_flag` equal to 0 specifies that `outSubWidthC` equals 2 and `outSubHeightC` equals 1. When `ChromaFormatIdc` equals 2 and `nnpfc_out_sub_c_flag` exists, the value of `nnpfc_out_sub_c_flag` must be equal to 1.

[0388] When ColourizationFlag equals 1, nnpfc_out_colour_format_idc specifies the color format of the image generated by NNPF, thus defining the values ​​of the variables outSubWidthC and outSubHeightC. nnpfc_out_colour_format_idc equals 1 specifies a 4:2:0 color format for the image generated by NNPF, and both outSubWidthC and outSubHeightC equal 2. nnpfc_out_colour_format_idc equals 2 specifies a 4:2:2 color format for the image generated by NNPF, with outSubWidthC equal to 2 and outSubHeightC equal to 1. nnpfc_out_colour_format_idc equals 3 specifies a 4:4:4 color format for the image generated by NNPF, with both outSubWidthC and outSubHeightC equal to 1. The value of nnpfc_out_colour_format_idc should not be 0.

[0389] When both ChromaUpsamplingFlag and ColourizationFlag are equal to 0, outSubWidthC and outSubHeightC are presumed to be equal to SubWidthC and SubHeightC, respectively.

[0390] `nnpfc_pic_width_num_minus1 + 1` and `nnpfc_pic_width_denom_minus1 + 1` specify the numerator and denominator of the resampling ratio of the width of the image generated by NNPF relative to the CroppedWidth, respectively. Both `nnpfc_pic_width_num_minus1` and `nnpfc_pic_width_denom_minus1` must be in the range of 0 to 65535 (inclusive).

[0391] The value of (nnpfc_pic_width_num_minus1 + 1) ÷ (nnpfc_pic_width_denom_minus1 + 1) must be in the range of 1 ÷ 16 to 16 (inclusive). When nnpfc_pic_width_num_minus1 and nnpfc_pic_width_denom_minus1 do not exist, their values ​​are presumed to be equal to 0.

[0392] The variable nnpfcOutputPicWidth, representing the width of the luminance sample array of the image generated by NNPF, is derived as follows:

[0393] nnpfcOutputPicWidth = Ceil( CroppedWidth ( nnpfc_pic_width_num_minus1 + 1 ) ÷ ( nnpfc_pic_width_denom_minus1 + 1 ) ) (77)

[0394] The requirement for bitstream consistency is that the value of nnpfcOutputPicWidth % outSubWidthC must be equal to 0.

[0395] `nnpfc_pic_height_num_minus1 + 1` and `nnpfc_pic_height_denom_minus1 + 1` specify the numerator and denominator of the resampling ratio of the height of the image generated by NNPF relative to the CroppedHeight, respectively. Both `nnpfc_pic_height_num_minus1` and `nnpfc_pic_height_denom_minus1` must be in the range of 0 to 65535 (inclusive).

[0396] The value of (nnpfc_pic_height_num_minus1 + 1) ÷ (nnpfc_pic_height_denom_minus1 + 1) must be in the range of 1 ÷ 16 to 16 (inclusive). When nnpfc_pic_height_num_minus1 and nnpfc_pic_height_denom_minus1 do not exist, their values ​​are presumed to be equal to 0.

[0397] The variable nnpfcOutputPicHeight, representing the height of the luminance sample array of the image generated by NNPF, is derived as follows:

[0398] nnpfcOutputPicHeight = Ceil( CroppedHeight ( nnpfc_pic_height_num_minus1 + 1 ) ÷ ( nnpfc_pic_height_denom_minus1 + 1 ) ) (78)

[0399] The requirement for bitstream consistency is that the value of nnpfcOutputPicHeight % outSubHeightC must be equal to 0.

[0400] When ResolutionResamplingFlag equals 1, at least one of the following conditions must be true:

[0401] — The value of nnpfcOutputPicWidth is not equal to CroppedWidth.

[0402] — The value of nnpfcOutputPicHeight is not equal to CroppedHeight.

[0403] `nnpfc_interpolated_pics[i]` specifies the number of interpolated images generated by NNPF between the i-th and (i+1)-th input images in NNPF. The value of `nnpfc_interpolated_pics[i]` must be in the range of 0 to 63 (inclusive). When the syntax element `nnpfc_interpolated_pics[i]` exists, the value of `nnpfc_interpolated_pics[i]` must be greater than 0 for at least one `i` value in the range of 0 to `nnpfc_num_input_pics_minus1 - 1` (inclusive).

[0404] Note 4. When the NNPF PictureRateUpsamplingFlag is equal to 1 and the NNPFASEI message that activates the NNPF has an nnpfa_persistence_flag of equal to 1, the value of nnpfc_interpolated_pics[i] is greater than 0 only for a single i value in the range of 0 to numInputPics - 1 (inclusive of boundary values).

[0405] The variables NumInpPicsInOutputTensor (which specifies the number of images with corresponding input images that exist in the NNPF output tensor), InpIdx[idx] (which specifies the index of the input image in the NNPF output tensor, which is the idx-th image with corresponding input images, in reverse output order), and numPicsInOutputTensor (which specifies the total number of images in the NNPF output tensor) are derived as follows:

[0406] for( i = 0, numPicsInOutputTensor = 0; i < numInputPics; i++ )

[0407] if( nnpfc_input_pic_filtering_flag[ i ] ) {

[0408] InpIdx[numPicsInOutputTensor] = i

[0409] numPicsInOutputTensor++

[0410] } (79)

[0411] NumInpPicsInOutputTensor = numPicsInOutputTensor

[0412] if(PictureRateUpsamplingFlag)

[0413] for( i = 0; i <= numInputPics - 2; i++ )

[0414] numPicsInOutputTensor += nnpfc_interpolated_pics[ i ]

[0415] `nnpfc_component_last_flag` equal to 1 indicates that the last dimension of both the NNPF input tensor and the NNPF output tensor is used for the current channel. `nnpfc_component_last_flag` equal to 0 indicates that the third dimension of both the NNPF input tensor and the NNPF output tensor is used for the current channel.

[0416] Note 5. The first dimension in both the input and output tensors is used for batch indexing, which is a common practice in some neural network frameworks. Although the formula in the semantics of this SEI message uses the batch size corresponding to a batch index equal to 0, the batch size used as input for the neural network inference process is determined by the post-processing implementation.

[0417] Note 6. For example, when nnpfc_inp_order_idc equals 3 and nnpfc_auxiliary_inp_idc equals 1, the input tensor has 7 channels, including four luminance matrices, two chrominance matrices, and one auxiliary input matrix. In this case, the procedure DeriveInputTensors() will derive each of these 7 channels of the input tensor one by one, and when a particular channel among these channels is processed, that channel is referred to as the current channel during the procedure.

[0418] `nnpfc_inp_format_idc` indicates the method for converting sample values ​​of the input image into NNPF input values. The value of `nnpfc_inp_format_idc` must be in the range of 0 to 255 (inclusive). Values ​​of `nnpfc_inp_format_idc` in the range of 2 to 255 (inclusive) are reserved for future specification by ITU-T | ISO / IEC and should not exist in the bitstream conforming to this version of the document. Decoders conforming to this version of the document must ignore NNPFC SEI messages with `nnpfc_inp_format_idc` in the range of 2 to 255 (inclusive).

[0419] When nnpfc_inp_format_idc equals 0, the input values ​​of NNPF are real numbers, and the functions InpY() and InpC() are defined as follows:

[0420] InpY( x ) = x ÷ ( ( 1 << BitDepth Y ) - 1 ) (80)

[0421] InpC( x )= x ÷ ( ( 1 << BitDepth C ) - 1 ) (81)

[0422] When nnpfc_inp_format_idc equals 1, the input values ​​of NNPF are unsigned integers, and the functions InpY() and InpC() are defined as follows:

[0423] shiftY = BitDepth Y - inpTensorBitDepth Y

[0424] if (inpTensorBitDepth) Y >= BitDepth Y )

[0425] InpY( x ) = x << ( inpTensorBitDepth Y - BitDepth Y (82)

[0426] else

[0427] InpY( x ) = Clip3(0, ( 1 << inpTensorBitDepth Y ) - 1,

[0428] ( x + ( 1 << ( shiftY - 1 ) ) ) >> shiftY )

[0429] shiftC = BitDepth C - inpTensorBitDepth C

[0430] if (inpTensorBitDepth) C >= BitDepth C )

[0431] InpC( x ) = x << ( inpTensorBitDepth C - BitDepth C (83)

[0432] else

[0433] InpC( x ) = Clip3(0, ( 1 << inpTensorBitDepth C ) - 1,

[0434] ( x + ( 1 << ( shiftC - 1 ) ) ) >> shiftC )

[0435] The variable inpTensorBitDepthY is inferred from the syntax element nnpfc_inp_tensor_luma_bitdepth_minus8, as specified below. The variable inpTensorBitDepthC is inferred from the syntax element nnpfc_inp_tensor_chroma_bitdepth_minus8, as specified below.

[0436] A value greater than 0 for nnpfc_auxiliary_inp_idc indicates that the auxiliary input data exists in the input tensor of NNPF. A value equal to 0 for nnpfc_auxiliary_inp_idc indicates that the auxiliary input data does not exist in the input tensor. A value equal to 1 for nnpfc_auxiliary_inp_idc indicates that the auxiliary input data is derived according to Equation 95.

[0437] The value of nnpfc_auxiliary_inp_idc must be in the range of 0 to 255 (inclusive). Values ​​of nnpfc_auxiliary_inp_idc in the range of 2 to 255 (inclusive) are reserved for future use by ITU-T | ISO / IEC and should not exist in bitstreams conforming to this version of the document. Decoders conforming to this version of the document must ignore NNPFC SEI messages with nnpfc_auxiliary_inp_idc in the range of 2 to 255 (inclusive).

[0438] nnpfc_inp_order_idc indicates the method for sorting the sample array of the input image to form the input tensor of NNPF.

[0439] The value of nnpfc_inp_order_idc must be in the range of 0 to 255 (inclusive). Values ​​of nnpfc_inp_order_idc in the range of 4 to 255 (inclusive) are reserved for future use by ITU-T | ISO / IEC and should not exist in bitstreams conforming to this version of the document. Decoders conforming to this version of the document must ignore NNPFC SEI messages with nnpfc_inp_order_idc in the range of 4 to 255 (inclusive).

[0440] When ChromaFormatIdc is not equal to 1, nnpfc_inp_order_idc should not be equal to 3.

[0441] When ChromaFormatIdc equals 0, nnpfc_inp_order_idc must equal 0.

[0442] When ChromaUpsamplingFlag equals 1, nnpfc_inp_order_idc should not equal 0.

[0443] Table 21 contains informative descriptions of the nnpfc_inp_order_idc values.

[0444] Table 21 — Description of nnpfc_inp_order_idc values

[0445]

[0446] Figure 6 The diagram shows the derivation of the four luminance channels (right) from the luminance component (left) when nnpfc_inp_order_idc equals 3.

[0447] `nnpfc_inp_tensor_luma_bitdepth_minus8` plus 8 specifies the bit depth of the luminance sample values ​​in the input integer tensor. The value of `inpTensorBitDepthY` is derived as follows:

[0448] inpTensorBitDepth Y = nnpfc_inp_tensor_luma_bitdepth_minus8 + 8 (84)

[0449] The value of nnpfc_inp_tensor_luma_bitdepth_minus8 must be in the range of 0 to 24 (inclusive) as a requirement for bitstream consistency.

[0450] `nnpfc_inp_tensor_chroma_bitdepth_minus8` plus 8 specifies the bit depth of the chroma sample values ​​in the input integer tensor. The value of `inpTensorBitDepthC` is derived as follows:

[0451] inpTensorBitDepth C = nnpfc_inp_tensor_chroma_bitdepth_minus8 + 8(85)

[0452] The value of nnpfc_inp_tensor_chroma_bitdepth_minus8 must be in the range of 0 to 24 (inclusive) as a requirement for bitstream consistency.

[0453] An nnpfc_out_format_idc value equal to 0 indicates that the sample values ​​output by NNPF are real numbers, where values ​​in the range of 0 to 1 (inclusive) are linearly mapped to unsigned integer values ​​in the range of 0 to (1 << bitDepth) - 1 (–inclusive) for any desired bit depth bitDepth in subsequent post-processing or display.

[0454] When nnpfc_out_format_idc is equal to 1, it indicates that the luma sample values output by the NNPF are unsigned integers in the range from 0 to (1 << outTensorBitDepthY) - 1 (inclusive of the boundary values), and the chroma sample values output by the NNPF are unsigned integers in the range from 0 to (1 << outTensorBitDepthC) - 1 (inclusive of the boundary values).

[0455] The value of nnpfc_out_format_idc shall be in the range from 0 to 255 (inclusive of the boundary values). Values of nnpfc_out_format_idc in the range from 2 to 255 (inclusive of the boundary values) are reserved for future specification by ITU-T | ISO / IEC and shall not be present in a bitstream conforming to this version of the document. A decoder conforming to this version of the document shall ignore NNPFC SEI messages where nnpfc_out_format_idc is in the range from 2 to 255 (inclusive of the boundary values).

[0456] nnpfc_out_order_idc indicates the output order of samples generated by the NNPF.

[0457] The value of nnpfc_out_order_idc shall be in the range from 0 to 255 (inclusive of the boundary values). Values of nnpfc_out_order_idc in the range from 4 to 255 (inclusive of the boundary values) are reserved for future use by ITU-T | ISO / IEC and shall not be present in a bitstream conforming to this version of the document. A decoder conforming to this version of the document shall ignore NNPFC SEI messages where nnpfc_out_order_idc is in the range from 4 to 255 (inclusive of the boundary values).

[0458] When ChromaUpsamplingFlag is equal to 1, nnpfc_out_order_idc shall not be equal to 0 or 3.

[0459] When ColourizationFlag is equal to 1, nnpfc_out_order_idc shall not be equal to 0.

[0460] Table 22 contains an informative description of the nnpfc_out_order_idc values.

[0461] Table 22 — Description of nnpfc_out_order_idc values

[0462]

[0463] `nnpfc_out_tensor_luma_bitdepth_minus8` plus 8 specifies the bit depth of the luminance sample values ​​in the output integer tensor. The value of `nnpfc_out_tensor_luma_bitdepth_minus8` must be in the range of 0 to 24 (inclusive). The value of `outTensorBitDepthY` is derived as follows:

[0464] outTensorBitDepth Y = nnpfc_out_tensor_luma_bitdepth_minus8 + 8 (86)

[0465] `nnpfc_out_tensor_chroma_bitdepth_minus8` plus 8 specifies the bit depth of the chroma sample values ​​in the output integer tensor. The value of `nnpfc_out_tensor_chroma_bitdepth_minus8` must be in the range of 0 to 24 (inclusive). The value of `outTensorBitDepthC` is derived as follows:

[0466] outTensorBitDepth C = nnpfc_out_tensor_chroma_bitdepth_minus8 + 8(87)

[0467] When BitDepthUpsamplingFlag equals 1, the value of nnpfc_out_format_idc must equal 1, and at least one of the following conditions must be true:

[0468] — nnpfc_out_tensor_luma_bitdepth_minus8 exists, and outTensorBitDepthY is greater than BitDepthY.

[0469] — nnpfc_out_tensor_chroma_bitdepth_minus8 exists, and outTensorBitDepthC is greater than BitDepthC.

[0470] When nnpfc_inp_tensor_luma_bitdepth_minus8, nnpfc_inp_tensor_chroma_bitdepth_minus8, nnpfc_out_tensor_luma_bitdepth_minus8, and nnpfc_out_tensor_chroma_bitdepth_minus8 exist, and outTensorBitDepthY is greater than inpTensorBitDepthY, outTensorBitDepthC should not be less than inpTensorBitDepthC. When nnpfc_inp_tensor_luma_bitdepth_minus8, nnpfc_inp_tensor_chroma_bitdepth_minus8, nnpfc_out_tensor_luma_bitdepth_minus8, and nnpfc_out_tensor_chroma_bitdepth_minus8 exist, and outTensorBitDepthC is greater than inpTensorBitDepthC, outTensorBitDepthY should not be less than inpTensorBitDepthY.

[0471] `nnpfc_separate_colour_description_present_flag` equal to 1 indicates that the SEI message syntax structure specifies different combinations of color primaries, transfer characteristics, matrix coefficients, and scaling and offset values ​​applied associated with the matrix coefficients for the image generated from NNPF. `nnpfc_separate_colour_description_present_flag` equal to 0 indicates that the combination of color primaries, transfer characteristics, matrix coefficients, and scaling and offset values ​​applied associated with the matrix coefficients for the image generated from NNPF is the same as the combination implied by the VUI parameters `vui_colour_primaries`, `vui_tranfer_characteristics`, `vui_matrix_coeffs`, and `vui_full_range_flag` indicated or presumed for CLVS.

[0472] nnpfc_colour_primaries has the same semantics as the vui_colour_primaries syntax element in sub-entry 7.3, except as follows:

[0473] — nnpfc_colour_primaries specifies the primary color of the image generated by NNPF as defined in the application SEI message, rather than the primary color used for CLVS.

[0474] — When nnpfc_colour_primaries does not exist in the NNPFC SEI message, the value of nnpfc_colour_primaries is presumed to be equal to vui_colour_primaries.

[0475] nnpfc_transfer_characteristics has the same semantics as the vui_transfer_characteristics syntax element in sub-entry 7.3, except as follows:

[0476] — nnpfc_transfer_characteristics specifies the transfer characteristics of images generated by NNPF as defined in the application SEI message, rather than the transfer characteristics used for CLVS.

[0477] — When nnpfc_transfer_characteristics does not exist in the NNPFC SEI message, the value of nnpfc_transfer_characteristics is presumed to be equal to vui_transfer_characteristics.

[0478] The nnpfc_matrix_coeffs describes the equations used to derive luminance and chrominance signals from green, blue, and red, or Y, Z, and X primary colors. Its semantics apply to images generated by applying the NNPF specified in this SEI message and are identical to the semantics specified in MatrixCoefficients in Recommendation ITU-T H.273 | ISO / IEC 23091-2, where BitDepthY and BitDepthC are equal to outTensorBitDepthY and outTensorBitDepthC, respectively.

[0479] When nnpfc_matrix_coeffs does not exist in the NNPFC SEI message, the value of nnpfc_matrix_coeffs is presumed to be equal to vui_matrix_coeffs.

[0480] nnpfc_matrix_coeffs should not be equal to 0 unless both of the following conditions are true:

[0481] — nnpfc_out_tensor_chroma_bitdepth_minus8 is equal to nnpfc_out_tensor_luma_bitdepth_minus8.

[0482] — nnpfc_out_order_idc equals 2, outSubHeightC equals 1, and outSubWidthC equals 1.

[0483] nnpfc_matrix_coeffs should not be equal to 8 unless one of the following conditions is true:

[0484] — nnpfc_out_tensor_chroma_bitdepth_minus8 is equal to nnpfc_out_tensor_luma_bitdepth_minus8.

[0485] — nnpfc_out_tensor_chroma_bitdepth_minus8 equals nnpfc_out_tensor_luma_bitdepth_minus8 + 1, nnpfc_out_order_idc equals 2, outSubHeightC equals 1, and outSubWidthC equals 1.

[0486] `nnpfc_full_range_flag` indicates the scaling and offset values ​​applied in association with the matrix coefficients, as specified by `nnpfc_matrix_coeffs`. Its semantics are identical to those specified by the `VideoFullRangeFlag` parameter in Recommendation ITU-T H.273 | ISO / IEC 23091-2. When not present, the value of `nnpfc_full_range_flag` is presumed to be 0.

[0487] A value of 1 for `nnpfc_chroma_loc_info_present_flag` indicates the presence of the `nnpfc_chroma_sample_loc_type_frame` syntax element in the NNPFC SEI message. A value of 0 for `nnpfc_chroma_loc_info_present_flag` indicates the absence of the `nnpfc_chroma_sample_loc_type_frame` syntax element in the NNPFC SEI message. When `nnpfc_chroma_loc_info_present_flag` does not exist, its value is presumed to be 0. The value of `nnpfc_chroma_loc_info_present_flag` must be 0 when `ColourizationFlag` is 0 or when `nnpfc_out_colour_format_idc` is not equal to 1.

[0488] When `nnpfc_chroma_sample_loc_type_frame` is not equal to 6 and `nnpfc_out_colour_format_idc` is equal to 1, it specifies the position of the chroma samples in the output image. Figure 1 As shown. `nnpfc_chroma_sample_loc_type_frame` equal to 6 and `nnpfc_out_colour_format_idc` equal to 1 indicate that the location of the chroma sample point is unknown, unspecified, or specified by other means not specified in this document. The value of `nnpfc_chroma_sample_loc_type_frame` must be in the range of 0 to 6 (inclusive).

[0489] nnpfc_overlap indicates the horizontal and vertical sample counts of the overlap between adjacent input tensors in NNPF. The value of nnpfc_overlap must be in the range of 0 to 16383 (inclusive).

[0490] `nnpfc_constant_patch_size_flag` equal to 1 indicates that NNPF accepts the exact patch size indicated by `nnpfc_patch_width_minus1` and `nnpfc_patch_height_minus1` as input. `nnpfc_constant_patch_size_flag` equal to 0 indicates that NNPF accepts any patch size with width `inpPatchWidth` and height `inpPatchHeight` as input, such that the width of the expanded patch (i.e., the patch plus the overlapping area) is equal to `inpPatchWidth + 2`. nnpfc_overlap) is nnpfc_extended_patch_width_cd_delta_minus1 + 1 + 2 The height of the expanded patch is a positive integer multiple of nnpfc_overlap, and it is equal to inpPatchHeight + 2. nnpfc_overlap) is nnpfc_extended_patch_height_cd_delta_minus1 + 1 + 2 A positive integer multiple of nnpfc_overlap.

[0491] When nnpfc_constant_patch_size_flag equals 1, nnpfc_patch_width_minus1 incremented by 1 indicates the horizontal sample count of the patch size required for the NNPF input. The value of nnpfc_patch_width_minus1 must be in the range of 0 to Min(32766, CroppedWidth - 1) (inclusive).

[0492] When nnpfc_constant_patch_size_flag equals 1, nnpfc_patch_height_minus1 incremented by 1 indicates the vertical sample count for the patch size required for the NNPF input. The value of nnpfc_patch_height_minus1 must be in the range of 0 to Min(32766, CroppedHeight - 1) (inclusive).

[0493] When nnpfc_constant_patch_size_flag equals 0, nnpfc_extended_patch_width_cd_delta_minus1 is incremented by 1 and 2. nnpfc_overlap indicates the common divisor of all allowed values ​​for the width of the extended patch required for the NNPF input. The value of nnpfc_extended_patch_width_cd_delta_minus1 must be in the range of 0 to Min(32766, CroppedWidth - 1) (inclusive).

[0494] When nnpfc_constant_patch_size_flag equals 0, nnpfc_extended_patch_height_cd_delta_minus1 is incremented by 1 and 2. nnpfc_overlap indicates the common divisor of all allowed values ​​for the height of the extended patch required for the NNPF input. The value of nnpfc_extended_patch_height_cd_delta_minus1 must be in the range of 0 to Min(32766, CroppedHeight - 1) (inclusive).

[0495] This makes the variables inpPatchWidth and inpPatchHeight the width and height of the small block, respectively.

[0496] If nnpfc_constant_patch_size_flag equals 0, then the following applies:

[0497] — The values ​​of inpPatchWidth and inpPatchHeight are provided by external means not specified in this document or set by the post-processor itself.

[0498] — inpPatchWidth + 2 The value of nnpfc_overlap must be nnpfc_extended_patch_width_cd_delta_minus1 + 1 + 2 The value must be a positive integer multiple of nnpfc_overlap, and inpPatchWidth must be less than or equal to CroppedWidth. inpPatchHeight + 2 The value of nnpfc_overlap must be nnpfc_extended_patch_height_cd_delta_minus1 + 1 + 2 The value must be a positive integer multiple of nnpfc_overlap, and inpPatchHeight must be less than or equal to CroppedHeight.

[0499] Otherwise (nnpfc_constant_patch_size_flag equals 1), the value of inpPatchWidth is set to equal to nnpfc_patch_width_minus1 + 1, and the value of inpPatchHeight is set to equal to nnpfc_patch_height_minus1 + 1.

[0500] The variables outPatchWidth, outPatchHeight, horCScaling, verCScaling, outPatchCWidth, and outPatchCHeight are derived as follows:

[0501] outPatchWidth = (nnpfcOutputPicWidth inpPatchWidth ) / CroppedWidth (88)

[0502] outPatchHeight = (nnpfcOutputPicHeight inpPatchHeight ) / CroppedHeight (89)

[0503] horCScaling = SubWidthC / outSubWidthC (90)

[0504] verCScaling = SubHeightC / outSubHeightC (91)

[0505] outPatchCWidth = outPatchWidth horCScaling (92)

[0506] outPatchCHeight = outPatchHeight verCScaling (93)

[0507] The requirement for bitstream consistency is outPatchWidth CroppedWidth must be equal to nnpfcOutputPicWidth inpPatchWidth and outPatchHeight CroppedHeight must be equal to nnpfcOutputPicHeight inpPatchHeight.

[0508] `nnpfc_padding_type` indicates the padding process when referencing sample locations outside the boundaries of the input image, as described in Table 23. The value of `nnpfc_padding_type` must be in the range of 0 to 15 (inclusive of boundary values). Values ​​of `nnpfc_padding_type` in the range of 5 to 15 (inclusive of boundary values) are reserved for future use by ITU-T | ISO / IEC and should not exist in the bitstream conforming to this version of this document. Decoders conforming to this version of this document must ignore NNPFC SEI messages with `nnpfc_padding_type` in the range of 5 to 15 (inclusive of boundary values).

[0509] Table 23 — Informative description of nnpfc_padding_type values

[0510]

[0511] nnpfc_luma_padding_val indicates the luminance value used for padding when nnpfc_padding_type is equal to 4. The value of nnpfc_luma_padding_val must be in the range of 0 to (1 << BitDepthY) - 1 (inclusive).

[0512] nnpfc_cb_padding_val indicates the Cb value used for padding when nnpfc_padding_type is equal to 4. The value of nnpfc_cb_padding_val must be in the range of 0 to (1 << BitDepthC) - 1 (inclusive).

[0513] nnpfc_cr_padding_val indicates the Cr value used for padding when nnpfc_padding_type is equal to 4. The value of nnpfc_cr_padding_val must be in the range of 0 to (1 << BitDepthC) - 1 (inclusive).

[0514] The function InpSampleVal( y,x, picHeight, picWidth, croppedPic, cIdx ) takes as input the vertical sample position y, the horizontal sample position x, the image height picHeight, the image width picWidth, the sample array croppedPic, and the component index cIdx (luminance is 0, Cb is 1, Cr is 2). It returns the value of sampleVal as derived below:

[0515] Note 7. For the input of the function InpSampleVal(), the vertical position is listed before the horizontal position to be compatible with the input tensor conventions of some inference engines.

[0516] if( nnpfc_padding_type == 0 )

[0517] if( y < 0 | | x < 0 | | y >= picHeight | | x >= picWidth )

[0518] sampleVal = 0

[0519] else

[0520] sampleVal = croppedPic[ x ][ y ] (94)

[0521] else if( nnpfc_padding_type == 1 )

[0522] sampleVal = croppedPic[ Clip3( 0, picWidth - 1, x ) ][ Clip3( 0,picHeight - 1, y ) ]

[0523] else if( nnpfc_padding_type == 2 )

[0524] sampleVal = croppedPic[ Reflect( picWidth - 1, x ) ][ Reflect(picHeight - 1, y ) ]

[0525] else if( nnpfc_padding_type == 3 ) {

[0526] if ( y >= 0 && y < picHeight )

[0527] sampleVal = croppedPic[ Wrap( picWidth - 1, x ) ][ y ]

[0528] } else if( nnpfc_padding_type == 4 )

[0529] if( y < 0 || x < 0 || y >= picHeight || x >= picWidth )

[0530] sampleVal = ( cIdx == 0 ? nnpfc_luma_padding_val :

[0531] ( cIdx == 1 ? nnpfc_cb_padding_val : nnpfc_cr_padding_val ) )

[0532] else

[0533] sampleVal = croppedPic[ x ][ y ]

[0534] When nnpfc_auxiliary_inp_idc is equal to 1, the variable strengthControlScaledVal is derived as follows:

[0535] for( i = 0; i < numInputPics; i++ )

[0536] if( nnpfc_inp_format_idc == 1 ) (95)

[0537] if( nnpfc_inp_order_idc == 0 || nnpfc_inp_order_idc == 2 ||

[0538] nnpfc_inp_order_idc == 3 )

[0539] strengthControlScaledVal[ i ] =

[0540] Floor ( StrengthControlVal[ i ] ( ( 1 << inpTensorBitDepth Y ) -1 ) )

[0541] else if( nnpfc_inp_order_idc == 1 )

[0542] strengthControlScaledVal[ i ] =

[0543] Floor (StrengthControlVal[i] ( ( 1 << inpTensorBitDepth C ) -1 ) )

[0544] else

[0545] strengthControlScaledVal[ i ] = StrengthControlVal[ i ]

[0546] Small blocks are rectangular arrays of samples from the components of an image (e.g., luminance or chrominance components).

[0547] The process of deriving the input tensor, `DeriveInputTensors()` (where the input tensor includes the given vertical coordinates `cTop` and horizontal coordinates `cLeft` of the top-left sample point of the small sample points), is defined as follows:

[0548] for( i = 0; i < numInputPics; i++ ) {

[0549] if( nnpfc_inp_order_idc == 0 )

[0550] for( yP = -nnpfc_overlap; yP < inpPatchHeight + nnpfc_overlap; yP++)

[0551] for( xP = -nnpfc_overlap; xP < inpPatchWidth + nnpfc_overlap; xP++ ){

[0552] inpVal = InpY( InpSampleVal( cTop + yP, cLeft + xP, CroppedHeight,

[0553] CroppedWidth, CroppedYPic[ i ], 0 ) )

[0554] yPovlp = yP + nnpfc_overlap

[0555] xPovlp = xP + nnpfc_overlap

[0556] if( !nnpfc_component_last_flag )

[0557] inputTensor[ 0 ][ i ][ 0 ][ yPovlp ][ xPovlp ] = inpVal

[0558] else

[0559] inputTensor[ 0 ][ i ][ yPovlp ][ xPovlp ][ 0 ] = inpVal

[0560] if( nnpfc_auxiliary_inp_idc = = 1 )

[0561] if( !nnpfc_component_last_flag )

[0562] inputTensor[ 0 ][ i ][ 1 ][ yPovlp ][ xPovlp ] =strengthControlScaledVal[ i ]

[0563] else

[0564] inputTensor[ 0 ][ i ][ yPovlp ][ xPovlp ][ 1 ] =strengthControlScaledVal[ i ]

[0565] }

[0566] else if( nnpfc_inp_order_idc = = 1 ) (96)

[0567] for( yP = -nnpfc_overlap; yP < inpPatchHeight + nnpfc_overlap; yP++)

[0568] for( xP = -nnpfc_overlap; xP < inpPatchWidth + nnpfc_overlap; xP++ ){

[0569] inpCbVal = InpC( InpSampleVal( cTop + yP, cLeft + xP, CroppedHeight / SubHeightC,

[0570] CroppedWidth / SubWidthC, CroppedCbPic[ i ], 1 ) )

[0571] inpCrVal = InpC( InpSampleVal( cTop + yP, cLeft + xP, CroppedHeight / SubHeightC,

[0572] CroppedWidth / SubWidthC, CroppedCrPic[ i ], 2 ) )

[0573] yPovlp = yP + nnpfc_overlap

[0574] xPovlp = xP + nnpfc_overlap

[0575] if( !nnpfc_component_last_flag ) {

[0576] inputTensor[ 0 ][ i ][ 0 ][ yPovlp ][ xPovlp ] = inpCbVal

[0577] inputTensor[ 0 ][ i ][ 1 ][ yPovlp ][ xPovlp ] = inpCrVal

[0578] } else {

[0579] inputTensor[ 0 ][ i ][ yPovlp ][ xPovlp ][ 0 ] = inpCbVal

[0580] inputTensor[ 0 ][ i ][ yPovlp ][ xPovlp ][ 1 ] = inpCrVal

[0581] }

[0582] if( nnpfc_auxiliary_inp_idc = = 1 )

[0583] if( !nnpfc_component_last_flag )

[0584] inputTensor[ 0 ][ i ][ 2 ][ yPovlp ][ xPovlp ] =strengthControlScaledVal[ i ]

[0585] else

[0586] inputTensor[ 0 ][ i ][ yPovlp ][ xPovlp ][ 2 ] =strengthControlScaledVal[ i ]

[0587] }

[0588] else if( nnpfc_inp_order_idc = = 2 )

[0589] for( yP = -nnpfc_overlap; yP < inpPatchHeight + nnpfc_overlap; yP++)

[0590] for( xP = -nnpfc_overlap; xP < inpPatchWidth + nnpfc_overlap; xP++ ){

[0591] yY = cTop + yP

[0592] xY = cLeft + xP

[0593] yC = yY / SubHeightC

[0594] xC = xY / SubWidthC

[0595] inpYVal = InpY( InpSampleVal( yY, xY, CroppedHeight,

[0596] CroppedWidth, CroppedYPic[ i ], 0 ) )

[0597] inpCbVal = InpC( InpSampleVal( yC, xC, CroppedHeight / SubHeightC,

[0598] CroppedWidth / SubWidthC, CroppedCbPic[ i ], 1 ) )

[0599] inpCrVal = InpC( InpSampleVal( yC, xC, CroppedHeight / SubHeightC,

[0600] CroppedWidth / SubWidthC, CroppedCrPic[ i ], 2 ) )

[0601] yPovlp = yP + nnpfc_overlap

[0602] xPovlp = xP + nnpfc_overlap

[0603] if( !nnpfc_component_last_flag ) {

[0604] inputTensor[ 0 ][ i ][ 0 ][ yPovlp ][ xPovlp ] = inpYVal

[0605] inputTensor[ 0 ][ i ][ 1 ][ yPovlp ][ xPovlp ] = inpCbVal

[0606] inputTensor[ 0 ][ i ][ 2 ][ yPovlp ][ xPovlp ] = inpCrVal

[0607] } else {

[0608] inputTensor[ 0 ][ i ][ yPovlp ][ xPovlp ][ 0 ] = inpYVal

[0609] inputTensor[ 0 ][ i ][ yPovlp ][ xPovlp ][ 1 ] = inpCbVal

[0610] inputTensor[ 0 ][ i ][ yPovlp ][ xPovlp ][ 2 ] = inpCrVal

[0611] }

[0612] if( nnpfc_auxiliary_inp_idc = = 1 )

[0613] if( !nnpfc_component_last_flag )

[0614] inputTensor[ 0 ][ i ][ 3 ][ yPovlp ][ xPovlp ] =strengthControlScaledVal[ i ]

[0615] else

[0616] inputTensor[ 0 ][ i ][ yPovlp ][ xPovlp ][ 3 ] =strengthControlScaledVal[ i ]

[0617] }

[0618] else if( nnpfc_inp_order_idc = = 3 )

[0619] for( yP = -nnpfc_overlap; yP < inpPatchHeight + nnpfc_overlap; yP++)

[0620] for( xP = -nnpfc_overlap; xP < inpPatchWidth + nnpfc_overlap; xP++ ){

[0621] yTL = cTop + yP 2

[0622] xTL = cLeft + xP 2

[0623] yBR = yTL + 1

[0624] xBR = xTL + 1

[0625] yC = cTop / 2 + yP

[0626] xC = cLeft / 2 + xP

[0627] inpTLVal = InpY( InpSampleVal( yTL, xTL, CroppedHeight,

[0628] CroppedWidth, CroppedYPic[ i ], 0 ) )

[0629] inpTRVal = InpY( InpSampleVal( yTL, xBR, CroppedHeight,

[0630] CroppedWidth, CroppedYPic[ i ], 0 ) )

[0631] inpBLVal = InpY( InpSampleVal( yBR, xTL, CroppedHeight,

[0632] CroppedWidth, CroppedYPic[ i ], 0 ) )

[0633] inpBRVal = InpY( InpSampleVal( yBR, xBR, CroppedHeight,

[0634] CroppedWidth, CroppedYPic[ i ], 0 ) )

[0635] inpCbVal = InpC( InpSampleVal( yC, xC, CroppedHeight / 2,

[0636] CroppedWidth / 2, CroppedCbPic[ i ], 1 ) )

[0637] inpCrVal = InpC( InpSampleVal( yC, xC, CroppedHeight / 2,

[0638] CroppedWidth / 2, CroppedCrPic[ i ], 2 ) )

[0639] yPovlp = yP + nnpfc_overlap

[0640] xPovlp = xP + nnpfc_overlap

[0641] if( !nnpfc_component_last_flag ) {

[0642] inputTensor[ 0 ][ i ][ 0 ][ yPovlp ][ xPovlp ] = inpTLVal

[0643] inputTensor[ 0 ][ i ][ 1 ][ yPovlp ][ xPovlp ] = inpTRVal

[0644] inputTensor[ 0 ][ i ][ 2 ][ yPovlp ][ xPovlp ] = inpBLVal

[0645] inputTensor[ 0 ][ i ][ 3 ][ yPovlp ][ xPovlp ] = inpBRVal

[0646] inputTensor[ 0 ][ i ][ 4 ][ yPovlp ][ xPovlp ] = inpCbVal

[0647] inputTensor[ 0 ][ i ][ 5 ][ yPovlp ][ xPovlp ] = inpCrVal

[0648] } else {

[0649] inputTensor[ 0 ][ i ][ yPovlp ][ xPovlp ][ 0 ] = inpTLVal

[0650] inputTensor[ 0 ][ i ][ yPovlp ][ xPovlp ][ 1 ] = inpTRVal

[0651] inputTensor[ 0 ][ i ][ yPovlp ][ xPovlp ][ 2 ] = inpBLVal

[0652] inputTensor[ 0 ][ i ][ yPovlp ][ xPovlp ][ 3 ] = inpBRVal

[0653] inputTensor[ 0 ][ i ][ yPovlp ][ xPovlp ][ 4 ] = inpCbVal

[0654] inputTensor[ 0 ][ i ][ yPovlp ][ xPovlp ][ 5 ] = inpCrVal

[0655] }

[0656] if( nnpfc_auxiliary_inp_idc == 1 )

[0657] if(!nnpfc_component_last_flag)

[0658] inputTensor[ 0 ][ i ][ 6 ][ yPovlp ][ xPovlp ] =strengthControlScaledVal[ i ]

[0659] else

[0660] inputTensor[ 0 ][ i ][ yPovlp ][ xPovlp ][ 6 ] =strengthControlScaledVal[ i ]

[0661] }

[0662] }

[0663] The procedure StoreOutputTensors() for deriving sample values ​​from the output tensor in order to generate images for NNPF in the sample arrays FilteredYPic, FilteredCbPic, and FilteredCrPic (where the output tensor has given vertical sample coordinates cTop and horizontal sample coordinates cLeft specifying the top-left sample position of the small block of samples included in the input tensor) is defined as follows:

[0664] for( i = 0; i < numPicsInOutputTensor; i++ ) {

[0665] if( nnpfc_out_order_idc == 0 )

[0666] for( yP = 0; yP < outPatchHeight; yP++)

[0667] for( xP = 0; xP < outPatchWidth; xP++ ) {

[0668] yY = cTop outPatchHeight / inpPatchHeight + yP

[0669] xY = cLeft outPatchWidth / inpPatchWidth + xP

[0670] if ( yY < nnpfcOutputPicHeight && xY < nnpfcOutputPicWidth )

[0671] if( !nnpfc_component_last_flag )

[0672] FilteredYPic[ i ][ xY ][yY ] = outputTensor[ 0 ][ i ][ 0 ][ yP ][ xP]

[0673] else

[0674] FilteredYPic[ i ][ xY ][ yY ] = outputTensor[ 0 ][ i ][ yP ][ xP ][ 0]}

[0675] else if( nnpfc_out_order_idc = = 1 ) (97)

[0676] for( yP = 0; yP < outPatchCHeight; yP++)

[0677] for( xP = 0; xP < outPatchCWidth; xP++ ) {

[0678] xSrc = cLeft horCScaling + xP

[0679] ySrc = cTop verCScaling + yP

[0680] if ( ySrc < nnpfcOutputPicHeight / outSubHeightC &&

[0681] xSrc < nnpfcOutputPicWidth / outSubWidthC )

[0682] if( !nnpfc_component_last_flag ) {

[0683] FilteredCbPic[ i ][ xSrc ][ ySrc ] = outputTensor[ 0 ][ i ][ 0 ][ yP][ xP ]

[0684] FilteredCrPic[ i ][ xSrc ][ ySrc ] = outputTensor[ 0 ][ i ][ 1 ][ yP][ xP ]

[0685] } else {

[0686] FilteredCbPic[ i ][ xSrc ][ ySrc ] = outputTensor[ 0 ][ i ][ yP ][ xP][ 0 ]

[0687] FilteredCrPic[ i ][ xSrc ][ ySrc ] = outputTensor[ 0 ][ i ][ yP ][ xP][ 1 ]

[0688] }

[0689] }

[0690] else if( nnpfc_out_order_idc = = 2 )

[0691] for( yP = 0; yP < outPatchHeight; yP++)

[0692] for( xP = 0; xP < outPatchWidth; xP++ ) {

[0693] yY = cTop outPatchHeight / inpPatchHeight + yP

[0694] xY = cLeft outPatchWidth / inpPatchWidth + xP

[0695] yC = yY / outSubHeightC

[0696] xC = xY / outSubWidthC

[0697] yPc = ( yP / outSubHeightC ) outSubHeightC

[0698] xPc = ( xP / outSubWidthC ) outSubWidthC

[0699] if ( yY < nnpfcOutputPicHeight && xY < nnpfcOutputPicWidth )

[0700] if( !nnpfc_component_last_flag ) {

[0701] FilteredYPic[ i ][ xY ][ yY ] = outputTensor[ 0 ][ i ][ 0 ][ yP ][ xP]

[0702] FilteredCbPic[ i ][ xC ][ yC ] = outputTensor[ 0 ][ i ][ 1 ][ yPc ][xPc ]

[0703] FilteredCrPic[ i ][ xC ][ yC ] = outputTensor[ 0 ][ i ][ 2 ][ yPc ][xPc ]

[0704] } else {

[0705] FilteredYPic[ i ][ xY ][ yY ] = outputTensor[ 0 ][ i ][ yP ][ xP ][ 0]

[0706] FilteredCbPic[ i ][ xC ][ yC ] = outputTensor[ 0 ][ i ][ yPc ][ xPc ][ 1 ]

[0707] FilteredCrPic[ i ][ xC ][ yC ] = outputTensor[ 0 ][ i ][ yPc ][ xPc ][ 2 ]

[0708] }

[0709] }

[0710] else if( nnpfc_out_order_idc = = 3 )

[0711] for( yP = 0; yP < outPatchHeight; yP++ )

[0712] for( xP = 0; xP < outPatchWidth; xP++ ) {

[0713] ySrc = cTop / 2 outPatchHeight / inpPatchHeight + yP

[0714] xSrc = cLeft / 2 outPatchWidth / inpPatchWidth + xP

[0715] if ( ySrc < nnpfcOutputPicHeight / 2 &&

[0716] xSrc < nnpfcOutputPicWidth / 2 )

[0717] if( !nnpfc_component_last_flag ) {

[0718] FilteredYPic[ i ][ xSrc 2 ][ ySrc 2 ] = outputTensor[ 0 ][ i ][ 0 ][ yP ][ xP ]

[0719] FilteredYPic[ i ][ xSrc 2 + 1 ][ ySrc 2 ] = outputTensor[ 0 ][i ][ 1 ][ yP ][ xP ]

[0720] FilteredYPic[ i ][ xSrc 2 ][ ySrc 2 + 1 ] = outputTensor[ 0 ][i ][ 2 ][ yP ][ xP ]

[0721] FilteredYPic[ i ][ xSrc 2 + 1][ ySrc 2 + 1 ] = outputTensor[ 0][ i ][ 3 ][ yP ][ xP ]

[0722] FilteredCbPic[ i ][ xSrc ][ ySrc ] = outputTensor[ 0 ][ i ][ 4 ][ yP][ xP ]

[0723] FilteredCrPic[ i ][ xSrc ][ ySrc ] = outputTensor[ 0 ][ i ][ 5 ][ yP][ xP ]

[0724] } else {

[0725] FilteredYPic[ i ][ xSrc 2 ][ ySrc 2 ] = outputTensor[ 0 ][ i ][ yP ][ xP ][ 0 ]

[0726] FilteredYPic[ i ][ xSrc 2 + 1 ][ ySrc 2 ] = outputTensor[ 0 ][i ][ yP ][ xP ][ 1 ]

[0727] FilteredYPic[ i ][ xSrc 2 ][ ySrc 2 + 1 ] = outputTensor[ 0 ][i ][ yP ][ xP ][ 2 ]

[0728] FilteredYPic[ i ][ xSrc 2 + 1][ ySrc 2 + 1 ] = outputTensor[ 0][ i ][ yP ][ xP ][ 3 ]

[0729] FilteredCbPic[ i ][ xSrc ][ ySrc ] = outputTensor[ 0 ][ i ][ yP ][ xP][ 4 ]

[0730] FilteredCrPic[ i ][ xSrc ][ ySrc ] = outputTensor[ 0 ][ i ][ yP ][ xP][ 5 ]

[0731] }

[0732] }

[0733] }

[0734] The NNPF PostProcessingFilter() is a target NNPF derived from the semantics of the NNPFA SEI message. The following example procedure can be used with NNPF PostProcessingFilter() to generate (multiple) filtered and / or interpolated images in a block-by-block manner, containing Y, Cb, and Cr sample arrays FilteredYPic, FilteredCbPic, and FilteredCrPic, respectively, as indicated by nnpfc_out_order_idc:

[0735] if( nnpfc_inp_order_idc == 0 | | nnpfc_inp_order_idc == 2 )

[0736] for( cTop = 0; cTop < CroppedHeight; cTop += inpPatchHeight )

[0737] for( cLeft = 0; cLeft < CroppedWidth; cLeft += inpPatchWidth ) {

[0738] inputTensor = DeriveInputTensors( )

[0739] outputTensor = PostProcessingFilter( inputTensor )

[0740] StoreOutputTensors(outputTensor)

[0741] }

[0742] else if( nnpfc_inp_order_idc == 1 )

[0743] for( cTop = 0; cTop < CroppedHeight / SubHeightC; cTop +=inpPatchHeight )

[0744] for( cLeft = 0; cLeft < CroppedWidth / SubWidthC; cLeft +=inpPatchWidth ) { (98)

[0745] inputTensor = DeriveInputTensors( )

[0746] outputTensor = PostProcessingFilter( inputTensor )

[0747] StoreOutputTensors( outputTensor )

[0748] }

[0749] else if( nnpfc_inp_order_idc = = 3 )

[0750] for( cTop = 0; cTop < CroppedHeight; cTop += inpPatchHeight 2 )

[0751] for( cLeft = 0; cLeft < CroppedWidth; cLeft += inpPatchWidth 2 ){

[0752] inputTensor = DeriveInputTensors( )

[0753] outputTensor = PostProcessingFilter( inputTensor )

[0754] StoreOutputTensors( outputTensor )

[0755] }

[0756] The NNPF-generated image with index i contains sample arrays FilteredYPic[i], FilteredCbPic[i], and FilteredCrPic[i] (if present), derived by Equation 98. The NNPF-generated image does not include overlapping regions.

[0757] The NNPF process consists of the process defined in Equation 98, and then outputs the NNPF generated images in ascending index order. All NNPF generated images that have been interpolated by NNPF are output, and those NNPF generated images corresponding to any input image of NNPF are output according to the semantics of the NNPFA SEI message.

[0758] A value of 1 for nnpfc_complexity_info_present_flag indicates the existence of one or more syntax elements that indicate the complexity of the NNPF associated with nnpfc_id. A value of 0 for nnpfc_complexity_info_present_flag indicates the absence of a syntax element that indicates the complexity of the NNPF associated with nnpfc_id.

[0759] An `nnpfc_parameter_type_idc` value of 0 indicates that the neural network uses only integer parameters. An `nnpfc_parameter_type_idc` value of 1 indicates that the neural network can use either floating-point or integer parameters. An `nnpfc_parameter_type_idc` value of 2 indicates that the neural network uses only binary parameters. An `nnpfc_parameter_type_idc` value of 3 is reserved for future use by ITU-T | ISO / IEC and should not exist in bitstreams conforming to this version of the document. Decoders conforming to this version of the document must ignore NNPFC SEI messages with `nnpfc_parameter_type_idc` equal to 3.

[0760] The values ​​0, 1, 2, and 3 for nnpfc_log2_parameter_bit_length_minus3 indicate that the neural network does not use parameters with bit lengths greater than 8, 16, 32, and 64, respectively. When nnpfc_parameter_type_idc exists and nnpfc_log2_parameter_bit_length_minus3 does not exist, the neural network does not use parameters with a bit length greater than 1.

[0761] `nnpfc_num_parameters_idc` indicates the maximum number of neural network parameters for NNPF, in powers of 2048. `nnpfc_num_parameters_idc` equal to 0 indicates that the maximum number of neural network parameters is unknown. The value of `nnpfc_num_parameters_idc` must be in the range of 0 to 52 (inclusive). Values ​​of `nnpfc_num_parameters_idc` greater than 52 are reserved for future use by ITU-T | ISO / IEC and should not exist in bitstreams conforming to this version of this document. Decoders conforming to this version of this document must ignore NNPFC SEI messages with `nnpfc_num_parameters_idc` greater than 52.

[0762] If the value of nnpfc_num_parameters_idc is greater than 0, then the variable maxNumParameters is deduced as follows:

[0763] maxNumParameters = ( 2 048 << nnpfc_num_parameters_idc ) - 1 (99)

[0764] The requirement that the number of neural network parameters in NNPF must be less than or equal to maxNumParameters is a requirement for bitstream consistency.

[0765] A value greater than 0 for nnpfc_num_kmac_operations_idc indicates that the maximum number of multiply-accumulate operations per sample in NNPF is less than or equal to nnpfc_num_kmac_operations_idc. 1,000. nnpfc_num_kmac_operations_idc equals 0, indicating that the maximum number of multiply-accumulate operations in the network is unknown. The value of nnpfc_num_kmac_operations_idc must be in the range of 0 to 2^32 - 2 (inclusive).

[0766] `nnpfc_total_kilobyte_size` greater than 0 indicates the total size in kilobytes required to store the uncompressed parameters of the neural network. The total size in bits is equal to or greater than the sum of the bits used to store each parameter. `nnpfc_total_kilobyte_size` is the total size in bits divided by 8000 and rounded down. `nnpfc_total_kilobyte_size` equal to 0 indicates that the total size required to store the parameters of the neural network is unknown. The value of `nnpfc_total_kilobyte_size` must be in the range of 0 to 2^32 - 2 (inclusive).

[0767] A value of 0 for nnpfc_num_metadata_extension_bits indicates that nnpfc_reserved_metadata_extension does not exist. A value greater than 0 for nnpfc_num_metadata_extension_bits indicates the length of nnpfc_reserved_metadata_extension in bits.

[0768] The value of nnpfc_num_metadata_extension_bits must be in the range of 0 to 2048 (inclusive). Values ​​of nnpfc_num_metadata_extension_bits in the range of 1 to 2048 (inclusive) are reserved for future use by ITU-T | ISO / IEC and should not exist in the bitstream conforming to this version of the document. Decoders conforming to this version of the document must allow any value of nnpfc_num_metadata_extension_bits in the range of 0 to 2048 (inclusive).

[0769] The nnpfc_reserved_metadata_extension should not exist in the bitstream conforming to this version of the document. However, decoders conforming to this version of the document must ignore the presence and value of nnpfc_reserved_metadata_extension. When present, the bit length of nnpfc_reserved_metadata_extension is equal to nnpfc_num_metadata_extension_bits.

[0770] nnpfc_alignment_zero_bit_b must be equal to 0 in the bitstream conforming to this version of the document. The decoder must ignore NNPFC SEI messages where nnpfc_reserved_zero_bit_b is not equal to 0.

[0771] nnpfc_payload_byte[i] contains the i-th byte of a bitstream conforming to ISO / IEC 15938-17. For all existing byte sequences of i values, nnpfc_payload_byte[i] must be a complete bitstream conforming to ISO / IEC 15938-17.

[0772] 3.5 CICP 3rd edition

[0773] In the latest proposed CICP specification [6] (i.e., CICP version 3), two new code point identifiers have been added, called YCgCo-Re and YCgCo-Ro. Their corresponding descriptions are as follows:

[0774] Matrix coefficients

[0775] Type: Unsigned integer, enumeration.

[0776] Range: 0 to 255 (inclusive), plus associated flags.

[0777] MatrixCoefficients describes the matrix coefficients used to derive luminance and chrominance signals from the green, blue, and red, or X, Y, and Z primary colors, as specified in Table 7 and the pseudocode operations outlined below.

[0778] The VideoFullRangeFlag flag may be provided along with the code point (see below).

[0779] VideoFullRangeFlag specifies the scaling and offset values ​​applied in relation to MatrixCoefficients. When it is not present or not specified, a value of 0 is generally presumed to be the default value for the video image.

[0780] An 8-bit field should be sufficient to represent MatrixCoefficients code points.

[0781] Depending on the application and the characteristics and format of the signal, such as the combination of chroma format sampling structure and the values ​​of BitDepthY and BitDepthC, specific values ​​of MatrixCoefficients can be disabled.

[0782] The interpretation of MatrixCoefficients is specified through the following pseudocode operations. ER, EG, and EB are defined as real-valued signals in the “linear domain” based on the indicated primary color (see 8.1) before the application of transmission characteristics (see 8.2).

[0783] For the purpose of representing YZX when MatrixCoefficients equals 0, in the following descriptions of pseudocode operations (11) to (19), (27) to (29), (33) to (35) and (48) to (50), the symbols R, G and B replace X, Y and Z respectively.

[0784] The nominal peak white is defined as having an ER of 1, an EG of 1, and an EB of 1.

[0785] Nominal black is defined as having ER equal to 0, EG equal to 0, and EB equal to 0.

[0786] The transfer characteristic function is applied to parameter x by being represented by (x)'.

[0787]

[0788] E' R = (E R )' (11)

[0789] E' G = (E G (12)

[0790] E' B = (E B (13)

[0791] In this case, the ranges of E'R, E'G, and E'B are defined as follows:

[0792] If TransferCharacteristics equals 11 or 12, or TransferCharacteristics equals 13 and MatrixCoefficients is not equal to 0, then E'R, E'G, and E'B are real numbers with a range of values ​​greater than the range of 0 to 1 (including boundary values), and their ranges are not specified in this document.

[0793] Otherwise, E'R, E'G, and E'B are real numbers in the range of 0 to 1 (including boundary values).

[0794] Otherwise, E'R, E'G, and E'B are real numbers in the range of 0 to 1 (including boundary values).

[0795] Otherwise (when MatrixCoefficients equals 14 or 15), signals E'L, E'M, and E'S are determined through the following ordered steps:

[0796] a) The real-valued signals EL, EM, and ES in the “linear domain” are determined as follows:

[0797] If MatrixCoefficients equals 14, then pseudocode operations (14) through (16) apply:

[0798] E L = (1 688) E R + 2 146 E G +262 E B ) ÷ 4 096 (14)

[0799] E M = (683) E R +2 951 E G + 462 E B ) ÷ 4 096 (15)

[0800] E S = (99) E R +309 E G +3 688 E B ) ÷ 4 096 (16)

[0801] Otherwise (MatrixCoefficients equals 15), pseudocode operations (17) through (19) apply:

[0802] E L = (1 747) E R + 2 169 E G +180 E B ) ÷ 4 096 (17)

[0803] E M = (673) E R +3 029 EG +394 E B ) ÷ 4 096 (18)

[0804] E S = (50) E R + 207 E G +3 839 E B ) ÷ 4 096 (19)

[0805] b) Signals E'L, E'M, and E'S are determined by applying the transfer characteristic function according to the pseudocode operations (20) to (22):

[0806] E' L = (E L )' (20)

[0807] E' M = (E M )' (twenty one)

[0808] E' S = (E S )' (twenty two)

[0809] When MatrixCoefficients equals 0, 8, 16, or 17, the variables BitDepthRGB and MaxValRGB are derived using the following ordered steps:

[0810] a) Variable BitDepth RGB The derivation is as follows:

[0811] - If MatrixCoefficients equals 0 or 8, the following applies:

[0812] BitDepth RGB = BitDepth Y (twenty three)

[0813] - Otherwise, if MatrixCoefficients equals 16, the following applies:

[0814] BitDepth RGB = BitDepth Y + 2 (24)

[0815] - Otherwise (MatrixCoefficients equals 17), the following applies:

[0816] BitDepth RGB = BitDepth Y + 1 (25)

[0817] b) Variable MaxVal RGB The derivation is as follows:

[0818] MaxVal RGB = ( 1 << BitDepth RGB ) - 1 (26)

[0819] The interpretation of MatrixCoefficients is defined as follows.

[0820] If VideoFullRangeFlag equals 0, the following applies:

[0821] If MatrixCoefficients equals 0, 8, 16, or 17, then pseudocode operations (27) through (29) apply:

[0822] R = Clip3( 0, MaxVal RGB , ( 1 << ( BitDepth RGB - 8 ) ) (219) E' R +16 ) ) (27)

[0823] G = Clip3( 0, MaxVal RGB , ( 1 << ( BitDepth RGB - 8 ) ) (219) E' G +16 ) ) (28)

[0824] B = Clip3(0, MaxVal RGB , ( 1 << ( BitDepth RGB - 8 ) ) (219) E' B +16 ) ) (29)

[0825] Otherwise, if MatrixCoefficients equals 1, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14 or 15, then pseudocode operations (30) through (32) apply:

[0826] Y = Clip1 Y ( Round( ( 1 << (BitDepth Y - 8 ) ) (219) E' Y + 16 ) )) (30)

[0827] Cb = Clip1 C ( Round( ( 1 << (BitDepth C - 8 ) ) (224) E' PB + 128) ) ) (31)

[0828] Cr = Clip1 C ( Round( ( 1 << (BitDepth C - 8 ) ) (224) E' PR + 128) ) ) (32)

[0829] Otherwise, if MatrixCoefficients equals 2, the interpretation of the MatrixCoefficients code points is unknown or determined by the application.

[0830] Otherwise (MatrixCoefficients are not equal to 0, 1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16 or 17), the interpretation of MatrixCoefficients code points is reserved for future definition by ITU-T | ISO / IEC.

[0831] Otherwise (VideoFullRangeFlag equals 1), the following pseudocode operation applies:

[0832] If MatrixCoefficients equals 0, 8, 16, or 17, then pseudocode operations (33) through (35) apply:

[0833] R = Clip3( 0, MaxVal RGB MaxVal RGB E' R (33)

[0834] G = Clip3( 0, MaxVal RGBMaxVal RGB E' G (34)

[0835] B = Clip3(0, MaxVal RGB MaxVal RGB E' B (35)

[0836] Otherwise, if MatrixCoefficients equals 1, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14 or 15, then pseudocode operations (36) through (38) apply:

[0837] Y = Clip1 Y ( Round( ( ( 1 << BitDepth Y ) - 1 ) E' Y ) ) (36)

[0838] Cb = Clip1 C ( Round( ( ( 1 << BitDepth C ) - 1 ) E' PB + ( 1 << (BitDepth C - 1 ) ) ) ) (37)

[0839] Cr = Clip1 C ( Round( ( ( 1 << BitDepth C ) - 1 ) E' PR + ( 1 << (BitDepth C - 1 ) ) ) ) (38)

[0840] Otherwise, if MatrixCoefficients equals 2, the interpretation of the MatrixCoefficients code points is unknown or determined by the application.

[0841] Otherwise (MatrixCoefficients are not equal to 0, 1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16 or 17), the interpretation of MatrixCoefficients code points is reserved for future definition by ITU-T | ISO / IEC.

[0842] When MatrixCoefficients equals 1, 4, 5, 6, 7, 9, 10, 11, 12, or 13, the constants KR and KB are specified as follows:

[0843] If MatrixCoefficients is not equal to 12 or 13, then the constants KR and KB are specified in Table 7.

[0844] Otherwise (where MatrixCoefficients equals 12 or 13), the constants KR and KB are calculated using chromaticity coordinates (xR, yR), (xG, yG), (xB, yB), and (xW, yW), as shown in Equations 39 and 40 below. The ColourPrimaries codepoints indicate the chromaticity coordinates of the red, green, blue, and white primary colors, as specified in Table 3. In some examples, an 8-bit field can represent the ColourPrimaries codepoints.

[0845] Table 3 – Explanation of ColourPrimaries Values

[0846]

[0847]

[0848]

[0849] (39)

[0850] (40)

[0851] The values ​​of zR, zG, zB, and zW are given by pseudocode operations (41) to (44):

[0852] z R = 1 - (x) R + y R (41)

[0853] z G = 1 - (x) G + y G (42)

[0854] z B = 1 - (x) B + y B (43)

[0855] z W = 1 - (x) W + y W(44)

[0856] The variables E'Y, E'PB, and E'PR (for MatrixCoefficients not equal to 0, 8, 16, or 17) or Y, Cb, and Cr (for MatrixCoefficients equal to 0, 8, 16, or 17) are specified as follows.

[0857] If MatrixCoefficients is not equal to 0, 8, 10, 11, 13, 14, 15, 16 or 17, then pseudocode operations (45) through (47) apply:

[0858] E' Y = K R E' R + (1-K) R - K B ) E' G + K B E' B (45)

[0859] E' PB = 0.5 (E' B - E' Y ) ÷ ( 1 - K B (46)

[0860] E' PR = 0.5 (E' R - E' Y ) ÷ ( 1 - K R (47)

[0861] E'Y is a real number, with a value of 0 associated with nominal black and a value of 1 associated with nominal white. E'PB and E'PR are real numbers, with values ​​of 0 associated with both nominal black and nominal white. When TransferCharacteristics is not equal to 11 or 12, E'Y is a real number with a value in the range of 0 to 1 (inclusive of boundary values). When TransferCharacteristics is not equal to 11 or 12, E'PB and E'PR are real numbers with values ​​in the range of -0.5 to 0.5 (inclusive of boundary values). When TransferCharacteristics is equal to 11 (IEC 61966-2-4) or 12 (Recommendation ITU-R BT.1361-0 Extended Gamut System), E'Y, E'PB, and E'PR are real numbers with a wider range not specified in this document.

[0862] Otherwise, if MatrixCoefficients equals 0, then pseudocode operations (48) through (50) apply:

[0863] Y = Round( G ) (48)

[0864] Cb = Round( B ) (49)

[0865] Cr = Round( R ) (50)

[0866] Otherwise, if MatrixCoefficients equals 8 and BitDepthC equals BitDepthY, then pseudocode operations (51) through (53) apply:

[0867] Y = Round( 0.5 G + 0.25 (R + B)) (51)

[0868] Cb = Round( 0.5 G - 0.25 ( R + B ) ) + (1 << ( BitDepth C - 1 )) (52)

[0869] Cr = Round( 0.5 (R - B) + (1 << ( BitDepth) C - 1 ) ) (53)

[0870] For the purpose of the YCgCo nomenclature used in Table 7, Cb and Cr in pseudocode operations (52) and (53) can be referred to as Cg and Co, respectively. The appropriate inversions of pseudocode operations (51) to (53) are given in pseudocode operations (54) to (57):

[0871] t = Y - ( Cb - ( 1 << ( BitDepth C - 1 ) ) ) (54)

[0872] G = Clip1 Y ( Y + ( Cb - ( 1 << ( BitDepth C - 1 ) ) ) ) (55)

[0873] B = Clip1 Y( t - ( Cr - ( 1 << ( BitDepth C - 1 ) ) ) ) (56)

[0874] R = Clip1 Y ( t + ( Cr - ( 1 << ( BitDepth C - 1 ) ) ) ) (57)

[0875] Otherwise, if MatrixCoefficients equals 8, 16, or 17, then pseudocode operations (58) through (61) apply:

[0876] Cr = Round( R ) - Round( B ) + (1 << ( BitDepth C - 1 ) ) (58)

[0877] t = Round( B ) + ( ( Cr - ( 1 << ( BitDepth C - 1 ) ) ) >> 1 ) (59)

[0878] Cb = Round( G ) - t + (1 << ( BitDepth C - 1 ) ) (60)

[0879] Y = t + ( ( Cb - ( 1 << ( BitDepth C - 1 ) ) ) >> 1 ) (61)

[0880] For the purpose of the YCgCo nomenclature used in Table 7, Cb and Cr in pseudocode operations (60) and (58) can be referred to as Cg and Co, respectively. The appropriate inversions of pseudocode operations (58) to (61) are given in pseudocode operations (62) to (65):

[0881] t = Y - ( ( Cb - ( 1 << ( BitDepth C - 1 ) ) ) >> 1 ) (62)

[0882] G = Clip3( 0, MaxVal RGB , t + ( Cb - ( 1 << ( BitDepth C - 1 ) ) ) ) (63)

[0883] B = Clip3(0, MaxVal RGB , t - ( ( Cr - ( 1 << ( BitDepth C - 1 ) ) ) >>1 ) ) (64)

[0884] R = Clip3( 0, MaxVal RGB , B + ( Cr - ( 1 << ( BitDepth C - 1 ) ) ) ) (65)

[0885] Otherwise, if MatrixCoefficients equals 10 or 13, the signal E'Y is determined by applying the transfer characteristic function, as given in pseudocode operations (66) and (67), and pseudocode operations (68) through (75) apply to the specifications for signals E'PB and E'PR:

[0886] E Y = K R E R + (1 - K) R - K B ) E G + K B E B (66)

[0887] E' Y = (E Y (67)

[0888] In this case, EY is defined from the “linear domain” signals of ER, EG, and EB before the application of the transfer characteristic function, and then the transfer characteristic function is applied to generate the signal E'Y. EY and E'Y are real values, with a value of 0 associated with nominal black and a value of 1 associated with nominal white.

[0889] E' PB = (E' B - E' Y ) ÷ ( 2 N B for -N B <= E' B - E' Y <= 0 (68)

[0890] E' PB = (E' B - E'Y ) ÷ ( 2 P B for 0 < E' B - E' Y <= P B (69)

[0891] E' PR = (E' R - E' Y ) ÷ ( 2 N R for -N R <= E' R - E' Y <= 0 (70)

[0892] E' PR = (E' R - E' Y ) ÷ ( 2 P R for 0 < E' R - E' Y <= P R (71)

[0893] The constants NB, PB, NR, and PR are determined by applying the transmission characteristic function to an expression involving the constants KB and KR, as given in pseudocode operations (72) to (75):

[0894] N B = (1 - K) B (72)

[0895] P B = 1 - (K) B )' (73)

[0896] N R = (1 - K) R )' (74)

[0897] P R = 1 - (K) R )' (75)

[0898] Otherwise, if MatrixCoefficients equals 11, then pseudocode operations (76) through (78) apply:

[0899] E' Y = E' G (76)

[0900] E' PB = ( 0.986 566 E' B - E' Y ) ÷ 2.0 (77)

[0901] E' PR = ( 0.991 902 E' Y ) ÷ 2.0 (78)

[0902] In this case, for the purpose of the Y'D'ZD'X naming scheme used in Table 7, E'PB can be called D'Z, and E'PR can be called D'X.

[0903] Otherwise, if MatrixCoefficients equals 14, the following applies:

[0904] If TransferCharacteristics is not equal to 18, then pseudocode operations (79) through (81) apply:

[0905] E' Y = 0.5 (E' L + E' M (79)

[0906] E' PB = (6 610) E' L - 13 613 E' M +7 003 E' S ) ÷ 4 096(80)

[0907] E' PR = (17 933) E' L - 17 390 E' M - 543 E' S ) ÷ 4 096(81)

[0908] Otherwise, pseudocode operations (82) through (84) apply:

[0909] E' Y = 0.5 (E' L + E'M (82)

[0910] E' PB = (3625) E' L - 7 465 E' M +3 840 E' S ) ÷ 4 096(83)

[0911] E' PR = (9500) E' L - 9 212 E' M - 288 E' S ) ÷ 4 096(84)

[0912] In these cases, for the purposes of the ICTCP nomenclature used in Table 7, the E'Y, E'PB, and E'PR of pseudocode operations (79), (80), and (81) or pseudocode operations (82), (83), and (84) can be referred to as I, CT, and CP, respectively. Pseudocode operations (79), (80), and (81) are specifically designed for TransferCharacteristics equal to 16 (PQ), while pseudocode operations (82), (83), and (84) are specifically designed for TransferCharacteristics equal to 18 (HLG).

[0913] Otherwise (MatrixCoefficients equals 15), the following applies:

[0914] E' Y = (1 638) E' L +1 638 E' M +820 E' S ) ÷ 4 096 (85)

[0915] E' PB = (18 248) E' L - 19 870 E' M +1 622 E' S ) ÷ 4 096(86)

[0916] E' PR = (3300) E' L +1 463 E' M - 4 763 E' S ) ÷ 4 096(87)

[0917] In this case, for the purpose of the IPT nomenclature used in Table 7, the E'Y, E'PB and E'PR of pseudocode operations (85), (86) and (87) can be referred to as I, P and T respectively.

[0918] Table 7 – Explanation of Matrix Coefficients Values

[0919]

[0920]

[0921] Note: In previous versions of this document, IEC 61966-2-1 sYCC was identified as corresponding to MatrixCoefficients equal to 1. Further careful examination determined that this representation should correspond to MatrixCoefficients equal to 5 (which is functionally equivalent to the value 6). Therefore, this document has been revised to correct this error.

[0922] 4. Technical problems solved by the disclosed embodiments

[0923] The example design of MatrixCoefficients and its references in various video codec standards have the following problems:

[0924] First, the third edition of the CICP law added two variants of YCgCo: YCgCo-Re and YCgCo-Ro. When referencing them, certain constraints should be specified.

[0925] Second, in CICP version 3, YCgCo-R, YCgCo-Re, and YCgCo-Ro are defined based on the luminance bit depth. However, the formulas are defined based on the chrominance bit depth. This can lead to undesirable interpretations of these formulas.

[0926] Third, when referencing MatrixCoeffcients in CICP, some constraints should be set in the relevant SEI message to prevent the use of meaningless values.

[0927] Fourth, for some bit depth values, the behavior is not defined in CICP.

[0928] Fifth, the chromaticity samples in YCgCo can exceed the range in CICP.

[0929] Sixth, some parameter combinations in CICP are unreasonable.

[0930] 5. List of solutions and implementation examples

[0931] To address the aforementioned issues, the following summarized methods are disclosed. These aspects should be considered as examples for explaining general concepts, and not interpreted in a narrow sense. Furthermore, these examples can be applied individually or in any combination.

[0932] 1) To address the shortcomings of problem 1, one or more of the following aspects are specified:

[0933] a. When using YCgCo-Re or YCgCo-Ro in CICP, the video or image must be in 4:4:4 chroma format.

[0934] b. When using YCgCo-Re in CICP, the chroma bit depth must be equal to the luminance bit depth.

[0935] i. Additionally, when using YCgCo-Re in CICP, an even bit depth is required.

[0936] c. When using YCgCo-Ro in CICP, the chroma bit depth must be equal to the luminance bit depth.

[0937] i. Additionally, when using YCgCo-Ro in CICP, an odd bit depth is required.

[0938] 2) To address the shortcomings of problem 2, one or more of the following aspects are specified:

[0939] a. The bit depth and maximum sample value in the RGB domain can be based on the chromaticity bit depth in the YCgCo format.

[0940] i. When using YCgCo-R, the bit depth in the RGB domain is equal to the chroma bit depth minus one.

[0941] ii. When using YCgCo-Re, the bit depth in the RGB domain is equal to the chroma bit depth minus two.

[0942] iii. When using YCgCo-Ro, the bit depth in the RGB domain is equal to the chroma bit depth minus one.

[0943] 3) To address the shortcomings of problem 3, constraints using MatrixCoefficients can be specified for various SEI messages related to RGB or YCgCo values:

[0944] a. Constraints can be specified for RGB or YCgCo related values ​​using the MatrixCoefficients in the SEI message for film grain characteristics.

[0945] i. The matrix coefficients of film grain characteristics should not be equal to 0 unless the luminance bit depth of the film grain characteristics is equal to the chrominance bit depth of the film grain characteristics.

[0946] ii. The matrix coefficients of film grain characteristics should not be equal to 8 unless the luminance bit depth of the film grain characteristics plus one equals the chrominance bit depth of the film grain characteristics.

[0947] iii. The matrix coefficients of film grain characteristics should not be equal to the values ​​indicating YCgCo-Re, unless the luminance bit depth of the film grain characteristics is equal to the chrominance bit depth of the film grain characteristics.

[0948] 1. Alternatively, the matrix coefficients of the film grain characteristics should not be equal to the values ​​indicating YCgCo-Re, unless the luminance bit depth of the film grain characteristics is equal to the chrominance bit depth of the film grain characteristics, and both are even numbers.

[0949] iv. The matrix coefficients of film grain characteristics should not be equal to the values ​​indicating YCgCo-Ro, unless the luminance bit depth of the film grain characteristics is equal to the chrominance bit depth of the film grain characteristics, and both are odd numbers.

[0950] 1. Alternatively, the matrix coefficients of the film grain characteristics should not be equal to the values ​​indicating YCgCo-Ro, unless the luminance bit depth of the film grain characteristics is equal to the chrominance bit depth of the film grain characteristics, and both are odd numbers.

[0951] b. Constraints can be specified in the SEI message for the neural network post-processing filter characteristics, such as MatrixCoefficients, for the correlation values ​​of YCgCo-Ro and YCgCo-Re.

[0952] i. If the output tensor is not in 4:4:4 chroma format, the matrix coefficients of the neural network post-processing filter characteristics should not be equal to the values ​​indicating YCgCo-Re.

[0953] ii. If the chroma bit depth or luminance bit depth does not exist, the matrix coefficients of the neural network post-processing filter characteristics should not be equal to the value indicating YCgCo-Re.

[0954] iii. If the chroma bit depth and the luminance bit depth are not equal, the matrix coefficients of the neural network post-processing filter characteristics should not be equal to the value indicating YCgCo-Re.

[0955] iv. If the chroma bit depth or luminance bit depth is not even, the matrix coefficients of the neural network post-processing filter characteristics should not be equal to the value indicating YCgCo-Re.

[0956] v. If the output tensor is not in 4:4:4 chroma format, the matrix coefficients of the neural network post-processing filter characteristics should not be equal to the values ​​indicating YCgCo-Ro.

[0957] vi. If the chroma bit depth or luminance bit depth does not exist, the matrix coefficients of the neural network post-processing filter characteristics should not be equal to the value indicating YCgCo-Ro.

[0958] vii. If the chroma bit depth and the luminance bit depth are not equal, the matrix coefficients of the neural network post-processing filter characteristics should not be equal to the value indicating YCgCo-Ro.

[0959] viii. If the chroma bit depth or luminance bit depth is not odd, the matrix coefficients of the neural network post-processing filter characteristics should not be equal to the value indicating YCgCo-Ro.

[0960] 4) To address the shortcomings of problem 4, bit depth values ​​less than a certain positive value should not be allowed in CICP.

[0961] a. In one example, the luminance and / or chrominance bit depth should be greater than or equal to N.

[0962] i. In one example, N equals 8, so that the current conversion from the signal indicating the primary color to the luminance / chrominance signal can be applied.

[0963] ii. Alternatively, in one example, N is greater than 2 to ensure that the bit depth of the RGB channels is greater than 0.

[0964] 1. In one example, N equals 3.

[0965] b. In one example, the bit depth value of the RGB channel should be greater than or equal to N.

[0966] i. In one example, N equals 8 so that the current conversion from the signal indicating the primary color to the RGB signal can be applied.

[0967] ii. In one example, when MatrixCoefficients indicates YCgCo-Re, the luminance and / or chrominance bit depth should be greater than or equal to (N + 2).

[0968] iii. In one example, when MatrixCoefficients indicates YCgCo-Ro, the luminance and / or chrominance bit depth must be greater than or equal to (N + 1).

[0969] 5) To address the shortcomings of problem 4, alternatively, the conversion from the signal indicating the primary color to RGB or luminance / chrominance signals can be extended to support cases with bit depth values ​​less than 8 bits, similar to cases with bit depth values ​​greater than 8 bits.

[0970] a. In one example, division by a constant can be used in the conversion from a signal indicating the primary color to an RGB or luminance / chrominance signal.

[0971] i. In one example, division by (1 << 8) is used in this transformation.

[0972] 6) To address the shortcomings of problem 5, the limiting operator can be used in the RGB to YCgCo conversion to ensure that the chroma values ​​are aligned with their bit depth.

[0973] 7) To address the shortcomings of problem 5, for YCgCo with a luminance / chrominance bit depth less than the RGB bit depth,

[0974] a. In one example, the RGB bit depth plus N equals the luminance bit depth.

[0975] i. In one example, N equals 2.

[0976] ii. In one example, N equals 1.

[0977] 8) To address the shortcomings of problem 6, when MatrixCoefficients indicates YCgCo-R, full_range_flag must be equal to 1.

[0978] a. When MatrixCoefficients indicates YCgCo-Re, full_range_flag should be equal to 1.

[0979] b. When MatrixCoefficients indicates YCgCo-Ro, full_range_flag should be equal to 1.

[0980] c. Alternatively, when MatrixCoefficients indicates YCgCo, full_range_flag must be equal to 1.

[0981] 9) To address the shortcomings of problem 5, we define Round'(x) = Sign(x). The alternative rounding function Round'(x) of Ceil(x) can be used to replace the current rounding function Round(x), where Ceil(x) is the smallest integer greater than or equal to x.

[0982] a. In one example, when calculating Y, Cb, and Cr for the YCgCo transform, i.e., when MatrixCoefficients is 8 and the luminance and chrominance bit depths are equal, Round(x) is replaced by Round'(x).

[0983] b. In one example, when calculating Cb and Cr for the YCgCo transform, i.e., when MatrixCoefficients is 8 and the luminance and chrominance bit depths are equal, Round(x) is replaced by Round'(x).

[0984] c. In one example, when calculating Y and Cb for the YCgCo transform, i.e., when MatrixCoefficients is 8 and the luminance and chrominance bit depths are equal, Round(x) is replaced by Round'(x).

[0985] i. Alternatively, in one example, when calculating Y and Cr for the YCgCo transform, i.e., when MatrixCoefficients is 8 and the luminance and chrominance bit depths are equal, Round(x) is replaced by Round'(x).

[0986] d. In one example, when calculating Cr for the YCgCo transform, i.e., when MatrixCoefficients is 8 and the luminance and chrominance bit depths are equal, Round(x) is replaced by Round'(x).

[0987] i. Alternatively, in one example, when calculating Cb for the YCgCo transform, i.e., when MatrixCoefficients is 8 and the luminance and chrominance bit depths are equal, Round(x) is replaced by Round'(x).

[0988] e. Alternatively, the alternative integer function can be defined as Round'(x) = Ceil(x-0.5), and all of the above sub-items can be applied.

[0989] 10) To address the shortcomings of problem 5, Round'(x) and Floor(x) defined in the above items can be used to replace the current rounding function Round(x).

[0990] a. In one example, Round(x) is replaced by Round'(x) in the calculation of the first two color channels and by Floor(x) in the calculation of the third color channel.

[0991] i. In one example, Round(x) is replaced by Round'(x) in the calculation of Y and Cb, and by Floor(x) in the calculation of Cr.

[0992] b. In one example, Round(x) is replaced by Round'(x) in the first color channel calculation and by Floor(x) in the second two color channel calculation.

[0993] c. In one example, Round(x) is replaced by Round'(x) in the first color channel calculation and by Floor(x) in the second color channel calculation.

[0994] i. In one example, Round(x) is replaced by Round'(x) in the Cb calculation and by Floor(x) in the Cr calculation.

[0995] d. Alternatively, the alternative integer function can be defined as Round'(x) = Ceil(x-0.5), and all of the above sub-items can be applied.

[0996] 11) Round'(x) as defined in item 9 can be applied to other integer transformations, integer systems, and number systems.

[0997] a. In one example, Round'(x) can be used to avoid range overflow.

[0998] b. In one example, Round'(x) can be used with other functions that convert real numbers to integers (such as Round(x), Floor(x), Ceil(x)) to avoid range overflow or balance the error distribution.

[0999] c. In one example, Round'(x) can be used in integer transformations and / or inverse transformations.

[1000] d. In one example, Round'(x) can be used in a digital random number generation system.

[1001] e. Alternatively, the alternative integer function can be defined as Round'(x) = Ceil(x-0.5), and all of the above sub-items can be applied.

[1002] 6. Examples

[1003] The following are some example implementations of the aspects outlined in Section 5 of the previous article.

[1004] The most relevant parts that have been added are shown in bold, or the parts that have been deleted are shown in bold italics. There may be other changes that are editable in nature and therefore not shown.

[1005] 6.1 First Embodiment

[1006] This embodiment relates to most of the items 1, 1.a, 1.b, 1.bi, 1.c, and 1.ci outlined in Section 5 of the previous article.

[1007] 7.3 VUI Parameter Semantics ...

[1009] vui_matrix_coeffs describes the equations used to derive luminance and chrominance signals from green, blue, and red, or Y, Z, and X primary colors. Its semantics are identical to those specified for MatrixCoefficients in Recommendation ITU-T H.273 | ISO / IEC 23091-2.

[1010] vui_matrix_coeffs should not be equal to 0 unless both of the following conditions are true:

[1011] — BitDepth C Equal to BitDepth Y .

[1012] — ChromaFormatIdc equals 3 (4:4:4 chroma format).

[1013] The provision that vui_matrix_coeffs equals 0 under all other conditions is reserved for future use by ITU-T | ISO / IEC.

[1014] vui_matrix_coeffs should not be equal to 8 unless one of the following conditions is true:

[1015] — BitDepth C Equal to BitDepth Y ,

[1016] — BitDepth C Equal to BitDepth Y +1, and ChromaFormatIdc equals 3 (4:4:4 chroma format).

[1017] The provision that vui_matrix_coeffs equals 8 under all other conditions is reserved for future use by ITU-T | ISO / IEC.

[1018] When the vui_matrix_coeffs syntax element is not present, the value of vui_matrix_coeffs is presumed to be equal to 2 (unknown or unspecified or determined by other means not specified in this document).

[1019] vui_matrix_coeffs should not be equal to 16 (YCgCo-Re) unless all of the following conditions are true:

[1020] — BitDepth C Equal to BitDepth Y .

[1021] — (BitDepth) C & 0x01) equals 0.

[1022] — ChromaFormatIdc equals 3 (4:4:4 chroma format).

[1023] The provision that vui_matrix_coeffs equals 16 under all other conditions is reserved for future use by ITU-T | ISO / IEC.

[1024] vui_matrix_coeffs should not be equal to 17 (YCgCo-Ro) unless all of the following conditions are true:

[1025] — BitDepth C Equal to BitDepth Y .

[1026] — (BitDepth) C & 0x01) equals 1.

[1027] — ChromaFormatIdc equals 3 (4:4:4 chroma format).

[1028] The provision that vui_matrix_coeffs equals 17 under all other conditions is reserved for future use by ITU-T | ISO / IEC. ...

[1030] 6.2 Second Embodiment

[1031] This embodiment relates to most of the items 2, 2.a, 2.ai, 2.a.ii, and 2.a.iii outlined in Section 5 of the previous article.

[1032] 8.3 Matrix Coefficients ...

[1034] When MatrixCoefficients equals 0, 8, 16, or 17, the variable BitDepth... RGB and MaxVal RGB Derivation using the following ordered steps:

[1035]

[1036] BitDepth RGB = BitDepth Y (twenty three)

[1037]

[1038] BitDepth RGB = BitDepth Y

[1039]

[1040] BitDepth RGB = BitDepth C - 1

[1041]

[1042] BitDepth RGB = BitDepth Y + 2 BitDepth C – 2 (24)

[1043]

[1044] BitDepth RGB = BitDepth Y + 1 BitDepth C - 1 (25)

[1045]

[1046] MaxVal RGB = ( 1 << BitDepth RGB ) - 1 (26) ...

[1048] 6.3 Third Embodiment

[1049] This embodiment relates to most of the items 13, 3.a, 3.ai, 3.a.ii, 3.a.iii, 3.a.iii.1, 3.a.iv, 3.a.iv.1 outlined in Section 5 of the preceding article.

[1050] 8.5.2 Semantics of SEI Messages for Film Grain Characteristics ...

[1052] fg_matrix_coeffs has the same semantics as that specified for the vui_matrix_coeffs syntax element in sub-entry 7.3, except as follows:

[1053] — fg_matrix_coeffs specifies the matrix coefficients for film grain characteristics as defined in the SEI message, rather than the matrix coefficients used for CLVS.

[1054] — When fg_matrix_coeffs does not exist in the film grain property SEI message, the value of fg_matrix_coeffs is presumed to be equal to vui_matrix_coeffs.

[1055] — For the semantics of VUI parameters, the allowed values ​​of fg_matrix_coeffs are not constrained by the chroma format of the decoded video image indicated by the value of ChromaFormatIdc.

[1056] fg_matrix_coeffs should not be equal to 0 unless fg_bit_depth_luma_minus8 is equal to fg_bit_depth_chroma_minus8.

[1057] fg_matrix_coeffs should not be equal to 8 unless one of the following conditions is true:

[1058] — fg_bit_depth_chroma_minus8 is equal to fg_bit_depth_luma_minus8.

[1059] — fg_bit_depth_chroma_minus8 equals fg_bit_depth_luma_minus8 + 1. ...

[1061] 6.4 Fourth Embodiment

[1062] This embodiment relates to most of the items 3, 3.a, 3.ai, 3.ai1, 3.a.ii, 3.a.ii.1 outlined in Section 5 of the previous article.

[1063] 8.28.2.2 Characteristics of Neural Network Post-Processing Filters and Semantics of SEI Messages ...

[1065] The `nnpfc_matrix_coeffs` parameter describes the equations used to derive luminance and chrominance signals from green, blue, and red, or Y, Z, and X primary colors. Its semantics apply to images generated by applying the NNPF specified in this SEI message and are identical to the semantics specified for `MatrixCoefficients` in Recommendation ITU-T H.273 | ISO / IEC 23091-2, where `BitDepth`... Y and BitDepth C They are respectively equal to outTensorBitDepth Y and outTensorBitDepth C .

[1066] When nnpfc_matrix_coeffs does not exist in the NNPFC SEI message, the value of nnpfc_matrix_coeffs is presumed to be equal to vui_matrix_coeffs.

[1067] nnpfc_matrix_coeffs should not be equal to 16 (YCgCo-Re) unless all of the following conditions are true:

[1068] Both nnpfc_out_tensor_chroma_bitdepth_minus8 and nnpfc_out_tensor_luma_bitdepth_minus8 exist.

[1069] – nnpfc_out_tensor_chroma_bitdepth_minus8 and nnpfc_out_tensor_luma_bitdepth_minus8 are both even numbers.

[1070] – nnpfc_out_tensor_chroma_bitdepth_minus8 is equal to nnpfc_out_tensor_luma_bitdepth_minus8.

[1071] – nnpfc_out_order_idc equals 2, outSubHeightC equals 1, and outSubWidthC equals 1.

[1072] nnpfc_matrix_coeffs should not be equal to 17 (YCgCo-Ro) unless all of the following conditions are true:

[1073] Both nnpfc_out_tensor_chroma_bitdepth_minus8 and nnpfc_out_tensor_luma_bitdepth_minus8 exist.

[1074] – nnpfc_out_tensor_chroma_bitdepth_minus8 and nnpfc_out_tensor_luma_bitdepth_minus8 are both odd numbers.

[1075] – nnpfc_out_tensor_chroma_bitdepth_minus8 is equal to nnpfc_out_tensor_luma_bitdepth_minus8.

[1076] – nnpfc_out_order_idc equals 2, outSubHeightC equals 1, and outSubWidthC equals 1. ...

[1078] 6.5 Example 5

[1079] This embodiment relates to items 1, 1.a, 1.b, 1.c, 5, 5.a, and 6, which are outlined in Section 5 of the previous article. The text changes are based on [6].

[1080] 8.3 Matrix Coefficients

[1081] Type: Unsigned integer, enumeration

[1082] Range: 0 to 255 (inclusive), plus associated flags.

[1083] MatrixCoefficients describes the matrix coefficients used to derive luminance and chrominance signals from the green, blue, and red or X, Y, and Z primary colors, as specified in Table 7 above and the pseudocode operations outlined below.

[1084] The value of MatrixCoefficients should not be equal to 0, 16, or 17 unless both of the following conditions are true:

[1085] – BitDepthY equals BitDepthC.

[1086] – The image color format is 4:4:4.

[1087] The provision that MatrixCoefficients equals 0, 16, or 17 under all other conditions is reserved for future use by ITU-T | ISO / IEC.

[1088] The value of MatrixCoefficients should not be equal to 8 unless one of the following conditions is true:

[1089] – BitDepthY equals BitDepthC.

[1090] – BitDepthC equals BitDepthY + 1, and the image chroma format is 4:4:4.

[1091] The provision that MatrixCoefficients equals 8 under all other conditions is reserved for future use by ITU-T | ISO / IEC.

[1092] The VideoFullRangeFlag flag may be provided along with the code point (see below). ...

[1094] 6.6 Example 6

[1095] This embodiment relates to items 4, 4.a, and 4.a.ii, which are outlined in Section 5 of the previous article. The text changes are based on [6].

[1096] 8.3 Matrix Coefficients ...

[1098] BitDepthY should be greater than or equal to 3. The provision that BitDepthY is less than 3 is reserved for future use by ITU-T | ISO / IEC.

[1099] When MatrixCoefficients equals 0, 8, 16, or 17, the variables BitDepthRGB and MaxValRGB are derived using the following ordered steps:

[1100]

[1101] BitDepth RGB = BitDepth Y (twenty three)

[1102]

[1103] BitDepth RGB = BitDepth Y + - 2 (24)

[1104]

[1105] BitDepth RGB = BitDepth Y + - 1 (25)

[1106]

[1107] MaxVal RGB = ( 1 << BitDepth RGB ) – 1 (26)

[1108] 6.7 Example 7

[1109] This embodiment relates to items 5, 5.a, and 5.ai, which are outlined in Section 5 of the previous article. The text changes are based on [6].

[1110] 8.3 Matrix Coefficients ...

[1112]

[1113] R = Clip3( 0, MaxVal RGB , ( 1 << ( BitDepth RGB - 8 ) ) ÷ ( 1 << 8 ) (219) E' R + 16 ) ) (27)

[1114] G = Clip3( 0, MaxVal RGB , ( 1 << ( BitDepth RGB - 8 ) ) ÷ ( 1 << 8 ) (219) E' G + 16 ) ) (28)

[1115] B = Clip3(0, MaxVal RGB , ( 1 << ( BitDepth RGB - 8 ) ) ÷ ( 1 << 8 ) (219) E' B + 16 ) ) (29)

[1116]

[1117] Y = Clip1 Y ( Round( ( 1 << ( BitDepth Y - 8 ) ) ÷ ( 1 << 8 ) (219) E' Y + 16 ) ) ) (30)

[1118] Cb = Clip1 C ( Round( ( 1 << ( BitDepth C - 8 ) ) ÷ ( 1 << 8 ) (224) E' PB + 128 ) ) ) (31)

[1119] Cr = Clip1 C ( Round( ( 1 << ( BitDepth C - 8 ) ) ÷ ( 1 << 8 ) (224) E' PR + 128 ) ) ) (32) ...

[1121] 6.8 Example 8

[1122] This embodiment relates to item 6, which was outlined in section 5 of the previous article. The text changes are based on [6].

[1123] 8.3 Matrix Coefficients ...

[1125]

[1126] Y = Round( 0.5 G + 0.25 (R + B)) (51)

[1127] Cb = Clip1 C (Round( 0.5 G - 0.25 ( R + B ) ) + (1 << ( BitDepth C - 1 ) ) ) (52)

[1128] Cr = Clip1 C (Round( 0.5 (R - B) + (1 << ( BitDepth) C - 1 ) ) ) (53) ...

[1130] 6.9 Example 9

[1131] This embodiment relates to item 6, which was outlined in section 5 of the previous article. The text changes are based on [6].

[1132] 8.3 Matrix Coefficients ...

[1134] When MatrixCoefficients equals 0, 8, 16, or 17, the variable BitDepth... RGB and MaxVal RGB Derivation using the following ordered steps:

[1135]

[1136] BitDepth RGB = BitDepth Y - 2 (23) ...

[1138]

[1139] Y = Round( 0.5 2 G + 0.25 (R + B)) (51)

[1140] Cb = Round( 0.5 2 G - 0.25 ( R + B ) ) + (1 << ( BitDepth C - 1 )) (52)

[1141] Cr = Round( 0.5 2 (R - B) + (1 << ( BitDepth) C- 1 ) ) (53)

[1142] 6.10 Example 10

[1143] This embodiment relates to items 9, 9.a, which are outlined in Section 5 of the previous article. The text changes are based on [6].

[1144] 5.4 Mathematical Functions

[1145] The following mathematical functions are defined as follows:

[1146] Abs(x) = (1)

[1147] Clip1 Y ( x ) = Clip3( 0, ( 1 << BitDepth Y ) - 1, x ) (2)

[1148] BitDepth Y It is the bit depth representing the corresponding luminance and color component signals.

[1149] Clip1 C ( x ) = Clip3( 0, ( 1 << BitDepth C ) - 1, x ) (3)

[1150] BitDepth C This refers to the bit depth representing the chroma component signal C. Generally, for different chroma component signals C, the BitDepth... C They can be different, for example, for C corresponding to Cb or Cr.

[1151] Clip3(x, y, z) = (4)

[1152] Floor(x) is the largest integer less than or equal to x (5).

[1153] Ceil(x) is the smallest integer greater than or equal to x.

[1154] Ln(x) is the natural logarithm of x (6)

[1155] Log10(x) is the logarithm of x to the base 10 (7)

[1156] Round(x) = Sign(x) Floor( Abs( x ) + 0.5 ) (8)

[1157] Round'(x) = Sign(x) Ceil( Abs( x ) - 0.5 )

[1158] Sign(x) = (9)

[1159] Sqrt(x) = √x (10)

[1160] 8.3 Matrix Coefficients ...

[1162]

[1163] Y = Round'( 0.5 G + 0.25 (R + B)) (51)

[1164] Cb = Round'( 0.5 G - 0.25 ( R + B ) ) + (1 << ( BitDepth C - 1 )) (52)

[1165] Cr = Round'( 0.5 (R - B) + (1 << ( BitDepth) C - 1 ) ) (53) ...

[1167] 6.11 Example 11

[1168] This embodiment relates to items 10, 10.a, and 10.ai, which were outlined in Section 5 of the previous article. The text changes are based on [6].

[1169] 5.4 Mathematical Functions

[1170] The following mathematical functions are defined as follows:

[1171] Abs(x) = (1)

[1172] Clip1 Y ( x ) = Clip3( 0, ( 1 << BitDepth Y ) - 1, x ) (2)

[1173] BitDepth YIt is the bit depth representing the corresponding luminance and color component signals.

[1174] Clip1 C ( x ) = Clip3( 0, ( 1 << BitDepth C ) - 1, x ) (3)

[1175] BitDepth C This refers to the bit depth representing the chroma component signal C. Generally, for different chroma component signals C, the BitDepth... C They can be different, for example, for C corresponding to Cb or Cr.

[1176] Clip3(x, y, z) = (4)

[1177] Floor(x) is the largest integer less than or equal to x (5).

[1178] Ceil(x) is the smallest integer greater than or equal to x.

[1179] Ln(x) is the natural logarithm of x (6)

[1180] Log10(x) is the logarithm of x to the base 10 (7)

[1181] Round(x) = Sign(x) Floor( Abs( x ) + 0.5 ) (8)

[1182] Round'(x) = Sign(x) Ceil( Abs( x ) - 0.5 )

[1183] Sign(x) = (9)

[1184] Sqrt(x) = √x (10)

[1185] 8.3 Matrix Coefficients ...

[1187]

[1188] Y = Round'( 0.5 G + 0.25 (R + B)) (51)

[1189] Cb = Round'( 0.5 G - 0.25 ( R + B ) ) + (1 << ( BitDepth C - 1 )) (52)

[1190] Cr = Round Floor (0.5) (R - B) + (1 << ( BitDepth) C - 1 ) ) (53) ...

[1192] 7. References

[1193] [1] ISO / IEC 23091-2 (current version).

[1194] [2] ITU-T and ISO / IEC, “Advanced video coding”, Recommendation ITU-T H.264 | ISO / IEC 14496-10 (current version).

[1195] [3] ITU-T and ISO / IEC, “Efficient video coding and decoding”, Recommendation ITU-T H.265 | ISO / IEC23008-2 (current version).

[1196] [4] ITU-T and ISO / IEC, “Multifunctional video coding and decoding”, Recommendation ITU-T H.266 | ISO / IEC 23090-3 (current version).

[1197] [5] ITU-T and ISO / IEC, “Multifunctional supplemental enhancement information messages for encoding and decoding video bitstreams”, Recommendation ITU-T H.274 | ISO / IEC 23002-7 (current version).

[1198] [6] "Preliminary text of ISO / IEC FDIS 23091-2:202x video CICP (3rd edition)," ISO / IEC SC29 / WG5 N00206

[1199] Figure 7This is a block diagram illustrating an example video processing system 4000 in which various embodiments disclosed herein may be implemented. Various implementations may include some or all of the components of system 4000. System 4000 may include an input 4002 for receiving video content. The video content may be received in a raw or uncompressed format, such as 8 or 10-bit multi-component pixel values, or it may be in a compressed or encoded format. Input 4002 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces (such as Ethernet, Passive Optical Network (PON), etc.) and wireless interfaces (such as Wi-Fi or cellular interfaces).

[1200] System 4000 may include an encoding / decoding component 4004 capable of implementing the various encoding / decoding or coding methods described in this disclosure. Encoding / decoding component 4004 can reduce the average bit rate from the video input 4002 to the output of encoding / decoding component 4004 to produce an encoded / decoded representation of the video. Encoding / decoding techniques are therefore sometimes referred to as video compression or video transcoding techniques. The output of encoding / decoding component 4004 may be stored or transmitted via a communication connection such as that represented by component 4006. The bitstream (or encoded / decoded) representation of the video received at input 4002, whether stored or communicated, may be used by component 4008 to generate pixel values ​​or transmit displayable video to display interface 4010. The process of generating user-visible video from the bitstream representation is sometimes referred to as video decompression. Furthermore, although some video processing operations are referred to as “encoding / decoding” operations or tools, it should be understood that encoding / decoding tools or operations are used by the encoder, and the corresponding decoding tools or operations that inversely convert the encoding / decoding result will be performed by the decoder.

[1201] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI), or DisplayPort, etc. Examples of storage interfaces include Serial Advanced Technology Attachment (SATA), Peripheral Component Interconnect (PCI), Integrated Drive Electronic Devices (IDE), etc. The embodiments described in this disclosure can be embodied in a variety of electronic devices, such as mobile phones, laptops, smartphones, or other devices capable of performing digital data processing and / or video display.

[1202] Figure 8This is a block diagram of an example video processing apparatus 4100. Apparatus 4100 can be used to implement one or more methods described herein. Apparatus 4100 can be embodied in a smartphone, tablet, computer, Internet of Things (IoT) receiver, etc. Apparatus 4100 may include one or more processors 4102, one or more memories 4104, and video processing circuitry 4106. The processors 4102(s) may be configured to implement one or more methods described herein. The memories 4104(s) may be used to store data and code for implementing the methods and embodiments described herein. The video processing circuitry 4106 may be used to implement some embodiments described herein in hardware circuitry. In some embodiments, the video processing circuitry 4106 may be at least partially included in the processor 4102, for example, a graphics coprocessor.

[1203] Figure 9 This is a flowchart of an example method 4200 for video processing. In step 4202, method 4200 determines that the video or image must be in a 4:4:4 chroma format when using Luminosity Green Chroma Orange Chroma Representation (YCgCo-R) with even bit depth (YCgCo-Re) or YCgCo-R Odd Bit Depth (YCgCo-Ro) in Code-Independent Code Points (CICP). In step 4204, a conversion between visual media data and a bitstream is performed based on CICP. This conversion may include encoding at the encoder, decoding at the decoder, or a combination thereof.

[1204] It should be noted that method 4200 can be implemented in an apparatus for processing video data, including a processor and a non-transitory memory having instructions thereon, such as a video encoder 4400, a video decoder 4500, and / or an encoder 4600. In this case, the instructions, when executed by the processor, cause the processor to perform method 4200. Furthermore, method 4200 can be executed by a non-transitory computer-readable medium, which includes a computer program product for use by a video encoding / decoding device. The computer program product includes computer-executable instructions stored on the non-transitory computer-readable medium, causing the video encoding / decoding device to perform method 4200 when executed by a processor.

[1205] Figure 10 This is a block diagram illustrating an example video encoding / decoding system 4300 that can utilize embodiments of the present disclosure. The video encoding / decoding system 4300 may include a source device 4310 and a destination device 4320. The source device 4310 generates encoded video data, and this source device 4310 may be referred to as a video encoding device. The destination device 4320 can decode the encoded video data generated by the source device 4310, and this destination device 4320 may be referred to as a video decoding device.

[1206] Source device 4310 may include video source 4312, video encoder 4314, and input / output (I / O) interface 4316. Video source 4312 may include sources such as video capture devices, interfaces for receiving video data from video content providers, and / or computer graphics systems for generating video data, or combinations of such sources. Video data may include one or more pictures. Video encoder 4314 encodes the video data from video source 4312 to generate a bitstream. The bitstream may include a sequence of bits forming a codec representation of the video data. The bitstream may include codec pictures and associated data. Codec pictures are codec representations of pictures. Associated data may include sequence parameter sets, picture parameter sets, and other syntax structures. I / O interface 4316 may include a modulator / demodulator (modem) and / or transmitter. Encoded video data may be transmitted directly to destination device 4320 via network 4330 through I / O interface 4316. Encoded video data may also be stored on storage medium / server 4340 for access by destination device 4320.

[1207] The destination device 4320 may include an I / O interface 4326, a video decoder 4324, and a display device 4322. The I / O interface 4326 may include a receiver and / or a modem. The I / O interface 4326 may acquire encoded video data from the source device 4310 or the storage medium / server 4340. The video decoder 4324 may decode the encoded video data. The display device 4322 may display the decoded video data to a user. The display device 4322 may be integrated with the destination device 4320 or may be external to the destination device 4320, wherein the destination device 4320 may be configured to interface with an external display device.

[1208] The video encoder 4314 and the video decoder 4324 can operate according to video compression standards, such as the High Efficiency Video Codec (HEVC) standard, the Multi-Functional Video Codec (VVC) standard, and other existing and / or further standards.

[1209] Figure 11 This is a block diagram illustrating an example of a video encoder 4400, which can be... Figure 10 The system 4300 shown includes a video encoder 4314. The video encoder 4400 can be configured to perform any or all embodiments of this disclosure. The video encoder 4400 includes multiple functional components. The embodiments described in this disclosure can be shared among the various components of the video encoder 4400. In some examples, a processor can be configured to perform any or all embodiments described in this disclosure.

[1210] The functional components of the video encoder 4400 may include a segmentation unit 4401; a prediction unit 4402, which may include a mode selection unit 4403, a motion estimation unit 4404, a motion compensation unit 4405, and an intra-frame prediction unit 4406; a residual generation unit 4407; a transform processing unit 4408; a quantization unit 4409; an inverse quantization unit 4410; an inverse transform unit 4411; a reconstruction unit 4412; a buffer 4413; and an entropy coding unit 4414.

[1211] In other examples, the video encoder 4400 may include more, fewer, or different functional components. In one example, the prediction unit 4402 may include an intra-block copy (IBC) unit. The IBC unit can perform prediction in an IBC mode, where at least one reference picture is the picture containing the current video block.

[1212] Furthermore, some components such as the motion estimation unit 4404 and the motion compensation unit 4405 can be highly integrated, but for illustrative purposes, they are shown separately in the example of the video encoder 4400.

[1213] The segmentation unit 4401 can segment an image into one or more video blocks. The video encoder 4400 and the video decoder 4500 can support various video block sizes.

[1214] The mode selection unit 4403 can select one of several encoding / decoding modes (intra-frame encoding / decoding or inter-frame encoding / decoding), for example, based on error results, and provide the resulting intra-frame or inter-frame encoded / decoded block to the residual generation unit 4407 to generate residual block data, and to the reconstruction unit 4412 to reconstruct the coded block for use as a reference image. In some examples, the mode selection unit 4403 can select an intra-frame / inter-frame joint prediction (CIIP) mode, where prediction is based on inter-frame prediction signals and intra-frame prediction signals. In the case of inter-frame prediction, the mode selection unit 4403 can also select a resolution for the block based on motion vectors (e.g., sub-pixel precision or integer pixel precision).

[1215] To perform inter-frame prediction on the current video block, motion estimation unit 4404 can generate motion information for the current video block by comparing one or more reference frames from buffer 4413 with the current video block. Motion compensation unit 4405 can determine a predicted video block for the current video block based on the motion information and decoded samples of images from buffer 4413 other than the image associated with the current video block.

[1216] The motion estimation unit 4404 and the motion compensation unit 4405 can perform different operations on the current video block, for example, depending on whether the current video block is in an I-band, P-band, or B-band.

[1217] In some examples, motion estimation unit 4404 can perform unidirectional prediction on the current video block, and can search for a reference video block for the current video block in the reference images of list 0 or list 1. Motion estimation unit 4404 can then generate a reference index indicating the reference image in list 0 or list 1, where the reference image contains the reference video block and a motion vector indicating the spatial displacement between the current video block and the reference video block. Motion estimation unit 4404 can output the reference index, prediction direction indicator, and motion vector as motion information for the current video block. Motion compensation unit 4405 can generate a predicted video block for the current block based on the reference video block indicated by the motion information of the current video block.

[1218] In other examples, motion estimation unit 4404 can perform bidirectional prediction on the current video block. Motion estimation unit 4404 can search for a reference video block for the current video block in the reference images in list 0, and can also search for another reference video block for the current video block in the reference images in list 1. Motion estimation unit 4404 can then generate reference indices indicating the reference images containing the reference video blocks in lists 0 and 1, and motion vectors indicating the spatial displacement between the reference video blocks and the current video block. Motion estimation unit 4404 can output the reference index and motion vector of the current video block as motion information for the current video block. Motion compensation unit 4405 can generate a predicted video block for the current video block based on the reference video blocks indicated by the motion information of the current video block.

[1219] In some examples, the motion estimation unit 4404 can output a complete set of motion information for use in the decoder's decoding process. In some examples, the motion estimation unit 4404 may not output a complete set of motion information for the current video. Instead, the motion estimation unit 4404 can reference the motion information of another video block to transmit the motion information of the current video block via a signal. For example, the motion estimation unit 4404 may determine that the motion information of the current video block is sufficiently similar to the motion information of neighboring video blocks.

[1220] In one example, the motion estimation unit 4404 may indicate a value to the video decoder 4500 in the syntax structure associated with the current video block, wherein the value indicates that the current video block has the same motion information as another video block.

[1221] In another example, motion estimation unit 4404 may identify another video block and motion vector difference (MVD) in the syntax structure associated with the current video block. The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the indicated video block. Video decoder 4500 may use the motion vector of the indicated video block and the motion vector difference to determine the motion vector of the current video block.

[1222] As discussed above, the video encoder 4400 can transmit motion vectors via signaling in a predictive manner. Two examples of predictive signaling techniques that can be implemented by the video encoder 4400 include Advanced Motion Vector Prediction (AMVP) and Merge Pattern Signaling.

[1223] Intra-prediction unit 4406 can perform intra-prediction on the current video block. When intra-prediction unit 4406 performs intra-prediction on the current video block, it can generate prediction data for the current video block based on decoded samples of other video blocks in the same frame. The prediction data for the current video block can include the predicted video block and various syntax elements.

[1224] The residual generation unit 4407 can generate residual data for the current video block by subtracting (or more) predicted video blocks from the current video block. The residual data for the current video block may include residual video blocks corresponding to different sample components of the samples in the current video block.

[1225] In other examples, such as in skip mode, there may be no residual data for the current video block, and the residual generation unit 4407 may not perform subtraction operations.

[1226] The transform processing unit 4408 can generate one or more transform coefficient video blocks for the current video block by applying one or more transforms to the residual video blocks associated with the current video block.

[1227] After the transform processing unit 4408 generates a transform coefficient video block associated with the current video block, the quantization unit 4409 can quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values ​​associated with the current video block.

[1228] The inverse quantization unit 4410 and the inverse transform unit 4411 can apply inverse quantization and inverse transform to the transform coefficient video block respectively to reconstruct the residual video block from the transform coefficient video block. The reconstruction unit 4412 can add the reconstructed residual video block to the corresponding samples of one or more predicted video blocks generated by the prediction unit 4402 to generate a reconstructed video block associated with the current block, which is stored in the buffer 4413.

[1229] After the video block is reconstructed by reconstruction unit 4412, a loop filtering operation can be performed to reduce video block artifacts in the video block.

[1230] Entropy encoding unit 4414 can receive data from other functional components of video encoder 4400. When entropy encoding unit 4414 receives data, it can perform one or more entropy encoding operations to generate entropy-encoded data and output a bitstream including the entropy-encoded data.

[1231] Figure 12 This is a block diagram illustrating an example of a video decoder 4500, which can be... Figure 10 The system 4300 shown includes a video decoder 4324. The video decoder 4500 can be configured to perform any or all embodiments of this disclosure. In the example shown, the video decoder 4500 includes multiple functional components. The embodiments described in this disclosure can be shared among the various components of the video decoder 4500. In some examples, a processor can be configured to perform any or all embodiments described in this disclosure.

[1232] In the example shown, the video decoder 4500 includes an entropy decoding unit 4501, a motion compensation unit 4502, an intra-frame prediction unit 4503, an inverse quantization unit 4504, an inverse transform unit 4505, a reconstruction unit 4506, and a buffer 4507. In some examples, the video decoder 4500 can perform a decoding process that is generally contrasted with the encoding process described with respect to the video encoder 4400.

[1233] The entropy decoding unit 4501 can retrieve the encoded bitstream. The encoded bitstream may include entropy-encoded video data (e.g., encoded video data blocks). The entropy decoding unit 4501 can decode the entropy-encoded video data, and the motion compensation unit 4502 can determine motion information from the entropy-decoded video data, including motion vectors, motion vector precision, reference image list index, and other motion information. The motion compensation unit 4502 can determine this information, for example, by executing AMVP and Merge modes.

[1234] The motion compensation unit 4502 can generate motion compensation blocks and can perform interpolation based on an interpolation filter. The identifier of the interpolation filter to be used, with sub-pixel accuracy, can be included in the syntax element.

[1235] The motion compensation unit 4502 can use the interpolation filter used by the video encoder 4400 during the encoding of the video block to calculate the sub-integer pixels for the reference block. The motion compensation unit 4502 can determine the interpolation filter used by the video encoder 4400 based on the received syntax information, and the motion compensation unit 4502 can use the interpolation filter to generate the prediction block.

[1236] The motion compensation unit 4502 may use some syntax information to determine the size of the blocks used to encode (multiple) frames and / or (multiple) stripes of the encoded video sequence, segmentation information describing how each macroblock of the image of the encoded video sequence is segmented, a mode indicating how each segment is encoded, one or more reference frames (and a list of reference frames) for each inter-frame codec block, and other information for decoding the encoded video sequence.

[1237] Intra-prediction unit 4503 can use, for example, an intra-prediction mode received in the bitstream to form prediction blocks from spatially adjacent blocks. Inverse quantization unit 4504 inverse quantizes (i.e., dequantizes) the quantized video block coefficients provided in the bitstream and decoded by entropy decoding unit 4501. Inverse transform unit 4505 applies the inverse transform.

[1238] The reconstruction unit 4506 can sum the residual block with the corresponding predicted block generated by the motion compensation unit 4502 or the intra-frame prediction unit 4503 to form a decoded block. If necessary, a deblocking filter can also be used to filter the decoded block to remove block artifacts. The decoded video block is then stored in a buffer 4507, which provides a reference block for subsequent motion compensation / intra-frame prediction and also generates decoded video for presentation on a display device.

[1239] Figure 13 This is a schematic diagram of an example encoder 4600. Encoder 4600 is suitable for implementing VVC techniques. Encoder 4600 includes three loop filters: a deblocking filter (DF) 4602, a sample adaptive compensation (SAO) 4604, and an adaptive loop filter (ALF) 4606. Unlike DF 4602, which uses predefined filters, SAO 4604 and ALF 4606 utilize the original samples of the current image, respectively, by adding an offset and by applying a finite impulse response (FIR) filter, and by utilizing the encoded / decoded side information through signal transmission offset and filter coefficients to reduce the mean square error between the original and reconstructed samples. ALF 4606 is located in the last processing stage of each image and can be considered as a tool to attempt to capture and repair artifacts caused by previous stages.

[1240] The encoder 4600 also includes an intra-frame prediction component 4608 and a motion estimation / compensation (ME / MC) component 4610 configured to receive input video. The intra-frame prediction component 4608 is configured to perform intra-frame prediction, while the ME / MC component 4610 is configured to perform inter-frame prediction using a reference image obtained from a reference image buffer 4612. Residual blocks from inter-frame or intra-frame prediction are fed into a transform (T) component 4614 and a quantization (Q) component 4616 to generate quantized residual transform coefficients, which are then fed into an entropy encoding / decoding component 4618. The entropy encoding / decoding component 4618 entropy-encodes and decodes the prediction results and quantized transform coefficients and transmits them to a video decoder (not shown). The quantization component output from the quantization component 4616 can be fed into an inverse quantization (IQ) component 4620, an inverse transform component 4622, and a reconstruction (REC) component 4624. REC component 4624 is able to output images to DF 4602, SAO 4604 and ALF 4606 for filtering before these images are stored in reference image buffer 4612.

[1241] The following provides a list of preferred solutions as examples.

[1242] The following solutions illustrate examples of the embodiments discussed herein.

[1243] 1. A method for processing media data, comprising: determining that a video or image should be in a 4:4:4 chroma format when using a luminance-green chroma-orange chroma representation (YCgCo-R) with an even bit depth (YCgCo-Re) or an odd bit depth (YCgCo-Ro) in code-code-independent code points (CICP); and performing a conversion between visual media data and a bitstream based on CICP.

[1244] 2. The method according to Solution 1, wherein when YCgCo-Re or YCgCo-Ro is used in CICP, the chroma bit depth should be equal to the luminance bit depth.

[1245] 3. The method according to any one of solutions 1-2, wherein when YCgCo-Re is used in CICP, the bit depth is even, or wherein when YCgCo-Ro is used in CICP, the bit depth is odd.

[1246] 4. The method according to any one of solutions 1-3, wherein the bit depth and maximum sample value in the red-green-blue (RGB) domain are based on the chromaticity bit depth in YCgCo format.

[1247] 5. The method according to any one of solutions 1-4, wherein when using YCgCo-R, the bit depth in the RGB domain is equal to the chroma bit depth minus one, or wherein when using YCgCo-Re, the bit depth in the RGB domain is equal to the chroma bit depth minus two, or wherein when using YCgCo-Ro, the bit depth in the RGB domain is equal to the chroma bit depth minus one.

[1248] 6. The method according to any one of solutions 1-5, wherein the constraint of using MatrixCoefficients is specified in the Supplemental Enhancement Information (SEI) message for the value associated with RGB or YCgCo.

[1249] 7. The method according to any one of solutions 1-6, wherein the constraints of MatrixCoefficients in the SEI message of film grain characteristics are specified for the values ​​associated with RGB or YCgCo.

[1250] 8. The method according to any one of solutions 1-7, wherein the matrix coefficients of the film grain characteristics shall not be equal to 0 unless the luminance bit depth of the film grain characteristics is equal to the chrominance bit depth of the film grain characteristics, or wherein the matrix coefficients of the film grain characteristics shall not be equal to 8 unless the luminance bit depth of the film grain characteristics plus one is equal to the chrominance bit depth of the film grain characteristics.

[1251] 9. The method according to any one of solutions 1-8, wherein the matrix coefficients of the film grain characteristics shall not be equal to the value indicating YCgCo-Re unless the luminance bit depth of the film grain characteristics is equal to the chrominance bit depth of the film grain characteristics, or wherein the matrix coefficients of the film grain characteristics shall not be equal to the value indicating YCgCo-Re unless the luminance bit depth of the film grain characteristics is equal to the chrominance bit depth of the film grain characteristics, and both are even numbers.

[1252] 10. The method according to any one of solutions 1-9, wherein the matrix coefficients of the film grain characteristics shall not be equal to the values ​​indicating YCgCo-Ro, unless the luminance bit depth of the film grain characteristics is equal to the chrominance bit depth of the film grain characteristics, and both are odd, or wherein the matrix coefficients of the film grain characteristics shall not be equal to the values ​​indicating YCgCo-Ro, unless the luminance bit depth of the film grain characteristics is equal to the chrominance bit depth of the film grain characteristics, and both are odd.

[1253] 11. The method according to any one of solutions 1-10, wherein the constraints of MatrixCoefficients in the SEI message of the neural network post-processing filter characteristics are specified for the values ​​associated with YCgCo-Ro and YCgCo-Re.

[1254] 12. The method according to any one of solutions 1-11, wherein when the output tensor is not in 4:4:4 chroma format, the matrix coefficients of the neural network post-processing filter characteristics should not be equal to the value indicating YCgCo-Re, or wherein when the chroma bit depth or the luma bit depth is not present, the matrix coefficients of the neural network post-processing filter characteristics should not be equal to the value indicating YCgCo-Re.

[1255] 13. The method according to any one of solutions 1-12, wherein the matrix coefficients of the neural network post-processing filter characteristics shall not be equal to the value indicating YCgCo-Re when the chroma bit depth and the luma bit depth are not equal, or wherein the matrix coefficients of the neural network post-processing filter characteristics shall not be equal to the value indicating YCgCo-Re when the chroma bit depth or the luma bit depth is not even.

[1256] 14. The method according to any one of solutions 1-13, wherein when the output tensor is not in 4:4:4 chroma format, the matrix coefficients of the neural network post-processing filter characteristics should not be equal to the value indicating YCgCo-Ro, or wherein when the chroma bit depth or luma bit depth is not present, the matrix coefficients of the neural network post-processing filter characteristics should not be equal to the value indicating YCgCo-Ro, or wherein when the chroma bit depth and luma bit depth are not equal, the matrix coefficients of the neural network post-processing filter characteristics should not be equal to the value indicating YCgCo-Ro.

[1257] 15. The method according to any one of solutions 1-14, wherein the conversion includes encoding visual media data into a bitstream.

[1258] 16. The method according to any one of solutions 1-14, wherein the conversion includes decoding visual media data from a bitstream.

[1259] 17. An apparatus for processing video data, comprising: a processor; and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform the method of any one of solutions 1-16.

[1260] 18. A non-transitory computer-readable medium comprising a computer program product for use by a video codec apparatus, the computer program product comprising computer-executable instructions stored on the non-transitory computer-readable medium, causing the video codec apparatus to perform the method described in any one of solutions 1-16 when executed by a processor.

[1261] 19. A non-transitory computer-readable recording medium storing a bitstream of video generated by a method performed by a video processing apparatus, wherein the method comprises: determining that a video or image should be in a 4:4:4 chroma format when using a luminance-green chroma-orange chroma representation (YCgCo-R) with an even bit depth (YCgCo-Re) or an odd bit depth (YCgCo-Ro) in code-code-independent code points (CICP); and generating the bitstream based on the determination.

[1262] 20. A method for storing a bitstream of video, comprising: determining that a video or image should be in a 4:4:4 chroma format when using a luminance-green chroma-orange chroma representation (YCgCo-R) with an even bit depth (YCgCo-Re) or an odd bit depth (YCgCo-Ro) in code-code-independent code points (CICP); generating the bitstream based on the determination; and storing the bitstream in a non-transitory computer-readable recording medium.

[1263] 21. A method, apparatus or system described in this disclosure.

[1264] In the described solution, the encoder conforms to the format rules by generating a codec representation based on those rules. In the described solution, the decoder parses the syntax elements in the codec representation using known information about their presence or absence, based on the format rules, to produce the decoded video.

[1265] In this disclosure, the term "video processing" can refer to video encoding, video decoding, video compression, or video decompression. For example, a video compression algorithm can be applied during the conversion from a pixel representation of a video to a corresponding bitstream representation, and vice versa. For example, the bitstream representation of the current video block can correspond to bits at co-positions or propagated at different positions in the bitstream defined by the syntax. For example, a macroblock can be encoded based on the error residual values ​​after transformation and encoding / decoding, and can also use bits from the header and other fields in the bitstream. Furthermore, during the conversion, the decoder can resolve the bitstream based on determination, knowing whether some fields may or may not be present, as described in the solutions above. Similarly, the encoder can determine whether to include or exclude specific syntax fields, and generate the codec representation accordingly by including or excluding syntax fields from the codec representation.

[1266] The disclosed and other solutions, examples, embodiments, modules, and functional operations described herein can be implemented in digital electronic circuits, or in computer software, firmware, or hardware, including the structures disclosed herein and their structural equivalents, or in one or more combinations thereof. The disclosed and other embodiments can be implemented as one or more computer program products, i.e., one or more computer program instruction modules encoded on a computer-readable medium for execution by a data processing apparatus or for controlling the operation of the data processing apparatus. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a storage device, a material composition affecting machine-readable propagation signals, or a combination thereof. The term "data processing apparatus" includes all means, devices, and machines for processing data, including, for example, a programmable processor, a computer, or multiple processors or computers. In addition to hardware, the apparatus may also include code that creates an execution environment for an associated computer program, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, or one or more combinations thereof. Propagation signals are artificially generated signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information for transmission to a suitable receiver device.

[1267] Computer programs (also known as programs, software, software applications, scripts, or code) can be written in any programming language, including compiled or interpreted languages, and can be deployed in any form, including standalone programs or modules, components, subroutines, or other units suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple co-located files (e.g., a file storing one or more modules, subroutines, or code portions). A computer program can be deployed to execute on one computer or on multiple computers located at a single site or distributed across multiple sites and interconnected by a communications network.

[1268] The processing and logic flows described in this disclosure can be executed by one or more programmable processors that execute one or more computer programs to perform functions by manipulating input data and generating outputs. The processing and logic flows can also be executed by special-purpose logic circuitry, and the apparatus can also be implemented as special-purpose logic circuitry, such as field-programmable gate arrays (FPGAs) or application-specific integrated circuits (ASICs).

[1269] Processors suitable for executing computer programs include, for example, general-purpose and special-purpose microprocessors, and any one or more processors in any type of digital computer. Typically, the processor receives instructions and data from read-only memory or random access memory, or both. The basic components of a computer are a processor that executes instructions and one or more storage devices that store the instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks. However, a computer does not necessarily have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and storage devices, including, for example, semiconductor storage devices such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable hard disks; magneto-optical disks; and CD-ROMs and DVD-ROMs. The processor and memory may be supplemented by or incorporated into special-purpose logic circuitry.

[1270] While this disclosure contains numerous details, these details should not be construed as limiting any subject matter or the scope of the claims, but rather as descriptions of features specific to particular embodiments of this disclosure. Certain features described in the context of individual embodiments in this disclosure may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented separately in multiple embodiments, or in any suitable sub-combination. Furthermore, although features may function in certain combinations as described above, and even were originally claimed in this manner, in some cases one or more features in a claimed combination may be removed from that combination, and the claimed combination may be for sub-combinations or variations thereof.

[1271] Similarly, although operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring such operations to be performed sequentially in the particular order or sequence shown, or requiring all shown operations to be performed in order to achieve the desired result. Furthermore, the partitioning of various system components in the embodiments described in this disclosure should not be construed as requiring such partitioning in all embodiments.

[1272] Only a few implementations and examples are described, and other implementations, improvements and variations may be made based on what is described and shown in this disclosure.

[1273] When there is no intervening component other than a line, trace, or other medium between the first and second components, the first component is directly coupled to the second component. When there is an intervening component other than a line, trace, or other medium between the first and second components, the first component is indirectly coupled to the second component. The term "coupled" and its variations include direct coupling and indirect coupling. The use of the term "about" means including a range of ±10% of the following figures, unless otherwise specified.

[1274] While several embodiments have been provided in this disclosure, it should be understood that the disclosed systems and methods may be embodied in many other specific forms without departing from the spirit or scope of this disclosure. The present examples are to be considered illustrative rather than restrictive and are not intended to be limited to the details set forth herein. For example, various elements or components may be combined or integrated into another system, or certain features may be omitted or not implemented.

[1275] Furthermore, the technologies, systems, subsystems, and methods described and illustrated as discrete or separate in the various embodiments can be combined or integrated with other systems, modules, technologies, or methods without departing from the scope of this disclosure. Other items shown or discussed as couplings can be directly connected or indirectly coupled or communicated through some interface, device, or intermediate component, whether electrically, mechanically, or otherwise. Those skilled in the art can identify examples of other changes, substitutions, and modifications, which can be made without departing from the spirit and scope of this disclosure.

Claims

1. A method for processing media data, comprising: According to the rules, the conversion between visual media data and the bitstream of the visual media data is performed based on code-independent code points (CICP). The rule stipulates that when the brightness green chroma orange chroma representation (YCgCo-R) even bit depth (YCgCo-Re) or YCgCo-R odd bit depth (YCgCo-Ro) is used in the CICP, the video or image of the visual media data should be in 4:4:4 chroma format.

2. The method as described in claim 1, wherein, The rule also stipulates that when YCgCo-Re or YCgCo-Ro is used in the CICP, the chroma bit depth should be equal to the luminance bit depth.

3. The method according to any one of claims 1-2, wherein, The rule also stipulates that when YCgCo-Re is used in the CICP, the bit depth must be even.

4. The method according to any one of claims 1-3, wherein, The rule also stipulates that when YCgCo-Ro is used in the CICP, the bit depth must be odd.

5. The method according to any one of claims 1-4, wherein, The bit depth and maximum sample value in the red, green, and blue (RGB) domains are based on the chromaticity bit depth in the YCgCo format.

6. The method according to any one of claims 1-5, wherein, When using YCgCo-R, the bit depth in the RGB domain is equal to the chroma bit depth minus one, or Wherein, when using the YCgCo-Re, the bit depth in the RGB domain is equal to the chroma bit depth minus two, or When using YCgCo-Ro, the bit depth in the RGB domain is equal to the chroma bit depth minus one.

7. The method according to any one of claims 1-6, wherein, Constraints on the use of matrix coefficients are specified in the Supplemental Enhancement Information (SEI) message for values ​​related to RGB or YCgCo.

8. The method according to any one of claims 1-7, wherein, The constraints for the matrix coefficients in the SEI message specify the use of film grain characteristics for values ​​related to RGB or YCgCo.

9. The method according to any one of claims 1-8, wherein, The rule also stipulates that the matrix coefficients of film grain characteristics should not be equal to 0, unless the luminance bit depth of the film grain characteristics is equal to the chrominance bit depth of the film grain characteristics.

10. The method according to any one of claims 1-9, wherein, The rule also stipulates that the matrix coefficients of film grain characteristics should not be equal to 8, unless the luminance bit depth of the film grain characteristics plus one equals the chrominance bit depth of the film grain characteristics.

11. The method according to any one of claims 1-10, wherein, The rule also stipulates that the matrix coefficients of the film grain characteristics should not be equal to the values ​​indicating the YCgCo-Re unless the luminance bit depth of the film grain characteristics is equal to the chrominance bit depth of the film grain characteristics, or, wherein the matrix coefficients of the film grain characteristics should not be equal to the values ​​indicating the YCgCo-Re unless the luminance bit depth of the film grain characteristics is equal to the chrominance bit depth of the film grain characteristics and both are even numbers.

12. The method according to any one of claims 1-11, wherein, The rule also stipulates that the matrix coefficients of the film grain characteristics should not be equal to the values ​​indicating the YCgCo-Ro, unless the luminance bit depth of the film grain characteristics is equal to the chrominance bit depth of the film grain characteristics and both are odd numbers.

13. The method according to any one of claims 1-12, wherein, The values ​​associated with YCgCo-Ro and YCgCo-Re are constrained by the matrix coefficients in the SEI message, which is a feature of the neural network post-processing filter.

14. The method according to any one of claims 1-13, wherein, The rule also stipulates that when the output tensor is not in the 4:4:4 chroma format, the matrix coefficients of the neural network post-processing filter characteristics should not be equal to the value indicating the YCgCo-Re.

15. The method according to any one of claims 1-14, wherein, The rule also stipulates that when the chroma bit depth or luminance bit depth is absent, the matrix coefficients of the neural network post-processing filter characteristics should not be equal to the value indicating YCgCo-Re.

16. The method according to any one of claims 1-15, wherein, The rule also stipulates that when the chroma bit depth and the luminance bit depth are not equal, the matrix coefficients of the neural network post-processing filter characteristics should not be equal to the value indicating YCgCo-Re.

17. The method according to any one of claims 1-16, wherein, The rule also stipulates that when the chroma bit depth or luminance bit depth is not even, the matrix coefficients of the neural network post-processing filter characteristics should not be equal to the value indicating YCgCo-Re.

18. The method according to any one of claims 1-17, wherein, The rule also stipulates that when the output tensor is not in the 4:4:4 chroma format, the matrix coefficients of the neural network post-processing filter characteristics should not be equal to the value indicating the YCgCo-Ro.

19. The method according to any one of claims 1-18, wherein, The rule also stipulates that when the chroma bit depth or luminance bit depth is absent, the matrix coefficients of the neural network post-processing filter characteristics should not be equal to the value indicating YCgCo-Ro.

20. The method according to any one of claims 1-19, wherein, The rule also stipulates that when the chroma bit depth and the luminance bit depth are not equal, the matrix coefficients of the neural network post-processing filter characteristics should not be equal to the value indicating YCgCo-Ro.

21. The method according to any one of claims 1-20, wherein, The rule also stipulates that when the chroma bit depth or luminance bit depth is not odd, the matrix coefficients of the neural network post-processing filter characteristics should not be equal to the value indicating YCgCo-Ro.

22. The method according to any one of claims 1-21, wherein, The conversion includes encoding the visual media data into the bitstream.

23. The method according to any one of claims 1-21, wherein, The conversion includes decoding the visual media data from the bitstream.

24. An apparatus for processing video data, comprising: processor; and a non-transitory memory thereon having instructions, wherein, when executed by the processor, the instructions cause the processor to perform the method as described in any one of claims 1-23.

25. A non-transitory computer-readable medium comprising a computer program product for use by a video codec apparatus, the computer program product comprising computer-executable instructions stored on the non-transitory computer-readable medium, causing the video codec apparatus to perform the method as described in any one of claims 1-23 when executed by a processor.

26. A non-transitory computer-readable recording medium storing a bitstream of video generated by a method performed by a video processing apparatus, wherein, The method includes: When using luminance-green chroma-orange chroma representation (YCgCo-R) with even bit depth (YCgCo-Re) or odd bit depth (YCgCo-R) in codepoints independent of code (CICP), the video or image should be in 4:4:4 chroma format; and The bit stream is generated based on the determination.

27. A method for storing a video bitstream, comprising: When using luminance-green chroma-orange chroma representation (YCgCo-R) with even bit depth (YCgCo-Re) or odd bit depth (YCgCo-R) in codepoints independent of code (CICP), the video or image should be in 4:4:4 chroma format. Based on the determination, a bit stream is generated; as well as The bit stream is stored in a non-transitory computer-readable recording medium.

Images