System and method for signaling neural network post filter differential interlaying information in video coding

Abstract

An apparatus capable of being configured to perform filtering based on information included in a neural network post-filter characteristic message is disclosed. In one example, the neural network post-filter characteristic message includes a syntax element indicating a number of output pictures generated for each interlaced input picture. In one example, the neural network post-filter characteristic message includes a syntax element that specifies whether an input picture is rotated.

Description

Technical Field

[0001] The present disclosure relates to video coding, and more particularly to a method for signaling a video encoding process. Systems and methods for obtaining neural network post-filter parameter information for a given frequency domain. Background Art

[0002] Digital video functions can be incorporated into various devices, including digital televisions, laptops or desktop computers, tablet computers, digital recording devices, digital media players, video game devices, cellular phones (including so-called smart phones), medical imaging devices, etc. Digital video can be encoded according to video coding standards. Video coding standards define the format of compatible bit streams that encapsulate coded video data. A compatible bit stream is a data structure that can be received and decoded by a video decoding device to generate reconstructed video data. The video coding standard also defines a decoding process, and a decoder that follows the decoding process can be said to be a compliant decoder. Video coding standards can be combined with video compression technology. Examples of video coding standards include ISO / IEC MPEG-4Visual and ITU-TH.264 (also known as ISO / IEC MPEG-4AVC), high-efficiency video coding (HEVC), and versatile video coding (VVC). HEVC is described in the High Efficiency Video Coding of Rec.ITU-T H.265 in November 2019, which is referred to as ITU-T H.265 in this article. VVC is described in Versatile Video Coding of Rec.ITU-T H.266, April 2022, which is incorporated herein by reference and is referred to herein as ITU-T H.266. Extensions and improvements to ITU-T H.266 are currently being considered to develop the next generation of video coding standards. For example, the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG) (collectively referred to as the Joint Video Study Group (JVET)) are working on standardizing enhanced video coding techniques that go beyond the functionality of the VVC standard. The enhanced compression model 7 (ECM 7), algorithm description of the enhanced compression model 7 (ECM 7), ISO / IEC JTC1 / SC29 document: JVET-AB2025 (October 20-28, 2022, Mainz, Germany), incorporated herein by reference, describes the coding features under the joint test model study, which is a potential enhanced video coding technology that goes beyond the functionality of ITU-T H.266. It should be noted that the coding features of the ECM 7 are implemented in the ECM reference software. As used herein, the term ECM may collectively refer to the algorithms included in the ECM 7 as well as the specific implementation of the ECM reference software.

[0003] Video compression technology can reduce the data requirements for storing and transmitting video data. Video compression technology can reduce data requirements by exploiting the redundancy inherent in video sequences. Video compression technology can divide the video sequence into smaller and continuous parts (i.e., a group of pictures within a video sequence, pictures within a group of pictures, regions within pictures, sub-regions within regions, etc.). Intra-frame prediction coding techniques (e.g., spatial prediction techniques within pictures) and inter-frame prediction techniques (i.e., techniques (time) between pictures) can be used to generate the difference between the unit video data to be encoded and the reference unit of the video data. The difference can be called residual data. The residual data can be encoded as quantized transform coefficients. Syntax elements can involve residual data and reference coding units (e.g., intra-frame prediction mode indexes and motion information). The residual data and syntax elements can be entropy encoded. Entropy-encoded residual data and syntax elements can be included in the data structure that forms a compatible bitstream. Summary of the invention

[0004] In general, the present disclosure describes various techniques for encoding video data. Specifically, the present disclosure describes techniques for signaling neural network post-filter parameter information in video encoding. It should be noted that although the techniques of the present disclosure are described with respect to ITU-T H.264, ITU-T H.265, ITU-T H.266, and ECM, the techniques of the present disclosure are generally applicable to video encoding. For example, in addition to those included in ITU-T H.264, ITU-T H.265, ITU-H.266, and ECM, the encoding techniques described herein may be incorporated into video encoding systems (including video encoding systems based on future video encoding standards), including video block structures, intra-frame prediction techniques, inter-frame prediction techniques, transform techniques, filtering techniques, and / or other entropy coding techniques. Therefore, references to ITU-T H.264, ITU-T H.265, ITU-T H.266, and / or ECM are for descriptive purposes and should not be interpreted as limiting the scope of the techniques described herein. In addition, it should be noted that the incorporation of references herein is for descriptive purposes and should not be construed as limiting or creating ambiguity regarding the terms used herein. For example, where a definition of a term provided in an incorporated reference differs from that provided in another incorporated reference and / or as used herein, the term should be interpreted in a manner that broadly includes each corresponding definition and / or in a manner that includes each specific definition in alternatives.

[0005] In one example, a method of encoding video data includes signaling a neural network post filter characteristics message, signaling a first syntax element from the neural network post filter characteristics message indicating a number of output pictures generated for each interlaced input picture, and signaling a second syntax element from the neural network post filter characteristics message specifying whether the input picture is rotated.

[0006] In one example, a device includes one or more processors configured to: signal a neural network post filter characteristics message, signal a first syntax element from the neural network post filter characteristics message indicating a number of output pictures generated for each interlaced input picture, and signal a second syntax element from the neural network post filter characteristics message specifying whether the input pictures are rotated.

[0007] In one example, a non-transitory computer-readable storage medium includes instructions stored thereon that, when executed, cause one or more processors of a device to: signal a neural network post filter characteristics message, signal a first syntax element from the neural network post filter characteristics message indicating a number of output pictures generated for each interlaced input picture, and signal a second syntax element from the neural network post filter characteristics message specifying whether the input pictures are rotated.

[0008] In one example, an apparatus includes means for signaling a neural network post filter characteristics message, means for signaling a first syntax element from the neural network post filter characteristics message indicating a number of output pictures generated for each interlaced input picture, and means for signaling a second syntax element from the neural network post filter characteristics message specifying whether the input pictures are rotated.

[0009] In one example, a method of decoding video data includes receiving a neural network post filter characteristics message, parsing a first syntax element from the neural network post filter characteristics message indicating a number of output pictures generated for each interlaced input picture, and parsing a second syntax element from the neural network post filter characteristics message specifying whether the input picture is rotated.

[0010] In one example, a device includes one or more processors configured to receive a neural network post filter characteristics message, parse a first syntax element from the neural network post filter characteristics message indicating a number of output pictures generated for each interlaced input picture, and parse a second syntax element from the neural network post filter characteristics message specifying whether the input pictures are rotated.

[0011] In one example, a non-transitory computer-readable storage medium includes instructions stored thereon that, when executed, cause one or more processors of a device to: receive a neural network post filter characteristics message, parse a first syntax element from the neural network post filter characteristics message indicating a number of output pictures generated for each interlaced input picture, and parse a second syntax element from the neural network post filter characteristics message specifying whether the input picture is rotated.

[0012] In one example, an apparatus includes components for receiving a neural network post filter characteristics message, components for parsing a first syntax element from the neural network post filter characteristics message indicating a number of output pictures generated for each interlaced input picture, and components for parsing a second syntax element from the neural network post filter characteristics message specifying whether the input pictures are rotated.

[0013] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a block diagram illustrating an example of a system that may be configured to encode and decode video data in accordance with one or more techniques of this disclosure. Figure 2 is a conceptual diagram illustrating encoded video data and corresponding data structures in accordance with one or more techniques of this disclosure. Figure 3 is a conceptual diagram illustrating a data structure that encapsulates encoded video data and corresponding metadata in accordance with one or more techniques of this disclosure. Figure 4 is a conceptual diagram illustrating an example of components that may be included in an implementation of a system that may be configured to encode and decode video data in accordance with one or more techniques of this disclosure. Figure 5 is a block diagram illustrating an example of a video encoder that may be configured to encode video data in accordance with one or more techniques of this disclosure. Figure 6is a block diagram illustrating an example of a video decoder that may be configured to decode video data in accordance with one or more techniques of this disclosure. Figure 7 is a conceptual diagram illustrating an example of a video component sampling format that may be used in accordance with one or more techniques of this disclosure. Figure 8 is a conceptual diagram illustrating an example of a packed data channel for a luma component, in accordance with one or more techniques of this disclosure. FIG. 9A to FIG. 9B is a conceptual diagram illustrating an example of combining fields to form an interlaced video image in accordance with one or more techniques of this disclosure. Fig.10 is a conceptual diagram illustrating an example of inputting an interlaced frame into a deinterlacing-based neural network in accordance with one or more techniques of this disclosure. Fig.11 is a conceptual diagram illustrating an example of rotation relative to a neural network based deinterlacing, in accordance with one or more techniques of this disclosure. DETAILED DESCRIPTION

[0015] The video content includes a video sequence consisting of a series of frames (or pictures). A series of frames may also be referred to as a group of pictures (GOP). Each video frame or picture may be divided into one or more regions. The region may be defined according to a set of rules for defining a basic unit (e.g., a video block) and a defined region. For example, the rule defining the region may be that the region must be an integer number of video blocks arranged in a rectangle. In addition, the video blocks in the region may be sorted according to a scanning mode (e.g., a raster scan). As used herein, the term "video block" may generally refer to a region of a picture, or may more specifically refer to a maximum array of sample values ​​that may be predictively encoded, its sub-partitions, and / or corresponding structures. In addition, the term "current video block" may refer to a region of a picture that is being encoded or decoded. A video block may be defined as an array of sample values. It should be noted that in some cases, a pixel value may be described as a sample value comprising a corresponding component of video data, which may also be referred to as a color component (e.g., brightness (Y) and chrominance (Cb and Cr) components or red, green, and blue components). It should be noted that in some cases, the terms "pixel value" and "sample value" may be used interchangeably. Additionally, in some cases, a pixel or sample may be referred to as a pel. A video sampling format (which may also be referred to as a chroma format) may define the number of chroma samples included in a video block relative to the number of luma samples included in the video block. For example, for a 4:2:0 sampling format, the sampling rate of the luma component is twice the sampling rate of the chroma components in both the horizontal and vertical directions. It should be noted that in some cases, the terms "luma" and "luminance" may be used interchangeably.

[0016] The video encoder may perform predictive coding on a video block and its sub-partitions. A video block and its sub-partitions may be referred to as a node. ITU-T H.264 specifies a macroblock including 16×16 luminance samples. That is, in ITU-T H.264, a picture is segmented into macroblocks. ITU-T H.265 specifies a similar coding tree unit (CTU) structure (which may be referred to as a maximum coding unit (LCU)). In ITU-TH.265, a picture is segmented into CTUs. In ITU-T H.265, for a picture, the CTU size may be set to include 16×16, 32×32, or 64×64 luminance samples. In ITU-T H.265, a CTU is composed of a corresponding coding tree block (CTB) for each component of the video data (e.g., luminance (Y) and chrominance (Cb and Cr)). It should be noted that a video having a luminance component and two corresponding chrominance components may be described as having two channels, i.e., a luminance channel and a chrominance channel. In addition, in ITU-T H.265, CTUs can be divided according to a quadtree (QT) partitioning structure, which allows the CTBs of a CTU to be divided into coding blocks (CBs). That is, in ITU-T H.265, CTUs can be divided into quadtree leaf nodes. According to ITU-T H.265, a luma CB together with two corresponding chroma CBs and associated syntax elements is referred to as a coding unit (CU). In ITU-TH.265, a signal can be sent to notify the minimum allowed size of a CB. In ITU-T H.265, the minimum allowed minimum size of a luma CB is 8×8 luma samples. In ITU-T H.265, the decision to encode a picture area using intra prediction or inter prediction is made at the CU level.

[0017] In ITU-T H.265, a CU is associated with a prediction unit structure having its root at the CU. In ITU-T H.265, the prediction unit structure allows the partitioning of luma CB and chroma CB to generate corresponding reference samples. That is, in ITU-T H.265, the luma CB and chroma CB can be partitioned into corresponding luma prediction blocks and chroma prediction blocks (PBs), where the PBs include blocks of sample values ​​to which the same prediction is applied. In ITU-T H.265, a CB can be divided into 1, 2, or 4 PBs. ITU-TH.265 supports PB sizes from 64×64 samples down to 4×4 samples. In ITU-T H.265, intra prediction data (e.g., intra prediction mode syntax elements) or inter prediction data (e.g., motion data syntax elements) corresponding to the PB are used to generate reference and / or predicted sample values ​​for the PB. ITU-T H.266 specifies a CTU with a maximum size of 128x128 luma samples. In ITU-H.266, CTU is divided according to a quadtree plus multi-type tree (QTMT or QT+MTT) structure. The QTMT structure in ITU-TH.266 allows the quadtree leaf nodes to be further divided by a binary tree (BT) structure. That is, in ITU-T H.266, the binary tree leaf nodes can be recursively divided vertically or horizontally. In addition, in ITU-T H.266, in addition to indicating binary segmentation, the multi-type tree can also indicate the so-called ternary (or ternary tree (TT)) segmentation. The ternary segmentation divides a block vertically or horizontally into three blocks. In the case of vertical TT segmentation, the block is segmented at a quarter of its width from the left edge and at a quarter of its width from the right edge, and in the case of horizontal TT segmentation, the block is segmented at a quarter of its height from the top edge and at a quarter of its height from the bottom edge.

[0018] As described above, each video frame or picture may be divided into one or more regions. For example, according to ITU-T H.265, each video frame or picture may be divided into one or more slices, and further divided into one or more tiles, wherein each slice includes a CTU sequence (e.g., arranged in a raster scan order), and wherein the tile is a CTU sequence corresponding to a rectangular area of ​​the picture. It should be noted that in ITU-T H.265, a slice is a sequence of one or more slice segments starting from an independent slice segment and containing all subsequent subordinate slice segments (if any) before the next independent slice segment (if any). Slice segments (such as slices) are CTU sequences. Therefore, in some cases, the terms "slice" and "slice segment" may be used interchangeably to indicate a sequence of CTUs arranged in a raster scan order. In addition, it should be noted that in ITU-T H.265, a tile may be composed of CTUs contained in more than one slice, and a slice may be composed of CTUs contained in more than one tile. However, ITU-T H.265 specifies that one or both of the following conditions should be met: (1) all CTUs in a slice belong to the same tile; and (2) all CTUs in a tile belong to the same slice.

[0019] With respect to ITU-T H.266, a slice needs to consist of an integer number of complete tiles or an integer number of continuous complete CTU rows within a tile, rather than just an integer number of CTUs. It should be noted that in ITU-T H.266, the slice design does not include slice fragments (i.e., there are no independent / dependent slice fragments). Therefore, in ITU-T H.266, a picture may include a single tile, wherein the single tile is contained within a single slice, or a picture may include multiple tiles, wherein the multiple tiles (or their CTU rows) may be contained within one or more slices. In ITU-T H.266, a picture is divided into tiles by specifying the corresponding height of a tile row and the corresponding width of a tile column. Therefore, in ITU-T H.266, a tile is a rectangular CTU area within a specific tile row and a specific tile column position. In addition, it should be noted that ITU-T H.266 stipulates that a picture can be divided into sub-pictures, wherein a sub-picture is a rectangular CTU area within a picture. The upper left CTU of a sub-picture may be located at any CTU position within a picture, where the sub-picture is constrained to include one or more slices. Therefore, unlike tiles, sub-pictures do not have to be restricted to specific row and column positions. It should be noted that sub-pictures can be used to encapsulate regions of interest within a picture, and the sub-bitstream extraction process can be used to decode and display only specific regions of interest. That is, as described in further detail below, the bitstream of encoded video data includes a sequence of network abstraction layer (NAL) units, wherein the NAL unit encapsulates encoded video data (i.e., video data corresponding to a picture slice), or the NAL unit encapsulates metadata (e.g., parameter sets) for decoding the video data, and the sub-bitstream extraction process forms a new bitstream by removing one or more NAL units from the bitstream.

[0020] Figure 2 is a conceptual diagram showing examples of pictures within picture groups partitioned according to tiles, slices, and sub-pictures. It should be noted that the techniques described herein can be applicable to tiles, slices, sub-pictures, their sub-partitions, and / or their equivalent structures. That is, regardless of how the pictures are divided into regions, the techniques described herein are generally applicable. For example, in some cases, the techniques described herein can be applicable to cases where tiles can be partitioned into so-called bricks, where a brick is a rectangular CTU row region within a particular tile. In addition, for example, in some cases, the techniques described herein can be applicable to cases where one or more tiles can be included in a so-called tile group, where a tile group includes an integer number of adjacent tiles. Figure 2 In the example shown, Pic3 is shown to include 16 tiles (ie, tile 0 to tile 15) and three slices (ie, slice 0 to slice 2). Figure 2In the example shown, slice 0 includes four tiles (i.e., tiles 0 to 3), slice 1 includes eight tiles (i.e., tiles 4 to 11), and slice 2 includes four tiles (i.e., tiles 12 to 15). Figure 2 As shown in the example of , Pic3 is shown to include two sub-pictures (i.e., sub-picture 0 and sub-picture 1), where sub-picture 0 includes slices 0 and slice 1 and where sub-picture 1 includes slice 2. As described above, sub-pictures can be used to encapsulate regions of interest within a picture, and a sub-bitstream extraction process can be used to selectively decode (and display) the regions of interest. For example, referring to Figure 2 , sub-picture 0 may correspond to the action portion of a sporting event presentation (e.g., a view of a field), and sub-picture 1 may correspond to a scrolling banner displayed during the sporting event presentation. By organizing pictures into sub-pictures in this manner, a viewer may be able to disable the display of a scrolling banner. That is, through the sub-bitstream extraction process, slice 2 NAL units may be removed from the bitstream (and therefore not decoded and / or displayed), and slice 0 NAL units and slice 1 NAL units may be decoded and displayed. How slices of a picture are encapsulated into corresponding NAL unit data structures and sub-bitstream extraction is described in further detail below.

[0021] As described above, the video sampling format (which may also be referred to as the chroma format) may define the number of chroma samples included in a CU relative to the number of luma samples included in the CU. For example, for a 4:2:0 sampling format, the sampling rate of the luma component is twice the sampling rate of the chroma components in both horizontal and vertical directions. Therefore, for a CU formatted according to the 4:2:0 format, the width and height of the sample array for the luma component are twice the width and height of each sample array for the chroma components. Figure 7 is a conceptual diagram illustrating an example of coding units formatted according to a 4:2:0 sample format. Figure 7 shows the relative position of chroma samples relative to luma samples within a CU. As mentioned above, a CU is usually defined based on the number of horizontal and vertical luma samples. Figure 7 As shown, a 16×16 CU formatted according to the 4:2:0 sample format includes 16×16 samples of the luma component and 8×8 samples for each chroma component. Figure 7In the example shown, the relative positions of chroma samples of adjacent video blocks of a 16×16 CU with respect to luma samples are shown. For a CU formatted according to the 4:2:2 format, the width of the sample array of the luma component is twice the width of the sample array of each chroma component, but the height of the sample array of the luma component is equal to the height of the sample array of each chroma component. In addition, for a CU formatted according to the 4:4:4 format, the sample array of the luma component has the same width and height as the sample array of each chroma component.

[0022] Table 1 shows how to specify the chroma format based on the value of the syntax element chroma_format_idc in ITU-T H.266. In addition, Table 1 shows how to specify the derived variables SubWidthC and SubHeightC according to the chroma format. SubWidthC and SubHeightC are used for deblocking, for example. Regarding Table 1, ITU-T H.266 provides the following: In monochrome sampling, there is only one sample array, which is nominally considered to be the brightness array. In 4:2:0 sampling, each of the two chroma arrays has half the height and half the width of the luma array. In 4:2:2 sampling, each of the two chroma arrays has the same height and half the width of the luma array. In 4:4:4 sampling, each of the two chroma arrays has the same height and the same width of the luma array. chroma_format_idc Chroma Format SubWidthC SubHeightC 0 monochrome 1 1 1 4:2:0 2 2 2 4:2:2 2 1 3 4:4:4 1 1 Table 1

[0023] For intra-frame prediction coding, the intra-frame prediction mode may specify the location of the reference sample within the picture. In ITU-T H.265, the possible intra-frame prediction modes defined include a plane (i.e., surface fitting) prediction mode, a DC (i.e., flat overall average) prediction mode, and 33 angle prediction modes (predMode: 2-34). In ITU-T H.266, the possible intra-frame prediction modes defined include a plane prediction mode, a DC prediction mode, and 65 angle prediction modes. In addition, in ITU-T H.266, additional intra-frame prediction tools are enabled, such as, for example, intra-frame sub-division mode and matrix-based intra-frame prediction. It should be noted that the plane prediction mode and the DC prediction mode may be referred to as a non-directional prediction mode, and the angle prediction mode may be referred to as a directional prediction mode. It should be noted that regardless of the number of possible prediction modes defined, the techniques described herein may be generally applicable.

[0024] For inter-frame prediction coding, a reference picture is determined, and a motion vector (MV) identifies samples in the reference picture used to generate a prediction for the current video block. For example, the current video block can be predicted using reference sample values ​​located in one or more previously encoded pictures, and the motion vector is used to indicate the position of the reference block relative to the current video block. The motion vector may describe, for example, the horizontal displacement component of the motion vector (i.e., MVx), the vertical displacement component of the motion vector (i.e., MVy), and the resolution of the motion vector (e.g., quarter-pixel precision, half-pixel precision, one-pixel precision, two-pixel precision, four-pixel precision). Previously decoded pictures (which may include pictures output before or after the current picture) may be organized into one or more reference picture lists and identified using reference picture index values. In addition, in inter-frame prediction coding, uni-prediction refers to generating predictions using sample values ​​from a single reference picture, and bi-prediction refers to generating predictions using corresponding sample values ​​from two reference pictures. That is, in uni-prediction, a single reference picture and a corresponding motion vector are used to generate a prediction for a current video block, while in bi-prediction, a first reference picture and a corresponding first motion vector and a second reference picture and a corresponding second motion vector are used to generate a prediction for a current video block. In bi-prediction, corresponding sample values ​​are combined (e.g., added, rounded and cropped, or averaged according to weights) to generate a prediction. Pictures and their regions may be classified based on which types of prediction modes are available for encoding their video blocks. That is, for regions with B type (e.g., B slices), bi-prediction, uni-prediction and intra-prediction modes may be utilized, for regions with P type (e.g., P slices), uni-prediction and intra-prediction modes may be utilized, and for regions with I type (e.g., I slices), only intra-prediction modes may be utilized. As described above, reference pictures are identified by reference indexes. For example, for P slices, there may be a single reference picture list RefPicList0, and for B slices, there may be a second independent reference picture list RefPicList1 in addition to RefPicList0. It should be noted that for uni-prediction in B slices, one of RefPicList0 or RefPicList1 may be used to generate the prediction. Furthermore, it should be noted that during the decoding process, at the start of decoding a picture, a reference picture list is generated from previously decoded pictures stored in the decoded picture buffer (DPB).

[0025] In addition, the coding standard may support various motion vector prediction modes. Motion vector prediction enables the value of the motion vector for the current video block to be derived based on another motion vector. For example, a set of candidate blocks with associated motion information can be derived from the spatial neighboring blocks and the temporal neighboring blocks of the current video block. In addition, the generated (or default) motion information can be used for motion vector prediction. Examples of motion vector prediction include advanced motion vector prediction (AMVP), temporal motion vector prediction (TMVP), so-called "merge" mode, and "skip" and "direct" motion reasoning. In addition, other examples of motion vector prediction include advanced temporal motion vector prediction (ATMVP) and space-time motion vector prediction (STMVP). In addition, in ITU-T H.266, the following inter-frame prediction modes are enabled: affine motion model, adaptive motion vector resolution, bidirectional optical flow, decoder-side motion vector correction, and geometric partitioning mode.

[0026] As described above, for inter-frame prediction coding, reference samples in previously encoded pictures are used to encode video blocks in the current picture. Previously encoded pictures that can be used as references when encoding the current picture are called reference pictures. It should be noted that the decoding order does not necessarily correspond to the picture output order, that is, the temporal order of pictures in a video sequence. In ITU-T H.266, when a picture is decoded, it is stored in a decoded picture buffer (DPB) (which may be called a frame buffer, a reference buffer, a reference picture buffer, etc.). In ITU-T H.266, pictures stored in the DPB are removed from the DPB when they are output and are no longer needed for encoding subsequent pictures. In ITU-T H.266, the determination of whether the picture should be removed from the DPB is called once for each picture after decoding the slice header, that is, at the beginning of decoding the picture. For example, the reference Figure 2 , Pic2 is shown as referenced to Pic1. Similarly, Pic3 is shown as referenced to Pic0. Figure 2, assuming that the number of pictures corresponds to the decoding order, the DPB will be filled as follows: after decoding Pic0, the DPB will include {Pic0}; at the beginning of decoding Pic1, the DPB will include {Pic0}; after decoding Pic1, the DPB will include {Pic0, Pic1}; at the beginning of decoding Pic2, the DPB will include {Pic0, Pic1}. Pic2 will then be decoded with reference to Pic1, and after decoding Pic2, the DPB will include {Pic0, Pic1, Pic2}. At the beginning of decoding Pic3, pictures Pic0 and Pic1 will be marked for removal from the DPB because they are not required for decoding Pic3 (or any subsequent pictures, not shown), and assuming that Pic1 and Pic2 have been output, the DPB will be updated to include {Pic0}. Pic3 will then be decoded with reference to Pic0. The process of marking pictures to remove them from the DPB can be called reference picture set (RPS) management.

[0027] As described above, intra prediction data or inter prediction data is used to generate reference sample values ​​for a block of sample values. The difference between the sample values ​​included in the current PB or another type of picture region structure and the associated reference samples (e.g., those generated using prediction) may be referred to as residual data. The residual data may include a corresponding difference array corresponding to each component of the video data. The residual data may be in the pixel domain. A transform such as a discrete cosine transform (DCT), a discrete sine transform (DST), an integer transform, a wavelet transform, or a conceptually similar transform may be applied to the difference array to generate transform coefficients. It should be noted that in ITU-T H.265 and ITU-T H.266, a CU is associated with a transform tree structure having its root at the CU level. The transform tree is divided into one or more transform units (TUs). That is, in order to generate transform coefficients, the array of difference values ​​may be divided (e.g., four 8×8 transforms may be applied to a 16×16 residual value array). For each component of the video data, such a subdivision of the difference values ​​may be referred to as a transform block (TB). It should be noted that in some cases a core transform and a subsequent secondary transform may be applied (in a video encoder) to generate transform coefficients. For a video decoder, the order of transforms is reversed.

[0028] The quantization process may be performed directly on the transform coefficients or residual sample values ​​(e.g., in the case of palette coded quantization). Quantization approximates the transform coefficients by limiting the amplitude to a set of specified values. Quantization essentially scales the transform coefficients in order to change the amount of data required to represent a set of transform coefficients. Quantization may include dividing the transform coefficients (or the values ​​obtained by adding the offset values ​​to the transform coefficients) by a quantization scaling factor and any associated rounding function (e.g., rounding to the nearest integer). The quantized transform coefficients may be referred to as coefficient level values. Inverse quantization (or "dequantization") may include multiplying the coefficient level values ​​by the quantization scaling factor, as well as any reciprocal rounding or offset addition operations. It should be noted that, as used herein, the term quantization process may refer to dividing by a scaling factor in some cases to generate a level value, and may refer to multiplying by a scaling factor in some cases to recover the transform coefficients. That is, the quantization process may refer to quantization in some cases, and to inverse quantization in some cases. In addition, it should be noted that although the quantization process is described in some of the examples below with respect to arithmetic operations associated with decimal notation, such description is for illustrative purposes and should not be construed as limiting. For example, the techniques described herein can be implemented in devices that use binary operations, etc. For example, the multiplication and division operations described herein can be implemented using shift operations, etc.

[0029] The quantized transform coefficients and syntax elements (e.g., syntax elements indicating the coding structure of the video block) may be entropy encoded according to an entropy coding technique. The entropy coding process includes encoding the syntax element values ​​using a lossless data compression algorithm. Examples of entropy coding techniques include content adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), probability interval partitioning entropy coding (PIPE), etc. The entropy-coded quantized transform coefficients and the corresponding entropy-coded syntax elements may form a compatible bitstream that can be used to reproduce the video data at the video decoder. The entropy coding process, such as CABAC, may include binarizing the syntax elements. Binarization refers to the process of converting the value of the syntax element into a sequence of one or more bits. These bits may be referred to as "bins". Binarization may include one or a combination of the following coding techniques: fixed length coding, unary coding, truncated unary coding, truncated Rice coding, Golomb coding, k-order exponential Golomb coding, and Golomb-Rice coding. For example, binarization may include representing the integer value 5 of the syntax element as 00000101 using an 8-bit fixed-length binarization technique, or representing the integer value 5 as 11110 using a unary coding binarization technique. As used herein, each of the terms fixed-length coding, unary coding, truncated unary coding, truncated Rice coding, Golomb coding, k-order exponential Golomb coding, and Golomb-Rice coding may refer to general implementations of these techniques and / or more specific implementations of these coding techniques. For example, a Golomb-Rice coding implementation may be specifically defined according to a video coding standard. In the example of CABAC, for a particular bin, the context provides a maximum probability state (MPS) value of the bin (i.e., the MPS of the bin is one of 0 or 1), and a probability value that the bin is an MPS or a minimum probability state (LPS). For example, the context may indicate that the MPS of the bin is 0, and the probability that the bin is 1 is 0.3. It should be noted that the context may be determined based on the value of the previously encoded bin including the bin in the current syntax element and the previously encoded syntax element. For example, the values ​​of syntax elements associated with neighboring video blocks may be used to determine the context of the current bin.

[0030] As described above, the video content includes a video sequence consisting of a series of pictures, and each picture can be divided into one or more regions. In ITU-T H.266, the coding representation of a picture includes a VCL NAL unit of a specific layer within an AU and contains all CTUs of the picture. For example, referring again to Figure 2, the coded representation of Pic3 is encapsulated in three coded slice NAL units (i.e., slice 0 NAL unit, slice 1 NAL unit, and slice 2 NAL unit). It should be noted that the term video coding layer (VCL) NAL unit is used as a general term for coded slice NAL units, that is, VCL NAL is a general term for slice NAL units of all types. As described above, and described in further detail below, NAL units can encapsulate metadata for decoding video data. NAL units that encapsulate metadata for decoding video sequences are generally referred to as non-VCL NAL units. Therefore, in ITU-T H.266, NAL units can be VCL NAL units or non-VCL NAL units. It should be noted that the VCL NAL unit includes slice header data, which provides information for decoding a specific slice. Therefore, in ITU-T H.266, information for decoding video data (which may be referred to as metadata in some cases) is not limited to being included in non-VCL NAL units. ITU-T H.266 specifies that a picture unit (PU) is a group of NAL units that are associated with each other according to a specified classification rule, are continuous in decoding order, and contain exactly one coded picture, and an access unit (AU) is a group of PUs that belong to different layers and contain coded pictures associated with the same time output from the DPB. ITU-T H.266 further specifies that a layer is a group of VCL NAL units and associated non-VCL NAL units that all have a specific value of the layer identifier. In addition, in ITU-T H.266, a PU consists of zero or one PH NAL unit, one coded picture (which consists of one or more VCL NAL units), and zero or more other non-VCL NAL units. In addition, in ITU-T H.266, a coded video sequence (CVS) is an AU sequence consisting of a CVSS AU and zero or more subsequent AUs that are non-CVSS AUs in decoding order (including all subsequent AUs up to (excluding) any subsequent AU that is a CVSS AU), wherein a coded video sequence start (CVSS) AU is an AU in which a PU exists for each layer in the CVS, and a coded picture in each existing picture unit is a coded layer video sequence start (CLVSS) picture. In ITU-T H.266, a coded layer video sequence (CLVS) is a PU sequence within the same layer, consisting of a CLVSS PU and zero or more subsequent non-CLVSS PUs in decoding order (including all subsequent PUs up to (excluding) any subsequent PU that is a CLVSS PU). That is, in ITU-T H.266, a bitstream can be described as including a sequence of AUs forming one or more CVSs.

[0031] Multi-layer video coding enables video presentation to be decoded / displayed as a presentation corresponding to a base layer of video data, and decoded / displayed as one or more additional presentations corresponding to an enhancement layer of video data. For example, a base layer can enable presentation of a video presentation with a basic quality level (e.g., high-definition presentation and / or a 30Hz frame rate), and an enhancement layer can enable presentation of a video presentation with an enhanced quality level (e.g., ultra-high-definition rendering and / or a 60Hz frame rate). The enhancement layer can be encoded by reference to the base layer. That is, for example, the picture in the enhancement layer can be encoded (e.g., using inter-layer prediction techniques) by reference to one or more pictures in the base layer (including a scaled version thereof). It should be noted that the layers can also be encoded independently of each other. In this case, there may be no inter-layer prediction between the two layers. Each NAL unit may include an identifier indicating the video data layer to which the NAL unit is associated. As described above, a sub-bitstream extraction process can be used to decode and display only a specific region of interest of a picture. In addition, a sub-bitstream extraction process can be used to decode and display only a specific video layer. Sub-bitstream extraction may refer to a process by which a device receiving a compatible or compliant bitstream forms a new compatible or compliant bitstream by discarding and / or modifying data in the received bitstream. For example, sub-bitstream extraction may be used to form a new compatible or compliant bitstream corresponding to a particular video representation (e.g., a high-quality representation).

[0032] In ITU-T H.266, each of a video sequence, GOP, picture, slice, and CTU may be associated with metadata describing video coding properties, and some types of metadata are encapsulated in non-VCL NAL units. ITU-T H.266 defines parameter sets that can be used to describe video data and / or video coding properties. Specifically, ITU-T H.266 includes the following four parameter sets: video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), and adaptation parameter set (APS), where SPS applies to zero or more integer CVSs, PPS applies to zero or more integer coded pictures, APS applies to zero or more slices, and VPS may be optionally referenced by SPS. PPS applies to one or more single coded pictures that reference it. In ITU-T H.266, parameter sets may be encapsulated as non-VCL NAL units and / or may be signaled as messages. ITU-T H.266 also includes a picture header (PH), which is encapsulated in its own NAL unit as a non-VCL NAL unit when signaled, or encapsulated in a slice header of a coded slice as part of a VCL NAL unit when signaled. In ITU-T H.266, the picture header is applied to all slices of a coded picture. ITU-TH.266 further enables signaling of decoding capability information (DCI) and supplementary enhancement information (SEI) messages. In ITU-T H.266, DCI and SEI messages assist processes related to decoding, display or other purposes, however, DCI and SEI messages may not be required to construct luminance or chrominance samples according to the decoding process. In ITU-T H.266, non-VCL NAL units can be used to signal DCI and SEI messages in the bitstream. In addition, DCI and SEI messages can be transmitted by a mechanism other than being present in the bitstream (i.e., signaling out of band).

[0033] Figure 3 An example of a bitstream including a plurality of CVSs is shown, wherein the CVSs include AUs, and the AUs include picture units. Figure 3 The example shown in corresponds to encapsulating Figure 2 An example of a slice NAL unit is shown in the example of . Figure 3 In the example shown, the corresponding picture unit of Pic3 includes three VCL NAL coded slice NAL units, namely slice 0 NAL unit, slice 1 NAL unit and slice 2 NAL unit, and two non-VCL NAL units, namely PPS NAL unit and PH NAL unit. Figure 3In , the header is the NAL unit header (i.e., not to be confused with the slice header). In addition, it should be noted that in Figure 3 In the example, other non-VCL NAL units not shown may be included in the CVS, such as SPS NAL units, VPS NAL units, SEI message NAL units, etc. In addition, it should be noted that in other examples, the PPS NAL unit for decoding Pic3 may be included elsewhere in the bitstream, for example, in the picture unit corresponding to Pic0, or may be provided by an external agency. In ITU-T H.266, the PH syntax structure may be present in the slice header of the VCL NAL unit or in the PH NAL unit of the current PU.

[0034] It should be noted that during the development of ITU-T H.266, convolutional neural network (CNN)-based techniques were studied, which showed the potential to remove artifacts and improve objective quality, but it was decided not to include such techniques in ITU-T H.266. However, CNN-based techniques are currently being considered for extension and / or improvement of ITU-T H.266. Some CNN-based techniques involve post-filtering. For example, "AHG11: Content-adaptive neural network post-filter" (teleconference, document JVET-Z0082-v2, referred to as JVET-Z0082 in this article) at the 26th meeting of ISO / IEC JTC1 / SC29 / WG11 held from April 20 to 29, 2022 describes a post-filter based on a content-adaptive neural network. It should be noted that in JVET-Z0082, content adaptation is achieved by overfitting the NN post-filter to the test video. In addition, it should be noted that the result of the overfitting process in JVET-Z0082 is a weight update. JVET-Z0082 describes where weight updates are encoded using ISO / IEC FDIS15938-17. Information Technology - Multimedia Content Description Interface - Part 17: Compression of Neural Networks for Multimedia Content Description and Analysis and Test Model for Incremental Compression of Neural Networks for Multimedia Content Description and Analysis (INCTM), N0179, February 2022, which may be collectively referred to as the MPEG NNR (Neural Network Representation) or Neural Network Coding (NNC) standard. JVET-Z0082 also describes where coding weight updates are signaled within the video bitstream as NNR post-filter SEI messages. The 26th ISO / IEC JTC1 / SC29 / WG11 meeting, "AHG9: NNR post-filter SEI message," held from April 20 to 29, 2022 (conference call, document JVET-Z0052-v1, referred to herein as JVET-Z0052), describes the NNR post-filter SEI message utilized by JVET-Z0082. Elements of the NN post-filter described in JVET-Z0082 and the NNR post-filter SEI messages described in JVET-Z0052 are adopted in "Additional SEI messages for VSEI (Draft 2)" of the 27th ISO / IEC JTC1 / SC29 / WG11 meeting, held from July 13 to 22, 2022 (conference call, document JVET-AA2006-v2, referred to herein as JVET-AA2006). JVET-AA2006 provides generic supplemental enhancement information messages for coded video bitstreams (VSEI).JVET-AA2006 specifies the syntax and semantics for a neural network post-filter characteristics SEI message and for a neural network post-filter activation SEI message. The neural network post-filter characteristics SEI message specifies a neural network that can be used as a post-processing filter. The use of a specified post-processing filter for a particular picture is indicated by a neural network post-filter activation SEI message. The 30th meeting of ISO / IEC JTC1 / SC29 / WG11, "Additional SEI messages for VSEI (Draft 4)", held in Antalya, Turkey from April 21 to 28, 2023 (document JVET-AD2006-v1 (referred to herein as JVET-AD2006)) also specifies the syntax and semantics for a neural network post-filter characteristics SEI message and for a neural network post-filter activation SEI message. JVET-AD2006 is described in more detail below. The technology described herein provides a technique for signaling a neural network post-filter message.

[0035] For the formulas used in this article, the following arithmetic operators can be used: +Addition - Subtraction * Multiplication, including matrix multiplication x y Exponentiation. Raises x to the power of y. In other contexts, such symbols are used for superscripts and are not intended to be interpreted as exponentiation. / Integer division truncates the result toward zero. For example, 7 / 4 and -7 / -4 truncate to 1, and -7 / 4 and 7 / -4 truncate to -1. ÷ is used to represent division in mathematical formulas when truncation or rounding is not intended. Used to represent division in mathematical formulas without the intent to truncate or round.

[0036] Additionally, the following math functions are available: Log2(x), the base 2 logarithm of x; Ceil(x), the smallest integer greater than or equal to x.

[0037] With respect to the exemplary syntax used herein, the following definitions of logical operators may be applied: x&&y Boolean logical "AND" of x and y x||y Boolean logical OR of x and y ! Boolean logic "NO" x? y:z If x is TRUE or not equal to 0, it evaluates to y; otherwise, it evaluates to z.

[0038] Additionally, the following relational operators can be applied: > greater than > = greater than or equal to < <=less than or equal to == is equal to ! = not equal to

[0039] Furthermore, it should be noted that among the syntax descriptors used in this document, the following descriptors may be applied: -b(8): Byte (8 bits) with any bit string pattern. The parsing of this descriptor is specified by the return value of the function read_bit(8). -f(n): A fixed-pattern bit string written using n bits (from left to right) starting from the leftmost bit. The parsing process of this descriptor is specified by the return value of the function read_bit(n). -se(v): signed integer order 0 Exp-Golomb coded syntax element, starting from the leftmost bit. -tb(v): truncated binary code using at most maxVal bits, where maxVal is defined in the semantics of the syntax element. -tu(v): truncated unary code using at most maxVal bits, where maxVal is defined in the semantics of the syntax element. -u(n): Use an unsigned integer of n bits. When n is "v" in the syntax table, the number of bits varies in a way that depends on the values ​​of other syntax elements. The parsing process of this descriptor is specified by the return value of the function read_bits(n), which is interpreted as the binary representation of the unsigned integer with the most significant bit written first. -ue(v): unsigned integer 0th order Exp-Golomb coded syntax element, starting from the leftmost bit. -st(v): A null-terminated string encoded as Universal Coded Character Set (UCS) Transmission Format-8 (UTF-8) characters, as specified in ISO / IEC 10646. The parsing process is specified as follows: st(v) starts at a byte-aligned position in the bitstream, and reads and returns a series of bytes from the bitstream (the series of bytes starting at the current position and continuing up to but not including the next byte-aligned byte equal to 0x00), and advances the bitstream pointer by (stringLength+1)*8 bit positions, where stringLength is equal to the number of bytes returned.

[0040] As described above, ITU-T H.266 defines NAL unit header semantics, which specifies the type of raw byte sequence payload (RBSP) data structure included in the NAL unit. Table 2 shows the syntax of the NAL unit header provided in ITU-T H.266. Table 2

[0041] ITU-T H.266 provides the following definitions for the corresponding syntax elements shown in Table 2. forbidden_zero_bit should be equal to 0. nuh_reserved_zero_bit shall be equal to 0. A value of 1 for nuh_reserved_zero_bit may be specified by ITU-T|ISO / IEC in the future. Although in this version of this specification, the value of nuh_reserved_zero_bit is required to be equal to 0, decoders conforming to this version of this specification shall allow a value of nuh_reserved_zero_bit equal to 1 to appear in the syntax and shall ignore (i.e., remove and discard from the bitstream) NAL units with nuh_reserved_zero_bit equal to 1. nuh_layer_id specifies the identifier of the layer to which the VCL NAL unit belongs or the identifier of the layer to which the non-VCL NAL unit applies. The value of nuh_layer_id shall be in the range of 0 to 55, inclusive. Other values ​​of nuh_layer_id are reserved for future use by ITU-T | ISO / IEC. Although in this version of this specification, the value of nuh_layer_id is required to be in the range of 0 to 55, inclusive, a decoder conforming to this version of this specification shall allow values ​​of nuh_layer_id greater than 55 to appear in the syntax and shall ignore (i.e., remove and discard from the bitstream) NAL units with nuh_layer_id greater than 55. The value of nuh_layer_id of all VCL NAL units of a coded picture shall be the same. The value of nuh_layer_id of a coded picture or PU is the value of nuh_layer_id of the VCL NAL unit of the coded picture or PU. When nal_unit_type is equal to PH_NUT or FD_NUT, nuh_layer_id shall be equal to the nuh_layer_id of the associated VCL NAL unit. When nal_unit_type is equal to EOS_NUT, nuh_layer_id shall be equal to one of the nuh_layer_id values ​​of the layer present in the CVS. NOTE—The value of nuh_layer_id for DCI, OPI, VPS, AUD, and EOB NAL units is not constrained. nuh_temporal_id_plus1 minus 1 specifies the temporal identifier of the NAL unit. The value of nuh_temporal_id_plus1 shall not be equal to 0. The variable TemporalId is derived as follows: TemporalId=nuh_temporal_id_plus1-1 When nal_unit_type is in the range of IDR_W_RADL to RSV_IRAP_11 (inclusive), TemporalId shall be equal to 0. When nal_unit_type is equal to STSA_NUT and vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]] is equal to 1, TemporalId shall be greater than 0. The value of TemporalId shall be the same for all VCL NAL units of an AU. The value of TemporalId of a coded picture, PU, ​​or AU is the value of TemporalId of the VCL NAL unit of the coded picture, PU, ​​or AU. The value of TemporalId of a sub-layer representation is the maximum value of the TemporalId of all VCL NAL units in the sub-layer representation. The value of TemporalId for non-VCL NAL units is constrained as follows: - If nal_unit_type is equal to DCI_NUT, OPI_NUT, VPS_NUT or SPS_NUT, TemporalId shall be equal to 0, and the TemporalId of the AU containing the NAL unit shall be equal to 0. - Otherwise, if nal_unit_type is equal to PH_NUT, TemporalId shall be equal to the TemporalId of the PU containing the NAL unit. - Otherwise, if nal_unit_type is equal to EOS_NUT or EOB_NUT, TemporalId shall be equal to 0. - Otherwise, if nal_unit_type is equal to AUD_NUT, FD_NUT, PREFIX_SEI_NUT, or SUFFIX_SEI_NUT, the TemporalId shall be equal to the TemporalId of the AU containing the NAL unit. - Otherwise, when nal_unit_type is equal to PPS_NUT, PREFIX_APS_NUT, or SUFFIX_APS_NUT, The TemporalId shall be greater than or equal to the TemporalId of the PU containing the NAL unit. NOTE—When the NAL unit is a non-VCL NAL unit, the value of TemporalId is equal to the minimum of the TemporalId values ​​of all AUs to which the non-VCL NAL unit applies. When nal_unit_type is equal to PPS_NUT, PREFIX_APS_NUT, or SUFFIX_APS_NUT, TemporalId may be greater than or equal to the TemporalId of the containing AU, since all PPSs and APSs may be included in the beginning of the bitstream (e.g., when they are transported out-of-band and the receiver places them at the beginning of the bitstream), with the first coded picture having TemporalId equal to 0. nal_unit_type specifies the NAL unit type, ie, the type of RBSP data structure contained in the NAL unit as specified in Table 3. NAL units with a nal_unit_type in the range of UNSPEC28..UNSPEC31 (inclusive), whose semantics are unspecified, shall not affect the decoding process specified in this specification. NOTE—NAL unit types in the range of UNSPEC_28..UNSPEC_31 may be used as determined by the application. The decoding process for these values ​​of nal_unit_type is not specified in this specification. Since different applications may use these NAL unit types for different purposes, special attention is expected when designing encoders that generate NAL units with these nal_unit_type values ​​and when designing decoders that interpret the content of NAL units with these nal_unit_type values. This specification does not define any management of these values. These nal_unit_type values ​​may only be used in contexts where "conflicts" in use (i.e., the meaning of the NAL unit content for the same nal_unit_type value is defined differently) are not important, or are not possible or are managed, for example, defined or managed in a controlling application or transport specification, or managed by the environment that controls the distribution of the bitstream. For purposes other than determining the amount of data in a DU of a bitstream, a decoder shall ignore (remove from the bitstream and discard) the contents of all NAL units that use a reserved value of nal_unit_type. NOTE—This requirement allows for the definition of compatible extensions of this specification in the future. Table 3 NOTE—A clean random access (CRA) picture may have associated RASL or RADL pictures present in the bitstream. NOTE—An Instantaneous Decoding Refresh (IDR) picture with nal_unit_type equal to IDR_N_LP does not have an associated leading picture present in the bitstream. An IDR picture with nal_unit_type equal to IDR_W_RADL does not have an associated RASL picture present in the bitstream, but may have an associated RADL picture in the bitstream. The value of nal_unit_type shall be the same for all VCL NAL units of a sub-picture. A sub-picture is said to have the same NAL unit type as the VCL NAL units of the sub-picture. For any particular picture's VCL NAL unit, the following applies: - If pps_mixed_nalu_types_in_pic_flag is equal to 0, the value of nal_unit_type shall be the same for all VCL NAL units of a picture, and a picture or PU is said to have the same NAL unit type as the coded slice NAL units of the picture or PU. - Otherwise (pps_mixed_nalu_types_in_pic_flag is equal to 1), all of the following constraints apply: -Picture should have at least two sub-pictures. - A VCL NAL unit of a picture shall have two or more different nal_unit_type values. - There shall be no VCL NAL units for pictures with nal_unit_type equal to GDR_NUT. - When the nal_unit_type of a VCL NAL unit of a picture is equal to nalUnitTypeA with a value of IDR_W_RADL, IDR_N_LP, or CRA_NUT, the nal_unit_type of other VCL NAL units of the picture shall be equal to nalUnitTypeA or TRAIL_NUT. The value of nal_unit_type shall be the same for all pictures of an IRAP or GDR AU. When sps_video_parameter_set_id is greater than 0, vps_max_tid_il_ref_pics_plus1[i][j] is equal to 0 (for j equal to GeneralLayerIdx[nuh_layer_id] and any value of i in the range of j+1 to vps_max_layers_minus1, inclusive), and pps_mixed_nalu_types_in_pic_flag is equal to 1, the value of nal_unit_type shall not be equal to IDR_W_RADL, IDR_N_LP, or CRA_NUT. Bitstream conformance requires that the following constraints apply: - When a picture is the leading picture of an IRAP picture, the picture shall be a RADL or RASL picture. - When a sub-picture is a leading sub-picture of an IRAP sub-picture, the sub-picture shall be a RADL or RASL sub-picture. - When a picture is not the leading picture of an IRAP picture, the picture shall not be a RADL or RASL picture. - When a sub-picture is not a leading sub-picture of an IRAP sub-picture, the sub-picture shall not be a RADL or RASL sub-picture. - RASL pictures shall not be present in the bitstream, these RASL pictures are associated with IDR pictures. - RASL sub-pictures shall not be present in the bitstream, these RASL sub-pictures are associated with IDR sub-pictures. - RADL pictures shall not be present in the bitstream, these RADL pictures are associated with IDR pictures with nal_unit_type equal to IDR_N_LP. NOTE—Random access at the location of an IRAP PU can be performed by discarding all PUs preceding the IRAP AU (and correctly decoding non-RASL pictures in the IRAP AU and all subsequent AUs in decoding order), provided that each parameter set is available (in the bitstream or by external means not specified in this specification) when referenced. - RADL sub-pictures shall not be present in the bitstream. These RADL sub-pictures have nal_unit_type equal to IDR_N_LP The IDR sub-picture is associated with the - The IRAP picture with nuh_layer_id equal to the specified layerId in decoding order and nuh_llayer_id equal to layerId Any pictures before it shall precede the IRAP picture in output order, and shall precede any RADL pictures associated with the IRAP picture in output order. -nuh_layer_id is equal to the specified value layerId and the sub-picture index is equal to the specified value subpicIdx in decoding order Any subpicture preceding the IRAP subpicture with nuh_layer_id equal to layerId and subpicture index equal to subpicIdx All RADL sub-pictures should precede the IRAP sub-picture and all its associated RADL sub-pictures in output order. -nuh_layer_id equals the specified value layerId in decoding order at the recovery point where nuh_layer_id equals layerId Any previous pictures should be placed before the picture at the recovery point in the output order. -nuh_layer_id is equal to the specified value layerId and the sub-picture index is equal to the specified value subpicIdx in decoding order. Any sub-picture in the recovery-point picture that precedes the sub-picture with nuh_layer_id equal to layerId and sub-picture index equal to subpicIdx shall precede that sub-picture in the recovery-point image in output order. - Any RASL pictures associated with a CRA picture shall be placed before any RADL pictures associated with the CRA picture in output order. Before picture. - Any RASL sub-picture associated with a CRA sub-picture shall appear before any RASL sub-picture associated with the CRA sub-picture in output order. Before any RADL sub-images. - Any RASL picture associated with a CRA picture whose nuh_layer_id is equal to a specific value layerId shall be output in the order The sequence is after any IRAP or GDR picture with nuh_layer_id equal to layerId, which precedes the CRA picture in decoding order. - nuh_layer_id associated with a CRA sub-picture is equal to a specific value layerId and the sub-picture index is equal to a specific value Any RASL sub-picture with nuh_layer_id equal to layerId and sub-picture index equal to subpicIdx shall follow in output order any IRAP or GDR sub-picture that precedes the CRA sub-picture in decoding order with nuh_layer_id equal to layerId and sub-picture index equal to subpicIdx. - If sps_field_seq_flag is equal to 0, the following applies: When nuh_layer_id is equal to the current map of the specified value layerId If the current picture is a leading picture associated with an IRAP picture, then the current picture shall precede all non-leading pictures associated with the same IRAP picture in decoding order. Otherwise (sps_field_seq_flag is equal to 1), let picA and picB be the first and last leading pictures associated with the IRAP picture in decoding order, respectively, and precede them in decoding order. There shall be at most one non-leading picture with nuh_layer_id equal to layerId before picA and in decoding order before picA There should be no non-leading picture with nuh_layer_id equal to layerId between picA and picB. - If sps_field_seq_flag is equal to 0, the following applies: When nuh_layer_id is equal to a specific value layerId and the sub-picture When the current sub-picture with slice index equal to a specific value subpicIdx is a leading sub-picture associated with an IRAP sub-picture, the current sub-picture should precede all non-leading sub-pictures associated with the same IRAP sub-picture in decoding order. (sps_field_seq_flag is equal to 1), let subpicA and subpicB be the first leading subpicture and the last leading subpicture associated with the IRAP subpicture in decoding order, respectively, and should exist before subpicA in decoding order at most one non-leading subpicture with nuh_layer_id equal to layerId and subpicture index equal to subpicIdx, and There shall not exist between picA and picB in decoding order a sub-picture with nuh_layer_id equal to layerId and sub-picture index equal to The non-leading picture of subpicIdx.

[0042] As provided in Table 3, the NAL unit may include a supplemental enhancement information (SEI) syntax structure. Tables 4 and 5 show the supplemental enhancement information (SEI) syntax structure provided in ITU-T H.266. Table 4 Table 5

[0043] Regarding Tables 4 and 5, ITU-T H.266 provides the following semantics: Each SEI message consists of variables that specify the type payloadType and size payloadSize of the SEI message payload. The derived SEI message payload size payloadSize is specified in bytes and shall be equal to the number of RBSP bytes in the SEI message payload. NOTE—The NAL unit byte sequence containing the SEI message may include one or more emulation prevention bytes (indicated by the emulation_prevention_three_byte syntax element). Since the payload size of the SEI message is specified in the RBSP bytes, the number of emulation prevention bytes is not included in the size of the SEI payload, payloadSize. payload_type_byte is a byte of the payload type of the SEI message. payload_size_byte is the payload size of the SEI message in bytes.

[0044] It should be noted that ITU-T H.266 defines payload types, and the 25th meeting of ISO / IEC JTC1 / SC29 / WG11, held from January 12 to 21, 2022, "Additional SEI messages for VSEI (Draft 6)" (teleconference, document JVET-Y2006-v1, which is incorporated herein by reference and referred to as JVET-Y2006) defines additional payload types. In addition, the 30th meeting of ISO / IEC JTC1 / SC29 / WG11, held in Antalya, Turkey from April 21 to 28, 2023, "New level and systems-related supplemental enhancement information for VVC (Draft 5)" (document JVET-AD2005-v1, which is incorporated herein by reference and referred to as JVET-AD2005-v1) defines additional payload types.

[0045] Table 6 generally shows the syntax structure of sei_payload(). That is, Table 6 shows the syntax structure of sei_payload(), but for the sake of brevity, not all possible types of payloads are included in Table 6. Table 6

[0046] Regarding Table 6, JVET-AD2005 provides the following semantics: sei_reserved_payload_extension_data shall not be present in a bitstream conforming to this version of this specification. However, decoders conforming to this version of this specification shall ignore the presence and value of sei_reserved_payload_extension_data. When present, the length in bits of sei_reserved_payload_extension_data is equal to 8*payloadSize-nEarlierBits-nPayloadZeroBits-1, where nEarlierBits is the number of bits in the sei_payload() syntax structure preceding the sei_reserved_payload_extension_data syntax element, and nPayloadZeroBits is the number of sei_payload_bit_equal_to_zero syntax elements at the end of the sei_payload() syntax structure. If more_data_in_payload() is TRUE and nPayloadZeroBits is not equal to 7 after parsing the SEI message syntax structure (eg, buffering_period() syntax structure), PayloadBits is set equal to 8*payloadSize-nPayloadZeroBits-1; otherwise, PayloadBits is set equal to 8*payloadSize. sei_payload_bit_equal_to_one should be equal to 1. sei_payload_bit_equal_to_zero should be equal to 0. NOTE—SEI messages with the same payloadType value are conceptually the same SEI messages regardless of whether they are contained in prefix or suffix SEI NAL units. NOTE—For the SEI messages specified in this specification and the VSEI specification (ITU-T H.274 | ISO / IEC 23002-7), the payloadType values ​​are aligned with similar SEI messages specified in AVC (Rec. ITU-T H.264 | ISO / IEC 14496-10) and HEVC (Rec. ITU-T H.265 | ISO / IEC 23008-2). The semantics and persistence scope of each SEI message is specified in the semantics specification of each specific SEI message. NOTE—Summarizes the persistence information of SEI messages.

[0047] JVET-AD2005 also provides the following: SEI messages having the syntax structure identified in [Table 6] may be used with the bitstreams specified in this specification, and these SEI messages are specified in other specifications, including Rec. ITU-T H.274 | ISO / IEC 23002-7, ISO / IEC 23001-11, or ISO / IEC 23090-13. When any of the specific SEI messages specified in Rec. ITU-T H.274 | ISO / IEC 23002-7, ISO / IEC 23001-11 or ISO / IEC 23090-13 are included in the bitstream specified in this specification, the SEI payload syntax shall be as specified in Rec. ITU-T H.274 | ISO / IEC 23002-7, ISO / IEC 23001-11 or ISO / IEC 23090-13, respectively, the syntax shall be included in the sei_payload() syntax structure as specified in clause D.2.1 and the payloadType values ​​specified in clause D.2.1 shall be used. The SEI payload syntax specified in Rec. ITU-T H.274 | ISO / IEC 23002-7, ISO / IEC 23001-11 or ISO / IEC 23090-13, respectively, shall apply. 23090-13, and in addition, any SEI message-specific constraints, variables, and semantics specified in this annex for that particular SEI message shall apply. As described above, the value of PayloadBits as specified in section D.2.2 is passed to the parser of the SEI message syntax structure specified in Rec. ITU-T H.274 | ISO / IEC 23002-7, ISO / IEC 23001-11, and ISO / IEC 23090-13. NOTE—The definition of an IRAP picture in the VSEI specification is as follows: A coded picture from which all pictures in the same layer in decoding order and output order can be decoded without first decoding any pictures in the same layer that are earlier in the decoding order in the coded video bitstream. Therefore, a GDR picture with ph_recovery_poc_cnt equal to 0 in a VVC bitstream is an IRAP picture according to the definition of an IRAP picture in the VSEI specification.

[0048] JVET-AD2005 also provides the following with respect to the use of the neural network post-filter characteristics SEI message: Let currPic be the cropped decoded output picture, the neural network post-processing filter (NNPF) defined by the neural network post-filter characteristics (NNPFC) SEI message activated by the neural network post-filter activation (NNPFA) SEI message, and currLayerId be the nuh_layer_id value of currPic. Bitstream conformance requirement When a picture unit contains an NNPFA SEI message, the value of ph_pic_output_flag in the picture header contained in the picture unit shall be equal to 1. NOTE—Since only the cropped decoded output pictures are used as input pictures of the NNPF, the value of ph_pic_output_flag in the picture header of the coded picture corresponding to each input picture of the NNPF is equal to 1. The variable pictureRateUpsamplingFlag is set equal to (nnpfc_purpose & 0x08)!=0. The variable numInputPics is set equal to nnpfc_num_input_pics_minus1+1. The variable numInferences is derived as follows: - The variable numPostRoll is set equal to the value of i if the following conditions exist such that nnpfc_interpolated_pics[i] is greater than 0 and the variable numInferences is set equal to 1+numPostRoll: pictureRateUpsamplingFlag is equal to 1; the NNPFA SEI message that activates this NNPF has nnpfa_persistence_flag equal to 1; nnpfc_interpolated_pics[i] is greater than 0 only for a single value of i greater than 0; and and currPic is the last picture in the bitstream in output order, with the last picture having a layer id equal to currLayerId nuh_layer_id. Otherwise, the variable numInferences is set equal to 1. For each value of j in the range 0 to numInferences-1 (inclusive), the following applies: - An array of inputPic[i] and inputPresentFlag[i] for i in the range 0 to numInputPics-1 (inclusive) indicating all input pictures and the presence of an input picture in the bitstream, respectively, is specified as follows: -When j is greater than 0, for each value of k in the range 0 to j-1 (inclusive), inputPic[k] is set to currPic, and inputPresentFlag[k] is set equal to 0. - The j-th input picture inputPic[j] is set equal to currPic, inputPresentFlag[j] is set equal to 1. -When numInputPics is greater than 1, the following applies to all i in the range of j+1 to numInputPics-j-1 (inclusive) For each value of , in increasing order of i: - inputPic[i] is set to prevPic and inputPresentFlag[i] is set to equal to 1: pictureRateUpsamplingFlag equal to 1, currPic is associated with a frame packing arrangement SEI message with fp_arrangement_type equal to 5 and with a specific value of fp_current_frame_is_frame0_flag, and there is In the cropped decoded output picture prevPic, the one with nuh_layer_id equal to currLayerId, preceding inputPic[i-1] in output order, and with fp_arrangement_type equal to 5 and with The last picture in output order among all the cropped decoded output pictures with the same value of fp_current_frame_is_frame0_flag. - Otherwise, inputPic[i] is set to prevPic and inputPresentFlag[i] is set to 1 if: pictureRateUpsamplingFlag is equal to 0, and there is a picture prevPic with a value equal to All cropped decoded layers that have nuh_layer_id of currLayerId and are before inputPic[i-1] in output order The last picture in the output order among the coded output pictures. - Otherwise, inputPic[i] is set to prevPic and inputPresentFlag[i] is set equal to 1 if the following conditions exist: currPic is not associated with a frame packing arrangement SEI message with fp_arrangement_type equal to 5, and there is a cropped decoded output picture prevPic that is the last picture in output order among all cropped decoded output pictures that have nuh_layer_id equal to currLayerId and that precede inputPic[i-1] in output order. - Otherwise, the following applies: - inputPic[i] is set to the same picture as inputPic[i-1] and inputPresentFlag[i] is set equal to 0. - Bitstream conformance requirement num_interpolated_pics[i-1] shall not be greater than zero. - For the purpose of interpreting the NNPFC SEI message, the following variables are specified: - If numInputPics is greater than 1 and there is a second NNPF defined by at least one NNPFC SEI message, activated by a NNPFA SEI message for currPic and with nnpfc_purpose equal to 4, the following applies: -CroppedWidth is set equal to nnpfc_pic_width_in_luma_samples defined for the second NNPF. -CroppedHeight is set equal to nnpfc_pic_height_in_luma_samples defined for the second NNPF. - Otherwise, the following applies: - CroppedWidth is set equal to the value of pps_pic_width_in_luma_samples-SubWidthC*(pps_conf_win_left_offset+pps_conf_win_right_offset) of currPic. - CroppedHeight is set equal to the value of pps_pic_height_in_luma_samples - SubHeightC * (pps_conf_win_top_offset + pps_conf_win_bottom_offset) of currPic. - the luma sample array CroppedYPic[i] and the chroma sample arrays CroppedCbPic[i] and CroppedCrPic[i], when present, are derived as follows for each value of i in the range 0 to numInputPics-1 (inclusive): -The variable sourcePic is exported as follows: - If inputPresentFlag[i] is equal to 1 or nnpfc_absent_input_pic_zero_flag is equal to 0, sourcePic is set to inputPic[i]. Otherwise (inputPresentFlag[i] is equal to 0 and nnpfc_absent_input_pic_zero_flag is equal to 1), sourcePic is set to the picture with a luma sample array of CroppedWidth×CroppedHeight samples equal to 0 and Cb and Cr sample arrays of (CroppedWidth / SubWidthC)×(CroppedHeight / SubHeightC) samples equal to 0. - If numInputPics is equal to 1, the following applies: - The luma sample array CroppedYPic[i] and the chroma sample arrays CroppedCbPic[i] and CroppedCrPic[i], when present, are set to the two-dimensional arrays of decoded sample values ​​of the Y, Cb and Cr components of sourcePic, respectively. - Otherwise (numInputPics is greater than 1), the following applies: -The variable sourceWidth is set equal to the pps_pic_width_in_luma_samples of sourcePic- SubWidthC*(pps_conf_win_left_offset+pps_conf_win_right_offset) value. -The variable sourceHeight is set equal to pps_pic_height_in_luma_samples of sourcePic- The value of SubHeightC*(pps_conf_win_top_offset+pps_conf_win_bottom_offset). -If sourceWidth is equal to CroppedWidth and sourceHeight is equal to CroppedHeight, then inputPic is set to be the same as sourcePic. - Otherwise (sourceWidth is not equal to CroppedWidth or sourceHeight is not equal to CroppedHeight), the following applies: - There shall be a NNPF, hereinafter referred to as super-resolution NNPF, defined by at least one NNPFC SEI message, activated by a NNPFA SEI message for sourcePic, and having nnpfc_purpose, nnpfc_pic_width_in_luma_samples equal to CroppedWidth, and nnpfc_pic_height_in_luma_samples equal to CroppedHeight. -resampledPic is set to the output of the neural network inference of the super-resolution NNPF with sourcePic as input. - Luma sample array CroppedYPic[i] and chroma sample array CroppedCbPic[i] and CroppedCrPic[i], when present, is set to a two-dimensional array of decoded sample values ​​of the Y, Cb, and Cr components of resampledPic, respectively. -BitDepthY and BitDepthC are both set equal to BitDepth. - ChromaFormatIdc is set equal to sps_chroma_format_idc. - Array StrengthControlVal[i] specifies the filter strength control values ​​for the input pictures of the NNPF for all values ​​of i in the range of 0 to numInputPics-1 (inclusive), derived as follows: -StrengthControlVal[i] is set to the value of (firstSliceQpY+QpBdOffset)÷(63+QpBdOffset), where firstSliceQpY is equal to the SliceQpY of the first slice of inputPic[i]. No more than two NNPFC SEI messages with the same nnpfc_id value should appear in one picture unit. When two NNPFC SEI messages exist in a picture unit with the same nnpfc_id value, these SEI messages should have different contents. When NNPFC SEI messages with the same nnpfc_id and different contents exist in the same picture unit, the two NNPFC SEI messages should be present in the same SEI NAL unit. When a picture contains NNPFA, the picture shall not be a non-output picture regardless of the value of nnpfa_cancel_flag in NNPFA.

[0049] As described above, JVET-AD2005 provides a NN post-filter supplemental enhancement information message. Specifically, JVET-AD2005 provides a neural network post-filter characteristics SEI message (payloadType==210) and a neural network post-filter activation SEI message (payloadType==211). Table 7 shows the syntax of the neural network post-filter characteristics SEI message provided in JVET-AD2006. It should be noted that the neural network post-filter characteristics SEI message can be referred to as NNPFC SEI. It should be noted that JVET-AD2006 is used for the following general process of the neural network post-filter: The input of this process is the bitstream BitstreamToFilter. The output of this process is a list of NNPF output pictures ListNnpfOutputPics. First, BitstreamToFilter is decoded and the list CroppedDecodedPictures is set to the list of cropped decoded pictures in output order resulting from decoding BitstreamToFilter. Second, as specified, for each cropped decoded picture in CroppedDecodedPictures for which one or more NNPFs are activated, the filtering process for one picture is repeatedly called in output order. The order of pictures in ListNnpfOutputPics is in output order. Within ListNnpfOutputPics, there shall be no more than one picture associated with any particular output time instance. When, for any particular picture in CroppedDecodedPictures, multiple NNPFs are activated and only one NNPF is allowed to be selected to apply (although any one of the NNPFs may be selected), the above constraints shall apply regardless of which NNPF is selected to apply to the particular picture.

[0050] In addition, it should be noted that JVET-AD2006 uses a neural network post-filter to perform the following filtering on an image: The filtering process specified in this subclause is applied to each cropped decoded picture in CroppedDecodedPictures for which one or more NNPFs are activated, which is referred to as the current picture. When the NNPF is applied to the current picture, a filtered and / or interpolated picture is generated by the NNPF by applying the NNPF process specified in the semantics of the NNPFC SEI message to the current picture in a block-by-block manner. When NNPF is applied to the current picture, the order in which the pictures generated by NNPF by applying the NNPF process are stored in the output tensor of NNPF is in the output order. When the applied NNPF is the last NNPF applied to the current picture, the pictures generated by the NNPF and output by the NNPF process are included in ListNnpfOutputPics in the same order as the pictures are stored in the output tensor of the NNPF. Table 7

[0051] Regarding Table 7, JVET-AD2006 provides the following semantics: The Neural Network Post-Filter Characteristics (NNPFC) SEI message specifies a neural network that can be used as a post-processing filter. The use of a specified Neural Network Post-Processing Filter (NNPF) for a particular picture is indicated by the Neural Network Post-Filter Activation (NNPFA) SEI message. The use of this SEI message requires the definition of the following variables: -The width and height of the input image in units of luminance samples, denoted in this article as CroppedWidth and CroppedHeight respectively. - The luma sample array CroppedYPic[idx] and the chroma sample arrays CroppedCbPic[idx] and CroppedCrPic[idx] of the input picture with index idx in the range 0 to numInputPics-1 (inclusive) are used as input to the NNPF when present. - The bit depth BitDepthY of the luma sample array of the input image. - The bit depth BitDepthC of the chroma sample array (if any) of the input picture. - a chroma format indicator, denoted herein by ChromaFormatIdc, as described in subclause 7.3. - When nnpfc_auxiliary_inp_idc is equal to 1, the filter strength control value array StrengthControlVal[idx] shall contain real numbers in the range 0 to 1 (inclusive) for the input pictures with index idx in the range 0 to numInputPics-1 (inclusive). The input picture with index 0 corresponds to the picture whose NNPF defined by the NNPFC SEI message is activated by the NNPFA SEI message. Input pictures with index i in the range of 1 to numInputPics-1, inclusive, precede the input picture with index i-1 in output order. The variables SubWidthC and SubHeightC are derived from ChromaFormatIdc as specified in Table 1. NOTE—There may be more than one NNPFC SEI message for the same picture. When more than one NNPFC SEI message with different nnpfc_id values ​​is present or activated for the same picture, they may have the same or different nnpfc_purpose and nnpfc_mode_idc values. nnpfc_purpose indicates the purpose of the NNPF, as specified in Table 8, where (nnpfc_urpose and bitMask) not equal to 0 indicates that the NNPF has a purpose associated with the bitMask value in Table 8. When nnpfc_purpose is greater than 0 and (nnpfc_urpose and bitMask) is equal to 0, the purpose associated with the bitMask value is not applicable to the NNPF. When nnpfc_pupose is equal to 0, the NNPF may be used as determined by the application. In bitstreams conforming to this version of this document, the value of nnpfc_purpose shall be in the range of 0 to 63, inclusive. Values ​​of 64 to 65 535, inclusive, for nnpfc_purpose are reserved for future use by ITU-T | ISO / IEC and shall not be present in bitstreams conforming to this version of this document. Decoders conforming to this version of this document shall ignore NNPFC SEI messages with nnpfc_purpose in the range of 64 to 65 535, inclusive. Table 8 The variables chromaUpsamplingFlag, resolutionResamplingFlag, pictureRateUpsamplingFlag, bitDepthUpsamplingFlag, and colourizationFlag specify whether nnpfc_purpose indicates that the purpose of the NNPF includes chroma upsampling, resolution resampling, picture rate upsampling, bit depth upsampling, and colourization, respectively, and are derived as follows: chromaUpsamplingFlag=((nnpfc_purpose&0x02)>0)? 1:0 resolutionResamplingFlag=((nnpfc_purpose&0x04)>0)? 1:0 pictureRateUpsamplingFlag=((nnpfc_purpose&0x08)>0)? 1:0 bitDepthUpsamplingFlag=((nnpfc_purpose&0x10)>0)? 1:0 colourizationFlag=((nnpfc_purpose&0x20)>0)? 1:0 NOTE - When ITU-T | ISO / IEC uses the reserved value of nnpfc_purpose in the future, the syntax of this SEI message may be extended with syntax elements whose presence is conditional on nnpfc_purpose being equal to that value. When ChromaFormatIdc is equal to 3, chromaUpsamplingFlag shall be equal to 0. When ChromaFormatIdc or chromaUpsamplingFlag is not equal to 0, colourizationFlag shall be equal to 0. When pictureRateUpsamplingFlag is equal to 1 and the input picture with index 0 is associated with a frame packing arrangement SEI message with fp_arrangement_type equal to 5, all input pictures are associated with a frame packing arrangement SEI message with fp_arrangement_type equal to 5 and with the same value of fp_current_frame_is_frame0_flag. nnpfc_id contains an identification number that can be used to identify the NNPF. The value of nnpfc_id should be in the range of 0 to 232-2, inclusive. The values ​​of nnpfc_id from 256 to 511, inclusive, and from 231 to 232-2, inclusive, are reserved for future use by ITU-T|ISO / IEC. A decoder conforming to this version of this document that encounters an NNPFC SEI message with a nnpfc_id in the range of 256 to 511, inclusive, or in the range of 231 to 232-2, inclusive, shall ignore the SEI message. When the NNPFC SEI message is the first NNPFC SEI message in decoding order with a particular nnpfc_id value within the current CLVS, the following applies: - This SEI message specifies the basic NNPF. - This SEI message is related to the current decoded picture and all subsequent decoded pictures of the current layer in output order until the end of the current CLVS. nnpfc_base_flag equal to 1 specifies that the SEI message specifies a base NNPF. nnpf_base_flag equal to 0 specifies that the SEI message specifies an update relative to the base NNPF. The following constraints apply to the value of nnpfc_base_flag: - nnpfc_base_flag shall be equal to 1 when the NNPFC SEI message is the first NNPFC SEI message in decoding order with a particular nnpfc_id value within the current CLVS. - When the NNPFC SEI message nnpfcB is not the first in decoding order with a specific nnpfc_id value within the current CLVS When the NNPFC SEI message is sent and the value of nnpfc_base_flag is equal to 1, the NNPFC SEI message shall be a repetition of the first NNPFC SEI message nnpfcA with the same nnpfc_id value in decoding order, that is, the payload content of nnpfcB shall be the same as Same as nnpfcA. When nnpfc_base_flag is equal to 0, the following applies: - This SEI message defines updates relative to the previous base NNPF in decoding order with the same nnpfc_id value. Updates are not cumulative, but each update is applied to the base NNPF specified by the first NNPFC SEI message in decoding order with a particular nnpfc_id value within the current CLVS. The SEI message obtains the NNPF defined by the SEI message relative to the update of the basic NNPF definition with the same nnpfc_id value. - This SEI message relates to the current decoded picture and all subsequent decoded pictures of the current layer in output order, until the end of the current CLVS, or until but not including subsequent NNPFCs within the current CLVS that follow the current decoded picture in output order and that are associated with the current CLVS in decoding order with nnpfc_base_flag equal to 0 and this particular nnpfc_id value. The decoded picture with which the SEI message is associated, whichever is earlier. nnpfc_mode_idc equal to 0 indicates that the SEI message contains an ISO / IEC 15938-17 bitstream that specifies a base NNPF (when nnpfc_base_flag is equal to 1) or an update relative to a base NNPF with the same nnpfc_id value (when nnpfc_base_flag is equal to 0). When nnpfc_base_flag is equal to 1, nnpfc_mode_idc equal to 1 specifies that the base NNPF associated with the nnpfc_id value is a neural network identified by a URI indicated by nnpfc_uri, which has a format identified by a tag URI nnpfc_tag_uri. When nnpfc_base_flag is equal to 0, nnpfc_mode_idc equal to 1 specifies that updates relative to the base NNPF with the same nnpfc_id value are defined by the URI indicated by nnpfc_uri, which has a format identified by the tag URI nnpfc_tag_uri. In bitstreams conforming to this version of this document, the value of nnpfc_mode_idc shall be in the range of 0 to 1, inclusive. Nnpfc_mode_idc values ​​of 2 to 255, inclusive, are reserved for future use by ITU-T|ISO / IEC and shall not be present in bitstreams conforming to this version of this document. Decoders conforming to this version of this document shall ignore NNPFC SEI messages with nnpfc_mode_idc in the range of 2 to 255, inclusive. nnpfc_mode_idc values ​​greater than 255 shall not be present in bitstreams conforming to this version of this document and are not reserved for future use. In bitstreams conforming to this version of this document, nnpfc_reserved_zero_bit_a shall be equal to 0. When nnpfc_reserved_zero_bit_a is not equal to 0, the decoder shall ignore the NNPFC SEI. nnpfc_tag_uri contains a tag URI having syntax and semantics as specified in IETF RFC 4151, identifying the format and associated information about a neural network used as a base NNPF or an update relative to a base NNPF having the same nnpfc_id value specified by nnpfc_uri. NOTE—nnpfc_tag_uri enables the format of a neural network specified by nnrpf_uri to be uniquely identified without the need for a central registry authority. nnpfc_tag_uri equal to "tag:iso.org,2023:15938-17" indicates that the neural network identified by nnpfc_uri complies with ISO / IEC 15938-17. nnpfc_uri contains a URI having syntax and semantics as specified in IETF Internet Standard 66, identifying a neural network used as a base NNPF or an update relative to a base NNPF having the same nnpfc_id value. nnpfc_property_present_flag equal to 1 specifies that syntax elements related to filter purpose, input format, output format and complexity are present. nnpfc_property_present_flag equal to 0 specifies that syntax elements related to filter purpose, input format, output format and complexity are not present. When nnpfc_base_flag is equal to 1, nnpfc_property_present_flag shall be equal to 1. When nnpfc_property_present_flag is equal to 0, all syntax elements that may only be present when nnpfc_property_present_flag is equal to 1 are inferred to have values ​​equal to the corresponding syntax elements in the NNPFC SEI message containing the base NNPF for which this SEI message provides updates, respectively. When the NNPFC SEI message nnpfcCurr is not the first NNPFC SEI message in decoding order with a particular nnpfc_id value within the current CLVS, is not a repetition of the first NNPFC SEI message with that particular nnpfc_id (i.e., the value of nnpfc_base_flag is equal to 0), and the value of nnpfc_property_present_flag is equal to 1, the following constraints apply: - The value of nnpfc_purpose in the NNPFC SEI message shall be the same as the value of nnpfc_purpose in the first NNPFC SEI message in decoding order with a specific nnpfc_id value within the current CLVS. - After nnpfc_property_present_flag in decoding order in the NNPFC SEI message and before The value of the syntax element preceding nnpfc_complexity_info_present_flag shall be the same as that of the element with a specific The nnpfc_id value is the same as the value of the corresponding syntax element in the first NNPFC SEI message in decoding order. - in the first NNPFC SEI message with a particular nnpfc_id value within the current CLVS in decoding order, nnpfc_complexity_info_present_flag shall be equal to 0, or nnpfc_complexity_info_present_flag shall be equal to 1 (denoted as nnpfcBase in the following), and all of the following conditions apply: - nnpfc_parameter_type_idc in nnpfcCurr should be equal to nnpfc_parameter_type_idc in nnpfcBase. - nnpfc_log2_parameter_bit_length_minus3 in nnpfcCurr (when present) should be less than or equal to nnpfc_log2_parameter_bit_length_minus3 in nnpfcBase. - If nnpfc_num_parameters_idc in nnpfcBase is equal to 0, then nnpfc_num_parameters_idc of nnpfcCurr shall be equal to 0. - Otherwise (nnpfc_num_parameters_idc in nnpfcBase is greater than 0), nnpfc_num_parameters_idc in nnpfcCurr shall be greater than 0 and less than or equal to nnpfc_num_parameters_idc in nnpfcBase. - If nnpfc_num_kmac_operations_idc in nnpfcBase is equal to 0, then nnpfc_num_kmac_operations_idc in nnpfcCurr shall be equal to 0. - Otherwise (nnpfc_num_kmac_operations_idc in nnpfcBase is greater than 0), nnpfc_num_kmac_operations_idc in nnpfcCurr shall be greater than 0 and less than or equal to nnpfc_num_kmac_operations_idc in nnpfcBase. - If nnpfc_total_kilobyte_size in nnpfcBase is equal to 0, then nnpfc_total_kilobyte_size in nnpfcCurr should be equal to 0. - Otherwise (nnpfc_total_kilobyte_size in nnpfcBase is greater than 0), nnpfc_total_kilobyte_size in nnpfcCurr should be greater than 0 and less than or equal to nnpfc_total_kilobyte_size in nnpfcBase. nnpfc_num_input_pics_minus1 plus 1 specifies the number of pictures used as input to the NNPF. The value of npfc_num_input_pics_minus1 shall be in the range of 0 to 63, inclusive. When pictureRateUpsamplingFlag is equal to 1, the value of nnpfc_num_input_pics_minus1 shall be greater than 0. The variable numInputPics that specifies the number of pictures used as input to NNPF is derived as follows: numInputPics=nnpfc_num_input_pics_minus1+1 nnpfc_input_pic_output_flag[i] equal to 1 indicates that for the i-th input picture, the NNPF generates a corresponding output picture. nnpfc_input_pic_output_flag[i] equal to 0 indicates that for the i-th input picture, the NNPF does not generate a corresponding output picture. When nnpfc_num_input_pics_minus1 is equal to 0, nnpfc_input_pic_output_flag[0] is inferred to be equal to 1. When pictureRateUpsamplingFlag is equal to 0 and nnpfc_num_input_pics_minus1 is greater than 0, nnpfc_input_pic_output_flag[i] shall be equal to 1 for at least one value of i in the range of 0 to nnpfc_num_input_pics_minus1, inclusive. nnpfc_absent_input_pic_zero_flag equal to 1 indicates that the NNPF expects that input pictures not present in the bitstream will be represented by sample arrays with sample values ​​equal to 0. nnpfc_absent_input_pic_flag equal to 0 indicates that the NNPF expects that input pictures not present in the bitstream will be represented by the closest input picture in output order within the bitstream. nnpfc_out_sub_c_flag specifies the values ​​of the variables outSubWidthC and outSubHeightC when chromaUpsamplingFlag is equal to 1. nnpfc_out_sub_c_flag equal to 1 specifies that outSubWidthC is equal to 1 and outSubHeightC is equal to 1. nnpfc_out_sub_c_flag equal to 0 specifies that outSubWidthC is equal to 2 and outSubHeightC is equal to 1. The value of nnpfc_out_sub_c_flag shall be equal to 1 when ChromaFormatIdc is equal to 2 and nnpfc_out_sub_c_flag is present. When colourizationFlag is equal to 1, nnpfc_out_colour_format_idc specifies the colour format of the NNPF output and therefore specifies the values ​​of the variables outSubWidthC and outSubHeightC. nnpfc_out_colour_format_idc equal to 1 specifies that the colour format of the NNPF output is the 4:2:0 format, and outSubWidthC and outSubHeightC are both equal to 2. nnpfc_out_colour_format_idc equal to 2 specifies that the colour format of the NNPF output is the 4:2:2 format, outSubWidthC is equal to 2 and outSubHeightC is equal to 1. nnpfc_out_colour_format_idc equal to 3 specifies that the colour format of the NNPF output is the 4:4:4 format, and outSubWidthC and outSubHeightC are both equal to 1. The value of nnpfc_out_colour_format_idc shall not be equal to 0. When chromaUpsamplingFlag and colourizationFlag are both equal to 0, outSubWidthC and outSubHeightC are inferred to be equal to SubWidthC and SubHeightC, respectively. nnpfc_pic_width_num_minus1 plus 1 and nnpfc_pic_width_denom_minus1 plus 1 specify the numerator and denominator of the resampling ratio of the NNPF output picture width relative to CroppedWidth, respectively. The value of (nnpfc_pic_width_num_minus1+1)÷(nnpfc_pic_width_denom_minus1+1) should be in the range of 1 / 16 to 16, inclusive. When nnpfc_pic_width_num_minus1 and nnpfc_pic_width_denom_minus1 are not present, the values ​​of nnpfc_pic_width_num_minus1 and nnpfc_pic_width_denom_minus1 are inferred to be equal to 0. The variable nnpfcOutputPicWidth represents the width of the luma sample array of the picture produced by applying the NNPF identified by nnpfc_id to the input picture, and is derived as follows: nnpfcOutputPicWidth=Ceil(CroppedWidth* (nnpfc_pic_width_num_minus1+1)+(nnpfc_pic_width_denom_minus1+1)) Bitstream conformance requires that the value of nnpfcOutputPicWidth % outSubWidthC shall be equal to 0. nnpfc_pic_height_num_minus1 plus 1 and nnpfc_pic_height_denom_minus1 plus 1 specify the numerator and denominator of the resampling ratio of the NNPF output picture height relative to the CroppedHeight, respectively. The value of (nnpfc_pic_height_num_minus1+1)÷(nnpfc_pic_height_denom_minus1+1) should be in the range of 1 / 16 to 16, inclusive. When nnpfc_pic_height_num_minus1 and nnpfc_pic_height_denom_minus1 are not present, the values ​​of nnpfc_pic_height_num_minus1 and nnpfc_pic_height_denom_minus1 are inferred to be equal to 0. The variable nnpfcOutputPicHeight represents the height of the luma sample array of the picture produced by applying the NNPF identified by nnpfc_id to the input picture, and is derived as follows: nnpfcOutputPicHeight=Ceil(CroppedHeight* (nnpfc_pic_height_num_minus1+1)+(nnpfc_pic_height_denom_minus1+1)) Bitstream conformance requires that the value of nnpfcOutputPicHeight % outSubHeightC shall be equal to 0. When nnpfc_pic_width_num_minus1, nnpfc_pic_width_denom_minus1, nnpfc_pic_height_num_minus1, and nnpfc_pic_height_denom_minus1 are present, at least one of the following shall be true: -The value of nnpfcOutputPicWidth is not equal to CroppedWidth. -The value of nnpfcOutputPicHeight is not equal to CroppedHeight. nnpfc_interpolated_pics[i] specifies the number of interpolated pictures generated by the NNPF between the i-th and (i+1)-th pictures used as input to the NNPF. The value of nnpfc_interpolated_pics[i] shall be in the range of 0 to 63, inclusive. For at least one value of i in the range of 0 to nnpfc_num_input_pics_minus1-1, inclusive, the value of nnpfc_interpolated_pics[i] shall be greater than 0. The variable NumInpPicsInOutputTensor that specifies the number of pictures that have a corresponding input picture and are present in the output tensor of the NNPF, the variable InpIdx[idx] that specifies the input picture index of the idx-th picture that is present in the output tensor of the NNPF and has a corresponding input picture, and the variable numOutputPics that specifies the total number of pictures that are present in the output tensor of the NNPF are derived as follows: nnpfc_component_last_flag equal to 1 indicates that the last dimension in the input tensor inputTensor is input to NNPF, and the output tensor outputTensor generated from NNPF is used for the current channel. nnpfc_component_last_flag equal to 0 indicates that the third dimension in the input tensor inputTensor is input to NNPF, and the output tensor outputTensor generated from NNPF is used for the current channel. Note—The first dimension in the input and output tensors is used for batch indexing, which is a practice in some neural network frameworks. Although the formulas in the semantics of this SEI message use a batch size corresponding to batch index equal to 0, it is up to the post-processing implementation to determine the batch size used as input for neural network inference. Note - For example, when nnpfc_inp_order_idc is equal to 3 and nnpfc_auxiliary_inp_idc is equal to 1, there are 7 channels in the input tensor, including four luma matrices, two chroma matrices, and one auxiliary input matrix. In this case, the DeriveInputTensors() process will derive each of these 7 channels of the input tensor one by one, and when a specific channel among these channels is processed, that channel is called the current channel during processing. nnpfc_inp_format_idc indicates a method of converting sample values ​​of an input picture into input values ​​input to the NNPF. When nnpfc_inp_format_idc is equal to 0, the input values ​​input to the NNPF are real numbers, and the functions InpY() and InpC() are specified as follows: InpY(x)=x÷((1<<BitDepth Y )-1) InpC(x)=x÷((1<<BitDepth C )-1) When nnpfc_inp_format_idc is equal to 1, the input value to the NNPF is an unsigned integer, and the functions InpY() and InpC() are specified as follows: The variable inpTensorBitDepthY is derived from the syntax element nnpfc_inp_tensor_luma_bitdepth_minus8, as specified below. The variable inpTensorBitDepthC is derived from the syntax element nnpfc_inp_tensor_chroma_bitdepth_minus8, as specified below. Values ​​of nnpfc_inp_format_idc greater than 1 are reserved for use by future ITU-T|ISO / IEC specifications and shall not be present in bitstreams conforming to this version of this document. Decoders conforming to this version of this document shall ignore NNPFC SEI messages containing reserved values ​​of nnpfc_inp_format_idc. nnpfc_auxiliary_inp_idc greater than 0 indicates that auxiliary input data is present in the input tensor of the NNPF. nnpfc_auxiliary_inp_idc equal to 0 indicates that auxiliary input data is not present in the input tensor. nnpfc_auxiliary_inp_idc equal to 1 specifies that the auxiliary input data is derived as specified. In bitstreams conforming to this version of this document, the value of nnpfc_auxiliary_inp_idc shall be in the range of 0 to 1, inclusive. Values ​​of nnpfc_auxiliary_inp_idc from 2 to 255, inclusive, are reserved for future use by ITU-T|ISO / IEC and shall not be present in bitstreams conforming to this version of this specification. Decoders conforming to this version of this document shall ignore NNPFCSEI messages with nnpfc_auxiliary_inp_idc in the range of 2 to 255, inclusive. Values ​​of nnpfc_auxiliary_inp_idc greater than 255 shall not be present in bitstreams conforming to this version of this document and are not reserved for future use. nnpfc_inp_order_idc indicates the method of sorting the sample array of the input picture to form the input tensor to the NNPF. In bitstreams conforming to this version of this document, the value of nnpfc_inp_order_idc shall be in the range of 0 to 3, inclusive. Values ​​of nnpfc_inp_order_idc from 4 to 255, inclusive, are reserved for future use by ITU-T|ISO / IEC and shall not be present in bitstreams conforming to this version of this document. Decoders conforming to this version of this document shall ignore NNPFC SEI messages with nnpfc_inp_order_idc in the range of 4 to 255, inclusive. Values ​​of nnpfc_inp_order_idc greater than 255 shall not be present in bitstreams conforming to this version of this document and are not reserved for future use. When ChromaFormatIdc is not equal to 1, nnpfc_inp_order_idc shall not be equal to 3. When ChromaFormatIdc is equal to 0, nnpfc_inp_order_idc shall be equal to 0. When chromaUpsamplingFlag is not equal to 1, nnpfc_inp_order_idc shall not be equal to 0. Table 9 contains informative descriptions of the nnpfc_inp_order_idc values. Table 9 nnpfc_inp_tensor_luma_bitdepth_minus8 specifies the bit depth of the luma sample values ​​in the input integer tensor by adding 8. The value of inpTensorBitDepthY is derived as follows: inpTensorBitDepthY = nnpfc_inp_tensor_luma_bitdepth_minus8 + 8 The bitstream compliance requires that the value of nnpfc_inp_tensor_luma_bitdepth_minus8 should be in the range of 0 to 24 (including the end values). nnpfc_inp_tensor_chroma_bitdepth_minus8 plus 8 specifies the bit depth of the chroma sample values in the input integer tensor. The value of inpTensorBitDepthC is derived as follows: inpTensorBitDepth C = nnpfc_inp_tensor_chroma_bitdepth_minus8 + 8 The bitstream compliance requires that the value of nnpfc_inp_tensor_chroma_bitdepth_minus8 should be in the range of 0 to 24 (including the end values). When nnpfc_auxiliary_inp_idc is equal to 1, the variable strengthControlScaledVal is derived as follows: A block is a rectangular array of samples from a component of a picture (e.g., a luminance or chrominance component). The DeriveInputTensors() procedure of the input tensor inputTensor for deriving the given vertical sample coordinate cTop and horizontal sample coordinate cLeft of the top-left sample position of the sample block included in the specified input tensor is specified as follows: nnpfc_out_format_idc being equal to 0 indicates that the sample values output by NNPF are real numbers in the range of 0 to 1 (including the end values), linearly mapped to the unsigned integer value range of 0 to (1 << bitDepth) - 1 (including the end values) for any desired bit depth bitDepth, for subsequent post-processing or display. nnpfc_out_format_idc equal to 1 indicates that the luma sample values output by the NNPF are unsigned integers in the range of 0 to (1<<outTensorBitDepthY)-1 (including the end values), and the chroma sample values output by the NNPF are unsigned integers in the range of 0 to (1<<outTensorBitDepthC)-1 (including the end values). Values of nnpfc_out_format_idc greater than 1 are reserved for future ITU-T|ISO / IEC specifications and shall not be present in the bitstream conforming to this version of this document. Decoders conforming to this version of this document shall ignore NNPFC SEI messages containing reserved values of nnpfc_out_format_idc. nnpfc_out_order_idc indicates the output order of samples generated from the NNPF. In the bitstream conforming to this version of this document, the value of nnpfc_out_order_idc shall be in the range of 0 to 3 (including the end values). Values of nnpfc_out_order_idc from 4 to 255 (including the end values) are reserved for future ITU-T|ISO / IEC use and shall not be present in the bitstream conforming to this version of this document. Decoders conforming to this version of this document shall ignore NNPFC SEI messages with nnpfc_out_order_idc in the range of 4 to 255 (including the end values). Values of nnpfc_out_order_idc greater than 255 shall not be present in the bitstream conforming to this version of this document and are not reserved for future use. - When chromaUpsamplingFlag is equal to 1, nnpfc_out_order_idc shall not be equal to 0 or 3. When colourizationFlag is equal to 1, nnpfc_out_order_idc shall not be equal to 0. Table 10 contains an informative description of the nnpfc_out_order_idc values. Table 10 nnpfc_out_tensor_luma_bitdepth_minus8 plus 8 specifies the bit depth of the luma sample values in the output integer tensor. The value of nnpfc_out_tensor_luma_bitdepth_minus8 shall be in the range of 0 to 24 (including the end values). The value of outTensorBitDepthY is derived as follows: outTensorBitDepthY =nnpfc_out_tensor_luma_bitdepth_minus8+8 nnpfc_out_tensor_chroma_bitdepth_minus8 specifies the bit depth of the chroma sample values ​​in the output integer tensor plus 8. The value of nnpfc_out_tensor_chroma_bitdepth_minus8 should be in the range of 0 to 24 (inclusive). The value of outTensorBitDepthC is derived as follows: outTensorBitDepth C =nnpfc_out_tensor_chroma_bitdepth_minus8+8 When bitDepthUpsamplingFlag is not equal to 1, the value of nnpfc_out_format_idc shall be equal to 1 and at least one of the following conditions shall be true: - nnpfc_out_tensor_luma_bitdepth_minus8 exists and outTensorBitDepthY is greater than BitDepthY. - nnpfc_out_tensor_chroma_bitdepth_minus8 exists and outTensorBitDepthC is greater than BitDepthC. When nnpfc_inp_tensor_luma_bitdepth_minus8, nnpfc_inp_tensor_chroma_bitdepth_minus8, nnpfc_out_tensor_luma_bitdepth_minus8, and nnpfc_out_tensor_chroma_bitdepth_minus8 exist and outTensorBitDepthY is greater than inpTensorBitDepthY, outTensorBitDepthC should not be less than inpTensorBitDepthC. When nnpfc_inp_tensor_luma_bitdepth_minus8, nnpfc_inp_tensor_chroma_bitdepth_minus8, nnpfc_out_tensor_luma_bitdepth_minus8, and nnpfc_out_tensor_chroma_bitdepth_minus8 exist and outTensorBitDepthC is greater than inpTensorBitDepthC, outTensorBitDepthC should not be less than inpTensorBitDepthC. The StoreOutputTensors() procedure for deriving sample values ​​in the filtered output sample arrays FilteredYPic, FilteredCbPic, and FilteredCrPic from the output tensor outputTensor given vertical sample coordinates cTop and horizontal sample coordinates cLeft of the top left sample position of the sample block included in the specified input tensor is specified as follows: nnpfc_separate_colour_description_present_flag equal to 1 indicates that a different combination of color primaries, transfer characteristics, matrix coefficients, and scaling and offset values ​​applied in association with the matrix coefficients for the pictures generated from the NNPF is specified in the SEI message syntax structure. nnpfc_separate_colour_description_present_flag equal to 0 indicates that the combination of color primaries, transfer characteristics, matrix coefficients, and scaling and offset values ​​applied in association with the matrix coefficients for the pictures generated from the NNPF is the same as indicated in the VUI parameters of the CLVS. nnpfc_colour_primaries has the same semantics as specified for the vui_colour_primaries syntax element, as follows: vui_colour_primaries indicates the chromaticity coordinates of the source color primaries. Its semantics are as specified in Rec.ITU-T H.273 | ISO / IEC 23091-2 for the ColourPrimaries parameter. When the vui_colour_primaries syntax element is not present, the value of vui_colour_primaries is inferred to be equal to 2 (the chromaticity is unknown or unspecified or determined by other means not specified in this specification). The value of vui_colour_primaries is identified as reserved for future use by Rec.ITU-T H.273 | ISO / IEC 23091-2 and should not be present in bitstreams conforming to this version of this specification. A decoder should interpret the reserved value of vui_color_primaries as equal to the value 2. The following exceptions apply: -nnpfc_colour_primaries specifies the color primaries of the picture produced by applying the NNPF specified in the SEI message, instead of the color primaries used for CLVS. - When nnpfc_colour_primaries is not present in the NNPFC SEI message, the value of nnpfc_colour_primaries is inferred to be equal to vui_colour_primaries. nnpfc_transfer_characteristics has the same semantics as specified for the vui_transfer_characteristics syntax element, as follows: vui_transfer_characteristics indicates the transfer characteristics function of the color representation. Its semantics are as specified for the TransferCharacteristics parameter in Rec.ITU-T H.273 | ISO / IEC 23091-2. When the vui_transfer_characteristics syntax element is not present, the value of vui_transfer_characteristics is inferred to be equal to 2 (the transfer characteristics are unknown or unspecified or determined by other means not specified in this specification). The value of vui_transfer_characteristics is identified as reserved for future use by Rec.ITU-T H.273 | ISO / IEC 23091-2 and should not be present in bitstreams conforming to this version of this specification. The decoder should interpret the reserved value of vui_transfer_characteristics as equal to the value 2. The following exceptions apply: -nnpfc_transfer_characteristics specifies the transfer characteristics of the picture generated by applying the NNPF specified in the SEI message, instead of the transfer characteristics used for CLVS. - When nnpfc_transfer_characteristics is not present in NNPFC SEI message, inference The value of nnpfc_transfer_characteristics is equal to vui_transfer_characteristics. nnpfc_matrix_coeffs describes the formulas used to derive luma and chroma signals from the green, blue, and red or Y, Z, and X primaries. Its semantics apply to the picture produced by applying the NNPF specified in this SEI message, and as specified in Rec.ITU-T H.273 | ISO / IEC 23091-2 for MatrixCoefficients, where BitDepthY and BitDepthC are equal to outTensorBitDepthY and outTensorBitDepthC, respectively. - When nnpfc_matrix_coeffs is not present in the NNPFC SEI message, the value of nnpfc_matrix_coeffs is inferred to be equal to vui_matrix_coeffs. nnpfc_matrix_coeffs should not be equal to 0 unless both of the following are true: -nnpfc_out_tensor_chroma_bitdepth_minus8 is equal to nnpfc_out_tensor_luma_bitdepth_minus8. -nnpfc_out_order_idc is equal to 2, outSubHeightC is equal to 1, and outSubWidthC is equal to 1. nnpfc_matrix_coeffs shall not be equal to 8 unless one of the following conditions is true: - nnpfc_out_tensor_chroma_bitdepth_minus8 is equal to nnpfc_out_tensor_luma_bitdepth_minus8. -nnpfc_out_tensor_chroma_bitdepth_minus8 is equal to nnpfc_out_tensor_luma_bitdepth_minus8+1, nnpfc_out_order_idc is equal to 2, outSubHeightC is equal to 1, and outSubWidthC is equal to 1. nnpfc_full_range_flag indicates the scaling and offset values ​​applied in association with the matrix coefficients as specified by nnpfc_matrix_coeffs. Its semantics are as specified for the VideoFullRangeFlag parameter in Rec. ITU-T H.273 | ISO / IEC 23091-2. When not present, the value of nnpfc_full_range_flag is inferred to be equal to 0. nnpfc_chroma_loc_info_present_flag equal to 1 specifies that the nnpfc_chroma_sample_loc_type_frame syntax element is present in the NNPFC information SEI message. nnpfc_chroma_loc_info_present_flag equal to 0 specifies that the nnpfc_chroma_sample_loc_type_frame syntax element is not present in the NNPFC information SEI message. The value of nnpfc_chroma_loc_info_present_flag shall be equal to 0 when colourizationFlag is equal to 0 or nnpfc_out_colour_format_idc is not equal to 1. When nnpfc_chroma_sample_loc_type_frame is not equal to 6 and nnpfc_out_colour_format_idc is equal to 1, nnpfc_chroma_sample_loc_type_frame specifies the locations of the chroma samples of the output picture. nnpfc_chroma_sample_loc_type_frame equal to 6 and nnpfc_out_colour_format_idc equal to 1 indicates that the locations of the chroma samples are unknown or unspecified or specified by other means not specified in this document. The value of nnpfc_chroma_sample_loc_type_frame shall be in the range of 0 to 6, inclusive. nnpfc_overlap indicates the overlapping horizontal and vertical sample counts of adjacent input tensors to the NNPF. The value of nnpfc_overlap should be in the range of 0 to 16 383 (inclusive). nnpfc_constant_patch_size_flag equal to 1 indicates that NNPF accepts as input exactly the patch size indicated by nnpfc_patch_width_minus1 and nnpfc_patch_height_minus1. nnpfc_constant_patch_size_flag equal to 0 indicates that NNPF accepts as input any patch size of width inpPatchWidth and height inpPatchHeight, such that the width of the extended patch (i.e., patch plus overlapped area) equal to inpPatchWidth+2*nnpfc_overlap is a positive integer multiple of nnpfc_extended_patch_width_cd_delta_minus1+1+2*nnpfc_overlap, and the height of the extended patch equal to inpPatchHeight+2*nnpfc_overlap is a positive integer multiple of nnpfc_extended_patch_height_cd_delta_minus1+1+2*nnpfc_overlap. When nnpfc_constant_patch_size_flag is equal to 1, nnpfc_patch_width_minus1 plus 1 indicates the horizontal sample count of the block size required for input to the NNPF. The value of nnpfc_patch_width_minus1 should be in the range of 0 to Min(32 766, CroppedWidth-1), inclusive. When nnpfc_constant_patch_size_flag is equal to 1, nnpfc_patch_height_minus1 plus 1 indicates the vertical sample count of the block size required for input to the NNPF. The value of nnpfc_patch_height_minus1 should be in the range of 0 to Min(32 766, CroppedHeight-1), inclusive. When nnpfc_constant_patch_size_flag is equal to 0, nnpfc_extended_patch_width_cd_delta_minus1 plus 1 plus 2*nnpfc_overlap indicates the common divisor of all allowed values ​​of the width of the extended patch required for input to the NNPF. The value of nnpfc_extended_patch_width_cd_delta_minus1 should be in the range of 0 to Min(32766, CroppedWidth-1), inclusive. When nnpfc_constant_patch_size_flag is equal to 0, nnpfc_extended_patch_height_cd_delta_minus1 plus 1 plus 2*nnpfc_overlap indicates the common divisor of all allowed values ​​of the height of the extended patch required for input to the NNPF. The value of nnpfc_extended_patch_height_cd_delta_minus1 should be in the range of 0 to Min(32766, CroppedHeight-1), inclusive. Let the variables inpPatchWidth and inpPatchHeight be the patch size width and patch size height respectively. If nnpfc_constant_patch_size_flag is equal to 0, the following applies: - The values ​​of inpPatchWidth and inpPatchHeight are provided by external means not specified in this document, or are set by the post-processor itself. -The value of inpPatchWidth+2*nnpfc_overlap should be The value of inpPatchHeight+2*nnpfc_overlap shall be a positive integer multiple of nnpfc_extended_patch_width_cd_delta_minus1+1+2*nnpfc_overlap, and inpPatchWidth shall be less than or equal to CroppedWidth. The value of inpPatchHeight+2*nnpfc_overlap shall be a positive integer multiple of nnpfc_extended_patch_height_cd_delta_minus1+1+2*nnpfc_overlap, and inpPatchHeight shall be less than or equal to CroppedHeight. Otherwise (nnpfc_constant_patch_size_flag is equal to 1), the value of inpPatchWidth is set equal to nnpfc_patch_width_minus1+1, and the value of inpPatchHeight is set equal to nnpfc_patch_height_minus1+1. The variables outPatchWidth, outPatchHeight, horCScaling, verCScaling, outPatchCWidth, and outPatchCHeight are derived as follows: outPatchWidth = (nnpfcOutputPicWidth * inpPatchWidth) / CroppedWidth outPatchHeight = (nnpfcOutputPicHeight * inpPatchHeight) / CroppedHeight horCScaling = SubWidthC / outSubWidthC verCScaling = SubHeightC / outSubHeightC outPatchCWidth = outPatchWidth * horCScaling outPatchCHeight = outPatchHeight * verCScaling The bitstream compliance requires that outPatchWidth * CroppedWidth shall be equal to nnpfcOutputPicWidth * inpPatchWidth, and outPatchHeight * CroppedHeight shall be equal to nnpfcOutputPicHeight * inpPatchHeight. nnpfc_padding_type indicates the padding process when referring to sample positions outside the boundaries of the input picture as described in Table 11. In the bitstream conforming to this version of this document, the value of nnpfc_padding_type shall be in the range from 0 to 4 (including the end values). The values of nnpfc_padding_type from 5 to 15 (including the end values) are reserved for future use by ITU-T|ISO / IEC and shall not be present in the bitstream conforming to this version of this document. The decoder conforming to this version of this document shall ignore the NNPFC SEI messages with nnpfc_padding_type in the range from 5 to 15 (including the end values). The values of nnpfc_padding_type greater than 15 shall not be present in the bitstream conforming to this version of this document and are not reserved for future use. nnpfc_padding_type describe 0 Zero padding 1 Copy Fill 2 Reflection Fill 3 Fill around 4 Fixed padding 5..15 reserve Table 11 nnpfc_luma_padding_val indicates the luma value used for padding when nnpfc_padding_type is equal to 4. The value of innpfc_luma_padding_val shall be in the range from 0 to (1 << BitDepthY) - 1 (including the end values). The nnpfc_cb_padding_val indicates the Cb value used for padding when nnpfc_padding_type is equal to 4. The value of nnpfc_cb_padding_val shall be in the range of 0 to (1<<BitDepthC)-1, inclusive of the end values. The nnpfc_cr_padding_val indicates the Cr value used for padding when nnpfc_padding_type is equal to 4. The value of nnpfc_cr_padding_val shall be in the range of 0 to (1<<BitDepthC)-1, inclusive of the end values. The function InpSampleVal(y, x, picHeight, picWidth, croppedPic, cIdx), which takes as input the vertical sample position y, the horizontal sample position x, the picture height picHeight, the picture width picWidth, the sample array croppedPic, and the component index cIdx (equal to 0 for luminance, 1 for Cb, and 2 for Cr), returns the value of sampleVal derived as follows: Note - For the input to the function InpSampleVal(), the vertical position is listed before the horizontal position to be compatible with the input tensor convention of some inference engines. NNPF PostProcessingFilter() is the target NNPF derived as in the semantics of the NNPFA SEI message. The following exemplary process can be used with NNPF PostProcessingFilter() to generate the filtered and / or interpolated pictures in a block-by-block manner, which respectively include the Y, Cb, and Cr sample arrays FilteredYPic, FilteredCbPic, and FilteredCrPic, as indicated by nnpfc_out_order_idc: The picture generated by NNPF with index i contains the sample arrays FilteredYPic[i], FilteredCbPic[i], and FilteredCrPic[i] (when present) derived as specified. The pictures generated by NNPF do not include overlapping regions. The NNPF process consists of the processes defined as specified, followed by outputting the pictures generated by NNPF in increasing index order, where all the pictures generated by NNPF that are interpolated are output, and those pictures generated by NNPF corresponding to any input pictures to NNPF are output as specified in the semantics of the NNPFA SEI message. nnpfc_complexity_info_present_flag equal to 1 specifies that one or more syntax elements indicating the complexity of the NNPF associated with nnpfc_id are present. nnpfc_complexity_info_present_flag equal to 0 specifies that no syntax elements indicating the complexity of the NNPF associated with nnpfc_id are present. nnpfc_parameter_type_idc equal to 0 indicates that the neural network uses only integer parameters. nnpfc_parameter_type_flag equal to 1 indicates that the neural network can use floating point or integer parameters. nnpfc_parameter_type_idc equal to 2 indicates that the neural network uses only binary parameters. nnpfc_parameter_type_idc equal to 3 is reserved for future use by ITU-T|ISO / IEC and shall not be present in bitstreams conforming to this version of this document. Decoders conforming to this version of this document shall ignore NNPFC SEI messages with nnpfc_parameter_type_idc equal to 3. nnpfc_log2_parameter_bit_length_minus3 equal to 0, 1, 2, and 3 indicates that the neural network does not use parameters with bit lengths greater than 8, 16, 32, and 64, respectively. When nnpfc_parameter_type_idc is present and nnpfc_log2_parameter_bit_length_minus3 is not present, the neural network does not use parameters with bit lengths greater than 1. nnpfc_num_parameters_idc indicates the maximum number of neural network parameters for the NNPF in powers of 2 048. nnpfc_num_parameters_idc equal to 0 indicates the maximum number of unknown neural network parameters. The value nnpfc_num_parameters_idc should be in the range of 0 to 52, inclusive. Values ​​of nnpfc_num_parameters_idc greater than 52 are reserved for future use by ITU-T|ISO / IEC and should not be present in bitstreams conforming to this version of this document. Decoders conforming to this version of this document shall ignore NNPFC SEI messages with nnpfc_num_parameters_idc greater than 52. If the value of nnpfc_num_parameters_idc is greater than 0, the variable maxNumParameters is derived as follows: maxNumParameters=(2048<<nnpfc_num_parameters_idc)-1 Bitstream conformance requires that the number of neural network parameters of an NNPF shall be less than or equal to maxNumParameters. nnpfc_num_kmac_operations_idc greater than 0 indicates that the maximum number of multiply-accumulate operations per sample of the NNPF is less than or equal to nnpfc_num_kmac_operations_idc*1000. nnpfc_num_kmac_operations_idc equal to 0 indicates an unknown maximum number of multiply-accumulate operations. The value of nnpfc_num_kmac_operations_idc shall be in the range of 0 to 232-2, inclusive. nnpfc_total_kilobyte_size greater than 0 indicates the total size in kilobytes required to store the uncompressed parameters for the neural network. The total size in bits is a number equal to or greater than the sum of the bits used to store each parameter. nnpfc_total_kilobyte_size is the total size in bits divided by 8000, rounded up. nnpfc_total_kilobyte_size equal to 0 indicates that the total size required to store the parameters for the neural network is unknown. The value of nnpfc_total_kilobyte_size should be in the range of 0 to 232-2, inclusive. nnpfc_metadata_extension_num_bits equal to 0 specifies that there is no nnpfc_reserved_metadata_extension. nnpfc_metadata_extension_num_bits greater than 0 specifies the length in bits of the nnpfc_reserved_metadata_extension. In this version of this document, nnpfc_metadata_extension_num_bits shall be equal to 0. Values ​​of nnpfc_metadata_extension_num_bits in the range of 1 to 2048, inclusive, are reserved for future use by ITU-T|ISO / IEC and shall not be present in bitstreams conforming to this version of this document. Decoders conforming to this version of this document shall allow any value of nnpfc_metadata_extension_num_bits in the range of 0 to 2048, inclusive. Values ​​of nnpfc_metadata_extension_num_bits greater than 2048 shall not be present in bitstreams conforming to this version of this document and are not reserved for future use. nnpfc_reserved_metadata_extension shall not be present in a bitstream conforming to this version of this document. However, decoders conforming to this version of this document shall ignore the presence and value of nnpfc_reserved_metadata_extension. When present, the length in bits of nnpfc_reserved_metadata_extension is equal to nnpfc_metadata_extension_num_bits. In bitstreams conforming to this version of this document, nnpfc_reserved_zero_bit_b shall be equal to 0. When nnpfc_reserved_zero_bit_b is not equal to 0, the decoder shall ignore the NNPFC SEI message. nnpfc_payload_byte[i] contains the i-th byte of a bitstream conforming to ISO / IEC 15938-17. The byte sequence nnpfc_payload_byte[i] for all current values ​​of i shall be a complete bitstream conforming to ISO / IEC 15938-17.

[0052] Table 12 shows the syntax of the neural network post-filter activation SEI message provided in JVET-AD2006. Table 12

[0053] Regarding Table 12, JVET-AD2006 provides the following semantics: The Neural Network Post Filter Activation (NNPFA) SEI message activates or deactivates the possible use of the target Neural Network Post-Processing Filter (NNPF) identified by nnpfa_target_id and nnpfa_target_base_flag for post-processing filtering of a set of pictures. For a specific picture for which the NNPF is activated, the target NNPF is derived as follows: - If nnpfa_target_base_flag is equal to 1, the target NNPF is the base NNPF with nnpfc_id equal to nnpfa_target_id. Otherwise (nnpfa_target_base_flag is equal to 0), the target NNPF is the NNPF specified by the last NNPFC SEI message with nnpfc_id equal to nnpfa_target_id that precedes the first VCL NAL unit of the current picture in decoding order and is not a repetition of the NNPFC SEI message containing the base NNPF. NOTE—There may be several NNPFA SEI messages for the same picture, for example, when the NNPF is intended for different purposes or filters different color components. nnpfa_target_id indicates the target NNPF, which is specified by one or more NNPFC SEI messages related to the current picture and with nnpfc_id equal to nnfpa_target_id. The value of nnpfa_target_id shall be in the range of 0 to 232-2, inclusive. The NNPFA SEI message with a specific value of nnpfa_target_id shall not be present in the current PU unless one or both of the following conditions are true: - Within the current CLVS, there is an NNPFC SEI message with nnpfc_id equal to a specific value of nnpfa_target_id present in a PU preceding the current PU in decoding order. - There is a NNPFC SEI message with nnpfc_id equal to a specific value of nnpfa_target_id in the current PU. When a PU includes both an NNPFC SEI message with a specific value of nnpfc_id and an NNPFA SEI message with nnpfa_target_id equal to the specific value of nnpfc_id, the NNPFC SEI message shall precede the NNPFA SEI message in decoding order. nnpfa_cancel_flag equal to 1 indicates that the persistence of the target NNPF established by any previous NNPFA SEI message with the same nnpfa_target_id as the current SEI message is canceled, i.e., the target NNPF is not used again unless it is activated by another NNPFA SEI message with the same nnpfa_target_id as the current SEI message and nnpfa_cancel_flag equal to 0. nnpfa_cancel_flag equal to 0 indicates that nnpfa_target_base_flag, nnpfa_persistence_flag and nnpfa_num_output_entries follow. nnpfa_target_base_flag equal to 1 specifies that the target NNPF is the base NNPF with nnpfc_id equal to nnpfa_target_id. nnpfa_target_base_flag equal to 0 specifies that the target NNPF is the NNPF specified by the last NNPFC SEI message with nnpfc_id equal to nnpfa_target_id that precedes the first VCL NAL unit of the current picture in decoding order and is not a repetition of the NNPFC SEI message containing the base NNPF. nnpfa_persistence_flag specifies the persistence of the target NNPF for the current layer. nnpfa_persistence_flag equal to 0 specifies the target NNPF that can be used for post-processing filtering of the current picture. nnpfa_persistence_flag equal to 1 specifies that the target NNPF can be used for post-processing filtering of the current picture and all subsequent pictures of the current layer in output order until one or more of the following conditions are true: - A new CLVS starts for the current layer. - End of bitstream. - The picture in the current layer that is associated with an NNPFA SEI message with the same nnpfa_target_id as the current SEI message and nnpfa_cancel_flag equal to 1, which follows the current picture in output order. NOTE—The target NNPF in the current layer associated with the NNPFA SEI message with the same nnpfa_target_id as the current SEI message and nnpfa_cancel_flag equal to 1 does not apply to this subsequent picture. Let nnpfcTargetPictures be the picture set to which the last NNPFC SEI message with nnpfc_id equal to nnpfa_target_id, preceding the current NNPFA SEI message in decoding order, belongs. Let nnpfaTargetPictures be the picture set for which the target NNPF is activated by the current NNPFA SEI message. Bitstream conformance requires that any picture included in nnpfaTargetPictures shall also be included in nnpfcTargetPictures. nnpfa_num_output_entries specifies the number of nnpfa_output_flag[i] syntax elements present in the NNPFA SEI message. The value of nnpfa_num_output_entries shall be in the range of 0 to NumInpPicsInOutputTensor, inclusive. nnpfa_output_flag[i] equal to 1 specifies that the NNPF generated picture corresponding to the input picture with InpIdx[i] is output through the NNPF process activated by this NNPFA SEI message, where the NNPF process is specified in the semantics of the NNPFC SEI message. nnpfa_output_flag[i] equal to 0 specifies that the NNPF generated picture corresponding to the input picture with index InpIdx[i] is not output through the NNPF process activated by this NNPFA SEI message. When nnpfa_num_output_entries is less than NumInpPicsInOutputTensor, for each value of i in the range of nnpfa_num_output_entries to NumInpPicsInOutputTensor-1 (including the end value), nnpfa_output_flag[i] is inferred to be equal to 1.

[0054] The neural network post filter characteristics provided in the JVET-AD2006 SEI message may not be ideal. Interlaced scanning combines the fields to form an interlaced image. FIG. 9A to FIG. 9B is a conceptual diagram showing an example of combining fields to form an interlaced video image in accordance with one or more techniques of this disclosure. Fig. 9AIn the example shown in , the field at time t is combined with the field at time t+1 to form two interlaced images. For example, two 1080p60 fields (i.e., 1920x1080 resolution progressive scan fields at 60 Hz) are combined to form two 1080i30 images. Fig. 9B In the example shown in , the field at time t is combined with the field at time t+1 to form four interlaced images. For example, two 1080p60 fields are combined to form four 540p60 images. Displaying interlaced images on a progressive scan display requires deinterlacing. A neural network post-processing filter based on deinterlacing has the ability to generate multiple images from a single input image. Fig.10 is a conceptual diagram showing an example in which an interlaced frame is input to a deinterlacing-based neural network (ie, DCNN) and output as two full-size frames. Fig.10 In the example shown, a deinterlacing-based neural network post-processing filter predicts a field at each of time t and time t+1, which is combined with the corresponding field of the input frame to form a full-size frame at time t and time t+1. The reference—Zhu, H. et al. (2017), "Real-time Deep Video Deinterlacing", CoRR, abs / 1708.00187—provides an example of a deinterlacing-based neural network that can be used based on Fig.10 The examples shown in and the techniques described herein are used. "Deep Deinterlacing" by Bernasconi, M. et al. (2020) provides another example of a deinterlacing-based neural network that can be used according to the techniques described herein. In Bernasconi, the input to the neural network includes three fields: even lines from an image at time t-1, odd lines from an image at time t, and even lines from an image at time t+1. The purpose of the neural network is to predict the even lines missing in the image at time t. It should be noted that if the input to such a deinterlacing neural network filter is an image that includes an interlaced field, only a portion of the information from certain images will be utilized. Since interlaced content typically alternates between even and odd lines, in Bernasconi's example, the neural network will process the relevant even or odd lines. Therefore, there may be many ways in which a deinterlacing-based neural network post-processing filter can operate. The signaling provided in JVET-AD2006 may not be sufficient to support a deinterlacing-based neural network post-processing filter. According to the techniques of this article, a signaling mechanism is provided to support interlaced scanning based neural network post-processing filters.

[0055] Figure 1 1 is a block diagram illustrating an example of a system that may be configured to encode (e.g., encode and / or decode) video data in accordance with one or more techniques of this disclosure. System 100 represents an example of a system that may encapsulate video data in accordance with one or more techniques of this disclosure. Figure 1 As shown, system 100 includes source device 102, communication medium 110, and target device 120. Figure 1 In the example shown, source device 102 may include any device configured to encode video data and transmit the encoded video data to communication medium 110. Target device 120 may include any device configured to receive the encoded video data via communication medium 110 and decode the encoded video data. Source device 102 and / or target device 120 may include computing devices equipped for wired and / or wireless communication, and may include, for example, set-top boxes, digital video recorders, televisions, desktop, laptop or tablet computers, game consoles, medical imaging devices, and mobile devices (including, for example, smart phones, cellular phones, personal gaming devices).

[0056] The communication medium 110 may include any combination of wireless and wired communication media and / or storage devices. The communication medium 110 may include coaxial cables, fiber optic cables, twisted pair cables, wireless transmitters and receivers, routers, switches, repeaters, base stations, or any other device that can be used to facilitate communication between various devices and sites. The communication medium 110 may include one or more networks. For example, the communication medium 110 may include a network configured to allow access to the World Wide Web, such as the Internet. The network may operate according to a combination of one or more telecommunication protocols. The telecommunication protocol may include proprietary aspects and / or may include standardized telecommunication protocols. Examples of standardized telecommunication protocols include digital video broadcasting (DVB) standards, advanced television system committee (ATSC) standards, integrated service digital broadcasting (ISDB) standards, cable data service interface specifications (DOCSIS) standards, global mobile communication systems (GSM) standards, code division multiple access (CDMA) standards, 3rd generation partnership project (3GPP) standards, European Telecommunications Standards Institute (ETSI) standards, Internet Protocol (IP) standards, wireless application protocol (WAP) standards, and Institute of Electrical and Electronics Engineers (IEEE) standards.

[0057] The storage device may include any type of device or storage medium capable of storing data. The storage medium may include a tangible or non-transitory computer-readable medium. The computer-readable medium may include an optical disk, a flash memory, a magnetic memory, or any other suitable digital storage medium. In some examples, the memory device or a portion thereof may be described as a non-volatile memory, and in other examples, a portion of the memory device may be described as a volatile memory. Examples of volatile memory may include random access memory (RAM), dynamic random access memory (DRAM), and static random access memory (SRAM). Examples of non-volatile memory may include a magnetic hard disk, an optical disk, a floppy disk, a flash memory, or an electrically programmable memory (EPROM) or an electrically erasable and programmable (EEPROM) memory. The storage device may include a memory card (e.g., a secure digital (SD) memory card), an internal / external hard disk drive, and / or an internal / external solid-state drive. Data may be stored on the storage device according to a defined file format.

[0058] Figure 4 is a conceptual diagram illustrating examples of components that may be included in a specific implementation of system 100. Figure 4 In the exemplary implementation shown, system 100 includes one or more computing devices 402A-402N, a television service network 404, a television service provider site 406, a wide area network 408, a local area network 410, and one or more content provider sites 412A-412N. Figure 4 The specific implementations shown in represent examples of a system that can be configured to allow digital media content (such as movies, live sporting events, etc.) and data and applications associated therewith and media presentations to be distributed to and accessed by multiple computing devices (such as computing devices 402A to 402N). Figure 4 In the example shown, computing devices 402A to 402N may include any device configured to receive data from one or more of television service network 404, wide area network 408, and / or local area network 410. For example, computing devices 402A to 402N may be equipped for wired and / or wireless communications and may be configured to receive services over one or more data channels, and may include televisions, including so-called smart televisions, set-top boxes, and digital video recorders. In addition, computing devices 402A to 402N may include desktop computers, laptop or tablet computers, game consoles, mobile devices (including, for example, "smart" phones, cellular phones, and personal gaming devices).

[0059] The television service network 404 is an example of a network configured to enable distribution of digital media content that may include television services. For example, the television service network 404 may include a public air television network, a public or subscription-based satellite television service provider network, and a public or subscription-based cable television provider network and / or a cloud or Internet service provider. It should be noted that, although in some examples, the television service network 404 may be primarily used to allow the provision of television services, the television service network 404 may also allow the provision of other types of data and services according to any combination of the telecommunications protocols described herein. In addition, it should be noted that in some examples, the television service network 404 may allow two-way communication between the television service provider site 406 and one or more of the computing devices 402A to 402N. The television service network 404 may include any combination of wireless and / or wired communication media. The television service network 404 may include coaxial cables, fiber optic cables, twisted pair cables, wireless transmitters and receivers, routers, switches, repeaters, base stations, or any other device that can be used to facilitate communication between various devices and sites. The television service network 404 may operate according to a combination of one or more telecommunications protocols. The telecommunication protocol may include proprietary aspects and / or may include a standardized telecommunication protocol. Examples of standardized telecommunication protocols include the DVB standard, the ATSC standard, the ISDB standard, the DTMB standard, the DMB standard, the Data Over Cable Service Interface Specification (DOCSIS) standard, the HbbTV standard, the W3C standard, and the UPnP standard.

[0060] Reference again Figure 4 , the television service provider site 406 may be configured to distribute television services via the television service network 404. For example, the television service provider site 406 may include one or more broadcast stations, cable television providers or satellite television providers, or Internet-based television providers. For example, the television service provider site 406 may be configured to receive transmissions (including television programs) via satellite uplink / downlink. In addition, as Figure 4 As shown, television service provider site 406 may be in communication with wide area network 408 and may be configured to receive data from content provider sites 412A through 412N. It should be noted that in some examples, television service provider site 406 may include a television studio and the content may originate from the television studio.

[0061] Wide area network 408 may include a packet-based network and operate according to a combination of one or more telecommunication protocols. Telecommunication protocols may include proprietary aspects and / or may include standardized telecommunication protocols. Examples of standardized telecommunication protocols include global system mobile communications (GSM) standards, code division multiple access (CDMA) standards, 3rd generation partnership project (3GPP) standards, European Telecommunications Standards Institute (ETSI) standards, European standards (EN), IP standards, wireless application protocols (WAP) standards, and Institute of Electrical and Electronics Engineers (IEEE) standards, such as, one or more IEEE 802 standards (e.g., Wi-Fi). Wide area network 408 may include any combination of wireless and / or wired communication media. Wide area network 408 may include coaxial cables, fiber optic cables, twisted pair cables, Ethernet cables, wireless transmitters and receivers, routers, switches, repeaters, base stations, or any other equipment that can be used to facilitate communication between various devices and sites. In one example, wide area network 408 may include the Internet. Local area network 410 may include a packet-based network and operate according to a combination of one or more telecommunication protocols. The local area network 410 may be distinguished from the wide area network 408 based on access levels and / or physical infrastructure. For example, the local area network 410 may include a secure home network.

[0062] Reference again Figure 4 , content provider sites 412A to 412N represent examples of sites that can provide multimedia content to television service provider sites 406 and / or computing devices 402A to 402N. For example, a content provider site may include a studio with one or more studio content servers, which are configured to provide multimedia files and / or streams to television service provider sites 406. In one example, content provider sites 412A to 412N may be configured to provide multimedia content using an IP suite. For example, a content provider site may be configured to provide multimedia content to a receiver device according to a real-time streaming protocol (RTSP), HTTP, etc. In addition, content provider sites 412A to 412N may be configured to provide data including hypertext-based content, etc. to one or more of receiver devices 402A to 402N and / or television service provider sites 406 via a wide area network 408. Content provider sites 412A to 412N may include one or more web servers. The data provided by data provider sites 412A to 412N may be defined according to a data format.

[0063] Reference again Figure 1, source device 102 includes video source 104, video encoder 106, data encapsulator 107 and interface 108. Video source 104 may include any device configured to capture and / or store video data. For example, video source 104 may include a camera and a storage device operably coupled thereto. Video encoder 106 may include any device configured to receive video data and generate a compatible bitstream representing the video data. A compatible bitstream may refer to a bitstream from which a video decoder can receive and reproduce video data. Various aspects of a compatible bitstream may be defined according to a video coding standard. When generating a compatible bitstream, video encoder 106 may compress the video data. The compression may be lossy (perceptible or imperceptible to the observer) or lossless. Figure 5 is a block diagram illustrating an example of a video encoder 500 that can implement the techniques described herein for encoding video data. It should be noted that although the exemplary video encoder 500 is illustrated as having different functional blocks, such illustration is intended for descriptive purposes and does not limit the video encoder 500 and / or its subcomponents to a particular hardware or software architecture. The functionality of the video encoder 500 may be implemented using any combination of hardware, firmware, and / or software implementations.

[0064] The video encoder 500 can perform intra-frame prediction encoding and inter-frame prediction encoding of a picture region and thus can be referred to as a hybrid video encoder. Figure 5 In the example shown, the video encoder 500 receives a source video block. In some examples, the source video block may include picture regions that have been divided according to a coding structure. For example, the source video data may include macroblocks, CTUs, CBs, sub-partitions thereof, and / or additional equivalent coding units. In some examples, the video encoder 500 may be configured to perform additional subdivisions of the source video block. It should be noted that the techniques described herein are generally applicable to video encoding regardless of how the source video data is divided before and / or during encoding. Figure 5 In the example shown, the video encoder 500 includes an adder 502, a transform coefficient generator 504, a coefficient quantization unit 506, an inverse quantization and transform coefficient processing unit 508, an adder 510, an intra-frame prediction processing unit 512, an inter-frame prediction processing unit 514, a filter unit 516, and an entropy coding unit 518. Figure 5 As shown, video encoder 500 receives source video blocks and outputs a bitstream.

[0065] exist Figure 5In the example shown, the video encoder 500 can generate residual data by subtracting the predicted video block from the source video block. The selection of the predicted video block is described in detail below. The summer 502 represents a component configured to perform the subtraction operation. In one example, the subtraction of the video block occurs in the pixel domain. The transform coefficient generator 504 applies a transform such as a discrete cosine transform (DCT), a discrete sine transform (DST), or a conceptually similar transform (for example, four 8×8 transforms can be applied to a 16×16 residual value array) to the residual block or its sub-partition to generate a set of residual transform coefficients. The transform coefficient generator 504 can be configured to perform any and all combinations of transforms included in the discrete triangular transform series, including its approximation. The transform coefficient generator 504 can output the transform coefficients to the coefficient quantization unit 506. The coefficient quantization unit 506 can be configured to perform quantization of the transform coefficients. The quantization process can reduce the bit depth associated with some or all coefficients. The degree of quantization can change the rate distortion (i.e., the relationship between bit rate and video quality) of the encoded video data. The degree of quantization can be modified by adjusting a quantization parameter (QP). The quantization parameter can be determined based on a slice level value and / or a CU level value (e.g., a CU delta QP value). The QP data may include any data used to determine the QP used to quantize a particular set of transform coefficients. Figure 5 As shown, the quantized transform coefficients (which may be referred to as level values) are output to the inverse quantization and transform coefficient processing unit 508. The inverse quantization and transform coefficient processing unit 508 may be configured to apply inverse quantization and inverse transformation to generate reconstructed residual data. Figure 5 As shown, at the summer 510, the reconstructed residual data can be added to the predicted video block. In this way, the encoded video block can be reconstructed, and the resulting reconstructed video block can be used to evaluate the coding quality of a given prediction, transformation and / or quantization. The video encoder 500 can be configured to perform multiple encoding rounds (e.g., performing encoding while changing one or more of the prediction, transformation parameters and quantization parameters). The rate distortion or other system parameters of the bitstream can be optimized based on the evaluation of the reconstructed video block. In addition, the reconstructed video block can be stored and used as a reference for predicting subsequent blocks.

[0066] Reference again Figure 5, the intra-frame prediction processing unit 512 may be configured to select an intra-frame prediction mode for a video block to be encoded. The intra-frame prediction processing unit 512 may be configured to evaluate a frame and determine an intra-frame prediction mode for encoding a current block. As described above, possible intra-frame prediction modes may include a planar prediction mode, a DC prediction mode, and an angular prediction mode. Furthermore, it should be noted that in some examples, the prediction mode of the chrominance component may be inferred based on the prediction mode of the luma prediction mode. The intra-frame prediction processing unit 512 may select an intra-frame prediction mode after performing one or more encoding rounds. Furthermore, in one example, the intra-frame prediction processing unit 512 may select a prediction mode based on a rate-distortion analysis. As Figure 5 As shown, the intra prediction processing unit 512 outputs intra prediction data (e.g., syntax elements) to the entropy coding unit 518 and the transform coefficient generator 504. As described above, the transform performed on the residual data can be mode-dependent (e.g., the secondary transform matrix can be determined based on the prediction mode).

[0067] Reference again Figure 5 , the inter-frame prediction processing unit 514 may be configured to perform inter-frame prediction encoding for the current video block. The inter-frame prediction processing unit 514 may be configured to receive a source video block and calculate a motion vector of a PU of the video block. The motion vector may indicate the displacement of a prediction unit of a video block within a current video frame relative to a prediction block within a reference frame. Inter-frame prediction encoding may use one or more reference pictures. In addition, motion prediction may be unidirectional prediction (using one motion vector) or bidirectional prediction (using two motion vectors). The inter-frame prediction processing unit 514 may be configured to select a prediction block by calculating a pixel difference determined by, for example, the sum of absolute differences (SAD), the sum of square differences (SSD), or other difference metrics. As described above, a motion vector may be determined and specified based on motion vector prediction. As described above, the inter-frame prediction processing unit 514 may be configured to perform motion vector prediction. The inter-frame prediction processing unit 514 may be configured to generate a prediction block using motion prediction data. For example, the inter-frame prediction processing unit 514 may locate a prediction video block ( Figure 5 ). It should be noted that the inter-frame prediction processing unit 514 can be further configured to apply one or more interpolation filters to the reconstructed residual block to calculate sub-integer pixel values ​​for motion estimation. The inter-frame prediction processing unit 514 can output the motion prediction data of the calculated motion vector to the entropy coding unit 518.

[0068] like Figure 5As shown, filter unit 516 receives the reconstructed video blocks and the encoding parameters and outputs the modified reconstructed video data. Filter unit 516 may be configured to perform deblocking and / or sample adaptive offset (SAO) filtering. SAO filtering is a nonlinear amplitude mapping that can be used to improve reconstruction by adding an offset to the reconstructed video data. It should be noted that, as Figure 5 As shown, the intra prediction processing unit 512 and the inter prediction processing unit 514 may receive the modified reconstructed video block via the filter unit 216. The entropy coding unit 518 receives the quantized transform coefficients and the prediction syntax data (i.e., the intra prediction data and the motion prediction data). It should be noted that in some examples, the coefficient quantization unit 506 may perform a scan of the matrix including the quantized transform coefficients before outputting the coefficients to the entropy coding unit 518. In other examples, the entropy coding unit 518 may perform the scan. The entropy coding unit 518 may be configured to perform entropy coding according to one or more of the techniques described herein. As such, the video encoder 500 represents an example of a device configured to generate encoded video data according to one or more techniques of the present disclosure.

[0069] Reference again Figure 1 , the data encapsulator 107 can receive the encoded video data and generate a compatible bitstream according to a defined data structure, for example, a sequence of NAL units. A device receiving the compatible bitstream can reproduce the video data therefrom. In addition, as described above, sub-bitstream extraction can refer to the process by which a device receiving a compatible bitstream forms a new compatible bitstream by discarding and / or modifying data in the received bitstream. It should be noted that the term conforming bitstream can be used instead of the term compatible bitstream. In one example, the data encapsulator 107 can be configured to generate syntax according to one or more techniques described herein. It should be noted that the data encapsulator 107 does not necessarily have to be located in the same physical device as the video encoder 106. For example, the functions described as being performed by the video encoder 106 and the data encapsulator 107 can be distributed in Figure 4 in the device shown.

[0070] As described above, the signaling provided in JVET-AD2006 may not be ideal. Specifically, the signaling provided in JVET-AD2006 may not be sufficient to support a neural network post-processing filter based on deinterlacing. In one example, according to the techniques of this article, a syntax element (e.g., nnpfc_per_output_deinterlaced_pics_flag) indicating the number of output pictures for each deinterlaced input picture to the NNPF may be signaled. It should be noted that some neural network post-filters for deinterlacing are trained based on top field first. In this case, the input picture needs to be flipped so that a neural network trained based on top field first can handle bottom field first input images (and vice versa). Fig.11 is a conceptual diagram illustrating an example of rotation relative to a deinterlacing-based neural network in accordance with one or more techniques of this disclosure. In one example, in accordance with the techniques herein, syntax may be provided to signal whether some of the input pictures need to be flipped. Further, it should be noted that a neural network trained to predict missing even rows in an image of interlaced content will fail when tested on images with odd rows missing (and vice versa). However, by applying a flip operation to the input and output of the deinterlacing neural network when necessary, it is possible to successfully predict both missing even and odd rows.

[0071] It should be noted that by applying a flipping operation (described below) to a model trained to predict missing even rows in an experiment using Bernasconi's model weights to predict missing odd rows in an image, a peak signal-to-noise ratio (PSNR) value comparable to that obtained when predicting the correct missing rows was achieved. Without using the flipping operation, a loss of more than 3 dB was incurred, when necessary. Table 13 summarizes the results of an exemplary experiment in the case of missing odd rows. Table 13

[0072] Table 14 shows an example of the relevant syntax of a neural network post-filter characteristics SEI message, in which a syntax element indicating the number of output pictures for each deinterlaced input picture to the NNPF is signaled according to the techniques of this document. Table 14

[0073] With respect to Table 14, in one example, the semantics may be based on the semantics provided above and the following semantics: nnpfc_purpose indicates the purpose of the NNPF, as specified in Table 15, where (nnpfc_urpose and bitMask) not equal to 0 indicates that the NNPF has a purpose associated with the bitMask value in Table 15. When nnpfc_purpose is greater than 0 and (nnpfc_urpose and bitMask) is equal to 0, the purpose associated with the bitMask value is not applicable to the NNPF. When nnpfc_pupose is equal to 0, the NNPF may be used as determined by the application and as specified by nnpfc_application_purpose_tag_uri. In bitstreams conforming to this version of this document, the value of nnpfc_purpose shall be in the range of 0 to 127, inclusive. Values ​​of nnpfc_purpose from 128 to 65 535, inclusive, are reserved for future use by ITU-T | ISO / IEC and shall not be present in bitstreams conforming to this version of this document. Decoders conforming to this version of this document shall ignore NNPFC SEI messages with nnpfc_purpose in the range of 128 to 65 535, inclusive. Table 15 The variables chromaUpsamplingFlag, resolutionResamplingFlag, pictureRateUpsamplingFlag, bitDepthUpsamplingFlag, and colourizationFlag specify whether nnpfc_purpose indicates that the purpose of the NNPF includes chroma upsampling, resolution resampling, picture rate upsampling, bit depth upsampling, and colourization, respectively, and are derived as follows: chromaUpsamplingFlag=((nnpfc_purpose&0x02)>0)? 1:0 resolutionResamplingFlag=((nnpfc_purpose&0x04)>0)? 1:0 pictureRateUpsamplingFlag=((nnpfc_purpose&0x08)>0)? 1:0 bitDepthUpsamplingFlag=((nnpfc_purpose&0x10)>0)? 1:0 colourizationFlag=((nnpfc_purpose&0x20)>0)? 1:0 deinterlacingFlag=((nnpfc_purpose&0x40)>0)? 1:0 NOTE - When ITU-T | ISO / IEC uses the reserved value of nnpfc_purpose in the future, the syntax of this SEI message may be extended with syntax elements whose presence is conditional on nnpfc_purpose being equal to that value. When ChromaFormatIdc is equal to 3, chromaUpsamplingFlag shall be equal to 0. When ChromaFormatIdc or chromaUpsamplingFlag is not equal to 0, colourizationFlag shall be equal to 0. When pictureRateUpsamplingFlag is equal to 1 and the input picture with index 0 is associated with a frame packing arrangement SEI message with fp_arrangement_type equal to 5, all input pictures are associated with a frame packing arrangement SEI message with fp_arrangement_type equal to 5 and with the same value of fp_current_frame_is_frame0_flag. nnpfc_per_output_deinterlaced_pics_flag equal to 1 indicates that two output pictures are generated for each interlaced input picture. nnpfc_per_output_deinterlaced_pics_flag equal to 0 indicates that one output picture is generated for each interlaced input picture. nnpfc_input_field_flip_flag[i] equal to 1 indicates that the i-th input picture to the deinterlacing post-processing filter is rotated 180 degrees counterclockwise. nnpfc_input_field_flip_flag[i] equal to 0 indicates that the i-th input picture to the deinterlacing post-processing filter is not rotated. or nnpfc_input_field_flip_flag[i] equal to 1 indicates that the i-th input picture is rotated 180 degrees counterclockwise during DeriveInputTensors() and StoreOutputTensor(). nnpfc_input_field_flip_flag[i] equal to 0 indicates that the i-th input picture is not rotated during DeriveInputTensors() and StoreOutputTensor(). or nnpfc_input_field_flip_flag[i] equal to 1 indicates that the i-th input picture is rotated 180 degrees counterclockwise during DeriveInputTensors(). nnpfc_input_field_flip_flag[i] equal to 0 indicates that the i-th input picture is not rotated during DeriveInputTensors(). In one example, the word "flip" may be used instead of "rotate 180 degrees." In one example, the words "clockwise" or "counterclockwise" may be removed. nnpfc_interpolated_pics[i] specifies the number of interpolated pictures generated by the NNPF between the i-th and (i+1)-th pictures used as input to the NNPF. The value of nnpfc_interpolated_pics[i] shall be in the range of 0 to 63, inclusive. For at least one value of i in the range of 0 to nnpfc_num_input_pics_minus1-1, inclusive, the value of nnpfc_interpolated_pics[i] shall be greater than 0. The variable NumInpPicsInOutputTensor that specifies the number of pictures that have a corresponding input picture and are present in the output tensor of the NNPF, the variable InpIdx[idx] that specifies the input picture index of the idx-th picture that is present in the output tensor of the NNPF and has a corresponding input picture, and the variable numOutputPics that specifies the total number of pictures that are present in the output tensor of the NNPF are derived as follows: In one example, because nnpfc_per_output_deinterlaced_pics_flag is signaled only when deinterlacingflag is equal to 1, the above formula can be written as follows:

[0074] It should be noted that the following conditions apply above and below: if(deinterlacingFlag&&nnpfc_input_field_flip_flag[i]) {...} can be split and written as: if(deinterlacingFlag) {if(nnpfc_input_field_flip_flag[i]){...} }

[0075] With respect to the example shown in Table 14, in one example, the process DeriveInputTensors() may be based on the following:

[0076] It should be noted that the following conditions apply above and below: if(deinterlacingFlag&&nnpfc_input_field_flip_flag[InpIdx[i] {...} can be split and written as: if(deinterlacingFlag) {if(nnpfc_input_field_flip_flag[InpIdx[1]]){...} }

[0077] With respect to the example shown in Table 14, in one example, the procedure StoreOutputTensors() may be based on the following:

[0078] It should be noted that in the above example, the semantics are defined as using clockwise rotation for DeriveInputTensors() and counterclockwise rotation for StoreOutputTensors(). In one example, the above semantics can be defined as using counterclockwise rotation for DeriveInputTensors() and clockwise rotation for StoreOutputTensors(). In this case, the process DeriveInputTensors() can be based on the following:

[0079] In this case, the procedure StoreOutputTensors() can be based on the following:

[0080] In one example, by utilizing the corresponding process described above, the above semantics can be defined as using counterclockwise rotation for DeriveInputTensors() and counterclockwise rotation for StoreOutputTensors(). In addition, in one example, by utilizing the corresponding process described above, the above semantics can be defined as using clockwise rotation for DeriveInputTensors() and clockwise rotation for StoreOutputTensors().

[0081] In one example, according to the techniques of this document, one or more additional syntax elements that provide information about the deinterlacing process NNPF can be signaled. In one example, a signal can be sent: a flag indicating whether the input picture is top field first or bottom field first, a flag indicating that the decoded picture input to the post-processing filter is composed of two interlaced fields together as a single picture or as separate fields, and / or a flag indicating whether the post-processing filter uses two fields or only one field in the interlaced picture for its deinterlacing process. Table 16 shows an example of the relevant syntax of the neural network post filter characteristics SEI message, in which the syntax elements indicating the deinterlacing process information are signaled according to the techniques of this document. Table 16

[0082] With respect to Table 16, in one example, the semantics may be based on the semantics provided above with respect to Table 14 and the following semantics: nnpfc_interlaced_input_type_flag equal to 1 indicates that the input to the deinterlacing post-processing filter consists of interlaced frames. nnpfc_interlaced_input_type_flag equal to 0 indicates that the input to the deinterlacing post-processing filter consists of fields. or nnpfc_interlaced_input_type_flag equal to 1 indicates that the input to the deinterlacing post-processing filter consists of interlaced pictures. nnpfc_interlaced_input_type_flag equal to 0 indicates that the input to the deinterlacing post-processing filter consists of fields. nnpfc_top_field_first_flag equal to 1 indicates that the decoded interlaced video sequence is top field first. nnpfc_top_field_first_flag equal to 0 indicates that the decoded interlaced video sequence is bottom field first. or nnpfc_top_field_first_flag equal to 1 indicates that the input to the deinterlacing post-processing filter is a top field first interlaced picture. nnpfc_top_field_first_flag equal to 0 indicates that the input to the deinterlacing post-processing filter is a bottom field first interlaced picture. or nnpfc_top_field_first_flag equal to 1 indicates that the input to the deinterlacing post-processing filter is a top field first interlaced picture or a field with the top field as the first picture in the CLVS. nnpfc_top_field_first_flag equal to 0 indicates that the input to the deinterlacing post-processing filter is a bottom field first interlaced picture or a field with the bottom field as the first picture in the CLVS. nnpfc_per_input_both_fields_flag[i] equal to 1 indicates that for the i-th input image to the post-processing filter, both its odd and even lines are used for deinterlacing by the post-processing filter. nnpfc_per_input_both fields_flag[i] equal to 0 indicates that for the i-th input image to the post-processing filter, only odd lines or only even lines are used for deinterlacing by the post-processing filter.

[0083] In one example, the syntax element nnpfc_top_field_first_flag may be conditionally signaled only when nnpfc_interlace_input_type_flag is equal to 1, as follows: if(nnpfc_interlaced_input_type_flag) nnpfc_top_field_first_flag

[0084] Regarding the syntax element nnpfc_top_field_first_flag, it should be noted that knowing whether the interlaced content is top field first or bottom field first is important for correct deinterlacing and correct display of the content. If the interlaced content is incorrectly interpreted as having the wrong field order (top first or bottom first), this may result in visual artifacts and distortion in the deinterlaced output. The field order also determines the order in which the fields are displayed to recreate a complete frame. By correctly identifying the field order, the deinterlacing algorithm can process the fields in the correct time order, thereby ensuring smooth motion and accurate reconstruction of the original image. Therefore, knowing the correct field order can be considered to be crucial for producing high-quality deinterlaced content and avoiding visual problems such as motion judder or combination artifacts.

[0085] Regarding the syntax element nnpfc_interlaced_input_type_flag, it should be noted that information about whether the input to the deinterlacing post-processing filter is in the form of fields or in the form of interlaced images is important for its efficient operation. For example, if the deinterlacing post-processing filter has been trained based on interlaced images (such as in Zhu), it is assumed that the input will be in the form of interlaced images. However, if the decoded pictures are in the form of fields, a pre-processing step is required to convert the decoded pictures into interlaced images before using these decoded pictures as input to the deinterlacing filter. By ensuring that the input format is aligned with the requirements of the deinterlacing filter, the best performance and accurate deinterlacing results can be achieved.

[0086] Regarding the syntax element nnpfc_per_input_both_fields_flag[i], it should be noted that in some cases, the deinterlacing neural network may utilize only half of the rows in the interlaced content to perform its function. As an example, as described above, for the neural network deinterlacing filter in Bernasconi, the input to the network consists of three fields: the even rows from the image at time t-1, the odd rows from the image at time t, and the even rows from the image at time t+1. The purpose of the neural network deinterlacing filter is to predict the even rows that are missing in the image at time t. If the input to such a deinterlacing neural network filter is an image that includes interlaced fields, only a portion of the information from certain images will be utilized. Because interlaced content typically alternates between even and odd rows, the network will focus on processing relevant even or odd rows. Therefore, irrelevant rows, such as odd rows from the image at time t-1, can be ignored.

[0087] In one example, according to the techniques herein, the syntax element nnpfc_per_output_deinterlaced_pics_flag in Table 14 and Table 16 may be replaced with the ue(v) syntax element based on the following semantics: nnpfc_num_output_deinterlaced_pics_per_input_minus1 plus 1 specifies the number of deinterlaced output pictures generated by the deinterlacing process NNPF. The value of nnpfc_num_output_deinterlaced_pics_per_input_minus1 should be in the range of 0 to 63 (inclusive). or The value of nnpfc_num_output_deinterlaced_pics_per_input_minus1 should be in the range of 0 to (nnpfc_num_input_pics_minus1+x), inclusive. Where x can be equal to 0 or 1 or 2 or 3 or any value in the range of 0 to 63 or 127 or 255.

[0088] In the example of signaling the syntax element nnpfc_num_output_deinterlaced_pics_per_input_minus1, numOutputPics, which specifies the total number of pictures present in the output tensor of the NNPF, may be derived as follows:

[0089] In one example, according to the techniques herein, rather than signaling instances of the syntax element nnpfc_input_field_flip_flag[i] (i=0; i<=nnpfc_num_input_pics_minus1; i++) as provided in Tables 14 and 16, the loops and instances of nnpfc_input_field_flip_flag[i] may be replaced with a single u(1) syntax element (which applies to all i in the range of 0 to nnpfc_num_input_pics_minus1, inclusive) based on the following semantics: nnpfc_input_field_flip_flag equal to 1 indicates that each input picture to the deinterlacing post-processing filter is rotated 180 degrees counter-clockwise. nnpfc_input_field_flip_flag equal to 0 indicates that each input to the deinterlacing post-processing filter is not rotated. or nnpfc_input_field_flip_flag equal to 1 indicates that each input picture is rotated 180 degrees counterclockwise during DeriveInputTensors() and StoreOutputTensor(). nnpfc_input_field_flip_flag equal to 0 indicates that each input picture is not rotated during DeriveInputTensors() and StoreOutputTensor(). or nnpfc_input_field_flip_flag equal to 1 indicates that each input picture is rotated 180 degrees counterclockwise during DeriveInputTensors(). nnpfc_input_field_flip_flag[i] equal to 0 indicates that each input picture is not rotated during DeriveInputTensors().

[0090] In this example, nnpfc_input_field_flip_flag[i] in the DeriveInputTensors() process and nnpfc_input_field_flip_flag[InpIdx[i]] in the StoreOutputTensor() process above can be replaced with nnpfc_input_field_flip_flag.

[0091] In this way, the video encoder 500 represents an example of a device configured to perform the following operations: sending a signal to notify a neural network post filter characteristics message, sending a signal to notify the neural network post filter characteristics message, sending a signal to notify the first syntax element from the neural network post filter characteristics message indicating the number of output pictures generated for each interlaced input picture, and sending a signal to notify the second syntax element from the neural network post filter characteristics message specifying whether the input picture is rotated.

[0092] As described above, the signaling provided in JVET-AD2006 may be less than ideal. Specifically, in JVET-AD2006, when the syntax element nnpfc_purpose is equal to 0, the NNPF may be used as determined by the application, and no information about the application-specific purpose is provided. In one example, according to the techniques herein, a signaling syntax element (e.g., nnpfc_application_purpose_tag_uri) may be sent when nnpfc_purpose is equal to 0 (i.e., when the NNPF purpose indicates an application-specific or application-determined use) to provide more details about the application-specific purpose in a decentralized manner. Tables 17A and 17B show examples of relevant syntax for a neural network post-filter characteristics SEI message, in which, according to the techniques herein, a signaling syntax element is sent to provide more details about the application-specific purpose. Table 17A Table 17B

[0093] With respect to Table 17A and Table 17B, in one example, the semantics may be based on the semantics provided above and the following semantics: nnpfc_purpose indicates the purpose of the NNPF, as specified in Table 8, where (nnpfc_urpose and bitMask) not equal to 0 indicates that the NNPF has a purpose associated with the bitMask value in Table 8. When nnpfc_purpose is greater than 0 and (nnpfc_urpose and bitMask) is equal to 0, the purpose associated with the bitMask value is not applicable to the NNPF. When nnpfc_pupose is equal to 0, the NNPF may be used as determined by the application and as specified by nnpfc_application_purpose_tag_uri. In bitstreams conforming to this version of this document, the value of nnpfc_purpose shall be in the range of 0 to 63, inclusive. Values ​​of nnpfc_purpose from 64 to 65535, inclusive, are reserved for future use by ITU-T | ISO / IEC and shall not be present in bitstreams conforming to this version of this document. Decoders conforming to this version of this document shall ignore NNPFC SEI messages with nnpfc_purpose in the range of 64 to 65535, inclusive. The variables chromaUpsamplingFlag, resolutionResamplingFlag, pictureRateUpsamplingFlag, bitDepthUpsamplingFlag, and colourizationFlag specify whether nnpfc_purpose indicates that the purpose of the NNPF includes chroma upsampling, resolution resampling, picture rate upsampling, bit depth upsampling, and colourization, respectively, and are derived as follows: chromaUpsamplingFlag=((nnpfc_purpose&0x02)>0)? 1:0 resolutionResamplingFlag=((nnpfc_purpose&0x04)>0)? 1:0 pictureRateUpsamplingFlag=((nnpfc_purpose&0x08)>0)? 1:0 bitDepthUpsamplingFlag=((nnpfc_purpose&0x10)>0)? 1:0 colourizationFlag=((nnpfc_purpose&0x20)>0)? 1:0 NOTE - When ITU-T | ISO / IEC uses the reserved value of nnpfc_purpose in the future, the syntax of this SEI message may be extended with syntax elements whose presence is conditional on nnpfc_purpose being equal to that value. When ChromaFormatIdc is equal to 3, chromaUpsamplingFlag shall be equal to 0. When ChromaFormatIdc or chromaUpsamplingFlag is not equal to 0, colourizationFlag shall be equal to 0. When pictureRateUpsamplingFlag is equal to 1 and the input picture with index 0 is associated with a frame packing arrangement SEI message with fp_arrangement_type equal to 5, all input pictures are associated with a frame packing arrangement SEI message with fp_arrangement_type equal to 5 and with the same value of fp_current_frame_is_frame0_flag. When nnpfc_purpose is equal to 0 (ie, when the NNPF purpose indicates an application-specific or application-determined usage), nnpfc_application_purpose_tag_uri specifies a tag URI with syntax and semantics as specified in IETF RFC 4151 that identifies the application-specific or application-determined purpose of the NNPF. NOTE - nnpfc_application_purpose_tag_uri enables the application-specific or application-determined purpose of an NNPF to be uniquely identified without the need for a central registration authority.

[0094] With respect to Table 17B, in one example, the semantics may be based on the semantics provided above and the following semantics: nnpfc_reserved_metadata_extension shall not be present in bitstreams conforming to this version of this document. However, decoders conforming to this version of this document shall ignore the presence and value of nnpfc_reserved_metadata_extension. When present, the length in bits of nnpfc_reserved_metadata_extension is equal to the length of nnpfc_metadata_extension_num_bits - (nnpfc_application_purpose_tag_uri) when nnpfc_purpose is equal to 0; otherwise, it is equal to nnpfc_metadata_extension_num_bits.

[0095] In one example, more than one nnpfc_application_purpose_tag_uri[i] value may be specified to allow multiple application-specific purposes of the NNPF to be defined in accordance with the techniques herein. Tables 18A and 18B show examples of relevant syntax for a neural network post-filter characteristics SEI message, where more than one syntax element may be signaled to provide more details about the application-specific purpose in accordance with the techniques herein. Table 18A Table 18B

[0096] With respect to Table 18A and Table 18B, in one example, the semantics may be based on the semantics provided above and the following semantics: nnpfc_purpose indicates the purpose of the NNPF, as specified in Table 8, where (nnpfc_urpose and bitMask) not equal to 0 indicates that the NNPF has a purpose associated with the bitMask value in Table 8. When nnpfc_purpose is greater than 0 and (nnpfc_urpose and bitMask) is equal to 0, the purpose associated with the bitMask value is not applicable to the NNPF. When nnpfc_pupose is equal to 0, the NNPF may be used as determined by the application and as specified by nnpfc_application_purpose_tag_uri. In bitstreams conforming to this version of this document, the value of nnpfc_purpose shall be in the range of 0 to 63, inclusive. Values ​​of 64 to 65 535, inclusive, for nnpfc_purpose are reserved for future use by ITU-T | ISO / IEC and shall not be present in bitstreams conforming to this version of this document. Decoders conforming to this version of this document shall ignore NNPFC SEI messages with nnpfc_purpose in the range of 64 to 65 535, inclusive. The variables chromaUpsamplingFlag, resolutionResamplingFlag, pictureRateUpsamplingFlag, bitDepthUpsamplingFlag, and colourizationFlag specify whether nnpfc_purpose indicates that the purpose of the NNPF includes chroma upsampling, resolution resampling, picture rate upsampling, bit depth upsampling, and colourization, respectively, and are derived as follows: chromaUpsamplingFlag=((nnpfc_purpose&0x02)>0)? 1:0 resolutionResamplingFlag=((nnpfc_purpose&0x04)>0)? 1:0 pictureRateUpsamplingFlag=((nnpfc_purpose&0x08)>0)? 1:0 bitDepthUpsamplingFlag=((nnpfc_purpose&0x10)>0)? 1:0 colourizationFlag=((nnpfc_purpose&0x20)>0)? 1:0 NOTE - When ITU-T | ISO / IEC uses the reserved value of nnpfc_purpose in the future, the syntax of this SEI message may be extended with syntax elements whose presence is conditional on nnpfc_purpose being equal to that value. When ChromaFormatIdc is equal to 3, chromaUpsamplingFlag shall be equal to 0. When ChromaFormatIdc or chromaUpsamplingFlag is not equal to 0, colourizationFlag shall be equal to 0. When pictureRateUpsamplingFlag is equal to 1 and the input picture with index 0 is associated with a frame packing arrangement SEI message with fp_arrangement_type equal to 5, all input pictures are associated with a frame packing arrangement SEI message with fp_arrangement_type equal to 5 and with the same value of fp_current_frame_is_frame0_flag. nnpfc_num_application_purposes_minus1 plus 1 specifies the number of nnpfc_application_purposes_tag_uri[i] values ​​signaled that specify the number of application-specific or application-determined purposes of the NNPF. The value of nnpfc_num_application_purposes_minus1 shall be in the range of 0 to 63 (or 127 or 255 or 1024 or some other upper limit), inclusive. When nnpfc_purpose is equal to 0 (ie, when the NNPF purpose indicates an application-specific or application-determined usage), nnpfc_application_purpose_tag_uri[i] specifies the i-th tag URI with syntax and semantics as specified in IETF RFC 4151, which identifies the i-th application-specific or application-determined purpose of the NNPF. NOTE - nnpfc_application_purpose_tag_uri[i] enables the application-specific or application-determined purpose of an NNPF to be uniquely identified without the need for a central registration authority. An NNPF with multiple nnpfc_application_purpose_tag_uri[i] provides the functionality specified by all individual nnpfc_application_purpose_tag_uri[i] values.

[0097] With respect to Table 18B, in one example, the semantics may be based on the semantics provided above and the following semantics: nnpfc_reserved_metadata_extension shall not be present in bitstreams conforming to this version of this document. However, decoders conforming to this version of this document shall ignore the presence and value of nnpfc_reserved_metadata_extension. When present, the length in bits of nnpfc_reserved_metadata_extension is equal to the sum of the length of nnpfc_metadata_extension_num_bits - (nnpfc_num_application_purposes_minus1) - (nnpfc_application_purpose_tag_uri[i]) when nnpfc_purpose is equal to 0, where i is in the range of 0 to nnpfc_num_application_purposes_minus1 (inclusive); otherwise, it is equal to nnpfc_metadata_extension_num_bits.

[0098] As described above, in JVET-AD2006, when the syntax element nnpfc_purpose is equal to 0, NNPF can be used as determined by the application, and no information about application-specific purposes is provided. It should be noted that in this case, in JVET-AD2006, the defined NNPF purposes (e.g., overall visual quality improvement, chroma upsampling, resolution upsampling, picture rate upsampling, bit depth upsampling, coloring) cannot be specified together with application-specific purposes. For example, NNPF may support one or more of the currently defined NNPF purposes, and include additional purposes that may be application-specific. In one example, according to the techniques of this article, bits are used to specify additional purposes of NNPF as determined by the application. This allows the purpose determined by the application to be used together with other NNPF purposes that have been defined or may be defined in the future. In one example, according to the techniques of this article, the semantics of the syntax element nnpfc_purpose may be based on the following: nnpfc_purpose indicates the purpose of the NNPF, as specified in Table 19, where (nnpfc_urpose and bitMask) not equal to 0 indicates that the NNPF has a purpose associated with the bitMask value in Table 19. When nnpfc_purpose is greater than 0 and (nnpfc_urpose and bitMask) is equal to 0, the purpose associated with the bitMask value is not applicable to the NNPF. nnpfc_purpose values ​​equal to 0 are reserved. In bitstreams conforming to this version of this document, the value of nnpfc_purpose shall be in the range of 0 to 127, inclusive. Values ​​of nnpfc_purpose from 128 to 65 535, inclusive, are reserved for future use by ITU-T | ISO / IEC and shall not be present in bitstreams conforming to this version of this document. Decoders conforming to this version of this document shall ignore NNPFC SEI messages with nnpfc_purpose in the range of 64 to 65 535, inclusive. Table 19 The variables chromaUpsamplingFlag, resolutionResamplingFlag, pictureRateUpsamplingFlag, bitDepthUpsamplingFlag, and colourizationFlag specify whether nnpfc_purpose indicates that the purpose of the NNPF includes chroma upsampling, resolution resampling, picture rate upsampling, bit depth upsampling, and colourization, respectively, and are derived as follows: chromaUpsamplingFlag=((nnpfc_purpose&0x02)>0)? 1:0 resolutionResamplingFlag=((nnpfc_purpose&0x04)>0)? 1:0 pictureRateUpsamplingFlag=((nnpfc_purpose&0x08)>0)? 1:0 bitDepthUpsamplingFlag=((nnpfc_purpose&0x10)>0)? 1:0 colourizationFlag=((nnpfc_purpose&0x20)>0)? 1:0 applicationPurposeFlag=((nnpfc_purpose&0x40)>0)? 1:0 NOTE - When ITU-T | ISO / IEC uses the reserved value of nnpfc_purpose in the future, the syntax of this SEI message may be extended with syntax elements whose presence is conditional on nnpfc_purpose being equal to that value. When ChromaFormatIdc is equal to 3, chromaUpsamplingFlag shall be equal to 0. When ChromaFormatIdc or chromaUpsamplingFlag is not equal to 0, colourizationFlag shall be equal to 0. When pictureRateUpsamplingFlag is equal to 1 and the input picture with index 0 is associated with a frame packing arrangement SEI message with fp_arrangement_type equal to 5, all input pictures are associated with a frame packing arrangement SEI message with fp_arrangement_type equal to 5 and with the same value of fp_current_frame_is_frame0_flag.

[0099] In one example, the semantics of nnpfc_purpose provided above can be used with the exemplary syntax and semantics provided above with respect to Tables 17A to 18B. That is, in each of Tables 17A to 18B, the condition "if (nnpfc_purpose == 0)" can be replaced with the condition "if (applicationPurposeFlag)".

[0100] Furthermore, it should be noted that although in the exemplary semantics, a specific bit (i.e., the 7th LSB) set to 1 is used to derive an applicationPurposeFlag equal to 1, in other examples, the meaning of this bit may be flipped (i.e., a bit equal to 0 may be used to derive an applicationPurposeFlag equal to 1). Furthermore, in other examples, instead of the 7th LSB, another bit may be used to indicate an application-determined or application-specific purpose. In another example, more than one bit (e.g., 2 bits) may be used to specify an application-determined or application-specific purpose.

[0101] In one example, according to the techniques herein, bits of the syntax element nnpfc_purpose that specify an application-specified purpose may be signaled for those cases where an application-specified purpose may be indicated in addition to one of the other defined NNPF purposes. However, for cases where the entire NNPF purpose is application-specified, a specific value of nnpfc_purpose equal to 0 is still defined. In one example, according to the techniques herein, the semantics of the syntax element nnpfc_purpose may be based on the following: nnpfc_purpose indicates the purpose of the NNPF, as specified in Table 19, where (nnpfc_urpose and bitMask) not equal to 0 indicates that the NNPF has a purpose associated with the bitMask value in Table 19. When nnpfc_purpose is greater than 0 and (nnpfc_urpose and bitMask) is equal to 0, the purpose associated with the bitMask value is not applicable to the NNPF. When nnpfc_pupose is equal to 0, the NNPF may be used as determined entirely by the application. In bitstreams conforming to this version of this document, the value of nnpfc_purpose shall be in the range of 0 to 127, inclusive. Values ​​of nnpfc_purpose from 128 to 65 535, inclusive, are reserved for future use by ITU-T | ISO / IEC and shall not be present in bitstreams conforming to this version of this document. Decoders conforming to this version of this document shall ignore NNPFC SEI messages with nnpfc_purpose in the range of 64 to 65 535, inclusive. The variables chromaUpsamplingFlag, resolutionResamplingFlag, pictureRateUpsamplingFlag, bitDepthUpsamplingFlag, and colourizationFlag specify whether nnpfc_purpose indicates that the purpose of the NNPF includes chroma upsampling, resolution resampling, picture rate upsampling, bit depth upsampling, and colourization, respectively, and are derived as follows: NOTE—When ITU-T | ISO / IEC uses the reserved value of nnpfc_purpose in the future, the syntax of this SEI message may be extended with syntax elements whose presence is conditional on nnpfc_purpose being equal to that value. When ChromaFormatIdc is equal to 3, chromaUpsamplingFlag shall be equal to 0. When ChromaFormatIdc or chromaUpsamplingFlag is not equal to 0, colourizationFlag shall be equal to 0. When pictureRateUpsamplingFlag is equal to 1 and the input picture with index 0 is associated with a frame packing arrangement SEI message with fp_arrangement_type equal to 5, all input pictures are associated with a frame packing arrangement SEI message with fp_arrangement_type equal to 5 and with the same value of fp_current_frame_is_frame0_flag.

[0102] In one example, the bit of the syntax element nnpfc_purpose indicating an application-determined or application-specific purpose may be used only when at least one of the other defined nnpfc_purpose bits is equal to 1. This prevents the use of only this bit to indicate a completely application-determined or application-specific nnpfc_purpose. In one example, this may be achieved by adding the following constraints to the exemplary semantics of nnpfc_purpose provided above: Bitstream conformance may require that when applicationPurposeFlag is equal to 1, at least one of chromaUpsamplingFlag, resolutionResamplingFlag, pictureRateUpsamplingFlag, bitDepthUpsamplingFlag, colourizationFlag (or other predefined NNPF purpose flags) should be equal to 1.

[0103] In another example, the derivation of applicationPurposeFlag in the exemplary semantics of nnpfc_purpose may be changed to the following:

[0104] In this case, applicationPurposeFlag is set equal to 1 when nnpfc_purpose is equal to 0 or when the application-determined or application-specific bit of nnpfc_purpose is set equal to 1 (ie, if (nnpfc_purpose & 0x40) is greater than 0).

[0105] In another example, a signaling flag (e.g., nnpfc_externally_specified_purpose_flag) may be signaled, where if the flag has a value of 1, the NNPF purpose is completely determined by the application and no signaling syntax element nnpfc_purpose is signaled. Additionally, if the flag has a value equal to 0, the signaling syntax element nnpfc_purpose is signaled. Table 20 shows an example of relevant syntax for a neural network post-filter characteristics SEI message, where a signaling flag may be signaled to provide more details about the application-specific purpose according to the techniques herein. Table 20

[0106] With respect to Table 20, in one example, the semantics may be based on the semantics provided above and the following semantics: nnpfc_externally_specified_purpose_flag equal to 1 specifies that the syntax element nnpfc_purpose is not present and the purpose of the NNPF is either externally specified or determined or specified by the application. nnpfc_externally_specified_purpose_flag equal to 0 specifies that the syntax element nnpfc_purpose is present and specifies the purpose of the NNPF.

[0107] In one example, the semantics of nnpfc_purpose provided above may be used with the exemplary syntax and semantics provided above with respect to Tables 17A to 18B. That is, in each of Tables 17A to 18B, the condition "if (nnpfc_purpose == 0)" may be replaced with the condition "if (nnpfc_externally_specified_purpose_flag)".

[0108] With respect to Table 20, the semantics of nnpfc_purpose may be based on the semantics provided above with respect to Table 7 (the semantics of nnpfc_purpose provided above include Table 19) or the following semantics: nnpfc_purpose indicates the purpose of the NNPF, as specified in Table 8, where (nnpfc_urpose and bitMask) not equal to 0 indicates that the NNPF has a purpose associated with the bitMask value in Table 8. When nnpfc_purpose is greater than 0 and (nnpfc_urpose and bitMask) is equal to 0, the purpose associated with the bitMask value is not applicable to the NNPF. nnpfc_purpose values ​​equal to 0 are reserved. In bitstreams conforming to this version of this document, the value of nnpfc_purpose shall be in the range of 0 to 63, inclusive. Values ​​of nnpfc_purpose from 64 to 65535, inclusive, are reserved for future use by ITU-T | ISO / IEC and shall not be present in bitstreams conforming to this version of this document. Decoders conforming to this version of this document shall ignore NNPFC SEI messages with nnpfc_purpose in the range of 64 to 65535, inclusive. The variables chromaUpsamplingFlag, resolutionResamplingFlag, pictureRateUpsamplingFlag, bitDepthUpsamplingFlag, and colourizationFlag specify whether nnpfc_purpose indicates that the purpose of the NNPF includes chroma upsampling, resolution resampling, picture rate upsampling, bit depth upsampling, and colourization, respectively, and are derived as follows: chromaUpsamplingFlag=((nnpfc_purpose&0x02)>0)? 1:∶0 resolutionResamplingFlag=((nnpfc_purpose&0x04)>0)? 1:0 pictureRateUpsamplingFlag=((nnpfc_purpose&0x08)>0)? 1:0 bitDepthUpsamplingFlag=((nnpfc_purpose&0x10)>0)? 1:0 colourizationFlag=((nnpfc_purpose&0x20)>0)? 1:0 NOTE - When ITU-T | ISO / IEC uses the reserved value of nnpfc_purpose in the future, the syntax of this SEI message may be extended with syntax elements whose presence is conditional on nnpfc_purpose being equal to that value. When ChromaFormatIdc is equal to 3, chromaUpsamplingFlag shall be equal to 0. When ChromaFormatIdc or chromaUpsamplingFlag is not equal to 0, colourizationFlag shall be equal to 0. When pictureRateUpsamplingFlag is equal to 1 and the input picture with index 0 is associated with a frame packing arrangement SEI message with fp_arrangement_type equal to 5, all input pictures are associated with a frame packing arrangement SEI message with fp_arrangement_type equal to 5 and with the same value of fp_current_frame_is_frame0_flag.

[0109] In this way, the video encoder 500 represents an example of a device configured to perform the following operations: sending a signaling neural network post filter characteristics message, sending a first syntax element in the signaling neural network post filter characteristics message indicating whether the purpose of the neural network post filter is specified by the application, and sending a second syntax element in the signaling neural network post filter characteristics message that specifies a universal resource identifier indicating a purpose specified by the application.

[0110] Reference again Figure 1 , interface 108 may include any device configured to receive data generated by data encapsulator 107 and transmit and / or store the data to a communication medium. Interface 108 may include a network interface card such as an Ethernet card, and may include an optical transceiver, a radio frequency transceiver, or any other type of device that can send and / or receive information. In addition, interface 108 may include a computer system interface that enables files to be stored on a storage device. For example, interface 108 may include a chipset that supports peripheral component interconnect (PCI) and high-speed peripheral component interconnect (PCIe) bus protocols, dedicated bus protocols, universal serial bus (USB) protocols, I2C, or any other logical and physical structure that can be used to interconnect peer devices.

[0111] Reference again Figure 1 , target device 120 includes interface 122, data decapsulator 123, video decoder 124 and display 126. Interface 122 may include any device configured to receive data from a communication medium. Interface 122 may include a network interface card such as an Ethernet card, and may include an optical transceiver, a radio frequency transceiver, or any other type of device that can receive and / or send information. In addition, interface 122 may include a computer system interface that allows compatible video bitstreams to be retrieved from a storage device. For example, interface 122 may include a chipset that supports PCI and PCIe bus protocols, dedicated bus protocols, USB protocols, I2C, or any other logical and physical structure that can be used to interconnect peer devices. Data decapsulator 123 may be configured to receive and parse any example grammatical structure described herein.

[0112] Video decoder 124 may include any device configured to receive a bitstream (e.g., sub-bitstream extraction) and / or acceptable variants thereof and reproduce video data therefrom. Display 126 may include any device configured to display video data. Display 126 may include one of a variety of display devices such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display. Display 126 may include a high definition display or an ultra high definition display. It should be noted that although in Figure 1 In the example shown, the video decoder 124 is described as outputting data to a display 126, but the video decoder 124 can be configured to output video data to various types of devices and / or subcomponents thereof. For example, the video decoder 124 can be configured to output video data to any communication medium, as described herein.

[0113] Figure 6 600 is a block diagram illustrating an example of a video decoder that may be configured to decode video data in accordance with one or more techniques of this disclosure (e.g., a decoding process for reference picture list construction described above). In one example, the video decoder 600 may be configured to decode transform data and reconstruct residual data from transform coefficients based on the decoded transform data. The video decoder 600 may be configured to perform intra-prediction decoding and inter-prediction decoding, and may therefore be referred to as a hybrid decoder. The video decoder 600 may be configured to parse any combination of the syntax elements described above in Tables 1 to 20. The video decoder 600 may decode video based on or in accordance with the above process and also based on the parsed values ​​in Tables 1 to 20.

[0114] exist Figure 6 In the example shown, the video decoder 600 includes an entropy decoding unit 602, an inverse quantization unit 604, an inverse transform coefficient processing unit 606, an intra-frame prediction processing unit 608, an inter-frame prediction processing unit 610, a summer 612, a post-filter unit 614, and a reference buffer 616. The video decoder 600 can be configured to decode video data in a manner consistent with a video encoding system. It should be noted that although the exemplary video decoder 600 shown has different functional blocks, such illustrations are intended for descriptive purposes and do not limit the video decoder 600 and / or its subcomponents to a specific hardware or software architecture. The functionality of the video decoder 600 can be implemented using any combination of hardware, firmware, and / or software implementations.

[0115] like Figure 6As shown, the entropy decoding unit 602 receives an entropy-encoded bitstream. The entropy decoding unit 602 may be configured to decode syntax elements and quantized coefficients from the bitstream according to a process that is inverse to the entropy encoding process. The entropy decoding unit 602 may be configured to perform entropy decoding according to any of the entropy encoding techniques described above. The entropy decoding unit 602 may determine the values ​​of syntax elements in the encoded bitstream in a manner consistent with the video coding standard. Figure 6 As shown, the entropy decoding unit 602 can determine the quantization parameter, quantization coefficient value, transformation data and prediction data from the bit stream. Figure 6 In the example shown, the inverse quantization unit 604 and the inverse transform coefficient processing unit 606 receive the quantized coefficient values ​​from the entropy decoding unit 602 and output reconstructed residual data.

[0116] See again Figure 6 , the reconstructed residual data may be provided to the summer 612. The summer 612 may add the reconstructed residual data to the predicted video block and generate the reconstructed video data. The predicted video block may be determined according to a predictive video technique, namely intra prediction and inter prediction. The intra prediction processing unit 608 may be configured to receive an intra prediction syntax element and retrieve the predicted video block from a reference buffer 616. The reference buffer 616 may include a memory device configured to store one or more frames of video data. The intra prediction syntax element may identify an intra prediction mode, such as the intra prediction mode described above. The inter prediction processing unit 610 may receive the inter prediction syntax element and generate a motion vector to identify a predicted block in one or more reference frames stored in the reference buffer 616. The inter prediction processing unit 610 may generate a motion compensated block, possibly performing interpolation based on an interpolation filter. An identifier of an interpolation filter for motion estimation with sub-pixel precision may be included in the syntax element. The inter prediction processing unit 610 may use an interpolation filter to calculate an interpolated value for a sub-integer pixel of a reference block. The post filter unit 614 may be configured to perform filtering on the reconstructed video data. For example, the post filter unit 614 may be configured to perform deblocking and / or sample adaptive offset (SAO) filtering, for example based on parameters specified in the bitstream. Furthermore, it should be noted that in some examples, the post filter unit 614 may be configured to perform dedicated arbitrary filtering (e.g., visual enhancement, such as mosquito noise removal). Figure 6 As shown, the video decoder 600 may output a reconstructed video block. In this way, the video encoder 600 represents an example of a device configured to receive a neural network post filter characteristics message, parse a first syntax element from the neural network post filter characteristics message indicating the number of output pictures generated for each interlaced input picture, and parse a second syntax element from the neural network post filter characteristics message specifying whether the input picture is rotated.

[0117] In one or more examples, the functions may be implemented by hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or codes on or transmitted via a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media corresponding to tangible media such as data storage media, or propagation media including, for example, any medium that facilitates the transmission of a computer program from one place to another according to a communication protocol. Thus, computer-readable media may generally correspond to: (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. The data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the techniques described in the present disclosure. A computer program product may include a computer-readable medium.

[0118] By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or any other medium that can be used to store the required program code in the form of instructions or data structures and can be accessed by a computer. Moreover, any connection is appropriately referred to as a computer-readable medium. For example, if a coaxial cable, optical fiber cable, twisted pair, digital subscriber line (DSL) or wireless technology such as infrared, radio and microwave are used to transmit instructions from a website, server or other remote source, then the coaxial cable, optical fiber cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave are included in the definition of the medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carriers, signals or other transient media, but are directed to non-transient tangible storage media. As used herein, disks and optical disks include compact disks (CDs), laser disks, optical optical disks, digital versatile disks (DVDs), floppy disks and Blu-ray disks, wherein disks usually copy data magnetically, while optical disks use lasers to copy data optically. The above combination should also be included in the scope of computer-readable media.

[0119] Instructions may be executed by one or more processors such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the term "processor" as used herein may refer to any of the aforementioned structures or any other structure suitable for implementing the techniques described herein. In addition, in some aspects, the functions described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a combined codec. Moreover, these techniques may be implemented entirely in one or more circuits or logic elements.

[0120] The techniques of the present disclosure may be implemented in a variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC), or a set of ICs (e.g., a chipset). Various components, modules, or units are described in this disclosure to emphasize functional aspects of a device configured to perform the disclosed techniques, but do not necessarily need to be implemented by different hardware units. Instead, as described above, the various units may be combined in a codec hardware unit, or provided by a collection of interoperating hardware units including one or more processors as described above, in combination with appropriate software and / or firmware.

[0121] In addition, each functional block or various features of the base station equipment and terminal equipment used in each of the above-mentioned embodiments can be realized or executed by a circuit (usually an integrated circuit or multiple integrated circuits). The circuit designed to perform the functions described in this specification may include a general-purpose processor, a digital signal processor (DSP), a dedicated or general-purpose integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices or discrete hardware components or a combination thereof. The general-purpose processor can be a microprocessor, or alternatively, the processor can be a conventional processor, a controller, a microcontroller or a state machine. The general-purpose processor or each of the above-mentioned circuits can be configured by a digital circuit, or can be configured by an analog circuit. In addition, when the technology of making an integrated circuit that replaces the current integrated circuit appears due to the progress of semiconductor technology, the integrated circuit produced by the technology can also be used.

[0122] Various examples have been described. These and other examples are within the scope of the following claims.

Claims

1. A method for performing neural network filtering on video data, the method comprising: Receiving a neural network post-filter characteristic message; parsing a first syntax element from the neural network post filter characteristics message indicating a number of output pictures generated for each interlaced input picture; as well as A second syntax element from the neural network post filter characteristics message is parsed that specifies whether the input picture is rotated.

2. A device comprising one or more processors, wherein the one or more processors are configured to: Receiving a neural network post-filter characteristic message; parsing a first syntax element from the neural network post filter characteristics message indicating a number of output pictures generated for each interlaced input picture; and A second syntax element from the neural network post filter characteristics message is parsed that specifies whether the input picture is rotated.

3. The device of claim 2, wherein the device comprises a video decoder.

Images