System and method for signaling post neural network filter parameter information in video coding

Abstract

A method of signaling post-neural network loop filter information of video data is disclosed. The method comprises the following steps: sending a signal to notify a neural network post-filter characteristic message; signaling a syntax element specifying a padding type corresponding to the neural network post-filter characteristic message, wherein the syntax element includes a value specifying a surround padding type and a value specifying a fixed padding type; and in the event that the fixed padding type is specified, signaling a syntax element indicating a sample value to be used for padding.

Description

Technical Field

[0001] The present disclosure relates to video encoding, and more particularly to systems and methods for signaling neural network post-filter parameter information for encoding video. Background Art

[0002] Digital video functions can be incorporated into various devices, including digital televisions, laptops or desktop computers, tablet computers, digital recording devices, digital media players, video game devices, cellular phones (including so-called smart phones), medical imaging devices, etc. Digital video can be encoded according to video coding standards. Video coding standards define the format of compatible bit streams that encapsulate coded video data. A compatible bit stream is a data structure that can be received and decoded by a video decoding device to generate reconstructed video data. Video coding standards can be combined with video compression technology. Examples of video coding standards include ISO / IEC MPEG-4Visual and ITU-T H.264 (also known as ISO / IEC MPEG-4AVC) and High Efficiency Video Coding (HEVC). HEVC is described in High Efficiency Video Coding (HEVC) of the ITU-T H.265 Recommendation of December 2016, which is incorporated herein by reference and is referred to herein as ITU-T H.265. ITU-TH.265 is being considered for expansion and improvement to develop the next generation of video coding standards. For example, the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG), collectively referred to as the Joint Video Study Group (JVET), have standardized video coding techniques with compression capabilities significantly exceeding the current HEVC standard. The Joint Exploration Model 7 (JEM 7), the algorithm description of the Joint Exploration Test Model 7 (JEM 7), ISO / IEC JTC1 / SC29 / WG11 document: JVET-G1001 (July 2017, Turin, Italy), incorporated herein by reference, describes the coding features studied by JVET under the Joint Test Model, which is a potential enhanced video coding technology that exceeds the capabilities of ITU-T H.265. It should be noted that the coding features of JEM 7 are implemented in the JEM reference software. As used herein, the term JEM may collectively refer to the algorithms included in JEM 7 and the specific implementation of the JEM reference software. In addition, in response to the "Joint Call for Proposals on Video Compression with Capabilities beyond HEVC" jointly issued by VCEG and MPEG, at the 10th meeting of ISO / IEC JTC1 / SC29 / WG11 held in San Diego, California (San Diego, CA) from April 16 to 20, 2018, various groups proposed multiple descriptions of video coding tools.Based on the various descriptions of the video coding tools, the final initial draft text of the video coding specification is described in "Versatile Video Coding (Draft 1)" at the 10th ISO / IEC JTC1 / SC29 / WG11 meeting held from April 16 to 20, 2018 in San Diego, California, document JVET-J1001-v2, which is incorporated herein by reference and is referred to as JVET-J1001. The development of the video coding standard by VCEG and MPEG is known as the Versatile Video Coding (VVC) project. "Versatile Video Coding (Draft 10)" at the 20th ISO / IEC JTC1 / SC29 / WG11 meeting held from October 7 to 16, 2020 (conference call, document JVET-T2001-v2, which is incorporated herein by reference and is referred to as JVET-T2001) represents the current iteration of the draft text of the video coding specification corresponding to the VVC project.

[0003] Video compression technology can reduce the data requirements for storing and transmitting video data. Video compression technology can reduce data requirements by exploiting the redundancy inherent in video sequences. Video compression technology can divide the video sequence into smaller and continuous parts (i.e., a group of pictures within a video sequence, pictures within a group of pictures, regions within pictures, sub-regions within regions, etc.). Intra-frame prediction coding techniques (e.g., spatial prediction techniques within pictures) and inter-frame prediction techniques (i.e., techniques (time) between pictures) can be used to generate the difference between the unit video data to be encoded and the reference unit of the video data. The difference can be called residual data. The residual data can be encoded as quantized transform coefficients. Syntax elements can involve residual data and reference coding units (e.g., intra-frame prediction mode indexes and motion information). The residual data and syntax elements can be entropy encoded. Entropy-encoded residual data and syntax elements can be included in the data structure that forms a compatible bitstream. Summary of the invention

[0004] In one example, a method for signaling neural network post-loop filter information of encoded data is provided, the method comprising: signaling a neural network post-filter characteristics message; signaling a syntax element specifying a fill type corresponding to the neural network post-filter characteristics message, wherein the syntax element includes a value specifying a surround fill type and a value specifying a fixed fill type; and, in a case where the fixed fill type is specified, signaling a syntax element indicating a sample value to be used for filling.

[0005] In one example, a method for applying a neural network in-loop filter to reconstruct video includes: receiving a neural network post-filter characteristics message; parsing a syntax element identifying a fill type corresponding to the neural network post-filter characteristics message, wherein the syntax element includes a value specifying a surround fill type and a value specifying a fixed fill type; and, if the fixed fill type is specified, parsing a syntax element indicating a sample value to be used for filling.

[0006] In one example, a device is provided that includes one or more processors configured to perform the following operations: receive a neural network post-filter characteristics message; parse a syntax element identifying a fill type corresponding to the neural network post-filter characteristics message, wherein the syntax element includes a value specifying a surround fill type and a value specifying a fixed fill type; and, if the fixed fill type is specified, parse a syntax element indicating a sample value to be used for filling. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 is a block diagram illustrating an example of a system that may be configured to encode and decode video data in accordance with one or more techniques of this disclosure. Figure 2 is a conceptual diagram illustrating encoded video data and corresponding data structures in accordance with one or more techniques of this disclosure. Figure 3 is a conceptual diagram illustrating a data structure that encapsulates encoded video data and corresponding metadata in accordance with one or more techniques of this disclosure. Figure 4 is a conceptual diagram illustrating an example of components that may be included in an implementation of a system that may be configured to encode and decode video data in accordance with one or more techniques of this disclosure. Figure 5 is a block diagram illustrating an example of a video encoder that may be configured to encode video data in accordance with one or more techniques of this disclosure. Figure 6 is a block diagram illustrating an example of a video decoder that may be configured to decode video data in accordance with one or more techniques of this disclosure. DETAILED DESCRIPTION

[0008] In general, the present disclosure describes various techniques for encoding video data. Specifically, the present disclosure describes techniques for signaling neural network post-filter parameter information for encoding video data. It should be noted that although the techniques of the present disclosure are described with respect to ITU-TH.264, ITU-TH.265, JEM, and JVET-T2001, the techniques of the present disclosure can be generally applied to video encoding. For example, in addition to those techniques included in ITU-TH.265, JEM, and JVET-T2001, the encoding techniques described herein can be incorporated into video encoding systems (including video encoding systems based on future video encoding standards), including video block structures, intra-frame prediction techniques, inter-frame prediction techniques, transform techniques, filtering techniques, and / or other entropy coding techniques. Therefore, references to ITU-TH.264, ITU-TH.265, JEM, and / or JVET-T2001 are for descriptive purposes and should not be interpreted as limiting the scope of the techniques described herein. In addition, it should be noted that the incorporation of references herein is for descriptive purposes and should not be construed as limiting or creating ambiguity regarding the terms used herein. For example, where a definition of a term provided in an incorporated reference differs from that provided in another incorporated reference and / or as used herein, the term should be interpreted in a manner that broadly includes each corresponding definition and / or in a manner that includes each specific definition in alternatives.

[0009] In one example, a method of encoding video data includes signaling a neural network post-filter characteristics message, and signaling a syntax element identifying a padding type corresponding to the neural network post-filter characteristics message, wherein the padding type includes cyclic padding.

[0010] In one example, a device includes one or more processors configured to perform the following operations: sending a signal to notify a neural network post-filter characteristics message, and sending a signal to notify a syntax element identifying a padding type corresponding to the neural network post-filter characteristics message, wherein the padding type includes loop padding.

[0011] In one example, a non-transitory computer-readable storage medium includes instructions stored thereon that, when executed, cause one or more processors of a device to signal a neural network post-filter characteristics message and to signal a syntax element identifying a padding type corresponding to the neural network post-filter characteristics message, wherein the padding type includes loop padding.

[0012] In one example, an apparatus includes: means for signaling a neural network post-filter characteristics message; and means for signaling a syntax element identifying a padding type corresponding to the neural network post-filter characteristics message, wherein the padding type includes cyclic padding.

[0013] In one example, a method of decoding video data includes receiving a neural network post-filter characteristics message, and parsing a syntax element identifying a padding type corresponding to the neural network post-filter characteristics message, wherein the padding type includes loop padding.

[0014] In one example, a device includes one or more processors configured to receive a neural network post-filter characteristics message, and parse a syntax element identifying a padding type corresponding to the neural network post-filter characteristics message, wherein the padding type includes loop padding.

[0015] In one example, a non-transitory computer-readable storage medium includes instructions stored thereon that, when executed, cause one or more processors of a device to receive a neural network post-filter characteristics message and parse a syntax element identifying a padding type corresponding to the neural network post-filter characteristics message, wherein the padding type includes loop padding.

[0016] In one example, an apparatus includes: means for receiving a neural network post-filter characteristics message; and means for parsing a syntax element identifying a padding type corresponding to the neural network post-filter characteristics message, wherein the padding type includes loop padding.

[0017] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description and drawings, and from the claims.

[0018] The video content includes a video sequence consisting of a series of frames (or pictures). A series of frames may also be referred to as a group of pictures (GOP). Each video frame or picture may be divided into one or more regions. The region may be defined according to a set of rules for defining a basic unit (e.g., a video block) and a defined region. For example, the rule defining the region may be that the region must be an integer number of video blocks arranged in a rectangle. In addition, the video blocks in the region may be sorted according to a scanning mode (e.g., a raster scan). As used herein, the term "video block" may generally refer to a region of a picture, or may more specifically refer to a maximum array of sample values ​​that may be predictively encoded, its sub-partitions, and / or corresponding structures. In addition, the term "current video block" may refer to a region of a picture that is being encoded or decoded. A video block may be defined as an array of sample values. It should be noted that in some cases, a pixel value may be described as a sample value comprising a corresponding component of video data, which may also be referred to as a color component (e.g., brightness (Y) and chrominance (Cb and Cr) components or red, green, and blue components). It should be noted that in some cases, the terms "pixel value" and "sample value" may be used interchangeably. Additionally, in some cases, a pixel or sample may be referred to as a pel. A video sampling format (which may also be referred to as a chroma format) may define the number of chroma samples included in a video block relative to the number of luma samples included in the video block. For example, for a 4:2:0 sampling format, the sampling rate of the luma component is twice the sampling rate of the chroma components in both the horizontal and vertical directions.

[0019] The video encoder may perform predictive coding on a video block and its sub-partitions. A video block and its sub-partitions may be referred to as a node. ITU-T H.264 specifies a macroblock including 16×16 luminance samples. That is, in ITU-T H.264, a picture is segmented into macroblocks. ITU-T H.265 specifies a similar coding tree unit (CTU) structure (which may be referred to as a maximum coding unit (LCU)). In ITU-TH.265, a picture is segmented into CTUs. In ITU-T H.265, for a picture, the CTU size may be set to include 16×16, 32×32, or 64×64 luminance samples. In ITU-T H.265, a CTU is composed of a corresponding coding tree block (CTB) for each component of the video data (e.g., luminance (Y) and chrominance (Cb and Cr)). It should be noted that a video having a luminance component and two corresponding chrominance components may be described as having two channels, i.e., a luminance channel and a chrominance channel. In addition, in ITU-T H.265, CTUs can be divided according to a quadtree (QT) partitioning structure, which allows the CTBs of a CTU to be divided into coding blocks (CBs). That is, in ITU-T H.265, CTUs can be divided into quadtree leaf nodes. According to ITU-T H.265, a luma CB together with two corresponding chroma CBs and associated syntax elements is referred to as a coding unit (CU). In ITU-TH.265, a signal can be sent to notify the minimum allowed size of a CB. In ITU-T H.265, the minimum allowed minimum size of a luma CB is 8×8 luma samples. In ITU-T H.265, the decision to encode a picture area using intra prediction or inter prediction is made at the CU level.

[0020] In ITU-T H.265, a CU is associated with a prediction unit structure having its root at the CU. In ITU-T H.265, the prediction unit structure allows the partitioning of luma CB and chroma CB to generate corresponding reference samples. That is, in ITU-T H.265, the luma CB and chroma CB can be partitioned into corresponding luma prediction blocks and chroma prediction blocks (PBs), wherein the PBs include blocks of sample values ​​to which the same prediction is applied. In ITU-T H.265, the CB can be divided into 1, 2, or 4 PBs. ITU-T H.265 supports PB sizes from 64×64 samples down to 4×4 samples. In ITU-TH.265, square PBs are supported for intra prediction, where the CB can form a PB or the CB can be partitioned into four square PBs. In ITU-TH.265, in addition to square PBs, rectangular PBs are also supported for inter prediction, where the CB can be halved vertically or horizontally to form a PB. In addition, it should be noted that in ITU-T H.265, for inter prediction, four asymmetric PB partitions are supported, where a CB is divided into two PBs at one quarter of the height (top or bottom) or width (left or right) of the CB. Intra prediction data (e.g., intra prediction mode syntax elements) or inter prediction data (e.g., motion data syntax elements) corresponding to the PB are used to generate reference and / or prediction sample values ​​for the PB.

[0021] JEM specifies a CTU with a maximum size of 256×256 luma samples. JEM specifies a quadtree plus binary tree (QTBT) block structure. In JEM, the QTBT structure allows the quadtree leaf nodes to be further divided by the binary tree (BT) structure. That is, in JEM, the binary tree structure allows the quadtree leaf nodes to be recursively divided vertically or horizontally. In JVET-T2001, CTUs are divided according to a quadtree plus multi-type tree (QTMT or QT+MTT) structure. The QTMT in JVET-T2001 is similar to the QTBT in JEM. However, in JVET-T2001, in addition to indicating binary partitioning, the multi-type tree can also indicate a so-called ternary (or ternary tree (TT)) partitioning. Ternary partitioning divides a block vertically or horizontally into three blocks. In case of vertical TT splitting, the block is split at one quarter of its width from the left edge and at one quarter of its width from the right edge, and in case of horizontal TT splitting, the block is split at one quarter of its height from the top edge and at one quarter of its height from the bottom edge.

[0022] As described above, each video frame or picture may be divided into one or more regions. For example, according to ITU-T H.265, each video frame or picture may be divided into one or more slices, and further divided into one or more tiles, wherein each slice includes a CTU sequence (e.g., arranged in raster scan order), and wherein the tile is a CTU sequence corresponding to a rectangular area of ​​the picture. It should be noted that in ITU-T H.265, a slice is a sequence of one or more slice segments starting from an independent slice segment and containing all subsequent subordinate slice segments (if any) before the next independent slice segment (if any). Slice segments (such as slices) are CTU sequences. Therefore, in some cases, the terms "slice" and "slice segment" may be used interchangeably to indicate a sequence of CTUs arranged in a raster scan order. In addition, it should be noted that in ITU-T H.265, a tile may be composed of CTUs contained in more than one slice, and a slice may be composed of CTUs contained in more than one tile. However, ITU-T H.265 specifies that one or both of the following conditions should be met: (1) all CTUs in a slice belong to the same tile; and (2) all CTUs in a tile belong to the same slice.

[0023] Regarding JVET-T2001, a slice needs to be composed of an integer number of complete tiles or an integer number of continuous complete CTU rows within a tile, rather than only an integer number of CTUs. It should be noted that in JVET-T2001, the slice design does not include a slice segment (ie, there is no independent / subordinate slice segment). Therefore, in JVET-T2001, a picture may include a single tile, wherein the single tile is contained in a single slice, or a picture may include multiple tiles, wherein the multiple tiles (or their CTU rows) may be contained in one or more slices. In JVET-T2001, the picture is divided into tiles by specifying the corresponding height of the tile row and the corresponding width of the tile column. Therefore, in JVET-T2001, a tile is a rectangular CTU area within a specific tile row and a specific tile column position. In addition, it should be noted that JVET-T2001 stipulates that a picture can be divided into sub-pictures, wherein the sub-picture is a rectangular CTU area within the picture. The upper left CTU of a sub-picture may be located at any CTU position within a picture, where the sub-picture is constrained to include one or more slices. Therefore, unlike tiles, sub-pictures do not have to be restricted to specific row and column positions. It should be noted that sub-pictures can be used to encapsulate regions of interest within a picture, and the sub-bitstream extraction process can be used to decode and display only specific regions of interest. That is, as described in further detail below, the bitstream of encoded video data includes a sequence of network abstraction layer (NAL) units, wherein the NAL unit encapsulates encoded video data (i.e., video data corresponding to a picture slice), or the NAL unit encapsulates metadata (e.g., parameter sets) for decoding the video data, and the sub-bitstream extraction process forms a new bitstream by removing one or more NAL units from the bitstream.

[0024] Figure 2 is a conceptual diagram showing examples of pictures within picture groups partitioned according to tiles, slices, and sub-pictures. It should be noted that the techniques described herein can be applicable to tiles, slices, sub-pictures, their sub-partitions, and / or their equivalent structures. That is, regardless of how the pictures are divided into regions, the techniques described herein are generally applicable. For example, in some cases, the techniques described herein can be applicable to cases where tiles can be partitioned into so-called bricks, where a brick is a rectangular CTU row region within a particular tile. In addition, for example, in some cases, the techniques described herein can be applicable to cases where one or more tiles can be included in a so-called tile group, where a tile group includes an integer number of adjacent tiles. Figure 2 In the example shown, Pic3 is shown to include 16 tiles (ie, tiles 0 to 15 ) and three slices (i.e., slice 0 to slice 2). Figure 2In the example shown, slice 0 includes four tiles (ie, tiles 0 to 3), and slice 1 includes eight tiles (ie, tiles 4 to 5). 11 ), and slice 2 includes four tiles (i.e., tiles 12 To Tile 15 ). In addition, Figure 2 As shown in the example of , Pic3 is shown to include two sub-pictures (i.e., sub-picture 0 and sub-picture 1), where sub-picture 0 includes slices 0 and slice 1 and where sub-picture 1 includes slice 2. As described above, sub-pictures can be used to encapsulate regions of interest within a picture, and a sub-bitstream extraction process can be used to selectively decode (and display) the regions of interest. For example, referring to Figure 2 , sub-picture 0 may correspond to the action portion of a sporting event presentation (e.g., a view of a field), and sub-picture 1 may correspond to a scrolling banner displayed during the sporting event presentation. By organizing pictures into sub-pictures in this manner, a viewer may be able to disable the display of a scrolling banner. That is, through the sub-bitstream extraction process, slice 2 NAL units may be removed from the bitstream (and therefore not decoded and / or displayed), and slice 0 NAL units and slice 1 NAL units may be decoded and displayed. How slices of a picture are encapsulated into corresponding NAL unit data structures and sub-bitstream extraction is described in further detail below.

[0025] For intra-frame prediction coding, the intra-frame prediction mode may specify the location of the reference sample within the picture. In ITU-TH.265, the possible intra-frame prediction modes defined include a plane (ie, surface fitting) prediction mode, a DC (ie, flat overall average) prediction mode, and 33 angular prediction modes (predMode:2-34). In JEM, the possible intra-frame prediction modes defined include a plane prediction mode, a DC prediction mode, and 65 angular prediction modes. It should be noted that the plane prediction mode and the DC prediction mode may be referred to as a non-directional prediction mode, and the angular prediction mode may be referred to as a directional prediction mode. It should be noted that regardless of the number of possible prediction modes defined, the techniques described herein may be generally applicable.

[0026] For inter-frame prediction coding, a reference picture is determined, and a motion vector (MV) identifies samples in the reference picture that are used to generate a prediction for the current video block. For example, the current video block may be predicted using reference sample values ​​located in one or more previously coded pictures, and a motion vector is used to indicate the position of the reference block relative to the current video block. The motion vector may describe, for example, the horizontal displacement component of the motion vector (i.e., the MV x ), the vertical displacement component of the motion vector (i.e., MV y) and the resolution of the motion vector (e.g., quarter pixel accuracy, half pixel accuracy, one pixel accuracy, two pixel accuracy, four pixel accuracy). Previously decoded pictures (which may include pictures output before or after the current picture) may be organized into one or more reference picture lists and identified using reference picture index values. In addition, in inter-frame prediction coding, uni-prediction refers to generating predictions using sample values ​​from a single reference picture, and bi-prediction refers to generating predictions using corresponding sample values ​​from two reference pictures. That is, in uni-prediction, a single reference picture and a corresponding motion vector are used to generate a prediction for the current video block, while in bi-prediction, a first reference picture and a corresponding first motion vector and a second reference picture and a corresponding second motion vector are used to generate a prediction for the current video block. In bi-prediction, corresponding sample values ​​are combined (e.g., added, rounded and cropped, or averaged according to weights) to generate a prediction. Pictures and their regions may be classified based on what types of prediction modes may be used to encode their video blocks. That is, for a region with a B type (e.g., a B slice), bi-prediction, uni-prediction, and intra-prediction modes can be utilized, for a region with a P type (e.g., a P slice), uni-prediction and intra-prediction modes can be utilized, and for a region with an I type (e.g., an I slice), only an intra-prediction mode can be utilized. As described above, a reference picture is identified by a reference index. For example, for a P slice, there may be a single reference picture list RefPicList0, and for a B slice, there may be a second independent reference picture list RefPicList1 in addition to RefPicList0. It should be noted that for a uni-prediction in a B slice, one of RefPicList0 or RefPicList1 may be used to generate a prediction. Furthermore, it should be noted that during the decoding process, when decoding of a picture begins, a reference picture list is generated from a previously decoded picture stored in a decoded picture buffer (DPB).

[0027] In addition, the coding standard may support various motion vector prediction modes. Motion vector prediction enables the value of the motion vector for the current video block to be derived based on another motion vector. For example, a set of candidate blocks with associated motion information can be derived from the spatial neighboring blocks and the temporal neighboring blocks of the current video block. In addition, the generated (or default) motion information can be used for motion vector prediction. Examples of motion vector prediction include advanced motion vector prediction (AMVP), temporal motion vector prediction (TMVP), so-called "merge" mode, and "skip" and "direct" motion reasoning. In addition, other examples of motion vector prediction include advanced temporal motion vector prediction (ATMVP) and space-time motion vector prediction (STMVP). For motion vector prediction, both the video encoder and the video decoder perform the same process to derive a set of candidates. Therefore, for the current video block, the same set of candidates is generated during encoding and decoding.

[0028] As described above, for inter-frame prediction coding, reference samples in previously encoded pictures are used to encode video blocks in the current picture. Previously encoded pictures that can be used as references when encoding the current picture are called reference pictures. It should be noted that the decoding order does not necessarily correspond to the picture output order, that is, the temporal order of pictures in a video sequence. In ITU-TH.265, when a picture is decoded, it is stored in a decoded picture buffer (DPB) (which may be called a frame buffer, a reference buffer, a reference picture buffer, etc.). In ITU-TH.265, pictures stored in the DPB are removed from the DPB when they are output and are no longer needed for encoding subsequent pictures. In ITU-TH.265, the determination of whether the picture should be removed from the DPB is called once for each picture after decoding the slice header, that is, at the beginning of decoding the picture. For example, the reference Figure 2 , Pic2 is shown as reference to Pic1. Similarly, Pic3 is shown as reference to Pic0. Figure 2, assuming that the number of pictures corresponds to the decoding order, the DPB will be filled as follows: after decoding Pic0, the DPB will include {Pic0}; at the beginning of decoding Pic1, the DPB will include {Pic0}; after decoding Pic1, the DPB will include {Pic0, Pic1}; at the beginning of decoding Pic2, the DPB will include {Pic0, Pic1}. Pic2 will then be decoded with reference to Pic1, and after decoding Pic2, the DPB will include {Pic0, Pic1, Pic2}. At the beginning of decoding Pic3, pictures Pic0 and Pic1 will be marked for removal from the DPB because they are not required for decoding Pic3 (or any subsequent pictures, not shown), and assuming that Pic1 and Pic2 have been output, the DPB will be updated to include {Pic0}. Pic3 will then be decoded with reference to Pic0. The process of marking pictures to remove them from the DPB can be called reference picture set (RPS) management.

[0029] As described above, intra prediction data or inter prediction data is used to generate reference sample values ​​for a block of sample values. The difference between the sample values ​​included in the current PB or another type of picture region structure and the associated reference samples (e.g., those generated using prediction) may be referred to as residual data. The residual data may include a corresponding difference array corresponding to each component of the video data. The residual data may be in the pixel domain. A transform such as a discrete cosine transform (DCT), a discrete sine transform (DST), an integer transform, a wavelet transform, or a conceptually similar transform may be applied to the difference array to generate transform coefficients. It should be noted that in ITU-TH.265 and JVET-T2001, a CU is associated with a transform tree structure having its root at the CU level. The transform tree is divided into one or more transform units (TUs). That is, in order to generate transform coefficients, the array of difference values ​​may be divided (e.g., four 8×8 transforms may be applied to a 16×16 residual value array). For each component of the video data, such a subdivision of the difference value may be referred to as a transform block (TB). It should be noted that in some cases a core transform and a subsequent secondary transform may be applied (in a video encoder) to generate transform coefficients. For a video decoder, the order of transforms is reversed.

[0030] The quantization process may be performed directly on the transform coefficients or residual sample values ​​(e.g., in the case of palette coded quantization). Quantization approximates the transform coefficients by limiting the amplitude to a set of specified values. Quantization essentially scales the transform coefficients in order to change the amount of data required to represent a set of transform coefficients. Quantization may include dividing the transform coefficients (or the values ​​obtained by adding the offset values ​​to the transform coefficients) by a quantization scaling factor and any associated rounding function (e.g., rounding to the nearest integer). The quantized transform coefficients may be referred to as coefficient level values. Inverse quantization (or "dequantization") may include multiplying the coefficient level values ​​by the quantization scaling factor, as well as any reciprocal rounding or offset addition operations. It should be noted that, as used herein, the term quantization process may refer to dividing by a scaling factor in some cases to generate a level value, and may refer to multiplying by a scaling factor in some cases to recover the transform coefficients. That is, the quantization process may refer to quantization in some cases, and to inverse quantization in some cases. In addition, it should be noted that although the quantization process is described in some of the examples below with respect to arithmetic operations associated with decimal notation, such description is for illustrative purposes and should not be construed as limiting. For example, the techniques described herein can be implemented in devices that use binary operations, etc. For example, the multiplication and division operations described herein can be implemented using bit shift operations, etc.

[0031] The quantized transform coefficients and syntax elements (e.g., syntax elements indicating the coding structure of the video block) may be entropy encoded according to an entropy coding technique. The entropy coding process includes encoding the syntax element values ​​using a lossless data compression algorithm. Examples of entropy coding techniques include content adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), probability interval partitioning entropy coding (PIPE), etc. The entropy-coded quantized transform coefficients and the corresponding entropy-coded syntax elements may form a compatible bitstream that can be used to reproduce the video data at the video decoder. The entropy coding process, such as CABAC, may include binarizing the syntax elements. Binarization refers to the process of converting the value of the syntax element into a sequence of one or more bits. These bits may be referred to as "bins". Binarization may include one or a combination of the following coding techniques: fixed length coding, unary coding, truncated unary coding, truncated Rice coding, Golomb coding, k-order exponential Golomb coding, and Golomb-Rice coding. For example, binarization may include representing the integer value 5 of the syntax element as 00000101 using an 8-bit fixed-length binarization technique, or representing the integer value 5 as 11110 using a unary coding binarization technique. As used herein, each of the terms fixed-length coding, unary coding, truncated unary coding, truncated Rice coding, Golomb coding, k-order exponential Golomb coding, and Golomb-Rice coding may refer to general implementations of these techniques and / or more specific implementations of these coding techniques. For example, a Golomb-Rice coding implementation may be specifically defined according to a video coding standard. In the example of CABAC, for a particular bin, the context provides a maximum probability state (MPS) value of the bin (i.e., the MPS of the bin is one of 0 or 1), and a probability value that the bin is an MPS or a minimum probability state (LPS). For example, the context may indicate that the MPS of the bin is 0, and the probability that the bin is 1 is 0.3. It should be noted that the context may be determined based on the value of the previously encoded bin including the bin in the current syntax element and the previously encoded syntax element. For example, the values ​​of syntax elements associated with neighboring video blocks may be used to determine the context of the current bin.

[0032] As described above, the sample values ​​of the reconstructed block may be different from the sample values ​​of the current video block being encoded. In addition, it should be noted that in some cases, encoding video data block by block may result in artifacts (e.g., so-called block artifacts, band artifacts, etc.). For example, block artifacts may cause the encoding block boundaries of the reconstructed video data to be visually perceived by the user. In this way, the reconstructed sample values ​​may be modified to minimize the difference between the sample values ​​of the encoded current video block and the reconstructed block and / or minimize artifacts introduced by the video encoding process. Such modifications may generally be referred to as filtering. It should be noted that filtering may occur as part of an in-loop filtering process or a post-loop (or post-filtering) filtering process. For an in-loop filtering process, the resulting sample values ​​of the filtering process may be used to predict the video block (e.g., stored in a reference frame buffer for subsequent encoding at a video encoder and subsequent decoding at a video decoder). For a post-loop filtering process, the resulting sample values ​​of the filtering process are output only as part of the decoding process (e.g., not used for subsequent encoding). For example, in the case of a video decoder, for an in-loop filtering process, the sample values ​​generated by the filtered reconstruction block will be used for subsequent decoding (e.g., stored in a reference buffer) and will be output (e.g., output to a display). For a post-loop filtering process, the reconstruction block will be used for subsequent decoding, and the sample values ​​generated by the filtered reconstruction block will be output and will be used for subsequent decoding.

[0033] Deblocking (or deblocking), deblocking filtering or applying a deblocking filter refers to the process of smoothing the boundaries of adjacent reconstructed video blocks (i.e., making the boundaries less noticeable to the observer). Smoothing the boundaries of adjacent reconstructed video blocks may include modifying the sample values ​​included in the rows or columns adjacent to the boundaries. JVET-T2001 provides a scenario in which a deblocking filter is applied to reconstructed sample values ​​as part of an in-loop filtering process. In addition to applying a deblocking filter as part of an in-loop filtering process, JVET-T2001 also provides a scenario in which a sample adaptive offset (SAO) filter may be applied in the in-loop filtering process. Generally speaking, SAO is a process in which deblocked sample values ​​in a region are modified by conditionally adding an offset value. Another type of filtering process includes a so-called adaptive loop filter (ALF). ALF using block-based adaptation is specified in JEM. In JEM, ALF is applied after the SAO filter. It should be noted that ALF can be applied to reconstructed samples independently of other filtering techniques. The process of applying the ALF specified in JEM at a video encoder can be summarized as follows: (1) each 2×2 block of the luminance component used to reconstruct the image is classified according to a classification index; (2) a filter coefficient set for each classification index is derived; (3) a filtering decision is determined for the luminance component; (4) a filtering decision is determined for the chrominance components; and (5) filter parameters (e.g., coefficients and decisions) are signaled. JVET-T2001 specifies a deblocking filter, SAO filter, and ALF filter, which can be described as being generally based on the deblocking filter, SAO filter, and ALF filter provided in ITU-T H.265 and JEM.

[0034] It should be noted that JVET-T2001 is referred to as a pre-release version of ITU-TH.266, and therefore, it is an almost finalized draft of the video coding standard produced by the VVC project, and therefore, can be referred to as the first version of the VVC standard (or VVC or VVC version 1 or ITU-H.266). It should be noted that during the VVC project, convolutional neural network (CNN)-based techniques were studied, which showed the potential for removing artifacts and improving objective quality, but it was decided not to include such techniques in the VVC standard. However, CNN-based techniques are currently being considered for extensions and / or improvements to VVC. Some CNN-based techniques involve post-filtering. For example, "AHG11: Content-adaptive neural network post-filter" (teleconference, document JVET-Z0082-v2, referred to as JVET-Z0082 in this article) at the 26th meeting of ISO / IEC JTC1 / SC29 / WG11 held from April 20 to 29, 2022 describes a post-filter based on a content adaptive neural network. It should be noted that in JVET-Z0082, content adaptation is achieved by overfitting the NN postfilter to the test video. Furthermore, it should be noted that the result of the overfitting process in JVET-Z0082 is a weight update. JVET-Z0082 describes where the weight updates are encoded using ISO / IEC FDIS15938-17. Information Technology - Multimedia Content Description Interface - Part 17: Compression of Neural Networks for Multimedia Content Description and Analysis and Test Model for Incremental Compression of Neural Networks for Multimedia Content Description and Analysis (INCTM), N0179, February 2022, which may be collectively referred to as the MPEG NNR (Neural Network Representation) or Neural Network Coding (NNC) standard. JVET-Z0082 also describes where the coding weight updates are signaled within the video bitstream as NNR postfilter SEI messages. The 26th ISO / IEC JTC1 / SC29 / WG11 meeting “AHG9: NNR post-filter SEI message” (conference call, document JVET-Z0052-v1, referred to herein as JVET-Z0052), held from April 20 to 29, 2022, described the NNR post-filter SEI message utilized by JVET-Z0082.The elements of the NN postfilter described in JVET-Z0082 and the NNR postfilter SEI messages described in JVET-Z0052 are adopted in "Additional SEI messages for VSEI (Draft 1)" of the 26th meeting of ISO / IEC JTC1 / SC29 / WG11, held from April 20 to 29, 2022 (conference call, document JVET-Z2006-v1, referred to herein as JVET-Z2006). JVET-Z2006 provides a generic supplemental enhancement information message for a coded video bitstream (VSEI). JVET-Z2006 also adopts elements of the NN post-filter supplemental enhancement information message described in "AHG9: NNR post-filter SEI" of the 26th ISO / IEC JTC1 / SC29 / WG11 meeting held from April 20 to 29, 2022 (conference call, document JVET-Z0244-v1, referred to herein as JVET-Z0244). JVET-Z2006 is described in more detail below. The techniques described herein provide techniques for signaling neural network parameters. For the formulas used in this article, the following arithmetic operators can be used: + Addition - Subtraction * Multiplication, including matrix multiplication x y Exponentiation. Raises x to the power of y. In other contexts, such symbols are used for superscripts and are not intended to be interpreted as exponentiation. / Integer division truncates the result towards zero. For example, 7 / 4 and -7 / -4 are truncated to 1, and -7 / 4 and 7 / -4 are truncated to -1. ÷ is used to represent division in mathematical formulas when truncation or rounding is not intended. Used to represent division in mathematical formulas without the intent to truncate or round. Additionally, the following math functions are available: Log2(x) is the logarithm of x with base 2; Ceil(x) is the smallest integer greater than or equal to x. With respect to the exemplary syntax used herein, the following definitions of logical operators may be applied: x&&y Boolean logical "AND" of x and y x∥y Boolean logical "or" of x and y ! Boolean logic "NO" x? y:z If x is TRUE or not equal to 0, it evaluates to y; otherwise, it evaluates to z. Additionally, the following relational operators can be applied: > Greater than >= greater than or equal to < less than <= less than or equal to == is equal to ! = Not equal to Furthermore, it should be noted that among the syntax descriptors used in this document, the following descriptors may be applied: -b(8): Byte (8 bits) with any bit string pattern. The parsing of this descriptor is specified by the return value of the function read_bit(8). -f(n): A fixed-mode bit string written using n bits (from left to right) starting from the leftmost bit. The parsing process of this descriptor is specified by the return value of the function read_bit(n). -se(v): signed integer order 0 Exp-Golomb coded syntax element, starting from the leftmost bit. -tb(v): truncated binary code using at most maxVal bits, where maxVal is defined in the semantics of the syntax element. -tu(v): truncated unary code using at most maxVal bits, where maxVal is defined in the semantics of the syntax element. -u(n): Use an unsigned integer of n bits. When n is "v" in the syntax table, the number of bits varies in a way that depends on the values ​​of other syntax elements. The parsing process of this descriptor is specified by the return value of the function read_bits(n), which is interpreted as the binary representation of the unsigned integer with the most significant bit written first. -ue(v): unsigned integer 0th order Exp-Golomb coded syntax element, starting from the leftmost bit.

[0035] As mentioned above, the video content includes a video sequence composed of a series of pictures, and each picture can be divided into one or more regions. In JVET-T2001, the coded representation of a picture includes a VCL NAL unit of a specific layer within an AU and contains all CTUs of the picture. For example, referring again to Figure 2, the coded representation of Pic3 is encapsulated in three coded slice NAL units (i.e., slice 0 NAL unit, slice 1 NAL unit, and slice 2 NAL unit). It should be noted that the term video coding layer (VCL) NAL unit is used as a general term for coded slice NAL units, that is, VCL NAL is a general term for slice NAL units of all types. As described above, and described in further detail below, NAL units can encapsulate metadata for decoding video data. NAL units that encapsulate metadata for decoding video sequences are generally referred to as non-VCL NAL units. Therefore, in JVET-T2001, NAL units can be VCL NAL units or non-VCL NAL units. It should be noted that the VCL NAL unit includes slice header data, which provides information for decoding a specific slice. Therefore, in JVET-T2001, information for decoding video data (which may be referred to as metadata in some cases) is not limited to being included in non-VCL NAL units. JVET-T2001 specifies that a picture unit (PU) is a group of NAL units that are associated with each other according to a specified classification rule, are consecutive in decoding order, and contain exactly one coded picture, and an access unit (AU) is a group of PUs that belong to different layers and contain coded pictures associated with the same time output from the DPB. JVET-T2001 further specifies that a layer is a group of VCL NAL units and their associated non-VCL NAL units that all have a specific value of the layer identifier. In addition, in JVET-T2001, a PU consists of zero or one PH NAL unit, one coded picture (which consists of one or more VCL NAL units), and zero or more other non-VCL NAL units. Furthermore, in JVET-T2001, a coded video sequence (CVS) is an AU sequence consisting of a CVSS AU and zero or more subsequent AUs that are non-CVSS AUs arranged in decoding order (including all subsequent AUs up to (excluding) any subsequent AU that is a CVSS AU), wherein a coded video sequence start (CVSS) AU is an AU in which a PU exists for each layer in the CVS, and a coded picture in each existing picture unit is a coded layer video sequence start (CLVSS) picture. In JVET-T2001, a coded layer video sequence (CLVS) is a PU sequence within the same layer, consisting of a CLVSS PU and zero or more subsequent PUs that are non-CLVSS PUs arranged in decoding order (including all subsequent PUs up to (excluding) any subsequent PU that is a CLVSS PU). That is, in JVET-T2001, a bitstream may be described as including a sequence of AUs forming one or more CVSs.

[0036] Multi-layer video coding enables video presentation to be decoded / displayed as a presentation corresponding to a base layer of video data, and decoded / displayed as one or more additional presentations corresponding to an enhancement layer of video data. For example, a base layer can enable presentation of a video presentation with a basic quality level (e.g., high-definition presentation and / or a 30Hz frame rate), and an enhancement layer can enable presentation of a video presentation with an enhanced quality level (e.g., ultra-high-definition rendering and / or a 60Hz frame rate). The enhancement layer can be encoded by reference to the base layer. That is, for example, the picture in the enhancement layer can be encoded (e.g., using inter-layer prediction techniques) by reference to one or more pictures in the base layer (including a scaled version thereof). It should be noted that the layers can also be encoded independently of each other. In this case, there may be no inter-layer prediction between the two layers. Each NAL unit may include an identifier indicating the video data layer to which the NAL unit is associated. As described above, a sub-bitstream extraction process can be used to decode and display only a specific region of interest of a picture. In addition, a sub-bitstream extraction process can be used to decode and display only a specific video layer. Sub-bitstream extraction may refer to a process by which a device receiving a compatible or compliant bitstream forms a new compatible or compliant bitstream by discarding and / or modifying data in the received bitstream. For example, sub-bitstream extraction may be used to form a new compatible or compliant bitstream corresponding to a particular video representation (e.g., a high-quality representation).

[0037] In JVET-T2001, each of a video sequence, GOP, picture, slice, and CTU can be associated with metadata describing video coding attributes, and some types of metadata are encapsulated in non-VCL NAL units. JVET-T2001 defines parameter sets that can be used to describe video data and / or video coding attributes. Specifically, JVET-T2001 includes the following four parameter sets: video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), and adaptive parameter set (APS), wherein SPS is applied to zero or more integer CVSs, PPS is applicable to zero or more integer coded pictures, APS is applicable to zero or more slices, and VPS can be optionally referenced by SPS. PPS is applied to a single coded picture that references it. In JVET-T2001, parameter sets can be encapsulated as non-VCL NAL units, and / or can be sent as message signals. JVET-T2001 also includes a picture header (PH), which is encapsulated as a non-VCL NAL unit. In JVET-T2001, the picture header is applied to all slices of the coded picture. JVET-T2001 further enables the signaling of decoding capability information (DCI) and supplementary enhancement information (SEI) messages. In JVET-T2001, DCI and SEI messages assist processes related to decoding, display or other purposes, however, DCI and SEI messages may not be required to construct luminance or chrominance samples according to the decoding process. In JVET-T2001, non-VCL NAL units can be used to signal DCI and SEI messages in the bitstream. In addition, DCI and SEI messages can be transmitted by some mechanism rather than by being present in the bitstream (i.e., signaling out of band).

[0038] Figure 3 An example of a bitstream including a plurality of CVSs is shown, wherein the CVSs include AUs, and the AUs include picture units. Figure 3 The example shown in corresponds to encapsulating Figure 2 An example of a slice NAL unit is shown in the example of . Figure 3 In the example shown, the corresponding picture unit of Pic3 includes three VCL NAL coded slice NAL units, namely slice 0 NAL unit, slice 1 NAL unit and slice 2 NAL unit, and two non-VCL NAL units, namely PPS NAL unit and PH NAL unit. Figure 3 In , the header is the NAL unit header (i.e., not to be confused with the slice header). In addition, it should be noted that in Figure 3In the example, other non-VCL NAL units not shown may be included in the CVS, such as SPS NAL units, VPS NAL units, SEI message NAL units, etc. In addition, it should be noted that in other examples, the PPS NAL unit for decoding Pic3 may be included elsewhere in the bitstream, for example, in the picture unit corresponding to Pic0, or may be provided by an external agency. As described in further detail below, in JVET-T2001, the PH syntax structure may be present in the slice header of the VCL NAL unit or in the PH NAL unit of the current PU. JVET-T2001 defines the NAL unit header semantics, which specifies the type of raw byte sequence payload (RBSP) data structure included in the NAL unit. Table 1 shows the syntax of the NAL unit header provided in JVET-T2001. Table 1 JVET-T2001 provides the following definitions for the corresponding syntax elements shown in Table 1. forbidden_zero_bit should be equal to 0. nuh_reserved_zero_bit shall be equal to 0. A value of 1 for the nuh_rescrved_zero bit may be specified by ITU-T|ISO / IEC in the future. Although in this version of this specification, the value of nuh_reserved_zero_bit is required to be equal to 0, decoders conforming to this version of this specification shall allow a value of nuh_reserved_zero_bit equal to 1 to appear in the syntax and shall ignore (i.e., remove and discard from the bitstream) NAL units with nuh_reserved_zero_bit equal to 1. nuh_layer_id specifies the identifier of the layer to which the VCL NAL unit belongs or the identifier of the layer to which the non-VCL NAL unit applies. The value of nuh_layer_id shall be in the range of 0 to 55, inclusive. Other values ​​of nuh_layer_id are reserved for future use by ITU-T | ISO / IEC. Although in this version of this specification, the value of nuh_layer_id is required to be in the range of 0 to 55, inclusive, a decoder conforming to this version of this specification shall allow values ​​of nuh_layer_id greater than 55 to appear in the syntax and shall ignore (i.e., remove and discard from the bitstream) NAL units with nuh_layer_id greater than 55. The value of nuh_layer_id of all VCL NAL units of a coded picture shall be the same. The value of nuh_layer_id of a coded picture or PU is the value of nuh_layer_id of the VCL NAL unit of the coded picture or PU. When nal_unit_type is equal to PH_NUT or FD_NUT, nuh_layer_id shall be equal to the nuh_layer_id of the associated VCL NAL unit. When nal_unit_type is equal to EOS_NUT, nuh_layer_id shall be equal to one of the nuh_layer_id values ​​of the layer present in the CVS. NOTE—The value of nuh_layer_id for DCI, OPI, VPS, AUD, and EOB NAL units is not constrained. nuh_temporal_id_plus1 minus 1 specifies the temporal identifier of the NAL unit. The value of nuh_temporal_id_plus1 shall not be equal to 0. The variable TemporalId is derived as follows: TemporalId=nuh_temporal_id_plus1-1 When nal_unit_type is in the range of IDR_W_RADL to RSV_IRAP_11 (inclusive), TemporalId shall be equal to 0. When nal_unit_type is equal to STSA_NUT and When vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]] is equal to 1, TemporalId should be greater than 0. The value of TemporalId shall be the same for all VCL NAL units of an AU. The value of TemporalId of a coded picture, PU, ​​or AU is the value of TemporalId of the VCL NAL unit of the coded picture, PU, ​​or AU. The value of TemporalId of a sublayer representation is the maximum value of the TemporalId of all VCL NAL units in the sublayer representation. The value of TemporalId for non-VCL NAL units is constrained as follows: - If nal_unit_type is equal to DCI_NUT, OPI_NUT, VPS_NUT or SPS_NUT, TemporalId shall be equal to 0, and the TemporalId of the AU containing the NAL unit shall be equal to 0. - Otherwise, if nal_unit_type is equal to PH_NUT, TemporalId shall be equal to the TemporalId of the PU containing the NAL unit. - Otherwise, if nal_unit_type is equal to EOS_NUT or EOB_NUT, TemporalId shall be equal to 0. - Otherwise, if nal_unit_type is equal to AUD_NUT, FD_NUT, PREFIX_SEI_NUT or SUFFIX_SEI_NUT, TemporalId shall be equal to the TemporalId of the AU containing the NAL unit. - Otherwise, when nal_unit_type is equal to PPS_NUT, PREFIX_APS_NUT, or SUFFIX_APS_NUT, TemporalId shall be greater than or equal to the TemporalId of the PU containing the NAL unit. NOTE—When the NAL unit is a non-VCL NAL unit, the value of TemporalId is equal to the minimum of the TemporalId values ​​of all AUs to which the non-VCL NAL unit applies. When nal_unit_type is equal to PPS_NUT, PREFIX_APS_NUT, or SUFFIX_APS_NUT, TemporalId may be greater than or equal to the TemporalId of the containing AU, since all PPSs and APSs may be included in the beginning of the bitstream (e.g., when they are transported out-of-band and the receiver places them at the beginning of the bitstream), with the first coded picture having TemporalId equal to 0. nal_unit_type specifies the NAL unit type, ie, the type of RBSP data structure contained in the NAL unit as specified in Table 2. NAL units with a nal_unit_type in the range of UNSPEC28..UNSPEC31 (inclusive), whose semantics are unspecified, shall not affect the decoding process specified in this specification. NOTE—NAL unit types in the range of UNSPEC_28..UNSPEC_31 may be used as determined by the application. The decoding process for these values ​​of nal_unit_type is not specified in this specification. Since different applications may use these NAL unit types for different purposes, special attention is expected when designing encoders that generate NAL units with these nal_unit_type values ​​and when designing decoders that interpret the content of NAL units with these nal_unit_type values. This specification does not define any management of these values. These nal_unit_type values ​​may only be used in contexts where "conflicts" in use (i.e., the meaning of the NAL unit content for the same nal_unit_type value is defined differently) are not important, or are not possible or are managed, for example, defined or managed in a controlling application or transport specification, or managed by the environment that controls the distribution of the bitstream. For purposes other than determining the amount of data in a DU of the bitstream, a decoder should ignore (remove from the bitstream and discard) the contents of all NAL units using reserved values ​​of nal_unit_type. NOTE—This requirement allows for compatible extensions of this specification to be defined in the future. Table 2 NOTE—A clean random access (CRA) picture may have associated RASL or RADL pictures present in the bitstream. NOTE—An Instantaneous Decoding Refresh (IDR) picture with nal_unit_type equal to IDR_N_LP does not have an associated leading picture present in the bitstream. An IDR picture with nal_unit_type equal to IDR W RADL does not have an associated RASL picture present in the bitstream, but may have an associated RADL picture in the bitstream. The value of nal_unit_type shall be the same for all VCL NAL units of a sub-picture. A sub-picture is said to have the same NAL unit type as the VCL NAL units of the sub-picture. For any particular picture's VCL NAL unit, the following applies: - If pps_mixed_nalu_types_in_pic_flag is equal to 0, the value of nal_unit_type shall be the same for all VCL NAL units of a picture, and a picture or PU is said to have the same NAL unit type as the coded slice NAL units of the picture or PU. - Otherwise (pps_mixed_nalu_types_in_pic_flag is equal to 1), all of the following constraints apply: -Picture should have at least two sub-pictures. - A VCL NAL unit of a picture shall have two or more different nal_unit_type values. - There shall be no VCL NAL units for pictures with nal_unit_type equal to GDR_NUT. - When the nal_unit_type of the VCL NAL unit of the picture is equal to the value of IDR_W_RADL, IDR_N_LP or When nalUnitTypeA is set to CRA_NUT, the nal_unit_type of other VCL NAL units of the picture should be equal to nalUnitTypeA or TRAIL_NUT. The value of nal_unit_type shall be the same for all pictures of an IRAP or GDR AU. When sps_video_parameter_set_id is greater than 0, vps_max_tid_il_ref_pics_plus1[i][j] is equal to 0 (for j equal to GeneralLayerIdx[nuh_layer_id] and any value of i in the range of j+1 to vps_max_layers_minus1, inclusive), and pps_mixed_nalu_types_in_pic_flag is equal to 1, the value of nal_unit_type shall not be equal to IDR_W_RADL, IDR_N_LP, or CRA_NUT. Bitstream conformance requires that the following constraints apply: - When a picture is the leading picture of an IRAP picture, the picture shall be a RADL or RASL picture. - When a sub-picture is a leading sub-picture of an IRAP sub-picture, the sub-picture shall be a RADL or RASL sub-picture. - When a picture is not the leading picture of an IRAP picture, the picture shall not be a RADL or RASL picture. - When a sub-picture is not a leading sub-picture of an IRAP sub-picture, the sub-picture shall not be a RADL or RASL sub-picture. - RASL pictures shall not be present in the bitstream, these RASL pictures are associated with IDR pictures. - RASL sub-pictures shall not be present in the bitstream, these RASL sub-pictures are associated with IDR sub-pictures. - RADL pictures shall not be present in the bitstream that are associated with an IDR picture with nal_unit_type equal to IDR_N_LP. NOTE—Random access at the location of an IRAP PU can be performed by discarding all PUs preceding the IRAP AU (and correctly decoding non-RASL pictures in the IRAP AU and all subsequent AUs in decoding order), provided that each parameter set is available (in the bitstream or by external means not specified in this specification) when referenced. - RADL sub-pictures shall not be present in the bitstream, these RADL sub-pictures are associated with IDR sub-pictures with nal_unit_type equal to IDR_N_LP. - Any pictures with nuh_layer_id equal to a particular value layerId that precede the IRAP picture with nuh_llayer_id equal to layerId in decoding order shall precede the IRAP picture in output order, and shall precede any RADL pictures associated with the IRAP picture in output order. - Any sub-picture with nuh_layer_id equal to a specific value layerId and sub-picture index equal to a specific value subpicIdx that precedes the IRAP sub-picture with nuh_layer_id equal to layerId and sub-picture index equal to subpicIdx in decoding order shall precede the IRAP sub-picture and all its associated RADL sub-pictures in output order. - Any pictures with nuh_layer_id equal to a particular value layerId that precede the recovery-point picture with nuh_layer_id equal to layerId in decoding order shall precede the recovery-point picture in output order. - Any sub-picture with nuh_layer_id equal to a particular value layerId and a sub-picture index equal to a particular value subpicIdx that precedes the sub-picture with nuh_layer_id equal to layerId and sub-picture index equal to subpicIdx in the recovery point picture in decoding order shall precede that sub-picture in the recovery point picture in output order. - Any RASL pictures associated with a CRA picture shall precede any RADL pictures associated with the CRA picture in output order. - Any RASL sub-picture associated with a CRA sub-picture shall precede any RADL sub-picture associated with the CRA sub-picture in output order. - Any RASL picture with nuh_layer_id equal to a particular value layerId associated with a CRA picture shall follow, in output order, any IRAP or GDR picture with nuh_layer_id equal to layerId that precedes the CRA picture in decoding order. - Any RASL sub-picture associated with a CRA sub-picture with nuh_layer_id equal to a specific value layerId and a sub-picture index equal to a specific value subpicIdx shall follow, in output order, any IRAP or GDR sub-picture with nuh_layer_id equal to layerId and sub-picture index equal to subpicIdx that precedes the CRA sub-picture in decoding order. - If sps_field_seq_flag is equal to 0, the following applies: when the current picture with nuh_layer_id equal to a specific value layerId is a leading picture associated with an IRAP picture, then the current picture shall precede all non-leading pictures associated with the same IRAP picture in decoding order. Otherwise (sps_field_seq_flag is equal to 1), let picA and picB be the first leading picture and the last leading picture associated with the IRAP picture in decoding order, respectively, there shall be at most one non-leading picture with nuh_layer_id equal to layerId before picA in decoding order, and there shall be no non-leading picture with nuh_layer_id equal to layerId between picA and picB in decoding order. - If sps_field_seq_flag is equal to 0, the following applies: when the current sub-picture with nuh_layer_id equal to a specific value layerId and sub-picture index equal to a specific value subpicIdx is a leading sub-picture associated with an IRAP sub-picture, then the current sub-picture shall precede all non-leading sub-pictures associated with the same IRAP sub-picture in decoding order. Otherwise (sps_field_seq_flag is equal to 1), let subpicA and subpicB be the first leading sub-picture and the last leading sub-picture associated with the IRAP sub-picture, respectively, in decoding order, there shall be at most one non-leading sub-picture with nuh_layer_id equal to layerId and sub-picture index equal to subpicIdx before subpicA in decoding order, and there shall be no non-leading picture with nuh_layer_id equal to layerId and sub-picture index equal to subpicIdx between picA and picB in decoding order. As provided in Table 2, the NAL unit may include a supplemental enhancement information (SEI) syntax structure, and Tables 3 and 4 show the supplemental enhancement information (SEI) syntax structure provided in JVET-T2001. Table 3 Table 4 For Tables 3 and 4, JVET-T2001 provides the following semantics: Each SEI message consists of variables that specify the type payloadType and size payloadSize of the SEI message payload. The derived SEI message payload size payloadSize is specified in bytes and shall be equal to the number of RBSP bytes in the SEI message payload. NOTE—The NAL unit byte sequence containing the SEI message may include one or more emulation prevention bytes (indicated by the emulation_prevention_three_byte syntax element). Since the payload size of the SEI message is specified in the RBSP bytes, the number of emulation prevention bytes is not included in the size of the SEI payload, payloadSize. payload_type_byte is a byte of the payload type of the SEI message. payload_size_byte is the payload size of the SEI message in bytes. It should be noted that JVET-T2001 defines payload types, and the 25th meeting of ISO / IEC JTC1 / SC29 / WG11 held from January 12 to 21, 2022, "Additional SEI messages for VSEI (Draft6)" (teleconference, document JVET-Y2006-V1, which is incorporated herein by reference and referred to as JVET-Y2006) defines additional payload types. Table 5 generally shows the sei_payload() syntax structure. That is, Table 5 shows the sei_payload() syntax structure, but for the sake of brevity, not all possible types of payloads not included in Table 5 are listed. Table 5 For Table 5, JVET-T2001 provides the following semantics. sei_reserved_payload_extension_data shall not be present in a bitstream conforming to this version of this specification. However, decoders conforming to this version of this specification shall ignore the presence and value of sei_reserved_payload_extension_data. When present, the length in bits of sei_reserved_payload_extension_data is equal to 8*payloadSize-nEarlierBits-nPayloadZeroBits-1, where nEarlierBits is the number of bits in the sei_payload() syntax structure preceding the sei_reserved_payload_extension_data syntax element, and nPayloadZeroBits is the number of sei_payload_bit_equal_to_zero syntax elements at the end of the sei_payload() syntax structure. If more_data_in_payload() is true and nPayloadZeroBits is not equal to 7 after parsing the SEI message syntax structure (eg, buffering_period() syntax structure), PayloadBits is set equal to 8*payloadSize-nPayloadZeroBits-1; otherwise, PayloadBits is set equal to 8*payloadSize. payload_bit_equal_to_one should be equal to 1. payload_bit_equal_to_zero shall be equal to 0. NOTE—SEI messages with the same payloadType value are conceptually the same SEI messages regardless of whether they are contained in prefix or suffix SEI NAL units. NOTE—For the SEI messages specified in this specification and the VSEI specification (ITU-T H.274 | ISO / IEC 23002-7), the payloadType values ​​are aligned with similar SEI messages specified in AVC (Rec. ITU-T H.264 | ISO / IEC 14496-10) and HEVC (Rec. ITU-T H.265 | ISO / IEC 23008-2). The semantics and persistence scope of each SEI message is specified in the semantics specification of each specific SEI message. NOTE—The persistence information of the SEI message is summarized in Table 142. JVET-T2001 also provides the following: The SEI message having the syntax structure identified in [Table 5] is specified in Rec. ITU-T H.274 ISO / IEC 23002-7 can be used with the bitstream specified in this specification. When any specific RecITU-T H.274 | ISO / IEC 23002-7 SEI message is included in a bitstream as specified in this specification, the SEI payload syntax shall be included in the sei_payload() syntax structure specified in [Table 5], the payloadType value specified in [Table 5] shall be used, and in addition, any SEI message specific constraints specified in this annex for the specific SEI message shall apply. As described above, the value of PayloadBits is passed to the parser of the SEI message syntax structure specified in Rec. ITU-T H.274 | ISO / IEC 23002-7.

[0039] As described above, JVET-Z2006 provides a NN post-filter supplemental enhancement information message. Specifically, JVET-Z2006 provides a neural network post-filter characteristics SEI message (payloadType==210) and a neural network post-filter activation SEI message (payloadType==211). Tables 6 and 7 show the syntax of the neural network post-filter characteristics SEI message provided in JVET-Z2006. Table 6 Table 7 For Table 6 and Table 7, JVET-Z2006 provides the following semantics. This SEI message specifies a neural network that can be used as a post-processing filter. The use of a specified post-processing filter for a particular picture is indicated by a Neural Network Post-Filter Activation SEI message. The use of this SEI message requires the definition of the following variables: - The cropped decoded output picture width and height in units of luma samples, denoted herein as InpPicWidthInLumaSamples and InpPicHeighInLumaSamples, respectively. - the luma sample array CroppedYPic[y][x] and the chroma sample arrays CroppedCbPic[y][x] and CroppedCrPic[y][x] (when present) of the cropped decoded output picture for vertical coordinate y and horizontal coordinate x, where the upper left corner of the sample array has coordinate y equal to 0 and coordinate x equal to 0. - The bit depth BitDepthY of the luma sample array of the cropped decoded output picture. - The bit depth BitDepthC of the chroma sample array (if any) of the cropped decoded output picture. - The chroma subsampling rate relative to luma represented as InpSubWidthC and InpSubHeightC. - When nnpfc_inp_order_idc is equal to 3, the initial luma quantization parameter value SliceQPY. When this SEI message specifies a neural network that can be used as a post-processing filter, the semantics specify the derivation of the luma sample array FilteredYPic[y][x], the chroma sample arrays FilteredCbPic[y][x], and FilteredCrPic[y][x], as indicated by the value of nnpfc_out_order_idc, which contain the output of the post-processing filter. nnpfc_id contains an identification number that can be used to identify the post-processing filter. The value of nnpfc_id should be between 0 and 2. 32 -2 (including the end value). nnpfc_id ranges from 256 to 511 (inclusive) and from 2 31 to 2 32 The value -2 (inclusive) is reserved for future use by ITU-T|ISO / IEC. 31 to 2 32 Decoders in the range -2 (inclusive) will ignore it. nnpfc_mode_idc equal to 0 specifies that the post-processing filter associated with the nnpfc_id value is determined by external means not specified in this specification. nnpfc_mode_idc equal to 1 specifies that the post-processing filter associated with the nnpfc_id value is a neural network represented by the ISO / IEC 15938-17 bitstream contained in this SEI message. The value of nnpfc_mode_idc shall be in the range of 0 to 255, inclusive. Values ​​of nnpfc_mode_idc greater than 1 are reserved for future ITU-T|ISO / IEC specifications and shall not be present in bitstreams conforming to this version of this specification. Decoders conforming to this version of this specification shall ignore SEI messages containing reserved values ​​of nnpfc_mode_idc. When the current CLVS contains, in decoding order, a previous neural network post-filter characteristics SEI message with a nnpfc_id value equal to the same value of nnpfc_id in this SEI message, at least one of the following conditions shall apply: - This SEI message has nnpfc_mode_idc equal to 1 and has the same content as the previous neural network postfilter characteristics SEI message, except that the values ​​of nnpfc_mode_idc and nnpfc_payload_byte[i] may be different in order to provide neural network updates. - This SEI message has the same content as the previous neural network post-filter characteristics SEI message When this SEI message is the first neural network post-filter characteristics SEI message with a particular nnpfc_id value within the current CLVS in decoding order, it specifies the basic post-processing filters that are relevant to the current decoded picture of the current layer and all subsequent decoded pictures in output order until the end of the current CLVS. When this SEI message is not the first neural network post-filter characteristics SEI message with a particular nnpfc_id value within the current CLVS in decoding order, this SEI message is relevant to the current decoded picture of the current layer and all subsequent decoded pictures in output order until the end of the current CLVS or the next neural network post-filter characteristics SEI message with the particular nnpfc_id value within the current CLVS in output order. nnpfc_purpose indicates the purpose of the post-processing filter as specified in Table 8. The value of nnpfc_purpose shall be between 0 and 2. 32 The nnpfc_purpose values ​​not appearing in Table 8 are reserved for future ITU-T|ISO / IEC specifications and shall not be present in bitstreams conforming to this version of this specification. Decoders conforming to this version of this specification shall ignore SEI messages containing reserved values ​​for nnpfc_purpose. Table 8 NOTE—When ITU-T | ISO / IEC uses the reserved value of nnpfc_purpose in the future, the syntax of this SEI message may be extended with syntax elements whose presence is conditional on nnpfc_purpose being equal to that value. nnpfc_out_sub_width_c_flag and nnpfc_out_sub_height_c_flag are used to derive variables outSubWidthC and outSubHeightC, respectively, which specify the chroma subsampling rate relative to luma in the picture resulting from post-processing filtering. When not present, nnpfc_out_sub_width_c_flag and nnpfc_out_sub_height_c_flag are both inferred to be equal to 0. When nnpfc_out_sub_width_c_flag and nnpfc_out_sub_height_c_flag are present, the sum of nnpfc_out_sub_width_c_flag and nnpfc_out_sub_height_c_flag shall be greater than 0. outSubWidthC=InpSubWidthC-nnpfc_out_sub_width_c_flag outSubHeightC=InpSubHeightC-nnpfc_out_sub_height_c_flag Bitstream conformance requires that both outSubWidthC and outSubHeightC be greater than 0. nnpfc_pic_width_in_luma_samples and nnpfc_pic_height_in_luma_samples specify the width and height, respectively, of the luma sample array of the picture produced by applying the post-processing filter identified by nnpfc_id to the cropped decoded output picture. When nnpfc_pic_width_in_luma_samples and nnpfc_pic_height_in_luma_samples are not present, they are inferred to be equal to InpPicWidthInLumaSamples and InpPicHeightInLumaSamples, respectively. nnpfc_component_last_flag equal to 0 specifies that the second dimension of the input tensor inputTensor is input to the post-processing filter, and the output tensor outputTensor produced from the post-processing filter is used for this channel. nnpfc_component_last_flag equal to 1 specifies that the last dimension of the input tensor inputTensor is input to the post-processing filter, and the output tensor outputTensor produced from the post-processing filter is used for this channel. Note—The specified values ​​of nnpfc_inp_sample_idc and nnpfc_out_sample_idc specify that the first dimension in the input and output tensors, respectively, is used for block indexing, which is a practice in some neural network frameworks. Although the semantics of this SEI message uses a block size equal to 1, it is up to the post-processing implementation to determine the block size used as input for neural network inference. NOTE—Color components are examples of channels. nnpfc_inp_sample_idc indicates the method of converting the sample values ​​of the cropped decoded output picture into input values ​​to the post-processing filter. When nnpfc_inp_sample_idc is equal to 0, 1, 2, or 3, the input values ​​to the post-processing filter are binary16, binary32, binary64, or binary128 floating point values, respectively, as specified in IEEE 754-2019, and the functions InpY and InPc are specified as follows: InpY(x)=x÷((1<<BitDepthY)-1) InpC(x)=x÷((1<<BitDepthC)-1) When nnpfc_inp_sample_idc is equal to 4, the input values ​​to the post-processing filter are unsigned integers, and the functions InpY and InpC are specified as follows: The variable inpTensorBitDepth is derived from the syntax element nnpfc_inp_tensor_bitdepth_minus8 as specified below. The value of nnpfc_inp_sample_idc shall be in the range of 0 to 255, inclusive. Values ​​of nnpfc_inp_sample_idc greater than 4 are reserved for future ITU-T|ISO / IEC specifications and shall not be present in bitstreams conforming to this version of this specification. Decoders conforming to this version of this specification shall ignore SEI messages containing reserved values ​​of nnpfc_inp_sample_idc. nnpfc_inp_tensor_bitdepth_mmus8 plus 8 specifies the bit depth of the luma sample values ​​in the input integer tensor. The value of inpTensorBitDepth is derived as follows: inpTensorBitDepth=nnpfc_inp_tensor_bitdepth_minus8+8 Bitstream conformance requires that the value of nnpfc_inp_tensor_bitdcpth_minus8 shall be in the range of 0 to 24, inclusive. nnpfc_inp_order_idc indicates a method for ordering the sample array of the cropped decoded output picture as input to the post-processing filter. Table 9 contains an informative description of the nnpfc_inp_order_idc value. The semantics of nnpfc_inp_order_idc in the range of 0 to 3 (including the end value) are specified in Table 11, which specifies the process for deriving the input tensor inputTensor for different values ​​of nnpfc_inp_order_idc, and specifies the given vertical sample coordinate cTop and horizontal sample coordinate cLeft of the top left sample position of the sample block included in the input tensor. When the chroma format of the cropped decoded output picture is not 4:2:0, nnpfc_inp_order_idc shall not be equal to 3. The value of nnpfc_inp_order_idc shall be in the range of 0 to 255 (including the end value). Values ​​of nnpfc_inp_order_idc greater than 3 are reserved for future specification by ITU-T|ISO / IEC and shall not be present in bitstreams conforming to this version of this specification. Decoders conforming to this version of this specification shall ignore SEI messages containing reserved values ​​of nnpfc_inp_order_idc. Table 9 A block is a rectangular array of samples of a component (eg, luma or chroma components) from a picture. nnpfc_constant_patch_size_flag equal to 0 specifies that the post-processing filter accepts as input any block size that is a positive integer multiple of the block size indicated by nnpfc_patch_width_minus1 and nnpfc_patch_height_minus1, and nnpfc_constant_patch_size_flag equal to 1 specifies that the post-processing filter accepts as input any block size that is a positive integer multiple of the block size indicated by nnpfc_patch_width_minus1 and nnpfc_patch_height_minus1. nnpfc_patch_widtht_minus1+1, when nnpfc_constant_patch_size_flag is equal to 1, specifies the horizontal sample count of the block size required for the input to the post-processing filter. When nnpfc_constant_patch_size_flag is equal to 0, any positive integer multiple of (nnpfc_patch_width mimis1+1) can be used as the horizontal sample count of the block size used for the input to the post-processing filter. The value of nnpfc_patch_width_minus1 should be in the range of 0 to 32766, inclusive. nnpfc_patch_height_minus1+1, when nnpfc_constant_patch_size_flag is equal to 1, specifies the vertical sample count of the block size required for the input to the post-processing filter. When nnpfc_constant_patch_size_flag is equal to 0, any positive integer multiple of (nnpfc_patch_height_minus1+1) can be used as the vertical sample count of the block size used for the input to the post-processing filter. The value of nnpfc_patch_height_minus1 should be in the range of 0 to 32766, inclusive. nnpfc_overlap specifies the overlapping horizontal and vertical sample counts of adjacent input tensors for the post-processing filter. The value of nnpfc_overlap should be in the range of 0 to 16383 (inclusive). The variables inpPatchWidth, inpPatchHeight, outPatchWidth, outPatchHeight, horCScaling, verCScaling, outPatchCWidth, outPatchCHeight, and overlapSize are derived as follows: Bitstream conformance requires that outPatchWidth*InpPicWidthlnLumaSamples shall be equal to nnpfc_pic_width_in luma samples*inpPatchWidth, and outPatchHeight*InpPicHeightlnLumaSamples shall be equal to nnpfc_pic_height_in_luma_sarnples*inpPatchHeight. nnpfc_padding_type specifies the padding process when referring to sample positions outside the boundaries of the cropped decoded output picture as described in Table 10. The value of nnpfc_padding_type shall be in the range of 0 to 15, inclusive. nnpfc_padding_type describe 0 Zero padding 1 Copy Fill 2 Reflection Fill 3..15 reserve Table 10 The function InpSampleVal(y,x,picHeight,picWidth,croppedPic) whose input is the vertical sample position y, the horizontal sample position x, the picture height picHeight, the picture width picWidth and the sample array croppedPic returns the value of sampleVal derived as follows: Table 11 nnpfc_complexity_idc greater than 0 specifies that there may be one or more syntax elements indicating the complexity of the post-processing filters associated with nnpfc_id. nnpfc_complexity_idc equal to 0 specifies that there are no syntax elements indicating the complexity of the post-processing filters associated with nnpfc_id. The value of mipfc_complexity_idc shall be in the range of 0 to 255, inclusive. Values ​​of nnpfc_complexity_idc greater than 1 are reserved for use by future ITU-T|ISO / IEC specifications and shall not be present in bitstreams conforming to this version of this specification. Decoders conforming to this version of this specification shall ignore SEI messages containing reserved values ​​of nnpfc_complexity_idc. nnpfc_out_sample_idc equal to 0, 1, 2, or 3 indicates that the sample values ​​output by the post-processing filter are binary16, binary32, binary64, or binary128 floating point values, respectively, as specified in IEEE 754-2019. The functions OutY and OutC for converting the luma sample values ​​and chroma sample values ​​output by the post-processing to integer values ​​at bit depths BitDepthY and BitDepthC, respectively, are specified as follows: nnpfc_out_sample_idc equal to 4 indicates that the sample values ​​output by the post-processing filter are unsigned integers. The functions OutY and OutC are specified as follows: The variable outTensorBitDepth is derived from the syntax element nnpfc_out_tensor_bitdepth_minus8 as described below. The value of nnpfc_out_sample_idc shall be in the range of 0 to 255, inclusive. Values ​​of nnpfc_out_sample_idc greater than 4 are reserved for future ITU-T|ISO / IEC specifications and shall not be present in bitstreams conforming to this version of this specification. Decoders conforming to this version of this specification shall ignore SEI messages containing reserved values ​​of nnpfc_out_sample_idc. nnpfc_out_tensor_bitdepth_mmus8 plus 8 specifies the bit depth of the sample values ​​in the output integer tensor. The value of outTensorBitDepth is derived as follows: outTensorBitDepth=nnpfc_out_tensor_bitdepth_minus8+8 Bitstream conformance requires that the value of nnpfcouttensor_bitdepth_minus8 should be in the range 0 to 24, inclusive. nnpfc_out_order_idc indicates the output order of samples generated from the post-processing filter. Table 12 contains an informative description of nnpfc_out_order_idc values. The semantics of nnpfc_out_order_idc in the range of 0 to 3 (including the end value) are specified in Table 13, which specifies the process for deriving sample values ​​in the filtered output sample arrays FilteredYPic, FilteredCbPic, and FilteredCrPic according to the output tensor outputTensor for different values ​​of nnpfc_out_order_idc, and specifies the given vertical sample coordinate cTop and horizontal sample coordinate cLeft of the top left sample position of the sample block included in the input tensor. When nnpfc_purpose is equal to 2 or 4, nnpfc_out_order_idc should not be equal to 3. The value of nnpfc_out_order_idc should be in the range of 0 to 255 (including the end value). Values ​​of nnpfc_out_order_idc greater than 3 are reserved for future specification by ITU-T|ISO / IEC and shall not be present in bitstreams conforming to this version of this specification. Decoders conforming to this version of this specification shall ignore SEI messages containing reserved values ​​of nnpfc_out_order_idc. Table 12 Table 13 The basic post-processing filters used for the cropped decoded output picture picA are the filters identified by the first neural network post-filter characteristics SEI message with a specific nnpfc_id value within CLVS in decoding order. If there is another neural network post-filter characteristics SEI message with the same nnpfc_id value, nnpfc_mode_idc equal to 1, with different content than the neural network post-filter characteristics SEI message defining the basic post-processing filter and related to the picture picA, the basic post-processing filter is updated by decoding the ISO / IEC 15938-17 bitstream in the neural network post-filter characteristics SEI message to obtain the post-processing filter PostProcessingFilter(). Otherwise, the post-processing filter PostProcessingFilter() is specified to be the same as the basic post-processing filter. The following process is used to filter the cropped decoded output picture using a post-processing filter PostProcessingFilter() to generate a filtered picture, which contains Y, Cb, Cr sample arrays FilteredYPic, FilteredCbPic, FilteredCrPic, respectively, as indicated by nnpfc_out_order_idc. nnpfc_reserved_zero_bit should be equal to 0. nnpfc_payload_byte[i] contains the i-th byte of a bitstream conforming to ISO / IEC 15938-17. The byte sequence nnpfc_payload_byte[i] for all current values ​​of i shall be a complete bitstream conforming to ISO / IEC 15938-17. nnpfc_parameter_type_flag equal to 0 indicates that the neural network uses only integer parameters, and nnpfc_parameter_type_flag equal to 1 indicates that the neural network can use floating point or integer parameters. nnpfc_log2_parameter_bit_length_minus3 equal to 0, 1, 2, and 3 instructs the neural network to not use parameters with bit lengths greater than 8, 16, 32, and 64, respectively. nnpfc_num_parameters_idc indicates the maximum number of neural network parameters for the post-processing filters in powers of 2048. nnpfc_num_parameters_idc equal to 0 indicates that the maximum number of neural network parameters is not specified. If the value of nnpfc_num_parameters_idc is greater than zero, the variable maxNumParameters is derived as follows: maxNumParameters=(2048<<nnfpc_num_parameters_idc)-1 Bitstream conformance requires that the number of neural network parameters of the post-processing filter shall be less than or equal to maxNumParameters. nnpfc_num_kmac_operations_idc greater than 0 specifies that the maximum number of multiply-accumulate operations per sample of the post-processing filter is less than or equal to nnpfc_num_kmac_operations_idc * 1000. nnpfc_num_kmac_operations_idc equal to 0 specifies that the maximum number of multiply-accumulate operations is not specified. Table 14 shows the syntax of the neural network post-filter activation SEI message provided in JVET-Z2006. Table 14 For Table 14, JVET-Z2006 provides the following semantics. This SEI message specifies the neural network post-processing filters that can be used as post-processing filters for the current picture. The neural network post-processing filter activation SEI message is only valid for the current picture. NOTE—There may be several neural network post-processing filter activation SEI messages for the same picture, for example, when the post-processing filters are intended for different purposes or filter different color components. nnpfa_id specifies that the neural network post-processing filters specified by one or more neural network post-processing filter characteristics SEI messages associated with the current picture and having nnpfc_id equal to nnfpa_id can be used for post-processing filtering of the current picture. Additionally, it should be noted that JVET-Z2006 provides the following for the use of the NN post-filter characteristics SEI message: For the purpose of interpreting the Neural Network Post-Filter Characteristics SEI message, the following variables are specified: -InpPicWidthInLumaSamples is set equal to pps_pic_width_in_luma_samples-SubWidthC*(pps_conf_win_left_offset+pps_conf_win_right_offset). -InpPicHeightInLumaSamples is set equal to pps_pic_height_in_luma_samples-SubHeightC*(pps_conf_win_top_offset+pps_conf_win_bottom_offset). - When present, the variables CroppedYPic[y][x], the chroma sample arrays CroppedCbPic[y][x], and CroppedCrPic[y][x] are set to 2-dimensional arrays of decoded sample values ​​of the 0th, 1st, and 2nd components, respectively, of the cropped decoded output picture to which the Neural Network Post-Filter Characteristics SEI message applies. -BitDepthY and BitDepthC are both set equal to BitDepth. -InpSubWidthC is set equal to SubWidthC. -InpSubHeightC is set equal to SubHeightC. -SliceQPY is set equal to SliceQp Y . When neural network post-filter characteristics SEI messages with the same nnpfc_id and different contents are present in the same picture unit, the two neural network post-filter characteristics SEI messages shall be present in the same SEI NAL unit.

[0040] The neural network post-filter characteristics SEI message provided in JVET-Z2006 may not be ideal. Specifically, the signaling in JVET-Z2006 may not be sufficient to indicate the various neural network post-filter parameters. According to the techniques described herein, additional syntax and semantics are provided for indicating the various neural network post-filter parameters.

[0041] Figure 1 1 is a block diagram illustrating an example of a system that may be configured to encode (e.g., encode and / or decode) video data in accordance with one or more techniques of this disclosure. System 100 represents an example of a system that may encapsulate video data in accordance with one or more techniques of this disclosure. Figure 1 As shown, system 100 includes source device 102, communication medium 110, and target device 120. Figure 1 In the example shown, source device 102 may include any device configured to encode video data and transmit the encoded video data to communication medium 110. Target device 120 may include any device configured to receive the encoded video data via communication medium 110 and decode the encoded video data. Source device 102 and / or target device 120 may include computing devices equipped for wired and / or wireless communication, and may include, for example, set-top boxes, digital video recorders, televisions, desktop, laptop or tablet computers, game consoles, medical imaging devices, and mobile devices (including, for example, smart phones, cellular phones, personal gaming devices).

[0042] The communication medium 110 may include any combination of wireless and wired communication media and / or storage devices. The communication medium 110 may include coaxial cables, fiber optic cables, twisted pair cables, wireless transmitters and receivers, routers, switches, repeaters, base stations, or any other device that can be used to facilitate communication between various devices and sites. The communication medium 110 may include one or more networks. For example, the communication medium 110 may include a network configured to allow access to the World Wide Web, such as the Internet. The network may operate according to a combination of one or more telecommunication protocols. The telecommunication protocol may include proprietary aspects and / or may include standardized telecommunication protocols. Examples of standardized telecommunication protocols include digital video broadcasting (DVB) standards, advanced television system committee (ATSC) standards, integrated service digital broadcasting (ISDB) standards, cable data service interface specifications (DOCSIS) standards, global mobile communication systems (GSM) standards, code division multiple access (CDMA) standards, 3rd generation partnership project (3GPP) standards, European Telecommunications Standards Institute (ETSI) standards, Internet Protocol (IP) standards, wireless application protocol (WAP) standards, and Institute of Electrical and Electronics Engineers (IEEE) standards.

[0043] The storage device may include any type of device or storage medium capable of storing data. The storage medium may include a tangible or non-transitory computer-readable medium. The computer-readable medium may include an optical disk, a flash memory, a magnetic memory, or any other suitable digital storage medium. In some examples, the memory device or a portion thereof may be described as a non-volatile memory, and in other examples, a portion of the memory device may be described as a volatile memory. Examples of volatile memory may include random access memory (RAM), dynamic random access memory (DRAM), and static random access memory (SRAM). Examples of non-volatile memory may include a magnetic hard disk, an optical disk, a floppy disk, a flash memory, or an electrically programmable memory (EPROM) or an electrically erasable and programmable (EEPROM) memory. The storage device may include a memory card (e.g., a secure digital (SD) memory card), an internal / external hard disk drive, and / or an internal / external solid-state drive. Data may be stored on the storage device according to a defined file format.

[0044] Figure 4 is a conceptual diagram illustrating examples of components that may be included in a specific implementation of system 100. Figure 4 In the exemplary implementation shown, system 100 includes one or more computing devices 402A-402N, a television service network 404, a television service provider site 406, a wide area network 408, a local area network 410, and one or more content provider sites 412A-412N. Figure 4The specific implementations shown in represent examples of a system that can be configured to allow digital media content (such as movies, live sporting events, etc.) and data and applications associated therewith and media presentations to be distributed to and accessed by multiple computing devices (such as computing devices 402A to 402N). Figure 4 In the example shown, computing devices 402A to 402N may include any device configured to receive data from one or more of television service network 404, wide area network 408, and / or local area network 410. For example, computing devices 402A to 402N may be equipped for wired and / or wireless communications and may be configured to receive services via one or more data channels, and may include televisions, including so-called smart televisions, set-top boxes, and digital video recorders. In addition, computing devices 402A to 402N may include desktop computers, laptops or tablet computers, game consoles, mobile devices (including, for example, "smart" phones, cellular phones, and personal gaming devices).

[0045] The television service network 404 is an example of a network configured to enable distribution of digital media content that may include television services. For example, the television service network 404 may include a public air television network, a public or subscription-based satellite television service provider network, and a public or subscription-based cable television provider network and / or a cloud or Internet service provider. It should be noted that, although in some examples, the television service network 404 may be primarily used to allow the provision of television services, the television service network 404 may also allow the provision of other types of data and services according to any combination of the telecommunications protocols described herein. In addition, it should be noted that in some examples, the television service network 404 may allow two-way communication between the television service provider site 406 and one or more of the computing devices 402A to 402N. The television service network 404 may include any combination of wireless and / or wired communication media. The television service network 404 may include coaxial cables, fiber optic cables, twisted pair cables, wireless transmitters and receivers, routers, switches, repeaters, base stations, or any other device that can be used to facilitate communication between various devices and sites. The television service network 404 may operate according to a combination of one or more telecommunications protocols. The telecommunication protocol may include proprietary aspects and / or may include a standardized telecommunication protocol. Examples of standardized telecommunication protocols include the DVB standard, the ATSC standard, the ISDB standard, the DTMB standard, the DMB standard, the Data Over Cable Service Interface Specification (DOCSIS) standard, the HbbTV standard, the W3C standard, and the UPnP standard.

[0046] Reference again Figure 4, the television service provider site 406 may be configured to distribute television services via the television service network 404. For example, the television service provider site 406 may include one or more broadcast stations, cable television providers or satellite television providers, or Internet-based television providers. For example, the television service provider site 406 may be configured to receive transmissions (including television programs) via satellite uplink / downlink. In addition, as Figure 4 As shown, television service provider site 406 may be in communication with wide area network 408 and may be configured to receive data from content provider sites 412A through 412N. It should be noted that in some examples, television service provider site 406 may include a television studio and the content may originate from the television studio.

[0047] Wide area network 408 may include a packet-based network and operate according to a combination of one or more telecommunication protocols. Telecommunication protocols may include proprietary aspects and / or may include standardized telecommunication protocols. Examples of standardized telecommunication protocols include global system mobile communications (GSM) standards, code division multiple access (CDMA) standards, 3rd generation partnership project (3GPP) standards, European Telecommunications Standards Institute (ETSI) standards, European standards (EN), IP standards, wireless application protocols (WAP) standards, and Institute of Electrical and Electronics Engineers (IEEE) standards, such as, one or more IEEE 802 standards (e.g., Wi-Fi). Wide area network 408 may include any combination of wireless and / or wired communication media. Wide area network 408 may include coaxial cables, fiber optic cables, twisted pair cables, Ethernet cables, wireless transmitters and receivers, routers, switches, repeaters, base stations, or any other equipment that can be used to facilitate communication between various devices and sites. In one example, wide area network 408 may include the Internet. Local area network 410 may include a packet-based network and operate according to a combination of one or more telecommunication protocols. The local area network 410 may be distinguished from the wide area network 408 based on access levels and / or physical infrastructure. For example, the local area network 410 may include a secure home network.

[0048] Reference again Figure 4, content provider sites 412A to 412N represent examples of sites that can provide multimedia content to television service provider sites 406 and / or computing devices 402A to 402N. For example, a content provider site may include a studio with one or more studio content servers, which are configured to provide multimedia files and / or streams to television service provider sites 406. In one example, content provider sites 412A to 412N may be configured to provide multimedia content using an IP suite. For example, a content provider site may be configured to provide multimedia content to a receiver device according to a real-time streaming protocol (RTSP), HTTP, etc. In addition, content provider sites 412A to 412N may be configured to provide data including hypertext-based content, etc. to one or more of receiver devices 402A to 402N and / or television service provider sites 406 via a wide area network 408. Content provider sites 412A to 412N may include one or more web servers. The data provided by data provider sites 412A to 412N may be defined according to a data format.

[0049] Reference again Figure 1 , source device 102 includes video source 104, video encoder 106, data encapsulator 107 and interface 108. Video source 104 may include any device configured to capture and / or store video data. For example, video source 104 may include a camera and a storage device operably coupled thereto. Video encoder 106 may include any device configured to receive video data and generate a compatible bitstream representing the video data. A compatible bitstream may refer to a bitstream from which a video decoder can receive and reproduce video data. Various aspects of a compatible bitstream may be defined according to a video coding standard. When generating a compatible bitstream, video encoder 106 may compress the video data. The compression may be lossy (perceptible or imperceptible to the observer) or lossless. Figure 5 is a block diagram illustrating an example of a video encoder 500 that can implement the techniques described herein for encoding video data. It should be noted that although the exemplary video encoder 500 is illustrated as having different functional blocks, such illustration is intended for descriptive purposes and does not limit the video encoder 500 and / or its subcomponents to a particular hardware or software architecture. The functionality of the video encoder 500 may be implemented using any combination of hardware, firmware, and / or software implementations.

[0050] The video encoder 500 can perform intra-frame prediction encoding and inter-frame prediction encoding of a picture region and thus can be referred to as a hybrid video encoder. Figure 5In the example shown, the video encoder 500 receives a source video block. In some examples, the source video block may include picture regions that have been divided according to a coding structure. For example, the source video data may include macroblocks, CTUs, CBs, sub-partitions thereof, and / or additional equivalent coding units. In some examples, the video encoder 500 may be configured to perform additional subdivisions of the source video block. It should be noted that the techniques described herein are generally applicable to video encoding regardless of how the source video data is divided before and / or during encoding. Figure 5 In the example shown, the video encoder 500 includes an adder 502, a transform coefficient generator 504, a coefficient quantization unit 506, an inverse quantization and transform coefficient processing unit 508, an adder 510, an intra-frame prediction processing unit 512, an inter-frame prediction processing unit 514, a filter unit 516, and an entropy coding unit 518. Figure 5 As shown, video encoder 500 receives source video blocks and outputs a bitstream.

[0051] exist Figure 5 In the example shown, the video encoder 500 can generate residual data by subtracting the predicted video block from the source video block. The selection of the predicted video block is described in detail below. The summer 502 represents a component configured to perform the subtraction operation. In one example, the subtraction of the video block occurs in the pixel domain. The transform coefficient generator 504 applies a transform such as a discrete cosine transform (DCT), a discrete sine transform (DST), or a conceptually similar transform (for example, four 8×8 transforms can be applied to a 16×16 residual value array) to the residual block or its sub-partition to generate a set of residual transform coefficients. The transform coefficient generator 504 can be configured to perform any and all combinations of transforms included in the discrete triangular transform series, including its approximation. The transform coefficient generator 504 can output the transform coefficients to the coefficient quantization unit 506. The coefficient quantization unit 506 can be configured to perform quantization of the transform coefficients. The quantization process can reduce the bit depth associated with some or all coefficients. The degree of quantization can change the rate distortion (i.e., the relationship between bit rate and video quality) of the encoded video data. The degree of quantization can be modified by adjusting a quantization parameter (QP). The quantization parameter can be determined based on a slice level value and / or a CU level value (e.g., a CU delta QP value). The QP data may include any data used to determine the QP used to quantize a particular set of transform coefficients. Figure 5 As shown, the quantized transform coefficients (which may be referred to as level values) are output to the inverse quantization and transform coefficient processing unit 508. The inverse quantization and transform coefficient processing unit 508 may be configured to apply inverse quantization and inverse transformation to generate reconstructed residual data. Figure 5As shown, at the summer 510, the reconstructed residual data can be added to the predicted video block. In this way, the encoded video block can be reconstructed, and the resulting reconstructed video block can be used to evaluate the coding quality of a given prediction, transformation and / or quantization. The video encoder 500 can be configured to perform multiple encoding rounds (e.g., performing encoding while changing one or more of the prediction, transformation parameters and quantization parameters). The rate distortion or other system parameters of the bitstream can be optimized based on the evaluation of the reconstructed video block. In addition, the reconstructed video block can be stored and used as a reference for predicting subsequent blocks.

[0052] Reference again Figure 5 , the intra-frame prediction processing unit 512 may be configured to select an intra-frame prediction mode for a video block to be encoded. The intra-frame prediction processing unit 512 may be configured to evaluate a frame and determine an intra-frame prediction mode for encoding a current block. As described above, possible intra-frame prediction modes may include a planar prediction mode, a DC prediction mode, and an angular prediction mode. Furthermore, it should be noted that in some examples, the prediction mode of the chrominance component may be inferred based on the prediction mode of the luma prediction mode. The intra-frame prediction processing unit 512 may select an intra-frame prediction mode after performing one or more encoding rounds. Furthermore, in one example, the intra-frame prediction processing unit 512 may select a prediction mode based on a rate-distortion analysis. As Figure 5 As shown, the intra prediction processing unit 512 outputs intra prediction data (e.g., syntax elements) to the entropy coding unit 518 and the transform coefficient generator 504. As described above, the transform performed on the residual data can be mode-dependent (e.g., the secondary transform matrix can be determined based on the prediction mode).

[0053] Reference again Figure 5 , the inter-frame prediction processing unit 514 may be configured to perform inter-frame prediction encoding for the current video block. The inter-frame prediction processing unit 514 may be configured to receive a source video block and calculate a motion vector of a PU of the video block. The motion vector may indicate the displacement of a prediction unit of a video block within a current video frame relative to a prediction block within a reference frame. Inter-frame prediction encoding may use one or more reference pictures. In addition, motion prediction may be unidirectional prediction (using one motion vector) or bidirectional prediction (using two motion vectors). The inter-frame prediction processing unit 514 may be configured to select a prediction block by calculating a pixel difference determined by, for example, the sum of absolute differences (SAD), the sum of square differences (SSD), or other difference metrics. As described above, a motion vector may be determined and specified based on motion vector prediction. As described above, the inter-frame prediction processing unit 514 may be configured to perform motion vector prediction. The inter-frame prediction processing unit 514 may be configured to generate a prediction block using motion prediction data. For example, the inter-frame prediction processing unit 514 may locate a prediction video block ( Figure 5). It should be noted that the inter-frame prediction processing unit 514 can be further configured to apply one or more interpolation filters to the reconstructed residual block to calculate sub-integer pixel values ​​for motion estimation. The inter-frame prediction processing unit 514 can output the motion prediction data of the calculated motion vector to the entropy coding unit 518.

[0054] like Figure 5 As shown, filter unit 516 receives the reconstructed video blocks and the encoding parameters and outputs the modified reconstructed video data. Filter unit 516 may be configured to perform deblocking and / or sample adaptive offset (SAO) filtering. SAO filtering is a nonlinear amplitude mapping that can be used to improve reconstruction by adding an offset to the reconstructed video data. It should be noted that, as Figure 5 As shown, the intra prediction processing unit 512 and the inter prediction processing unit 514 may receive the modified reconstructed video block via the filter unit 216. The entropy coding unit 518 receives the quantized transform coefficients and the prediction syntax data (i.e., the intra prediction data and the motion prediction data). It should be noted that in some examples, the coefficient quantization unit 506 may perform a scan of the matrix including the quantized transform coefficients before outputting the coefficients to the entropy coding unit 518. In other examples, the entropy coding unit 518 may perform the scan. The entropy coding unit 518 may be configured to perform entropy coding according to one or more of the techniques described herein. As such, the video encoder 500 represents an example of a device configured to generate encoded video data according to one or more techniques of the present disclosure.

[0055] Reference again Figure 1 , the data encapsulator 107 can receive the encoded video data and generate a compatible bitstream according to a defined data structure, for example, a sequence of NAL units. A device receiving the compatible bitstream can reproduce the video data therefrom. In addition, as described above, sub-bitstream extraction can refer to the process by which a device receiving a compatible bitstream forms a new compatible bitstream by discarding and / or modifying data in the received bitstream. It should be noted that the term conforming bitstream can be used instead of the term compatible bitstream. In one example, the data encapsulator 107 can be configured to generate syntax according to one or more techniques described herein. It should be noted that the data encapsulator 107 does not necessarily have to be located in the same physical device as the video encoder 106. For example, the functions described as being performed by the video encoder 106 and the data encapsulator 107 can be distributed in Figure 4 in the device shown. As described above, the signaling provided in JVET-Z2006 may be insufficient. Specifically, the signaling of the padding type in JVET-Z2006 may not be sufficient to signal the type of padding that may be used. For example, it may be useful to allow signaling of one or more of wrap-around padding, fixed or constant padding, maximum padding, minimum padding, median padding, and / or average padding, where examples of these respective padding types may be as follows: Circular fill (or ring fill or wrap around fill): Fills using the wrapping of a vector around an axis. Use the values ​​at the beginning to fill the end, and the values ​​at the end to fill the beginning. It is used to create a wrap around effect after being applied on the input; this allows information to be seamlessly replicated across the border. This type of padding may be useful for omnidirectional or 360 degree video. Wrap_padding([1,2,3,4,5],(2,3))=[4 5 1 2 3 4 5 1 2 3] Fixed padding: pads with a specified value. You can use fixed padding based on your application needs. For example, you can do a mid-level gray padding (i.e., nnpfc_padding_val is equal to 128). Fixed_padding([1,2,3,4,5],(2,3),128)=[128 128 1 2 3 4 5 128 128 128] Max / Min / Median / Average Fill: Fill using the maximum / minimum / median / average values. These types of padding allow specific values ​​to be chosen in the array to be replicated across the boundaries. It allows values ​​to be chosen that are more specific to the input data, rather than random values ​​or zeros. maximum_padding([1, 2, 3, 4, 5], (2, 3)) = [5 5 1 2 3 4 5 5 5 5]

[0056] Table 15 shows the syntax of an exemplary neural network post-filter characteristics SEI message according to the techniques herein. Table 15 For Table 15, the semantics may be based on the semantics provided above and based on the following: nnpfc_padding_type specifies the padding process when referencing sample positions outside the boundaries of the cropped decoded output picture as described in Table 15A. The value of nnpfc_padding_type shall be in the range of 0 to 15, inclusive. nnpfc_padding_type describe 0 Zero padding 1 Copy Fill 2 Reflection Fill 3 Circular fill (or ring fill or wrap around fill) 4 Fixed Fill (or Constant Fill) 5 Minimum Fill 6 Maximum value filling 7 Average value filling 8 Median Fill 9..15 reserve Table 15A nnpfc_padding_val specifies the value to be used for padding when nnpfc_padding_type is equal to 4 (or equal to fixed or constant padding). or nnpfc_padding_val specifies the value to be used for padding when nnpfc_padding_type indicates fixed padding. The function InpSampleVal(y,x,picHeight,picWidth,croppedPic) whose input is the vertical sample position y, the horizontal sample position x, the picture height picHeight, the picture width picWidth and the sample array croppedPic returns the value of sampleVal derived as follows: Minimum(croppedPic) returns the minimum value of the samples in the croppedPic array. Maximum(croppedPic) returns the maximum value of the samples in the croppedPic array. Mean(croppedPic) returns the mean of all samples in the croppedPic array. Median(croppedPic) returns the median of the sample In other examples, a subset of the above fill types may be defined as provided in Table 15B: npfc_padding_type describe 0 Zero padding 1 Copy Fill 2 Reflection Fill 3 Fill around 4 Fixed padding 5..15 reserve Table 15B

[0057] In another example, the fixed fill may be referred to as a constant fill. In another example, fixed padding may replace zero padding, since zero padding is a special case of fixed padding with nnpfc_padding_val equal to 0. Therefore, in one example, the padding type may be defined as provided in Table 15C. nnpfc_padding_type describe 0 Fixed padding 1 Copy Fill 2 Reflection Fill 3 Fill around 4..15 reserve Table 15C in,

[0058] In one example, according to the techniques herein, for fixed padding, a padding value may be signaled for each of the luma component and the chroma component. Table 16 shows the syntax of an exemplary neural network post-filter characteristics SEI message according to the techniques herein. Table 16 In one example, the signaling of nnpfc_cb_padding_val and nnpfc_cr_padding_val may be further conditioned on the chroma format. For example, for monochrome, ie, for a ChromaFormatIdc (or sps_chroma_format_idc) value of 0 indicating that the picture has only a luma component, these two syntax elements may not be signaled. For Table 16, the semantics may be based on the semantics provided above and based on the following: nnpfc_luma_padding_val specifies the luma value to be used for padding when nnpfc_padding_type is equal to 4 (ie, fixed padding). nnpfc_cb_padding_val specifies the Cb value to be used for padding when nnpfc_padding_type is equal to 4 (ie, fixed padding). nnpfc_cr_padding_val specifies the Cr value to be used for padding when nnpfc_padding_type is equal to 4 (ie, fixed padding). In another example, the semantics may be based on the following: nnpfc_luma_paddirig_val specifies the luma value to be used to pad the luma sample array when nnpfc_padding_type is equal to 4 (ie, fixed padding). nnpfc_cb_padding_val specifies the Cb value to be used to pad the Cb sample array when nnpfc_padding_type is equal to 4 (ie, fixed padding). nnpfc_cr_padding_val specifies the Cr value to be used to pad the Cr sample array when nnpfc_padding_type is equal to 4 (ie, fixed padding). In another example, the semantics may be based on the following: nnpfc_luma_padding_val specifies the value to be used to pad the luma sample array when nnpfc_padding_type is equal to 4 (ie, fixed padding). nnpfc_cb_padding_val specifies the value used to pad the Cb sample array when nnpfc_padding_type is equal to 4 (ie, fixed padding). nnpfc_cr_padding_val specifies the value to be used to pad the Cr sample array when nnpfc_padding_type is equal to 4 (ie, fixed padding). In one example, a common value for padding values ​​for both Cb and Cr may be signaled instead of separate values ​​(nnpfc_cb_padding_val, nnpfc_cr_padding_val). For example, the syntax element nnpfc_cbcr_padding_val may specify the value to be used to pad the Cb sample array and the Cr sample array when nnpfc_padding_type is equal to 4 (ie, fixed padding). in, The function InpSampleVal(y,x,picHeight,picWidth,croppedPic), whose input is the vertical sample position y, the horizontal sample position x, the picture height picHeight, the picture width picWidth and the sample array croppedPic, returns the value of sampleVal or SampleValY, SampleValCb, SampleYCr derived as follows, where SampleValY represents the array of luma sample values, SampleValCb represents the array of Cb (chroma) sample values, and SampleValCr represents the array of Cr (chroma) sample values. In one example, SampleValY can be referred to as sampleVal[0], SampleValCb can be referred to as sampleVal[1] and SampleValCr can be referred to as sampleVal[2], basically, SampleValY (or sampleVal[0]) is the luma sample value, SampleValCb (or sampleVal[1]) is the Cb (chroma) sample value, and SampleValCr (or sampleVal[2]) is the Cr (chroma) sample value. For example, based on the following: In one example, an additional other condition may be added to the portion of the above pseudocode where the other portion (i.e., when y is less than 0 or y is greater than or equal to picHeight) specifies that no padding or some other padding (e.g., zero or constant or copied or reflected padding may be used): In another example, zero padding may be added on the top and bottom borders, such as shown below: In another example, it can be assumed that "y" never goes beyond the bounds (i.e., no padding on top and bottom, wrapping / looping / ringing padding only along the width dimension), such as as follows: In one example, an additional condition may be added to the portion of the pseudocode above where if y is less than 0 or y is greater than or equal to picHeight, then no padding or some other padding (e.g., zero or constant or replicated or reflected padding may be used) is specified: In another example, wrap-around filling can be done as follows: the left side of the image is filled by the rightmost part of the image, and vice versa. In the vertical axis, the top left corner of the filled image is filled by the top right corner, and vice versa. In one example, this can be done as follows: in, The signaling of complexity elements in JVET-Z2006 may be insufficient. In one example, according to the techniques of this article, new values ​​can be used to indicate the type of neural network parameters. For example, a binary parameter represents a binary neural network in which weights and biases take only two values. An example binarized neural network or BNN network is described as having weights and activations constrained to +1 or -1. The weights and biases of the BNN are constrained to -1 or 1: and the trained network is used for the task of image classification. In addition, the double floating point parameter specifies that the network parameters use double precision. For example, a typical floating point value is in the range of [-3.4e+38, +3.4e+38], while a double floating point value has a range of [-1.7e+308, +1.7e+308]. In one example, according to the techniques of this article, nnpfc_complexity_element() can be based on the syntax provided in Table 17. Table 17 For Table 17, the semantics may be based on the semantics provided above as well as the following semantics: nnpfc_parameter_type_idc equal to 0 indicates that the neural network uses only integer parameters. nnpfc_parameter_type_flag equal to 1 indicates that the neural network can use floating point or integer parameters. nnpfc_parameter_type_idc equal to 2 indicates that the neural network can only use binary parameters. nnpfc_parameter_type_idc equal to 3 indicates that the neural network can use double floating point, floating point, or integer parameters. In one example, the semantics of nnpfc_parameter_type_idc may be based on the following: nnpfc_parameter_type_idc indicates the type of parameter used by the neural network. The value of nnpfc_parameter_type_flag is between 0 and 3 (inclusive). Table 18 describes the different values. nnpfc_parameter_type_idc describe 0 Integer parameters 1 Floating point parameters 2 Binary parameters 3 Double floating point parameters Table 18 In one example, according to the techniques herein, nnpfc_complexity_element() may be based on the syntax provided in Table 19. Table 19 As provided in Table 19, nnpfc_log2_parameter_bit_length_minus3 is signaled only when nnpfc_parameter_type_idc is not equal to 2. For Table 18, the semantics may be based on the semantics provided above, and in one example, the semantics of nnpfc_log2_parameter bit_length_minus3 may be based on the following: nnpfc_log2_parameter_bit_length_niinus3 equal to 0, 1, 2, and 3 indicates that the neural network does not use parameters with bit lengths greater than 8, 16, 32, and 64, respectively. When nnpfc_parameter_type_idc is present and nnpfc_log2_parameter_bit_length_minus3 is not present, the neural network does not use parameters with bit lengths greater than 1. In one example, according to the techniques herein, a syntax element nnpfc_log2_parameter_bit_length_flag may be included in nnpfc_complexity_element(). In one example, this syntax element is signaled only when nnpfc_parameter_type_idc indicates that the neural network uses binary parameters. The syntax element indicates that the bit length of the neural network parameter does not exceed 1. In one example, according to the techniques herein, nnpfc_complexity_element() may be based on the syntax provided in Table 20. Table 20 For Table 20, the semantics may be based on the semantics provided above, and in one example, the semantics of nnpfc_log2_parameter_bit_length_flag may be based on the following: nnpfc_log2_parameter_bit_length_flag equal to 1 indicates that the neural network does not use parameters with a bit length greater than 1.

[0059] It should be noted that the neural network post-processing filter can be used to upsample the frame rate of the decoded video. JVET-Z2006 does not provide sufficient signaling to indicate a neural network post-processing filter that upsamples the frame rate. In one example, according to the techniques of this document, a signal can be sent to notify frame rate upsampling as an additional purpose of the post-processing filter. In one example, according to the techniques of this document, a neural network post-processing filter can take two or more decoded output pictures as input and interpolate one or more pictures between the pictures it takes as input. Table 21 shows the syntax of an exemplary neural network post-filter characteristics SEI message according to the techniques of this document. Table 21 For Table 21, the semantics may be based on the semantics provided above and based on the following: nnpfc_purpose indicates the purpose of the post-processing filter as specified in Table 22. The value of nnpfc_purpose shall be between 0 and 2. 32 nnpfc_purpose values ​​not appearing in Table 22 are reserved for use by future ITU-T|ISO / IEC specifications and shall not be present in bitstreams conforming to this version of this specification. Decoders conforming to this version of this specification shall ignore SEI messages containing reserved values ​​for nnpfc_purpose. Table 22 nnpfc_number_of_input_pictures_minus1 plus 1 specifies the number of decoded output pictures used as input to the post-processing filter. nnpfc_number_of_interpolated_pictures_minus1 plus 1 specifies the total number of interpolated pictures shown by the post-processing filter. Alternatively, in one example, the syntax elements nnpfc_number_of_input_pictures_minus1 and nnpfc_number_of_interpolated_pictures_minus1 may be signaled without minus1 coding as follows: nnpfc_number_of_input_pictures specifies the number of decoded output pictures used as input to the post-processing filters. nnpfc_number_of_interpolated_pictures specifies the total number of interpolated pictures shown by the post-processing filter. In other examples, the fixed length codes u(v), u(2), u(3), or u(4) may be used to signal any of the following syntax elements: nnpfc_number_of_input_pictures, nnpfc_number_of_input_pictures_minus1, nnpfc_number_of_interpolated_pictures, or nnpfc_numbcr_of_interpolated_pictures_minus1. JVET-Z2006 provides for some syntax elements where invalid values ​​may be signaled. In one example, according to the techniques herein, the semantics of the syntax elements may be modified to provide restrictions to prohibit signaling invalid values. In one example, according to the techniques herein, the semantics of the syntax element nnpfc_constant_patch_size_flag may be based on the following: nnpfc_constant_patch_size_flag equal to 0 specifies that the post-processing filter accepts as input any block size that is a positive integer multiple of the block size indicated by nnpfc_patch_width_minus1 and nnpfc_patch_height_min_us1. When nnpfc_constant_patch_size_flag is equal to 0, the block size width shall be less than or equal to InpPicWidthlnLumaSamples. When nnpfc_constant_patch_size_flag is equal to 0, the block size height shall be less than or equal to InpPicHeightlnLumaSamples. nnpfc_constant_patch_size_flag equal to 1 specifies that the post-processing filter accepts as input exactly the block size indicated by nnpfc_patch_width_mitius1 and nnpfc_patch_height_minus1. The variables inpPatchWidth, inpPatchHeight, outPatchWidth, outPatchHeight, horCScaling, verCScaling, outPatchCWidth, outPatchCHeight, and overlapSize are derived as follows: Bitstream conformance requires that outPatchWidth*InpPicWidthInLumaSamples shall be equal to nnpfc_pic_width_in_luma_samples*inpPatchWidth, and outPatchHeight* InpPicHeightlnLumaSamples should be equal to nnpfc_pic_height_in_luma_samples*inpPatchHeight. In one example, according to the techniques herein, the grammatical element The semantics of nnpfc_patch_width_minus1 and nnpfc-patch_height_minus1 can be based on the following: nnpfc_patch_width_minus1+1, when nnpfc_constant_patch_size_flag is equal to 1, specifies the horizontal sample count of the block size required for the input to the post-processing filter. When nnpfc_constant_patch_size_flag is equal to 0, any positive integer multiple of (nnpfc_patch_width_minus1+1) can be used as the horizontal sample count of the block size used for the input to the post-processing filter. The value of nnpfc_patch_width_minus1 shall be in the range of 0 to 32766 (inclusive), and nnpfc_patch_width_minus1 shall be less than InpPicWidthInLumaSamples. nnpfc_patch_height_minus1+1, when nnpfc_constant_patch_size_flag is equal to 1, specifies the vertical sample count of the block size required for the input to the post-processing filter. When nnpfc_constant_patch_size_flag is equal to 0, any positive integer multiple of (nnpfc_patch_height_minus1+1) can be used as the vertical sample count of the block size used for the input to the post-processing filter. The value of nnpfc_patch_height_minus1 should be in the range of 0 to 32766 (inclusive), and nnpfc_patch_height_minus1 should be less than InpPicHeightInLumaSamples.

[0060] JVET-Z2006 does not provide sufficient signaling to indicate support for accepting neural network post filters of any block size. In one example, according to the techniques of this document, signaling is provided to support accepting neural network post filters of any block size. Table 23 shows the syntax of an exemplary neural network post filter characteristics SEI message according to the techniques of this document. Table 23 For Table 23, the semantics may be based on the semantics provided above and based on the following: nnpfc_constant_patch_size_idc equal to 0 specifies that the post-processing filter accepts as input any block size that is a positive integer multiple of the block size indicated by nnpfc_patch_width_minus1 and nnpfc_patch_height_minus1. nnpfc_constant_patch_size_flag equal to 1 specifies that the post-processing filter accepts as input any block size that is a positive integer multiple of the block size indicated by nnpfc_patch_width_minus1 and nnpfc_patch_height_minus1, and nnpfc_constant_patch_size_flag equal to 2 specifies that the post-processing filter accepts any block size. The value 3 is reserved. In one example, according to techniques herein, nnpfc_patch_width_minus1 and nnpfc_patch_height_minus1 may not be signaled when the post-processing filter accepts any block size. Table 24 shows the syntax of an exemplary neural network post-filter characteristics SEI message according to the techniques herein. Table 24 For Table 24, the semantics may be based on the semantics provided above. In one example, the above conditional signaling may be specified as follows: In one example, ue(v) encoding may be used for nnpfc_constant_patch_size_idc.

[0061] In one example, instead of using idc to define a new value specifying that any block size may be used, a separate new flag may be signaled. Table 25 shows the syntax of an exemplary neural network post-filter characteristics SEI message according to the techniques herein. Table 25 For Table 25, the semantics may be based on the semantics provided above as well as the following semantics. nnpfc_any_patch_size_flag equal to 1 specifies that the post-processing filter accepts any block size, nnpfc_any_patch_size_flag equal to 0 specifies that the information about the block size accepted by the post-processing filter is specified via the syntax elements nnpfc_constant_patch_size_flag, nnpfc_patch_width_minus1, nnpfc_patch_height_minus1. nnpfc_constant_patch_size_flag equal to 0 specifies that the post-processing filter accepts as input any block size that is a positive integer multiple of the block size indicated by nnpfc_patch_width_minus1 and nnpfc_patch_height_minus1. nnpfc_constant_patch_size_flag equal to 1 specifies that the post-processing filter accepts as input any block size that is a positive integer multiple of the block size indicated by nnpfc_patch_width_minus1 and nnpfc_patch_height_minus1. JVET-Z2006 does not provide sufficient signaling to indicate the upper limits on the width and height dimensions of the neural network postfilter inputs accepted for any block size. It should be noted that increasing the input size results in an increase in inference time. At a certain input size, the inference time becomes impractical. Tables 26 and 27 show examples showing that increasing the input size results in an increase in inference time for the network architecture presented in JVET-Z0082. Table 26 shows an example of inference time versus input size on an Intel(R) Xeon(R) Silver4214CPU@2.20GHz. Table 27 shows an example of inference time versus input size on an NVIDIA RTX A4000. For Tables 26 and Table 27, using random input. Input Dimensions Inference time in seconds 72x72 0.026 144x144 0.038 288x288 0.37 576x576 1.46 1152x1152 6.6 2304x2304 24.8 4608x4608 Processor is turned off Table 26 Input Dimensions Inference time in seconds 72x72 0.02 144x144 0.02 288x288 0.03 576x576 0.04 1152x1152 Insufficient memory error Table 27 In one example, according to the techniques herein, syntax elements based on the following may be included in a neural network post-filter characteristics SEI message to provide an estimated upper bound on the maximum input width and height dimensions acceptable to the network. nnpfc_max_patch_size: If nnpfc_constant_pntch_size_flag is equal to 0 or 2, this flag is sent to set an estimated upper limit on the maximum input width and height dimensions acceptable to the network. The maximum dimensions are calculated as follows: max_width=nnpfc_max_patch_size*(nnpfc_patch_width_minus1+1) max_height=nnpfc_max_patch_size*(nnpfc_patch_height_minus1+1) JVET-Z2006 does not provide sufficient signaling to indicate the number of frames concatenated in the channel dimension of the neural network postfilter input. Multiple frames can be concatenated in the channel dimension without significantly increasing the computational cost, so it may be important to signal whether the input is formed by one or more frames. Some experiments were performed to demonstrate that an increase in the channel dimension does not lead to an increase in computational complexity. Table 28 shows the results of an experiment using the network architecture provided in JVET-Z0082 and changing the first convolution according to the change in channel dimension, using a random input with fixed width and height dimensions (72x72). Table 28 shows an example of inference time compared to channel size on an Intel(R) Xeon(R) Silver 4214CPU@2.20GHz. Table 28 shows that increasing the channel size does not lead to an increase in inference time. Channel size Inference time in seconds 10 0.03 160 0.029 Table 28 Because the concatenation of multiple frames in the channel dimension does not result in an increase in inference time, the neural network post-filter can be trained on multiple or one frame. This leads to the need to have information in the SEI message indicating whether the network is used with one frame or multiple frames as input. In one example, according to the technology of this article, a syntax element based on the following content can be included in the neural network post-filter characteristics SEI message to indicate that multiple frames are concatenated in the channel dimension. nnpfc_number_of_input_frames_in_channel_dimension: Indicates the number of frames used as input to the post-processing filter. The input frames are concatenated in the channel dimension. nnpfc_number_of_output_frames_in_channel_dimension: Indicates the number of output frames of the post-processing filters. The output frames are concatenated in the channel dimension. It should be noted that the 21st ISO / IEC JTC1 / SC29 / WG11 meeting "AHG9 / AHG11: SET message for carriage of neural network information for postfiltering" held from January 6 to 15, 2021 (teleconference, document JVET-U0091-v3, referred to in this article as JVET-U0091) described the syntax element num_nn_input_ref_pic with the following semantics: num_nn_input_refpic specifies the number of input reference pictures. num_nn_input_ref_pic equal to 0 specifies that the current output picture of the decoder is the only input data of the neural network. num_nn_input_ref_pic greater than 0 specifies that the number of reference pictures used as input data of the neural network is num_nn_input_ref_pic-1. It is alleged that JVET-U0091 does not describe how to concatenate multiple frames. In an atypical case, when multiple frames are concatenated, an extra dimension representing time is added to the input. The time dimension indicates the number of frames, and in the case where such input contains this extra dimension, the use of 3D convolution becomes necessary, which leads to a significant increase in complexity and computational cost. In one example, according to the technology of this article, the frames are concatenated in the channel dimension, which does not lead to an increase in computational cost or an increase in inference time. That is, in one example according to the technology of this article, the input may have the following dimensions: (block size, width, height, channel * time). In one example, two syntax elements may be included in the neural network post-filter characteristics SEI message. These syntax elements may indicate the number of frames used in the input of the post-processing filter and the number of frames in its output. In one example, these syntax elements may be called nnpfc_number_of_input_frames_in_channel_dimension and nnpfc_number_of_output_frames_in_channel_dimension and are based on the semantics defined above. JVET-Z2006 does not provide sufficient signaling to indicate an upper limit on the block size of the neural network post-filter input. It should be noted that increasing the block size results in an increase in inference time. At a certain block size, the inference time becomes impractical. Tables 29 and 30 show examples showing that increasing the block size results in an increase in inference time for the network architecture provided in JVET-Z0082. Table 29 shows an example of inference time versus block size on an Intel(R) Xeon(R) Silver 4214CPU@2.20GHz. Table 30 shows an example of inference time versus block size on an NVIDIA RTX A4000. For Tables 29 and 30, random inputs with fixed width and height dimensions are used. Block size Inference time in seconds 1 0.023 2 0.049 6 1.19 512 8.86 2048 Processor is turned off Table 29 Block size Inference time in seconds 1 0.009 4 0.01 64 0.03 128 0.18 256 Insufficient memory error Table 30 In one example, according to the techniques herein, syntax elements based on the following may be included in a neural network post-filter characteristics SEI message to provide a recommended maximum block size acceptable to the network. nnpfc_max_batch_size: Indicates the recommended maximum block size for the input to the post-processing filters.

[0062] In this way, video encoder 500 represents an example of a device configured to perform the following operations: sending a signal to notify a neural network post-filter characteristics message, and sending a signal to notify a syntax element identifying a padding type corresponding to the neural network post-filter characteristics message, wherein the padding type includes loop padding.

[0063] Reference again Figure 1 , interface 108 may include any device configured to receive data generated by data encapsulator 107 and transmit and / or store the data to a communication medium. Interface 108 may include a network interface card such as an Ethernet card, and may include an optical transceiver, a radio frequency transceiver, or any other type of device that can send and / or receive information. In addition, interface 108 may include a computer system interface that enables files to be stored on a storage device. For example, interface 108 may include support for peripheral component interconnect (PCI) and high-speed peripheral component interconnect (PCIe) bus protocols, proprietary bus protocols, universal serial bus (USB) protocols, I 2 C's chipset, or any other logical and physical structure that can be used to interconnect peer devices.

[0064] Reference again Figure 1, target device 120 includes interface 122, data decapsulator 123, video decoder 124, and display 126. Interface 122 may include any device configured to receive data from a communication medium. Interface 122 may include a network interface card such as an Ethernet card, and may include an optical transceiver, a radio frequency transceiver, or any other type of device that can receive and / or send information. In addition, interface 122 may include a computer system interface that allows a compatible video bitstream to be retrieved from a storage device. For example, interface 122 may include a computer system that supports PCI and PCIe bus protocols, proprietary bus protocols, USB protocols, I 2 The data decapsulator 123 may be configured to receive and parse any of the example grammatical structures described herein.

[0065] Video decoder 124 may include any device configured to receive a bitstream (e.g., sub-bitstream extraction) and / or acceptable variants thereof and reproduce video data therefrom. Display 126 may include any device configured to display video data. Display 126 may include one of a variety of display devices such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display. Display 126 may include a high definition display or an ultra high definition display. It should be noted that although in Figure 1 In the example shown, the video decoder 124 is described as outputting data to a display 126, but the video decoder 124 can be configured to output video data to various types of devices and / or subcomponents thereof. For example, the video decoder 124 can be configured to output video data to any communication medium, as described herein.

[0066] Figure 6 600 is a block diagram illustrating an example of a video decoder that may be configured to decode video data in accordance with one or more techniques of this disclosure (e.g., a decoding process for reference picture list construction described above). In one example, the video decoder 600 may be configured to decode transform data and reconstruct residual data from transform coefficients based on the decoded transform data. The video decoder 600 may be configured to perform intra-frame prediction decoding and inter-frame prediction decoding, and may therefore be referred to as a hybrid decoder. The video decoder 600 may be configured to parse any combination of the syntax elements described above in Tables 1 to 30. The video decoder 600 may decode video based on or in accordance with the above process and also based on the parsed values ​​in Tables 1 to 30.

[0067] exist Figure 6In the example shown, the video decoder 600 includes an entropy decoding unit 602, an inverse quantization unit 604, an inverse transform coefficient processing unit 606, an intra-frame prediction processing unit 608, an inter-frame prediction processing unit 610, a summer 612, a post-filter unit 614, and a reference buffer 616. The video decoder 600 can be configured to decode video data in a manner consistent with a video encoding system. It should be noted that although the exemplary video decoder 600 shown has different functional blocks, such illustrations are intended for descriptive purposes and do not limit the video decoder 600 and / or its subcomponents to a specific hardware or software architecture. The functionality of the video decoder 600 can be implemented using any combination of hardware, firmware, and / or software implementations.

[0068] like Figure 6 As shown, the entropy decoding unit 602 receives an entropy-encoded bitstream. The entropy decoding unit 602 may be configured to decode syntax elements and quantized coefficients from the bitstream according to a process that is inverse to the entropy encoding process. The entropy decoding unit 602 may be configured to perform entropy decoding according to any of the entropy encoding techniques described above. The entropy decoding unit 602 may determine the values ​​of syntax elements in the encoded bitstream in a manner consistent with the video coding standard. Figure 6 As shown, the entropy decoding unit 602 can determine the quantization parameter, quantization coefficient value, transformation data and prediction data from the bit stream. Figure 6 In the example shown, the inverse quantization unit 604 and the inverse transform coefficient processing unit 606 receive the quantized coefficient values ​​from the entropy decoding unit 602 and output reconstructed residual data.

[0069] See again Figure 6, the reconstructed residual data may be provided to the summer 612. The summer 612 may add the reconstructed residual data to the predicted video block and generate the reconstructed video data. The predicted video block may be determined according to a predictive video technique, namely intra prediction and inter prediction. The intra prediction processing unit 608 may be configured to receive an intra prediction syntax element and retrieve the predicted video block from a reference buffer 616. The reference buffer 616 may include a memory device configured to store one or more frames of video data. The intra prediction syntax element may identify an intra prediction mode, such as the intra prediction mode described above. The inter prediction processing unit 610 may receive the inter prediction syntax element and generate a motion vector to identify a predicted block in one or more reference frames stored in the reference buffer 616. The inter prediction processing unit 610 may generate a motion compensated block, possibly performing interpolation based on an interpolation filter. An identifier of an interpolation filter for motion estimation with sub-pixel precision may be included in the syntax element. The inter prediction processing unit 610 may use an interpolation filter to calculate an interpolated value for a sub-integer pixel of a reference block. The post-filter unit 614 may be configured to perform filtering on the reconstructed video data. For example, the post-filter unit 614 may be configured to perform deblocking and / or sample adaptive offset (SAO) filtering, for example based on parameters specified in the bitstream. Furthermore, it should be noted that in some examples, the post-filter unit 614 may be configured to perform dedicated arbitrary filtering (e.g., visual enhancement, such as mosquito noise removal). Figure 6 As shown, the video decoder 600 may output a reconstructed video block. In this manner, the video decoder 600 represents an example of a device configured to receive a neural network post-filter characteristics message, and parse a syntax element identifying a padding type corresponding to the neural network post-filter characteristics message, wherein the padding type includes loop padding.

[0070] In one or more examples, the functions may be implemented by hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or codes on or transmitted via a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media corresponding to tangible media such as data storage media, or propagation media including any medium that facilitates the transmission of a computer program from one place to another, for example, according to a communication protocol. Thus, computer-readable media may generally correspond to: (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. The data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the techniques described in the present disclosure. A computer program product may include a computer-readable medium.

[0071] By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or any other medium that can be used to store the required program code in the form of instructions or data structures and can be accessed by a computer. Moreover, any connection is appropriately referred to as a computer-readable medium. For example, if a coaxial cable, optical fiber cable, twisted pair, digital subscriber line (DSL) or wireless technology such as infrared, radio and microwave are used to transmit instructions from a website, server or other remote source, then the coaxial cable, optical fiber cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave are included in the definition of the medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carriers, signals or other transient media, but are directed to non-transient tangible storage media. As used herein, disks and optical disks include compact disks (CDs), laser disks, optical optical disks, digital versatile disks (DVDs), floppy disks and Blu-ray disks, wherein disks usually copy data magnetically, while optical disks use lasers to copy data optically. The above combination should also be included in the scope of computer-readable media.

[0072] Instructions may be executed by one or more processors such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the term "processor" as used herein may refer to any of the aforementioned structures or any other structure suitable for implementing the techniques described herein. In addition, in some aspects, the functions described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a combined codec. Moreover, these techniques may be implemented entirely in one or more circuits or logic elements.

[0073] The techniques of the present disclosure may be implemented in a variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC), or a set of ICs (e.g., a chipset). Various components, modules, or units are described in this disclosure to emphasize functional aspects of a device configured to perform the disclosed techniques, but do not necessarily need to be implemented by different hardware units. Instead, as described above, the various units may be combined in a codec hardware unit, or provided by a collection of interoperating hardware units including one or more processors as described above, in combination with appropriate software and / or firmware.

[0074] In addition, each functional block or various features of the base station equipment and terminal equipment used in each of the above-mentioned embodiments can be realized or executed by a circuit (usually an integrated circuit or multiple integrated circuits). The circuit designed to perform the functions described in this specification may include a general-purpose processor, a digital signal processor (DSP), a dedicated or general-purpose integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices or discrete hardware components or a combination thereof. The general-purpose processor can be a microprocessor, or alternatively, the processor can be a conventional processor, a controller, a microcontroller or a state machine. The general-purpose processor or each of the above-mentioned circuits can be configured by a digital circuit, or can be configured by an analog circuit. In addition, when the technology of making an integrated circuit that replaces the current integrated circuit appears due to the progress of semiconductor technology, the integrated circuit produced by the technology can also be used.

[0075] Various examples have been described. These and other examples are within the scope of the following claims. <Cross Reference>

[0076] This nonprovisional application claims priority under 35 U.S.C. 119 to provisional application No. 63 / 358,058, filed on July 1, 2022, and application No. 63 / 389,356, filed on July 14, 2022, the entire contents of which are hereby incorporated by reference.

Claims

1. A method for signaling neural network post-loop filter information for video data, the method comprising: Sending a signal to inform the neural network of the post-filter characteristic message; signaling a syntax element specifying a padding type corresponding to the neural network post-filter characteristics message, wherein the syntax element includes a value specifying a wrap-around padding type and a value specifying a fixed padding type; and In case the fixed padding type is specified, a syntax element indicating the sample values ​​to be used for padding is signaled.

2. A method for applying a neural network in-loop filter to reconstruct a video, the method comprising: Receive a neural network post-filter characteristic message; parsing a syntax element identifying a padding type corresponding to the neural network post-filter characteristics message, wherein the syntax element includes a value specifying a wrap-around padding type and a value specifying a fixed padding type; as well as In case the fixed padding type is specified, syntax elements indicating sample values ​​to be used for padding are parsed.

3. A device comprising one or more processors configured to: Receive a neural network post-filter characteristic message; parsing a syntax element identifying a padding type corresponding to the neural network post-filter characteristics message, wherein the syntax element includes a value specifying a wrap-around padding type and a value specifying a fixed padding type; and In case the fixed padding type is specified, syntax elements indicating sample values ​​to be used for padding are parsed.

Images