Systems and methods for performing adaptive resolution changes in video encoding

Abstract

The present disclosure relates to video coding, and more specifically to techniques for adaptive resolution changes. According to one aspect of the present invention, a motion compensated interpolation filter is selected based on a value of a scaling factor, wherein the motion compensated interpolation filter is specified by 16 fractional sample positions and 8 interpolation filter coefficients corresponding to each of the fractional sample positions.

Description

Technical Field

[0001] This disclosure relates to video coding, and more specifically to techniques for adaptive resolution changes. Background Technology

[0002] Digital video functionality can be integrated into a wide variety of devices, including digital televisions, laptops or desktops, tablets, digital recording devices, digital media players, video game consoles, cellular phones (including so-called smartphones), medical imaging equipment, and more. Digital video can be encoded according to video coding standards. Video coding standards define the format for encapsulating encoded video data into compliant bitstreams. A compliant bitstream is a data structure that can be received and decoded by video decoding devices to generate reconstructed video data. Video coding standards can be combined with video compression techniques. Examples of video coding standards include ISO / IEC MPEG-4 Visual and ITU-T H.264 (also known as ISO / IEC MPEG-4 AVC) and High Efficiency Video Coding (HEVC). HEVC is described in the ITU-T H.265 Recommendation of December 2016, which is incorporated herein by reference and referred to herein as ITU-T H.265. Extensions and improvements to ITU-T H.265 are currently under consideration for developing next-generation video coding standards. For example, the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG) (collectively referred to as the Joint Video Study Group (JVET)) are working to standardize video coding technologies with compression capabilities significantly exceeding the current HEVC standard. The Joint Exploratory Model 7 (JEM 7), the algorithm description of Joint Exploratory Test Model 7 (JEM 7), and the ISO / IEC JTC1 / SC29 / WG11 document: JVET-G1001 (July 2017, Turin, Italy), which are incorporated herein by reference, describe the coding features of the JVET under the Joint Test Model Study, a technology that represents a potential enhancement to video coding beyond the capabilities of ITU-T H.265. It should be noted that the coding features of JEM 7 are implemented in the JEM reference software. As used herein, the term JEM can refer collectively to the algorithms included in JEM 7 and the specific implementations in the JEM reference software. In addition, in response to the “Joint Call for Proposals on Video Compression with Capabilities beyond HEVC” jointly issued by VCEG and MPEG, various groups presented multiple descriptions of video coding tools at the 10th meeting of ISO / IEC JTC1 / SC29 / WG11 held in San Diego, CA, from April 16 to 20, 2018.Based on various descriptions of video coding tools, the final initial draft text of the video coding specification was described in "Versatile Video Coding (Draft 1)," also known as document JVET-J1001-v2, presented at the 10th meeting of ISO / IEC JTC1 / SC29 / WG11 held in San Diego, California, April 16-20, 2018. This document is incorporated herein by reference and referred to as JVET-J1001. Current development of the next-generation video coding standard for JVET and MPEG is known as the Universal Video Coding (VVC) project. "Versatile Video Coding (Draft 5)" (document JVET-N1001-v8, which is incorporated herein by reference and referred to as JVET-N1001), presented at the 14th meeting of ISO / IEC JTC1 / SC29 / WG11 held in Geneva, Switzerland, March 19-27, 2019, represents a new version of the draft text of the video coding specification corresponding to the VVC project. The “Versatile Video Coding (Draft 6)” (document JVET-O2001-vE, which is incorporated herein by reference and referred to as JVET-O2001) from the 15th meeting of ISO / IEC JTC1 / SC29 / WG11 held in Gothenburg, Sweden from July 3 to 12, 2019, refers to the current version of the draft text of the video coding specification corresponding to the VVC project.

[0003] Video compression techniques reduce the data requirements for storing and transmitting video data. Video compression reduces data requirements by utilizing the inherent redundancy in video sequences. It can further divide a video sequence into smaller, consecutive segments (i.e., a set of images within a video sequence, images within a set of images, regions within images, sub-regions within regions, etc.). Intra-frame predictive coding techniques (e.g., spatial prediction within images) and inter-frame prediction techniques (i.e., temporal techniques between images) can be used to generate the difference between the unit of video data to be encoded and a reference unit of the video data. This difference can be called residual data. Residual data can be encoded as quantized transform coefficients. Syntax elements can relate to the residual data and the reference coding unit (e.g., intra-frame predictive mode index and motion information). Entropy coding can be applied to the residual data and syntax elements. The entropy-coded residual data and syntax elements can be included in the data structure that forms a compliant bitstream. Summary of the Invention

[0004] In one example, a method for video decoding is provided, which includes: determining a scaling factor corresponding to downsampling between a reference image and a current image; and wherein a motion-compensated interpolation filter is specified by 16 fractional sample locations and 8 interpolation filter coefficients corresponding to each of these fractional sample locations.

[0005] In one example, an apparatus for encoding video data is provided, the apparatus including one or more processors configured to: determine a scaling factor corresponding to downsampling between a reference image and a current image; and select a motion-compensated interpolation filter based on the value of the scaling factor, wherein the motion-compensated interpolation filter is specified by 16 fractional sample locations and 8 interpolation filter coefficients corresponding to each of these fractional sample locations. Attached Figure Description

[0006] [ Figure 1 ] Figure 1 This is a conceptual diagram illustrating an example of a set of pictures encoded according to quadtree / multitree partitioning based on one or more techniques of this disclosure.

[0007] [ Figure 2A ] Figure 2A This is a conceptual diagram illustrating an example of encoding video data blocks according to one or more techniques disclosed herein.

[0008] [ Figure 2B ] Figure 2B This is a conceptual diagram illustrating an example of encoding video data blocks according to one or more techniques disclosed herein.

[0009] [ Figure 3 ] Figure 3 This is a conceptual diagram illustrating examples of video component sampling formats usable according to one or more techniques of this disclosure.

[0010] [ Figure 4 ] Figure 4 This is a conceptual diagram illustrating a data structure for encapsulating and encoding video data and corresponding metadata according to one or more techniques disclosed herein.

[0011] [ Figure 5 ] Figure 5 This is a block diagram illustrating an example of a system that can be configured to encode and decode video data according to one or more techniques of this disclosure.

[0012] [ Figure 6 ] Figure 6 This is a block diagram illustrating an example of a video encoder that can be configured to encode video data according to one or more techniques of this disclosure.

[0013] [ Figure 7 ] Figure 7 This is a block diagram illustrating an example of a video decoder that can be configured to decode video data according to one or more techniques of this disclosure. Detailed Implementation

[0014] Generally speaking, this disclosure describes various techniques for encoding video data. Specifically, this disclosure describes techniques for adaptive resolution changes. It should be noted that although the techniques disclosed herein relate to ITU-T H.264, ITU-T H.265, JEM, JVET-N1001, and JVET-O2001, the techniques disclosed herein are generally applicable to video coding. For example, in addition to those techniques included in ITU-T H.265, JEM, JVET-N1001, and JVET-O2001, the coding techniques described herein can be incorporated into video coding systems (including video coding systems based on future video coding standards), including video block structures, intra-frame prediction techniques, inter-frame prediction techniques, transform techniques, filtering techniques, and / or other entropy coding techniques. Therefore, references to ITU-T H.264, ITU-T H.265, JEM, JVET-N1001, and JVET-O2001 are for descriptive purposes and should not be construed as limiting the scope of the techniques described herein. Furthermore, it should be noted that the inclusion of references in this paper by way of citation is for descriptive purposes and should not be construed as limiting or creating ambiguity regarding the terminology used herein. For example, where a definition of a term is provided in one of the incorporated references that differs from that in another incorporated reference and / or as used herein, the term should be interpreted in a manner that broadly includes each corresponding definition and / or in a manner that includes each particular definition in alternatives.

[0015] In one example, one approach includes determining an adaptive resolution scaling factor and, based on that adaptive resolution scaling factor, determining a motion-compensated interpolation filter.

[0016] In one example, a device includes one or more processors configured to determine an adaptive resolution scaling factor and, based on the adaptive resolution scaling factor, determine a motion-compensated interpolation filter.

[0017] In one example, a non-transitory computer-readable storage medium includes instructions stored thereon that, when executed, cause one or more processors of the device to determine an adaptive resolution scaling factor and, based on the adaptive resolution scaling factor, determine a motion-compensated interpolation filter.

[0018] In one example, an apparatus includes means for determining an adaptive resolution scaling factor and means for determining a motion-compensated interpolation filter based on the adaptive resolution scaling factor.

[0019] Details of one or more examples are set forth in the following figures and description. Other features, objects, and advantages will become apparent from the description, figures, and claims.

[0020] Video content comprises a sequence of frames (or images). A series of frames may also be referred to as a group of pictures (GOP). Each video frame or image may be divided into one or more regions. Regions may be defined based on basic units (e.g., video blocks) and a set of rules defining regions. For example, a rule defining a region may be that a region must be an integer number of video blocks arranged in a rectangle. Furthermore, video blocks within a region may be ordered according to a scanning mode (e.g., raster scan). As used herein, the term "video block" may generally refer to a region of an image, or more specifically, to the largest array of sample values ​​that can be predictably encoded, its sub-partitions, and / or corresponding structures. Additionally, the term "current video block" may refer to the region of an image that is being encoded or decoded. A video block may be defined as an array of sample values. It should be noted that in some cases, pixel values ​​may be described as sample values ​​comprising the corresponding components of the video data, which may also be referred to as color components (e.g., luminance (Y) and chrominance (Cb and Cr) components or red, green, and blue components). It should be noted that in some cases, the terms "pixel value" and "sample value" are used interchangeably. Furthermore, in some cases, a pixel or sample can be referred to as a pel. A video sampling format (also known as a chroma format) can be defined relative to the number of luminance samples included in a video block, thus defining the number of chroma samples included in the video block. For example, in a 4:2:0 sampling format, the luminance component's sampling rate is twice the sampling rate of the chroma components in both the horizontal and vertical directions.

[0021] Video encoders perform predictive coding on video blocks and their sub-partitions. Video blocks and their sub-partitions can be referred to as nodes. ITU-T H.264 specifies macroblocks comprising 16×16 luma samples. That is, in ITU-T H.264, pictures are segmented into macroblocks. ITU-T H.265 specifies a similar Coded Tree Unit (CTU) structure (which may be referred to as a Maximum Coded Unit (LCU)). In ITU-T H.265, pictures are segmented into CTUs. In ITU-T H.265, for a picture, the CTU size can be set to include 16×16, 32×32, or 64×64 luma samples. In ITU-T H.265, a CTU consists of a corresponding Coded Tree Block (CTB) for each component of the video data (e.g., luma (Y) and chrominance (Cb and Cr)). It should be noted that a video with one luma component and two corresponding chrominance components can be described as having two channels, namely, a luma channel and a chrominance channel. Furthermore, in ITU-T H.265, CTUs can be partitioned according to a quadtree (QT) partitioning structure, which allows the CTU's CTB to be divided into coded blocks (CBs). That is, in ITU-T H.265, a CTU can be divided into quadtree leaf nodes. According to ITU-T H.265, a luma CB, along with two corresponding chroma CBs and associated syntax elements, is called a coding unit (CU). In ITU-T H.265, the minimum permissible size of a CB can be signaled. In ITU-T H.265, the minimum permissible size of a luma CB is 8×8 luma samples. In ITU-T H.265, the decision to code a picture region using intra-frame prediction or inter-frame prediction is made at the CU level.

[0022] In ITU-T H.265, a CU (Cubic Component Unit) is associated with a Prediction Unit (PU) structure that has its root at the CU. In ITU-T H.265, the PU structure allows the segmentation of the Luminance CB (Cubic Block) and Chromaticity CB to generate corresponding reference samples. That is, in ITU-T H.265, the Luminance CB and Chromaticity CB can be segmented into corresponding Luminance Prediction Blocks (PBs) and Chromaticity Prediction Blocks (PBs), where each PB comprises a block of sample values ​​to which the same prediction is applied. In ITU-T H.265, a CB can be divided into one, two, or four PBs. ITU-T H.265 supports PB sizes from 64×64 samples down to 4×4 samples. In ITU-T H.265, square PBs are supported for intra-frame prediction, where a CB can form a PB or can be segmented into four square PBs. In addition to square PBs, ITU-T H.265 also supports rectangular PBs for inter-frame prediction, where a CB can be halved vertically or horizontally to form a PB. Furthermore, it should be noted that in ITU-T H.265, for inter-frame prediction, four asymmetric PB partitions are supported, where the CB is divided into two PBs at one-quarter of the height (top or bottom) or width (left or right) of the CB. Intra-frame prediction data (e.g., intra-frame prediction mode syntax elements) or inter-frame prediction data (e.g., motion data syntax elements) corresponding to the PB are used to generate reference and / or prediction sample values ​​for the PB.

[0023] JEM specifies a CTU with a maximum size of 256×256 luminance samples. JEM specifies a Quadtree Plus Binary Tree (QTBT) block structure. In JEM, the QTBT structure allows the quadtree leaf nodes to be further partitioned by a binary tree (BT) structure. That is, in JEM, the binary tree structure allows the quadtree leaf nodes to be recursively partitioned vertically or horizontally. In JVET-N1001 and JVET-O2001, the CTU is partitioned according to a Quadtree Plus Multi-Type Tree (QTMT or QT+MTT) structure. The QTMT in JVET-N1001 and JVET-O2001 is similar to the QTBT in JEM. However, in JVET-N1001 and JVET-O2001, in addition to indicating binary partitioning, the multi-type tree can also indicate so-called ternary (or ternary tree (TT)) partitioning. Ternary partitioning divides a block vertically or horizontally into three blocks. In the case of a vertical TT division, the block is divided at one-quarter of its width from the left edge and at one-quarter of its width from the right edge; and in the case of a horizontal TT division, the block is divided at one-quarter of its height from the top edge and at one-quarter of its height from the bottom edge. See again. Figure 1 , Figure 1 This illustrates an example where a CTU is partitioned into quadtree leaf nodes, and these quadtree leaf nodes are further partitioned based on either BT or TT partitioning. That is, in Figure 1 In the diagram, dashed lines indicate additional binary and ternary partitions in a quadtree.

[0024] As described above, each video frame or picture can be divided into one or more regions. For example, according to ITU-T H.265, each video frame or picture can be divided into one or more slices, and further divided into one or more tiles, wherein each slice includes a sequence of CTUs (e.g., arranged in raster scan order), and wherein a tile is a sequence of CTUs corresponding to a rectangular area of ​​the picture. It should be noted that, in ITU-T H.265, a slice is a sequence of one or more slice segments that begin with an independent slice segment and include all subsequent subordinate slice segments (if any) preceding the next independent slice segment (if any). A slice segment (such as a piece) is a sequence of CTUs. Therefore, in some cases, the terms "slice" and "slice segment" are used interchangeably to refer to a sequence of CTUs arranged in raster scan order. Furthermore, it should be noted that, in ITU-T H.265, a tile may consist of CTUs contained in more than one slice, and a slice may consist of CTUs contained in more than one tile. However, ITU-T H.265 specifies that one or both of the following conditions must be met: (1) all CTUs in a slice belong to the same tile; and (2) all CTUs in a tile belong to the same slice.

[0025] Regarding JVET-N1001 and JVET-O2001, slices need to consist of an integer number of tiles, not just an integer number of CTUs. In JVET-N1001 and JVET-O2001, a tile is a rectangular row region of CTUs within a specific tile in an image. Furthermore, in JVET-N1001 and JVET-O2001, a tile can be divided into multiple tiles, each tile consisting of one or more rows of CTUs within the tile. Tiles not divided into multiple tiles are also referred to as tiles. However, tiles that are a proper subset of a tile are not referred to as tiles. Therefore, some video coding techniques may or may not support slices comprising a set of CTUs that do not form an image. Additionally, it should be noted that in some cases, slices may need to consist of an integer number of complete tiles, and in such cases, the slice is referred to as a tile group. The techniques described herein are applicable to tiles, slices, tiles, and / or tile groups. Figure 1 This is a concept diagram illustrating an example of a group of images including slices. Figure 1 In the example shown, Pic3 is depicted as comprising two slices (i.e., slice 0 and slice 1). Figure 1In the example shown, slice 0 includes one brick, namely brick 0, and slice 1 includes two bricks, namely brick 1 and brick 2. It should be noted that in some cases, slice 0 and slice 1 may meet the requirements of a tile and / or a tile group and be classified as a tile and / or a tile group.

[0026] For intra-frame predictive coding, the intra-frame prediction mode can specify the location of a reference sample within the image. In ITU-T H.265, the defined possible intra-frame prediction modes include planar (i.e., surface-fitting) prediction modes, DC (i.e., flat global average) prediction modes, and 33 angular prediction modes (predMode: 2-34). In JEM, the defined possible intra-frame prediction modes include planar prediction modes, DC prediction modes, and 65 angular prediction modes. It should be noted that planar prediction modes and DC prediction modes can be referred to as non-directional prediction modes, and angular prediction modes can be referred to as directional prediction modes. It should be noted that the techniques described herein are generally applicable regardless of the number of defined possible prediction modes.

[0027] For inter-frame predictive coding, a reference picture is determined, and motion vectors (MVs) identify samples in that reference picture used to generate predictions for the current video block. For example, reference sample values ​​located in one or more previously encoded pictures can be used to predict the current video block, and motion vectors are used to indicate the position of the reference block relative to the current video block. Motion vectors can describe, for example, the horizontal displacement component of the motion vector (i.e., MV). x ), the vertical displacement component of the motion vector (i.e., MV) yThe resolution of the motion vectors (e.g., quarter-pixel precision, half-pixel precision, one-pixel precision, two-pixel precision, four-pixel precision) is used. Previously decoded images (which may include images output before or after the current image) can be organized into one or more lists of reference images and identified using reference image index values. Furthermore, in inter-frame predictive coding, single prediction refers to generating a prediction using sample values ​​from a single reference image, while dual prediction refers to generating a prediction using corresponding sample values ​​from two reference images. That is, in single prediction, a single reference image and its corresponding motion vector are used to generate a prediction for the current video block, while in dual prediction, a first reference image and its corresponding first motion vector, and a second reference image and its corresponding second motion vector are used to generate a prediction for the current video block. In dual prediction, the corresponding sample values ​​are combined (e.g., added, rounded, and cropped, or averaged according to weights) to generate a prediction. Images and their regions can be classified based on which types of prediction patterns are available for encoding their video blocks. In other words, for regions of type B (e.g., B slices), dual prediction, single prediction, and intra-prediction modes can be used; for regions of type P (e.g., P slices), single prediction and intra-prediction modes can be used; and for regions of type I (e.g., I slices), only intra-prediction mode can be used. As described above, reference images are identified by reference indices. For example, for P slices, a single reference image list RefPicList0 can exist, and for B slices, in addition to RefPicList0, a second independent reference image list RefPicList1 can exist. It should be noted that for single prediction in B slices, either RefPicList0 or RefPicList1 can be used to generate the prediction. Furthermore, it should be noted that during the decoding process, at the start of decoding an image, a reference image list is generated from previously decoded images stored in the Decoding Image Buffer (DPB).

[0028] Furthermore, the coding standard supports various motion vector prediction modes. Motion vector prediction enables the derivation of motion vector values ​​for the current video block based on another motion vector. For example, a set of candidate blocks with associated motion information can be derived from the spatially and temporally adjacent blocks of the current video block. Additionally, the generated (or default) motion information can be used for motion vector prediction. Examples of motion vector prediction include Advanced Motion Vector Prediction (AMVP), Temporal Motion Vector Prediction (TMVP), the so-called "merge" mode, and "skip" and "direct" motion inference. Other examples of motion vector prediction include Advanced Temporal Motion Vector Prediction (ATMVP) and Spatial-Temporal Motion Vector Prediction (STMVP). For motion vector prediction, both the video encoder and video decoder perform the same process to derive a set of candidates. Therefore, for the current video block, the same set of candidates is generated during encoding and decoding.

[0029] As mentioned above, for inter-frame predictive coding, reference samples from previously encoded images are used to encode video blocks in the current image. The previously encoded image that can be used as a reference when encoding the current image is called the reference image. It should be noted that the decoding order does not necessarily correspond to the image output order, i.e., the temporal order of images in the video sequence. In ITU-T H.265, when an image is decoded, it is stored in a decoded image buffer (DPB) (which may be called a frame buffer, reference buffer, reference image buffer, etc.). In ITU-T H.265, images stored in the DPB are removed from the DPB when output and are no longer needed for encoding subsequent images. In ITU-T H.265, after decoding the slice header, i.e., at the start of image decoding, a determination is made once for each image whether it should be removed from the DPB. For example, the reference... Figure 1 Pic3 is shown with reference to Pic2. Similarly, Pic4 is shown with reference to Pic1. Regarding Figure 1Assuming the number of images corresponds to the decoding order, the DPB will be populated as follows: After decoding Pic1, the DPB will include {Pic1}; at the start of decoding Pic2, the DPB will include {Pic1}; after decoding Pic2, the DPB will include {Pic1, Pic2}; at the start of decoding Pic3, the DPB will include {Pic1, Pic2}. Then, Pic3 will be decoded with reference to Pic2, and after decoding Pic3, the DPB will include {Pic1, Pic2, Pic3}. At the start of decoding Pic4, images Pic2 and Pic3 will be marked for removal from the DPB because they are not required for decoding Pic4 (or any subsequent images, not shown), and assuming Pic2 and Pic3 have already been output, the DPB will be updated to include {Pic1}. Pic4 will then be decoded with reference to Pic1. The process of marking images to remove them from the DPB can be called Reference Picture Set (RPS) management.

[0030] As described above, intra-frame prediction data or inter-frame prediction data is used to generate reference sample values ​​for blocks of sample values. The difference between sample values ​​included in the current PB or another type of picture region structure and the associated reference samples (e.g., those generated using prediction) can be referred to as residual data. Residual data can include a corresponding array of differences corresponding to each component of the video data. The residual data may be in the pixel domain. Transformations such as Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), integer transform, wavelet transform, or conceptually similar transforms can be applied to the difference array to generate transform coefficients. It should be noted that in ITU-T H.265, JVET-N1001, and JVET-O2001, the CU is associated with a Transform Unit (TU) structure having its root at the CU level. That is, to generate transform coefficients, the array of differences can be partitioned (e.g., four 8×8 transforms can be applied to a 16×16 residual array). Such a subdivision of the differences for each component of the video data can be referred to as a Transform Block (TB). It should be noted that in some cases, a core transform and a subsequent quadratic transform can be applied (in a video encoder) to generate transform coefficients. For a video decoder, the order of the transforms is reversed.

[0031] Quantization can be performed directly on transform coefficients or residual sample values ​​(e.g., in the case of palette-encoded quantization). Quantization approximates transform coefficients by limiting the amplitude to a specified set of values. Quantization essentially scales the transform coefficients to change the amount of data needed to represent a set of transform coefficients. Quantization may include dividing the transform coefficient (or the value obtained by adding an offset value to the transform coefficient) by a quantization scaling factor and any associated rounding function (e.g., rounding to the nearest integer). The quantized transform coefficients may be referred to as coefficient bit values. Inverse quantization (or “dequantization”) may include multiplying the coefficient bit value by the quantization scaling factor, and any reciprocal rounding or offset addition operations. It should be noted that, as used herein, the term quantization process may refer in some cases to division by a scaling factor to generate a bit value, and in some cases to multiplication by a scaling factor to recover the transform coefficient. That is, quantization process may refer to quantization in some cases and inverse quantization in others. Furthermore, it should be noted that although some examples below describe quantization processes for arithmetic operations related to decimal notation, such descriptions are for illustrative purposes and should not be construed as limiting. For example, the techniques described herein can be implemented in devices using binary arithmetic, etc. For example, the multiplication and division operations described herein can be implemented using bit shifting operations, etc.

[0032] Entropy coding techniques can be used to entropy-encode quantized transform coefficients and syntax elements (e.g., syntax elements indicating the coding structure of video blocks). The entropy coding process involves encoding the syntax element values ​​using a lossless data compression algorithm. Examples of entropy coding techniques include Content Adaptive Variable Length Coding (CAVLC), Context Adaptive Binary Arithmetic Coding (CABAC), Probability Interval Partition Entropy Coding (PIPE), etc. The entropy-encoded quantized transform coefficients and the corresponding entropy-encoded syntax elements can form a compliant bitstream that can be used to reproduce video data at the video decoder. The entropy coding process, such as CABAC, may include binarizing the syntax elements. Binarization is the process of converting the values ​​of syntax elements into a sequence of one or more bits. These bits may be referred to as "bins". Binarization may include one or a combination of the following coding techniques: fixed-length coding, unary coding, truncated unary coding, truncated Rice coding, Golomb coding, k-order exponential Golomb coding, and Golomb-Rice coding. For example, binarization may include representing the integer value 5 of a syntax element as 00000101 using an 8-bit fixed-length binarization technique, or representing the integer value 5 as 11110 using a unary coding binarization technique. As used herein, each of the terms fixed-length coding, unary coding, truncated unary coding, truncated Rice coding, Golomb coding, k-order exponential Golomb coding, and Golomb-Rice coding may refer to a general implementation of these techniques and / or a more specific implementation of these coding techniques. For example, a Golomb-Rice coding implementation may be specifically defined according to a video coding standard. In the CABAC example, for a particular bin, the context provides the bin's maximum probability state (MPS) value (i.e., the bin's MPS is either 0 or 1), and the probability value that the bin is in either the MPS or minimum probability state (LPS). For example, the context may indicate that the bin's MPS is 0 and the probability that the bin is 1 is 0.3. It should be noted that the context may be determined based on the values ​​of bins in the previous encodings of the current syntax element and bins in the previously encoded syntax elements. For example, the value of a syntax element associated with an adjacent video block can be used to determine the context of the current bin.

[0033] Figures 2A to 2B This is a conceptual diagram illustrating an example of encoding video data blocks. (Example:) Figure 2A As shown, bit-order values ​​are generated by subtracting a set of predicted values ​​from the current video data block to produce a residual, performing a transformation on the residual, and quantizing the transform coefficients. This process encodes the current block of video data (e.g., the CB corresponding to a video component). Figure 2B As shown, the current video data block is decoded by performing inverse quantization on the bit-order values, performing an inverse transform, and adding a set of predicted values ​​to the resulting residual. It should be noted that, in Figures 2A to 2BIn the example, the sample values ​​of the reconstructed block differ from the sample values ​​of the current video block being encoded. Thus, the encoding can be considered lossy. However, the difference in sample values ​​can be considered acceptable or imperceptible to the viewer of the reconstructed video.

[0034] In addition, such as Figures 2A to 2B As shown, a scaling factor array is used to generate coefficient bit values. In ITU-T H.265, the scaling factor array is generated by selecting a scaling matrix and multiplying each entry in the scaling matrix by a quantization scaling factor. In ITU-T H.265, the scaling matrix is ​​selected in part based on the prediction mode and color components, where scaling matrices of the following sizes are defined: 4×4, 8×8, 16×16, and 32×32. It should be noted that in some examples, the scaling matrix may provide the same value for each entry (i.e., scaling all coefficients by a single value). In ITU-T H.265, the value of the quantization scaling factor can be determined by the quantization parameter QP. In ITU-T H.265, for an 8-bit bit depth, QP can take 52 values ​​from 0 to 51, and a change of 1 in QP typically corresponds to a change of approximately 12% in the value of the quantization scaling factor. Furthermore, in ITU-T H.265, a set of transform coefficients' QP values ​​can be derived using predicted quantization parameter values ​​(which may be referred to as predicted QP values ​​or QP prediction values) and optionally signaled quantization parameter increment values ​​(which may be referred to as QP increment values ​​or incremental QP values). In ITU-T H.265, quantization parameters can be updated for each CU, and corresponding quantization parameters can be derived for each of the luminance and chrominance channels. See again. Figure 2A The quantized transform coefficients are encoded into a bitstream. The quantized transform coefficients and syntax elements (e.g., syntax elements indicating the coding structure of video blocks) can be entropy-coded using entropy coding techniques.

[0035] The video sampling format (also known as the chroma format) can be defined relative to the number of luminance samples included in the CU. For example, for a 4:2:0 sampling format, the sampling rate of the luminance component is twice that of the sampling rates of the chroma components in both the horizontal and vertical directions. Therefore, for a CU formatted according to the 4:2:0 format, the width and height of the sample arrays used for the luminance components are twice the width and height of each sample array used for the chroma components. Figure 3 This is a conceptual diagram illustrating an example of a coding unit formatted according to the 4:2:0 sample format. Figure 3 This shows the relative positions of the chromaticity samples with respect to the luminance samples within the CU. As mentioned above, the CU is typically defined based on the number of horizontal and vertical luminance samples. Therefore, as... Figure 3As shown, a 16×16 CU formatted according to the 4:2:0 sample format includes 16×16 samples for the luma component and 8×8 samples for each chroma component. Furthermore, in the example shown in Figure 2, the relative positions of the chroma samples relative to the luma samples of adjacent video blocks in the 16×16 CU are illustrated. For a CU formatted according to the 4:2:2 format, the width of the luma component sample array is twice the width of the sample array for each chroma component, but the height of the luma component sample array is equal to the height of the sample array for each chroma component. Furthermore, for a CU formatted according to the 4:4:4 format, the luma component sample array has the same width and height as the sample array for each chroma component.

[0036] The following arithmetic operators can be used for the formulas used in this article:

[0037] Addition

[0038] - Subtraction

[0039] Multiplication, including matrix multiplication

[0040] x y Exponentiation. Specifies x as a power of y. In other contexts, this symbol is used as a superscript rather than intended to be interpreted as exponentiation.

[0041] / is an integer division operation that truncates the result towards zero. For example, 7 / 4 and -7 / -4 are truncated to 1, and -7 / 4 and 7 / -4 are truncated to -1.

[0042] ÷ is used to represent division in mathematical formulas when it is not intended to truncate or round.

[0043] x / y is used to represent division in mathematical formulas when not intended for truncation or rounding.

[0044] x%y modulus. The remainder when x is divided by y, defined only for integers x and y where x ≥ 0 and y > 0.

[0045] In addition, the following logical operators can be used:

[0046] x&&y The Boolean logic "and" of x and y

[0047] The Boolean OR operation between x and y.

[0048] ! Boolean logic "No"

[0049] x? y:z If x is TRUE or not equal to 0, evaluate to y; otherwise, evaluate to z.

[0050] In addition, the following relational operators can be used:

[0051] > Greater than

[0052] ≥ greater than or equal to

[0053] < less than

[0054] <= less than or equal to

[0055] == equal to

[0056] != not equal to

[0057] In addition, the following bitwise operators can be used:

[0058] & Bitwise "AND". When operating on integer variables, the two's complement representation of the integer value is used. When operating on binary variables that contain fewer bits than another variable, the shorter variable is extended by adding more significant bits equal to 0.

[0059] | Bitwise "OR". When operating on integer variables, the two's complement representation of the integer value is used. When operating on binary variables that contain fewer bits than another variable, the shorter variable is extended by adding more significant bits equal to 0.

[0060] ^ Bitwise "XOR". When operating on integer variables, the two's complement representation of the integer value is used. When operating on binary variables that contain fewer bits than another variable, the shorter variable is extended by adding more significant bits equal to 0.

[0061] x>>y The two's complement integer representation of x is arithmetically shifted right by y binary bits. This function is only defined for non-negative integer values of y. Bits shifted into the most significant bit (MSB) due to the right shift have the value of the MSB of x before the shift operation.

[0062] x<<y The two's complement integer representation of x is arithmetically shifted left by y binary bits. This function is only defined for non-negative integer values of y. Bits shifted into the least significant bit (LSB) due to the left shift have the value equal to 0.

[0063] In addition, the following assignment operators can be used:

[0064] = Assignment operator

[0065] ++ Increment, i.e., x++ is equivalent to x = x + 1; when used in array indexing, the value of the variable is evaluated before the increment operation.

[0066] – Decrement, i.e., x–- is equivalent to x = x - 1; when used in array indexing, the value of the variable is evaluated before the decrement operation.

[0067] += Increments by a specified amount, that is, x+=3 is equivalent to x=x+3, and x+=(-3) is equivalent to x=x+(-3).

[0068] -= Decrease by a specified amount, that is, x-=3 is equivalent to x=x–3, and x-=(-3) is equivalent to x=x–(-3).

[0069] In addition, the following mathematical functions can be used:

[0070]

[0071] Floor(x), the largest integer less than or equal to x.

[0072] Log2(x) is the base-2 logarithm of x.

[0073]

[0074]

[0075] Furthermore, it should be noted that the following descriptors can be applied among the grammatical descriptors used in this paper:

[0076] -b(8): Bytes (8 bits) with any bit string pattern. The parsing process for this descriptor is specified by the return value of the function read_bit(8).

[0077] -f(n): A fixed-pattern bit string written using n bits (from left to right), starting from the leftmost bit. The parsing process for this descriptor is specified by the return value of the function read_bit(n).

[0078] -se(v): Syntax element for signed integer zero-order Exp-Golomb encoding, starting from the leftmost bit.

[0079] -tb(v): Uses a truncated binary code with up to maxVal bits, where maxVal is defined in the semantics of the syntax element.

[0080] -tu(v): Uses a truncated unary code with up to maxVal bits, where maxVal is defined in the semantics of the syntax element.

[0081] -u(n): Uses an n-bit unsigned integer. When n is "v" in the syntax table, the number of bits varies depending on the values ​​of other syntax elements. The parsing process for this descriptor is specified by the return value of the function read_bits(n), which is interpreted as the binary representation of the unsigned integer, with the most significant bit written first.

[0082] -ue(v): Syntax element of unsigned integer 0-order Exp-Golomb encoding, starting from the leftmost bit.

[0083] As mentioned above, for inter-frame predictive coding, motion vectors include a resolution specifying the sample precision. Specifically, for fractional sample resolution, a corresponding interpolation procedure is used to generate predictions. JVET-N1001 provides the following fractional sample interpolation procedure:

[0084] The input to this process is:

[0085] - Luminance position (xSb, ySb), which specifies the top-left sample of the current coded sub-block relative to the top-left luminance sample of the current image.

[0086] - The variable sbWidth specifies the width of the current encoded sub-block.

[0087] - The variable sbHeight specifies the height of the current encoded sub-block.

[0088] - Motion vector offset mvOffset

[0089] - Refined motion vector refMvLX

[0090] - The selected reference image sample array refPicLX,

[0091] -Bidirectional optical flow flag bdofFlag

[0092] - The variable cIdx specifies the color component index of the current block.

[0093] The output of this process is:

[0094] -Predict sample values ​​in array predSamplesLX ((sbWidth+bdofOffset)×(sbHeight+bdofOffset)).

[0095] The bidirectional optical flow boundary offset bdofOffset is exported as follows:

[0096] bdofOffset = bdofFlag? 2:0

[0097] - If cIdx equals 0, then the following applies:

[0098] Let (xIntL, yIntL) be the brightness position given in full sample units, and (xFracL, yFracL) be the offset given in 1 / 16 sample units. These variables are used only in this clause to specify the fractional sample positions within the reference sample array refPicLX.

[0099] -For each luminance sample location (x) within the predicted luminance sample array predSamplesLXL =0..sbWidth–1+bdofOffset,y L =0..sbHeight–1+bdofOffset), corresponding to the predicted brightness sample value predSamplesLX[x L ][y L Export as follows:

[0100] -variable xInt L yInt L xFrac L and yFrac L Export as follows:

[0101] xInt L =xSb+(refMvLX[0]>>4)+x L

[0102] yInt L =ySb+(refMvLX[1]>>4)+y L

[0103] xFrac L =refMvLXf 0]&15

[0104] yFrac L =refMvLX[1]&15

[0105] - If bdofFlag equals TRUE and one or more of the following conditions are true, then by calling (xInt) L ,yInt L (xFrac) L ,yFrac L The process of obtaining integer luminance samples, using ) and refPicLX as input, derives the predicted luminance sample values ​​predSamplesLX[x]. L ][y L ]:

[0106] -x L It equals 0.

[0107] -x L It equals sbWidth+1.

[0108] -y L It equals 0.

[0109] -y L It equals sbHeight + 1.

[0110] -Otherwise, the following applies:

[0111] - The motion vector mvLX is set to be equal to (refMvLX-mvOffset).

[0112] - For dir = 0..1, the list padVal[dir] is exported as follows:

[0113] - The variable disp is exported as follows:

[0114] disp=(refMvLX[dir]>>4)–(mvLX[dir]>>4)+(dir==0?x L :y L )

[0115] - If disp is less than 0, then padVal[dir] is set to equal to disp.

[0116] Otherwise, if disp is greater than (dir == 0 ? sbWidth: sbHeight) - 1, then padVal[dir] is set to equal to disp - ((dir == 0 ? sbWidth: sbHeight) - 1).

[0117] Otherwise, padVal[dir] is set to 0.

[0118] - The predicted luminance sample values ​​predSamplesLXf[x] are derived by calling the following luminance sample 8-tap interpolation filtering process with (xIntL,yIntL), (xFracL,yFracL), refPicLX, sbWidth, sbHeight, (xSb,ySb), and padVal as inputs. L ][y L ].

[0119] - Otherwise (cIdx is not equal to 0), the following applies:

[0120] Let (xIntC, yIntC) be the chromaticity position given in full sample units, and (xFracC, yFracC) be the offset given in 1 / 32 sample units. These variables are used only in this clause to specify the general fractional sample positions within the reference sample array refPicLX.

[0121] - For each chromaticity sample position (xC = 0..sbWidth-1, yC = 0..sbHeight-1) within the predicted chromaticity sample array predSamplesLX, the corresponding predicted chromaticity sample value predSamplesLXf[xC][yC] is derived as follows:

[0122] The variables xIntc, yIntc, xFracc, and yFracc are derived as follows:

[0123] xIntc=(xSb / SubWidthC)+(mvLX[0]>>5)+xc

[0124] yIntc=(ySb / SubHeightC)+(mvLX[1]>>5)+yc

[0125] xFracc=mvLX[0]&31

[0126] yFracc=mvLX[1]&31

[0127] - The motion vector mvLX is set to be equal to (refMvLX-mvOffset).

[0128] - For dir = 0..1, the list padVal[dir] is exported as follows:

[0129] - The variable disp is exported as follows:

[0130] disp=(refMvLX[dir]>>4)–(mvLX[dir]>>4)+(dir==0?xc:yc)

[0131] - If disp is less than 0, then padVal[dir] is set to equal to disp.

[0132] Otherwise, if disp is greater than (dir == 0 ? sbWidth / SubWidthC:sbHeight / SubWidthC) - 1, then padVal[dir] is set to equal to disp - ((dir == 0 ? sbWidth / SubWidthC:sbHeight / SubWidthC) - 1).

[0133] Otherwise, padVal[dir] is set to 0.

[0134] - The predicted sample values ​​predSamplesLXf[xC][yC] are derived by calling the following chromaticity sample interpolation procedure with (xIntC,yIntC), (xFracC,yFracC), refPicLX, and padVal as inputs.

[0135] Luminance sample interpolation filtering process :

[0136] The input to this process is:

[0137] -Luminance position in full sample units (xInt) L,yInt L ),

[0138] -Luminance position in fractional sample units (xFrac) L ,yFrac L ),

[0139] -Luminance reference sample array refPicLX L ,

[0140] - The variable sbWidth specifies the width of the current child block.

[0141] - The variable sbHeight specifies the height of the current child block.

[0142] - Brightness position (xSb, ySb), which specifies the top-left sample of the current sub-block relative to the top-left brightness sample of the current image.

[0143] - A list padVal[dir], where dir = 0 or 1, specifies the padding direction and amount of the reference sample. The output of this process is the predicted luminance sample value predSampleLXL.

[0144] The variables shift1, shift2, and shift3 are exported as follows:

[0145] - The variable shift1 is set to equal Min(4, BitDepth) Y -8), variable shift2 is set to 6, and variable shift3 is set to Max(2, 14-BitDepth). Y ).

[0146] - The variable picW is set to equal pic_width_in_luma_samples and the variable picH is set to equal pic_height_in_luma_samples.

[0147] For equal to xFrac L or yFrac L For each 1 / 16 fractional sample location p, the brightness interpolation filter coefficient f L [p] is specified in Table 1A.

[0148] Luminance interpolation filter coefficient fPad L [0] is set to equal f L [xFrac L ], and fPadL[1] is set to equal f L [yFrac L And based on padVal[dir], modify it as follows, where dir equals 0.

[0149] S1:

[0150] - If padVal[dir] equals -2, then fPad L [dir] should be modified as follows:

[0151] fPad L [dir][2]=fPad L [dir][2]+fPad L [dir][1]+fPad L [dir][0]

[0152] fPad L [dir][0] = 0

[0153] fPad L [dir][1] = 0

[0154] Otherwise, if padVal[dir] equals -1, then fPad[dir] is modified as follows:

[0155] fPad L [dir][1]=fPad L [dir][1]+fPad L [dir][0]

[0156] iPad L [dir][0] = 0

[0157] Otherwise, if padVal[dir] equals 1, then fPad L [dir] should be modified as follows:

[0158] fPad L [dir][6]=fPad L [dir][6]+fPad L [dir][7]

[0159] fPad L [dir][7]=0

[0160] Otherwise, if padVal[dir] equals 2, then fPad L [dir] should be modified as follows:

[0161] fPad L [dir][5]=fPad L [dir][5]+fPad L [dir][6]+fPad L [dir][7]

[0162] fPad L [dir][6]=0

[0163] fPad L [dir][7]=0

[0164] When MotionModelIdc[xSb][ySb] is greater than 0 and sbWidth equals 4 and sbHeight

[0165] When it equals 4, fPad L The [dir] value is modified as follows, where dir equals 0 and 1:

[0166] fPad L [dir][1]=fPad L [dir][0]+fPad L [dir][1]

[0167] fPad L [dir][0] = 0

[0168] fPad L [dir][6]=fPad L [dir][6]+fPad L [dir][7]

[0169] fPad L [dir][7]=0

[0170] Luminance position (xInt) in full sample units i ,yInt i The following is derived, where i = 0..7:

[0171] xInt i =Clip3(0,picW–1,sps_ref_wraparound_enabled_flag?ClipH((sps_ref_wraparound_offset_minus1+1)*MinCbSizeY,picW,xIntL+i–3):xIntL+i–3)

[0172] yInt i =Clip3(0,picH–1,yIntL+i–3)

[0173] Predicted brightness sample value predSampleLX L Export as follows:

[0174] -If xFrac L and yFracL If both are equal to 0, then predSampleLX L The values ​​are derived as follows:

[0175] prcdSampleLX,=refPicLX L [xInt3][yInt3]< <shift3

[0176] -Otherwise, if xFrac L Not equal to 0 and yFrac L If it equals 0, then predSampleLX L The values ​​are derived as follows:

[0177]

[0178] -Otherwise, if xFrac L Equal to 0 and yFrac L If it is not equal to 0, then predSampleLX L The values ​​are derived as follows:

[0179]

[0180] -Otherwise, if xFrac L Not equal to 0 and yFrac L If it is not equal to 0, then predSampleLX L The values ​​are derived as follows:

[0181] The sample array temp[n] (where n = 0..7) is derived as follows:

[0182]

[0183] -Predicted brightness sample value predSampleLX L Export as follows:

[0184]

[0185]

[0186]

[0187] Table 1A

[0188] Luminance sample interpolation filtering process :

[0189] The input to this process is:

[0190] -Luminance position in full sample units (xInt) L ,yInt L ),

[0191] -Luminance reference sample array refPicLX L ,

[0192] The output of this process is the predicted luminance sample value, predSampleLX. L

[0193] The variable shift is set to equal Max(2, 14-BitDepth). Y ).

[0194] The variable picW is set to equal pic_width_in_luma_samples and the variable picH is set to equal pic_height_in_luma_samples.

[0195] The brightness position (xInt, yInt) calculated in units of the entire sample is derived as follows:

[0196] xInt=Clip3(0,picW–1,sps_ref_wraparound_enabled_flag?

[0197] ClipH((sps_ref_wraparound_offset_minus1+1)*MinCbSizeY,picW,xInt L ):

[0198] xInt L )

[0199] yInt = Clip3(0, picH – 1, yInt) L )

[0200] Predicted brightness sample value predSampleLX L Export as follows:

[0201] predSampleLX L =refPicLX L [xInt][yInt]< <shift3

[0202] Colorimetric sample interpolation process :

[0203] The input to this process is:

[0204] - Chromaticity position in full sample units (xInt) C ,yInt C ),

[0205] - Chromaticity position in 1 / 32 fractional sample units (xFrac)C ,yFrac C ),

[0206] -Colorimetric reference sample array refPicLX C .

[0207] The output of this process is the predicted chromaticity sample value, predSampleLX. C

[0208] The variables shift1, shift2, and shift3 are exported as follows:

[0209] - Variable shift1 is set to equal Min(4, BitDepthc-8), variable shift2 is set to equal 6, and variable shift3 is set to equal Max(2, 14-BitDepthc-8). C ).

[0210] -Variable picW C It is set to equal pic_width_in_luma_samples / SubWidthC and the variable picH C It is set to equal to pic_height_in_luma_samples / SubHeightC.

[0211] For equal to xFracc or yFrac C For each 1 / 32 fractional sample location p, the chroma interpolation filter coefficient f C [p] is specified in Table 2A.

[0212] The chromaticity interpolation filter coefficients fPadc[0] are set to be equal to f C [xFrac L ], and fPad C [1] was set to equal f C [yFrac L And based on padVal[dir], modify it as follows, where dir equals 0 and 1:

[0213] - If padVal[dir] equals -1, then fPad C [dir] should be modified as follows:

[0214] fPad C [dir][1]=fPad C [dir][1]+fPad C [dir][0]

[0215] fPadc[dir][0]=0

[0216] Otherwise, if padVal[dir] equals 1, then fPad C [dir] should be modified as follows:

[0217] fPad C [dir][2]=fPad C [dir][2]+fPad C [dir][3]

[0218] fPad C [dir][3]=0

[0219] The variable xOffset is set to equal to

[0220] (sps ref wraparound offset minus1+1)*MinCbSizeY) / SubWidthC.

[0221] Chromaticity position (xInt) in full sample units i ,yInt i The following is derived, where i = 0..3:

[0222] xInt i =Clip3(0,picW–1,

[0223] sps_ref_wraparound_enabled_flag? ClipH(xOffset,picW C ,xInt C +i–1):xInt C +i–1)

[0224] yInt i =Clip3(0,picH C –1,yInt C +i–1).

[0225] Predicted chromaticity sample value predSampleLX C Export as follows:

[0226] -If xFrac C and yFrac C If both are equal to 0, then predSampleLX C The values ​​are derived as follows:

[0227] predSampleLX C =refPicLX C [xInt1][yInt1]< <shift3

[0228] -Otherwise, if xFrac C Not equal to 0 and yFrac C If it equals 0, then predSampleLX C The values ​​are derived as follows:

[0229]

[0230] -Otherwise, if xFrac C Equal to 0 and yFrac C If it is not equal to 0, then predSampleLX C The values ​​are derived as follows:

[0231]

[0232] -Otherwise, if xFrac C Not equal to 0 and yFrac C If it is not equal to 0, then predSampleLX C The values ​​are derived as follows:

[0233] The sample array temp[n] (where n = 0..3) is derived as follows:

[0234]

[0235] -Predicted chromaticity sample values ​​predSampleLX C Export as follows:

[0236] predSampleLX C =(fPadC1[1][0]*temp[0]+

[0237] fPad C [1][1]*temp[1]+

[0238] fPad C [1][2]*temp[2]+

[0239] fPad C [1][3]*temp[3])>>shift2

[0240]

[0241]

[0242] Table 2A

[0243] In addition, JVET-O2001 provides the following fractional sample interpolation procedure:

[0244] The input to this process is:

[0245] - Luminance position (xSb, ySb), which specifies the top-left sample of the current coded sub-block relative to the top-left luminance sample of the current image.

[0246] - The variable sbWidth specifies the width of the current encoded sub-block.

[0247] - The variable sbHeight specifies the height of the current encoded sub-block.

[0248] - Motion vector offset mvOffset

[0249] - Refined motion vector refMvLX

[0250] - The selected reference image sample array refPicLX,

[0251] - Half-sample interpolation filter index hpellfIdx,

[0252] -Bidirectional optical flow flag bdofFlag

[0253] - The variable cIdx specifies the color component index of the current block.

[0254] The output of this process is:

[0255] -Predicted sample values ​​(sbWidth+brdExtSize)×(sbHeight+brdExtSize)predSamplesLX.

[0256] The predicted block boundary extension size brdExtSize is derived as follows:

[0257] brdExtSize=(bdofFlag||(inter_affine_flag[xSb][ySb]&&

[0258] sps_affine_prof_enabled_flag))? 2:0

[0259] The variable fRefWidth is set to be equal to the PicOutputWidthL of the reference image in the brightness sample.

[0260] The variable fRefHeight is set to equal to the PicOutputHeightL of the reference image in the brightness sample. The motion vector mvLX is set to equal to (refMvLX - mvOffset).

[0261] - If cIdx equals 0, then the following applies:

[0262] - The scaling factor and its fixed-point representation are defined as

[0263] hori_scale_fp=((fRefWidth<<14)+(PicOutputWidthL>>1)) / PicOutputWidthL

[0264] vert_scale_fp=((fRefHeight<<14)+(PicOutputHeightL>>1)) / PicOutputHeightL

[0265] Let (xIntL, yIntL) be the brightness position given in full sample units, and (xFracL, yFracL) be the offset given in 1 / 16 sample units. These variables are used only in this clause to specify the fractional sample positions within the reference sample array refPicLX.

[0266] - Top-left coordinates of the boundary block used for reference sample filling

[0267] (xSbInt L ,ySbInt L ) is set to equal to

[0268] (xSb+(mvLX[0]>>4),ySb+(mvLX[1]>>4)).

[0269] -For each luminance sample location (x) within the predicted luminance sample array predSamplesLX L =O..sbWidth-1+brdExtSize,y L =O..sbHeight-1+brdExtSize), corresponding to the predicted brightness sample value predSamplesLXf[x L ][y L Export as follows:

[0270] -Let (refxSb,refySb) L ) and (refx L ,refy L ) represents the brightness position pointed to by the motion vector (refMvLX[0], refMvLX[1]) given in 1 / 16 sample units. The variable refxSb L refx L ,refySb L and refy L Export as follows:

[0271] refxSb L=((xSb<<4)+refMvLX[0])*hori_scale_fp

[0272] refx L =((Sign(refxSb)*((Abs(refxSb)+128)>>8)

[0273] +x L *((hori_scale_fp+8)>>4))+32)>>6

[0274] refySb L =((ySb<<4)+refMvLX[1])*vert scale fp

[0275] refyL=((Sign(refySb)*((Abs(refySb)+128)>>8)+yL*((vert_scale_fp+8)>>4))+32)>>6

[0276] -variable xInt L yInt L xFrac L and yFrac L Export as follows:

[0277] xInt L =refxt>>4

[0278] yInt L =refy L >>4

[0279] xFrac L =refx L &15

[0280] y Frac I.=refy L &15

[0281] - If bdofFlag equals TRUE (or sps_affine_prof_enabled_flag equals TRUE, and inter_affine_flag[xSb][ySb] equals TRUE) and one or more of the following conditions are true, then by calling (xInt) L +(xFrac L >>3)-1), yInt L +(yFrac L >>3)-1) and refPicLX are used as inputs to specify the luminance integer sample acquisition process to derive the predicted luminance sample value predSamplesLX[xL ][y L ].

[0282] -x L It equals 0.

[0283] -x L It equals sbWidth+1.

[0284] -y L It equals 0.

[0285] -y L It equals sbHeight + 1.

[0286] - Otherwise, by calling the following methods with (xIntL-(brdExtSize>0?1:0), yIntL-(brdExtSize>0?1:0)), (xFracL, yFracL), (xSbInt) as follows: L ,ySbInt L The 8-tap interpolation filtering process, which takes refPicLX, hpellfIdx, sbWidth, sbHeight and (xSb, ySb) as inputs, is used to derive the predicted luminance sample values ​​predSamplesLX[xL][yL].

[0287] - Otherwise (cIdx is not equal to 0), the following applies:

[0288] Let (xIntC, yIntC) be the chromaticity position given in full sample units, and (xFracC, yFracC) be the offset given in 1 / 32 sample units. These variables are used only in this clause to specify the general fractional sample positions within the reference sample array refPicLX.

[0289] - The top-left coordinates (xSbIntC, ySbIntC) of the boundary block used for filling the reference sample are set to equal ((xSb / SubWidthC)+(mvLX[0]>>5),(ySb / SubHeightC)+(mvLX[1]>>5)).

[0290] - For each chromaticity sample position (xC = 0..sbWidth-1, yC = 0..sbHeight-1) within the predicted chromaticity sample array predSamplesLX, the corresponding predicted chromaticity sample value predSamplesLXf[xC][yC] is derived as follows:

[0291] -Let (refxSbc,refySb) C ) and (refx C ,refy CThe position of brightness is indicated by the motion vector (mvLX[0], mvLX[1]) given in 1 / 32 sample units. The variables refxSbc, refySbc, and refx are... C and refy C Export as follows:

[0292] refxSbc=((xSb / SubWidthC<<5)+mvLX[0])*hori_scale_fp

[0293] refx C =((Sign(refxSbc)*((Abs(refxSbc)+256)>>9)+xC*((hori scale_fp+8)>>4))+16)>>5

[0294] refySb C =((ySb / SubHeightC<<5)+mvLX[1])*vert_scale_fp

[0295] refyc=((Sign(refySbc)*((Abs(refySbc)+256)>>9)+yC*((vert_scale_fp+8)>>4))+16)<<5

[0296] -variable xInt C yInt C xFrac C and yFrac C Export as follows:

[0297] xIntc = refx C >>5

[0298] yIntC = refyc >> 5

[0299] xFracC=refyC&31

[0300] yFracC=refyC&31

[0301] The predicted sample values ​​predSamplesLXf[xC][yC] are derived by calling the following procedure, which is specified with (xIntC,yIntC), (xFracC,yFracC), (xSbIntC,ySbIntC), sbWidth, sbHeight, and refPicLX as inputs.

[0302] Luminance sample interpolation filtering process

[0303] The input to this process is:

[0304] -Luminance position in full sample units (xInt) L ,yInt L ),

[0305] -Luminance position in fractional sample units (xFrac) L ,yFrac L ),

[0306] -Luminance position based on full sample cells (xSbInt) L ,ySbInt L This specifies the top-left sample of the boundary block used to fill the reference sample relative to the top-left brightness sample of the reference image.

[0307] -Luminance reference sample array refPicLX L ,

[0308] - Half-sample interpolation filter index hpellfIdx,

[0309] - The variable sbWidth specifies the width of the current child block.

[0310] - The variable sbHeight specifies the height of the current child block.

[0311] - Brightness position (xSb, ySb), which specifies the top-left sample of the current sub-block relative to the top-left brightness sample of the current image.

[0312] The output of this process is the predicted luminance sample value, predSampleLX. L

[0313] The variables shift1, shift2, and shift3 are exported as follows:

[0314] - Variable shift1 is set to equal Min(4, BitDepthy-8), variable shift2 is set to equal 6, and variable shift3 is set to equal Max(2, 14-BitDepthy).

[0315] - The variable picW is set to equal pic_width_in_luma_samples and the variable picH is set to equal pic_height_in_luma_samples.

[0316] For equal to xFrac L or yFrac L For each 1 / 16 fractional sample location p, the brightness interpolation filter coefficient f L [p]Export as follows:

[0317] - If MotionModelIdc[xSb][ySb] is greater than 0 and sbWidth and sbHeight are both equal to 4, then the luminance interpolation filter coefficient f L [p] is specified in Table 1C.

[0318] - Otherwise, the luminance interpolation filter coefficients f L [p] As specified in Table 1B according to hpellfIdx.

[0319] Luminance position (xInt) in full sample units i ,yInt i The following is derived, where i = 0..7:

[0320] - If subpic_treated_as_pic_flag[SubPicIdx] equals 1, then the following applies:

[0321] xInt i =Clip3(SubPicLeftBoundaryPos,SubPicRightBoundaryPos,xInt L +i–3)

[0322] yInt i =Clip3(SubPicTopBoundaryPos,SubPicBotBoundaryPos,yInt L +i–3)

[0323] - Otherwise (subpic_treated_as_pic_flag[SubPicIdx] equals 0), the following applies:

[0324] xInt i =Clip3(0,picW–1,sps_ref_wraparound_enabled_flag?

[0325] ClipH((sps_ref_wraparound_offset_minus1+1)*MinCbSizeY,picW,xInt L +i–3):xInt L +i–3)

[0326] yInt i =Clip3(0,picH–1,yInt) L +i–3)

[0327] The brightness position, calculated per full sample unit, is further modified as follows, where i = 0..7:

[0328] xInt i =Clip3(xSbInt) L –3,xSbInt L +sbWidth+4,xInt i )

[0329] yInt i =Clip3(ySbInt) L –3,ySbInt L +sbHeight+4,yInt i )

[0330] Predicted brightness sample value predSampleLX L Export as follows:

[0331] -If xFrac L and yFrac L If both are equal to 0, then predSampleLX L The values ​​are derived as follows:

[0332] predSampleLX L =refPicLX L [xInt3][yInt3]< <shift3

[0333] -Otherwise, if xFrac L Not equal to 0 and yFrac L If it equals 0, then predSampleLX L The values ​​are derived as follows:

[0334]

[0335] -Otherwise, if xFrac L Equal to 0 and yFrac L If it is not equal to 0, then predSampleLX L The values ​​are derived as follows:

[0336]

[0337] -Otherwise, if xFrac L Not equal to 0 and yFrac L If it is not equal to 0, then predSampleLX L The values ​​are derived as follows:

[0338] The sample array temp[n] (where n = 0..7) is derived as follows:

[0339]

[0340] -Predicted brightness sample value predSampleLX L Export as follows:

[0341]

[0342]

[0343] Table 1B

[0344]

[0345] Table 1C

[0346] Brightness integer sample acquisition process

[0347] The input to this process is:

[0348] -Luminance position in full sample units (xInt) L ,yInt L ),

[0349] -Luminance reference sample array refPicLX L ,

[0350] The output of this process is the predicted luminance sample value, predSampleLX. L

[0351] The variable shift is set to equal Max(2, 14-BitDepth). Y ).

[0352] The variable picW is set to equal pic_width_in_luma_samples and the variable picH is set to equal pic_height_in_luma_samples.

[0353] The brightness position (xInt, yInt) calculated in units of the entire sample is derived as follows:

[0354] xInt=Clip3(0,picW–1,sps_ref_wraparound_enabled_flag?

[0355] ClipH((sps_ref_wraparound_offset_minus1+1)*MinCbSizeY,picW,xInt L ):xInt L )

[0356] yInt = Clip3(0, picH – 1, yInt)L )

[0357] Predicted brightness sample value predSampleLX L Export as follows:

[0358] predSampleLX L =relPicLX,[xInt][yInt]< <shifts

[0359] Colorimetric sample interpolation process

[0360] The input to this process is:

[0361] - Chromaticity position in full sample units (xInt) C ,yInt C ),

[0362] - Chromaticity position in 1 / 32 fractional sample units (xFrac) C ,yFrac C ),

[0363] - Chromaticity position based on full sample units (xSbInt) C ,ySbInt C This specifies the top-left sample of the boundary block used to fill the reference sample relative to the top-left chroma sample of the reference image.

[0364] - The variable sbWidth specifies the width of the current child block.

[0365] - The variable sbHeight specifies the height of the current child block.

[0366] -Colorimetric reference sample array refPicLX C .

[0367] The output of this process is the predicted chromaticity sample value, predSampleLX. C

[0368] The variables shift1, shift2, and shift3 are exported as follows:

[0369] - The variable shift1 is set to equal Min(4, BitDepth) C -8), variable shift2 is set to 6, and variable shift3 is set to Max(2, 14-BitDepth). C ).

[0370] -Variable picW CIt is set to equal pic_width_in_luma_samples / SubWidthC and the variable picH C It is set to equal to pic_height_in_luma_samples / SubHeightC.

[0371] For equal to xFrac C or yFrac C For each 1 / 32 fractional sample location p, the chroma interpolation filter coefficient f C [p]

[0372] Specify in Table 2B.

[0373] The variable xOffset is set to equal to

[0374] (sps_ref_wraparound_offset_minus1+1)*MinCbSizeY) / SubWidthC.

[0375] The chromaticity positions (xInti, yInti) in full sample units are derived as follows, where i = 0.3:

[0376] - If subpic_treated_as_pic_flag[SubPicIdx] equals 1, then the following applies:

[0377] xInt i =Clip3(SubPicLeftBoundaryPos / SubWidthC,SubPicRightBoundaryPos / SubWidthC,xInt L +i)yInt i =Clip3(SubPicTopBoundaryPos / SubHeightC,SubPicBotBoundaryPos / SubHeightC,yInt L +i)

[0378] - Otherwise (subpic_treated_as_pic_flag[SubPicIdx] equals 0), then the following applies: xInt i =Clip3(0,picW C –1,

[0379] sps_ref_wraparound_enabled_flag? ClipH(xOffset,picW C ,xInt C +i–1):xIntC +i–1)

[0380] yInti = Clip3(0, picH C –1,yInt C +i–1)

[0381] Chromaticity position (xInt) in full sample units i ,yInt i Further modifications are as follows, where i = 0..3:

[0382] xInt i =Clip3(xSbIntC–1,xSbIntC+sbWidth+2,xInt i )

[0383] yInt i =Clip3(ySbIntC–1,ySbIntC+sbHeight+2,yInt i )

[0384] Predicted chromaticity sample value predSampleLX C Export as follows:

[0385] -If xFrac C and yFrac C If both are equal to 0, then predSampleLX C The values ​​are derived as follows:

[0386] predSampleLX C =refPicLX C [xInt i ][yInt i ]< <shift3

[0387] -Otherwise, if xFrac C Not equal to 0 and yFrac C If the value is 0, then the value of predSampleLXc is derived as follows:

[0388] predSampleLXc=(X;=o fc[xFrac C ][i]*refPicLXc[xInt i [yInti])>>shiftl

[0389] - Otherwise, if xFracc equals 0 and yFrac C If it is not equal to 0, the value of predSampleLXc is derived as follows:

[0390] predSampleLXc=(SF=ofc[yFrac C ][i]*refPicLXc[x! nti][yInti])>>shift1

[0391] -Otherwise, if xFrac C Not equal to 0 and yFrac C If it is not equal to 0, the value of predSampleLXc is derived as follows:

[0392] The sample array temp[n] (where n = 0..3) is derived as follows:

[0393] temp[n] = (^? ＝o fc[xFrac C ][i]*refPicLXc[xInti][yInt n ])>>shift1

[0394] - The predicted chromaticity sample value predSampleLXc is exported as follows:

[0395] predSampleLXc=(f C [yFrac C ][0]*temp[0]+

[0396] f C [yFrac C [1]*temp[1]+

[0397] f C [yFracc][2]*temp[2]+

[0398] f C [yFracc][3]*temp[3])>>shift2

[0399]

[0400]

[0401] Table 2B

[0402] It should be noted that, according to the fractional sample interpolation process provided in JVET-O2001, there are two inter-frame prediction modes that use a 6-tap filter for interpolation filtering. That is, Table 1C is the 6-tap filter for affine motion (i.e., MotionModelIdc[xSb][ySb] is greater than 0, and sbWidth and sbHeight are both equal to 4, corresponding to affine motion), and Table 1B derives the 6-tap filter for semi-pellet adaptive motion vector resolution (AMVR) (i.e., hpelIfIdx == 1 corresponds to semi-pellet AVMR, i.e., p = 0, 8).

[0403] It should be noted that although the fractional sample interpolation process differs between JVET-N1001 and JVET-O2001 (e.g., based on the inclusion of a 6-tap filter in JVET-O2001), the two processes are similar because the luminance interpolation filter coefficients f are specified. L [p] and chromaticity interpolation filter coefficients f C [p]. As detailed below, according to the techniques described herein, different sets of interpolation filter coefficients can be used conditionally (e.g., according to a scaling factor). Therefore, the techniques described herein are generally applicable to fractional sample interpolation processes. That is, for example, the brightness interpolation filter coefficients f L [p] and / or chroma interpolation filter coefficients f C [p] This can be conditionally determined (e.g., according to a scaling factor) in a manner independent of other aspects of the fractional sample interpolation process. Generally, according to the techniques described herein, typical use case interpolation filter coefficients that may cause severe aliasing artifacts when scaling occurs can be replaced with filters having low-pass characteristics to reduce aliasing artifacts. Such filters with low-pass characteristics can be, for example, Lanczos filters generated using a Lanczos window selected to provide the desired low-pass filtering effect. Therefore, in some cases, the filters described in Tables 1A to 2B above can be referred to as typical use case filters.

[0404] As further described above, video content comprises a video sequence consisting of a series of frames (or images), and each video frame or image can be divided into one or more regions. The encoded video sequence (CVS) can be encapsulated (or structured) into a series of access units, where each access unit includes video data configured as a Network Abstraction Layer (NAL) unit. A bitstream can be described as a sequence of NAL units comprising one or more CVSs. It should be noted that multi-layer extensions enable video rendering to include a base layer and one or more additional enhancement layers. For example, the base layer may enable video rendering with a basic quality level (e.g., high-definition rendering and / or a 30Hz frame rate), and the enhancement layer may enable video rendering with an enhanced quality level (e.g., ultra-high-definition rendering and / or a 60Hz frame rate). Enhancement layers can be encoded by referencing the base layer. That is, for example, images in the enhancement layer can be encoded by referencing one or more images (including scaled versions of them) in the base layer (e.g., using inter-layer prediction techniques). Each NAL unit may include an identifier indicating the video data layer associated with the NAL unit. It should be noted that sub-bitstream extraction can refer to the process by which a device receiving a compatible or conforming bitstream forms a new compatible or conforming bitstream by discarding and / or modifying data in the received bitstream. For example, sub-bitstream extraction can be used to form a new compatible or conforming bitstream corresponding to a specific video representation (e.g., a high-quality representation). Layers can also be encoded independently of each other. In this case, inter-layer prediction may not exist between two layers.

[0405] refer to Figure 1In the example shown, each video data slice included in Pic3 (i.e., slice 0, slice 1) is shown as encapsulated in a NAL unit. In JVET-N1001 and JVET-O2001, each of the video sequence, GOP, picture, slice, and CTU can be associated with metadata describing video coding attributes. JVET-N1001 and JVET-O2001 define parameter sets that can be used to describe video data and / or video coding attributes. Specifically, JVET-N1001 and JVET-O2001 include five types of parameter sets: Decoding Parameter Set (DPS), Video Parameter Set (VPS), Sequence Parameter Set (SPS), Picture Parameter Set (PPS), and Adaptive Parameter Set (APS). In JVET-N1001 and JVET-O2001, parameter sets can be encapsulated as special types of NAL units or can be signaled as messages. NAL units that include encoded video data (e.g., slices) can be referred to as VCL (Video Coding Layer) NAL units, and NAL units that include metadata (e.g., parameter sets) can be referred to as non-VCL NAL units. Furthermore, JVET-N1001 and JVET-O2001 enable the signaling of Supplemental Enhancement Information (SEI) messages. In JVET-N1001 and JVET-O2001, SEI messages assist in processes related to decoding, display, or other purposes; however, SEI messages may not be necessary to construct luma or chroma samples through the decoding process. In JVET-N1001 and JVET-O2001, SEI messages can be signaled in the bitstream using non-VCL NAL units. Moreover, SEI messages can be transmitted in some way, rather than existing in the bitstream (i.e., signaled out of band). Figure 4 An example of a bitstream comprising multiple CVSs is shown, where each CVS is represented by a NAL unit included in its respective access unit. Figure 4 In the example shown, the non-VCL NAL unit includes the corresponding parameter set NAL unit (i.e., sequence parameter set (SPS) and image parameter set (PPS) NAL unit), SEI message NAL unit, and access unit delimiter NAL unit. It should be noted that in Figure 4 In this context, HEADER is the header for NAL units.

[0406] As described above, a video block can be defined as an array of sample values ​​with a video sampling format. The total number of samples, specified as width multiplied by height, including the image, can be referred to as the image resolution (e.g., 1920×1080 luminance samples). In JVET-N1001 and JVET-O2001, the resolution and video sampling format of each image included in the encoded video sequence are specified in the corresponding Sequence Parameter Set (SPS). Table 3 shows the Sequence Parameter Set syntax provided in JVET-N1001.

[0407]

[0408]

[0409]

[0410]

[0411]

[0412] Table 3

[0413] Regarding Table 3, JVET-N1001 provides the following semantics:

[0414] When sps_decoding_parameter_set_id is greater than 0, it specifies the value of dps_decoding_parameter_set_id of the DPS referenced by the SPS. When sps_decoding_parameter_set_id is equal to 0, the SPS does not reference the DPS, and no DPS is active when decoding is performed on each CVS that references the SPS.

[0415] When sps_video_parameter_set_id is greater than 0, it specifies the value of vps_video_parameter_set_id of the VPS referenced by the SPS. When sps_video_parameter_set_id is equal to 0, the SPS does not reference the VPS, and no VPS is active when decoding is performed on each CVS that references the SPS.

[0416] Increasing 1 to sps_max_sub_layers_minus1 specifies the maximum number of temporal sublayers that may exist in each CVS referencing SPS. The value of sps_max_sub_layers_minus1 should be in the range of 0 to 6 (inclusive).

[0417] In this version of the bitstream conforming to this specification, sps_reserved_zero_5bits should be equal to 0. Other values ​​for sps_reserved_zero_5bits are reserved for future use by ITU-T|ISO / IEC.

[0418] `gra_enabled_flag` equal to 1 indicates that GRA images may exist in CVS that references SPS. `gra_enabled_flag` equal to 0 indicates that GRA images do not exist in CVS that references SPS.

[0419] sps_seq_parameter_set_id provides an identifier for SPS for reference by other syntax elements. The value of sps_seq_parameter_set_id should be in the range of 0 to 15 (inclusive).

[0420] As specified, chroma_format_idc specifies the chroma sampling relative to the luminance sampling. The value of chroma_format_idc should be in the range of 0 to 3 (inclusive).

[0421] A `separate_colour_plane_flag` value of 1 indicates that the three color components of the 4:4:4 chroma format are encoded separately. A `separate_colour_plane_flag` value of 0 indicates that the color components are not encoded separately. If `separate_colour_plane_flag` does not exist, it is inferred to be 0. When `separate_colour_plane_flag` is 1, the encoded image consists of three separate components, each composed of encoded samples of a color plane (Y, Cb, or Cr) using a monochrome encoding syntax. In this case, each color plane is associated with a specific `colour_plane_id` value.

[0422] Note—There is no correlation in the decoding process between color planes with different color_plane_id values. For example, the decoding process of a monochrome image with one color_plane_id value does not use any data from monochrome images with different color plane_id values ​​for inter-frame prediction.

[0423] Based on the value of separate_colour_plane_flag, the value of the variable ChromaArrayType is specified as follows:

[0424] - If separate_colour_plane_flag equals 0, then ChromaArrayType is set to equal chroma_format_idc.

[0425] Otherwise (if separate_colour_plane_flag equals 1), ChromaArrayType is set to 0.

[0426] `pic_width_in_luma_samples` specifies the width of each decoded image, in luminance samples. `pic_width_in_luma_samples` should not be equal to 0 and should be an integer multiple of `MinCbSizeY`.

[0427] `pic_height_in_luma_samples` specifies the height of each decoded image, in luminance samples. `pic_height_in_luma_samples` should not be equal to 0 and should be an integer multiple of `MinCbSizeY`.

[0428] A conformance_window_flag value of 1 indicates that the clipping window offset parameter is immediately following the conformance_window_flag value in SPS. A conformance_window_flag value of 0 indicates that the clipping window offset parameter is not present.

[0429] `conf_win_left_offset`, `conf_win_right_offset`, `conf_win_top_offset`, and `conf_win_bottom_offset` specify the samples of the image output from the decoding process in CVS, based on the rectangular region specified in the coordinates of the image to be output. When `conformance_window_flag` equals 0, it is inferred that the values ​​of `conf_win_left_offset`, `conf_win_right_offset`, `conf_win_top_offset`, and `conf_win_bottom_offset` are equal to 0.

[0430] The fit-to-cropping window contains brightness samples with horizontal image coordinates from SubWidthC*conf_win_left_offset to pic_width_in_luma_samples - (SubWidthC*conf_win_right_offset+1) and vertical image coordinates from SubHeightC*conf_win_top_offset to pic_height_in_luma_samples - (SubHeightC*conf_win_bottom_offset+1) (inclusive of end values).

[0431] The value of SubWidthC*(conf_win_left_offset+conf_win_right_offset) should be less than pic_width__in_luma_samples, and the value of SubHeightC*(conf_win_top_offset+conf_win_bottom_offset) should be less than pic_height_in_luma_samples.

[0432] When ChromaArrayType is not equal to 0, the corresponding specified sample of the two chroma matrices is a sample with image coordinates (x / SubWidthC, y / SubHeightC), where (x, y) are the image coordinates of the specified luminance sample.

[0433] Note—The fit-to-crop-window offset parameter is only applied during output. All internal decoding processes are applied to the uncropped image size.

[0434] bit_depth_luma_minus8 specifies the luminance array BitDepth as follows. Y The values ​​of the bit depth and brightness quantization parameter range offset QpBdOffsety for the sample:

[0435] BitDepth Y =8 + bitdepthluma_minus8

[0436] QpBdOffset Y = 6 * bit_depth_luma_minus8

[0437] bit_depth_luma_minus8 should be in the range of 0 to 8 (inclusive).

[0438] bit_depth_chroma_minus8 specifies the chroma array BitDepth as follows.C The bit depth and colorimetric parameter range offset QpBdOffset of the sample C Value:

[0439] BitDepth C =8 + bit_depth_chroma_minus8

[0440] QpBdOffset C = 6 * bit_depth_chroma_minus8

[0441] bit_depth_chroma_minus8 should be in the range of 0 to 8 (inclusive).

[0442] The following specifies the value of the variable MaxPicOrderCntLsb used in the decoding process of image order counting:

[0443] MaxPicOrderCntLsb=2 (log2_max_pic_order_cat_lab_minus4+4)

[0444] The value of log2_max_pic_order_cnt_lsb_minus4 should be in the range of 0 to 12 (inclusive).

[0445] A value of 1 for `sps_sub_layer_ordering_info_present_flag` indicates that `sps_max_dec_pic_buffering_minus1[i]`, `sps_max_num_reorder_pics[i]`, and `sps_max_latency_increase_plus1[i]` exist for `sps_max_sub_layers_minus1+1` sublayers. A value of 0 for `sps_sub_layer_ordering_info_present_flag` indicates that the values ​​of `sps_max_dec_pic_buffering_minus1[sps_max_sub_layers_minus1]`, `sps_max_num_reorder_pics[sps_max_sub_layers_minus1]`, and `sps_max_latency_increase_plus1[sps_max_sub_layers_minus1]` apply to all sublayers.

[0446] The increment of 1 in `sps_max_dec_pic_buffering_minus1[i]` specifies the maximum required size of the CVS decoding picture buffer when `HighestTid` equals `i`, in units of picture storage buffers. The value of `sps_max_dec_pic_buffering_minus1[i]` should be in the range of 0 to `MaxDpbSize-1` (inclusive), where `MaxDpbSize` is specified elsewhere. When `i` is greater than 0, `sps_max_dec_pic_buffering_minus1[i]` should be greater than or equal to `sps_max_dec_pic_buffering_minus1i-1`. When there is no sps_max_dec_jric_buffering_minus1[i] in the range of 0 to sps_max_sub_layers_minus1-1 (inclusive), it is inferred that sps_max_dec_jric_buffering_minus1[sps_max_sub_layers_minus1] is equal to sps_max_dec_pic_buffering_minus1[sps_max_sub_layers_minus1] since sps_sub_layer_ordering_info_present_flag is equal to 0.

[0447] `sps_max_num_reorder_pics[i]` indicates the maximum number of pictures allowed in the decoding order that precede any picture in the CVS and in the output order when `HighestTid` equals `i`. The value of `sps_max_num_reorder_pics[i]` should be in the range of 0 to `sps_max_dec_pic_buffering_minus1[i]` (inclusive). When `i` is greater than 0, `sps_max_num_reorder_pics[i]` should be greater than or equal to `sps_max_num_reorder_pics[i-1]`. When `i` is in the range of 0 to `sps_max_sub_layers_minus1-1` (inclusive), and `sps_max_num_reorder_pics[i]` does not exist, it is inferred to be equal to `sps_max_num_reorder_pics[sps_max_sub_layers_minus1]` since `sps_sub_layer_ordering_info_present_flag` equals 0.

[0448] sps_max_latency_increase_plus1[i] is not equal to 0 and is used to calculate the value of SpsMaxLatencyPictures[i]. This value specifies the maximum number of pictures that can be before any picture in the CVS in the output order and after that picture in the decoding order when HighestTid equals i.

[0449] When sps_max_latency_increase_plus1[i] is not equal to 0, the value of SpsMaxLatencyPictures[i] is specified as follows:

[0450] SpsMaxLatencyPictures[i] =

[0451] sps_max_num_reorder_pics[i]+sps_max_latency_increase_plus1[i]-1

[0452] When sps_max_latency_increase_plus1[i] equals 0, it does not represent the corresponding limit.

[0453] The value of sps_max_latency_increase_plus1[i] should be between 0 and 2. 32 The range is -2 (inclusive).

[0454] When there is no sps_max_latency_increase_plus1[i] for i in the range of 0 to sps_max_sub_layers_minus1-1 (inclusive), it is inferred that sps_sub_layer_ordering_info_present_flag is equal to sps_max_latency_increase_plus1[sps_max_sub_layers_minus1].

[0455] A long_term_ref_pics_flag value of 0 indicates that no LTRP is used for inter-frame prediction of any coded pictures in CVS. A long_term_ref_pics_flag value of 1 indicates that LTRP can be used for inter-frame prediction of one or more coded pictures in CVS.

[0456] `sps_idr_rpl_present_flag` equal to 1 indicates that the slice header of the IDR image contains a reference image list syntax element. `sps_idr_rpl_present_flag` equal to 0 indicates that the slice header of the IDR image does not contain a reference image list syntax element.

[0457] The value of rpl1_same_as_rpl0_flag equal to 1 indicates that the syntax structures num_ref_pic_lists_in_sps[1] and ref_pic_list_struct(1,rplsIdx) do not exist, and the following applies:

[0458] - It is inferred that the value of num_ref_pic_lists_in_sps[1] is equal to the value of num_ref_pic_lists_in_sps[0].

[0459] - Infer that the value of each syntax element in ref_pic_list_struct(1,rplsIdx) is equal to the value of the corresponding syntax element in ref_pic_list_struct(0,rplsIdx) for rplsIdx in the range 0 to num_ref_pic_lists_in_sps[0]-1.

[0460] num_ref_pic_lists_in_sps[i] represents the number of ref_pic_list_struct(listIdx,rplsIdx) syntax structures included in the SPS, where listIdx is equal to i. The value of num_ref_pic_lists_in_sps[i] should be in the range of 0 to 64 (inclusive).

[0461] Note—For each value of listIdx (equal to 0 or 1), the decoder should allocate memory for a total of num_ref_pic_list_struct(listIdx,rplsIdx) syntax structures, because there may be a ref_pic_list_struct(listIdx,rplsIdx) syntax structure in the slice header of the current image that directly signals this.

[0462] The value of qtbtt_dual_tree_intra_flag equal to 1 indicates that, for I slices, each CTU is divided into coding units with 64×64 luma samples using implicit quadtree partitioning, and these coding units are the roots of two separate coding_tree syntax structures for luma and chroma.

[0463] The increment of 2 in log2_ctu_size_minus2 specifies the size of the luminance coding tree block for each CTU.

[0464] log2_min_luma_coding_block_size_minus2 plus 2 specifies the minimum luma coding block size.

[0465] Variables CtbLog2SizeY, CtbSizeY, MinCbLog2SizeY, MinCbSizeY, MinTbLog2SizeY,

[0466] MaxTbLog2SizeY, MinTbSizeY, MaxTbSizeY, PicWidthlnCtbsY, PicHeightlnCtbsY, PicSizelnCtbsY, PicWidthlnMinCb sY, PicHeightlnMinCbsY, PicSizelnMinCbsY, PicSizelnSamplesY, PicWidthlnSamplesC and PicHeightlnSamplesC are exported as follows:

[0467] CtbLog2SizeY=log2_ctu_size_minus2+2

[0468] CtbSizeY = 1 <CtbLog2SizeY

[0469] MinCbLog2SizeY=log2_min_luma_coding_block_size_minus2+2

[0470] MinCbSizeY=1< <MinCbLog2SizeY

[0471] MinTbLog2SizeY=2

[0472] MaxTbLog2SizeY = 6

[0473] MinTbSizeY=1< <MinTbLog2SizeY

[0474] MaxTbSizeY=1< <MaxTbLog2SizeY

[0475] PicWidthlnCtbsY=Ceil(pic_width_in_luma_samples CtbSizeY)

[0476] PicHeightlnCtbsY=Ceil(pic_height_in_luma_samples CtbSizeY)

[0477] PicSizelnCtbsY=PicWidthlnCtbsY*PicHeightlnCtbsY

[0478] PicWidthlnMinCbsY=pic_width_in_luma_samples / MinCbSizeY

[0479] PicHeightlnMinCbsY=pic_height_in_luma_samples I MinCbSizeY

[0480] PicSizelnMinCbsY=PicWidthlnMinCbsY*PicHeightlnMinCbsY

[0481] PicSizelnSamplesY=pic_width_in_luma_samples*pic_height_in_luma_samples

[0482] PicWidthlnSamplesC=pic_width_in_luma_samples / SubWidthC

[0483] PicHeightlnSamplesC=pic_height_in_luma_samples / SubHeightC

[0484] The variables CtbWidthC and CtbHeightC specify the width and height of the array for each chroma CTB, respectively. These two variables are derived as follows:

[0485] - If chroma_format_idc equals 0 (monochrome) or separate_colour_plane_flag equals 1, then CtbWidthC and CtbHeightC both equal 0.

[0486] - Otherwise, CtbWidthC and CtbHeightC are exported as follows:

[0487] CtbWidthC=CtbSizeY / SubWidthC

[0488] CtbHeightC=CtbSizeY / SubHeightC

[0489] For log2BlockWidth in the range of 0 to 4 and log2BlockHeight in the range of 0 to 4 (including the end values), call the specified upper-right diagonal scan order array initialization process with 1 << log2BlockWidth and 1 << log2BlockHeight as inputs, and assign the output to DiagScanOrder[log2BlockWidth][log2BlockHeight].

[0490] A partition_constraints_override_enabled_flag equal to 1 specifies the presence of a partition_constraints_override_flag in the slice header of a slice that references the SPS. A partition_constraints_override_enabled_flag equal to 0 specifies the absence of a partition_constraints_override_enabled_flag in the slice header of a slice that references the SPS.

[0491] sps_log2_diff_min_qt_min_cb_intra_slice_luma specifies the default difference between the base-2 logarithm of the minimum size in the luma samples of the luma leaf blocks resulting from quadtree partitioning of a CTU and the base-2 logarithm of the minimum coded block size in the luma samples of the luma CUs in a slice with slice_type equal to 2 (I) that references the SPS. When the partition_constraints_override_flag is equal to 1, this default difference can be overridden by slice_log2_diff_min_qt_min_cb_luma present in the slice header of a slice that references the SPS. The value of sps_log2_diff_min_qt_min_cb_intra_slice_luma should be in the range of 0 to CtbLog2SizeY - MinCbLog2SizeY (including the end values). The base-2 logarithm of the minimum size in the luma samples of the luma leaf blocks resulting from quadtree partitioning of a CTU is derived as follows:

[0492] MinQtLog2SizeIntraY = sps_log2_diff_min_qt_min_cb_intra_slice_luma + MinCbLog2SizeY

[0493] `sps_log2_diff_mm_qt_min_cb_inter_slice` specifies the default difference between the base-2 logarithm of the smallest size of the luminance sample in the luminance blade block generated by quadtree segmentation of the CTU and the base-2 logarithm of the smallest luminance coded block size in the luminance sample of the luminance CU in a slice with `slice_type` equal to 0 (B) or 1 (P) referencing the SPS. This default difference can be overridden by `slice_log2_diff_min_qt_min_cb_luma` present in the slice header of the slice referencing the SPS when `partition_constraints_override_flag` is equal to 1. The value of `sps_log2_diff_min_qt_min_cb_inter_slice` should be in the range of 0 to `CtbLog2SizeY - MinCbLog2SizeY` (inclusive). The base-2 logarithm of the smallest size of the luminance sample in the luminance blade block generated by quadtree segmentation of the CTU is derived as follows:

[0494] MinQtLog2SizeInterY=sps_log2_diff_min_qt_min_cb_inter_slice+MinCbLog2SizeY

[0495] `sps_max_mtt_hierarchy_depth_inter_slice` specifies the default maximum hierarchical depth of the coding unit generated by multi-type tree splitting of quadtree leaves in slices referencing SPS with `slice_type` equal to 0 (B) or 1 (P). When `partition_constraints_override_flag` equals 1, the default maximum hierarchical depth can be overridden by `slice_max_mtt_hierarchy_depth_luma` present in the slice header of the slice referencing SPS. The value of `sps_max_mtt_hierarchy_depth_intcr_slice` should be in the range of 0 to `CtbLog2SizeY - MinCbLog2SizeY` (inclusive).

[0496] `sps_max_mtt_hierarchy_depth_intra_slice_luma` specifies the default maximum hierarchical depth of the coding units generated by multi-type tree splitting of quadtree leaves in slices referencing SPS with `slice_type` equal to 2(I). When `partition_constraints_override_flag` equals 1, the default maximum hierarchical depth can be overridden by `slice_max_mtt_hierarchy_depth_luma` present in the slice header of the slice referencing SPS. The value of `sps_max_mtt_hierarchy_depth_intra_slice_luma` should be in the range of 0 to `CtbLog2SizeY - MinCbLog2SizeY` (inclusive).

[0497] `sps_log2_diff_max_bt_min_qt_intra_slice_luma` specifies the base-2 logarithm of the maximum size (width or height) of the luminance samples in a luminance coded block that can be binary-partitioned, and the base-2 logarithm of the minimum size (width or height) of the luminance samples in a luminance blade block resulting from quadtree partitioning of the CTU in a slice with `slice_type` equal to 2(I) referencing the SPS. This default difference can be overridden by `slice_log2_diff_max_bt_min_qt_luma` present in the slice header of the slice referencing the SPS when `partition_constraints_override_flag` is equal to 1. The value of `sps_log2_diff_max_bt_min_qt_intra_slice_luma` should be in the range of 0 to `CtbLog2SizeY - MinQtLog2SizeIntraY` (inclusive). If sps_log2_diff_max_bt_min_qt_intra_slice_luma does not exist, it is inferred that the value of sps_log2_diff_max_bt_min_qt_intra_slice_luma is equal to 0.

[0498] `sps_log2_diff_max_tt_min_qt_intra_slice_luma` specifies the base-2 logarithm of the maximum size (width or height) of the luminance samples in a luminance coding block that can be segmented using ternary partitioning, and the base-2 logarithm of the minimum size (width or height) of the luminance blade block resulting from quadtree partitioning of the CTU in a slice referencing SPS with slice_type equal to 2(I). This default difference can be overridden by `slice_log2_diff_max_tt_min_qt_luma` present in the slice header of the slice referencing SPS when `partition_constraints_override_flag` is equal to 1. The value of `sps_log2_diff_max_bt_min_qt_intra_slice_luma` should be in the range of 0 to `CtbLog2SizeY - MinQtLog2SizeIntraY` (inclusive). If sps_log2_diff_max_bt_min_qt_intra_slice_luma does not exist, it is inferred that the value of sps_log2_diff_max_tt_min_qt_intra_slice_luma is equal to 0.

[0499] `sps_log2_diff_max_bt_min_qt_inter_slice` specifies the base-2 logarithm of the maximum size (width or height) of the luminance samples in a luminance coded block that can be partitioned into binary segments, and the base-2 logarithm of the minimum size (width or height) of the luminance samples in a luminance blade block resulting from quadtree partitioning of the CTU in a slice with a slice_type equal to 0 (B) or 1 (P) referencing the SPS. This default difference can be overridden by `slice_log2_diff_max_bt_min_qt_luma` present in the slice header of the slice referencing the SPS when `partition_constraints_override_flag` is equal to 1. The value of `sps_log2_diff_max_bt_min_qt_inter_slice` should be in the range of 0 to `CtbLog2SizeY - MinQtLog2SizeInterY` (inclusive). If sps_log2_diff_max_bt_min_qt_inter_slice does not exist, it is inferred that the value of sps_log2_diff_max_bt_min_qt_inter_slice is equal to 0.

[0500] `sps_log2_diff_max_tt_min_qt_inter_slice` specifies the base-2 logarithm of the maximum size (width or height) of the luminance samples in a luminance coding block that can be segmented using ternary partitioning, and the base-2 logarithm of the minimum size (width or height) of the luminance blade block resulting from quadtree partitioning of the CTU in a slice referencing SPS with slice_type equal to 0 (B) or 1 (P). This default difference can be overridden by `slice_log2_diff_max_tt_min_qt_luma` present in the slice header of the slice referencing SPS when `partition_constraints_override_flag` is equal to 1. The value of `sps_log2_diff_max_tt_min_qt_inter_slice` should be in the range of 0 to `CtbLog2SizeY - MinQtLog2SizeInterY` (inclusive). If sps_log2_diff_max_tt_min_qt_inter_slice does not exist, it is inferred that the value of sps_log2_diff_max_tt_min_qt_inter_slice is equal to 0.

[0501] `sps_log2_diff_min_qt_min_cb_intra_slice_chroma` specifies the base-2 logarithm of the smallest size of the luminance samples in the chroma leaf blocks resulting from quadtree partitioning of a chroma CTU with `treeType` equal to `DUAL_TREE_CHROMA`, and the base-2 logarithm of the smallest code block size in the luminance samples of a chroma CU with `treeType` equal to `DUAL_TREE_CHROMA` in a slice with `slice_type` equal to 2(I) referencing SPS. This default difference can be overridden by `slice_log2_diff_min_qt_min_cb_chroma` present in the slice header of the slice referencing SPS when `partition_constraints_override_flag` is equal to 1. The value of `sps_log2_diff_min_qt_min_cb_intra_slice_chroma` should be in the range of 0 to `CtbLog2SizeY - MinCbLog2SizeY` (inclusive). When it does not exist, the value of `sps_log2_diff_min_qt_min_cb_intra_slice_chroma` is inferred to be 0. The base-2 logarithm of the smallest size of the chroma leaf block luminance sample generated by quadtree segmentation of a CTU with `treeType` equal to `DUAL_TREE_CHROMA` is derived as follows:

[0502] MinQtLog2SizeIntraC=sps_log2dif'fminqt_rninchintra_slicc_chroma+MinCbLog2SizeY

[0503] `sps_max_mtt_hierarchy_depth_intra_slice_chroma` specifies the default maximum hierarchical depth of chroma coding units generated by multi-type tree segmentation of chroma quadtree leaves with `treeType` equal to `DUAL_TREE_CHROMA` in slices referencing `SPS` with `slice_type` equal to 2(I). When `partition_constraints_override_flag` equals 1, the default maximum hierarchical depth can be overridden by `slice_max_mtt_hierarchy_depth_chroma` present in the slice header of the slice referencing `SPS`. The value of `sps_max_mtt_hierarchy_depth_intra_slice_chroma` should be in the range of 0 to `CtbLog2SizeY - MinCbLog2SizeY` (inclusive). If it does not exist, the value of `sps_max_mtt_hierarchy_depth_intra_slice_chroma` is inferred to be 0.

[0504] `sps_log2_diff_max_qt_min_cb_intra_slice_chroma` specifies the base-2 logarithm of the maximum size (width or height) of the luminance samples in a chroma-coded block that can be binary-partitioned, and the base-2 logarithm of the minimum size (width or height) of the luminance samples in a chroma CTU with `treeType` equal to `DUAL_TREE_CHROMA` in a slice with `slice_type` equal to 2(I) referencing the SPS. This default difference can be overridden by `slice_log2_diff_max_bt_min_qt_chroma` present in the slice header of the slice referencing the SPS when `partition_constraints_override_flag` equals 1. The value of `sps_log2_diff_max_bt_min_qt_intra_slice_chroma` should be in the range of 0 to `CtbLog2SizeY - MinQtLog2SizeIntraC` (inclusive). When sps_log2_diff_max_bt_min_qt_intra_slice_chroma does not exist, it is inferred that the value of sps_log2_diff_max_bt_min_qt_intra_slice_chroma is equal to 0.

[0505] `sps_log2_diff_max_tt_min_qt_intra_slice_chroma` specifies the base-2 logarithm of the maximum size (width or height) of the luminance samples in a chroma-coded block that can be partitioned using ternary segmentation, and the base-2 logarithm of the minimum size (width or height) of the luminance samples in a chroma CTU with `treeType` equal to `DUAL_TREE_CHROMA` in a slice with `slice_type` equal to 2(I) referencing the SPS. This default difference can be overridden by `slice_log2_diff_max_tt_min_qt_chroma` present in the slice header of the slice referencing the SPS when `partition_constraints_override_flag` is equal to 1. The value of `sps_log2_diff_max_tt_min_qt_intra_slice_chroma` should be in the range of 0 to `CtbLog2SizeY - MinQtLog2SizeIntraC` (inclusive). When sps_log2_diff_max_tt_min_qt_intra_slice_chroma does not exist, it is inferred that the value of sps_log2_diff_max_tt_min_qt_intra_slice_chroma is equal to 0.

[0506] `sps_sao_enabled_flag` equal to 1 specifies that the sample adaptive offset process is applied to the reconstructed image after the unblocking filter process. `sps_sao_enabled_flag` equal to 0 specifies that the sample adaptive offset process is not applied to the reconstructed image after the unblocking filter process.

[0507] A value of 0 for sps_alf_enabled_flag disables the adaptive loop filter. A value of 1 for sps_alf_cnablcd_flag enables the adaptive loop filter.

[0508] A value of 0 for sps_pcm_enabled_flag indicates that PCM-related syntax (pcm_sample_bit_depth_luma_minus1, pcm_sample_bit_depth_chroma_minus1, log2_min_pcm_luma_coding_block_size_minus3, log2_diff_max_min_pcm_luma_coding_block_size, pcm_loop_filter_disabled_flag, pcm_flag, pcm_alignment_zero_bit syntax element, and pcm_sample() syntax structure) does not exist in CVS.

[0509] Note—When MinCbLog2SizeY equals 6 and sps_pcm_enabled_flag equals 1, there is no syntax related to PCM sample data in CVS (pcm_flag, pcm_alignment_zero_bit syntax element, and pcm_sample() syntax structure) because the maximum size of the encoded block for transmitting syntax related to PCM sample data is limited to less than or equal to Min(CtbLog2SizeY, 5). Therefore, MinCbLog2SizeY equal to 6 and sps_pcm_enabled_flag equal to 1 are not appropriate settings for transmitting PCM sample data in CVS.

[0510] pcm_sample_bit_depth_luma_minus1 specifies the number of bits used to represent each PCM sample value of the luminance component as follows:

[0511] PcmBitDepth Y =pcm_sample_bit_depth_luma_minus1+1

[0512] The value of PcmBitDepthy should be less than or equal to the value of BitDepthy.

[0513] pcm_sample_bit_depth_chroma_minus1 specifies the number of bits used to represent each PCM sample value of the chroma component as follows:

[0514] PcmBitDepthc=pcm_sample_bit_depth_chroma_minus1+1

[0515] The value of PcmBitDepthc should be less than or equal to BitDepth. CThe value of pcm_sample_bit_depth_chroma_minus1 is not used during decoding when ChromaArrayType is equal to 0, and the decoder should ignore its value.

[0516] log2_min_pcm_luma_coding_block_size_minus3 plus 3 specifies the minimum size of the coding block when pcm_flag is equal to 1.

[0517] The variable Log2MinIpcmCbSizeY is set to equal log2_min_pcm_luma_coding_block_size_minus3+3. The value of Log2MinIpcmCbSizeY should be within the range of Min(MinCbLog2SizeY,5) to Min(CtbLog2SizeY,5) (inclusive).

[0518] log2_diff_max_min_pcm_luma_coding_block_size specifies the difference between the maximum and minimum sizes of the coding block when pcm_flag is equal to 1.

[0519] The variable Log2MaxIpcmCbSizeY is set to be equal to log2_diff_max_min_pcm_luma_coding_block_size + Log2MinIpcmCbSizeY. The value of Log2MaxIpcmCbSizeY should be less than or equal to Min(CtbLog2SizeY,5).

[0520] The pcm_loop_filter_disabled_flag specifies whether to disable the loop filtering process on reconstructed samples in coding units where pcm_flag is equal to 1:

[0521] - If pcm_loop_filter_disabled_flag equals 1, then the deblocking filter, sample adaptive offset filter, and adaptive loop filter are disabled on the reconstructed samples in the coding unit where pcm_flag equals 1.

[0522] Otherwise (pcm_loop_filter_disabled_flag value equals 0), the deblocking filter, sample adaptive offset filter, and adaptive loop filter are not disabled on reconstructed samples in coding units where pcm_flag equals 1.

[0523] If pcm_loop_filter_disabled_flag does not exist, it is inferred that it is equal to 0.

[0524] A value of 1 for `sps_ref_wraparound_enabled_flag` specifies that horizontal wraparound motion compensation is applied in inter-frame prediction. A value of 0 for `sps_ref_wraparound_enabled_flag` specifies that horizontal wraparound motion compensation is not applied. When it does not exist, the value of `sps_ref_wraparound_enabled_flag` is inferred to be 0.

[0525] Increasing `sps_ref_wraparound_offset_minus1` by 1 specifies the offset used to calculate the horizontal wraparound position, in MinCbSizeY luminance samples. The value of `ref_wraparound_offset_minus1` should be in the range of (CtbSizeY / MinCbSizeY)+1 to (pic_width_in_luma_samples / MinCbSizeY)-1 (inclusive).

[0526] `sps_temporal_mvp_enabled_flag` equal to 1 indicates that `slice_temporal_mvp_enabled_flag` exists in the slice header of slices in CVS whose `slice_type` is not equal to 'I'. `sps_temporal_mvp_enabled_flag` equal to 0 indicates that `slice_temporal_mvp_enabled_flag` does not exist in the slice header and that the temporal motion vector predictor is not used in CVS.

[0527] `sps_sbtmvp_enabled_flag` equal to 1 specifies that a sub-block-based temporal motion vector predictor can be used in CVS for image decoding, where all slices have a `slice_type` not equal to 1. `sps_sbtmvp_enabled_flag` equal to 0 specifies that the sub-block-based temporal motion vector predictor is not used in CVS. When `sps_sbtmvp_enabled_flag` does not exist, it is inferred to be equal to 0.

[0528] A value of 1 for sps_amvr_enabled_flag specifies that adaptive motion vector differential resolution is used in motion vector coding. A value of 0 for amvr_enabled_flag specifies that adaptive motion vector differential resolution is not used in motion vector coding.

[0529] A value of 0 for sps_bdof_enabled_flag indicates that bidirectional optical flow inter-frame prediction is disabled. A value of 1 for sps_bdof_enabled_flag indicates that bidirectional optical flow inter-frame prediction is enabled.

[0530] A value of 1 for sps_smvd_enabled_flag indicates that symmetric motion vector difference can be used in motion vector decoding. A value of 0 for sps_smvd_enabled_flag indicates that symmetric motion vector difference is not used in motion vector encoding.

[0531] A value of 1 for `sps_affine_amvr_enabled_flag` specifies that adaptive motion vector difference resolution is used in motion vector coding in affine inter-frame mode. A value of 0 for `sps_affine_amvr_enabled_flag` specifies that adaptive motion vector difference resolution is not used in motion vector coding in affine inter-frame mode.

[0532] A value of 1 for sps_dmvr_enabled_flag indicates that inter-frame dual prediction based on decoder motion vector correction is enabled. A value of 0 for sps_dmvr_enabled_flag indicates that inter-frame dual prediction based on decoder motion vector correction is disabled.

[0533] A value of 1 for sps_mmvd_enabled_flag indicates that the merge mode with motion vector difference is enabled. A value of 0 for sps_mmvd_enabled_flag indicates that the merge mode with motion vector difference is disabled.

[0534] A value of 1 for sps_isp_enabled_flag indicates that intra-prediction with sub-partitions is enabled. A value of 0 for sps_isp_enabled_flag indicates that intra-prediction with sub-partitions is disabled.

[0535] A value of 1 for sps_mrl_enabled_flag indicates that intra-prediction with multiple reference lines is enabled. A value of 0 for sps_mrl_enabled_flag indicates that intra-prediction with multiple reference lines is disabled.

[0536] A value of 1 for sps_mip_enabled_flag indicates that matrix-based intra-frame prediction is enabled.

[0537] A value of 0 for sps_mrl_enabled_flag indicates that matrix-based intra-frame prediction is disabled.

[0538] A value of 0 for sps_cclm_enabled_flag disables intra-frame prediction of the cross-component linear model from the luma component to the chroma component. A value of 1 for sps_cclm_enabled_flag enables intra-frame prediction of the cross-component linear model from the luma component to the chroma component.

[0539] `sps_cclm_colocated_chroma_flag` equal to 1 specifies that the left and top-left sampled luminance samples in the intra-frame prediction of the cross-component linear model are juxtaposed with the top-left luminance sample. `sps_cclm_colocated_chroma_flag` equal to 0 specifies that the left and top-left sampled luminance samples in the intra-frame prediction of the cross-component linear model are horizontally co-located with the top-left luminance sample, but vertically shifted by 0.5 luminance sample units relative to the top-left luminance sample.

[0540] `sps_mts_enabled_flag` equal to 1 indicates that both `sps_explicit_mts_intra_enabled_flag` and `sps_explicit_mts_inter_enabled_flag` exist in the sequence parameter set RBSP syntax. `sps_mts_enabled_flag` equal to 0 indicates that neither `sps_explicit_mts_intra_enabled_flag` nor `sps_explicit_mts_inter_enabled_flag` exist in the sequence parameter set RBSP syntax.

[0541] A value of 1 for `sps_explicit_mts_intra_enabled_flag` indicates that `tu_mts_idx` may exist in the transform unit syntax for intra-coding units. A value of 0 for `sps_explicit_mts_intra_enabled_flag` indicates that `tu_mts_idx` does not exist in the transform unit syntax for intra-coding units. When it does not exist, it is inferred that the value of `sps_explicit_mts_intra_enabled_flag` is 0.

[0542] A value of 1 for `sps_explicit_mts_inter_enabled_flag` indicates that `tu_mts_idx` may exist in the transform unit syntax used for inter-coding units. A value of 0 for `sps_explicit_mts_inter_enabled_flag` indicates that `tu_mts_idx` does not exist in the transform unit syntax used for inter-coding units. When it does not exist, it is inferred that the value of `sps_explicit_mts_inter_enabled_flag` is 0.

[0543] A value of 0 for sps_sbt_enabled_flag disables subblock transform for inter-frame prediction CUs. A value of 1 for sps_sbt_enabled_flag enables subblock transform for inter-frame prediction CUs.

[0544] A value of 0 for `sps_sbt_max_size_64_flag` specifies that the maximum CU width and height allowed for subblock transformations is 32 luminance samples. A value of 1 for `sps_sbt_max_size_64_flag` specifies that the maximum CU width and height allowed for subblock transformations is 64 luminance samples.

[0545] MaxSbtSize=sps_sbtmax_size_64_flag? 64:32

[0546] `sps_affine_enabled_flag` specifies whether affine-based motion compensation can be used for inter-frame prediction. If `sps_affine_enabled_flag` equals 0, the syntax should be constrained so that affine-based motion compensation is not used in CVS, and `inter_affine_flag` and `cu_affine_type_flag` are not present in the CVS coding unit syntax. Otherwise (`sps_affine_enabled_flag` equals 1), affine-based motion compensation can be used in CVS.

[0547] `sps_affine_type_flag` specifies whether motion compensation based on a 6-parameter affine model can be used for inter-frame prediction. If `sps_affine_type_flag` equals 0, the syntax should be constrained so that motion compensation based on a 6-parameter affine model is not used in CVS, and `cu_affine_type_flag` does not exist in the coding unit syntax of CVS. Otherwise (`sps_affine_type_flag` equals 1), motion compensation based on a 6-parameter affine model can be used in CVS. When it does not exist, the value of `sps_affine_type_flag` is inferred to be 0.

[0548] `sps_bcw_enabled_flag` specifies whether double prediction with CU weights can be used for inter-frame prediction. If `sps_bcw_enabled_flag` equals 0, the syntax should be constrained so that double prediction with CU weights is not used in CVS, and `bcw_idx` does not exist in the CVS coding unit syntax. Otherwise (`sps_bcw_enabled_flag` equals 1), double prediction with CU weights can be used in CVS.

[0549] `sps_ibc_enabled_flag` equal to 1 indicates that the current image reference can be used in image decoding in CVS. `sps_ibc_enabled_flag` equal to 0 indicates that the current image reference is not used in CVS. If `sps_ibc_enabled_flag` does not exist, it is inferred to be equal to 0.

[0550] `sps_ciip_enabled_flag` specifies that the ciip_flag may exist in the coding unit syntax used for inter-frame coding units. `sps_ciip_enabled_flag` equal to 0 indicates that the ciip_flag does not exist in the coding unit syntax used for inter-frame coding units.

[0551] `sps_fpel_mmvd_enabled_flag` equal to 1 indicates that the merge mode with motion vector difference is using integer sample precision. `sps_fpel_mmvd_enabled_flag` equal to 0 indicates that the merge mode with motion vector difference can use fractional sample precision.

[0552] `sps_triangle_enabled_flag` specifies whether triangle-based motion compensation can be used for inter-frame prediction. `sps_triangle_enabled_flag` equal to 0 specifies that the syntax should be constrained so that triangle-based motion compensation is not used in CVS, and that `merge_triangle_split_dir`, `merge_triangle_idx0`, and `merge_triangle_idx1` are not present in the CVS coding unit syntax. `sps_triangle_enabled_flag` equal to 1 specifies that triangle-based motion compensation can be used in CVS.

[0553] A value of 1 for `sps_lmcs_enabled_flag` specifies that a luminance map with chroma scaling is used in CVS. A value of 0 for `sps_lmcs_enabled_flag` specifies that a luminance map with chroma scaling is not used in CVS.

[0554] A value of 1 for sps_lfnst_enabled_flag indicates that lfnst_idx may be present in the residual coding syntax used for intra-coding units. A value of 0 for sps_lfhst_enabled_flag indicates that lfnst_idx is not present in the residual coding syntax used for intra-coding units.

[0555] sps_ladf_enabled_flag equal to 1 indicates that sps_num_ladf_intervals_minus2, sps_ladf_lowest_interval_qp_offset, sps_ladf_qp_offset[i] and sps_ladf_delta_threshold_minus1[i] exist in SPS.

[0556] Increasing 1 to sps_num_ladf_intervals_minus2 specifies the number of sps_ladf_delta_threshold_minus1[i] and sps_ladf_qp_offset[i] syntax elements present in SPS. The value of sps_num_ladf_intervals_minus2 should be in the range of 0 to 3 (inclusive).

[0557] sps_ladf_lowest_interval_qp_offset specifies the offset used to derive the specified variable qP. The value of sps_ladf_lowest_interval_qp_offset should be in the range of 0 to 63 (inclusive).

[0558] sps_ladf_qp_offset[i] specifies the offset array used to derive the specified variable qP. The value of sps_ladf_qp_offset[i] should be in the range of 0 to 63 (inclusive).

[0559] `sps_ladf_delta_threshold_minus1[i]` is used to calculate the value of `SpsLadfIntervalLowerBound[i]`, which specifies the lower bound of the i-th luminance intensity level interval. The value of `sps_ladf_delta_threshold_minus1[i]` should be between 0 and 2. BitDopthY The range is -3 (inclusive).

[0560] The value of SpsLadflntervalLowerBoundf[0] is set to equal to 0.

[0561] For each value of i in the range of 0 to sps_num_ladf_intervals_minus2 (inclusive), the variable

[0562] SpsLadflntervalLowerBound[i+1] is exported as follows:

[0563] SpsLadflntervalLowerBound[i+1]=SpsLadflntervalLowerBound[i]

[0564] +sps_ladf_delta_threshold_minus1[i]+1

[0565] A timing_info_present_flag value of 1 indicates that the syntax elements num_units_in_tick, time_scale, and hrd_parameters_present_flag exist in the SPS RBSP syntax structure. A timing_info_present_flag value of 0 indicates that num_units_in_tick, time_scale, and hrd_parameters_present_flag do not exist in the SPS RBSP syntax structure.

[0566] `num_units_in_tick` is the number of time units of a clock operating at a frequency of `time_scale` Hz, which corresponds to an increment of the clock tick counter (called a clock tick). `num_units_in_tick` should be greater than 0. A clock tick in seconds is equal to the quotient of `num_units_in_tick` divided by `time_scale`. For example, when the picture rate of a video signal is 25Hz, `time_scale` can be equal to 27,000,000 and `num_units_in_tick` can be equal to 1,080,000, and therefore the clock tick can be equal to 0.04 seconds.

[0567] `time_scale` is the number of time units that elapse within one second. For example, the `time_scale` of a time coordinate system using a 27MHz clock is 27,000,000. The value of `time_scale` should be greater than 0.

[0568] A value of 1 for hrd_parameters_present_flag indicates that the syntax structure hrd_parameters() exists within the SPS RBSP syntax structure. A value of 0 for hrd_parameters_present_flag indicates that the syntax structure hrd_parameters() does not exist within the SPS RBSP syntax structure.

[0569] A value of 1 for `vui_parameters_present_flag` indicates that the syntax structure `vui_parameters()` exists within the SPS RBSP syntax structure. A value of 0 for `vui_parameters_present_flag` indicates that the syntax structure `vui_parameters()` does not exist within the SPS RBSP syntax structure.

[0570] A value of 0 for sps_extension_flag indicates that the sps_extension_data_flag syntax structure does not exist in the SPS RBSP syntax structure. A value of 1 for sps_extension_flag indicates that the sps_extension_data_flag syntax structure exists in the SPS RBSP syntax structure.

[0571] The `sps_extension_data_flag` flag can have any value. Its presence and value do not affect the decoder conforming to the configuration file specified in this version of the specification. Decoders conforming to this version of the specification should ignore all `sps_extension_data_flag` syntax elements.

[0572] In addition, Table 4 shows the image parameter set syntax provided in JVET-N1001.

[0573]

[0574]

[0575]

[0576] Table 4

[0577] Regarding Table 4, JVET-N1001 provides the following semantics:

[0578] The `pps_pic_parameter_set_id` identifier is used by other syntax elements for reference. The value of `pps_pic_parameter_set_id` should be in the range of 0 to 63 (inclusive).

[0579] pps_seq_parameter_set_id specifies the value of sps_seq_parameter_set_id for the active SPS. The value of pps_seq_parameter_set_id should be in the range of 0 to 15 (inclusive).

[0580] A value of 1 for `output_flag_present_flag` indicates that the `pic_output_flag` syntax element exists in the slice header referencing PPS. A value of 0 for `output_flag_present_flag` indicates that the `pic_output_flag` syntax element does not exist in the slice header referencing PPS.

[0581] A single_tile_in_pic_flag value of 1 indicates that there is only one tile in each image referencing PPS. A single_tile_in_pic_flag value of 0 indicates that there is more than one tile in each image referencing PPS.

[0582] Note—If there are no further tile divisions within a tile, the entire tile is called a tile. When an image contains only a single tile without further tile divisions, the image is called a single tile.

[0583] For bitstream compliance requirements, the value of single_tile_in_pic_flag should be the same for all PPSs active within CVS.

[0584] A `uniform_filespacing_flag` of 1 specifies that tile column boundaries and the same tile row boundaries are evenly distributed across the entire image, and the tile column and row boundaries are signaled using the syntax elements `tile_cols_width_minus1` and `tile_rows_height_minus1`. A `uniform_tile_spacing_flag` of 0 specifies that tile column boundaries and the same tile row boundaries can be evenly or non-uniformly distributed across the entire image, and the tile column and row boundaries are signaled using the syntax elements `num_tile_columns_minus1` and `num_tile_rows_minus1`, as well as a list of syntax elements `tile_column_width_minus1[i]` and `tile_row_height_minus1[i]`. If the `uniform_tile_spacing_flag` does not exist, it is inferred that the value of `uniform_tile_spacing_flag` is equal to 1.

[0585] Increasing `tile_cols_width_minus1` by 1 specifies the width of the tile columns excluding the rightmost tile column of the image, in CTB, when `uniform_tile_spacing_flag` equals 1. The value of `tile_cols_width_minus1` should be in the range of 0 to `PicWidthlnCtbsY-1` (inclusive). If it does not exist, the value of `tile_cols_width_minus1` is assumed to be equal to `PicWidthlnCtbsY-1`.

[0586] Incrementing `tile_rows_height_minus1` by 1 specifies the height of the tile rows, excluding the bottom tile row of the image, in CTB when `uniform_tile_spacing_flag` is equal to 1. The value of `tile_rows_height_minus1` should be in the range of 0 to `PicHeightlnCtbsY-1` (inclusive). If it does not exist, it is assumed that the value of `tile_rows_height_minus1` is equal to `PicHeightlnCtbsY-1`.

[0587] `num_tile_columns_minus1` incremented by 1 specifies the number of tile columns used to divide the image when `uniform_tile_spacing_flag` equals 0. The value of `num_tile_columns_minus1` should be in the range of 0 to `PicWidthlnCtbsY-1` (inclusive). When `single_tile_in_pic_flag` equals 1, the value of `num_tile_columns_minus1` is inferred to be 0. Otherwise, when `uniform_tile_spacing_flag` equals 1, the value of `num_tile_columns_minus1` is inferred based on the specification.

[0588] Incrementing `num_tile_rows_minus1` by 1 specifies the number of tile rows to divide the image when `uniform_tile_spacing_flag` equals 0. The value of `num_tile_rows_minus1` should be in the range of 0 to `PicHeightlnCtbsY-1` (inclusive). When `single_tile_in_pic_flag` equals 1, the value of `num_tile_rows_minus1` is inferred to be 0. Otherwise, when `uniform_tile_spacing_flag` equals 1, the value of `num_tile_rows_minus1` is inferred based on the specification.

[0589] The variable NumTilesInPic is set to equal to

[0590] (num_tile_columns_minus1+1)*(num_tile_rows_minus1+1).

[0591] When single_tile_in_pic_flag equals 0, NumTilesInPic should be greater than 1.

[0592] tile_column_width_minus1[i] incremented by 1 specifies the width of the i-th tile column, in CTB.

[0593] tile_row_height_minus1[i] incremented by 1 specifies the height of the i-th tile row, in CTB.

[0594] A `brick_splitting_present_flag` value of 1 indicates that one or more tiles referencing PPS images can be split into two or more tiles. A `brick_splitting_present_flag` value of 0 indicates that tiles not referencing PPS images are split into two or more tiles.

[0595] A value of 1 for `brick_split_flag[i]` indicates that the i-th tile is split into two or more bricks. A value of 0 for `brick_split_flag[i]` indicates that the i-th tile is not split into two or more bricks. If the i-th tile does not exist, it is assumed that the value of `brick_split_flag[i]` is 0.

[0596] A `uniform_brick_spacing_flag[i]` equal to 1 indicates that the horizontal brick boundaries are uniformly distributed on the i-th brick, and the horizontal brick boundaries are signaled using the syntax element `brick_height_minus1[i]`. A `uniform_brick_spacing_flag[i]` equal to 0 indicates that the horizontal brick boundaries may be uniformly distributed or non-uniformly distributed on the i-th brick, and the horizontal brick boundaries are signaled using a list of syntax elements `num_brick_rows_minus1[i]` and `brick_row_height_minus1[i][j]`. If it does not exist, the value of `uniform_brick_spacing_flag[i]` is assumed to be 1.

[0597] Incrementing `brick_height_minus1[i]` by 1 specifies the height of the row of bricks in the i-th tile, excluding the bottom bricks, when `uniform_brick_spacing_flag[i]` equals 1, in CTB units. When present, `brick_height_minus1` should be between 0 and RowHeight[i] - 2 (inclusive). When absent, it is assumed that `brick_height_minus1[i]` equals RowHeight[i] - 1.

[0598] The increment of `mm_brick_rows_minus1[i]` by 1 specifies the number of bricks to divide the i-th tile when `uniform_brick_spacing_flag[i]` equals 0. When present, the value of `num_brick_rows_minus1[i]` should be in the range of 1 to `RowHeight[i] - 1` (inclusive). If `brick_split_flag[i]` equals 0, the value of `num_brick_rows_minus1[i]` is inferred to be 0. Otherwise, if `uniform_brick_spacing_flag[i]` equals 1, the value of `num_brick_rows_minus1[i]` is inferred based on the specification.

[0599] The increment of brick_row_height_minus1[i][j] by 1 specifies the height of the j-th brick in the i-th tile when uniform_tile_spacing_flag is equal to 0, in CTB units.

[0600] Export the following variables, and when uniform_tile_spacing_flag equals 1, infer the values ​​of num_tile_columns_minus1 and num_tile_rows_minus1, and for each i in the range from 0 to NumTilesInPic-1 (inclusive), when uniform_brick_spacing_flag[i] equals 1, infer the value of num_brick_rows_minus1[i] by calling the specified CTB raster and brick scan conversion procedure:

[0601] - A list RowHeight[j], where j is in the range 0 to num_tile_rows_minus1 (inclusive), specifying the height of the j-th tile row in CTB.

[0602] - The list CtbAddrRsToBs[ctbAddrRs], where ctbAddrRs is in the range of 0 to PicSizelnCtbsY-1 (inclusive), specifies the conversion from CTB addresses in the CTB raster scan of the image to CTB addresses in the brick scan.

[0603] - A list CtbAddrBsToRs[ctbAddrBs], where ctbAddrBs is in the range of 0 to PicSizelnCtbsY-1 (inclusive), which specifies the conversion from the CTB address in the brick scan to the CTB address in the image's CTB raster scan.

[0604] - A list BrickId[ctbAddrBs], where ctbAddrBs is in the range of 0 to PicSizeInCtbsY-1 (inclusive), which specifies the translation from the CTB address to the brick ID in the brick scan.

[0605] - A list NumCtusInBrick[brickIdx], where brickIdx is in the range of 0 to NumBricksInPic-1 (inclusive), which specifies the conversion from the brick index to the number of CTUs in the brick.

[0606] - List FirstCtbAddrBs[brickIdx], where brickIdx is in the range of 0 to NumBricksInPic-1 (inclusive), which specifies the translation from the brick ID to the CTB address in the brick scan of the first CTB in the brick.

[0607] A single_brick_per_slice_flag value of 1 indicates that each slice referencing the PPS includes one brick. A single_brick_per_slice_flag value of 0 indicates that slices referencing the PPS may include more than one brick. When it does not exist, the value of single_brick_per_slice_flag is assumed to be 1.

[0608] A `rect_slice_flag` value of 0 indicates that the blocks within each slice are in the raster scan order and no slice information is signaled in the PPS. A `rect_slice_flag` value of 1 indicates that the blocks within each slice cover a rectangular area of ​​the image and slice information is signaled in the PPS. If it does not exist, it is assumed that `rect_slice_flag` is equal to 1.

[0609] `num_slices_in_pic_minus1` incremented by 1 specifies the number of slices referencing PPS in each image. The value of `num_slices_in_pic_minus1` should be in the range of 0 to `NumBricksInPic-1` (inclusive). If it does not exist and `single_brick_per_slice_flag` is equal to 1, it is inferred that the value of `num_slices_in_pic_minus1` is equal to `NumBricksInPic-1`.

[0610] `top_left_brick_idx[i]` specifies the brick index of the top-left corner of the i-th slice. For any `i` not equal to `j`, the value of `top_left_brick_idx[i]` should not be equal to the value of `top_left_brick_idx[j]`. If it does not exist, it is inferred that the value of `top_left_brick_idx[i]` is equal to `i`. The length of the `top_left_brick_idx[i]` syntax element is Ceil(Log2(NumBricksInPic) bits.

[0611] `bottom_right_brickidx_delta[i]` specifies the difference between the brick index of the bottom-right corner of the i-th slice and `top_left_brick_idx[i]`. When `single_brick_per_slice_flag` equals 1, it is inferred that `bottom_right_brick_idx_delta[i]` is 0. The length of the `bottom_right_brick_idx_delta[i]` syntax element is Ceil(Log2(NumBricksInPic-top_left_brick_idx[i])) bits.

[0612] Bitstream compliance requirements: a slice should include multiple complete tiles or only one tile in sequential, consecutive complete tiles.

[0613] The variables NumBricksInSlice[i] and BricksToSliceMap[j] specify the number of bricks in the i-th slice and the mapping from bricks to slices, respectively. These variables are derived as follows:

[0614]

[0615] A `loop_filter_across_bricks_enabled_flag` value of 1 specifies that in-loop filtering can be performed across brick boundaries in images referencing the PPS. A `loop_filter_across_bricks_enabled_flag` value of 0 specifies that in-loop filtering can be performed without crossing brick boundaries in images referencing the PPS. In-loop filtering operations include unblocking filters, sample adaptive offset filters, and adaptive loop filters. When not present, the value of `loop_filter_across_bricks_enabled_flag` is inferred to be 1.

[0616] A `loop_filter_across_slices_enabled_flag` value of 1 specifies that loop filtering can be performed across slice boundaries in an image that spans the reference PPS. A `loop_filter_across_slice_enabled_flag` value of 0 specifies that loop filtering can be performed across slice boundaries in an image that does not span the reference PPS. Loop filtering operations include unblocking filters, sample adaptive offset filters, and adaptive loop filter operations. When not present, the value of `loop_filter_across_slices_enabled_flag` is inferred to be 0.

[0617] A `signalled_slice_id_flag` value of 1 indicates that the slice ID of each slice is signaled. A `signalled_slice_id_flag` value of 0 indicates that the slice ID is not signaled. When `rect_slice_flag` is equal to 0, it is inferred that the value of `signalled_slice_id_flag` is equal to 0.

[0618] Increasing 1 to `signalled_slice_id_length_minus1` specifies the number of bits used to represent the syntax element `slice_id[i]` (if present) and the syntax element `slice_address` in the slice header. The value of `signalled_slice_id_length_minus1` should be in the range of 0 to 15 (inclusive). If it does not exist, the value of `signalled_slice_id_length_minus1` is inferred to be equal to `Ceil(Log2(num_slices_in_pic_minus1+1))-1`.

[0619] `slice_id[i]` specifies the slice ID of the i-th slice. The length of the `slice_id[i]` syntax element is `signalled_slice_id_length_minus1+1` bits. If it does not exist, for each i in the range 0 to `num_slices_in_pic_minus1` (inclusive), the value of `slice_id[i]` is inferred to be equal to i.

[0620] `entropy_coding_sync_enabled_flag` equal to 1 specifies that a specific synchronization procedure of the context variable is called before the CTU of the first CTB in the CTB row of each brick in each image including the referenced PPS is decoded, and after the CTU of the first CTB in the CTB row of each brick in each image including the referenced PPS is decoded. `entropy_coding_sync_enabled_flag` equal to 0 specifies that the specific synchronization procedure of the context variable does not need to be called before the CTU of the first CTB in the CTB row of each brick in each image including the referenced PPS is decoded, and after the CTU of the first CTB in the CTB row of each brick in each image including the referenced PPS is decoded.

[0621] For bitstream compliance requirements, the value of `entropy_coding_sync_enabled_flag` should be the same for all PPSs active within CVS.

[0622] A cabac_init_present_flag value of 1 indicates that the cabac_init_flag exists in the slice header referencing PPS. A cabac_init_present_flag value of 0 indicates that the cabac_init_flag does not exist in the slice header referencing PPS.

[0623] Increment 1 by num_ref_idx_default_active_minus1[i]. When i equals 0, specify the inferred value of the variable NumRefIdxActive[0] for slice P or slice B, where num_ref_idx_active_override_flag equals 0. When i equals 1, specify the inferred value of NumRefldxActive[1] for slice B, where num_ref_idx_active_override_flag equals 0. The value of num_ref_idx_default_active_minus1[i] should be in the range of 0 to 14 (inclusive).

[0624] A value of 0 for rpll_idx_present_flag indicates that ref_pic_list_sps_flag[1] and ref_pic_list_idx[1] do not exist in the slice header. A value of 1 for rpll_idx_present_flag indicates that ref_pic_list_sps_flag[1] and ref_pic_list_idx[1] may exist in the slice header.

[0625] `init_qp_minus26` plus 26 specifies the initial value of `SliceQpy` for each slice referencing PPS. The initial value of `SliceQpy` is modified at the slice level when `slice_qp_delta` is decoded to a non-zero value. The value of `init_qp_minus26` should be within the range of -(26 + QpBdOffset). Y The range is from +37 to +37 (inclusive).

[0626] A transform_skip_enabled_flag value of 1 indicates that the transform_skip_flag may exist in the transform unit syntax. A transform_skip_enabled_flag value of 0 indicates that the transform_skip_flag does not exist in the transform unit syntax.

[0627] log2_transform_skip_max_size_minus2 specifies the maximum block size to be skipped during transformation and should be in the range of 0 to 3.

[0628] If it does not exist, it is inferred that the value of log2_transform_skip_max_size_minus2 is equal to 0.

[0629] The variable MaxTsSize is set to equal to 1 << (log2_transform_skip_max_size_minus2+2).

[0630] A flag of 1 for `cu_qp_delta_enabled_flag` indicates that the `cu_qp_delta_subdiv` syntax element exists in the PPS and that `cu_qp_delta_abs` may exist in the transform unit syntax. A flag of 0 for `cu_qp_delta_enabled_flag` indicates that the `cu_qp_delta_subdiv` syntax element does not exist in the PPS and that `cu_qp_delta_abs` does not exist in the transform unit syntax.

[0631] `cu_qp_delta_subdiv` specifies the maximum `cbSubdiv` value of the encoding unit that transmits `cu_qp_delta_abs` and `cu_qp_delta_sign_flag`. The range of values ​​for `cu_qp_delta_subdiv` is specified as follows:

[0632] - If slice_type equals I, then the value of cu_qp_delta_subdiv should be in the range of 0 to 2*(log2_ctu_size_minus2-log2_min_qt_size_intra_slice_minus2+MaxMttDepthY) (inclusive).

[0633] - Otherwise (slice_type is not equal to I), the value of cu_qp_delta_subdiv should be in the range of 0 to 2*(log2_ctu_size_minus2-log2_min_qt_size_inter_slice_minus2+MaxMttDepthY) (inclusive).

[0634] If it does not exist, the value of cu_qp_delta_subdiv is inferred to be 0.

[0635] pps_cb_qp_offset and pps_cr_qp_offset specify the values ​​used for exporting Qp. Cb and Qp' Cr Brightness quantization parameter Qp' Y The offset. The values ​​of pps_cb_qp_offset and pps_cr_qp_offset should be in the range of -12 to +12 (inclusive). When ChromaArrayType equals 0, pps_cb_qp_offset and pps_cr_qp_offset are not used during decoding, and the decoder should ignore their values.

[0636] A value of 1 for `pps_slice_chroma_qp_offsets_present_flag` indicates that the `slice_cb_qp_offset` and `slice_cr_qp_offset` syntax elements exist in the associated slice header. A value of 0 for `pps_slice_chroma_qp_offsets_present_flag` indicates that these syntax elements do not exist in the associated slice header. When `ChromaArrayType` is 0, `pps_slice_chroma_qp_offsets_present_flag` should be 0.

[0637] A weighted_pred_flag value of 0 indicates that weighted predictions are not applied to the P-slice, while a weighted_pred_flag value of 1 indicates that weighted predictions are applied to the P-slice.

[0638] A weighted_bipred_flag value of 0 specifies that the default weighted prediction is applied to the B-slice. A weighted_bipred_flag value of 1 specifies that the weighted prediction is applied to the B-slice.

[0639] A deblocking filter control present flag of 1 indicates that the deblocking filter control syntax element exists in the PPS. A deblocking filter control present flag of 0 indicates that the deblocking filter control syntax element does not exist in the PPS.

[0640] A `deblocking_filter_override_enabled_flag` value of 1 indicates that the `deblocking_filter_override_flag` is present in the slice header of images referencing PPS. A `deblocking_filter_override_enabled_flag` value of 0 indicates that the `deblocking_filter_override_flag` is not present in the slice header of images referencing PPS. When it is not present, it is assumed that the value of `deblocking_filter_override_flag` is 0.

[0641] A value of 1 for `pps_deblocking_filter_disabled_flag` specifies that the deblocking filter operation should not be applied to slices of PPS where `slice_deblocking_filter_disabled_flag` is not present. A value of 0 for `pps_deblocking_filter_disabled_flag` specifies that the deblocking filter operation should be applied to slices of PPS where `slice_deblocking_filter_disabled_flag` is not present. When it is not present, the value of `pps_deblocking_filter_disabled_flag` is assumed to be 0.

[0642] `pps_beta_offset_div2` and `pps_tc_offset_div2` specify the default unblocking parameter offsets (divided by 2) applied to the P and tC values ​​of slices referencing PPS, unless the default unblocking parameter offsets are overridden by unblocking parameter offsets present in the slice header of the slice referencing PPS. The values ​​of `pps_beta_offset_div2` and `pps_tc_offset_div2` should both be in the range of -6 to 6 (inclusive). If they do not exist, it is assumed that the values ​​of `pps_beta_offset_div2` and `pps_tc_offset_div2` are equal to 0.

[0643] `pps_loop_filter_across_virtual_boundaries_disabled_flag` equal to 1 specifies that in-loop filtering operations are disabled on virtual boundaries in images referencing PPS. `pps_loop_filter_across_virtual_boundaries_disabled_flag` equal to 0 specifies that this type of disabling of in-loop filtering operations is not applied in images referencing PPS. In-loop filtering operations include unblocking filters, sample adaptive offset filters, and adaptive loop filter operations. When not present, it is inferred that the value of `pps_loop_filter_across_virtual_boundaries_disabled_flag` is equal to 0.

[0644] `pps_num_ver_virtual_boundaries` specifies the number of `pps_virtual_boundaries_pos_x[i]` syntax elements that exist in PPS. If `pps_virtual_boundariespos_x[i]` does not exist, it is inferred to be equal to 0.

[0645] `pps_virtual_boundaries_pos_x[i]` is used to calculate the value of `PpsVirtualBoundariesPosX[i]`, which specifies the position of the i-th vertical virtual boundary in luminance samples. The number of bits used to represent `pps_virtual_boundaries_pos_x[i]` is `Ceil(Log2(pic_width_in_luma_samples)-3)`. `pps_virtual_boundaries_pos_x[i]` should be in the range of 1 to `Ceil(pic_width_in_luma_samples÷8)-1` (inclusive).

[0646] The position of the vertical virtual boundary, PpsVirtualBoundariesPosX[i], is exported as follows:

[0647] PpsVirtualBoundariesPosX[i]=pps_virtual_boundaries_pos_x[i]*8

[0648] The distance between any two vertical virtual boundaries should be greater than or equal to CtbSizeY luminance samples.

[0649] pps_num_hor_virtual_boundaries specifies the number of pps_virtual_boundaries_pos_y[i] syntax elements that exist in PPS. If pps_num_hor_virtual_boundaries does not exist, it is inferred to be equal to 0.

[0650] `pps_virtual_boundaries_pos_y[i]` is used to calculate the value of `PpsVirtualBoundariesPosY[i]`, which specifies the position of the i-th horizontal virtual boundary in luminance samples. The number of bits used to represent `pps_virtual_boundaries_pos_y[i]` is `Ceil(Log2(pic_height_in_luma_samples)-3)`. `pps_virtual_boundaries_pos_y[i]` should be in the range of 1 to `Ceil(pic_height_in_luma_samples÷8)-1` (inclusive).

[0651] The position of the horizontal virtual boundary, PpsVirtualBoundariesPosY[i], is derived as follows:

[0652] PpsVirtualBoundariesPosY[i]=pps_virtual_boundaries_pos_y[i]*8

[0653] The distance between any two horizontal virtual boundaries should be greater than or equal to CtbSizeY luminance samples.

[0654] A value of 0 for pps_extension_flag indicates that the pps_extension_data_flag syntax element does not exist in the PPS RBSP syntax structure. A value of 1 for pps_extension_flag indicates that the pps_extension_data_flag syntax element exists in the PPS RBSP syntax structure.

[0655] The `pps_extension_data_flag` flag can have any value. Its presence and value do not affect the decoder conforming to the configuration file specified in this version of the specification. Decoders conforming to this version of the specification should ignore all `pps_extension_data_flag` syntax elements.

[0656] Adaptive Resolution Variation (ARC) refers to the variation of image resolution within CVS. It should be noted that Adaptive Resolution Variation can also be called Reference Image Resampling (RRR). Therefore, although the term ARC is used in this document, it can be used interchangeably with RRR or other equivalent terms in some cases. That is, ARC addresses situations where the resolution of the stored reference image may differ from the resolution of the current image. The current image can be decoded using predictions from a reference image generated by downsampling and / or downsampling the stored reference image (i.e., the stored reference image has a higher resolution than the current image) or upsampling and / or upsampling the stored reference image (the stored reference image has a lower resolution than the current image). ARC can be used in many different scenarios. Exemplary scenarios where ARC downsampling may be useful include: situations where the encoded resolution is reduced due to a (significant) decrease in available bandwidth; situations where the resolution is reduced due to system-triggered events, such as changes in the active speaker in a multi-party session, or switching between "main video" and screen sharing; situations where the resolution is reduced due to user-triggered events, such as switching from full screen to window / thumbnail, or rotating a handheld device from landscape to portrait; and / or situations where the resolution received in an Adaptive Bitrate (ABR) streaming service is reduced when switching from a higher bitrate representation to a lower bitrate representation.

[0657] Several different proposals have been made to add support for ARC in VVC. One exemplary proposal is by Chen et al., “AHG 19: Adaptive Resolution Change,” JVET-N0279, March 2019 (referred to as Chen in this paper), which describes signaling adaptive resolution changes to the parameter set and modifications to the current motion compensation prediction process when there is a resolution change between the current image and its reference image. Specifically, Chen describes using a block-based, one-step approach for the case where the stored reference image has a different resolution than the current image. That is, Chen provides modifications to the motion compensation prediction process in JVET-N1001 limited to motion vector scaling and subpixel position derivation, and cases where a separate resampling process is not performed on the reference image. Furthermore, the existing motion compensation interpolator remains unchanged in Chen's work.

[0658] Specifically, regarding ARC in the signal notification parameter set, Chen provides the following cases where the syntax elements pic_width_in_luma_samples and pic_height_in_luma_samples in the SPS syntax structure are replaced by the syntax elements max_pic_width_in_luma_samples and max_pic_height_in_luma_samples with the following semantics:

[0659] `max_pic_width_in_luma_samples` specifies the maximum width of the decoded image referencing SPS, in luminance samples. `max_pic_width_in_luma_samples` should not be equal to 0 and should be an integer multiple of `MinCbSizeY`.

[0660] `max_pic_height_in_luma_samples` specifies the maximum height of the decoded image using SPS, in luminance samples. `max_pic_height_in_luma_samples` should not be equal to 0 and should be an integer multiple of `MinCbSizeY`.

[0661] Chen also provided examples of adding the syntax provided in Table 5 to the PPS syntax structure:

[0662]

[0663] Table 5

[0664] Regarding Table 5, Chen provided the following semantics:

[0665] `pic_size_different_from_max_flag` equal to 1 indicates that PPS signals a different image width or height than the referenced `max_pic_width_in_luma_samples` and `max_pic_height_in_luma_sample` in SPS. `pic_size_different_from_max_flag` equal to 0 indicates that `pic_width_in_luma_samples` and `pic_height_in_luma_sample` are the same as the referenced `max_pic_width_in_luma_samples` and `max_pic_height_in_luma_sample` in SPS.

[0666] `pic_width_in_luma_samples` specifies the width of each decoded image, in luminance samples. `pic_width_in_luma_samples` should not be equal to 0 and should be an integer multiple of `MinCbSizeY`. If `pic_width_in_luma_samples` does not exist, it is inferred to be equal to `max_pic_width_in_luma_samples`.

[0667] `pic_height_in_luma_samples` specifies the height of each decoded image, in luminance samples. `pic_height_in_luma_samples` should not be equal to 0 and should be an integer multiple of `MinCbSizeY`. If `pic_height_in_luma_samples` does not exist, it is inferred to be equal to `max_pic_height_in_luma_samples`.

[0668] For bitstream compliance requirements, the horizontal and vertical scaling ratios for each active reference image should be between 1 / 8 and 2 (inclusive). The scaling ratios are defined as follows:

[0669] -horizontal_scaling_ratio=((reference_pic_width_in_luma_samples<<14)+(pic_width_in_lu ma_samples / 2)) / pic_width_in_luma_samples

[0670] -vertical_scaling_ratio=((reference_pic_hei ght_in_luma_samples<<14)+(pic_height_in_lu ma_samples / 2)) / pic_height_in_luma_samples

[0671] Regarding motion vector scaling and subpixel position derivation, Chen provides the case where all motion vectors are normalized to the current image grid rather than their corresponding reference image grid. He also explains how, when resolution changes occur, both motion vectors and reference blocks are scaled simultaneously during motion compensation prediction, with the scaling range limited to [1 / 8, 2], i.e., magnification is limited to 1:8 and reduction to 2:1. Specifically, Chen provides the following brightness scaling process:

[0672] The scaling factor and its fixed-point representation are defined as follows:

[0673]

[0674]

[0675] The scaling process consists of two parts:

[0676] 1. Map the top-left pixel of the current block to the reference image;

[0677] 2. Use horizontal and vertical steps to address the reference positions of other pixels in the current block.

[0678] If the coordinates of the top-left pixel of the current block are (x, y), then the sub-pixel position (x', y') pointed to by the motion vector (mvX, mvY) in the reference image (1 / 16th pixel unit) is specified as follows:

[0679] The horizontal position in the reference image is

[0680] x'=((x<<4)+mvX)hori_scale_fp,

[0681] Furthermore, x' was further reduced to retain only 10 fractional places.

[0682] x'=Sign(x')((Abs(x')+(1<<7))>>8).

[0683] Similarly, the vertical position in the reference image is...

[0684] y'=((y<<4)+mvY)vert_scale_fp,

[0685] And y' was further reduced to

[0686] y'=Sign(y')((Abs(y')+(1<<7))>>8).

[0687] At this point, the reference position of the top-left pixel of the current block is at (x', y'). The positions of other reference sub-pixels / pixels are calculated using the horizontal and vertical step sizes relative to (x', y'). These step sizes are derived from the aforementioned horizontal and vertical scaling factors with a precision of 1 / 1024 pixels as follows:

[0688] x_step=(hori_scale_fp+8)>>4,

[0689] y_step=(vert_scale_fp+8)>>4.

[0690] For example, if a pixel in the current block is in column i and row j, farthest from the top-left corner pixel, then the horizontal and vertical coordinates of its corresponding reference pixel are derived by the following formula:

[0691] x'i =x'+i*x_step,

[0692] y' i =y'+j*y_step.

[0693] In subpixel interpolation, x' must be... i and y' i Decomposed into full-pixel portions and fractional pixels

[0694] part:

[0695] The full pixel portion used for addressing the reference block is equal to

[0696] (x' i +32)>>10,

[0697] (y' j +32)>>10.

[0698] The fractional pixel portion used to select the interpolation filter is equal to

[0699] Δx = ((x'i+32)>>6)&15,

[0700] Δy=f(y' y +32)>>6)&15.

[0701] Once the full-pixel and fractional-pixel positions within the reference image are determined, the existing motion-compensated interpolator can be used without any additional changes. The full-pixel positions will be used to obtain reference block patches from the reference image, and the fractional-pixel positions will be used to select the appropriate interpolation filter.

[0702] In addition, Chen provides the following chroma scaling process:

[0703] When the chroma format is 4:2:0, the chroma motion vector has 1 / 32 pixel precision. Aside from adjustments related to the chroma format, the scaling process for the chroma motion vector and chroma reference block is almost identical to the scaling process for the luma block.

[0704] The coordinates of the top-left pixel of the current chroma block are (x... C ,y C When ), the initial horizontal and vertical positions in the reference chromaticity image are:

[0705] x C '=((x C <<5)+mvX)·hori_scale_fp,

[0706] y C '=((y C<<5)+mvY)·vert_cale_fp,

[0707] Where mvX and mvY are the original luminance motion vectors, but they should now be checked with 1 / 32 pixel precision.

[0708] Further reduce x C 'and y C To maintain 1 / 1024 pixel precision

[0709] x C =Sign(x) C ')·((Abs(x C ')+(1<<8))>>9),

[0710] y C =Sign(y C ')·((Abs(y C ')+(1<<8))>>9).

[0711] Compared to the associated brightness formula, the upper right offset adds an extra bit.

[0712] The step size used is the same as that used for luminance. For a chroma pixel at (i,j) relative to the top-left pixel, the horizontal and vertical coordinates of its reference pixel are derived by the following formula:

[0713] x C ' i =x C '+i*x_step,

[0714] y C ' j =y C '+j*y_step.

[0715] In subpixel interpolation, x C ' i and y C ' j It is also broken down into full-pixel parts and fractional-pixel parts:

[0716] The full pixel portion used for addressing the reference block is equal to

[0717] (x C ' i +16)>>10,

[0718] (y C ' j +16)>>10.

[0719] The fractional pixel portion used to select the interpolation filter is equal to

[0720] Δx=((x C ' i +16)>>5)&31,

[0721] Δy=((y C ' j +16)>>5)&31.

[0722] The techniques used to enable ARC provided in Chen may be less than ideal. Specifically, according to the techniques in Chen, when the current image uses a lower resolution different from the reference image, the reference image is scaled down from a reference image stored at a higher resolution, resulting in potentially less than ideal image quality. That is, when the techniques provided in Chen are used in conjunction with the motion interpolation filter provided in JVET-N1001 described above, severe aliasing artifacts may occur, especially at relatively large scaling ratios. This disclosure describes examples of filters with low-pass characteristics that can be used in ARC use cases where the reference image is larger than the current image. Furthermore, signaling ARC parameters in Chen may be less than ideal. This disclosure describes examples of techniques for signaling ARC parameters. It should be noted that the exemplary techniques described herein can be used in ARC methods, including methods of scaling without resampling the reference image and / or creating new reference images based on reference images with different resolutions.

[0723] Figure 5 This is a block diagram illustrating an example of a system that can be configured to encode (e.g., encode and / or decode) video data according to one or more techniques of this disclosure. System 100 represents an example of a system that can perform video encoding using the partitioning techniques according to one or more techniques of this disclosure. Figure 5 As shown, system 100 includes source device 102, communication medium 110, and target device 120. Figure 5 In the example shown, source device 102 may include any device configured to encode video data and transmit the encoded video data to communication medium 110. Target device 120 may include any device configured to receive and decode the encoded video data via communication medium 110. Source device 102 and / or target device 120 may include computing devices equipped for wired and / or wireless communication, and may include set-top boxes, digital video recorders, televisions, desktop computers, laptops or tablets, game consoles, mobile devices including, for example, "smart" phones, cellular phones, personal gaming devices, and medical imaging equipment.

[0724] Communication medium 110 may include any combination of wireless and wired communication media and / or storage devices. Communication medium 110 may include coaxial cable, fiber optic cable, twisted-pair cable, wireless transmitters and receivers, routers, switches, repeaters, base stations, or any other device that can be used to facilitate communication between various devices and sites. Communication medium 110 may include one or more networks. For example, communication medium 110 may include a network configured to allow access to the World Wide Web, such as the Internet. The network may operate according to a combination of one or more telecommunications protocols. Telecommunication protocols may include proprietary aspects and / or may include standardized telecommunications protocols. Examples of standardized telecommunications protocols include the Digital Video Broadcasting (DVB) standard, the Advanced Television Systems Committee (ATSC) standard, the Integrated Services Digital Broadcasting (ISDB) standard, the Cable Data Services Interface Specification (DOCSIS) standard, the Global System for Mobile Communications (GSM) standard, the Code Division Multiple Access (CDMA) standard, the 3rd Generation Partnership Project (3GPP) standard, the European Telecommunications Standards Institute (ETSI) standard, the Internet Protocol (IP) standard, the Wireless Application Protocol (WAP) standard, and the Institute of Electrical and Electronics Engineers (IEEE) standard.

[0725] Storage devices can include any type of device or storage medium capable of storing data. Storage media can include tangible or non-transitory computer-readable media. Computer-readable media can include optical discs, flash memory, magnetic storage, or any other suitable digital storage medium. In some examples, a memory device or a portion thereof may be described as non-volatile memory, and in other examples, a portion of a memory device may be described as volatile memory. Examples of volatile memory can include random access memory (RAM), dynamic random access memory (DRAM), and static random access memory (SRAM). Examples of non-volatile memory can include magnetic hard disks, optical discs, floppy disks, flash memory, or electrically programmable memory (EPROM) or electrically erasable and programmable (EEPROM) memory. Storage devices can include memory cards (e.g., secure digital (SD) memory cards), internal / external hard disk drives, and / or internal / external solid-state drives. Data can be stored on the storage device according to defined file formats.

[0726] Refer again Figure 5Source device 102 includes a video source 104, a video encoder 106, and an interface 108. The video source 104 may include any device configured to capture and / or store video data. For example, the video source 104 may include a camera and a storage device operatively coupled thereto. The video encoder 106 may include any device configured to receive video data and generate a compliant bitstream representing the video data. A compliant bitstream may refer to a bitstream from which a video decoder can receive and reproduce video data. Aspects of a compliant bitstream may be defined according to a video coding standard. When generating a compliant bitstream, the video encoder 106 may compress the video data. Compression may be lossy (perceptible or imperceptible) or lossless. The interface 108 may include any device configured to receive a compliant video bitstream and transmit and / or store the compliant video bitstream to a communication medium. The interface 108 may include a network interface card such as an Ethernet card and may include an optical transceiver, an RF transceiver, or any other type of device capable of transmitting and / or receiving information. Furthermore, interface 108 may include a computer system interface that allows compatible video bitstreams to be stored on a storage device. For example, interface 108 may include protocols supporting Peripheral Component Interconnect (PCI) and High-Speed ​​Peripheral Component Interconnect (PCIe) bus protocols, dedicated bus protocols, Universal Serial Bus (USB) protocols, and I / O protocols. 2 C's chipset or any other logical and physical structure that can be used to interconnect peer devices.

[0727] Refer again Figure 5 The target device 120 includes an interface 122, a video decoder 124, and a display 126. Interface 122 may include any device configured to receive compatible video bitstreams from a communication medium. Interface 108 may include a network interface card such as an Ethernet card, and may include an optical transceiver, an RF transceiver, or any other type of device capable of receiving and / or transmitting information. Furthermore, interface 122 may include a computer system interface that allows retrieval of compatible video bitstreams from a storage device. For example, interface 122 may include protocols supporting PCI and PCIe bus protocols, dedicated bus protocols, USB protocols, and I / O protocols. 2 The chipset of C, or any other logical and physical structure that can be used to interconnect peer devices. The video decoder 124 may include any device configured to receive compatible bitstreams and / or acceptable variations thereof, and reproduce video data from them. The display 126 may include any device configured to display video data. The display 126 may include one of a variety of display devices such as a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or another type of display. The display 126 may include a high-definition display or an ultra-high-definition display. It should be noted that, although in Figure 3In the example shown, video decoder 124 is described as outputting data to display 126, but video decoder 124 can be configured to output video data to various types of devices and / or their sub-components. For example, video decoder 124 can be configured to output video data to any communication medium, as described herein.

[0728] Figure 6 This is a block diagram illustrating an example of a video encoder 200 that can implement the techniques described herein for encoding video data. It should be noted that although the exemplary video encoder 200 is shown as having different functional blocks, such illustrations are intended for descriptive purposes and do not limit the video encoder 200 and / or its sub-components to a particular hardware or software architecture. The functionality of the video encoder 200 can be implemented using any combination of hardware, firmware, and / or software implementations. In one example, the video encoder 200 may be configured to encode video data according to the techniques described herein. The video encoder 200 may perform intra-frame predictive coding and inter-frame predictive coding of picture regions, and thus may be referred to as a hybrid video encoder. Figure 6 In the example shown, video encoder 200 receives a source video block. In some examples, the source video block may include picture regions that have been partitioned according to the coding structure. For example, source video data may include macroblocks, CTUs, CBs, their sub-partitions, and / or additional equivalent coding units. In some examples, video encoder 200 may be configured to perform additional subdivision of the source video block. It should be noted that some of the techniques described herein are generally applicable to video coding, regardless of how the source video data is partitioned before and / or during encoding. Figure 6 In the example shown, the video encoder 200 includes a summer 202, a transform coefficient generator 204, a coefficient quantization unit 206, an inverse quantization / transformation processing unit 208, a summer 210, an intra-frame prediction processing unit 212, an inter-frame prediction processing unit 214, a filter unit 216, and an entropy coding unit 218.

[0729] like Figure 6As shown, video encoder 200 receives source video blocks and outputs a bitstream. Video encoder 200 generates residual data by subtracting a predicted video block from the source video block. Summer 202 represents the component configured to perform this subtraction operation. In one example, the subtraction of the video block occurs in the pixel domain. Transform coefficient generator 204 applies a transform, such as a Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), or a conceptually similar transform, to its residual block or sub-partition (e.g., four 8×8 transforms can be applied to a 16×16 residual value array) to generate a set of residual transform coefficients. Transform coefficient generator 204 can be configured to perform any and all combinations of transforms included in the discrete trigonometric transform series. As mentioned above, in ITU-T H.265, TB is limited to the following sizes: 4×4, 8×8, 16×16, and 32×32. In one example, the transform coefficient generator 204 can be configured to perform transforms based on arrays of sizes 4×4, 8×8, 16×16, and 32×32. In another example, the transform coefficient generator 204 can be further configured to perform transforms based on arrays of other sizes. Specifically, in some cases, performing transforms on rectangular arrays composed of different values ​​may be useful. In one example, the transform coefficient generator 204 can be configured to perform transforms based on array sizes of 2×2, 2×4N, 4M×2, and / or 4M×4N. In one example, a two-dimensional (2D) M×N inverse transform can be implemented as a one-dimensional (1D) M-point inverse transform followed by a 1D N-point inverse transform. In one example, a 2D inverse transform can be implemented as a 1D N-point vertical transform followed by a 1D N-point horizontal transform. In another example, a 2D inverse transform can be implemented as a 1D N-point horizontal transform followed by a 1D N-point vertical transform. The transform coefficient generator 204 can output transform coefficients to the coefficient quantization unit 206.

[0730] Coefficient quantization unit 206 can be configured to perform quantization of the transform coefficients. As described above, the degree of quantization can be modified by adjusting the quantization parameters. Coefficient quantization unit 206 can be further configured to determine the quantization parameters and output QP data (e.g., data for determining the quantization group size and / or incremental QP value), which the video decoder can use to reconstruct the quantization parameters to perform inverse quantization during video decoding. It should be noted that in other examples, one or more additional or alternative parameters (e.g., scaling factors) can be used to determine the quantization level. The techniques described herein can generally be applied to determine the quantization level of transform coefficients corresponding to another component of video data based on the quantization level of transform coefficients corresponding to one component of the video data.

[0731] See you again Figure 6The quantization transform coefficients are output to the inverse quantization / transform processing unit 208. The inverse quantization / transform processing unit 208 can be configured to apply inverse quantization and inverse transform to generate reconstructed residual data. Figure 6 As shown, at summer 210, reconstructed residual data can be added to the predicted video block. This allows for the reconstruction of the encoded video block, which can then be used to evaluate the coding quality of a given prediction, transform, and / or quantization. The video encoder 200 can be configured to perform multiple coding rounds (e.g., coding while changing one or more of the prediction, transform, and quantization parameters). The rate-distortion or other system parameters of the bitstream can be optimized based on the evaluation of the reconstructed video block. Furthermore, the reconstructed video block can be stored and used as a reference for predicting subsequent blocks.

[0732] As described above, intra-frame prediction can be used to encode video blocks. The intra-frame prediction processing unit 212 can be configured to select an intra-frame prediction mode for the video block to be encoded. The intra-frame prediction processing unit 212 can be configured to evaluate frames and / or regions thereof and determine the intra-frame prediction mode to be used for encoding the current block. Figure 6 As shown, the intra-frame prediction processing unit 212 outputs intra-frame prediction data (e.g., syntax elements) to the entropy coding unit 218 and the transform coefficient generator 204. As mentioned above, the transform performed on the residual data can depend on the mode. As mentioned above, possible intra-frame prediction modes can include planar prediction mode, DC prediction mode, and angle prediction mode. Furthermore, in some examples, the prediction for the chrominance component can be inferred from the intra-frame prediction used for the luma prediction mode. The inter-frame prediction processing unit 214 can be configured to perform inter-frame prediction coding for the current video block. The inter-frame prediction processing unit 214 can be configured to receive the source video block and calculate the motion vector of the PU of the video block. The motion vector can indicate the displacement of the PU (or similar coding structure) of the video block in the current video frame relative to the prediction block in the reference frame. Inter-frame prediction coding can use one or more reference pictures. Furthermore, motion prediction can be unidirectional prediction (using one motion vector) or bidirectional prediction (using two motion vectors). Inter-frame prediction processing unit 214 can be configured to select prediction blocks by calculating pixel differences determined by, for example, sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metrics. As described above, motion vectors can be determined and specified based on motion vector prediction. As described above, inter-frame prediction processing unit 214 can be configured to perform motion vector prediction. Inter-frame prediction processing unit 214 can be configured to generate prediction blocks using motion prediction data. For example, inter-frame prediction processing unit 214 can locate prediction video blocks within a frame buffer ( Figure 6(Not shown in the image). It should be noted that the inter-frame prediction processing unit 214 can be further configured to apply one or more interpolation filters to the reconstructed residual block to compute sub-integer pixel values ​​for motion estimation. The inter-frame prediction processing unit 214 can output the motion prediction data of the computed motion vectors to the entropy coding unit 218. Figure 6 As shown, the inter-frame prediction processing unit 214 can receive reconstructed video blocks via the filter unit 216.

[0733] As mentioned above, aliasing artifacts may occur when the techniques provided in Chen are used in conjunction with the motion interpolation filter provided in JVET-N1001. Referring to Tables 1A and 2A above, the motion interpolation filter in JVET-N1001 does not constitute a low-pass filter and does not provide good quality when applied to downsampled scenes. It should be noted that while the motion interpolation filter in JVET-N1001 can provide acceptable results for scaling ratios close to 1, severe aliasing artifacts will be observed at large scaling ratios. It should be noted that, according to the techniques described herein, ARC can be implemented using a two-step approach, where an additional reference image is created by resampling an existing reference image. The additional reference image can be stored in a decoded image buffer (DBP) or a temporary storage buffer. The filters described below can be used in one-step and / or two-step ARC methods to provide an additional reference image unaffected by aliasing artifacts.

[0734] In one example, one or more syntax elements are signaled in the parameter set or slice header to indicate which of the predefined filter coefficient sets will be used for interpolation filtering. By allowing such signaling, the encoding can be tuned to use different filters for the same scaling ratio, for example, based on the characteristics of the content.

[0735] In one example, one or more thresholds are used to define or determine the filter coefficients used for interpolation filtering. Such thresholds may be predefined in a parameter set or signaled.

[0736] In one example, according to the techniques described herein, if the downsampling factor is greater than 1.5:1, the following filter coefficients in Table 6A can be used in the luminance sample interpolation filtering process:

[0737]

[0738] Table 6A

[0739] It should be noted that for the exemplary filters shown in Table 6A, the sum of the filter coefficients is 128. Therefore, the resulting filter value can be obtained by dividing by 128 or alternatively by shifting left by 7.

[0740] Table 6B shows a set of alternative exemplary filter coefficients. In Table 6B, the sum of the filter coefficients is 64. Therefore, the resulting filter value can be obtained by dividing by 64 or alternatively by shifting left by 6.

[0741]

[0742] Table 6B

[0743] It should be noted that the filters in Tables 6A and 6B can be referred to as 1.5 low-scaling filters.

[0744] Table 6C provides alternative filters. It should be noted that the filter coefficients provided in Table 6C are better at preserving higher frequencies than the filters in Tables 6A and 6B. In other words, the filters provided in Table 6C can be considered sharper than those provided in Tables 6A and 6B, and can provide better downsampling results, for example, with a factor of 2.

[0745]

[0746] Table 6C

[0747] Furthermore, in one example, in addition to or alternatively, according to the techniques described herein, if the downsampling factor is greater than 3:1, the following filter coefficients in Table 7A can be used in the luminance sample interpolation filtering process:

[0748]

[0749]

[0750] Table 7A

[0751] It should be noted that for the exemplary filters in Table 7A, the sum of the filter coefficients is 128. Therefore, the resulting filter value can be obtained by dividing by 128 or alternatively by shifting left by 7.

[0752] Table 7B shows a set of alternative exemplary filter coefficients. In Table 7B, the sum of the filter coefficients is 64. Therefore, the resulting filter value can be obtained by dividing by 64 or alternatively by shifting left by 6 bits.

[0753]

[0754]

[0755] Table 7B

[0756] It should be noted that the filters in Tables 7A and 7B can be referred to as 3.0 high-scaling filters.

[0757] Furthermore, in one example, in accordance with the techniques described herein, in addition to or alternatively, if the downsampling factor is greater than 1.3:1 but less than 1.8:1, the following exemplary filter coefficients in Table 7C can be used in the luminance sample interpolation filtering process. It should be noted that the filters in Table 7C can be referred to as 1.8 medium scaling filters.

[0758]

[0759] Table 7C

[0760] In other words, according to the technique described herein, the luminance sample interpolation filtering process can conditionally select and utilize the filter coefficients used for the luminance sample interpolation filtering process based on the downsampling factor. Specifically, the luminance sample interpolation filtering described in JVET-N1001 can be modified based on whether the downsampling factor is greater than 1.5:1 and / or greater than 3:1 to conditionally utilize the filter coefficients included in Tables 1A, 6A to 6B, and / or 7A to 7B. Furthermore, the luminance sample interpolation filtering described in JVET-O2001 can be modified based on whether the downsampling factor is greater than 1.8:1 and / or greater than 3:1 to conditionally utilize the filter coefficients included in Tables 1B to 1C, 6A to 6B, and / or 7A to 7B. It should be noted that the two exemplary filters in Tables 6A and 7A are integer versions of Lanczos filters, with scaling factors of 2:1 and 4:1, and cutoff frequencies of 0.9π and 0.8π, respectively. Furthermore, either the luminance sample interpolation filter described in JVET-N1001 or JVET-O2001 can be modified based on whether the downsampling factor is greater than 1.3:1 but less than 1.8:1 to conditionally utilize the filter coefficients included in Table 7C. It should be noted that the examples in Table 7C are integer versions of the Lanczos filter with a scaling factor of 1.5:1 and a cutoff frequency of 0.95π.

[0761] In one example, according to the techniques described herein, if the downsampling factor is greater than 1.5:1, the following filter coefficients in Table 8A can be used in the chroma sample interpolation filtering process:

[0762]

[0763]

[0764] Table 8A

[0765] It should be noted that for the filters in Table 8A, the sum of the filter coefficients is 128. Therefore, the resulting filter value can be obtained by dividing by 128 or alternatively by shifting it to the left by 7.

[0766] Table 8B shows a set of exemplary alternative filter coefficients. In Table 8B, the sum of the filter coefficients is 64. Therefore, the resulting filter value can be obtained by dividing by 64 or alternatively by shifting left by 6 bits.

[0767]

[0768]

[0769] Table 8B

[0770] It should be noted that the filters in Tables 8A and 8B can be referred to as 1.5 low-scaling filters.

[0771] In one example, according to the techniques described herein, if the downsampling factor is greater than 3:1, the following filter coefficients in Table 9A can be used in the chroma sample interpolation filtering process.

[0772]

[0773]

[0774] Table 9A

[0775] It should be noted that for the exemplary filters in Table 9A, the sum of the filter coefficients is 128. Therefore, the resulting filter value can be obtained by dividing by 128 or alternatively by shifting left by 7.

[0776] Table 9B shows a set of alternative exemplary filter coefficients. In Table 9B, the sum of the filter coefficients is 64. Therefore, the resulting filter value can be obtained by dividing by 64 or alternatively by shifting left by 6.

[0777]

[0778]

[0779] Table 9B

[0780] It should be noted that the filters in Tables 9A and 9B can be referred to as 3.0 high-scaling filters.

[0781] Furthermore, in one example, according to the techniques described herein, in addition to or alternatively, if the downsampling factor is greater than 1.3:1 but less than 1.8:1, the following filter coefficients in Table 9C can be used in the luminance sample interpolation filtering process. It should be noted that the filters in Table 9C can be referred to as 1.8 medium scaling filters.

[0782]

[0783]

[0784] Table 9C

[0785] In other words, according to the technique described herein, the chroma sample interpolation filtering process can conditionally select and utilize the filter coefficients used for the chroma sample interpolation filtering process based on the downsampling factor. Specifically, the chroma sample interpolation filtering described in JVET-N1001 can be modified based on whether the downsampling factor is greater than 1.5:1 and / or greater than 3:1 to conditionally utilize the filter coefficients included in Tables 2A, 8A to 8B, and / or 9A to 9B. Furthermore, the luminance sample interpolation filtering described in JVET-O2001 can be modified based on whether the downsampling factor is greater than 1.5:1 and / or greater than 3:1 to conditionally utilize the filter coefficients included in Tables 2A, 8A to 8B, and / or 9A to 9B. Furthermore, either the chroma sample interpolation filtering described in JVET-N1001 or JVET-O2001 can be modified based on whether the downsampling factor is greater than 1.3:1 but less than 1.8:1 to conditionally utilize the filter coefficients included in Table 9C. It should be noted that, according to the techniques described herein, the coefficients used for the luminance sample interpolation filtering process and / or the chroma sample interpolation filtering process are based on the downsampling factor.

[0786] In other words, in one example, the filter coefficients used for the luminance sample interpolation filtering process and the chrominance sample interpolation filtering process can be based on the downsampling factor, as described below:

[0787] If the downsampling factor is less than 1.5, then use the typical interpolation filter;

[0788] Otherwise, if the downsampling factor > 3.0, then use a 3.0 high-scaling interpolation filter;

[0789] Otherwise, use a 1.5 low-scaling interpolation filter.

[0790] or

[0791] If the downsampling factor is less than 1.3, then use the typical interpolation filter.

[0792] Otherwise, if the downsampling factor > 1.8, a 3.0 high-scaling interpolation filter is used;

[0793] Otherwise, use a 1.8 medium-scaling interpolation filter.

[0794] If the downsampling factor is greater than 3, then use a 3.0 high-scaling interpolation filter;

[0795] Otherwise, if the downsampling filter is greater than 1.8, then use a 1.8 medium-scaling interpolation filter;

[0796] Otherwise, if the downsampling filter is greater than 1.3, then use a 1.5 low-scaling interpolation filter;

[0797] Otherwise, use the typical interpolation filter.

[0798] It should be noted that, in general, according to the techniques described herein, various downsampling factor thresholds can be defined and / or signaled for the application of scaling interpolation filters at various levels.

[0799] As described above, in JVET-O2001, affine and half-pixel AMVR inter-frame prediction modes use a 6-tap filter for interpolation filtering. According to the techniques described herein, when one or both of these inter-frame prediction modes are used with ARC, in some cases it may be desirable to use a 6-tap filter adjusted for ARC. In one example, such an adjusted filter can be implemented according to exemplary Tables 9D through 9F, where the exemplary filter in Table 9D is optimized for a 1.5:1 scaling factor, the exemplary filter in Table 9E is optimized for a 2:1 scaling factor, and the exemplary filter in Table 9F is optimized for a 4x scaling factor. In other words, in one example, according to the techniques described herein, for affine modes, the luminance sample interpolation filter described in JVET-O2001 can be modified based on a scaling factor to conditionally utilize the filter coefficients included in Tables 1C, 9D, 9E, and / or 9F, and / or for half-pixel AMVR, the luminance sample interpolation filter can be modified based on a scaling factor to conditionally utilize the filter coefficients included in Tables 1B, 9D, 9E, and / or 9F.

[0800]

[0801]

[0802] Table 9D

[0803]

[0804] Table 9E

[0805]

[0806] Table 9F

[0807] In one example, the interpolation filter coefficients for affine mode and / or half-pixel AMVR can be chosen as follows:

[0808] If the downsampling factor is less than 1.3, then use the typical interpolation filter.

[0809] Otherwise, if the downsampling factor is greater than 3, use Table 9F;

[0810] Otherwise, if the downsampling factor is >1.8, use Table 9E;

[0811] Otherwise, use Table 9D.

[0812] It should be noted that, in general, according to the techniques described herein, various downsampling factor thresholds can be defined and / or signaled to apply various levels of scaling interpolation filters for affine mode and / or half-pixel AMVR cases.

[0813] As described above, JVET-N0279 proposes a process for determining corresponding positions in reference images of different resolutions using a fixed-point representation of a scaling factor. According to the techniques described herein, the different motion-compensated interpolation filters described in Tables 1A to 2B and Tables 6A to 9F above can be conditionally used based on the scaling factor in conjunction with the ARC process described in JVET-N0279. That is, for example, using the fixed-point precision from JVET-N0279, where the scaling factor value has 14 binary decimals, the proposed thresholds would be 24576 (for 1.5:1) and 49152 (for 3:1), respectively. Therefore, according to the techniques described herein, in addition to the derivation of full-pixel and fractional-pixel positions proposed in JVET-N0279, the following can also be added to the luminance scaling process in JVET-N0279.

[0814] If scale_fp > 49152, then for xFrac... L or yFrac L For each 1 / 16 fractional sample location p, the brightness interpolation filter coefficient f L [p] is specified in Table 7A.

[0815] Otherwise, if scale_fp > 24576, then for xFrac... L or yFrac L For each 1 / 16 fractional sample location p, the brightness interpolation filter coefficient f L [p] is specified in Table 6.

[0816] Otherwise, for xFrac L or yFrac L For each 1 / 16 fractional sample location p, the brightness interpolation filter coefficient f L [p] is specified in Table 1A.

[0817] And / or

[0818] The following can be added to the chroma scaling procedure in JVET-N0279:

[0819] If scale_fp > 49152, then for xFrac... C or yFracC For each 1 / 32 fractional sample location p, the brightness interpolation filter coefficient f C [p] is specified in Table 9A.

[0820] Otherwise, if scale_fp > 24576, then for xFrac... C or yFrac C For each 1 / 16 fractional sample location p, the brightness interpolation filter coefficient f C [p] is specified in Table 8.

[0821] Otherwise, for xFrac C or yFrac C For each 1 / 16 fractional sample location p, the brightness interpolation filter coefficient f C [p] is specified in Table 2A.

[0822] In another example,

[0823] If scale_fp > 49152, then for xFrac... C or yFrac C For each 1 / 16 fractional sample location p, the brightness interpolation filter coefficient f C [p]Specified in Table 9A,

[0824] Otherwise, if scale_fp > 24576, then for values ​​equal to xFracc or yFrac C For each 1 / 16 fractional sample location p, the brightness interpolation filter coefficient f C [p] is specified in Table 8.

[0825] Otherwise, for xFrac C or yFrac C For each 1 / 16 fractional sample location p, the brightness interpolation filter coefficient f C [p] is specified in Table 2A.

[0826] Where scale_fp is the fixed-point scaling factor for the corresponding direction (horizontal or vertical).

[0827] like Figure 6 As shown, filter unit 216 receives reconstructed video blocks and encoding parameters, and outputs modified reconstructed video data. Filter unit 216 can be configured to perform deblocking, Sample Adaptive Offset (SAO) filtering, Adaptive Loop Filtering (ALF), etc. SAO filtering is a nonlinear amplitude mapping that can be used to improve reconstruction by adding an offset to the reconstructed video data. It should be noted that, as... Figure 5As shown, the intra-frame prediction processing unit 212 and the inter-frame prediction processing unit 214 can receive modified reconstructed video blocks via the filter unit 216. The entropy coding unit 218 receives quantized transform coefficients and prediction syntax data (i.e., intra-frame prediction data and motion prediction data). The entropy coding unit 218 can be configured to perform entropy coding according to one or more techniques described herein. Thus, the video encoder 200 represents an example of a device configured to determine an adaptive resolution scaling factor and, based on that adaptive resolution scaling factor, determine a motion-compensated interpolation filter according to one or more techniques of this disclosure.

[0828] As described above, Chen describes adaptive resolution variation in the signaling notification parameter set. The adaptive resolution variation in Chen's signaling notification may not be ideal. In one example, according to the techniques described herein, the syntax elements pic_width_in_luma_samples and pic_height_in_luma_samples in the SPS syntax structure can be replaced by the syntax elements pic_width_unit_minus1 and pic_height_unit_minus1, respectively, and the syntax elements max_pic_width_in_pic_width_units_minus1 and max_pic_height_in_pic_height_units_minus1 can be added to the SPS syntax structure, as provided in Table 10.

[0829]

[0830] Table 10

[0831] Regarding Table 10, in one example, the semantics of pic_width_unit_minus1, pic_height_unit_minus1, max_pic_width_in_pic_width_units_minus1, and max_pic_height_in_pic_height_units_minus1 can be based on the following:

[0832] The increment of 1 in pic_width_unit_minus1 specifies the unit used to signal the width-related syntax elements.

[0833] The increment of 1 in pic_height_unit_minus1 specifies the unit used to signal the syntax elements related to height.

[0834] The increment of max_pic_width_in_pic_width_units_minus1 by 1 specifies the maximum width of the decoded image that references SPS, in units of ((pic_width_unit_minus1+l)*MinCbSizeY) luminance samples.

[0835] The increment of max_pic_height_in_pic_height_units_minus1 by 1 specifies the maximum height of the decoded image that references SPS, in units of ((pic_height_unit_minus1-l)*MinCbSizeY) luminance samples.

[0836] Furthermore, based on the techniques described in this article, in one example, the syntax provided in Table 11 can be added to the PPS syntax structure:

[0837]

[0838] Table 11

[0839] Regarding Table 11, in one example, the semantics of pic_size_different_from_max_flag, pic_width_in_pic_width_units_minus1, and pic_height_in_luma_samples_pic_height_umt_minus1 can be based on the following:

[0840] `pic_size_different_from_max_flag` equal to 1 indicates that PPS signals a different image width or height than `max_pic_width_in_pic_width_units_minus1` and `max_pic_height_in_pic_height_units_minus1` referenced in SPS. `pic_size_different_from_max_flag` equal to 0 indicates that `pic_width_in_pic_width_units_minus1` and `pic_height_in_pic_height_units_minus1` are the same as `max_pic_width_in_pic_width_units_minus1` and `max_pic_height_in_pic_height_units_minus1` referenced in SPS.

[0841] `pic_width_in_pic_width_units_minus1` specifies the width of each decoded image, in units of ((pic_width_unit_minus1+1)*MinCbSizeY) luminance samples. If `pic_width_in_pic_width_units_minus1` does not exist, it is inferred to be equal to `max_pic_width_in_pic_width_units_minus1`.

[0842] `pic_height_in_pic_height_unit_minus1` specifies the height of each decoded image, in units of ((pic_height_unit_minus1+1)*MinCbSizeY) luminance samples. If `pic_height_in_pic_width_units_minus1` does not exist, it is inferred to be equal to `max_pic_height_in_pic_height_units_minus1`.

[0843] For bitstream compliance requirements, the horizontal and vertical scaling ratios for each active reference image should be between 1 / 8 and 2 (inclusive). The scaling ratios are defined as follows:

[0844] -horizontal_scaling_ratio=(((reference_pic_width_in_pic_width_units_minus1+1)<<14)+((pic_width_in_pic_width_units_minus1+1) / 2)) / pic_width_in_(pic_width_units_minus1+1)

[0845] -vertical_scaling_ratio=(((reference_pic_height_in_pic_height_units_minus1+1)<<14)+((pic_height_in_pic_height_units_minus1+1) / 2)) / (pic_height_in_pic_height_units_minus1+1)

[0846] Furthermore, in one example, a single `pic_size_unit` syntax element can be added instead of two new syntax elements. That is, the width and height can be specified with a single syntax element, as shown in Table 11A.

[0847]

[0848] Table 11A

[0849] `pic_size_unit` specifies the unit used to signal syntax elements related to width and height. `pic_size_unit` should not be equal to 0 and should be an integer multiple of `MinCbSizeY`.

[0850] `max_pic_width_in_pic_size_units_minus1` plus 1 specifies the maximum width of the decoded image referencing SPS, in units of `pic_size_unit` luminance samples.

[0851] `max_pic_height_in_pic_size_units_minus1` plus 1 specifies the maximum height of the decoded image referencing SPS, in units of `pic_size_unit` luminance samples.

[0852] Furthermore, in one example, a single `pic_size_unit_minus1` syntax element can be added instead of two new syntax elements. That is, the width and height can be specified with a single syntax element, an example of which is shown in Table 11B.

[0853]

[0854] Table 11B

[0855] The increment 1 in pic_size_unit_minus1 specifies the unit used to signal syntax elements related to width and height. The increment 1 in pic_size_unit_minus1 should be an integer multiple of MinCbSizeY.

[0856] The variable PicSizeUnit is exported as follows:

[0857] PicSizeUnit=(pic_size_unit_minus1+1)

[0858] `max_pic_width_injpic_size_units_minus1` plus 1 specifies the maximum width of the decoded image referencing SPS, in units of `pic_size_unit` luminance samples.

[0859] `max_pic_height_in_pic_size_umts_minus1` plus 1 specifies the maximum height of the decoded image referencing SPS, in units of `pic_size_unit` luminance samples.

[0860] In another variant, pic_size_unit_minus1 can be defined using the following semantics:

[0861] The increment of 1 in pic_size_unit_minus1 specifies the unit used to signal syntax elements related to width and height.

[0862] The variable PicSizeUnit is exported as follows:

[0863] PicSizeUnit=(pic_size_unit_minus1+1)*MinCbSizeY

[0864] In one example, the image width and height can be indicated by subtracting one encoding and using MinCbSizeY samples. In one example, any of the syntax elements provided above and indicated as having u(v) encoding can alternatively use u(v) encoding based on configuration files, layers, or level size constraints.

[0865] In one example, according to the techniques described herein, the scaling ratio can be explicitly signaled, for example, by signaling the floating-point ratio. In another example, according to the techniques described herein, the syntax provided in Table 12 can be added to the PPS syntax structure:

[0866]

[0867] Table 12

[0868] Regarding Table 12, in one example, the semantics of pic_size_different_from_max_flag, horizontal_scaling_ratio_fp, and vertical_scaling_ratio_fp can be based on the following:

[0869] `pic_size_different_from_max_flag` equal to 1 indicates that PPS signals an image width or height that is different from the maximum image width and maximum image height specified in the referenced SPS (e.g., `max_pic_width_in_pic_width_units_minus1` and `max_pic_height_in_pic_height_units_minus1`). `pic_size_different_from_max_flag` equal to 0 indicates that the image width and image height of the images applied by this PPS are the same as the maximum image width and maximum image height specified in the referenced SPS (e.g., `max_pic_width_in_pic_width_units_minus1` and `max_pic_height_in_pic_height_units_minus1`).

[0870] `horizontal_scaling_ratio_fp` specifies a fixed or floating-point scaling ratio that, when multiplied by the maximum width of the decoded image signaled in the SPS, specifies the width of the image referencing the PPS.

[0871] vertical_scaling_ratio_fp specifies a fixed or floating-point scaling ratio that, when multiplied by the maximum height of the decoded image signaled in the SPS, specifies the height of the image referencing the PPS.

[0872] In one example, according to the techniques described herein, the scaling ratio can be explicitly signaled, for example, by signaling the numerator and denominator of the ratio. In another example, according to the techniques described herein, the syntax provided in Table 13 can be added to the PPS syntax structure:

[0873]

[0874] Table 13

[0875] Regarding Table 13, in one example, the semantics of pic_size_different_from_max_flag, horizontal_scaling_ratio_numerator_minus1, horizontal_scaling_ratio_denominator_minus1, vertical_scaling_ratio_numerator_minus1, and vertical_scaling_ratio_denominator_minus1 can be based on the following:

[0876] `pic_size_different_from_max_flag` equal to 1 specifies an image width or height that is different from the maximum image width and maximum image height specified in the referenced SPS (e.g., `max_pic_width_in_pic_width_units_minus1` and `max_pic_height_in_pic_height_units_minus1`). `pic_size_different_from_max_flag` equal to 0 specifies that the image width and image height of the PPS application are the same as the maximum image width and maximum image height specified in the referenced SPS (e.g., `max_pic_width_in_pic_width_units_minus1` and `max_pic_height_in_pic_height_units_minus1`).

[0877] The increment of 1 in `horizontal_scaling_ratio_numerator_minus1` specifies the numerator of the fixed-point or floating-point scaling ratio, which, when multiplied by the maximum width of the decoded image signaled in the SPS, specifies the width of the image referencing the PPS.

[0878] The increment of 1 in `horizontal_scaling_ratio_denominator_minus1` specifies the denominator of the fixed-point or floating-point scaling ratio. This denominator, when multiplied by the maximum width of the decoded image signaled in the SPS, specifies the width of the image referencing the PPS.

[0879] vertical_scaling_ratio_numerator_minus1 plus 1 specifies the numerator of the fixed-point or floating-point scaling ratio, which specifies the height of the image referencing the PPS when multiplied by the maximum height of the decoded image signaled in the SPS.

[0880] The increment of vertical_scaling_ratio_denominator_minus1 by 1 specifies the denominator of the fixed-point or floating-point scaling ratio. This denominator, when multiplied by the maximum height of the decoded image signaled in the SPS, specifies the height of the image referencing the PPS.

[0881] In one example, according to the techniques described herein, for ARC, a signal can be sent to notify the snapping window. In another example, according to the techniques described herein, the syntax provided in Table 14 can be added to the PPS syntax structure:

[0882]

[0883] Table 14

[0884] Regarding Table 14, in one example, the semantics of various syntactic elements can be based on the following:

[0885] `pic_size_different_from_max_flag` equal to 1 indicates that PPS signals a different image width or height than `max_pic_width_in_pic_width_units_minus1` and `max_pic_height_in_pic_height_units_minus1` as specified in SPS. `pic_size_different_from_max_flag` equal to 0 indicates that the image width and height of the images used by this PPS application are the same as `max_pic_width_in_pic_width_units_minus1` and `max_pic_height_in_pic_height_units_minus1` as specified in SPS.

[0886] `pic_width_in_luma_samples` specifies the width of each decoded image, in luminance samples. `pic_width_in_luma_samples` should not be equal to 0 and should be an integer multiple of `MinCbSizeY`. If `pic_width_in_luma_samples` does not exist, it is inferred to be equal to the maximum width of the image specified in the referenced SPS (e.g., `max_pic_width_in_luma_samples`).

[0887] `pic_height_in_luma_samples` specifies the height of each decoded image, in luminance samples. `pic_height_in_luma_samples` should not be equal to 0 and should be an integer multiple of `MinCbSizeY`. If `pic_height_in_luma_samples` does not exist, it is inferred to be equal to the maximum width of the image specified in the referenced SPS (e.g., `max_pic_height_in_luma_samples`).

[0888] A value of 1 for pps_conformance_window_flag indicates that the clipping window offset parameter is immediately following the PPS parameter. A value of 0 for conformance_window_flag indicates that the clipping window offset parameter is not present.

[0889] `pps_conf_win_left_offset`, `pps_conf_win_right_offset`, `pps_conf_win_top_offset`, and `pps_conf_win_bottom_offset` specify the sample of the image output from the decoding process in CVS based on the rectangular region specified in the coordinates of the image to be output. When `conformance_window_flag` equals 0, it is inferred that the values ​​of `pps_conf_win_left_offset`, `pps_conf_win_right_offset`, `pps_conf_win_top_offset`, and `pps_conf_win_bottom_offset` are equal to 0.

[0890] The fit-to-crop window contains brightness samples with horizontal image coordinates from SubWidthC*pps_conf_win_left_offset to pic_width_in_pic_size_units*pic_size_unit-(SubWidthC*pps_conf_win_right_offset+1) and vertical image coordinates (including end values) from SubHeightC*pps_conf_win_top_offset to pic_height_in_pic_size_units*pic_size_unit-(SubHeightC*pps_conf_win_bottom_offset+1).

[0891] The value of SubWidthC*(pps_conf_win_left_offset+pps_conf_win_right_offset) should be less than pic_width_in_pic_size_units*pic_size_unit, and the value of SubHeightC*(pps_conf_win_top_offset+pps_conf_win_bottom_offset) should be less than pic_height_in_pic_size_units*pic_size_unit.

[0892] When ChromaArrayType is not equal to 0, the corresponding specified sample of the two chroma matrices is a sample with image coordinates (x / SubWidthC, y / SubHeightC), where (x, y) are the image coordinates of the specified luminance sample.

[0893] Note—The fit-to-crop window offset parameter is only applied during output. All internal decoding processes apply to the uncropped image size.

[0894] In another example according to the techniques described herein, for ARC, the snap-in window can be signaled. In one example, according to the techniques described herein, the syntax provided in Table 14A can be added to the PPS syntax structure:

[0895]

[0896] Table 14A

[0897] Regarding Table 14A, in one example, the semantics of various syntax elements can be based on the following:

[0898] `pic_size_different_from_max_flag` equal to 1 indicates that PPS signals a different image width or height than `max_pic_width_in_pic_size_units_minus1` and `max_pic_height_in_pic_size_units_minus1` referenced in SPS. `pic_size_different_from_max_flag` equal to 0 indicates that `pic_width_in_pic_size_units_minus1` and `pic_height_in_pic_size_units_minus1` are the same as `max_pic_width_in_pic_size_units_minus1` and `max_pic_height_in_pic_size_units_minus1` referenced in SPS.

[0899] `pic_width_in_pic_size_units_minus1` specifies the width of each decoded image, in units of `pic_size_unit_luma_samples` luminance samples. If `pic_width_in_pic_size_units_minus1` does not exist, it is inferred to be equal to `max_pic_width_in_pic_size_units_minus1`.

[0900] `pic_height_in_size_umt_minus1` specifies the height of each decoded image, in units of `pic_size_unit_luma_samples` luminance samples. If `pic_height_in_pic_size_units_minus1` does not exist, it is inferred to be equal to `max_pic_height_in_pic_size_units_minus1`.

[0901] A value of 1 for pps_conformance_window_flag indicates that the clipping window offset parameter is immediately following the PPS parameter. A value of 0 for conformance_window_flag indicates that the clipping window offset parameter is not present.

[0902] `pps_conf_win_left_offset`, `pps_conf_win_right_offset`, `pps_conf_win_top_offset`, and `pps_conf_win_bottom_offset` specify the sample of the image output from the decoding process in CVS based on the rectangular region specified in the coordinates of the image to be output. When `conformance_window_flag` equals 0, it is inferred that the values ​​of `pps_conf_win_left_offset`, `pps_conf_win_right_offset`, `pps_conf_win_top_offset`, and `pps_conf_win_bottom_offset` are equal to 0.

[0903] The fit-to-crop window contains brightness samples with horizontal image coordinates from SubWidthC*pps_conf_win_left_offset to pic_width_in_pic_size_units*pic_size_unit-(SubWidthC*pps_conf_win_right_offset+1) and vertical image coordinates (including end values) from SubHeightC*pps_conf_win_top_offset to pic_height_in_pic_size_units*pic_size_unit-(SubHeightC*pps_conf_win_bottom_offset+1).

[0904] The value of SubWidthC*(pps_conf_win_left_offset+pps_conf_win_right_offset) should be less than pic_width_in_pic_size_units*pic_size_unit, and the value of SubHeightC*(pps_conf_win_top_offset+pps_conf_win_bottom_offset) should be less than pic_height_in_pic_size_units*pic_size_unit.

[0905] When ChromaArrayType is not equal to 0, the corresponding specified sample of the two chroma matrices is a sample with image coordinates (x / SubWidthC,y|SubHeightC), where (x,y) are the image coordinates of the specified luminance sample.

[0906] Note—The fit-to-crop-window offset parameter is only applied during output. All internal decoding processes are applied to the uncropped image size.

[0907] In another example according to the techniques described herein, for ARC, the snap-in window can be signaled. In one example, according to the techniques described herein, the syntax provided in Table 14B can be added to the PPS syntax structure:

[0908]

[0909] Table 14B

[0910] Regarding Table 14B, in one example, the semantics of various syntax elements can be based on the following:

[0911] `pic_size_different_from_max_flag` equal to 1 indicates that PPS signals a different image width or height than `max_pic_width_in_pic_size_units_minus1` and `max_pic_height_in_pic_size_units_minus1` referenced in SPS. `pic_size_different_from_max_flag` equal to 0 indicates that `pic_width_in_pic_size_units_minus1` and `pic_height_in_pic_size_units_minus1` are the same as `max_pic_width_in_pic_size_units_minus1` and `max_pic_height_in_pic_size_units_minus1` referenced in SPS.

[0912] `pic_width_in_pic_size_units_minus1` specifies the width of each decoded image, in units of `PicSizeUnit` luminance samples. If `pic_width_in_pic_size_units_minus1` does not exist, it is inferred to be equal to `max_pic_width_in_pic_size_units_minus1`.

[0913] `pic_height_in_size_unit_minus1` specifies the height of each decoded image, in units of `PicSizeUnit` luminance samples. If `pic_height_in_pic_size_units_minus1` does not exist, it is inferred to be equal to `max_pic_height_in_pic_size_units_minus1`.

[0914] A value of 1 for `pps_conformance_window_flag` indicates that the clipping window offset parameter is immediately following the parameter in PPS. A value of 0 for `conformance_window_flag` indicates that the clipping window offset parameter does not exist. When it does not exist, the value of `pps_conformance_window_flag` is presumed to be 0.

[0915] `pps_conf_win_left_offset`, `pps_conf_win_right_offset`, `pps_conf_win_top_offset`, and `pps_conf_win_bottom_offset` specify the sample of the image output from the decoding process in CVS based on the rectangular region specified in the coordinates of the image to be output. When `pic_size_different_from_max_flag` equals 1 and `conformance_window_flag` equals 0, it is inferred that the values ​​of `pps_conf_win_left_offset`, `pps_conf_win_right_offset`, `pps_conf_win_top_offset`, and `pps_conf_win_bottom_offset` are equal to 0. When pic_size_different_from_max_flag equals 0, it is inferred that the values ​​of pps_conf_win_left_offset, pps_conf_win_right_offset, pps_conf_win_top_offset, and pps_conf_win_bottom_offset are equal to sps_conf_win_left_offset, sps_conf_win_right_offset, sps_conf_win_top_offset, and sps_conf_win_bottom_offset, respectively.

[0916] The fit-to-cropping window contains brightness samples with horizontal image coordinates from SubWidthC*pps_conf_win_left_offset to pic_width_in_pic_size_units*PicSizeUnit)-(SubWidthC*pps_conf_win_right_offset+1) and vertical image coordinates (including endpoints) from SubHeightC*pps_conf_win_top_offset to pic_height_in_pic_size_units*PicSizeUnit)-(SubHeightC*pps_conf_win_bottom_offset+1).

[0917] The value of SubWidthC*(pps_conf_win_left_offset+pps_conf_win_right_offset) should be less than pic_width_in_pic_size_units*PicSizeUnit, and the value of SubHeightC*(pps_conf_win_top_offset+pps_conf_win_bottom_offset) should be less than pic_height_in_pic_size_units*PicSizeUnit.

[0918] The variables PicOutputWidthL and PicOutputHeightL are derived as follows:

[0919] PicOutputWidthL=pic_width_in_pic_size_units*PicSizeUnit-(SubWidthC*pps_conf_win_right_offset+1)–SubWidthC*pps_conf_win_left_offset

[0920] PicOutputHeightL=pic_height_in_pic_size_units*PicSizeUnit–SubHeightC*pps_conf_wim_top_offset

[0921] When ChromaArrayType is not equal to 0, the corresponding specified sample of the two chroma matrices is a sample with image coordinates (x / SubWidthC, y / SubHeightC), where (x, y) are the image coordinates of the specified luminance sample.

[0922] Note—The fit-to-crop-window offset parameter is only applied during output. All internal decoding processes are applied to the uncropped image size.

[0923] In another variant example, a separate syntax element can be added to the PPS to indicate the image size unit. Therefore, in this case, the image size unit is indeed needed from the SPS. This allows for fully parallel and independent parsing and manipulation based on the SPS and PPS. Table 14C shows the syntax elements related to the image size unit in the PPS.

[0924]

[0925] Table 14C

[0926] Regarding Table 14C, in one example, the semantics of various syntax elements can be based on the following:

[0927] `pps_pic_size_unit_minus1plus1` specifies the unit of the syntax element used to signal the width and height of the image referencing this PPS. `pps_pic_size_unit_minus1` plus 1 should be an integer multiple of `MinCbSizeY`.

[0928] The variable PPSPicSizeUnit is exported as follows:

[0929] PPSPicSizeUnit=(pic_size_unit_minus1+1)

[0930] In another variant, pps_pic_size_unit_minus1 can be defined using the following semantics:

[0931] pps_pic_size_unit_minus1plus1 specifies the unit of the syntax element used to signal the width and height of the image that references the PPS.

[0932] The variable PPSPicSizeUnit is exported as follows:

[0933] PPSPicSizeUnit=(pic_size_unit_minus1+1)*MinCbSizeY

[0934] `pps_conf_win_left_offset`, `pps_conf_win_right_offset`, `pps_conf_win_top_offset`, and `pps_conf_win_bottom_offset` specify the sample of the image output from the decoding process in CVS based on the rectangular region specified in the coordinates of the image to be output. When `pic_size_distinct_from_max_flag` equals 1 and `conformance_window_flag` equals 0, it is inferred that the values ​​of `pps_conf_win_left_offset`, `pps_conf_win_right_offset`, `pps_conf_win_top_offset`, and `pps_conf_win_bottom_offset` are equal to 0. When pic_size_different_from_max_flag equals 0, it is inferred that the values ​​of pps_conf_win_left_offset, pps_conf_win_right_offset, pps_conf_win_top_offset, and pps_conf_win_bottom_offset are equal to sps_conf_win_left_offset, sps_conf_win_right_offset, sps_conf_win_top_offset, and sps_conf_win_bottom_offset, respectively.

[0935] The fit-to-cropping window contains brightness samples with horizontal image coordinates from SubWidthC*pps_conf_win_left_offset to pic_width_in_pic_size_units*PPSPicSizeUnit)-(SubWidthC*pps_conf_win_right_offset+1) and vertical image coordinates (including end values) from SubHeightC*pps_conf_win_top_offset to pic_height_in_pic_size_units*PPSPicSizeUnit)-(SubHeightC*pps_conf_win_bottom_offset+1).

[0936] The value of SubWidthC*(pps_conf_win_left_offset+pps_conf_win_right_offset) should be less than pic_width_in_pic_size_units*PPSPicSizeUnit, and the value of SubHeightC*(pps_conf_win_top_offset+pps_conf_win_bottom_offset) should be less than pic_height_in_pic_size_units*PPSPicSizeUnit.

[0937] The variables PicOutputWidthL and PicOutputHeightL are derived as follows:

[0938] PicOutputWidthL=pic_width_in_pic_size_units*PPSPicSizeUnit-(SubWidthC*pps_conf_win_right_offset+1)–SubWidthC*pps_conf_win_left_offset

[0939] PicOutputHeightL=pic_height_in_pic_size_units*PPSPicSizeUnit–SubHeightC*pps_conf_win_top_offset

[0940] When ChromaArrayType is not equal to 0, the corresponding specified sample of the two chroma matrices is a sample with image coordinates (x / SubWidthC, y / SubHeightC), where (x, y) are the image coordinates of the specified luminance sample.

[0941] Note that the fit-to-crop window offset parameter is only applied during output. All internal decoding processes are applied to the uncropped image size.

[0942] In one example, according to the techniques described herein, the actual output width and height can be signaled, and the encoded resolution can then be derived as a minimum resolution greater than or equal to the output resolution and a multiple of the minimum block size. In another example, according to the techniques described herein, the syntax provided in Table 14 can be added to the PPS syntax structure:

[0943]

[0944] Table 15

[0945] Regarding Table 15, in one example, the semantics of pic_size_different_from_max_fla, pic_width_in_luma_samples, and pic_height_inluma_samples can be based on the following:

[0946] `pic_size_different_from_max_flag` equal to 1 indicates that PPS signals an image width or height that is different from the maximum image width and maximum image height specified in the referenced SPS (e.g., `max_pic_width_in_pic_width_units_minus1` and `max_pic_height_in_pic_height_units_minus1`). `pic_size_different_from_max_flag` equal to 0 indicates that the image width and image height of the images applied by this PPS are the same as the maximum image width and maximum image height specified in the referenced SPS (e.g., `max_pic_width_in_pic_width_units_minus1` and `max_pic_height_in_pic_height_units_minus1`).

[0947] `pic_width_in_luma_samples` specifies the width of each decoded image, in luminance samples. `pic_width_in_luma_samples` should not be equal to 0 and should be an integer multiple of `MinCbSizeY`. If `pic_width_in_luma_samples` does not exist, it is inferred to be equal to the maximum width of the image specified in the referenced SPS (e.g., `max_pic_width_in_luma_samples`).

[0948] `pic_height_in_luma_samples` specifies the height of each decoded image, in luminance samples. `pic_height_in_luma_samples` should not be equal to 0 and should be an integer multiple of `MinCbSizeY`. If `pic_height_in_luma_samples` does not exist, it is inferred to be equal to the maximum width of the image specified in the referenced SPS (e.g., `max_pic_height_in_luma_samples`).

[0949] In one example, a scaling factor is proposed to be calculated based on the output width and output height of both the current image and the reference image. If the scaling factor is represented in a 14-bit fixed-point representation, as in JVET-N0279, then the scaling factor can be calculated as:

[0950] HoriScaleFp=(RefPicOutputWidthL<<14) / PicOutputWidthL

[0951] VertScaleFp=(RefPicOutputHeightL<<14) / PicOutputHeightL

[0952] Where RefPicOutputWidthL and RefPicOutputHeightL are the PicOutputWidthL and PicOutputHeightL of the reference image, respectively.

[0953] In one example, according to the techniques described in this article, the ARC process can be based on the following:

[0954] In the two-step method, the entire image with a resolution different from the maximum resolution at different resolutions is upsampled / downsampled and stored in the DPB.

[0955] 1. If the current image will be stored twice in the DPB at different resolutions, a flag is signaled in the slice. If so, the resampling process is invoked immediately after the image is decoded.

[0956] 2. A simple fixed resampling filter that provides good performance for 1:2 and 2:1 scaling cases.

[0957] 3. Relative to the width and height of the signal notification in SPS, both the encoded resolution and the resampled resolution must be within the range of [1 / 4, 1].

[0958] 4. If the resolution is not equal to the signal notification value in the SPS, then either the width or the height must be at most relative to the signal notification value in the SPS.

[0959] 5. The coded image can be predicted only from images with the same resolution in the DPB. If no reference image with the same resolution exists in the DPB, the coded image must be encoded intra-frame.

[0960] 6. Lower-resolution images occupy one full DPB image slot out of the total number of available image slots at full resolution. This means that memory allocation and indexing in the DPB do not need to be flexible.

[0961] 7. Which image output from the DPB can be signaled, assumed (e.g., always the highest resolution or always the encoded resolution), or controlled by external means (i.e., decoder selection)?

[0962] When using a two-step approach, general spatial scalability can be handled using the ARC scheme, which has the following differences / additions:

[0963] 1. Higher resolutions use higher layer_ids.

[0964] 2. Predicting from a higher layer_id to a lower layer_id is not allowed.

[0965] 3. Only base layer images with the same POC can be used for interlayer prediction.

[0966] 4. The basal layer image is resampled (using the ARC resampling process) and can be used to predict the current image in the higher layers.

[0967] 5. Each layer has its own DPB, and images cannot be added or removed from the DPBs of other layers.

[0968] Furthermore, in one example, according to the techniques described in this article, the ARC process can be based on the following:

[0969] In the one-step approach, the decoder image is stored in the DPB at its original resolution. When a block from the image is used as a reference, downsampling / upsampling is performed on a block-by-block basis.

[0970] The primary goal / objective of these steps is to minimize processing and implementation overhead to support ARC. It is reasonable to assume that ARC will be used where sufficient DPB space is available (e.g., low latency), so no optimization is needed to store more images when “reduced resolution images are encoded”.

[0971] The default / simplest behavior would be to mark both images as "not used for prediction" simultaneously, meaning that images of different resolutions cannot be removed without removing corresponding images of the same resolution.

[0972] The corresponding DPB processing for the one-step method:

[0973] 1. The resolution of the current image is signaled in the slice header or SPS (or PPS).

[0974] 2. After decoding, the current image is stored in the DPB without further processing.

[0975] 3. Scaling is performed by adjusting motion compensation to take into account the scaling ratio.

[0976] 4. The scaling factor can be limited to [1 / 4, 1], but it is not absolutely necessary to limit the scaling factor. The extreme case is from 360p to 8K, with a scaling factor of 12. However, the benefit is minimal compared to directly encoding 8K images as intra-frame images.

[0977] 5. The resolution can be arbitrarily close to "full resolution" because there is no cost associated with resampling.

[0978] 6. Lower-resolution images occupy one full DPB image slot out of the total number of available image slots at full resolution. This means that memory allocation and indexing in the DPB do not need to be flexible.

[0979] 7. The resolution of the output image from the DPB can be signaled, assumed (e.g., always full resolution or always encoded resolution), or controlled by external means (i.e.,

[0980] Decoder selection)

[0981] When using a one-step approach, general space scalability can be handled with the ARC scheme, which has the following differences / additions:

[0982] 1. Higher resolutions use higher layer_ids.

[0983] 2. Predicting from a higher layer_id to a lower layer_id is not allowed.

[0984] 3. The POC must be unique for all images in all layers.

[0985] 4. The “POC offset” is used at higher levels to indicate which images correspond to the same time instances as the images in the base layer (this offset may be used when scaling motion vectors in MV prediction, but it is unclear how much gain will be gained).

[0986] 5. Each layer has its own DPB, and images cannot be added or removed from the DPBs of other layers.

[0987] Thus, video encoder 200 represents an example of a device configured to signal adaptive resolution change parameters according to one or more techniques disclosed herein.

[0988] Figure 7 This is a block diagram illustrating an example of a video decoder configured to decode video data according to one or more techniques described herein. In one example, the video decoder 300 may be configured to reconstruct video data based on one or more of the techniques described above. That is, the video decoder 300 may operate in a manner reversible from the video encoder 200 described above. The video decoder 300 may be configured to perform intra-frame predictive decoding and inter-frame predictive decoding, and thus may be referred to as a hybrid decoder. Figure 7In the example shown, the video decoder 300 includes an entropy decoding unit 302, an inverse quantization unit 304, an inverse transform processing unit 306, an intra-frame prediction processing unit 308, an inter-frame prediction processing unit 310, a summer 312, a filter unit 314, and a reference buffer 316. The video decoder 300 can be configured to decode video data in a manner consistent with a video coding system that implements one or more aspects of a video coding standard. It should be noted that although the exemplary video decoder 300 shown has different functional blocks, such illustrations are intended for descriptive purposes and do not limit the video decoder 300 and / or its sub-components to a particular hardware or software architecture. The functionality of the video decoder 300 can be implemented using any combination of hardware, firmware, and / or software implementations.

[0989] like Figure 7 As shown, the entropy decoding unit 302 receives an entropy-encoded bitstream. The entropy decoding unit 302 can be configured to decode the quantization syntax elements and quantization coefficients from the bitstream according to a process that is the inverse of the entropy encoding process. The entropy decoding unit 302 can be configured to perform entropy decoding according to any of the entropy encoding techniques described above. The entropy decoding unit 302 can parse the encoded bitstream in a manner consistent with video coding standards. The video decoder 300 can be configured to parse the encoded bitstream, wherein the encoded bitstream is generated based on the techniques described above.

[0990] Refer again Figure 7 The inverse quantization unit 304 receives quantization transform coefficients (i.e., bit values) and quantization parameter data from the entropy decoding unit 302. The quantization parameter data may include any and all combinations of the aforementioned incremental QP values ​​and / or quantization group size values. The video decoder 300 and / or the inverse quantization unit 304 may be configured to determine the QP value for inverse quantization based on the value signaled by the video encoder and / or through video attributes and / or encoding parameters. That is, the inverse quantization unit 304 may operate in a manner inverse of the aforementioned coefficient quantization unit 206. For example, the inverse quantization unit 304 may be configured to infer predetermined values, allowed quantization group sizes, etc., according to the aforementioned techniques. The inverse quantization unit 304 may be configured to apply inverse quantization. The inverse transform processing unit 306 may be configured to perform an inverse transform to generate reconstructed residual data. The techniques performed by the inverse quantization unit 304 and the inverse transform processing unit 306 may be similar to the techniques performed by the aforementioned inverse quantization / transform processing unit 208. The inverse transform processing unit 306 can be configured to apply inverse DCT, inverse DST, inverse integer transform, indivisible quadratic transform (NSST), or conceptually similar inverse transform procedures to transform the coefficients in order to generate residual blocks in the pixel domain. Furthermore, as mentioned above, whether a specific transform (or the type of specific transform) is performed can depend on the intra-frame prediction mode. For example... Figure 7As shown, the reconstructed residual data can be provided to the summer 312. The summer 312 can add the reconstructed residual data to the predicted video block and generate reconstructed video data. The predicted video block can be determined based on the predicted video technique (i.e., intra-frame prediction and inter-frame prediction).

[0991] Intra-prediction processing unit 308 may be configured to receive intra-prediction syntax elements and retrieve predicted video blocks from reference buffer 316. Reference buffer 316 may include a memory device configured to store one or more video data frames. The intra-prediction syntax elements may identify intra-prediction modes, such as those described above. In one example, intra-prediction processing unit 308 may use one or more techniques from the intra-prediction coding techniques described herein to reconstruct the video block. Inter-prediction processing unit 310 may receive inter-prediction syntax elements and generate motion vectors to identify predicted blocks in one or more reference frames stored in reference buffer 316. Inter-prediction processing unit 310 may generate motion-compensated blocks, possibly performing interpolation based on interpolation filters. Identifiers for interpolation filters used for motion estimation with sub-pixel precision may be included in the syntax elements. Inter-prediction processing unit 310 may use interpolation filters to compute interpolated values ​​for sub-integer pixels of the reference blocks. That is, for example, inter-prediction processing unit 310 may use one or more of the interpolation filters described above based on one or more ARC parameters. Filter unit 314 can be configured to perform filtering on the reconstructed video data. For example, filter unit 314 can be configured to perform deblocking and / or SAO filtering, as described above with respect to filter unit 216. Furthermore, it should be noted that in some examples, filter unit 314 can be configured to perform dedicated arbitrary filtering (e.g., visual enhancement). Figure 7 As shown, the video decoder 300 can output reconstructed video blocks. Thus, the video decoder 300 represents an example of a device configured to determine an adaptive resolution scaling factor according to one or more techniques of this disclosure and to determine a motion-compensated interpolation filter based on that adaptive resolution scaling factor. Thus, the video decoder 300 represents an example of a device configured to parse syntax elements indicating adaptive resolution variation parameters according to one or more techniques of this disclosure.

[0992] In one or more examples, the functionality may be implemented by hardware, software, firmware, or any combination thereof. If implemented in software, the functionality may be stored as one or more instructions or code on or transmitted over a computer-readable medium and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium corresponding to a tangible medium such as a data storage medium, or a propagation medium that includes, for example, any medium facilitating the transfer of a computer program from one place to another according to a communication protocol. Thus, a computer-readable medium may generally correspond to: (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementing the techniques described in this disclosure. Computer program products may include computer-readable media.

[0993] By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or any other medium that can be used to store required program code in the form of instructions or data structures and that is accessible by a computer. Furthermore, any connection is appropriately referred to as a computer-readable medium. For example, coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but rather refer to non-transient tangible storage media. As used herein, disks and optical discs include compact optical discs (CDs), laser discs, optical discs, digital versatile optical discs (DVDs), floppy disks, and Blu-ray discs, where disks typically copy data magnetically, while optical discs use lasers to copy data optically. Combinations of the above should also be included within the scope of computer-readable media.

[0994] Instructions can be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the term "processor" as used herein can refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein. Furthermore, in some aspects, the functionality described herein can be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into combined codecs. Moreover, these techniques can be implemented entirely within one or more circuit or logic elements.

[0995] The techniques disclosed herein can be implemented in various devices or apparatuses, including wireless mobile phones, integrated circuits (ICs), or a set of ICs (e.g., chipsets). Various components, modules, or units are described in this disclosure to emphasize functional aspects of a device configured to perform the disclosed techniques, but they do not necessarily need to be implemented through different hardware units. Rather, as described above, various units can be combined in a codec hardware unit, or provided through an interoperable hardware unit comprising a collection of one or more processors as described above, combined with suitable software and / or firmware.

[0996] Furthermore, each functional block or feature of the base station equipment and terminal equipment used in each of the above embodiments can be implemented or executed by circuitry (typically one or more integrated circuits). Circuitry designed to perform the functions described in this specification may include general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, or combinations thereof. The general-purpose processor may be a microprocessor, or alternatively, it may be a conventional processor, controller, microcontroller, or state machine. The general-purpose processor or each of the above circuitry may be configured by digital circuitry or by analog circuitry. Furthermore, when advancements in semiconductor technology lead to the development of technologies for manufacturing integrated circuits that replace current integrated circuits, integrated circuits produced using such technologies can also be used.

[0997] Various examples have been described. These and other examples are within the scope of the following claims.

[0998] <Summary of the Invention>

[0999] In one example, a video encoding method is provided, which includes: determining an adaptive resolution scaling factor; and determining a motion-compensated interpolation filter based on the adaptive resolution scaling factor.

[1000] In one example, a method for video decoding is provided, which includes: determining an adaptive resolution scaling factor; and determining a motion-compensated interpolation filter based on the adaptive resolution scaling factor.

[1001] In one example, the method also includes signaling one or more syntax elements that indicate an adaptive resolution scaling factor and / or a motion-compensated interpolation filter.

[1002] In one example, the method also includes parsing one or more syntax elements that indicate an adaptive resolution scaling factor and / or a motion-compensated interpolation filter.

[1003] In one example, an apparatus for encoding video data is provided, the apparatus including one or more processors configured to perform any and all combinations of these steps.

[1004] In one example, the device is provided, which includes a video encoder.

[1005] In one example, the device is provided, which includes a video decoder.

[1006] In one example, a system is provided that includes: a device including a video encoder; and the device including a video decoder.

[1007] In one example, an apparatus for encoding video data includes means for performing any and all combinations of steps.

[1008] In one example, a non-transitory computer-readable storage medium is provided, which includes instructions stored thereon that, when executed, cause one or more processors of a device for encoding video data to perform any and all combinations of steps.

[1009] In one example, a method for video decoding is provided, which includes: determining a scaling factor corresponding to downsampling between a reference image and a current image; and wherein a motion-compensated interpolation filter is specified by 16 fractional sample locations and 8 interpolation filter coefficients corresponding to each of these fractional sample locations.

[1010] In one example, the method is provided where the motion-compensated interpolation filter is selected as an integer version of a Lanczos filter with a scaling factor of 1.5:1 and a cutoff frequency of 0.95π.

[1011] In one example, the method is provided, where the interpolation filter coefficients [-1,-5,17,42,17,-5,-1,0] correspond to fractional sample position 0.

[1012] In one example, a method is provided in which selecting a motion-compensated interpolation filter based on the value of a scaling factor includes selecting a motion-compensated interpolation filter when the scaling factor is greater than 1.3.

[1013] In one example, a method is provided in which selecting a motion-compensated interpolation filter based on the value of a scaling factor includes selecting a motion-compensated interpolation filter when the scaling factor is less than 1.8.

[1014] In one example, an apparatus for encoding video data is provided, the apparatus including one or more processors configured to: determine a scaling factor corresponding to downsampling between a reference image and a current image; and select a motion-compensated interpolation filter based on the value of the scaling factor, wherein the motion-compensated interpolation filter is specified by 16 fractional sample locations and 8 interpolation filter coefficients corresponding to each of these fractional sample locations.

[1015] In one example, the device is provided in which the motion-compensated interpolation filter is selected as an integer version of a Lanczos filter with a scaling factor of 1.5:1 and a cutoff frequency of 0.95π.

[1016] In one example, the device is provided, where the interpolation filter coefficients [-1,-5,17,42,17,-5,-1,0] correspond to fractional sample position 0.

[1017] In one example, the device is provided in which selecting the motion-compensated interpolation filter based on the value of the scaling factor includes selecting the motion-compensated interpolation filter when the scaling factor is greater than 1.3.

[1018] In one example, the device is provided in which selecting the motion-compensated interpolation filter based on the value of the scaling factor includes selecting the motion-compensated interpolation filter when the scaling factor is less than 1.75.

[1019] In one example, the device is provided, which includes a video decoder.

[1020] <Cross-reference>

[1021] This non-provisional patent application claims priority under 35 USC § 119 to provisional applications 62 / 865,295, filed June 23, 2019; 62 / 865,841, filed June 24, 2019; 62 / 893,791, filed August 29, 2019; and 62 / 907,612, filed September 28, 2019, the entire contents of which are incorporated herein by reference.

Claims

1. A method for decoding an encoded bitstream, the method comprising: The scaling factor is determined by using the width of the reference image and the width of the current image; as well as Based on the scaling factor value being greater than a first threshold and less than or equal to a second threshold, it is determined which filter coefficients will be used for the interpolation filter, wherein the interpolation filter is an 8-tap filter defined for 16 fractional sample positions and includes interpolation filter coefficients corresponding to each of the 16 fractional sample positions, and for each tap 0 to 7 of fractional sample position 0, the interpolation filter includes interpolation filter coefficients equal to [-1, -5, 17, 42, 17, -5, -1, 0].

2. A method for encoding video data, the method comprising: The scaling factor is determined by using the width of the reference image and the width of the current image; as well as Based on the scaling factor value being greater than a first threshold and less than or equal to a second threshold, it is determined which filter coefficients will be used for the interpolation filter, wherein the interpolation filter is an 8-tap filter defined for 16 fractional sample positions and includes interpolation filter coefficients for each of the 16 fractional sample positions, and for each tap 0 to 7 of fractional sample position 0, the interpolation filter includes interpolation filter coefficients equal to [-1, -5, 17, 42, 17, -5, -1, 0].

3. An apparatus for decoding an encoded bitstream, the apparatus comprising one or more processors configured to perform the following operations: The scaling factor is determined by using the width of the reference image and the width of the current image; and Based on the scaling factor value being greater than a first threshold and less than or equal to a second threshold, it is determined which filter coefficients will be used in the interpolation filter, wherein... The interpolation filter is an 8-tap filter defined for 16 fractional sample positions and includes interpolation filter coefficients corresponding to each of the 16 fractional sample positions. For each tap 0 to 7 of fractional sample position 0, the interpolation filter includes interpolation filter coefficients equal to [-1, -5, 17, 42, 17, -5, -1, 0].

4. An apparatus for encoding video data, the apparatus comprising one or more processors configured to perform the following operations: The scaling factor is determined by using the width of the reference image and the width of the current image; and Based on the scaling factor value being greater than a first threshold and less than or equal to a second threshold, it is determined which filter coefficients will be used in the interpolation filter, wherein... The interpolation filter is an 8-tap filter defined for 16 fractional sample positions and includes interpolation filter coefficients for each of the 16 fractional sample positions. For each tap 0 to 7 of fractional sample position 0, the interpolation filter includes interpolation filter coefficients equal to [-1, -5, 17, 42, 17, -5, -1, 0].

Images