Coding device and non-transitory computer-readable recording medium

Abstract

The invention discloses an encoding apparatus and a non-transitory computer-readable recording medium. The encoding device is configured to: receive a first array of predicted sample values; receiving a second array of predicted sample values; determining a scaling value based on the color component index value and a video sampling format of the video data; and generating a third array of predicted sample values by applying a blend value to the first array of predicted sample values and the second array of predicted sample values, wherein the blend value is based on the scaled value.

Description

[0001] This application is a divisional application of Chinese patent application "System and method for performing inter-frame predictive coding in video coding" (application number: 202080062841.9), filed on September 3, 2020. Technical Field

[0002] This disclosure relates to video coding, and more specifically to techniques for performing inter-frame prediction. Background Technology

[0003] Digital video functionality can be integrated into a wide variety of devices, including digital televisions, laptops or desktops, tablets, digital recording devices, digital media players, video game consoles, cellular phones (including so-called smartphones), medical imaging equipment, and more. Digital video can be encoded according to video coding standards. Video coding standards define the format for encapsulating and encoding compatible bitstreams of video data. A compatible bitstream is a data structure that can be received and decoded by video decoding devices to generate reconstructed video data. Video coding standards can be combined with video compression techniques. Examples of video coding standards include ISO / IEC MPEG-4 Visual and ITU-T H.264 (also known as ISO / IEC MPEG-4 AVC) and High Efficiency Video Coding (HEVC). HEVC is described in the ITU-T H.265 Recommendation of December 2016, which is incorporated herein by reference and referred to herein as ITU-T H.265. Extensions and improvements to ITU-T H.265 are currently under consideration for the development of next-generation video coding standards. For example, the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG) (collectively referred to as the Joint Video Study Group (JVET)) are working to standardize video coding technologies with compression capabilities significantly exceeding the current HEVC standard. The Joint Exploratory Model 7 (JEM7), the algorithm description of Joint Exploratory Test Model 7 (JEM 7), and the ISO / IEC JTC1 / SC29 / WG11 document: JVET-G1001 (July 2017, Turin, Italy), which are incorporated herein by reference, describe the coding features of the JVET under the Joint Test Model Study, a technology that represents a potential enhancement to video coding beyond the capabilities of ITU-T H.265. It should be noted that the coding features of JEM 7 are implemented in the JEM reference software. As used herein, the term JEM can refer collectively to the algorithms included in JEM 7 and the specific implementations in the JEM reference software. In addition, in response to the “Joint Call for Proposals on Video Compression with Capabilities beyond HEVC” jointly released by VCEG and MPEG, various groups presented multiple descriptions of video coding tools at the 10th meeting of ISO / IEC JTC1 / SC29 / WG11 held in San Diego, CA, from April 16 to 20, 2018.Based on various descriptions of video coding tools, the final initial draft text of the video coding specification is described in "Versatile Video Coding (Draft 1)," also known as document JVET-J1001-v2, presented at the 10th meeting of ISO / IEC JTC1 / SC29 / WG11 held in San Diego, California, April 16-20, 2018. This document is incorporated herein by reference and referred to as JVET-J1001. Current development of the next-generation video coding standard for VCEG and MPEG is known as the Universal Video Coding (VVC) project. "Versatile Video Coding (Draft 6)" (document JVET-O2001-vE, which is incorporated herein by reference and referred to as JVET-O2001), presented at the 15th meeting of ISO / IEC JTC1 / SC29 / WG11 held in Gothenburg, Sweden, July 3-12, 2019, represents the current version of the draft text of the video coding specification corresponding to the VVC project.

[0004] Video compression techniques reduce the data requirements for storing and transmitting video data. Video compression can reduce data requirements by utilizing the inherent redundancy in video sequences. It can further divide a video sequence into smaller, consecutive parts (i.e., a set of images within a video sequence, images within a set of images, regions within images, sub-regions within regions, etc.). Intra-frame predictive coding techniques (e.g., spatial prediction within images) and inter-frame prediction techniques (i.e., temporal techniques between images) can be used to generate the difference between the unit of video data to be encoded and a reference unit of the video data. This difference can be called residual data. The residual data can be encoded as quantized transform coefficients. Syntax elements can relate to the residual data and the reference coding unit (e.g., intra-frame predictive mode index and motion information). Entropy coding can be applied to the residual data and syntax elements. The entropy-coded residual data and syntax elements can be included in a data structure that forms a compatible bitstream. Summary of the Invention

[0005] In one example, a method for decoding video data includes: receiving a first array of predicted sample values; receiving a second array of predicted sample values; determining a scaling value based on color component index values ​​and a video sampling format of the video data; and generating a third array of predicted sample values ​​by applying a blended value to the first and second arrays of predicted sample values, wherein the blended value is based on the scaling value.

[0006] In one example, a device including one or more processors is configured to: receive a first array of predicted sample values; receive a second array of predicted sample values; determine a scaling value based on color component index values ​​and a video sampling format of video data; and generate a third array of predicted sample values ​​by applying a blended value to the first and second arrays of predicted sample values, wherein the blended value is based on the scaling value. Attached Figure Description

[0007] [ Figure 1 ] Figure 1 This is a conceptual diagram illustrating an example of a set of pictures encoded according to quadtree / multitree partitioning based on one or more techniques of this disclosure.

[0008] [ Figure 2A ] Figure 2A This is a conceptual diagram illustrating an example of encoding video data blocks according to one or more techniques disclosed herein.

[0009] [ Figure 2B ] Figure 2B This is a conceptual diagram illustrating an example of encoding video data blocks according to one or more techniques disclosed herein.

[0010] [ Figure 3 ] Figure 3 This is a conceptual diagram illustrating examples of video component sampling formats usable according to one or more techniques of this disclosure.

[0011] [ Figure 4 ] Figure 4 This is a conceptual diagram illustrating a data structure for encapsulating and encoding video data and corresponding metadata according to one or more techniques disclosed herein.

[0012] [ Figure 5 ] Figure 5 This is a block diagram illustrating an example of a system that can be configured to encode and decode video data according to one or more techniques of this disclosure.

[0013] [ Figure 6 ] Figure 6 This is a block diagram illustrating an example of a video encoder that can be configured to encode video data according to one or more techniques of this disclosure.

[0014] [ Figure 7 ] Figure 7 This is a block diagram illustrating an example of a video decoder that can be configured to decode video data according to one or more techniques of this disclosure. Detailed Implementation

[0015] Generally, this disclosure describes various techniques for encoding video data. Specifically, this disclosure describes techniques for performing inter-frame prediction. It should be noted that although the techniques disclosed herein are described with respect to ITU-T H.264, ITU-T H.265, JEM, and JVET-O2001, the techniques disclosed herein are generally applicable to video coding. For example, in addition to those techniques included in ITU-T H.265, JEM, and JVET-O2001, the coding techniques described herein can be incorporated into video coding systems (including video coding systems based on future video coding standards), including video block structures, intra-frame prediction techniques, inter-frame prediction techniques, transform techniques, filtering techniques, and / or other entropy coding techniques. Therefore, references to ITU-T H.264, ITU-T H.265, JEM, and / or JVET-O2001 are for descriptive purposes and should not be construed as limiting the scope of the techniques described herein. Furthermore, it should be noted that the inclusion of references in this paper by way of citation is for descriptive purposes and should not be construed as limiting or creating ambiguity regarding the terminology used herein. For example, where the definition of a term provided in an incorporated reference differs from that in another incorporated reference and / or as used herein, the term should be interpreted in a manner that broadly includes each corresponding definition and / or in a manner that includes each specific definition in an alternative context.

[0016] In one example, an apparatus for encoding video data includes one or more processors configured to: receive a first predictive sample value array; receive a second predictive sample value array; determine a scaling value based on color component index values ​​and the video sampling format of the video data; generate a third predictive sample value array by applying a mixing matrix to the first and second predictive sample value arrays, wherein the mixing matrix is ​​based on the scaling value; and perform video encoding using the third predictive sample value array.

[0017] In one example, a non-transitory computer-readable storage medium includes instructions stored thereon that, when executed, cause one or more processors of the device to: receive a first array of predicted sample values; receive a second array of predicted sample values; determine a scaling value based on color component index values ​​and a video sampling format of the video data; generate a third array of predicted sample values ​​by applying a blending matrix to the first and second arrays of predicted sample values, wherein the blending matrix is ​​based on the scaling value; and perform video encoding using the third array of predicted sample values.

[0018] In one example, an apparatus includes: means for receiving a first array of predicted sample values; means for receiving a second array of predicted sample values; means for determining a scaling value based on color component index values ​​and a video sampling format of video data; means for generating a third array of predicted sample values ​​by applying a mixing matrix to the first and second arrays of predicted sample values, wherein the mixing matrix is ​​based on the scaling value; and means for performing video encoding using the third array of predicted sample values.

[0019] Details of one or more examples are set forth in the following figures and description. Other features, objects, and advantages will become apparent from the description, figures, and claims.

[0020] Video content comprises a sequence of frames (or images). A series of frames may also be referred to as a group of pictures (GOP). Each video frame or image may be divided into one or more regions. A region may be defined according to a set of rules that define the region based on basic units (e.g., video blocks). For example, a rule defining a region may be that the region must be an integer number of video blocks arranged in a rectangle. Furthermore, the video blocks within a region may be ordered according to a scanning mode (e.g., raster scan). As used herein, the term "video block" may generally refer to a region of an image, or more specifically, to the largest array of sample values ​​that can be predictably encoded, its sub-partitions, and / or corresponding structures. Additionally, the term "current video block" may refer to the region of an image that is being encoded or decoded. A video block may be defined as an array of sample values. It should be noted that in some cases, pixel values ​​may be described as sample values ​​comprising the corresponding components of the video data, which may also be referred to as color components (e.g., luminance (Y) and chrominance (Cb and Cr) components or red, green, and blue components). It should be noted that in some cases, the terms "pixel value" and "sample value" are used interchangeably. Furthermore, in some cases, a pixel or sample can be referred to as a pel. A video sampling format (also known as a chroma format) can be defined by the number of chroma samples included in a video block, relative to the number of luminance samples included in the video block. For example, in a 4:2:0 sampling format, the luminance component is sampled at twice the rate of the chroma components in both the horizontal and vertical directions.

[0021] Video encoders perform predictive coding on video blocks and their sub-partitions. Video blocks and their sub-partitions can be referred to as nodes. ITU-T H.264 specifies macroblocks comprising 16×16 luma samples. That is, in ITU-T H.264, pictures are segmented into macroblocks. ITU-T H.265 specifies a similar coding tree unit (CTU) structure (which can be referred to as a maximum coding unit (LCU)). In ITU-T H.265, pictures are segmented into CTUs. In ITU-T H.265, for a picture, the CTU size can be set to include 16×16, 32×32, or 64×64 luma samples. In ITU-T H.265, a CTU consists of a corresponding coding tree block (CTB) for each component of the video data (e.g., luma (Y) and chroma (Cb and Cr)). It should be noted that a video with one luma component and two corresponding chroma components can be described as having two channels, namely, a luma channel and a chroma channel. Furthermore, in ITU-T H.265, CTUs can be partitioned according to a quadtree (QT) partitioning structure, which allows the CTU's CTB to be divided into coded blocks (CBs). That is, in ITU-T H.265, a CTU can be divided into quadtree leaf nodes. According to ITU-T H.265, a luma CB, along with two corresponding chroma CBs and associated syntax elements, is called a coding unit (CU). In ITU-T H.265, the minimum permissible size of a CB can be signaled. In ITU-T H.265, the minimum permissible size of a luma CB is 8×8 luma samples. In ITU-T H.265, the decision to code a picture region using intra-frame prediction or inter-frame prediction is made at the CU level.

[0022] In ITU-T H.265, a CU (Cubic Component Unit) is associated with a Prediction Unit (PU) structure that has its root at the CU. In ITU-T H.265, the PU structure allows the partitioning of the Luminance CB (Cubic Block) and Chromaticity CB to generate corresponding reference samples. That is, in ITU-T H.265, the Luminance CB and Chromaticity CB can be partitioned into corresponding Luminance Prediction Blocks (CBs) and Chromaticity Prediction Blocks (PBs), where each PB comprises a block of sample values ​​to which the same prediction is applied. In ITU-T H.265, a CB can be divided into one, two, or four PBs. ITU-T H.265 supports PB sizes from 64×64 samples down to 4×4 samples. In ITU-T H.265, square PBs are supported for intra-frame prediction, where a CB can form a PB or can be partitioned into four square PBs. In addition to square PBs, ITU-T H.265 also supports rectangular PBs for inter-frame prediction, where a CB can be halved vertically or horizontally to form a PB. Furthermore, it should be noted that in ITU-T H.265, for inter-frame prediction, four asymmetric PB partitions are supported, where the CB is divided into two PBs at one-quarter of the height (top or bottom) or width (left or right) of the CB. Intra-frame prediction data (e.g., intra-frame prediction mode syntax elements) or inter-frame prediction data (e.g., motion data syntax elements) corresponding to the PB are used to generate reference and / or prediction sample values ​​for the PB.

[0023] JEM specifies a CTU with a maximum size of 256×256 luminance samples. JEM specifies a Quadtree Plus Binary Tree (QTBT) block structure. In JEM, the QTBT structure allows the quadtree leaf nodes to be further divided by a binary tree (BT) structure. That is, in JEM, the binary tree structure allows the quadtree leaf nodes to be recursively divided vertically or horizontally. In JVET-O2001, CTUs are divided according to a Quadtree Plus Multi-Type Tree (QTMT or QT+MTT) structure. QTMT in JVET-O2001 is similar to QTBT in JEM. However, in JVET-O2001, in addition to indicating binary partitioning, the multi-type tree can also indicate so-called ternary (or ternary tree (TT)) partitioning. Ternary partitioning divides a block vertically or horizontally into three blocks. In the case of a vertical TT split, the block is divided at one-quarter of its width from the left edge and at one-quarter of its width from the right edge; and in the case of a horizontal TT split, the block is divided at one-quarter of its height from the top edge and at one-quarter of its height from the bottom edge. See again. Figure 1 , Figure 1 This illustrates an example where a CTU is partitioned into quadtree leaf nodes, and these quadtree leaf nodes are further partitioned based on either BT or TT partitioning. That is, in Figure 1 In the diagram, dashed lines indicate additional binary and ternary partitions in a quadtree.

[0024] As described above, each video frame or picture can be divided into one or more regions. For example, according to ITU-T H.265, each video frame or picture can be divided into one or more slices, and further divided into one or more tiles, wherein each slice includes a sequence of CTUs (e.g., arranged in raster scan order), and wherein a tile is a sequence of CTUs corresponding to a rectangular area of ​​the picture. It should be noted that, in ITU-T H.265, a slice is a sequence of one or more slice segments that begin with an independent slice segment and include all subsequent subordinate slice segments (if any) preceding the next independent slice segment (if any). A slice segment (such as a piece) is a sequence of CTUs. Therefore, in some cases, the terms "slice" and "slice segment" are used interchangeably to refer to a sequence of CTUs arranged in raster scan order. Furthermore, it should be noted that, in ITU-T H.265, a tile may consist of CTUs contained in more than one slice, and a slice may consist of CTUs contained in more than one tile. However, ITU-T H.265 specifies that one or both of the following conditions must be met: (1) all CTUs in a slice belong to the same tile; and (2) all CTUs in a tile belong to the same slice.

[0025] Regarding JVET-O2001, slices need to consist of an integer number of tiles, not just an integer number of CTUs. In JVET-O2001, a tile is a rectangular row region of CTUs within a specific tile in an image. Furthermore, in JVET-O2001, a tile can be divided into multiple tiles, each tile consisting of one or more rows of CTUs within the tile. Tiles not divided into multiple tiles are also referred to as tiles. However, tiles that are a proper subset of a tile are not referred to as tiles. Therefore, some video coding techniques may or may not support slices comprising a set of CTUs that do not form an image. Additionally, it should be noted that in some cases, slices may need to consist of an integer number of complete tiles, and in such cases, the slice is referred to as a tile group. The techniques described herein are applicable to tiles, slices, tiles, and / or tile groups. Figure 1 This is a concept diagram illustrating an example of a group of images including slices. Figure 1 In the example shown, Pic3 is depicted as comprising two slices (i.e., slice 0 and slice 1). Figure 1 In the example shown, slice 0 includes one brick, namely brick 0, and slice 1 includes two bricks, namely brick 1 and brick 2. It should be noted that in some cases, slice 0 and slice 1 may meet the requirements of a tile and / or a tile group and be classified as a tile and / or a tile group.

[0026] The number of chroma samples included in a CU can be defined relative to the number of luma samples included in the CU. For example, in a 4:2:0 sampling format, the sampling rate of the luma component is twice the sampling rate of the chroma components in both the horizontal and vertical directions. Therefore, for a CU formatted according to the 4:2:0 format, the width and height of the sample arrays used for the luma components are twice the width and height of each sample array used for the chroma components. Figure 3 This is a conceptual diagram illustrating an example of a coding unit formatted according to the 4:2:0 sample format. Figure 3 This shows the relative positions of the chromaticity samples with respect to the luminance samples within the CU. As mentioned above, the CU is typically defined based on the number of horizontal and vertical luminance samples. Therefore, as... Figure 3 As shown, the 16×16 CU, formatted according to the 4:2:0 sample format, includes 16×16 samples for the luma component and 8×8 samples for each chroma component. Furthermore, in Figure 3 The example shown illustrates the relative positions of chroma samples to luma samples for adjacent video blocks in a 16×16 CU. For a CU formatted in 4:2:2 format, the width of the luma component sample array is twice the width of the chroma component sample array, but the height of the luma component sample array is equal to the height of the chroma component sample array. Furthermore, for a CU formatted in 4:4:4 format, the luma component sample array has the same width and height as the chroma component sample array.

[0027] Table 1 shows how to specify the chroma format in JVET-O2001 based on the values ​​of the syntax elements chroma_format_idc and separate_colour_plane_flag included in JVET-O2001. Furthermore, Table 1 shows how to derive the variables SubWidthC and SubHeightC based on the chroma format. SubWidthC and SubHeightC are used, for example, for unblocking. Regarding Table 1, JVET-O2001 provides the following specifications:

[0028] In monochrome sampling, there is only one sample array, which is nominally regarded as a luminance array.

[0029] In 4:2:0 sampling, each of the two chroma arrays has half the height and half the width of the luminance array.

[0030] In 4:2:2 sampling, each of the two chroma arrays has the same height and half the width of the luminance array.

[0031] In 4:4:4 sampling, the following applies based on the value of separate_colour_plane_flag:

[0032] - If separate_colour_plane_flag equals 0, then each of the two chromaticity arrays has the same height and width as the luminance array.

[0033] Otherwise (separate_colour_plane_flag equals 1), the three color planes are processed separately as monochrome sampled images.

[0034]

[0035] It should be noted that for sampling formats, such as the 4:2:0 sample format, the chroma position type can be specified. That is, for example, for the 4:2:0 sample format, horizontal and vertical offset values ​​indicating relative spatial positioning can be specified for the chroma samples relative to the luminance samples. Table 2 provides the definitions of HorizontalOffsetC and VerticalOffsetC for the five chroma position types provided in JVET-O2001.

[0036]

[0037] For intra-frame predictive coding, the intra-frame prediction mode can specify the location of a reference sample within the image. In ITU-T H.265, the defined possible intra-frame prediction modes include planar (i.e., surface-fitting) prediction modes, DC (i.e., flat global average) prediction modes, and 33 angular prediction modes (predMode: 2-34). In JEM, the defined possible intra-frame prediction modes include planar prediction modes, DC prediction modes, and 65 angular prediction modes. It should be noted that planar prediction modes and DC prediction modes can be referred to as non-directional prediction modes, and angular prediction modes can be referred to as directional prediction modes. It should be noted that the techniques described herein are generally applicable regardless of the number of defined possible prediction modes.

[0038] For inter-frame predictive coding, a reference picture is determined, and motion vectors (MVs) identify samples in that reference picture used to generate predictions for the current video block. For example, reference sample values ​​located in one or more previously encoded pictures can be used to predict the current video block, and motion vectors are used to indicate the position of the reference block relative to the current video block. Motion vectors can describe, for example, the horizontal displacement component of the motion vector (i.e., MV). x ), the vertical displacement component of the motion vector (i.e., MV) yThe resolution of the motion vectors (e.g., quarter-pixel precision, half-pixel precision, one-pixel precision, two-pixel precision, four-pixel precision) is used. Previously decoded images (which may include images output before or after the current image) can be organized into one or more lists of reference images and identified using reference image index values. Furthermore, in inter-frame predictive coding, single prediction refers to generating a prediction using sample values ​​from a single reference image, while dual prediction refers to generating a prediction using corresponding sample values ​​from two reference images. That is, in single prediction, a single reference image and its corresponding motion vector are used to generate a prediction for the current video block, while in dual prediction, a first reference image and its corresponding first motion vector, and a second reference image and its corresponding second motion vector are used to generate a prediction for the current video block. In dual prediction, the corresponding sample values ​​are combined (e.g., added, rounded, and cropped, or averaged according to weights) to generate a prediction. Images and their regions can be classified based on which types of prediction patterns are available for encoding their video blocks. In other words, for regions of type B (e.g., B slices), dual prediction, single prediction, and intra-prediction modes can be used; for regions of type P (e.g., P slices), single prediction and intra-prediction modes can be used; and for regions of type I (e.g., I slices), only intra-prediction mode can be used. As described above, reference images are identified by reference indices. For example, for P slices, a single reference image list RefPicList0 can exist, and for B slices, in addition to RefPicList0, a second independent reference image list RefPicList1 can exist. It should be noted that for single prediction in B slices, either RefPicList0 or RefPicList1 can be used to generate the prediction. Furthermore, it should be noted that during the decoding process, at the start of decoding an image, a reference image list is generated from previously decoded images stored in the Decoding Image Buffer (DPB).

[0039] Furthermore, the coding standard supports various motion vector prediction modes. Motion vector prediction enables the derivation of motion vector values ​​for the current video block based on another motion vector. For example, a set of candidate blocks with associated motion information can be derived from the spatially and temporally adjacent blocks of the current video block. Additionally, the generated (or default) motion information can be used for motion vector prediction. Examples of motion vector prediction include Advanced Motion Vector Prediction (AMVP), Temporal Motion Vector Prediction (TMVP), the so-called "merge" mode, and "skip" and "direct" motion inference. Other examples of motion vector prediction include Advanced Temporal Motion Vector Prediction (ATMVP) and Spatial-Temporal Motion Vector Prediction (STMVP). For motion vector prediction, both the video encoder and video decoder perform the same process to derive a set of candidates. Therefore, for the current video block, the same set of candidates is generated during encoding and decoding.

[0040] As mentioned above, for inter-frame predictive coding, reference samples from previously encoded images are used to encode video blocks in the current image. The previously encoded image that can be used as a reference when encoding the current image is called the reference image. It should be noted that the decoding order does not necessarily correspond to the image output order, i.e., the temporal order of images in the video sequence. In ITU-T H.265, when an image is decoded, it is stored in a decoded image buffer (DPB) (which may be called a frame buffer, reference buffer, reference image buffer, etc.). In ITU-T H.265, images stored in the DPB are removed from the DPB when output and are no longer needed for encoding subsequent images. In ITU-T H.265, after decoding the slice header, i.e., at the start of image decoding, a determination is made once for each image whether it should be removed from the DPB. For example, the reference... Figure 1 Pic3 is shown with reference to Pic2. Similarly, Pic4 is shown with reference to Pic1. Regarding Figure 1Assuming the number of images corresponds to the decoding order, the DPB will be populated as follows: After decoding Pic1, the DPB will include {Pic1}; at the start of decoding Pic2, the DPB will include {Pic1}; after decoding Pic2, the DPB will include {Pic1, Pic2}; at the start of decoding Pic3, the DPB will include {Pic1, Pic2}. Then, Pic3 will be decoded with reference to Pic2, and after decoding Pic3, the DPB will include {Pic1, Pic2, Pic3}. At the start of decoding Pic4, images Pic2 and Pic3 will be marked for removal from the DPB because they are not required for decoding Pic4 (or any subsequent images, not shown), and assuming Pic2 and Pic3 have already been output, the DPB will be updated to include {Pic1}. Pic4 will then be decoded with reference to Pic1. The process of marking images to remove them from the DPB can be called Reference Picture Set (RPS) management.

[0041] As described above, intra-frame prediction data or inter-frame prediction data is used to generate reference sample values ​​for blocks of sample values. The difference between sample values ​​included in the current PB or another type of picture region structure and the associated reference samples (e.g., those generated using prediction) can be referred to as residual data. Residual data can include a corresponding array of differences corresponding to each component of the video data. The residual data may be in the pixel domain. Transformations such as Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), integer transform, wavelet transform, or conceptually similar transforms can be applied to the array of differences to generate transform coefficients. It should be noted that in ITU-T H.265 and JVET-O2001, the CU is associated with a Transform Unit (TU) structure having its root at the CU level. That is, to generate transform coefficients, the array of differences can be partitioned (e.g., four 8×8 transforms can be applied to a 16×16 residual array). Such a subdivision of the differences for each component of the video data can be referred to as a Transform Block (TB). It should be noted that in some cases, a core transform and a subsequent second transform can be applied (in a video encoder) to generate transform coefficients. For a video decoder, the order of the transforms is reversed.

[0042] Quantization can be performed directly on transform coefficients or residual sample values ​​(e.g., in the case of palette-encoded quantization). Quantization approximates transform coefficients by limiting the amplitude to a specified set of values. Quantization essentially scales the transform coefficients to change the amount of data needed to represent a set of transform coefficients. Quantization may include dividing the transform coefficient (or the value obtained by adding an offset value to the transform coefficient) by a quantization scaling factor and any associated rounding function (e.g., rounding to the nearest integer). The quantized transform coefficients may be referred to as coefficient bit values. Inverse quantization (or “dequantization”) may include multiplying the coefficient bit value by the quantization scaling factor, and any reciprocal rounding or offset addition operations. It should be noted that, as used herein, the term quantization process may refer in some cases to division by a scaling factor to generate a bit value, and in some cases to multiplication by a scaling factor to recover the transform coefficients. That is, quantization process may refer to quantization in some cases and inverse quantization in others. Furthermore, it should be noted that although some examples below describe quantization processes for arithmetic operations related to decimal notation, such descriptions are for illustrative purposes and should not be construed as limiting. For example, the techniques described herein can be implemented in devices using binary arithmetic, etc. For example, the multiplication and division operations described herein can be implemented using bit shifting operations, etc.

[0043] Entropy coding techniques can be used to entropy-encode quantized transform coefficients and syntax elements (e.g., syntax elements indicating the coding structure of video blocks). The entropy coding process involves encoding the syntax element values ​​using a lossless data compression algorithm. Examples of entropy coding techniques include Content Adaptive Variable Length Coding (CAVLC), Context Adaptive Binary Arithmetic Coding (CABAC), Probability Interval Partition Entropy Coding (PIPE), etc. The entropy-encoded quantized transform coefficients and the corresponding entropy-encoded syntax elements can form a compatible bitstream that can be used to reproduce video data at the video decoder. The entropy coding process, such as CABAC, may include binarizing the syntax elements. Binarization is the process of converting the values ​​of syntax elements into a sequence of one or more bits. These bits may be referred to as "bins". Binarization may include one or a combination of the following coding techniques: fixed-length coding, unary coding, truncated unary coding, truncated Rice coding, Golomb coding, k-order exponential Golomb coding, and Golomb-Rice coding. For example, binarization may include representing the integer value 5 of a syntax element as 00000101 using an 8-bit fixed-length binarization technique, or representing the integer value 5 as 11110 using a unary coding binarization technique. As used herein, each of the terms fixed-length coding, unary coding, truncated unary coding, truncated Rice coding, Golomb coding, k-order exponential Golomb coding, and Golomb-Rice coding may refer to a general implementation of these techniques and / or a more specific implementation of these coding techniques. For example, a Golomb-Rice coding implementation may be specifically defined according to a video coding standard. In the CABAC example, for a particular bin, the context provides the bin's maximum probability state (MPS) value (i.e., the bin's MPS is either 0 or 1), and the probability value that the bin is either the MPS or the minimum probability state (LPS). For example, the context may indicate that the bin's MPS is 0 and the probability that the bin is 1 is 0.3. It should be noted that the context may be determined based on the values ​​of bins in the previous encodings of the current syntax element and bins in the previously encoded syntax elements. For example, the value of a syntax element associated with an adjacent video block can be used to determine the context of the current bin.

[0044] The following arithmetic operators can be used for the formulas used in this article:

[0045]

[0046] In addition, the following logical operators can be used:

[0047] x&&y The Boolean logic "and" of x and y

[0048] The Boolean OR operation between x and y.

[0049] !Boolean logic "No"

[0050] x?y:z If x is TRUE or not equal to 0, evaluate to y; otherwise, evaluate to z.

[0051] In addition, the following relational operators can be used:

[0052]

[0053] In addition, the following bitwise operators can be used:

[0054]

[0055] In addition, the following assignment operators can be used:

[0056]

[0057] In addition, the following mathematical functions can be used:

[0058]

[0059] Floor(x), the largest integer less than or equal to x.

[0060] Log2(x) is the base-2 logarithm of x.

[0061]

[0062]

[0063] Round(x) = Sign(x) * Floor(Abs(x)+0.5)

[0064]

[0065] Furthermore, it should be noted that the following descriptors can be applied among the grammatical descriptors used in this paper:

[0066] - b(8): A byte (8 bits) with any bit string pattern. The parsing process for this descriptor is specified by the return value of the function read_bit(8).

[0067] - f(n): A fixed-pattern bit string written using n bits (from left to right), starting from the leftmost bit. The parsing process for this descriptor is specified by the return value of the function read_bit(n).

[0068] - se(v): Syntax element of signed integer 0-order Exp-Golomb encoding, starting from the leftmost bit.

[0069] - tb(v): A truncated binary code using at most maxVal bits, where maxVal is defined in the semantics of the syntax element.

[0070] - tu(v): A truncated unary code using at most maxVal bits, where maxVal is defined in the semantics of the syntax element.

[0071] - u(n): Uses an n-bit unsigned integer. When n is "v" in the syntax table, the number of bits varies depending on the values ​​of other syntax elements. The parsing process for this descriptor is specified by the return value of the function read_bits(n), which is interpreted as a binary representation of the unsigned integer, with the most significant bit written first.

[0072] - ue(v): Syntax element of unsigned integer 0-order Exp-Golomb encoding, starting from the leftmost bit.

[0073] Figures 2A to 2B This is a conceptual diagram illustrating an example of encoding video data blocks. (Example:) Figure 2A As shown, bit-level values ​​are generated by subtracting a set of predicted values ​​from the current video data block to produce a residual, performing a transformation on the residual, and quantizing the transform coefficients. The current block of video data (e.g., the CB corresponding to a video component) is then encoded. The bit-level values ​​are encoded into a bitstream. (See diagram.) Figure 2B As shown, the current video data block is decoded by performing inverse quantization on the bit-order values, performing an inverse transform, and adding a set of predicted values ​​to the resulting residual. It should be noted that, in Figures 2A to 2B In the example, the sample values ​​of the reconstructed block differ from the sample values ​​of the current video block being encoded. Thus, the encoding can be considered lossy. However, the difference in sample values ​​can be considered acceptable or imperceptible to the viewer of the reconstructed video.

[0074] In addition, such as Figures 2A to 2B As shown, a scaling factor array is used to generate coefficient bit-order values. In ITU-T H.265, the scaling factor array is generated by selecting a scaling matrix and multiplying each entry in the scaling matrix by a quantization scaling factor. The scaling matrix can be selected in part based on the prediction mode and color components. It should be noted that in some examples, the scaling matrix can provide the same value for each entry (i.e., scaling all coefficients by a single value). The value of the quantization scaling factor can be determined by the quantization parameter QP. Furthermore, the QP values ​​of a set of transform coefficients can be derived using predicted quantization parameter values ​​(which may be referred to as predicted QP values ​​or QP prediction values) and optionally signaled quantization parameter increment values ​​(which may be referred to as QP increment values ​​or incremented QP values). The quantization parameters can be updated for each CU, and the corresponding quantization parameters can be derived for each of the luminance and chrominance channels.

[0075] refer to Figure 1 In the example shown, each video data slice included in Pic3 (i.e., slice 0, slice 1) is shown as encapsulated in a NAL unit. In JVET-O2001, each of the video sequence, GOP, picture, slice, and CTU can be associated with metadata describing video coding attributes. JVET-O2001 defines parameter sets that can be used to describe video data and / or video coding attributes. Specifically, JVET-O2001 includes five types of parameter sets: Decoding Parameter Set (DPS), Video Parameter Set (VPS), Sequence Parameter Set (SPS), Picture Parameter Set (PPS), and Adaptive Parameter Set (APS). In JVET-O2001, parameter sets can be encapsulated as special types of NAL units or can be signaled as messages. NAL units that include encoded video data (e.g., slices) can be referred to as VCL (Video Coding Layer) NAL units, and NAL units that include metadata (e.g., parameter sets) can be referred to as non-VCL NAL units. Furthermore, JVET-O2001 enables the signaling of Supplemental Enhancement Information (SEI) messages. In JVET-O2001, SEI messages assist in processes related to decoding, display, or other purposes; however, SEI messages may not be necessary to construct luminance or chrominance samples during the decoding process. In JVET-O2001, SEI messages can be signaled in the bitstream using non-VCL NAL units. Additionally, SEI messages can be transmitted in a manner other than existing within the bitstream (i.e., signaled out-of-band). Figure 4 An example of a bitstream comprising multiple CVSs is shown, where each CVS is represented by a NAL unit included in its respective access unit. Figure 4 In the example shown, the non-VCL NAL unit includes the corresponding parameter set NAL unit (i.e., sequence parameter set (SPS) and image parameter set (PPS) NAL unit), SEI message NAL unit, and access unit delimiter NAL unit. It should be noted that in Figure 4 In this context, HEADER is the header for NAL units.

[0076] As mentioned above, for inter-frame predictive coding, reference sample values ​​located in one or more previously coded images can be used to predict the current video patch. JVET-O2001 includes so-called triangle-based motion compensation. In triangle-based motion compensation, two triangle predictions are used to predict a rectangular video patch. That is, for prediction, the rectangular video patch is divided into two triangles about the diagonal (from the top left corner to the bottom right corner) or about the reverse diagonal (from the top right corner to the bottom left corner). Each triangle prediction can be generated using the motion vector of each triangle itself and the reference frame index. Furthermore, a rectangular prediction for predicting the rectangular video patch can be generated by performing an adaptive weighting process on the diagonal edges of adjacent triangle predictions. That is, triangle predictions can be mixed. A detailed description of an example of triangle-based motion compensation is provided in “CE10.3.1.b: Triangular prediction unitmode” (document JVET-L0124-v2), presented at the 12th meeting of ISO / IEC JTC1 / SC29 / WG11 held in Macau, China from October 3 to 12, 2018.

[0077] JVET-O2001 specifies the following decoding procedure for triangular inter-frame blocks:

[0078] The input to this process is:

[0079] - Luminance position (xCb, yCb), which specifies the top-left sample of the current coding block relative to the top-left luminance sample of the current image.

[0080] - The variable cbWidth specifies the width of the current coded block in the luminance sample.

[0081] - The variable cbHeight specifies the height of the current coded block in the luminance sample.

[0082] Luminance motion vectors mvA and mvB with a fractional sample accuracy of -1 / 16.

[0083] - Chromaticity motion vectors mvCA and mvCB

[0084] -Refer to indices refIdxA and refIdxB,

[0085] - Predicted list flags predListFlagA and predListFlagB.

[0086] The output of this process is:

[0087] - The (cbWidth)×(cbHeight) array of brightness prediction samples, predSamples L ,

[0088] - array predSamples of (cbWidth / SubWidthC) × (cbHeight / SubHeightC) for chromaticity prediction samples of component Cb Cb ,

[0089] - array predSamples of (cbWidth / SubWidthC) × (cbHeight / SubHeightC) for chromaticity prediction samples of component Cr Cr .

[0090] Let predSamplesLA L and predSamplesLB L It is a (cbWidth)×(cbHeight) array of predicted brightness sample values, predSamplesLA Cb ,predSamplesLB Cb ,predSamplesLA Cr and predSamplesLB Cr It is a (cbWidth / SubWidthC)×(cbHeight / SubHeightC) array that predicts the chromaticity sample values.

[0091] Export predSamples using the following ordered steps L ,predSamples Cb and predSamples Cr :

[0092] 1. For N, which is either A or B, the following applies:

[0093] - A reference image is derived by calling a specified reference image selection process with X set to equal predListFlagN and refIdxX set to equal refIdxN as input. This reference image consists of an ordered two-dimensional array of luminance samples, refPicLN. L Two ordered two-dimensional arrays refPicLN for chromaticity samples Cb and refPicLN Cr composition.

[0094] - By calling the luminance block width sbWidth (with luminance position (xCb, yCb), luminance block height sbHeight (set to cbWidth), motion vector offset mvOffset (set to (0, 0)), motion vector mvLX (set to mvN), and refPicLN (set to refPicLN).L The array predSamplesLNL is derived by using the reference array refPicLXL, the variable bdofFlag set to FALSE, and the variable cIdx set to 0 as inputs to a specified fractional sample interpolation procedure.

[0095] - By calling the code block width sbWidth (with luminance position (xCb, yCb), set to cbWidth / SubWidthC, the code block height sbHeight (set to cbHeight / SubHeightC), the motion vector offset mvOffset (set to (0, 0)), the motion vector mvLX (set to mvCN), and the refPicLN (set to refPicLN) Cb The reference array refPicLX Cb The array predSamplesLN is derived by using a specified fractional sample interpolation procedure with variables bdofFlag set to FALSE and cIdx set to 1 as inputs. Cb .

[0096] - By calling the code block width sbWidth (with luminance position (xCb, yCb), set to cbWidth / SubWidthC, the code block height sbHeight (set to cbHeight / SubHeightC), the motion vector offset mvOffset (set to (0, 0)), the motion vector mvLX (set to mvCN), and the refPicLN (set to refPicLN) Cr The reference array refPicLX Cr The array predSamplesLNcr is derived by using a specified fractional sample interpolation procedure with the variable bdofFlag set to FALSE and the variable cIdx set to 2 as inputs.

[0097] 2. Set the splitting direction of the merge triangle mode variable triangleDir to equal merge_triangle_split_dir[xCb][yCb][which indicates the diagonal or anti-diagonal direction].

[0098] 3. Export the predicted samples (predSamples) within the current luminance coding block by calling the weighted sample prediction procedure specified below for the triangular merging mode. L [x L ][y L ], x L = 0..cbWidth – 1 and y L= 0..cbHeight –1, where nCbW is the block width equal to cbWidth, nCbH is the block height equal to cbHeight, and the sample array predSamplesLA is... L and predSamplesLB L The variables triangleDir and cIdx, which is equal to 0, are used as inputs.

[0099] 4. Export the predicted samples (predSamples) within the current chroma component Cb encoding block by calling the weighted sample prediction procedure specified below for the triangular merging mode. Cb [xc][yc], xc = 0..cbWidth / SubWidthC – 1 and yc = 0..cbHeight / SubHeightC – 1, where the coded block width nCbW is set to be equal to cbWidth / SubWidthC, the coded block height nCbH is set to be equal to cbHeight / SubHeightC, and the sample array predSamplesLA is set to... Cb and predSamplesLB Cb The variables triangleDir and cIdx, which is equal to 1, are used as inputs.

[0100] 5. Export the predicted samples (predSamples) within the current chroma component Cr encoding block by calling the weighted sample prediction procedure specified below for the triangular merging mode. Cr [xc][yc], xc = 0..cbWidth / SubWidthC – 1 and yc = 0..cbHeight / SubHeightC – 1, where the coded block width nCbW is set to be equal to cbWidth / SubWidthC, the coded block height nCbH is set to be equal to cbHeight / SubHeightC, and the sample array predSamplesLA is set to... Cr and predSamplesLB Cr The variables triangleDir and cIdx, which is equal to 2, are used as inputs.

[0101] 6. Use the luma code block position (xCb, yCb), luma code block width cbWidth, luma code block height cbHeight, division direction triangleDir, luma motion vectors mvA and mvB, reference indices refIdxA and refIdxB, and prediction list flags predListFlagA and predListFlagB as input to call the motion vector storage procedure for merging triangular patterns.

[0102] Weighted Sample Prediction Process for Triangular Merging Pattern

[0103] The input to this process is:

[0104] - Two variables, nCbW and nCbH, specify the width and height of the current code block.

[0105] - Two (nCbW)×(nCbH) arrays, predSamplesLA and predSamplesLB,

[0106] - The variable triangleDir specifies the division direction.

[0107] - Specifies the variable cIdx for the color component index.

[0108] The output of this process is an array of (nCbW)×(nCbH) predicted sample values, pbSamples.

[0109] The variable nCbR is derived as follows:

[0110] nCbR = (nCbW > nCbH) ? (nCbW / nCbH) : (nCbH / nCbW)

[0111] The variable bitDepth is exported as follows:

[0112] - If cIdx equals 0, then set bitDepth to equal BitDepthy.

[0113] Otherwise, set bitDepth to equal BitDepthc.

[0114] The variables shiftl and offsetl are exported as follows:

[0115] - Set the variable shiftl to equal Max(5, 17 - bitDepth).

[0116] - Set the variable offsetl to equal 1 << (shiftl – 1).

[0117] Based on the values ​​of triangleDir, wS, and cIdx, predict samples pbSamples[x][y], x = 0..nCbW – 1 and y = 0..nCbH – 1, as derived below:

[0118] - The variable wIdx is exported as follows:

[0119] - If cIdx equals 0 and triangleDir equals 0, then the following applies:

[0120] wIdx = (nCbW > nCbH) ? (Clip3(0, 8, (x / nCbR – y) + 4))

[0121] : (Clip3(0, 8, (x - y / nCbR) + 4))

[0122] - Otherwise, if cIdx equals 0 and triangleDir equals 1, then the following applies:

[0123] wIdx = (nCbW > nCbH) ? (Clip3(0, 8, (nCbH - 1 - x / nCbR - y) + 4))

[0124] (Clip3(0, 8, (nCbW – 1 – x – y / nCbR) + 4))

[0125] - Otherwise, if cIdx is greater than 0 and triangleDir is equal to 0, then the following applies:

[0126] wIdx = (nCbW > nCbH) ? (Clip3(0, 4, (x / nCbR – y) + 2))v

[0127] : (Clip3(0, 4, (x – y / nCbR) + 2))

[0128] - Otherwise (if cIdx is greater than 0 and triangleDir equals 1), the following applies:

[0129] wIdx = (nCbW > nCbH) ? (Clip3(0, 4, (nCbH - 1 - x / nCbR - y) + 2))

[0130] (Clip3(0, 4, (nCbW – 1 – x – y / nCbR) + 2))

[0131] - The following uses wIdx and cIdx to derive the variable wValue, which specifies the height of the predicted sample:

[0132] wValue = (cIdx = = 0) ? Clip3(0, 8, wIdx) : Clip3(0, 8, wIdx * 2)

[0133] - The predicted sample values ​​are derived as follows:

[0134] pbSamples[ x ][ y ] = Clip3(0, (1 << bitDepth) – 1,

[0135] (predSamplesLA[ x ][ y ] * wValue +

[0136] predSamplesLB[ x ][ y ] * (8 - wValue) + offsetl) >> shiftl)

[0137] It should be noted that in JVET-O2001, the weighted sample prediction process based on triangle motion compensation may not be ideal. Specifically, the blending function used for triangle prediction does not consider chroma sampling formats other than 4:2:0. In one example, according to the techniques described herein, the blending function can be simplified and / or extended to accommodate a practical chroma sampling format (which could be 4:2:2 or 4:4:4).

[0138] Figure 5 This is a block diagram illustrating an example of a system that can be configured to encode (e.g., encode and / or decode) video data according to one or more techniques of this disclosure. System 100 represents an example of a system that can perform video encoding using one or more techniques of this disclosure. Figure 5 As shown, system 100 includes source device 102, communication medium 110, and target device 120. Figure 5 In the example shown, source device 102 may include any device configured to encode video data and transmit the encoded video data to communication medium 110. Target device 120 may include any device configured to receive and decode the encoded video data via communication medium 110. Source device 102 and / or target device 120 may include computing devices equipped for wired and / or wireless communication, and may include set-top boxes, digital video recorders, televisions, desktop computers, laptops or tablets, game consoles, mobile devices including, for example, "smart" phones, cellular phones, personal gaming devices, and medical imaging equipment.

[0139] Communication medium 110 may include any combination of wireless and wired communication media and / or storage devices. Communication medium 110 may include coaxial cable, fiber optic cable, twisted-pair cable, wireless transmitters and receivers, routers, switches, repeaters, base stations, or any other device that can be used to facilitate communication between various devices and sites. Communication medium 110 may include one or more networks. For example, communication medium 110 may include a network configured to allow access to the World Wide Web, such as the Internet. The network may operate according to a combination of one or more telecommunications protocols. Telecommunication protocols may include proprietary aspects and / or may include standardized telecommunications protocols. Examples of standardized telecommunications protocols include the Digital Video Broadcasting (DVB) standard, the Advanced Television Systems Committee (ATSC) standard, the Integrated Services Digital Broadcasting (ISDB) standard, the Cable Data Services Interface Specification (DOCSIS) standard, the Global System for Mobile Communications (GSM) standard, the Code Division Multiple Access (CDMA) standard, the 3rd Generation Partnership Project (3GPP) standard, the European Telecommunications Standards Institute (ETSI) standard, the Internet Protocol (IP) standard, the Wireless Application Protocol (WAP) standard, and the Institute of Electrical and Electronics Engineers (IEEE) standard.

[0140] Storage devices can include any type of device or storage medium capable of storing data. Storage media can include tangible or non-transitory computer-readable media. Computer-readable media can include optical discs, flash memory, magnetic storage, or any other suitable digital storage medium. In some examples, a memory device or a portion thereof may be described as non-volatile memory, and in other examples, a portion of a memory device may be described as volatile memory. Examples of volatile memory can include random access memory (RAM), dynamic random access memory (DRAM), and static random access memory (SRAM). Examples of non-volatile memory can include magnetic hard disks, optical discs, floppy disks, flash memory, or electrically programmable memory (EPROM) or electrically erasable and programmable (EEPROM) memory. Storage devices can include memory cards (e.g., secure digital (SD) memory cards), internal / external hard disk drives, and / or internal / external solid-state drives. Data can be stored on the storage device according to defined file formats.

[0141] Refer again Figure 5Source device 102 includes a video source 104, a video encoder 106, and an interface 108. Video source 104 may include any device configured to capture and / or store video data. For example, video source 104 may include a camera and a storage device operatively coupled thereto. Video encoder 106 may include any device configured to receive video data and generate a compliant bitstream representing the video data. A compliant bitstream may refer to a bitstream from which a video decoder can receive and reproduce video data. Aspects of a compliant bitstream may be defined according to a video coding standard. When generating a compliant bitstream, video encoder 106 may compress the video data. Compression may be lossy (perceptible or imperceptible) or lossless. Interface 108 may include any device configured to receive a compliant video bitstream and transmit and / or store the compliant video bitstream to a communication medium. Interface 108 may include a network interface card such as an Ethernet card and may include an optical transceiver, an RF transceiver, or any other type of device capable of transmitting and / or receiving information. Furthermore, interface 108 may include a computer system interface that allows compatible video bitstreams to be stored on a storage device. For example, interface 108 may include protocols supporting Peripheral Component Interconnect (PCI) and Peripheral Component Fast Interconnect (PCIe) bus protocols, dedicated bus protocols, Universal Serial Bus (USB) protocols, and I / O protocols. 2 C or any other logical and physical structure that can be used to interconnect peer devices.

[0142] Refer again Figure 5 The target device 120 includes an interface 122, a video decoder 124, and a display 126. Interface 122 may include any device configured to receive compatible video bitstreams from a communication medium. Interface 108 may include a network interface card such as an Ethernet card, and may include an optical transceiver, an RF transceiver, or any other type of device capable of receiving and / or transmitting information. Furthermore, interface 122 may include a computer system interface that allows retrieval of compatible video bitstreams from a storage device. For example, interface 122 may include protocols supporting PCI and PCIe bus protocols, dedicated bus protocols, USB protocols, etc. 2 The chipset of C, or any other logical and physical structure that can be used to interconnect peer devices. The video decoder 124 may include any device configured to receive compatible bitstreams and / or acceptable variations thereof, and reproduce video data from them. The display 126 may include any device configured to display video data. The display 126 may include one of a variety of display devices such as a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or another type of display. The display 126 may include a high-definition display or an ultra-high-definition display. It should be noted that, although in Figure 3In the example shown, video decoder 124 is described as outputting data to display 126, but video decoder 124 can be configured to output video data to various types of devices and / or their sub-components. For example, video decoder 124 can be configured to output video data to any communication medium, as described herein.

[0143] Figure 6 This is a block diagram illustrating an example of a video encoder 200 that can implement the techniques described herein for encoding video data. It should be noted that although the exemplary video encoder 200 is shown as having different functional blocks, such illustrations are intended for descriptive purposes and do not limit the video encoder 200 and / or its sub-components to a particular hardware or software architecture. The functionality of the video encoder 200 can be implemented using any combination of hardware, firmware, and / or software implementations. In one example, the video encoder 200 may be configured to encode video data according to the techniques described herein. The video encoder 200 may perform intra-frame predictive coding and inter-frame predictive coding of picture regions, and thus may be referred to as a hybrid video encoder. Figure 6 In the example shown, video encoder 200 receives a source video block. In some examples, the source video block may include picture regions that have been partitioned according to the coding structure. For example, source video data may include macroblocks, CTUs, CBs, their sub-partitions, and / or another equivalent coding unit. In some examples, video encoder 200 may be configured to perform additional subdivision of the source video block. It should be noted that some of the techniques described herein are generally applicable to video coding, regardless of how the source video data is partitioned before and / or during encoding. Figure 6 In the example shown, the video encoder 200 includes a summer 202, a transform coefficient generator 204, a coefficient quantization unit 206, an inverse quantization / transformation processing unit 208, a summer 210, an intra-frame prediction processing unit 212, an inter-frame prediction processing unit 214, a filter unit 216, and an entropy coding unit 218.

[0144] like Figure 6As shown, video encoder 200 receives source video blocks and outputs a bitstream. Video encoder 200 generates residual data by subtracting a predicted video block from the source video block. Summer 202 represents the component configured to perform this subtraction operation. In one example, the subtraction of the video block occurs in the pixel domain. Transform coefficient generator 204 applies a transform, such as Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), or a conceptually similar transform, to its residual block or sub-partition (e.g., four 8×8 transforms can be applied to a 16×16 residual value array) to generate a set of residual transform coefficients. Transform coefficient generator 204 can be configured to perform any and all combinations of transforms included in the discrete trigonometric transform series. As mentioned above, in ITU-T H.265, TB is limited to the following sizes: 4×4, 8×8, 16×16, and 32×32. In one example, the transform coefficient generator 204 can be configured to perform transforms based on arrays of sizes 4×4, 8×8, 16×16, and 32×32. In another example, the transform coefficient generator 204 can be further configured to perform transforms based on arrays of other sizes. Specifically, in some cases, performing transforms on rectangular arrays composed of different values ​​may be useful. In one example, the transform coefficient generator 204 can be configured to perform transforms based on array sizes of 2×2, 2×4N, 4M×2, and / or 4M×4N. In one example, a two-dimensional (2D) M×N inverse transform can be implemented as a one-dimensional (1D) M-point inverse transform followed by a 1D N-point inverse transform. In one example, a 2D inverse transform can be implemented as a 1D N-point vertical transform followed by a 1D N-point horizontal transform. In another example, a 2D inverse transform can be implemented as a 1D N-point horizontal transform followed by a 1D N-point vertical transform. The transform coefficient generator 204 can output transform coefficients to the coefficient quantization unit 206.

[0145] Coefficient quantization unit 206 can be configured to perform quantization of the transform coefficients. As described above, the degree of quantization can be modified by adjusting the quantization parameters. Coefficient quantization unit 206 can be further configured to determine the quantization parameters and output QP data (e.g., data for determining the quantization group size and / or incremental QP value), which the video decoder can use to reconstruct the quantization parameters to perform inverse quantization during video decoding. It should be noted that in other examples, one or more additional or alternative parameters (e.g., scaling factors) can be used to determine the quantization level. The techniques described herein are generally applicable to determining the quantization level of transform coefficients corresponding to another component of video data based on the quantization level of transform coefficients corresponding to one component of the video data.

[0146] See you again Figure 6The quantized transform coefficients are output to the inverse quantization / transform processing unit 208. The inverse quantization / transform processing unit 208 can be configured to apply inverse quantization and inverse transform to generate reconstructed residual data. Figure 6 As shown, at summer 210, reconstructed residual data can be added to the predicted video block. This allows for the reconstruction of the encoded video block, which can then be used to evaluate the coding quality of a given prediction, transform, and / or quantization. The video encoder 200 can be configured to perform multiple coding rounds (e.g., coding while changing one or more of the prediction, transform, and quantization parameters). The rate-distortion or other system parameters of the bitstream can be optimized based on the evaluation of the reconstructed video block. Furthermore, the reconstructed video block can be stored and used as a reference for predicting subsequent blocks.

[0147] As described above, intra-frame prediction can be used to encode video blocks. The intra-frame prediction processing unit 212 can be configured to select an intra-frame prediction mode for the video block to be encoded. The intra-frame prediction processing unit 212 can be configured to evaluate frames and / or regions thereof and determine the intra-frame prediction mode to be used for encoding the current block. Figure 6 As shown, the intra-prediction processing unit 212 outputs intra-prediction data (e.g., syntax elements) to the entropy coding unit 218 and the transform coefficient generator 204. As mentioned above, the transform performed on the residual data can depend on the mode. As mentioned above, possible intra-prediction modes can include planar prediction mode, DC prediction mode, and angle prediction mode. Furthermore, in some examples, the prediction for the chromaticity component can be inferred from the intra-prediction used for the luminance prediction mode.

[0148] Inter-frame prediction processing unit 214 can be configured to perform inter-frame prediction coding for the current video block. Inter-frame prediction processing unit 214 can be configured to receive a source video block and calculate the motion vector of the PU (Power Unit) of the video block. The motion vector can indicate the displacement of the PU (or similar coding structure) of the video block within the current video frame relative to the prediction block within a reference frame. Inter-frame prediction coding can use one or more reference images. Furthermore, motion prediction can be unidirectional prediction (using one motion vector) or bidirectional prediction (using two motion vectors). Inter-frame prediction processing unit 214 can be configured to select a prediction block by calculating pixel differences determined by, for example, sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metrics. As described above, motion vectors can be determined and specified based on motion vector prediction. As described above, inter-frame prediction processing unit 214 can be configured to perform motion vector prediction. Inter-frame prediction processing unit 214 can be configured to generate a prediction block using motion prediction data. For example, inter-frame prediction processing unit 214 can locate the prediction video block within the frame buffer (…). Figure 6(Not shown in the image). It should be noted that the inter-frame prediction processing unit 214 can be further configured to apply one or more interpolation filters to the reconstructed residual block to compute sub-integer pixel values ​​for motion estimation. The inter-frame prediction processing unit 214 can output the motion prediction data of the computed motion vectors to the entropy coding unit 218. Figure 6 As shown, the inter-frame prediction processing unit 214 can receive reconstructed video blocks via the filter unit 216.

[0149] As mentioned above, in JVET-O2001, the weighted sample prediction process based on triangle motion compensation may be less than ideal. In one example, according to the techniques described herein, the weighted sample prediction process for triangle motion compensation can determine the size of the prediction sample array based on the color component index and the sampling format. In one example, according to the techniques described herein, the weighted sample prediction process for triangle motion compensation can be based on the following:

[0150] Weighted Sample Prediction Process for Triangular Merging Pattern

[0151] The input to this process is:

[0152] - Two variables, nCbW and nCbH, specify the width and height of the current code block.

[0153] - Two (nCbW)×(nCbH) arrays, predSamplesLA and predSamplesLB,

[0154] - The variable triangleDir specifies the division direction.

[0155] - Specifies the variable cIdx for the color component index.

[0156] The output of this process is an array pbSamples of (nCbW)×(nCbH) predicted sample values. The variables nCbWH and nCbHW are derived as follows:

[0157] nCbWH = Max (1, nCbW / nCbH)

[0158] nCbHW = Max (1, nCbH / nCbW)

[0159] Based on the value of cIdx, the variables bitDepth and scale are derived as follows:

[0160] - If cIdx equals 0, then set bitDepth to equal BitDepth. Y And set scale to equal 1.

[0161] Otherwise, set bitDepth to equal BitDepthc. If Min(nCbW, nCbH) equals Min(nCbW *SubWidthC, nCbH * SubHeightC), then set scale to equal 1; otherwise, set it to equal 2.

[0162] The variables shiftl and offsetl are exported as follows:

[0163] - Set the variable shiftl to equal Max(5, 17 - bitDepth).

[0164] - Set the variable offsetl to equal 1 << (shiftl – 1).

[0165] Based on the value of triangleDir, predict samples pbSamples[x][y], x = 0..nCbW – 1 and y = 0..nCbH – 1, as derived below:

[0166] - The variable wIdx is exported as follows:

[0167] - If triangleDir equals 0, then the following applies:

[0168] wIdx = x / nCbWH * scale – y / nCbHW * scale

[0169] - Otherwise (if triangleDir equals 1), the following applies:

[0170] wIdx = (nCbW – 1 – x) / nCbWH * scale – y / nCbHW * scale

[0171] - The following uses wIdx and cIdx to derive the variable wValue, which specifies the height of the predicted sample:

[0172] wValue = Clip3(0, 8, wIdx + 4)

[0173] - The predicted sample values ​​are derived as follows:

[0174] pbSamples[ x ][ y ] = Clip3(0, (1 << bitDepth) – 1,

[0175] (predSamplesLA[ x ][ y ] * wValue +

[0176] predSamplesLB[ x ][ y ] * (8 – wValue) + offsetl) >> shiftl)

[0177] It should be noted that, with a sampling format of 4:2:2, for a video block with 16×8 luminance samples, the corresponding chrominance array is 8×8 (width × height). According to the weighted sample prediction process specified in JVET-O2001, for a triangleDir value equal to 0, the following blending matrix (wValue) is generated for luminance and chrominance:

[0178]

[0179] It should be noted that, in this case, horizontally magnifying the chromaticity matrix will not produce a matrix similar to the luminance matrix.

[0180] Based on the weighted sample prediction process specified in this paper, for a triangleDir equal to 0, the following blending matrix (wValue) is generated for luminance and chrominance:

[0181]

[0182] Therefore, the weighted sample prediction process specified by JVET-O2001 and the weighted sample prediction process specified by the technique in this paper will produce different wValue values ​​in some cases. For example, in the above case, for chroma, at (x,y)=(1, 2), the wValue is equal to 2 according to the weighted sample prediction process specified by JVET-O2001, while it is equal to 3 according to the technique in this paper. Different wValue values ​​will produce different arrays pbSamples[x][y]. The array pbSamples[x][y] generated by the technique in this paper can improve coding efficiency in some cases.

[0183] Thus, video encoder 200 represents an example of a device configured to perform the following operations: receiving a first predictive sample value array; receiving a second predictive sample value array; determining a scaling value based on color component index values ​​and the video sampling format of the video data; generating a third predictive sample value array by applying a blending matrix to the first and second predictive sample value arrays, wherein the blending matrix is ​​based on the scaling value; and performing video encoding using the third predictive sample value array.

[0184] like Figure 6As shown, filter unit 216 receives reconstructed video blocks and coding parameters, and outputs modified reconstructed video data. Filter unit 216 can be configured to perform deblocking, sample adaptive offset (SAO) filtering, adaptive loop filtering (ALF), etc. SAO filtering is a nonlinear amplitude mapping that can be used to improve reconstruction by adding an offset to the reconstructed video data. It should be noted that, as Figure 5 As shown, intra-frame prediction processing unit 212 and inter-frame prediction processing unit 214 can receive modified reconstructed video blocks via filter unit 216. Entropy coding unit 218 receives quantized transform coefficients and prediction syntax data (i.e., intra-frame prediction data and motion prediction data). Entropy coding unit 218 can be configured to perform entropy coding according to one or more of the techniques described herein.

[0185] Figure 7 This is a block diagram illustrating an example of a video decoder configured to decode video data according to one or more techniques described herein. In one example, the video decoder 300 may be configured to reconstruct video data based on one or more of the techniques described above. That is, the video decoder 300 may operate in a manner reversible from the video encoder 200 described above. The video decoder 300 may be configured to perform intra-frame predictive decoding and inter-frame predictive decoding, and thus may be referred to as a hybrid decoder. Figure 7 In the example shown, the video decoder 300 includes an entropy decoding unit 302, an inverse quantization unit 304, an inverse transform processing unit 306, an intra-frame prediction processing unit 308, an inter-frame prediction processing unit 310, a summer 312, a filter unit 314, and a reference buffer 316. The video decoder 300 can be configured to decode video data in a manner consistent with a video coding system that implements one or more aspects of a video coding standard. It should be noted that although the exemplary video decoder 300 shown has different functional blocks, such illustrations are intended for descriptive purposes and do not limit the video decoder 300 and / or its sub-components to a particular hardware or software architecture. The functionality of the video decoder 300 can be implemented using any combination of hardware, firmware, and / or software implementations.

[0186] like Figure 7 As shown, the entropy decoding unit 302 receives an entropy-encoded bitstream. The entropy decoding unit 302 can be configured to decode the quantization syntax elements and quantization coefficients from the bitstream according to a process that is the inverse of the entropy encoding process. The entropy decoding unit 302 can be configured to perform entropy decoding according to any of the entropy encoding techniques described above. The entropy decoding unit 302 can parse the encoded bitstream in a manner consistent with video coding standards. The video decoder 300 can be configured to parse the encoded bitstream, wherein the encoded bitstream is generated based on the techniques described above.

[0187] Refer again Figure 7 The inverse quantization unit 304 receives quantization transform coefficients (i.e., bit values) and quantization parameter data from the entropy decoding unit 302. The quantization parameter data may include any and all combinations of the aforementioned incremental QP values ​​and / or quantization group size values. The video decoder 300 and / or the inverse quantization unit 304 may be configured to determine the QP value for inverse quantization based on the value signaled by the video encoder and / or through video attributes and / or encoding parameters. In other words, the inverse quantization unit 304 may operate in a manner inverse of the aforementioned coefficient quantization unit 206. For example, the inverse quantization unit 304 may be configured to infer predetermined values, allowed quantization group sizes, derive quantization parameters, etc., according to the aforementioned techniques. The inverse quantization unit 304 may be configured to apply inverse quantization. The inverse transform processing unit 306 may be configured to perform an inverse transform to generate reconstructed residual data. The techniques performed by the inverse quantization unit 304 and the inverse transform processing unit 306 may be similar to the techniques performed by the aforementioned inverse quantization / transform processing unit 208. The inverse transform processing unit 306 can be configured to apply inverse DCT, inverse DST, inverse integer transform, indivisible quadratic transform (NSST), or conceptually similar inverse transform procedures to transform the coefficients in order to generate residual blocks in the pixel domain. Furthermore, as mentioned above, whether a specific transform (or the type of specific transform) is performed can depend on the intra-frame prediction mode. Figure 7 As shown, the reconstructed residual data can be provided to the summer 312. The summer 312 can add the reconstructed residual data to the predicted video block and generate reconstructed video data. The predicted video block can be determined based on the predicted video technique (i.e., intra-frame prediction and inter-frame prediction).

[0188] Intra-prediction processing unit 308 may be configured to receive intra-prediction syntax elements and retrieve predicted video blocks from reference buffer 316. Reference buffer 316 may include a memory device configured to store one or more video data frames. The intra-prediction syntax elements may identify intra-prediction modes, such as those described above. In one example, intra-prediction processing unit 308 may reconstruct video blocks according to one or more of the intra-prediction coding techniques described herein. Inter-prediction processing unit 310 may receive inter-prediction syntax elements and generate motion vectors to identify predicted blocks in one or more reference frames stored in reference buffer 316. Inter-prediction processing unit 310 may generate motion-compensated blocks, possibly performing interpolation based on interpolation filters. Identifiers for interpolation filters used for motion estimation with sub-pixel precision may be included in the syntax elements. Inter-prediction processing unit 310 may use interpolation filters to compute interpolated values ​​for sub-integer pixels of the reference blocks. In one example, inter-prediction processing unit 310 may reconstruct video blocks using one or more of the inter-prediction coding techniques described herein. Filter unit 314 can be configured to perform filtering on the reconstructed video data. For example, filter unit 314 can be configured to perform deblocking and / or SAO filtering, as described above with respect to filter unit 216. Furthermore, it should be noted that in some examples, filter unit 314 can be configured to perform dedicated arbitrary filtering (e.g., visual enhancement). Figure 7 As shown, the video decoder 300 can output reconstructed video blocks. Thus, the video decoder 300 represents an example of a device configured to perform the following operations: receiving a first predictive sample value array; receiving a second predictive sample value array; determining a scaling value based on color component index values ​​and the video sampling format of the video data; generating a third predictive sample value array by applying a mixing matrix to the first and second predictive sample value arrays, wherein the mixing matrix is ​​based on the scaling value; and performing video encoding using the third predictive sample value array.

[0189] In one or more examples, the functionality may be implemented by hardware, software, firmware, or any combination thereof. If implemented in software, the functionality may be stored as one or more instructions or code on or transmitted over a computer-readable medium and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium corresponding to a tangible medium such as a data storage medium, or a propagation medium that includes, for example, any medium facilitating the transfer of a computer program from one place to another according to a communication protocol. Thus, a computer-readable medium may generally correspond to: (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementing the techniques described in this disclosure. Computer program products may include computer-readable media.

[0190] By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or any other medium that can be used to store required program code in the form of instructions or data structures and that is accessible by a computer. Furthermore, any connection is appropriately referred to as a computer-readable medium. For example, coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but rather refer to non-transient tangible storage media. As used herein, disks and optical discs include compact optical discs (CDs), laser discs, optical discs, digital versatile optical discs (DVDs), floppy disks, and Blu-ray discs, where disks typically copy data magnetically, while optical discs use lasers to copy data optically. Combinations of the above should also be included within the scope of computer-readable media.

[0191] Instructions can be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the term "processor" as used herein can refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein. Furthermore, in some aspects, the functionality described herein can be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into combined codecs. Moreover, these techniques can be implemented entirely within one or more circuit or logic elements.

[0192] The techniques disclosed herein can be implemented in various devices or apparatuses, including wireless mobile phones, integrated circuits (ICs), or a set of ICs (e.g., chipsets). Various components, modules, or units are described in this disclosure to emphasize functional aspects of a device configured to perform the disclosed techniques, but they do not necessarily need to be implemented through different hardware units. Rather, as described above, various units can be combined in a codec hardware unit, or provided through an interoperable hardware unit comprising a collection of one or more processors as described above, combined with suitable software and / or firmware.

[0193] Furthermore, each functional block or feature of the base station equipment and terminal equipment used in each of the above embodiments can be implemented or performed by circuitry (typically one or more integrated circuits). Circuitry designed to perform the functions described in this specification may include general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, or combinations thereof. The general-purpose processor may be a microprocessor, or alternatively, it may be a conventional processor, controller, microcontroller, or state machine. The general-purpose processor or each of the above circuitry can be configured by digital circuitry or by analog circuitry. Furthermore, when advancements in semiconductor technology lead to the development of technologies for producing integrated circuits that replace current integrated circuits, integrated circuits produced using such technologies can also be used.

[0194] Various examples have been described. These and other examples are within the scope of the following claims.

[0195] <Summary of the Invention>

[0196] In one example, a method for encoding video data includes: receiving a first array of predicted sample values; receiving a second array of predicted sample values; determining a scaling value based on color component index values ​​and a video sampling format of the video data; generating a third array of predicted sample values ​​by applying a blending matrix to the first and second arrays of predicted sample values, wherein the blending matrix is ​​based on the scaling value; and performing video encoding using the third array of predicted sample values.

[0197] In one example, a method is provided in which performing video encoding using a third predicted sample value array includes decoding the video data by adding residuals to the third predicted sample value array.

[0198] In one example, the method is provided, which also includes encoding video data by subtracting the third predicted sample value array from the current video block, whereby video encoding is performed using a third predicted sample value array.

[0199] In one example, the method is provided where the color component index value indicates the chroma component, and the video sample format of the video data is 4:2:2.

[0200] In one example, the method is provided where the color component index value indicates the chroma component, and the video sample format of the video data is 4:4:4.

[0201] In one example, a device for encoding video data includes one or more processors configured to perform any and all combinations of these steps.

[0202] In one example, the device is provided, which includes a video encoder.

[0203] In one example, the device is provided, which includes a video decoder.

[0204] In one example, a system includes: a device that includes a video encoder; and the device includes a video decoder.

[0205] In one example, an apparatus for encoding video data includes means for performing any and all combinations of steps.

[0206] In one example, a non-transitory computer-readable storage medium includes instructions stored thereon that, when executed, cause one or more processors of a device for encoding video data to perform any and all combinations of steps.

[0207] In one example, a method for decoding video data includes: receiving a first array of predicted sample values; receiving a second array of predicted sample values; determining a scaling value based on color component index values ​​and a video sampling format of the video data; and generating a third array of predicted sample values ​​by applying a blended value to the first and second arrays of predicted sample values, wherein the blended value is based on the scaling value.

[0208] In one example, a method is provided in which determining the scaling value based on the color component index value and the video sampling format of the video data includes determining a scaling value of 2 for the horizontal component and a scaling value of 1 for the vertical component, provided that the color component index value indicates the chroma component and the video sampling format of the video data is 4:2:2.

[0209] In one example, a method is provided in which determining the scaling value based on the color component index value and the video sampling format of the video data includes determining a scaling value of 1 for the horizontal component and a scaling value of 1 for the vertical component, provided that the color component index value indicates the chroma component and the video sampling format of the video data is 4:4:4.

[0210] In one example, a device including one or more processors is configured to: receive a first array of predicted sample values; receive a second array of predicted sample values; determine a scaling value based on color component index values ​​and a video sampling format of video data; and generate a third array of predicted sample values ​​by applying a blended value to the first and second arrays of predicted sample values, wherein the blended value is based on the scaling value.

[0211] In one example, the device is provided in which determining the scaling value based on the color component index value and the video sampling format of the video data includes determining a scaling value of 2 for the horizontal component and a scaling value of 1 for the vertical component when the color component index value indicates the chroma component and the video sampling format of the video data is 4:2:2.

[0212] In one example, the device is provided in which determining the scaling value based on the color component index value and the video sampling format of the video data includes determining a scaling value of 1 for the horizontal component and a scaling value of 1 for the vertical component, provided that the color component index value indicates the chroma component and the video sampling format of the video data is 4:4:4.

[0213] In one example, the device is provided, which includes a video decoder.

[0214] <Cross-reference>

[0215] This non-provisional application claims priority to provisional application 62 / 896,500, filed September 5, 2019, pursuant to section 119 of title 35 of the United States Code, the entire contents of which are incorporated herein by reference.

Claims

1. An encoding device comprising one or more processors, said one or more processors being configured to: Using the first motion vector, the first predicted sample value array is derived according to the fractional sample interpolation process; Using the second motion vector, a second predicted sample value array is derived according to the fractional sample interpolation process; The horizontal and vertical positions are scaled based on the color component index values ​​and the video sampling format of the video data to derive the blended values; as well as A third predicted sample value array is generated by adding the following two items together: (1) the product of the mixed value and the first predicted sample value array, and (2) (i) the product of the mixed value and (ii) the second predicted sample value array.

2. The encoding device according to claim 1, wherein, The one or more processors scale the horizontal and vertical positions based on different size values, provided that the color component index value indicates the chroma component and the video sampling format of the video data is 4:2:

2.

3. The encoding device according to claim 1, wherein, The one or more processors scale the horizontal and vertical positions based on the same size value, provided that the color component index value indicates the chroma component and the video sampling format of the video data is 4:4:

4.

4. A non-transitory computer-readable recording medium storing a program for causing a computer to perform the following operations: Using the first motion vector, the first predicted sample value array is derived according to the fractional sample interpolation process; Using the second motion vector, a second predicted sample value array is derived according to the fractional sample interpolation process; The horizontal and vertical positions are scaled based on the color component index values ​​and the video sampling format of the video data to derive the blended values; as well as A third predictive sample value array is generated by applying the mixed value to the first and second predictive sample value arrays.

Images