Systems and methods for performing inter prediction encoding in video encoding

Abstract

A method of decoding video data is disclosed. The method includes receiving a first array of prediction sample values, receiving a second array of prediction sample values, determining a scaling value based on a color component index value and a video sampling format of the video data, and generating a third array of prediction sample values by applying a blending value to the first array of prediction sample values and the second array of prediction sample values, wherein the blending value is based on the scaling value.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to video coding, and more specifically to techniques for performing inter prediction. BACKGROUND

[0002] Digital video capabilities can be incorporated into a wide range of devices, including digital televisions, laptop or desktop computers, tablet computers, digital recording devices, digital media players, video gaming devices, cellular telephones, including so-called smartphones, medical imaging devices, and the like. Digital video can be coded according to a video coding standard. Video coding standards define the format of a compliant bitstream encapsulating encoded video data. A compliant bitstream is a data structure that can be received and decoded by a video decoding device to generate reconstructed video data. Video coding standards can incorporate video compression techniques. Examples of video coding standards include ISO / IEC MPEG-4 Visual and ITU-T H.264 / AVC (also known as ISO / IEC MPEG-4 AVC) and High-Efficiency Video Coding (HEVC). HEVC is described in High Efficiency Video Coding (HEVC), ITU-T H.265 Recommendation, December 2016, which is incorporated by reference, and is referred to herein as ITU-T H.265. Extensions and improvements to ITU-T H.265 are currently being considered for the development of next generation video coding standards. For example, the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG) (collectively, the Joint Video Exploration Team (JVET)) are working on standardizing video coding technology with compression capabilities significantly exceeding those of the current HEVC standard. Joint Exploration Test Model 7 (JEM 7), Algorithm Description of Joint Exploration Test Model 7 (JEM 7), ISO / IEC JTC1 / SC29 / WG11 Document: JVET-G1001, July 2017, Torino, IT, which is incorporated by reference herein, describes the coding features under exploration by the JVET in the Joint Test Model development, which is a potential enhancement to the functionality of ITU-T H.265. It is noted that the coding features of JEM 7 are implemented in the JEM reference software. As used herein, the term JEM collectively refers to the algorithms included in JEM 7 and the implementation of the JEM reference software. Further, in response to the “Joint Call for Proposals on Video Compression with Capabilities beyond HEVC,” jointly issued by VCEG and MPEG, multiple groups submitted descriptions of video coding tools at the 10th Meeting of ISO / IEC JTC1 / SC29 / WG11, April 16-20, 2018, San Diego, CA.According to various descriptions of video coding tools, a final initial draft text of a video coding specification is described in “Versatile Video Coding (Draft 1),” i.e., document JVET-J1001-v2, 10th Meeting of ISO / IEC JTC1 / SC29 / WG11, San Diego, CA, USA, April 16-20, 2018, which is incorporated by reference herein and referred to as JVET-J1001. The current development of a next generation video coding standard by VCEG and MPEG is referred to as the Versatile Video Coding (VVC) project. “Versatile Video Coding (Draft 6),” i.e., document JVET-O2001-vE, 15th Meeting of ISO / IEC JTC1 / SC29 / WG11, Gothenburg, SE, July 3-12, 2019, which is incorporated by reference herein and referred to as JVET-O2001, represents the current version of the draft text of a video coding specification corresponding to the VVC project.

[0003] Video compression techniques enable the storage and transmission of video data at a lower bit rate, while maintaining the visual quality of the video data. Video compression techniques can reduce the data required for storing and transmitting video data by exploiting the inherent redundancies in a video sequence. Video compression techniques can sub-divide a video sequence into successively smaller portions (i.e., a group of pictures within a video sequence, a picture within a group of pictures, a region within a picture, a sub-region within a region, etc.). Intra-picture prediction encoding techniques (e.g., spatial prediction techniques within a picture) and inter-picture prediction techniques (i.e., techniques across pictures (temporal)) can be used to generate difference values between the video data to be encoded and a reference unit of video data. The difference values can be referred to as residual data. The residual data can be encoded as quantized transform coefficients. Syntax elements can relate to the residual data and the reference coding unit (e.g., intra-prediction mode indices and motion information). The residual data and the syntax elements can be entropy coded. The entropy coded residual data and syntax elements can be included in a data structure that forms a compliant bitstream. SUMMARY

[0004] In one example, a method of decoding video data includes receiving a first array of prediction sample values, receiving a second array of prediction sample values, determining a scaling value based on a color component index value and a video sampling format of the video data, and generating a third array of prediction sample values by applying a blending value to the first array of prediction sample values and the second array of prediction sample values, wherein the blending value is based on the scaling value.

[0005] In one example, an apparatus comprising one or more processors configured to: receive a first array of prediction sample values; receive a second array of prediction sample values; determine a scaling value based on a color component index value and a video sampling format of video data; and generate a third array of prediction sample values by applying a blending value to the first array of prediction sample values and the second array of prediction sample values, wherein the blending value is based on the scaling value. BRIEF DESCRIPTION OF DRAWINGS

[0006] [ FIG. 1 ] FIG. 1 is a conceptual diagram illustrating an example of a set of pictures encoded according to quad-tree multi-type tree partitioning according to one or more techniques of this disclosure.

[0007] [ FIG. 2A ] FIG. 2A is a conceptual diagram illustrating an example of encoding a block of video data according to one or more techniques of this disclosure.

[0008] [ FIG. 2B ] FIG. 2B is a conceptual diagram illustrating an example of encoding a block of video data according to one or more techniques of this disclosure.

[0009] [ FIG. 3 ] FIG. 3 is a conceptual diagram illustrating an example of video component sampling formats that can be used according to one or more techniques of this disclosure.

[0010] [ FIG. 4 ] FIG. 4 is a conceptual diagram illustrating a data structure that encapsulates encoded video data and corresponding metadata according to one or more techniques of this disclosure.

[0011] [ FIG. 5 ] FIG. 5 is a block diagram illustrating an example of a system that can be configured to encode and decode video data according to one or more techniques of this disclosure.

[0012] [ FIG. 6 ] FIG. 6 is a block diagram illustrating an example of a video encoder that can be configured to encode video data according to one or more techniques of this disclosure.

[0013] [ FIG. 7 ] FIG. 7 is a block diagram illustrating an example of a video decoder that can be configured to decode video data according to one or more techniques of this disclosure. DETAILED DESCRIPTION

[0014] In general, this disclosure describes various techniques for encoding video data. In particular, this disclosure describes techniques for performing inter prediction. It should be noted that although the techniques of this disclosure are described with respect to ITU-T H.264, ITU-T H.265, JEM, and JVET-O2001, the techniques of this disclosure can be applied universally to video encoding. For example, in addition to those techniques included in ITU-T H.265, JEM, and JVET-O2001, the encoding techniques described herein can be incorporated into video encoding systems, including video encoding systems based on future video coding standards, including video block structures, intra prediction techniques, inter prediction techniques, transform techniques, filtering techniques, and / or other entropy encoding techniques. Accordingly, references to ITU-T H.264, ITU-T H.265, JEM, and / or JVET-O2001 are for descriptive purposes and should not be construed as limiting the scope of the techniques described herein. Moreover, it should be noted that the incorporation of documents by reference herein is for descriptive purposes and should not be construed as limiting or creating ambiguity with respect to terms used herein. For example, where a definition of a term provided in an incorporated reference differs from another incorporated reference and / or as used herein, then the term should be interpreted in a manner that broadly includes each respective definition and / or in a manner that specifically includes each particular definition among the alternatives.

[0015] In one example, a device for encoding video data includes one or more processors configured to: receive a first array of prediction sample values; receive a second array of prediction sample values; determine a scaling value based on a color component index value and a video sampling format of the video data; generate a third array of prediction sample values by applying a blending matrix to the first array of prediction sample values and the second array of prediction sample values, wherein the blending matrix is based on the scaling value; and perform video encoding using the third array of prediction sample values.

[0016] In one example, a non-transitory computer-readable storage medium includes instructions stored thereon that, when executed, cause one or more processors of a device to: receive a first array of prediction sample values; receive a second array of prediction sample values; determine a scaling value based on a color component index value and a video sampling format of the video data; generate a third array of prediction sample values by applying a blending matrix to the first array of prediction sample values and the second array of prediction sample values, wherein the blending matrix is based on the scaling value; and perform video encoding using the third array of prediction sample values.

[0017] In one example, an apparatus comprises means for receiving a first array of prediction sample values; means for receiving a second array of prediction sample values; means for determining a scaling value based on a color component index value and a video sampling format of the video data; means for generating a third array of prediction sample values by applying a blending matrix to the first array of prediction sample values and the second array of prediction sample values, wherein the blending matrix is based on the scaling value; and means for performing video encoding using the third array of prediction sample values.

[0018] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description and drawings, and from the claims.

[0019] Video content includes a video sequence composed of a series of frames (or pictures). The series of frames can also be referred to as a group of pictures (GOP). Each video frame or picture can be divided into one or more regions. The regions can be defined according to a base unit (e.g., a video block) and a set of rules that define the regions. For example, the rules that define the regions can be that the regions must be an integer number of video blocks arranged in a rectangle. In addition, the video blocks in a region can be ordered according to a scan pattern (e.g., raster scan). As used herein, the term “video block” can generally refer to a region of a picture, or can more specifically refer to a maximum array of sample values that can be predictively coded, sub-partitions thereof, and / or corresponding structures. In addition, the term “current video block” can refer to a region of a picture that is being coded or decoded. A video block can be defined as an array of sample values. It should be noted that in some cases, pixel values can be described as including sample values of respective components of video data, which can also be referred to as color components (e.g., luma (Y) and chroma (Cb and Cr) components or red, green, and blue components). It should be noted that in some cases, the terms “pixel value” and “sample value” can be used interchangeably. In addition, in some cases, a pixel or sample can be referred to as a pel. A video sampling format (which can also be referred to as a chroma format) can define a number of chroma samples included in a video block relative to a number of luma samples included in the video block. For example, for a 4:2:0 sampling format, the sampling rate of the luma component is twice that of the chroma components in both the horizontal and vertical directions.

[0020] A video encoder can perform predictive coding on video blocks and sub-partitions thereof. Video blocks and sub-partitions thereof can be referred to as nodes. ITU-T H.264 specifies macroblocks comprising 16x16 luma samples. That is, in ITU-T H.264, pictures are segmented into macroblocks. ITU-T H.265 specifies a similar coding tree unit (CTU) structure (which can be referred to as a largest coding unit (LCU)). In ITU-T H.265, pictures are segmented into CTUs. In ITU-T H.265, for a picture, the CTU size can be set to comprise 16x16, 32x32, or 64x64 luma samples. In ITU-T H.265, a CTU is composed of respective coding tree blocks (CTBs) for each component of video data (e.g., luma (Y) and chroma (Cb and Cr)). It should be noted that video having one luma component and two corresponding chroma components can be described as having two channels, i.e., a luma channel and a chroma channel. Further, in ITU-T H.265, a CTU can be partitioned according to a quadtree (QT) partitioning structure, which causes the CTBs of the CTU to be partitioned into coding blocks (CBs). That is, in ITU-T H.265, a CTU can be partitioned into quadtree leaf nodes. According to ITU-T H.265, one luma CB along with two corresponding chroma CBs and associated syntax elements are referred to as a coding unit (CU). In ITU-T H.265, a minimum allowed size of a CB can be signaled. In ITU-T H.265, the minimum allowed minimum size of a luma CB is 8x8 luma samples. In ITU-T H.265, the decision to code a picture region using intra prediction or inter prediction is made at the CU level.

[0021] In ITU-T H.265, a CU is associated with a prediction unit (PU) structure having its root at the CU. In ITU-T H.265, the PU structure allows partitioning of luma CBs and chroma CBs to generate corresponding reference samples. That is, in ITU-T H.265, a luma CB and a chroma CB can be partitioned into respective luma prediction blocks and chroma prediction blocks (PBs), where a PB includes sample values to which the same prediction is applied. In ITU-T H.265, a CB can be divided into 1, 2, or 4 PBs. ITU-T H.265 supports PB sizes from 64x64 samples down to 4x4 samples. In ITU-T H.265, square PBs are supported for intra prediction, where a CB can form a PB or a CB can be partitioned into four square PBs. In ITU-T H.265, in addition to square PBs, rectangular PBs are supported for inter prediction, where a CB can be halved vertically or horizontally to form PBs. Further, it should be noted that in ITU-T H.265, for inter prediction, four asymmetric PB partitions are supported, where a CB is partitioned into two PBs at one quarter of the height (top or bottom) or width (left or right) of the CB. Intra prediction data (e.g., intra prediction mode syntax elements) or inter prediction data (e.g., motion data syntax elements) corresponding to a PB are used to generate reference and / or predicted sample values for the PB.

[0022] JEM specifies a CTU having a maximum size of 256x256 luma samples. JEM specifies a quadtree plus binary tree (QTBT) block structure. In JEM, the QTBT structure allows quadtree leaf nodes to be further divided by a binary tree (BT) structure. That is, in JEM, the binary tree structure allows quadtree leaf nodes to be recursively divided vertically or horizontally. In JVET-O2001, a CTU is partitioned according to a quadtree plus multi-type tree (QTMT or QT+MTT) structure. The QTMT in JVET-O2001 is similar to the QTBT in JEM. However, in JVET-O2001, in addition to indicating binary splits, the multi-type tree can also indicate so-called ternary (or ternary tree (TT)) splits. A ternary split divides one block into three blocks either vertically or horizontally. In the case of a vertical TT split, the block is split at one quarter of its width from the left edge and at one quarter of its width from the right edge, and in the case of a horizontal TT split, the block is split at one quarter of its height from the top edge and at one quarter of its height from the bottom edge. Again referring to FIG. 1 , FIG. 1 An example is shown in which a CTU is partitioned into quadtree leaf nodes and the quadtree leaf nodes are further partitioned according to BT splits or TT splits. That is, in FIG. 1 , the dashed lines indicate additional binary and ternary splits in the quadtree.

[0023] As noted above, each video frame or picture can be divided into one or more regions. For example, according to ITU-T H.265, each video frame or picture can be partitioned to include one or more slices, and further partitioned to include one or more tiles, where each slice includes a sequence of CTUs (e.g., arranged in a raster scan order), and where a tile is a sequence of CTUs corresponding to a rectangular region of a picture. It should be noted that in ITU-T H.265, a slice is a sequence of one or more slice segments starting with an independent slice segment and containing all subsequent dependent slice segments (if any) before the next independent slice segment (if any). A slice segment (like a slice) is a sequence of CTUs. Thus, in some cases, the terms “slice” and “slice segment” can be used interchangeably to indicate a sequence of CTUs arranged in a raster scan order. Further, it should be noted that in ITU-T H.265, a tile can be composed of CTUs contained in more than one slice, and a slice can be composed of CTUs contained in more than one tile. However, ITU-T H.265 specifies that one or both of the following conditions should be met: (1) all CTUs in a slice belong to the same tile; and (2) all CTUs in a tile belong to the same slice.

[0024] With respect to JVET-O2001, a slice needs to be composed of an integer number of bricks, rather than just an integer number of CTUs. In JVET-O2001, a brick is a rectangular CTU row region within a particular tile in a picture. Further, in JVET-O2001, a tile can be partitioned into multiple bricks, each composed of one or more CTU rows within the tile. A tile that is not partitioned into multiple bricks is also referred to as a brick. However, a brick that is a proper subset of a tile is not referred to as a tile. Thus, in some video coding techniques, a slice including a group of CTUs that do not form a rectangular region of a picture can or can not be supported. Further, it should be noted that in some cases, a slice can need to be composed of an integer number of complete tiles, and in such cases, the slice is referred to as a tile group. The techniques described herein can apply to bricks, slices, tiles, and / or tile groups. FIG. 1 is a conceptual diagram illustrating an example of a group of pictures including slices. In FIG. 1 In the example shown, Pic3 is shown to include two slices (i.e., slice 0 and slice 1). In FIG. 1 In the example shown, slice 0 includes one brick, i.e., brick 0, and slice 1 includes two bricks, i.e., brick 1 and brick 2. It should be noted that in some cases, slice 0 and slice 1 can meet the requirements of a tile and / or tile group and be classified as a tile and / or tile group.

[0025] A video sampling format (which can also be referred to as a chroma format) can define a number of chroma samples included in a CU relative to a number of luma samples included in the CU. For example, for a 4:2:0 sampling format, the sampling rate of the luma component is twice the sampling rate of the chroma components in both the horizontal and vertical directions. Thus, for a CU formatted according to the 4:2:0 format, the width and height of the sample array for the luma component is twice the width and height of each of the sample arrays for the chroma components. FIG. 3 is a conceptual diagram illustrating an example of a coding unit formatted according to a 4:2:0 sample format. FIG. 3 The relative positions of the chroma samples relative to the luma samples within the CU are illustrated. As noted above, a CU is typically defined according to the number of horizontal and vertical luma samples. Thus, as illustrated in FIG. 3 a 16x16 CU formatted according to the 4:2:0 sample format includes 16x16 samples of the luma component and 8x8 samples for each of the chroma components. Moreover, in the example illustrated in FIG. 3 the relative positions of the chroma samples relative to the luma samples of neighboring video blocks of the 16x16 CU are illustrated. For a CU formatted according to the 4:2:2 format, the width of the sample array of the luma component is twice the width of the sample array of each of the chroma components, but the height of the sample array of the luma component is equal to the height of the sample array of each of the chroma components. Moreover, for a CU formatted according to the 4:4:4 format, the sample array of the luma component has the same width and height as the sample array of each of the chroma components.

[0026] Table 1 illustrates how the chroma format is specified in JVET-O2001 based on the values of the syntax elements chroma_format_idc and separate_colour_plane_flag included in JVET-O2001. Moreover, Table 1 illustrates how the variables SubWidthC and SubHeightC are derived according to the chroma format. SubWidthC and SubHeightC are used, for example, for deblocking. In connection with Table 1, JVET-O2001 provides the following specification:

[0027] In monochrome sampling, there is only one sample array, which is nominally treated as a luma array.

[0028] In 4:2:0 sampling, each of the two chroma arrays has half the height and half the width of the luma array.

[0029] In 4:2:2 sampling, each of the two chroma arrays has the same height as the luma array and half the width.

[0030] In 4:4:4 sampling, the following applies depending on the value of separate colour plane flag:

[0031] - If separate colour plane flag is equal to 0, each of the two chroma arrays has the same height and width as the luma array.

[0032] - Otherwise (separate colour plane flag is equal to 1), the three colour planes are treated separately as monochrome sample pictures.

[0033] chroma_format_idc separate_colour_plane_flag chroma format SubWidth C SubHeight C 0 0 monochrome 1 1 1 0 4:2:0 2 2 2 0 4:2:2 2 1 3 0 4:4:4 1 1 3 1 4:4:4 1 1

[0034] Table 1

[0035] It should be noted that for a sample format, such as a 4:2:0 sample format, a chroma location type can be specified. That is, for example, for a 4:2:0 sample format, a chroma sample can be specified relative to a luma sample with a horizontal offset value and a vertical offset value that indicate a relative spatial positioning. Table 2 provides definitions for HorizontalOffsetC and VerticalOffsetC for 5 chroma location types provided in JVET-O2001.

[0036] ChromaLocType HorizontalOffsetC VerticalOffsetC 0 0 0.5 1 0.5 0.5 2 0 0 3 0.5 0 4 0 1 5 0.5 1

[0037] Table 2

[0038] For intra-predictive coding, an intra-prediction mode can specify the location of reference samples within the picture. In ITU-T H.265, the possible intra-prediction modes that have been defined include a planar (i.e., surface-fitting) prediction mode, a DC (i.e., flat overall average) prediction mode, and 33 angular prediction modes (predMode: 2-34). In JEM, the possible intra-prediction modes that have been defined include a planar prediction mode, a DC prediction mode, and 65 angular prediction modes. It should be noted that the planar prediction mode and the DC prediction mode can be referred to as non-directional prediction modes, and the angular prediction modes can be referred to as directional prediction modes. It should be noted that the techniques described herein can be universally applicable regardless of the number of possible prediction modes that have been defined.

[0039] For inter-predictive coding, a reference picture is determined, and a motion vector (MV) identifies a sample in that reference picture that is used to generate a prediction for a current video block. For example, a current video block can be predicted using reference sample values located in one or more previously coded pictures, and a motion vector is used to indicate the location of the reference block relative to the current video block. The motion vector can describe, for example, a horizontal displacement component of the motion vector (i.e., MVx) and a vertical displacement component of the motion vector (i.e., MVy). x), a vertical displacement component of the motion vector (i.e., MV y ) and a resolution of the motion vector (e.g., quarter-pixel precision, half-pixel precision, one-pixel precision, two-pixel precision, four-pixel precision). Previously decoded pictures, which can include pictures output before or after the current picture, can be organized into one or more reference picture lists and identified using reference picture index values. Further, in inter prediction coding, single prediction refers to generating a prediction using sample values from a single reference picture and double prediction refers to generating a prediction using corresponding sample values from two reference pictures. That is, in single prediction, a single reference picture and corresponding motion vector are used to generate a prediction for a current video block, while in double prediction, a first reference picture and corresponding first motion vector and a second reference picture and corresponding second motion vector are used to generate a prediction for a current video block. In double prediction, the corresponding sample values are combined (e.g., added, rounded and clamped, or averaged according to weights) to generate the prediction. Pictures and regions thereof can be classified based on which types of prediction modes can be used to code video blocks thereof. That is, for regions having a B type (e.g., B slices), double prediction, single prediction and intra prediction modes can be utilized, for regions having a P type (e.g., P slices), single prediction and intra prediction modes can be utilized, and for regions having an I type (e.g., I slices), only intra prediction modes can be utilized. As noted above, reference pictures are identified by reference index. For example, for P slices, there can be a single reference picture list, RefPicListO, and for B slices, in addition to RefPicListO, there can be a second independent reference picture list, RefPicListl. It should be noted that for single prediction in B slices, one of RefPicListO or RefPicListl can be used to generate a prediction. Further, it should be noted that during the decoding process, when starting to decode a picture, the reference picture lists are generated from previously decoded pictures stored in a decoded picture buffer (DPB).

[0040] Furthermore, coding standards can support various motion vector prediction modes. Motion vector prediction enables derivation of a value for a motion vector of a current video block based on another motion vector. For example, a set of candidate blocks with associated motion information can be derived from spatially and temporally neighboring blocks of the current video block. Furthermore, the generated (or default) motion information can be used for motion vector prediction. Examples of motion vector prediction include advanced motion vector prediction (AMVP), temporal motion vector prediction (TMVP), so-called "merge" mode, and "skip" and "direct" motion inference. Furthermore, other examples of motion vector prediction include advanced temporal motion vector prediction (ATMVP) and spatial-temporal motion vector prediction (STMVP). For motion vector prediction, both the video encoder and the video decoder perform the same process to derive a set of candidates. Thus, for a current video block, the same set of candidates is generated during encoding and decoding.

[0041] As noted above, for inter prediction coding, reference samples in previously coded pictures are used to code a video block in a current picture. Previously coded pictures that are available as references when coding the current picture are referred to as reference pictures. It is noted that the decoding order does not necessarily correspond to the picture output order, i.e., the temporal order of pictures in a video sequence. In ITU-T H.265, when a picture is decoded, it is stored to a decoded picture buffer (DPB) (which can be referred to as a frame buffer, reference buffer, reference picture buffer, etc.). In ITU-T H.265, pictures stored to the DPB are removed from the DPB when they are output and are no longer needed for coding subsequent pictures. In ITU-T H.265, a determination of whether a picture should be removed from the DPB is invoked once per picture after decoding the slice header, i.e., at the beginning of decoding a picture. For example, a reference FIG. 1 , Pic3 is shown to reference Pic2. Similarly, Pic4 is shown to reference Pic1. With respect to FIG. 1, assuming the number of pictures corresponds to the decoding order, the DPB would be populated as follows: after decoding Pic1, the DPB would include {Pic1}; at the start of decoding Pic2, the DPB would include {Pic1}; after decoding Pic2, the DPB would include {Pic1, Pic2}; at the start of decoding Pic3, the DPB would include {Pic1, Pic2}. Pic3 would then be decoded with reference to Pic2, and after decoding Pic3, the DPB would include {Pic1, Pic2, Pic3}. At the start of decoding Pic4, pictures Pic2 and Pic3 would be marked for removal from the DPB because they are not needed for decoding Pic4 (or any subsequent pictures, not shown), and assuming Pic2 and Pic3 have been output, the DPB would be updated to include {Pic1}. Pic4 would then be decoded with reference to Pic1. The process of marking pictures for removal from the DPB can be referred to as reference picture set (RPS) management.

[0042] As noted above, intra prediction data or inter prediction data is used to produce reference sample values for a block of sample values. The difference between sample values included in a current PB or another type of picture region structure and associated reference samples (e.g., those generated using prediction) can be referred to as residual data. Residual data can include a respective array of difference values corresponding to each component of the video data. Residual data can be in the pixel domain. A transform such as a discrete cosine transform (DCT), a discrete sine transform (DST), an integer transform, a wavelet transform, or a conceptually similar transform can be applied to the array of difference values to produce transform coefficients. It should be noted that in ITU-T H.265 and JVET-O2001, a CU is associated with a transform unit (TU) structure having its roots at the CU level. That is, to generate transform coefficients, the array of difference values can be partitioned (e.g., four 8x8 transforms can be applied to a 16x16 array of residual values). Such sub-division of difference values can be referred to as a transform block (TB) for each component of the video data. It should be noted that in some cases, a core transform and a subsequent secondary transform can be applied (in a video encoder) to generate transform coefficients. For a video decoder, the order of the transforms is reversed.

[0043] The quantization process can be performed directly on transform coefficient or residual sample values (e.g., for palette coding quantization). Quantization approximates transform coefficients by limiting their amplitudes to a specified set of values. Quantization essentially scales transform coefficients in order to vary the amount of data needed to represent a set of transform coefficients. Quantization can include dividing transform coefficients (or values resulting from adding an offset value to transform coefficients) by a quantization scaling factor and any associated rounding function (e.g., rounding to the nearest integer). Quantized transform coefficients can be referred to as coefficient level values. Dequantization (or “dequantization”) can include multiplying coefficient level values by a quantization scaling factor, and any inverse rounding or offset addition operations. It should be noted that, as used herein, the term quantization process can refer to dividing by a scaling factor to generate level values in some cases, and multiplying by a scaling factor to recover transform coefficients in some cases. That is, the quantization process can refer to quantization in some cases, and dequantization in some cases. Further, it should be noted that while the quantization process is described in some of the examples below with respect to arithmetic operations related to decimal notation, such descriptions are for illustrative purposes and should not be interpreted as limiting. For example, the techniques described herein can be implemented in devices that use binary operations, etc. For example, the multiplication and division operations described herein can be implemented using shift operations, etc.

[0044] Quantized transform coefficients and syntax elements (e.g., syntax elements indicative of a coding structure of a video block) can be entropy encoded according to an entropy encoding technique. The entropy encoding process includes encoding the values of the syntax elements using lossless data compression algorithms. Examples of entropy encoding techniques include content adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), probability interval partitioning entropy coding (PIPE), and the like. The entropy encoded quantized transform coefficients and corresponding entropy encoded syntax elements can form a compliant bitstream that can be used to reproduce the video data at a video decoder. The entropy encoding process, e.g., CABAC, can include binarization of the syntax elements. Binarization refers to the process of converting the value of a syntax element into a sequence of one or more bits. These bits can be referred to as “bins.” Binarization can include one or a combination of the following encoding techniques: fixed length encoding, unary encoding, truncated unary encoding, truncated Rice encoding, Golomb encoding, k-th order exponential Golomb encoding, and Golomb-Rice encoding. For example, binarization can include representing the integer value 5 of a syntax element as 00000101 using an 8-bit fixed length binarization technique, or as 11110 using a unary encoding binarization technique. As used herein, each of the terms fixed length encoding, unary encoding, truncated unary encoding, truncated Rice encoding, Golomb encoding, k-th order exponential Golomb encoding, and Golomb-Rice encoding can refer to general implementations of these techniques and / or more specific implementations of these encoding techniques. For example, a Golomb-Rice encoding implementation can be specifically defined according to a video coding standard. In the example of CABAC, for a particular bin, a context provides a most probable state (MPS) value for the bin (i.e., the MPS for the bin is one of 0 or 1), and a probability value for the bin being the MPS or a least probable state (LPS). For example, a context can indicate that the MPS for a bin is 0, and the probability of the bin being 1 is 0.3. It should be noted that the context can be determined based on values of previously encoded bins, including the current syntax element and previously coded syntax elements. For example, values of syntax elements associated with neighboring video blocks can be used to determine the context for a current bin.

[0045] With respect to the equations used herein, the following arithmetic operators can be used:

[0046] + addition

[0047] - subtraction

[0048] * multiplication, including matrix multiplication

[0049] x y power. x is designated as the power of y. In other contexts, such notation is used for superscripts and is not intended to be interpreted as a power.

[0050] Integer division that truncates the result toward zero. For example, 7 / 4 and -7 / -4 are truncated to 1, and -7 / 4 and 7 / -4 are truncated to -1.

[0051] ÷ Used in mathematical formulas to denote division, in cases where truncation or rounding is not intended.

[0052] Used in mathematical formulas to denote division, in cases where truncation or rounding is not intended.

[0053] x % y The modulus of x divided by y, defined only for integer x and y where x >= 0 and y > 0.

[0054] In addition, the following logical operators can be used:

[0055] x && y Boolean logical "and" of x and y

[0056] x || y Boolean logical "or" of x and y

[0057] ! Boolean logical "not"

[0058] x? y : z Evaluates to y if x is TRUE or not equal to 0; otherwise, evaluates to z.

[0059] In addition, the following relational operators can be used:

[0060] > Greater than

[0061] >= Greater than or equal to

[0062] < Less than

[0063] <= Less than or equal to

[0064] == Equal to

[0065] != Not equal to

[0066] In addition, the following bitwise operators can be used:

[0067] & Bitwise "and". When operating on integral type variables, the binary complement representation of the integer values is operated on. When operating on binary variables that contain fewer bits than the other variable, the shorter variable is extended by adding more significant bits equal to 0.

[0068] | Bitwise "or". When operating on integral type variables, the binary complement representation of the integer values is operated on. When operating on binary variables that contain fewer bits than the other variable, the shorter variable is extended by adding more significant bits equal to 0.

[0069] Bitwise XOR. When operating on integral variables, the binary complement representation of the integer values is operated on. When operating on binary variables containing fewer bits than another variable, the shorter variable is extended by adding more significant bits equal to 0.

[0070] x » y Arithmetic right shift of the binary complement integer representation of x by y binary digits. The function is defined only for non-negative integer values of y. Bits shifted into the most significant bit (MSB) as a result of the right shift have a value equal to the MSB of x before the shift operation.

[0071] x << y Arithmetic left shift of the binary complement integer representation of x by y binary digits. The function is defined only for non-negative integer values of y. Bits shifted into the least significant bit (LSB) as a result of the left shift have a value equal to 0.

[0072] In addition, the following assignment operators can be used:

[0073] = assignment operator

[0074] ++ increment, i.e., x++ is equivalent to x = x + 1; when used in an array index, the value of the variable is evaluated before the increment operation.

[0075] -- decrement, i.e., x-- is equivalent to x = x - 1; when used in an array index, the value of the variable is evaluated before the decrement operation.

[0076] + = increment by a specified amount, i.e., x += 3 is equivalent to x = x + 3, and x += (-3) is equivalent to x = x + (-3).

[0077] - = decrement by a specified amount, i.e., x -= 3 is equivalent to x = x - 3, and x -= (-3) is equivalent to x = x - (-3).

[0078] In addition, the following defined mathematical functions can be used:

[0079]

[0080] Floor(x), the largest integer less than or equal to x.

[0081] Log2(x), the logarithm of x to the base 2.

[0082]

[0083]

[0084] Round(x) = Sign(x) * Floor(Abs(x) + 0.5)

[0085]

[0086] Furthermore, it should be noted that in the syntax descriptors used herein, the following descriptors can apply:

[0087] - b(8): Byte (8 bits) with any bit string pattern. The parsing process for this descriptor is specified by the return value of the function read_bit(8).

[0088] - f(n): Fixed pattern bit string written using n bits (from left to right) from the leftmost bit. The parsing process for this descriptor is specified by the return value of the function read_bit(n).

[0089] - se(v): Syntax element of signed integer 0th order Exp-Golomb coding from the leftmost bit.

[0090] - tb(v): Truncated binary code using at most maxVal bits, where maxVal is defined in the semantics of the syntax element.

[0091] - tu(v): Truncated unary code using at most maxVal bits, where maxVal is defined in the semantics of the syntax element.

[0092] - u(n): Unsigned integer using n bits. When n is "v" in the syntax table, the number of bits varies in a way that depends on the value of other syntax elements. The parsing process for this descriptor is specified by the return value of the function read_bits(n), which is interpreted as the binary representation of an unsigned integer written first with the most significant bits.

[0093] - ue(v): Syntax element of unsigned integer 0th order Exp-Golomb coding from the leftmost bit.

[0094] FIG. 2A to FIG. 2B is a conceptual diagram illustrating an example of encoding a block of video data. As shown in FIG. 2A , a current block of video data (e.g., a CB corresponding to a video component) is encoded by generating residual values from a set of prediction values, performing a transform on the residual values, and quantizing the transform coefficients to generate bitplane values. The bitplane values are encoded into a bitstream. As shown in FIG. 2B , a current block of video data is decoded by performing inverse quantization on the bitplane values, performing an inverse transform, and adding a set of prediction values to the resulting residual values. It should be noted that in the example of FIG. 2A to FIG. 2B , the sample values of the reconstructed block are different from the sample values of the current video block that was encoded. As such, the encoding can be considered lossy. However, the difference in sample values can be considered acceptable or imperceptible to a viewer of the reconstructed video.

[0095] Additionally, as FIG. 2A to FIG. 2BAs shown, the coefficient bit-shift values are generated using a scaling factor array. In ITU-T H.265, the scaling factor array is generated by selecting a scaling matrix and multiplying each entry in the scaling matrix by a quantization scaling factor. The scaling matrix can be selected based in part on the prediction mode and the color component. It should be noted that in some examples, the scaling matrix can provide the same value for each entry (i.e., scale all coefficients according to a single value). The value of the quantization scaling factor can be determined by a quantization parameter, QP. In addition, the QP value for a set of transform coefficients can be derived using a predicted quantization parameter value (which can be referred to as a predicted QP value or QP predicted value) and optionally signaled quantization parameter delta value (which can be referred to as a QP delta value or delta QP value). The quantization parameter can be updated for each CU, and a respective quantization parameter can be derived for each of the luma and chroma channels.

[0096] Referring to FIG. 1 In the example shown in JVET-O2001, each slice of video data included in Pic3 (i.e., slice 0 slice 1) is shown as being encapsulated in a NAL unit. In JVET-O2001, each of a video sequence, a GOP, a picture, a slice, and a CTU can be associated with metadata describing video coding properties. JVET-O2001 defines parameter sets that can be used to describe video data and / or video coding properties. Specifically, JVET-O2001 includes the following five types of parameter sets: a decoding parameter set (DPS), a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), and an adaptation parameter set (APS). In JVET-O2001, parameter sets can be encapsulated as special types of NAL units, or can be signaled as messages. NAL units that include encoded video data (e.g., slices) can be referred to as VCL (video coding layer) NAL units, and NAL units that include metadata (e.g., parameter sets) can be referred to as non-VCL NAL units. In addition, JVET-O2001 enables supplemental enhancement information (SEI) messages to be signaled. In JVET-O2001, SEI messages assist processes related to decoding, display, or other purposes, however, SEI messages can not be required to construct luma or chroma samples through the decoding process. In JVET-O2001, SEI messages can be signaled in the bitstream using non-VCL NAL units. In addition, SEI messages can be conveyed in some manner other than being present in the bitstream (i.e., signaled out-of-band). FIG. 4 An example of a bitstream including a plurality of CVSs is shown, where a CVS is represented by NAL units included in a respective access unit. In FIG. 4In the illustrated example, the non-VCL NAL units include respective parameter set NAL units (i.e., sequence parameter set (SPS) and picture parameter set (PPS) NAL units), SEI message NAL units, and access unit delimiter NAL units. It should be noted that, in FIG. 4 In the illustrated example, the non-VCL NAL units include respective parameter set NAL units (i.e., sequence parameter set (SPS) and picture parameter set (PPS) NAL units), SEI message NAL units, and access unit delimiter NAL units. It should be noted that, in

[0097] As noted above, for inter prediction coding, reference sample values located in one or more previously coded pictures can be used to predict a current video block. JVET-O2001 includes so-called triangular-based motion compensation. In triangular-based motion compensation, a rectangular video block is predicted using two triangular predictions. That is, to perform the prediction, the rectangular video block is split into two triangles about a diagonal line (from the top-left corner to the bottom-right corner) or about an inverse diagonal line (from the top-right corner to the bottom-left corner). Each triangle prediction can be generated using its own motion vector and reference frame index. Moreover, a rectangular prediction for predicting the rectangular video block can be generated by performing an adaptive weighting process on the diagonal edges of the neighboring triangle predictions. That is, the triangle predictions can be blended. A detailed description of an example of triangular-based motion compensation is provided in “CE10.3.1.b: Triangular prediction unit mode,” ISO / IEC JTC1 / SC29 / WG11 12th Meeting, Macau, CN, 3-12 Oct 2018, Document JVET-L0124-v2.

[0098] JVET-O2001 specifies the following decoding process for triangular inter blocks:

[0099] The inputs to this process are:

[0100] - a luma position (xCb, yCb) specifying the top-left sample of the current coding block relative to the top-left luma sample of the current picture,

[0101] - a variable cbWidth specifying the width of the current coding block in luma samples,

[0102] - a variable cbHeight specifying the height of the current coding block in luma samples,

[0103] - luma motion vectors mvA and mvB of 1 / 16 fractional sample accuracy,

[0104] - chroma motion vectors mvCA and mvCB,

[0105] - reference indices refldxA and refldxB,

[0106] - prediction list markers predListFlagA and predListFlagB.

[0107] The output of the process is:

[0108] - an array of (cbWidth) x (cbHeight) of luma prediction samples predSamples L ,

[0109] - an array of (cbWidth / SubWidthC) x (cbHeight / SubHeightC) of chroma prediction samples predSamples Cb ,

[0110] - an array of (cbWidth / SubWidthC) x (cbHeight / SubHeightC) of chroma prediction samples predSamples Cr .

[0111] Let predSamplesLA L and predSamplesLB L be arrays of (cbWidth) x (cbHeight) of luma prediction sample values, predSamplesLA Cb , predSamplesLB Cb , predSamplesLA Cr and predSamplesLB Cr be arrays of (cbWidth / SubWidthC) x (cbHeight / SubHeightC) of chroma prediction sample values.

[0112] predSamples L , predSamples Cb and predSamples Cr are derived using the following ordered steps:

[0113] 1. For each N of A and B, the following applies:

[0114] - the reference picture is derived by invoking the specified reference picture selection process with X having the setting equal to predListFlagN and refIdxX having the setting equal to refIdxN as inputs, the reference picture consisting of an ordered two-dimensional array of luma samples refPicLN L and two ordered two-dimensional arrays of chroma samples refPicLN Cb and refPicLN Cr .

[0115] - the array predSamplesLNL is derived by invoking the specified fractional sample interpolation process with luma position (xCb, yCb), the luma coding block width sbWidth set equal to cbWidth, the luma coding block height sbHeight set equal to cbHeight, the motion vector offset mvOffset set equal to (0, 0), the motion vector mvLX set equal to mvN, and the reference array refPicLXL set equal to refPicLN L , the variable bdofFlag set equal to FALSE, and the variable cldx set equal to 0 as inputs.

[0116] - the array predSamplesLNL is derived by invoking the specified fractional sample interpolation process with luma position (xCb, yCb), the coding block width sbWidth set equal to cbWidth / SubWidthC, the coding block height sbHeight set equal to cbHeight / SubHeightC, the motion vector offset mvOffset set equal to (0, 0), the motion vector mvLX set equal to mvCN, and the reference array refPicLX Cb set equal to refPicLN Cb , the variable bdofFlag set equal to FALSE, and the variable cldx set equal to 1 as inputs. Cb .

[0117] - the array predSamplesLNcr is derived by invoking the specified fractional sample interpolation process with luma position (xCb, yCb), the coding block width sbWidth set equal to cbWidth / SubWidthC, the coding block height sbHeight set equal to cbHeight / SubHeightC, the motion vector offset mvOffset set equal to (0, 0), the motion vector mvLX set equal to mvCN, and the reference array refPicLX Cr set equal to refPicLN Cr , the variable bdofFlag set equal to FALSE, and the variable cldx set equal to 2 as inputs.

[0118] 2. Set the split direction of the merge triangle mode variable triangleDir equal to merge_triangle_split_dir[ xCb ][ yCb ][ which indicates diagonal or inverse diagonal direction ].

[0119] 3. The prediction samples predSamples within the current luma coding block are derived by invoking the weighted sample prediction process for triangle merge mode specified by L [x L ][y L ], x L = 0..cbWidth - 1 and y L = 0..cbHeight - 1, with the coding block width nCbW set equal to cbWidth, the coding block height nCbH set equal to cbHeight, the sample arrays predSamplesLA L and predSamplesLB L , the variable triangleDir and cldx equal to 0 as inputs.

[0120] 4. The prediction samples predSamples within the current chroma component Cb coding block are derived by invoking the weighted sample prediction process for triangle merge mode specified by Cb [xc][yc], xc = 0..cbWidth / SubWidthC - 1 and yc = 0..cbHeight / SubHeightC - 1, with the coding block width nCbW set equal to cbWidth / SubWidthC, the coding block height nCbH set equal to cbHeight / SubHeightC, the sample arrays predSamplesLA Cb and predSamplesLB Cb , the variable triangleDir and cldx equal to 1 as inputs.

[0121] 5. The prediction samples predSamples within the current chroma component Cr coding block are derived by invoking the weighted sample prediction process for triangle merge mode specified by Cr [xc][yc], xc = 0..cbWidth / SubWidthC - 1 and yc = 0..cbHeight / SubHeightC - 1, with the coding block width nCbW set equal to cbWidth / SubWidthC, the coding block height nCbH set equal to cbHeight / SubHeightC, the sample arrays predSamplesLA Cr and predSamplesLB Cr , the variable triangleDir and cldx equal to 2 as inputs.

[0122] 6. The motion vector storage process for the merge triangle mode is invoked with the luma coding block position (xCb, yCb), the luma coding block width cbWidth, the luma coding block height cbHeight, the split direction triangleDir, the luma motion vectors mvA and mvB, the reference indices refldxA and refldxB, and the prediction list flags predListFlagA and predListFlagB as inputs.

[0123] Weighted sample prediction process for triangle merge mode

[0124] The inputs of this process are:

[0125] - two variables nCbW and nCbH specifying the width and height of the current coding block,

[0126] - two (nCbW) x (nCbH) arrays predSamplesLA and predSamplesLB,

[0127] - a variable triangleDir specifying the split direction,

[0128] - a variable cldx specifying the color component index.

[0129] The output of this process is an (nCbW) x (nCbH) array pbSamples of predicted sample values.

[0130] The variable nCbR is derived as follows:

[0131] nCbR = (nCbW > nCbH)? (nCbW / nCbH) : (nCbH / nCbW)

[0132] - if cldx is equal to 0, bitDepth is set equal to BitDepthy.

[0133] - otherwise, bitDepth is set equal to BitDepthc.

[0134] The variables shiftl and offsetl are derived as follows:

[0135] - variable shiftl is set equal to Max(5, 17 - bitDepth).

[0136] - variable offsetl is set equal to 1 « (shiftl - 1).

[0137] The prediction sample pbSamples[ x ][ y ], x = 0..nCbW - 1 and y = 0..nCbH - 1, is derived as follows depending on the values of triangleDir, wS and cldx:

[0138] The variable wldx is derived as follows:

[0139] If cldx is equal to 0 and triangleDir is equal to 0, the following applies:

[0140] wldx = ( nCbW > nCbH )? ( Clip3( 0, 8, ( x / nCbR - y ) + 4 ) ) : ( Clip3( 0, 8, ( x - y / nCbR ) + 4 ) )

[0141] Otherwise, if cldx is equal to 0 and triangleDir is equal to 1, the following applies:

[0142] wldx = ( nCbW > nCbH )? ( Clip3( 0, 8, ( nCbH - 1 - x / nCbR - y ) + 4 ) ) : ( Clip3( 0, 8, ( nCbW - 1 - x - y / nCbR ) + 4 ) )

[0143] Otherwise, if cldx is greater than 0 and triangleDir is equal to 0, the following applies:

[0144] wldx = ( nCbW > nCbH )? ( Clip3( 0, 4, ( x / nCbR - y ) + 2 ) ) : ( Clip3( 0, 4, ( x - y / nCbR ) + 2 ) )

[0145] Otherwise (if cldx is greater than 0 and triangleDir is equal to 1), the following applies:

[0146] wldx = ( nCbW > nCbH )? ( Clip3( 0, 4, ( nCbH - 1 - x / nCbR - y ) + 2 ) ) : ( Clip3( 0, 4, ( nCbW - 1 - x - y / nCbR ) + 2 ) )

[0147] The variable wValue specifying the height of the specified prediction sample is derived as follows using wldx and cldx:

[0148] wValue = ( cldx == 0 )? Clip3( 0, 8, wldx ) : Clip3( 0, 8, wldx * 2 )

[0149] The prediction sample value is derived as follows:

[0150] pbSamples[ x ][ y ] = Clip3( 0, ( 1 « bitDepth ) - 1, ( predSamplesLA[ x ][ y ] * wValue + predSamplesLB[ x ][ y ] * ( 8 - wValue ) + offsetl ) » shiftl )

[0151] It should be noted that in JVET-O2001, the weighted sample prediction process for triangle-based motion compensation can not be ideal. Specifically, the blending function used for triangle prediction does not account for chroma sampling formats other than 4:2:0. In one example, the blending function can be simplified and / or extended to accommodate actual chroma sampling formats (which can be 4:2:2 or 4:4:4) according to the techniques herein.

[0152] FIG. 5 FIG. 1 is a block diagram illustrating an example of a system that can be configured to encode (e.g., encode and / or decode) video data according to one or more techniques of this disclosure. System 100 represents an example of a system that can perform video encoding using the partitioning techniques according to one or more techniques of this disclosure. As shown, system 100 includes source device 102, communication medium 110, and destination device 120. In the example shown, source device 102 can include any device configured to encode video data and transmit the encoded video data to communication medium 110. Destination device 120 can include any device configured to receive encoded video data via communication medium 110 and decode the encoded video data. Source device 102 and / or destination device 120 can include computing devices equipped for wired and / or wireless communication, and can include set-top boxes, digital video recorders, televisions, desktop computers, laptop or tablet computers, game consoles, mobile devices including, for example, “smart” phones, cellular telephones, personal gaming devices, and medical imaging devices. FIG. 5 FIG. 5

[0153] ​​Communication media 110 can include any combination of wireless and wired communication media and / or storage devices. Communication media 110 can include coaxial cables, fiber optic cables, twisted pair cables, wireless transmitters and receivers, routers, switches, repeaters, base stations, or any other equipment that can be useful to facilitate communications between various devices and stations. Communication media 110 can include one or more networks. For example, communication media 110 can include a network configured to allow access to the World Wide Web, for example, the Internet. The network can operate according to one or more combinations of telecommunications protocols. The telecommunications protocols can include proprietary aspects and / or can include standardized telecommunications protocols. Examples of standardized telecommunications protocols include Digital Video Broadcasting (DVB) standards, Advanced Television Systems Committee (ATSC) standards, Integrated Services Digital Broadcasting (ISDB) standards, Data Over Cable Service Interface Specification (DOCSIS) standards, Global System for Mobile Communications (GSM) standards, Code Division Multiple Access (CDMA) standards, Third Generation Partnership Project (3GPP) standards, European Telecommunications Standards Institute (ETSI) standards, Internet Protocol (IP) standards, Wireless Application Protocol (WAP) standards, and Institute of Electrical and Electronics Engineers (IEEE) standards.

[0154] Storage devices can include any type of device or storage medium capable of storing data. Storage media can include tangible or non-transitory computer-readable media. Computer- readable media can include optical, flash memory, magnetic memory, or any other suitable digital memory media. In some examples, a memory device or portions thereof can be described as non-volatile storage, and in other examples, portions of memory devices can be described as volatile storage. Examples of volatile memory can include random access memory (RAM), dynamic random access memory (DRAM), and static random access memory (SRAM). Examples of non-volatile memory can include magnetic hard discs, optical discs, floppy discs, flash memories, or forms of electrically programmable memory (EPROM) or electrically erasable and programmable (EEPROM) memory. Storage devices can include memory cards (e.g., Secure Digital (SD) memory cards), internal / external hard disk drives, and / or internal / external solid state drives. Data can be stored on storage devices in accordance with defined file formats.

[0155] Referring again to FIG. 5The source device 102 includes a video source 104, a video encoder 106, and an interface 108. The video source 104 can include any device configured to capture and / or store video data. For example, the video source 104 can include a video camera and a storage device operably coupled to the video camera. The video encoder 106 can include any device configured to receive video data and generate a compliant bitstream representing the video data. A compliant bitstream can refer to a bitstream from which a video decoder can receive and reproduce video data. Aspects of a compliant bitstream can be defined according to a video coding standard. When generating a compliant bitstream, the video encoder 106 can compress the video data. The compression can be lossy (perceptible or non-perceptible) or lossless. The interface 108 can include any device configured to receive a compliant video bitstream and transmit and / or store the compliant video bitstream to a communication medium. The interface 108 can include a network interface card, such as an Ethernet card, and can include an optical transceiver, a radio frequency transceiver, or any other type of device that can send and / or receive information. Further, the interface 108 can include a computer system interface that can allow the compliant video bitstream to be stored on a storage device. For example, the interface 108 can include a chipset, such as a Peripheral Component Interconnect (PCI) and Peripheral Component Interconnect Express (PCIe) bus protocol chipset, a proprietary bus protocol chipset, a Universal Serial Bus (USB) protocol chipset, an I 2 or any other logical and physical structure(s) that can be used to interconnect peer devices.

[0156] Referring again to FIG. 5 The destination device 120 includes an interface 122, a video decoder 124, and a display 126. The interface 122 can include any device configured to receive a compliant video bitstream from a communication medium. The interface 108 can include a network interface card, such as an Ethernet card, and can include an optical transceiver, a radio frequency transceiver, or any other type of device that can receive and / or send information. Further, the interface 122 can include a computer system interface that can allow the compliant video bitstream to be retrieved from a storage device. For example, the interface 122 can include a chipset, such as a PCI and PCIe bus protocol chipset, a proprietary bus protocol chipset, a USB protocol chipset, an I 2 The video decoder 124 can include any device configured to receive a compliant bitstream and / or acceptable variations thereof, and reproduce video data therefrom. The display 126 can include any device configured to display video data. The display 126 can include one of various display devices such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display. The display 126 can include a high definition display or an ultra-high definition display. It should be noted that although the source device 102 and the destination device 120 are illustrated as separate devices, the source device 102 and the destination device 120 can be integrated in a single device. FIG. 3In the illustrated example, video decoder 124 is described as outputting data to display 126, but video decoder 124 can be configured to output video data to various types of devices and / or subcomponents thereof. For example, video decoder 124 can be configured to output video data to any communication medium, as described herein.

[0157] FIG. 6 is a block diagram illustrating an example of a video encoder 200 that can implement the techniques described herein for encoding video data. It should be noted that although the example video encoder 200 is illustrated as having distinct functional blocks, such an illustration is for descriptive purposes and does not limit video encoder 200 and / or its sub-components to a particular hardware or software architecture. The functions of video encoder 200 can be realized using any combination of hardware, firmware, and / or software implementations. In one example, video encoder 200 can be configured to encode video data in accordance with the techniques described herein. Video encoder 200 can perform intra-predictive encoding and inter-predictive encoding of picture regions and, as such, can be referred to as a hybrid video encoder. In FIG. 6 In the illustrated example, video encoder 200 receives a source video block. In some examples, the source video block can include a picture region that has been partitioned according to an encoding structure. For example, the source video data can include macroblocks, CTUs, CBs, sub-partitions thereof, and / or another equivalent encoding unit. In some examples, video encoder 200 can be configured to perform additional partitioning of the source video block. It should be noted that regardless of how the source video data is partitioned prior to and / or during encoding, some of the techniques described herein can be generally applicable to video encoding. FIG. 6 In the illustrated example, video encoder 200 includes summer 202, transform coefficient generator 204, coefficient quantization unit 206, inverse quantization / transform processing unit 208, summer 210, intra-prediction processing unit 212, inter-prediction processing unit 214, filter unit 216, and entropy encoding unit 218.

[0158] As FIG. 6As shown, video encoder 200 receives source video blocks and outputs a bitstream. Video encoder 200 can generate residual data by subtracting a prediction video block from a source video block. Summer 202 represents the component configured to perform this subtraction operation. In one example, the subtraction of video blocks occurs in the pixel domain. Transform coefficient generator 204 applies a transform, such as a Discrete Cosine Transform (DCT), a Discrete Sine Transform (DST), or a conceptually similar transform, to its residual block or sub-partition (e.g., four 8x8 transforms can be applied to a 16x16 array of residual values) to generate a set of residual transform coefficients. Transform coefficient generator 204 can be configured to perform any and all combinations of the transforms included in the family of discrete trigonometric transforms. As described above, in ITU-T H.265, TBs are restricted to the following sizes 4x4, 8x8, 16x16, and 32x32. In one example, transform coefficient generator 204 can be configured to perform transforms according to arrays of size 4x4, 8x8, 16x16, and 32x32. In one example, transform coefficient generator 204 can be further configured to perform transforms according to arrays of other sizes. Specifically, in some cases, it can be useful to perform transforms on rectangular arrays of different values. In one example, transform coefficient generator 204 can be configured to perform transforms according to the following array sizes: 2x2, 2x4N, 4Mx2, and / or 4Mx4N. In one example, a two-dimensional (2D) MxN inverse transform can be implemented as a one-dimensional (1D) M-point inverse transform followed by a 1D N-point inverse transform. In one example, a 2D inverse transform can be implemented as a 1D N-point vertical transform followed by a 1D N-point horizontal transform. In one example, a 2D inverse transform can be implemented as a 1D N-point horizontal transform followed by a 1D N-point vertical transform. Transform coefficient generator 204 can output transform coefficients to coefficient quantization unit 206.

[0159] Coefficient quantization unit 206 can be configured to perform quantization of transform coefficients. As described above, the degree of quantization can be modified by adjusting a quantization parameter. Coefficient quantization unit 206 can be further configured to determine a quantization parameter and output QP data (e.g., data used to determine quantization group sizes and / or delta QP values) that a video decoder can use to reconstruct the quantization parameter to perform inverse quantization during video decoding. It should be noted that in other examples, one or more additional or alternative parameters can be used to determine a quantization bit- shift (e.g., a scaling factor). The techniques described herein can generally be applied to determining a quantization bit-shift for transform coefficients corresponding to one component of video data based on a quantization bit-shift for transform coefficients corresponding to another component of video data.

[0160] Referring again to FIG. 6quantized transform coefficients are output to inverse quantization / transform processing unit 208. Inverse quantization / transform processing unit 208 can be configured to apply inverse quantization and inverse transform to generate reconstructed residual data. As shown in FIG. 6 summer 210. In this way, the encoded video block can be reconstructed and the resulting reconstructed video block can be used to evaluate the coding quality of a given prediction, transform, and / or quantization. Video encoder 200 can be configured to perform multiple coding passes (e.g., perform encoding while varying one or more of the prediction, transform parameters, and quantization parameters). Rate-distortion or other system parameters of the bitstream can be optimized based on the evaluation of the reconstructed video block. Moreover, the reconstructed video block can be stored and used as a reference to predict subsequent blocks.

[0161] As noted above, video blocks can be encoded using intra prediction. Intra prediction processing unit 212 can be configured to select an intra prediction mode for a video block to be encoded. Intra prediction processing unit 212 can be configured to evaluate the frame and / or regions thereof and determine an intra prediction mode to use to encode the current block. As shown in FIG. 6 Intra prediction processing unit 212 outputs intra prediction data (e.g., syntax elements) to entropy encoding unit 218 and transform coefficient generator 204. As noted above, the transform performed on the residual data can depend on the mode. As noted above, possible intra prediction modes can include a planar prediction mode, a DC prediction mode, and an angular prediction mode. Moreover, in some examples, prediction for chroma components can be inferred from intra prediction for luma prediction modes.

[0162] Inter prediction processing unit 214 can be configured to perform inter prediction encoding for a current video block. Inter prediction processing unit 214 can be configured to receive a source video block and calculate motion vectors for PUs of the video block. Motion vectors can indicate a displacement of a PU (or similar coding structure) of a video block within a current video frame relative to a prediction block within a reference frame. Inter prediction encoding can use one or more reference pictures. Moreover, motion prediction can be single-directional prediction (using one motion vector) or bi-directional prediction (using two motion vectors). Inter prediction processing unit 214 can be configured to select a prediction block by calculating pixel differences determined by, for example, sum of absolute difference (SAD), sum of squared difference (SSD), or other difference metric. As noted above, motion vectors can be determined and specified according to motion vector prediction. As noted above, inter prediction processing unit 214 can be configured to perform motion vector prediction. Inter prediction processing unit 214 can be configured to generate a prediction block using motion prediction data. For example, inter prediction processing unit 214 can locate a prediction video block within a frame buffer FIG. 6The inter prediction processing unit 214 can be further configured to apply one or more interpolation filters to the reconstructed residual block to calculate sub-integer pixel values for motion estimation. The inter prediction processing unit 214 can output the motion prediction data of the calculated motion vectors to the entropy encoding unit 218. As shown, the inter prediction processing unit 214 can receive the reconstructed video block via the filter unit 216. FIG. 6

[0163] As noted above, the weighted sample prediction process for triangle-based motion compensation can not be ideal in JVET-O2001. In one example, in accordance with the techniques herein, the weighted sample prediction process for triangle-based motion compensation can determine a size of a prediction sample array based on a color component index and a sampling format. In one example, in accordance with the techniques herein, the weighted sample prediction process for triangle-based motion compensation can be based on the following:

[0164] Weighted sample prediction process for triangle merge mode

[0165] The inputs to this process are:

[0166] - two variables nCbW and nCbH specifying the width and height of the current code block,

[0167] - two (nCbW) x (nCbH) arrays predSamplesLA and predSamplesLB,

[0168] - a variable triangleDir specifying the partition direction,

[0169] - a variable cldx specifying the color component index.

[0170] The output of this process is an (nCbW) x (nCbH) array pbSamples of prediction sample values. The variables nCbWH and nCbHW are derived as follows:

[0171] nCbWH = Max(1, nCbW / nCbH)

[0172] nCbHW = Max(1, nCbH / nCbW)

[0173] Depending on the value of cldx, the variables bitDepth, scale are derived as follows:

[0174] - if cldx is equal to 0, set bitDepth equal to BitDepth Y and set scale equal to 1.

[0175] ​- Otherwise, set bitDepth equal to BitDepthc, set scale equal to 1 if Min(nCbW, nCbH) is equal to Min(nCbW * SubWidthC, nCbH * SubHeightC), and set scale equal to 2 otherwise.

[0176] The variables shiftl and offsetl are derived as follows:

[0177] - Set the variable shiftl equal to Max(5, 17 - bitDepth).

[0178] - Set the variable offsetl equal to 1 « (shiftl - 1).

[0179] Depending on the value of triangleDir, the prediction sample pbSamples[x][y], x = 0..nCbW - 1 and y = 0..nCbH - 1, is derived as follows:

[0180] - The variable wldx is derived as follows:

[0181] - If triangleDir is equal to 0, the following applies:

[0182] wldx = x / nCbWH * scale - y / nCbHW * scale

[0183] - Otherwise (if triangleDir is equal to 1), the following applies:

[0184] wldx = (nCbW - 1 - x) / nCbWH * scale - y / nCbHW * scale

[0185] - The variable wValue specifying the height of the prediction sample is derived as follows using wldx and cldx:

[0186] wValue = Clip3(0, 8, wldx + 4)

[0187] - The prediction sample value is derived as follows:

[0188] pbSamples[x][y] = Clip3(0, (1 « bitDepth) - 1, (predSamplesLA[x][y] * wValue + predSamplesLB[x][y] * (8 - wValue) + offsetl) » shiftl)

[0189] It should be noted that in the case of a sampling format of 4:2:2, for a video block having 16x8 luma samples, the corresponding chroma array is 8x8 (widthxheight). According to the weighted sample prediction process specified in JVET-O2001, for triangleDir equal to 0, the following blending matrices (wValue values) are produced for luma and chroma: 4 4 5 5 6 6 7 7 8 8 8 8 8 8 8 8 3 3 4 4 5 5 6 6 7 7 8 8 8 8 8 8 2 2 3 3 4 4 5 5 6 6 7 7 8 8 8 8 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 0 0 1 1 2 2 3 3 4 4 5 5 6 6 7 7 0 0 0 0 1 1 2 2 3 3 4 4 5 5 6 6 0 0 0 0 0 0 1 1 2 2 3 3 4 4 5 5 0 0 0 0 0 0 0 0 1 1 2 2 3 3 4 4 4 6 8 8 8 8 8 8 2 4 6 8 8 8 8 8 0 2 4 6 8 8 8 8 0 0 2 4 6 8 8 8 0 0 0 2 4 6 8 8 0 0 0 0 2 4 6 8 0 0 0 0 0 2 4 6 0 0 0 0 0 0 2 4

[0206] It should be noted that in this case, the horizontal upscaled chroma matrix will not produce a matrix similar to the luma matrix.

[0207] According to the weighted sample prediction process specified in accordance with the techniques herein, for triangleDir equal to 0, the following blending matrices (wValue values) are produced for luma and chroma: 4 4 5 5 6 6 7 7 8 8 8 8 8 8 8 8 3 3 4 4 5 5 6 6 7 7 8 8 8 8 8 8 2 2 3 3 4 4 5 5 6 6 7 7 8 8 8 8 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 0 0 1 1 2 2 3 3 4 4 5 5 6 6 7 7 0 0 0 0 1 1 2 2 3 3 4 4 5 5 6 6 0 0 0 0 0 0 1 1 2 2 3 3 4 4 5 5 0 0 0 0 0 0 0 0 1 1 2 2 3 3 4 4 4 5 6 7 8 8 8 8 3 4 5 6 7 8 8 2 3 4 5 6 7 8 1 2 3 4 5 6 7 0 1 2 3 4 5 6 0 0 1 2 3 4 5 0 0 0 1 2 3 4 0 0 0 0 1 2 3

[0224] Thus, the weighted sample prediction process specified according to JVET-O2001 and the weighted sample prediction process specified according to the techniques herein can produce different wValue values in some cases. For example, for the above case, for chroma, at (x, y) = (1, 2), according to the weighted sample prediction process specified according to JVET-O2001, wValue equals 2, while according to the techniques herein, wValue equals 3. The different wValue values result in different arrays pbSamples[x][y]. The array pbSamples[x][y] produced according to the techniques herein can improve coding efficiency in some cases.

[0225] As such, video encoder 200 represents an example of a device configured to receive a first array of prediction sample values; receive a second array of prediction sample values; determine a scaling value based on a color component index value and a video sampling format of video data; generate a third array of prediction sample values by applying a blending matrix to the first array of prediction sample values and the second array of prediction sample values, wherein the blending matrix is based on the scaling value; and perform video encoding using the third array of prediction sample values.

[0226] As FIG. 6As shown, filter unit 216 receives the reconstructed video blocks and coding parameters and outputs modified reconstructed video data. Filter unit 216 can be configured to perform deblocking, sample adaptive offset (SAO) filtering, adaptive loop filtering (ALF), and the like. SAO filtering is a non-linear amplitude mapping that can be used to improve reconstruction by adding an offset to the reconstructed video data. It should be noted that, as FIG. 5 As shown, intra prediction processing unit 212 and inter prediction processing unit 214 can receive the modified reconstructed video blocks via filter unit 216. Entropy encoding unit 218 receives quantized transform coefficients and prediction syntax data (i.e., intra prediction data and motion prediction data). Entropy encoding unit 218 can be configured to perform entropy encoding in accordance with one or more of the techniques described herein.

[0227] FIG. 7 is a block diagram illustrating an example of a video decoder that can be configured to decode video data in accordance with one or more techniques of this disclosure. In one example, video decoder 300 can be configured to reconstruct video data based on one or more of the techniques described above. That is, video decoder 300 can operate in a reciprocal manner to video encoder 200 described above. Video decoder 300 can be configured to perform intra prediction decoding and inter prediction decoding, and thus can be referred to as a hybrid decoder. In FIG. 7 In the example shown, video decoder 300 includes an entropy decoding unit 302, an inverse quantization unit 304, an inverse transform processing unit 306, an intra prediction processing unit 308, an inter prediction processing unit 310, a summer 312, a filter unit 314, and a reference buffer 316. Video decoder 300 can be configured to decode video data in a manner consistent with a video coding system that can implement one or more aspects of a video coding standard. It should be noted that although the example video decoder 300 is shown with distinct functional blocks or units, such an illustration is for descriptive purposes only and does not limit video decoder 300 and / or its sub-components to a particular hardware or software architecture. The functionality of video decoder 300 can be implemented using any combination of hardware, firmware, and / or software implementations.

[0228] As FIG. 7 shown, entropy decoding unit 302 receives an entropy encoded bitstream. Entropy decoding unit 302 can be configured to decode quantized syntax elements and quantized coefficients from the bitstream according to a process reciprocal to the entropy encoding process. Entropy decoding unit 302 can be configured to perform entropy decoding in accordance with any of the entropy encoding techniques described above. Entropy decoding unit 302 can parse the encoded bitstream in a manner consistent with a video coding standard. Video decoder 300 can be configured to parse an encoded bitstream, where the encoded bitstream is generated based on the techniques described above.

[0229] Referring again to FIG. 3, FIG. 7 Quantization parameter data can include any and all combinations of the delta QP values and / or quantization group size values described above. Video decoder 300 and / or inverse quantization unit 304 can be configured to determine QP values for inverse quantization based on values signaled by the video encoder and / or by video properties and / or coding parameters. That is, inverse quantization unit 304 can operate in a reciprocal manner to quantization unit 206 described above. For example, inverse quantization unit 304 can be configured to infer predetermined values, allowed quantization group sizes, derive quantization parameters, etc. according to the techniques described above. Inverse quantization unit 304 can be configured to apply inverse quantization. Inverse transform processing unit 306 can be configured to perform inverse transforms to generate reconstructed residual data. The techniques performed by inverse quantization unit 304 and inverse transform processing unit 306 can be similar to the techniques performed by inverse quantization / transform processing unit 208 described above. Inverse transform processing unit 306 can be configured to apply inverse DCTs, inverse DSTs, inverse integer transforms, inverse non-separable secondary transforms (NSSTs), or conceptually similar inverse transform processes to transform coefficients in order to produce a residual block in the pixel domain. Moreover, as described above, whether a particular transform is performed (or the type of particular transform) can depend on the intra prediction mode. As shown in FIG. 3, the reconstructed residual data can be provided to summer 312. Summer 312 can add the reconstructed residual data to a predicted video block and generate reconstructed video data. The predicted video block can be determined according to the prediction video techniques (i.e., intra prediction and inter prediction). FIG. 7

[0230] ​Intra prediction processing unit 308 can be configured to receive intra prediction syntax elements and retrieve a predicted video block from reference buffer 316. Reference buffer 316 can comprise a memory device configured to store one or more frames of video data. The intra prediction syntax elements can identify an intra prediction mode, such as the intra prediction modes described above. In one example, intra prediction processing unit 308 can reconstruct the video block according to one or more of the intra prediction coding techniques described herein. Inter prediction processing unit 310 can receive inter prediction syntax elements and generate motion vectors to identify a predicted block in one or more reference frames stored in reference buffer 316. Inter prediction processing unit 310 can generate a motion compensated block, possibly performing interpolation based on an interpolation filter. An identifier for an interpolation filter used for motion estimation with sub-pixel accuracy can be included in the syntax elements. Inter prediction processing unit 310 can use the interpolation filter to calculate interpolated values for sub-integer pixels of the reference block. In one example, inter prediction processing unit 310 can reconstruct the video block using one or more of the inter prediction coding techniques described herein. Filter unit 314 can be configured to perform filtering on the reconstructed video data. For example, filter unit 314 can be configured to perform deblocking and / or SAO filtering, as described above with respect to filter unit 216. In addition, it should be noted that, in some examples, filter unit 314 can be configured to perform specialized arbitrary filtering (e.g., visual enhancement). As FIG. 7 As shown, video decoder 300 can output a reconstructed video block. As such, video decoder 300 represents an example of a device configured to receive an array of first prediction sample values; receive an array of second prediction sample values; determine a scaling value based on a color component index value and a video sampling format of the video data; generate an array of third prediction sample values by applying a blending matrix to the array of first prediction sample values and the array of second prediction sample values, wherein the blending matrix is based on the scaling value; and perform video coding using the array of third prediction sample values.

[0231] In one or more examples, the functions described can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions can be stored on or transmitted over as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media can include computer-readable storage media, which corresponds to a tangible medium such as data storage media, or communication media including any medium that facilitates transfer of a computer program from one place to another, e.g., according to a communication protocol. In this manner, computer- readable media generally can correspond to (1) tangible computer-readable storage media which is non-transitory or (2) a communication medium such as a signal or carrier wave. Data storage media can be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code and / or data structures for implementation of the techniques described in this disclosure. A computer program product can include a computer-readable medium.

[0232] By way of example, and not limitation, such computer-readable storage media can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage, or other magnetic storage devices, flash memory, or any other storage medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any

[0233] Instructions can be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Accordingly, the term "processor," as used herein can refer to any of the foregoing structure or any other structure suitable for implementation of the techniques described herein. In addition, in some aspects, the functionality described herein can be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated in a combined codec. Also, the techniques could be fully implemented in one or more circuits or logic elements.

[0234] The techniques of this disclosure can be implemented in a variety of devices or apparatuses including a wireless phone, a set top box, an integrated circuit (IC) or a set of ICs (e.g., a chip set). Various components, modules, or units described in the disclosure can be implemented as hardware, software, or firmware, or any combination thereof. Hardware implementations can include digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), general purpose processors, or other programmable logic devices. Software implementations can include routines executed using one or more processors, such as a central processing unit (CPU), an ASIC, an FPGA, or the like. Hardware and software implementations are equally effective.

[0235] Furthermore, each functional block or various features of the base station device and the terminal device used in each of the above-described embodiments can be implemented as an integrated circuit or a plurality of integrated circuits that are constituted as an integrated circuit core. Also, the integrated circuit core can be provided as a system on a chip (SoC) that is a system in which an integrated circuit or a plurality of integrated circuits and a processor are integrated on the same substrate.

[0236] Various examples have been described. These and other examples are within the scope of the following claims.

[0237] <SUMMARY>

[0238] In one example, a method of encoding video data includes receiving a first array of prediction sample values; receiving a second array of prediction sample values; determining a scaling value based on a color component index value and a video sample format of the video data; generating a third array of prediction sample values by applying a blending matrix to the first array of prediction sample values and the second array of prediction sample values, wherein the blending matrix is based on the scaling value; and performing video encoding using the third array of prediction sample values.

[0239] In one example, the method is provided wherein performing video encoding using the third array of prediction sample values includes decoding the video data by adding a residual to the third array of prediction sample values.

[0240] In one example, the method is provided further comprising wherein performing video encoding using the third array of prediction sample values includes encoding the video data by subtracting the third array of prediction sample values from a current video block.

[0241] In one example, the method is provided wherein the color component index value indicates a chroma component and the video sample format of the video data is 4:2:2.

[0242] In one example, the method is provided wherein the color component index value indicates a chroma component and the video sample format of the video data is 4:4:4.

[0243] In one example, an apparatus for encoding video data includes one or more processors configured to perform any and all combinations of the steps.

[0244] In one example, the apparatus is provided wherein the apparatus includes a video encoder.

[0245] In one example, the apparatus is provided wherein the apparatus includes a video decoder.

[0246] In one example, a system includes an apparatus including a video encoder; and the apparatus includes a video decoder.

[0247] In one example, an apparatus for encoding video data includes means for performing any and all combinations of the steps.

[0248] In one example, a non-transitory computer-readable storage medium including instructions stored thereon that, when executed, cause one or more processors of an apparatus for encoding video data to perform any and all combinations of the steps.

[0249] In one example, a method of decoding video data includes receiving a first array of prediction sample values, receiving a second array of prediction sample values, determining a scaling value based on a color component index value and a video sampling format of the video data, and generating a third array of prediction sample values by applying a blending value to the first array of prediction sample values and the second array of prediction sample values, wherein the blending value is based on the scaling value.

[0250] In one example, the method is provided where determining the scaling value based on the color component index value and the video sampling format of the video data includes determining a scaling value of 2 for a horizontal component and a scaling value of 1 for a vertical component if the color component index value indicates a chroma component and the video sampling format of the video data is 4:2:2.

[0251] In one example, the method is provided where determining the scaling value based on the color component index value and the video sampling format of the video data includes determining a scaling value of 1 for a horizontal component and a scaling value of 1 for a vertical component if the color component index value indicates a chroma component and the video sampling format of the video data is 4:4:4.

[0252] In one example, an apparatus comprising one or more processors configured to receive a first array of prediction sample values, receive a second array of prediction sample values, determine a scaling value based on a color component index value and a video sampling format of the video data, and generate a third array of prediction sample values by applying a blending value to the first array of prediction sample values and the second array of prediction sample values, wherein the blending value is based on the scaling value.

[0253] In one example, the apparatus is provided where determining the scaling value based on the color component index value and the video sampling format of the video data includes determining a scaling value of 2 for a horizontal component and a scaling value of 1 for a vertical component if the color component index value indicates a chroma component and the video sampling format of the video data is 4:2:2.

[0254] In one example, the apparatus is provided where determining the scaling value based on the color component index value and the video sampling format of the video data includes determining a scaling value of 1 for a horizontal component and a scaling value of 1 for a vertical component if the color component index value indicates a chroma component and the video sampling format of the video data is 4:4:4.

[0255] In one example, the apparatus is provided where the apparatus comprises a video decoder.

[0256] <cross-reference>

[0257] This nonprovisional application claims priority under 35 U.S.C. § 119 to provisional application 62 / 896,500, filed September 5, 2019, the entire contents of which are hereby incorporated by reference.

Claims

1. A method of decoding video data, the method comprising: receiving a first array of prediction sample values, wherein the first array of prediction sample values is derived from a fractional sample interpolation process using a first motion vector; receiving a second array of prediction sample values, wherein the second array of prediction sample values is derived from the fractional sample interpolation process using a second motion vector; based on a color component index value not being equal to 0, scaling a horizontal position and a vertical position according to a video sampling format of the video data, and using the horizontal position and the vertical position to generate a blending value, wherein the blending value specifies a weight of a prediction sample; and generating a third array of prediction sample values by using a value obtained by adding (1) a product of the blending value and the first array of prediction sample values, and (2) a product of (i) eight minus the blending value and (ii) the second array of prediction sample values.

2. The method of claim 1, wherein scaling a horizontal position and a vertical position comprises: based on different size values, scaling the horizontal position and the vertical position in a case that the color component index value indicates a chroma component and the video sampling format of the video data is 4:2:

2.

3. The method of claim 1, wherein scaling a horizontal position and a vertical position comprises: based on same size values, scaling the horizontal position and the vertical position in a case that the color component index value indicates a chroma component and the video sampling format of the video data is 4:4:

4.

4. A device comprising one or more processors configured to: receiving a first array of prediction sample values, wherein, receive a first array of prediction sample values, wherein the first array of prediction sample values is derived from a fractional sample interpolation process using a first motion vector; receive a second array of prediction sample values, wherein the second array of prediction sample values is derived from the fractional sample interpolation process using a second motion vector; based on a color component index value not being equal to 0, scale a horizontal position and a vertical position according to a video sampling format of the video data, and use the horizontal position and the vertical position to generate a blending value, wherein the blending value specifies a weight of a prediction sample; and generate a third array of prediction sample values by using a value obtained by adding (1) a product of the blending value and the first array of prediction sample values, and (2) a product of (i) eight minus the blending value and (ii) the second array of prediction sample values.

5. The device of claim 4, wherein the one or more processors, based on different size values, scale the horizontal position and the vertical position in a case that the color component index value indicates a chroma component and the video sampling format of the video data is 4:2:

2.

6. The device of claim 4, wherein the one or more processors, based on same size values, scale the horizontal position and the vertical position in a case that the color component index value indicates a chroma component and the video sampling format of the video data is 4:4:

4.

7. The apparatus of claim 4, wherein the apparatus is a video decoder.

Images