Alignment of prediction weights in video coding

Abstract

A method of video processing is described. The method includes determining, for conversion between a current block of video and a codec representation of the video, suitability of a geometric partitioning mode for the current block based on characteristics of the current block; and performing the conversion based on the determination.

Description

[0001] Cross-reference to related applications

[0002] In accordance with the patent law and / or rules applicable under the Paris Convention, this application promptly claims priority and benefit to International Patent Application No. PCT / CN2019 / 103903, filed September 1, 2019, and International Patent Application No. PCT / CN2019 / 110490, filed October 10, 2019. The entire disclosure of the foregoing applications is incorporated herein by reference as part of the disclosure of this application. Technical Field

[0003] This patent document relates to video encoding and decoding. Background Art

[0004] Despite advancements in video compression, digital video still accounts for the largest share of bandwidth usage on the Internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth demand for digital video is expected to continue to grow. Summary of the Invention

[0005] The apparatus, system, and method relate to digital video encoding and decoding, and more particularly to video and image encoding and decoding, wherein inter-frame prediction is used by segmenting video blocks into triangles and arbitrary geometries.

[0006] In one example aspect, a method for video processing is disclosed. The method includes: determining chroma weights according to rules, the chroma weights being used to determine a chroma prediction block of a current block of a video by predicting a mixed chroma block; and performing a conversion between the current block and a codec representation of the video based on the determination, wherein the rules specify the determination of the chroma weights from the luminance weights of the co-bit luminance blocks of the current block; wherein the current block is encoded using a geometric segmentation mode.

[0007] In another example aspect, a different method for video processing is disclosed. The method includes: determining chroma weights according to rules used to determine a chroma prediction block of the chroma block of a current block of a video by predicting a mixed chroma block; and performing a conversion between the current block and a codec representation of the video based on the determination, wherein the rules depend on the characteristics of co-bit luma blocks and / or the characteristics of the current block, wherein the current block is encoded and decoded using a geometric segmentation mode.

[0008] In another example, a different method for video processing is disclosed. This method includes: performing a conversion between a current block of video and a codec representation of the video, wherein during the conversion, a predicted block of the current block is determined by mixing the predictions of the current block according to a mixing weight mask, wherein the current block is codec-decoded using a geometric segmentation pattern.

[0009] In another example, a different method for video processing is disclosed. This method includes: performing a conversion between a current block of video and a codec representation of the video; determining the applicability of a geometric segmentation mode to the current block based on its characteristics; and performing the conversion based on the determination.

[0010] In another example, a different method for video processing is disclosed. This method includes performing a conversion between a current block of video and a codec representation of the video, wherein the codec representation conforms to format rules specifying that, if a geometric segmentation mode is disabled for the current block during the conversion, syntax elements describing the geometric segmentation mode are not included in the codec representation.

[0011] In another representative aspect, the above methods are implemented in processor-executable code and stored in a computer-readable medium.

[0012] In another representative aspect, apparatus configured or operable to perform the methods described above is disclosed. The apparatus may include a processor programmed to implement the methods.

[0013] In another representative aspect, video decoder devices can implement the methods described in this paper.

[0014] The above or other aspects and features of the disclosed technology will be described in more detail in the drawings, description and claims. Attached Figure Description

[0015] Figure 1 Example locations of spatial domain merging candidates are shown.

[0016] Figure 2 An example of candidate pairs considered for redundancy checks of spatial domain merging candidates is shown.

[0017] Figure 3 This is a diagram illustrating the scaling of motion vectors in the temporal merging candidate.

[0018] Figure 4 These are examples of candidate positions for temporal merging candidates C0 and C1.

[0019] Figure 5 This is an example of inter-frame prediction based on triangulation.

[0020] Figure 6 An example of unidirectional prediction MV selection for the triangular segmentation pattern is shown.

[0021] Figure 7 An example of the weights used in the mixing process is shown.

[0022] Figure 8 An example of the weights used in the mixing process of an 8x16 TPM block (WD6) is shown.

[0023] Figure 9 Example weights are shown for setting up an 8x16 TPM prediction block.

[0024] Figure 10 This is a block diagram of an example video processing system in which the techniques disclosed herein can be implemented.

[0025] Figure 11 This is a block diagram illustrating an example implementation of a hardware platform for video processing.

[0026] Figure 12 This is a flowchart of an example method for video processing.

[0027] Figure 13A An example of a TPM design in VTM6.0 is shown.

[0028] Figure 13B An example of a TPM design proposal is shown.

[0029] Figure 14 An example of GEO boundary description is shown.

[0030] Figure 15A An example of 32 angle schemes for angle quantization of the geometric merging pattern is shown.

[0031] Figure 15B Examples of 24 angle schemes for angle quantization of the geometric merging pattern are shown.

[0032] Figures 16A to 16D A flowchart of an example method for video processing is shown. Detailed Implementation

[0033] The embodiments of the disclosed techniques can be applied to existing video codec standards (e.g., HEVC, H.265) and future standards to improve compression performance. Section headings in this document are used to improve readability and do not in any way limit the discussion and embodiments (and / or implementations) to the respective sections. 1. Summary of the Invention

[0035] This document covers video encoding and decoding technologies. Specifically, it concerns techniques related to inter-frame prediction and video encoding and decoding. These techniques can be applied to existing video encoding and decoding standards like HEVC or to standards yet to be finalized (Multi-Functional Video Codec). They may also be applicable to future video standards or video codecs.

[0036] 2. Initial Discussion

[0037] Video codec standards have primarily evolved through the development of well-known ITU-T and ISO / IEC standards. ITU-T developed H.261 and H.263, while ISO / IEC developed MPEG-1 and MPEG-4 Visual. The two organizations jointly developed the H.262 / MPEG-2 video standard, the H.264 / MPEG-4 Advanced Video Coding (AVC) standard, and the H.265 / HEVC standard. Starting with H.262, video codec standards are based on a hybrid video codec architecture, utilizing temporal prediction plus transform coding. To explore future video codec technologies beyond HEVC, the Joint Video Exploration Team (JVET) was jointly established by VCEG and MPEG in 2015. Since then, JVET has adopted many new methods and incorporated them into reference software called the Joint Exploration Model (JEM). JVET meetings are held quarterly, and the goal of new codec standards is to reduce the bitrate by 50% compared to HEVC. At the JVET meeting in April 2018, the new video codec standard was officially named Multifunctional Video Codec (VVC), and the first version of the VVC Test Model (VTM) was released at that time. With continuous efforts to contribute to VVC standardization, new codec technologies have been adopted into the VVC standard at each JVET meeting. Then, the VVC working draft and test model VTM are updated after each meeting. The VVC project now aims for Technical Completion (FDIS) at the meeting in June 2020.

[0038] 2.1. Extended Combined Forecast

[0039] In VTM, a list of merge candidates is constructed sequentially by including the following five categories:

[0040] 1) Airspace MVP from the airspace neighboring CU

[0041] 2) Temporal MVP from co-located CU

[0042] 3) Historical MVPs from FIFO tables

[0043] 4) Paired average MVP

[0044] 5) Zero MV.

[0045] The signaling in the stripe header informs the size of the merge list, and the maximum allowed size of the merge list in the VTM is 6. For each CU encoded in merge mode, the index of the best merge candidate is encoded using truncated unary binarization (TU). The first binary number (bin) of the merge index is encoded using context encoding, and bypass encoding is used for the other binary numbers.

[0046] This section provides the process for generating each type of merge candidate.

[0047] 2.1.1. Spatial Domain Candidate Derivation

[0048] The derivation of spatial domain merging candidates in VVC is the same as that in HEVC. Figure 1 Up to four merge candidates are selected from the candidates located at the positions depicted. The derived order is A0, B0, B1, A1, and B2. Position B2 is considered only if any CU at positions A0, B0, B1, or A1 is unavailable (e.g., because it belongs to another stripe or slice) or if it is an intra-frame codec. After adding the candidate at position A1, a redundancy check is performed on the addition of the remaining candidates. This redundancy check ensures that candidates with the same motion information are excluded from the list, thereby improving codec efficiency. To reduce computational complexity, not all possible candidate pairs are considered in the aforementioned redundancy check. Instead, only those with the same motion information are considered. Figure 2 The arrows in the list link pairs, and a candidate is added to the list only if the corresponding candidate used for redundancy checking does not have the same motion information.

[0049] 2.1.2. Temporal Candidate Derivation

[0050] In this step, only one candidate is added to the list. Specifically, in the export of this temporal merging candidate, scaled motion vectors are exported based on the co-located CU belonging to the co-located reference image. The list of reference images to be used for the export of the co-located CU is explicitly signaled in the strip header. Figure 3 The dashed lines in the diagram represent the scaled motion vectors used to obtain temporal merge candidates. These vectors are scaled from the motion vectors of the co-located CUs using POC distances tb and td, where tb is defined as the POC difference between the current image and the reference image of the current image, and td is defined as the POC difference between the reference image and the co-located image of the co-located image. The reference image index of the temporal merge candidate is set to zero.

[0051] Figure 3 This is a diagram illustrating the scaling of motion vectors in the temporal merging candidate.

[0052] The position of the time-domain candidate is selected between candidate C0 and C1, such as Figure 4 As depicted. If the CU at position C0 is unavailable, is intra-frame encoded, or is outside the current line of the CTU, then position C1 is used. Otherwise, position C0 is used in the export of the temporal merge candidate.

[0053] Figure 4 Examples of candidate positions C0 and C1 for time-domain merging are shown.

[0054] 2.1.3. Historical Merge Candidate Derivation

[0055] Historically based MVP (HMVP) merge candidates are added to the merge list following the spatial MVP and TMVP. In this method, motion information from previous codec blocks is stored in a table and used as the MVP for the current CU. A table with multiple HMVP candidates is maintained during the encoding / decoding process. The table is reset (cleared) when a new CTU row is encountered. Whenever a non-sub-block intra-codec CU exists, the associated motion information is added to the last entry of the table as a new HMVP candidate.

[0056] In VTM, the HMVP table size S is set to 6, indicating that up to 6 history-based MVP (HMVP) candidates can be added to the table. When a new motion candidate is inserted into the table, a first-in, first-out (FIFO) rule of constraints is utilized, where a redundancy check is first applied to look for the existence of the same HMVP in the table. If found, the same HMVP is removed from the table and all subsequent HMVP candidates are moved forward.

[0057] HMVP candidates can be used in the process of constructing the merge candidate list. The last few HMVP candidates in the table are checked sequentially and inserted into the candidate list after the TMVP candidates. Redundancy checks are applied to the spatial or temporal merge candidates on the HMVP candidates.

[0058] To reduce the number of redundant check operations, the following simplification is introduced:

[0059] 1. The number of HMVP candidates used to generate the merge list, set to (N<=4)? M:(8–N), where N indicates the number of existing candidates in the merge list, and M indicates the number of available HMVP candidates in the table.

[0060] 2. Once the total number of available merge candidates reaches the maximum allowed merge candidates minus 1, the process of building the merge candidate list by HMVP is terminated.

[0061] 2.1.4. Pairwise average merging candidate derivation

[0062] Pairwise averaged candidates are generated by averaging predefined pairs of candidates in the existing merge candidate list. The predefined pairs are defined as {(0,1),(0,2),(1,2),(0,3),(1,3),(2,3)}, where the number represents the merge index of the merge candidate list. Averaged motion vectors are calculated for each reference list. If two motion vectors are available in a list, they are averaged, even if they point to different reference images; if only one motion vector is available, that one is used directly; if no motion vector is available, the list remains invalid.

[0063] If the merge list is not full after pairwise average merge candidates are added, insert zero MVPs at the end until the maximum number of merge candidates is reached.

[0064] 2.2. Triangulation for Inter-Frame Prediction

[0065] In VTM, Triangle Partitioning (TPM) mode is supported for inter-frame prediction. TPM is applied only to CUs with 64 samples or more, and is encoded / decoded in skip or merge mode, not in regular merge mode, MMVD mode, CIIP mode, or sub-block merge mode. CU-level flags indicate whether TPM is applied.

[0066] When using this pattern, use either diagonal or anti-diagonal partitioning. Figure 5 The CU is uniformly divided into two triangular segments. Each triangular segment in the CU is used for inter-frame prediction using its own motion; each segment only allows unidirectional prediction, i.e., each segment has one motion vector and one reference index. Unidirectional prediction motion constraints are applied to ensure that, as with regular bidirectional prediction, each CU requires only two motion-compensated predictions. The unidirectional prediction motion for each segment is directly derived from the merge candidate list constructed for the extended merge prediction in 2.1, and the selection of unidirectional prediction motion from the given merge candidates in the list follows the process in 2.2.1.

[0067] Figure 5 This is an example of inter-frame prediction based on triangulation.

[0068] If the triangulation pattern is used for the current CU, a flag indicating the direction of the triangulation (diagonal or anti-diagonal) and two merge indices (one for each triangulation) are further signaled. After predicting each triangulation, a blending process with adaptive weights is used to adjust the sample values ​​along the diagonal or anti-diagonal edges. This is the prediction signaling for the entire CU, and as in other prediction patterns, the transformation and quantization processes are applied to the entire CU. Finally, the motion field of the CU predicted using the triangulation pattern is stored in 4x4 cells, as shown in 2.2.3.

[0069] 2.2.1. Construction of One-Way Prediction Candidate List

[0070] Given a merge candidate index, unidirectional predicted motion vectors are derived from the merge candidate list constructed using the extended merge prediction process in 2.1, as follows: Figure 6 The example provided illustrates this. For candidates in the list, their LX motion vector (where X equals the parity of the merged candidate index value) is used as the unidirectional predicted motion vector for the triangular segmentation pattern. Figure 6In this context, these motion vectors are labeled with "x". If the corresponding LX motion vector is absent, the L(1-X) motion vectors of the same candidate in the expanded and merged prediction candidate list are used as the unidirectional prediction motion vectors for the triangular segmentation pattern.

[0071] 2.2.2. Blending along the edges of the triangular division

[0072] After each triangulation is predicted using its own motion, a blend is applied to both prediction signaling notifications to derive samples around the diagonal or anti-diagonal edges. The following weights are used during the blending process:

[0073] • Luminosity values ​​{7 / 8, 6 / 8, 5 / 8, 4 / 8, 3 / 8, 2 / 8, 1 / 8} and chromaticity values ​​{6 / 8, 4 / 8, 2 / 8}, such as Figure 7 As shown.

[0074] Figure 7 An example of the weights used in the mixing process is shown.

[0075] 2.2.3. Sports Field Storage

[0076] The motion vectors of the CU encoded and decoded in triangular segmentation mode are stored in 4x4 units. Depending on the position of each 4x4 unit, either unidirectional or bidirectional prediction motion vectors are stored. Segmentation 1 and Segmentation 2 respectively indicate Mv1 and Mv2 as unidirectional prediction motion vectors. If the 4x4 unit is located in... Figure 7 In the example shown, if the 4x4 cell is in an unweighted region, then either Mv1 or Mv2 is stored. Otherwise, if the 4x4 cell is in a weighted region, the bidirectional predicted motion vector is stored. The bidirectional predicted motion vector is derived from Mv1 and Mv2 according to the following procedure:

[0077] 1) If Mv1 and Mv2 are from different lists of reference images (one from L0 and the other from L1), then Mv1 and Mv2 are simply combined to form a bidirectional predicted motion vector.

[0078] 2) Otherwise, if Mv1 and Mv2 come from the same list, and without loss of generality, then assume they both come from L0. In this case,

[0079] 2.a) If a reference image for Mv2 (or Mv1) is present in L1, then Mv2 (or Mv1) is converted into an L1 motion vector using that reference image in L1. The two motion vectors are then combined to form a bidirectional predicted motion vector;

[0080] Otherwise, instead of bidirectional predicted motion, only unidirectional predicted motion Mv1 is stored.

[0081] 2.3. Specifications for Triangular Partitioning in VVC WD6

[0082] The following specifications for the deblocking filtering process are extracted from the latest VVC working draft JVET-O2001-vE.

[0083] 2.3.1. Syntax for Merging Data

[0084]

[0085]

[0086]

[0087] 2.3.2. Merging Data Semantics

[0088] ciip_flag[x0][y0] specifies whether to apply combined inter-frame image merging and intra-frame image prediction to the current codec unit. The array indices x0 and y0 specify the position (x0, y0) of the top-left luminance sample of the codec block under consideration relative to the top-left luminance sample of the image.

[0089] When ciip_flag[x0][y0] does not exist, it is inferred as follows:

[0090] – If all of the following conditions are true, then ciip_flag[x0][y0] is inferred to be equal to 1:

[0091] –sps_ciip_enabled_flag equals 1.

[0092] –general_merge_flag[x0][y0] equals 1.

[0093] –merge_subblock_flag[x0][y0] equals 0.

[0094] –regular_merge_flag[x0][y0] equals 0.

[0095] –cbWidth is less than 128.

[0096] –cbHeight is less than 128.

[0097] –cbWidth*cbHeight is greater than or equal to 64.

[0098] Otherwise, ciip_flag[x0][y0] is inferred to be equal to 0.

[0099] When ciip_flag[x0][y0] equals 1, the variable IntraPredModeY[x][y] is set to equal INTRA_PLANAR, where x = x0..x0 + cbWidth – 1 and y = y0..y0 + cbHeight – 1.

[0100] The variable MergeTriangleFlag[x0][y0] (which specifies whether triangle-based motion compensation is used to generate prediction samples for the current codec unit when decoding B-strips) is derived as follows:

[0101] – If all of the following conditions are true, then MergeTriangleFlag[x0][y0] will be set to 1:

[0102] –sps_triangle_enabled_flag equals 1.

[0103] –slice_type equals B.

[0104] –general_merge_flag[x0][y0] equals 1.

[0105] –MaxNumTriangleMergeCand is greater than or equal to 2.

[0106] –cbWidth*cbHeight is greater than or equal to 64.

[0107] –regular_merge_flag[x0][y0] equals 0.

[0108] –merge_subblock_flag[x0][y0] equals 0.

[0109] –ciip_flag[x0][y0] equals 0.

[0110] Otherwise, set MergeTriangleFlag[x0][y0] to 0.

[0111] `merge_triangle_split_dir[x0][y0]` specifies the direction of the merge triangle pattern. The array indices x0 and y0 specify the position (x0, y0) of the top-left luminance sample of the codec block under consideration relative to the top-left luminance sample of the image.

[0112] If merge_triangle_split_dir[x0][y0] does not exist, it is inferred to be equal to 0.

[0113] merge_triangle_idx0[x0][y0] specifies the first merge candidate index of the triangle-based motion compensation candidate list, where x0 and y0 specify the position (x0, y0) of the top-left luminance sample of the codec block under consideration relative to the top-left luminance sample of the image.

[0114] If merge_triangle_idx0[x0][y0] does not exist, it is inferred to be equal to 0.

[0115] merge_triangle_idx1[x0][y0] specifies the second merge candidate index of the triangle-based motion compensation candidate list, where x0 and y0 specify the position (x0, y0) of the top-left luminance sample of the codec block under consideration relative to the top-left luminance sample of the image.

[0116] If merge_triangle_idx1[x0][y0] does not exist, it is inferred to be equal to 0.

[0117] 2.3.3. Derivation process of trigonometric motion vector components and reference index

[0118] 2.3.3.1. Overall

[0119] The input for this process is:

[0120] – The brightness position (xCb, yCb) of the top-left sample of the current luminance block relative to the top-left sample of the current image.

[0121] – The variable cbWidth specifies the width of the current codec block in the luminance sample.

[0122] – The variable cbHeight specifies the height of the current codec block in the luminance sample.

[0123] The output of this process is:

[0124] – Brightness motion vectors mvA and mvB with a precision of 1 / 16 fraction of a sample point

[0125] – Chromaticity motion vectors mvCA and mvCB with 1 / 32 fractional sample point precision

[0126] –Refer to indices refIdxA and refIdxB,

[0127] – Prediction list flags predListFlagA and predListFlagB.

[0128] The following call is used to derive the luminance motion vectors of the triangular merging pattern specified in Clause 2.3.3.2: the luminance position (xCb, yCb), variables cbWidth and cbHeight are taken as input, and the output is the luminance motion vectors mvA and mvB, the reference indices refIdxA and refIdxB, and the prediction list flags predListFlagA and predListFlagB.

[0129] Use the following call to derive the luminance motion vector as described in Clause 2.3.3.3: take mvA and refIdxA as inputs, and output mvCA.

[0130] Use the following call to derive the luminance motion vector as described in Clause 2.3.3.3: take mvB and refIdxB as inputs, and output mvCB.

[0131] 2.3.3.2. Derivation process of brightness motion vector in merged triangular mode

[0132] This process is only called when MergeTriangleFlag[xCb][yCb] equals 1, where (xCb, yCb) specifies the top-left sample of the current luminance block relative to the top-left luminance sample of the current image.

[0133] The input for this process is:

[0134] – The brightness position (xCb, yCb) of the top-left sample of the current luminance block relative to the top-left sample of the current image.

[0135] – The variable cbWidth specifies the width of the current codec block in the luminance sample.

[0136] – The variable cbHeight specifies the height of the current codec block in the luminance sample.

[0137] The output of this process is:

[0138] – Brightness motion vectors mvA and mvB with a precision of 1 / 16 fraction of a sample point

[0139] –Refer to indices refIdxA and refIdxB,

[0140] – Prediction list flags predListFlagA and predListFlagB.

[0141] Motion vectors mvA and mvB, reference indices refIdxA and refIdxB, and prediction list flags predListFlagA and predListFlagB are derived by the following steps in sequence:

[0142] – Use the following call to derive the luminance motion vectors of the merge mode as specified in Clause 8.5.2.2: luminance position (xCb, yCb), variables cbWidth and cbHeight as inputs, and output luminance motion vectors mvL0[0][0], mvL1[0][0], reference indices refIdxL0, refIdxL1, prediction list using flags predFlagL0[0][0] and predFlagL1[0][0], bidirectional prediction weight index bcwIdx, and merge candidate list mergeCandList.

[0143] – Variables m and n (the merge indices of 0 and 1 for the triangular segmentation, respectively) are derived using merge_triangle_idx0[xCb][yCb] and merge_triangle_idx1[xCb][yCb] as follows:

[0144] m=merge_triangle_idx0[xCb][yCb] (8-475)

[0145] n=merge_triangle_idx1[xCb][yCb]+(merge_triangle_idx1[xCb][yCb]>=m)? 1:0 (8-476)

[0146] – Let refIdxL0M and refIdxL1M, predFlagL0M and predFlagL1M, and mvL0M and mvL1M be the reference index of the merge candidate M (M = mergeCandList[m]) at position m in the merge candidate list mergeCandList, the prediction list utilization flag, and the motion vector.

[0147] – The variable X is set to equal to (m&0x01).

[0148] – When predFlagLXM equals 0, X is set to equal to (1-X).

[0149] – Applications are as follows:

[0150] mvA[0]=mvLXM[0] (8-477)

[0151] mvA[1]=mvLXM[1] (8-478)

[0152] refIdxA=refIdxLXM (8-479)

[0153] predListFlagA = X (8-480)

[0154] – Let refIdxL0N and refIdxL1N, predFlagL0N and predFlagL1N, and MvL0N and mvL1N be the reference index of the merge candidate N (N = mergeCandList[n]) at position m in the merge candidate list mergeCandList, the prediction list using flags, and the motion vector.

[0155] – The variable X is set to equal to (n&0x01).

[0156] – When predFlagLXN equals 0, X is set to equal to (1-X).

[0157] – Applications are as follows:

[0158] mvB[0]=mvLXN[0] (8-481)

[0159] mvB[1]=mvLXN[1] (8-482)

[0160] refIdxB=refIdxLXN (8-483)

[0161] predListFlagB = X (8-484)

[0162] 2.3.3.3. Derivation process of chromaticity motion vector

[0163] The input for this process is:

[0164] – Brightness motion vector mvLX with 1 / 16 fractional sample point precision,

[0165] –Refer to index refIdxLX.

[0166] The output of this process is a chromaticity motion vector mvCLX with a precision of 1 / 32 fractional sample points.

[0167] The chromaticity motion vector is derived from the corresponding luminance motion vector.

[0168] The chroma motion vector mvCLX is exported as follows:

[0169] mvCLX[0]=mvLX[0]*2 / SubWidthC (8-435)

[0170] mvCLX[1]=mvLX[1]*2 / SubHeightC (8-436)

[0171] 2.3.4. Decoding process of triangular inter-frame blocks

[0172] 2.3.4.1. Overall

[0173] This procedure is called when decoding the codec unit with MergeTriangleFlag[xCb][yCb] equal to 1.

[0174] The input for this process is:

[0175] – Luminance position (xCb, yCb): Specifies the top-left luminance sample of the current codec block relative to the top-left luminance sample of the current image.

[0176] – The variable cbWidth specifies the width of the current codec block in the luminance sample.

[0177] – The variable cbHeight specifies the height of the current codec block in the luminance sample.

[0178] – Brightness motion vectors mvA and mvB with a precision of 1 / 16 fraction of a sample point

[0179] – Chromaticity motion vectors mvCA and mvCB.

[0180] –Refer to indices refIdxA and refIdxB,

[0181] – Prediction list flags predListFlagA and predListFlagB.

[0182] The output of this process is:

[0183] – The array of (cbWidth)x(cbHeight) predSamples for brightness prediction L ,

[0184] – array predSamples of chromaticity prediction samples for component Cb (cbWidth / SubWidthC) x (cbHeight / SubHeightC) Cb ,

[0185] – The chromaticity prediction sample array predSamples for component Cr (cbWidth / SubWidthC) x (cbHeight / SubHeightC) Cr ,

[0186] Let predSamplesLA L and predSamplesLB L For the predicted luminance sample values, there is a (cbWidth) x (cbHeight) array, and predSamplesLA Cb ,predSamplesLB Cb ,predSamplesLA Crand predSamplesLB Cr It is an array of (cbWidth / SubWidthC)x(cbHeight / SubHeightC) of predicted chromaticity sample values.

[0187] predSamples L ,predSamples Cb and predSamples Cr Derived by the following steps in sequence:

[0188] 1. For N to be each of A and B, the following applies:

[0189] – composed of an ordered two-dimensional array of brightness samples, refPicLN L The two ordered two-dimensional arrays refPicLN of chromaticity samples Cb and refPicLN Cr The constructed reference image is derived using the following procedure specified in Item 8.5.6.2 of VVC WD6: X set to equal predListFlagN and refIdxX set to equal refIdxN as input.

[0190] – array predSamplesLN L It is derived using the following procedure specified in Item 8.5.6.3 of VVC WD6: Luminance position (xCb, yCb), Luminance block width sbWidth set to equal cbWidth, Luminance block height sbHeight set to equal cbHeight, Motion vector offset mvOffset set to equal (0,0), Motion vector mvLX set to equal mvN, and refPicLN set to equal refPicLN. L The reference array refPicLX L The variable bdofFlag, set to FALSE, and the variable cIdx, set to 0, are used as inputs.

[0191] – array predSamplesLN Cb It is derived using the following procedure specified in Item 8.5.6.3 of VVC WD6: luminance position (xCb, yCb), codec block width sbWidth set to equal cbWidth / SubWidthC, codec block height sbHeight set to equal cbHeight / SubHeightC, motion vector offset mvOffset set to equal (0,0), motion vector mvLX set to equal mvCN, and refPicLN set to equal... CbThe reference array refPicLX Cb The variable bdofFlag, set to FALSE, and the variable cIdx, set to 1, are used as inputs.

[0192] – array predSamplesLN Cr It is derived using the following procedure specified in Item 8.5.6.3 of VVC WD6: luminance position (xCb, yCb), codec block width sbWidth set to equal cbWidth / SubWidthC, codec block height sbHeight set to equal cbHeight / SubHeightC, motion vector offset mvOffset set to equal (0,0), motion vector mvLX set to equal mvCN, and refPicLN set to equal... Cr The reference array refPicLX Cr The variable bdofFlag, set to FALSE, and the variable cIdx, set to 2, are used as inputs.

[0193] 2. Set the split direction variable triangleDir of the merge triangle mode to equal merge_triangle_split_dir[xCb][yCb].

[0194] 3. Predicted Samples within the current luminance codec block L [x L ][y L ], where x L =0..cbWidth-1 and y L =0..cbHeight-1 is derived using the following weighted sample prediction procedure via the triangular merging mode specified in Clause 2.3.4.2: the codec block width nCbW is set to equal cbWidth, the codec block height nCbH is set to equal cbHeight, and the sample array predSamplesLA is... L and predSamplesLB L The variables triangleDir and cIdx, which are equal to 0, are used as inputs.

[0195] 4. PredSamples within the current chroma component Cb encoding / decoding block Cb [x C ][y C ], where x C =0..cbWidth / SubWidthC-1 and y C=0..cbHeight / SubHeightC-1 is derived using the weighted sample prediction procedure specified in Clause 2.3.4.2 by calling the triangular merging mode: the codec block width nCbW is set to be equal to cbWidth / SubWidthC, the codec block height nCbH is set to be equal to cbHeight / SubHeightC, and the sample array predSamplesLA is... Cb and predSamplesLB Cb The variables triangleDir and cIdx, which are equal to 1, are used as inputs.

[0196] 5. PredSamples within the current chrominance component Cr encoding / decoding block Cr [x C ][y C ], where x C =0..cbWidth / SubWidthC-1 and y C =0..cbHeight / SubHeightC-1 is derived using the weighted sample prediction procedure specified in Clause 2.3.4.2 by calling the triangular merging mode: the codec block width nCbW is set to be equal to cbWidth / SubWidthC, the codec block height nCbH is set to be equal to cbHeight / SubHeightC, and the sample array predSamplesLA is... Cr and predSamplesLB Cr The variables triangleDir and cIdx, which are equal to 2, are used as inputs.

[0197] 6. Use the following call to the motion vector stored procedure of the merged triangle pattern specified in Clause 2.3.4.3: luma codec block position (xCb, yCb), luma codec block width cbWidth, luma codec block height cbHeight, segmentation direction triangleDir, luma motion vectors mvA and mvB, reference indices refIdxA and refIdxB, and prediction list flags predListFlagA and predListFlagB as input.

[0198] 2.3.4.2. Weighted Sample Prediction Process of Triangular Merging Model

[0199] The input for this process is:

[0200] – Two variables, nCbW and nCbH, specify the width and height of the current codec block.

[0201] – Two (nCbW)x(nCbH) arrays, predSamplesLA and predSamplesLB,

[0202] – The variable triangleDir specifies the splitting direction.

[0203] – The variable cIdx specifies the color component index.

[0204] The output of this process is an array of (nCbW)x(nCbH) predicted sample values, pbSamples.

[0205] The variable nCbR is exported as follows:

[0206] nCbR=(nCbW>nCbH)? (nCbW / nCbH):(nCbH / nCbW)

[0207] (8-841)

[0208] The variable bitDepth is exported as follows:

[0209] - If cIdx equals 0, then bitDepth is set to equal BitDepth. Y .

[0210] Otherwise, bitDepth is set to equal BitDepth. C .

[0211] The variables shift1 and offset1 are exported as follows:

[0212] - The variable shift1 is set to equal Max(5, 17-bitDepth).

[0213] - Set the variable offset1 to equal 1 << (shift1 - 1).

[0214] Depending on the values ​​of triangleDir, wS, and cIdx, predict sample points pbSamples[x][y], where x = 0..nCbW-1 and y = 0..nCbH-1, derived as follows:

[0215] The variable wIdx is exported as follows:

[0216] – If cIdx equals 0 and triangleDir equals 0, then the following applies:

[0217] wIdx=(nCbW>nCbH)? (Clip3(0,8,(x / nCbR-y)+4))(8-842)

[0218] :(Clip3(0,8,(xy / nCbR)+4))

[0219] Otherwise, if cIdx equals 0 and triangleDir equals 1, then the following applies:

[0220] wIdx=(nCbW>nCbH)? (Clip3(0,8,(nCbH-1-x / nCbR-y)+4))(8-843)

[0221] (Clip3(0,8,(nCbW-1-xy / nCbR)+4))

[0222] Otherwise, if cIdx is greater than 0 and triangleDir is equal to 0, then the following applies:

[0223] wIdx=(nCbW>nCbH)? (Clip3(0,4,(x / nCbR-y)+2))(8-844)

[0224] :(Clip3(0,4,(xy / nCbR)+2))

[0225] Otherwise, (if cIdx is greater than 0 and triangleDir is equal to 1), the following applies:

[0226] wIdx=(nCbW>nCbH)? (Clip3(0,4,(nCbH-1-x / nCbR-y)+2))(8-845)

[0227] (Clip3(0,4,(nCbW-1-xy / nCbR)+2))

[0228] – The variable wValue, which specifies the weights of the predicted samples, is derived using wIdx and cIdx as follows:

[0229] wValue=(cIdx==0)? Clip3(0,8,wIdx):Clip3(0,8,wIdx*2)(8-846)

[0230] – The predicted sample values ​​are exported as follows:

[0231] pbSamples[x][y] = Clip3(0, (1< <bitDepth)-1,(predSamplesLA[x][y]*wValue+(8-847)

[0232] predSamplesLB[x][y]*(8-wValue)+offset1)>>shift1)

[0233] 2.3.4.3. Motion Vector Storage Procedure in Triangle Merging Mode

[0234] This procedure is called when decoding the codec unit with MergeTriangleFlag[xCb][yCb] equal to 1.

[0235] The input for this process is:

[0236] – Luminance position (xCb, yCb): Specifies the top-left luminance sample of the current codec block relative to the top-left luminance sample of the current image.

[0237] – The variable cbWidth specifies the width of the current codec block in the luminance sample.

[0238] – The variable cbHeight specifies the height of the current codec block in the luminance sample.

[0239] – The variable triangleDir specifies the splitting direction.

[0240] – Brightness motion vectors mvA and mvB with a precision of 1 / 16 fraction of a sample point

[0241] –Refer to indices refIdxA and refIdxB,

[0242] – Prediction list flags predListFlagA and predListFlagB.

[0243] The variables numSbX and numSbY, which specify the number of 4x4 blocks in the current codec block in the horizontal and vertical directions, are set to equal numSbX = cbWidth >> 2 and numSbY = cbHeight >> 2.

[0244] The variable minSb is set to equal Min(numSbX,numSbY)-1.

[0245] The variable cbRatio is exported as follows:

[0246] cbRatio=(cbWidth>cbHeight)? (cbWidth / cbHeight):(cbHeight / cbWidth)(8-848)

[0247] For each 4x4 sub-block at sub-block index (xSbIdx, ySbIdx), where xSbIdx = 0..numSbX-1 and ySbIdx = 0..numSbY-1, the following applies:

[0248] – The variables xIdx and yIdx are derived as follows:

[0249] xIdx=(cbWidth>cbHeight)? (xSbIdx / cbRatio):xSbIdx(8-849)

[0250] yIdx=(cbWidth>cbHeight)? ySbIdx:(ySbIdx / cbRatio)(8-850)

[0251] – The variable sType is exported as follows:

[0252] – If triangleDir equals 0, then the following applies:

[0253] sType=(xIdx==yIdx)? 2:((xIdx>yIdx)?0:1)

[0254] (8-851)

[0255] – Otherwise (triangleDir equals 1), the following applies:

[0256] sType=(xIdx+yIdx==minSb)? 2:((xIdx+yIdx <minSb)?0:1)(8-852)

[0257] – Depending on the value of sType, the following assignments are made:

[0258] – If sType equals 0, the following applies:

[0259] predFlagL0=(predListFlagA==0)? 1:0

[0260] (8-853)

[0261] predFlagL1=(predListFlagA==0)? 0:1 (8-854)

[0262] refIdxL0=(predListFlagA==0)? refIdxA:-1 (8-855)

[0263] refIdxL1=(predListFlagA==0)? -1:refIdxA (8-856)

[0264] mvL0[0]=(predListFlagA==0)? mvA[0]:0 (8-857)

[0265] mvL0[1]=(predListFlagA==0)? mvA[1]:0 (8-858)

[0266] mvL1[0]=(predListFlagA==0)? 0:mvA[0] (8-859)

[0267] mvL1[1]=(predListFlagA==0)? 0:mvA[1] (8-860)

[0268] Otherwise, if sType equals 1 or (sType equals 2 and predListFlagA+predListFlagB is not equal to 1), then the following applies:

[0269] predFlagL0=(predListFlagB==0)? 1:0 (8-861)

[0270] predFlagL1=(predListFlagB==0)? 0:1 (8-862)

[0271] refIdxL0=(predListFlagB==0)? refIdxB:-1 (8-863)

[0272] refIdxL1=(predListFlagB==0)? -1:refIdxB (8-864)

[0273] mvL0[0]=(predListFlagB==0)? mvB[0]:0 (8-865)

[0274] mvL0[1]=(predListFlagB==0)? mvB[1]:0 (8-866)

[0275] mvL1[0]=(predListFlagB==0)? 0:mvB[0] (8-867)

[0276] mvL1[1]=(predListFlagB==0)? 0:mvB[1] (8-868)

[0277] – Otherwise (sType equals 2 and predListFlagA + predListFlagB equals 1), the following applies:

[0278] predFlagL0 = 1 (8-869)

[0279] predFlagL1=1 (8-870)

[0280] refIdxL0 = (predListFlagA == 0)? refIdxA : refIdxB (8 - 871)

[0281] refIdxL1 = (predListFlagA == 0)? refIdxB : refIdxA (8 - 872)

[0282] mvL0[0] = (predListFlagA == 0)? mvA[0] : mvB[0] (8 - 873)

[0283] mvL0[1] = (predListFlagA == 0)? mvA[1] : mvB[1] (8 - 874)

[0284] mvL1[0] = (predListFlagA == 0)? mvB[0] : mvA[0] (8 - 875)

[0285] mvL1[1] = (predListFlagA == 0)? mvB[1] : mvA[1] (8 - 876)

[0286] – For x = 0..3 and y = 0..3, make the following assignments:

[0287] MvL0[(xSbIdx << 2)+x][(ySbIdx << 2)+y] = mvL0 (8 - 877)

[0288] MvL1[(xSbIdx << 2)+x][(ySbIdx << 2)+y] = mvL1 (8 - 878)

[0289] RefIdxL0[(xSbIdx << 2)+x][(ySbIdx << 2)+y] = refIdxL0 (8 - 879)

[0290] RedIdxL1[(xSbIdx << 2)+x][(ySbIdx << 2)+y] = refIdxL1 (8 - 880)

[0291] PredFlagL0[(xSbIdx << 2)+x][(ySbIdx << 2)+y] = predFlagL0 (8 - 881)

[0292] PredFlagL1[(xSbIdx << 2)+x][(ySbIdx << 2)+y] = predFlagL1 (8 - 882)

[0293] 2.4. Geometric Merge Mode (GEO)

[0294] In JVET-P0068, the GEO merging pattern is being used as an extension of the existing TPM in VVC. GEO uses the same predictive merging concept as TPM, but expands the merging mask to up to 140 different patterns, with 32 angular and 5 distance offsets. The merging mask for the GEO pattern is derived from the distance and partition boundaries of the sample locations using three lookup tables. In this pattern, angular segmentation is used to divide the blocks, such that at least one segment has non-horizontal and non-vertical boundaries.

[0295] 2.4.1. Conceptual Description

[0296] Figure 13A and 13B The diagram illustrates the TPM in VTM-6.0 and the additional shapes proposed for non-rectangular inter-frame blocks.

[0297] Similar to the proposed TPM, inter-frame GEO segmentation is permitted for unidirectional prediction blocks of at least 8×8 to achieve the same memory bandwidth on the decoder side as dual prediction blocks. Motion vector predictions from the GEO segmentation are aligned with those from the TPM. Also in the TPM, the blending of the two predictions is applied at the internal boundaries.

[0298] The boundary of the geometric merging pattern is determined by angle. and distance offset ρ i Description, such as Figure 14 As shown. Angle This represents the quantized angle between 0 and 360 degrees, and the distance offset ρ. i The quantization offset ρ represents the maximum distance. max Furthermore, partitioning directions that overlap with binary tree partitioning and TPM partitioning are excluded.

[0299] 2.4.2. Quantification of angle and distance.

[0300] Quantize angles with a fixed step size between 0 and 360 degrees.

[0301] In CE4-1.1, CE4-1.2a with 108 modes, and CE4-1.14, the angle is quantized from 0 to 360 degrees in steps of 11.25 degrees. like Figure 15A A total of 32 angles are shown.

[0302] Although in CE4-1.2b with 80 modes, the metric angle is still 11.25. However, the near-vertical orientation angle (near-horizontal demarcation boundary) was removed because, in natural values, the target and motion are mostly horizontal. Figure 15B The diagram illustrates a scheme that reduces the number of angles to 24.

[0303] Starting from the maximum possible distance ρ with a fixed step size max Quantization distance ρ i . ρ max The value of can be geometrically derived from equation (1), where w or h equals 8 and is scaled by the shorter side length in log2. For When the temperature is equal to 0 degrees, ρ max It equals w / 2, and for When the angle is equal to 90 degrees, ρ max It equals h / 2. Returning the shifted "1.0" sample is to avoid dividing the edges too close to the corners.

[0304]

[0305] In CE4-1.1 and CE4-1.14, the quantization distance ρ is set to 5 steps. i Considering that there are a total of 140 partitioning patterns for 32 angles, binary tree and TPM partitioning are excluded.

[0306] In CE4-1.2a, which has 108 modes, the quantization distance ρ is set to 4 steps. i Considering that there are a total of 108 partitioning patterns for 32 angles, binary tree and TPM partitioning are excluded.

[0307] In the CE4-1.2b with 80 modes, the quantization distance ρ is set to 4 steps. i Considering that there are a total of 80 partitioning patterns for 24 angles, binary tree and TPM partitioning are excluded.

[0308] Table 1 summarizes the number of angles, distances, and modes for CE testing.

[0309] Table 1. Number of angles, number of distances, and number of division patterns.

[0310] CE4-1.1 CE4-1.2a CE4-1.2b Number of angles 32 32 24 Number of distances 5 4 4 Number of partitioning patterns 140 108 80

[0311] 2.4.3. Blending Operation of Luminosity Blocks

[0312] In the same geometric merging mode as TPM mode, the final predictor P has 3-bit mixing masks W0 and W1. B As in equation (2)

[0313] P B =(W0P0+W1P1+4)>>3 (2)

[0314] Using equations (3), (4), and (5), a lookup table is used to derive the blending mask of the geometric merging pattern from the distance of the sample point locations and the dividing boundary.

[0315] distFromLine=((x<<1)+1)*Dis[displacementX]+((y<<1)+1))*Dis[displacementY]-rho (3)

[0316] distScaled=Clip3(0, 26, (abs(distFromLine)+4)>>3) (4)

[0317] sampleWeightL[x][y]=distFromLine<=0? GeoFilter[distScaled]: 8-GeoFilter[distScaled] (5)

[0318] This involves three lookup tables: Dis[.] with 32 entries, StepDis[.] with 36 entries, and GeoFilter[.] with 26 entries.

[0319] It is guaranteed that the lower left sample of the current block is predicted from P0. In other words, when the lower left sample's distFromLine is negative, W0 equals sampleWeightL[x][y], and W1 equals 8-W0. Otherwise (if the lower left sample's distFromLine is positive), W1 equals sampleWeightL[x][y], and W0 equals 8-W1.

[0320] The actual computational complexity derived from the geometric hybrid mask comes from equation (3), since all remaining operations are performed using lookup tables.

[0321] In the VTM software implementation, equation (3) requires one addition per sample point and one addition per sample point row. For example, in an 8x8 CU, each sample point requires 1.125 additions and 0.015625 multiplications.

[0322] To process each 4x4 cell in parallel, for example in an 8x8 CU, each sample requires 1.125 additions and 0.00625 multiplications;

[0323] To process each row (e.g., 8x8 CU) in parallel, each sample requires 1 addition and 0.125 multiplications;

[0324] To process all samples in the CU in parallel, two multiplications and one addition are required for each sample.

[0325] Table 2 summarizes the worst-case computational complexity (8x8) for each sample point:

[0326] Table 2 Worst-case complexity analysis

[0327]

[0328] For more details on hybrid operations, please refer to section "8.5.7.3 Weighted Sample Prediction Mode of Geometric Merging Mode" in the attached draft specification revision document.

[0329] 2.4.4. Blending of Color Blocks

[0330] The sample weights calculated for the luminance samples are subsampled and used for chroma mixing without any computation. The chroma sample weight at coordinates (x, y) is set to be equal to the luminance sample weight at coordinates (2x, 2y) with respect to the top-left sample of the luminance block.

[0331] 2.4.5. Exporting Motion Vectors

[0332] The same merge list export process used for TPM is used to export the motion vector for each segment of the GEO block. Each segment is predicted only by unidirectional prediction.

[0333] 2.4.6. Motion Vector Storage

[0334] In CE4-1.1 and CE4-1.2, the luminance samples at the four corners of the 4x4 motion storage unit are weighted and summed. The sum is then compared to two thresholds to determine whether one of two unidirectional predictive motion messages or bidirectional predictive motion message is stored. The bidirectional predictive motion message is derived using the same procedure as in TPM.

[0335] In CE4-1.14, the motion vector storage process is further simplified. The distance between the center position of the 4x4 motion storage cell and the partition boundary is calculated and compared with a fixed threshold to determine whether to store unidirectional or bidirectional predictive motion information for that 4x4 motion storage cell. The sign of the distance indicates which unidirectional predictive motion information should be stored in the case of unidirectional predictive storage. In CE4-1.14, the correlation between the mixing mask and motion storage is removed.

[0336] 2.4.7. Mode Signaling Notification

[0337] According to the proposed method, the GEO mode is signaled together with the TPM mode as an additional merging mode.

[0338] Table 3 Syntax Elements Introduced by the Proposal

[0339] if(!ciip_flag[x0][y0]&&MaxNumTriangleMergeCand>1){ if(cbWidth>=8&&cbHeight>=8) merge_geo_flag[x0][y0] ae(v) if(merge_geo_flag[x0][y0]) merge_geo_idx[x0][y0] ae(v) else merge_triangle_split_dir[x0][y0] ae(v) merge_triangle_idx0[x0][y0] ae(v) merge_triangle_idx1[x0][y0] ae(v) }

[0340] The merge_geo_flag[][] is signaled using the 4CABAC context model. The first three values ​​depend on the output of the adjacent blocks above and to the left, and the fourth value depends on the aspect ratio of the current block. merge_geo_flag[][] indicates whether the current block uses GEO mode or TPM mode, and it is similar to a "most likely mode" flag.

[0341] geo_partition_idx[][] is used to store angles. and distance ρ i The index of the lookup table. The geo_partition_idx encoding / decoding truncates the binary number and uses bypass binarization.

[0342] 3. Examples of the technical problems solved by the technical solutions described herein

[0343] Several issues exist in the latest VVC working draft WD6 (JVET-O2001-v14), described as follows:

[0344] (1) In WD6, for the blending process of two triangular segments, the chroma weights are not aligned with the luminance weights, such as... Figure 8 As shown, this can cause visual artifacts.

[0345] (2) In WD6, the weight settings for triangular prediction do not consider multiple chroma formats such as 4:2:2 and 4:4:4, for example. Figure 8 As shown.

[0346] (3) In WD6, chroma only allows even weights, which is inconsistent with luminance, because both even and odd integers are allowed for the luminance component, such as... Figure 8 As shown.

[0347] (4) In WD6, TPM is allowed for 4xN and Nx4 blocks, where weighted blending is required for all pixels, which may be undesirable.

[0348] (5) In WD6, TPM is allowed for blocks with a width-to-height ratio greater than 2, which may be undesirable.

[0349] (6) GEO and TPM are applied with independent signaling notification and independent computation for hybrid weighted masks and motion storage masks, respectively.

[0350] Figure 8 An example of the weights used in the mixing process of an 8x16 TPM block (WD6) is shown.

[0351] 4. Examples of Implementation Methods and Technologies

[0352] The items listed below should be understood as examples illustrating general concepts. These entries should not be interpreted in a narrow sense. Furthermore, these items can be combined in any way.

[0353] The term 'TPM' can refer to an encoding / decoding method that divides a block into two or more sub-regions, and the transformation is applied to the entire block. The term 'TPM' can indicate a triangulation prediction mode and / or a geometric merging mode as an extension of the triangulation prediction mode.

[0354] Weighted sampling points in the TPM mixing process

[0355] 1. The weights of TPM chromaticity prediction samples can be aligned with the weights of co-located luminance prediction samples.

[0356] a) In one example, the weights of the TPM codec chroma blocks (e.g., Cb blocks and / or Cr blocks) can be set according to the weights of the co-bit luminance blocks.

[0357] b) In one example, the weights of the TPM codec chroma blocks can be a subset of the weights of the co-bit luminance blocks.

[0358] c) In one example, for each position within a block, a chroma block with dimension M×N can be assigned the same weight as a luminance block with dimension M×N.

[0359] 2. The weight of TPM chromaticity prediction samples can depend on the width and height of the co-occurrence luminance block and / or the color form, including the chromaticity subsampling ratio.

[0360] a) In one example, for a 4:4:4 chroma format, the weight of the TPM chroma prediction sample can be the same as the weight of the co-position luminance prediction sample for each position within the block.

[0361] b) In one example, for 4:2:0 and 4:2:2 chroma formats, the weights of TPM chroma prediction samples can be subsampled from the weights of TPM luminance prediction samples.

[0362] i. For a W×H TPM prediction block, where W is the block width and H is the block height, subWidthC and subHeightC represent the chromaticity subsampling ratios in the width and height directions, respectively. Assuming the weights of the luminance prediction block are represented by a two-dimensional array WeightY[x][y], where x = 0…(W-1) and y = 0…(H-1), then the weights of the co-bit chromaticity prediction block, WeightC[x][y], where x = 0…(W / subWidthC-1) and y = 0…(H / subHeightC-1), can be calculated from WeightY[f(x)][g(y)].

[0363] 1) In one example, f(x) = x*subWidthC + offsetX, g(y) = y*subHeightC + OffsetY, for example, OffsetX = OffsetY = 0.

[0364] ii. Assume that the weight of position (x,y) in the WxH TPM luma block is calculated by w(x,y), for example, w(x,y)=a*x+b*y+c, where x=0…W-1 and y=0…H-1 are the coordinates of the luma sample, and a, b, c are integers depending on W and / or H. In one example, the weight of position (x',y') in the co-bit TPM chroma block can be calculated as w(f(x'),g(y')), for example, w(f(x'),g(y'))=a*(subWidthC*x')+b*(subHeightC*y')+c, where x'=0…W / subWidthC-1 and y'=0…H / subHeightC-1 are the coordinates of the chroma sample.

[0365] c) In one example, the weights used in TPM can depend only on the dimensions of the block (width and / or height) and can be the same for different chroma components.

[0366] i. For example, a chromaticity component with size W*H can use the same weights as a luminance component with size W*H.

[0367] ii. In one example, TPM can be disabled for chroma blocks of such size when no weight is defined for the luma block size.

[0368] d) In one example, the weights used in TPM can depend on the dimensions of the block (width and / or height) and the color components of the block.

[0369] i. In one example, the weights can be different for different color components.

[0370] ii. In one example, the weights can be the same for both chromaticity components.

[0371] 1) Alternatively, the weights used for the luminance component and the chrominance component can be different.

[0372] 3. The weight of a TPM prediction sample can be equal to an integer X.

[0373] a) In one example, odd or even integers can be assigned as weights X (such as X = 0…8) for TPM chromaticity prediction samples.

[0374] b) The weights of TPM luminance / chromaticity prediction samples can be clipped to the range [M,N], such as M=0, N=8.

[0375] c) In one example, the TPM weight can be less than zero.

[0376] 4. In one example, the mixed weight mask of the TPM / GEO block can be predefined as N tables (such as N>0).

[0377] 5. In one example, the mixed weight mask of the TPM / GEO block can be calculated from the computation equation.

[0378] Common problems of TPM

[0379] Let W represent the block width and H represent the block height.

[0380] 6. Whether TPM is enabled or disabled may depend on the ratio of block width to height, such as max(H,W) / min(H,W).

[0381] a) Alternatively, enabling or disabling TPM may depend on the difference between the block width and height, for example, Abs(Log2(cbWidth)-Log2(cbHeight)), where Abs(x) returns the absolute value of x and Log2(x) returns the base-2 logarithm of the number x.

[0382] b) TPM may not be permitted for blocks with a width-to-height ratio or a height-to-width ratio greater than X (e.g., X = 2).

[0383] i. In one example, for a W×H prediction block, if W / H>2, TPM may be disabled.

[0384] ii. In one example, for a W×H prediction block, TPM may be disabled if H / W>2.

[0385] 7. Whether TPM is allowed may depend on the maximum transformation size.

[0386] a) In one example, TPM may not be allowed for blocks with a width and / or height greater than the maximum transform size.

[0387] 8. Whether TPM is allowed may depend on the maximum CU size.

[0388] a) In one example, TPM may not be allowed for blocks with a width and / or height equal to the maximum CU size.

[0389] 9. TPM may not be allowed for blocks with a block width greater than N and / or a block height greater than M.

[0390] a) In one example, N = M = 64.

[0391] 10. TPM may not be allowed for blocks with a block width of N and / or a block height of M.

[0392] a) In one example, N = M = 4.

[0393] 11. TPM may not be allowed for certain chroma formats.

[0394] a) In one example, TPM may not be allowed for 4:0:0 chroma format.

[0395] b) In one example, TPM may not be allowed for 4:4:4 chroma format.

[0396] c) In one example, TPM may not be allowed for 4:2:2 chroma format.

[0397] 12. If the two reference images used in TPM have different resolutions, TPM may not be allowed.

[0398] a) Alternatively, TPM may not be allowed if one of the two reference images used in TPM has a different resolution than the current image.

[0399] 13. When TPM is disabled or not allowed, signaling may not be provided to TPM syntax elements (such as merge_triangle_split_dir, merge_triangle_idx0, and merge_triangle_idx1).

[0400] a) When no signaling is given to a syntax element, it may be inferred as 0.

[0401] b) When TPM is disabled or not allowed, TPM-related semantic variables (such as MergeTriangleFlag) may be inferred as 0.

[0402] 14. The above bullet points may apply to the Triangular Prediction Model (TPM) and / or the Geometric Merging Model (GEO). In other words, TPM may refer to GEO.

[0403] Unification of TPM and GEO

[0404] 15. TPM and GEO can be unified.

[0405] a) In one example, TPM can be considered a subset of GEO.

[0406] i. Alternatively, GEO can be considered a subset of TPM.

[0407] ii. For example, if the codec tool A (or equivalent "mode A" or simply "A") (such as TPM) is considered a subset of the codec tool B (or equivalent "mode B" or simply "B") (such as GEO), the methods disclosed below can be applied.

[0408] 1) A and B can be signaled as one mode.

[0409] a) In one example, mode A and mode B can share the same control flag(s) at the SPS / VPS / APS / PPS / stripe / sub - picture / slice / brick / VPDU / CTU / TU / CU / PU / picture - header / stripe - header level.

[0410] b) In one example, mode A can be signaled as a subset of B.

[0411] i. For example, B is defined as a prediction mode containing N (such as N > 1) sub - modes represented as {M0, M1, M2…, M N-1}, and A can be defined as a prediction mode containing X (such as X < N) sub - modes represented as {M0, M k0 , M k1 …, M kX-1}, where {M0, M k0 , M k1 …, M kX-1} is a subset of {M0, M1, M2…, M N-1}.

[0412] ii. In one example, a first syntax element is signaled to indicate whether mode B is applied. A second syntax element indicating whether mode A is applied is signaled depending on whether mode B is applied.

[0413] 1. In one example, the second syntax element is signaled only if mode B is applied.

[0414] 2) A and B can share at least one computational logic to generate a hybrid weight mask.

[0415] a) In one example, the hybrid weight masks of A and B can be derived from the same lookup table.

[0416] 3) A and B can share at least one computational logic to generate a motion storage mask.

[0417] a) In one example, the motion storage masks of A and B can be derived from the same lookup table.

[0418] b) In one example, at least one computational logic can be used to calculate the hybrid weights for both the TPM mode and the GEO mode.

[0419] i. In one example, the computational logic of TPM can be used for GEO.

[0420] 1) Alternatively, the computational logic of GEO can be used for TPM.

[0421] c) In one example, at least one computational logic can be used to compute the motion storage mask for both TPM mode and GEO mode.

[0422] i. In one example, the computational logic of TPM can be used for GEO.

[0423] 1) Alternatively, the computational logic of GEO can be used for TPM.

[0424] TPM / GEO motion storage mask generation

[0425] 16. The motion vector storage mask for TPM can be generated in the same way as GEO.

[0426] a) Alternatively, the motion vector storage mask for GEO can be generated in the same manner as TPM.

[0427] b) In one example, a motion vector storage mask can be generated and / or stored to indicate a specific combination of block width and block height, representing the inter-frame prediction direction (such as unidirectional or bidirectional prediction).

[0428] i. In one example, a motion vector storage mask can be generated only for permissible combinations of block width and block height.

[0429] c) In one example, each element of the motion vector storage mask indicates: which motion vector is stored between two sub-segments; and / or how many motion vectors (such as one motion vector or two motion vectors) and / or inter-frame prediction direction (such as unidirectional or bidirectional prediction) are stored for a 4x4 sub-block.

[0430] d) In one example, the motion vector storage mask for the TPM / GEO block can be predefined as N tables (such as N>0).

[0431] e) In one example, the motion vector storage mask of the TPM / GEO block can be calculated from the computation equation.

[0432] 17. The motion vectors of TPM / GEO can be stored in 4x4 cells.

[0433] a) In one example, each of the two sub-segments of the TPM / GEO has its own motion vector for motion compensation, but the motion vectors stored as candidates for spatial / temporal motion vectors are in 4x4 cells.

[0434] i. In one example, each 4x4 sub-block of a TPM / GEO can have different motion vectors stored in a buffer.

[0435] 18. For sub-blocks belonging to the mixed region of TPM / GEO, both L0 motion vector and L1 motion vector can be stored.

[0436] a) In one example, a blended region can refer to the region where two sub-divisions overlap.

[0437] b) In one example, for those 4x4 sub-blocks outside the mixing region of the TPM / GEO block, the unidirectional prediction (such as L0 or L1) motion vectors of the sub-segment can be stored.

[0438] c) In one example, for those 4x4 sub-blocks belonging to the mixed region of the TPM / GEO block, the bidirectional predicted motion vectors of the sub-segmentation of both can be stored.

[0439] d) In one example, for those 4x4 sub-blocks belonging to the mixed region of the TPM / GEO block, if the sub-segments have motion vectors from the same direction, the minimum / maximum / average / weighted motion vector between the two motion vectors can be stored.

[0440] TPM and GEO's Block Removal

[0441] 19. Whether and / or how a deblocking process is applied to the codec block may depend on whether the codec block is encoded in TPM mode or GEO mode.

[0442] a) In one example, the boundary between two sub-blocks in two different segments can be filtered during the deblocking filtering stage.

[0443] i. In one example, the boundary strength (BS) is equal to 1 in this case.

[0444] ii. In one example, the boundary strength (BS) is equal to 2 in this case.

[0445] 20. For the boundary between two sub-blocks in a TPM / GEO codec block, deblocking can be triggered.

[0446] a) In one example, if one of the two sub-blocks next to the inner TU edge of the TPM / GEO mode has a non-zero coefficient, deblocking can be triggered regardless of whether there is a motion difference between the two sub-blocks.

[0447] b) There may be non-zero coefficients in either of the sub-blocks.

[0448] i. In one example, if two sub-blocks next to the inner TU edge of the TPM / GEO mode have all zero coefficients, but the motion difference between the two sub-blocks is large enough, it is still possible to trigger the deblocking.

[0449] ii. Alternatively, if the two sub-blocks next to the inner TU edge of the TPM / GEO mode have all zero coefficients, but the motion difference between the two sub-blocks is large enough, then deblocking may not be triggered.

[0450] c) Whether deblocking is triggered for the edge of two sub-blocks encoded and decoded in TPM / GEO mode can depend on whether the edge is a TU edge or an MV edge of the two sub-blocks.

[0451] i. If the motion difference is large enough for the MV edge in TPM / GEO mode, deblocking can be triggered.

[0452] ii. If a non-zero coefficient exists in a sub-block next to the TU edge in TPM / GEO mode, deblocking can be triggered.

[0453] iii. When the filtered edge is a TU edge or an MV edge, deblocking can be triggered if any of the following conditions are met:

[0454] 1) If the motion difference is large enough.

[0455] 2) There is a non-zero coefficient in either of the two adjacent sub-blocks.

[0456] d) The “TU edge” mentioned above refers to the actual transform unit edge, and the “MV edge” mentioned above refers to the PU edge or the sub-block edge aligned with the filter grid.

[0457] e) The “motor difference” mentioned above can indicate the following situation.

[0458] i. The motion vector difference between the two sub-blocks is greater than T (e.g., T = 1 pixel or 1 / 2 pixel, or 8 in units of 1 / 16 luminance samples).

[0459] ii. Different reference frame indexes

[0460] iii. Different reference POCs

[0461] iv. Different numbers of reference frames

[0462] Regarding configurable CTU size and maximum transformation size

[0463] 21. Whether to apply ISP may not depend on the maximum transform size and / or minimum transform size.

[0464] a) In one example, signaling notifications for ISP flags (such as intra_subpartitions_mode_flag) may not depend on whether the width of the current block is less than or equal to the maximum transform size, and / or may not depend on whether the height of the current block is less than or equal to the maximum transform size.

[0465] b) In one example, signaling notifications for ISP flags (such as intra_subpartitions_mode_flag) may not depend on whether the height multiplied by the width of the current block is greater than the square of the minimum transform size.

[0466] c) In one example, signaling notifications for ISP flags (such as intra_subpartitions_mode_flag) may depend on whether the height multiplied by the width of the current block is greater than 16.

[0467] d) In one example, signaling notifications for ISP flags (such as intra_subpartitions_mode_flag) may depend on whether the width of the current block is less than or equal to 64, and / or on whether the height of the current block is less than or equal to 64.

[0468] 22. ISP can be applied when the dimension of the codec block is greater than the maximum transform size.

[0469] a) In one example, when the ISP codec block is larger than the maximum transform size, the ISP block can be implicitly divided recursively until the sub-segment reaches size 64.

[0470] b) In one example, when the ISP codec block is larger than the maximum transform size, the ISP block can be implicitly divided recursively until the sub-segment reaches the size of the maximum transform.

[0471] 23. When the dimension of the codec block is greater than or equal to 128, CIIP and / or TPM and / or GEO can be applied.

[0472] a) In one example, the maximum CTU size can be set to greater than 128.

[0473] b) In one example, CIIP can be used for blocks with a block size greater than or equal to 128.

[0474] c) In one example, TPM and / or GEO can be applied to blocks with a block size greater than 128.

[0475] 24. When the dimension of the codec block is greater than 128, signaling can be used to notify the merging of data.

[0476] a) In one example, merge flags (such as regular_merge_flag, mmvd_merge_flag, mmvd_cand_flag, mmvd_distance_idx, mmvd_direction_idx, merge_idx, ciip_flag, merge_triangle_split_dir, merge_triangle_idx0, merge_triangle_idx1) may depend on whether the dimension of the codec block is smaller than the maximum CTU size.

[0477] 25. If the block width and / or block height are equal to or greater than X (such as X = 64 or 128), the value of pred_mode_ibc_flag can be inferred to be 0.

[0478] a) In one example, if the block width and block height are greater than 64, the value of pred_mode_ibc_flag can be inferred to be 0.

[0479] b) In one example, if the block width and / or block height is greater than 64, the value of pred_mode_ibc_flag can be inferred to be 0.

[0480] 26. When the dimension of the codec block is greater than 128, signaling can be used to notify cu_skip_flag and / or pred_mode_flag.

[0481] Generally remove squares

[0482] 27. The image-level deblocking parameter offsets for β and tC can be different for each component.

[0483] a) In one example, the image-level deblocking parameter offsets Cb and Cr for brightness can be different and indicated by different syntax elements.

[0484] b) Alternatively, the picture-level deblocking parameter offsets for the joint_cb_cr encoding / decoding modes can be different and indicated by different syntax elements.

[0485] 28. The strip-level deblocking parameter offsets for β and tC can be different for each component.

[0486] a) In one example, the strip level deblocking parameter offsets Cb and Cr of brightness can be different and indicated by different syntax elements.

[0487] b) Alternatively, the picture-level deblocking parameter offsets for the joint_cb_cr encoding / decoding modes can be different and indicated by different syntax elements.

[0488] 29. The chroma QP used to derive chroma deblocking parameters can be based on image-level chroma QP offset and CU-level chroma QP offset, but is independent of strip-level chroma QP offset.

[0489] a) In one example, the chroma QP used to derive the chroma deblocking parameters can depend on pps_cb_qp_offset, pps_cr_qp_offset, pps_cbcr_qp_offset, and CuQpOffset. Cb CuQpOffset Cr and CuQpOffset CbCr However, it is unrelated to slice_cb_qp_offset, slice_cr_qp_offset, and slice_cbcr_qp_offset.

[0490] 5. Examples

[0491] The following is an example implementation that can be applied to the VVC specification. The modifications are based on the latest VVC working draft (JVET-O2001-v14). Newly added sections are highlighted in bold italics, and sections deleted from the VVC working draft are marked with double brackets (e.g., [[a]] indicates the deletion of the character "a").

[0492] Example #1, Explanation of TPM Luminance and Chromaticity Weights

[0493] TPM chromaticity weights are aligned with luminance weights based on block width, block height, and chromaticity subsampling ratio. Figure 9 An example weight setting for an 8x16 TPM prediction block is shown.

[0494] 5.1. Example #2, regarding the alignment of TPM chromaticity weight and TPM luminance weight

[0495] 8.5.7 Decoding process of triangular inter-frame blocks

[0496] 8.5.7.1 Overall

[0497] This procedure is called when decoding the codec unit with MergeTriangleFlag[xCb][yCb] equal to 1.

[0498] The input for this process is:

[0499] – The luminance position (xCb, yCb) specifies the top-left luminance sample of the current codec block relative to the top-left luminance sample of the current image.

[0500] – The variable cbWidth specifies the width of the current codec block in the luminance sample.

[0501] – The variable cbHeight specifies the height of the current codec block in the luminance sample.

[0502] – Brightness motion vectors mvA and mvB with a precision of 1 / 16 fraction of a sample point

[0503] – Chromaticity motion vectors mvCA and mvCB.

[0504] –Refer to indices refIdxA and refIdxB,

[0505] – Prediction list flags predListFlagA and predListFlagB.

[0506] The output of this process is:

[0507] –The array predSamplesL of (cbWidth)x(cbHeight) predicted brightness samples.

[0508] – array predSamples of chromaticity prediction samples for component Cb (cbWidth / SubWidthC) x (cbHeight / SubHeightC) Cb ,

[0509] – The chromaticity prediction sample array predSamples for component Cr (cbWidth / SubWidthC) x (cbHeight / SubHeightC) Cr ,

[0510] Let predSamplesLA L and predSamplesLB L Let predSamplesLB be an array of (cbWidth) x (cbHeight) predicted luminance sample values. Cb ,predSamplesLA Cr and predSamplesLB Cr It is an array of (cbWidth / SubWidthC)x(cbHeight / SubHeightC) of predicted chromaticity sample values.

[0511] predSamples L ,predSamples Cb and predSamples Cr Derived by the following steps in sequence:

[0512] 1. For N to be each of A and B, the following applies:

[0513] – Use the following procedure, specified in Clause 8.5.6.2 of VVC WD6, to derive an ordered two-dimensional array refPicLN of luminance samples. L The two ordered two-dimensional arrays refPicLN of chromaticity samples Cb and refPicLN Cr The reference image is constructed as follows: X is set to be equal to predListFlagN and refIdxX is set to be equal to refIdxN as input.

[0514] – array predSamplesLN L It is derived using the following fractional sample interpolation procedure specified in Clause 8.5.6.3: luminance position (xCb, yCb), luminance codec block width sbWidth set to equal cbWidth, luminance codec block height sbHeight set to equal cbHeight, motion vector offset mvOffset set to equal (0,0), motion vector mvLX set to equal mvN, and refPicLN set to equal refPicLN. L The reference array refPicLX L The variable bdofFlag, set to FALSE, and the variable cIdx, set to 0, are used as inputs.

[0515] – array predSamplesLN Cb It is derived using the following fractional sample interpolation procedure specified in Clause 8.5.6.3: luminance position (xCb, yCb), codec block width sbWidth set to equal cbWidth / SubWidthC, codec block height sbHeight set to equal cbHeight / SubHeightC, motion vector offset mvOffset set to equal (0,0), motion vector mvLX set to equal mvCN, and refPicLN set to equal... Cb The reference array refPicLX Cb The variable bdofFlag, set to FALSE, and the variable cIdx, set to 1, are used as inputs.

[0516] – array predSamplesLN CrIt is derived using the following fractional sample interpolation procedure specified in Clause 8.5.6.3: luminance position (xCb, yCb), codec block width sbWidth set to equal cbWidth / SubWidthC, codec block height sbHeight set to equal cbHeight / SubHeightC, motion vector offset mvOffset set to equal (0,0), motion vector mvLX set to equal mvCN, and refPicLN set to equal... Cr The reference array refPicLX Cr The variable bdofFlag, set to FALSE, and the variable cIdx, set to 2, are used as inputs.

[0517] 2. Set the split direction variable triangleDir of the merge triangle mode to equal merge_triangle_split_dir[xCb][yCb].

[0518] 3. The predicted samples predSamplesL[xL][yL] within the current luma codec block, where xL = 0..cbWidth-1 and yL = 0..cbHeight-1, are derived using the following weighted sample prediction procedure via the triangular merging mode specified in Clause 8.5.7.2: the codec block width nCbW set to equal cbWidth, the codec block height nCbH set to equal cbHeight, the sample arrays predSamplesLAL and predSamplesLBL, and the variables triangleDir and cIdx equal to 0 as inputs.

[0519] 4. The predicted samples predSamplesCb[xC][yC] within the current chroma component Cb codec block, where xC = 0..cbWidth / SubWidthC-1 and yC = 0..cbHeight / SubHeightC-1, are derived using the following weighted sample prediction procedure by invoking the triangular merging mode specified in Clause 8.5.7.2: the luminance codec block width nCbW set to equal cbWidth[[ / SubWidthC]], the luminance codec block height nCbH set to equal cbHeight[[ / SubHeightC]], the sample arrays predSamplesLACb and predSamplesLBCb, and the variables triangleDir and cIdx equal to 1 as inputs.

[0520] 5. The predicted samples predSamplesCr[xC][yC] within the current chroma component Cr codec block, where xC = 0..cbWidth / SubWidthC-1 and yC = 0..cbHeight / SubHeightC-1, are derived using the following weighted sample prediction procedure by invoking the triangular merging mode specified in Clause 8.5.7.2: the luminance codec block width nCbW set to equal cbWidth[[ / SubWidthC]], the luminance codec block height nCbH set to equal cbHeight[[ / SubHeightC]], the sample arrays predSamplesLACr and predSamplesLBCr, and the variables triangleDir and cIdx equal to 2 as inputs.

[0521] 6. Use the following call to the motion vector stored procedure of the merged triangle pattern specified in Clause 8.5.7.3: luma codec block (xCb, yCb), luma codec block width cbWidth, luma codec block height cbHeight, segmentation direction triangleDir, luma motion vectors mvA and mvB, reference indices refIdxA and refIdxB, and prediction list flags predListFlagA and predListFlagB as input.

[0522] 8.5.7.2 Weighted Sample Prediction Process of Triangular Merging Model

[0523] The input for this process is:

[0524] – Two variables, nCbW and nCbH, specify the width and height of the current luma codec block.

[0525] – Two (nCbW / SubWidthC)x(nCbH / SubHeightC) arrays, predSamplesLA and predSamplesLB.

[0526] – The variable triangleDir specifies the splitting direction.

[0527] – The variable cIdx specifies the color component index.

[0528] The output of this process is an array pbSamples of predicted sample values ​​(nCbW / SubWidthC)x(nCbH / SubHeightC).

[0529] The variable nCbR is exported as follows:

[0530] nCbR=(nCbW>nCbH)? (nCbW / nCbH):(nCbH / nCbW) (8-841)

[0531] The variable bitDepth is exported as follows:

[0532] - If cIdx equals 0, then bitDepth is set to equal BitDepth. Y .

[0533] Otherwise, bitDepth is set to equal BitDepth. C .

[0534] The variables shift1 and offset1 are exported as follows:

[0535] - The variable shift1 is set to equal Max(5, 17-bitDepth).

[0536] - Set the variable offset1 to equal 1 << (shift1 - 1).

[0537] Depending on the values ​​of triangleDir[[, wS and cIdx]], predict sample points pbSamples[x][y], where x = 0..nCbW / SubWidthC-1 and y = 0..nCbH / SubHeightC-1, derived as follows:

[0538] The variables xIdx and yIdx are derived as follows:

[0539] xIdx=(cIdx==0)? x:x*SubWidthC

[0540] yIdx=(cIdx==0)? y:y*SubHeightC

[0541] – The variable [[wIdx]]wValue, which specifies the weights of the predicted samples, is derived using wIdx and cIdx as follows:

[0542] – If [[cIdx equals 0 and]]triangleDir equals 0, then the following applies:

[0543] [[wIdx]]wValue=(nCbW>nCbH)? (Clip3(0,8,(xIdx / nCbR-yIdx)+4)) (8-842)

[0544] :(Clip3(0,8,(xIdx-yIdx / nCbR)+4))

[0545] Otherwise, if cIdx equals 0 and triangleDir equals 1, then the following applies:

[0546] [[wIdx]]wValue=(nCbW>nCbH)? (Clip3(0,8,(nCbH-1-xIdx / nCbR-yIdx)+4))(8-843)

[0547] (Clip3(0,8,(nCbW-1-xIdx-yIdx / nCbR)+4))

[0548] Otherwise, if cIdx is greater than 0 and triangleDir is equal to 0, then the following applies:

[0549] wIdx=(nCbW>nCbH)? (Clip3(0,4,(x / nCbR-y)+2))(8-844)

[0550] :(Clip3(0,4,(xy / nCbR)+2))

[0551] Otherwise, (if cIdx is greater than 0 and triangleDir is equal to 1), the following applies:

[0552] wIdx=(nCbW>nCbH)? (Clip3(0,4,(nCbH-1-x / nCbR-y)+2)) (8-845)

[0553] (Clip3(0,4,(nCbW-1-xy / nCbR)+2))

[0554] – The variable wValue, which specifies the weights of the predicted samples, is derived using wIdx and cIdx as follows:

[0555] wValue=(cIdx==0)? Clip3(0,8,wIdx):Clip3(0,8,wIdx*2) (8-846)]]

[0556] – The predicted sample values ​​are exported as follows:

[0557] pbSamples[x][y] = Clip3(0, (1< <bitDepth)-1,(predSamplesLA[x][y]*wValue+(8-847)

[0558] predSamplesLB[x][y]*(8-wValue)+offset1)>>shift1)

[0559] 5.2. Example #3, TPM regarding the adjustment of the block width-to-height ratio

[0560] 7.3.8.7 Syntax for Merging Data

[0561]

[0562]

[0563]

[0564] 7.4.9.7 Merging Data Semantics

[0565] The variable MergeTriangleFlag[x0][y0] (which specifies whether triangle-based motion compensation is used to generate prediction samples for the current codec unit when decoding B-strips) is derived as follows:

[0566] – If all of the following conditions are true, then MergeTriangleFlag[x0][y0] will be set to 1:

[0567] –sps_triangle_enabled_flag equals 1.

[0568] –slice_type equals B.

[0569] –general_merge_flag[x0][y0] equals 1.

[0570] –MaxNumTriangleMergeCand is greater than or equal to 2.

[0571] –cbWidth*cbHeight is greater than or equal to 64.

[0572] –regular_merge_flag[x0][y0] equals 0.

[0573] –merge_subblock_flag[x0][y0] equals 0.

[0574] –ciip_flag[x0][y0] equals 0.

[0575] –Abs(Log2(cbWidth)-Log2(cbHeight)) is less than or equal to 2

[0576] Otherwise, set MergeTriangleFlag[x0][y0] to 0.

[0577] 5.3. Example #4, regarding TPM adjustment for block width <128 and height <128

[0578] 7.3.8.7 Syntax for Merging Data

[0579]

[0580]

[0581]

[0582] 7.4.9.7 Merging Data Semantics

[0583] The variable MergeTriangleFlag[x0][y0] (which specifies whether triangle-based motion compensation is used to generate prediction samples for the current codec unit when decoding B-strips) is derived as follows:

[0584] – If all of the following conditions are true, then MergeTriangleFlag[x0][y0] will be set to 1:

[0585] –sps_triangle_enabled_flag equals 1.

[0586] –slice_type equals B.

[0587] –general_merge_flag[x0][y0] equals 1.

[0588] –MaxNumTriangleMergeCand is greater than or equal to 2.

[0589] –cbWidth*cbHeight is greater than or equal to 64.

[0590] –regular_merge_flag[x0][y0] equals 0.

[0591] –merge_subblock_flag[x0][y0] equals 0.

[0592] –ciip_flag[x0][y0] equals 0.

[0593] –cbWidth is less than 128.

[0594] –cbHeight is less than 128.

[0595] Otherwise, set MergeTriangleFlag[x0][y0] to 0. 5.4. Regarding TPM adjustment with block width > 4 and height > 4

[0596] 7.3.8.7 Syntax for Merging Data

[0597]

[0598]

[0599] 7.4.9.7 Merging Data Semantics

[0600] The variable MergeTriangleFlag[x0][y0] (which specifies whether triangle-based motion compensation is used to generate prediction samples for the current codec unit when decoding B-strips) is derived as follows:

[0601] – If all of the following conditions are true, then MergeTriangleFlag[x0][y0] will be set to 1:

[0602] –sps_triangle_enabled_flag equals 1.

[0603] –slice_type equals B.

[0604] –general_merge_flag[x0][y0] equals 1.

[0605] –MaxNumTriangleMergeCand is greater than or equal to 2.

[0606] –cbWidth*cbHeight is greater than or equal to 64.

[0607] –regular_merge_flag[x0][y0] equals 0.

[0608] –merge_subblock_flag[x0][y0] equals 0.

[0609] –ciip_flag[x0][y0] equals 0.

[0610] –cbWidth is greater than 4.

[0611] –cbHeight is greater than 4.

[0612] Otherwise, set MergeTriangleFlag[x0][y0] to 0.

[0613] 5.5. Example #6, Regarding ISP signaling notifications unrelated to minimum and maximum transform sizes

[0614] 7.3.8.5 Encoding / Decoding Unit Syntax

[0615]

[0616]

[0617] 5.6. Example #7, regarding ISP applied to blocks larger than the maximum transform size

[0618] 7.3.8.6 Encoding / Decoding Unit Syntax

[0619]

[0620] 5.7. Example #8, regarding ISP application for block sizes greater than 16 pixels

[0621] 7.3.8.7 Encoding / Decoding Unit Syntax

[0622]

[0623]

[0624] 5.8. Example of MV rounding

[0625] Changes to the draft work

[0626] The working draft specified in JVET-O2001-v14 has been changed as follows. Newly added sections are highlighted in bold italics. Removed sections are marked with double brackets.

[0627] 8.5.5.3 Derivation process of sub-block-based temporal merging candidates

[0628] The input for this process is:

[0629] –…

[0630] The output of this process is:

[0631] –…

[0632] …

[0633] – Use the following call to round the motion vector as specified in Clause 8.5.2.14: take mvX set to equal tempMv[0], rightShift set to equal to 4, and leftShift set to equal to 0 as inputs, and take the rounded tempMv[0] as output.

[0634] – Use the following call to round the motion vector as specified in Clause 8.5.2.14: take mvX set to equal tempMv[1], rightShift set to equal to 4, and leftShift set to equal to 0 as inputs, and take the rounded tempMv[1] as output.

[0635] – For xSbIdx = 0..numSbX-1 and ySbIdx = 0..numSbY-1, the motion vector mvLXSbCol[xSbIdx][ySbIdx] and the prediction list are derived using the flag predFlagLXSbCol[xSbIdx][ySbIdx] as follows:

[0636] – The brightness position (xSb, ySb) of the top-left sample of the current encoding / decoding sub-block relative to the top-left luminance sample of the current image is exported as follows:

[0637] xSb=xCb+xSbIdx*sbWidth+sbWidth / 2 (8-551)

[0638] ySb=yCb+ySbIdx*sbHeight+sbHeight / 2 (8-552)

[0639] The positions (xColSb, yColSb) of the colocation sub-blocks inside –ColPic are derived as follows.

[0640] – The following applies:

[0641] yColSb=Clip3(yCtb,

[0642] Min(CurPicHeightInSamplesY-1,yCtb+(1<<CtbLog2SizeY)-1),ySb+[[(]]tempMv[1][[> >4)]]) (8-553

[0643] – If subpic_treated_as_pic_flag[SubPicIdx] equals 1, then the following applies:

[0644] xColSb=Clip3(xCtb,

[0645] Min(SubPicRightBoundaryPos,xCtb+(1<<CtbLog2SizeY)+3),xSb+[[(]]tempMv[0][[> >4)]]) (8-554

[0646] – Otherwise (subpic_treated_as_pic_flag[SubPicIdx] equals 0), the following applies:

[0647] xColSb=Clip3(xCtb,

[0648] Min(CurPicWidthInSamplesY-1,xCtb+(1<<CtbLog2SizeY)+3),xSb+[[(]]tempMv[0][[> >4)]]) (8-555

[0649] …

[0650] 8.5.5.4 Process of exporting basic motion data based on sub-block temporal merging

[0651] The input for this process is:

[0652] –...

[0653] The output of this process is:

[0654] –...

[0655] The variable tempMv is set as follows:

[0656] tempMv[0]=0 (8-558)

[0657] tempMv[1]=0 (8-559)

[0658] The variable currPic specifies the current image.

[0659] If availableFlagA1 equals TRUE, then the following applies:

[0660] – If all of the following conditions are true, then tempMv is set to equal mvL0A1:

[0661] –predFlagL0A1 equals 1.

[0662] –DiffPicOrderCnt(ColPic,RefPicList[0][refIdxL0A1]) equals 0,

[0663] Otherwise, if all of the following conditions are true, then tempMv is set to equal mvL1A1:

[0664] –slice_type equals B,

[0665] –predFlagL1A1 equals 1,

[0666] –DiffPicOrderCnt(ColPic,RefPicList[1][refIdxL1A1]) equals 0.

[0667] – Use the following call to round the motion vector as specified in Clause 8.5.2.14: take mvX set to equal tempMv[0], rightShift set to equal to 4, and leftShift set to equal to 0 as inputs, and take the rounded tempMv[0] as output.

[0668] – Use the following call to round the motion vector as specified in Clause 8.5.2.14: take mvX set to equal tempMv[1], rightShift set to equal to 4, and leftShift set to equal to 0 as inputs, and take the rounded tempMv[1] as output.

[0669] The positions of the co-occurrence blocks (xColCb, yColCb) inside ColPic are exported as follows.

[0670] – Applications are as follows:

[0671] yColCb=Clip3(yCtb,

[0672] Min(CurPicHeightInSamplesY-1,yCtb+(1<<CtbLog2SizeY)-1),yColCtrCb+[[(]]tempMv[1][[> >4)]]) (8-560

[0673] – If subpic_treated_as_pic_flag[SubPicIdx] equals 1, then the following applies:

[0674] xColCb=Clip3(xCtb,

[0675] Min(SubPicRightBoundaryPos,xCtb+(1<<CtbLog2SizeY)+3),xColCtrCb+[[(]]tempMv[0][[> >4)]]) (8-561

[0676] – Otherwise, if subpic_treated_as_pic_flag[SubPicIdx] equals 0, the following applies:

[0677] xColCb=Clip3(xCtb,

[0678] Min(CurPicWidthInSamplesY-1,xCtb+(1<<CtbLog2SizeY)+3),xColCtrCb+[[(]]tempMv[0][[> >4)]]) (8-562

[0679] 5.9. Example of a Sub-TMVP

[0680] 8.5.5.2 Derivation process of motion vectors and reference indices in sub-block merging mode

[0681] The input for this process is:

[0682] – The brightness position (xCb, yCb) of the top-left sample of the current luminance block relative to the top-left sample of the current image.

[0683] – Two variables, cbWidth and cbHeight, specify the width and height of the luminance codec block.

[0684] The output of this process is:

[0685] – The number of luminance codec sub-blocks in the horizontal direction, numSbX, and the number of luminance codec sub-blocks in the vertical direction, numSbY.

[0686] –Refer to indices refIdxL0 and refIdxL1,

[0687] – The prediction list utilizes flag arrays predFlagL0[xSbIdx][ySbIdx] and predFlagL1[xSbIdx][ySbIdx].

[0688] – Luminance sub-block motion vector arrays mvL0[xSbIdx][ySbIdx] and mvL1[xSbIdx][ySbIdx] with a precision of 1 / 16 fraction of a sample, where xSbIdx = 0..numSbX–1, ySbIdx = 0..numSbY–1,

[0689] – A chromaticity sub-block motion vector array mvCL0[xSbIdx][ySbIdx] and mvCL1[xSbIdx][ySbIdx] with a precision of 1 / 32 fractional sample points, where xSbIdx = 0..numSbX–1, ySbIdx = 0..numSbY–1,

[0690] – Bidirectional prediction weight index bcwIdx.

[0691] The variables numSbXAff and numSbYAff are set as follows.

[0692] numSbXAff=cbWidth>>2

[0693] numSbYAff=cbHeight>>2

[0694] The variables numSbX, numSbY, and the subblock merge candidate list subblockMergeCandList are derived by the following ordered steps:

[0695] 1. When sps_sbtmvp_enabled_flag equals 1, the following applies:

[0696] – For exports of availableFlagA1, refIdxLXA1, predFlagLXA1, and mvLXA1, the following applies:

[0697] – The luminance position (xNbA1, yNbA1) inside the adjacent luminance codec block is set to equal to (xCb-1, yCb+cbHeight-1).

[0698] – Use the following call to the adjacent block applicability derivation procedure as specified in Clause 6.4.4: the current luminance position (xCurr, yCurr) set to equal (xCb, yCb), the adjacent luminance position (xNbA1, yNbA1), checkPredModeY set to equal TRUE, and cIdx set to equal 0 as inputs, and the output assigned to the block applicability flag availableA1.

[0699] – The variables availableFlagA1, refIdxLXA1, predFlagLXA1, and mvLXA1 are exported as follows:

[0700] – If availableA1 equals FALSE, then availableFlagA1 is set to equal to 0, both components of mvLXA1 are set to equal to 0, refIdxLXA1 is set to equal to -1 and predFlagLXA1 is set to equal to 0, where X is 0 or 1, and bcwIdxA1 is set to equal to 0.

[0701] – Otherwise, set availableFlagA1 to equal 1 and make the following allocation:

[0702] mvLXA1=MvLX[xNbA1][yNbA1] (8-498)

[0703] refIdxLXA1=RefIdxLX[xNbA1][yNbA1] (8-499)

[0704] predFlagLXA1=PredFlagLX[xNbA1][yNbA1] (8-500)

[0705] – Use the following derivation procedure for sub-block-based temporal merging candidates as specified in Clause 8.5.5.3: Luminance position (xCb, yCb), luminance codec block width cbWidth, luminance codec block height cbHeight, availability flag availableFlagA1, reference index refIdxLXA1, prediction list utilization flag predFlagLXA1, and motion vector mvLXA1 as output, and the output is availability flag availableFlagSbCol, level The number of luminance codec blocks in the directional direction is numSbXCol and the number of luminance codec blocks in the vertical direction is numSbYCol, the reference index is refIdxLXSbCol, the luminance motion vector is mvLXSbCol[xSbIdx][ySbIdx] and the prediction list uses the flag predFlagLXSbCol[xSbIdx][ySbIdx], where xSbIdx = 0..numSbXCol-1, ySbIdx = 0..numSbYCol–1, and X is 0 or 1.

[0706] 2. When `sps_affine_enabled_flag` equals 1, the sample point positions (xNbA0, yNbA0), (xNbA1, yNbA1), (xNbA2, yNbA2), (xNbB0, yNbB0), (xNbB1, yNbB1), (xNbB2, yNbB2), (xNbB3, yNbB3), and the variables `numSbXAff` and `numSbYAff` are derived as follows:

[0707] (xA0,yA0)=(xCb-1,yCb+cbHeight)

[0708] (8-501)

[0709] (xA1,yA1)=(xCb-1,yCb+cbHeight-1)

[0710] (8-502)

[0711] (xA2,yA2)=(xCb-1,yCb) (8-503)

[0712] (xB0,yB0)=(xCb+cbWidth,yCb-1)

[0713] (8-504)

[0714] (xB1,yB1)=(xCb+cbWidth-1,yCb-1)

[0715] (8-505)

[0716] (xB2,yB2)=(xCb-1,yCb-1)

[0717] (8-506)

[0718] (xB3,yB3)=(xCb,yCb-1) (8-507)

[0719] [[numSbXAff=cbWidth>>2 (8-508)

[0720] numSbYAff=cbHeight>>2(8-509)]]

[0721] 3. When `sps_affine_enabled_flag` equals 1, the variable `availableFlagA` is set to `FALSE` and the following applies to (xNbA0, yNbA0) to (xNbA1, yNbA1)... k ,yNbA k ):

[0722] …

[0723] 8. When numCurrMergeCand is less than MaxNumSubblockMergeCand, repeat the following until numCurrMergeCand equals MaxNumSubblockMergeCand, where mvZero[0] and mvZero[1] are both equal to 0:

[0724] –zeroCand m The reference index and prediction list utilize flags and motion vectors, where m equals (numCurrMergeCand - numOrigMergeCand) and are derived as follows:

[0725] refIdxL0ZeroCand m =0 (8-515)

[0726] predFlagL0ZeroCand m =1 (8-516)

[0727] cpMvL0ZeroCand m [0] = mvZero (8-517)

[0728] cpMvL0ZeroCandm [1] = mvZero (8-518)

[0729] cpMvL0ZeroCand m [2] = mvZero (8-519)

[0730] refIdxL1ZeroCand m =(slice_type==B)? 0:-1 (8-520)

[0731] predFlagL1ZeroCand m =(slice_type==B)? 1:0 (8-521)

[0732] cpMvL1ZeroCand m [0] = mvZero (8-522)

[0733] cpMvL1ZeroCand m [1] = mvZero (8-523)

[0734] cpMvL1ZeroCand m [2] = mvZero (8-524)

[0735] motionModelIdcZeroCand m =1 (8-525)

[0736] bcwIdxZeroCand m =0 (8-526)

[0737] – Add candidate zeroCand to the end of subblockMergeCandList m , where m equals (numCurrMergeCand - numOrigMergeCand), and numCurrMergeCand increments by 1, as follows:

[0738] subblockMergeCandList[numCurrMergeCand++]=zeroCand m (8-527)

[0739] The variables refIdxL0, refIdxL1, predFlagL0[xSbIdx][ySbIdx], predFlagL1[xSbIdx][ySbIdx], mvL0[xSbIdx][ySbIdx], mvL1[xSbIdx][ySbIdx], mvCL0[xSbIdx][ySbIdx], and mvCL1[xSbIdx][ySbIdx], where xSbIdx = 0..numSbX-1 and ySbIdx = 0..numSbY-1, are derived as follows:

[0740] – If subblockMergeCandList[merge_subblock_idx[xCb][yCb]] equals SbCol, numSbX is set to equal numSbXCol, and numSbY is set to equal numSbYCol, then the bidirectional prediction weight index bcwIdx is set to equal 0, and the following applies when X is 0 or 1:

[0741] refIdxLX=refIdxLXSbCol (8-528)

[0742] – For xSbIdx=0..numSbX-1 and ySbIdx=0..numSbY-1, the following applies:

[0743] predFlagLX[xSbIdx][ySbIdx]=predFlagLXSbCol[xSbIdx][ySbIdx] (8-529)

[0744] mvLX[xSbIdx][ySbIdx][0]=mvLXSbCol[xSbIdx][ySbIdx][0] (8-530)

[0745] mvLX[xSbIdx][ySbIdx][1]=mvLXSbCol[xSbIdx][ySbIdx][1] (8-531)

[0746] – When predFlagLX[xSbIdx][ySbIdx] equals 1, use the following call to derive the chromaticity motion vector as in Item 8.5.2.13: take mvLX[xSbIdx][ySbIdx] and refIdxLX as inputs and output mvCLX[xSbIdx][ySbIdx].

[0747] – Make the following allocations: x = xCb..xCb + cbWidth-1 and y = yCb..yCb + cbHeight-1:

[0748] MotionModelIdc[x][y]=0 (8-532)

[0749] – Otherwise (subblockMergeCandList[merge_subblock_idx[xCb][yCb]] is not equal to SbCol), numSbX is set to equal to numSbXAff, numSbY is set to equal to numSbYAff, and the following applies when X is 0 or 1:

[0750] – Using N as the candidate list for merging subblocks, subblockMergeCandList (N = subblockMergeCandList[merge_subblock_idx[xCb][yCb]]), candidates located at position merge_subblock_idx[xCb][yCb] are selected, make the following allocation:

[0751] Figure 10 This is a block diagram of an example video processing system 1900 that can implement the various techniques disclosed herein. Various implementations may include some or some components of system 1900. System 1900 may include an input 1902 for receiving video content. The video content may be received in a raw or uncompressed format (e.g., 8 or 10-bit multi-component pixel values), or in a compressed or encoded format. Input 1902 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces (such as Ethernet, Passive Optical Networking (PON), etc.) and wireless interfaces (such as Wi-Fi or cellular interfaces).

[0752] System 1900 may include a codec component 1904 capable of implementing the various codec or encoding methods described in this document. Codec component 1904 can reduce the average bit rate of the video from input 1902 to the output of codec component 1904 to produce a codec representation of the video. Therefore, codec techniques are sometimes referred to as video compression or video transcoding techniques. The output of codec component 1904 can be stored or transmitted via connected communication, as represented by component 1906. The stored or communicated bitstream (or codec) representation of the video received at input 1902 can be used by component 1908 to generate pixel values ​​or displayable video that is sent to display interface 1910. The process of generating user-visible video from the bitstream representation is sometimes referred to as video decompression. Furthermore, although some video processing operations are referred to as "codec" operations or tools, it should be understood that codec tools or operations are used at the encoder, and the corresponding decoding tools or operations that will be performed by the decoder to reverse the codec results are also executed.

[0753] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI), or DisplayPort. Examples of storage interfaces include SATA (Serial Advanced Technology Accessory), PCI, IDE, etc. The technologies described in this document can be implemented in a variety of electronic devices, such as mobile phones, laptops, smartphones, or other devices capable of performing digital data processing and / or video display.

[0754] Figure 11 This is a block diagram of a video processing device 1100. Device 1100 can be used to implement one or more of the methods described herein. Device 1100 can be implemented as a smartphone, tablet computer, Internet of Things (IoT) receiver, etc. Device 1100 may include one or more processors 1102, one or more memories 1104, and video processing hardware 1106. The processors(multiple) 1102 can be configured to implement one or more methods described herein. The memories(multiple) 1104 can be used to store data and code for implementing the methods and techniques described herein. The video processing hardware 1106 can be used to implement some of the techniques described herein in hardware circuitry. In some embodiments, hardware 1106 may be at least partially located within processor 1102, such as a graphics coprocessor.

[0755] Some embodiments of the technology disclosed herein include determining or enabling a video processing tool or mode. In one example, when a video processing tool or mode is enabled, the encoder will use or implement the tool or mode in the processing of video blocks, but not necessarily modify the resulting bitstream based on the use of the tool or mode. In other words, when a video processing tool or mode is determined or enabled, the conversion from video blocks to a video bitstream will use the video processing tool or mode. In another example, when a video processing tool or mode is enabled, the decoder will utilize knowledge that the bitstream has already been modified based on the video processing tool or mode to process the bitstream. In other words, using a video processing tool or mode determined or enabled, the conversion from a video bitstream to video blocks will be performed.

[0756] Some embodiments of the technology disclosed herein include deciding or determining to disable video processing tools or modes. In one example, when video processing tools or modes are disabled, the encoder will not use any tools or modes in the conversion of video blocks to a bitstream representation of the video. In another example, when video processing tools or modes are disabled, the decoder will process the bitstream knowing that no modifications to the bitstream are being made based on the decision or determination to disable video processing tools or modes.

[0757] In this document, the term "video processing" can refer to video encoding, video decoding, video compression, or video decompression. For example, a video compression algorithm can be applied during the conversion from the pixel representation of a video to its corresponding bitstream representation, and vice versa. The bitstream representation or encoding / decoding representation of the current video block can, for example, correspond to bits or extended bits at different locations within the bitstream. For example, a video block can be encoded using the error residuals of the transformation and encoding / decoding, and using bits in the header and other fields in the bitstream.

[0758] In some embodiments, the following first set of terms may be implemented.

[0759] The following terms may be implemented in conjunction with the additional technologies described in the items listed in the previous chapters (e.g., item 1).

[0760] 1. A method for video processing (e.g., Figure 12 The method shown (1200) includes: for the conversion between a video unit and a video unit codec representation including a luma block and a chroma block co-located with the luma block, determining (1202) chroma weights for the conversion of the chroma block using a triangular partitioning mode (TPM) by aligning the chroma weights for the conversion of the luma block with the luma weights; and performing (1204) the conversion based on the determined result.

[0761] 2. The method of Clause 1, wherein the chromaticity weight is determined as a function of the luminance weight.

[0762] 3. The method of any of Clauses 1-2, wherein the chromaticity weight is a subset of the luminance weight.

[0763] 4. The method of any of Clauses 1-3, wherein for the equal-sized portions of the luminance block that coincide with the chrominance block, the chrominance weight is equal to the luminance weight.

[0764] The following terms may be implemented in conjunction with the additional technologies described in the items listed in the previous chapters (e.g., Item 2).

[0765] 5. A video processing method, comprising: for a conversion between a video unit comprising a luma block and a chroma block co-located with the luma block and a codec representation of the video unit, determining chroma weights for a conversion of the chroma block using a triangular partitioning mode (TPM) based on characteristics of the luma block or characteristics of the video unit; and performing the conversion based on the determined result.

[0766] 6. The method of Clause 5, wherein the characteristics of the luminance block include the height or width of the luminance block.

[0767] 7. The method of any one of Clauses 5-6, wherein the characteristics of the video unit include the color format of the video unit or the chroma subsampling ratio of the video unit.

[0768] 8. The method as described in any of Clauses 5-7, wherein the chromaticity weights also depend on the color component identifiers of the chromaticity blocks.

[0769] The following terms may be implemented in conjunction with the additional technologies described in the items listed in the previous chapters (e.g., item 3).

[0770] 9. The method as described in any of Clauses 1-8, wherein the chromaticity weight and / or luminance weight are equal to integers.

[0771] The following terms may be implemented in conjunction with the additional technologies described in the items listed in the previous chapters (e.g., item 4).

[0772] 10. A video processing method, comprising: converting between video units and encoded / decoded representations of video units that include luma blocks and chroma blocks co-located with the luma blocks; determining, based on characteristics of the video units, whether to use a triangular partitioning mode (TPM) for the conversion; and performing the conversion based on the determined result.

[0773] 11. The method of Item 10, wherein the property is a dimensional ratio equal to max(H,W) / min(H,W), where max and min are functions of maximum and minimum values, and H and W are the height and width of the pixels of the video unit.

[0774] 12. The method of Item 10, wherein the property is a dimension ratio equal to Abs(Log2(cbWidth)-Log2(cbHeight)), where Abs is an absolute value function, and cbWidth and cbHeight are the pixel width and pixel height containing the chroma block.

[0775] 13. The method of Clause 10, wherein the determined result is that TPM is disabled due to a dimensionality ratio greater than 2.

[0776] The following terms may be implemented in conjunction with the additional technologies described in the items listed in the previous chapters (e.g., item 5).

[0777] 14. The method of Clause 10, wherein the characteristics of the video unit include the maximum transform size for the conversion of the video.

[0778] 15. The method of Clause 14, wherein the use of TPM is disabled because the height or width of the video unit is greater than the maximum transform size.

[0779] The following terms may be implemented in conjunction with the additional technologies described in the items listed in the previous chapters (e.g., item 6).

[0780] 16. The method of Clause 10, wherein the characteristics of the video unit include the maximum codec unit size used during the conversion of the video.

[0781] 17. The method of Clause 16, wherein the use of TMP is disabled because the height or width of the cell is equal to the size of the maximum codec cell.

[0782] The following terms may be implemented in conjunction with the additional technologies described in the items listed in the previous chapters (e.g., item 7).

[0783] 18. The method of Clause 10, wherein the characteristics of the video unit include the height or width of the video unit, and wherein it is determined that the use of TPM is disabled because the height is greater than N or the width is greater than M.

[0784] The following terms may be implemented in conjunction with the additional technologies described in the items listed in the previous chapters (e.g., item 8).

[0785] 19. The method of Clause 10, wherein the characteristics of the video unit include the height or width of the video unit, and wherein it is determined that the use of TPM is disabled due to a height of N or a width of M.

[0786] The following terms may be implemented in conjunction with the additional technologies described in the items listed in the previous chapters (e.g., item 9).

[0787] 20. The method of Clause 10, wherein the characteristics of the video unit include the chroma format of the video unit, and wherein it is determined that the use of TPM is disabled because the chroma format is a specific format.

[0788] 21. The method as described in Clause 20, wherein the particular format is 4:0:0.

[0789] The following terms may be implemented in conjunction with the additional technologies described in the items listed in the previous chapters (e.g., item 10).

[0790] 22. The method of Clause 1, wherein the characteristics of the video unit include the resolution of the reference image used in the conversion of the video unit, and wherein the use of TPM is disabled due to the different resolutions of each other.

[0791] The following terms may be implemented in conjunction with the additional technologies described in the items listed in the previous chapters (e.g., item 11).

[0792] 23. The method of any of the preceding clauses, wherein if it is determined that TPM mode is disabled, the codec represents a syntax element that omits TMP syntax elements.

[0793] 24. The method of any one of clauses 1 to 23, wherein the conversion includes encoding the video into a codec representation.

[0794] 25. The method of any one of clauses 1 to 23, wherein the conversion includes decoding the encoding / decoding representation to generate pixel values ​​of the video.

[0795] 26. A video decoding apparatus comprising a processor configured to implement the methods referenced in any one or more of the provisions 1 to 25.

[0796] 27. A video encoding apparatus comprising a processor configured to implement the methods referenced in any one or more of the provisions 1 to 25.

[0797] 28. A computer program product having computer code stored thereon, said code, when executed by a processor, causing the processor to implement the methods referenced in any one or more of clauses 1 to 25.

[0798] 29. This document describes a method, apparatus, or system.

[0799] The second set of clauses describes certain features and methods of the technology disclosed in the preceding sections (e.g., Items 1 to 5).

[0800] 1. A method for video processing (e.g., Figure 16A The method 1610 shown includes: determining (1612) chroma weights according to rules, the chroma weights being used to determine a chroma prediction block of the chroma block of the current block of a video by predicting the chroma blocks of the mixed chroma blocks; and performing (1614) a conversion between the current block and the codec representation of the video according to the determination, wherein the rules specify that the chroma weights are determined from the luminance weights of the co-bit luminance blocks of the current block; wherein the current block is codeced using a geometric segmentation mode.

[0801] 2. The method as described in Clause 1, wherein the rule specifies that the chromaticity weights are a subset of the luminance weights.

[0802] 3. The method as described in Clause 1 or 2, wherein the chroma format of the current block is 4:2:0, 4:2:2, or 4:4:4.

[0803] 4. The method as described in Clause 1, wherein the rule specifies that the chromaticity weight applied to the chromaticity block of size MxN is the same as the luminance weight applied to the luminance block of size MxN, where M and N are integers greater than 0.

[0804] 5. The method as described in Clause 4, wherein the chroma format of the current block is 4:4:4.

[0805] 6. A method for video processing (e.g., Figure 16BThe method 1620 shown includes: determining (1622) chroma weights according to rules, the chroma weights being used to determine a chroma prediction block of the chroma block of the current block of the video by prediction of the mixed chroma blocks; and performing (1624) a conversion between the current block and the codec representation of the video according to the determination, wherein the rules depend on the characteristics of the co-bit luma blocks and / or the characteristics of the current block.

[0806] The current block is encoded and decoded using a geometric segmentation mode.

[0807] 7. The method as described in Clause 6, wherein the characteristics of the co-bit luminance block include the height and / or width of the co-bit luminance block, and the characteristics of the current block include the color format of the current block and / or the chroma subsampling ratio of the current block.

[0808] 8. The method as described in Clause 6, wherein the rule specifies that, for the chroma block having a 4:4:4 chroma format, the chroma weight is the same as the luminance weight applied to the co-bit luminance block.

[0809] 9. The method as described in Clause 6, wherein the rule specifies that, for the chroma block having a 4:2:0 or 4:2:2 chroma format, the chroma weights are subsampled from the luminance weights applied to the co-bit luminance block.

[0810] 10. The method as described in Clause 9, wherein the chroma weight WeightC[x][y] is calculated by WeightY[f(x)][g(y)], where x is an integer between 0 and W / subWidthC-1, y is an integer between 0 and H / subHeightC-1, W and H are the height and width of the chroma block, subWidthC and subHeightC indicate the chroma subsampling ratios in the width and height directions, respectively, and WeightY[a][b] indicates the luminance weight, where a is an integer between 0 and W-1, and b is an integer between 0 and H-1.

[0811] 11. The method as described in Clause 6, wherein the rule specifies that the chromaticity weight depends on the size of the co-bit luminance block and is the same for different color components.

[0812] 12. The method as described in Clause 11, wherein the chromaticity component of size WxH uses the same weight as the luminance component of size WxH, where W and H are integers indicating the width and height of the co-occurring luminance block, respectively.

[0813] 13. The method as described in Clause 6, wherein the rule specifies that the chroma weight depends on the size of the co-bit luma block and the color components of the video unit.

[0814] 14. The method as described in Clause 13, wherein the chromaticity weights are different for different color components of the video unit.

[0815] 15. The method as described in Clause 13, wherein the chromaticity weights are the same for both chromaticity components.

[0816] 16. The method of any one of Clauses 6 to 15, wherein the chromaticity weights and / or the luminance weights are integers.

[0817] 17. The method as described in Clause 16, wherein both odd integer weights and even integer weights are permitted.

[0818] 18. The method of any one of clauses 6 to 17, wherein the chromaticity weights and / or the luminance weights are clipped to the range [M, N], where M and N are integers.

[0819] 19. A method for video processing (e.g., Figure 16C The method 1630 shown includes: performing (1632) a conversion between a current block of a video and a codec representation of the video, wherein during the conversion, a predicted block of the current block is determined by mixing the prediction of the current block according to a mixing weight mask, wherein the mixing weight mask is determined according to a rule; wherein the current block is encoded and decoded using a geometric segmentation mode.

[0820] 20. The method as described in Clause 19, wherein the rule specifies that the hybrid weight mask is derived based on one or more predefined tables containing N elements, where N is an integer greater than 0.

[0821] 21. The method as described in Clause 19, wherein the rule specifies that the hybrid weight mask is calculated from a computational equation.

[0822] 22. The method of any one of clauses 1 to 21, wherein the current block is encoded and decoded using a geometric segmentation mode, wherein two or more sub-regions of the current block are obtained by segmenting the current block along an angle, horizontally, or vertically.

[0823] 23. The method of any one of clauses 1 to 22, wherein the conversion includes encoding the video into the codec representation.

[0824] 24. The method of any one of clauses 1 to 22, wherein the conversion includes decoding the codec representation to generate the video.

[0825] 25. A video processing apparatus comprising a processor configured to implement the methods described in any one or more of clauses 1 to 24.

[0826] 26. A computer-readable medium storing program code, which, when executed, causes a processor to implement the methods described in any one or more of clauses 1 to 24.

[0827] 27. A computer-readable medium storing an encoded or bitstream representation generated according to any one of the methods described above.

[0828] The third set of clauses describes certain features and methods of the technology disclosed in previous sections (e.g., Entries 6 to 14).

[0829] 1. A method for video processing (e.g., Figure 16D The method 1640 shown includes: converting between a current block of a video and a codec representation of the video; determining (1642) the applicability of a geometric segmentation mode to the current block based on the characteristics of the current block; and performing (1644) the conversion based on the determination.

[0830] 2. The method as described in Clause 1, wherein the characteristics of the current block include a dimension ratio obtained as max(H,W) / min(H,W), where max and min are functions of maximum and minimum values, and H and W are the height and width of the current block.

[0831] 3. The method as described in Clause 1, wherein the characteristic of the current block includes the difference between the width and height of the current block.

[0832] 4. The method as described in Clause 3, wherein the difference is obtained as Abs(Log2(cbWidth)-Log2(cbHeight)), where Abs is an absolute value function, and cbWidth and cbHeight are the width and height of the codec block including the current block.

[0833] 5. The method as described in Clause 1, wherein the determination determines that the geometric segmentation mode is disabled because the ratio between the width (W) and height (H) of the current block is greater than X, where X is an integer.

[0834] 6. The method as described in Clause 5, wherein the determination determines that the geometric segmentation mode is disabled because the W / H ratio is greater than T1, where T1 is an integer.

[0835] 7. As described in Clause 6, wherein T1 equals 2.

[0836] 8. The method as described in Clause 5, wherein the determination determines that the geometric segmentation mode is disabled because the H / W ratio is greater than T2, where T2 is an integer.

[0837] 9. As described in Clause 8, wherein T2 equals 2.

[0838] 10. The method as described in Clause 1, wherein the characteristic of the current block includes the maximum transform size for the transformation of the video.

[0839] 11. The method as described in Clause 10, wherein the determination determines that the geometric segmentation mode is disabled because the height or width of the current block is greater than the maximum transform size.

[0840] 12. The method as described in Clause 1, wherein the characteristics of the current block include the maximum codec unit size for the conversion of the video.

[0841] 13. The method as described in Clause 12, wherein the determination determines that the geometric segmentation mode is disabled because the height or width of the current block is equal to the maximum codec unit size.

[0842] 14. The method of Clause 1, wherein the characteristics of the current block include the height or width of the current block, and wherein the determination determines that the geometric segmentation mode is disabled because the height is greater than N and / or the width is greater than M, wherein N and M are integers.

[0843] 15. The method as described in Clause 14, wherein N = M = 64.

[0844] 16. The method of Clause 1, wherein the characteristics of the current block include the height or width of the current block, and wherein the determination determines that the geometric segmentation mode is disabled due to the height being N and / or the width being M.

[0845] 17. The method as described in Clause 16, wherein N = M = 4.

[0846] 18. The method of Clause 1, wherein the characteristics of the current block include the chroma format of the current block, and wherein the determination determines that the geometric segmentation mode is disabled because the chroma format is a specific format.

[0847] 19. The method as described in Clause 18, wherein the particular format is 4:0:0, 4:4:4, or 4:2:2.

[0848] 20. The method as described in Clause 1, wherein the characteristics of the current block include the resolution of the reference image used in the transformation of the current block, and wherein the determination determines that the geometric segmentation mode is disabled because the resolutions are different from each other.

[0849] 21. The method of Clause 1, wherein the characteristic of the current block includes the resolution of a reference image used in the transformation of the current block, and wherein the determination determines that the geometric segmentation mode is disabled because the resolution of one of the reference images is different from the resolution of the current image including the current block.

[0850] 22. The method of any one of the preceding clauses, wherein the transformation comprises determining a weighted mixture of predictions for the current block as two or more sub-regions, and wherein a single transform is applied to the residual of the current block during encoding, or a single inverse transform is applied to the coefficients of the current block parsed from the codec representation during decoding.

[0851] 23. A method for video processing (e.g., Figure 16C The method 1630 shown includes: performing (1632) a conversion between a current block of a video and a codec representation of the video, wherein the codec representation conforms to a format rule that specifies that, in the event that a geometric segmentation mode is disabled for the current block during the conversion, syntax elements describing the geometric segmentation mode are not included in the codec representation.

[0852] 24. The method as described in Clause 23, wherein the format rule specifies that if the syntax element is not included in the codec representation, the syntax element is inferred to be 0.

[0853] 25. The method as described in Clause 23, wherein the format rule specifies that, due to the unavailability of the geometric segmentation pattern, the semantic variable associated with the geometric segmentation pattern is inferred to be 0.

[0854] 26. The method as described in Clause 25, wherein the semantic variable includes a flag indicating that the geometric segmentation mode is disabled.

[0855] 27. The method of any one of clauses 1 to 26, wherein the current block is encoded and decoded using the geometric segmentation pattern, wherein two or more sub-regions of the current block are obtained by segmenting the current block along an angle, horizontally, or vertically.

[0856] 28. The method of any one of Clauses 1 to 27, wherein performing the conversion includes generating the codec representation from the current block.

[0857] 29. The method of any one of clauses 1 to 27, wherein performing the conversion includes generating the current block from the codec representation.

[0858] 30. A video processing apparatus comprising a processor configured to implement the methods described in any one or more of clauses 1 to 29.

[0859] 31. A computer-readable medium storing program code, which, when executed, causes a processor to implement the method described in any one or more of clauses 1 to 29.

[0860] 32. A computer-readable medium storing an encoded or bitstream representation generated according to any one of the methods described above.

[0861] In the above terms, performing a conversion includes using the results of previously determined steps during the encoding or decoding operation to achieve the conversion result.

[0862] Other schemes, examples, embodiments, modules, and functional operations disclosed in this document may be implemented in digital electronic circuits or in computer software, firmware, or hardware, including the structures disclosed in this document and their equivalents, or combinations thereof. The disclosed and other embodiments may be implemented as one or more computer program products, such as one or more computer program instruction modules encoded on a computer-readable medium for execution by a data processing apparatus or for controlling the operation of a data processing device. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a complex influencing machine-readable propagating signaling notifications, or a combination thereof. The term "data processing apparatus" encompasses all devices, apparatuses, and machines for processing data, including, for example, programmable processors, computers, or multiple processors or computers. In addition to hardware, the apparatus may also include code that creates an execution environment for the computer program in question, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination thereof. Propagating signaling notifications are artificially generated signaling notifications, such as machine-generated telecommunication signaling notifications, optical signaling notifications, or electromagnetic signaling notifications, which are generated to encode information for transmission to a suitable receiver device.

[0863] Computer programs (also known as programs, software, software applications, scripts, or code) can be written in any programming language, including compiled or interpreted languages, and can be deployed in any form, including standalone programs or modules, components, subroutines, or other units suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple co-located files (e.g., a file storing one or more modules, subroutines, or code portions). A computer program can be deployed to execute on one computer or on multiple computers located at a single site or distributed across multiple sites and interconnected by a communication network.

[0864] The processes and logic flows described in this document can be performed by one or more programmable processors executing one or more computer programs to perform functions by manipulating input data and producing outputs. The processes and logic flows can also be performed by special-purpose logic circuitry (e.g., FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits)), and the device can be implemented as special-purpose logic circuitry (e.g., FPGAs or ASICs).

[0865] Processors suitable for executing computer programs include, for example, both general-purpose and special-purpose microprocessors, and any one or more processors in any type of digital computer. Typically, a processor receives instructions and data from read-only memory or random access memory, or both. The basic components of a computer are a processor for executing instructions and one or more storage devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices (e.g., magneto-optical, magneto-optical, or optical disc) for storing data, or operatively coupled to receive data from or transfer data to a mass storage device (e.g., magneto-optical, magneto-optical, or optical disc), or both. However, a computer does not necessarily have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by dedicated logic circuitry or integrated into dedicated logic circuitry.

[0866] While this patent document contains numerous details, these details should not be construed as limiting any subject matter or the scope of the claims, but rather as descriptions of features specific to particular embodiments of a particular technology. In this patent document, certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented separately in multiple embodiments or in various suitable sub-combinations. Furthermore, although features may be described above as operating in certain combinations and even initially claimed in this way, in certain circumstances one or more features from the claimed combination may be removed from the combination, and the claimed combination may be for sub-combinations or variations thereof.

[0867] Similarly, although operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring such operations to be performed in the specific order or sequence shown, or to perform all the operations shown, in order to achieve the desired result. Furthermore, the separation of various system components in the embodiments described in this patent document should not be construed as requiring such separation in all embodiments.

[0868] Only a few implementations and examples are described, and other implementations, enhancements and variations can be made based on what is described and shown in this patent document.

Claims

1. A video processing method, comprising: For the conversion between the current block of the video and the bitstream of the video, the applicability of the geometric segmentation mode to the current block is determined based on the characteristics of the current block; as well as The conversion is performed based on the determination. The characteristics of the current block include the precision of the reference image used in the transformation of the current block, and the determination determines whether the geometric segmentation mode is disabled due to the difference in precision between the reference images, or... The characteristics of the current block include the precision of the reference images used in the transformation of the current block, and wherein the determination determines that the geometric segmentation mode is disabled because the precision of one of the reference images is different from the precision of the current image including the current block.

2. The method as described in claim 1, wherein, The characteristics of the current block include the dimension ratio obtained as max(H,W) / min(H,W), where max and min are the maximum and minimum value functions, and H and W are the height and width of the current block.

3. The method as described in claim 1, wherein, The characteristics of the current block include the difference between the width and height of the current block.

4. The method of claim 3, wherein, The difference is obtained as Abs(Log2(cbWidth)-Log2(cbHeight)), where Abs is the absolute value function, and cbWidth and cbHeight are the width and height of the codec block including the current block.

5. The method of claim 1, wherein, The determination determines that the geometric segmentation mode is disabled because the ratio between the width (W) and height (H) of the current block is greater than X, where X is an integer.

6. The method of claim 5, wherein, The determination determines that the geometric segmentation mode is disabled because the W / H ratio is greater than T1, where T1 is an integer.

7. The method of claim 6, wherein, T1 equals 2.

8. The method of claim 5, wherein, The determination determines that the geometric segmentation mode is disabled because the H / W ratio is greater than T2, where T2 is an integer.

9. The method of claim 8, wherein, T2 equals 2.

10. The method of claim 1, wherein, The characteristics of the current block include the maximum transform size for the transformation of the video.

11. The method of claim 10, wherein, The determination determines that the geometric segmentation mode is disabled because the height or width of the current block is greater than the maximum transform size.

12. The method of claim 1, wherein, The characteristics of the current block include the maximum codec unit size for the conversion of the video.

13. The method of claim 12, wherein, The determination determines that the geometric segmentation mode is disabled because the height or width of the current block is equal to the maximum codec unit size.

14. The method of claim 1, wherein, The characteristics of the current block include the height or width of the current block, and the determination determines that the geometric segmentation mode is disabled because the height is greater than N and / or the width is greater than M, where N and M are integers.

15. The method of claim 14, wherein, N = M = 64.

16. The method of claim 1, wherein, The characteristics of the current block include the height or width of the current block, and the determination determines that the geometric segmentation mode is disabled because the height is N and / or the width is M.

17. The method of claim 16, wherein, N = M = 4.

18. The method of claim 1, wherein, The characteristics of the current block include the chroma format of the current block, and wherein the determination determines that the geometric segmentation mode is disabled because the chroma format is a specific format.

19. The method of claim 18, wherein, The specific format is 4:0:0, 4:4:4, or 4:2:

2.

20. The method of claim 1, wherein, The transformation includes determining a prediction block for the current block as a weighted mixture of predictions from two or more sub-regions, wherein a single transform is applied to the residual of the current block during encoding, or a single inverse transform is applied to the coefficients of the current block parsed from the bitstream during decoding.

21. The method as described in claim 1, in, The bitstream conforms to a format rule that stipulates that, if the geometric segmentation mode is disabled for the current block during the conversion, the syntax elements describing the geometric segmentation mode are not included in the bitstream.

22. The method of claim 21, wherein, The formatting rules stipulate that if the syntax element is not included in the bitstream, the syntax element is inferred to be 0.

23. The method of claim 21, wherein, The formatting rules stipulate that, since the geometric segmentation pattern is unavailable, the semantic variables related to the geometric segmentation pattern are inferred to be 0.

24. The method of claim 23, wherein, The semantic variables include flags indicating that the geometric segmentation mode is disabled.

25. The method of claim 1, wherein, The current block is encoded and decoded using the geometric segmentation mode, in which two or more sub-regions of the current block are obtained by segmenting the current block along an angle, horizontally, or vertically.

26. The method according to any one of claims 1 to 25, wherein, Performing the conversion includes generating the bit stream from the current block.

27. The method according to any one of claims 1 to 25, wherein, Performing the conversion includes generating the current block from the bitstream.

28. A video processing apparatus comprising a processor configured to implement the method of any one of claims 1 to 27.

29. A computer-readable medium storing program code, which, when executed, causes a processor to perform the method of any one of claims 1 to 27.

Images