Coding and decoding method and device

Abstract

The present invention provides a method for decoding or encoding, the method comprising: determining whether an inter-frame prediction mode called weighted prediction is enabled for a current block; and if weighted prediction is enabled, modifying a part of a reconstruction process of the current block related to the weighted prediction and / or to at least another prediction tool in a first set of prediction tools, the first set including a triangle prediction mode, a geometric prediction mode, a local illumination compensation mode, a combined intra-frame inter-frame prediction mode, and a mode that can refine motion information for motion compensation based on an optical flow model.

Description

1. Technical Field

[0001] At least one embodiment of the present invention generally relates to a method and apparatus for video encoding or decoding, and more specifically to a method and apparatus for video encoding or decoding using a prediction sample weighting process or an optical flow-based motion information refinement process. 2. Background technology

[0002] In order to achieve high compression efficiency, video coding schemes typically use prediction and transformation to exploit spatial and temporal redundancy in video content. During encoding, the image of the video content is divided into pixel blocks, and then these blocks are partitioned into one or more sub-blocks, hereinafter referred to as original sub-blocks. Intra-frame or inter-frame prediction is then applied to each sub-block to exploit intra-frame or inter-frame image correlation. Regardless of the prediction method used (intra-frame or inter-frame), a prediction value sub-block is determined for each original sub-block. Then, the sub-block representing the difference between the original sub-block and the prediction value sub-block (usually represented as a prediction error sub-block or a prediction residual sub-block) is transformed, quantized, and entropy encoded to generate an encoded video stream. In order to reconstruct the video, the compressed data is decoded by the inverse process corresponding to the transformation, quantization, and entropy encoding.

[0003] A joint collaborative team of ITU-T and ISO / IEC experts, called the Joint Video Experts Team (JVET), is developing an international standard called Versatile Video Coding (VVC), in which new compression tools are introduced. For example, in VVC, the number of compression tools that use a prediction sample weighting process (hereinafter referred to as PSW tools) has increased slightly. At the same time, new tools have emerged that aim to apply refinements of motion information based on the concept of optical flow, hereinafter referred to as OF tools. These PSW tools (or OF tools) use similar concepts, and when these tools are combined, the improvements in compression provided by each tool alone are not cumulative. In addition, when these tools (PSW tools and OF tools) are combined, contradictory effects may occur, so the combination may lead to a reduction in compression efficiency.

[0004] It is desirable to propose solutions that allow avoiding incompatible combinations of PSW tools and / or OF tools. It is also desirable to propose solutions that avoid the joint use of PSW tools and / or OF tools, when the use of only one of these tools would result in similar compression efficiency but reduced encoding / decoding complexity, or when the combination of PSW tools and / or OF tools would result in unacceptable complexity. 3. Summary of the invention

[0005] In a first aspect, one or more embodiments of the present invention provide a method for decoding, the method comprising:

[0006] determining, from the encoded video stream, whether to enable an inter-prediction mode called weighted prediction for a current block, wherein a residual block is calculated as a difference between the current block and a weighted version of a reference block (in the case of unidirectional inter-prediction) or a weighted average of two reference blocks (in the case of bidirectional inter-prediction); and,

[0007] If weighted prediction is enabled, modify a portion of a reconstruction process of the current block associated with the weighted prediction and / or with at least another prediction tool in a first set of prediction tools, the first set comprising a triangle prediction mode, a geometric prediction mode, a local illumination compensation mode, a combined intra-frame and inter-frame prediction mode, and a mode for refining motion information for motion compensation based on an optical flow model.

[0008] In a second aspect, one or more embodiments of the present invention provide a device for decoding, the device comprising:

[0009] means for determining from a coded video stream whether to enable an inter-prediction mode called weighted prediction for a current block, wherein a residual block is calculated as a difference between the current block and a weighted version of a reference block (in the case of unidirectional inter-prediction) or a weighted average of two reference blocks (in the case of bidirectional inter-prediction); and,

[0010] A modification device applied when weighted prediction is enabled, which modifies a part of the reconstruction process of the current block related to weighted prediction and / or at least another prediction tool in a first set of prediction tools, the first set including a triangle prediction mode, a geometric prediction mode, a local illumination compensation mode, a combined intra-frame and inter-frame prediction mode, and a mode for refining motion information for motion compensation based on an optical flow model.

[0011] In one embodiment, weighted prediction is enabled when at least one weight used by weighted prediction mentioned in a slice header of a slice including the current block is different from a default weight.

[0012] In one embodiment, the reconstruction process of the current block is modified by preventing parsing or inference of any syntax elements associated with tools in the first set for the current block from the encoded video stream.

[0013] In one embodiment, weighted prediction is enabled when a reference picture providing a reference block for inter prediction of a current block is associated with a weight different from a default weight.

[0014] In one embodiment, determining to enable weighted prediction for the current block includes parsing or inferring syntax elements associated with tools in the first set in a bitstream portion of the encoded video stream representing the current block.

[0015] In one embodiment, the reconstruction process of the current block is modified by preventing the application of the tools in the first set, even if syntax elements of the encoded video stream associated with the tools in the first set indicate that the tools in the first set need to be applied to the current block.

[0016] In one embodiment, the reconstruction process of a current block specified in an encoded video stream is modified by replacing the application of a tool from a first set with the application of an alternative inter-frame prediction tool.

[0017] In one embodiment, the reconstruction process of the current block is modified by changing the semantics of the syntax elements to change the interpretation of the syntax elements associated with the tools in the first set.

[0018] In one embodiment, the reconstruction process of the current block is modified by changing the weighted prediction and the execution order of the tools in the first set.

[0019] In one embodiment, the reconstruction process of the current block is modified by mixing weighted prediction and at least one tool from the first set into a single weighting stage.

[0020] In a third aspect, one or more embodiments of the present invention provide a method for encoding a bitstream portion representing a current block in a coded video stream, the method comprising:

[0021] determining whether an inter prediction mode called weighted prediction is enabled for the current block, in which a residual block is calculated as a difference between the current block and a weighted version of a reference block (in the case of unidirectional inter prediction) or a weighted average of two reference blocks (in the case of bidirectional inter prediction); and,

[0022] If weighted prediction is enabled, modify a portion of the encoding process of the current block associated with the weighted prediction and / or with at least another prediction tool in a first set of prediction tools, the first set including a triangle prediction mode, a geometric prediction mode, a local illumination compensation mode, a combined intra-frame and inter-frame prediction mode, and a mode that can refine motion information for motion compensation based on an optical flow model.

[0023] In a fourth aspect, one or more embodiments of the present invention provide a device for encoding a bit stream portion representing a current block in a coded video stream, the device comprising:

[0024] means for determining whether to enable an inter prediction mode called weighted prediction for the current block, wherein a residual block is calculated as a difference between the current block and a weighted version of a reference block (in the case of unidirectional inter prediction) or a weighted average of two reference blocks (in the case of bidirectional inter prediction); and,

[0025] A modification device applied when weighted prediction is enabled, which modifies a part of the reconstruction process of the current block related to the weighted prediction and / or to at least another prediction tool in a first set of prediction tools, the first set including a triangle prediction mode, a geometric prediction mode, a local illumination compensation mode, a combined intra-frame and inter-frame prediction mode, and a mode that can refine motion information for motion compensation based on an optical flow model.

[0026] In one embodiment, weighted prediction is enabled when at least one weight used by weighted prediction encoded in a slice header of a slice including the current block is different from a default weight.

[0027] In one embodiment, the reconstruction process of the current block is modified by preventing any use of prediction tools from the first set, the bitstream portion of the encoded video stream representing the current block not including syntax elements related to prediction tools from the first set.

[0028] In one embodiment, weighted prediction is enabled when a reference picture providing a reference block for inter prediction of a current block is associated with a weight different from a default weight.

[0029] In one embodiment, the reconstruction process of the current block is modified by reconstructing the current block without applying a prediction tool from a first set selected for encoding the current block, and a syntax element representing the selected prediction tool from the first set is encoded in a bitstream portion of the encoded video stream representing the current block.

[0030] In one embodiment, the reconstruction process of the current block is modified by reconstructing the current block by replacing a prediction tool from a first set selected for encoding the current block with an alternative inter-frame prediction tool, and a syntax element representing the selected prediction tool from the first set is encoded in a bitstream portion of the encoded video stream representing the current block.

[0031] In one embodiment, the reconstruction process of the current block is modified by modifying the weighted prediction and the order of application of the tools in the first set.

[0032] In one embodiment, the reconstruction process of the current block is modified by mixing weighted prediction and at least one tool from the first set into a single weighting stage.

[0033] In a fifth aspect, one or more embodiments of the present invention provide a device, comprising the apparatus according to the second aspect or the fourth aspect.

[0034] In a sixth aspect, one or more embodiments of the present invention provide a signal comprising data generated by the method according to the third aspect or by the device according to the fourth aspect.

[0035] In a seventh aspect, one or more embodiments of the embodiments of the present invention provide a computer program, comprising program code instructions for implementing the decoding method according to the first aspect or for implementing the encoding method according to the third aspect.

[0036] In an eighth aspect, one or more embodiments of the embodiments of the present invention provide an information storage device storing program code instructions for implementing the decoding method according to the first aspect or for implementing the encoding method according to the third aspect. 4. Description of the drawings

[0037] Figure 1 An example of the partitioning that a pixel image of an original video undergoes is shown;

[0038] Figure 2 A method for encoding a video stream is schematically depicted;

[0039] Figure 3 A method for decoding an encoded video stream is schematically depicted;

[0040] Figure 4A Schematically illustrates an example of a hardware architecture of a processing module capable of implementing an encoding module or a decoding module, in which various aspects and embodiments are implemented;

[0041] Figure 4B A block diagram illustrating an example of a system in which various aspects and embodiments are implemented;

[0042] Figure 5A Schematically illustrates an implementation of a triangle prediction mode;

[0043] Figure 5B Schematically illustrates an implementation of a geometric prediction mode;

[0044] Figure 5C Schematically illustrates the adaptive weighting process applied in triangular prediction mode;

[0045] Figure 6 schematically showing an example of application of an optical flow prediction refinement mode;

[0046] Figure 7 is a high-level representation of an implementation of the reconstruction process performed during the encoding process;

[0047] Figure 8 is a high-level representation of an implementation of the reconstruction process performed during the decoding process;

[0048] Fig. 9 The modified parts of the refactoring process are disclosed;

[0049] Fig.10 An example of partitioning a block into two sub-blocks of equal size is schematically shown;

[0050] Fig.11 An example of partitioning a block into two sub-blocks of unequal size is schematically shown;

[0051] Fig.12 schematically illustrates an example parsing process corresponding to deactivation of a PSW or OF tool, including not applying a conventional reconstruction process to the PSW or OF tool; and,

[0052] Fig.13 An example reconfiguration process corresponding to deactivation of a PSW or OF tool is schematically represented, including not applying a conventional reconfiguration process to the PSW or OF tool. 5. Specific implementation methods

[0053] The following description of the embodiments is in the context of VVC. However, these embodiments are not limited to the video encoding / decoding method corresponding to VVC, and are applicable to other video encoding / decoding methods, and are also applicable to other image encoding / decoding methods in which PSW tools and / or OF tools are used and these tools can be used in combination.

[0054] At least one of the following embodiments allows preventing the combination of incompatible PSW tools and / or OF tools.

[0055] At least one of the following embodiments allows reducing the complexity of the encoding and / or decoding process by preventing combinations of PSW tools and / or OF tools that do not result in sufficient compression gain or result in unacceptable complexity.

[0056] Figure 1 An example of the partitioning that a pixel image 11 of an original video 10 undergoes is shown. Here, a pixel is considered to consist of three components: one luminance component and two chrominance components. However, the following embodiments are applicable to images consisting of pixels comprising another number of components (e.g., grayscale images in which pixels comprise one component), or images consisting of pixels comprising three color components and a transparency component and / or a depth component.

[0057] The image is divided into multiple coded entities. First, Figure 1As indicated by reference number 13 in , in VVC, an image is divided into a grid of blocks called coding tree units (CTUs). A CTU consists of N×N blocks of luma samples and two corresponding blocks of chroma samples. N is typically a power of two, such as a maximum value of "128". Secondly, the image is divided into one or more CTU groups. For example, the image may be divided into one or more tile rows and tile columns, where a tile is a sequence of CTUs covering a rectangular area of ​​the image. In some cases, a tile may be divided into one or more bricks, each brick consisting of at least one CTU row within the tile. On top of the concepts of tiles and bricks, there is another coding entity, called a slice, which may contain at least one tile of an image or at least one brick of a tile.

[0058] Two slicing modes are supported, raster scan slicing mode and rectangular slicing mode. In raster scan slicing mode, a slice contains a sequence of tiles from a tile raster scan of an image. In rectangular slicing mode, a slice contains at least one brick of an image forming a rectangular region of the image. The bricks within a rectangular slice are arranged in the order of the brick raster scan of the slice.

[0059] exist Figure 1 In the example, as indicated by reference numeral 12, the image 11 is divided into three slices S1, S2 and S3 in a raster scan slice mode, each slice includes a plurality of tiles (not shown), and each tile includes only one brick.

[0060] like Figure 1As indicated by reference numeral 14 in the figure, a CTU can be partitioned into the form of a hierarchical tree of one or more sub-blocks called coding units (CUs). The CTU is the root (i.e., parent node) of the hierarchical tree and can be partitioned into multiple CUs (i.e., child nodes). If each CU is not further partitioned into smaller CUs, each CU becomes a leaf of the hierarchical tree; or if each CU is further partitioned into smaller CUs (i.e., child nodes), each CU becomes a parent node of the smaller CU. Although the standard HEVC (ISO / IEC 23008-2–MPEG-H Part 2, High Efficiency Video Coding / ITU-T H.265) uses only one type of hierarchical tree called a quadtree, VVC allows the hierarchical tree to mix multiple types of hierarchical trees, including quadtrees, binary trees, and ternary trees. In the quadtree, a CTU (or CU) can be partitioned into "4" square CUs of equal size (i.e., can be the parent node of "4" square CUs of equal size). In a binary tree, a CTU (or CU) may be partitioned horizontally or vertically into "2" rectangular CUs of equal size. In a ternary tree, a CTU (or CU) may be partitioned horizontally or vertically into "3" rectangular CUs. For example, a CU with a height of N and a width of M is vertically (or horizontally) partitioned into a first CU with a height of N (or N / 4) and a width of M / 4 (or M), a second CU with a height of N (or N / 2) and a width of M / 2 (or M), and a third CU with a height of N (or N / 4) and a width of M / 4 (or M).

[0061] exist Figure 1 In the example of , CTU 14 is first partitioned into "4" square CUs using quadtree type partitioning. The top left CU is a leaf of the hierarchical tree because it is not further partitioned, i.e., it is not the parent node of any other CU. The top right CU is further partitioned into "4" smaller square CUs using quadtree type partitioning again. The bottom right CU is vertically partitioned into "2" rectangular CUs using binary tree type partitioning. The bottom left CU is vertically partitioned into "3" rectangular CUs using ternary tree type partitioning.

[0062] During image encoding, the partitioning is adaptive and each CTU is partitioned to optimize the compression efficiency of the CTU criterion.

[0063] The concepts of prediction unit (PU) and transform unit (TU) appear in HEVC. In fact, in HEVC, the coding entity for prediction (i.e. PU) and the coding entity for transform (i.e. TU) can be subdivisions of CU. For example, Figure 1 As shown in FIG. 1 , a CU of size 2N×2N can be divided into PUs of size N×2N or PUs of size 2N×N 1411. In addition, the CU can be divided into "4" TUs of size N×N 1412 or TUs of size The “16” TUs.

[0064] It can be noted that in VVC, except for some special cases, the boundaries of TU and PU are aligned on the boundaries of CU. Therefore, a CU generally includes one TU and one PU. Some PSW tools allow a CU to be partitioned into multiple PUs. For example, a prediction mode called triangle prediction mode (TPM) allows a CU to be partitioned into two triangle PUs of equal size. Figure 5A Denotes two PUs obtained by partitioning a rectangular CU according to TPM mode. While TPM mode allows partitioning a CU into two triangular PUs of equal size, a second mode called geometric prediction mode (GEO) allows partitioning a CU into two PUs of unequal size. Figure 5B Represents a rectangular CU partitioned into two PUs according to GEO mode.

[0065] In this application, the term "block" or "image block" may be used to refer to any of CTU, CU, PU and TU. In addition, the term "block" or "image block" may be used to refer to macroblocks, partitions and sub-blocks specified in H.264 / AVC or other video coding standards, and more generally to arrays of samples of various sizes.

[0066] In this application, the terms "reconstruction" and "decoding" are used interchangeably, the terms "pixel" and "sample" are used interchangeably, and the terms "image", "picture", "sub-picture", "slice" and "frame" are used interchangeably. Usually, but not necessarily, the term "reconstruction" is used at the encoding end, while "decoding" is used at the decoding end.

[0067] Figure 2 Schematically depicts a method for encoding a video stream performed by an encoding module. Variations of this encoding method are envisioned, but for clarity, the following description Figure 2 encoding method without describing all expected variations.

[0068] Before encoding, the current original image 201 of the video may be pre-encoded ( Figure 2 For example, a color transformation is applied to the current raw image (e.g., conversion from RGB 4:4:4 to YCbCr 4:2:0), or a remapping is applied to the components of the current raw image 201 in order to obtain a signal distribution that is more resilient to compression (e.g., using histogram equalization of one of the color components).

[0069] The encoding of the current raw image 201 begins during step 202 with the partitioning of the current raw image 201, as described with respect to Figure 1 As described. Therefore, the current image 201 is partitioned into CTU, CU, PU, ​​TU, etc. For each block, the encoding module determines a coding mode between intra prediction and inter prediction.

[0070] Intra-prediction consists in predicting the pixels of the current block from a prediction block derived from pixels of a reconstructed block located in the causal vicinity of the current block to be encoded according to an intra-prediction method during step 203. The result of the intra-prediction is a prediction direction indicating which pixels of the nearby blocks are used, and a residual block obtained by calculating the difference between the current block and the prediction block.

[0071] Inter prediction consists in predicting the pixels of the current block from a block of pixels of an image preceding or following the current image (called reference block) (this image is called reference image). During the encoding of the current block according to the inter prediction method, the reference image block closest to the current block is determined by a motion estimation step 204 according to a similarity criterion. During step 204, a motion vector indicating the position of the reference block in the reference image is determined. Said motion vector is used during a motion compensation step 205 during which a residual block is calculated in the form of a difference between the current block and the reference block.

[0072] In the first video compression standards, the above-mentioned unidirectional inter prediction mode was the only inter mode available.With the evolution of video compression standards, the family of inter modes has grown significantly and now includes many different inter modes.

[0073] The first evolution of inter-mode is bidirectional inter prediction (or B-mode). In B-mode, the current block is associated with two motion vectors, specifying two reference blocks in two different images. This allows the calculation of the prediction value block of the residual block of the current block, which is the average of the two reference blocks. The prediction value biPred of the current block is calculated as follows:

[0074] biPred=(w0.pred0+w1.pred1+1) / 2 (1)

[0075] Wherein pred0 is the motion compensation block prediction value obtained in the list L0 of the reference image stored in the buffer of the decoded image (commonly called the decoded picture buffer (DPB)); pred1 is the motion compensation block prediction value obtained in the list L1 of the reference image stored in the DPB; in the case of bidirectional inter-frame prediction, w0 and w1 are weights equal to one.

[0076] Several generalizations of bidirectional inter - frame prediction are proposed, where the weights w0 and w1 can be different. Weighted prediction (WP) can be considered a generalization of bidirectional inter - frame prediction in some aspects. In WP, in the case of unidirectional inter - frame prediction, the residual block is calculated as the difference between the current block and the weighted version of the reference block, or in the case of bidirectional inter - frame prediction, the residual block is calculated as the difference between the current block and the weighted average of two reference blocks. WP can be generalized such that the residual block is calculated as the difference between the current block and the weighted version of the reference block or the weighted average of multiple reference blocks. WP can be enabled at the sequence level in the sequence header (referred to as the Sequence Parameter Set (SPS) in VVC), or at the picture level in the picture header (referred to as the Picture Parameter Set (PPS) in VVC), or at the slice level in the slice header. WP defines the weight w for each CTU group associated with each component of each reference picture i of each reference image list (L0 and L1) stored in the DPB i and the offset off i (e.g., generally at the slice header level). If the current block is encoded with unidirectional WP, the predicted sample pred(x, y) at the position (x, y) of the current block is calculated as follows:

[0077] pred(x, y) = ((w0.pred0(x, y)+(1 << (shift - 1))) >> shift)+Off0

[0078] where pred0(x, y) is the motion - compensated predicted value sample obtained from the reference image list L0 stored in the DPB and spatially corresponding to pred(x, y), w0 is the weight, Off0 is the offset value, shift is the shift value, (y << x) is the left shift of x bits of the binary representation of y, and (y >> x) is the right shift of x bits of the binary representation of y.

[0079] If the current block is encoded with bidirectional WP, the predicted sample pred(x, y) at the position (x, y) of the current block is calculated as follows:

[0080] pred(x, y) = ((w0.pred0(x, y)+w1.pred1(x, y)+Off 01 ) >> (shift + 1)) (2)

[0081] where Off 01 =(Off0 + Off1 + 1) << shift

[0082] where pred1(x, y) is the motion - compensated predicted value sample obtained from the reference image list L1 stored in the DPB and spatially corresponding to pred(x, y), w1 is the weight, and Off1 is the offset value.

[0083] In order to maintain increased numerical precision when weighting samples, intermediate sample values ​​can be stored and manipulated with increased bit depth precision. In this case, at the end of the prediction calculation process, the final (desired) bit depth sample prediction precision (bitDepth) is obtained by the last right bit shift. For example, the reference picture in the DPB is stored with precision bitDepth, but the intermediate motion compensated samples are stored in the intermediate buffer with increased precision (bitDepth+sp). In this case, the right shift value shift in the previous two equations can be replaced by (shift+sp) to ultimately rescale the predicted samples to precision bitDepth. It should be noted that a similar intermediate bit depth precision increase process is typically used for all prediction tools that use the sample weighting process.

[0084] While WP is enabled in the sequence header (SPS) and picture header (PPS) and the associated weights and offsets are specified in the slice header, a new mode called CU-level weighted bi-prediction (BCW) allows weights to be signaled at the block level. When the BCW mode is applied to the current block, the prediction value sample pred(x, y) of the current block is calculated as follows:

[0085] pred(x,y)=((8-w).pred0(x,y)+w.pred1(x,y)+4)>>3

[0086] Where pred0(x,y) is a motion compensated prediction value sample obtained in the list L0 of the reference image stored in the DPB and spatially corresponds to pred(x,y), pred1(x,y) is a motion compensated prediction value sample obtained in the list L1 of the reference image stored in the DPB and spatially corresponds to pred(x,y), and w is a weight obtained in a set of five weights (w∈{-2,3,4,5,10}). The weight w is determined in one of the following two ways: 1) For a non-merged CU, the weight index bcw_idx is signaled after the motion vector difference; 2) For a merged CU, the weight index bcw_idx is inferred from the neighboring block based on the merge candidate index.

[0087] About Figure 5A and Figure 5BThe presented triangle prediction mode (TPM) and geometric prediction mode (GEO) are specific forms of inter-frame mode. As mentioned above, in TPM and GEO modes, the block is divided into two PUs (i.e., two partitions). Information representing the boundary separating the two PUs is encoded for the current block. The information includes a syntax element (merge_triangle_split_dir for TPM mode or wedge_partition_idx for GEO mode) representing the edge direction (i.e., the direction and / or position of the boundary). When using TPM or GEO mode, each PU of the current block uses its own motion information for inter-frame prediction. For each PU, only unidirectional prediction is allowed. Two merge indexes (merge-idx-0 and merge-idx-1) are encoded corresponding to a list of motion vector prediction value candidates with associated reference indexes, which is constructed by spatial and temporal neighbors (based on a conventional merge candidate list) and is constructed alternately by list L0 and list L1. In version 6.0 of the VVC standard described in document JVET-O2001 (hereinafter referred to as JVET-O2001), in the case of TPM, the values ​​of (merge-idx-0 and merge-idx-1) are derived from the syntax elements (merge_triangle_idx0 and merge_triangle_idx1). In the case of GEO, the values ​​of merge-idx-0 and merge-idx-1 are derived from the syntax elements wedge_partition_idx0 and wedge_partition_idx1. After predicting each PU, a blending process with adaptive weights is used to adjust the boundary between two PUs (e.g., Figure 5A The sample value of the boundary between PU 501 and PU 502 in . Figure 5C The adaptive weighting process applied in TPM mode is schematically shown. Figure 5C In FIG. 5 , an 8×8 block 50 encoded in TPM mode includes two triangular partitions 509 and 510. The samples in partition 509, represented in black, are integrally predicted according to the prediction value Pred0. The samples in partition 510, represented in white, are integrally predicted according to the prediction value Pred1. The samples at the boundary between partitions 509 and 510 are the weighted average of the samples predicted according to the prediction values ​​Pred0 and Pred1, with the weights applied in the weighted average depending on the position of the samples relative to the boundary. Figure 5C In the example, samples with identifier equal to "1" are calculated as follows:

[0088]

[0089] Where Pred0(x,y) is the sample at position (x,y) in the predicted value Pred0, Pred1(x,y) is the sample at position (x,y) in the predicted value Pred1, and the adaptive weights and Equal to and

[0090] The sample with identifier equal to "2" is calculated as follows:

[0091]

[0092] in and Equal to and

[0093] The sample with identifier equal to "4" is calculated as follows:

[0094]

[0095] in and Equal to and

[0096] The sample with identifier equal to "6" is calculated as follows:

[0097]

[0098] in and Equal to and

[0099] The sample with identifier equal to "7" is calculated as follows:

[0100]

[0101] in and Equal to and

[0102] During the standardization process of VVC, a new inter-mode was proposed, called Local Illumination Compensation (LIC), which aims to compensate for the illumination changes that may occur between the current block and its reference block in inter-frame prediction. When this tool is activated for the current block, some prediction parameters are calculated based on some reconstructed image samples located to the left and / or top of the current block and reference image samples located to the left and / or top of the reference block identified by the motion vector. The LIC model based on the calculated prediction parameters is then applied to the prediction samples of the reference block. The prediction samples Pred corrected using LIC are corr (x,y) can be calculated as follows:

[0103] Pred corr (x,y)=((a.Pred(x,y)+(1<<(shift-1)))>>shift)+b

[0104] Where Pred corr (x,y) is the sample at position (x,y) corrected by LIC, Pred(x,y) is the motion compensated prediction sample value at position (x,y), shift is the value of the binary shift as stated above for WP, and a and b are LIC parameters.

[0105] LIC mode can be enabled in HLS and signaled at the block level.

[0106] A new tool called Optical Flow (OF) based on the concept of optical flow, which assumes that the movement of objects is smooth, is also proposed to VVC. In the OF tool, motion information for motion compensation can be refined based on the optical flow model. In VVC, the OF tool includes a first mode called Bidirectional Optical Flow (BDOF) mode, a second mode called Optical Flow Prediction Refinement (PROF) mode, and a third mode called Decoder Side Motion Refinement (DMVR). However, there are other OF tools.

[0107] BDOF is used to refine the bidirectional prediction signal of a block at the 4×4 sub-block level. For each 4×4 sub-block of the block, a motion refinement is calculated by minimizing the difference between the samples of the two reference blocks used for bidirectional prediction. The motion refinement is then used to adjust the values ​​of the samples obtained by bidirectional prediction in the 4×4 sub-block. To do this, a new term is added to the traditional bidirectional prediction

[0108]

[0109] Where pred is the predicted value sub-block, I 0 is the first motion compensation reference sub-block, I 1 is the second motion compensation reference sub-block, and are the horizontal gradient and vertical gradient of the i-th reference sub-block (k={0,1}), (v x ,v y ) is the motion vector between the first reference subblock and the second reference subblock, and σ is the time difference between the image including the first reference subblock and the image including the second reference subblock. For more details about the BDOF mode, see Section 3.4.9 of the document JVET-O2002-V1: Algorithm Description of Generic Video Coding and Test Model 6 (VTM 6), Jianle Chen, Yan Ye, and Seung Hwan Kim, 15th Conference: Gothenburg, Sweden, July 3-12, 2019, hereinafter referred to as JVET-O2002.

[0110] In PROF mode, when the parametric motion model is known, the offset correction ΔI can be derived from the optical flow model, as in the case of affine motion compensation. The offset correction ΔI is then added to the prediction samples of the current block I(i,j) to obtain the corrected prediction samples I′(i,j):

[0111] ΔI(i,j)=g x (i,j)*Δv x (i,j)+g y (i,j)*Δv y (i,j)

[0112] I′(i,j)=I(i,j)+ΔI(i,j)

[0113] where g x (i,j) and g y (i, j) are the horizontal local gradient and vertical local gradient at position (i, j) in the current image, respectively, Δv x ,Δv y ) is the difference between the motion vector V(i,j) at position (i,j) calculated using the parametric motion model and the motion vector V of the block to which the sample at position (i,j) belongs calculated using the position-dependent motion model (i.e., the rotation model). B The difference between. Figure 6 An application example of the PROF mode is schematically shown.

[0114] Still in the context of bidirectional inter prediction, DMVR aims to refine the motion vector by testing some displacements around the decoded motion vector. Multiple candidate motion vector pairs are tested. Each pair consists of a motion vector MV specifying the first reference block refBlock_L0 and a motion vector MV specifying the second reference block refBlock_L1 The motion vector pair is selected to minimize the motion vector MV refBlock_L0 The reference block is identified by the motion vector MVrefBlock_L1 Specifies the difference between reference blocks in a pair of motion vectors.

[0115] In VVC, a new tool called combined intra and inter prediction (CIIP) is also proposed. As its name indicates, the CIIP mode combines (mixes) the inter prediction signal with the intra prediction signal. The inter prediction signal in the CIIP mode is derived using the same inter prediction process applied to the conventional merge mode (i.e., the motion information of the current block is derived from the neighboring blocks). The intra prediction signal is derived after the conventional intra prediction process with the planar mode. Then, the weighted average method is used to combine the intra and inter prediction signals, where the weight values ​​are calculated according to the coding modes of the top and left neighboring blocks. More precisely, the CIIP prediction value is obtained using the following formula:

[0116] P CIIP =(w A .P inter +w B .P intra +off)>>shift

[0117] Where P inter is the inter-frame prediction value, P intra is the intra prediction value, off and shift are constant offset and constant shift equal to two, respectively, and w A and w B is a weight that depends on the coding mode of the top and left neighboring blocks.

[0118] In the following, BCW, TPM, GEO, CIIP, LIC modes are considered to belong to the set of PSW tools. BDOF, PROF and DMVR modes are considered to belong to OF tools. It should be noted that the names of the PSW and OF tools listed above are the names currently used in JVET-O2002 or JVET-O2001. These names are only descriptive terms. Therefore, the use of other names for the PSW and OF tools is not excluded.

[0119] During the selection step 206 , the prediction mode that optimizes compression performance is selected by the encoding module among the tested prediction modes (intra prediction mode, inter prediction mode, including for example PSW and OF tools) according to the rate / distortion criterion (ie RDO criterion).

[0120] When a prediction mode is selected, the residual block is transformed during step 207 and quantized during step 209. It is noted that the encoding module may skip the transformation and directly apply quantization to the untransformed residual signal. When the current block is encoded according to the intra-frame prediction mode, the prediction direction and the transformed and quantized residual block are encoded by the entropy encoder during step 210. When the current block is encoded according to the inter-frame prediction, the motion vector of the block is predicted according to a prediction vector selected from a set of motion vectors corresponding to reconstructed blocks located near the block to be encoded, when appropriate. Next, during step 210, the motion information is encoded by the entropy encoder in the form of a motion residual and an index for identifying the prediction vector. During step 210, the transformed and quantized residual block is encoded by the entropy encoder. It is noted that the encoding module may bypass the transformation and quantization, that is, apply entropy coding to the residual without applying the transformation or quantization process. The result of the entropy coding is inserted into the encoded video stream 211. Metadata may be attached to the encoded video stream 211.

[0121] After the quantization step 209, the current block is reconstructed so that the pixels corresponding to the block can be used for future predictions. This reconstruction phase is also called a prediction loop. Therefore, inverse quantization is applied to the transformed and quantized residual block during step 212, and an inverse transform is applied during step 213. According to the prediction mode for the block obtained during step 214, the prediction block of the block is reconstructed. If the current block is encoded according to the inter-frame prediction mode, the encoding module applies motion compensation using the motion vector of the current block during step 216 when appropriate in order to identify a reference block for the current block. If the current block is encoded according to the intra-frame prediction mode, during step 215, the reference block of the current block is reconstructed using the prediction direction corresponding to the current block. The reference block and the reconstructed residual block are added in order to obtain the reconstructed current block.

[0122] After reconstruction, during a step 217, an in-loop post-filtering is applied to the reconstructed blocks, which is intended to reduce coding artifacts. This post-filtering is called in-loop post-filtering because it occurs in the prediction loop to obtain at the encoder the same reference image as at the decoder, thereby avoiding drift between the encoding process and the decoding process. For example, in HEVC, in-loop post-filtering includes deblocking filtering and SAO (Sample Adaptive Offset) filtering. During the entropy coding step 210, parameters representing the activation or deactivation of the in-loop deblocking filter and the characteristics of the in-loop deblocking filter when activated are introduced in the coded video stream 211.

[0123] When a block is reconstructed, it is inserted into a reconstructed image stored in a memory 219 storing reconstructed images of the DPB during a step 218. The reconstructed image thus stored can then be used as a reference image for other images to be coded.

[0124] Figure 3 Schematically depicts the steps performed by the decoding module for Figure 2 The method described herein is a method for decoding the encoded video stream 211 encoded by the method. Variations of this decoding method are contemplated, but for the sake of clarity, the following description is Figure 3 without describing all anticipated variations.

[0125] The decoding is done block by block. For the current block, it starts with entropy decoding of the current block during a step 310. Entropy decoding allows obtaining the prediction mode of the block.

[0126] If the block has been coded according to inter prediction mode, entropy decoding allows obtaining the prediction vector index, the motion residual and the residual block, if appropriate.During a step 308, the motion vector of the current block is reconstructed using the prediction vector index and the motion residual.

[0127] If the block has been coded according to an intra prediction mode, entropy decoding allows obtaining the prediction direction and the residual block. Steps 312, 313, 314, 315, 316 and 317 implemented by the decoding module are identical in all respects to steps 212, 213, 214, 215, 216 and 217, respectively, implemented by the encoding module. In step 318, the decoded block is saved in a decoded image and the decoded image is stored in a reference image memory 319 storing the DPB. When the decoding module decodes a given image, the image stored in the reference image memory 319 is identical to the image stored in the reference image memory 219 by the encoding module during the encoding of said given image. The decoded image may also be output by the decoding module, for example for display.

[0128] The decoded image can be further processed after decoding ( Figure 3 ), for example, an inverse color transform (e.g., a transform from YCbCr 4:2:0 to RGB 4:4:4) or an inverse remapping that performs the inverse of the remapping performed during the pre-encoding process.

[0129] As mentioned above, the selection of the prediction mode of the block is based on the RDO criterion. This selection criterion rarely considers the impact of the selected prediction mode on the encoding and / or decoding complexity. However, it is known that some compression tools can significantly increase the encoding / decoding complexity. For example, the combination of WP with another PSW tool is equivalent to cascading two weighting stages (one weighting stage for WP and the second weighting stage for another PSW tool), which increases the encoding and decoding complexity. In addition, during the weighting process, two intermediate storage buffers should be used to improve the intermediate bit depth accuracy.

[0130] Furthermore, the RDO criterion may consider combinations of prediction tools, but generally never considers the order of the tools in the combination. However, depending on the tools in the combination, some orders may be more desirable than others.

[0131] Figure 4A Schematically illustrates an example of a hardware architecture of a processing module 40 capable of implementing an encoding module or a decoding module modified according to different aspects and embodiments, wherein the encoding module or the decoding module can respectively implement Figure 2 The method used to encode and Figure 3 Method for decoding. As a non-limiting example, the processing module 40 includes the following items connected by a communication bus 405: a processor or CPU (central processing unit) 400 including one or more microprocessors, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture; a random access memory (RAM) 401; a read-only memory (ROM) 402; a storage unit 403, which may include a non-volatile memory and / or a volatile memory, including but not limited to an electrically erasable programmable read-only memory (EEPROM), a read-only memory (ROM), a programmable read-only memory (PROM), a random access memory (RAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), a flash memory, a disk drive and / or an optical drive, or a storage medium reader, such as an SD (Secure Digital) card reader and / or a hard disk drive (HDD) and / or a network accessible storage device; at least one communication interface 404, the at least one communication interface is used to exchange data with other modules, devices or equipment. The communication interface 404 may include but is not limited to a transceiver configured to transmit and receive data through a communication channel. The communication interface 404 may include, but is not limited to, a modem or a network card.

[0132] If the processing module 40 implements a decoding module, the communication interface 404 enables, for example, the processing module 40 to receive an encoded video stream and provide a decoded video stream. If the processing module 40 implements an encoding module, the communication interface 404 enables, for example, the processing module 40 to receive raw image data to be encoded and provide an encoded video stream.

[0133] The processor 400 is capable of executing instructions loaded into the RAM 401 from the ROM 402, an external memory (not shown), a storage medium or a communication network. When the processing module 40 is powered on, the processor 400 is capable of reading instructions from the RAM 401 and executing them. These instructions form a computer program that enables, for example, the processor 400 to implement the Figure 3 Describes the decoding method or Figure 2 The encoding method described, the decoding and encoding methods include the following various aspects and embodiments described in this document.

[0134] All or part of the algorithms and steps of the encoding or decoding method may be implemented in software form by executing a set of instructions by a programmable machine such as a DSP (digital signal processor) or a microcontroller, or may be implemented in hardware form by a machine or dedicated components such as an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).

[0135] Figure 4B A block diagram of an example of a system 4 in which various aspects and embodiments are implemented is shown. System 4 may be embodied as a device including various components described below, and is configured to perform one or more aspects and embodiments described in this document. Examples of such devices include, but are not limited to, various electronic devices, such as personal computers, laptop computers, smart phones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. The elements of system 4 may be embodied in a single integrated circuit (IC), multiple ICs, and / or discrete components, either individually or in combination. For example, in at least one embodiment, system 4 includes a processing module 40 that implements a decoding module or an encoding module. However, in another embodiment, system 4 may include a first processing module 40 that implements a decoding module and a second processing module 40 that implements an encoding module, or a processing module 40 that implements a decoding module and an encoding module. In various embodiments, system 40 is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or through a dedicated input and / or output port. In various embodiments, system 4 is configured to implement one or more aspects described in this document.

[0136] The system 4 includes at least one processing module 40 that can implement one or both of an encoding module or a decoding module.

[0137] Inputs to processing module 40 may be provided through various input modules as shown in block 42. Such input modules include, but are not limited to: (i) a radio frequency (RF) module that receives an RF signal transmitted over the air, for example, by a broadcaster; (ii) a component (COMP) input module (or a group of COMP input modules); (iii) a universal serial bus (USB) input module; and / or (iv) a high-definition multimedia interface (HDMI) input module. Figure 4B Other examples not shown include composite video.

[0138] In various embodiments, the input module of block 42 has associated corresponding input processing elements as known in the art. For example, the RF module may be associated with elements suitable for: (i) selecting a desired frequency (also referred to as selecting a signal, or band limiting a signal to a frequency band), (ii) down-converting the selected signal, (iii) again band-limiting to a narrower frequency band to select a signal band that may be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired packet stream. The RF module of various embodiments includes one or more elements for performing these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF portion may include a tuner that performs various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or near baseband frequency) or to baseband. In a set-top box embodiment, RF module and its associated input processing element receive the RF signal transmitted by wired (for example, cable) medium, and perform frequency selection by filtering, down-conversion and filtering to required frequency band again.Various embodiments rearrange the order of above-mentioned (and other) elements, remove some elements in these elements, and / or add other elements of similar or different functions.Adding element can include inserting element between existing element, for example, inserting amplifier and analog-to-digital converter.In various embodiments, RF module comprises antenna.

[0139] In addition, the USB and / or HDMI modules may include corresponding interface processors for connecting the system 4 to other electronic devices across the USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented, for example, within a separate input processing IC or within the processing module 40 as desired. Similarly, various aspects of USB or HDMI interface processing may be implemented, for example, within a separate interface IC or within the processing module 40 as desired. The demodulated, error corrected, and demultiplexed stream is provided to the processing module 40.

[0140] The various elements of the system 4 may be disposed within an integrated housing. Within the integrated housing, the various elements may be interconnected and data may be transmitted between these elements using a suitable connection arrangement (e.g., an internal bus known in the art, including an inter-IC (I2C) bus, wiring, and a printed circuit board). For example, in the system 4, the processing module 40 is interconnected with the other elements of the system 4 via a bus 405.

[0141] The communication interface 404 of the processing module 40 allows the system 4 to communicate over the communication channel 41. For example, the communication channel 41 may be implemented in a wired and / or wireless medium.

[0142] In various embodiments, a wireless network such as a Wi-Fi network, for example IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers) is used to transmit data streams or otherwise provide it to the system 4. The Wi-Fi signals of these embodiments are received by a communication channel 41 and a communication interface 404 suitable for Wi-Fi communication. The communication channel 41 of these embodiments is usually connected to an access point or a router, which provides access to an external network including the Internet, for allowing streaming applications and other cloud communications. Other embodiments use a set-top box to provide streaming data to the system 4, which delivers data through the HDMI connection of the input block 42. Still other embodiments use the RF connection of the input block 42 to provide streaming data to the system 4. As described above, various embodiments provide data in a non-streaming manner. In addition, various embodiments use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0143] System 4 can provide output signals to various output devices, including display 46, speaker 47 and other peripheral devices 48. Display 46 of various embodiments includes one or more of, for example, a touch screen display, an organic light emitting diode (OLED) display, a curved display and / or a foldable display. Display 46 can be used for a television, a tablet computer, a laptop computer, a cellular phone (mobile phone) or other devices. Display 46 can also be integrated with other components (for example, as in a smart phone), or it can be separate (for example, an external monitor of a laptop computer). In various examples of embodiments, other peripheral devices 46 include one or more of an independent digital video disc (or digital versatile disc, both terms are DVR), a disc player, a stereo system and / or a lighting system. Various embodiments use one or more peripheral devices 46 that provide functions based on the output of system 4. For example, a disc player performs the function of playing the output of system 4.

[0144] In various embodiments, control signals are transmitted between system 4 and display 46, speaker 47, or other peripheral device 48 using signaling such as AV.Link, consumer electronics control (CEC), or other communication protocols that enable device-to-device control with or without user intervention. Output devices may be communicatively coupled to system 4 via dedicated connections through respective interfaces 43, 44, and 45. Alternatively, output devices may be connected to system 4 via communication interface 404 using communication channel 41. Display 46 and speaker 47 may be integrated into a single unit with other components of system 4 in an electronic device such as a television. In various embodiments, display interface 43 includes a display driver, such as, for example, a timing controller (T Con) chip.

[0145] For example, if the RF portion of input 42 is part of a separate set-top box, the display 46 and speaker 47 may alternatively be separate from one or more of the other components. In various embodiments where the display 46 and speaker 47 are external components, the output signals may be provided via dedicated output connections including, for example, an HDMI port, a USB port, or a COMP output.

[0146] Various implementations participate in decoding. As used in this application, "decoding" may encompass, for example, all or part of a process performed on a received encoded video stream to produce a final output suitable for display. In various embodiments, such processes include one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and prediction. In various embodiments, such processes also include or alternatively include processes performed by decoders of various implementations described in this application, such as for determining whether certain PSWs and OF tools are compatible for a CU.

[0147] As a further example, in one embodiment, "decoding" refers only to entropy decoding ( Figure 3 Whether the phrase "decoding process" refers specifically to a subset of operations or broadly to a broader decoding process will be clear based on the context of the specific description and is believed to be well understood by those skilled in the art.

[0148] Various specific implementations participate in encoding. In a manner similar to the discussion above about "decoding", "encoding" as used in this application may cover, for example, all or part of the process performed on an input video sequence to produce an encoded video stream. In various embodiments, such processes include one or more processes typically performed by an encoder, such as partitioning, prediction, transformation, quantization, and entropy coding. In various embodiments, such processes also include or alternatively include processes performed by encoders of various specific implementations described in this application, such as for determining whether two or more PSWs and / or OF tools are compatible for a CU.

[0149] As a further example, in one embodiment, "encoding" refers to encoding mode selection ( Figure 2 Step 206 in ) and entropy coding ( Figure 2 Whether the phrase "encoding process" refers specifically to a subset of operations or broadly to a broader encoding process will be clear based on the context of the specific description and is believed to be well understood by those skilled in the art.

[0150] It should be noted that the syntax element names used in this article are descriptive terms. Therefore, they do not exclude the use of other syntax element names.

[0151] When the figures are presented as flow charts, it should be understood that they also provide block diagrams of corresponding devices. Similarly, when the figures are presented as block diagrams, it should be understood that they also provide flow charts of corresponding methods / processes.

[0152] Various embodiments refer to rate-distortion optimization. Specifically, during the encoding process, the balance or compromise between rate and distortion is usually considered. Rate-distortion optimization is usually expressed as minimizing a rate-distortion function, which is a weighted sum of rate and distortion. There are different methods to solve the rate-distortion optimization problem. For example, these methods can be based on extensive testing of all coding options (including all considered modes or coding parameter values), and fully evaluate their coding costs and the associated distortion of the reconstructed signal after encoding and decoding. Faster methods can also be used to reduce coding complexity, especially for the calculation of approximate distortion based on prediction or prediction residual signals rather than reconstructed residual signals. A mixture of these two methods can also be used, such as by using approximate distortion for only some of the possible coding options, and using complete distortion for other coding options. Other methods only evaluate a subset of possible coding options. More generally, many methods use any of various techniques to perform optimization, but optimization is not necessarily a complete evaluation of both coding costs and associated distortions.

[0153] Implementations and aspects described herein may be implemented, for example, in a method or process, an apparatus, a software program, a data stream, or a signal. Even if discussed only in the context of a single form of implementation (e.g., discussed only as a method), the implementation of features discussed may also be implemented in other forms (e.g., an apparatus or program). An apparatus may be implemented, for example, in appropriate hardware, software, and firmware. A method may be implemented, for example, in a processor, which generally refers to a processing device,

[0154] The processing device includes, for example, a computer, a microprocessor, an integrated circuit or a programmable logic device. The processor also includes a communication device, such as, for example, a computer, a cell phone, a portable / personal digital assistant ("PDA"), and other devices that facilitate information communication between end users.

[0155] Reference to "one embodiment" or "an embodiment" or "an implementation" or "an implementation" and other variations thereof means that a particular feature, structure, characteristic, etc. described in connection with the embodiment is included in at least one embodiment. Thus, the appearance of the phrase "in one embodiment" or "in an embodiment" or "in one implementation" or "in an implementation" and any other variations thereof in various places throughout this application are not necessarily all referring to the same embodiment.

[0156] Additionally, the present application may refer to “determining” various information. Determining information may include, for example, estimating information, calculating information, predicting information, retrieving information from a memory, or obtaining information, for example, from another device, module, or from a user, for example, one or more of the following.

[0157] Furthermore, the present application may refer to "accessing" various information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from a memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0158] Additionally, the present application may refer to "receiving" various information. Like "accessing," receiving is intended to be a broad term. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from a memory). Furthermore, "receiving" generally involves, in one way or another, during operations such as, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0159] It should be understood that, for example, in the case of “A / B,” “A and / or B,” and “at least one of A and B,” “one or more of A and B,” use of any of the following “ / ,” “and / or,” and “at least one,” “one or more” is intended to encompass selection of only the first listed option (A), or only the second listed option (B), or both options (A and B). As a further example, in the case of “A, B, and / or C,” and “at least one of A, B, and C,” “one or more of A, B, and C,” such phrases are intended to encompass selection of only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first listed option and the second listed option (A and B), or only the first listed option and the third listed option (A and C), or only the second listed option and the third listed option (B and C), or all three options (A and B and C). As will be apparent to one of ordinary skill in this and related arts, this may be extended to as many items as listed.

[0160] Moreover, as used herein, the word "signaling" refers to (among other things) indicating something to the corresponding decoder. For example, in certain embodiments, the encoder uses PSW (or OF) tool parameters to signal the PSW (or OF) tool. Like this, in one embodiment, the same parameters are used on both the encoder side and the decoder side. Therefore, for example, the encoder can transmit (explicit signaling) specific parameters to the decoder so that the decoder can use the same specific parameters. On the contrary, if the decoder already has specific parameters and others, signaling can be used without transmitting (implicit signaling) to simply allow the decoder to know and select specific parameters. By avoiding transmitting any actual function, bit saving is achieved in various embodiments. It should be understood that signaling can be implemented in various ways. For example, in various embodiments, one or more grammatical elements, tags, etc. are used to signal information to the corresponding decoder. Although the verb form of the word "signal (signaling)" is related to the above, the word "signal (signal)" can also be used as a noun herein.

[0161] It will be apparent to one of ordinary skill in the art that a specific implementation may generate various signals formatted to carry, for example, storable or transmittable information. The information may include, for example, instructions for performing a method or data generated by one of the specific implementations. For example, a signal may be formatted to carry a coded video stream of the embodiment. Such a signal may be formatted, for example, as an electromagnetic wave (e.g., using a radio frequency portion of a spectrum) or a baseband signal. Formatting may include, for example, encoding a coded video stream and modulating a carrier using the coded video stream. The information carried by the signal may be, for example, analog or digital information. It is known that the signal may be transmitted over a variety of different wired or wireless links. The signal may be stored on a processor readable medium.

[0162] Figure 7 is Figure 2 A high-level representation of an implementation of the reconstruction process performed during the encoding process.

[0163] Mainly during steps 214 and 216, for each block of the image Figure 7 process.

[0164] In step 70 , the processing module 40 determines whether WP is enabled for the current block.

[0165] If WP is enabled, then step 71 is entered after step 70. The processing module 40 modifies a part of the encoding process of the current block related to WP and / or at least one PSW or OF tool.

[0166] If WP is disabled, step 70 is followed by step 72. During step 72, the usual encoding process as specified in document JVET-02001 is applied to the current block.

[0167] In a first implementation of step 70, the processing module 40 determines the value of a first variable PPS_weighted_pred_flag representing a syntax element pps_weighted_pred_flag and / or a second variable PPS_weighted_bipred_flag representing a syntax element pps_weighted_bipred_flag. As defined in JVET-02001, the syntax element pps_weighted_pred_flag (or pps_weighted_bipred_flag) indicates, at the level of a picture header (PPS), whether unidirectional weighted prediction (or bidirectional weighted prediction) is enabled for a P slice (or B slice) referring to the PPS. If one of the variables PPS_weighted_pred_flag or PPS_weighted_bipred_flag is equal to one (i.e., unidirectional weighted prediction or bidirectional weighted prediction is enabled) in the PPS referred to by the slice including the current block, the processing module 40 considers that WP is enabled for the current block. Otherwise, the processing module 40 considers that WP is disabled for the current block.

[0168] In some cases, WP may be enabled at the PPS level, but the weights for applying WP may be equal to the default weights (i.e., each weight for WP is equal to one), which is equivalent to transforming WP into a simple unidirectional or bidirectional inter prediction. In VVC-based implementations, when reference picture i is associated with the default weights, the syntax elements luma_weight_l0_flag[i] and chroma_weight_l0_flag[i] are equal to zero.

[0169] In a second implementation of step 70, in addition to checking the values ​​of variables PPS_weighted_pred_flag and PPS_weighted_bipred_flag, processing module 40 also checks the values ​​of weights for WP. When enabled by syntax elements pps_weighted_pred_flag and pps_weighted_bipred_flag, the weights for WP prediction are encoded in the slice header (as indicated in Sections 7.3.6.1 and 7.3.6.2 of JVET-O2001). The slice header includes information indicating the weight values ​​for each reference image used to predict a block of the current slice. In a second implementation, if at least one weight value indicated in the slice header of the slice including the current block is different from a default value, processing module 40 considers WP to be enabled for the current block. Otherwise, processing module 40 considers WP to be disabled for the current block and that WP is a simple unidirectional or bidirectional inter-frame prediction.

[0170] When the first and second embodiments of step 70 are applied, in the first embodiment of step 71, the processing module 40 modifies a portion of the encoding process of the current block by preventing the use of the PSW or OF tool during the encoding process. In other words, if a WP is selected for the current block, the PSW and the OF tool are not considered as prediction tools that can be used for the current block, and information related to the PSW or the OF tool is not encoded in the encoded video stream 211. In other words, the bitstream portion of the encoded video stream 211 representing the current block does not include syntax elements related to the PSW or the OF tool.

[0171] Using the information included in the PPS and the slice header, the processing module 40 is able to determine whether a WP with a non-default weight can be applied to a block included in the same slice as the current block. Specifically, each information representing a weight included in the slice header indicates that when providing a prediction value for inter-frame prediction of a block of the slice, the reference image associated with the weight is weighted by the weight. The information included in the slice header does not allow determination of which block a WP with a non-default weight is actually applied to. In order to determine whether a WP with a non-default weight is applied to the current block, it is necessary to identify which reference image provides a prediction value to the current block. In a third embodiment of step 70, after having checked the values ​​of the variables PPS_weighted_pred_flag and PPS_weighted_bipred_flag and the weights used by the WP, the processing module 40 determines which reference image provides a prediction value to the current block. If at least one reference image that provides a prediction value to the current block is associated with a non-default weight, the processing module 40 determines that weighted prediction is enabled for the current block. Otherwise, the processing module 40 considers that WP is disabled for the current block.

[0172] When the third implementation of step 70 is applied, in the second implementation of step 71, the processing module 40 modifies a part of the reconstruction process of the current block by not applying the PSW or OF tool during the reconstruction process, even if the PSW or OF tool was selected during step 206 and syntax elements corresponding to the PSW or OF tool are encoded in the encoded video stream 211. However, in this case, the conventional WP reconstruction process is applied.

[0173] When the third implementation of step 70 is applied, in the third implementation of step 71, deactivating the PSW or OF tool includes not applying a conventional reconstruction process for the PSW or OF tool, even if a syntax element corresponding to the PSW or OF tool is encoded in the encoded video stream 211. In this implementation, the processing module 40 applies an alternative inter-frame prediction tool to the current block instead of the PSW or OF tool specified in the encoded video stream 211. However, in the third implementation of step 71, the conventional WP reconstruction process is applied.

[0174] In a first example of the third implementation of step 71 (hereinafter referred to as example (1)), WP and TPM modes are enabled for the current block.

[0175] In a variant (1a) of example (1), during step 71, the TPM reconstruction process is replaced by a bidirectional inter-frame prediction (B-mode) reconstruction process as described above. Applying equation (1), the prediction value bipred of the current block with w0=w1=1 is determined for all prediction value samples. The prediction values ​​pred0 and pred1 are generated using the motion information associated with the two partitions of the TPM mode.

[0176] In a variation (1b) of example (1), the TPM reconstruction process is replaced by a bidirectional WP reconstruction process as described above. Equation (2) is applied to determine the prediction value bipred of the current block using the WP weights w0 and w1 associated with the reference indexes of the prediction values ​​pred0 and pred1 and the offset values ​​Off0 and Off1 for all samples. The prediction values ​​pred0 and pred1 are generated using the motion information associated with the two partitions of the TPM mode.

[0177] In variation (1c) of example (1), the syntax element (merge_triangle_split_dir) indicating the edge direction is reinterpreted to indicate whether variation (1a) or (1b) should be applied.

[0178] In a variation (1d) of example (1), the TPM reconstruction process is replaced by a process in which the initial CU is divided into two vertical rectangular PUs or two horizontal rectangular PUs, one of which is predicted using pred0 and the other is predicted using pred1, such as Fig.10 Depicted in . The prediction values ​​pred0 and pred1 are generated using the motion information associated with the two partitions of the TPM pattern. In an implementation of variant (1d), the partitioning into two rectangular PUs generates two rectangular PUs of unequal sizes. Fig.11 An example of a CU partitioned into rectangular PUs of unequal sizes is described.

[0179] In variant (1e) of variant (1d), the syntax element (merge_triangle_split_dir) indicating the edge direction is reinterpreted to indicate whether the partition is vertical or horizontal.

[0180] Fig.12 The parsing process of variant (1f) corresponding to example (1) is schematically represented. In variant (1f), a new syntax element biReg is introduced (eg, coded on one bit).

[0181] In step 1201 , the processing module 40 determines whether WP is enabled at the HLS (High Level Syntax) level (eg, in PPS) for the current block.

[0182] If WP is enabled at the HLS level, step 1204 is entered after step 1201. During step 1204, the processing module 40 determines the value of the syntax element biReg by decoding the syntax element. If biReg is equal to a first value (for example, if biReg=0), the processing module 40 determines to apply the reconstruction process of variant (1a) to the current block. If biReg is equal to a second value (for example, if biReg=1), the processing module 40 determines to apply the reconstruction process of variant (1b) to the current block.

[0183] If WP is not activated at the HLS level for the current block, then after step 1201, step 1202 and step 1203 are entered. During step 1202, the processing module 40 determines the value of the syntax element merge_triangle_split_dir indicating the edge direction of the TPM mode by decoding the syntax element. In step 1203, the processing module determines the value of the syntax element biReg by decoding the syntax element.

[0184] In a first implementation of variant (1f), if biReg is equal to a first value (e.g., if biReg=0) at step 1203, the processing module 40 determines to apply the reconstruction process of variant (1a) to the current block. If biReg is equal to a second value (e.g., if biReg=1) at step 1203, the processing module 40 determines to apply the reconstruction process of TPM mode to the current block.

[0185] In a second implementation of variant (1f), if biReg is equal to a first value (e.g., if biReg=0) at step 1203, the processing module 40 determines to apply a reconstruction process of the TPM mode to the current block. If biReg is equal to a second value (e.g., if biReg=1) at step 1203, the processing module 40 determines to apply a reconstruction process of variant (1e) to the current block. In other words, if biReg=1, the current block (i.e., the current CU) is partitioned into two rectangular PUs, and the syntax element merge_triangle_split_dir is used to determine the edge direction.

[0186] Fig.13 The reconstruction process corresponding to variant (1g) of example (1) is schematically represented.

[0187] In step 1301 , the processing module 40 determines whether a WP with a non-default weight is activated for the current block.

[0188] If WP is disabled or enabled using the default weight of the current block, step 1301 is followed by step 1302. During step 1302, the processing module applies a TPM reconstruction process to the current block.

[0189] If WP with non-default weight is enabled for the current block, step 1303 is performed after step 1301. In an implementation of step 1303, the processing module 40 determines whether the reference images including the prediction values ​​pred0 and pred1 specified by the motion information associated with the two partitions of the TPM mode have the same POC (picture order count) (i.e., have the same picture number in display order). If the two reference images have the same POC, step 1304 is performed after step 1303.

[0190] In an embodiment of step 1304 , the processing module 40 applies the reconstruction process of variant ( 1 a ) during step 1304 .

[0191] In an embodiment of step 1304 , the processing module 40 applies the reconstruction process of variant ( 1 b ) during step 1304 .

[0192] In the implementation of step 1304, as in variant (1c), the processing module 40 uses the syntax element merge_triangle_split_dir to determine which variant between variants (1a) and (1b) to apply.

[0193] In the implementation of step 1304, the processing module 40 applies the TPM reconstruction process without applying the WP reconstruction process, even if WP has been enabled in this case.

[0194] If, at step 1303, the processing module 40 determines that the two reference images have different POC values, then step 1302 is performed after step 1303.

[0195] In another implementation of step 1303, the processing module 40 determines whether the POC value PIC1 of one of the reference images in the reference images is less than the POC value PIC of the image including the current block curr , and whether the POC value PIC2 of the other reference image is greater than the POC value PIC curr . If (POC1 < POC curr < POC2), then the processing module 40 performs step 1302. Otherwise, the processing module 40 performs step 1304. In this implementation of step 1303, all implementations of step 1304 can be applied.

[0196] In another implementation of step 1303, the processing module 40 determines whether the two reference images have the same POC and whether the WP weights are the same. If the two reference images have the same POC and the WP weights are the same, then step 1304 is performed after step 1303. Otherwise, step 1302 is performed after step 1303. In this implementation of step 1303, all implementations of step 1304 can be applied.

[0197] In the second example of the third implementation of step 71 (hereinafter referred to as example (2)), WP and GEO modes are enabled for the current block.

[0198] In the first variant (2a) of example (2), during step 71, the GEO reconstruction process is replaced with the bidirectional inter prediction reconstruction process as described above. Equation (1) is applied to determine the predicted value bipred of the current block where w0 = w1 = 1. The motion information associated with the two partitions of the GEO mode is used to generate the predicted values pred0 and pred1. It can be seen that variant (2a) is derived from variant (1a).

[0199] Similarly, variants (2b, 2c, 2d, 2e, 2f, and 2g) can be derived from variants (1b, 1c, 1d, 1e, 1f, and 1g).

[0200] In a variant (2b) of example (2) derived from variant (1b), the GEO reconstruction process is replaced by a conventional bidirectional WP inter-prediction (B-mode) reconstruction process.

[0201] In variant (2c) of example (2) derived from variant (1c), the syntax element wedge_partition_idx indicating the edge direction is reinterpreted to indicate whether variant (2a) or (2b) should be applied. However, while the syntax element merge_triangle_split_dir is a binary syntax element capable of indicating two edge directions, the syntax element wedge_partition_idx has a larger granularity. In some implementations, the syntax element wedge_partition_idx may indicate "82" different edge directions. In one implementation, half of the possible values ​​of wedge_partition_idx (e.g., values ​​"0" to "40") indicate that variant (2a) is applied, while the other half of the possible values ​​of wedge_partition_idx (e.g., values ​​"41" to "81") indicate that variant (2b) is applied.

[0202] In variant (2d) of example (2) derived from variant (1d), the GEO reconstruction process is replaced with reconstruction of two vertical or horizontal PUs as described in variant (1d). In variant (2e), the partition direction (vertical or horizontal) is determined using the syntax element wedge_partition_idx indicating the edge direction. In one embodiment, half of the possible values ​​of wedge_partition_idx (e.g., values ​​"0" to "40") indicate that variant (2a) is applied, while the other half of the possible values ​​of wedge_partition_idx (e.g., values ​​"41" to "81") indicate that variant (2b) is applied.

[0203] In variant (2f), a new syntax element biReg is used.

[0204] In step 1201 , the processing module 40 determines whether WP is enabled at the HLS (High Level Syntax) level (eg, in PPS) for the current block.

[0205] If WP is enabled at the HLS level, step 1204 is entered after step 1201. During step 1204, the processing module 40 determines the value of the syntax element biReg by decoding the syntax element. If biReg is equal to a first value (for example, if biReg=0), the processing module 40 determines to apply the reconstruction process of variant (2a) to the current block. If biReg is equal to a second value (for example, if biReg=1), the processing module 40 determines to apply the reconstruction process of variant (2b) to the current block.

[0206] If WP is not activated at the HLS level for the current block, then after step 1201, step 1202 and step 1203 are entered. During step 1202, the processing module 40 determines the value of the syntax element wedge_partition_idx indicating the edge direction of the GEO mode by decoding the syntax element. In step 1203, the processing module determines the value of the syntax element biReg by decoding the syntax element.

[0207] In a first implementation of variant (2f), if biReg is equal to a first value (e.g., if biReg=0) at step 1203, the processing module 40 determines to apply the reconstruction process of variant (1a) to the current block. If biReg is equal to a second value (e.g., if biReg=1) at step 1203, the processing module 40 determines to apply the reconstruction process of the GEO mode to the current block.

[0208] In a second implementation of variant (2f), if biReg is equal to a first value (e.g., if biReg=0) at step 1203, the processing module 40 determines to apply the reconstruction process of the GEO mode to the current block. If biReg is equal to a second value (e.g., if biReg=1) at step 1203, the processing module 40 applies the reconstruction process of variant (2e) to the current block. In other words, if biReg=1, the current block (i.e., the current CU) is partitioned into two rectangular PUs, and the syntax element wedge_partition_idx is used to determine the edge direction.

[0209] In a third implementation of variant (2f), if biReg is equal to the first value (e.g., if biReg=0) at step 1203, the processing module 40 uses the syntax element wedge_partition_idx to determine the partition of the current block. For example, a value of wedge_partition_idx between "0" and "40" indicates a horizontal partition of the current block:

[0210] Values ​​"0" to "7" correspond to partitions with two PUs of equal size, e.g. Fig.10The left side indicates;

[0211] Values ​​"8" to "15" correspond to a partition with a first PU of height equal to H / 4 (where H is the height of the current block) and a second PU of height equal to 3×H / 4 ( Fig.11 )

[0212] Values ​​"16" to "23" correspond to partitions with a first PU of height equal to 3×H / 4 and a second PU of height equal to H / 4 ( Fig.11 )

[0213] Values ​​"24" to "31" correspond to a partition with a first PU having a height equal to H / 3 and a second PU having a height equal to 2×H / 3;

[0214] • Values ​​"32" to "40" correspond to partitions with a first PU having a height equal to 2×H / 3 and a second PU having a height equal to H / 3.

[0215] A value between "41" and "82" of wedge_partition_idx indicates a vertical partition of the current block:

[0216] Values ​​"41" to "48" correspond to partitions with two PUs of equal size, such as Fig.10 As indicated on the right;

[0217] Values ​​"49" to "56" correspond to a partition with a first PU of height equal to W / 4 (where W is the width of the current block) and a second PU of width equal to 3×W / 4 ( Fig.11 );

[0218] Values ​​"57" to "64" correspond to a partition with a first PU of width equal to 3×W / 4 and a second PU of width equal to W / 4 ( Fig.11 )

[0219] Values ​​"65" to "72" correspond to a partition with a first PU having a width equal to W / 3 and a second PU having a width equal to 2×W / 3;

[0220] • Values ​​"73" to "81" correspond to partitions with a first PU having a width equal to 2×W / 3 and a second PU having a width equal to W / 3.

[0221] If biReg is equal to the second value (eg, if biReg=1) at step 1203, the processing module 40 determines to apply the reconstruction process of the GEO mode to the current block. Each PU is then predicted using the motion information of the GEO mode.

[0222] In variant (2g), in step 1301, the processing module 40 determines whether a WP with a non-default weight is activated for the current block.

[0223] If WP is disabled or enabled using the default weight of the current block, step 1301 is followed by step 1302. During step 1302, the processing module applies a GEO mode reconstruction process to the current block.

[0224] If WP with non-default weight is enabled for the current block, step 1303 is performed after step 1301. In an implementation of step 1303, the processing module 40 determines whether the reference images including the prediction values ​​pred0 and pred1 specified by the motion information associated with the two partitions of the TPM mode have the same POC (picture order count) (i.e., have the same picture number in display order). If the two reference images have the same POC, step 1304 is performed after step 1303.

[0225] In an embodiment of step 1304 , the processing module 40 applies the reconstruction process of variant ( 2 a ) during step 1304 .

[0226] In an embodiment of step 1304 , the processing module 40 applies the reconstruction process of variant ( 2b ) during step 1304 .

[0227] In an implementation of step 1304 , as in variant ( 2c ), processing module 40 uses the syntax element wedge_partition_idx to determine which variant between variants ( 2a ) and ( 2b ) to apply.

[0228] In an implementation of step 1304, processing module 40 uses the syntax element wedge_partition_idx to determine the partition of the current block. For example, a value of wedge_partition_idx between "0" and "40" indicates a horizontal partition of the current block:

[0229] Values ​​"0" to "7" correspond to partitions with two PUs of equal size, e.g. Fig.10 The left side indicates;

[0230] Values ​​"8" to "15" correspond to a partition with a first PU of height equal to H / 4 (where H is the height of the current block) and a second PU of height equal to 3×H / 4 ( Fig.11 )

[0231] Values ​​"16" to "23" correspond to partitions with a first PU of height equal to 3×H / 4 and a second PU of height equal to H / 4 ( Fig.11 )

[0232] · Values from "24" to "31" correspond to a partition with a first PU of height equal to H / 3 and a second PU of height equal to 2×H / 3;

[0233] · Values from "32" to "40" correspond to a partition with a first PU of height equal to 2×H / 3 and a second PU of height equal to H / 3.

[0234] Values of wedge_partition_idx between "41" and "82" indicate the vertical partition of the current block:

[0235] · Values from "41" to "48" correspond to a partition with two PUs of equal size, as Fig.10 shown on the right;

[0236] · Values from "49" to "56" correspond to a partition with a first PU of height equal to W / 4 (where W is the width of the current block) and a second PU of width equal to 3×W / 4 ( Fig.11 upper right figure);

[0237] · Values from "57" to "64" correspond to a partition with a first PU of width equal to 3×W / 4 and a second PU of width equal to W / 4 ( Fig.11 lower right figure);

[0238] · Values from "65" to "72" correspond to a partition with a first PU of width equal to W / 3 and a second PU of width equal to 2×W / 3;

[0239] · Values from "73" to "81" correspond to a partition with a first PU of width equal to 2×W / 3 and a second PU of width equal to W / 3.

[0240] If, at step 1303, the processing module 40 determines that the two reference images have different POC values, then step 1302 is performed after step 1303.

[0241] In another implementation of step 1303, the processing module 40 determines whether the POC value POC1 of one of the reference images in the reference images is less than the POC value POC of the image including the current block curr , and whether the POC value POC2 of the other reference image is greater than the POC value POC curr . If (POC1 < POC curr < POC2), then the processing module 40 performs step 1302. Otherwise, the processing module applies to perform step 1304.

[0242] In the variants (1a, 1b) of example (1) and the variants (2a, 2b) of example (2) of the third implementation of step 71, the syntax element indicating the edge direction (merge_triangle_split_dir in TPM mode or wedge_partition_idx in GEO mode) becomes useless. In another variant of the first example and the second example of the third implementation of step 71, the two syntax elements are reinterpreted. In other words, in this variant, the semantics of the two syntax elements are changed. For example:

[0243] merge-idx0=merge_triangle_idx0;

[0244] delta_idx=merge_triangle_split_dir; and

[0245] merge-idx1=delta-idx+merge_triangle_idx1.

[0246] In a third example of the third implementation of step 71, WP and GEO modes are enabled for the current block. In this case, during step 71, the GEO reconstruction process is replaced by a general bidirectional inter-frame prediction reconstruction process. Applying equation (1), the prediction value bipred of the current block is determined, where the weight depends on the number of samples included in each partition. For example, referring to Figure 5B , w0 is proportional to the number of samples in partition 505, and w1 is proportional to the number of samples in partition 506. If the block including partitions 505 and 506 is a 16×8 block including "128" samples, partition 505 includes NB1 samples and partition 506 includes NB2 samples, for example w0=NB1 / 128 and w1=NB2 / 128. The prediction values ​​pred0 and pred1 are generated using the motion information associated with the two partitions of the GEO mode. In Figure 5B In FIG. 5 , the motion information of partition 505 is used to generate the prediction value pred0, while the motion information of partition 506 is used to generate the prediction value pred1.

[0247] In a third exemplary variation of the third embodiment of step 71, weights w0 and w1 are selected from a set of weights authorized for BCW, i.e., ∈{-2,3,4,5,10}. For example, a first partition that includes more samples than a second partition is associated with a higher set of weights {-2,3,4,5,10} than the second partition. For example, again referring to Figure 5B, the weight w0 associated with partition 505 is equal to "3", while the weight w1 associated with partition 506 is equal to "5." Likewise, the motion information of partition 505 is used to generate the prediction value pred0, while the motion information of partition 506 is used to generate the prediction value pred1.

[0248] In a fourth example of the third implementation of step 71, WP and TPM or GEO mode are enabled for the current block. In this case, a modified TPM or GEO mode is applied to the current block. In the modified TPM or GEO mode, a partitioning of the current block into two partitions (two PUs) specified by the TPM or GEO mode is applied. A prediction value associated with each partition is identified and motion compensated using motion information corresponding to the partition. In a first variation of the fourth example of the third implementation, after predicting each partition, no motion is applied along a boundary between two partitions (e.g., Figure 5C In a second variation of the fourth example of the third embodiment, after predicting each partition, a blending process is used to adjust the sample values ​​along the boundary between two partitions (e.g., Figure 5C However, instead of using the sample value of Figure 5C The adaptive weighting described above applies a simple average to samples located along the boundary. Figure 5C In the example, the sample values ​​for samples with identifiers 1, 2, 4, 6, and 7 are calculated as follows:

[0249]

[0250] Where Pred0(x,y) is the sample at position (x,y) in the predicted value Pred0, Pred1(x,y) is the sample at position (x,y) in the predicted value Pred1, and the adaptive weights and equal

[0251] In a third variant of the fourth example of the third embodiment, the blending process of the TPM or GEO mode is enabled or disabled by at least one high-level syntax (HLS) element. When at least one HLS element indicates that the blending process is enabled, if WP is enabled for the current block, the blending process of the TPM or GEO mode is disabled. The value of the HLS element can also be inferred. Also in this case, when the inferred value of at least one HLS element indicates that the blending process is enabled, if WP is enabled for the current block, the blending process of the TPM or GEO mode is disabled.

[0252] For example, the HLS element is a syntax element sps_blending_ctrl_flag at the sequence level (i.e., SPS level). The HLS element sps_blending_ctrl_flag indicates whether the blending process of TPM and GEO can be controlled at the slice level (i.e., sps_blending_ctrl_flag=1) or not controlled at the slice level (i.e., sps_blending_ctrl_flag=0). If not present, the value of the HLS element sps_blending_ctrl_flag can be inferred from other syntax elements. In one embodiment, the HLS element sps_blending_ctrl_flag is present in the SPS header when the syntax element sps_triangle_enabled_flag is equal to true and the syntax element sps_weighted_pred_flag is equal to false. The syntax element sps_triangle_enabled_flag specifies whether TPM or GEO mode is available for inter prediction for blocks of a P slice referring to an SPS (if sps_triangle_enabled_flag = true) or not for inter prediction (if sps_triangle_enabled_flag = false). The syntax element sps_weighted_pred_flag equal to true specifies that weighted prediction may be applied for a P slice referring to an SPS. sps_weighted_pred_flag equal to false specifies that weighted prediction is not applied for a P slice referring to an SPS.

[0253] if(sps_triangle_enabled_flag&&! sps_weighted_pred_flag) sps_blending_ctrl_flag u(1)

[0254] Table TAB_A

[0255] Table TAB_A describes an example of SPS level syntax related to the HLS element sps_blending_ctrl_flag.

[0256] When a slice refers to an SPS that indicates through sps_blending_ctrl_flag that the activation or deactivation of the TPM or GEO blending process can be controlled at the slice level, the slice level syntax element slice_blending_off_flag indicates whether the blending process is actually activated (slice_blending_off_flag=1) or not activated (slice_blending_off_flag=0) for the slice.

[0257]

[0258] Table TAB_B

[0259] Table TAB_B describes an example of slice level syntax associated with the syntax element slice_blending_off_flag.

[0260] It can be seen that when the condition set is met, the syntax element slice_blending_off_flag is present in the slice header. For example, the condition set includes:

[0261] TPM or GEO mode is enabled for the current slice (sps_triangle_enabled_flag = true);

[0262] The control of the blending process of slice level TPM or GEO is enabled (sps_blending_ctrl_flag = true);

[0263] WP is disabled for P slices (!(pps_weighted_pred_flag&&slice_type==P));

[0264] · WP is disabled for B slices (!(pps_weighted_bipred_flag&&slice_type==B)).

[0265] If the syntax element slice_blending_off_flag is not present, it is inferred to be '0'.

[0266] At the block level, the variable blending_off_flag specifies whether TPM or GEO mode blending is applied to the current block.

[0267] set up:

[0268] cIdx represents the color component V index of the current block;

[0269] A and B represent two TPM (or GEO) partitions of the current block;

[0270] predListFlagA and predListFlagB represent the prediction list flags of A and B;

[0271] refIdxA and refIdxB represent the reference indexes of partitions A and B

[0272] The variable blending_off_flag is exported as follows:

[0273] Initialize blending_off_flag to the value of the syntax element slice_blending_off_flag;

[0274] If the variable blending_off_flag is equal to "1", the variable blending_off_flag is not modified;

[0275] Otherwise, if cIdx is equal to “0” and ((predListFlagX=0 and the luma weight associated with the reference index refIdxX is a non-default weight (i.e., luma_weight_l0_flag[refIdxX] is equal to “1”)) or (predListFlagX=1 and the luma weight associated with the reference index refIdxX is a non-default weight (i.e., luma_weight_l1_flag[refIdxX] is equal to 1))), then the variable blending_off_flag is set to be equal to “1” (i.e., the blending process of the TPM or GEO mode is not activated for the luma component of the current block),

[0276] · Otherwise, if cIdx is not equal to "0" and ((predListFlagX=0 and the chroma weight associated with the reference index refIdxX is a non-default weight (i.e., chroma_weight_l0_flag[refIdxX] is equal to 1)) or (predListFlagX=1 and the chroma weight associated with the reference index refIdxX is a non-default weight (i.e., chroma_weight_l1_flag[refIdxX] is equal to 1))), then blending_off_flag is set to be equal to "1" (i.e., the blending process of TPM or GEO mode is not activated for the chroma components of the current block).

[0277] In a variant, the SPS level HLS syntax (sps_triangle_enabled_flag, sps_weighted_pred_flag, sps_blending_ctrl_flag) may be replaced with the PPS level HLS syntax (pps_triangle_enabled_flag, pps_weighted_pred_flag, pps_blending_ctrl_flag) with the same semantics.

[0278] In a fourth variation of the fourth example of the third embodiment, when WP is enabled at the HLS level for the current block, the blending process of the TPM or GEO mode is not activated.

[0279] In a fifth variation of the fourth example of the third embodiment, when WP is enabled at the HLS level and at least one WP weight is equal to a non-default weight of the current block, blending processing of the TPM or GEO mode is not activated.

[0280] The VVC standard specified in JVET-O2001 specifies the execution order of prediction tools. For example, in the current version of VVC, when WP is enabled, WP is applied after PROF mode. In the fourth implementation of step 71, the execution order of WP and PSW or OF tools is modified. In the example of the fourth implementation of step 71, WP is applied before PROF mode. In one variant, the choice of applying WP after or before PROF mode depends on the WP weight and offset value. For example, if the weight is better than "1", WP is executed before PROF mode, otherwise WP is executed after PROF mode, so PROF mode is applicable to a wide range of signals.

[0281] In the current version of VVC described in JVET-O2001, DMVR and BDOF modes are disabled when WP is enabled. In a fifth implementation of step 71, WP is applied before DMVR or BDOF mode. This increases coding complexity (two additional buffers for storing references weighted with WP, and one additional weighting stage for motion compensated prediction), but improves prediction accuracy.

[0282] In a sixth implementation of step 71, to reduce complexity, PROF mode is disabled when WP is enabled.

[0283] In the current version of VVC described in JVET-O2001, when WP is enabled, WP is applied before the CIIP mode blending process. In a seventh implementation of step 71, in order to reduce complexity, when WP is enabled, CIIP mode is disabled.

[0284] In an eighth implementation of step 71, when both WP and CIIP modes are enabled for the current block, the regular CIIP mode is applied to the current block, but the WP weight and offset are set to default values ​​for the current block.

[0285] So far, it is considered that the WP and PSW or OF tool processes are cascaded, that is, they are executed sequentially. The result of this cascade is that if a first tool (e.g., WP) is applied before a second tool (e.g., PSW or OF tool), the second tool is applied to the result of the first tool. In the ninth embodiment of step 71, attention is paid to the combination of WP and at least one PSW tool. The modification of the coded portion of the current block associated with weighted prediction and / or at least one PSW tool includes mixing WP and PSW tools at a single weighted (blending) level. The mixed tools can be WP and CIIP or WP and GEO or WP and TPM or WP and LIC and CIIP. PSW tools can also be mixed together. For example, in this case, the mixed tools are LIC and TPM or BCW and TPM or BCW and CIIP. During the mixing process, the weights and offsets of the mixed tools are combined so that a single weighted (blending) process is applied.

[0286] PSW tools can be classified into two subgroups: a first subgroup of PSW tools, where the offset is performed after the right shift; and a second subgroup of PSW tools, where the offset is performed before the right shift. The first subgroup includes BCW mode, CIIP mode, and bidirectional WP. The second subgroup includes unidirectional WP and LIC mode.

[0287] For example, suppose (a i ,b i ,s i ) represent the sample weight, offset and right shift value associated with PSW tool “i” respectively, and pred(x,y) represents the corresponding sample prediction.

[0288] If instrument i=0 is combined with instrument i=1, instruments "0" and "1" belong to the first subgroup of PSW instruments, the prediction sample can be obtained using the following equation:

[0289] pred(x,y)=((a0.a1.pred0(x,y)+a1(b0<<s0)+(1<<(s0+s1-1)))>>(s0+s1))+b1

[0290] where pred0(x,y) is the motion compensated prediction value sample taken in the list L0 of reference pictures stored in the DPB and spatially corresponding to pred(x,y). This equation applies when unidirectional WP and LIC modes are combined.

[0291] If instruments "0" and "1" belong to the second subgroup of PSW instruments, the forecast sample can be obtained using the following equation:

[0292] pred(x,y)=(a0.a1.pred0(x,y)+a1(b0<<s0)+(b1<<s0)+(1<<(s0+s1-1)))>>(s0+s1)

[0293] This equation applies when combining bidirectional WP and BCW mode or bidirectional WP and CIIP mode or CIIP mode and BCW mode.

[0294] Similar equations can be derived for the combination of the tools of the first subset and the tools of the second subset and for the combination of more than two PSW tools.

[0295] Figure 8 is a high-level representation of an implementation of the reconstruction process performed during the decoding process.

[0296] It can be noted that the processes applied during the encoding process and the decoding process are symmetrical. Therefore, many of the embodiments described with respect to the encoding process have symmetrical embodiments during the decoding process.

[0297] Mainly during steps 314, 308 and 316 for each block of the current image Figure 8 process.

[0298] In step 80 , the processing module 40 determines whether WP is enabled for the current block.

[0299] If WP is enabled, step 80 is followed by step 81. During step 81, the processing module 40 modifies a part of the reconstruction process of said current block related to WP and / or to at least one PSW or OF tool.

[0300] If WP is disabled, step 82 is entered after step 80. During step 82, the usual reconstruction process as specified in document JVET-O2001 is applied to the current block.

[0301] In a first implementation of step 80, the processing module 40 checks the values ​​of the syntax elements pps_weighted_pred_flag and pps_weighted_bipred_flag in the PPS referred to by the slice including the current block according to the encoded video stream 211. If one of the syntax elements pps_weighted_pred_flag or pps_weighted_bipred_flag is equal to one, the processing module 40 considers that WP is enabled for the current block. Otherwise, the processing module 40 considers that WP is disabled for the current block.

[0302] In a second implementation of step 80, in addition to checking the values ​​of the syntax elements pps_weighted_pred_flag and pps_weighted_bipred_flag from the PPS referred to by the slice including the current block, the processing module 40 also checks the value of the weight of the WP in the slice header for the slice. If at least one of the weight values ​​is different from the default value, the processing module 40 considers that WP is enabled for the current block. Otherwise, the processing module 40 considers that WP is disabled for the current block.

[0303] When the first or second implementation of step 80 is applied, in the first implementation of step 81, the processing module 40 modifies a part of the reconstruction process of the current block by not parsing or inferring syntax elements related to the PSW or OF tool in the coded video stream. However, in this case, the conventional WP reconstruction process is applied.

[0304] Fig. 9 An example of a portion of the reconstruction process modified according to the first embodiment or the second embodiment of step 80 is disclosed.

[0305] In step 901, the processing module 40 checks a first condition combination, which includes a condition CIIP indicating whether the CIIP mode is enabled for the current block, a condition TPM indicating whether the TPM mode is enabled for the current block, a condition wp_disabled indicating whether the WP is disabled for the current block, and a condition OtherConditions described in JVET_O2001. If the first condition combination (OtherConditions and ((CIIP) or ((TPM and wp_disabled))) is true, during step 902, the processing module 40 parses the syntax element regular_merge_flag specified in the document JVET-O2001 according to the encoded video stream 211. After step 902, step 903 is entered. When the first condition combination is false in step 901, step 903 is directly entered after step 901. In this case, the value of the syntax element regular_merge_flag is inferred by the processing module 40 according to the values ​​of other syntax elements specified by the standard VVC (regular_merge_flag = (general_merge_flag &&! merge_subblock_flag)).

[0306] In step 903, the processing module 40 checks the value of the syntax element regular_merge_flag. If regular_merge_flag=true, the processing module 40 determines during step 904 that the current block uses the merge mode and decodes the regular merge parameters.

[0307] If regular_merge_flag=false, the processing module 40 checks the second condition combination ((OtherConditions and ((CIIP) and (TPM and wp_disabled)))) during step 905 . If the second condition combination is false, the process proceeds to step 906 after step 905 . Otherwise, the process proceeds to step 907 after step 905 .

[0308] During step 906, processing module 40 infers the value of the syntax element ciip_flag and the value of the syntax element MergeTriangleFlag. Ciip_flag=1 indicates that the CIIP mode is applicable to the current block. ciip_flag=0 indicates that the CIIP mode is not applicable to the current block. MergeTriangleFlag=1 indicates that the TPM mode is applicable to the current block. MergeTriangleFlag=0 indicates that the TPM mode is not applicable to the current block. In step 906, one CIIP mode or TPM mode is applicable. The parameters of the CIIP or TPM mode are also inferred by processing module 40.

[0309] During a step 907 , the processing module 40 decodes the syntax element ciip_flag from the encoded video stream 211 .

[0310] During step 908 , the processing module 40 checks the value of the syntax element ciip_flag. If ciip_flag=1, then after step 908 , the process proceeds to step 909 . If ciip_flag=0, then after step 908 , the process proceeds to step 910 .

[0311] During step 909 , the processing module 40 decodes the parameters of the CIIP mode from the encoded video stream 211 .

[0312] During step 910 , the processing module 40 decodes the parameters of the TPM mode from the encoded video stream 211 .

[0313] From About Fig. 9 As can be seen from the described algorithm, TPM parameters are inferred (step 906) or parsed (step 910) from the encoded video stream 211 only when WP is disabled.

[0314] In one embodiment, the modification of the reconstruction process according to the first embodiment or the second embodiment of step 80 implies a modification of the merge data syntax as described in section 7.3.8.7 of JVET-02001.

[0315]

[0316]

[0317] TAB1

[0318] The syntax changes for the merge data syntax are shown in bold in table TAB1.

[0319] In another embodiment, the modification of the reconstruction process according to the first embodiment or the second embodiment of step 80 implies a modification of the slice header syntax as described in Section 7.3.6.1 of JVET-02001.

[0320]

[0321] TAB2

[0322] As can be seen from Table TAB2, the syntax element max_num_merge_cand_minus_max_num_triangle_cand appears in the slice header only when WP is disabled. The syntax element max_num_merge_cand_minus_max_num_triangle_cand indicates the maximum number of TPM mode candidates supported in the slice. This syntax element is useless when it is known that TPM mode is disabled, which is the case when WP is enabled for the current slice. Therefore, during the decoding process, when WP is enabled for the slice, the processing module 40 is not intended to parse the syntax element max_num_merge_cand_minus_max_num_triangle_cand.

[0323] exist Fig. 9In a variant of the algorithm, conditional TPM is replaced by conditional GEO. Conditional GEO indicates whether GEO mode is enabled for the current block. In this variant, GEO parameters are inferred (step 906) or parsed (step 910) from the encoded video stream 211 only when WP is disabled. In addition, in table TAB1, the syntax elements sps_triangle_enabled_flag, MaxNumTriangleMergeCand, merge_triangle_split_dir, merge_triangle_idx0, merge_triangle_idx1 are replaced by syntax elements sps_wedge_enabled_flag, MaxNumWedgeMergeCand, wedge_partition_idx, merge_wedge_idx0 and merge_wedge_idx1, respectively. In table TAB2, the syntax elements sps_triangle_enabled_flag pps_max_num_merge_cand_minus_max_num_triangle_cand_minus1 and max_num_merge_cand_minus_max_num_triangle_cand are replaced by sps_wedge_enabled_flag pps_max_num_merge_cand_minus_max_num_wedge_cand_minus1 and max_num_merge_cand_minus_max_num_wedge_cand, respectively.

[0324] exist Fig. 9 In a variant of the algorithm of , the condition CIIP is replaced by the condition (CIIP and wp_disabled). In this variant, the CIIP parameter is inferred (step 906) or parsed (step 909) from the coded video stream 211 only when WP is disabled. In this case, the merge data syntax is modified as follows (modifications are indicated in bold in Table TAB3):

[0325]

[0326]

[0327] TAB3

[0328] As already mentioned above, parsing of the PPS involved in the slice comprising the current block and the PPS of the slice header of said slice does not allow the processing module 40 to determine whether a WP with a non-default weight is actually applied to the current block. In order to obtain this information, the processing module 40 needs to parse the syntax elements representing the current block (i.e. the coding unit syntax as described in Section 7.3.8.5 of JVET-O2001). In particular, parsing of the syntax elements representing the current block allows determining the values ​​of the syntax elements ref_idx_l0 and ref_idx_l1. The syntax element ref_idx_l0 (or ref_idx_l1) represents an index in the list L0 (or in the list L1) of the reference image of the DPB of the image providing a prediction value for the current block. Using the index ref_idx_l0 (or ref_idx_l1), the processing module 40 can determine whether at least one reference image providing a prediction value for the current block is weighted using a non-default weight based on the flag luma_weight_l0_flag[] for luma and the flag chroma_weight_l0_flag[] for chroma (or luma_weight_l1_flag[] and chroma_weight_l1_flag[]) encoded in the slice header. In practice, a flag equal to "1" specifies that the corresponding weight has a non-default value. A flag equal to "0" specifies that the weight has a default value.

[0329] For some PSW or OF tools, the syntax element ref_idx_lx can only be accessed by parsing the syntax of the PSW or OF tool. This is especially true for TPM, GEO, CIIP modes. For example, in Fig. 9In the third embodiment of step 80, after checking the values ​​of the syntax elements pps_weighted_pred_flag and pps_weighted_bipred_flag in the PPS referred to by the slice including the current block, and the value of the weight used by the WP from the slice header of the slice, the processing module 40 parses or infers the syntax elements related to the PSW or OF tool of the current block to obtain the value of the index ref_idx_lx (where lx is equal to l0 or l1). According to the indexes ref_idx_l0 and ref_idx_l1, the processing module 40 determines the value of the flag luma_weight_lx_flag[ref_idx_lx] or chroma_weight_lx_flag[ref_idx_lx]. If at least one of the flags is equal to "1", the processing module 40 considers that WP is enabled for the current block (ie, if the condition cond is false, where for bidirectional prediction, cond = (luma_weight_l0_flag[ref_idx_l0] == 0

[0330] &&luma_weight_l1_flag[ref_idx_l1]==0

[0331] &&chroma_weight_l0_flag[ref_idx_l0]==0

[0332] &&chroma_weight_l1_flag[ref_idx_l1]==0),

[0333] Or for unidirectional prediction, cond = (luma_weight_lx_flag[ref_idx_lx] = = 0

[0334] &&chroma_weight_lx_flag[ref_idx_lx]==0), the processing device 40 considers that WP is enabled).

[0335] Otherwise, the processing module 40 considers WP disabled for the current block. In other words, if each flag luma_weight_lx_flag[ref_idx_lx] and chroma_weight_lx_flag[ref_idx_lx] associated with the current block is equal to "0", the processing device 40 considers WP disabled (ie, if the condition cond is true, the processing device 40 considers WP disabled).

[0336] When the third implementation of step 80 is applied, in the second implementation of step 81, the processing module 40 modifies a portion of the reconstruction process of the current block by parsing or inferring syntax elements related to the PSW or OF tool in the coded video stream 211 without applying the PSW or OF tool to the current block. In other words, even if the PSW or OF tool is specified in the coded video stream 211, the reconstruction process corresponding to the PSW or OF tool is disabled. However, in this case, the conventional WP reconstruction process is applied.

[0337] When the third implementation of step 80 is applied, in the third implementation of step 81, deactivating the PSW or OF tool includes parsing or inferring syntax elements related to the PSW or OF tool during reconstruction of the current block, but not applying a conventional reconstruction process to the PSW or OF tool. In this implementation, the processing module 40 applies an alternative inter-frame prediction tool to the current block instead of the PSW or OF tool specified in the encoded video stream 211. However, in the third implementation of step 81, a conventional WP reconstruction process is applied.

[0338] In a first example of the third implementation of step 81 (hereinafter referred to as example (3)), WP and TPM modes are enabled for the current block.

[0339] In a variant (3a) of example (3), during step 81, the TPM reconstruction process is replaced by a bidirectional inter-frame prediction (B-mode) reconstruction process as described above. Applying equation (1), the prediction value bipred of the current block with w0=w1=1 is determined for all prediction value samples. The prediction values ​​pred0 and pred1 are generated using the motion information associated with the two partitions specified in the syntax element related to the TPM mode. It can be seen that the variant (3a) applied during the decoding process is the same as the variant (1a) applied during the encoding process.

[0340] Similarly, variants (3b, 3c, 3d, 3e, 3f, and 3g) that are respectively the same as variants (1b, 1c, 1d, 1e, 1f, and 1g) may be applied during the decoding process in place of variant (3a).

[0341] In the second example of the third implementation of step 81 (hereinafter referred to as example (4)), the WP and GEO modes are enabled for the current block.

[0342] In a first variant (4a) of example (2), during step 81, the GEO reconstruction process is replaced by a bidirectional inter-frame prediction reconstruction process as described above. Applying equation (1), the prediction value bipred of the current block with w0=w1=1 is determined. The prediction values ​​pred0 and pred1 are generated using the motion information associated with the two partitions specified in the syntax element related to the GEO mode. It can be seen that the variant (4a) applied during the decoding process is the same as the variant (2a) applied during the encoding process.

[0343] Similarly, variants (4b, 4c, 4d, 4e, 4f, and 4g), which are respectively the same as variants (2b, 2c, 2d, 2e, 2f, and 2g), may be applied during the decoding process in place of variant (4a).

[0344] In the variants (3a), (3b), (4a) and (4b) of the examples (3) and (4) of the third implementation of step 81, the syntax element indicating the edge direction (merge_triangle_split_dir for TPM mode or wedge_partition_idx for GEO mode) becomes useless. In another variant of the examples (3) and (4) of the third implementation of step 81, the two syntax elements are reinterpreted. In other words, in this variant, the semantics of the two syntax elements are changed. For example:

[0345] merge-idx0=merge_triangle_idx0;

[0346] delta_idx=merge_triangle_split_dir; and

[0347] merge-idx1=delta-idx+merge_triangle_idx1.

[0348] In a third example of a third implementation of step 81, WP and GEO modes are enabled for the current block. In this case, during step 81, the GEO reconstruction process is replaced by a general bidirectional inter-frame prediction reconstruction process. Applying equation (1), the prediction value bipred of the current block is determined, where the weight depends on the number of samples included in each partition. For example, referring to Figure 5B, w0 is proportional to the number of samples in partition 505, and w1 is proportional to the number of samples in partition 506. If the block including partitions 505 and 506 is a 16×8 block including "128" samples, partition 505 includes NB1 samples and partition 506 includes NB2 samples, e.g., w0=NB1 / 128 and w1=NB2 / 128. The prediction values ​​pred0 and pred1 are generated using the motion information associated with the two partitions specified in the syntax element associated with the GEO mode. Figure 5B In FIG. 5 , the motion information of partition 505 is used to generate the prediction value pred0, while the motion information of partition 506 is used to generate the prediction value pred1.

[0349] In a third exemplary variation of the third embodiment of step 81, weights w0 and w1 are selected from a set of weights authorized for BCW, i.e., ∈{-2,3,4,5,10}. For example, a first partition that includes more samples than a second partition is associated with a higher set of weights {-2,3,4,5,10} than the second partition. For example, again referring to Figure 5B , the weight w0 associated with partition 505 is equal to "3", while the weight w1 associated with partition 506 is equal to "5." Likewise, the motion information of partition 505 is used to generate the prediction value pred0, while the motion information of partition 506 is used to generate the prediction value pred1.

[0350] In a fourth example of the third implementation of step 81, WP and TPM or GEO modes are enabled for the current block. In this case, a modified TPM or GEO mode is applied to the current block. In this modified TPM or GEO mode, a partitioning of the current block into two partitions (two PUs) specified by the TPM or GEO syntax element in the coded video stream 211 is applied. A prediction value associated with each partition is identified and motion compensated using motion information corresponding to the partition. After predicting each partition, a blending process is used to adjust the boundaries between the two partitions (e.g., Figure 5C However, instead of using the sample value of Figure 5C The adaptive weighting described above applies a simple average to samples located along the boundary. Figure 5C In the example, the sample values ​​for samples with identifiers 1, 2, 4, 6, and 7 are calculated as follows:

[0351]

[0352] Where Pred0(x,y) is the sample at position (x,y) in the predicted value Pred0, Pred1(x,y) is the sample at position (x,y) in the predicted value Pred1, and the adaptive weights and equal

[0353] As about Figure 7 As mentioned, the VVC standard specified in JVET-O2001 specifies the execution order of prediction tools. For example, in the current version of VVC, when WP is enabled, WP is applied after PROF mode. In the fourth implementation of step 81, the execution order of the WP reconstruction process and the PSW or OF tool reconstruction process is modified. In the example of the fourth implementation, WP is applied before PROF mode. In a variant, the choice of applying WP after or before PROF mode depends on the WP weight and the offset value. For example, if the weight is better than "1", WP is executed before PROF mode, otherwise WP is executed after PROF mode, so PROF mode is applicable to a wide range of signals.

[0354] In the current version of VVC described in JVET-O2001, DMVR and BDOF modes are disabled when WP is enabled. In a fifth implementation of step 81, WP is applied before the DMVR and BDOF reconstruction process. This increases decoding complexity (two additional buffers for storing references weighted with WP, and an additional weighting level for motion compensated prediction), but improves prediction accuracy. In fact, both DMVR and BDOF are bidirectional prediction tools (reference image ref-0 and reference image ref-1 are used to calculate the current prediction), performing local (sub-block) motion vector refinement by assuming the following:

[0355] 1. The local motion is constant between ref-0 and ref-1, and

[0356] 2. The local illumination change between ref-0 and ref-1 is caused only by a slight misalignment of the motion compensated subblock of ref-0 (pred-0) with the motion compensated subblock of ref-1 (pred-1). This misalignment of the prediction sample values ​​(pred-0 vs. pred-1) allows the derivation of MV refinement (DMVR) or prediction sample offset correction (BDOF) for application to the current subblock bi-prediction samples.

[0357] In the case where the local illumination changes are (also) caused by illumination scene changes, assumption "2" is no longer valid. The DMVR and BDOF processes may even reduce the bidirectional prediction accuracy. However, if WP is applied to pred-0 and pred-1 before DMVR / BDOF, the WP will correct the motion compensation sub-block ref-0 and the motion compensation sub-block ref-1 so that the illumination changes are corrected to be the same as the current sub-block. Then the optical flow model becomes valid, and DMVR and BDOF actually improve the prediction accuracy.

[0358] In a sixth implementation of step 81, to reduce complexity, PROF mode is disabled when WP is enabled.

[0359] In the current version of VVC described in JVET-O2001, when WP is enabled, WP is applied before the CIIP mode blending process. In a seventh implementation of step 81, in order to reduce complexity, when WP is enabled, CIIP mode is disabled.

[0360] In an eighth implementation of step 81, when both WP and CIIP modes are enabled for the current block, the regular CIIP mode is applied to the current block, but the WP weight and offset are set to default values ​​for the current block.

[0361] So far, it is considered that the WP and PSW or OF tool processes are cascaded, that is, executed sequentially. The result of this cascade is that if a first tool (e.g., WP) is applied before a second tool (e.g., PSW or OF tool), the second tool is applied to the result of the first tool. In the ninth embodiment of step 81, attention is paid to the combination of WP and at least one PSW tool. The modification of part of the reconstruction process of the current block associated with weighted prediction and / or at least one PSW tool includes mixing WP and PSW tools at a single weighted (blending) level. The mixed tools can be WP and CIIP or WP and GEO or WP and TPM or WP and LIC and CIIP. PSW tools can also be mixed together. For example, in this case, the mixed tools are LIC and TPM or BCW and TPM or BCW and CIIP. During the mixing process, the weights and offsets of the mixed tools are combined so that a single weighted (blending) process is applied.

[0362] PSW tools can be classified into two subgroups: a first subgroup of PSW tools, where the offset is performed after the right shift; and a second subgroup of PSW tools, where the offset is performed before the right shift. The first subgroup includes BCW mode, CIIP mode, and bidirectional WP. The second subgroup includes unidirectional WP and LIC mode.

[0363] For example, suppose (a i ,b i ,s i ) represent the sample weight, offset and right shift value associated with PSW tool “i” respectively, and pred(x,y) represents the corresponding sample prediction.

[0364] If instrument i=0 is combined with instrument i=1, instruments "0" and "1" belong to the first subgroup of PSW instruments, the prediction sample can be obtained using the following equation:

[0365] pred(x,y)=((a0.a1.pred0(x,y)+a1(b0<<s0)+(1<<(s0+s1-1)))>>(s0+s1))+b1

[0366] where pred0(x,y) is the motion compensated prediction value sample taken in the list L0 of reference pictures stored in the DPB and spatially corresponding to pred(x,y). This equation applies when unidirectional WP and LIC modes are combined.

[0367] If instruments "0" and "1" belong to the second subgroup of PSW instruments, the forecast sample can be obtained using the following equation:

[0368] pred(x,y)=(a0.a1.pred0(x,y)+a1(b0<<s0)+(b1<<s0)+(1<<(s0+s1-1)))>>(s0+s1)

[0369] This equation applies when combining bidirectional WP and BCW mode or bidirectional WP and CIIP mode or CIIP mode and BCW mode.

[0370] Similar equations can be derived for the combination of the tools of the first subset and the tools of the second subset and for the combination of more than two PSW tools.

[0371] For example, when combining a one-way WP and a LIC, instrument "0" is a two-way WP and instrument "1" is a LIC:

[0372] ·a0=LumaWeightL0[refIdxL0];

[0373] ·s0=Luma_log2_weight_denom+SP;

[0374] ·s0=Luma_offset_LO[refIdxL0]<<(BitDepthY-8).

[0375] The syntax elements LumaWeightL0, refIdxL0, Luma_log2_weight_denom, Luma_offset_LO, and BitDepthY are defined in JVET-O2001.

[0376] a1=a;

[0377] s1 = shift;

[0378] ·b1=b.

[0379] where a, b and shift are the LIC parameters defined above.

[0380] A number of embodiments are described above. Features of these embodiments may be provided individually or in any combination. In addition, embodiments may include one or more of the following features, devices, or aspects, individually or in any combination, across various claim categories and types:

[0381] • Depending on the activation or deactivation of the WP, determine whether the PSW or OF tool can be applied during the reconstruction of the current block.

[0382] • Determine whether the PSW or OF tool is replaced with an alternative inter prediction tool during reconstruction of the current block, depending on the activation or deactivation of the WP.

[0383] • Determine, based on activation or deactivation of a WP, whether to modify the application of said tools for the current block relative to the conventional application of PSW or OF tools.

[0384] • If both PSW tools and WP are enabled for the current block, determine whether the PSW tools and WP are merged in a single weighting stage.

[0385] • Determine the execution order of WP and PSW or OF tools during reconstruction of the current block.

[0386] • A bitstream or signal comprising one or more of the described syntax elements or variants thereof.

[0387] • Insertion of syntax elements in the signaling, which enables the decoder to decode the CU with or without PSW or OF tools in a manner corresponding to that used by the encoder.

[0388] • Creating and / or transmitting and / or receiving and / or decoding a bitstream or signal comprising one or more of the described syntax elements or variations thereof.

[0389] A television, set-top box, mobile phone, tablet or other electronic device implementing at least one of the described embodiments.

[0390] A television, set-top box, mobile phone, tablet or other electronic device that implements at least one of the described embodiments and displays the resulting images (e.g., using a monitor, screen or other type of display).

[0391] A television, set-top box, mobile phone, tablet or other electronic device that tunes (e.g., using a tuner) to a channel to receive a signal comprising an encoded video stream and performs at least one of the described embodiments.

[0392] A television, set-top box, mobile phone, tablet or other electronic device that receives a signal including an encoded video stream over the air (e.g., using an antenna) and performs at least one of the described embodiments.

Claims

1. A method for decoding, the method comprising: determining whether to enable a weighted prediction mode for a current block based on the video, wherein a residual block is based on a difference between the current block and a weighted version of a reference block or a weighted average of a plurality of reference blocks; as well as In response to the weighted prediction mode being enabled, modifying a portion of a reconstruction process of the current block associated with the weighted prediction mode and the at least another prediction tool by changing an execution order of the weighted prediction mode and at least another prediction tool, the other prediction tool being a tool for a prediction sample weighting process using a set of tools including a CU-level weighted bi-prediction mode, a triangle prediction mode, a geometric prediction mode, a combined intra-frame inter-frame prediction, and a local illumination compensation tool, or a tool for refining motion information for motion compensation based on an optical flow model; wherein the weighted prediction mode is enabled when a reference image providing a reference block for inter-frame prediction of the current block is associated with a weight different from a default weight, and wherein determining to enable the weighted prediction mode for the current block comprises parsing or inferring syntax elements related to the at least another prediction tool in a portion of video data representing the current block.

2. A method for encoding a current block, the method comprising: determining whether to enable a weighted prediction mode for the current block, wherein a residual block is based on a difference between the current block and a weighted version of a reference block or a weighted average of two reference blocks; as well as In response to the weighted prediction mode being enabled, modifying a portion of the encoding process of the current block associated with the weighted prediction mode and the at least another prediction tool by changing an execution order of the weighted prediction mode and at least another prediction tool, the other prediction tool being a tool for a prediction sample weighting process using a set of tools including a CU-level weighted bi-prediction mode, a triangle prediction mode, a geometric prediction mode, a combined intra-frame inter-frame prediction, and a local illumination compensation tool, or a tool that can refine motion information for motion compensation based on an optical flow model; wherein the weighted prediction mode is enabled when a reference image providing a reference block for inter-frame prediction of the current block is associated with a weight different from a default weight, and wherein determining to enable the weighted prediction mode for the current block comprises parsing or inferring syntax elements related to the at least another prediction tool in a portion of video data representing the current block.

3. A device for decoding, the device comprising: means for determining from a video whether to enable a weighted prediction mode for a current block, wherein a residual block is based on a difference between the current block and a weighted version of a reference block or a weighted average of a plurality of reference blocks; as well as Modification means applied in response to weighted prediction being enabled, the modification means modifying a part of the reconstruction process of the current block associated with the weighted prediction mode and at least another prediction tool by changing the execution order of the weighted prediction mode and at least another prediction tool, the other prediction tool being a tool for a prediction sample weighting process using a set of tools including a CU-level weighted bi-prediction mode, a triangle prediction mode, a geometric prediction mode, a combined intra-frame inter-frame prediction, and a local illumination compensation tool, or a tool for refining motion information for motion compensation based on an optical flow model; wherein the weighted prediction mode is enabled when a reference image providing a reference block for inter-frame prediction of the current block is associated with a weight different from a default weight, and wherein determining to enable the weighted prediction mode for the current block comprises parsing or inferring syntax elements related to the at least another prediction tool in a portion of video data representing the current block.

4. A device for encoding a current block, the device comprising: means for determining whether to enable a weighted prediction mode for the current block, wherein a residual block is based on a difference between the current block and a weighted version of a reference block or a weighted average of a plurality of reference blocks; as well as a modification device applied in response to the weighted prediction mode being enabled, the modification device modifying a part of the reconstruction process of the current block related to the weighted prediction mode and the at least another prediction tool by changing the execution order of the weighted prediction mode and the at least another prediction tool, the other prediction tool being a tool for a prediction sample weighting process using a set of tools including a CU-level weighted bi-prediction mode, a triangle prediction mode, a geometric prediction mode, a combined intra-frame inter-frame prediction, and a local illumination compensation tool, or a tool that can refine motion information for motion compensation based on an optical flow model; wherein the weighted prediction mode is enabled when a reference image providing a reference block for inter-frame prediction of the current block is associated with a weight different from a default weight, and wherein determining to enable the weighted prediction mode for the current block comprises parsing or inferring syntax elements related to the at least another prediction tool in a portion of video data representing the current block.

5. A computer program product, comprising program code instructions for implementing the decoding method according to claim 1 or for implementing the encoding method according to claim 2.

6. An information storage device storing program code instructions for implementing the decoding method according to claim 1 or for implementing the encoding method according to claim 2.

Images