Method and device for video processing and medium

Abstract

The embodiment of the invention provides a solution for video processing. A method for video processing is proposed. The method comprises: during a conversion between a target video block of a video and a bit stream of the video, obtaining prediction information of a target sub-block in the target video block based on a type of a prediction sample in the target sub-block of the target video block, the target video block being encoded and decoded by a geometric partitioning tool; and performing conversion based on the prediction information of the target sub-block.

Description

Technical Field

[0001] Embodiments of the present disclosure generally relate to video coding techniques, and more particularly, to prediction for geometrically partitioning coding blocks. Background Art

[0002] Nowadays, digital video functions are being applied to all aspects of people's lives. For video encoding / decoding, various types of video compression technologies have been proposed, such as MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4 Part 10 Advanced Video Codec (AVC), ITU-T H.265 High Efficiency Video Codec (HEVC) standard, and Versatile Video Codec (VVC) standard. However, the encoding and decoding efficiency of conventional video encoding and decoding technologies is generally low, which is undesirable. Summary of the invention

[0003] Embodiments of the present disclosure provide solutions for video processing.

[0004] In a first aspect, a method for video processing is proposed. The method comprises: during conversion between a target video block of a video and a bitstream of the video, obtaining prediction information of a target subblock in the target video block based on a type of prediction samples in a target subblock of the target video block, the target video block being encoded and decoded by a geometric partitioning tool; and performing conversion based on the prediction information of the target subblock. Compared with conventional schemes, the proposed method can advantageously improve encoding and decoding effectiveness and encoding and decoding efficiency.

[0005] In a second aspect, a device for processing video data is provided, wherein the device for processing video data comprises a processor and a non-volatile memory with instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform the method according to the first aspect.

[0006] In a third aspect, a non-transitory computer-readable recording medium is provided. The non-transitory computer-readable recording medium stores a code stream of a video generated by a method executed by a video processing device, wherein the method comprises: obtaining prediction information of a target sub-block in a target video block of the video based on a prediction sample type in the target sub-block, the target video block being encoded and decoded by a geometric partitioning tool; and generating a bit stream based on the prediction information of the target sub-block.

[0007] In a fourth aspect, another method for storing a code stream of a video is provided. The method comprises: obtaining prediction information of a target sub-block based on a prediction sample type in a target sub-block in a target video block of the video, wherein the target video block is encoded and decoded by a geometric partitioning tool; generating a bit stream based on the prediction information of the target sub-block; and storing the bit stream in a non-transitory computer-readable recording medium.

[0008] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become more apparent through the following detailed description with reference to the accompanying drawings. In the exemplary embodiments of the present disclosure, the same reference numerals generally refer to the same components.

[0010] Figure 1 A block diagram showing an example video encoding and decoding system according to some embodiments of the present disclosure is shown;

[0011] Figure 2 A block diagram of an example video encoder according to some embodiments of the present disclosure is shown;

[0012] Figure 3 A block diagram of an example video decoder according to some embodiments of the present disclosure is shown;

[0013] Figure 4 is a schematic diagram of the intra prediction mode;

[0014] Figure 5A A schematic diagram of the top reference is shown;

[0015] Figure 5B A schematic diagram of the left reference is shown;

[0016] Figure 6 A schematic diagram showing the discontinuity problem when the direction exceeds 45°;

[0017] Fig. 7A A schematic diagram showing the sample definition used by PDPC applied to diagonal top right intra mode;

[0018] Figure 7B A schematic diagram showing the definition of samples used by PDPC applied to the diagonal bottom left intra mode;

[0019] Figure 7C A schematic diagram showing the sample definition used by PDPC applied to the upper right frame mode in the adjacent diagonal;

[0020] Fig.7D A schematic diagram showing the sample definition used by PDPC applied to the lower left corner frame mode in the adjacent diagonal;

[0021] Figure 8 A schematic diagram showing an example of four reference rows of a neighboring prediction block;

[0022] Fig. 9A and Fig. 9B A schematic diagram of sub-division is shown;

[0023] Fig.10 A schematic diagram of a matrix-weighted intra prediction process is shown;

[0024] Fig.11 A schematic diagram showing the positions of spatial merging candidates;

[0025] Fig.12 A schematic diagram showing candidate pairs considered for redundancy checking of spatial merging candidates;

[0026] Fig.13 A schematic diagram showing motion vector scaling for temporal merging candidates is shown;

[0027] Fig.14 The candidate positions of the temporal merge candidates C0 and C1 are shown;

[0028] Fig.15A and Fig. 15B A schematic diagram of the MMVD search point is shown;

[0029] Fig.16 A schematic diagram showing an extended CU region used in BDOF;

[0030] Fig.17 A schematic diagram of a symmetric MVD pattern is shown;

[0031] Fig.18 Decoding side motion vector refinement is shown;

[0032] Fig.19 Schematic diagram showing the top and left neighboring blocks used in CIIP weight derivation;

[0033] Fig. 20 A schematic diagram showing an example of a GPM segmentation grouped by the same angle;

[0034] Fig.21 A schematic diagram showing unidirectional prediction MV selection for geometric partitioning mode;

[0035] Fig. 22 A schematic diagram showing an example of generating a warp weight w0 using a geometric partitioning mode;

[0036] Fig.23 A schematic diagram showing the proposed intra-block decoding process;

[0037] Fig.24 A schematic diagram showing the calculation of HoG from a template with a width of 3 pixels;

[0038] Fig.25 Schematic diagram showing prediction fusion by weighted averaging of two HoG modes and planes;

[0039] Fig.26 An example of sub-block based motion / mode information storage for a GPM codec block is shown;

[0040] Fig. 27 A flowchart showing a method according to an embodiment of the present disclosure; and

[0041] Fig.28 A block diagram of a computing device is shown in which various embodiments of the present disclosure may be implemented.

[0042] Throughout the drawings, same or similar reference numbers generally refer to same or similar elements. DETAILED DESCRIPTION

[0043] The principle of the present disclosure will now be described with reference to some embodiments. It should be understood that these embodiments are described only for the purpose of illustrating and helping those skilled in the art to understand and implement the present disclosure, without implying any limitation on the scope of the present disclosure. In addition to the methods described below, the disclosure described herein can also be implemented in various ways.

[0044] In the following description and claims, unless defined otherwise, all scientific and technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present disclosure belongs.

[0045] References in this disclosure to "one embodiment," "an embodiment," "an example embodiment," and the like indicate that the described embodiment may include a particular feature, structure, or characteristic, but not every embodiment must include the particular feature, structure, or characteristic. Furthermore, these phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in conjunction with an example embodiment, whether or not explicitly described, it is considered to be within the knowledge of those skilled in the art to affect such feature, structure, or characteristic in relation to other embodiments.

[0046] It should be understood that although the terms "first" and "second" etc. can be used to describe various elements, these elements should not be limited to these terms. These terms are only used to distinguish one element from another element. For example, a first element can be referred to as a second element, and similarly, a second element can be referred to as a first element without departing from the scope of the exemplary embodiments. As used herein, the term "and / or" includes any and all combinations of one or more of the listed terms.

[0047] The terms used herein are only used for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments. As used herein, the singular forms "a", "an", and "the" are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms "including", "comprising", and / or "having" when used herein indicate the presence of the features, elements, and / or components, etc., but do not exclude the presence or addition of one or more other features, elements, components, and / or combinations thereof.

[0048] Example Environment

[0049] Figure 1 1 is a block diagram illustrating an example video codec system 100 that may utilize the techniques of the present disclosure. As shown, the video codec system 100 may include a source device 110 and a destination device 120. The source device 110 may also be referred to as a video encoding device, and the destination device 120 may also be referred to as a video decoding device. In operation, the source device 110 may be configured to generate encoded video data, and the destination device 120 may be configured to decode the encoded video data generated by the source device 110. The source device 110 may include a video source 112, a video encoder 114, and an input / output (I / O) interface 116.

[0050] Video source 112 may include a source such as a video capture device. Examples of a video capture device include, but are not limited to, an interface that receives video data from a video content provider, a computer graphics system for generating video data, and / or combinations thereof.

[0051] The video data may include one or more pictures. The video encoder 114 encodes the video data from the video source 112 to generate a bitstream. The bitstream may include a bit sequence that forms a coded representation of the video data. The bitstream may include coded pictures and associated data. The coded pictures are coded representations of pictures. The associated data may include sequence parameter sets, picture parameter sets, and other grammatical structures. The I / O interface 116 may include a modulator / demodulator and / or a transmitter. The coded video data may be directly transmitted to the destination device 120 via the network 130A via the I / O interface 116. The coded video data may also be stored on a storage medium / server 130B for access by the destination device 120.

[0052] The destination device 120 may include an I / O interface 126, a video decoder 124, and a display device 122. The I / O interface 126 may include a receiver and / or a modem. The I / O interface 126 may obtain encoded video data from the source device 110 or the storage medium / server 130B. The video decoder 124 may decode the encoded video data. The display device 122 may display the decoded video data to the user. The display device 122 may be integrated with the destination device 120, or may be outside the destination device 120, which is configured to be connected to an external display device interface.

[0053] The video encoder 114 and the video decoder 124 may operate according to a video compression standard, such as the High Efficiency Video Codec (HEVC) standard, the Versatile Video Codec (VVC) standard, and other existing and / or future standards.

[0054] Figure 2 is a block diagram showing an example of a video encoder 200 according to some embodiments of the present disclosure, which may be Figure 1 An example of a video encoder 114 in the system 100 is shown.

[0055] Video encoder 200 may be configured to implement any or all of the techniques of this disclosure. Figure 2 In the example of , video encoder 200 includes multiple functional components. The techniques described in this disclosure can be shared between the various components of video encoder 200. In some examples, a processor can be configured to perform any or all of the techniques described in this disclosure.

[0056] In some embodiments, the video encoder 200 may include a partitioning unit 201, a prediction unit 202, a residual generation unit 207, a transformation unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transformation unit 211, a reconstruction unit 212, a buffer 213 and an entropy coding and decoding unit 214, and the prediction unit 202 may include a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205 and an intra-frame prediction unit 206.

[0057] In other examples, the video encoder 200 may include more, fewer, or different functional components. In one example, the prediction unit 202 may include an intra-block copy (IBC) unit. The IBC unit may perform prediction in an IBC mode, where at least one reference picture is a picture where the current video block is located.

[0058] Furthermore, although some components (such as the motion estimation unit 204 and the motion compensation unit 205) may be integrated, for the purpose of explanation, these components are described in detail below. Figure 2 are shown separately in the example.

[0059] The partitioning unit 201 may partition a picture into one or more video blocks. The video encoder 200 and the video decoder 300 (which will be discussed in detail below) may support various video block sizes.

[0060] The mode selection unit 203 may select one of a plurality of codec modes (intra-frame coding or inter-frame coding), for example, based on the error result, and provide the generated intra-frame codec block or inter-frame codec block to the residual generation unit 207 to generate residual block data, and to the reconstruction unit 212 to reconstruct the codec block for use as a reference picture. In some examples, the mode selection unit 203 may select a combination of intra-frame and inter-frame prediction (CIIP) modes, where the prediction is based on an inter-frame prediction signal and an intra-frame prediction signal. In the case of inter-frame prediction, the mode selection unit 203 may also select a resolution for the motion vector (e.g., sub-pixel precision or integer pixel precision) for the block.

[0061] To perform inter-frame prediction on the current video block, the motion estimation unit 204 may generate motion information for the current video block by comparing the current video block with one or more reference frames from the buffer 213. The motion compensation unit 205 may determine a predicted video block for the current video block based on the motion information and decoded samples of pictures from the buffer 213 other than the picture associated with the current video block.

[0062] The motion estimation unit 204 and the motion compensation unit 205 may perform different operations on the current video block, for example, depending on whether the current video block is in an I slice, a P slice, or a B slice. As used herein, an "I slice" may refer to a portion of a picture consisting of macroblocks, all of which are based on macroblocks within the same picture. Furthermore, as used herein, in some aspects, a "P slice" and a "B slice" may refer to a portion of a picture consisting of macroblocks that are independent of macroblocks in the same picture.

[0063] In some examples, the motion estimation unit 204 may perform unidirectional prediction on the current video block, and the motion estimation unit 204 may search the reference pictures of list 0 or list 1 to find the reference video block for the current video block. The motion estimation unit 204 may then generate a reference index and a motion vector, the reference index indicating the reference picture in list 0 or list 1 containing the reference video block, and the motion vector indicating the spatial displacement between the current video block and the reference video block. The motion estimation unit 204 may output the reference index, the prediction direction indicator, and the motion vector as the motion information of the current video block. The motion compensation unit 205 may generate a predicted video block for the current video block based on the reference video block indicated by the motion information of the current video block.

[0064] Alternatively, in other examples, the motion estimation unit 204 may perform bidirectional prediction on the current video block. The motion estimation unit 204 may search the reference pictures in list 0 for a reference video block for the current video block, and may also search the reference pictures in list 1 for another reference video block for the current video block. The motion estimation unit 204 may then generate a plurality of reference indexes and a plurality of motion vectors, the plurality of reference indexes indicating a plurality of reference pictures in list 0 and list 1 containing a plurality of reference video blocks, and the plurality of motion vectors indicating a plurality of spatial displacements between the plurality of reference video blocks and the current video block. The motion estimation unit 204 may output the plurality of reference indexes and the plurality of motion vectors of the current video block as motion information of the current video block. The motion compensation unit 205 may generate a predicted video block for the current video block based on the plurality of reference video blocks indicated by the motion information of the current video block.

[0065] In some examples, motion estimation unit 204 may output a complete set of motion information for use in a decoding process by a decoder. Alternatively, in some embodiments, motion estimation unit 204 may signal motion information of a current video block with reference to motion information of another video block. For example, motion estimation unit 204 may determine that motion information of a current video block is sufficiently similar to motion information of a neighboring video block.

[0066] In one example, motion estimation unit 204 may indicate to video decoder 300 a value in a syntax structure associated with the current video block that indicates that the current video block has the same motion information as another video block.

[0067] In another example, the motion estimation unit 204 may identify another video block and a motion vector difference (MVD) in a syntax structure associated with the current video block. The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the indicated video block. The video decoder 300 may use the motion vector of the indicated video block and the motion vector difference to determine the motion vector of the current video block.

[0068] As discussed above, the video encoder 200 may signal motion vectors in a predictive manner.Two examples of prediction signaling techniques that may be implemented by the video encoder 200 include Advanced Motion Vector Prediction (AMVP) and merge mode signaling.

[0069] The intra prediction unit 206 may perform intra prediction on the current video block. When performing intra prediction on the current video block, the intra prediction unit 206 may generate prediction data for the current video block based on decoded samples of other video blocks in the same picture. The prediction data for the current video block may include a prediction video block and various syntax elements.

[0070] The residual generation unit 207 may generate residual data for the current video block by subtracting (e.g., indicated by a minus sign) the prediction video block(s) of the current video block from the current video block. The residual data of the current video block may include residual video blocks corresponding to different sample portions of samples in the current video block.

[0071] In other examples, such as in skip mode, there may be no residual data for the current video block, and the residual generation unit 207 may not perform a subtraction operation.

[0072] Transform unit 208 may generate one or more transform coefficient video blocks for a current video block by applying one or more transforms to the residual video block associated with the current video block.

[0073] After transform unit 208 generates a transform coefficient video block associated with the current video block, quantization unit 209 may quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values ​​associated with the current video block.

[0074] The inverse quantization unit 210 and the inverse transform unit 211 may apply inverse quantization and inverse transform to the transform coefficient video block, respectively, to reconstruct a residual video block from the transform coefficient video block. The reconstruction unit 212 may add the reconstructed residual video block to corresponding samples from one or more prediction video blocks generated by the prediction unit 202 to generate a reconstructed video block associated with the current video block for storage in the buffer 213.

[0075] After reconstruction unit 212 reconstructs the video block, a loop filtering operation may be performed to reduce video blocking artifacts in the video block.

[0076] The entropy coding unit 214 may receive data from other functional components of the video encoder 200. When the data is received, the entropy coding unit 214 may perform one or more entropy coding operations to generate entropy coded data and output a bitstream including the entropy coded data.

[0077] Figure 3 is a block diagram showing an example of a video decoder 300 according to some embodiments of the present disclosure, which may be Figure 1 An example of a video decoder 124 in the system 100 is shown.

[0078] Video decoder 300 may be configured to perform any or all of the techniques of this disclosure. Figure 3In the example of , video decoder 300 includes multiple functional components. The techniques described in this disclosure can be shared between the various components of video decoder 300. In some examples, a processor can be configured to perform any or all of the techniques described in this disclosure.

[0079] exist Figure 3 In the example of , the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, and a reconstruction unit 306 and a buffer 307. In some examples, the video decoder 300 can perform a decoding process that is generally opposite to the encoding process described with respect to the video encoder 200.

[0080] The entropy decoding unit 301 can retrieve the encoded bitstream. The encoded bitstream may include entropy encoded video data (e.g., encoded video data blocks). The entropy decoding unit 301 can decode the entropy encoded video data, and the motion compensation unit 302 can determine motion information from the entropy decoded video data, the motion information including motion vectors, motion vector precision, reference picture list indexes, and other motion information. The motion compensation unit 302 can determine the information, for example, by performing AMVP and merge mode. AMVP is used, including deriving several most likely candidates based on data and reference pictures of neighboring PBs. The motion information typically includes horizontal and vertical motion vector displacement values, one or two reference picture indexes, and in the case of prediction areas in B strips, also includes an identification of which reference picture list is associated with each index. As used herein, in some aspects, "merge mode" may refer to deriving motion information from blocks that are adjacent in space or time.

[0081] The motion compensation unit 302 may generate motion compensated blocks, possibly performing interpolation based on interpolation filters.Identifiers for the interpolation filters used with sub-pixel precision may be included in the syntax elements.

[0082] The motion compensation unit 302 may calculate interpolated values ​​for sub-integer pixels of a reference block using interpolation filters used by the video encoder 200 during encoding of the video block. The motion compensation unit 302 may determine the interpolation filters used by the video encoder 200 based on received syntax information, and the motion compensation unit 302 may use the interpolation filters to generate a prediction block.

[0083] The motion compensation unit 302 may use at least part of the syntax information to determine the size of blocks used to encode (multiple) frames and / or (multiple) slices of the encoded video sequence, partition information describing how each macroblock of a picture of the encoded video sequence is partitioned, a mode indicating how each partition is encoded, one or more reference frames (and reference frame lists) for each inter-frame coding block, and other information for decoding the encoded video sequence. As used herein, in some aspects, a "slice" may refer to a data structure that can be decoded independently of other slices of the same picture in terms of entropy coding, signal prediction, and residual signal reconstruction. A slice may be an entire picture, or it may be a region of a picture.

[0084] The intra prediction unit 303 may use, for example, an intra prediction mode received in the bitstream to form a prediction block from spatially neighboring blocks. The inverse quantization unit 304 inverse quantizes (i.e., dequantizes) the quantized video block coefficients provided in the bitstream and decoded by the entropy decoding unit 301. The inverse transform unit 305 applies an inverse transform.

[0085] The reconstruction unit 306 may obtain the decoded block, for example, by adding the residual block to the corresponding prediction block generated by the motion compensation unit 302 or the intra prediction unit 303. If necessary, a deblocking filter may also be applied to filter the decoded block to remove blocking artifacts. The decoded video block is then stored in a buffer 307, which provides reference blocks for subsequent motion compensation / intra prediction, and the buffer 307 also generates the decoded video for presentation on a display device.

[0086] Some example embodiments of the present disclosure will be described in detail below. It should be noted that the section titles used in this document are for ease of understanding, and the embodiments disclosed in the section are not limited to that section. In addition, although some embodiments are described with reference to general video codecs or other specific video codecs, the disclosed technology is also applicable to other video coding and decoding technologies. In addition, although some embodiments describe the video encoding steps in detail, it should be understood that the corresponding decoding steps of canceling the encoding will be implemented by the decoder. In addition, the term video processing includes video coding or compression, video decoding or decompression, and video transcoding, in which video pixels are represented from one compression format to another compression format or at different compression bit rates.

[0087] 1. Overview

[0088] The present disclosure relates to video coding technology, and more specifically, to generating a prediction block from more than one combination, wherein each combination can be obtained from a different coding technology. It can be applied to existing video coding standards such as HEVC and VVC, and can also be applied to future video coding standards or video codecs.

[0089] 2. Background

[0090] Video codec standards have evolved primarily through the development of the well-known ITU-T and ISO / IEC standards. ITU-T developed H.261 and H.263, ISO / IEC developed MPEG-1 and MPEG-4 Visual, and the two organizations jointly developed H.262 / MPEG-2 Video and H.264 / MPEG-4 Advanced Video Codec (AVC) and H.265 / HEVC standards. Since H.262, video codec standards have been based on a hybrid video codec structure, in which temporal prediction plus transform codec is used. In order to explore future video codec technologies beyond HEVC, VCEG and MPEG jointly founded the Joint Video Exploration Team (JVET) in 2015. JVET meetings are held simultaneously every quarter, and the new video codec standard was officially named Versatile Video Codec (VVC) at the JVET meeting in April 2018, when the first version of the VVC Test Model (VTM) was released. The VVC working draft and the test model VTM are updated after each meeting. The VVC project achieved technical completion (FDIS) at the July 2020 meeting.

[0091] 2.1. Codec Tools

[0092] 2.1.1. Intra-frame prediction

[0093] 2.1.1.1. Intra-mode codec with 67 intra-prediction modes

[0094] To capture arbitrary edge directions present in natural videos, the number of directional intra modes in VVC is expanded to 65 from the 33 used in HEVC. Figure 4 A schematic diagram 400 of intra prediction modes is shown. Figure 4 New directional modes not in HEVC are depicted as dashed arrows, and planar and DC modes remain unchanged. These more dense directional intra prediction modes apply to all block sizes and luma and chroma intra prediction.

[0095] In VVC, for non-square blocks, several traditional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes.

[0096] In HEVC, each intra-coded block has a square shape, and the length of each of its sides is a power of 2. Therefore, no division operation is required to generate intra predictors using DC mode. In VVC, blocks can have a rectangular shape, which in general requires the use of a division operation for each block. To avoid division operations for DC prediction, only the longer sides are used to calculate the average value for non-square blocks.

[0097] 2.1.1.2. Intra-mode encoding and decoding

[0098] In order to keep the complexity of the most probable mode (MPM) list generation low, a 6MPM intra mode encoding and decoding method is adopted considering two available adjacent intra modes. The following three aspects are considered to construct the MPM list:

[0099] – Default intra mode

[0100] – Neighboring intra mode

[0101] – Export Intra Mode

[0102] Regardless of whether the MRL and ISP codecs are applied, a unified 6-MPM list is used for intra blocks. The MPM list is constructed based on the intra modes of the neighboring blocks on the left and above. Assuming that the mode on the left is denoted as Left and the mode of the block above is denoted as Above, the unified MPM list is constructed as follows:

[0103] – When a neighboring block is not available, its internal mode is set to planar by default.

[0104] – If both Left and Above modes are non-angle modes:

[0105] – MPM list → {plane, DC, V, H, V-4, V+4}

[0106] – If one of the Left and Above modes is an angle mode and the other is a non-angle mode:

[0107] – Set mode Max as the larger mode in Left and Above

[0108] – MPM list → {Plane, Max, DC, Max-1, Max+1, Max-2}

[0109] – If Left and Above are both angles, and they are different:

[0110] – Set mode Max as the larger mode in Left and Above

[0111] – If the difference between the Left mode and the Above mode is in the range of 2 to 62 (inclusive)

[0112] – MPM list → {Plane, Left, Above, DC, Max-1, Max+1}

[0113] -otherwise

[0114] – MPM List → {Flat, Left, Above, DC, Max-2, Max+2}

[0115] – If Left and Above are both angles, and they are the same:

[0116] – MPM list → {Plane, Left, Left-1, Left+1, DC, Left-2}

[0117] In addition, the first binary bit of the mpm index codeword is CABAC context coding. A total of three contexts are used, corresponding to whether the current intra block is MRL-enabled, ISP-enabled, or a normal intra block.

[0118] During the 6-MPM list generation process, pruning is used to remove repeated patterns so that only unique patterns can be included in the MPM list. For entropy coding and decoding of the 61 non-MPM patterns, truncated binary code (TBC) is used.

[0119] 2.1.1.3. Wide-angle intra prediction for non-square blocks

[0120] The conventional angular intra prediction direction is defined as 45 degrees to -135 degrees clockwise. In VVC, for non-square blocks, several conventional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes. The replaced mode is transmitted through the signal using the original mode index, which is remapped to the index of the wide-angle mode after parsing. The total number of intra prediction modes remains unchanged, i.e. 67, and the intra mode encoding and decoding method remains unchanged.

[0121] Figure 5A A schematic diagram 500 is shown with top reference.

[0122] Figure 5B A schematic diagram 550 is shown for left reference.

[0123] To support these prediction directions, a top reference of length 2W+1 is defined as Figure 5A As shown, and the left reference of length 2H+1 is defined as Figure 5B shown.

[0124] The number of patterns that are replaced in the wide-angle direction mode depends on the aspect ratio of the block.

[0125] The alternative intra prediction modes are shown in Table 1.

[0126] Table 1 - Intra prediction modes replaced by wide angle mode

[0127]

[0128] Figure 6 A schematic diagram 600 is shown of a discontinuity where the orientation exceeds 45 degrees. Figure 6 As shown, in the case of wide-angle intra prediction, two vertically adjacent prediction samples can use two non-adjacent reference samples. Therefore, a low-pass reference sample filter and side smoothing are applied to wide-angle prediction to reduce the negative impact of the increased gap Δpα. If the wide-angle mode represents a non-fractional offset. There are 8 modes in the wide-angle mode that meet this condition, and the 8 modes are [-14, -12, -10, -6, 72, 76, 78, 80]. When a block is predicted by these modes, the samples in the reference buffer are directly copied without applying any interpolation. With this modification, the number of samples that need to be smoothed is reduced. In addition, it aligns the design of non-fractional modes in traditional prediction mode and wide-angle mode.

[0129] In VVC, 4:2:2 and 4:4:4 as well as 4:2:0 chroma formats are supported. The chroma derivation mode (DM) derivation table for the 4:2:2 chroma format was originally ported from HEVC, which expanded the number of entries from 35 to 67 to be consistent with the expansion of intra prediction modes. Since the HEVC specification does not support prediction angles below -135 degrees and above 45 degrees, the luma intra prediction modes ranging from 2 to 5 are mapped to 2. Therefore, the chroma DM derivation table for the 4:2:2: chroma format is updated by replacing the values ​​of the entries of the mapping table to more accurately convert the prediction angles of the chroma blocks.

[0130] 2.1.1.4. Mode Dependent Intra-Frame Smoothing (MDIS)

[0131] A four-tap intra-frame interpolation filter is used to improve the accuracy of directional intra-frame prediction. In HEVC, a two-tap linear interpolation filter is used to generate intra-frame prediction blocks in directional prediction mode (i.e., excluding planar and DC predictors). In VVC, a simplified 6-bit 4-tap Gaussian interpolation filter is used only for directional intra-frame mode. The non-directional intra-frame prediction process is unchanged. The selection of the 4-tap filter is performed according to the MDIS condition of the directional intra-frame prediction mode that provides non-fractional displacement, that is, all the following directional modes are excluded: 2, HOR_IDX, DIA_IDX, VER_IDX, 66.

[0132] Depending on the intra prediction mode, the following reference sample processing is performed:

[0133] – Directional intra prediction modes are classified into one of the following groups:

[0134] – vertical mode or horizontal mode (HOR_IDX, VER_IDX),

[0135] – Diagonal mode, indicating that the angle is a multiple of 45 degrees (2, DIA_IDX, VDIA_IDX),

[0136] – Other directional modes;

[0137] – If the directional intra prediction mode is classified as belonging to group A, no filter is applied to the reference samples to generate the predicted samples;

[0138] – Otherwise, if the mode belongs to group B, the [1,2,1] reference sample filter may be applied (depending on the MDIS condition) to the reference samples to further copy these filtered values ​​to the intra predictor according to the selected direction, but no interpolation filter is applied;

[0139] Otherwise, if the mode is classified as belonging to group C, only the intra reference sample interpolation filter is applied to the reference samples to generate predicted samples at fractional or integer positions falling between the reference samples according to the selected direction (no reference sample filtering is performed).

[0140] 2.1.1.5. Position-dependent intra prediction combination

[0141] In VVC, the intra prediction results for DC, planar, and several angular modes are further modified by the Position Dependent Intra Prediction Combination (PDPC) method. PDPC is an intra prediction method that calls for a combination of boundary reference samples and HEVC-style intra prediction with filtered boundary reference samples. PDPC is applied to the following intra modes without signaling: planar, DC, horizontal, vertical, bottom-left corner mode and its eight neighboring corner modes, and top-right corner mode and its eight neighboring corner modes.

[0142] The prediction sample pred(x',y') is predicted using a linear combination of the intra prediction mode (DC, planar, angular) and the reference sample according to the following equation 3-8:

[0143] pred(x',y')=(wL×R -1,y’ +wT×R x’,-1 -wTL×R -1,-1 +(64-wL-wT+wTL)×pred(x',y')+32)>>6 (2-1)

[0144] Its R x,-1 ,R -1,y represent the reference samples located at the top and left boundaries of the current sample (x, y), and R -1,-1 Represents the reference sample located at the upper left corner of the current block.

[0145] If PDPC is applied to DC, planar, horizontal and vertical intra modes, no additional boundary filtering is required, as required in the case of HEVC DC mode boundary filtering or horizontal / vertical mode edge filtering. The PDPC process for DC mode and planar mode is the same, and clipping operations are avoided. For angular modes, the pdpc scaling factor is adjusted so that no range check is required, and the angle condition for enabling pdpc is removed (using scaling >= 0). In addition, the PDPC weights are based on 32 in all angular mode cases. The PDPC weights depend on the prediction mode, as shown in Table 2. PDPC is applied to blocks with width and height both greater than or equal to 4.

[0146] Fig. 7A A schematic diagram 700 is shown showing the sample definition used by PDPC applied to diagonal top-right intra mode. Figure 7B A diagram 720 showing the definition of samples used by PDPC applied to diagonal bottom-left intra mode. Figure 7C A schematic diagram 740 is shown showing the sample definition used for PDPC applied to the upper right frame mode in the adjacent diagonal. Fig.7D A schematic diagram 760 is shown showing the sample definition used by PDPC applied to the lower left corner frame mode in the adjacent diagonal. 7A to 7D The reference samples (R) of PDPC applied on various prediction modes are shown. x,-1 ,R -1,y and R -1,-1 ). The prediction sample pred(x',y') is located at (x',y') in the prediction block. For example, the reference sample R x,-1 The coordinate x of is given by: x = x' + y' + 1, with reference to sample R -1,y The coordinate y of is similarly given by: y = x' + y' + 1 for the diagonal pattern. For other angular patterns, the reference sample R x,-1 and R -1,y can be located at fractional sample positions. In this case, the sample value of the nearest integer sample position is used.

[0147] Table 2 - Example of PDPC weights according to prediction mode

[0148]

[0149]

[0150] 2.1.1.6. Multiple Reference Line (MRL) Intra Prediction

[0151] Multiple reference line (MRL) intra prediction uses more reference lines for intra prediction. Figure 8 Schematic diagram 800 shows an example of four reference lines of a neighboring prediction block. Figure 8In Figure 1, an example of 4 reference lines is depicted, where the samples of segments A and F are not extracted from the reconstructed neighboring samples, but are filled with the closest samples from segments B and E, respectively. HEVC intra picture prediction uses the nearest reference line (i.e., reference line 0). In MRL, 2 additional lines are used (reference line 1 and reference line 3).

[0152] The index of the selected reference row (mrl_idx) is signaled and used to generate the intra predictor. For reference row indices greater than 0, only additional reference row modes are included in the MPM list, and only the MPM index is signaled and not the rest of the modes. The reference row index is signaled before the intra prediction mode, and if a non-zero reference row index is signaled, the intra prediction mode does not include the planar mode.

[0153] MRL is disabled for the first row of a block within a CTU to prevent the use of extended reference samples outside the current CTU row. In addition, PDPC will be disabled when additional rows are used. For MRL mode, the derivation of DC values ​​in DC intra-frame prediction mode for non-zero reference row indexes is aligned with the derivation of reference row index 0. MRL requires storage of 3 neighboring luma reference rows with a CTU to generate predictions. The downsampling filtering of the Cross Component Linear Model (CCLM) tool also requires 3 neighboring luma reference rows. The definition of MRL using the same 3 rows is consistent with CCLM to reduce the storage requirements of the decoder.

[0154] 2.1.1.7. Intra-frame sub-partitioning (ISP)

[0155] Intra sub-partitioning (ISP) divides the luma intra prediction block into 2 or 4 sub-partitions vertically or horizontally depending on the block size. For example, the minimum block size of ISP is 4×8 (or 8×4). If the block size is larger than 4×8 (or 8×4), the corresponding block will be divided into four sub-partitions. We note that M×128 (M≤64) and 128×N (N≤64) ISP blocks may generate potential problems for 64×64 VDPU. For example, an M×128 CU in the single-tree case has one M×128 luma TB and two corresponding Chroma TB. If the CU uses ISP, then the luma TB will be divided into 4 M×32TBs (can only be split horizontally), each of which is smaller than a 64×64 block. However, in the current ISP design, the chroma blocks are indivisible. Therefore, the size of both chroma components will be larger than a 32×32 block. Similarly, a 128×NCU using ISP can also create a similar situation. Therefore, these two situations are problems for the 64×64 decoder pipeline. Therefore, the CU size that can use ISP is limited to a maximum value of 64×64. Fig. 9A An example diagram 910 of sub-partitioning of 4x8 CUs and 8x4 CUs is shown. Fig. 9B An example diagram 920 shows sub-partitioning of a CU other than 4x8, 8x4, and 4x4. All sub-partitions satisfy the condition of having at least 16 samples.

[0156] In ISP, 1xN / 2xN sub-block prediction is not allowed to depend on the reconstructed values ​​of previously decoded 1xN / 2xN sub-blocks of the codec block, so that the minimum prediction width of the sub-block becomes four samples. For example, an 8xN (N>4) codec block encoded using ISP with vertical partitioning is partitioned into two prediction regions of size 4xN and four transforms of size 2xN. In addition, a 4xN codec block encoded using ISP with vertical partitioning is predicted using a full 4xN block; each of the four transforms is used. Although 1xN and 2xN transform sizes are allowed, it is asserted that the transforms of these blocks in the 4xN region can be performed in parallel. For example, when a 4xN prediction region contains four 1xN transforms, there is no transform in the horizontal direction; the transform in the vertical direction can be performed as a single 4xN transform in the vertical direction. Similarly, when a 4xN prediction region contains two 2xN transform blocks, the transform operations of the two 2xN blocks in each direction (horizontal and vertical) can be performed in parallel. Therefore, processing these smaller blocks does not increase latency compared to processing intra blocks for 4x4 regular codec.

[0157] Table 3 - Entropy coding coefficient group size

[0158] Block size Coefficient group size 1×N, N≥16 1×16 N×1, N≥16 16×1 2×N, N≥8 2×8 N×2, N≥8 8×2 All other possible M×N situations 4×4

[0159] For each sub-partition, a reconstructed sample is obtained by adding the residual signal to the prediction signal. Here, the residual signal is generated by processes such as entropy decoding, inverse quantization, and inverse transformation. Therefore, the reconstructed sample values ​​of each sub-partition can be used to generate a prediction for the next sub-partition, and each sub-partition is processed repeatedly. In addition, the first sub-partition to be processed is the upper left sample containing the CU, and then the sub-partitions continue downward (horizontal partitioning) or to the right (vertical partitioning). Therefore, the reference samples used to generate the sub-partition prediction signal are only located on the left and above the row. All sub-partitions share the same intra-frame mode. The following is a summary of the interaction of ISP with other codec tools.

[0160] -Multiple Reference Lines (MRL): If the MRL index of a block is not 0, the ISP codec mode will be inferred to be 0, so the ISP mode information will not be sent to the decoder.

[0161] - Entropy coding coefficient group size: As shown in Table 3, the size of the entropy coding sub-block has been modified to have 16 samples in all possible cases. It is worth noting that the new size only affects blocks generated by ISP where one dimension is less than 4 samples. In all other cases, the coefficient group maintains 4×4 dimensions.

[0162] -CBF encoding: It is assumed that at least one subpartition has a non-zero CBF. Thus, if n is the number of subpartitions, and the first n-1 subpartitions already yield zero CBF, the CBF of the nth subpartition is inferred to be 1.

[0163] -MPM usage: The MPM flag will be inferred as the one for blocks encoded and decoded via ISP mode, the MPM list is modified to exclude DC mode and prioritize horizontal intra mode for ISP horizontal partitioning and vertical intra mode for ISP vertical partitioning.

[0164] – Transform size restriction: All ISP transforms with length greater than 16 points use DCT-II.

[0165] –PDPC: When the CU uses ISP codec mode, the PDPC filter will not be applied to the resulting sub-partition.

[0166] –MTS flag: If the CU uses ISP codec mode, the MTS CU flag will be set to 0 and will not be sent to the decoder. Therefore, the encoder will not perform RD tests on the different available transforms for each result sub-partition. The transform selection for ISP mode will instead be fixed and will be selected based on the intra mode used, the processing order and the block size. Therefore, no signaling is required. For example, let t H and t V The horizontal and vertical transforms are selected for the w×h sub-partition, where w is the width and h is the height. The transforms are then selected according to the following rules:

[0167] - If w=1 or h=1, there is no horizontal transform or vertical transform, respectively.

[0168] – If w=2 and w>32, t H =DCT-II

[0169] – If h=2 and h>32, t V =DCT-II

[0170] – Otherwise, the transformation is selected as shown in Table 4.

[0171] Table 4 - Transform selection depending on intra mode

[0172]

[0173] In ISP mode, all 67 intra prediction modes are allowed. PDPC is also applied if the corresponding width and height are at least 4 samples long. In addition, the conditions for intra interpolation filter selection no longer exist, and in ISP mode, cubic (DCT-IF) filtering is always used for fractional position interpolation.

[0174] 2.1.1.8. Matrix Weighted Intra Prediction (MIP)

[0175] The matrix weighted intra prediction (MIP) method is an intra prediction technique newly added to VVC. To predict samples of a rectangular block of width W and height H, the matrix weighted intra prediction (MIP) takes a row of H reconstructed neighboring boundary samples on the left side of the block and a row of W reconstructed neighboring boundary samples above the block as input. If the reconstructed samples are not available, they are generated like traditional intra prediction. Fig.10 A schematic diagram 1000 of a matrix-weighted intra prediction process is shown. The generation of the prediction signal is based on three steps, namely, averaging, matrix-vector multiplication, and linear interpolation, as Fig.10 shown.

[0176] Average neighboring samples

[0177] Among the boundary samples, four samples or eight samples are selected by averaging based on the block size and shape. Specifically, input boundary bdry top and bdry left It is reduced to a smaller size by averaging the neighboring boundary samples according to a predefined rule that depends on the size of the block. and Then, the two reduced boundaries and Connect to the reduced boundary vector bdry red , so the size of the reduced boundary vector for blocks of shape 4×4 is four, and the size of the reduced boundary vector for blocks of all other shapes is eight. If mode refers to a MIP mode, this connection is defined as follows:

[0178]

[0179] Matrix Multiplication

[0180] The average sample is taken as input, a matrix-vector multiplication is performed, and then an offset is added. The result is a downscaled prediction signal on a subsampled set of samples in the original block. red Generate a reduced prediction signal pred red, , the reduced prediction signal is of width W red and height H red Here, W red and H red is defined as:

[0181]

[0182]

[0183] The reduced prediction signal pred is calculated by computing the matrix-vector product and adding an offset red :

[0184] pred red =A.bdry red +b.

[0185] Here, A is a matrix, and if W = H = 4, then it has W red ·H red rows and 4 columns, in all other cases there are 8 columns. b is W red ·H red The matrix A and the offset vector b are taken from S0, S1, S 2. One of the sets. The index idx=idx(W,H) is defined as follows:

[0186]

[0187] Here, each coefficient of matrix A is represented with 8 bits of precision. Set S0 consists of 16 matrices Each matrix has 16 rows and 4 columns, and 16 offset vectors Each offset vector has a size of 16. This set of matrices and offset vectors is used for blocks of size 4×4. Set S1 consists of 8 matrices Each matrix has 16 rows and 8 columns, and 8 offset vectors Each offset vector has a size of 16. Set S2 consists of 6 matrices Each matrix has 64 rows and 8 columns, and 6 offset vectors Each offset vector has a size of 64.

[0188] Interpolation

[0189] The prediction signals at the remaining positions are generated from the prediction signals on the subsampled set by linear interpolation, which is a single-step linear interpolation in each direction. The interpolation is performed first in the horizontal direction and then in the vertical direction, regardless of the shape of the block or the size of the block.

[0190] MIP mode signaling and coordination with other codec tools

[0191] For each codec unit (CU) in intra mode, a flag is sent indicating whether the MIP mode is to be applied. If the MIP mode is to be applied, the MIP mode (predModeIntra) is signaled. For the MIP mode, the transposed flag (isTransposed) is used to determine whether the mode is transposed, and the MIP mode identifier (modeId) is used to determine the matrix used for a given MIP mode, which is derived as follows:

[0192] isTransposed=predModeIntra&1

[0193] modeId=predModeIntra>>1 (2-6)

[0194] The MIP codec mode is coordinated with other codecs by taking into account the following aspects:

[0195] – Enable LFNST on MIPs on large blocks. Here the LFNST transform in planar mode is used

[0196] – Reference sample derivation for MIP is performed like in conventional intra prediction mode

[0197] – For the upsampling step used in MIP prediction, the original reference samples are used instead of the downsampled samples

[0198] – Perform clipping before upsampling instead of after upsampling

[0199] – MIP is allowed to be up to 64×64 regardless of the maximum transform size

[0200] – The number of MIP modes is 32 for sizeId=0, 16 for sizeId=1, and 12 for sizeId=2.

[0201] 2.1.2. Inter-frame prediction

[0202] For each inter-predicted CU, the motion parameters include motion vectors, reference picture indices and reference picture list usage indices, as well as additional information required by the new codec features of VVC that will be used for inter-prediction sample generation. The motion parameters can be transmitted through signals in an explicit or implicit manner. When a CU is encoded and decoded in skip mode, the CU is associated with one PU and has no significant residual coefficients, no encoded motion vector increments or reference picture indices. A merge mode is specified, whereby the motion parameters of the current CU are obtained from neighboring CUs, which include spatial candidates and temporal candidates, as well as additional arrangements introduced in VVC. Merge mode can be applied to any inter-predicted CU, not just skip mode. An alternative to merge mode is the explicit transmission of motion parameters, where the motion vectors, the corresponding reference picture indices for each reference picture list and the reference picture list usage flags, as well as other required information, are explicitly transmitted through signals for each CU.

[0203] In addition to the inter-frame coding and decoding functions in HEVC, VVC also includes some new and refined inter-frame prediction coding and decoding tools, as shown below:

[0204] – Extended Consolidated Forecast

[0205] – Merge Mode with MVD (MMVD)

[0206] – Symmetrical MVD (SMVD) signaling

[0207] – Affine motion compensated prediction

[0208] – Sub-block based temporal motion vector prediction (SbTMVP)

[0209] – Adaptive Motion Vector Resolution (AMVR)

[0210] – Motion field storage: 1 / 16 luminance sample MV storage and 8x8 motion field compression

[0211] – Bi-prediction with CU-level weights (BCW)

[0212] – Bidirectional Optical Flow (BDOF)

[0213] –Decoder-side motion vector refinement (DMVR)

[0214] – Geometric Partitioning Mode (GPM)

[0215] – Combined Inter and Intra Prediction (CIIP)

[0216] The text below provides details of those inter prediction methods specified in VVC.

[0217] 2.1.2.1. Extended Merge Prediction

[0218] In VVC, the merge candidate list is constructed by including the following five types of candidates in order:

[0219] 1) Spatial MVP from spatially adjacent CUs

[0220] 2) Time MVP from the same CU

[0221] 3) History-based MVP from FIFO table

[0222] 4) Paired Average MVP

[0223] 5) Zero MV.

[0224] The size of the merge list is signaled in the sequence parameter set header, and the maximum allowed size of the merge list is 6. For each CU code in merge mode, the index of the best merge candidate is encoded and decoded using truncated unary binarization (TU). The first bin of the merge index is encoded and decoded using context, while bypass encoding and decoding is used for other bins.

[0225] The derivation process of merge candidates for each category is provided in this section. As in HEVC, VVC also supports parallel derivation of merge candidate lists for all CUs within a certain size region.

[0226] 2.1.2.2 Spatial Candidate Export

[0227] The derivation of spatial merge candidates in VVC is the same as that in HEVC, except that the positions of the first two merge candidates are swapped. Fig.11 A schematic diagram 1100 showing the locations of spatial merging candidates is shown. Fig.11 Among the candidates at the positions shown, up to four merge candidates are selected. The derivation order is B0, A0, B1, A1 and B2. Position B2 is considered only when one or more CUs at positions B0, A0, B1 and A1 are not available (for example, because it belongs to another slice or tile) or is intra-coded. After the candidate at position A1 is added, a redundancy check is performed on the addition of the remaining candidates, which ensures that candidates with the same motion information are excluded from the list, thereby improving the coding efficiency. In order to reduce computational complexity, not all possible candidate pairs are considered in the mentioned redundancy check. Fig.12 is a diagram 1200 illustrating candidate pairs considered for redundancy checking of spatial merging candidates. Fig.12 The pairs are linked by arrows in , and candidates are added to the list only if the corresponding candidates used for redundancy checking do not have the same motion information.

[0228] 2.1.2.3 Time Candidate Export

[0229] In this step, only one candidate is added to the list. In particular, in the derivation of this temporal merge candidate, the scaled motion vector is derived based on the co-located CU belonging to the co-located reference picture. The reference picture list to be used for deriving the co-located CU is explicitly signaled in the slice header. Fig.13 Schematic diagram 1300 of motion vector scaling for temporal merge candidates is shown. Fig.13 As shown in the dashed line, the scaled motion vector for the temporal merge candidate is obtained, which is scaled from the motion vector of the co-located CU using the POC distances, tb and td, where tb is defined as the POC difference between the reference picture of the current picture and the current picture, and td is defined as the POC difference between the reference picture of the co-located picture and the co-located picture. The reference picture index of the temporal merge candidate is set equal to zero.

[0230] Fig.14 A schematic diagram 1400 is shown of candidate positions for temporal merging of candidates C0 and C1. Fig.14As shown, the position of the temporal candidate is selected between candidates C0 and C1. If the CU at position C0 is not available and is intra-coded, or the CU at position C0 is outside the current row of the CTU, position C1 is used. Otherwise, position C0 is used in the derivation of the temporal merge candidate.

[0231] 2.1.2.4. History-based Merge Candidate Derivation

[0232] History-based MVP (HMVP) merge candidates are added to the merge list after spatial MVP and TMVP. In this method, the motion information of the previous codec block is stored in a table and used as the MVP of the current CU. A table with multiple HMVP candidates is maintained during the encoding / decoding process. The table is reset (cleared) when a new CTU row is encountered. Whenever there is a non-sub-block inter-coded CU, the associated motion information is added as a new HMVP candidate to the last entry of the table.

[0233] The HMVP table size S is set to 6, which indicates that up to 6 history-based MVP (HMVP) candidates can be added to the table. When a new motion candidate is inserted into the table, a constrained first-in-first-out (FIFO) rule is used, where a redundancy check is first applied to find if there is an identical HMVP in the table. If found, the identical HMVP is removed from the table, and then all HMVP candidates are moved forward.

[0234] HMVP candidates can be used in the merge candidate list construction process. The nearest HMVP candidates in the table are checked in order and inserted into the candidate list after the TMVP candidate. Redundancy check is applied to HMVP candidates to spatial or temporal merge candidates.

[0235] To reduce the number of redundant checking operations, the following simplifications are introduced:

[0236] 1. The number of HMPV candidates for merge list generation is set to (N<=4)?M:(8-N), where N indicates the number of existing candidates in the merge list and M indicates the number of available HMVP candidates in the table.

[0237] 2. Once the total number of available merge candidates reaches the maximum allowed merge candidates minus 1, the merge candidate list construction process from HMVP is terminated.

[0238] 2.1.2.5. Pairwise Average Merging Candidate Derivation

[0239] The pairwise average candidate is generated by averaging the predefined candidate pairs in the existing merge candidate list, and the predefined pairs are defined as {(0, 1), (0, 2), (1, 2), (0, 3), (1, 3), (2, 3)}, where the number represents the merge index of the merge candidate list. The average motion vector is calculated separately for each reference list. If two motion vectors are available in one list, they are averaged even if they point to different reference pictures; if only one motion vector is available, it is used directly; if no motion vector is available, this list is kept invalid.

[0240] When the merge list is not full after adding pairwise average merge candidates, zero MVPs will be inserted at the end until the maximum number of merge candidates is encountered.

[0241] 2.1.2.6. Merging estimated regions

[0242] The merge estimation region (MER) allows the merge candidate list to be derived independently for a CU in the same merge estimation region (MER). For generating the merge candidate list for the current CU, candidate blocks in the same MER as the current CU are not included. In addition, the update process for the history-based motion vector prediction candidate list is updated only when (xCb+cbWidth)>>Log2ParMrgLevel is greater than xCb>>Log2ParMrgLevel and (yCb+cbHeight)>>Log2parMrglevel is greater than (yCb>>Log2ParMrgLevel), and where (xCb, yCb) is the top left luma sample position of the current CU in the picture, and (cbWidth, cbHeight) is the CU size. The MER size is selected at the encoder end and signaled in the sequence parameter set as log2_parallel_merge_level_minus2.

[0243] 2.1.3. Merge Mode with MVD (MMVD)

[0244] In addition to the merge mode that uses implicitly derived motion information directly for prediction sample generation of the current CU, a merge mode with motion vector difference (MMVD) is introduced in VVC. The MMVD flag is signaled immediately after the skip flag and merge flag are sent to specify whether the MMVD mode is used for the CU.

[0245] In MMVD, after a merge candidate is selected, it is further refined by the MVD information transmitted by the signal. Further information includes a merge candidate flag, an index specifying the magnitude of motion, and an index indicating the direction of motion. In MMVD mode, one of the first two candidates in the merge list is selected to be used as the MV basis. The merge candidate flag is transmitted by the signal to specify which one to use.

[0246] The distance index specifies the motion magnitude information and indicates a predefined offset from the starting point. Fig.15A A schematic diagram 1500 showing MMVD search points for an L0 reference is shown. Fig. 15B A schematic diagram 1550 of MMVD search points for L1 reference is shown. Figure 15A-15B As shown in Table 5, the offset is added to the horizontal component or the vertical component of the starting MV.

[0247] Table 5: Relationship between distance index and predefined offset

[0248]

[0249] The direction index indicates the direction of the MVD relative to the starting point. The direction index can indicate four directions, as shown in Table 6. Note that the meaning of the MVD symbol can change depending on the information of the starting MV. When the starting MV is a unidirectional prediction MV or a bidirectional prediction MV, where both lists point to the same side of the current picture (i.e., both referenced POCs are greater than the POC of the current picture or both are less than the POC of the current picture), the symbol in Table 6 specifies the sign of the MV offset added to the starting MV. When the starting MV is a bidirectional prediction MV, where the two MVs point to different sides of the current picture (i.e., one referenced POC is greater than the POC of the current picture and the other referenced POC is less than the POC of the current picture), the symbol in Table 6 specifies the sign of the MV offset added to the list0 MV component of the starting MV, and the sign of the list1 MV has the opposite value.

[0250] Table 6: Sign of MV offset specified by direction index

[0251] Direction Index 00 01 10 11 x-axis + – N / A N / A y-axis N / A N / A + –

[0252] 2.1.3.1. Bidirectional Prediction with CU-Level Weights (BCW)

[0253] In HEVC, a bidirectional prediction signal is generated by averaging two prediction signals obtained from two different reference pictures and / or using two different motion vectors. In VVC, the bidirectional prediction mode is extended beyond simple averaging to allow weighted averaging of the two prediction signals.

[0254] Pbi-pred =((8-w)*P0+w*P1+4)>>3 (2-7)

[0255] Five weights are allowed in weighted average bidirectional prediction, w∈{-2,3,4,5,10}. For each bidirectional prediction CU, the weight w is determined in one of two ways: 1) For non-merged CUs, the weight index is signaled after the motion vector difference; 2) For merged CUs, the weight index is inferred from neighboring blocks based on the merge candidate index. BCW is only applied to CUs with 256 or more luma samples (i.e., the CU width multiplied by the CU height is greater than or equal to 256). For low-latency pictures, all 5 weights will be used. For non-low-latency pictures, only 3 weights (w∈{3,4,5}) are used.

[0256] – At the encoder, a fast search algorithm is applied to find the weight index without significantly increasing the encoder complexity. These algorithms are summarized below. The VTM software and documentation can be referenced for further details. When combined with AMVR, the unequal weights of 1-pixel and 4-pixel motion vector accuracy are only conditionally checked if the current picture is a low-latency picture.

[0257] - When combined with affine, affine ME will be performed for unequal weights if and only if the affine mode is selected as the current best mode.

[0258] – When the two reference pictures in bidirectional prediction are the same, unequal weights are only checked conditionally.

[0259] – Unequal weights are not searched when certain conditions are met, which depend on the POC distance between the current picture and its reference pictures, the codec QP, and the temporal level.

[0260] The BCW weight index is encoded using one context encoded bit followed by a bypass encoded bit. The first context encoded bit indicates whether equal weights are used; and if unequal weights are used, an additional bit is signaled using the bypass codec to indicate which unequal weights are used.

[0261] Weighted prediction (WP) is a codec tool supported by the H.264 / AVC and HEVC standards for efficient encoding and decoding of video content in fading conditions. Support for WP has also been added to the VVC standard. WP allows weighting parameters (weights and offsets) to be transmitted by signal for each reference picture list L0 and each reference picture in list L1. Then, during motion compensation, the weights and offsets of the corresponding reference pictures are applied. WP and BCW are designed for different types of video content. In order to avoid interaction between WP and BCW (which will complicate the VVC decoder design), if the CU uses WP, the BCW weight index is not transmitted by signal, and w is inferred to be 4 (ie, equal weights are applied). For the merged CU, the weight index is inferred from the neighboring blocks based on the merge candidate index. This can be applied to the normal merge mode and the inherited affine merge mode. For the constructed affine merge mode, affine motion information is constructed based on the motion information of up to 3 blocks. The BCW index of the CU using the constructed affine merge mode is simply set equal to the BCW index of the first control point MV.

[0262] In VVC, CIIP and BCW cannot be applied jointly to a CU. When a CU is encoded and decoded using CIIP mode, the BCW index of the current CU is set to 2, for example, equal weight.

[0263] 2.1.3.2. Bidirectional Optical Flow (BDOF)

[0264] The Bidirectional Optical Flow (BDOF) tool is included in VVC. BDOF, formerly known as BIO, is included in JEM. Compared to the JEM version, BDOF in VVC is a simpler version that requires much less computation, especially in terms of the number of multiplications and the size of the multipliers.

[0265] BDOF is used to refine the bidirectional prediction signal of a CU at the 4×4 sub-block level. BDOF is applied to a CU if all of the following conditions are met:

[0266] – The CU is encoded using “true” bi-prediction mode, i.e. one of the two reference pictures precedes the current picture in display order, and the other of the two reference pictures follows the current picture in display order

[0267] – The distances from the two reference images to the current image (i.e., POC difference) are the same

[0268] – Both reference images are short-term reference images.

[0269] –CU is not encoded or decoded using affine mode or ATMVP merge mode

[0270] –CU has more than 64 luma samples

[0271] – CU height and CU width are both greater than or equal to 8 luma samples

[0272] –BCW weight index indicates equal weights

[0273] – Current CU does not have WP enabled

[0274] –CIIP mode is not used for the current CU

[0275] BDOF is applied only to the luma component. As its name suggests, BDOF mode is based on the concept of optical flow, which assumes that the motion of objects is smooth. For each 4×4 sub-block, motion refinement (v x ,v y ) is calculated by minimizing the difference between L0 prediction samples and L1 prediction samples. Then motion refinement is used to adjust the bi-prediction sample values ​​in the 4x4 sub-block. The following steps are applied in the BDOF process.

[0276] First, by directly calculating the difference between two adjacent samples, the horizontal and vertical gradients of the two prediction signals, and k=0,1,is calculated, that is,

[0277]

[0278]

[0279] Among them I (k) (i, j) is the sample value at the prediction signal coordinate (i, j) in list k, k=0,1, and shift1 is calculated based on the luma bit depth bitDepth as shift1=max(6, bitDepth-6).

[0280] Then, the autocorrelations and cross-correlations of the gradients S1, S2, S3, S5, and S6 are calculated as follows:

[0281]

[0282] in,

[0283]

[0284] where Ω is a 6×6 window around a 4×4 sub-block, and n a and n b The values ​​are set to min(1, bitDepth-11) and min(4, bitDepth-8) respectively.

[0285] Then the cross-correlation and autocorrelation terms are used to refine the motion (v x ,vy ) is exported using:

[0286]

[0287] in th′ BIO =2 max(5,BD-7) . is the round-down (floor) function, and

[0288] Based on the motion refinement and gradients, the following adjustments are computed for each sample in the 4×4 sub-block:

[0289]

[0290] Finally, the BDOF samples of the CU are calculated by adjusting the bidirectional prediction samples as shown below:

[0291] pred BDOF (x,y)=(I (0) (x,y)+I (1) (x,y)+b(x,y)+o offset )>>shift (2-13)

[0292] These values ​​are chosen so that the multipliers in the BDOF process do not exceed 15 bits and the maximum bit width of the intermediate parameters in the BDOF process remains within 32 bits.

[0293] In order to derive the gradient value, some prediction samples I in the list k (k = 0, 1) outside the current CU boundary are (k) (i,j) needs to be generated. Fig.16 FIG. 4 is a schematic diagram showing an extended CU region used in BDOF. Fig.16 As depicted, BDOF in VVC uses an extended row / column around the boundary of the CU. In order to control the computational complexity of generating prediction samples outside the boundary, the prediction samples in the extended area (white positions) are generated by directly taking the reference samples of the nearby integer positions (using the floor() operation on the coordinates) without interpolation, and a conventional 8-tap motion compensated interpolation filter is used to generate the prediction samples within the CU (gray positions). These extended sample values ​​are only used for gradient calculations. For the rest of the steps in the BDOF process, if any sample values ​​and gradient values ​​outside the CU boundary are needed, they are filled (i.e. repeated) from their nearest neighbors.

[0294] When the width and / or height of a CU is greater than 16 luma samples, it will be split into sub-blocks with width and / or height equal to 16 luma samples, and the sub-block boundaries are regarded as CU boundaries in the BDOF process. The maximum unit size of the BDOF process is limited to 16x16. For each sub-block, the BDOF process can be skipped. When the SAD between the initial L0 prediction sample and the L1 prediction sample is less than a threshold, the BDOF process is not applied to the sub-block. The threshold is set equal to (8*W*(H>>1), where W represents the sub-block width and H represents the sub-block height. In order to avoid the additional complexity of SAD calculation, the SAD between the initial L0 prediction sample and the L1 prediction sample calculated in the DVMR process is reused here.

[0295] If BCW is enabled for the current block, that is, the BCW weight index indicates unequal weights, bidirectional optical flow is disabled. Similarly, if WP is enabled for the current block, that is, luma_weight_lx_flag of any of the two reference pictures is 1, BDOF is also disabled; BDOF is also disabled when the CU is encoded and decoded using symmetric MVD mode or CIIP mode.

[0296] 2.1.4. Symmetric MVD encoding and decoding

[0297] In VVC, in addition to conventional unidirectional prediction mode MVD signaling and bidirectional prediction mode MVD signaling, symmetric MVD mode is applied for bidirectional prediction MVD signaling. In symmetric MVD mode, motion information including reference picture indexes of both list 0 and list 1 and MVD of list 1 is not transmitted by signal but derived.

[0298] The decoding process of the symmetric MVD mode is as follows:

[0299] 1) At the stripe level, the variables BiDirPredFlag, RefIdxSymL0 and RefIdxSymL1 are derived as follows:

[0300] – If mvd_l1_zero_flag is 1, BiDirPredFlag is set equal to 0.

[0301] – Otherwise, if the nearest reference picture in list 0 and the nearest reference picture in list 1 form a forward and backward reference picture pair or a backward and forward reference picture pair, then BiDirPredFlag is set to 1, and both the list 0 reference picture and the list 1 reference picture are short-term reference pictures. Otherwise, BiDirPredFlag is set to 0.

[0302] 2) At CU level, if the CU is bidirectionally predicted and BiDirPredFlag is equal to 1, the symmetric mode flag indicating whether the symmetric mode is used is explicitly signaled.

[0303] When the symmetric mode flag is true, only mvp_l0_flag, mvp_l1_flag and MVD0 are explicitly signaled. The reference indexes of list 0 and list 1 are set equal to the reference picture pair, respectively. MVD1 is set equal to (-MVD0). The final motion vector is shown in the following formula.

[0304]

[0305] Fig.17 Schematic diagram of symmetric MVD mode. In the encoder, symmetric MVD motion estimation starts with initial MV evaluation. A set of initial MV candidates includes MVs obtained from unidirectional prediction search, MVs obtained from bidirectional prediction search, and MVs from AMVP lists. For example, there is a list 0 reference picture 1710 and a list 1720 for the current picture 1720.

[0306] 1 Refer to Figure 1730. The one with the lowest distortion cost is selected as the one for symmetry

[0307] Initial MV for MVD motion search.

[0308] 2.1.5. Decoder-side Motion Vector Refinement (DMVR)

[0309] To improve the accuracy of merge mode MV, decoder-side motion vector refinement based on bilateral matching is applied in VVC. In bidirectional prediction operation, the refined MV is searched around the initial MV in reference picture list L0 and reference picture list L1. The BM method calculates the distortion between two candidate blocks in reference picture list L0 and list L1. Fig.18 Decoding side motion vector refinement is shown. Fig.18 As shown, based on each MV candidate around the initial MV, the SAD between block 1810 and block 1812 is calculated, where for the current picture 1802, block 1810 is in the reference picture 1801 in list L0, and block 1812 is in the reference picture 1803 in list L1. The MV candidate with the lowest SAD becomes the refined MV and is used to generate a bidirectional prediction signal.

[0310] In VVC, DMVR can be applied to CUs that are coded or decoded using the following modes and functions:

[0311] – CU-level merge mode with bi-predictive MV

[0312] – One reference picture is in the past and the other is in the past relative to the current picture.

[0313] Future – The distances from the two reference images to the current image (i.e., POC differences) are the same

[0314] – Both reference images are short-term reference images

[0315] –CU has more than 64 luma samples

[0316] – CU height and CU width are both greater than or equal to 8 luma samples

[0317] –BCW weight index indicates equal weights

[0318] – The current block is not WP enabled

[0319] –CIIP mode is not used for the current block

[0320] The refined MV derived by the DMVR process is used to generate inter-frame prediction samples and is also used for temporal motion vector prediction for future picture coding. The original MV is used in the deblocking process and is also used for spatial motion vector prediction for future CU coding.

[0321] Additional features of DMVR are mentioned in the following sub-clauses.

[0322] 2.1.5.1. Search solution

[0323] In DVMR, the search point is around the initial MV, and the MV offset obeys the MV difference mirror rule. In other words, any point examined by DMVR represented by the candidate MV pair (MV0, MV1) follows the following two equations:

[0324] MV0′=MV0+MV_offset (2-15)

[0325] MV1′=MV1-MV_offset (2-16)

[0326] Wherein, MV_offset represents the refinement offset between the initial MV and the refinement MV in one of the reference pictures. The refinement search range is two integer luma samples starting from the initial MV. The search includes an integer sample offset search phase and a fractional sample refinement phase.

[0327] The integer sample offset search uses a 25-point full search. First, the SAD of the initial MV pair is calculated. If the SAD of the initial MV pair is less than the threshold, the integer sample stage of DMVR is terminated. Otherwise, the SAD of the remaining 24 points is calculated and checked in raster scan order. The point with the smallest SAD is selected as the output of the integer sample offset search stage. In order to reduce the impact of DMVR refinement uncertainty, it is proposed to support the original MV in the DMVR process. The SAD between the reference blocks referenced by the initial MV candidates is reduced by 1 / 4 of the SAD value.

[0328] The integer sample search is followed by fractional sample refinement. To save computational complexity, fractional sample refinement is derived using the parametric error surface equation instead of an additional search using SAD comparison. Fractional sample refinement is conditionally called based on the output of the integer sample search stage. When the integer sample search stage ends with a center with minimum SAD in the first or second iteration search, fractional sample refinement is further applied.

[0329] In the sub-pixel offset estimation based on the parameter error surface, the cost of the center position and the costs of the four neighboring positions from the center are used to fit a two-dimensional parabolic error surface equation of the following form

[0330] E(x,y)=A(x min ) 2 +B(yy min ) 2 +C (2-17)

[0331] Where (x min ,y min ) corresponds to the fractional position with the minimum cost, and C corresponds to the minimum cost value. By solving the above equation using the cost values ​​of the five search points, (x min ,y min ) is calculated as:

[0332] x min =(E(-1,0)-E(1,0)) / (2(E(-1,0)+E(1,0)-2E(0,0))) (2-18)

[0333] y min =(E(0,-1)-E(0,1)) / (2((E(0,-1)+E(0,1)-2E(0,0))) (2-19)

[0334] x min and min The value of is automatically clamped between -8 and 8, since all cost values ​​are positive, and the minimum value is E(0,0). This corresponds to a half-pixel offset with 1 / 16-pixel MV precision in VVC. The calculated fraction (xmin ,y min ) is added to the integer distance refinement MV to obtain a sub-pixel accurate refinement delta MV.

[0335] 2.1.5.2. Bilinear interpolation and sample padding

[0336] In VVC, the resolution of MV is 1 / 16 luma sample. Samples at fractional positions are interpolated using an 8-tap interpolation filter. In DMVR, the search points are around the initial fractional pixel MV with integer sample offsets, so samples at these fractional positions need to be interpolated for the DMVR search process. In order to reduce computational complexity, a bilinear interpolation filter is used to generate fractional samples for the search process in DMVR. Another important effect is that by using a bilinear filter, DVMR does not access more reference samples than the normal motion compensation process within a 2-sample search range. After the refined MV is obtained through the DMVR search process, an ordinary 8-tap interpolation filter is applied to generate the final prediction. In order not to access more reference samples for the normal MC process, samples will be filled from those available samples, which are not required for the interpolation process based on the original MV, but are required for the interpolation process based on the refined MV.

[0337] 2.1.5.3. Maximum DMVR processing unit

[0338] When the width and / or height of a CU is greater than 16 luma samples, it will be further split into sub-blocks with width and / or height equal to 16 luma samples. The maximum unit size of the DMVR search process is limited to 16x16.

[0339] 2.1.6. Combined Inter and Intra Prediction (CIIP)

[0340] In VVC, when a CU is encoded and decoded in merge mode, if the CU contains at least 64 luma samples (i.e., the CU width multiplied by the CU height is equal to or greater than 64), and if both the CU width and the CU height are less than 128 luma samples, an additional flag is transmitted by signaling to indicate whether the combined inter / intra prediction (CIIP) mode is applied to the current CU. Fig.19 A schematic diagram 1900 is shown of the top and left neighboring blocks used in the CIIP weight derivation. As its name suggests, CIIP prediction combines the inter prediction signal with the intra prediction signal. The inter prediction signal P in CIIP mode inter is derived using the same inter prediction process applied to the normal merge mode; and the intra prediction signal P is derived after the normal intra prediction process for the planar mode intraThen, the intra and inter prediction signals are combined using weighted averaging, where the weight values ​​are calculated depending on the coding mode of the top and left neighboring blocks as follows ( Fig.19 Depicted):

[0341] - If the top neighbor is available and intra-coded, set isIntraTop to 1, otherwise set isIntra top to 0;

[0342] - If the left neighbor is available and intra-frame decoding is performed, isIntraLeft is set to 1, otherwise isIntralLeft is set to 0;

[0343] -If (isIntraLeft+isIntraTop) is equal to 2, then wt is set to 3;

[0344] - Otherwise, if (isIntraLeft + isIntraTop) is equal to 1, then wt is set to 2;

[0345] - Otherwise, set wt to 1.

[0346] The CIIP forecast is constructed as follows:

[0347] P CIIP =((4-wt)*P inter +wt*P intra +2)>>2 (2-20)

[0348] 2.1.7. Geometric Partitioning Model (GPM)

[0349] In VVC, geometric partitioning mode is supported for inter prediction. The geometric partitioning mode is signaled using a CU level flag as a merge mode. Other merge modes include normal merge mode, MMVD mode, CIIP mode, and sub-block merge mode. For each possible CU size w×h=2 m ×2 n , where m,n∈{3…6} and excluding 8x64 and 64x8, the geometric partitioning mode supports a total of 64 partitions.

[0350] Fig. 20 Schematic diagram 2000 shows an example of GPM segmentation grouped by the same angle. When this mode is used, the CU is segmented into two parts by a geometrically positioned straight line ( Fig. 20). The position of the partition line is mathematically derived from the angle and offset parameters of the specific partition. Each part of the geometric partition in the CU uses its own motion for inter-frame prediction; each partition only allows unidirectional prediction, that is, each part has a motion vector and a reference index. Unidirectional prediction motion constraints are applied to ensure that only two motion compensated predictions are required for each CU, which is the same as traditional bidirectional prediction.

[0351] If geometric partitioning mode is used for the current CU, a geometric partitioning index indicating the partitioning mode (angle and offset) of the geometric partitioning and two merge indices (one for each partition) are further transmitted through the signal. The number of maximum GPM candidate sizes is explicitly transmitted through the signal in the SPS, and the syntax binarization for the GPM merge index is specified. After predicting each part of the geometric partitioning, a hybrid process with adaptive weights is used to adjust the sample values ​​along the edges of the geometric partitioning. This is the prediction signal for the entire CU, and the transformation and quantization processes will be applied to the entire CU as in other prediction modes. Finally, the motion field of the CU predicted using the geometric partitioning mode is stored.

[0352] 2.1.7.1. One-way prediction candidate list construction

[0353] The unidirectional prediction candidate list is derived directly from the merge candidate list constructed according to the extended merge prediction process. Denote n as the index of the unidirectional prediction motion in the geometric unidirectional prediction candidate list 2110. The LX motion vector of the nth extended merge candidate is used as the nth unidirectional prediction motion vector of the geometric partitioning mode, with X equal to the parity of n. Fig.21 is a schematic diagram 2100 showing unidirectional prediction MV selection for geometric partitioning mode. These motion vectors are Fig.21 If the corresponding LX motion vector of the n-th extended merge candidate does not exist, the L(1-X) motion vector of the same candidate is used instead as the unidirectional prediction motion vector of the geometric partitioning mode.

[0354] 2.1.7.2. Blending along geometric partition edges

[0355] After predicting each part of the geometric partition using its own motion, blending is applied to the two prediction signals to derive samples around the geometric partition edges. The blending weights for each position of the CU are derived based on the distance between the individual position and the partition edge.

[0356] The distance from position (x,y) to the partition edge is derived as:

[0357]

[0358]

[0359]

[0360]

[0361] where i,j are the angle and offset indices of the geometric partition, which depend on the geometric partition index transmitted by the signal. x,j and ρ y,j The sign of depends on the angle index i.

[0362] The weight of each part of the geometric partition is derived as follows:

[0363] wIdxL(x,y)=partIdx? 32+d(x,y):32-d(x,y) (2-25)

[0364]

[0365] w1(x,y)=1-w0(x,y) (2-27)

[0366] partIdx depends on the angle index i. Fig. 22 A schematic diagram 2200 of mixing weights w0 using a set partitioning pattern is shown.

[0367] 2.1.7.3. Motion Field Storage for Geometry Partitioning Patterns

[0368] Mv1 from the first part of the geometric partitioning, Mv2 from the second part of the geometric partitioning, and the combination Mv of Mv1 and Mv2 are stored in the motion field of the CU encoded in the geometric partitioning mode.

[0369] The type of motion vector stored for each individual position in the motion field is determined as:

[0370] sType=abs(motionIdx)<32?2:(motionIdx≤0?(1-partIdx):partIdx) (2-43)

[0371] Where motionIdx is equal to d(4x+2,4y+2), which is recalculated according to equation (2-36). partIdx depends on the angle index i.

[0372] If sType is equal to 0 or 1, then Mv0 or Mv1 is stored in the corresponding motion field, otherwise, if sTType is equal to 2, the combined Mv from Mv0 and Mv2 is stored. The combined Mv is generated using the following process:

[0373] 1) If Mv1 and Mv2 are from different reference picture lists (one from L0 and the other from L1), then Mv1 and Mv2 are simply combined to form a bidirectional prediction motion vector.

[0374] 2) Otherwise, if Mv1 and Mv2 are from the same list, only the unidirectional predicted motion Mv2 is stored.

[0375] 2.1.8 Geometric Partitioning Mode (GPM) with Merged Motion Vector Difference (MMVD)

[0376] GPM in VVC is extended by applying motion vector refinement on top of the existing GPM unidirectional MV. A flag is first signaled for the GPMCU to specify whether this mode is used. If this mode is used, each geometric partition of the GPM CU can further decide whether to signal MVD. If MVD is signaled for the geometric partition, the motion of the partition is further refined by the signaled MVD information after selecting the GPM merge candidate. All other procedures are the same as in GPM.

[0377] Similar to MMVD, MVD is represented by a pair of distance and direction. There are 9 candidate distances (1 / 4 pixel, 1 / 2 pixel, 1 pixel, 2 pixels, 3 pixels, 4 pixels, 6 pixels, 8 pixels, 16 pixels) and 8 candidate directions (four horizontal directions / vertical directions and four diagonal directions) involving GPM and MMVD (GPM-MMVD). In addition, when pic_fpel_mmvd_enabled_flag is equal to 1, MVD is shifted left by 2 like MMVD.

[0378] 2.1.9 Geometric Partitioning Pattern (GPM) with Template Matching (TM)

[0379] Template matching is applied to GPM. When GPM mode is enabled for a CU, a CU level flag is signaled to indicate whether TM is applied to two geometric partitions. TM is used to refine the motion information of each geometric partition. When TM is selected, the template is constructed using the left, top, or left and top neighboring samples according to the partition angle, as shown in Table 7. In Table 7, A represents the use of the top sample, L represents the use of the left sample, and L+A represents the use of the left and top samples. Then, the motion is refined by minimizing the difference between the current template and the template in the reference picture using the same search pattern with the merge mode of the half-pixel interpolation filter disabled.

[0380] Table 7 Templates for the first and second geometric partitions

[0381]

[0382] The GPM candidate list is constructed as follows:

[0383] 1. Interleaved list-0MV candidates and list-1MV candidates are directly derived from the regular merge candidate list, where list-0MV candidates have higher priority than list-1MV candidates. A pruning method based on an adaptive threshold of the current CU size is applied to remove redundant MV candidates.

[0384] 2. Interleaved list-1 MV candidates and list-0 MV candidates are further derived directly from the regular merge candidate list, where list-1 MV candidates have higher priority than list-0 MV candidates. The same pruning method as the adaptive threshold is also applied to remove redundant MV candidates.

[0385] 3. Zero MV candidates will be filled until the GPM candidate list is full.

[0386] GPM-MMVD and GPM-TM are enabled for only one GPM CU. This is done by first signaling the GPM-MMVD syntax. When both GPM-MMVD control flags are equal to false (i.e., GPM-MMVD is disabled for both GPM partitions), a GPM-TM flag is signaled to indicate whether template matching is applied to both GPM partitions. Otherwise (at least one GPM-MMVD flag is equal to true), the value of the GPM-TM flag is inferred to be false.

[0387] 2.1.10 Multiple Hypothesis Prediction (MHP)

[0388] The present disclosure adopts multi-hypothesis prediction. In the inter-frame AMVP mode, the conventional merge mode and the MMVD mode, at most two additional predictors are transmitted through the signal. The obtained overall prediction signal is iteratively accumulated using each additional prediction signal.

[0389] p n+1 =(1-α n+1 ) n +α n+1 h n+1

[0390] The weight factor α is specified according to the following table:

[0391] add_hyp_weight_idx α 0 1 / 4 1 -1 / 8

[0392] For inter-AMVP mode, MHP is applied only when unequal weights in BCW are selected in bi-prediction mode.

[0393] 2.1.11 Decoder-side Intra Mode Derivation (DIMD)

[0394] Three angular modes are selected from the Histogram of Gradients (HoG) computed from the neighboring pixels of the current block. Once the three modes are selected, their predictors are calculated normally and then their weighted average is used as the final predictor for the block. To determine the weights, the corresponding magnitude in the HoG is used for each of the three modes. The DIMD mode is used as an alternative prediction mode and is always checked in FullRD mode.

[0395] The current version of DIMD has been modified in terms of signaling, HoG calculation, and prediction fusion. The purpose of this modification is to improve the codec performance as well as address the complexity issues raised during the last meeting (i.e., throughput of 4x4 blocks). The following sections describe the modifications in each aspect.

[0396] 2.1.11.1 Signals

[0397] Fig.23 A schematic diagram 2300 of the proposed intra-block decoding process is shown. Fig.23 Shown is the order of parsing flags / indexes in VTM5 integrated with the proposed DIMD.

[0398] It can be seen that the DIMD flag of the block is first parsed using a single CABAC context, which is initialized to a default value of 154.

[0399] If flag == 0, parsing continues normally.

[0400] Otherwise (if flag == 1), only the ISP index is parsed and the following flags / indexes are inferred to be zero: BDPCM flag, MIP flag, MRL index. In this case, the entire IPM parsing is also skipped.

[0401] During the parsing phase, when a regular non-DIMD block queries the IPM of its DIMD neighbors, the pattern PLANAR_IDX is used as a virtual IPM for the DIMD block.

[0402] 2.1.11.2 Texture Analysis

[0403] Fig.24 A schematic diagram 2400 of HoG calculation from a template with a width of 3 pixels is shown. Texture analysis of DIMD includes the calculation of the histogram of gradients (HoG) (e.g. Fig.24 ). HoG calculation is performed by applying horizontal and vertical Sobel filters on pixels in a template of width 3 around the block. However, if the above template pixels fall into different CTUs, they will not be used for texture analysis.

[0404] Once the calculation is complete, the IPM corresponding to the two highest histogram bins is selected for the block.

[0405] In previous versions, all pixels on the line in the template were involved in the HoG calculation. However, the current version improves the throughput of this process by applying the Sobel filter more sparsely on 4x4 blocks. For this purpose, only one pixel from the left and one pixel from above are used. This is done in Fig.24 is shown in .

[0406] Besides reducing the number of operations for gradient computation, this property also simplifies the process of selecting the best 2 patterns from the HoG, since the resulting HoG cannot have more than two non-zero magnitudes.

[0407] 2.1.11.3 Prediction Fusion

[0408] This method uses a fusion of three predictors for each block. However, the choice of prediction mode is different and a combined hypothesis intra prediction method is used, where the planar mode is considered to be combined with other modes when computing the intra prediction candidates. In the current version, the two IPMs corresponding to the two highest HoG stripes are combined with the planar mode.

[0409] The prediction fusion is applied as a weighted average of the three predictors mentioned above. For this purpose, the weight of the plane is fixed to 21 / 64 (~1 / 3). Then, the remaining 43 / 64 (~2 / 3) weight is shared between the two HoG IPMs, proportional to the amplitude of their HoG stripes. Fig.25 This process is shown visually.

[0410] Fig.25 An illustrative process 2500 of prediction fusion via weighted averaging of two HoG modes and planes is shown.

[0411] 2.1.12 Template-based Intra Mode Derivation (TIMD)

[0412] The TIMD mode is derived from the MPM using neighboring templates. The TIMD mode is used as an additional intra prediction method for the CU.

[0413] 2.1.12.1 TIMD Mode Export

[0414] For each intra prediction mode in the MPM, the SATD between the prediction and reconstructed samples of the template is calculated. The intra prediction mode with the smallest SATD is selected as the TIMD mode and used for intra prediction of the current CU. Position-dependent intra prediction combining (PDPC) is included in the derivation of the TIMD mode.

[0415] 2.1.12.2TIMD Signaling

[0416] A flag is signaled in the sequence parameter set (SPS) to enable / disable the proposed method. When the flag is true, a CU level flag is signaled to indicate whether the proposed TIMD method is used. The TIMD flag follows the MIP flag. If the TIMD flag is equal to true, the remaining syntax elements related to the luma intra prediction mode, including the MRL, ISP, and normal parsing stages of the luma intra prediction mode, are skipped.

[0417] 2.1.12.3 Modification of MPM list construction in TIMD mode export

[0418] During the construction of the MPM list, the intra prediction modes of neighboring blocks are derived as planes when they are inter-coded. To improve the accuracy of the MPM list, the propagated intra prediction modes are derived using motion vectors and reference pictures when neighboring blocks are inter-coded and used for the construction of the MPM list. This modification only applies to the export of TIMD mode.

[0419] 3. Question

[0420] There are several problems in the existing video coding and decoding technology, which need to be further improved to obtain higher coding and decoding gain.

[0421] (1) The combination of multiple hypothesis prediction (such as CIIP, MHP, etc.) and other encoding and decoding tools needs to be carefully designed.

[0422] (2) The coding and decoding methods used to generate the combination of multiple hypothesis prediction blocks need to be carefully designed.

[0423] (3) Currently, three types of motion will be stored for GPM mode coded blocks, where combined motion is stored for sub-blocks located in mixed regions. Furthermore, if a GPM block contains both inter and intra subdivisions / regions, the storage of intra mode information is not defined.

[0424] 4. Embodiments of the present disclosure

[0425] The following embodiments of the present disclosure should be considered as examples to explain the general concept. These disclosures should not be interpreted in a narrow manner. In addition, these disclosures can be combined in any way.

[0426] The term “video unit” or “codec unit” or “block” may refer to a codec tree block (CTB), a codec tree unit (CTU), a codec block (CB), CU, PU, ​​TU, PB, TB.

[0427] In the present disclosure, regarding “blocks encoded and decoded using mode N”, the “mode N” here can be a prediction mode (for example, MODE_INTRA, MODE_INTER, MODE_PLT, MODE_IBC, etc.) or a coding and decoding technology (for example, AMVP, merge, SMVD, BDOF, PROF, DMVR, AMVR, TM, affine, CIIP, GPM, MMVD, BCW, HMVP, SbTMVP, etc.).

[0428] "Multiple hypothesis prediction" in the present disclosure may refer to any codec that combines / mixes more than one prediction / combination / hypothesis into one for later reconstruction. For example, the combination / hypothesis may be INTER mode codec, INTRA mode codec, or any other codec mode / method, such as CIIP, GPM, MHP, etc.

[0429] In the following discussion, the "base hypothesis" of the multiple hypothesis prediction block may refer to a first hypothesis / prediction having a first set of weight values.

[0430] In the following discussion, an "additional hypothesis" of a multiple hypothesis prediction block may refer to a second hypothesis / prediction having a second set of weight values.

[0431] Composition of multiple hypothesis forecasts

[0432] 1. In one example, mode X may not be allowed to generate hypotheses for a multi-hypothesis prediction block that is encoded using multi-hypothesis prediction mode Y.

[0433] 1) For example, the base hypothesis of a multi-hypothesis prediction block may not be allowed to be encoded or decoded by mode X.

[0434] 2) For example, additional hypotheses of a multi-hypothesis prediction block may not be allowed to be encoded or decoded by mode X.

[0435] 3) For example, for blocks of codec X, it may never signal any block-level codec information related to mode Y.

[0436] 4) For example, X is a palette codec block (eg, PLT mode).

[0437] 5) Alternatively, mode X may be allowed to be used to generate hypotheses for multi-hypothesis prediction blocks that are encoded and decoded using mode Y.

[0438] a) For example, X is a symmetric MVD codec (eg, SMVD) mode.

[0439] b) For example, X is based on a template matching based technique.

[0440] c) For example, X is based on a bilateral matching based technique.

[0441] d) For example, X is a combined intra and inter prediction (eg, CIIP) mode.

[0442] e) For example, X is a geometry partitioning prediction (eg, GPM) mode.

[0443] 6) Mode Y can be CIIP, GPM or MHP.

[0444] 2. CIIP can be used for blocks together with mode X (such as GPM, or MMVD, or Affine).

[0445] 1) In one example, at least one hypothesis in the GPM is generated by CIIP. In other words, at least one hypothesis in the GPM is generated as a weighted sum of at least one inter prediction and one intra prediction.

[0446] 2) In one example, at least one hypothesis in CIIP is generated by GPM. In other words, at least one hypothesis in CIIP is generated as a weighted sum of at least two inter-frame predictions.

[0447] 3) In one example, at least one hypothesis in the CIIP is generated by MMVD.

[0448] 4) In one example, at least one hypothesis in the CIIP is generated by affine prediction.

[0449] 5) In one example, whether mode X can be used with CIIP may depend on codec information such as block dimensions.

[0450] 6) In one example, whether mode X can be used with CIIP can be signaled from the encoder to the decoder.

[0451] a) In one example, signaling can be conditioned by codec information such as block dimensions.

[0452] 3. In one example, one or more hypotheses of a multiple hypothesis prediction block may be generated based on a position-dependent prediction combination (eg, PDPC).

[0453] 1) For example, the hypothesized prediction samples may first be processed through PDPC and then used to generate the multiple hypothesis prediction block.

[0454] 2) For example, a predictor obtained based on PDPC taking into account neighboring sample values ​​can be used to generate a hypothesis.

[0455] 3) For example, predictors obtained based on gradient-based PDPC that takes into account the gradients of neighboring samples can be used to generate hypotheses.

[0456] a) For example, gradient-based PDPC can be applied to the codec assumptions of intra mode (planar, DC, horizontal, vertical or diagonal mode).

[0457] 4) For example, the PDPC predictor may not be based on prediction samples within the current block.

[0458] a) For example, the PDPC predictor may be based only on the prediction (or reconstruction) samples of the neighboring current block.

[0459] b) For example, the PDPC predictor may be based on both the prediction (or reconstructed) samples neighboring the current block and the prediction (or reconstructed) samples inside the current block.

[0460] 4. In one example, a multiple hypothesis prediction block may be generated based on a decoder-side refinement technique.

[0461] 1) For example, decoder-side refinement techniques may be applied to one or more hypotheses of a multi-hypothesis prediction block.

[0462] 2) For example, decoder-side refinement techniques can be applied to multiple hypothesis prediction blocks.

[0463] 3) For example, the decoder side refinement technique can be based on decoder side template matching (e.g., TM), decoder side bilateral matching (e.g., DMVR), or decoder side bidirectional optical flow (e.g., BDOF) or prediction refinement using optical flow (PROF).

[0464] 4) For example, a multi-hypothesis prediction block can be encoded and decoded using CIIP, MHP, GPM or any other multi-hypothesis prediction mode.

[0465] 5) For example, the INTER predicted motion data of the multiple hypothesis blocks (eg, CIIP) may be further refined by decoder side template matching (TM), and / or decoder side bilateral matching (DMVR) and / or decoder side bidirectional optical flow (BDOF).

[0466] 6) For example, the INTER prediction samples of the multiple hypothesis blocks (e.g., CIIP) can be further refined by decoder-side template matching (TM), and / or decoder-side bilateral matching (DMVR), and / or decoder-side bidirectional optical flow (BDOF) or prediction refinement using optical flow (PROF).

[0467] 7) For example, the INTRA prediction part of the multiple hypothesis blocks (e.g., CIIP, MHP, etc.) can be further refined through decoder-side mode derivation (e.g., DIMD), decoder-side intra-frame template matching, etc.

[0468] 8) Refined intra prediction mode / motion information of multiple hypotheses blocks may be disabled to predict subsequent blocks to be encoded / decoded in the same slice / tile / picture / sub-picture.

[0469] 9) Alternatively, the decoder-side refinement technique may not be applied to the multiple hypothesis prediction blocks.

[0470] a) For example, decoder-side refinement techniques may not be allowed for MHP codec blocks.

[0471] 5. For blocks coded with multiple hypothesis prediction (e.g., coded with CIIP, MHP), it is recommended to derive the blocks into multiple sub-blocks / sub-partitions / partitions.

[0472] 1) In one example, multiple sets of motion information may be transmitted / derived via signals.

[0473] a) In one example, for each sub-block / sub-partition / partition, a set of motions may be derived.

[0474] 2) In one example, the final prediction of a sub-block / sub-partition may depend only on a set of motion information associated with it.

[0475] a) Alternatively, the final prediction of a sub-block / sub-partition may only depend on more than one set of motion information associated with it.

[0476] 6. In one example, if a multi-hypothesis prediction unit (e.g., a codec unit) contains more than one sub-block / sub-partition / partition, where the size of each sub-block / sub-partition / partition is smaller than the size of the entire multi-hypothesis prediction unit, the following rules may be applied:

[0477] 1) For example, multiple hypothesis prediction units can be partitioned in a unified manner.

[0478] a) For example, the multi-hypothesis prediction unit can be divided into rectangular or square sub-blocks.

[0479] b) For example, the multi-hypothesis prediction unit may be divided into M×N sub-blocks.

[0480] i. For example, M=N.

[0481] ii. For example, M!=N.

[0482] iii. For example, M=4 or 8 or 16.

[0483] iv. For example, N=4 or 8 or 16.

[0484] v. For example, M is equal to the width of the entire multi-hypothesis prediction unit, and N is less than the height of the entire multi-hypothesis prediction unit.

[0485] vi. For example, M is less than the width of the entire multi-hypothesis prediction unit, and N is equal to the height of the entire multi-hypothesis prediction unit.

[0486] c) For example, the multi-hypothesis prediction unit can be divided into triangular sub-blocks.

[0487] i. For example, the multi-hypothesis prediction unit can be divided into two diagonal triangles.

[0488] 2) For example, the multi-hypothesis prediction units may be divided in a non-uniform / irregular manner.

[0489] a) For example, the multi-hypothesis prediction unit can be divided by a diagonal line or a straight line (eg, GPM division, etc.).

[0490] b) For example, the multiple hypothesis prediction units can be divided by curves.

[0491] 3) For example, whether a sub-block / sub-partition / partition / hypothesis of a multiple hypothesis prediction unit is intra-coded may depend on the partition information of the multiple hypothesis prediction unit.

[0492] a) For example, it may depend on the angle of the dividing line.

[0493] i. For example, which GPM partition is intra-mode coded may depend on the GPM partition mode (or GPM partition angle, or GPM partition distance).

[0494] ii. For example, one or more lookup tables (or mapping tables) may be predefined for the correspondence between a GPM partition mode (or a GPM partition angle, or a GPM partition distance) and which sub-block / sub-partition / partition / hypothesis is intra-coded.

[0495] b) For example, it may depend on the number of neighboring samples (outside the entire MHPU) adjacent to the sub-block / sub-partition / partition / hypothesis (and this also depends on how the MHPU is partitioned).

[0496] 4) For example, in the case where a subblock / subpartition / partition / hypothesis of the entire multi-hypothesis prediction unit is intra-mode coded, what intra mode is allowed for the subblock / subpartition / partition / hypothesis may depend on the partition information.

[0497] a) For example, whether to use horizontal intra mode, vertical intra mode, diagonal intra mode or other intra mode may depend on the partition information of the multi-hypothesis prediction unit.

[0498] b) For example, a predefined set of intra modes may be defined depending on whether above and / or left neighbor samples are available for this sub-block / sub-partition / partition / hypothesis.

[0499] i. For example, when a sub-block / sub-partition / partition / hypothesis has no left neighboring samples outside the entire multiple-hypothesis codec unit but adjacent to the current sub-block / sub-partition / partition / hypothesis (the size of the sub-block / sub-partition / partition / hypothesis partition is smaller than the multiple-hypothesis codec unit), horizontal or near-horizontal intra-frame mode may not be allowed.

[0500] ii. For example, when a sub-block / sub-partition / partition / hypothesis has no upper neighboring samples outside the entire multi-hypothesis coding unit but adjacent to the current sub-block / sub-partition / partition / hypothesis, vertical or near-vertical intra mode may not be allowed.

[0501] c) For example, what intra modes are allowed for GPM partitioning may depend on the GPM partitioning mode (or GPM partitioning angle, or GPM partitioning distance).

[0502] i. For example, a predefined set of intra modes may be defined depending on the GPM partition shape / angle / distance / mode.

[0503] ii. For example, one or more lookup tables (or mapping tables) may be predefined for the correspondence between GPM partition modes (or GPM partition angles, or GPM partition distances) and what intra modes are allowed for intra-coded subblocks / subpartitions / partitions / hypotheses.

[0504] a) For example, at most one intra mode may be allowed for a GPM partition.

[0505] b) For example, a set of predefined intra modes may be allowed for GPM partitioning.

[0506] iii. Alternatively, what intra mode is used for a GPM partition may depend on available neighboring samples outside the entire GPM codec unit but adjacent to the current GPM partition (the size of a GPM partition is smaller than a GPM codec unit).

[0507] a) For example, if a GPM partition has no left neighboring samples but has top neighboring samples adjacent to the current GPM partition, horizontal or nearly horizontal intra mode of left-to-right prediction may be allowed for the current GPM partition.

[0508] b) For example, if a GPM partition has no upper neighboring samples but has left neighboring samples adjacent to the current GPM partition, vertical or nearly vertical intra mode predicted from top to bottom may be allowed for the current GPM partition.

[0509] c) For example, if a GPM partition has neither upper neighboring samples nor neighboring samples adjacent to the current GPM partition, intra mode is not allowed for the current GPM partition.

[0510] i. Optionally, in such a case, specific intra modes other than horizontal / vertical / near-horizontal / near-vertical intra modes may be allowed for the current GPM partition.

[0511] 5) In one example, it is assumed that the prediction unit cannot be divided into sub-blocks / sub-partitions / partitions in a sharp clipping manner. Instead, the weighted values ​​of the prediction samples in the unit can be determined by splitting the sub-blocks / sub-partitions / partitions.

[0512] a) If a unit is partitioned into multiple sub-blocks / sub-partitions / partitions and the prediction samples for each sub-block / sub-partition / partition are derived independently, the unit is partitioned into sub-blocks / sub-partitions / partitions in a sharpness-clipped manner.

[0513] b) If a unit is conceptually divided into multiple sub-blocks / sub-partitions / partitions, but the prediction samples for each sub-block / sub-partition / partition are not independently derived, then the unit is not divided into sub-blocks / sub-partitions / partitions in a sharp clipping manner.

[0514] c) In one example, a first weighted value for a first prediction at a first position in a first sub-block / sub-partition / partition may be greater than a second weighted value for a first prediction at a second position in a second sub-block / sub-partition / partition.

[0515] i. For example, the first prediction may be an intra prediction, the first sub-block / sub-partition / partition may be considered an intra-coded sub-block / sub-partition / partition, and the second sub-block / sub-partition / partition may be considered an intra-coded sub-block / sub-partition / partition.

[0516] d) Alternatively, also in this case, the indication of the partitioning information is no longer signaled.

[0517] 6) In one example, the derivation of weighting values ​​used in multiple hypothesis prediction may depend on whether the hypothesis prediction unit (eg, codec unit) contains more than one sub-block / sub-partition / partition.

[0518] a) In one example, weighting values ​​may be derived over the relative sample positions in each sub-block / sub-partition / partition.

[0519] i. In one example, a first weighting value at a first relative sample position in a first sub-block / sub-partition / partition may be equal to a second weighting value at the same relative sample position in a second sub-block / sub-partition / partition.

[0520] b) Alternatively, weighting values ​​may be derived towards relative sample positions in the entire hypothesized prediction unit.

[0521] c) In one example, different weights may be used for different dimensions of sub-blocks / sub-partitions / partitions.

[0522] 7) The partitioning / weighting values ​​used in a block decoded by multiple hypothesis prediction may depend on the information being decoded, the color components, the color format, etc.

[0523] a) In one example, the chroma components follow the partitioning rules applied to the luma components.

[0524] i. Alternatively, the chroma components have different partitioning rules applied to the luma components.

[0525] b) In one example, the chroma components follow the weighting value derivation rules applied to the luma component.

[0526] i. Alternatively, furthermore, the weighting values ​​applied to the chroma components may be shared / derived from the luma component.

[0527] 8) The above method can also be applied to the bullet points mentioned in bullet point 5.

[0528] CIIP / MHP inter-frame component

[0529] 7. For example, virtual / generated motion data (eg, including motion vectors, prediction directions, reference indices, etc.) may be used for multiple hypothesis prediction (eg, CIIP, MHP, GPM, etc.).

[0530] 1) Virtual / generated motion data can be generated in basic blocks in a basic block manner. For example, a basic block can be a 4×4 block.

[0531] a) In one example, the motion data of a basic block may depend on how hypothesis prediction is performed on the basic block, such as a weighted value on the basic block, a partitioning method on the basic block, motion data of one prediction of multiple hypothesis predictions on the basic block, and the like.

[0532] 2) For example, the prediction direction (L0, L1 or bi) may be derived according to a predefined rule.

[0533] a) For example, if only motion information for L0 can be found among all hypothetical predictions for a basic block, the prediction direction of the basic block may be set to unidirectional prediction L0.

[0534] b) For example, if only motion information for L0 can be found among all hypothetical predictions for the basic block, the prediction direction of the basic block may be set to the unidirectional prediction L1.

[0535] c) For example, if motion information of two directions can be found in all hypothesized predictions of the basic block, the prediction direction of the basic block can be set to bi.

[0536] 3) For example, the virtual / generated motion may be a bi-predicted motion created according to predefined rules.

[0537] a) For example, a virtual / generated BI motion may be constructed from a candidate L0 motion from the first candidate list and a candidate L1 motion from the second candidate list.

[0538] i. For example, the first candidate list and / or the second candidate list may be predefined.

[0539] ii. For example, the first candidate list may be an AMVP candidate list, a MERGE candidate list, a new candidate list constructed based on GPM / AMVP / MERGE candidates, or any other motion candidate list.

[0540] iii. For example, the second candidate list may be a MERGE candidate list, an AMVP candidate list, a new candidate list constructed based on GPM / AMVP / MERGE candidates, or any other motion candidate list.

[0541] iv. Alternatively, the first candidate list is different from the second candidate list.

[0542] v. Alternatively, the first candidate list may be the same as the second candidate list.

[0543] 4) For example, the virtual / generated motion may be a unidirectional predicted motion created according to predefined rules.

[0544] a) For example, a virtual / generated unidirectional motion may be constructed from candidate L0 or L1 motions from the third candidate list.

[0545] i. For example, the third candidate list may be an AMVP candidate list, a MERGE candidate list, a new candidate list constructed based on GPM / AMVP / MERGE candidates, or any other motion candidate list.

[0546] 5) For example, if the L0 / L1 / BI motion is from the MERGE candidate list, the merge candidate index can be transmitted by signaling.

[0547] a) Alternatively, the merge candidate index can be implicitly derived from the decoder derivation method (e.g., based on template matching, or based on bilateral matching, etc.)

[0548] 6) For example, if the L0 / L1 / BI motion is from the AMVP candidate list, the motion vector difference (eg, MVD) may be signaled.

[0549] a) In addition, the AMVP candidate index may be transmitted via a signal.

[0550] i. Alternatively, the AMVP candidate index can be implicitly derived from the decoder derivation method (e.g., based on template matching, or based on bilateral matching, etc.)

[0551] b) Alternatively, the motion vector difference may be derived implicitly from a decoder derivation method (eg based on template matching, or based on bilateral matching, etc.).

[0552] 7) For example, virtual / generated motion data can be used to generate prediction blocks, and the resulting prediction blocks can be used to calculate the final predicted video unit (e.g., multiple hypothesis prediction blocks, new coding modes).

[0553] a) Alternatively, motion / sample refinement can be further applied to the generated prediction block.

[0554] i. For example, motion / sample refinement can be template matching (TM), bilateral matching, decoder-derived motion vector refinement (e.g., DMVR), multi-pass decoder-derived motion vector refinement (e.g., MPDMVR), BODF, PROF, etc.

[0555] 8) For example, the virtual / generated motion data can be used in subsequent procedures, such as the deblocking process.

[0556] 9) For example, virtual / generated motion data may be used to predict motion data in subsequent blocks.

[0557] CIIP / MHP intra-frame component

[0558] 8. For example, the intra portion of a multi-hypothesis prediction block (eg, CIIP, MHP, GPM, etc.) may be determined based on predefined rules.

[0559] 1) For example, the intra portion of a multi-hypothesis prediction block (eg, CIIP, MHP, GPM, etc.) can be derived based on fusion-based intra prediction.

[0560] a) For example, fusion-based intra prediction may refer to predicting blocks mixed from more than one intra mode.

[0561] b) For example, fusion-based intra prediction can be generated from the first X intra modes from a predefined set of intra modes.

[0562] i. For example, the first X (such as X>1) intra modes may be the modes with the lowest cost.

[0563] a) In addition, the cost may be calculated based on a template matching method or a bilateral matching method.

[0564] i. For example, a template matching based approach can be used to sort a set of predefined intra modes and select the best X modes for the intra portion of the multiple hypothesis block.

[0565] b) In addition, the cost may be calculated based on a quality metric (eg, SAD / SATD / MSE, etc.) using information of neighboring samples.

[0566] c) Additionally, the cost can be computed based on the Histogram of Gradients (HoG) from neighboring samples.

[0567] ii. For example, the predefined intra mode set may include planar mode, and / or normal intra mode, and / or intra mode from an MPM list, etc.

[0568] c) For example, the weights used for mixing / fusion of multiple prediction samples may depend on the intra prediction angle / direction.

[0569] i. Alternatively, the weights used for mixing / fusion of multiple prediction samples may depend on the GPM partition mode, and / or the GPM partition angle, and / or the GPM partition distance.

[0570] d) For example, the weights used for mixing / fusion of multiple prediction samples may be block / partition / sub-block based (eg, different blocks / partitions / sub-blocks may have different weights).

[0571] i. Alternatively, the weights used for mixing / fusion of multiple prediction samples may be sample-based (eg, different weights may be assigned to different samples).

[0572] 9. For example, the intra portion of a multi-hypothesis prediction block (eg, CIIP, MHP, GPM, etc.) may be determined based on a decoder-derived method.

[0573] 1) In one example, the decoder may determine an intra prediction mode derivation (DIMD).

[0574] 2) In one example, it can be determined by template-based intra prediction mode derivation (TIMD).

[0575] Weighting factor design and storage

[0576] 10. In one example, when an intra-predicted sample is mixed with another predicted sample (which may be inter-coded, or intra-coded, or predicted from other mixtures), what mixing / fusion weights to use may depend on the codec information.

[0577] 1) For example, the rules for deriving the mixing weights may depend on the predicted patterns of the samples being mixed.

[0578] a) For example, different hypothesis combinations such as "Intra+Intra", "Intra+Inter" or "Inter+Inter" may be different.

[0579] 2) For example, the blending weights of intra and inter / intra may depend on one of the prediction modes of the intra-predicted samples used for blending / fusion.

[0580] 3) For example, more than one set of blending / fusion weights may be defined for a particular fusion method based on what intra-mode is used for a video unit.

[0581] a) For example, different weight sets can be defined based on classification according to intra-frame modes, such as horizontal mode, vertical mode, wide-angle mode, diagonal mode, anti-diagonal mode, intra-frame mode in which samples are predicted from top and left neighboring samples (e.g., intra-frame mode indexes corresponding to angles greater than horizontal, less than 18), intra-frame mode in which samples are predicted from top neighboring samples (e.g., intra-frame mode indexes corresponding to angles less than vertical, with indices greater than 50), intra-frame mode in which samples are predicted from left neighboring samples (e.g., intra-frame mode indices greater than horizontal (e.g., 18) but less than vertical (e.g., 50)), and the like.

[0582] b) For example, the weight setting can be based on the rules of weight definition / classification in existing codec tools (such as PDPC, CIIP, etc.).

[0583] 4) For example, more than one set of blending / fusion weights may be defined for a specific fusion method based on which sub-block / sub-unit the current sample belongs to.

[0584] a) For example, different samples may have different weights.

[0585] b) For example, samples belonging to different sub-blocks may have different weights.

[0586] c) For example, a sub-block may have a non-rectangular shape.

[0587] 5) The weighting values ​​may depend on the color components.

[0588] a) In one example, a weighted value on a first (eg, chrominance) component may be derived based on a corresponding weighted value on a second (eg, luma) component.

[0589] 11. For example, intra mode information (eg, GPM, MHP, CIIP, etc.) of a multiple hypothesis prediction block may be stored on an M×M unit basis (such as M=4, or 8, or 16).

[0590] a) For example, for an M×M unit located at a mixed area, where all sub-blocks / sub-partitions / partitions / hypotheses within the MxM unit are encoded and decoded by INTRA, the intra-frame mode of the sub-block / sub-partition / partition / hypothesis is stored, which may depend on (i) partition information (e.g., partition angle / distance / mode, etc.); (ii) the size of the sub-block / sub-partition / partition / hypothesis; iii) intra-frame mode information; and (iv) a predefined rule.

[0591] b) For example, for an M×M unit located at a mixed area containing intra-coded and inter-coded sub-blocks / sub-partitions / partitions / hypotheses, whether to store motion data or intra-mode information may depend on (i) a predefined rule; (ii) intra-mode information; (iii) inter-frame motion data; (iv) partition information (e.g., partition angle / distance / mode, etc.), and (v) the size of the sub-block / sub-partition / partition / hypothesis.

[0592] c) For example, the above-mentioned M×M unit-based intra-mode storage can be used for multi-prediction modes that divide the coding unit into more than one sub-block / sub-partition / partition (eg, GPM, etc.).

[0593] d) For example, the above-mentioned M×M unit-based intra-mode storage can be used for multi-prediction modes that do not divide the coding unit into sub-blocks / sub-partitions / partitions (e.g., CIIP, MHP, etc.).

[0594] e) For example, the above-mentioned intra mode storage based on M×M units can be used to predict the intra prediction mode in the subsequent block.

[0595] General rights

[0596] 12. Whether and / or how to apply the method disclosed above can be transmitted by a signal at the sequence level / picture group level / picture level / slice level / tile group level, such as in a sequence header / picture header / SPS / VPS / DPS / DCI / PPS / APS / slice header / tile group header.

[0597] 13. Whether and / or how to apply the methods disclosed above can be transmitted by signals in PB / TB / CB / PU / TU / CU / VPDU / CTU / CTU row / slice / tile / sub-picture / other kinds of regions containing more than one sample or pixel.

[0598] Whether and / or how to apply the above disclosed method may depend on the coded information, such as block size, color format, single / dual tree partitioning, color component, slice / picture type.

[0599] In the present disclosure, GPM specifies a prediction method that divides a codec unit into at least two sub-partitions / partitions, and the dividing line can be a slant line or a straight line. In addition, each partition of the GPM video unit can use a separate prediction method (e.g., intra-frame, inter-frame, non-inter-frame, L0 prediction or L1 prediction). Alternatively, at least two intermediate prediction blocks are generated using a separate prediction method, and a final prediction block is generated by a weighted sum of the intermediate prediction blocks, where the weighted value is determined based on the partitioning method. On the other hand, the transformation of the GPM video unit is based on the entire video unit rather than the sub-partitions / partitions. In another example, GPM can generate multiple sets of motion information and the final prediction is based on weighted prediction signals from different sets of motion information; or it can generate a final prediction based on a hybrid prediction method (e.g., intra / inter / palette / IBC).

[0600] GPM intra-intra prediction, GPM intra-inter prediction

[0601] 14. In one example, the storage of the codec information of the intra part of the GPM intra-intra prediction (or GPM intra-inter prediction) may follow the following rules:

[0602] 1) For example, codec information may be stored on an MxN basis.

[0603] a) For example, M = 2 or 4 or 8 luma samples.

[0604] b) For example, M may be equal to a non-dyadic value.

[0605] c) For example, N=M.

[0606] 2) For example, the intra-coded divided codec information storage may be stored based on zero MV.

[0607] 3) For example, intra-coded divided codec information storage may be stored based on a reference index equal to -1.

[0608] 4) For example, intra-coded divided codec information storage may be stored based on a reference index equal to a reference index of a current slice / picture.

[0609] 5) For example, intra-coded divided codec information storage may be stored based on the actual intra prediction mode / angle / direction used to derive intra prediction.

[0610] a) For example, the actual intra prediction mode / angle / direction of the GPM partition may not belong to one of the regular intra mode indices.

[0611] i. For example, the actual intra prediction mode / angle / direction of the GPM partition can be mapped to one of the regular intra mode indexes for codec information storage.

[0612] b) Alternatively, the actual intra prediction mode / angle / direction used to derive the intra prediction may not be stored.

[0613] i. For example, the codec information storage of the intra-coded partition may be based on a default inter-motion (such as zero MV).

[0614] c) Alternatively, the storage of codec information for an intra-coded partition may be based on a default intra-mode (not necessarily an intra-mode index for the partition).

[0615] i. For example, the default intra mode may be planar mode.

[0616] 6) For example, at a mixed area (eg, an intra-inter blending area along a GPM dividing line), whether to store intra-coded information or inter-coded information may be predefined.

[0617] a) In one example, if a sample belongs to a mixed region, more than one weight value in the GPM for the sample is not equal to 0.

[0618] b) For example, intra-coded information may always be stored.

[0619] c) For example, inter-coded information may always be stored.

[0620] d) Alternatively, whether to store intra-frame or inter-frame coded information may depend on division information (division line, division mode index, division angle, division distance, etc.).

[0621] 7) For example, at a blended region (eg, an intra-intra blended region along a GPM dividing line), the coded information of which partition is stored may be predefined.

[0622] a) In one example, if a sample belongs to a mixed region, more than one weight value in the GPM for the sample is not equal to zero.

[0623] b) Alternatively, whether to store the encoded information of the first partition or the second partition may depend on the partition information (partition line, partition mode index, partition angle, partition distance, etc.).

[0624] c) In one example, whether to store the coded information of the first partition or the second partition may depend on two intra prediction modes.

[0625] 8) For example, the stored GPM intra-frame partitioning codec information may be used by subsequent encoded / decoded blocks, such as for MPM list construction of codec units subsequent to the current GPM block.

[0626] 9) For example, the stored GPM intra-frame divided codec information can be used for the deblocking process.

[0627] 15. In one example, a multiple hypothesis prediction block may be generated based on more than one intra prediction.

[0628] 1) For example, multiple hypothesis prediction More than one hypothesis of a block (eg, based on the entire block, or based on sub-blocks / partitions) may be intra predicted.

[0629] a) For example, an MHP block may include more than one intra-coded hypothesis.

[0630] b) For example, the CIIP block may include at least two intra predictions.

[0631] c) For example, both partitions of the GPM block may be intra-mode coded.

[0632] d) For example, the intra modes of two partitions may not be allowed to be the same.

[0633] e) For example, the intra mode of one of the two partitions may be signaled in the bitstream.

[0634] i. For example, other partitioned intra modes can be derived implicitly.

[0635] ii. For example, the intra-mode of the first partition may be excluded from the encoded representation of the second partition.

[0636] f) For example, two divided intra modes may be signaled in the bitstream.

[0637] g) For example, two partitioned intra modes can be derived implicitly.

[0638] h) For example, two intra predictions of two partitions may be weighted mixed.

[0639] i. For example, two intra predictions of a GPM block can be weighted mixed.

[0640] ii. For example, all samples within a partition may have the same weighting factor.

[0641] iii. For example, different samples may have different weighting factors.

[0642] iv. For example, the weighting value may depend on the partitioning method of the GPM block.

[0643] v. For example, the weighting value may depend on at least one intra prediction mode.

[0644] 2) For example, the multi-hypothesis prediction block may be partitioned by one or more slanted or straight partition lines (eg, GPM partition lines).

[0645] a) For example, the partitioning mode (angle, direction, partitioning mode index) can be signaled in the bitstream.

[0646] b) For example, the partitioning mode may be signaled in the same manner as the GPM partitioning mode index.

[0647] c) For example, the partitioning mode can be implicitly derived based on codec information.

[0648] 3) For example, one or more syntax elements (eg, flags) may be signaled to indicate at the decoder side whether intra prediction for a certain (GPM) partition is derived.

[0649] a) For example, a CU-based flag may be signaled for the entire block.

[0650] b) For example, a partition-based flag may be signaled for a certain partition of a block.

[0651] c) For example, the intra prediction derived by the decoder may be DIMD or TIMD, etc.

[0652] 16. In one example, multiple hypothesis prediction blocks may be allowed for P slices / pictures where only the L0 reference list is available, and / or B slices / pictures where both the L0 reference list and the L1 reference list are available.

[0653] 1) For example, GPM may be allowed for P slices / pictures.

[0654] 2) For example, the two partitions of a GPM block may be intra prediction and inter prediction (aka GPM intra-inter block).

[0655] a) For example, inter prediction may be L0 prediction or L1 prediction.

[0656] b) For example, the partitioned intra-mode may be predefined or signaled.

[0657] 3) For example, the two partitions of a GPM block may be one intra prediction and another intra prediction.

[0658] a) For example, the intra modes of two partitions may not be allowed to be the same.

[0659] 4) For example, the two partitions of a GPM block may be one inter prediction and another inter prediction.

[0660] a) For example, a GPM block may include two L0 predictions.

[0661] b) For example, a GPM block may include two L1 predictions.

[0662] c) For example, the motion information of two predictions (eg, merge index, motion vector, reference index, etc.) may not be allowed to be the same.

[0663] d) For example, when two partitions are predicted from the same prediction direction (eg, L0 or L1), the motion vectors of the two partitions may be added together or averaged for mixed region motion storage.

[0664] i. For example, if the prediction direction and reference index of two partitions are the same, the motion vectors of the two partitions can be directly added or averaged for motion storage of the mixed region.

[0665] ii. For example, if the prediction directions of the two partitions are the same but the reference indices are different, the motion storage of the mixed region can be based on a motion vector scaling process.

[0666] e) For example, when two partitions are predicted from the same prediction direction (eg, L0 or L1), the motion vector of the partition with the smaller reference index may be stored.

[0667] f) For example, when two partitions are predicted from the same prediction direction (eg, L0 or L1), the motion vector of the partition with the smaller |MVx|+|MVy| may be stored.

[0668] 5) For example, the GPM candidate list can be constructed based on regular merging candidates with a specific prediction direction such as L0

[0669] 6) For example, the GPM candidate list for P slices may be constructed differently from the GPM candidate list for B slices.

[0670] a) For example, the GPM candidate list for P slices may be a subset of the GPM candidate list for B slices.

[0671] 17. For example, whether syntax elements (e.g., flags) for CU-based GPM template matching are signaled may depend on / be subject to whether intra-inter coding (e.g., one partition is intra-coded and the other partition is inter-coded) is used for the GPM block.

[0672] 1) For example, in the case where the GPM block is encoded and decoded by intra-inter prediction, CU-based GPM template matching (where GPM partitions are all refined by template matching) may not be allowed to be further applied.

[0673] a) For example, in case CU-based GPM template matching is not allowed for GPM Intra-Inter blocks, the flag of CU-level TM is not signaled but inferred to some value.

[0674] 2) Alternatively, whether intra-inter codec is allowed for a GPM block may depend on / be subject to whether CU-based GPM template matching is used for the block.

[0675] a) For example, in case of using CU-based GPM template matching, GPM intra-inter prediction may not be allowed to be further applied.

[0676] b) For example, in case GPM intra-inter prediction is not allowed for GPM blocks, the intra-coded information is not signaled in the bitstream.

[0677] 3) Alternatively, GPM Intra-Inter blocks may be allowed to use partition-based GPM template matching (where inter-coded GPM partitions are allowed to be refined by template matching).

[0678] a) For example, in the case where partition-based GPM template matching is allowed for GPM intra-inter blocks (partition-based GPM template matching is allowed to be applied to inter-coded partitions), a flag for the inter-coded partitions can be signaled to specify whether the motion of the partition is further refined by template matching.

[0679] 18. In one example, multiple hypothesis prediction blocks may be allowed for I slices / pictures.

[0680] 1) For example, GPM may be allowed for I slices / pictures.

[0681] a) For example, a GPM block may include two non-inter predictions.

[0682] b) For example, non-inter prediction may be intra prediction, IBC prediction or palette prediction.

[0683] c) For example, different intra modes may be used for the two partitions of the GPM block.

[0684] d) For example, sample-based weighting factors may be used to blend / fuse the two partitions of a GPM block.

[0685] 2) For example, CIIP may be allowed for I slices / pictures.

[0686] a) For example, the CIIP block may include intra prediction and non-inter prediction.

[0687] b) For example, non-inter prediction may be intra prediction, IBC prediction or palette prediction.

[0688] c) For example, different intra modes may be used for the two predictions of the CIIP block.

[0689] d) For example, block-based weighting factors may be used to blend / fuse the two predictions of the CIIP block.

[0690] 3) For example, MHP may be allowed for I slices / pictures.

[0691] a) For example, an MHP block may include multiple non-inter predictions.

[0692] b) For example, non-inter prediction may be intra prediction, IBC prediction or palette prediction.

[0693] c) For example, different intra modes may be used for multiple hypotheses of the MHP block.

[0694] d) For example, block-based weighting factors can be used to mix / fuse multiple hypotheses of the MHP block.

[0695] 4) If the IBC involves GPM / CIIP / MHP, information such as BV for the IBC can be transmitted through the signal.

[0696] 5) If the palette involves GPM / CIIP / MHP, information on the palette such as palette index can be transmitted through the signal.

[0697] Miscellaneous

[0698] 19. In one example, for a particular codec, the shape of the template for a video unit may depend on the availability of neighboring samples.

[0699] 1) When the upper samples are available but the left samples are not available (for example, the template exceeds the left boundary of the picture, or the current block is located in the first row of the picture), the template only includes the upper samples.

[0700] 2) When the left samples are available but the upper samples are not available (for example, the template exceeds the upper boundary of the picture, or the current block is located in the first column of the picture), the template only includes the left samples.

[0701] 3) When the left sample and the upper sample are not available (for example, the current block is located in the first row and the first column of the picture), the template is not used.

[0702] 4) Alternatively, a virtual template may be used, where at least one sample of the template is generated from a certain average value (eg filled with default sample values ​​depending on the internal bit depth).

[0703] a) In one example, padding may be used to fill in unavailable samples.

[0704] 5) This template can be used for MV / BV export based on template matching.

[0705] 6) The template can be used for intra-frame prediction derivation based on template matching.

[0706] 20. In one example, filter coefficients, clipping values ​​may be allowed to be values ​​not equal to a power of 2.

[0707] 1) In one example, the filter coefficients of the CCALF may be based on values ​​that are not equal to a power of 2.

[0708] 2) In one example, the clipping value of some codec tools (eg, non-linear clipping in ALF, etc.) may not be a power of 2.

[0709] 21. In one example, chrominance and luma can share similar filter shapes.

[0710] 1) The filter shapes can be the same, but the filter lengths can be different.

[0711] 2) For example, assuming that an MxN diamond / cross filter is used for the luma component of a loop filter (e.g., ALF, CCALF, etc.), its associated chroma components may be allowed to use similar diamond / cross filters with dimensions of (M>>SubWidthC)x(N>>SubHeightC), where SubWidthC and SubHeightC depend on the chroma format sampling structure.

[0712] a) For example, for 4:2:0 chroma format, SubWidthC=SubHeightC=2.

[0713] b) For example, for 4:4:4 chroma format, SubWidthC=SubHeightC=1.

[0714] c) For example, for 4:2:2 chroma format, SubWidthC=2 and SubHeightC=1.

[0715] 3) In one example, if the chroma format is 4:4:4, chroma and luma can share the same filter shape.

[0716] General rights

[0717] 22. Whether and / or how to apply the method disclosed above can be transmitted by a signal at the sequence level / picture group level / picture level / slice level / tile group level, such as in a sequence header / picture header / SPS / VPS / DPS / DCI / PPS / APS / slice header / tile group header.

[0718] 23. Whether and / or how to apply the methods disclosed above may be signaled in PB / TB / CB / PU / TU / CU / VPDU / CTU / CTU line / slice / tile / sub-picture / other types of regions containing more than one sample or pixel. Whether and / or how to apply the methods disclosed above may depend on the coded information, such as block size, color format, single / dual tree partitioning, color component, slice / picture type.

[0719] It should be noted that the following terms are not limited to the specific terms defined in the existing standards. Any variant of the codec tool is also applicable. For example, the term "GPM" may represent a codec method that divides a block into two or more sub-regions, wherein at least one sub-region cannot be represented by any existing partitioning structure (e.g., QT / BT / TT). In another example, the term "GPM" may represent a codec block in which at least one final prediction signal of the codec block is generated by a weighted sum of two or more auxiliary prediction signals associated with the GPM sub-region. For example, the term "GPM" may indicate a geometric merge mode (GEO) and / or a geometric partition mode (GPM), and / or a wedge prediction mode, and / or a triangle prediction mode (TPM), and / or a GPM block with motion vector difference (GPM MMVD), and / or a GPM block with motion refinement based on template matching (GPM TM), and / or a GPM with inter-frame and intra-frame, and / or any variant based on GPM.

[0720] Fig.26 An example diagram 2600 is shown of sub-block based motion / mode information storage for a GPM coded block. Fig.26 In , the prediction samples within the dark sub-block (crossing the GPM dividing line) are mixed from sub-region A and sub-region B.

[0721] Motion / mode storage for GPM codec blocks

[0722] 24. In one example, if a GPM subblock contains inter- and intra-predicted samples (e.g., Fig.26 As shown in the dark sub-block in the figure, it is assumed that sub-region-A is inter-frame coded and sub-region-B is intra-frame coded), and the sub-block can be regarded as an intra-frame coded sub-block in the coding and decoding and / or in-loop filtering process of the subsequent video block.

[0723] 1) In one example, the motion stored for such a GPM sub-block may always be perceived as unavailable.

[0724] 2) For example, the motion vector is stored as a zero vector, and the reference index is stored as a certain value (eg, -1), indicating that the sub-block has no reference picture.

[0725] 3) Alternatively, the motion information stored for such a GPM sub-block may always be equal to the motion information of the inter-coded sub-region.

[0726] a) For example, no matter which of the two sub-regions is inter-coded, the motion information stored for the GPM sub-block may always be equal to the motion information of the inter-coded sub-region.

[0727] 4) In general, adaptive / selective motion information storage can be applied to such GPM sub-blocks.

[0728] a) For example, whether the stored motion is perceived as unusable, or equal to the motion of the inter-coded sub-region may depend on the coded information.

[0729] i. Further, the encoded and decoded information includes but is not limited to partition information (such as GPM partition mode, and / or GPM partition angle, and / or GPM partition direction), and / or weight index, and / or GPM block / sub-block position and / or GPM block / sub-block size.

[0730] 5) For example, the stored motion information can be used for subsequent processing of the current GPM block, such as deblocking.

[0731] 6) For example, the stored motion information may be used as temporal motion information for coding / prediction of future blocks, where the future blocks are within subsequent coded pictures in coding order.

[0732] 7) For example, the stored motion information can be used as spatial motion information for encoding / prediction of future blocks, where the future blocks are within the current picture.

[0733] 8) For example, the stored motion information may be used for loop filtering, such as deblocking filtering.

[0734] 25. In one example, the inter prediction process for generating prediction samples for an inter-coded sub-region may follow the rules detailed below.

[0735] 1) For example, inter-coded sub-regions can always be unidirectionally predicted.

[0736] 2) First, the sub-region of inter-frame coding can be bidirectionally predicted.

[0737] 3) In addition, the sub-region of inter-frame coding can be unidirectionally predicted.

[0738] 4) For example, the above rules may be applicable to the case where at least one GPM subblock in the entire GPM block contains both inter-frame and intra-frame prediction samples.

[0739] a) In addition, the above rules may be applied to the case where at least one GPM subblock in the entire GPM block contains intra-prediction samples.

[0740] b) In addition, the above rules may be applied to the case where at least one GPM subblock in the entire GPM block contains inter-frame prediction samples.

[0741] 26. In one example, if a GPM subblock contains inter- and intra-predicted samples (e.g., Fig.26 As shown in the dark sub-block in the GPM, assuming that sub-region-A is inter-frame coded and sub-region-B is intra-frame coded), the intra-frame mode information stored for such a GPM sub-block can always be equal to the intra-frame mode information of the sub-region of intra-frame coded.

[0742] 1) For example, no matter which of the two sub-regions is intra-coded, the intra-mode information stored in the GPM sub-block can always be equal to the intra-mode information of the intra-coded sub-region.

[0743] 2) However, the intra-mode information stored for such GPM sub-blocks may always be considered unusable.

[0744] 3) In general, adaptive / selective intra-mode information storage can be applied to such GPM sub-blocks.

[0745] a) For example, whether the stored intra-mode is perceived as unusable, or equal to the intra-mode of the intra-coded sub-region may depend on the coded information.

[0746] i. Further, the encoded and decoded information includes but is not limited to partition information (such as GPM partition mode, and / or GPM partition angle, and / or GPM partition direction), and / or weight index, and / or GPM block / sub-block position and / or GPM block / sub-block size.

[0747] 4) For example, the stored intra-frame mode information can be used for subsequent processing of the current GPM block, such as deblocking.

[0748] 5) For example, the stored intra mode information may be used as temporal intra mode information for future block coding / prediction (eg, TIMD), where the future block is within a subsequently coded picture in coding order.

[0749] 6) For example, the stored intra mode information can be used as spatial domain intra mode information for encoding / prediction of future blocks, where the future blocks are within the current image.

[0750] 27. In one example, when the prediction of a GPM subblock is a mixture of more than one inter-frame prediction samples (e.g., Fig.26 As shown in the dark sub-block in , assuming that sub-region A is inter-frame coded and sub-region B is also inter-frame coded), the motion stored for such a GPM sub-block can always be equal to the motion information of one sub-region.

[0751] 1) For example, the motion stored for such a GPM sub-block may include motion information from at most one of the two sub-regions.

[0752] a. In addition, the stored motion information of such GPM sub-blocks can always be unidirectionally predicted.

[0753] i. Therefore, the stored motion information of such a GPM sub-block may be bi-directionally predicted (eg, in the case where sub-region A is bi-directionally predicted or sub-region B is bi-directionally predicted).

[0754] ii. Of course, the motion information of such GPM sub-blocks stored can be unidirectionally predicted.

[0755] b. In addition, a total of two types of motion (e.g., sub-region A motion or sub-region B motion) may be stored for such a GPM sub-block. The motion storage for such a GPM sub-block may not be / never be of a third type, e.g., combined / constructed from both sub-region A motion and sub-region B motion.

[0756] 2) For example, whether to store the sub-region A motion or the sub-region B motion (but never the combined motion of these two) of such a GPM sub-block may depend on the codec information.

[0757] ii. Further, the encoded and decoded information includes but is not limited to partition information (such as GPM partition mode, and / or GPM partition angle, and / or GPM partition direction), and / or weight index, and / or GPM block / sub-block position and / or GPM block / sub-block size.

[0758] 28. In one example, if the prediction of a GPM subblock is a mixture of multiple intra prediction samples (e.g., Fig.26 As shown in the dark sub-block in FIG, assuming that sub-region A is intra-frame coded and decoded (sub-region B is also intra-frame coded and decoded), the intra-frame mode information stored in such a GPM sub-block can always be equal to the intra-frame mode information of a sub-region.

[0759] 1) For example, the intra-mode information stored for such a GPM sub-block may always be equal to the intra-mode information of sub-region A.

[0760] 2) For example, the intra-mode information stored for such a GPM sub-block may always be equal to the intra-mode information of sub-region B.

[0761] 3) For example, whether to store sub-region A intra-mode information or sub-region B intra-mode information (but never a combination of both) for such a GPM sub-block may depend on the codec information.

[0762] i. Further, the encoded and decoded information includes but is not limited to partition information (such as GPM partition mode, and / or GPM partition angle, and / or GPM partition direction), and / or weight index, and / or GPM block / sub-block position and / or GPM block / sub-block size.

[0763] 4) Alternatively, constructed / converted / mapped intra-modes may be stored for such GPM sub-blocks.

[0764] 5) Alternatively, more than one intra-mode may be stored for such a GPM sub-block.

[0765] a) For example, for such a GPM sub-block, the intra-frame mode information of the sub-region A and the intra-frame mode information of the sub-region B may be stored simultaneously.

[0766] 29. For example, the aforementioned GPM block may be a block of GPM coding without motion refinement.

[0767] 30. For example, the aforementioned GPM block may be a block of GPM codec with motion refinement.

[0768] 31. For example, the above-mentioned GPM block can be a GPM MMVD block.

[0769] 32. For example, the above-mentioned GPM block may be a GPM TM (Template Matching) block.

[0770] 5. Examples

[0771] Embodiments of the present disclosure relate to geometric partitioning tools. As used herein, the term "geometric partitioning tool" or "geometric partitioning mode (GPM) tool" refers to any codec tool that divides a block into two or more sub-regions, at least one of which cannot be split. Generated by any existing partitioning structure (e.g., QT / BT / TT). For example, the term "GPM" can represent a codec block in which at least one final prediction signal of the codec block is generated by a weighted sum of two or more auxiliary prediction signals associated with the GPM sub-region. For example, the term "GPM" can indicate a geometric merge mode (GEO), and / or a geometric partitioning mode (GPM), and / or a wedge prediction mode, and / or a triangle prediction mode (TPM), and / or a GPM block with motion vector difference (GPM MMVD), and / or a GPM block with motion refinement based on template matching (GPMTM), and / or a GPM with inter-frame and intra-frame, and / or any variant based on GPM.

[0772] It should be noted that the following terms are not limited to the specific terms defined in existing standards, and any variants of the codec tool are also applicable.

[0773] As used herein, the term "block" may refer to a codec block (CB), a codec unit (CU), a prediction unit (PU), a transform unit (TU), a prediction block (PB), or a transform unit block (TB).

[0774] Fig. 27 2700 is a flowchart of a method 2700 for video processing according to some embodiments of the present disclosure. The method 2700 may be implemented during conversion between a target video block of a video and a bitstream of the video. Fig. 27 As shown, method 2700 starts at 2702, where prediction information of a target sub-block in a target video block is obtained based on the type of prediction sample in the target sub-block. The target video block is encoded and decoded by a geometric partitioning tool. For example, the obtained prediction information can be stored. The geometric partitioning tool can include a geometric merge mode (GEO), a geometric partitioning mode (GPM), a wedge prediction mode, a triangle prediction mode, a GPM with motion vector difference (GPM MMVD), a GPM block refinement with template matching-based motion (GPM TM), a GPM with inter-frame and intra-frame, or a variant codec tool based on GPM.

[0775] As used herein, the term "target video block" may also be referred to as a "geometric partitioning block" or a "GPM-coded block." For example, the target video block may include a geometric partitioning mode (GPM)-coded block without motion refinement, a GPM-coded block with motion refinement, a GPM block with motion vector difference (GPM MMVD), or a GPM block with template matching (GPM TM).

[0776] At block 2704, conversion is performed based on the prediction information of the target sub-block. For example, the conversion may include encoding the target video block into a bitstream. For another example, the conversion may include decoding the target video block from the bitstream.

[0777] According to an embodiment of the present disclosure, it is proposed that prediction information of a sub-block in a target block, such as motion information or intra-frame mode information, can be obtained. The obtained prediction information of the sub-block can be stored for further encoding or decoding or prediction processing. This can improve the prediction effect and encoding and decoding efficiency.

[0778] In some embodiments, if the target sub-block includes intra-frame prediction samples and inter-frame prediction samples, the intra-frame coded information of the target sub-block can be used for at least one of the following: coding and decoding of subsequent video blocks, or in-loop filtering process. Fig.26 The sub-blocks shown crossing the GPM dividing line include both intra-frame prediction samples and inter-frame prediction samples. In other words, in the encoding and decoding and / or in-loop filtering process of subsequent video blocks, the target sub-block can be regarded as an intra-frame coded sub-block.

[0779] In some embodiments, at step 2702, if the target subblock includes intra-prediction samples and inter-prediction samples, motion information of the target subblock may be obtained. Alternatively or additionally, the obtained motion information may be stored. In some embodiments, the motion information may not be available. For example, the motion information may always be perceived as not available.

[0780] In some embodiments, the motion information may include a zero vector with a corresponding reference index. The corresponding reference index indicates that the target sub-block has no reference picture. In other words, the motion vector may be stored as a zero vector, and the reference index may be stored as a certain value (e.g., -1) indicating that the sub-block has no reference picture.

[0781] Alternatively or additionally, in a certain embodiment, at step 2702, motion information of inter-frame prediction samples of the target sub-block may be obtained. For example, in some embodiments, the target sub-block may include an intra-frame region and an inter-frame region. The prediction samples of the intra-frame region are intra-frame predictions, and the prediction samples of the inter-frame region are inter-frame predictions. In this case, at block 2702, motion information of the inter-frame region may be obtained. In other words, the motion information stored for the GPM sub-block may always be equal to the motion information of the sub-region of the inter-frame codec.

[0782] In some embodiments, adaptive or selective motion information storage can be applied to the target sub-block. Alternatively or additionally, it can be determined based on the coded information of the target video block whether the motion information of the target sub-block is unavailable or equal to the motion information of the inter-frame prediction sample. The motion information of the target sub-block can be stored based on the determination.

[0783] For example, the coded information of the target video block includes at least one of the following: partition information, weight index, position of a GPM block or a GPM subblock, or dimension of a GPM block or a GPM subblock. The partition information may include at least one of the following: a GPM partition mode, a GPM partition angle, or a GPM partition direction.

[0784] In some embodiments, at block 2704, the motion information may be used for subsequent processes in the conversion. The subsequent processes may include a deblocking process. That is, the obtained motion information or the stored motion information may be used for subsequent processing of the current GPM block, such as deblocking. Alternatively or additionally, in some embodiments, at block 2704, the motion information may be used for loop filtering in the conversion, such as deblocking filtering.

[0785] In some embodiments, the obtained motion information can be used as temporal motion information for encoding or predicting another block. The other block can be located in a subsequent coded picture of the video in the codec order. For example, the stored motion information can be used as temporal motion information for encoding / predicting a future block, where the future block is located in a subsequent coded picture in the codec order.

[0786] Alternatively or additionally, in some embodiments, the obtained motion information can be used as spatial motion information to encode or predict a further block. The other block can be within the current picture of the video. That is, the stored motion information can be used as spatial motion information for encoding / predicting a future block, where the future block is within the current picture.

[0787] In some embodiments, inter-frame prediction samples of the inter-frame coded area of ​​the target video block can be generated based on predetermined rules. In other words, the inter-frame prediction process of generating prediction samples of the inter-frame coded sub-area can follow certain rules. For example, the predetermined rules may include at least one of the following: a rule indicating a unidirectionally predicted inter-frame coded area, a rule indicating a bidirectionally predicted inter-frame coded area. For example, the sub-area of ​​the inter-frame codec can always be unidirectionally predicted. First, the sub-area of ​​the inter-frame codec can be bidirectionally predicted. Alternatively or additionally, the sub-area of ​​the inter-frame codec can be unidirectionally predicted. In some embodiments, if at least one sub-block of the target video block includes at least one of an inter-frame prediction sample or an intra-frame prediction sample, the at least one sub-block includes the target sub-block, and the inter-frame prediction sample can be generated according to the predetermined rules. That is, the above rules can be applied to the case where at least one GPM sub-block in the entire GPM block contains both inter-frame and intra-frame prediction samples. The above rules can also be applied to the case where at least one GPM sub-block of the entire GPM block contains intra-frame prediction samples. Of course, the above rules can be applied to the case where at least one GPM sub-block in the entire GPM block contains inter-frame prediction samples.

[0788] In some embodiments, in step 2702, if the target sub-block includes intra-frame prediction samples and inter-frame prediction samples, the intra-frame mode information of the intra-frame prediction samples of the target sub-block can be obtained. For example, the intra-frame mode information of the intra-frame coding area of ​​the target sub-block can be obtained. The intra-frame coding area is intra-frame coded. That is, no matter which of the two sub-areas is intra-frame coded, the intra-frame mode information stored for the GPM sub-block can always be equal to the intra-frame mode information of the intra-frame coding. Sub-area. In some embodiments, the obtained intra-frame mode information can be stored.

[0789] In some embodiments, intra-mode information is not available. For example, intra-mode information of a target sub-block may always be perceived as not available.

[0790] Alternatively or additionally, in some embodiments, adaptive or selective intra-frame mode information storage may be applied to the target sub-block. For example, in some embodiments, it may be determined whether the intra-frame mode information of the target sub-block is unavailable or equal to the intra-frame mode information of the intra-frame prediction sample based on the coded information of the target video block. The intra-frame mode information of the target sub-block may be stored based on the determination. For example, the coded information of the target video block includes at least one of the following: segmentation information, weight index, position of the GPM block or GPM sub-block, or dimension of the GPM block or GPM sub-block. Among them, the partition information may include at least one of the following: GPM partition mode, GPM partition angle, or GPM partition direction.

[0791] In some embodiments, at block 2704, the intra-mode information may be used for subsequent processes in the conversion. For example, the subsequent process may include a deblocking process. That is, the stored intra-mode information, such as the intra-mode information, may be used for subsequent processing of the current GPM block, such as deblocking.

[0792] Alternatively or additionally, in some embodiments, the obtained intra-frame mode information can be used as temporal intra-frame mode information for encoding or predicting further blocks. The other block can be located in a subsequent coded picture of the video in the codec order. Encoding or predicting another block may include applying template-based intra-frame mode derivation (TIMD). For example, the stored intra-frame mode information can be used as temporal intra-frame mode information for future block encoding or prediction such as TIMD. The future block is located in a subsequent coded picture in the codec order.

[0793] In some embodiments, the obtained intra-frame mode information can be used as spatial intra-frame mode information for encoding or predicting another block, the other block being within the current picture of the video. For example, the stored intra-frame mode information can be used as spatial intra-frame mode information for encoding or predicting a future block. The future block is within the current picture.

[0794] In some embodiments, at block 2702, if the prediction sample includes at least two inter-prediction samples, motion information of the target subblock may be obtained. In some embodiments, the obtained motion information may be stored.

[0795] In some embodiments, motion information may be obtained in an inter-coded region. Inter-prediction samples are coded in an inter-coded region. For example, the inter-coded region is one of two inter-coded regions of the target sub-block. In other words, the motion information stored for the target sub-block may include motion information from at most one of the two sub-regions.

[0796] In some embodiments, the motion information may include at least one of the following: unidirectional prediction or bidirectional prediction. For example, the stored motion information of the target sub-block may always be unidirectionally predicted. Therefore, the stored motion information of the target sub-block may be bidirectionally predicted. For example, in the case where at least one inter-frame coded region of the target sub-block is bidirectionally predicted, the motion information includes bidirectional prediction. Therefore, the stored motion information of the target sub-block may be unidirectionally predicted.

[0797] Alternatively or additionally, in some embodiments, two types of motion information may be obtained for a target sub-block. Each of the two types of motion information is associated with a corresponding inter-frame coded region of the target sub-block. In other words, a total of two types of motion information (e.g., Fig.26In some embodiments, the third type of motion information may not exist in the motion information of the target sub-block. For example, the third type of motion information may include combining or constructing motion information from the first motion information in the first inter-frame encoded and decoded area and the second motion information in the second inter-frame encoded and decoded area. In other words, the motion storage of the target GPM sub-block may not be or never be of the third type, for example, a combination or construction of both the sub-region A motion and the sub-region B motion.

[0798] In some embodiments, whether to store the first motion information of the first inter-frame encoded region of the target sub-block or the second motion information of the second inter-frame encoded region of the target sub-block can be based on the information of the encoded and decoded target sub-block. The corresponding motion information can be stored based on the determination. In some embodiments, the stored motion information does not contain the combined motion information of the first motion information and the second motion information. For example, the encoded and decoded information of the target video block may include at least one of the following: segmentation information, weight index, position of the GPM block or GPM sub-block, or dimension of the GPM block or GPM sub-block. Among them, the partition information may include at least one of the following: GPM partition mode, GPM partition angle, or GPM partition direction.

[0799] In some embodiments, at block 2702, if the prediction sample includes at least two intra-frame prediction samples, intra-frame mode information of the target subblock may be obtained. In some embodiments, the obtained intra-frame mode information may be stored. For example, intra-frame mode information in an intra-frame coding area may be obtained. The intra-frame prediction sample is coded in the intra-frame coding area. The intra-frame coded area may be one of the two intra-frame coded areas of the target subblock.

[0800] In some embodiments, whether to store the first intra-frame mode information of the encoded and decoded area of ​​the first frame of the target sub-block or the second intra-frame mode information of the encoded and decoded area of ​​the second frame of the target sub-block can be determined based on the encoded and decoded information of the target sub-block. Each intra-frame mode information can be stored based on this determination. In some embodiments, the combined intra-frame mode information of the first and second intra-frame mode information may not exist in the stored intra-frame mode information. That is, the combination of the first intra-frame mode information and the second intra-frame mode information may never be stored. In some embodiments, the encoded and decoded information of the target video block may include at least one of the following: segmentation information, weight index, position of the GPM block or GPM sub-block, or dimension of the GPM block or GPM sub-block. Among them, the partition information may include at least one of the following: GPM partition mode, GPM partition angle, or GPM partition direction.

[0801] Alternatively or additionally, in some embodiments, in order to obtain intra-frame mode information, at least one of the following may be obtained for the target sub-block: a constructed intra-frame mode, a converted intra-frame mode, or a mapped intra-frame mode. Intra-frame mode. In other words, a constructed or converted or mapped intra-frame mode may be stored for the current GPM sub-block.

[0802] In some embodiments, intra-frame mode information of multiple intra-frame coded regions of the target sub-block may be obtained. For example, more than one intra-frame coded region may include a first intra-frame coded region and a second intra-frame coded region. In other words, intra-frame mode information of sub-region A and intra-frame mode information of sub-region B may be stored for the target sub-block.

[0803] In some embodiments, the bitstream of the video may be stored in a non-transitory computer-readable recording medium. The bitstream of the video may be generated by a method performed by a video processing device. According to the method, prediction information of a target sub-block in a target video block of the video may be obtained based on the type of prediction sample in the target sub-block. The target video block may be encoded and decoded by a geometric partitioning tool. A bitstream may be generated based on the prediction information of the target sub-block.

[0804] In some embodiments, prediction information of a target sub-block in a target video block of a video may be obtained based on the type of prediction sample in the target sub-block. The target video block may be encoded and decoded by a geometric partitioning tool. A bitstream of the video may be generated based on the prediction information of the target sub-block. The bitstream may be stored in a non-transitory computer-readable recording medium.

[0805] According to an embodiment of the present disclosure, it is proposed that prediction information of a sub-block in a target video block can be obtained and stored. By using prediction information such as motion information or intra-frame mode information, the encoding and decoding process can be improved, thereby improving the effectiveness of target video block prediction and encoding and decoding efficiency.

[0806] Implementations of the present disclosure may be described in terms of the following clauses, and features of these clauses may be combined in any reasonable way.

[0807] Clause 1. A method for video processing, comprising: during conversion between a target video block of a video and a bitstream of the video, obtaining prediction information of a target sub-block in the target video block based on the type of prediction samples in a target sub-block of the target video block, the target video block being encoded and decoded by a geometric partitioning tool; and performing the conversion based on the prediction information of the target sub-block.

[0808] Clause 2. The method according to Clause 1 further includes: if the target subblock includes intra-frame prediction samples and inter-frame prediction samples, using at least one item of intra-frame coding information of the target subblock in at least one of the following: coding and decoding of subsequent video blocks, or an in-loop filtering process.

[0809] Clause 3. A method according to Clause 1 or Clause 2, wherein obtaining the prediction information of the target subblock based on the type of the prediction sample comprises: if the target subblock includes an intra-frame prediction sample and an inter-frame prediction sample, obtaining motion information of the target subblock.

[0810] Clause 4. The method of clause 3, wherein the motion information is not available.

[0811] Clause 5. The method of clause 3, wherein the motion information comprises a zero vector with a corresponding reference index indicating that there is no reference picture for the target subblock.

[0812] Clause 6. The method of clause 3, wherein obtaining the motion information comprises: obtaining the motion information of the inter-prediction samples of the target subblock.

[0813] Clause 7. A method according to Clause 3, wherein the target subblock includes an intra-frame area and an inter-frame area, the prediction samples in the intra-frame area are intra-frame predicted, the prediction samples in the inter-frame area are inter-frame predicted, and wherein obtaining the motion information includes: obtaining the motion information of the inter-frame area.

[0814] Clause 8. The method of any of clauses 3-7, further comprising: applying adaptive or selective motion information storage to the target sub-block.

[0815] Clause 9. The method according to any one of Clauses 3-8 further includes: determining whether the motion information of the target sub-block is unavailable or equal to the motion information of the inter-frame prediction sample based on the encoded and decoded information of the target video block; and storing the motion information for the target sub-block based on the determination.

[0816] Clause 10. The method of clause 9, wherein the encoded information of the target video block comprises at least one of: partition information, a weight index, a position of a GPM block or a GPM subblock, or a dimension of a GPM block or a GPM subblock.

[0817] Clause 11. The method according to clause 10, wherein the segmentation information includes at least one of the following: a GPM segmentation mode, a GPM segmentation angle, or a GPM segmentation direction.

[0818] Clause 12. The method of any of clauses 3-11, wherein performing the conversion comprises using the motion information for subsequent processes in the conversion.

[0819] Clause 13. The method of clause 12, wherein the subsequent process comprises a deblocking process.

[0820] Clause 14. A method according to any of clauses 3-11, wherein the obtained motion information is used as temporal motion information for encoding or predicting another block, the other block being located in a subsequent coded picture of the video in coding order.

[0821] Clause 15. A method according to any of clauses 3-11, wherein the obtained motion information is used as spatial motion information for encoding or predicting another block, the other block being located within a current picture of the video.

[0822] Clause 16. The method of any of clauses 3-11, wherein performing the conversion comprises using the motion information for loop filtering in the conversion.

[0823] Clause 17. The method of clause 15, wherein the loop filtering comprises deblocking filtering.

[0824] Clause 18. The method according to any one of clauses 1-17, further comprising: generating inter-frame prediction samples of the inter-frame encoded region of the target video block based on a predetermined rule.

[0825] Clause 19. The method of clause 18, wherein the predetermined rule comprises at least one of: a rule indicating a unidirectionally predictive inter-frame coded region, or a rule indicating a bidirectionally predictive inter-frame coded region.

[0826] Clause 20. A method according to Clause 18 or Clause 19, wherein generating the inter-frame prediction sample comprises: if at least one sub-block of the target video block includes at least one of an inter-frame prediction sample or an intra-frame prediction sample, and the at least one sub-block includes the target sub-block, then generating the inter-frame prediction sample based on the predetermined rule.

[0827] Clause 21. A method according to any one of Clauses 1-2, wherein obtaining the prediction information of the target subblock based on the type of the prediction sample includes: if the target subblock includes intra-frame prediction samples and inter-frame prediction samples, then obtaining intra-frame mode information of the intra-frame prediction samples of the target subblock.

[0828] Clause 22. The method according to clause 21, wherein obtaining the intra mode information of the target subblock comprises: obtaining intra mode information of an intra-coded region of the target subblock, the intra-coded region being intra-coded.

[0829] Clause 23. A method as described in clause 21 or clause 22, wherein the intra-mode information is not available.

[0830] Clause 24. A method according to any one of clauses 21-23, further comprising: applying adaptive or selective intra-mode information storage to the target sub-block.

[0831] Clause 25. The method according to any one of Clauses 21-24 further includes: determining whether the intra-frame mode information of the target sub-block is unavailable or equal to the intra-frame mode information of the intra-frame prediction sample based on the encoded and decoded information for the target video block; and storing the intra-frame mode information for the target sub-block based on the determination.

[0832] Clause 26. The method of clause 25, wherein the coded information of the target video block comprises at least one of: partition information, a weight index, a position of a GPM block or a GPM sub-block, or a dimension of a GPM block or a GPM sub-block.

[0833] Clause 27. The method according to clause 26, wherein the segmentation information includes at least one of the following: a GPM segmentation mode, a GPM segmentation angle, or a GPM segmentation direction.

[0834] Clause 28. The method of any of clauses 21-27, wherein performing the conversion comprises using the intra-mode information for subsequent processes in the conversion.

[0835] Clause 29. The method of clause 28, wherein the subsequent process comprises a deblocking process.

[0836] Clause 30. A method according to any one of clauses 21-27, wherein the obtained intra-frame mode information is used as temporal intra-frame mode information for encoding or predicting another block, the other block being located in a subsequent encoded picture of the video in the encoding and decoding order.

[0837] Clause 31. The method of clause 30, wherein encoding or predicting the other block comprises applying template-based intra mode derivation (TIMD).

[0838] Clause 32. A method according to any one of clauses 21-27, wherein the obtained intra-frame mode information is used as spatial intra-frame mode information for encoding or predicting another block, the other block being located in a current picture of the video.

[0839] Clause 33. A method according to clause 1 or clause 2, wherein obtaining the prediction information of the target subblock based on the type of the prediction sample includes: if the prediction sample includes at least two inter-frame prediction samples, obtaining motion information for the target subblock.

[0840] Clause 34. The method of clause 33, wherein obtaining the motion information comprises obtaining motion information in an inter-coded region in which the inter-prediction samples are coded.

[0841] Clause 35. The method of clause 34, wherein the inter-coded region is one of two inter-coded regions for the target subblock.

[0842] Clause 36. A method according to any one of clauses 33-35, wherein the motion information comprises at least one of: unidirectional prediction, or bidirectional prediction.

[0843] Clause 37. The method of clause 36, wherein the motion information comprises a bi-directional prediction if at least one inter-coded region of the target sub-block is bi-directionally predicted.

[0844] Clause 38. The method according to clause 33, wherein obtaining the motion information comprises: obtaining two types of motion information for the target sub-block, each type of the two types of motion information being associated with a corresponding inter-frame coded region of the target sub-block.

[0845] Clause 39. The method of clause 38, wherein a third type of motion information is absent from the motion information for the target subblock.

[0846] Clause 40. The method of clause 39, wherein the third type of motion information comprises combining or constructing motion information from first motion information in a first inter-frame encoded region and second motion information in a second inter-frame encoded region.

[0847] Clause 41. The method according to any one of clauses 33-40 further includes: determining whether to store first motion information of a first inter-frame encoded and decoded region of the target sub-block or second motion information of a second inter-frame encoded and decoded region of the target sub-block based on the encoded and decoded information for the target sub-block; and storing the corresponding motion information based on the determination.

[0848] Clause 42. The method of clause 41, wherein the stored motion information does not include combined motion information of the first motion information and the second motion information.

[0849] Clause 43. A method according to clause 41 or clause 42, wherein the encoded information of the target video block includes at least one of the following: partition information, a weight index, a position of a GPM block or a GPM sub-block, or a dimension of a GPM block or a GPM sub-block.

[0850] Clause 44. The method according to clause 43, wherein the segmentation information includes at least one of the following: a GPM segmentation mode, a GPM segmentation angle, or a GPM segmentation direction.

[0851] Clause 45. A method according to clause 1 or clause 2, wherein obtaining the prediction information of the target subblock based on the type of the prediction sample includes: if the prediction sample includes at least two intra-frame prediction samples, obtaining intra-frame mode information for the target subblock.

[0852] Clause 46. The method of clause 45, wherein obtaining the intra mode information comprises obtaining the intra mode information in an intra-coded region in which intra-prediction samples are coded.

[0853] Clause 47. The method of clause 46, wherein the intra-coded region is one of two intra-coded regions for the target subblock.

[0854] Clause 48. The method according to any one of Clauses 45-47 further includes: determining whether to store first intra-frame mode information of a first intra-frame encoded and decoded area of ​​the target sub-block or second intra-frame mode information of a second intra-frame encoded and decoded area of ​​the target sub-block based on the encoded and decoded information for the target sub-block; and storing the corresponding intra-frame mode information based on the determination.

[0855] Clause 49. The method of clause 48, wherein the stored intra-mode information does not include combined intra-mode information of the first intra-mode information and the second intra-mode information.

[0856] Clause 50. The method of clause 48 or clause 49, wherein the encoded information of the target video block comprises at least one of: partition information, a weight index, a position of a GPM block or a GPM subblock, or a dimension of a GPM block or a GPM subblock.

[0857] Clause 51. The method of clause 50, wherein the segmentation information comprises at least one of: a GPM segmentation mode, a GPM segmentation angle, or a GPM segmentation direction.

[0858] Clause 52. The method according to clause 45, wherein obtaining the intra-frame mode information comprises: obtaining at least one of the following for the target subblock: a constructed intra-frame mode, a converted intra-frame mode, or a mapped intra-frame mode.

[0859] Clause 53. The method of clause 45, wherein obtaining the intra-mode information comprises obtaining intra-mode information for more than one intra-coded region of the target subblock.

[0860] Clause 54. The method of clause 53, wherein the more than one intra-coded regions include a first intra-coded region and a second intra-coded region.

[0861] Clause 55. A method according to any one of clauses 1-54, wherein the target video block comprises one of: a geometric partition mode (GPM) encoded block without motion refinement, a geometric partition mode (GPM) encoded block with motion refinement, a GPM with motion vector difference (GPM MMVD) block, or a GPM with template matching based motion refinement (GPM TM) block.

[0862] Clause 56. A method according to any one of clauses 1-54, wherein the geometry partitioning tool includes at least one of the following: geometry merge mode (GEO), geometry partitioning mode (GPM), wedge prediction mode, triangle prediction mode, GPM with motion vector difference (GPM MMVD), GPM with template matching based motion refinement (GPM TM) block, GPM with inter-frame and intra-frame, or a variant codec tool based on GPM.

[0863] Clause 57. The method of any of clauses 1-56, wherein the converting comprises encoding the target video block into the bitstream.

[0864] Clause 58. The method of any of clauses 1-56, wherein the converting comprises decoding the target video block from the bitstream.

[0865] Clause 59. An apparatus for processing video data, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform the method of any of clauses 1-58.

[0866] Clause 60. A non-transitory computer-readable storage medium storing instructions for causing a processor to perform the method of any of clauses 1-58.

[0867] Item 61. A non-transitory computer-readable recording medium storing a bitstream of a video generated by a method performed by a video processing device, wherein the method comprises: obtaining prediction information of a target subblock based on a prediction sample type in a target subblock in a target video block of the video, wherein the target video block is encoded and decoded by a geometric partitioning tool; and generating the bitstream based on the prediction information of the target subblock.

[0868] Item 62. A method for storing a bitstream of a video, comprising: obtaining prediction information of a target subblock based on a prediction sample type in a target subblock in a target video block of the video, wherein the target video block is encoded and decoded by a geometric partitioning tool; generating the bitstream based on the prediction information of the target subblock; and storing the bitstream in a non-transitory computer-readable recording medium.

[0869] Example Device

[0870] Fig.28 A block diagram of a computing device 2800 in which various embodiments of the present disclosure may be implemented is shown. The computing device 2800 may be implemented as a source device 110 (or video encoder 114 or 200) or a destination device 120 (or video decoder 124 or 300), or may be included in a source device 110 (or video encoder 114 or 200) or a destination device 120 (or video decoder 124 or 300).

[0871] It should be understood that Fig.28 The computing device 2800 shown in FIG. 2 is for illustrative purposes only and is not intended to in any way imply any limitation on the functionality and scope of the embodiments of the present disclosure.

[0872] like Fig.28 As shown, computing device 2800 includes a general computing device 2800. Computing device 2800 may include at least one or more processors or processing units 2810, memory 2820, storage unit 2830, one or more communication units 2840, one or more input devices 2850, and one or more output devices 2860.

[0873] In some embodiments, computing device 2800 can be implemented as any user terminal or server terminal with computing power. The server terminal can be a server provided by a service provider, a large computing device, etc. The user terminal can be, for example, any type of mobile terminal, fixed terminal or portable terminal, including mobile phones, stations, units, devices, multimedia computers, multimedia tablet computers, Internet nodes, communicators, desktop computers, laptop computers, notebook computers, netbook computers, personal communication systems (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio broadcast receivers, electronic book devices, gaming devices, or any combination thereof, and including accessories and peripherals of these devices or any combination thereof. It is conceivable that computing device 2800 can support any type of interface to the user (such as "wearable" circuit devices, etc.).

[0874] The processing unit 2810 may be a physical processor or a virtual processor and may implement various processes based on a program stored in the memory 2820. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capability of the computing device 2800. The processing unit 2810 may also be referred to as a central processing unit (CPU), a microprocessor, a controller, or a microcontroller.

[0875] The computing device 2800 typically includes various computer storage media. Such media can be any media accessible by the computing device 2800, including but not limited to volatile media and non-volatile media, or removable media and non-removable media. The memory 2820 can be a volatile memory (e.g., a register, a cache, a random access memory (RAM)), a non-volatile memory (such as a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM) or flash memory) or any combination thereof. The storage unit 2830 can be any removable or non-removable medium, and can include machine-readable media, such as a memory, a flash drive, a disk or other media that can be used to store information and / or data and can be accessed in the computing device 2800.

[0876] The computing device 2800 may also include additional removable / non-removable storage media, volatile / non-volatile storage media. Fig.28 Although not shown in the figure, a magnetic disk drive for reading from and / or writing to a removable nonvolatile magnetic disk and an optical disk drive for reading from and / or writing to a removable nonvolatile optical disk may be provided. In this case, each drive may be connected to the bus (not shown) via one or more data medium interfaces.

[0877] The communication unit 2840 communicates with another computing device via a communication medium. In addition, the functions of the components in the computing device 2800 can be implemented by a single computing cluster or multiple computing machines that can communicate via a communication connection. Therefore, the computing device 2800 can operate in a networked environment using logical connections to one or more other servers, networked personal computers (PCs), or other general network nodes.

[0878] Input device 2850 may be one or more of various input devices, such as a mouse, keyboard, trackball, voice input device, etc. Output device 2860 may be one or more of various output devices, such as a display, speaker, printer, etc. With the aid of communication unit 2840, computing device 2800 may also communicate with one or more external devices (not shown), such as storage devices and display devices, and computing device 2800 may also communicate with one or more devices that enable a user to interact with computing device 2800, or any device that enables computing device 2800 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.), if necessary. Such communication may be performed via an input / output (I / O) interface (not shown).

[0879] In some embodiments, some or all components of the computing device 2800 may also be arranged in a cloud computing architecture rather than being integrated in a single device. In a cloud computing architecture, components may be provided remotely and work together to implement the functions described in the present disclosure. In some embodiments, cloud computing provides computing, software, data access and storage services, which will not require the end user to know the physical location or configuration of the system or hardware that provides these services. In various embodiments, cloud computing provides services via a wide area network (e.g., the Internet) using a suitable protocol. For example, a cloud computing provider provides applications via a wide area network, which can be accessed through a web browser or any other computing component. The software or components of the cloud computing architecture and the corresponding data may be stored on a remote server. The computing resources in a cloud computing environment may be merged or distributed at the location of a remote data center. Cloud computing infrastructures may provide services through a shared data center, although they appear as a single access point for users. Therefore, cloud computing architectures may be used to provide components and functions described herein from a service provider at a remote location. Alternatively, they may be provided by a conventional server, or installed directly or otherwise on a client device.

[0880] In an embodiment of the present disclosure, the computing device 2800 may be used to implement video encoding / decoding. The memory 2820 may include one or more video codec modules 2825 having one or more program instructions. These modules can be accessed and executed by the processing unit 2810 to perform the functions of the various embodiments described herein.

[0881] In an example embodiment of performing video encoding, the input device 2850 may receive video data as input 2870 to be encoded. The video data may be processed by, for example, the video encoding module 2825 to generate an encoded bitstream. The encoded bitstream may be provided as output 2880 via the output device 2860.

[0882] In an example embodiment of performing video decoding, the input device 2850 may receive an encoded bitstream as input 2870. The encoded bitstream may be processed by, for example, the video encoding module 2825 to generate decoded video data. The decoded video data may be provided as output 2880 via the output device 2860.

[0883] Although the present disclosure has been specifically shown and described with reference to the preferred embodiments of the present disclosure, it will be understood by those skilled in the art that various changes can be made in form and detail without departing from the spirit and scope of the present application as defined by the appended claims. These changes are intended to be encompassed by the scope of the present application. Therefore, the foregoing description of the embodiments of the present application is not intended to be limiting.

Claims

1. A method for video processing, comprising: During conversion between a target video block of a video and a bitstream of the video, obtaining prediction information of the target sub-block in the target video block based on a type of prediction samples in the target sub-block of the target video block, the target video block being encoded and decoded by a geometric partitioning tool; as well as The converting is performed based on the prediction information of the target subblock.

2. The method according to claim 1, further comprising: If the target subblock includes intra-frame prediction samples and inter-frame prediction samples, at least one item of intra-frame coding and decoding information of the target subblock is used in at least one of the following: The encoding and decoding of subsequent video chunks, or In-loop filtering process.

3. The method according to claim 1 or claim 2, wherein obtaining the prediction information of the target sub-block based on the type of the prediction sample comprises: If the target subblock includes intra-frame prediction samples and inter-frame prediction samples, motion information of the target subblock is obtained. The method of claim 3 , wherein the motion information is not available. 5 . The method of claim 3 , wherein the motion information comprises a zero vector with a corresponding reference index indicating that there is no reference picture for the target subblock.

6. The method according to claim 3, wherein obtaining the motion information comprises: The motion information of the inter-frame prediction sample of the target sub-block is obtained.

7. The method according to claim 3, wherein the target subblock includes an intra region and an inter region, the prediction samples in the intra region are intra predicted, the prediction samples in the inter region are inter predicted, and The acquisition of the motion information includes: The motion information of the inter-frame area is obtained.

8. The method according to any one of claims 3 to 7, further comprising: Adaptive or selective motion information storage is applied to the target sub-block.

9. The method according to any one of claims 3 to 8, further comprising: Determining, based on the coded and decoded information of the target video block, whether the motion information of the target sub-block is unavailable or equal to the motion information of the inter-frame prediction sample; as well as The motion information for the target sub-block is stored based on the determining.

10. The method of claim 9, wherein the coded information of the target video block comprises at least one of: Split information, Weight index, The location of a GPM block or GPM subblock, or The dimensions of a GPM block or GPM subblock.

11. The method according to claim 10, wherein the segmentation information comprises at least one of the following: GPM division mode, GPM division angle, or GPM division direction.

12. The method according to any one of claims 3 to 11, wherein performing the conversion comprises: The motion information is used for subsequent processes in the conversion. The method of claim 12 , wherein the subsequent process comprises a deblocking process.

14. The method according to any one of claims 3 to 11, wherein the obtained motion information is used as temporal motion information for encoding or predicting another block, the other block being located in a subsequent coded picture of the video in coding order.

15. The method according to any one of claims 3-11, wherein the obtained motion information is used as spatial motion information for encoding or predicting another block, the other block being located in a current picture of the video.

16. The method according to any one of claims 3 to 11, wherein performing the conversion comprises: The motion information is used in the conversion for loop filtering. The method of claim 15 , wherein the loop filtering comprises deblocking filtering.

18. The method according to any one of claims 1 to 17, further comprising: Inter-frame prediction samples of the inter-frame encoded region of the target video block are generated based on a predetermined rule.

19. The method according to claim 18, wherein: The predetermined rule includes at least one of the following: Rules indicating regions of unidirectionally predicted inter-frame codecs, or Indicates rules for bi-predictive inter-coded regions.

20. The method of claim 18 or claim 19, wherein generating the inter-frame prediction samples comprises: If at least one subblock of the target video block includes at least one of an inter-frame prediction sample or an intra-frame prediction sample, the at least one subblock includes the target subblock, and the inter-frame prediction sample is generated based on the predetermined rule.

21. The method according to any one of claims 1-2, wherein obtaining the prediction information of the target sub-block based on the type of the prediction sample comprises: If the target subblock includes intra-prediction samples and inter-prediction samples, intra-mode information of the intra-prediction samples of the target subblock is obtained.

22. The method according to claim 21, wherein acquiring the intra mode information of the target subblock comprises: Intra-frame mode information of an intra-frame coded and decoded region of the target sub-block is obtained, where the intra-frame coded and decoded region is intra-frame coded and decoded.

23. A method according to claim 21 or claim 22, wherein the intra mode information is not available.

24. The method according to any one of claims 21 to 23, further comprising: Adaptive or selective intra mode information storage is applied to the target sub-block.

25. The method according to any one of claims 21 to 24, further comprising: Determining, based on the coded and decoded information for the target video block, whether the intra mode information of the target subblock is unavailable or equal to the intra mode information of the intra prediction sample; as well as The intra mode information for the target subblock is stored based on the determining.

26. The method of claim 25, wherein the coded information of the target video block comprises at least one of: Split information, Weight index, The location of a GPM block or GPM subblock, or The dimensions of a GPM block or GPM subblock.

27. The method according to claim 26, wherein the segmentation information comprises at least one of the following: GPM division mode, GPM division angle, or GPM division direction.

28. The method of any one of claims 21-27, wherein performing the conversion comprises: The intra mode information is used for subsequent processes in the conversion.

29. The method of claim 28, wherein the subsequent process comprises a deblocking process.

30. The method according to any one of claims 21-27, wherein the obtained intra-frame mode information is used as temporal intra-frame mode information for encoding or predicting another block, the other block being located in a subsequent coded picture of the video in coding order.

31. The method of claim 30, wherein encoding or predicting the another block comprises: Template-based intra mode derivation (TIMD) is applied.

32. The method according to any one of claims 21-27, wherein the obtained intra-frame mode information is used as spatial domain intra-frame mode information for encoding, decoding or predicting another block, the other block being located in a current picture of the video.

33. The method according to claim 1 or claim 2, wherein obtaining the prediction information of the target sub-block based on the type of the prediction sample comprises: If the prediction samples include at least two inter-frame prediction samples, motion information for the target subblock is obtained.

34. The method of claim 33, wherein obtaining the motion information comprises: Motion information in an inter-coded region is obtained, in which the inter-prediction samples are coded.

35. The method of claim 34, wherein the inter-coded region is one of two inter-coded regions for the target subblock.

36. The method according to any one of claims 33 to 35, wherein the motion information comprises at least one of the following: One-way prediction, or Bidirectional prediction.

37. The method of claim 36, wherein the motion information comprises bi-directional prediction in the case where at least one inter-coded region of the target subblock is bi-directionally predicted.

38. The method of claim 33, wherein obtaining the motion information comprises: Two types of motion information for the target sub-block are obtained, where each type of the two types of motion information is associated with a corresponding inter-frame coded region of the target sub-block. The method according to claim 38 , wherein the motion information for the target subblock does not contain third type of motion information.

40. The method of claim 39, wherein the third type of motion information comprises combining or constructing motion information from first motion information in a first inter-frame encoded region and second motion information in a second inter-frame encoded region.

41. The method according to any one of claims 33 to 40, further comprising: Based on the coded information for the target subblock, determining whether to store the first motion information of the first inter-frame coded region of the target subblock or the second motion information of the second inter-frame coded region of the target subblock; as well as Corresponding motion information is stored based on the determination. The method according to claim 41 , wherein the stored motion information does not include combined motion information of the first motion information and the second motion information.

43. The method of claim 41 or claim 42, wherein the coded information of the target video block comprises at least one of: Split information, Weight index, The location of a GPM block or GPM subblock, or The dimensions of a GPM block or GPM subblock.

44. The method of claim 43, wherein the segmentation information comprises at least one of: GPM division mode, GPM division angle, or GPM division direction.

45. The method according to claim 1 or claim 2, wherein obtaining the prediction information of the target sub-block based on the type of the prediction sample comprises: If the prediction samples include at least two intra prediction samples, intra mode information for the target subblock is obtained.

46. ​​The method of claim 45, wherein obtaining the intra-mode information comprises: The intra mode information in an intra-coded region is obtained, and intra prediction samples are coded in the intra-coded region.

47. The method of claim 46, wherein the intra-coded region is one of two intra-coded regions for the target subblock.

48. The method according to any one of claims 45 to 47, further comprising: Based on the coded information for the target subblock, determining whether to store first intra-frame mode information of a first intra-frame coded region of the target subblock or second intra-frame mode information of a second intra-frame coded region of the target subblock; as well as Corresponding intra-mode information is stored based on the determination.

49. The method of claim 48, wherein the stored intra-frame mode information does not include combined intra-frame mode information of the first intra-frame mode information and the second intra-frame mode information.

50. The method of claim 48 or claim 49, wherein the coded information of the target video block comprises at least one of: Split information, Weight index, The location of a GPM block or GPM subblock, or The dimensions of a GPM block or GPM subblock.

51. The method of claim 50, wherein the segmentation information comprises at least one of: GPM division mode, GPM division angle, or GPM division direction.

52. The method of claim 45, wherein obtaining the intra-mode information comprises: At least one of the following items is obtained for the target sub-block: Constructed intra mode, The converted intra mode, or Mapped intra mode.

53. The method of claim 45, wherein obtaining the intra-mode information comprises: Intra-mode information of more than one intra-frame coded regions of the target sub-block is obtained.

54. The method of claim 53, wherein the more than one intra-codec regions include a first intra-codec region and a second intra-codec region.

55. The method of any one of claims 1-54, wherein the target video block comprises one of: Geometric Partitioning Mode (GPM) coded blocks without motion refinement, Geometric Partitioning Mode (GPM) coded blocks with motion refinement, GPM with motion vector difference (GPM MMVD) blocks, or GPM with template matching based motion refinement (GPM TM) block.

56. The method of any one of claims 1-54, wherein the geometric partitioning tool comprises at least one of: Geometry merge mode (GEO), Geometric Partitioning Mode (GPM), Wedge forecasting model, Triangular prediction model, GPM with Motion Vector Difference (GPM MMVD), GPM with template matching based motion refinement (GPM TM) block, GPM with inter- and intra-frames, or GPM-based variant encoding and decoding tools.

57. The method of any one of claims 1-56, wherein the converting comprises encoding the target video block into the bitstream.

58. The method of any one of claims 1-56, wherein the converting comprises decoding the target video block from the bitstream.

59. An apparatus for processing video data, comprising a processor and a non-volatile memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform the method according to any one of claims 1-58.

60. A non-transitory computer-readable storage medium storing instructions for causing a processor to execute the method according to any one of claims 1-58.

61. A non-transitory computer-readable recording medium storing a bitstream of a video generated by a method performed by a video processing device, wherein the method comprises: Obtain prediction information of the target sub-block based on a prediction sample type in a target sub-block in a target video block of the video, wherein the target video block is encoded and decoded by a geometric partitioning tool; as well as The bitstream is generated based on the prediction information of the target subblock.

62. A method for storing a bitstream of a video, comprising: Obtain prediction information of the target sub-block based on a prediction sample type in a target sub-block in a target video block of the video, wherein the target video block is encoded and decoded by a geometric partitioning tool; generating the bitstream based on the prediction information of the target sub-block; as well as The bit stream is stored in a non-transitory computer-readable recording medium.

Images