Method and device for video processing and medium

Abstract

Embodiments of the present disclosure provide a scheme for video processing. A method for video processing is proposed. The method comprises, for a conversion between a current video block of the video and a bitstream of the video, obtaining a value of a metric in a direction for a mixing region included in a target region associated with the current video block, the value being determined from a plurality of predetermined values for the metric, the value of the sample for the mixing region is determined based on the value of the sample for the first part of the target region and the value of the sample for the second part of the target region; and performing a conversion based on the value for the metric.

Description

Technical Field

[0001] Embodiments of the present disclosure relate generally to video processing techniques, and more particularly, to geometric partitioning modes (GPMs). Background Art

[0002] Nowadays, digital video functions are being applied to all aspects of people's lives. For video encoding / decoding, various types of video compression technologies have been proposed, such as MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4 Part 10 Advanced Video Codec (AVC), ITU-T H.265 High Efficiency Video Codec (HEVC) standard, and Versatile Video Codec (VVC) standard. However, the encoding and decoding efficiency and encoding and decoding quality of video encoding and decoding technologies are generally expected to be further improved. Summary of the invention

[0003] Embodiments of the present disclosure provide a solution for video processing.

[0004] In a first aspect, a method for video processing is provided. The method includes: for conversion between a current video block of a video and a bitstream of the video, obtaining a value of a metric for a mixed region in a direction, the mixed region being included in a target region associated with the current video block, the value being determined from a plurality of predetermined values ​​for the metric, the value for a sample of the mixed region being determined based on a value for a sample of a first portion of the target region and a value for a sample of a second portion of the target region; and performing conversion based on the value for the metric.

[0005] According to the method of the first aspect of the present disclosure, the value of the metric (such as width) for the mixed area is determined from a plurality of predetermined values ​​for the metric. Compared with the traditional solution where the value for the metric is fixed, the proposed method can advantageously support the adaptive size of the mixed area. Thus, the coding efficiency and coding quality can be improved.

[0006] In a second aspect, a device for video processing is provided. The device includes a processor and a non-volatile memory having instructions thereon. The instructions, when executed by the processor, cause the processor to perform the method according to the first aspect of the present disclosure.

[0007] In a third aspect, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium stores instructions, and the instructions enable a processor to execute the method according to the first aspect of the present disclosure.

[0008] In a fourth aspect, another non-transitory computer-readable recording medium is proposed. The non-transitory computer-readable recording medium stores a bitstream of a video generated by a method performed by an apparatus for video processing. The method includes: obtaining a value for a metric in a direction for a mixed region, the mixed region being included in a target region associated with a current video block of the video, the value being determined from a plurality of predetermined values ​​for the metric, the value for a sample of the mixed region being determined based on a value for a sample of a first portion of the target region and a value for a sample of a second portion of the target region; and generating a bitstream based on the value for the metric.

[0009] In a fifth aspect, a method for storing a bitstream of a video is provided. The method includes: obtaining a value of a metric for a mixed region in a direction, the mixed region being included in a target region associated with a current video block of the video, the value being determined from a plurality of predetermined values ​​for the metric, the value of a sample for the mixed region being determined based on a value of a sample for a first portion of the target region and a value of a sample for a second portion of the target region; generating a bitstream based on the value for the metric; and storing the bitstream in a non-transitory computer-readable recording medium.

[0010] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become more apparent through the following detailed description with reference to the accompanying drawings. In the exemplary embodiments of the present disclosure, the same reference numerals generally refer to the same components.

[0012] Figure 1 A block diagram showing an example video encoding and decoding system according to some embodiments of the present disclosure is shown;

[0013] Figure 2 A block diagram of an example video encoder according to some embodiments of the present disclosure is shown;

[0014] Figure 3 A block diagram of an example video decoder according to some embodiments of the present disclosure is shown;

[0015] Figure 4 A schematic diagram showing an intra-frame prediction mode is shown;

[0016] Figure 5A A schematic diagram of the top reference is shown;

[0017] Figure 5BA schematic diagram of the left reference is shown;

[0018] Figure 6 A schematic diagram showing discontinuity when the orientation exceeds 45 degrees;

[0019] Fig. 7A A schematic diagram showing the definition of samples used by PDPC applied to diagonal top right intra mode;

[0020] Figure 7B A schematic diagram showing the definition of samples used by PDPC applied to the diagonal bottom left intra mode;

[0021] Figure 7C A schematic diagram showing the definition of samples used by PDPC applied to the adjacent diagonal upper right intra mode;

[0022] Fig.7D A schematic diagram showing the definition of samples used by PDPC applied to the adjacent diagonal lower left intra mode;

[0023] Figure 8 An example diagram showing four reference rows adjacent to a prediction block;

[0024] Fig. 9A and 9B An example of sub-division depending on block size is shown;

[0025] Fig.10 A schematic diagram of a matrix-weighted intra prediction process is shown;

[0026] Fig.11 A schematic diagram showing the positions of spatial merging candidates;

[0027] Fig.12 A schematic diagram showing candidate pairs considered for redundancy checking of spatial merging candidates;

[0028] Fig.13 A schematic diagram showing motion vector scaling for temporal merging candidates is shown;

[0029] Fig.14 A schematic diagram showing candidate positions for temporal merging candidates is shown;

[0030] Fig.15 A schematic diagram of the MMVD search point is shown;

[0031] Fig.16 A schematic diagram showing an extended CU region used in BDOF;

[0032] Fig.17 shows a schematic diagram for an illustration of a symmetric MVD mode;

[0033] Fig.18 Decoding side motion vector refinement is shown;

[0034] Fig.19 Schematic diagram showing the top and left neighboring blocks used in CIIP weight derivation;

[0035] Fig. 20 A schematic diagram showing an example of a GPM segmentation grouped by the same angle;

[0036] Fig.21 A schematic diagram showing unidirectional prediction MV selection for geometric partitioning mode;

[0037] Fig. 22 A schematic diagram showing an exemplary generation of a bending weight w0 using a geometric partitioning mode is shown;

[0038] Fig.23 A schematic diagram of a low frequency non-separable transform (LFNST) process is shown;

[0039] Fig.24 Examples of SBT locations, types, and transformation types are shown;

[0040] Fig.25 An example of a ROI for LFNST16 is shown;

[0041] Fig.26 An example of a ROI for LFNST8 is shown;

[0042] Fig. 27 A schematic diagram of discontinuity measurement is shown;

[0043] Fig.28 A schematic diagram of a luma mapping architecture with chroma scaling is shown;

[0044] Fig.29A A schematic diagram showing an example of two partitions of a hybrid CU according to an embodiment of the present disclosure;

[0045] Fig.29B A schematic diagram showing an example of mixing two top templates according to an embodiment of the present disclosure;

[0046] Fig.30 A flowchart of a method for video processing according to an embodiment of the present disclosure is shown;

[0047] Fig.31A An example of mixing two partitions according to an embodiment of the present disclosure is shown;

[0048] Fig.31B Another example of mixing two partitions according to an embodiment of the present disclosure is shown;

[0049] Fig.31C shows yet another example of mixing two partitions according to an embodiment of the present disclosure; and

[0050] Fig.32 A block diagram of a computing device is shown in which various embodiments of the present disclosure may be implemented.

[0051] Throughout the drawings, same or similar reference numbers generally refer to same or similar elements. DETAILED DESCRIPTION

[0052] The principle of the present disclosure will now be described with reference to some embodiments. It should be understood that these embodiments are described only for the purpose of illustrating and helping those skilled in the art to understand and implement the present disclosure, without implying any limitation on the scope of the present disclosure. In addition to the methods described below, the disclosure described herein can also be implemented in various ways.

[0053] In the following description and claims, unless otherwise defined, all scientific and technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present disclosure belongs.

[0054] References in this disclosure to "one embodiment," "an embodiment," "an example embodiment," etc. indicate that the described embodiment may include a particular feature, structure, or characteristic, but not every embodiment must include the particular feature, structure, or characteristic. Furthermore, these phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in conjunction with an example embodiment, whether or not explicitly described, it is considered to be within the knowledge of those skilled in the art to affect such feature, structure, or characteristic in relation to other embodiments.

[0055] It should be understood that although the terms "first" and "second" etc. can be used to describe various elements, these elements should not be limited to these terms. These terms are only used to distinguish one element from another element. For example, a first element can be referred to as a second element, and similarly, a second element can be referred to as a first element without departing from the scope of the exemplary embodiments. As used herein, the term "and / or" includes any and all combinations of one or more of the listed terms.

[0056] The terms used herein are only used for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments. As used herein, the singular forms "a", "an", and "the" are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms "including", "comprising", and / or "having" when used herein indicate the presence of the features, elements, and / or components, etc., but do not exclude the presence or addition of one or more other features, elements, components, and / or combinations thereof. Example Environment

[0057] Figure 1 1 is a block diagram illustrating an example video codec system 100 that may utilize the techniques of the present disclosure. As shown, the video codec system 100 may include a source device 110 and a destination device 120. The source device 110 may also be referred to as a video encoding device, and the destination device 120 may also be referred to as a video decoding device. In operation, the source device 110 may be configured to generate encoded video data, and the destination device 120 may be configured to decode the encoded video data generated by the source device 110. The source device 110 may include a video source 112, a video encoder 114, and an input / output (I / O) interface 116.

[0058] The video source 112 may include a source such as a video capture device. Examples of a video capture device include, but are not limited to, an interface for receiving video data from a video content provider, a computer graphics system for generating video data, and / or a combination thereof.

[0059] The video data may include one or more pictures. The video encoder 114 encodes the video data from the video source 112 to generate a bitstream. The bitstream may include a bit sequence that forms a coded representation of the video data. The bitstream may include coded pictures and associated data. The coded pictures are coded representations of pictures. The associated data may include sequence parameter sets, picture parameter sets, and other grammatical structures. The I / O interface 116 may include a modulator / demodulator and / or a transmitter. The coded video data may be directly transmitted to the destination device 120 via the network 130A via the I / O interface 116. The coded video data may also be stored on a storage medium / server 130B for access by the destination device 120.

[0060] The destination device 120 may include an I / O interface 126, a video decoder 124, and a display device 122. The I / O interface 126 may include a receiver and / or a modem. The I / O interface 126 may obtain encoded video data from the source device 110 or the storage medium / server 130B. The video decoder 124 may decode the encoded video data. The display device 122 may display the decoded video data to the user. The display device 122 may be integrated with the destination device 120, or may be outside the destination device 120, which is configured to be connected to an external display device interface.

[0061] The video encoder 114 and the video decoder 124 may operate according to a video compression standard, such as the High Efficiency Video Codec (HEVC) standard, the Versatile Video Codec (VVC) standard, and other existing and / or future standards.

[0062] Figure 2is a block diagram showing an example of a video encoder 200 according to some embodiments of the present disclosure, which may be Figure 1 An example of a video encoder 114 in the system 100 is shown.

[0063] Video encoder 200 may be configured to implement any or all of the techniques of this disclosure. Figure 2 In the example of , video encoder 200 includes multiple functional components. The techniques described in this disclosure can be shared between the various components of video encoder 200. In some examples, a processor can be configured to perform any or all of the techniques described in this disclosure.

[0064] In some embodiments, the video encoder 200 may include a partitioning unit 201, a prediction unit 202, a residual generation unit 207, a transformation unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transformation unit 211, a reconstruction unit 212, a buffer 213 and an entropy coding and decoding unit 214, and the prediction unit 202 may include a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205 and an intra-frame prediction unit 206.

[0065] In other examples, the video encoder 200 may include more, fewer, or different functional components. In one example, the prediction unit 202 may include an intra-block copy (IBC) unit. The IBC unit may perform prediction in an IBC mode, where at least one reference picture is a picture where the current video block is located.

[0066] Furthermore, although some components (such as the motion estimation unit 204 and the motion compensation unit 205) may be integrated, for the purpose of explanation, these components are described in detail below. Figure 2 are shown separately in the example.

[0067] The partitioning unit 201 may partition a picture into one or more video blocks. The video encoder 200 and the video decoder 300 (which will be discussed in detail below) may support various video block sizes.

[0068] The mode selection unit 203 may select one of a plurality of codec modes (intra-frame coding or inter-frame coding), for example, based on the error result, and provide the generated intra-frame codec block or inter-frame codec block to the residual generation unit 207 to generate residual block data, and to the reconstruction unit 212 to reconstruct the codec block for use as a reference picture. In some examples, the mode selection unit 203 may select a combination of intra-frame and inter-frame prediction (CIIP) modes, where the prediction is based on an inter-frame prediction signal and an intra-frame prediction signal. In the case of inter-frame prediction, the mode selection unit 203 may also select a resolution for the motion vector (e.g., sub-pixel precision or integer pixel precision) for the block.

[0069] To perform inter-frame prediction on the current video block, the motion estimation unit 204 may generate motion information for the current video block by comparing the current video block with one or more reference frames from the buffer 213. The motion compensation unit 205 may determine a predicted video block for the current video block based on the motion information and decoded samples of pictures from the buffer 213 other than the picture associated with the current video block.

[0070] The motion estimation unit 204 and the motion compensation unit 205 may perform different operations on the current video block, for example, depending on whether the current video block is in an I slice, a P slice, or a B slice. As used herein, an "I slice" may refer to a portion of a picture consisting of macroblocks, all of which are based on macroblocks within the same picture. Furthermore, as used herein, in some aspects, a "P slice" and a "B slice" may refer to a portion of a picture consisting of macroblocks that are independent of macroblocks in the same picture.

[0071] In some examples, the motion estimation unit 204 may perform unidirectional prediction on the current video block, and the motion estimation unit 204 may search the reference pictures of list 0 or list 1 to find the reference video block for the current video block. The motion estimation unit 204 may then generate a reference index and a motion vector, the reference index indicating the reference picture in list 0 or list 1 containing the reference video block, and the motion vector indicating the spatial displacement between the current video block and the reference video block. The motion estimation unit 204 may output the reference index, the prediction direction indicator, and the motion vector as the motion information of the current video block. The motion compensation unit 205 may generate a predicted video block for the current video block based on the reference video block indicated by the motion information of the current video block.

[0072] Alternatively, in other examples, the motion estimation unit 204 may perform bidirectional prediction on the current video block. The motion estimation unit 204 may search the reference pictures in list 0 for a reference video block for the current video block, and may also search the reference pictures in list 1 for another reference video block for the current video block. The motion estimation unit 204 may then generate a plurality of reference indexes and a plurality of motion vectors, the plurality of reference indexes indicating a plurality of reference pictures in list 0 and list 1 containing a plurality of reference video blocks, and the plurality of motion vectors indicating a plurality of spatial displacements between the plurality of reference video blocks and the current video block. The motion estimation unit 204 may output the plurality of reference indexes and the plurality of motion vectors of the current video block as motion information of the current video block. The motion compensation unit 205 may generate a predicted video block for the current video block based on the plurality of reference video blocks indicated by the motion information of the current video block.

[0073] In some examples, motion estimation unit 204 may output a complete set of motion information for use in a decoding process by a decoder. Alternatively, in some embodiments, motion estimation unit 204 may signal motion information of a current video block with reference to motion information of another video block. For example, motion estimation unit 204 may determine that motion information of a current video block is sufficiently similar to motion information of a neighboring video block.

[0074] In one example, motion estimation unit 204 may indicate to video decoder 300 a value in a syntax structure associated with the current video block that indicates that the current video block has the same motion information as another video block.

[0075] In another example, the motion estimation unit 204 may identify another video block and a motion vector difference (MVD) in a syntax structure associated with the current video block. The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the indicated video block. The video decoder 300 may use the motion vector of the indicated video block and the motion vector difference to determine the motion vector of the current video block.

[0076] As discussed above, the video encoder 200 may signal motion vectors in a predictive manner.Two examples of prediction signaling techniques that may be implemented by the video encoder 200 include Advanced Motion Vector Prediction (AMVP) and merge mode signaling.

[0077] The intra prediction unit 206 may perform intra prediction on the current video block. When performing intra prediction on the current video block, the intra prediction unit 206 may generate prediction data for the current video block based on decoded samples of other video blocks in the same picture. The prediction data for the current video block may include a prediction video block and various syntax elements.

[0078] The residual generation unit 207 may generate residual data for the current video block by subtracting (e.g., indicated by a minus sign) the prediction video block(s) of the current video block from the current video block. The residual data of the current video block may include residual video blocks corresponding to different sample portions of samples in the current video block.

[0079] In other examples, such as in skip mode, there may be no residual data for the current video block, and the residual generation unit 207 may not perform a subtraction operation.

[0080] Transform unit 208 may generate one or more transform coefficient video blocks for a current video block by applying one or more transforms to the residual video block associated with the current video block.

[0081] After transform unit 208 generates a transform coefficient video block associated with the current video block, quantization unit 209 may quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values ​​associated with the current video block.

[0082] The inverse quantization unit 210 and the inverse transform unit 211 may apply inverse quantization and inverse transform to the transform coefficient video block, respectively, to reconstruct a residual video block from the transform coefficient video block. The reconstruction unit 212 may add the reconstructed residual video block to corresponding samples from one or more prediction video blocks generated by the prediction unit 202 to generate a reconstructed video block associated with the current video block for storage in the buffer 213.

[0083] After reconstruction unit 212 reconstructs the video block, a loop filtering operation may be performed to reduce video blocking artifacts in the video block.

[0084] The entropy coding unit 214 may receive data from other functional components of the video encoder 200. When the data is received, the entropy coding unit 214 may perform one or more entropy coding operations to generate entropy coded data and output a bitstream including the entropy coded data.

[0085] Figure 3 is a block diagram showing an example of a video decoder 300 according to some embodiments of the present disclosure, which may be Figure 1 An example of a video decoder 124 in the system 100 is shown.

[0086] Video decoder 300 may be configured to perform any or all of the techniques of this disclosure. Figure 3 In the example of , video decoder 300 includes multiple functional components. The techniques described in this disclosure can be shared between the various components of video decoder 300. In some examples, a processor can be configured to perform any or all of the techniques described in this disclosure.

[0087] exist Figure 3 In the example of , the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, a reconstruction unit 306 and a buffer 307. In some examples, the video decoder 300 can perform a decoding process that is generally opposite to the encoding process described with respect to the video encoder 200.

[0088] The entropy decoding unit 301 can retrieve the encoded bitstream. The encoded bitstream may include entropy encoded video data (e.g., encoded video data blocks). The entropy decoding unit 301 can decode the entropy encoded video data, and the motion compensation unit 302 can determine motion information from the entropy decoded video data, the motion information including motion vectors, motion vector precision, reference picture list indexes, and other motion information. The motion compensation unit 302 can determine the information, for example, by performing AMVP and merge mode. AMVP is used, including deriving several most likely candidates based on data and reference pictures from adjacent PBs. The motion information typically includes horizontal and vertical motion vector displacement values, one or two reference picture indexes, and in the case of prediction areas in B strips, also includes an identification of which reference picture list is associated with each index. As used herein, in some aspects, "merge mode" may refer to deriving motion information from blocks that are adjacent in space or time.

[0089] The motion compensation unit 302 may generate a motion compensated block, possibly performing interpolation based on an interpolation filter.Identifiers for the interpolation filters used with sub-pixel precision may be included in the syntax elements.

[0090] The motion compensation unit 302 may calculate interpolation values ​​for sub-integer pixels of a reference block using interpolation filters used by the video encoder 200 during encoding of the video block. The motion compensation unit 302 may determine the interpolation filters used by the video encoder 200 based on received syntax information, and the motion compensation unit 302 may use the interpolation filters to generate a prediction block.

[0091] The motion compensation unit 302 may use at least part of the syntax information to determine the size of blocks used to encode (multiple) frames and / or (multiple) slices of the encoded video sequence, partition information describing how each macroblock of a picture of the encoded video sequence is partitioned, a mode indicating how each partition is encoded, one or more reference frames (and reference frame lists) for each inter-frame coding block, and other information for decoding the encoded video sequence. As used herein, in some aspects, a "slice" may refer to a data structure that can be decoded independently of other slices of the same picture in terms of entropy coding, signal prediction, and residual signal reconstruction. A slice may be an entire picture, or it may be a region of a picture.

[0092] The intra prediction unit 303 may use, for example, an intra prediction mode received in the bitstream to form a prediction block from spatially neighboring blocks. The inverse quantization unit 304 inverse quantizes (i.e., dequantizes) the quantized video block coefficients provided in the bitstream and decoded by the entropy decoding unit 301. The inverse transform unit 305 applies an inverse transform.

[0093] The reconstruction unit 306 may obtain the decoded block, for example, by adding the residual block to the corresponding prediction block generated by the motion compensation unit 302 or the intra prediction unit 303. If necessary, a deblocking filter may also be applied to filter the decoded block to remove blocking artifacts. The decoded video block is then stored in a buffer 307, which provides reference blocks for subsequent motion compensation / intra prediction, and the buffer 307 also generates the decoded video for presentation on a display device.

[0094] Some example embodiments of the present disclosure will be described in detail below. It should be noted that the section titles used in this document are for ease of understanding, and the embodiments disclosed in the section are not limited to that section. In addition, although some embodiments are described with reference to general video codecs or other specific video codecs, the disclosed technology is also applicable to other video coding and decoding technologies. In addition, although some embodiments describe the video encoding steps in detail, it should be understood that the corresponding decoding steps of canceling the encoding will be implemented by the decoder. In addition, the term video processing includes video coding or compression, video decoding or decompression, and video transcoding, in which video pixels are represented from one compression format to another compression format or at different compression bit rates. 1. Brief Overview The present disclosure relates to video coding technology. Specifically, the present disclosure is about inter-frame and intra-frame prediction in image / video coding. It can be applied to existing video coding standards such as HEVC and VVC. It can also be applied to future video coding standards or video codecs. 2. Introduction Video codec standards have evolved primarily through the development of the well-known ITU-T and ISO / IEC standards. ITU-T developed H.261 and H.263, ISO / IEC developed MPEG-1 and MPEG-4 Visual, and the two organizations jointly developed H.262 / MPEG-2 Video and H.264 / MPEG-4 Advanced Video Codec (AVC) and H.265 / HEVC standards. Since H.262, video codec standards have been based on a hybrid video codec structure, in which temporal prediction plus transform codec is used. In order to explore future video codec technologies beyond HEVC, VCEG and MPEG jointly founded the Joint Video Exploration Team (JVET) in 2015. JVET meetings are held simultaneously every quarter, and the new video codec standard was officially named Versatile Video Codec (VVC) at the JVET meeting in April 2018, and the first version of the VVC Test Model (VTM) was released at that time. The VVC working draft and the test model VTM are updated after each meeting. The VVC project achieved technical completion (FDIS) at the July 2020 meeting. 2.1. Existing Codec Tools 2.1.1. Intra-frame prediction 2.1.1.1. Intra-mode codec with 67 intra-prediction modes To capture arbitrary edge directions present in natural videos, the number of directional intra modes in VVC is extended to 65 from the 33 used in HEVC. Figure 4 New directional modes not in HEVC are depicted as red dashed arrows, and planar and DC modes remain unchanged. These denser directional intra prediction modes apply to all block sizes and both luma and chroma intra prediction. In VVC, for non-square blocks, several traditional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes. In HEVC, each intra-coded block has a square shape, and the length of each of its sides is a power of 2. Therefore, no division operation is required to generate intra predictors using DC mode. In VVC, blocks can have a rectangular shape, which in general requires the use of a division operation for each block. To avoid division operations for DC prediction, only the longer sides are used to calculate the average value for non-square blocks. 2.1.1.2. Intra-mode coding In order to keep the complexity of the most probable mode (MPM) list generation low, a 6MPM intra mode encoding and decoding method is adopted considering two available adjacent intra modes. The following three aspects are considered to construct the MPM list: – Default intra mode; – Neighboring intra mode; – Export intra mode. Regardless of whether the MRL and ISP codecs are applied, a unified 6-MPM list is used for intra blocks. The MPM list is constructed based on the intra modes of the left and upper neighboring blocks. Assuming that the mode on the left is denoted as Left and the mode of the upper block is denoted as Above, the unified MPM list is constructed as follows: – When a neighboring block is not available, its intra mode is set to planar by default. – If both Left and Above modes are non-corner modes: – MPM list → {plane, DC, V, H, V-4, V+4} – If one of the Left and Above modes is a corner mode and the other is a non-corner mode: – Set mode Max as the larger mode in Left and Above – MPM list → {Plane, Max, DC, Max-1, Max+1, Max-2} – If Left and Above are both angles, and they are different: – Set mode Max as the larger mode in Left and Above – If the difference between the Left mode and the Above mode is in the range of 2 to 62 (inclusive) – MPM list → {Plane, Left, Above, DC, Max-1, Max+1} -otherwise – MPM List → {Flat, Left, Above, DC, Max-2, Max+2} – If Left and Above are both angles, and they are the same: – MPM list → {Plane, Left, Left-1, Left+1, DC, Left-2} In addition, the first binary bit of the MPM index codeword is CABAC context coded. A total of three contexts are used, corresponding to whether the current intra block is MRL-enabled, ISP-enabled, or a regular intra block. During the 6MPM list generation process, pruning is used to remove repeated patterns so that only unique patterns can be included in the MPM list. For entropy coding and decoding of the 61 non-MPM patterns, truncated binary code (TBC) is used. 2.1.1.3. Wide-angle intra prediction for non-square blocks Conventional angular intra prediction directions are defined as 45 degrees to -135 degrees clockwise. In VVC, for non-square blocks, several conventional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes. The replaced modes are signaled using the original mode index, which is remapped to the index of the wide-angle mode after parsing. The total number of intra prediction modes remains unchanged, i.e. 67, and the intra mode encoding and decoding method remains unchanged. Figure 5A and 5B Reference samples for wide-angle intra prediction are shown. To support these prediction directions, a top reference of length 2W+1 and a left reference of length 2H+1 are defined as Figure 5A and 5B shown. The number of alternative modes in the wide-angle direction mode depends on the aspect ratio of the block. The alternative intra prediction modes are shown in Table 1. Table 1 - Intra prediction modes replaced by wide angle mode Aspect Ratio Alternative intra prediction modes W / H==16 Mode 12, 13, 14, 15 W / H==8 Mode 12, 13 W / H==4 Mode 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 W / H==2 Mode 2, 3, 4, 5, 6, 7, W / H==1 none W / H==1 / 2 Mode 61, 62, 63, 64, 65, 66 W / H==1 / 4 Modes 57, 58, 59, 60, 61, 62, 63, 64, 65, 66 W / H==1 / 8 Mode 55, 56 W / H==1 / 16 Modes 53, 54, 55, 56 like Figure 6As shown, in the case of wide-angle intra prediction, two vertically adjacent prediction samples can use two non-adjacent reference samples. Therefore, a low-pass reference sample filter and side smoothing are applied to wide-angle prediction to reduce the increased gap Δp α negative impact. If the wide-angle mode represents a non-fractional offset. There are 8 modes in the wide-angle mode that meet this condition, and the 8 modes are [-14, -12, -10, -6, 72, 76, 78, 80]. When a block is predicted by these modes, the samples in the reference buffer are directly copied without applying any interpolation. With this modification, the number of samples that need to be smoothed is reduced. In addition, it aligns the design of non-fractional modes in the traditional prediction mode and the wide-angle mode. In VVC, 4:2:2 and 4:4:4 as well as 4:2:0 chroma formats are supported. The chroma derivation mode (DM) derivation table for the 4:2:2 chroma format was originally ported from HEVC, which expanded the number of entries from 35 to 67 to keep pace with the expansion of intra prediction modes. Since the HEVC specification does not support prediction angles below -135 degrees and above 45 degrees, the luma intra prediction modes ranging from 2 to 5 are mapped to 2. Therefore, the chroma DM derivation table for the 4:2:2 chroma format is updated by replacing some values ​​of the entries of the mapping table to more accurately convert the prediction angles of the chroma blocks. 2.1.1.4. Mode-dependent intra-frame smoothing (MDIS) A four-tap frame interpolation filter is used to improve the accuracy of directional intra prediction. In HEVC, a two-tap linear interpolation filter is used to generate intra prediction blocks in directional prediction mode (i.e., excluding planar and DC predictors). In VVC, a simplified 6-bit 4-tap Gaussian interpolation filter is used only for directional intra mode. The non-directional intra prediction process is unchanged. The selection of the 4-tap filter is performed according to the MDIS condition of the directional intra prediction mode that provides non-fractional displacement, i.e., all the following directional modes are excluded: 2, HOR_IDX, DIA_IDX, VER_IDX, 66. Depending on the intra prediction mode, the following reference sample processing is performed: – Directional intra prediction modes are classified into one of the following groups: – vertical mode or horizontal mode (HOR_IDX, VER_IDX), – Diagonal mode, indicating that the angle is a multiple of 45 degrees (2, DIA_IDX, VDIA_IDX), – Other directional modes; – If the directional intra prediction mode is classified as belonging to group A, no filter is applied to the reference samples to generate the predicted samples; – Otherwise, if the mode belongs to group B, the [1, 2, 1] reference sample filter may be applied (depending on the MDIS condition) to the reference samples to further copy these filtered values ​​to the intra predictor according to the selected direction, But no interpolation filter is applied; Otherwise, if the mode is classified as belonging to group C, only the intra reference sample interpolation filter is applied to the reference samples to generate predicted samples at fractional or integer positions falling between the reference samples according to the selected direction (no reference sample filtering is performed). 2.1.1.5. Position-dependent intra prediction combination In VVC, the intra prediction results for DC, planar, and several angular modes are further modified by the Position-Dependent Intra Prediction Combination (PDPC) method. PDPC is an intra prediction method that calls for a combination of HEVC-style intra prediction without filtering and with filtering boundary reference samples. PDPC is applied to the following intra modes without signaling: planar, DC, horizontal, vertical, bottom-left corner mode and its eight neighboring corner modes, and top-right corner mode and its eight neighboring corner modes. The prediction sample pred(x', y') is predicted using a linear combination of the intra prediction mode (DC, planar, angular) and the reference sample according to the following equation 3-8: pred(x',y')=(wL×R -1，y’ +wT×R x’，-1 -wTL×R -1，-1 +(64-wL-wT+wTL)×pred(x',y')+32)>>6 (2-1) Where R x，-1 , R -1，y denote the reference samples located at the top and left boundaries of the current sample (x, y), and R -1，-1 Represents the reference sample located at the upper left corner of the current block. If PDPC is applied to DC, planar, horizontal and vertical intra modes, no additional boundary filtering is required, as required in the case of HEVC DC mode boundary filtering or horizontal / vertical mode edge filtering. The PDPC process for DC mode and planar mode is the same, and clipping operations are avoided. For angular modes, the pdpc scaling factor is adjusted so that no range check is required, and the angle condition for enabling pdpc is removed (using scaling >= 0). In addition, the PDPC weights are based on 32 in all angular mode cases. The PDPC weights depend on the prediction mode, as shown in Table 2. PDPC is applied to blocks with width and height both greater than or equal to 4. FIG7 shows the reference samples (R x，-1 , R-1，y and R -1，-1 ). The prediction sample pred(x', y') is located at (x', y') in the prediction block. For example, the reference sample R x，-1 The coordinate x of is given by: x = x' + y' + 1, with reference to sample R -1，y The coordinate y of is similarly given by: y = x' + y' + 1 for the diagonal mode. For other angular modes, the reference sample R x，-1 and R -1，y can be located at fractional sample positions. In this case, the sample value of the nearest integer sample position is used. Table 2 - Example of PDPC weights according to prediction mode Prediction Model w wxya wxya Diagonally upper right 16>>((y’<<1)>>shift) 16>>((x’<<1)>>shift) 0 Diagonal lower left 16>>((y’<<1)>>shift) 16>>((x’<<1)>>shift) 0 Adjacent diagonal upper right 32>>((y’<<1)>>shift) 0 0 Adjacent diagonal lower left 0 32>>((x’<<1)>>shift) 0 2.1.1.6. Multiple Reference Line (MRL) Intra Prediction Multiple reference line (MRL) intra prediction uses more reference lines for intra prediction. Figure 8 In Figure 1, an example of 4 reference lines is depicted, where the samples of segments A and F are not extracted from the reconstructed neighboring samples, but are filled with the closest samples from segments B and E, respectively. HEVC intra picture prediction uses the nearest reference line (i.e., reference line 0). In MRL, 2 additional lines are used (reference line 1 and reference line 3). The index of the selected reference row (mrl_idx) is signaled and used to generate the intra predictor. For reference row indices greater than 0, only additional reference row modes are included in the MPM list, and only the mpm index is signaled without the remaining modes. The reference row index is signaled before the intra prediction mode, and if a non-zero reference row index is signaled, the intra prediction mode does not include the planar mode. MRL is disabled for the first row of a block within a CTU to prevent the use of extended reference samples outside the current CTU row. In addition, PDPC will be disabled when additional rows are used. For MRL mode, the derivation of DC values ​​in DC intra prediction mode for non-zero reference row index is aligned with the derivation for reference row index 0. MRL requires storage of the CTU and 3 adjacent luma reference rows to generate the prediction. The Cross Component Linear Model (CCLM) tool also requires 3 adjacent luma reference rows for its downsampling filtering. The definition of MRL using the same 3 rows is consistent with CCLM to reduce the storage requirements of the decoder. 2.1.1.7. Intra-frame Sub-Partitioning (ISP) Intra sub-partitioning (ISP) divides the luma intra prediction block into 2 or 4 sub-partitions vertically or horizontally depending on the block size. For example, the minimum block size of ISP is 4×8 (or 8×4). If the block size is larger than 4×8 (or 8×4), the corresponding block will be divided into four sub-partitions. It can be noted that M×128 (M≤64) and 128×N (N≤64) ISP blocks may generate potential problems for 64×64 VDPU. For example, an M×128 CU in a single-tree case has one M×128 luma TB and two corresponding Chroma TB. If the CU uses ISP, then the luma TB will be split into 4 M×32TBs (only horizontal splitting is possible), each of which is smaller than a 64×64 block. However, in the current ISP design, the chroma blocks are indivisible. Therefore, the size of both chroma components will be larger than a 32×32 block. Similarly, a 128×NCU using ISP can also create a similar situation. Therefore, both situations are problems for a 64×64 decoder pipeline. Therefore, the CU size that can use ISP is limited to a maximum value of 64×64. Fig. 9A and 9B Examples of two possibilities are shown. All sub-partitions satisfy the condition of having at least 16 samples. In ISP, 1xN / 2xN sub-block prediction is not allowed to depend on the reconstructed values ​​of previously decoded 1xN / 2xN sub-blocks of the codec block, so that the minimum prediction width of the sub-block becomes four samples. For example, an 8xN (N>4) codec block encoded using ISP with vertical partitioning is partitioned into two prediction regions of size 4xN and four transforms of size 2xN. In addition, a 4xN codec block encoded using ISP with vertical partitioning is predicted using a full 4xN block; each of the four transforms is used. Although 1xN and 2xN transform sizes are allowed, it is asserted that the transforms of these blocks in the 4xN region can be performed in parallel. For example, when a 4xN prediction region contains four 1xN transforms, there is no transform in the horizontal direction; the transform in the vertical direction can be performed as a single 4xN transform in the vertical direction. Similarly, when a 4xN prediction region contains two 2xN transform blocks, the transform operations of the two 2xN blocks in each direction (horizontal and vertical) can be performed in parallel. Therefore, processing these smaller blocks does not increase latency compared to processing intra blocks for 4x4 regular codec. Fig. 9A An example of sub-partitioning for 4×8 CU and 8×4 CU is shown, and Fig. 9B Examples of sub-partitioning for CUs other than 4x8, 8x4, and 4x4 are shown. Table 3 - Entropy coding coefficient group size Block size Coefficient group size 1×N, N≥16 1×16 N×1, N≥16 16×1 2×N, N≥8 2×8 N×2, N≥8 8×2 All other possible M×N situations 4×4 For each sub-partition, a reconstructed sample is obtained by adding the residual signal to the prediction signal. Here, the residual signal is generated by processes such as entropy decoding, inverse quantization, and inverse transformation. Therefore, the reconstructed sample values ​​of each sub-partition can be used to generate a prediction for the next sub-partition, and each sub-partition is processed repeatedly. In addition, the first sub-partition to be processed is the upper left sample containing the CU, and then the sub-partitions continue downward (horizontal partitioning) or to the right (vertical partitioning). Therefore, the reference samples used to generate the sub-partition prediction signal are only located on the left and above the row. All sub-partitions share the same intra-frame mode. The following is a summary of the interaction of ISP with other codec tools. – Multiple Reference Line (MRL): If the MRL index of a block is not 0, the ISP codec mode will be inferred to be 0, so the ISP mode information will not be sent to the decoder. – Entropy coding coefficient group size: As shown in Table 3, the size of the entropy coding sub-block has been modified to have 16 samples in all possible cases. It is worth noting that the new size only affects blocks produced by ISP where one of the dimensions is less than 4 samples. In all other cases, the coefficient group remains 4×4 in size. – CBF codec: It is assumed that at least one subpartition has a non-zero CBF. Thus, if n is the number of subpartitions, and the first n-1 subpartitions already yield zero CBF, the CBF of the nth subpartition is inferred to be 1. – MPM usage: The MPM flag will be inferred as the one for blocks encoded and decoded via ISP mode, and the MPM list is modified to exclude DC mode and prioritize horizontal intra mode for ISP horizontal partitioning and vertical intra mode for ISP vertical partitioning. – Transform size restriction: All ISP transforms with length greater than 16 points use DCT-II. –PDPC: When the CU uses ISP codec mode, the PDPC filter will not be applied to the resulting sub-partition. –MTS flag: If the CU uses ISP codec mode, the MTS CU flag will be set to 0 and will not be sent to the decoder. Therefore, the encoder will not perform RD tests on the different available transforms for each result sub-partition. The transform selection for ISP mode will instead be fixed and will be chosen based on the intra mode used, the processing order and the block size. Therefore, no signaling is required. For example, let t H and t V The horizontal and vertical transforms are selected for the w×h sub-partition, where w is the width and h is the height. The transforms are then selected according to the following rules: - If w=1 or h=1, there is no horizontal transform or vertical transform, respectively. – If w=2 and w>32, t H=DCT-II. – If h=2 and h>32, t V =DCT-II. – Otherwise, the transformation is selected as shown in Table 4. Table 4 - Transform selection depending on intra mode In ISP mode, all 67 intra prediction modes are allowed. PDPC is also applied if the corresponding width and height are at least 4 samples long. In addition, the conditions for intra interpolation filter selection no longer exist, and in ISP mode, cubic (DCT-IF) filtering is always used for fractional position interpolation. 2.1.1.8. Matrix Weighted Intra Prediction (MIP) The matrix weighted intra prediction (MIP) method is an intra prediction technique newly added to VVC. To predict samples of a rectangular block of width W and height H, the matrix weighted intra prediction (MIP) takes a row of H reconstructed neighboring boundary samples on the left side of the block and a row of W reconstructed neighboring boundary samples above the block as input. If the reconstructed samples are not available, they are generated like traditional intra prediction. The generation of the prediction signal is based on the following three steps, which are averaging, matrix-vector multiplication, and linear interpolation, as shown in Fig.10 shown. 3.3.6.1 Average Neighboring Samples Among the boundary samples, four samples or eight samples are selected by averaging based on the block size and shape. Specifically, input boundary bdry top and bdry left It is reduced to a smaller size by averaging the neighboring boundary samples according to a predefined rule that depends on the size of the block. and Then, the two reduced boundaries and Connect to the reduced boundary vector bdry red , so the size of the reduced boundary vector for blocks of shape 4×4 is four, and the size of the reduced boundary vector for blocks of all other shapes is eight. If mode refers to a MIP mode, this connection is defined as follows: 3.3.6.2 Matrix Multiplication The average sample is taken as input, a matrix-vector multiplication is performed, and then an offset is added. The result is a downscaled prediction signal on a subsampled set of samples in the original block. red Generate a reduced prediction signal pred red ,, the reduced prediction signal is of width W red and height H redHere, W red and H red is defined as: The reduced prediction signal pred is calculated by computing the matrix-vector product and adding an offset red : pred red =A.bdry red +b. Here, A is a matrix, and if W = H = 4, then it has W red ·H red rows and 4 columns, in all other cases there are 8 columns. b is W red ·H red The matrix A and the offset vector b are taken from one of the sets S0, S1, S2. The index idx=idx(W,H) is defined as follows: Here, each coefficient of matrix A is represented with 8 bits of precision. Set S0 consists of 16 matrices Each matrix has 16 rows and 4 columns, and 16 offset vectors Each offset vector has a size of 16. This set of matrices and offset vectors is used for blocks of size 4×4. Set S1 consists of 8 matrices Each matrix has 16 rows and 8 columns, and 8 offset vectors Each offset vector has a size of 16. Set S2 consists of 6 matrices Each matrix has 64 rows and 8 columns, and 6 offset vectors Each offset vector has a size of 64. 3.3.6.3 Interpolation The prediction signals at the remaining positions are generated from the prediction signals on the subsampled set by linear interpolation, which is a single-step linear interpolation in each direction. The interpolation is performed first in the horizontal direction and then in the vertical direction, regardless of the shape or size of the block. 3.3.6.4 Signaling of MIP mode and coordination with other codec tools For each codec unit (CU) in intra mode, a flag is sent indicating whether the MIP mode is to be applied. If the MIP mode is to be applied, the MIP mode (predModeIntra) is signaled. For the MIP mode, the transposed flag (isTransposed) is used to determine whether the mode is transposed, and the MIP mode identifier (modeId) is used to determine the matrix used for a given MIP mode, which is derived as follows: isTransposed=predModeIntra&1 modeId=predModeIntra>>1 (2-6) The MIP codec mode is coordinated with other codec tools by considering the following aspects: - Enable LFNST on MIPs on large blocks. LFNST transform in planar mode is used here. - Reference sample derivation for MIP is performed like conventional intra prediction mode. - For the upsampling step used in MIP prediction, the original reference samples are used instead of the downsampled samples. - Perform clipping before upsampling instead of after upsampling. - MIPs are allowed to be up to 64x64 regardless of the maximum transform size. - The number of MIP modes is 32 for sizeld=0, 16 for sizeld=1 and 12 for sizeld=2. 2.1.2. Inter-frame prediction For each inter-predicted CU, the motion parameters include motion vectors, reference picture indices and reference picture list usage indices, as well as additional information required by the new codec features of VVC that will be used for inter-prediction sample generation. The motion parameters can be transmitted through signals in an explicit or implicit manner. When a CU is encoded and decoded in skip mode, the CU is associated with one PU and has no significant residual coefficients, no encoded motion vector increments or reference picture indices. A merge mode is specified, whereby the motion parameters of the current CU are obtained from neighboring CUs, which include spatial candidates and temporal candidates, as well as additional arrangements introduced in VVC. Merge mode can be applied to any inter-predicted CU, not just skip mode. An alternative to merge mode is the explicit transmission of motion parameters, where the motion vectors, the corresponding reference picture indices for each reference picture list and the reference picture list usage flags, as well as other required information, are explicitly transmitted through signals for each CU. In addition to the inter-frame coding and decoding functions in HEVC, WC also includes some new and refined inter-frame prediction coding and decoding tools, as follows: - Extended Merger Forecast. – Merge Mode with MVD (MMVD). – Symmetrical MVD (SMVD) signaling. – Affine motion compensated prediction. – Sub-block based temporal motion vector prediction (SbTMVP). – Adaptive Motion Vector Resolution (AMVR). – Sports Field Storage: 1 / 16 thLuma samples MV storage and 8x8 motion field compression. – Bi-prediction with CU level weight (BCW). – Bidirectional Optical Flow (BDOF). – Decoder side motion vector refinement (DMVR). – Geometric Partitioning Mode (GPM). – Combined Inter and Intra Prediction (CIIP). The text below provides details of those inter prediction methods specified in VVC. 2.1.2.1. Extended Merge Prediction In VVC, the merge candidate list is constructed by including the following five types of candidates in order: 1) Spatial MVP from spatially neighboring CUs. 2) Time MVP from the same CU. 3) Table from FIFO based on historical MVP. 4) Average MVP in pairs. 5) Zero MV. The size of the merge list is signaled in the sequence parameter set header, and the maximum allowed size of the merge list is 6. For each CU code in merge mode, the index of the best merge candidate is encoded and decoded using truncated unary binarization (TU). The first bin of the merge index is encoded and decoded using context, while bypass encoding and decoding is used for the other bins. The derivation process of merge candidates for each category is provided in this section. As in HEVC, VVC also supports parallel derivation of merge candidate lists for all CUs within a certain size region. 2.1.2.1.1. Spatial candidate export The derivation of spatial merge candidates in VVC is the same as that in HEVC, except that the positions of the first two merge candidates are swapped. Fig.11 Among the candidates at the positions shown, up to four merge candidates are selected. The derivation order is B0, A0, B1, A1 and B2. Position B2 is considered only when one or more CUs at positions B0, A0, B1 and A1 are not available (for example, because it belongs to another slice or tile) or is intra-coded. After the candidate at position A1 is added, a redundancy check is performed on the addition of the remaining candidates, which ensures that candidates with the same motion information are excluded from the list, thereby improving the coding efficiency. In order to reduce computational complexity, not all possible candidate pairs are considered in the mentioned redundancy check. Instead, only the possible candidate pairs are considered. Fig.12 The pairs are linked by arrows in , and candidates are added to the list only if the corresponding candidates used for redundancy checking do not have the same motion information. 2.1.2.1.2. Time candidate export In this step, only one candidate is added to the list. In particular, in the derivation of this temporal merge candidate, the scaled motion vector is derived based on the co-located CU belonging to the co-located reference picture. The reference picture list to be used for deriving the co-located CU is explicitly signaled in the slice header. Fig.13 As shown by the dashed line in , the scaled motion vector for the temporal merge candidate is obtained, which is scaled from the motion vector of the co-located CU using the POC distances, tb and td, where tb is defined as the POC difference between the reference picture of the current picture and the current picture, and td is defined as the POC difference between the reference picture of the co-located picture and the co-located picture. The reference picture index of the temporal merge candidate is set equal to zero. like Fig.14 As shown, the position of the temporal candidate is selected between candidates C0 and C1. If the CU at position C0 is not available and is intra-coded, or the CU at position C0 is outside the current row of the CTU, position C1 is used. Otherwise, position C0 is used in the derivation of the temporal merge candidate. 2.1.2.1.3. History-based Merge Candidate Derivation History-based MVP (HMVP) merge candidates are added to the merge list after spatial MVP and TMVP. In this method, the motion information of the previous codec block is stored in a table and used as the MVP of the current CU. A table with multiple HMVP candidates is maintained during the encoding / decoding process. The table is reset (cleared) when a new CTU row is encountered. Whenever there is a non-sub-block inter-coded CU, the associated motion information is added as a new HMVP candidate to the last entry of the table. The HMVP table size S is set to 6, which indicates that up to 6 history-based MVP (HMVP) candidates can be added to the table. When a new motion candidate is inserted into the table, a constrained first-in, first-out (FIFO) rule is used, where a redundancy check is first applied to find if there is an identical HMVP in the table. If found, the identical HMVP is removed from the table, and then all HMVP candidates are moved forward. HMVP candidates can be used in the merge candidate list construction process. The nearest HMVP candidates in the table are checked in order and inserted into the candidate list after the TMVP candidate. Redundancy check is applied to HMVP candidates to spatial or temporal merge candidates. To reduce the number of redundant checking operations, the following simplifications are introduced: 1. The number of HMPV candidates for merge list generation is set to (N<=4)?M:(8-N), where N indicates the number of existing candidates in the merge list and M indicates the number of available HMVP candidates in the table. 2. Once the total number of available merge candidates reaches the maximum allowed merge candidates minus 1, the merge candidate list construction process from HMVP is terminated. 2.1.2.1.4. Pairwise average merge candidate derivation The pairwise average candidate is generated by averaging the predefined candidate pairs in the existing merge candidate list, and the predefined pairs are defined as {(0, 1), (0, 2), (1, 2), (0, 3), (1, 3), (2, 3)}, where the number represents the merge index of the merge candidate list. The average motion vector is calculated separately for each reference list. If two motion vectors are available in one list, they are averaged even if they point to different reference pictures; if only one motion vector is available, it is used directly; if no motion vector is available, this list is kept invalid. When the merge list is not full after adding pairwise average merge candidates, zero MVPs will be inserted at the end until the maximum number of merge candidates is encountered. 2.1.2.2. Merging estimated regions The merge estimation region (MER) allows the merge candidate list to be derived independently for a CU in the same merge estimation region (MER). For generating the merge candidate list for the current CU, candidate blocks in the same MER as the current CU are not included. In addition, the update process for the history-based motion vector prediction candidate list is updated only when (xCb+cbWidth)>>Log2ParMrgLevel is greater than xCb>>Log2ParMrgLevel and (yCb+cbHeight)>>Log2parMrglevel is greater than (yCb>>Log2ParMrgLevel), and where (xCb, yCb) is the top left luma sample position of the current CU in the picture, and (cbWidth, cbHeight) is the CU size. The MER size is selected at the encoder end and signaled in the sequence parameter set as log2_parallel_merge_level_minus2. 2.1.2.3. Merge Mode with MVD (MMVD) In addition to the merge mode that uses implicitly derived motion information directly for prediction sample generation of the current CU, a merge mode with motion vector difference (MMVD) is introduced in VVC. The MMVD flag is signaled immediately after the skip flag and merge flag are sent to specify whether the MMVD mode is used for the CU. In MMVD, after a merge candidate is selected, it is further refined by the MVD information transmitted by the signal. Further information includes a merge candidate flag, an index specifying the magnitude of motion, and an index indicating the direction of motion. In MMVD mode, one of the first two candidates in the merge list is selected to be used as the MV basis. The merge candidate flag is transmitted by the signal to specify which one to use. The distance index specifies the motion magnitude information and indicates a predefined offset from the starting point. Fig.15 As shown in Table 5, the offset is added to the horizontal component or the vertical component of the starting MV. Table 5 - Relationship between distance index and predefined offset The direction index indicates the direction of the MVD relative to the starting point. The direction index can indicate four directions, as shown in Table 6. Note that the meaning of the MVD symbol can change depending on the information of the starting MV. When the starting MV is a unidirectional prediction MV or a bidirectional prediction MV, where both lists point to the same side of the current picture (i.e., both referenced POCs are greater than the POC of the current picture or both are less than the POC of the current picture), the symbol in Table 6 specifies the sign of the MV offset added to the starting MV. When the starting MV is a bidirectional prediction MV, where the two MVs point to different sides of the current picture (i.e., one referenced POC is greater than the POC of the current picture and the other referenced POC is less than the POC of the current picture), the symbol in Table 6 specifies the sign of the MV offset added to the list0 MV component of the starting MV, and the sign of the list1 MV has the opposite value. Table 6 - Sign of MV offset specified by direction index Direction Index 00 01 10 11 x-axis + - N / A N / A y-axis N / A N / A + - 2.1.2.4. Bidirectional Prediction with CU-Level Weights (BCW) In HEVC, a bidirectional prediction signal is generated by averaging two prediction signals obtained from two different reference pictures and / or using two different motion vectors. In VVC, the bidirectional prediction mode is extended beyond simple averaging to allow weighted averaging of the two prediction signals. P bi-pred =((8-w)*P0+w*P1+4)>>3 (2-7) Five weights, w∈{-2, 3, 4, 5, 10}, are allowed in weighted average bidirectional prediction. For each bidirectional prediction CU, the weight w is determined in one of two ways: 1) For non-merged CUs, the weight index is signaled after the motion vector difference; 2) For merged CUs, the weight index is inferred from neighboring blocks based on the merge candidate index. BCW is only applied to CUs with 256 or more luma samples (i.e., the CU width multiplied by the CU height is greater than or equal to 256). For low-latency pictures, all 5 weights will be used. For non-low-latency pictures, only 3 weights (w∈{3, 4, 5}) are used. – At the encoder, a fast search algorithm is applied to find the weight index without significantly increasing the encoder complexity. These algorithms are summarized below. When combined with AMVR, if the current picture is a low-delay picture, Then only the unequal weights of 1-pixel and 4-pixel motion vector precision are checked conditionally. - When combined with affine, affine ME will be performed for unequal weights if and only if the affine mode is selected as the current best mode. – When the two reference pictures in bidirectional prediction are the same, unequal weights are only checked conditionally. – Unequal weights are not searched when certain conditions are met, which depend on the POC distance between the current picture and its reference pictures, the codec QP, and the temporal level. The BCW weight index is encoded using a binary bit for context encoding followed by a binary bit for bypass encoding. The first binary bit for context encoding indicates whether equal weights are used; and if unequal weights are used, an additional binary bit is signaled using bypass encoding to indicate which unequal weights are used. Weighted prediction (WP) is a codec tool supported by the H.264 / AVC and HEVC standards for efficient encoding and decoding of video content in fading conditions. Support for WP has also been added to the VVC standard. WP allows weighting parameters (weights and offsets) to be transmitted by signal for each reference picture list L0 and each reference picture in list L1. Then, during motion compensation, the weights and offsets of the corresponding reference pictures are applied. WP and BCW are designed for different types of video content. In order to avoid interaction between WP and BCW (which will complicate the VVC decoder design), if the CU uses WP, the BCW weight index is not transmitted by signal, and w is inferred to be 4 (ie, equal weights are applied). For the merged CU, the weight index is inferred from the neighboring blocks based on the merge candidate index. This can be applied to the normal merge mode and the inherited affine merge mode. For the constructed affine merge mode, affine motion information is constructed based on the motion information of up to 3 blocks. The BCW index of the CU using the constructed affine merge mode is simply set equal to the BCW index of the first control point MV. In VVC, CIIP and BCW cannot be applied jointly to a CU. When a CU is encoded and decoded using CIIP mode, the BCW index of the current CU is set to 2, for example, equal weight. 2.1.2.5. Bidirectional Optical Flow (BDOF) The Bidirectional Optical Flow (BDOF) tool is included in VVC. BDOF, formerly known as BIO, is included in JEM. Compared to the JEM version, BDOF in VVC is a simpler version that requires much less computation, especially in terms of the number of multiplications and the size of the multipliers. BDOF is used to refine the bidirectional prediction signal of a CU at the 4×4 sub-block level. BDOF is applied to a CU if all of the following conditions are met: - The CU is encoded using "true" bi-prediction mode, ie, one of the two reference pictures precedes the current picture in display order, and the other of the two reference pictures follows the current picture in display order. – The distances from the two reference images to the current image (i.e., POC difference) are the same. – Both reference images are short-term reference images. –CU is not encoded or decoded using affine mode or ATMVP merge mode. – CU has more than 64 luma samples. – Both CU height and CU width are greater than or equal to 8 luma samples. – BCW weight index indicates equal weight. – The current CU does not have WP enabled. –CIIP mode is not used for the current CU. BDOF is applied only to the luma component. As its name suggests, BDOF mode is based on the concept of optical flow, which assumes that the motion of objects is smooth. For each 4×4 sub-block, motion refinement (v x , v y ) is calculated by minimizing the difference between L0 prediction samples and L1 prediction samples. Then motion refinement is used to adjust the bi-prediction sample values ​​in the 4x4 sub-block. The following steps are applied in the BDOF process. First, by directly calculating the difference between two adjacent samples, the horizontal and vertical gradients of the two prediction signals, and is calculated, that is, Among them I (k) (i, j) is the sample value at the prediction signal coordinate (i, j) in list k, k=0, 1, and shift1 is calculated as shi(t1=max(6, bitDepth-6) based on the luma bit depth bitDepth. Then, the autocorrelations and cross-correlations of the gradients S1, S2, S3, S5, and S6 are calculated as follows in where Ω is a 6×6 window around a 4×4 sub-block, and n a and n b The values ​​are set to min(1, bitDepth-11) and min(4, bitDepth-8) respectively. Then, using the cross-correlation and autocorrelation terms, motion refinement (v x , v y ) is exported using: in th′ BIO =2 max(5，BD-7) . is the round-down (floor) function, and Based on the motion refinement and gradients, the following adjustments are computed for each sample in the 4×4 sub-block: Finally, the BDOF samples of the CU are calculated by adjusting the bidirectional prediction samples as shown below: pred BDOF (x, y) = (I (0) (x,y)+I(1) (x, y) + b(x, y) + o offset )>>shift (2-13) These values ​​are chosen so that the multipliers in the BDOF process do not exceed 15 bits and the maximum bit width of the intermediate parameters in the BDOF process remains within 32 bits. To derive the gradient value, some prediction samples I ( k )(i, j) needs to be generated. Fig.16 As depicted, BDOF in VVC uses an extended row / column around the boundary of the CU. In order to control the computational complexity of generating prediction samples outside the boundary, the prediction samples in the extended area (white positions) are generated by directly taking the reference samples of the nearby integer positions (using the floor() operation on the coordinates) without interpolation, and a conventional 8-tap motion compensated interpolation filter is used to generate the prediction samples within the CU (gray positions). These extended sample values ​​are only used for gradient calculations. For the rest of the steps in the BDOF process, if any sample values ​​and gradient values ​​outside the CU boundary are needed, they are filled (i.e. repeated) from their nearest neighbors. When the width and / or height of a CU is greater than 16 luma samples, it will be split into sub-blocks with width and / or height equal to 16 luma samples, and the sub-block boundaries are regarded as CU boundaries in the BDOF process. The maximum unit size of the BDOF process is limited to 16x16. For each sub-block, the BDOF process can be skipped. When the SAD between the initial L0 prediction sample and the L1 prediction sample is less than a threshold, the BDOF process is not applied to the sub-block. The threshold is set equal to (8*W*(H>>1), where W represents the sub-block width and H represents the sub-block height. In order to avoid the additional complexity of SAD calculation, the SAD between the initial L0 prediction sample and the L1 prediction sample calculated in the DVMR process is reused here. If BCW is enabled for the current block, that is, the BCW weight index indicates unequal weights, bidirectional optical flow is disabled. Similarly, if WP is enabled for the current block, that is, luma_weight_lx_flag of any of the two reference pictures is 1, BDOF is also disabled; BDOF is also disabled when the CU is encoded and decoded using symmetric MVD mode or CIIP mode. 2.1.2.6. Symmetric MVD encoding and decoding In VVC, in addition to conventional unidirectional prediction mode MVD signaling and bidirectional prediction mode MVD signaling, symmetric MVD mode is applied for bidirectional prediction MVD signaling. In symmetric MVD mode, motion information including reference picture indexes of both list 0 and list 1 and MVD of list 1 is not transmitted by signal but derived. The decoding process of the symmetric MVD mode is as follows: 1) At the stripe level, the variables BiDirPredFlag, RefIdxSymL0 and RefIdxSymL1 are derived as follows: - If mvd_11_zero_flag is 1, BiDirPredFlag is set equal to 0. Otherwise, if the nearest reference picture in list 0 and the nearest reference picture in list 1 form a forward and backward reference picture pair or a backward and forward reference picture pair, then BiDirPredFlag is set to 1, and both the list 0 reference picture and the list 1 reference picture are short-term reference pictures. Otherwise, BiDirPredFlag is set to 0. 2) At CU level, if the CU is bidirectionally predicted and BiDirPredFlag is equal to 1, the symmetric mode flag indicating whether the symmetric mode is used is explicitly signaled. Fig.17 A schematic diagram of a symmetric MVD pattern is shown. When the symmetric mode flag is true, only mvp_10_flag, mvp_11_flag and MVD0 are explicitly signaled. The reference indexes of list 0 and list 1 are set equal to the reference picture pair, respectively. MVD1 is set equal to (-MVD0). The final motion vector is shown in the following formula. In the encoder, symmetric MVD motion estimation starts with initial MV evaluation. A set of initial MV candidates includes MVs obtained from unidirectional prediction search, MVs obtained from bidirectional prediction search, and MVs from the AMVP list. The one with the lowest distortion cost is selected as the initial MV for symmetric MVD motion search. 2.1.2.7. Decoder-side Motion Vector Refinement (DMVR) To improve the accuracy of the merge mode MV, decoder-side motion vector refinement based on bilateral matching is applied in VVC. In the bidirectional prediction operation, the refined MV is searched around the initial MV in the reference picture list L0 and the reference picture list L1. The BM method calculates the distortion between two candidate blocks in the reference picture list L0 and the list L1. Fig.18As shown, based on each MV candidate around the initial MV, the SAD between the red blocks is calculated. The MV candidate with the lowest SAD becomes the refined MV and is used to generate the bidirectional prediction signal. In VVC, DMVR can be applied to CUs that are coded or decoded using the following modes and functions: - CU level merge mode with bi-predictive MV. -Relative to the current picture, one reference picture is in the past and the other reference picture is in the future. - The distances from the two reference pictures to the current picture (ie, the POC difference) are the same. - Both reference images are short-term reference images. -CU has more than 64 luma samples. - CU height and CU width are both greater than or equal to 8 luma samples. -BCW weight index indicates equal weight. -Currently the block is not WP enabled. -CIIP mode is not used for the current block. The refined MV derived by the DMVR process is used to generate inter-frame prediction samples and is also used for temporal motion vector prediction for future picture coding. The original MV is used in the deblocking process and is also used for spatial motion vector prediction for future CU coding. Additional features of DMVR are mentioned in the following sub-clauses. 2.1.2.7.1. Search scheme In DVMR, the search point is around the initial MV, and the MV offset obeys the MV difference mirror rule. In other words, any point examined by DMVR represented by the candidate MV pair (MV0, MV1) follows the following two equations: MV0′=MV0+MV_offset (2-15) MV1′=MV1-MV_offset (2-16) Wherein, MV_offset represents the refinement offset between the initial MV and the refinement MV in one of the reference pictures. The refinement search range is two integer luma samples starting from the initial MV. The search includes an integer sample offset search phase and a fractional sample refinement phase. The integer sample offset search uses a 25-point full search. First, the SAD of the initial MV pair is calculated. If the SAD of the initial MV pair is less than the threshold, the integer sample stage of DMVR is terminated. Otherwise, the SAD of the remaining 24 points is calculated and checked in raster scan order. The point with the smallest SAD is selected as the output of the integer sample offset search stage. In order to reduce the impact of DMVR refinement uncertainty, it is proposed to support the original MV in the DMVR process. The SAD between the reference blocks referenced by the initial MV candidates is reduced by 1 / 4 of the SAD value. The integer sample search is followed by fractional sample refinement. To save computational complexity, fractional sample refinement is derived using the parametric error surface equation instead of an additional search using SAD comparison. Fractional sample refinement is conditionally called based on the output of the integer sample search stage. When the integer sample search stage ends with a center with minimum SAD in the first or second iteration search, fractional sample refinement is further applied. In the sub-pixel offset estimation based on the parametric error surface, the cost at the center position and the costs at the four neighboring positions from the center are used to fit a two-dimensional parabolic error surface equation of the following form: E(x, y) = A(x min ) 2 +B(yy min ) 2 +C (2-17) Where (x min ,y min ) corresponds to the fractional position with the minimum cost, and C corresponds to the minimum cost value. By solving the above equation using the cost values ​​of the five search points, (x min ,y min ) is calculated as: x min =(E(-1,0)-E(1,0)) / (2(E(-1,0)+E(1,0)-2E(0,0))) (2-18) y min =(E(0,-1)-E(0,1)) / (2((E(0,-1)+E(0,1)-2E(0,0))) (2-19) x min and min The value of is automatically clamped between -8 and 8, since all cost values ​​are positive, and the minimum value is E(0,0). This corresponds to a half-pixel offset with 1 / 16-pixel MV precision in VVC. The calculated fraction (x min ,y min ) is added to the integer distance refinement MV to obtain a sub-pixel accurate refinement delta MV. 2.1.2.7.2. Bilinear interpolation and sample padding In VVC, the resolution of MV is 1 / 16 luma sample. Samples at fractional positions are interpolated using an 8-tap interpolation filter. In DMVR, the search points are around the initial fractional pixel MV with integer sample offsets, so samples at these fractional positions need to be interpolated for the DMVR search process. In order to reduce computational complexity, a bilinear interpolation filter is used to generate fractional samples for the search process in DMVR. Another important effect is that by using a bilinear filter, DVMR does not access more reference samples than the normal motion compensation process within a 2-sample search range. After the refined MV is obtained through the DMVR search process, an ordinary 8-tap interpolation filter is applied to generate the final prediction. In order not to access more reference samples for the normal MC process, samples will be filled from those available samples, which are not required for the interpolation process based on the original MV, but are required for the interpolation process based on the refined MV. 2.1.2.7.3. Maximum DMVR processing unit When the width and / or height of a CU is greater than 16 luma samples, it will be further split into sub-blocks with width and / or height equal to 16 luma samples. The maximum unit size of the DMVR search process is limited to 16x16. 2.1.2.8. Combined Inter and Intra Prediction (CIIP) In VVC, when a CU is encoded and decoded in merge mode, if the CU contains at least 64 luma samples (i.e., the CU width multiplied by the CU height is equal to or greater than 64), and if both the CU width and the CU height are less than 128 luma samples, an additional flag is transmitted by signal to indicate whether the combined inter / intra prediction (CIIP) mode is applied to the current CU. As its name suggests, CIIP prediction combines the inter prediction signal with the intra prediction signal. The inter prediction signal P in CIIP mode inter is derived using the same inter prediction process applied to the normal merge mode; and the intra prediction signal P is derived after the normal intra prediction process for the planar mode intra Then, the intra and inter prediction signals are combined using weighted averaging, where the weight values ​​are calculated depending on the coding mode of the top and left neighboring blocks as follows ( Fig.19 as depicted in ): - if the top neighbor is available and intra-coded, set isIntraTop to 1, otherwise set isIntratop to 0; - If the left neighbor is available and intra-frame decoding is performed, isIntraLeft is set to 1, otherwise isIntralLeft is set to 0; -If (isIntraLeft+isIntraTop) is equal to 2, then wt is set to 3; - Otherwise, if (isIntraLeft + isIntraTop) is equal to 1, then wt is set to 2; - Otherwise, set wt to 1. The CIIP forecast is constructed as follows: P CIIP =((4-wt)*P inter +wt*P intra +2)>>2 (2-20) 2.1.2.9. Geometric Partitioning Model (GPM) In VVC, geometric partitioning mode is supported for inter prediction. The CU level flag is used as a merge mode to signal the geometric partitioning mode. Other merge modes include normal merge mode, MMVD mode, CIIP mode, and sub-block merge mode. For each possible CU size w×h=2 m ×2 n , where m, n ∈ {3…6} and excluding 8x64 and 64x8, the geometric partitioning mode supports a total of 64 partitions. When this mode is used, the CU is split into two parts by a geometrically positioned straight line (e.g. Fig. 20 ). The position of the partition line is mathematically derived from the angle and offset parameters of the specific partition. Each part of the geometric partition in the CU uses its own motion for inter-frame prediction; each partition only allows unidirectional prediction, that is, each part has a motion vector and a reference index. Unidirectional prediction motion constraints are applied to ensure that only two motion compensated predictions are required for each CU, which is the same as traditional bidirectional prediction. If geometric partitioning mode is used for the current CU, a geometric partitioning index indicating the partitioning mode (angle and offset) of the geometric partitioning and two merge indices (one for each partition) are further transmitted through the signal. The number of maximum GPM candidate sizes is explicitly transmitted through the signal in the SPS, and the syntax binarization for the GPM merge index is specified. After predicting each part of the geometric partitioning, a hybrid process with adaptive weights is used to adjust the sample values ​​along the edges of the geometric partitioning. This is the prediction signal for the entire CU, and the transformation and quantization processes will be applied to the entire CU as in other prediction modes. Finally, the motion field of the CU predicted using the geometric partitioning mode is stored. 2.1.2.9.1. One-way prediction candidate list construction The unidirectional prediction candidate list is derived directly from the merge candidate list constructed according to the extended merge prediction process. Denote n as the index of the unidirectional prediction motion in the geometric unidirectional prediction candidate list. The LX motion vector of the nth extended merge candidate is used as the nth unidirectional prediction motion vector of the geometric partitioning mode, with X equal to the parity of n. These motion vectors are Fig.21 If the corresponding LX motion vector of the n-th extended merge candidate does not exist, the L(1-X) motion vector of the same candidate is used instead as the unidirectional prediction motion vector of the geometric partitioning mode. 2.1.2.9.2. Blending Along Geometric Partition Edges After predicting each part of the geometric partition using its own motion, blending is applied to the two prediction signals to derive samples around the geometric partition edges. The blending weights for each position of the CU are derived based on the distance between the individual position and the partition edge. The distance from position (x, y) to the partition edge is derived as: where i, j are the angle and offset indices of the geometric partition, which depend on the geometric partition index transmitted by the signal. x，j and ρ y，j The sign of depends on the angle index i. The weight of each part of the geometric partition is derived as follows: wIdxL(x,y)=partIdx? 32+d(x,y):32-d(x,y) (2-25) W1(x,y)=1-w0(x,y) (2-27) partIdx depends on the angle index i. An example of weight w0 is Fig. 22 Shown in. 2.1.2.9.3. Motion Field Storage for Geometry Partitioning Mode Mv1 from the first part of the geometric partition, Mv2 from the second part of the geometric partition, and the combination of Mv1 and Mv2 v Stored in the motion field of the CU encoded in the geometry partitioning mode. The type of motion vector stored for each individual position in the motion field is determined as: sType=abs(motionIdx)<32?2: (motionIdx≤0?(1-partIdx):partIdx) (2-28) Where motionIdx is equal to d(4x+2, 4y+2). partIdx depends on the angle index i. If sType is equal to 0 or 1, then Mv0 or Mv1 is stored in the corresponding motion field, otherwise, if sTType is equal to 2, then the value from Mv0 or Mv1 is stored. v 0 and M v 2 Combination M v . Combination M v It is generated using the following process: 1) If Mv1 and Mv2 come from different reference picture lists (one from L0 and the other from L1), then Mv1 and Mv2 v 2 are simply combined to form a bi-directionally predicted motion vector. 2) Otherwise, if Mv1 and Mv2 are from the same list, only the unidirectional predicted motion Mv2 is stored. 2.1.2.9.4. GPM with Inter and Intra Prediction (GPM Inter-Intra) With GPM Inter-Intra, in addition to merging candidates for each non-rectangular partitioned area in the CU to which GPM is applied, a predefined intra prediction mode for the geometric dividing line can also be selected. In the proposed method, a flag from the encoder is used to determine whether the intra prediction mode or the inter prediction mode is for each GPM separation area. When in inter prediction mode, the unidirectional prediction signal is generated by the MV from the merged candidate list. On the other hand, when in intra prediction mode, the unidirectional prediction signal is generated from the neighboring pixels used for the intra prediction mode, which is specified by an index from the encoder. The variation of possible intra prediction modes is restricted by the geometry. Finally, the two unidirectional prediction signals are mixed in the same way as ordinary GPM. 2.1.2.10.Multiple Hypothesis Prediction (MHP) Over inter-AMVP mode, normal merge mode and MMVD mode, up to two additional predictors are signaled. The resulting overall prediction signal is iteratively accumulated with each additional prediction signal. p n+1 =(1-α n+1 ) n +α n+1 h n+1 The weight factor α is specified according to the following table: add_hyp_weight_idx α 0 1 / 4 1 -1 / 8 For inter-AMVP mode, MHP is applied only when unequal weights in BCW are selected in bi-prediction mode. 2.1.2.11.OBMC When OBMC is applied, the top and left boundary pixels of the CU are refined using the motion information of neighboring blocks with weighted prediction as described in JVET-L0101. The conditions under which OBMC should not be applied are as follows: When OBMC is disabled at SPS level. When the current block has intra mode or IBC mode. When the current block applies LIC. When the current luminance block area is less than or equal to 32. Sub-block boundary OBMC is performed by applying the same blending to the top, left, bottom and right sub-block boundary pixels using the motion information of the neighboring sub-blocks. It is enabled for sub-block based codecs: Affine AMVP mode; Affine merge mode and sub-block based temporal motion vector prediction (SbTMVP); Sub-block based bilateral matching. 2.1.3. Transformation and quantization 2.1.3.1. Large Block Size Transforms with High Frequency Zeroing In VVC, large block size transforms up to 64×64 are enabled, which are mainly used for higher resolution videos, such as 1080p and 4K sequences. For transform blocks of size (width or height, or width and height) equal to 64, the high-frequency transform coefficients are adjusted to zero so that only low-frequency coefficients are retained. For example, for an M×N transform block, M is the block width and N is the block height, when M is equal to 64, only the left 32 columns of transform coefficients are retained. Similarly, when N is equal to 64, only the first 32 rows of transform coefficients are retained. When the transform skip mode is used for large blocks, the entire block will be used without zeroing any values. In addition, transform shifts are eliminated in transform skip mode. VTM also supports a configurable maximum transform size in SPS, allowing the encoder to flexibly select a transform size of up to 32 lengths or 64 lengths according to the needs of the specific implementation. 2.1.3.2. Multiple Transform Selection (MTS) for Core Transformations In addition to DCT-II, which has been adopted in HEVC, the Multiple Transform Selection (MTS) scheme is also used to encode the residual of the blocks that are inter- and intra-coded. It uses multiple selected transforms in DCT8 / DST7. The newly introduced transform matrices are DST-VII and DCT-VIII. Table 7 shows the basic functions of the selected DST / DCT. Table 7 - Transform basis functions of DCT-II / VIII and DSTVII for N-point input To maintain the orthogonality of the transform matrix, the quantization of the transform matrix is ​​more accurate than that in HEVC. To keep the intermediate values ​​of the transform coefficients within 16 bits, all coefficients are 10 bits after horizontal and vertical transforms. To control the MTS scheme, separate enable flags are specified at the SPS level for intra and inter frames respectively. When MTS is enabled at SPS, a CU level flag is signaled to indicate whether MTS is applied. Here, MTS is applied only to luma. When one of the following conditions applies, MTS signaling will be skipped. – The position of the last significant coefficient of the luma TB is less than 1 (ie DC only). – The last significant coefficient of the luminance TB is located within the MTS zeroing region. If the MTS CU flag is equal to 0, DCT2 is applied in both directions. However, if the MTS CU flag is equal to 1, two other flags are additionally signaled to indicate the transform types in the horizontal and vertical directions, respectively. The transform and signaling mapping table is shown in Table 8. By eliminating intra-mode and block shape dependencies, the transform selection of ISP and implicit MTS is unified. If the current block is in ISP mode or the current block is an intra-frame block and both intra-frame and inter-frame explicit MTS are turned on, only DST7 is used for horizontal and vertical transform cores. In terms of transform matrix accuracy, an 8-bit main transform core is used. Therefore, all transform cores used in HEVC remain the same, including 4-point DCT-2 and DST-7, 8-point, 16-point and 32-point DCT-2. In addition, other transform cores (including 64-point DCT-2, 4-point DCT-8, 8-point, 16-point, 32-point DST-7 and DCT-8) all use 8-bit main transform cores. Table 8 - Transformation and signaling mapping table To reduce the complexity of large-size DST-7 and DCT-8, high-frequency transform coefficients are cleared to zero for DST-7 and DCT-8 blocks of size (width or height, or width and height) equal to 32. Only coefficients in the 16×16 low-frequency region are retained. As in HEVC, the residual of a block can be coded or decoded using transform skip mode. To avoid syntax coding redundancy, the transform skip flag is not signaled when the CU level MTS_CU_flag is not equal to 0. Note that when LFNST or MIP is activated for the current CU, the implicit MTS transform is set to DCT2. In addition, when MTS is enabled for an inter-coded block, implicit MTS can still be enabled. 2.1.3.3. Low Frequency Non-Separable Transform (LFNST) In VVC, as Fig.23 shown, the LFNST is applied between the forward main transform and quantization (at the encoder) and between de-quantization and the inverse main transform (at the decoder side). In the LFNST, a 4×4 non-separable transform or an 8×8 non-separable transform is applied according to the block size. For example, the 4×4 LFNST is applicable to small blocks (i.e., min(width, height) < 8), and the 8×8 LFNST is applicable to larger blocks (i.e., min(width, height) > 4). The application of the non-separable transform used in the LFNST is described below using the input as an example. Applying the 4×4 LFNST, the 4×4 input block X is first represented as a vector The non-separable transform is calculated as where represents the transform coefficient vector and T is a 16x16 transform matrix. Subsequently, the 16×1 coefficient vector is reorganized into a 4×4 block using the scan order (horizontal, vertical, or diagonal) of the block. The coefficients with smaller indices will be placed in the 4×4 coefficient block together with smaller scan indices. 2.1.3.3.1. Simplified Non-Separable Transform The LFNST (Low Frequency Non-Separable Transform) applies the non-separable transform based on the direct matrix multiplication method, enabling a single-round implementation without multiple iterations. However, it is necessary to reduce the dimension of the non-separable transform matrix to minimize the computational complexity and the storage space for storing the transform coefficients. Therefore, the simplified non-separable transform (or RST) method is used in the LFNST. The main idea of the simplified non-separable transform is to map an N-dimensional vector (for an 8x8 NSST, N is usually equal to 64) to an R-dimensional vector in a different space, where N / R (R < N) is the reduction factor. Thus, the RST matrix is no longer an N×N matrix but becomes an R×N matrix, as follows: Where the R rows of the transform are the R basis of the N-dimensional space. The inverse transform matrix of RT is the transpose of its forward transform. For 8×8 LFNST, a reduction factor of 4 is applied, and the 64×64 direct matrix (conventional 8x8 inseparable transform matrix size) is reduced to a 16×48 direct matrix. Therefore, the 48×16 inverse RST matrix is ​​used on the decoder side to generate the core (main) transform coefficients in the 8×8 upper left region. When a 16×48 matrix is ​​applied instead of 16×64 with the same transform set configuration, each matrix obtains 48 input data from three 4×4 blocks in the upper left 8×8 block (excluding the lower right 4x4 block). With the help of dimensionality reduction, the memory usage for storing all LFNST matrices is reduced from 10KB to 8KB, and the performance degradation is also reasonable. In order to reduce complexity, LFNST is restricted to be applicable only when all coefficients outside the first coefficient subgroup are insignificant. Therefore, when LFNST is applied, all main transform coefficients must be zero. This allows the LFNST index signal to be adjusted on the last significant position, thus avoiding an extra coefficient scan in the current LFNST design, which is only needed to check the significant coefficients at a specific position. The worst-case processing of LFNST (in terms of multiplications per pixel) limits the inseparable transforms for 4×4 and 8×8 blocks to 8×16 and 8×48 transforms, respectively. In these cases, the last significant scan position must be less than 8 when LFNST is applied, and less than 16 for other sizes. For blocks of shape 4×N and N×4 with N>8, the proposed restriction means that LFNST is now applied only once and only to the 4×4 region in the upper left corner. Since all the only major coefficients are zero when LFNST is applied, the number of operations required for the major transform is reduced in this case. From the encoder's perspective, the quantization of the coefficients is significantly simplified when testing the LFNST transform. The first 16 coefficients (in scan order) must be quantized to the maximum extent possible with rate-distortion optimization, and the remaining coefficients are forced to zero. 2.1.3.3.2.LFNST Transform Selection There are a total of 4 transform sets in LFNST, each with 2 inseparable transform matrices (kernels). The mapping from intra prediction modes to transform sets is predefined, as shown in Table 9. If the current block uses one of the three CCLM modes (INTRA_LT_CCLM, INTRA_T_CCLM, or INTRA_L_CCLM) (81 <= intra prediction mode <= 83), transform set 0 is selected for the current chroma block. For each transform set, the selected inseparable secondary transform candidate is further specified by an explicitly signaled LFNST index. After the transform coefficients, the index is signaled once in the bitstream for each intra CU. Table 9 - Transformation selection table predModeIntra Transform Set Index predModeIntra<0 1 0<=predModeIntra<=1 0 2<=predModeIntra<=12 1 13<=predModeIntra<=23 2 24<=predModeIntra<=44 3 45<=predModeIntra<=55 2 56<=predModeIntra<=80 1 81<=intra prediction mode<=83 0 2.1.3.3.3.LFNST index signaling and interaction with other tools Since LFNST is restricted to apply only when all coefficients outside the first coefficient subgroup are insignificant, the LFNST index encoding depends on the position of the last significant coefficient. In addition, the LFNST index is context-encoded but does not depend on the intra prediction mode, and only the first binary bit is context-encoded. In addition, LFNST is applied to intra CUs in intra and inter slices, and to both luma and chroma. If dual tree is enabled, the LFNST indexes for luma and chroma will be signaled separately. For inter slices (dual tree is disabled), a single LFNST index is signaled and used for luma and chroma. Considering the implicit partitioning (TU tiling) of large CUs larger than 64×64 due to the existing maximum transform size limit (64×64), the LFNST index search may increase the data buffer by a factor of four for a certain number of debit pipeline stages. Therefore, the maximum size allowed for LFNST is limited to 64×64. Note that LFNST is enabled only at DCT2. LFNST index signaling is placed before MTS index signaling. Using a scaling matrix for perceptual quantization does not clearly indicate that the scaling matrix specified for the master matrix is ​​useful for the LFNST coefficients. Therefore, scaling matrices for LFNST coefficients are not allowed. For single-tree partitioning mode, chroma LFNST is not applied. 2.1.3.4. Sub-Block Transform (SBT) In VTM, sub-block transform is introduced for inter-predicted CUs. In this transform mode, only a sub-part of the residual block is encoded and decoded for the CU. When cu_cbf is equal to 1 for an inter-predicted CU, cu_sbt_flag can be signaled to indicate whether the entire residual block or a sub-part of the residual block is encoded and decoded. For the former case, the inter-MTS information is further parsed to determine the transform type of the CU. In the latter case, a part of the residual block is encoded and decoded by inferring the adaptive transform, while the other part of the residual block is cleared. When SBT is used for an inter-coded CU, the SBT type and SBT position information are signaled in the bitstream. There are two SBT types and two SBT positions, such as Fig.24As shown. For SBT-V (or SBT-H), the TU width (or height) can be equal to half of the CU width (or height) or 1 / 4 of the CU width (or height), resulting in 2:2 partitioning or 1:3 / 3:1 partitioning. 2:2 partitioning is similar to binary tree (BT) partitioning, while 1:3 / 3:1 partitioning is similar to asymmetric binary tree (ABT) partitioning. In ABT partitioning, only small areas contain non-zero residuals. If one dimension of the CU is 8 (in units of luminance samples), 1:3 / 3:1 partitioning along that dimension is not allowed. A CU has a maximum of 8 SBT modes. Position-dependent transform kernel selection is applied to luma transform blocks in SBT-V and SBT-H (chroma TBs always use DCT-2). Different kernel transforms are associated with the two positions, SBT-H and SBT-V. More specifically, the horizontal and vertical transforms for each SBT position are in Fig.24 For example, the horizontal and vertical transforms for SBT-V position 0 are DCT-8 and DST-7, respectively. When one side of the residual TU is larger than 32, the transforms for both dimensions are set to DCT-2. Therefore, the sub-block transforms jointly specify the TU tiling, cbf, and the horizontal and vertical core transform types for the residual block. SBT does not apply to CUs coded in combined inter-intra mode. 2.1.3.5. Maximum transform size and zeroing of transform coefficients The CTU size and the maximum transform size (i.e., all MTS transform kernels) are extended to 256, where the largest intra-coded block size can be 128×128. For UHD sequences, the maximum CTU size is set to 256; otherwise it is set to 128. During the primary transform process, no normalized zeroing operation is applied to the transform coefficients. However, if LFNST is applied, the main transform coefficients outside the LFNST region will be normalized to zero. 2.1.3.6. Enhanced MTS for intra-frame coding and decoding In the current VVC design [1], for MTS, only DST7 and DCT8 transform kernels are used for intra-frame and inter-frame encoding and decoding. Other major transforms are used, including DCT5, DST4, DST1, and the identity transform (IDT). The MTS setting also depends on the TU size and intra-frame mode information. 16 different TU sizes are considered, and for each TU size, 5 different categories are considered based on the intra-frame mode information. For each category, 4 different transform pairs are considered, the same as VVC. Note that although a total of 80 different categories are considered, some of these different classes often share the exact same set of transforms. Therefore, there are 58 (less than 80) unique entries in the resulting LUT. For angular modes, the joint symmetry of TU shape and intra prediction is taken into account. Therefore, mode i (i>34) with TU shape A×B will be mapped to the same class corresponding to mode j=(68–i) with TU shape B×A. However, for each transform pair, the order of horizontal and vertical transform kernels is swapped. For example, for a 16×4 block of mode 18 (horizontal prediction) and a 4×16 block of mode 50 (vertical prediction), they are mapped to the same class. However, the vertical and horizontal transform kernels are swapped. For wide-angle modes, the transform set is determined using the closest traditional angular mode. For example, mode 2 is used for all modes between -2 and -14. Similarly, mode 66 is used for modes 67 to 80. The MTS index [0, 3] is signaled using a 2-bit fixed length codec. 2.1.3.7. Secondary Transformation: Extension of LFNST with Large Kernel The LFNST design in VVC is extended as follows: The number of LFNST sets (S) and candidates (C) is extended to S=35 and C=3, and the LFNST set (lfnstTrSetIdx) for a given intra mode (predModeIntra) is derived according to the following formula: oFor predModeIntra<2, lfnstTrSetIdx is equal to 2; o for predModeIntra in [0, 34], lfnstTrSetIdx = predModeIntra; o For predModeIntra in [35, 66], lfnstTrSetIdx = 68 - predModeIntra. Three different kernels, LFNST4, LFNST8, and LFNST16, are defined to indicate the LFNST kernel set, which are applied to 4×N / N×4 (N≥4), 8×N / Nx8 (N≥8), and M×N (M, N≥16), respectively. The kernel size is given by the following formula: (LFSNT4, LFNST8*, LFNST16*)=(16x16, 32x64, 32x96). The forward LFNST is applied to the upper left low frequency region, which is called the region of interest (ROI). When LFNST is applied, the primary transform coefficients present in the region outside the ROI are zeroed, which is unchanged from the VVC standard. The ROI for LFNST16 is Fig.25It is shown in . It consists of six 4×4 sub-blocks, which are consecutive in scanning order. Since the number of input samples is 96, the transformation matrix of forward LFNST16 can be R×96. In the present disclosure, R is selected as 32, and accordingly 32 coefficients (two 4×4 sub-blocks) are generated from forward LFNST16, which are placed in coefficient scanning order. The ROI for LFNST8 is Fig.26 The forward LFNST8 matrix can be R×64, with R chosen to be 32. The generated coefficients are positioned in the same way as LFNST16. The mapping from intra prediction modes to these sets is shown in Table 10. Table 10. Mapping of intra prediction modes to LFNST set indices 2.1.3.8. Symbol prediction The basic idea of ​​the coefficient sign prediction methods (JVET-D0031 and JVET-J0021) is to compute the reconstruction residuals for both negative and positive combinations of the available transform coefficients and select the hypothesis that minimizes the cost function. To obtain the best sign, the cost function is defined as a measure of discontinuity across block boundaries, such as Fig. 27 It measures all hypotheses and chooses the one with the smallest cost as the predictor of the sign of the coefficient. The cost function is defined as the sum of the absolute second-order derivatives in the residual domain of the upper rows and left columns as follows: Where R is the reconstructed neighbor, P is the prediction of the current block, and r is the residual hypothesis. Each block can only be calculated once (-R -1 +2R0-P1) term and only the residual assumption is subtracted. 2.1.4. Luminance Mapping with Chroma Scaling (LMCS) In VVC, a codec tool called "luminance mapping with chroma scaling" (LMCS) is added as a new processing block before loop filtering. LMCS has two main components: 1) in-loop mapping of luma components based on an adaptive piecewise linear model; 2) for chroma components, luma-dependent chroma residual scaling is applied. Fig.28 shows the LMCS architecture from the decoder perspective. Fig.28 The light blue shaded blocks in represent where processing is applied in the mapped domain; and these include inverse quantization, inverse transform, luma intra prediction, and adding the luma prediction to the luma residual. Fig.28The unshaded blocks in indicate where processing is applied in the original (i.e., unmapped) domain; and these include loop filters such as deblocking, ALF and SAO, motion compensated prediction, chroma intra prediction, adding chroma prediction to chroma residual, and storing the decoded picture as a reference picture. Fig.28 The light yellow shaded blocks in the are the new LMCS functional blocks, including the forward and reverse mapping of the luma signal, and the luma-dependent chroma scaling process. Like most other tools in VVC, LMCS can be enabled / disabled at the sequence level using the SPS flag. 2.1.4.1. Brightness Mapping with a Piecewise Linear Model The in-loop mapping of the luma component adjusts the dynamic range of the input signal by redistributing codewords within the dynamic range to improve compression efficiency. Luma mapping uses a forward mapping function FwdMap and a corresponding inverse mapping function InvMap. The FwdMap function is signaled using a piecewise linear model with 16 equal segments. The InvMap function does not need to be signaled but is derived from the FwdMap function. The luminance mapping model is signaled in the adaptive parameter set (APS) syntax structure with aps_params_type set to 1 (LMCS_APS). A maximum of 4 LMCS APS can be used in a coded video sequence. Only 1 LMCS APS can be used for a picture. The luminance mapping model is signaled using a piecewise linear model. The piecewise linear model divides the dynamic range of the input signal into 16 equal parts, and for each part, its linear mapping parameters are represented by the number of codewords assigned to the part. Take 10-bit input as an example. By default, each of the 16 fragments is allocated 64 codewords. The number of codewords transmitted by the signal is used to calculate the scaling factor and adjust the mapping function of the fragment accordingly. At the slice level, the LMCS enable flag is signaled to indicate if Fig.28 Whether the LMCS process shown is applied to the current slice. If LMCS is enabled for the current slice, aps_id is signaled in the slice header to identify the APS that carries the luma mapping parameters. Each i-th fragment, i=0...15 of the FwdMap piecewise linear model is defined by two input pivot points InputPivot[] and two output (mapped) pivot points MappedPivot[]. InputPivot[] and MappedPivot[] are calculated as follows (assuming 10-bit video): 1)OrgCW=64 2) For i = 0:16, InputPivot[i] = i * OrgCW 3) For i = 0:16, MappedPivot[i] is calculated as follows: MappedPivot[0] = 0; for(i=0;i<1 6;i++) MappedPivot[i+1]=MappedPivot[i]+SignalledCW[i] where SignalledCW[i] is the number of codewords signaled for the i-th segment. like Fig.28 As shown, for the inter-frame coded block, motion compensation prediction is performed in the mapping domain. In other words, the motion compensation prediction block Y is calculated based on the reference signal in the DPB. pred Finally, the FwdMap function is applied to map the brightness prediction block in the original domain to the mapping domain, Y′ pred =FwdMap(Y pred ). For intra-coded blocks, the FwdMap function is not applied because intra prediction is performed in the mapping domain. Calculate the reconstructed block Y r Finally, the InvMap function is applied to convert the reconstructed brightness value in the mapping domain back to the reconstructed brightness value in the original domain. The InvMap function is applicable to intra-coded and inter-coded luma blocks. The brightness mapping process (forward and / or inverse mapping) can be implemented using a lookup table (LUT) or using on-the-fly calculations. If LUTs are used, then FwdMapLUT and InvMapLUT can be pre-calculated and pre-stored for use at the tile group level, and forward and inverse mapping can be simply implemented as FwdMap(Y pred )=FwdMapLUT[Y pred ] and InvMap(Y r )=InvMapLUT[Y r ]. Alternatively, immediate calculation can be used. Take the forward mapping function FwdMap as an example. To find out which fragment a brightness sample belongs to, the sample value is shifted right by 6 bits (equivalent to 16 equal fragments). Then, the linear model parameters of the fragment are retrieved and applied immediately to calculate the mapped brightness value. Let i be the fragment index, a1 and a2 be InputPivot[i] and InputPivot[i+1] respectively, and b1 and b2 be MappedPivot[i] and MappedPivot[i+1] respectively. The calculation of the FwdMap function is as follows: FwdMap(Y pred )=((b2-b1) / (a2-a1))*(Y pred -a1)+b1 The InvMap function can be calculated on the fly in a similar way. Typically, the fragments in the mapping domain are of unequal size, so the most straightforward inverse mapping process will require comparisons in order to find out which fragment the current sample value belongs to. This comparison increases the complexity of the decoder. Therefore, VVC imposes bitstream constraints on the value of the output pivot point MappedPivot[i] as shown below. Assume that the range of the mapping domain (for 10-bit video, the range is [0, 1023]) is divided into 32 equal fragments. If MappedPivot[i] is not a multiple of 32, then MappedPivot[i+1] and MappedPivot[i] cannot belong to the same fragment among the 32 fragments of the same size, that is, MappedPivot[i+1] >> (BitDepth Y -5) must not be equal to MappedPivot[i]>>(BitDepth Y Due to this bitstream constraint, the InvMap function can also be implemented using a simple right shift of 5 bits (corresponding to 32 equally sized fragments) in order to find out which fragment a sample value belongs to. 2.1.4.2. Luma-Dependent Chroma Residual Scaling Chroma residual scaling is intended to compensate for the interaction between luma signals and their corresponding chroma signals. Whether chroma residual scaling is enabled is also signaled at the slice level. If luma mapping is enabled, an additional flag is emitted to indicate whether luma-dependent chroma residual scaling is enabled. When luma mapping is not used, luma-dependent chroma residual scaling is disabled. Additionally, luma-dependent chroma residual scaling is always disabled for chroma blocks with an area less than or equal to 4. The chroma residual scaling depends on the average of the neighboring luma samples reconstructed on the top and / or left of the current VPDU. If the current CU is inter-128x128, inter-128x64, and inter-64x128, the chroma residual scaling factor derived for the CU associated with the first VPDU is used for all chroma transform blocks in that CU. Denote apgYr as the average of the neighboring luma samples reconstructed (see Fig.28 ). C ScaleInv The value of is calculated as follows: 1) Based on the InvMap function, find the index Y of the piecewise linear model to which avgYr belongs Idx . 2) C ScaleInv =cScaleInv[Y Idx ], where cScaleInv[] is a 16-segment LUT pre-computed based on the value of SignalledCW[i] and the offset value for the chroma residual scaling process signaled in APS. Unlike brightness mapping performed based on samples, C ScaleInv is a constant for the entire chrominance block. ScaleInv , chroma residual scaling is applied as follows: Encoder side: C ResScale =C Res *C Scale =C Res / C ScaleInv Decoder side: C Res =C ResScale / C Scale =C ResScale *C ScaleInv 2.1.5. Image border filling 2.1.5.1. Image border filling methods in existing codec standards (such as H.264 / AVC, H.265 / HEVC and H.266 / VVC) Copy / repeat padding is used to expand the image to a larger size. More specifically, the reference image is expanded to form a larger image by the following steps: 1) The boundary samples at the left boundary of the reference picture are copied to the left; 2) The boundary samples located at the right boundary of the reference image are copied to the right; 3) The boundary samples at the top boundary of the reference image are copied to the top; 4) Boundary samples located at the bottom boundary of the reference picture are copied below. For current picture coding, when the motion vector of the current block points to a reference block that is (partially or completely) outside the reference picture, the prediction block of the current block will be generated from padded samples outside the reference picture boundary. 2.1.5.2. Research on Motion Compensated Boundary Filling When the decoder performs motion compensation, if the motion vector points to a block outside the frame boundary, part of the reference block is not available. For each area with a size of 4xM or Mx4 along the boundary of the reference picture to be filled, M is the expected frame boundary extension, and the motion vector is derived from the nearest 4x4 block inside the frame. If the nearest 4x4 block is intra-coded, a zero motion vector is used. If the nearest 4x4 block is coded using bidirectional inter-frame prediction, only motion vectors pointing to pixels away from the frame boundary are used for motion compensation for filling. After the motion vector is derived, motion compensation is then performed to obtain the pixels in the filled area, taking into account the average pixel value offset between the nearest 4x4 block and its corresponding block in the reference picture. 3. Question There are some problems with the existing video coding and decoding technology, which need to be further improved to obtain higher coding and decoding gain. 1. Generally speaking, OBMC mixes multiple inter predictions in the original domain, while intra prediction is performed in the LMCS reshaped domain. When applying GPM inter-intra, it is necessary to consider how to apply OBMC on top of GPM inter-intra prediction. 2. Template matching based methods usually compare samples between two templates through a cost function (such as SAD / SATD of samples within a template constructed only from neighboring samples). When multiple template matching schemes are allowed in the codec, it is necessary to consider how to use this technique. 3. Multiple reference rows / columns can be applied to different codecs using the rules / constraints that are enforced. 4. Intra mode candidates can be generated from previous historical candidate lists / tables. 5. Partition information can be derived based on a history table with partition states of previously coded blocks. 6. In the VVC standard and the existing ECM4.0, the prediction samples of the two GPM partitions around the GPM partition line are weighted mixed. However, the GPM mode and its variant modes (such as template-based GPM partition mode reordering, GPM with template matching, GPM with motion vector difference, GPM inter-intra, etc.) are designed based on a fixed mixing width and cannot adapt to various video contents. CABAC codecs for syntax elements with sorted priority / selection rate / probability can be further designed. 7. RPR needs to be designed taking into account some specific coding tools (such as methods based on template matching). 8. Image border padding requires large memory buffers, which needs further study. 4. Detailed solution The detailed solutions below should be considered as examples to explain the general concept. These solutions should not be interpreted in a narrow way. Moreover, these solutions can be combined in any way. The term “video unit” or “codec unit” or “block” may refer to a codec tree block (CTB), a codec tree unit (CTU), a codec block (CB), a CU, a PU, a TU, a PB, a TB. In the present disclosure, regarding “blocks encoded and decoded in mode N”, the “mode N” here can be a prediction mode (for example, MODE_INTRA, MODE_INTER, MODE_PLT, MODE_IBC, etc.) or a coding and decoding technology (for example, AMVP, merge, SMVD, BDOF, PROF, DMVR, AMVR, TM, affine, CIIP, GPM, GEO, TPM, MMVD, BCW, HMVP, SbTMVP, etc.). It should be noted that the following terms are not limited to the specific terms defined in existing standards, and any changes in codec tools are also applicable. Consistency between GPM inter-frame, intra-frame, OBMC and LMCS 1. For example, OBMC can be performed first on inter prediction in the original domain, then forward mapping is applied, and the output can be further processed by the program. a. For example, the inter-frame prediction may be GPM inter-frame prediction. b. For example, the inter-frame prediction may be the inter-frame prediction of the GPM variant mode. c. For example, a further procedure could be to weight with GPM intra prediction in the reshaped domain. d. For example, OBMC may first be applied to the GPM inter prediction in the original domain to obtain the original domain OBMC compensated GPM inter prediction. Then, the forward LMCS mapping is applied to the original domain OBMC compensated prediction to obtain the reshaped domain OBMC compensated GPM inter prediction. Afterwards, the reshaped domain GPM intra prediction is further weighted mixed with the reshaped OBMC compensated prediction to obtain the final OBMC compensated GPM inter-intra hybrid prediction in the reshaped domain. 2. For example, the first inter prediction can be forward mapped to the reshaped domain first, and then processed by the program to obtain the second prediction. The forward mapping can be applied to other inter predictions participating in the OBMC process. In this way, OBMC is finally performed in the reshaped domain. a. For example, the first inter-frame prediction may be a GPM inter-frame prediction. b. For example, the first inter-frame prediction may be an inter-frame prediction of a GPM variant mode. c. This procedure can be mixed with GPM intra prediction weighting. d. For example, first, the forward LMCS mapping can be applied to GPM inter prediction to obtain the reshaped domain GPM inter prediction. Secondly, the reshaped domain GPM inter prediction is further weighted mixed with the reshaped domain GPM intra prediction to obtain the reshaped domain GPM hybrid prediction. Thirdly, the forward LMCS mapping is applied to other inter predictions used in the OBMC process. Finally, OBMC is applied in the reshaped domain to mix the reshaped domain GPM inter-intra prediction and other reshaped domain inter predictions. 3. For example, GPM intra prediction is first back-mapped to the original domain and then weighted mixed with GPM inter prediction in the original domain. Afterwards, OBMC is performed in the original domain. Finally, forward mapping is applied to the OBMC-compensated prediction. a. For example, first, the reverse LMCS mapping can be applied to the GPM intra prediction to obtain the original domain GPM intra prediction. Secondly, the original domain GPM intra prediction is further weighted mixed with the original domain GPM inter prediction to obtain the original domain GPM hybrid prediction. Thirdly, OBMC is applied to the original domain GPM hybrid prediction to obtain the original domain OBMC compensated prediction. Finally, the forward LMCS mapping is applied to obtain the reshaped domain OBMC compensated GPM inter-intra prediction. Template matching based 4. For example, more than one template matching method may be enabled for a video unit. a. For example, a video unit can be encoded and decoded using the following intra-frame prediction method: i. TIMD and / or its variants; ii. DIMD and / or its variants; iii. MPM candidate ranking. b. For example, the video unit may be encoded and decoded by one or more of the following inter-frame prediction methods (multiple inter-frame prediction methods may be applied in sequence): i. GPM and / or its variants, such as GPM-TM; ii. CIIP and / or its variants, such as CIIP-TM; iii. conventional mergers and / or their variants, such as TM-mergers; iv. AMVP and / or its variants, such as TM-amvp; v.MMVD and / or its variants, such as TM-MMVD; vi. Affine and / or its variants, such as TM-affine. c. For example, motion candidate re-ranking methods (eg, ARMC) may be based on more than one type of template matching scheme. i. For example, more than one type of template matching scheme may be used to reorder conventional merge candidates. ii. For example, more than one type of template matching scheme may be used to reorder CIIP merge candidates. iii. For example, more than one type of template matching scheme may be used to reorder GPM merge candidates. iv. For example, more than one type of template matching scheme may be used to re-rank the MMVD merging candidates. v. For example, more than one type of template matching scheme may be used to re-rank AMVP candidates. vi. For example, more than one type of template matching scheme may be used to re-rank affine (affine merge, and / or affine amvp) candidates. vii. Additionally, or alternatively, different numbers of motion candidates may be allowed depending on the template matching scheme and / or prediction method. d. For example, multiple template matching methods can be based on different cost functions, that is, one template matching uses cost function A and another template matching uses cost function B. i. For example, the cost function may be based on a measure of discontinuity across block boundaries. ii. For example, the cost function can be based on the sum of the absolute second-order derivatives in the residual domain for the upper rows and left columns, as shown below: Where R is the reconstructed neighbor, P is the prediction of the current block, and a and b are variables. e. For example, which template matching method is used for a video block can be signaled as a syntax element (eg, mode index, syntax parameter, flag, etc.). f. For example, different template matching methods can be based on different template positions. i. Templates can have different sample rows. ii. The template may include samples to the left and above the current block. iii. The template may only include samples to the left of the current block. iv. The template may contain only samples above the current block. v. The template may include samples of at least one neighboring sample of the current block and at least one prediction sample. vi. The template can contain only neighboring samples. vii. The template may only include prediction samples of the current block. Motion-based candidates 5. For example, the codec information of more than one column and / or more than one row of neighboring coded blocks may be used for the current video block coding. a. For example, a video unit may be encoded and decoded using affine merging, affine Amvp, normal merging, sub-block merging, CIIP, GPM, AMVP-merging, AMVP, intra, and any variants thereof, etc. b. For example, the coding information may be intra mode, inter prediction method, motion vector, reference index, prediction direction, coordinates / positions of neighboring coded blocks. c. For example, the size / size of neighboring blocks may be measured by the size / size of PU / TU / CU / sub-block or a fixed granularity such as 4x4 or 8x8 or 16x16. d. For example, consider M rows of neighboring blocks, where M may be a function of the distance between the current PU / TU / CU / subblock and the current Variable on how far the top border of the CTU / CTB is. i. For example, suppose that at most X1 (e.g., X1 is a constant) rows of neighboring blocks are considered, and from the current The distance from the top edge of the PU / TU / CU / sub-block to the current CTU / CTB is given by D=func(curblk, ctb_top_boundary) (for example, D is a variable), then M=X1-D. ii. Additionally, alternatively, the value of M complies with the invented rule only when the current PU / TU / CU / sub-block is not located at the top boundary of the CTU / CTB. iii. Additionally, alternatively, if the current PU / TU / CU / subblock is located at the top boundary of the CTU / CTB (eg, D=0), then M=X2 (eg, X2 is a predefined constant). iv. For example, X2 = 1 or 2 or 3 or 4 or other constant value. v. For example, X1 is a constant. vi. For example, X1 may be different from X2. e. Alternatively, consider M rows of neighboring blocks, where M can be a predefined constant (eg, M>1). i. For example, if the PU / TU / CU / sub-block is not located at the top boundary of the CTU / CTB, this rule may be applied to the PU / TU / CU / sub-block. ii. Alternatively, if the PU / TU / CU / sub-block is located at the top boundary of the CTU / CTB, this rule may be applied to the PU / TU / CU / sub-block. f. For example, consider N columns of neighboring blocks, where N may be determined by the distance between the current PU / TU / CU / subblock and the current Variable for how far the left border of CTU / CTB is. i. For example, assuming that at most Y1 (e.g., Y1 is a constant) rows of neighboring blocks are considered, and the distance from the current PU / TU / CU / subblock to the left boundary of the current CTU / CTB is expressed as DD=func(curblk, ctb_top_boundary) (e.g., DD is a variable), then M=Y1-DD. ii. Additionally, alternatively, the value of M complies with the invented rule only when the current PU / TU / CU / sub-block is not located at the left boundary of the CTU / CTB. iii. Additionally, alternatively, if the current PU / TU / CU / subblock is located at the left boundary of the CTU / CTB (eg, DD=0), M=Y2 (eg, Y2 is a predefined constant). iv. For example, Y2 = 1 or 2 or 3 or 4 or other constant value. v. For example, Y1 is a constant. vi. For example, Y1 may be different from Y2. g. For example, consider a neighboring block of N columns, where N can be a constant (eg, N>1). i. For example, if the PU / TU / CU / sub-block is not located at the left boundary of the CTU / CTB, this rule may be applied to the PU / TU / CU / sub-block. ii. For example, if the PU / TU / CU / sub-block is located at the left boundary of the CTU / CTB, this rule may be applied to the PU / TU / CU / sub-block. h. For example, each PU / TU / CU / subblock of M rows and / or N columns of neighboring blocks may be checked until the total number of valid candidates meets a predefined value. i. Alternatively, some PUs / TUs / CUs / sub-blocks may be checked according to a rule (eg, every two of them, etc.). ii. Additionally, similarity checks may be applied to count valid candidates. i. Additionally, alternatively, an adaptive re-ordering based method (eg, ARMC) may be used to order the motion / mode candidates of M rows and / or N columns of neighboring blocks. 6. For example, whether and / how to apply a template-based approach may depend on the location of the current block. a. For example, if the current block is located at the upper boundary of a CTU (or other area, such as a VPDU), the template-based method is not used. b. For example, the template-based method is used only when the current block is located at the upper boundary of the CTU (or other area, such as VPDU). c. For example, if the current block is located at the left boundary of a CTU (or other region such as a VPDU), the template-based method is not used. d. For example, the template-based method is used only when the current block is located at the left boundary of a CTU (or other region such as a VPDU). e. For example, if the current block is located at the upper boundary of a CTU (or other area, such as a VPDU), the template does not contain samples above the current block. f. For example, the template may include samples above the current block only when the current block is located at the upper boundary of the CTU (or other area, such as VPDU). g. For example, if the current block is located at the left boundary of a CTU (or other region such as a VPDU), the template does not include samples on the left side of the current block. h. For example, the template may contain samples on the left side of the current block only when the current block is located at the left boundary of the CTU (or other area such as VPDU). Based on history table 7. For example, at least one history-based intra-mode table may be maintained for a video unit codec. a. For example, the history table is updated on the fly using the intra mode information of the decoded block. b. For example, pruning / redundancy / similarity checking may be used to add new candidates to the history table. c. For example, a history-based intra-mode table may be maintained for the codec blocks in the current picture. i. For example, at most K1 intra-mode candidates from previously coded blocks in the current picture may be maintained in a table. d. For example, a history-based intra-mode table may be maintained for codec blocks in a reference picture. i. For example, at most K2 intra mode candidates from the identified block in the reference picture are maintained in a table. e. For example, the MPM list construction for the current intra block codec can be based on the intra modes in the history table. i. For example, a predefined number of intra mode candidates may be selected from a history table based on a rule. ii. For example, intra mode candidates may be first sorted (eg, via a template-based approach) and then some of them are selected for MPM list generation. f. Alternatively, the intra modes in the table may be understood as predefined modes for current intra block coding. 8. For example, at least one history-based partition information table may be maintained for a video unit codec. a. For example, the history table is updated on the fly using the decoded CU / PU / TU / CTU / CB / PB / TB / CTB partition information. i. For example, partition information of previously coded blocks in the current picture may be considered. ii. Additionally, alternatively, partition information of a block coded in the time domain in a reference picture may be considered. b. For example, pruning / redundancy / similarity checking may be used to add new candidates to the history table. c. For example, a candidate in the partition table may consist of a partition depth (eg, QT depth, MTT depth), the size of the codec tree node, partition tree information (eg, BT or TT, vertical or horizontal for each depth), etc. d. For example, the partition information of the current block (eg, codec tree node) can be derived from the candidate index of the history-based partition table. i. For example, the candidate index may be signaled / presented in the bitstream. ii. For example, the partition information of the current block (eg, codec tree node) may not be directly transmitted through a signal. e. For example, whether to use history-based partition information derivation can be signaled in the bitstream. i. For example, a block-based (eg, codec tree node) syntax element (eg, a flag or mode index) may be signaled for the current block, indicating whether the partition information is directly signaled or derived from a history-based partition table. ii. For example, the SPS / PPS / PH / SH / CTU / CTB flags may be signaled to indicate that the history-based partition table approach is allowed at a video unit level higher than blocks. Adaptive hybrid width related (template, GPM) In the present disclosure, the term "video unit" or "codec unit" may refer to a template, a partition, a sub-partition, a codec tree block (CTB), a codec tree unit (CTU), a codec block (CB), a CU, a PU, a TU, a PB, a TB. 9. An adaptively changeable mixing region may be applied to blend / merge / mix the first template / partition / sub-partition / sub-block and the second template / partition / sub-partition / sub-block of a video unit. a. In one example, when fusing at least two templates / partitions / sub-partitions / sub-blocks, the fused samples in the mixed area may be derived as a weighted sum of first corresponding samples in the first template / partition / sub-partition / sub-block and second corresponding samples in the second template / partition / sub-partition / sub-block. b. In one example, when fusing at least two templates / partitions / sub-partitions / sub-blocks, the fused samples outside the mixed area may be set equal to the first corresponding samples in the first template / partition / sub-partition / sub-block, or equal to the second corresponding samples in the second template / partition / sub-partition / sub-block. c. For example, the fused template / partition / sub-partition / sub-block may be used to derive at least one cost. i. The cost may be used to derive at least one MV. ii. The cost may be used to derive at least one reference picture. iii. Costs can be used to derive at least one model. iv. Costs can be used to derive at least one GPM hybrid method. d. In one example, the blending area may be determined by the blending width. i. For example, Fig.29A A schematic diagram showing an example of mixing two partitions of a CU (i.e., partition 0 and partition 1), where "D" represents the mixing width. Fig.29A As shown in , the blending area of ​​a video unit (e.g., CU / PU / TU) can be based on the blending width (e.g., Fig.29A D). ii. For example, Fig.29B A schematic diagram showing an example of mixing two top templates (i.e., top template 0 and top template 1) is shown, where "D" represents the mixing width. Fig.29B As shown in , a blending region (eg, a template, a portion of a template) of a video unit can be based on a blending width (eg, Fig.29B D). e. For example, a video unit may contain more than one partition. i. For example, the video unit may be encoded and decoded using the GPM mode and / or its variant modes (eg, GPM segmentation mode reordering based on template matching, GPM with template matching, GPM with motion vector difference, GPM inter-intra, etc.). ii. Furthermore, the video unit may be a reference video unit for a GPM (and / or variants thereof) codec block. iii. Alternatively, for example, the video unit may be coded using multiple hypothesis predictions other than GPM (and / or variants thereof). f. For example, video units can be encoded and decoded using a template-based approach. i. Alternatively, video units may not be encoded or decoded using a template-based approach. g. For example, a weighted blending process based on an adaptive blending region may be applied to blend the first template and the second template, wherein samples around the dividing line may be fused / blended / merged from samples of the first template and samples of the second template. i. For example, according to the partition pattern / line of the video unit, neighboring samples on the left and / or above the video unit may be divided into a first template and a second template. ii. For example, a template may be constructed from more than one sample adjacent to a video unit. 1) For example, a template may construct a row and / or a column of samples. a. Alternatively, the template can construct more than one row of samples. b. Additionally, alternatively, the template may construct more than one column of samples. 2) For example, the first template may be constructed from samples adjacent to the first partition of the video unit. a. In addition, a second template may be constructed from samples adjacent to a second partition of the video unit. h. For example, the same blend width (and / or rule to derive the blend width) may be used for template sample blending (ie, template samples are outside the video unit) and block sample blending (ie, block samples are inside the video unit). i. Alternatively, different blending widths (and / or rules for deriving blending widths) may be used. i. For example, the blending widths for more than one template / partition / sub-partition / sub-block used to blend a video unit may not be equal to a fixed value. i. For example, the blend width may depend on the dimensions / size of the template (such as width and / or height). ii. For example, the blending width may depend on the dimensions / size (such as width and / or height) of the video unit. iii. For example, the mixing width may depend on the decoded information. 1) For example, it may depend on the prediction method used for the video unit. 2) For example, it may depend on a signaling index / parameter / variable. 3) For example, it can depend on the template cost based on the neighboring sample information. iv. For example, the blend width may depend on predefined set(s) (or lookup table(s)). 1) For example, at least one predefined set (or lookup table) may be defined. 2) For example, more than one set (or lookup table) may be defined. a. For example, which set (or lookup table) is used for a video unit can be based on the dimensions (eg, width and / or height) of the video unit. b. For example, which set (or lookup table) is used for a video unit may be based on the CTU size of the video sequence. c. For example, which set (or lookup table) is used for a video unit may be based on the resolution of the video sequence. d. For example, which set (or lookup table) is used for a video unit may be based on another syntax element (eg, a syntax element signaled at a higher level than the block level). 3) For example, more than one element may be defined in a predefined set (or lookup table). 4) For example, at least one element may be defined in a predefined set (or a lookup table). v. For example, the mixing width may depend on the decoded information and a predefined lookup table. 1) For example, the mixing width can be derived based on the signaling index / parameter / variable and a predefined lookup table. vi. Alternatively, the blend width may be equal to a fixed value (eg, predefined). vii. For example, the final hybrid template can be used for template cost derivation of video units. j. For example, a syntax element may be signaled at the video unit level indicating the mixing width of the video unit and / or template. i. For example, a first syntax element may be signaled indicating the mixing width to be used for the video unit. 1) For example, fixed length codec can be used for its binarization. 2) Alternatively, variable length coding can be used for its binarization. a. For example, the first binary bit may indicate whether a non-zero blend width is used for the video unit, while the other binary bits may indicate which non-zero blend width is used. 3) For example, the blend width for a video unit may be looked up from a predefined lookup table. ii. For example, at least one binary bit of the syntax element may be context coded, and / or the context model may depend on the coded information of the left / above block. iii. Alternatively, at least one binary bit of the syntax element may be bypass coded. k. For example, the blend width for a video unit may be derived implicitly based on the template cost. i. For example, more than one stencil cost may be calculated for a video unit, each result from a hypothetical blend width, and the blend width that results in the lowest stencil cost may be used as the target blend width for the video unit. 10. In one example, a mixing method (such as a mixing area or a mixing width) used in a codec tool such as GPM or GMVD or GPM-TM or inter-intra GPM or GPM mode reordering can be determined using a first signaling message and / or a first decoder derived information. a. In one example, a mixing method and / or mixing area in a mode reordering such as GPM or GMVD or GPM-TM or inter-intra GPM or GPM may be determined using both at least one signaling message and at least one decoder derived information. b. In one example, the first message may be signaled in SPS / PPS / APS / picture header / slice header / CTU / CU / PU / TU. c. The first piece of information may include the block width (W) and / or height (H). d. In one example, whether the mixing method and / or the mixing area is determined by using the first signaling message or by using the first decoder-derived information may be determined by the second signaling message and / or the second decoder-derived information. e. In one example, the second message may be signaled in SPS / PPS / APS / picture header / slice header / CTU / CU / PU / TU. f. The second piece of information may include the block width (W) and / or height (H). i. For example, if W>=T1 and H>=T2, the hybrid method can be determined through the message. ii. For example, if W<=T1 and H<=T2, the hybrid method can be determined by the message. iii. For example, if W>=T1 or H>=T2, the hybrid method can be determined through the message. iv. For example, if W<=T1 or H<=T2, the hybrid method can be determined through the message. g. In one example, the signaling rule of the first message or the second message may depend on the third decoder derivation information. i. The third decoder derived information may include W and / or H. h. The hybrid method may be determined jointly by the first signaling message and the first decoder derived information. i. In one example, the mixing methods for different components (such as luma and chroma) can be determined in the same way or in different ways. i. In one example, the blending method for luma is performed in an adaptive manner, but for chroma it is performed in a fixed manner. 11. For example, the maximum value of the blending weight may depend on the blending method (such as blending area or blending width). a. For example, the maximum value of the blending weight can be derived based on the blending width. i. For example, a higher precision (ie, a larger value) for the maximum value of the blending weights may be used for video units with a wider blending width. b. For example, the maximum value of the blending weight may be derived based on whether an adaptive blending width is used for a video unit. c. For example, the maximum value of the blending weight can be derived based on whether adaptive blending width is allowed (at a higher level, such as SPS / PPS / PH / SH). i. For example, the maximum value of the mixing weight may be derived based on a high-level syntax element indicating whether adaptive mixing width is allowed at such a level. ii. For example, if a fixed mixing width is used (ie, adaptive mixing width is not allowed), the maximum value of the mixing weight may be equal to T1 (such as T1=8, etc.), otherwise, the maximum value of the mixing weight may be equal to T2 (such as T2=32, etc.). d. For example, an adaptive maximum of mixing weights can be used for video units. i. For example, the maximum value of the mixing weights of the first template / partition / sub-partition / sub-block and the second template / partition / sub-partition / sub-block used to mix the video unit may depend on the signaling index / parameter. ii. For example, the maximum value of the mixing weights to be used can be implicitly derived from the codec information. e. Alternatively, the maximum value of the mixing weight can be equal to a fixed value (eg, 32, 16, 64, etc.). 12. For example, motion and / or intra-mode storage for a multi-hypothesis coded video unit may depend on the blending width for the video unit. a. For example, the multi-hypothesis encoding and decoding mode can be: i. Conventional GPM mode; ii. Template matching based re-ranking for GPM segmentation mode; iii. GPM with template matching; iv. GPM with motion vector difference; v.GPM inter-intra; vi. CIIP and / or its variants; vii. MHP and / or its variants; viii. BCW and / or its variants. b. For example, how to store motion information of a multi-hypothesis coded video unit may depend on the mix width used for the video unit. i. For example, if the blend width is greater than a predefined value (such as a threshold), the blend / combined / bi-predicted motion may be stored for the sub-blocks within the blend region. c. For example, how to store intra / inter mode information for inter-intra codec video units (such as GPM Inter-Intra, CIIP, etc.) may depend on the mixing width used for the video unit. i. For example, if the blend width is greater than a predefined value (such as a threshold), the generated pattern may be stored for the sub-blocks within the blend area. CABAC codec related 13. Golomb-Rice codec can be used for syntax elements with sorted priority / selection rate / probability. a. For example, the syntax elements may be the following syntax parameters / variables / elements following the reordering based approach. i. Merged index, BM merged index, TM merged index, affine merged index, GPM merged index, etc. ii.AMVP index. iii. AMVP merges candidate indexes. iv. MHP additional merge hypothesis index and / or MHP additional amvp hypothesis index. v. GPM segmentation mode index. vi. GPM MMVD candidate index. vii. GPM MMVD distance index. viii. GPM MMVD step index. ix. GPM intra mode index. x.GPM mixed-width index. RPR related 14. When RPR is enabled, the following can be deactivated / disabled / disabled: a. Methods related to intra-frame template matching. i. For example, ARMC, TM merge mode, AMVP-MERGE mode, MMVD-TM mode, Affine MMVD-TM mode, CIIP-TM mode, GPM-TM mode, GPM segmentation mode reordering mode, etc. b.DMVR related methods i. For example, AMVP-MERGE mode, BM mode, ADMVR mode, etc. c. For example, specific methods can be disabled by syntax flags. d. For example, a specific method can be disabled by setting a specific parameter to a large value to deactivate the method. Image border filling related 15. Picture border padding may be applied in at least one direction (eg, horizontally and / or vertically). a. For example, image border padding can be applied only in the horizontal direction. b. For example, image border padding can be applied only in the vertical direction. c. For example, picture border padding can be applied in both horizontal and vertical directions. General 16. Whether and / or how to apply the above-disclosed method can be transmitted by a signal at the sequence level / picture group level / picture level / slice level / tile group level, for example in the sequence header / picture header / SPS / VPS / DPS / DCI / PPS / APS / slice header / tile group header. 17. Whether and / or how the above disclosed method can be applied when a PB / TB / CB / PU / TU / CU / VPDU / CTU / CTU row / slice / tile / sub-picture / other type of area containing more than one sample or pixel is transmitted by a signal. 18. Whether and / or how to apply the above disclosed method may depend on the coded information, such as block size, color format, single / dual tree partitioning, color component, slice / picture type.

[0095] More details of embodiments of the present disclosure related to sample adjustment will be described below. The embodiments of the present disclosure should be considered as examples to explain general concepts and should not be interpreted in a narrow manner. In addition, these embodiments can be applied individually or in combination in any way.

[0096] As used herein, the term "block" may refer to a codec tree block (CTB), a codec tree unit (CTU), a codec block (CB), a codec unit (CU), a prediction unit (PU), a transform unit (TU), a prediction block (PB), a transform block (TB), a video processing unit including a plurality of samples / pixels, etc. A block may be rectangular or non-rectangular.

[0097] Fig.30 3000 is a flowchart of a method 3000 for video processing according to some embodiments of the present disclosure. The method 3000 may be implemented during a conversion between a current video block of a video and a bitstream of the video. Fig.30As shown, method 3000 starts at 3002, where a value for a metric of a mixed region in a direction is obtained. The mixed region is included in a target region associated with a current video block. By way of example and not limitation, the target region may include a coding unit (CU), a prediction unit (PU), a transform unit (TU), a template, a portion of a template, etc.

[0098] In some embodiments, the value for the metric is determined from a plurality of predetermined values ​​for the metric. This will be described in detail below. The value for the sample of the mixed area is determined based on the value of the sample of the first part for the target area and the value of the sample of the second part for the target area. The process can also be described as fusing, merging or mixing the first and second parts. In one example, the value for the sample of the mixed area can be determined as a weighted sum of the value of the sample of the first part and the value of the sample of the second part. In another example, the value for the sample of the mixed area can be set to be equal to the value of the sample of the first part or the value of the sample of the second part.

[0099] In some embodiments, the measurement may be the width between the two sides of the mixed region, that is, the distance between the two sides of the mixed region. The measurement may be in a direction perpendicular to the dividing line. This will refer to Figure 31A-Figure 31C Detailed description. Fig.31A An example of mixing two partitions (ie, partition 0 and partition 1) according to an embodiment of the present disclosure is shown. Partition 0 and partition 1 are divided by a partition line 3110. Fig.31A As shown, both mixing regions (i.e., mixing region 0 and mixing region 1) are rectangular. In this case, the metric for mixing region 0 is represented as D0, which corresponds to the width of the rectangle, i.e., the distance between the dividing line 3110 and the line 3112. Similarly, the metric for mixing region 1 is represented as D1, which corresponds to the width of the rectangle, i.e., the distance between the dividing line 3110 and the line 3114. In some alternative embodiments, the two mixing regions (i.e., mixing region 0 and mixing region 1) as a whole can be regarded as a single mixing region, and thus the sum of the widths D0 and D1 can be regarded as the metric of the single mixing region.

[0100] Fig.31B Another example of mixing two partitions (ie, partition 2 and partition 3) according to an embodiment of the present disclosure is shown. Partition 2 and partition 3 are divided by a dividing line 3120. Fig.31BAs shown, mixing region 2 is a parallelogram and mixing region 3 is a pentagon. In this case, the measurement for mixing region 2 is represented as D2, which corresponds to the height of the parallelogram, that is, the distance between dividing line 3120 and line 3122. The measurement for mixing region 3 is represented as D3, which corresponds to the distance between dividing line 3120 and line 3124. It should be understood that D2 can also be referred to as the width of mixing region 2, and D3 can also be referred to as the width of mixing region 3. In some alternative embodiments, the two mixing regions (i.e., mixing region 2 and mixing region 3) as a whole can be regarded as a single mixing region, and therefore the sum of widths D2 and D3 can be regarded as a measurement of a single mixing region.

[0101] Fig.31C Another example of mixing two partitions (ie, partition 4 and partition 5) according to an embodiment of the present disclosure is shown. Partition 4 and partition 5 are divided by a dividing line 3130. Fig.31C As shown, both mixing regions (i.e., mixing region 4 and mixing region 5) are trapezoidal. In this case, the measurement for mixing region 4 is represented as D4, which corresponds to the height of the trapezoid, i.e., the distance between dividing line 3130 and line 3132. Similarly, the measurement for mixing region 5 is represented as D5, which corresponds to the height of the trapezoid, i.e., the distance between dividing line 3130 and line 3134. It should be understood that D4 may also be referred to as the width of mixing region 4, and D5 may also be referred to as the width of mixing region 5. In some embodiments, the two mixing regions (i.e., mixing region 4 and mixing region 5) as a whole may be regarded as a single mixing region, and therefore the sum of widths D4 and D5 may be regarded as a measurement of a single mixing region.

[0102] It should be understood that the above illustrations and / or examples are described for illustration purposes only, and the scope of the present disclosure is not limited in this regard.

[0103] At 3004, a conversion is performed based on the value for the metric. In one example, the conversion may include encoding the current video block into a bitstream. Alternatively or additionally, the conversion may include decoding the current video block from a bitstream.

[0104] In view of the above, the value (such as width) of the mixed area for measurement is determined according to a plurality of predetermined values ​​for the measurement. Compared with the traditional solution where the value for measurement is fixed, the proposed method can advantageously support the adaptive size of the mixed area. Thus, the coding efficiency and coding quality can be improved.

[0105] In some embodiments, the first portion and / or the second portion may include a template of the current video block, a partition of the current video block, a sub-partition of the current video block, a sub-block of the current video block, or the like.

[0106] In some embodiments, the cost may be determined based on the target region. In one example, based on the cost, a motion vector (MV) may be determined from a plurality of MVs for the current video block. In another example, based on the cost, a reference picture may be determined from a plurality of reference pictures for the current video block. In another example, based on the cost, a partitioning mode may be determined from a plurality of partitioning modes for the current video block. In yet another example, based on the cost, a geometric partitioning mode (GPM) hybrid scheme is determined from a plurality of GPM hybrid schemes for the current video block.

[0107] In some embodiments, the mixed area can be determined based on the value used for the metric. In some embodiments, the current video block can include more than one partition. In some embodiments, the current video block can be encoded and decoded using a GPM-based mode or multi-hypothesis prediction. Additionally or alternatively, the current video block can be a reference video block of another video block of the video. The other video block is different from the current video block and is encoded and decoded using a GPM-based mode. As an example and not limitation, the GPM-based mode can include one of the following: GPM mode, template matching-based reordering for GPM segmentation mode, GPM with template matching, GPM with motion vector difference, or GPM inter-frame-intra-frame mode.

[0108] In some embodiments, the current video block may be encoded using a template-based encoding and decoding tool. Alternatively, the current video block may not be encoded using a template-based encoding and decoding tool.

[0109] In some embodiments, the first portion may be a first template of the current video block, and the second portion may be a second template of the current video block. The mixed region may be around a dividing line between the first template and the second template (also referred to herein as a "dividing line"). Based on the width of the mixed region, a weighted mixing process may be applied to the first template and the second template. That is, a weighted mixing process based on an adaptive mixed region may be applied to mix the first template and the second template. For example, samples around the dividing line may be mixed from samples of the first template and samples of the second template.

[0110] In some embodiments, the left neighboring samples of the current video block and / or the upper neighboring samples of the current video block can be divided into a first template and a second template based on a partitioning mode or a partitioning line of the current video block. Additionally, the first template and / or the second template may include more than one sample adjacent to the current video block. In one example, the first template or the second template may include a row of samples adjacent to the current video block or a column of samples adjacent to the current video block. In another example, the first template or the second template may include more than one row of samples adjacent to the current video block or more than one column of samples adjacent to the current video block.

[0111] In some embodiments, the first template may include samples adjacent to a first partition of the current video block. Additionally, the second template may include samples adjacent to a second partition of the current video block that is different from the first partition.

[0112] In some embodiments, the value of the metric for the mixed region for the mixed template sample may be the same as the value of the metric for the mixed region for the mixed block sample. Additionally or alternatively, the value of the metric for the mixed region for the mixed template sample may be determined according to the same rules as the value of the metric for the mixed region for the mixed block sample. Fig.29B , the width of the mixing area for mixing samples of top template 0 and top template 1 is the same as the width of the mixing area for mixing samples of partition 0 and partition 1 of the video block.

[0113] In some alternative embodiments, the value of the metric for the mixed region for the mixed template sample may be different from the value of the metric for the mixed region for the mixed block sample. Additionally or alternatively, the value of the metric for the mixed region for the mixed template sample may be determined according to a rule different from the value of the metric for the mixed region for the mixed block sample.

[0114] In some embodiments, the value for the metric may be different from the value of the metric for a mixed region in another target region associated with another video block of the video.The other video block may be different from the current video block.

[0115] In some embodiments, the value for the measurement is determined from a plurality of predetermined values ​​based on the size of the template of the current video block. For example, the value for the measurement is determined from a plurality of predetermined values ​​based on the width and / or height of the template. In some alternative or additional embodiments, the value for the measurement is determined from a plurality of predetermined values ​​based on the size of the current video block. For example, the value for the measurement is determined from a plurality of predetermined values ​​based on the width and / or height of the current video block.

[0116] In some embodiments, the value for the metric is determined from a plurality of predetermined values ​​based on information decoded from the bitstream (i.e., decoded information). As an example and not limitation, the value for the metric is determined from a plurality of predetermined values ​​based on a prediction scheme for the current video block, an index indicated in the bitstream, a parameter indicated in the bitstream, a variable indicated in the bitstream, or a template cost depending on information of neighboring samples of the current video block. It should be understood that the above illustrations and / or examples are described for descriptive purposes only. The scope of the present disclosure is not limited in this regard.

[0117] In some embodiments, multiple predetermined values ​​may be stored in a lookup table. In some embodiments, multiple predetermined values ​​may be included in a first set of predetermined values. As an example and not limitation, based on the size of the current video block, the width of the current video block, the height of the current video block, the codec tree unit (CTU) size of the video, the resolution of the video, or the first syntax element indicated in the bitstream, the first set of predetermined values ​​is determined from multiple sets of predetermined values. In addition, each set of predetermined values ​​in the multiple sets of predetermined values ​​may be stored in a lookup table. It should be understood that the above illustrations and / or examples are described only for the purpose of description. The scope of the present disclosure is not limited in this regard.

[0118] In some embodiments, the first syntax element may be indicated at a level higher than the block level. Additionally or alternatively, a set of predetermined values ​​in the plurality of sets of predetermined values ​​may include at least one value. In an example, a set of predetermined values ​​in the plurality of sets of predetermined values ​​may include more than one value.

[0119] In some embodiments, the value used for the metric may be fixed for video blocks encoded in the same mode.In some embodiments, the target region may be a template of the current video block, and the template cost of the current video block may be determined based on sample values ​​of the template.

[0120] In some embodiments, a second syntax element indicating a value for the metric may be included in the bitstream.For example, the second syntax element may be indicated at the level of the current video block (ie, video unit level).

[0121] In some embodiments, a fixed length codec or a variable length codec may be used for binarization of the second syntax element. In some embodiments, a first binary bit of the second syntax element may indicate whether a non-zero blend width is used for the current video block, and another binary bit of the second syntax element may indicate a non-zero blend width for the current video block.

[0122] In some embodiments, at least one binary bit of the second syntax element may be context-coded. For example, the context model may depend on the codec information of the left neighboring video block or the upper neighboring video block of the current video block. In some alternative embodiments, at least one binary bit of the second syntax element may be bypass-coded.

[0123] In some embodiments, based on the template costs determined for at least a portion of the plurality of predetermined values, the value for the metric may be determined from the plurality of predetermined values. As an example and not limitation, the value having the lowest template cost may be determined as the value for the metric.

[0124] In some embodiments, based on first information included in the bitstream and / or second information not present in the bitstream, a value for a metric, a mixing region, or a mixing scheme used in a GPS-based codec tool may be determined. For example, the first information included in the bitstream may be a message transmitted with a signal, and the second information not present in the bitstream may be information derived by a decoder.

[0125] In some embodiments, based on third information included in the bitstream and / or fourth information not present in the bitstream, information about whether to determine the mixing area and / or the mixing scheme used in the GPM-based codec tool based on the first information or the second information can be determined. For example, the third information included in the bitstream can be a signaling message, and the fourth information not present in the bitstream can be information derived by the decoder.

[0126] In some embodiments, the GPM-based codec tool may include GPM, GMVD, GPM-TM, inter-frame-intra-frame GPM, GPM mode reordering, etc. In some embodiments, the first information may be indicated in a sequence parameter set (SPS), a picture parameter set (PPS), an adaptive parameter set (APS), a picture header (PH), a slice header (SH), a CTU, a CU, a PU, or a TU. Additionally, the second information may include the width of the current video block and / or the height of the current video block. In some embodiments, the third information may be indicated in an SPS, a PPS, an APS, a PH, a SH, a CTU, a CU, a PU, or a TU. In addition, the fourth information may include the width of the current video block and / or the height of the current video block.

[0127] In some embodiments, if the width is greater than or equal to the first threshold and the height is greater than or equal to the second threshold, the mixing area and / or the mixing scheme is determined based on the first information or the second information. Alternatively, if the width is less than or equal to the first threshold and the height is less than or equal to the second threshold, the mixing area and / or the mixing scheme is determined based on the first information or the second information.

[0128] In some alternative embodiments, if the width is greater than or equal to the first threshold or the height is greater than or equal to the second threshold, the mixing area and / or the mixing scheme may be determined based on the first information or the second information. Alternatively, if the width is less than or equal to the first threshold or the height is less than or equal to the second threshold, the mixing area and / or the mixing scheme may be determined based on the first information or the second information.

[0129] In some embodiments, the signaling rule of the first information or the third information may depend on fifth information that is not present in the bitstream. For example, the fifth information that is not present in the bitstream may be information derived by a decoder. As an example and not limitation, the fifth information may include at least one of the following: the width of the current video block, or the height of the current video block.

[0130] In some embodiments, a mixing scheme for a first component of a current video block may be determined in the same manner as a second component of the current video block that is different from the first component. Alternatively, the mixing scheme for the first component may be determined in a manner different from the second component. For example, the mixing scheme for the first component may be determined in an adaptive manner, and the mixing scheme for the second component may be determined in a fixed manner. In one example, the first component may be a luminance component, and the second component may be a chrominance component. Alternatively, the first component may be a chrominance component, and the second component may be a luminance component.

[0131] In some embodiments, the maximum value of the mixing weight used to determine the value of the sample for the mixed area can depend on the mixing scheme, the mixed area, or the value used for the metric. For example, the maximum value of the mixing weight for the value of the metric can be less than the maximum value of the mixing weight for another value greater than the value of the metric.

[0132] In some embodiments, based on whether the adaptive blend width is used for another video block of the video, a maximum value of a blend weight used to determine the value of the sample of the blend region of the other video block may be determined. In some embodiments, based on information about whether the adaptive blend width is allowed at a target level higher than the level of the current video block, a maximum value of the blend weight used to determine the value of the sample for the blend region may be determined. By way of example and not limitation, the target level may be an SPS level, a PPS level, a PH level, or a SH level.

[0133] In some embodiments, based on a syntax element indicated in the bitstream at a target level higher than the level of the current video block, a maximum value of a blend weight for determining the value of samples for the blend region may be determined. The syntax element may indicate whether adaptive blend width is allowed at the target level. By way of example and not limitation, the target level may be an SPS level, a PPS level, a PH level, or a SH level.

[0134] In some embodiments, if a fixed blend width is used for another video block of the video, the maximum value of the blend weights for determining the values ​​of the samples of the blend region of the other video block may be equal to the first value. If an adaptive blend width is used for another video block, the maximum value of the blend weights for determining the values ​​of the samples of the blend region of the other video block may be equal to the second value. As an example and not a limitation, the first value may be less than the second value. For example, the first value may be 8 and the second value may be 32.

[0135] In some embodiments, the maximum value of the mixing weight used to determine the value of the sample for the mixed region can be adaptive. In one example, the maximum value of the mixing weight used to determine the value of the sample for the mixed region can depend on an index or parameter indicated in the bitstream. In another example, based on the codec information of the video, the maximum value of the mixing weight used to determine the value of the sample for the mixed region can be determined.

[0136] In some embodiments, the maximum value of the blending weight used to determine the value of the sample for the blending region may be equal to a fixed value. By way of example and not limitation, the fixed value may be 32, 16, or 64.

[0137] In some embodiments, the current video block may be encoded using a multiple hypothesis encoding mode. Information about how to store motion information and / or intra mode information for the current video block may depend on the value used for the metric. For example, if the value used for the metric is greater than a first predefined value, then for the sub-blocks within the target region, a blended motion, a combined motion, or a bidirectionally predicted motion may be stored.

[0138] As examples and not limitations, the multi-hypothesis coding mode may be a conventional GPM mode, template matching based reordering for a GPM segmentation mode, GPM with template matching, GPM with motion vector difference, GPM inter-intra, a mode based on combined inter- and intra-prediction (CIIP), a mode based on multiple hypothesis prediction (MHP), or a mode based on bidirectional prediction (BCW) with CU level weights. It should be understood that the above illustrations and / or examples are described for descriptive purposes only. The scope of the present disclosure is not limited in this regard.

[0139] In some embodiments, the current video block can be encoded and decoded using an inter-intra codec mode. Information about how to store intra mode information or inter mode information for the current video block can depend on the value used for the metric. For example, the inter-intra codec mode can include GPM inter-intra or CIIP. In one example, if the value used for the metric is greater than a second predefined value, the generated mode is stored for the sub-blocks within the target area.

[0140] In some embodiments, if the candidates used in the codec tool are reordered based on priority, selectivity, or probability, a third syntax element indicating one of the candidates can be encoded and decoded using the Columbus-Rice codec. For example, the third syntax element can be an index to one of the candidates, and the candidates can include one of the following: a candidate for a merge mode, a candidate for a bilateral matching (BM) merge mode, a candidate for a template matching (TM) merge mode, a candidate for an affine merge mode, a candidate for a GPM merge mode, a candidate for an advanced motion vector prediction (AMVP), a candidate for an AMVP merge, a candidate for an MHP additional merge hypothesis, a candidate for an MHP additional AMVP hypothesis, a candidate for a GPM partition mode, a candidate for a GPM MMVD, a candidate for a GPM MMVD distance, a candidate for a GPM MMVD step size, a candidate for a GPM intra-frame mode, or a candidate for a GPM mixed width. It should be understood that the above illustrations and / or examples are described for descriptive purposes only. The scope of the present disclosure is not limited in this regard.

[0141] In some embodiments, if reference picture resampling is enabled, at least one of the following codec tools may be disabled: a codec tool associated with inter-frame template matching or a codec tool associated with decoder-side motion vector refinement (DMVR). As an example and not limitation, the codec tool associated with inter-frame template matching may be an adaptive reordering of merge candidates (ARMC) mode, a TM merge mode, an AMVP-MERGE mode, an MMVD-TM mode, an affine MMVD-TM mode, a CIIP-TM mode, a GPM-TM mode, or a GPM segmentation mode reordering mode. In addition, the codec tool associated with DMVR may be an AMVP-MERGE mode, a BM mode, or an adaptive DMVR mode. It should be understood that the above illustrations and / or examples are described for descriptive purposes only. The scope of the present disclosure is not limited in this regard.

[0142] In some embodiments, at least one coding tool may be disabled using a syntax flag.As an example and not limitation, at least one coding tool may be disabled by setting a value for a parameter to a predetermined value, such as a relatively large value.

[0143] In some embodiments, picture border padding can be applied in at least one direction. In one example, the at least one direction can include a horizontal direction and a vertical direction. In some alternative embodiments, picture border padding can be applied in a single direction. For example, the single direction can be a horizontal direction or a vertical direction.

[0144] According to another embodiment of the present disclosure, a non-transitory computer-readable recording medium is provided. The non-transitory computer-readable recording medium stores a bitstream of a video generated by a method performed by a device for video processing. In the method, a value for a metric in a mixed area in one direction is obtained. The mixed area is included in a target area associated with a current video block of the video. The value is determined from a plurality of predetermined values ​​for the metric. The value of the sample for the mixed area is determined based on the value of the sample for the first part of the target area and the value of the sample for the second part of the target area. In addition, a bitstream is generated based on the value for the metric.

[0145] According to some other embodiments of the present disclosure, a method for storing a bitstream of a video is provided. In the method, a value for a metric of a mixed region in one direction is obtained. The mixed region is included in a target region associated with a current video block of the video. The value is determined from a plurality of predetermined values ​​for the metric. The value of the sample for the mixed region is determined based on the value of the sample for the first part of the target region and the value of the sample for the second part of the target region. In addition, a bitstream is generated based on the value for the metric, and the bitstream is stored in a non-transitory computer-readable recording medium.

[0146] Embodiments of the present disclosure may be described in terms of the following clauses, and features of these clauses may be combined in any reasonable way.

[0147] Clause 1. A method for video processing, comprising: for a conversion between a current video block of a video and a bitstream of the video, obtaining a value for a metric in a direction for a mixed region included in a target region associated with the current video block, the value being determined from a plurality of predetermined values ​​for the metric, the value for a sample of the mixed region being determined based on a value for a sample of a first portion of the target region and a value for a sample of a second portion of the target region; and performing the conversion based on the value for the metric.

[0148] Clause 2. The method of clause 1, wherein the metric is the width between two sides of the mixing region.

[0149] Clause 3. A method according to any one of clauses 1 to 2, wherein the first part or the second part comprises one of the following: a template of the current video block, a partition of the current video block, a sub-partition of the current video block, or a sub-block of the current video block.

[0150] Clause 4. A method according to any one of clauses 1 to 3, wherein the value of the sample for the mixed region is determined as a weighted sum of the value of the sample for the first portion and the value of the sample for the second portion.

[0151] Clause 5. The method of any one of clauses 1 to 3, wherein the value of the sample for the mixed region is equal to the value of the sample for the first portion or the value of the sample for the second portion.

[0152] Clause 6. The method of any one of clauses 1 to 5, wherein a cost is determined based on the target area.

[0153] Clause 7. The method of clause 6, wherein based on the cost, a motion vector (MV) is determined from a plurality of MVs for the current video block.

[0154] Clause 8. The method of clause 6, wherein based on the cost, a reference picture is determined from a plurality of reference pictures for the current video block.

[0155] Clause 9. The method of clause 6, wherein based on the cost, a partitioning mode is determined from a plurality of partitioning modes for the current video block.

[0156] Clause 10. The method of clause 6, wherein based on the cost, a geometric partitioning mode (GPM) mixing scheme is determined from a plurality of GPM mixing schemes for the current video block.

[0157] Clause 11. The method of any one of clauses 1 to 10, wherein the mixed region is determined based on the value for the metric.

[0158] Clause 12. A method according to any one of clauses 1 to 11, wherein the target area comprises one of the following: a coding unit (CU), a prediction unit (PU), a transform unit (TU), a template, or a part of a template.

[0159] Clause 13. The method of any one of clauses 1 to 12, wherein the current video block comprises more than one partition.

[0160] Clause 14. The method of any one of clauses 1 to 13, wherein the current video block is encoded using a GPM-based mode or multi-hypothesis prediction.

[0161] Clause 15. The method of any one of clauses 1 to 14, wherein the current video block is a reference video block of another video block of the video, the other video block being different from the current video block and being encoded using a GPM-based mode.

[0162] Clause 16. A method according to any one of clauses 14 to 15, wherein the GPM-based mode comprises one of the following: GPM mode, template matching-based reordering for GPM segmentation mode, GPM with template matching, GPM with motion vector difference, or GPM inter-frame-intra-frame mode.

[0163] Clause 17. The method of any one of clauses 1 to 13, wherein the current video block is encoded using a template-based encoding tool.

[0164] Clause 18. The method of any one of clauses 1 to 13, wherein the current video block is not encoded using a template based encoding tool.

[0165] Clause 19. A method according to any one of clauses 1 to 18, wherein the first part is a first template of the current video block, the second part is a second template of the current video block, the mixed area is around a dividing line between the first template and the second template, and based on the width of the mixed area, a weighted mixing process is applied to the first template and the second template.

[0166] Clause 20. The method according to clause 19, wherein based on the partitioning manner of the current video block, at least one of the following is divided into the first template and the second template: a left neighboring sample of the current video block, or an upper neighboring sample of the current video block.

[0167] Clause 21. The method of clause 19, wherein the first template or the second template comprises more than one sample neighboring the current video block.

[0168] Clause 22. The method of clause 19, wherein the first template or the second template comprises at least one of: a row of samples adjacent to the current video block, or a column of samples adjacent to the current video block.

[0169] Clause 23. The method of clause 19, wherein the first template or the second template comprises more than one row of samples adjacent to the current video block, or more than one column of samples adjacent to the current video block.

[0170] Clause 24. The method of Clause 19, wherein the first template comprises samples adjacent to a first partition of the current video block.

[0171] Clause 25. The method of Clause 19, wherein the second template comprises samples adjacent to a second partition of the current video block that is different than the first partition.

[0172] Clause 26. A method according to any one of clauses 1 to 25, wherein the value of the metric for the mixed area for the mixed template sample is the same as the value of the metric for the mixed area for the mixed block sample, or the value of the metric for the mixed area for the mixed template sample is determined according to the same rule as the value of the metric for the mixed area for the mixed block sample.

[0173] Clause 27. A method according to any one of clauses 1 to 25, wherein the value of the metric for a mixed area for a mixed template sample is different from the value of the metric for a mixed area for a mixed block sample, or the value of the metric for a mixed area for a mixed template sample is determined according to a rule different from the value of the metric for a mixed area for a mixed block sample.

[0174] Clause 28. A method according to any one of clauses 1 to 27, wherein the value for the metric is different from the value of the metric for a mixed area in another target area associated with another video block of the video, the other video block being different from the current video block.

[0175] Clause 29. The method of any one of clauses 1 to 28, wherein obtaining the value for the metric comprises determining the value for the metric from the plurality of predetermined values ​​based on a size of a template of the current video block.

[0176] Clause 30. A method according to any one of clauses 1 to 28, wherein obtaining the value for the metric comprises: determining the value for the metric from the multiple predetermined values ​​based on at least one of the following: the width of the template of the current video block, or the height of the template of the current video block.

[0177] Clause 31. The method of any one of clauses 1 to 28, wherein obtaining the value for the metric comprises determining the value for the metric from among the plurality of predetermined values ​​based on a size of the current video block.

[0178] Clause 32. A method according to any one of clauses 1 to 28, wherein obtaining the value for the metric comprises: determining the value for the metric from the multiple predetermined values ​​based on at least one of the following: the width of the current video block, or the height of the current video block.

[0179] Clause 33. A method according to any one of clauses 1 to 28, wherein obtaining the value for the metric comprises: determining the value for the metric from the plurality of predetermined values ​​based on information decoded from the bitstream.

[0180] Clause 34. A method according to any one of clauses 1 to 28, wherein obtaining the value for the metric comprises: determining the value for the metric from the multiple predetermined values ​​based on one of: a prediction scheme for the current video block, an index indicated in the bitstream, a parameter indicated in the bitstream, a variable indicated in the bitstream, or a template cost that depends on information of neighboring samples of the current video block.

[0181] Clause 35. The method of any one of clauses 1 to 34, wherein the plurality of predetermined values ​​are stored in a lookup table.

[0182] Clause 36. The method of any one of clauses 1 to 35, wherein the plurality of predetermined values ​​are included in a first set of predetermined values.

[0183] Clause 37. A method according to clause 36, wherein obtaining the value for the metric comprises: determining the first set of predetermined values ​​from multiple sets of predetermined values ​​based on one of: the size of the current video block, the width of the current video block, the height of the current video block; the codec tree unit (CTU) size of the video, the resolution of the video, or a first syntax element indicated in the bitstream.

[0184] Clause 38. The method of clause 37, wherein the first syntax element is indicated at a level higher than a block level.

[0185] Clause 39. The method of any one of clauses 37 to 38, wherein a set of predetermined values ​​in the plurality of sets of predetermined values ​​comprises more than one value.

[0186] Clause 40. The method of any one of clauses 37 to 38, wherein a set of predetermined values ​​in the plurality of sets of predetermined values ​​comprises at least one value.

[0187] Clause 41. The method of any of clauses 1 to 40, wherein the value for the metric is fixed for video blocks encoded in the same mode.

[0188] Clause 42. The method of any one of clauses 1 to 41, wherein the target region is a template of the current video block, and the template cost of the current video block is determined based on sample values ​​of the template.

[0189] Clause 43. A method according to any of clauses 1 to 42, wherein a second syntax element indicating the value for the metric is included in the bitstream.

[0190] Clause 44. The method of clause 43, wherein the second syntax element is indicated at a level of the current video block.

[0191] Clause 45. The method of any one of clauses 43 to 44, wherein a fixed length codec or a variable length codec is used for binarization of the second syntax element.

[0192] Clause 46. A method according to any one of clauses 43 to 45, wherein a first binary bit of the second syntax element indicates whether a non-zero blend width is used for the current video block, and another binary bit of the second syntax element indicates a non-zero blend width for the current video block.

[0193] Clause 47. A method according to any one of clauses 43 to 45, wherein at least one binary bit of the second syntax element is context coded.

[0194] Clause 48. The method of clause 47, wherein the context model depends on codec information of a left neighboring video block or an upper neighboring video block of the current video block.

[0195] Clause 49. A method according to any one of clauses 43 to 45, wherein at least one binary bit of the second syntax element is bypass coded.

[0196] Clause 50. The method of any one of clauses 1 to 28, wherein the value for the metric is determined from the plurality of predetermined values ​​based on a template cost determined for at least a portion of the plurality of predetermined values.

[0197] Clause 51. The method of clause 50, wherein the value having the lowest template cost is determined as the value for the metric.

[0198] Clause 52. A method according to any one of clauses 1 to 51, wherein the value for the metric, the mixing region or the mixing scheme used in a GPM-based coding tool is determined based on at least one of: first information included in the bitstream, or second information not present in the bitstream.

[0199] Clause 53. A method according to any one of clauses 1 to 51, wherein information about determining the mixing area and / or mixing scheme used in a GPM-based coding tool based on first information included in the bitstream or second information not present in the bitstream is determined based on at least one of: third information included in the bitstream, or fourth information not present in the bitstream.

[0200] Clause 54. The method of any one of clauses 52 to 53, wherein the GPM-based codec tool comprises one of: GPM, GMVD, GPM-TM, Inter-Intra GPM, or GPM mode reordering.

[0201] Clause 55. A method according to any one of clauses 52 to 54, wherein the first information is indicated in one of the following: a sequence parameter set (SPS), a picture parameter set (PPS), an adaptation parameter set (APS), a picture header (PH), a slice header (SH), a CTU, a CU, a PU, or a TU.

[0202] Clause 56. The method of any one of clauses 52 to 55, wherein the second information comprises at least one of: a width of the current video block, or a height of the current video block.

[0203] Clause 57. A method according to any one of clauses 53 to 56, wherein the third information is indicated in one of the following: SPS, PPS, APS, PH, SH, CTU, CU, PU, ​​or TU.

[0204] Clause 58. The method according to any one of clauses 53 to 57, wherein the fourth information comprises at least one of the following: a width of the current video block, or a height of the current video block.

[0205] Clause 59. A method according to any one of clauses 53 to 58, wherein if the width is greater than or equal to a first threshold and the height is greater than or equal to a second threshold, the mixing area and / or the mixing scheme is determined based on the first information or the second information.

[0206] Clause 60. A method according to any one of clauses 53 to 58, wherein if the width is less than or equal to a first threshold and the height is less than or equal to a second threshold, the mixing area and / or the mixing scheme is determined based on the first information or the second information.

[0207] Clause 61. A method according to any one of clauses 53 to 58, wherein if the width is greater than or equal to a first threshold or the height is greater than or equal to a second threshold, the mixing area and / or the mixing scheme is determined based on the first information or the second information.

[0208] Clause 62. A method according to any one of clauses 53 to 58, wherein if the width is less than or equal to a first threshold or the height is less than or equal to a second threshold, the mixing area and / or the mixing scheme is determined based on the first information or the second information.

[0209] Clause 63. A method according to any of clauses 52 to 62, wherein a signalling rule of the first information or the third information depends on fifth information not present in the bitstream.

[0210] Clause 64. The method of clause 63, wherein the fifth information comprises at least one of: a width of the current video block, or a height of the current video block.

[0211] Clause 65. The method of any of clauses 52 to 64, wherein the mixing scheme for the first component of the current video block is determined in the same manner as a second component of the current video block that is different from the first component.

[0212] Clause 66. The method of any of clauses 52 to 64, wherein the mixing scheme for the first component of the current video block is determined differently than a second component of the current video block that is different from the first component.

[0213] Clause 67. A method according to any one of clauses 52 to 64, wherein the mixing scheme for a first component of the current video block is determined in an adaptive manner, and the mixing scheme for a second component of the current video block that is different from the first component is determined in a fixed manner.

[0214] Clause 68. A method according to any one of clauses 65 to 67, wherein the first component is a luma component and the second component is a chroma component, or the first component is a chroma component and the second component is a luma component.

[0215] Clause 69. A method according to any one of clauses 1 to 68, wherein the maximum value of the mixing weight used to determine the value of the sample for the mixing area depends on one of: the mixing scheme, the mixing area, or the value for the metric.

[0216] Clause 70. The method of clause 69, wherein a maximum value of a blending weight for the value of the metric is less than a maximum value of a blending weight for another value of the metric that is greater than the value.

[0217] Clause 71. A method according to any one of clauses 1 to 68, wherein a maximum value of a blending weight is determined based on whether an adaptive blending width is used for another video block of the video, and the blending weight is used to determine the value of a sample of a blending region of the other video block.

[0218] Clause 72. A method according to any one of clauses 1 to 68, wherein a maximum value of a blending weight for determining the values ​​of samples for the blending region is determined based on information about whether adaptive blending width is allowed at a target level above the level of the current video block.

[0219] Clause 73. A method according to any one of clauses 1 to 68, wherein a maximum value of a mixing weight for determining the value of samples for the mixed region is determined based on a syntax element indicated in the bitstream at a target level higher than the level of the current video block, the syntax element indicating whether adaptive mixing width is allowed at the target level.

[0220] Clause 74. The method according to any one of clauses 72 to 73, wherein the target level comprises one of the following: SPS level, PPS level, PH level, or SH level.

[0221] Clause 75. A method according to any one of clauses 1 to 68, wherein if a fixed blending width is used for another video block of the video, the maximum value of the blending weights used to determine the values ​​of samples of the blending area for the other video block is equal to a first value, and if an adaptive blending width is used for the other video block, the maximum value of the blending weights used to determine the values ​​of samples of the blending area for the other video block is equal to a second value.

[0222] Clause 76. The method of clause 75, wherein the first value is less than the second value.

[0223] Clause 77. A method according to any one of clauses 1 to 68, wherein a maximum value of a mixing weight used to determine the value for the sample of the mixing region is adaptive.

[0224] Clause 78. A method according to any of clauses 1 to 68, wherein a maximum value of a mixing weight used to determine the value for the sample of the mixed region depends on an index or parameter indicated in the bitstream.

[0225] Clause 79. A method according to any one of clauses 1 to 68, wherein a maximum value of a mixing weight used to determine the value for the sample of the mixed region is determined based on codec information of the video.

[0226] Clause 80. A method according to any one of clauses 1 to 68, wherein a maximum value of a mixing weight used to determine the value for the sample of the mixing region is equal to a fixed value.

[0227] Clause 81. A method according to any one of clauses 1 to 80, wherein the current video block is encoded using a multi-hypothesis encoding and decoding mode, and information about how to store motion information and / or intra-frame mode information of the current video block depends on the value used for the metric.

[0228] Clause 82. A method according to clause 81, wherein the multi-hypothesis encoding and decoding mode includes one of the following: conventional GPM mode, template matching based reordering for GPM partitioning mode, GPM with template matching, GPM with motion vector difference, GPM inter-intra, a mode based on combined inter- and intra-prediction (CIIP), a mode based on multiple hypothesis prediction (MHP), or a mode based on bidirectional prediction (BCW) with CU level weights.

[0229] Clause 83. A method according to any one of clauses 1 to 82, wherein if the value for the metric is greater than a first predefined value, then for a sub-block within the target area, mixed motion, combined motion or bidirectionally predicted motion is stored.

[0230] Clause 84. A method according to any one of clauses 1 to 80, wherein the current video block is encoded and decoded using an inter-intra codec mode, and information about how to store intra mode information or inter mode information for the current video block depends on the value used for the metric.

[0231] Clause 85. The method of clause 84, wherein the inter-intra codec mode comprises GPM inter-intra or CIIP.

[0232] Clause 86. A method according to any one of clauses 1 to 85, wherein if the value for the metric is greater than a second predefined value, the generated pattern is stored for a sub-block within the target area.

[0233] Clause 87. A method according to any one of clauses 1 to 86, wherein if the candidates used in the codec tool are reordered based on priority, selectivity or probability, a third syntax element indicating one of the candidates is encoded using Golomb-Rice encoding.

[0234] Clause 88. A method according to any one of clauses 1 to 86, wherein the third grammatical element is an index to one of the candidates, the candidate comprising one of: a candidate for merge mode, a candidate for bilateral matching (BM) merge mode, a candidate for template matching (TM) merge mode, a candidate for affine merge mode, a candidate for GPM merge mode, a candidate for advanced motion vector prediction (AMVP), a candidate for AMVP merge, a candidate for MHP additional merge hypothesis, a candidate for MHP additional AMVP hypothesis, a candidate for GPM partition mode, a candidate for GPM MMVD, a candidate for GPM MMVD distance, a candidate for GPM MMVD step size, a candidate for GPM intra mode, or a candidate for GPM mixing width.

[0235] Clause 89. A method according to any one of clauses 1 to 88, wherein if reference picture resampling is enabled, at least one of the following codec tools is disabled: codec tools related to intra-frame template matching, or codec tools related to decoder-side motion vector refinement (DMVR).

[0236] Clause 90. A method according to clause 89, wherein the codec tools associated with the intra-frame template matching include at least one of the following: adaptive reordering of merge candidates (ARMC) mode, TM merge mode, AMVP-MERGE mode, MMVD-TM mode, affine MMVD-TM mode, CIIP-TM mode, GPM-TM mode, or GPM segmentation mode reordering mode.

[0237] Clause 91. A method according to any one of clauses 89 to 90, wherein the DMVR-related codec tool includes at least one of the following: AMVP-MERGE mode, BM mode, or adaptive DMVR mode.

[0238] Clause 92. The method of any one of clauses 89 to 91, wherein the at least one codec tool is disabled using a syntax flag.

[0239] Clause 93. The method of any one of clauses 89 to 91, wherein the at least one codec tool is disabled by setting a value for a parameter to a predetermined value.

[0240] Clause 94. The method of any one of clauses 1 to 93, wherein picture border padding is applied in at least one direction.

[0241] Clause 95. The method of clause 94, wherein the at least one direction comprises a horizontal direction and a vertical direction.

[0242] Clause 96. A method according to any one of clauses 1 to 93, wherein picture border padding is applied in a single direction.

[0243] Clause 97. The method of clause 96, wherein the single direction is a horizontal direction or a vertical direction.

[0244] Clause 98. The method of any one of clauses 1 to 97, wherein the converting comprises encoding the current video block into the bitstream.

[0245] Clause 99. The method of any one of clauses 1 to 97, wherein the converting comprises decoding the current video block from the bitstream.

[0246] Clause 100. An apparatus for video processing, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform a method according to any one of clauses 1 to 99.

[0247] Clause 101. A non-transitory computer-readable storage medium storing instructions that cause a processor to perform the method of any one of clauses 1 to 99.

[0248] Item 102. A non-transitory computer-readable recording medium storing a bitstream of a video generated by a method performed by a device for video processing, wherein the method comprises: obtaining a value of a metric for a mixed region in a direction, the mixed region being included in a target region associated with a current video block of the video, the value being determined from a plurality of predetermined values ​​for the metric, the value of a sample for the mixed region being determined based on a value of a sample for a first portion of the target region and a value of a sample for a second portion of the target region; and generating the bitstream based on the value for the metric.

[0249] Clause 103. A method for storing a bitstream of a video, comprising: obtaining a value of a metric for a mixed region in a direction, the mixed region being included in a target region associated with a current video block of the video, the value being determined from a plurality of predetermined values ​​for the metric, the value of a sample for the mixed region being determined based on a value of a sample for a first portion of the target region and a value of a sample for a second portion of the target region; generating the bitstream based on the value for the metric; and storing the bitstream in a non-transitory computer-readable recording medium. Example Device

[0250] Fig.32A block diagram of a computing device 3200 in which various embodiments of the present disclosure may be implemented is shown. The computing device 3200 may be implemented as a source device 110 (or video encoder 114 or 200) or a destination device 120 (or video decoder 124 or 300), or may be included in a source device 110 (or video encoder 114 or 200) or a destination device 120 (or video decoder 124 or 300).

[0251] It should be understood that Fig.32 The computing device 3200 shown in the figure is for illustrative purposes only and is not intended to in any way imply any limitation on the functionality and scope of the embodiments of the present disclosure.

[0252] like Fig.32 As shown, computing device 3200 includes a general computing device 3200. Computing device 3200 may include at least one or more processors or processing units 3210, memory 3220, storage unit 3230, one or more communication units 3240, one or more input devices 3250, and one or more output devices 3260.

[0253] In some embodiments, the computing device 3200 can be implemented as any user terminal or server terminal with computing capabilities. The server terminal can be a server provided by a service provider, a large computing device, etc. The user terminal can be, for example, any type of mobile terminal, fixed terminal or portable terminal, including a mobile phone, a station, a unit, a device, a multimedia computer, a multimedia tablet computer, an Internet node, a communicator, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an electronic book device, a gaming device, or any combination thereof, and includes accessories and peripherals of these devices, or any combination thereof. It is conceivable that the computing device 3200 can support any type of interface to the user (such as a "wearable" circuit device, etc.).

[0254] The processing unit 3210 may be a physical processor or a virtual processor and may implement various processes based on a program stored in the memory 3220. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capability of the computing device 3200. The processing unit 3210 may also be referred to as a central processing unit (CPU), a microprocessor, a controller, or a microcontroller.

[0255] The computing device 3200 typically includes various computer storage media. Such media can be any media accessible by the computing device 3200, including but not limited to volatile media and non-volatile media, or removable media and non-removable media. The memory 3220 can be a volatile memory (e.g., a register, a cache, a random access memory (RAM)), a non-volatile memory (such as a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM) or flash memory) or any combination thereof. The storage unit 3230 can be any removable or non-removable medium, and can include machine-readable media, such as a memory, a flash drive, a disk, or other media that can be used to store information and / or data and can be accessed in the computing device 3200.

[0256] The computing device 3200 may also include additional removable / non-removable storage media, volatile / non-volatile storage media. Fig.32 Although not shown in the figure, a magnetic disk drive for reading from and / or writing to a removable nonvolatile magnetic disk and an optical disk drive for reading from and / or writing to a removable nonvolatile optical disk may be provided. In this case, each drive may be connected to the bus (not shown) via one or more data medium interfaces.

[0257] The communication unit 3240 communicates with another computing device via a communication medium. In addition, the functions of the components in the computing device 3200 can be implemented by a single computing cluster or multiple computing machines that can communicate via a communication connection. Therefore, the computing device 3200 can operate in a networked environment using logical connections to one or more other servers, networked personal computers (PCs), or other general network nodes.

[0258] Input device 3250 may be one or more of various input devices, such as a mouse, keyboard, trackball, voice input device, etc. Output device 3260 may be one or more of various output devices, such as a display, speaker, printer, etc. With the aid of communication unit 3240, computing device 3200 may also communicate with one or more external devices (not shown), such as storage devices and display devices, and computing device 3200 may also communicate with one or more devices that enable a user to interact with computing device 3200, or any device that enables computing device 3200 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.), if necessary. Such communication may be performed via an input / output (I / O) interface (not shown).

[0259] In some embodiments, some or all components of the computing device 3200 may also be arranged in a cloud computing architecture rather than being integrated in a single device. In a cloud computing architecture, components may be provided remotely and work together to implement the functions described in the present disclosure. In some embodiments, cloud computing provides computing, software, data access and storage services, which will not require the end user to know the physical location or configuration of the system or hardware that provides these services. In various embodiments, cloud computing provides services via a wide area network (such as the Internet) using a suitable protocol. For example, a cloud computing provider provides applications via a wide area network, which can be accessed through a web browser or any other computing component. The software or components of the cloud computing architecture and the corresponding data may be stored on a remote server. The computing resources in a cloud computing environment may be merged or distributed at the location of a remote data center. Cloud computing infrastructure may provide services through a shared data center, although they appear as a single access point for users. Therefore, the cloud computing architecture may be used to provide the components and functions described herein from a service provider at a remote location. Alternatively, they may be provided by a conventional server, or installed directly or otherwise on a client device.

[0260] In an embodiment of the present disclosure, the computing device 3200 may be used to implement video encoding / decoding. The memory 3220 may include one or more video encoding / decoding modules 3225 having one or more program instructions. These modules can be accessed and executed by the processing unit 3210 to perform the functions of the various embodiments described herein.

[0261] In an example embodiment performing video encoding, input device 3250 may receive video data as input 3270 to be encoded. The video data may be processed by, for example, video codec module 3225 to generate an encoded bitstream. The encoded bitstream may be provided as output 3280 via output device 3260.

[0262] In an example embodiment performing video decoding, input device 3250 may receive an encoded bitstream as input 3270. The encoded bitstream may be processed by, for example, video codec module 3225 to generate decoded video data. The decoded video data may be provided as output 3280 via output device 3260.

[0263] Although the present disclosure has been specifically shown and described with reference to the preferred embodiments of the present disclosure, it will be understood by those skilled in the art that various changes can be made in form and detail without departing from the spirit and scope of the present application as defined by the appended claims. These changes are intended to be encompassed by the scope of the present application. Therefore, the foregoing description of the embodiments of the present application is not intended to be limiting.

Claims

1. A method for video processing, comprising: For a transition between a current video block of a video and a bitstream of the video, obtaining a value of a metric for a mixed region in a direction, the mixed region being included in a target region associated with the current video block, the value being determined from a plurality of predetermined values ​​for the metric, the value for a sample of the mixed region being determined based on a value for a sample of a first portion of the target region and a value for a sample of a second portion of the target region; as well as The converting is performed based on the value for the metric.

2. The method of claim 1, wherein the metric is the width between two sides of the mixing region.

3. The method according to any one of claims 1 to 2, wherein the first part or the second part comprises one of the following: the template of the current video block, the division of the current video block, sub-division of the current video block, or A sub-block of the current video block.

4. The method according to any one of claims 1 to 3, wherein the value of the sample for the mixed region is determined as a weighted sum of the value of the sample for the first portion and the value of the sample for the second portion.

5. The method according to any one of claims 1 to 3, wherein the value of the sample for the mixed area is equal to the value of the sample for the first part or the value of the sample for the second part.

6. The method according to any one of claims 1 to 5, wherein a cost is determined based on the target area.

7. The method of claim 6, wherein based on the cost, a motion vector (MV) is determined from a plurality of MVs for the current video block.

8. The method of claim 6, wherein based on the cost, a reference picture is determined from a plurality of reference pictures for the current video block.

9. The method of claim 6, wherein based on the cost, a partitioning mode is determined from a plurality of partitioning modes for the current video block.

10. The method of claim 6, wherein based on the cost, a geometric partitioning mode (GPM) mixing scheme is determined from a plurality of GPM mixing schemes for the current video block.

11. The method according to any one of claims 1 to 10, wherein the mixing region is determined based on the value for the metric.

12. The method according to any one of claims 1 to 11, wherein the target area comprises one of: Codec Unit (CU), Prediction Unit (PU), Transformation Unit (TU), template, or Part of the template.

13. The method of any one of claims 1 to 12, wherein the current video block comprises more than one partition.

14. The method according to any one of claims 1 to 13, wherein the current video block is encoded using a GPM-based mode or multi-hypothesis prediction.

15. The method of any one of claims 1 to 14, wherein the current video block is a reference video block of another video block of the video, the other video block being different from the current video block and being encoded and decoded using a GPM-based mode.

16. The method according to any one of claims 14 to 15, wherein the GPM-based mode comprises one of the following: GPM mode, Template matching based re-ranking for GPM segmentation mode, GPM with template matching, GPM with motion vector difference, or GPM inter-intra mode.

17. The method of any one of claims 1 to 13, wherein the current video block is encoded using a template-based encoding tool.

18. The method of any one of claims 1 to 13, wherein the current video block is not encoded using a template-based encoding tool.

19. The method according to any one of claims 1 to 18, wherein the first part is a first template of the current video block, the second part is a second template of the current video block, the mixed area is around a dividing line between the first template and the second template, and based on the width of the mixed area, a weighted mixing process is applied to the first template and the second template.

20. The method of claim 19, wherein based on a partitioning manner of the current video block, at least one of the following is divided into the first template and the second template: The left neighboring sample of the current video block, or The upper neighboring samples of the current video block.

21. The method of claim 19, wherein the first template or the second template includes more than one sample adjacent to the current video block.

22. The method of claim 19, wherein the first template or the second template comprises at least one of the following: A row of samples adjacent to the current video block, or A column of samples adjacent to the current video block.

23. The method of claim 19, wherein the first template or the second template includes more than one row of samples adjacent to the current video block, or more than one column of samples adjacent to the current video block.

24. The method of claim 19, wherein the first template comprises samples adjacent to a first partition of the current video block.

25. The method of claim 19, wherein the second template comprises samples adjacent to a second partition of the current video block that is different than the first partition.

26. A method according to any one of claims 1 to 25, wherein the value of the metric for a mixed region for a mixed template sample is the same as the value of the metric for a mixed region for a mixed block sample, or The value of the metric for the mixed region for the mixed template samples is determined according to the same rules as the value of the metric for the mixed region for the mixed block samples.

27. A method according to any one of claims 1 to 25, wherein the value of the metric for a mixed region for a mixed template sample is different from the value of the metric for a mixed region for a mixed block sample, or The value of the metric for the mixing region for the mixing template samples is determined according to a different rule than the value of the metric for the mixing region for the mixing block samples.

28. The method of any one of claims 1 to 27, wherein the value for the metric is different from a value of the metric for a mixed region in another target region associated with another video block of the video, the other video block being different from the current video block.

29. The method of any one of claims 1 to 28, wherein obtaining the value for the metric comprises: The value for the metric is determined from the plurality of predetermined values ​​based on a size of a template of the current video block.

30. The method of any one of claims 1 to 28, wherein obtaining the value for the metric comprises: The value for the metric is determined from the plurality of predetermined values ​​based on at least one of: The width of the template of the current video block, or The height of the template of the current video block.

31. The method of any one of claims 1 to 28, wherein obtaining the value for the metric comprises: The value for the metric is determined from the plurality of predetermined values ​​based on a size of the current video block.

32. The method of any one of claims 1 to 28, wherein obtaining the value for the metric comprises: The value for the metric is determined from the plurality of predetermined values ​​based on at least one of: the width of the current video block, or The height of the current video block.

33. The method of any one of claims 1 to 28, wherein obtaining the value for the metric comprises: The value for the metric is determined from the plurality of predetermined values ​​based on information decoded from the bitstream.

34. The method of any one of claims 1 to 28, wherein obtaining the value for the metric comprises: The value for the metric is determined from the plurality of predetermined values ​​based on: a prediction scheme for the current video block, an index indicated in said bitstream, the parameters indicated in the bitstream, a variable indicated in the bitstream, or The template cost depends on the information of neighboring samples of the current video block.

35. A method according to any one of claims 1 to 34, wherein the plurality of predetermined values ​​are stored in a look-up table.

36. A method according to any one of claims 1 to 35, wherein the plurality of predetermined values ​​are comprised in a first set of predetermined values.

37. The method of claim 36, wherein obtaining the value for the metric comprises: The first set of predetermined values ​​is determined from a plurality of sets of predetermined values ​​based on one of: The size of the current video block, the width of the current video block, The height of the current video block; the codec tree unit (CTU) size of the video, the resolution of the video, or A first syntax element indicated in the bitstream.

38. The method of claim 37, wherein the first syntax element is indicated at a level higher than a block level.

39. A method according to any one of claims 37 to 38, wherein a set of predetermined values ​​in the plurality of sets of predetermined values ​​comprises more than one value.

40. The method of any one of claims 37 to 38, wherein a set of predetermined values ​​in the plurality of sets of predetermined values ​​comprises at least one value.

41. The method of any one of claims 1 to 40, wherein the value for the metric is fixed for video blocks encoded in the same mode.

42. The method of any one of claims 1 to 41, wherein the target region is a template of the current video block, and the template cost of the current video block is determined based on sample values ​​of the template.

43. The method of any one of claims 1 to 42, wherein a second syntax element indicating the value for the metric is included in the bitstream.

44. The method of claim 43, wherein the second syntax element is indicated at a level of the current video block.

45. The method according to any one of claims 43 to 44, wherein a fixed length codec or a variable length codec is used for binarization of the second syntax element.

46. ​​The method of any one of claims 43 to 45, wherein a first binary bit of the second syntax element indicates whether a non-zero blend width is used for the current video block, and another binary bit of the second syntax element indicates a non-zero blend width for the current video block.

47. The method of any one of claims 43 to 45, wherein at least one binary bit of the second syntax element is context-coded.

48. The method of claim 47, wherein the context model depends on codec information of a left neighboring video block or an upper neighboring video block of the current video block.

49. The method according to any one of claims 43 to 45, wherein at least one binary bit of the second syntax element is bypass coded.

50. A method according to any one of claims 1 to 28, wherein the value for the metric is determined from the plurality of predetermined values ​​based on a template cost determined for at least a portion of the plurality of predetermined values.

51. The method of claim 50, wherein the value having the lowest template cost is determined as the value for the metric.

52. The method of any one of claims 1 to 51, wherein the value for the metric, the mixing region or the mixing scheme used in a GPM-based codec tool is determined based on at least one of: The first information included in the bit stream, or The second information is not present in the bitstream.

53. The method according to any one of claims 1 to 51, wherein the information about determining the mixing area and / or mixing scheme used in a GPM-based codec tool based on first information included in the bitstream or second information not present in the bitstream is determined based on at least one of the following: The third information included in the bit stream, or Fourth information not present in the bitstream.

54. The method of any one of claims 52 to 53, wherein the GPM-based codec tool comprises one of: GPM, GMVD, GPM-TM, Inter-Intra GPM, or GPM mode reordering.

55. The method according to any one of claims 52 to 54, wherein the first information is indicated in one of: Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Adaptive Parameter Set (APS), Picture header (PH), Strip Header (SH), CTU, CU, PU, or TU.

56. The method according to any one of claims 52 to 55, wherein the second information comprises at least one of the following: the width of the current video block, or The height of the current video block.

57. The method according to any one of claims 53 to 56, wherein the third information is indicated in one of: SPS, PPS, APS, PH, SH, CTU, CU, PU, or TU.

58. The method according to any one of claims 53 to 57, wherein the fourth information comprises at least one of the following: the width of the current video block, or The height of the current video block.

59. A method according to any one of claims 53 to 58, wherein if the width is greater than or equal to a first threshold and the height is greater than or equal to a second threshold, the mixing area and / or the mixing scheme is determined based on the first information or the second information.

60. The method according to any one of claims 53 to 58, wherein if the width is less than or equal to a first threshold and the height is less than or equal to a second threshold, the mixing area and / or the mixing scheme is determined based on the first information or the second information.

61. A method according to any one of claims 53 to 58, wherein if the width is greater than or equal to a first threshold or the height is greater than or equal to a second threshold, the mixing area and / or the mixing scheme is determined based on the first information or the second information.

62. A method according to any one of claims 53 to 58, wherein if the width is less than or equal to a first threshold or the height is less than or equal to a second threshold, the mixing area and / or the mixing scheme is determined based on the first information or the second information.

63. The method according to any one of claims 52 to 62, wherein a signaling rule of the first information or the third information depends on fifth information not present in the bitstream.

64. The method according to claim 63, wherein the fifth information comprises at least one of the following: the width of the current video block, or The height of the current video block.

65. The method of any one of claims 52 to 64, wherein the mixing scheme for the first component of the current video block is determined in the same manner as a second component of the current video block that is different from the first component.

66. The method of any one of claims 52 to 64, wherein the mixing scheme for the first component of the current video block is determined differently than a second component of the current video block that is different from the first component.

67. The method according to any one of claims 52 to 64, wherein the mixing scheme for a first component of the current video block is determined in an adaptive manner, and the mixing scheme for a second component of the current video block different from the first component is determined in a fixed manner.

68. A method according to any one of claims 65 to 67, wherein the first component is a luminance component and the second component is a chrominance component, or The first component is a chrominance component, and the second component is a luma component.

69. A method according to any one of claims 1 to 68, wherein a maximum value of a mixing weight used to determine the value of a sample for the mixing region depends on one of: a mixing scheme, the mixing region, or the value for the metric.

70. The method of claim 69, wherein a maximum value of a blending weight for the value of the metric is less than a maximum value of a blending weight for another value of the metric that is greater than the value.

71. The method according to any one of claims 1 to 68, wherein a maximum value of a blending weight is determined based on whether an adaptive blending width is used for another video block of the video, and the blending weight is used to determine the value of the sample of the blending area of ​​the other video block.

72. A method according to any one of claims 1 to 68, wherein a maximum value of a blending weight for determining the value of samples for the blending region is determined based on information about whether adaptive blending width is allowed at a target level higher than the level of the current video block.

73. A method according to any one of claims 1 to 68, wherein a maximum value of a mixing weight for determining the value of samples for the mixed region is determined based on a syntax element indicated in the bitstream at a target level higher than the level of the current video block, the syntax element indicating whether adaptive mixing width is allowed at the target level.

74. The method of any one of claims 72 to 73, wherein the target level comprises one of: SPS level, PPS level, pH level, or SH level.

75. The method of any one of claims 1 to 68, wherein if a fixed blend width is used for another video block of the video, a maximum value of blend weights used to determine values ​​of samples of a blend region for the another video block is equal to a first value, and If an adaptive blend width is used for the other video block, a maximum value of blend weights used to determine values ​​of samples of a blend region for the other video block is equal to a second value.

76. The method of claim 75, wherein the first value is less than the second value.

77. A method according to any one of claims 1 to 68, wherein a maximum value of a mixing weight used to determine the value for the sample of the mixing region is adaptive.

78. A method according to any one of claims 1 to 68, wherein a maximum value of a mixing weight used to determine the value for samples of the mixed region depends on an index or parameter indicated in the bitstream.

79. The method according to any one of claims 1 to 68, wherein a maximum value of a mixing weight used to determine the value for the sample of the mixed region is determined based on codec information of the video.

80. A method according to any one of claims 1 to 68, wherein a maximum value of a mixing weight used to determine the value for the sample of the mixing region is equal to a fixed value.

81. The method according to any one of claims 1 to 80, wherein the current video block is encoded using a multi-hypothesis encoding mode, and information about how to store motion information and / or intra-frame mode information of the current video block depends on the value used for the metric.

82. The method of claim 81, wherein the multi-hypothesis coding mode comprises one of: Conventional GPM mode, Template matching based re-ranking for GPM segmentation mode, GPM with template matching, GPM with motion vector difference, GPM inter-intra, Based on the combined inter- and intra-prediction (CIIP) mode, Models based on Multiple Hypothesis Prediction (MHP), or Mode based on bidirectional prediction with CU level weights (BCW).

83. The method of any one of claims 1 to 82, wherein if the value for the metric is greater than a first predefined value, then for a sub-block within the target area, mixed motion, combined motion or bi-directionally predicted motion is stored.

84. The method according to any one of claims 1 to 80, wherein the current video block is encoded using an inter-intra codec mode, and information about how to store intra mode information or inter mode information for the current video block depends on the value used for the metric.

85. The method of claim 84, wherein the inter-intra codec mode comprises GPM inter-intra or CIIP.

86. A method according to any one of claims 1 to 85, wherein if the value for the metric is greater than a second predefined value, the generated pattern is stored for a sub-block within the target area.

87. A method according to any one of claims 1 to 86, wherein if candidates used in a codec tool are reordered based on priority, selectivity or probability, a third syntax element indicating one of the candidates is encoded using Golomb-Rice encoding.

88. The method of any one of claims 1 to 86, wherein the third syntax element is an index of one of the candidates, the candidate comprising one of: For candidates of merge mode, For candidates of the bilateral matching (BM) merge mode, For candidates of Template Matching (TM) merge mode, For candidates of affine merge mode, For candidates of GPM merge mode, For candidates of Advanced Motion Vector Prediction (AMVP), For candidates of AMVP merger, For the candidate of MHP additional merging hypothesis, For the candidate of MHP additional AMVP hypothesis, For the candidates of GPM segmentation mode, For GPM MMVD candidates, For the candidates of GPM MMVD distance, For the candidate GPM MMVD step size, A candidate for GPM intra mode, or Candidates for GPM mix width.

89. The method of any one of claims 1 to 88, wherein if reference picture resampling is enabled, at least one of the following codecs is disabled: Codec tools related to intra-frame template matching, or Codec tools related to decoder-side motion vector refinement (DMVR).

90. The method according to claim 89, wherein the codec tools related to the intra-frame template matching include at least one of the following: Adaptive Reordering of Merge Candidates (ARMC) mode, TM merge mode, AMVP-MERGE mode, MMVD-TM mode, Affine MMVD-TM mode, CIIP-TM mode, GPM-TM mode, or GPM split mode reorder mode.

91. The method according to any one of claims 89 to 90, wherein the DMVR-related codec tools include at least one of the following: AMVP-MERGE mode, BM mode, or Adaptive DMVR mode.

92. The method of any one of claims 89 to 91, wherein the at least one codec tool is disabled using a syntax flag.

93. The method of any one of claims 89 to 91, wherein the at least one codec tool is disabled by setting a value for a parameter to a predetermined value.

94. A method according to any one of claims 1 to 93, wherein picture border padding is applied in at least one direction.

95. The method of claim 94, wherein the at least one direction comprises a horizontal direction and a vertical direction.

96. A method according to any one of claims 1 to 93, wherein picture border padding is applied in a single direction.

97. The method of claim 96, wherein the single direction is a horizontal direction or a vertical direction.

98. The method of any one of claims 1 to 97, wherein the converting comprises encoding the current video block into the bitstream.

99. The method of any one of claims 1 to 97, wherein the converting comprises decoding the current video block from the bitstream.

100. An apparatus for video processing, comprising a processor and a non-volatile memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform a method according to any one of claims 1 to 99.

101. A non-transitory computer-readable storage medium storing instructions that cause a processor to perform a method according to any one of claims 1 to 99.

102. A non-transitory computer-readable recording medium storing a bitstream of a video generated by a method performed by an apparatus for video processing, wherein the method comprises: obtaining a value of a metric in a direction for a mixed region included in a target region associated with a current video block of the video, the value being determined from a plurality of predetermined values ​​for the metric, the value for a sample of the mixed region being determined based on a value for a sample of a first portion of the target region and a value for a sample of a second portion of the target region; as well as The bitstream is generated based on the value for the metric.

103. A method for storing a bitstream of a video, comprising: obtaining a value of a metric in a direction for a mixed region included in a target region associated with a current video block of the video, the value being determined from a plurality of predetermined values ​​for the metric, the value for a sample of the mixed region being determined based on a value for a sample of a first portion of the target region and a value for a sample of a second portion of the target region; generating the bitstream based on the value for the metric; as well as The bit stream is stored in a non-transitory computer-readable recording medium.

Images