Method and device for video processing and medium

Abstract

Embodiments of the present disclosure provide a solution for video processing, and provide a method for video processing, the method comprising: for a conversion between a current video block of a video and a bitstream of the video, determining a symbol prediction for a block vector difference (BVD) of the current video block; determining the BVD based at least on the symbol prediction of the BVD; and carrying out conversion based on BVD.

Description

Technical Field

[0001] Embodiments of the present disclosure generally relate to video coding techniques, and more particularly, to block vector construction and motion candidate list construction. Background Art

[0002] Nowadays, digital video functions are being applied to all aspects of people's lives. For video encoding / decoding, various types of video compression technologies have been proposed, such as MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4 Part 10 Advanced Video Codec (AVC), ITU-T H.265 High Efficiency Video Codec (HEVC) standard, and Versatile Video Codec (VVC) standard. However, the encoding and decoding efficiency of conventional video encoding and decoding technologies is usually very low, which is undesirable. Summary of the invention

[0003] The embodiments of the present disclosure provide a solution for video processing.

[0004] In a first aspect, a method for video processing is proposed. The method comprises: determining a sign prediction of a block vector difference (BVD) of the current video block for a conversion between a current video block of a video and a bitstream of the video; determining the BVD based at least on the sign prediction of the BVD; and performing the conversion based on the BVD. The method according to the first aspect of the present disclosure determines the BVD of the current video block based on the sign prediction of the BVD. Compared with the conventional scheme, the method proposed in the first aspect can advantageously improve the coding effectiveness and coding efficiency.

[0005] In a second aspect, another method for video processing is proposed. The method includes: for the conversion between a current video block of a video and a bitstream of the video, determining a cost associated with the current video block based on at least one of the following: a template matching cost associated with the current video block, a bilateral matching cost associated with the current video block, or a block boundary discontinuity measure associated with the current video block; performing processing on the current video block based on the cost; and performing the conversion based on the processing. The method according to the second aspect of the present disclosure determines a motion candidate list by adding motion candidates of different candidate types. Compared with traditional schemes, the method proposed in the second aspect can advantageously improve coding effectiveness and coding efficiency.

[0006] In a third aspect, a device for processing video data is provided. The video data processing device comprises a processor and a non-volatile memory having instructions thereon. The instructions, when executed by the processor, cause the processor to perform the method according to the first or second aspect of the present disclosure.

[0007] In a fourth aspect, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium stores instructions for causing a processor to execute the method according to the first or second aspect of the present disclosure.

[0008] In a fifth aspect, another non-transitory computer-readable recording medium is provided. The non-transitory computer-readable storage medium stores a bitstream of a video generated by a method performed by a video processing device. The method includes: determining a sign prediction of a block vector difference (BVD) of a current video block of the video; determining the BVD based on at least the sign prediction of the BVD; and generating the bitstream based on the BVD.

[0009] In a sixth aspect, a method for storing a bitstream of a video is provided. The method comprises: determining a sign prediction of a block vector difference (BVD) of a current video block of the video; determining the BVD based at least on the sign prediction of the BVD; generating the bitstream based on the BVD; and storing the bitstream in a non-transitory computer-readable recording medium.

[0010] In a seventh aspect, another non-transitory computer-readable recording medium is provided. The non-transitory computer-readable recording medium stores a bitstream of a video generated by a method performed by a video processing device. The method includes: determining a cost associated with a current video block of the video based on at least one of the following: a template matching cost associated with the current video block, a bilateral matching cost associated with the current video block, or a block boundary discontinuity metric associated with the current video block; performing processing on the current video block based on the cost; and generating the bitstream based on the processing.

[0011] In an eighth aspect, a method for storing a bitstream of a video is provided. The method includes: determining a cost associated with a current video block of the video based on at least one of: a template matching cost associated with the current video block, a bilateral matching cost associated with the current video block, or a block boundary discontinuity metric associated with the current video block; performing processing on the current video block based on the cost; generating the bitstream based on the processing; and storing the bitstream in a non-transitory computer-readable recording medium.

[0012] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become more apparent through the following detailed description with reference to the accompanying drawings. In the exemplary embodiments of the present disclosure, the same reference numerals generally refer to the same components.

[0014] Figure 1 A block diagram of an example video encoding and decoding system according to some embodiments of the present disclosure is shown;

[0015] Figure 2 A block diagram of a first example video encoder is shown according to some embodiments of the present disclosure;

[0016] Figure 3 A block diagram of an example video decoder is shown according to some embodiments of the present disclosure;

[0017] Figure 4 An example diagram showing example locations of spatial merge candidates is shown;

[0018] Figure 5 An example diagram showing candidate pairs considered for redundancy checking of spatial domain merging candidates is shown;

[0019] Figure 6 An example diagram showing example motion vector scaling for a temporal merge candidate is shown;

[0020] Figure 7 An example diagram showing candidate positions for time domain merging candidates C0 and C1 is shown;

[0021] Figure 8 An example diagram showing VVC spatial neighboring blocks of a current block is shown;

[0022] Fig. 9 An example diagram showing the relationship between a virtual block and a current block is shown;

[0023] Fig.10 An example diagram of spatial neighboring blocks used to derive spatial merge candidates is shown;

[0024] Fig.11A and Fig. 11B The SbTMVP process in VVC is shown respectively;

[0025] Figures 12A-12D The current CTU processing order and the available samples in the current CTU and the left CTU are shown respectively;

[0026] Fig.13 Neighboring samples used to calculate SAD are shown;

[0027] Fig.14 Neighboring samples used to calculate the SAD of sub-CU level motion information are shown;

[0028] Fig.15 An example diagram showing the classification process is shown;

[0029] Fig.16 An example diagram illustrating a reordering process in an encoder is shown;

[0030] Fig.17 An example diagram illustrating a reordering process in a decoder is shown;

[0031] Fig.18 shows an example diagram illustrating performing template matching on a search area around an initial MV;

[0032] Fig.19 An example graph showing template matching predictions is shown;

[0033] Fig. 20 An example diagram showing the intra-frame template matching search area used;

[0034] Fig.21 An example diagram showing a template and its reference sample used in TIMD;

[0035] Fig. 22 An example diagram showing a display template and a reference sample of the template is shown;

[0036] Fig.23 An example diagram showing templates in reference list 0 and reference list 1 and reference samples of the templates is shown;

[0037] Fig.24 An example diagram showing a template and a reference sample of the template using motion information of a sub-block of a current block for a block having sub-block motion is shown;

[0038] Fig.25 An example diagram showing the use of motion information of each sub-template for a block with sub-block motion is shown;

[0039] Fig.26 An example diagram showing a template for a block with OBMC and a reference sample of the template is shown;

[0040] Fig. 27 An example diagram showing motion estimation of a rectangular block with hash values ​​of square sub-blocks is shown;

[0041] Fig.28 An example luminance mapping with a chroma scaling architecture is shown;

[0042] Fig.29 shows example pairings in the merge candidate reordering ARMC-TM and additional pairing candidates after reordering;

[0043] Fig. 30A An example diagram showing candidate locations for spatial domain candidates is shown;

[0044] Fig. 30B An example diagram showing candidate locations for time domain candidates is shown;

[0045] Fig.31 An example diagram showing the derivation of a sub-CUbv motion field from a corresponding co-located sub-CU by applying motion shifts from spatial neighbors is shown;

[0046] Fig.32 An example diagram showing example intra-frame template matching is shown;

[0047] Fig.33A A schematic diagram showing a reference template outside the current picture is shown;

[0048] Fig.33B A schematic diagram showing cropping the BV so that the reference template is located inside the current picture is shown;

[0049] Fig.34 The non-adjacent locations used are shown;

[0050] Fig.35 A schematic diagram showing spatial candidates for IBC merge / AMVP candidate list is shown;

[0051] Fig.36 A schematic diagram showing a template and a reference sample of the template is shown;

[0052] Fig.37 An example location for TMVP use is shown;

[0053] Fig.38 An example of generating a HAPC is shown;

[0054] Fig.39 The spatial neighbors used to derive affine merge candidates are shown;

[0055] Fig.40 An example of affine merge candidates constructed from non-adjacent neighbors is shown;

[0056] Fig.41 An example of an affine motion model based on control points is shown;

[0057] Fig.42 An example affine MVF for each sub-block is shown;

[0058] Fig.43 Example positions of inherited affine motion predictors are shown;

[0059] Fig.44 An example control point motion vector inheritance is shown;

[0060] Fig.45 Example positions of candidate positions for constructing an affine merge pattern are shown;

[0061] Fig.46 An exemplary diagram showing the use of motion vectors for the proposed combined method;

[0062] Fig.47 An example sub-block MV VSB and a pixel Δv(i,j) are shown (arrows);

[0063] Fig.48 Example neighboring reconstructed blocks and a current prediction block are shown;

[0064] Fig.49A and Fig.49B The division methods of the angle modes are shown respectively;

[0065] Figure 50A-Figure 50C The available IPM candidates are shown respectively;

[0066] Fig.50D An example of a GPM with intra and intra prediction is shown;

[0067] Fig.51 The edges on the template are shown;

[0068] Fig.52 Example neighboring reconstructed blocks and a current prediction block are shown;

[0069] Fig.53A An example reference template outside the valid IBC reference area is shown;

[0070] Fig.53B An example showing positioning of a reference template by a BV of a corresponding BV candidate relative to a current block instead of a current template is shown;

[0071] Fig.54 A flowchart of a method for video processing according to some embodiments of the present disclosure is shown;

[0072] Fig.55 Another flow chart of a method for video processing according to some embodiments of the present disclosure is shown;

[0073] Fig.56 A block diagram of a computing device is shown in which various embodiments of the present disclosure may be implemented.

[0074] In the drawings, same or similar reference numbers generally refer to same or similar elements. DETAILED DESCRIPTION

[0075] The principle of the present disclosure will now be described with reference to some embodiments. It should be understood that these embodiments are described only for the purpose of illustrating and helping those skilled in the art to understand and implement the present disclosure, and do not imply any limitation on the scope of the present disclosure. In addition to the methods described below, the disclosure described herein can also be implemented in various ways.

[0076] In the following description and claims, unless otherwise defined, all scientific and technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present disclosure belongs.

[0077] References in this disclosure to "one embodiment," "an embodiment," "an example embodiment," etc. indicate that the described embodiment may include a particular feature, structure, or characteristic, but not every embodiment must include the particular feature, structure, or characteristic. Furthermore, these phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in conjunction with an example embodiment, whether or not explicitly described, it is considered to be within the knowledge of those skilled in the art to affect such feature, structure, or characteristic in relation to other embodiments.

[0078] It should be understood that although the terms "first" and "second" etc. can be used to describe various elements, these elements should not be limited to these terms. These terms are only used to distinguish one element from another element. For example, a first element can be referred to as a second element, and similarly, a second element can be referred to as a first element without departing from the scope of the exemplary embodiments. As used herein, the term "and / or" includes any and all combinations of one or more of the listed terms.

[0079] The terms used herein are only used for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments. As used herein, the singular forms "a", "an", and "the" are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms "including", "comprising", and / or "having" when used herein indicate the presence of the features, elements, and / or components, etc., but do not exclude the presence or addition of one or more other features, elements, components, and / or combinations thereof. Example Environment

[0080] Figure 11 is a block diagram illustrating an example video codec system 100 that may utilize the techniques of the present disclosure. As shown, the video codec system 100 may include a source device 110 and a destination device 120. The source device 110 may also be referred to as a video encoding device, and the destination device 120 may also be referred to as a video decoding device. In operation, the source device 110 may be configured to generate encoded video data, and the destination device 120 may be configured to decode the encoded video data generated by the source device 110. The source device 110 may include a video source 112, a video encoder 114, and an input / output (I / O) interface 116.

[0081] Video source 112 may include a source such as a video capture device. Examples of a video capture device include, but are not limited to, an interface that receives video data from a video content provider, a computer graphics system for generating video data, and / or combinations thereof.

[0082] The video data may include one or more pictures. The video encoder 114 encodes the video data from the video source 112 to generate a bitstream. The bitstream may include a bit sequence that forms a coded representation of the video data. The bitstream may include coded pictures and associated data. The coded pictures are coded representations of pictures. The associated data may include sequence parameter sets, picture parameter sets, and other grammatical structures. The I / O interface 116 may include a modulator / demodulator and / or a transmitter. The coded video data may be directly transmitted to the destination device 120 via the network 130A via the I / O interface 116. The coded video data may also be stored on a storage medium / server 130B for access by the destination device 120.

[0083] The destination device 120 may include an I / O interface 126, a video decoder 124, and a display device 122. The I / O interface 126 may include a receiver and / or a modem. The I / O interface 126 may obtain encoded video data from the source device 110 or the storage medium / server 130B. The video decoder 124 may decode the encoded video data. The display device 122 may display the decoded video data to the user. The display device 122 may be integrated with the destination device 120, or may be outside the destination device 120, which is configured to be connected to an external display device interface.

[0084] The video encoder 114 and the video decoder 124 may operate according to a video compression standard, such as the High Efficiency Video Codec (HEVC) standard, the Versatile Video Codec (VVC) standard, and other existing and / or further standards.

[0085] Figure 2is a block diagram showing an example of a video encoder 200 according to some embodiments of the present disclosure, which may be Figure 1 An example of a video encoder 114 in the system 100 is shown.

[0086] Video encoder 200 may be configured to implement any or all of the techniques of this disclosure. Figure 2 In the example of , video encoder 200 includes multiple functional components. The techniques described in this disclosure can be shared between the various components of video encoder 200. In some examples, a processor can be configured to perform any or all of the techniques described in this disclosure.

[0087] In some embodiments, the video encoder 200 may include a partitioning unit 201, a prediction unit 202, a residual generation unit 207, a transformation unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transformation unit 211, a reconstruction unit 212, a buffer 213 and an entropy coding unit 214, and the prediction unit 202 may include a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205 and an intra-frame prediction unit 206.

[0088] In other examples, the video encoder 200 may include more, fewer, or different functional components. In one example, the prediction unit 202 may include an intra-block copy (IBC) unit. The IBC unit may perform prediction in an IBC mode, where at least one reference picture is a picture where the current video block is located.

[0089] Furthermore, although some components (such as the motion estimation unit 204 and the motion compensation unit 205) may be integrated, for the purpose of explanation, these components are described in detail below. Figure 2 are shown separately in the example.

[0090] The partitioning unit 201 may partition a picture into one or more video blocks. The video encoder 200 and the video decoder 300 may support various video block sizes.

[0091] The mode selection unit 203 may select one of a plurality of coding modes (intra-coding or inter-coding) based on the error result, for example, and provide the generated intra-coding block or inter-coding block to the residual generation unit 207 to generate residual block data, and to the reconstruction unit 212 to reconstruct the coding block for use as a reference picture. In some examples, the mode selection unit 203 may select a combination of intra- and inter-prediction (CIIP) modes, where the prediction is based on an inter-prediction signal and an intra-prediction signal. In the case of inter-prediction, the mode selection unit 203 may also select a resolution for the motion vector (e.g., sub-pixel precision or integer pixel precision) for the block.

[0092] To perform inter-frame prediction on the current video block, the motion estimation unit 204 may generate motion information for the current video block by comparing the current video block with one or more reference frames from the buffer 213. The motion compensation unit 205 may determine a predicted video block for the current video block based on the motion information and decoded samples of pictures from the buffer 213 other than the picture associated with the current video block.

[0093] The motion estimation unit 204 and the motion compensation unit 205 may perform different operations on the current video block, for example, depending on whether the current video block is in an I slice, a P slice, or a B slice. As used herein, an "I slice" may refer to a portion of a picture consisting of macroblocks, all of which are based on macroblocks within the same picture. Furthermore, as used herein, in some aspects, a "P slice" and a "B slice" may refer to a portion of a picture consisting of macroblocks that are independent of macroblocks in the same picture.

[0094] In some examples, the motion estimation unit 204 may perform unidirectional prediction on the current video block, and the motion estimation unit 204 may search the reference pictures of list 0 or list 1 to find the reference video block for the current video block. The motion estimation unit 204 may then generate a reference index and a motion vector, the reference index indicating the reference picture in list 0 or list 1 containing the reference video block, and the motion vector indicating the spatial displacement between the current video block and the reference video block. The motion estimation unit 204 may output the reference index, the prediction direction indicator, and the motion vector as the motion information of the current video block. The motion compensation unit 205 may generate a predicted video block for the current video block based on the reference video block indicated by the motion information of the current video block.

[0095] Alternatively, in other examples, the motion estimation unit 204 may perform bidirectional prediction on the current video block. The motion estimation unit 204 may search the reference pictures in list 0 for a reference video block for the current video block, and may also search the reference pictures in list 1 for another reference video block for the current video block. The motion estimation unit 204 may then generate a plurality of reference indexes and a plurality of motion vectors, the plurality of reference indexes indicating a plurality of reference pictures in list 0 and list 1 containing a plurality of reference video blocks, and the plurality of motion vectors indicating a plurality of spatial displacements between the plurality of reference video blocks and the current video block. The motion estimation unit 204 may output the plurality of reference indexes and the plurality of motion vectors of the current video block as motion information of the current video block. The motion compensation unit 205 may generate a predicted video block for the current video block based on the plurality of reference video blocks indicated by the motion information of the current video block.

[0096] In some examples, motion estimation unit 204 may output a complete set of motion information for use in a decoding process by a decoder. Alternatively, in some embodiments, motion estimation unit 204 may signal motion information of a current video block with reference to motion information of another video block. For example, motion estimation unit 204 may determine that motion information of a current video block is sufficiently similar to motion information of a neighboring video block.

[0097] In one example, motion estimation unit 204 may indicate to video decoder 300 a value in a syntax structure associated with the current video block that indicates that the current video block has the same motion information as another video block.

[0098] In another example, the motion estimation unit 204 may identify another video block and a motion vector difference (MVD) in a syntax structure associated with the current video block. The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the indicated video block. The video decoder 300 may use the motion vector of the indicated video block and the motion vector difference to determine the motion vector of the current video block.

[0099] As discussed above, the video encoder 200 may signal motion vectors in a predictive manner.Two examples of prediction signaling techniques that may be implemented by the video encoder 200 include Advanced Motion Vector Prediction (AMVP) and merge mode signaling.

[0100] The intra prediction unit 206 may perform intra prediction on the current video block. When the intra prediction unit 206 performs intra prediction on the current video block, the intra prediction unit 206 may generate prediction data for the current video block based on decoded samples of other video blocks in the same picture. The prediction data for the current video block may include a prediction video block and various syntax elements.

[0101] The residual generation unit 207 may generate residual data for the current video block by subtracting (e.g., indicated by a minus sign) the prediction video block(s) of the current video block from the current video block. The residual data of the current video block may include residual video blocks corresponding to different sample portions of samples in the current video block.

[0102] In other examples, such as in skip mode, there may be no residual data for the current video block, and the residual generation unit 207 may not perform a subtraction operation.

[0103] Transform processing unit 208 may generate one or more transform coefficient video blocks for a current video block by applying one or more transforms to the residual video block associated with the current video block.

[0104] After transform processing unit 208 generates a transform coefficient video block associated with the current video block, quantization unit 209 may quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values ​​associated with the current video block.

[0105] The inverse quantization unit 210 and the inverse transform unit 211 may apply inverse quantization and inverse transform to the transform coefficient video block, respectively, to reconstruct a residual video block from the transform coefficient video block. The reconstruction unit 212 may add the reconstructed residual video block to corresponding samples from one or more prediction video blocks generated by the prediction unit 202 to generate a reconstructed video block associated with the current video block for storage in the buffer 213.

[0106] After reconstruction unit 212 reconstructs the video block, a loop filtering operation may be performed to reduce video blocking artifacts in the video block.

[0107] The entropy coding unit 214 may receive data from other functional components of the video encoder 200. When the entropy coding unit 214 receives the data, the entropy coding unit 214 may perform one or more entropy coding operations to generate entropy coded data and output a bitstream including the entropy coded data.

[0108] Figure 3 is a block diagram showing an example of a video decoder 300 according to some embodiments of the present disclosure, which may be Figure 1 An example of a video decoder 124 in the system 100 is shown.

[0109] Video decoder 300 may be configured to perform any or all of the techniques of this disclosure. Figure 3 In the example of , video decoder 300 includes multiple functional components. The techniques described in this disclosure can be shared between the various components of video decoder 300. In some examples, a processor can be configured to perform any or all of the techniques described in this disclosure.

[0110] exist Figure 3 In the example of , the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, and a reconstruction unit 306 and a buffer 307. In some examples, the video decoder 300 can perform a decoding process that is generally opposite to the encoding process described with respect to the video encoder 200.

[0111] The entropy decoding unit 301 can retrieve the encoded bitstream. The encoded bitstream may include entropy encoded video data (e.g., encoded video data blocks). The entropy decoding unit 301 can decode the entropy encoded video data, and the motion compensation unit 302 can determine motion information from the entropy decoded video data, the motion information including motion vectors, motion vector precision, reference picture list indexes, and other motion information. The motion compensation unit 302 can determine the information, for example, by performing AMVP and merge mode. AMVP is used, including deriving several most likely candidates based on data and reference pictures of neighboring PBs. The motion information typically includes horizontal and vertical motion vector displacement values, one or two reference picture indexes, and in the case of prediction areas in B strips, also includes an identification of which reference picture list is associated with each index. As used herein, in some aspects, "merge mode" may refer to deriving motion information from blocks that are adjacent in the spatial or temporal domain.

[0112] The motion compensation unit 302 may generate a motion compensated block, possibly performing interpolation based on interpolation filtering.An identifier for the interpolation filter used with sub-pixel precision may be included in the syntax element.

[0113] The motion compensation unit 302 may calculate interpolated values ​​for sub-integer pixels of a reference block using interpolation filtering used by the video encoder 200 during encoding of the video block. The motion compensation unit 302 may determine the interpolation filtering used by the video encoder 200 based on received syntax information, and the motion compensation unit 302 may use the interpolation filtering to generate a prediction block.

[0114] The motion compensation unit 302 may use at least part of the syntax information to determine the size of blocks used to encode (multiple) frames and / or (multiple) slices of the encoded video sequence, partition information describing how each macroblock of a picture of the encoded video sequence is partitioned, a mode indicating how each partition is encoded, one or more reference frames (and reference frame lists) for each inter-coded block, and other information for decoding the encoded video sequence. As used herein, in some aspects, a "slice" may refer to a data structure that can be decoded independently of other slices of the same picture in terms of entropy coding, signal prediction, and residual signal reconstruction. A slice may be an entire picture, or it may be a region of a picture.

[0115] The intra prediction unit 303 may use, for example, an intra prediction mode received in the bitstream to form a prediction block from spatially neighboring blocks. The inverse quantization unit 304 inverse quantizes (i.e., dequantizes) the quantized video block coefficients provided in the bitstream and decoded by the entropy decoding unit 301. The inverse transform unit 305 applies an inverse transform.

[0116] The reconstruction unit 306 may obtain the decoded block, for example, by adding the residual block to the corresponding prediction block generated by the motion compensation unit 302 or the intra prediction unit 303. If necessary, a deblocking filter may also be applied to filter the decoded block in order to remove blocking artifacts. The decoded video block is then stored in a buffer 307, which provides reference blocks for subsequent motion compensation / intra prediction, and the buffer 307 also generates the decoded video for presentation on a display device.

[0117] Some exemplary embodiments of the present disclosure will be described in detail below. It should be noted that the section titles used in this document are for ease of understanding, rather than limiting the embodiments disclosed in the section to that section. In addition, although some embodiments are described with reference to general video codecs or other specific video codecs, the disclosed technology is also applicable to other video codec technologies. In addition, although some embodiments describe the video encoding steps in detail, it should be understood that the corresponding decoding steps of the unencoding will be implemented by the decoder. In addition, the term video processing includes video encoding or compression, video decoding or decompression, and video transcoding, in which video pixels are represented from one compression format to another compression format or at different compression bit rates. 1. Brief Overview The present disclosure relates to video coding technology, and more specifically, to IBC prediction and related technologies in video coding, which can be applied to existing HEVC, VVC and other video coding standards, and can also be applied to future video coding standards or video codecs. 2. Introduction Video codec standards have evolved primarily through the development of the well-known ITU-T and ISO / IEC standards. ITU-T developed H.261 and H.263, ISO / IEC developed MPEG-1 and MPEG-4 Visual, and the two organizations jointly developed H.262 / MPEG-2 Video and H.264 / MPEG-4 Advanced Video Codec (AVC) and H.265 / HEVC standards. Since H.262, video codec standards have been based on a hybrid video codec structure, in which temporal prediction plus transform codec is used. In order to explore future video codec technologies beyond HEVC, VCEG and MPEG jointly founded the Joint Video Exploration Team (JVET) in 2015. JVET meetings are held simultaneously every quarter, and the new video codec standard was officially named Versatile Video Codec (VVC) at the JVET meeting in April 2018, when the first version of the VVC Test Model (VTM) was released. The VVC working draft and the test model VTM are updated after each meeting. The VVC project achieved technical completion (FDIS) at the July 2020 meeting. 2.1. Extended Merge Prediction In VVC, the merge candidate list is constructed by including the following five types of candidates in order: 1) Airspace MVP from airspace neighbor CU 2) Time domain MVP from the same CU 3) History-based MVP from FIFO table 4)4) Paired Average MVP 5) Zero MV. The size of the merge list is signaled in the sequence parameter set header, and the maximum allowed size of the merge list is 6. For each CU code in merge mode, the index of the best merge candidate is encoded using truncated unary binarization (TU). The first bin of the merge index is encoded using context, while bypass coding is used for other bins. The derivation process of merge candidates for each category is provided in this section. As in HEVC, VVC also supports parallel derivation of merge candidate lists for all CUs within a certain size area. 2.1.1. Spatial Candidate Derivation The derivation of spatial merge candidates in VVC is the same as that in HEVC, except that the positions of the first two merge candidates are swapped. Figure 4 An example diagram 400 showing example locations of spatial merge candidates is shown. Figure 4 Among the candidates at the positions shown, up to four merge candidates are selected. The derivation order is B1, A1B0, A0, and B2. Position B2 is considered only when one or more CUs at positions B0, A0, B1, and A1 are not available (for example, because it belongs to another slice or tile) or is intra-coded. After the candidate at position A1 is added, a redundancy check is performed on the addition of the remaining candidates, which ensures that candidates with the same motion information are excluded from the list, thereby improving coding efficiency. Figure 5 An example diagram 500 showing candidate pairs considered for redundancy check of spatial merge candidates is shown. In order to reduce computational complexity, not all possible candidate pairs are considered in the redundancy check mentioned. Instead, only Figure 5 The pairs linked with arrows in , and candidates are added to the list only if the corresponding candidates for redundancy check do not have the same motion information. 2.2.2. Time Domain Candidate Derivation In this step, only one candidate is added to the list. In particular, in the derivation of this temporal merge candidate, the scaled motion vector is derived based on the co-located CU belonging to the co-located reference picture. The reference picture list to be used for deriving the co-located CU is explicitly signaled in the slice header. Figure 6An example diagram 600 showing example motion vector scaling of temporal merging candidates is shown. Figure 6 As shown by the dotted line in , the scaled motion vector of the temporal merge candidate is obtained, which is scaled from the motion vector of the co-located CU using the POC distances tb and td, where tb is defined as the POC difference between the reference picture of the current picture and the current picture, and td is defined as the POC difference between the reference picture of the co-located picture and the co-located picture. The reference picture index of the temporal merge candidate is set equal to zero. Figure 7 An example diagram 700 showing candidate positions of temporal merging candidates C0 and C1 is shown. The position of the temporal candidate is selected between candidates C0 and C1, such as Figure 7 If the CU at position C0 is not available, intra-coded, or outside the current row of the CTU, position C1 is used. Otherwise, position C0 is used in the derivation of the temporal merge candidate. 2.1.3. History-based Merge Candidate Derivation History-based MVP (HMVP) merge candidates are added to the merge list after spatial MVP and TMVP. In this method, the motion information of the previously coded blocks is stored in a table and used as the MVP of the current CU. A table with multiple HMVP candidates is maintained during the encoding / decoding process. The table is reset (cleared) when a new CTU row is encountered. Whenever there is a non-sub-block inter-coded CU, the associated motion information is added to the last entry of the table as a new HMVP candidate. The HMVP table size S is set to 6, which indicates that up to 6 history-based MVP (HMVP) candidates can be added to the table. When a new motion candidate is inserted into the table, a constrained first-in, first-out (FIFO) rule is used, where a redundancy check is first applied to find out whether the same HMVP exists in the table. If found, the same HMVP is removed from the table and then all HMVP candidates are moved forward. HMVP candidates can be used in the merge candidate list construction process. Check the closest HMVP candidates in the table in order and insert them into the candidate list after the TMVP candidate. Apply redundancy check to the HMVP candidates to make spatial or temporal merge candidates. To reduce the number of redundant checking operations, the following simplifications are introduced: 1. The number of HMPV candidates for merge list generation is set to (N<=4)?M:(8-N), where N indicates the number of existing candidates in the merge list and M indicates the number of available HMVP candidates in the table. 2. Once the total number of available merge candidates reaches the maximum allowed merge candidates minus 1, the merge candidate list construction process from HMVP is terminated. 2.1.4. Pairwise Average Merging Candidate Derivation The pairwise average candidate is generated by averaging the predetermined candidate pairs in the existing merge candidate list, and the predetermined pairs are defined as {(0, 1), (0, 2), (1, 2), (0, 3), (3, 1), (2, 3)}, where the number represents the merge index of the merge candidate list. The average motion vector is calculated separately for each reference list. If two motion vectors are available in one list, they are averaged even if they point to different reference pictures; if only one motion vector is available, it is used directly; if no motion vector is available, this list is kept invalid. When the merge list is not full after adding pairwise average merge candidates, zero MVPs will be inserted at the end until the maximum number of merge candidates is encountered. 2.2. New Merge Candidates 2.2.1 Non-adjacent merging candidate export Figure 8 An example diagram 800 showing VVC spatial neighboring blocks of a current block is shown. In VVC, Figure 8 The five spatial neighboring blocks and one temporal neighboring block shown are used to derive merge candidates. It is proposed to derive additional merge candidates from locations that are not adjacent to the current block using the same style as in VVC. To achieve this, for each search round i, a virtual block is generated based on the current block as follows: First, the relative position of the virtual block to the current block is calculated using the following formula: Offsetx=-i×gridX, Offsety=-i×gridY Where Offsetx and Offsetty represent the offset of the upper left corner of the virtual block relative to the lower left corner of the current block, and gridX and gridY are the width and height of the search grid. Second, the width and height of the virtual block are calculated using the following formula: newWidth=i×2×gridX+currWidth newHeight=i×2×gridY+currHeight. Where currWidth and currHeight are the width and height of the current block. newWidth and newHeight are the width and height of the new virtual block. gridX and gridY are currently set to currWidth and currHeight respectively. Fig. 9 The virtual blocks in the i-th search round are shown. Fig. 9An example diagram showing the relationship between a virtual block and a current block is shown. After generating a virtual block, blocks Ai, Bi, Ci, Di, and Ei can be regarded as VVC spatial neighboring blocks of the virtual block, and their positions are obtained using the same style in VVC. Obviously, if the search round i is 0, the virtual block is the current block. In this case, blocks Ai, Bi, Ci, Di, and Ei are spatial neighboring blocks used in VVC merge mode. When constructing the merge candidate list, pruning is performed to ensure that each element in the merge candidate list is unique. The maximum search round is set to 1, which means that five non-adjacent spatial neighboring blocks are used. Non-adjacent spatial merge candidates are inserted into the merge list after the temporal merge candidates in the order of B1->A1->C1->D1->E1. 2.2.2 Non-adjacent airspace candidates Non-adjacent spatial merging candidates are inserted after the TMVP in the regular merging candidate list. Fig.10 An example diagram 1000 showing spatial neighboring blocks used to derive spatial merge candidates is shown. The pattern of spatial merge candidates is as follows: Fig.10 As shown in Figure 2. The distance between non-adjacent spatial candidates and the current codec block is based on the width and height of the current codec block. The line buffer limit is not applied. 2.2.3STMVP It is proposed to use three spatial merging candidates and one temporal merging candidate to derive an average candidate as the STMVP candidate. The STMVP is inserted before the upper left spatial merge candidate. The STMVP candidate is pruned along with all previous merge candidates in the merge list. For spatial domain candidates, the first three candidates in the current merged candidate list are used. For the time domain candidates, the same positions as the VTM / HEVC co-location positions are used. For spatial candidates, the first, second and third candidates inserted into the current merge candidate list before STMVP are denoted as F, S and T. The temporal candidate having the same position as the VTM / HEVC co-location position used in TMVP is denoted Col. The motion vector of the STMVP candidate in prediction direction X (denoted as mvLX) is derived as follows: 1) If the reference indices of the four merge candidates are all valid and equal to zero in the prediction direction X (X=0 or 1), mvLX=(mvLX_F+mvLX_S+mvLX_T+mvLX_Col)>>2 2) If the reference indexes of three of the four merge candidates are valid and equal to zero in the prediction direction X (X=0 or 1), mvLX=(mvLX_F×3+mvLX_S×3+mvLX_Col×2)>>3 or mvLX=(mvLX_F×3+mvLX_T×3+mvLX_Col×2)>>3 or mvLX=(mvLX_S×3+mvLX_T×3+mvLX_Col×2)>>3 3) If the reference indexes of two of the four merge candidates are valid and equal to zero in the prediction direction X (X=0 or 1), mvLX=(mvLX_F+mvLX_Col)>>1 or mvLX=(mvLX_S+mvLX_Col)>>1 or mvLX=(mvLX_T+mvLX_Col)>>1 NOTE: If time domain candidates are not available, STMVP mode is turned off. 2.2.4 Merge List Size If both non-adjacent merge candidates and STMVP merge candidates are considered, the size of the merge list is signaled in the sequence parameter set header and the maximum allowed size of the merge list is increased (e.g., 8). 2.3. Sub-block based temporal motion vector prediction (SbTMVP) VVC supports the sub-block-based temporal motion vector prediction (SbTMVP) method. Similar to the temporal motion vector prediction (TMVP) in HEVC, SbTMVP uses the motion field in the co-located picture to improve the motion vector prediction and merge mode of the CU in the current picture. The same co-located picture used by TMVP is used for SbTMVP. SbTMVP differs from TMVP in the following two main aspects: –TMVP predicts motion at CU level, but SbTMVP predicts motion at sub-CU level; – While TMVP pre-fetches the temporal motion vector from the co-located block in the co-located picture (the co-located block is the bottom-right or center block relative to the current CU), SbTMVP applies motion shifting before pre-fetching temporal motion information from the co-located picture, where the motion shifting is obtained from the motion vector from one of the spatial neighboring blocks of the current CU. SbTMVP process Fig.11A and Fig. 11B As shown in Figure 2, SbTMVP predicts the motion vector of the sub-CU within the current CU in two steps. Fig.11A An example diagram showing the spatial neighboring blocks used by SbTMVP is shown. In the first step, check Fig.11A If A1 has a motion vector that uses the co-located picture as its reference picture, then that motion vector is selected as the motion shift to be applied. If no such motion is identified, then the motion shift is set to (0, 0). Fig. 11B It is shown that the sub-CU motion field is derived by applying the motion shift from the spatial neighbors and scaling the motion information from the corresponding co-located sub-CU. In the second step, the motion shift identified in step 1 is applied (i.e., added to the coordinates of the current block) to obtain the sub-CU level motion information (motion vector and reference index) from the co-located picture, as shown in Fig. 11B shown. Fig. 11B The example in assumes that the motion shift is set to the motion of block A1. Then, for each sub-CU, the motion information of its corresponding block in the co-located picture (the minimum motion grid covering the center sample) is used to derive the motion information of the sub-CU. After the motion information of the co-located sub-CU is identified, it is converted into the reference index and motion vector of the current sub-CU in a manner similar to the TMVP process of HEVC, where temporal motion scaling is applied to align the reference picture of the temporal motion vector with the reference picture of the current CU. In VVC, a sub-block based merge list containing a combination of both SbTMVP candidates and affine merge candidates is used for signaling based on sub-block merge mode. The SbTMVP mode is enabled / disabled by a sequence parameter set (SPS) flag. If the SbTMVP mode is enabled, the SbTMVP predictor is added as the first entry in the list of sub-block based merge candidates, followed by the affine merge candidates. The size of the sub-block based merge list is signaled in the SPS, and the maximum allowed size of the sub-block based merge list in VVC is 5. The sub-CU size used in SbTMVP is fixed to 8x8, and like the affine merge mode, the SbTMVP mode is only applicable to CUs whose width and height are both greater than or equal to 8. The encoding logic of the additional SbTMVP merge candidate is the same as that of other merge candidates, that is, for each CU in a P or B slice, an additional RD check is performed to decide whether to use the SbTMVP candidate. 2.4. Intra-block copy (IBC) Intra-block copying (IBC) is a tool used by HEVC to extend SCC. As we all know, IBC significantly improves the encoding and decoding efficiency of screen content materials. Since the IBC mode is implemented as a block-level codec mode, block matching (BM) is performed on the encoder to find the best block vector (or motion vector) for each CU. Here, the block vector is used to indicate the displacement from the current block to the reference block, which has been reconstructed within the current picture. The luminance block vector of the CU encoded and decoded by IBC is integer precision. The chrominance block vector can also be rounded to integer precision. When combined with AMVR, the IBC mode can switch between 1-pixel and 4-pixel motion vector precision. The CU encoded and decoded by IBC is regarded as a third prediction mode other than intra-frame prediction mode or inter-frame prediction mode. The IBC mode is applicable to CUs whose width and height are both less than or equal to 64 luminance samples. At the encoder side, hash-based motion estimation is performed for IBC. The encoder performs RD checks for blocks with width or height no greater than 16 luma samples. For non-merge mode, a block vector search is first performed using a hash-based search. If the hash search does not return a valid candidate, a local search based on block matching is performed. In hash-based search, the hash key matching (32-bit CRC) between the current block and the reference block is extended to all allowed block sizes. The hash key calculation for each position in the current picture is based on 4x4 sub-blocks. For larger current blocks, when all hash keys of all 4×4 sub-blocks match the hash keys of the corresponding reference positions, the hash key is determined to match the hash key of the reference block. If the hash keys of multiple reference blocks are found to match the hash key of the current block, the block vector cost of each matched reference is calculated, and the one with the smallest cost is selected. In the block matching search, the search range is set to cover the previous CTU and the current CTU. At the CU level, the IBC mode is signaled using a flag, which can be signaled as either IBC AMVP mode or IBC Skip / Merge mode as shown below: – IBC skip / merge mode: The merge candidate index is used to indicate which block vectors from the neighboring candidate IBC codec block list are used to predict the current block. The merge list consists of spatial, HMVP and pairwise candidates. – IBC AMVP mode: Block vector differences are coded the same way as motion vector differences. The block vector prediction method uses two candidates as predictors, one from the left neighbor and one from the top neighbor (if IBC coded). When either neighbor is not available, the default block vector will be used as the predictor. A flag is signaled to indicate the block vector predictor index. 2.4.1 Simplification of IBC vector prediction The BV predictors for merge mode and AMVP mode in IBC will share a common predictor list, which consists of the following elements: · Two adjacent airspace locations (such as Figure 4 A0, B0) shown, 5 HMVP entries, The default is the zero vector. For merge mode, the first 6 entries of this list are used at most; for AMVP mode, the first 2 entries of this list will be used. And this list meets the requirements of shared merge list area (sharing the same list in SMR). 2.4.2 IBC Reference Area In order to reduce memory consumption and decoder complexity, IBC in VVC only allows the reconstruction of a predetermined area, which includes the area of ​​the current CTU and some areas of the left CTU. Figures 12A-12D Example diagrams 1210 , 1230 , 1250 , and 1270 showing the current CTU processing order and available samples in the current CTU and the left CTU are shown. Figures 12A-12D The reference area of ​​the IBC mode is shown, where each block represents a 64x64 luma sample unit. Depending on the location of the current codec CU position within the current CTU, the following applies: – If the current block falls into the upper left 64x64 block of the current CTU, then in addition to the reconstructed samples in the current CTU, the CPR mode can also be used to refer to the reference samples in the lower right 64x64 block of the left CTU. Using the CPR mode, the current block can also refer to the reference samples in the lower left 64x64 block of the left CTU and the reference samples in the upper right 64x64 block of the left CTU. – Using CPR mode, if the current block falls into the upper right 64x64 block of the current CTU, in addition to the samples already reconstructed in the current CTU, if the luma position (0,64) relative to the current CTU has not been reconstructed, the current block can also refer to the reference samples in the lower left 64x64 block and the lower right 64x64 block of the left CTU; otherwise, the current block can also refer to the reference samples in the lower right 64x64 block of the left CTU. – Using CPR mode, if the current block falls into the lower left 64x64 block of the current CTU, then in addition to the samples that have been reconstructed in the current CTU, if the luma position (64,0) relative to the current CTU has not been reconstructed, the current block can also refer to the reference samples in the upper right 64x64 block and the lower right 64x64 block of the left CTU. Otherwise, using CPR mode, the current block can also refer to the reference samples in the lower right 64x64 block of the left CTU. – If the current block falls into the lower right 64x64 block of the current CTU, only the CPR mode can be used to refer to the already reconstructed samples in the current CTU. This restriction allows the IBC mode to be implemented using local on-chip memory for hardware implementation. 2.4.3 Interaction between IBC and other codecs The interaction between IBC mode and other inter-frame codec tools in VVC, such as paired merge candidates, history-based motion vector predictor (HMVP), combined intra / inter prediction mode (CIIP), merge mode with motion vector difference (MMVD) and geometric partition mode (GPM), is as follows: – IBC can be used with pairwise merge candidates and HMVP. A new pairwise IBC merge candidate can be generated by averaging two IBC merge candidates. For HMVP, IBC motion is inserted into the history buffer for future reference. – IBC cannot be used in conjunction with the following interframe tools: Affine Motion, CIIP, MMVD, and GPM. – When DUAL_TREE partitioning is used, IBC is not allowed for the chroma codec block. Unlike the HEVC screen content codec extension, for IBC prediction, the current picture is no longer included as one of the reference pictures in reference picture list 0. The derivation process of motion vectors for IBC mode excludes all neighboring blocks in inter mode and vice versa. The following IBC design aspects are applied: – IBC shares the same process as regular MV merging, including pairwise merging candidates and history-based motion predictors, but TMVP and zero vectors are not allowed as they are invalid for IBC mode. – Separate HMVP buffers for regular MV and IBC (5 candidates each). – Block vector constraints are implemented as bitstream consistency constraints. The encoder needs to ensure that there are no invalid vectors in the bitstream. If the merge candidate is invalid (out of range or 0), the merge must not be used. This bitstream consistency constraint is expressed in the form of a virtual buffer as described below. – For deblocking, IBC is handled as inter mode. - If the current block is encoded using IBC prediction mode, AMVR does not use quarter pixels; instead, AMVR is signaled to only indicate whether the MV is an inter pixel or 4 integer pixels. - The number of IBC merge candidates may be signaled separately from the number of normal, sub-block and geometric merge candidates in the slice header. The concept of virtual buffer is used to describe the allowed reference area for IBC prediction mode and valid block vectors. ctbSize represents the CTU size, and the width of the virtual buffer ibcBuf is wIbcBuf=128x128 / ctbSize, and the height is hIbcBuf=ctbSize. For example, for a CTU size of 128x128, the size of ibcBuf is also 128x128; for a CTU size of 64x64, the size of ibcBuf is 256x64; for a CTU size of 32x32, the size of ibcBuf is 512x32. The size of the VPDU is min(ctbSize, 64) in each dimension, Wv=min(ctbSize, 64). The virtual IBC buffer, ibcBuf is maintained as follows. – At the start of decoding each CTU line, flush the entire ibcBuf with an invalid value of -1. – When starting to decode the VPDU (xVPDU, yVPDU) corresponding to the upper left corner of the picture, set ibcBuf[x][y] = -1, where x = xVPDU% wIbcBuf, ..., xVPDU% wIbcBuf+W v -1;y=yVPDU%ctbSize,…,yVPDU%ctbSize+W v -1. – After decoding the CU containing (x, y) relative to the upper left corner of the picture, set –ibcBuf[x%wIbcBuf][y%ctbSize]=recSample[x][y]. For a block covering coordinates (x, y), the block is valid if the following is true for the block vector bv = (bv[0], bv[1]); otherwise the block is invalid: ibcBuf[(x+bv[0])%wIbcBuf][(y+bv[1])%ctbSize] should not be equal to -1. 2.4.4IBC Virtual Buffer Test The luma block vector bvL (a luma block vector with 1 / 16 fractional sampling accuracy) shall comply with the following constraints: –CtbSizeY is greater than or equal to ((yCb+(bvL[1]>>4))&(CtbSizeY-1))+cbHeight. – IbcVirBuf[0][(x+(bvL[0]>>4))&(IbcBufWidthY-1)][(y+(bvL[1]>>4))&(CtbSizeY-1)] is not equal to -1 for x=xCb..xCb+cbWidth-1 and y=yCb..yCb+cbHeight-1. Otherwise, bvL is considered as an invalid bv. Samples are processed in CTB units. The array size for each luma CTB is CtbSizeY in samples in both width and height. – (xCb, yCb) is the luminance position of the upper left sample of the current luminance codec block relative to the upper left luminance sample of the current picture, –cbWidth specifies the width of the current codec block in luma samples. –cbHeight specifies the height of the current codec block in luma samples. 2.5. Adaptive Merging Candidate Re-ranking Based on Template Matching In order to improve the encoding and decoding efficiency, after the merge candidate list is constructed, the order of each merge candidate is adjusted according to the template matching cost. The merge candidates are arranged in ascending order according to the template matching cost in the list and operated in the form of subgroups. Fig.13 An example graph 1300 showing neighboring samples used to calculate SAD is shown. Fig.14 An example diagram 1400 showing neighboring samples used to calculate the SAD of sub-CU level motion information is shown. The template matching cost is measured by the SAD (sum of absolute differences) between the neighboring samples of the current CU and its corresponding reference samples. Fig.13 As shown, if the merge candidate contains dual prediction motion information, the corresponding reference sample is the average of the corresponding reference sample in reference list 0 and the corresponding reference sample in reference list 1. Fig.14 As shown, if the merge candidate contains sub-CU level motion information, the corresponding reference samples are composed of adjacent samples of the corresponding reference sub-block. Fig.15 A schematic diagram 1500 of the classification process is shown. Fig.15 As shown, the classification process operates in the form of subgroups. The first three merge candidates are classified together. The following three merge candidates are classified together. The template size (width of the left template or height of the upper template) is 1. The subgroup size is 3. 2.6. Adaptive Merge Candidate List We can assume that the number of merge candidates is 8. We take the first 5 merge candidates as the first subgroup and the last 3 merge candidates as the second subgroup (i.e., the last subgroup). Fig.16 An example diagram 1600 showing the reordering process in an encoder is shown. For the encoder, after building the merge candidate list at block 1602, some merge candidates are adaptively reordered in ascending order of merge candidate cost, such as Fig.16 shown. More specifically, in box 1604, the template matching costs of the merge candidates in all subgroups except the last subgroup are calculated; then in box 1606, the merge candidates in its own subgroup except the last subgroup are reordered; finally, in box 1608, a final merge candidate list is obtained. Fig.17 An example diagram 1700 showing the reordering process in a decoder is shown. For the decoder, after building the merge candidate list, some / no merge candidates are adaptively reordered in ascending order of merge candidate cost, such as Fig.17 As shown. Fig.17 In , the subgroup where the selected (signal) merging candidate is located is called the selected subgroup. More specifically, at block 1702, if the selected merge candidate is located in the last subgroup, at block 1704, after the selected merge candidate is derived, the merge candidate list construction process is terminated, at block 1706, reordering is not performed and the merge candidate list is not changed; otherwise, the execution process is as follows: In box 1708, after all merge candidates in the selected subgroup are derived, the merge candidate list construction process is terminated; in box 1710, the template matching cost for the merge candidates in the selected subgroup is calculated; in box 1712, the merge candidates in the selected subgroup are reordered; finally, in box 1714, a new merge candidate list is obtained. For encoders and decoders: The template matching cost is derived as a function of T and RT, where T is the set of samples in the template and RT is the set of reference samples for the template. When deriving the reference samples of the template of the merge candidate, the motion vector of the merge candidate is rounded to integer pixel precision. It can also be derived using 8-tap or 12-tap luma interpolation filtering. The reference samples of the template (RT) for bidirectional prediction are derived by weighted averaging the reference samples of the template in reference list 0 (RT0) and the reference samples of the template in reference list 1 (RT1) as follows: RT=((8-w)*RT0+w*RT1+4)>>3 The weight of the reference template in reference list 0 (8-w) and the weight of the reference template in reference list 1 (w) are determined by the BCW index of the merge candidate. BCW indexes equal to {0, 1, 2, 3, 4} correspond to w equal to {-2, 3, 4, 5, 10} respectively. If the local illumination compensation (LIC) flag of the merge candidate is true, the reference samples of the template are derived using the LIC method. The template matching cost is calculated based on the sum of absolute differences (SAD) between T and RT. The template size is 1. This means that the width of the left template and / or the height of the upper template is 1. If the codec mode is MMVD, the merge candidates used to derive the base merge candidate are not reordered. If the coding mode is GPM, the merge candidates used to derive the unidirectional prediction candidate list are not reordered. 2.7. Template Matching (TM) Template Matching (TM) is a decoder-side MV derivation method used to refine the motion information of the current CU by finding the closest match between a template in the current picture (i.e., the top and / or left neighboring blocks of the current CU) and a block in a reference picture (i.e., the same size as the template). Fig.18 An example diagram 1800 showing template matching performed on a search area around an initial MV is shown. Fig.18 As shown, in the [-8, +8] pixel search range, a better MV is searched around the initial motion of the current CU. This paper adopts template matching with two modifications: the search step size is determined based on the AMVR mode, and the TM can cascade the bilateral matching process in the merge mode. In AMVP mode, MVP candidates are determined based on template matching error to select the one that achieves the minimum difference between the current block template and the reference block template, and then TM performs MV refinement only on this specific MVP candidate. TM refines this MVP candidate by using an iterative diamond search, starting with full-pixel MVD accuracy (or 4 pixels for 4-pixel AMVR mode) within the [-8, +8] pixel search range. The AMVP candidate can be further refined by using a cross search with full-pixel MVD accuracy (or 4 pixels for 4-pixel AMVR mode), and then using half-pixels and quarter-pixels in turn according to the AMVR mode specified in Table 3. This search process ensures that the MVP candidate still maintains the same MV accuracy as indicated by the AMVR mode after the TM process. Table 1. Search patterns of AMVR and merge patterns with AMVR In merge mode, a similar search method is applied to the merge candidates indicated by the merge index. As shown in Table 1, TM can be performed all the way down to 1 / 8 pixel MVD accuracy, or skip accuracy beyond half-pixel MVD accuracy, depending on whether alternative interpolation filtering is used based on the merged motion information (used when AMVR is half-pixel mode). In addition, when TM mode is enabled, template matching can work as an independent process between block-based and sub-block-based bilateral matching (BM) methods or as an additional MV refinement process, depending on whether BM can be enabled according to its enabling condition check. At the encoder side, the TM merge mode will perform MV refinement on each merge candidate. 2.8. Intra-frame template matching Template Matching Prediction (TMP) is a special intra prediction mode that copies the best prediction block from the reconstructed part of the current frame, and an L-shaped template of the current frame is matched with the current template. Fig.19 An example diagram 1900 showing template matching prediction is shown. For a predetermined search range, the encoder searches for the template most similar to the current template in the reconstructed part of the current frame and uses the corresponding block as the prediction block. The encoder then signals the use of this mode and the reverse operation is performed on the decoder side. Template matching prediction is a coding tool mainly applicable to screen content coding and decoding. A prediction signal is generated at the decoder side by matching the L-shaped causal neighbors of the current block with another block in a predetermined search area. Fig. 20 An example diagram 2000 showing the intra-frame template matching search area used is shown. Specifically, the search range is divided into 3 areas: R1: In the current CTU, R2: upper left corner outside the current CTU, R3: Above the current CTU, R4: On the left side of the current CTU. In each region, the decoder searches for the template with the least SAD relative to the current region and uses its corresponding block as the prediction block. The sizes of all regions (SearchRange_w, SearchRange_h) are set proportionally to the block size (BlkW, BlkH) so that there is a fixed number of SAD comparisons per pixel. That is: SearchRange_w=a*BlkW SearchRange_h=a*BlkH where "a" is a constant that controls the gain / complexity tradeoff. 2.9. Template-based MPM Intra Mode Export Fig.21An example diagram 2100 showing the templates used in TIMD and their reference samples is shown. The TIMD mode is derived from the MPM using neighboring templates. The TIMD mode is used as an additional intra prediction method for a CU. Fig.21 As shown, for each candidate mode, the template and its reference samples are used to generate the prediction samples of the template. The cost is calculated as the sum of absolute transform differences (SATD) between the prediction samples and the reconstructed samples of the template. The intra prediction mode with the minimum cost is selected as the TIMD mode and used for intra prediction of the CU. 2.9.1 TIMD Mode Export For each intra prediction mode in the MPM, the SATD between the prediction and reconstructed samples of the template is calculated. The intra prediction mode with the smallest SATD is selected as the TIMD mode for intra prediction of the current CU. The derivation of TIMD mode supports position-dependent intra prediction combination (PDPC) and gradient PDPC. 2.9.2TIMD Signaling A flag is signaled in the sequence parameter set (SPS) to enable / disable the proposed method. When the flag is true, a CU level flag is issued to indicate whether the proposed TIMD method is used. The TIMD flag is signaled immediately after the MIP flag. If the TIMD flag is equal to true, the rest of the syntax elements related to the luma intra prediction mode are skipped. 2.9.3 Interaction with new codecs in ECM-1.0 When the DIMD flag is equal to true, the TIMD flag is not signaled and is set to false. TIMD is allowed to be combined with ISP and MRL. When TIMD is combined with ISP or MRL and the TIMD flag is equal to true, the derived TIMD mode is used as the intra prediction mode for ISP or MRL. When the secondary MPM is enabled, both the primary MPM and the secondary MPM are used to derive the TIMD mode. The 6-tap interpolation filter is not used in the export of TIMD mode. 2.9.4 Modification of MPM table construction in TIMD mode derivation In the process of constructing the MPM list, when neighboring blocks are inter-encoded, their intra prediction modes are derived as planes. In order to improve the accuracy of the MPM list, when neighboring blocks are inter-encoded, the propagated intra prediction modes are derived using motion vectors and reference pictures and used for constructing the MPM list. 2.10. Adaptive Merge Candidate List In the following, a template is a set of reconstructed samples that are adjacent or non-adjacent to the current block. The reference samples of the template are derived according to the same motion information of the current block. For example, the reference samples of the template are a mapping of the template that depends on the motion information. In this case, the reference samples of the template are located by the motion vector of the motion information in the reference picture indicated by the reference index of the motion information. Fig. 22 An example diagram 2200 showing templates and reference samples of the templates is shown. Fig. 22 Examples are shown, where RT represents the reference sample of the current template T. When the merge candidate utilizes bidirectional prediction, the reference sample of the template of the merge candidate is represented by RT, and RT can be generated according to RT0 derived from the reference picture in reference picture list 0 and RT1 derived from the reference picture in reference picture list 1. In one example, RT0 includes a set of reference samples on the reference picture of the current block indicated by the reference index of the merge candidate that references the reference picture in reference list 0, where the MV of the merge candidate references reference list 0). In one example, RT1 includes a set of reference samples on the reference picture of the current block indicated by the reference index of the merge candidate that references the reference picture in reference list 1, where the MV of the merge candidate references reference list 1). Fig.23 An example diagram 2300 showing a template and reference samples of the template in reference list 0 and reference list 1 is shown. In one example, the reference samples (RT) of the templates used for bidirectional prediction are derived by taking an equally weighted average of the reference samples (RT0) of the templates in reference list 0 and the reference samples (RT1) of the templates in reference list 1. An example is as follows: RT=(RT0+RT1+1)>>1 In one example, the reference sample (RT) of the template used for bidirectional prediction bi-pred ) is derived by taking a weighted average of the reference samples of the template in reference list 0 (RT0) and the reference samples of the template in reference list 1 (RT1). An example is as follows: RT=((2 N -w)*RT0+w*RT1+2 N-1 )>>N, for example, N=3. In one example, the weight of the reference templates in reference list 0 such as (8-w) and the weight of the reference templates in reference list 1 such as (w) may be determined by the BCW index of the merge candidate. The merge candidates can be divided into several groups according to some criteria. Each group is called a subgroup. For example, we can take adjacent spatial and temporal merge candidates as the first subgroup and the remaining merge candidates as the second subgroup; in another example, we can also take the first N (N≥2) merge candidates as the first subgroup, the following M (M≥2) merge candidates as the second subgroup, and the remaining merge candidates as the third subgroup of candidates. Note that the proposed method can be applied to the candidate list construction process of merging inter-frame codec blocks (e.g., translational motion), affine codec blocks; or other motion candidate list construction processes (e.g., AMVP list, IBC AMVP list, and IBC merge list). W and H are the width and height of the current block (e.g., luminance block). The following description takes the merge candidate list construction process as an example: 1. The merging candidates can be adaptively rearranged in the final merging candidate list according to one or some criteria. a. In one example, firstly, part or all of the process of building the current merge candidate list is called, and then the candidates in the list are reordered. i. Alternatively, the candidates in the first subgroup may be reordered and they should be added before those in the second subgroup, where the first subgroup is added before the second subgroup. (i) In one example, multiple merge candidates of a first category may be first derived and then reordered within the first category; then merge candidates from the second category may be determined based on the reordered candidates in the first category (eg, how pruning is applied). ii. Alternatively, the first merge candidate in the first category may be compared with the second merge candidate in the second category to decide the order of the first or second merge candidate in the final merge candidate list. b. In one example, the merge candidates may be adaptively rearranged before being retrieved. i. In one example, the process of adaptively arranging merge candidates may be processed before obtaining merge candidates to be used in the motion compensation process. c. In one example, if the width of the current block is greater than the height of the current block, the above candidate is added before the left candidate. d. In one example, if the width of the current block is smaller than the height of the current block, the above candidate is added after the left candidate. e. Whether to adaptively rearrange merge candidates may depend on the selected merge candidate or the selected merge candidate index. i. In one example, if the selected merge candidate is in the last subgroup, the merge candidates are not adaptively re-arranged. f. In one example, costs are assigned to the merge candidates, and the merge candidates are adaptively reordered in ascending order of their costs. i. In one example, the cost of merging candidates may be a template matching cost. ii. In one example, the template is a set of reconstructed samples that are adjacent or non-adjacent to the current block. iii. The template matching cost is derived as a function of T and RT, where T is the set of samples in the template and RT is the set of reference samples for the template. (i) How to obtain the reference sample of the template of the merge candidate can depend on the motion information of the merge candidate a) In one example, when deriving reference samples for a template, the motion vector of a merge candidate is rounded to integer pixel precision, where an integer motion vector may be its closest integer motion vector. b) In one example, when deriving the reference samples of the template, an N-tap interpolation filter is used to obtain the reference samples of the template at the sub-pixel position. For example, N can be 2, 4, 6 or 8. c) In one example, when deriving reference samples for a template, the motion vectors of the merge candidates may be scaled to a given reference picture (eg, for each reference picture list, if available). d) For example, Fig. 22 As shown, a reference sample of the template of the merge candidate is obtained on a reference picture of the current block indicated by the reference index of the merge candidate, the reference picture having the MV of the merge candidate or a modified MV (eg, according to bullet points a)-b)). e) For example, when the merge candidate utilizes bidirectional prediction, the reference sample of the template of the merge candidate is represented by RT, and RT can be generated according to RT0 derived from the reference picture in reference picture list 0 and RT1 derived from the reference picture in reference picture list 1. [1] In one example, RT0 includes a set of reference samples on a reference picture of the current block indicated by a reference index of a merge candidate that references a reference picture in reference list 0, where the MV of the merge candidate references reference list 0). [2] In one example, RT1 includes a set of reference samples on a reference picture of the current block indicated by a reference index of a merge candidate that references a reference picture in reference list 1, where the MV of the merge candidate references 1 (reference list 1). [3] For example Fig.23 shown. f) In one example, the reference samples (RT) of the templates used for bidirectional prediction are derived by taking an equally weighted average of the reference samples (RT0) of the templates in reference list 0 and the reference samples (RT1) of the templates in reference list 1. An example is as follows: RT=(RT0+RT1+1)>>1 g) In one example, the reference sample (RT) of the template used for bidirectional prediction bi-pred ) is derived by taking a weighted average of the reference samples of the template in reference list 0 (RT0) and the reference samples of the template in reference list 1 (RT1). An example is as follows: RT=((2 N -w)*RT0+w*RT1+2 N-1 )>>N, for example, N=3. h) In one example, the weights of reference templates in reference list 0 such as (8-w) and the weights of reference templates in reference list 1 such as (w) may be determined by the BCW index of the merge candidate. [1] In one example, the BCW index is equal to 0 and w is set equal to -2. [2] In one example, the BCW index is equal to 1 and w is set equal to 3. [3] In one example, the BCW index is equal to 2 and w is set equal to 4. [4] In one example, the BCW index is equal to 3 and w is set equal to 5. [5] In one example, the BCW index is equal to 4 and w is set equal to 10. i) In one example, if the local illumination compensation (LIC) flag of the merge candidate is true, the reference samples of the template are derived using the LIC method. (ii) The cost can be calculated based on the sum of absolute differences (SAD) of T and RT. a) Alternatively, the cost may be calculated based on the Sum of Absolute Transformed Difference (SATD) of T and RT. b) Alternatively, the cost may be calculated based on the sum of squared differences (SSD) of T and RT. c) Alternatively, the cost may be calculated based on weighted SAD / weighted SATD / weighted SSD. (iii) In addition to the SAD calculated in (ii), the cost may also consider the continuity (Boundary_SAD) between RT and reconstructed samples adjacent or non-adjacent to T. For example, reconstructed samples to the left and / or above adjacent or non-adjacent to T are considered. a) In one example, the cost may be calculated based on SAD and Boundary_SAD. [1] In one example, the cost may be calculated as (SAD + w*Boundary_SAD). w may be predetermined, signaled, or derived from decoded information. 2. Whether and / or how to reorder merge candidates may depend on the category of the merge candidates. a. In one example, only adjacent spatial and temporal merging candidates may be reordered. b. In one example, only adjacent spatial merging candidates, STMVP merging candidates, and temporal merging candidates may be reordered. c. In one example, only adjacent spatial, STMVP, temporal, and non-adjacent spatial merging candidates may be reordered. d. In one example, only adjacent spatial, STMVP, temporal, non-adjacent spatial, and HMVP merging candidates may be reordered. e. In one example, only adjacent spatial domain, STMVP, temporal domain, HMVP of non-adjacent spatial domain and pairwise average merging candidates may be reordered. f. In one example, only adjacent spatial, temporal, HMVP and pairwise average merging candidates may be reordered. g. In one example, only adjacent spatial, temporal and HMVP merging candidates may be reordered. h. In one example, only adjacent spatial merging candidates may be reordered. i. In one example, only the first subgroup may be reordered. j. In one example, the last subgroup cannot be reordered. k. In one example, only the top N merge candidates may be reordered. i. In one example, N is set equal to 5. 1. In one example, for candidates that are not reordered, they are arranged in the merge candidate list according to the original order. m. In one example, candidates that are not reordered may be placed after candidates to be reordered. n. In one example, candidates that are not reordered may be placed before candidates to be reordered. o. In one example, some combinations of the above items (a-k) can be reordered. p. Different subgroups can be reordered individually. q. Two candidates in different subgroups cannot be compared and / or re-ranked. r. If the first subgroup is before the second subgroup, then after reordering, the first candidate in the first subgroup needs to be placed before the second candidate in the second subgroup. 3. Whether and / or how to reorder merge candidates may depend on the codec mode. a. In one example, if the coding mode is the normal merge mode, the merge candidates may be reordered. b. In one example, if the codec mode is MMVD, the merge candidates used to derive the base merge candidate are not reordered. i. Alternatively, the reordering method may be different for MMVD mode and other merge modes. c. In one example, if the coding mode is CIIP, the merge candidate for combining with intra prediction is based on the reordered merge candidates. i. Alternatively, the reordering method may be different for CIIP mode and other merge modes. d. In one example, if the coding mode is GPM, the merge candidates used to derive the unidirectional prediction candidate list are not reordered. i. Alternatively, the reordering method may be different for GPM mode and other merge modes. e. In one example, if the coding mode is the triangulation mode, the merge candidates used to derive the unidirectional prediction candidate list are not reordered. i. Alternatively, the reordering method may be different for triangle mode and other merging modes. f. In one example, if the coding mode is a sub-block based merge mode, candidate merges based on part or all of the sub-blocks are reordered. i. Alternatively, the reordering method may be different for sub-block based merge mode and other merge modes. ii. In one example, merging candidates based on unidirectional prediction sub-blocks are not reordered. iii. In one example, SbTMVP candidates are not reordered. iv. In one example, the constructed affine candidates are not reordered. v. In one example, zero-filled affine candidates are not reordered. 4. Whether and / or how to reorder merging candidates may depend on the available number of neighboring spatial and / or STMVP and / or temporal merging candidates. 5. Whether the merge candidates need to be reordered may depend on decoded information (eg, the width and / or height of the CU). a. In one example, if the height is greater than or equal to M, the width is greater than or equal to N, and the width*height is greater than or equal to R, the merge candidates may be reordered. i. In one example, M, N, and R are set equal to 8, 8, and 128. ii. In one example, M, N, and R are set equal to 16, 16, and 512. b. In one example, if the height is greater than or equal to M and the width is greater than or equal to N, the merge candidates may be reordered. i. In one example, M and N are set equal to 8 and 8. ii. In one example, M and N are set equal to 16 and 16. 6. The subgroup size can be adaptive. a. In one example, the subgroup size is determined based on the available number of adjacent spatial and / or STMVP and / or temporal merging candidates denoted as N. i. In one example, if N is less than M and greater than Q, the subgroup size is set to N; ii. In one example, if N is less than or equal to Q, no reordering is performed; iii. In one example, if N is greater than or equal to M, the subgroup size is set to M. iv. In one example, M and Q are set equal to 5 and 1 respectively. (i) Alternatively, M and / or Q may be predetermined, signaled, or derived from decoded information. b. In one example, the subgroup size is determined according to the available number of adjacent spatial and temporal merging candidates denoted as N. i. In one example, if N is less than M and greater than Q, the subgroup size is set to N; ii. In one example, if N is less than or equal to Q, no reordering is performed; iii. In one example, if N is greater than or equal to M, the subgroup size is set to M. iv. In one example, M and Q are set equal to 5 and 1 respectively. 7. The template shape can be adaptive. a. In one example, the template may only include neighboring samples left for the current block. b. In one example, the template may only include neighboring samples above the current block. c. In one example, the template shape is selected according to the CU shape. d. In one example, the width of the left template is selected according to the CU height. i. For example, if H<=M, the size of the left template is w1xH; otherwise, the size of the left template is w2xH. e. In one example, M, w1, and w2 are set equal to 8, 1, and 2, respectively. f. In one example, the height of the upper template is selected according to the CU width. i. For example, if W<=N, the size of the upper template is Wxh1; otherwise, the size of the upper template is Wxh2. (i) In one example, N, h1, and h2 are set equal to 8, 1, and 2, respectively. g. In one example, the width of the left template is selected according to the CU width. i. For example, if W<=N, the size of the left template is w1xH; otherwise, the size of the left template is w2xH. (i) In one example, N, w1, and w2 are set equal to 8, 1, and 2, respectively. h. In one example, the height of the upper template is selected according to the CU height. i. For example, if H<=M, the size of the upper template is Wxh1; otherwise, the size of the upper template is Wxh2. (i) In one example, M, h1, and h2 are set equal to 8, 1, and 2, respectively. i. In one example, samples of the template and reference samples of the template samples may be subsampled or downsampled before being used to calculate the cost. i. Whether and / or how subsampling is performed may depend on the CU size. ii. In one example, no subsampling is performed on the short edges of the CU. 8. In the above examples, the merge candidate is one of the candidates included in the final merge candidate list (eg, after pruning). a. Alternatively, the merge candidate is one derived from a given spatial or temporal block or HMVP table, or derived in other ways, even though it may not be included in the final merge candidate list. 9. Templates can include samples of specific color components. a. In one example, the template includes samples of the luma component only. 10. Whether to apply adaptive merge candidate list reordering can depend on the message transmitted by signal in VPS / SPS / PPS / sequence header / picture header / slice header / CTU / CU. It can also be based on the area of ​​the signal. For example, the picture is divided into groups of CTU / CU evenly or unevenly, and a flag is encoded and decoded for each group to indicate whether to apply merge candidate list reordering. 2.11. Adaptive Motion Candidate List 1. The motion candidates in the motion candidate list of the block may be adaptively rearranged according to one or some criteria to derive a reordered motion candidate list, and the block is encoded / decoded according to the reordered motion candidate list. a. The motion candidates in the motion candidate list of blocks that are not in the regular merge candidate list may be adaptively rearranged according to one or some criteria to derive a reordered motion candidate list. b. In one example, whether and / or how to reorder motion candidates may depend on the codec mode (e.g., affine merge, affine AMVP, regular merge, regular AMVP, GPM, TPM, MMVD, TM merge, CIIP, GMVD, affine MMVD). c. In one example, whether and / or how to reorder motion candidates may depend on the category of the motion candidates (eg, spatial, temporal, STMVP, HMVP, pairwise, SbTMVP, constructive affine, inherited affine). d. In one example, the motion candidate list may be an AMVP candidate list. e. In one example, the motion candidate list may be a merge candidate list. f. In one example, the motion candidate list may be an affine merge candidate list. g. In one example, the motion candidate list may be a sub-block based merge candidate list. h. In one example, the motion candidate list may be a GPM merge candidate list. i. In one example, the motion candidate list may be a TPM merge candidate list. j. In one example, the motion candidate list may be a TM merge candidate list. k. In one example, the motion candidate list may be a candidate list of an MMVD codec block. 1. In one example, the motion candidate list may be a candidate list of a DMVR codec block. 2. How to adaptively rearrange motion candidates in a motion candidate list may depend on decoded information, for example, the category of the motion candidate, the category of the motion candidate list, and the encoding and decoding tool. a. In one example, for different motion candidate lists, different criteria may be used to re-rank the motion candidate lists. i. In one example, the criteria may include how to select a template. ii. In one example, the criteria may include how to calculate the template cost. iii. In one example, the criteria may include how many candidates in the candidate list and / or how many subgroups need to be reordered. b. In one example, motion candidates in a motion candidate list are first adaptively rearranged to construct a fully rearranged candidate list or a partially rearranged candidate list, and then at least one motion candidate indicated by at least one index is retrieved from the rearranged candidate list to derive final motion information to be used by the current block. c. In one example, the motion candidates before refinement are first adaptively rearranged (e.g., using TM for TM-encoded blocks; adding MVD for MMVD-encoded blocks) to construct a fully rearranged candidate list or a partially rearranged candidate list. Then, at least one motion candidate indicated by at least one index is retrieved from the rearranged candidate list, and refinement (e.g., using TM for TM-encoded blocks; adding MVD for MMVD-encoded blocks) is applied to the retrieved motion candidate to derive final motion information of the current block. d. In one example, refinement is applied to at least one motion candidate in the motion candidate list (e.g., using TM for TM codec blocks; adding MVD for MMVD codec blocks), and then adaptively rearranges them to construct a fully rearranged candidate list or a partially rearranged candidate list, and then at least one motion candidate indicated by at least one index is retrieved from the rearranged candidate list to derive final motion information without any further refinement for the current block. 3. In one example, new MERGE / AMVP motion candidates may be generated based on the candidate re-ranking. i. For example, the candidate L0 motion and L1 motion may be reordered separately. ii. For example, a new bi-prediction merging candidate may be constructed by combining one from the reordered L0 motion and another from the reordered L1 motion. iii. For example, new uni-directional prediction merging candidates can be generated by reordering L0 or L1 motion. 2.12. Adaptive Motion Candidate List For sub-block motion prediction, if the sub-block size is Wsub*Hsub, the height of the upper template is Ht, and the width of the left template is Wt, the upper template can be regarded as a composition of several sub-templates with a size of Wsub*Ht, and the left template can be regarded as a composition of several sub-templates with a size of Wt*Hsub. After deriving the reference sample of each sub-template in a similar manner as described above, the reference sample of the template is derived. Fig.24 An example diagram 2400 showing a template of a block having sub-block motion using motion information of a sub-block of a current block and a reference sample of the template is shown. Fig.25An example diagram 2500 showing a template of a block with sub-block motion using motion information of each sub-template and a reference sample of the template is shown. It should be noted that the terms mentioned below are not limited to specific terms defined in existing standards. Any changes in codecs also apply. For example, the term "GPM" is used to represent any codec that derives two sets of motion information and uses the derived information and the partitioning pattern to obtain the final prediction, for example, TPM is also considered a GPM. Note that the proposed method can be applied to the merge candidate list construction process of inter-frame codec blocks (e.g., translational motion), affine codec blocks, or IBC codec blocks; or other motion candidate list construction processes (e.g., normal AMVP list, affine AMVP list, and IBC AMVP list). W and H are the width and height of the current block (eg, luma block). 1. In one example, if the coding mode is TM merging, some or all TM merging candidates may be reordered. a. In one example, if the coding mode is TM merging, part or all of the original TM merging candidates may be reordered before the TM refinement process. b. Alternatively, if the coding mode is TM merging, some or all of the refined TM merging candidates may be reordered after the TM refinement process. c. Alternatively, if the coding mode is TM merging, the TM merging candidates may not be reordered. d. Alternatively, the reordering method may be different for TM merge mode and other merge modes. 2. In one example, if the coding mode is a sub-block based merge mode, candidate merges based on part or all of the sub-blocks may be reordered. a. Alternatively, the reordering method may be different for the sub-block based merge mode and other merge modes. b. In one example, a template can be divided into sub-templates. Each sub-template can have individual motion information. i. In one example, a cost for a reordering candidate may be derived based on the cost of each sub-template. For example, the cost for a reordering candidate may be calculated as the sum of the costs of all sub-templates. For example, the cost of a sub-template may be calculated as SAD, SATD, SSD, or any other distortion measure between a sub-template and its corresponding reference sub-template. c. In one example, to derive reference samples of the sub-template, the motion information of the sub-blocks in the first row and first column of the current block may be used. i. In one example, the motion information of a sub-template can be derived (eg, copied) from its neighboring sub-block in the current block. Fig.24 shown. d. In one example, in order to derive a reference sample of a sub-template, the motion information of the sub-template may be derived without referring to the motion information of the sub-block in the current block. Fig.25 shown. i. In one example, the motion information of each sub-template is calculated according to the affine model of the current block. (i) In one example, the motion vector of the center sample of each sub-block containing the sub-template calculated according to the affine model of the current block is regarded as the motion vector of the sub-template. (ii) In one example, the motion vector of the center sample of each sub-template calculated according to the affine model of the current block is regarded as the motion vector of the sub-template. (iii) For the 4-parameter affine motion model, the motion vector at the sample position (x, y) in the block is derived as: (iv) For the 6-parameter affine motion model, the motion vector at the sample position (x, y) in the block is derived as: (v) For (iii) and (iv), the coordinates of the upper left corner, upper right corner, and lower left corner of the current block are (0, 0), (W, 0), and (0, H), and the motion vectors of the upper left corner, upper right corner, and lower left corner of the current block are (mv 0x , mv 0y )、(mv 1x , mv 1y ) and (mv 2x , mv 2y ). (vi) In one example, the coordinates (x, y) in the above equation may be set equal to the position in the template or the position of the sub-template. For example, the coordinates (x, y) may be set equal to the center position of the sub-template. e. In one example, this scheme can be applied to affine merge candidates. f. In one example, this scheme can be applied to affine AMVP candidates. g. In one example, this scheme can be applied to SbTMVP merge candidates. h. In one example, this scheme can be applied to GPM merge candidates. i. In one example, the scheme can be applied to TPM merge candidates. j. In one example, this scheme can be applied to TM to refine merge candidates. k. In one example, this scheme can be applied to DMVR to refine merge candidates. 1. In one example, this scheme can be applied to MULTI_PASS_DMVR to refine merge candidates. 3. In one example, if the codec mode is MMVD, the merge candidates used to derive the base merge candidate may be reordered. a. In one example, a reordering process may be applied to the merge candidates before they are refined by a signaled or derived MVD. b. For example, the reordering method may be different for MMVD mode and other merge modes. 4. In one example, if the coding mode is MMVD, the merge candidates after MMVD refinement may be reordered. a. In one example, a reordering process may be applied to the merge candidates after they are refined by a signaled or derived MVD. b. For example, the reordering method may be different for MMVD mode and other merge modes. 5. In one example, if the coding mode is affine MMVD, the merge candidates from which the base merge candidate is derived may be reordered. a. In one example, a reordering process may be applied to affine merge candidates before they are refined by a signaled or derived MVD. b. For example, the reordering method may be different for the affine MMVD mode and other merge modes. 6. In one example, if the coding mode is affine MMVD, the merge candidates after affine MMVD refinement may be reordered. a. In one example, a re-ranking process may be applied to affine merge candidates after they are refined by a signaled or derived MVD. b. For example, the reordering method may be different for the affine MMVD mode and other merge modes. 7. In one example, if the codec mode is GMVD, the merge candidates used to derive the base merge candidate may be reordered. a. In one example, a reordering process may be applied to the merge candidates before they are refined by a signaled or derived MVD. b. For example, the reordering method may be different for GMVD mode and other merge modes. 8. In one example, if the coding mode is GMVD, the merge candidates after GMVD refinement may be reordered. a. In one example, a reordering process may be applied to the merge candidates after they are refined by a signaled or derived MVD. b. For example, the reordering method may be different for GMVD mode and other merge modes. 9. In one example, if the codec mode is GPM, the merge candidates may be reordered. a. In one example, before the merge candidates are used to derive the GPM candidate list for each partition (also referred to as the unidirectional prediction candidate list of the GPM), a reordering process may be applied to the original merge candidates. b. In one example, if the coding mode is GPM, the merge candidates in the unidirectional prediction candidate list may be reordered. c. In one example, a GPM unidirectional prediction candidate list may be constructed based on re-ranking. i. In one example, a candidate with bi-directional prediction (also referred to as a bi-directional prediction candidate) may be separated into two uni-directional prediction candidates. (i) If the number of original merging candidates is M, at most 2M unidirectional prediction candidates can be separated from them. ii. In one example, unidirectional prediction candidates separated from bidirectional prediction candidates may be put into an initial unidirectional prediction candidate list. iii. In one example, the candidates in the initial unidirectional prediction candidate list may be reordered using the template matching cost. iv. In one example, the first N unidirectional prediction candidates with smaller template matching costs can be used as the final GPM unidirectional prediction candidates. For example, N is equal to M. d. In one example, after deriving the GPM unidirectional prediction candidate list, a combined bidirectional prediction list of partition 0 and partition 1 is constructed, and then the bidirectional prediction list is reordered. i. In one example, if the number of GPM unidirectional prediction candidates is M, the number of combined bidirectional prediction candidates is M*(M-1). e. Alternatively, the reordering method may be different for GPM mode and other merge modes. 2.13. Adaptive motion candidate list It should be noted that the terms mentioned below are not limited to specific terms defined in existing standards. Any changes in codecs also apply. For example, the term "GPM" is used to represent any codec that derives two sets of motion information and uses the derived information and the partitioning pattern to obtain the final prediction. For example, TPM is also considered a GPM. Note that the proposed method can be applied to the merge candidate list construction process of inter-frame codec blocks (e.g., translational motion), affine codec blocks, or IBC codec blocks; or other motion candidate list construction processes (e.g., normal AMVP list, affine AMVP list, and IBC AMVP list). W and H are the width and height of the current block (eg, luma block). 1. The reference sample (RT) of the template or sub-template used for bidirectional prediction is obtained by equally weighted averaging the reference sample (RT0) of the template or sub-template in reference list 0 and the reference sample (RT1) of the template or sub-template in reference list 1. An example is as follows: RT(x,y)=(RT0(x,y)+RT1(x,y)+1)>>1 2. The reference sample (RT) of the template or sub-template used for bidirectional prediction is obtained by weighted averaging the reference sample (RT0) of the template or sub-template in reference list 0 and the reference sample (RT1) of the template or sub-template in reference list 1. a. An example is as follows: RT(x, y)=(2 N -w)*RT0(x,y)+w*RT1(x,y)+2 N-1 )>>N, for example, N=3. b. The weights may be determined by the BCW index, and may be derived on the fly or predetermined or determined by the weights used in the weighted forecast. c. In one example, the weight of a reference template such as (8-w) in reference list 0 and the weight of a reference template such as (w) in reference list 1 may be determined by the BCW index of the merge candidate. i. In one example, the BCW index is equal to 0 and w is set to -2. ii. In one example, the BCW index is equal to 1 and w is set to 3. iii. In one example, the BCW index is equal to 2 and w is set to 4. iv. In one example, BCW index equals 3 and w is set to 5. v. In one example, BCW index equals 4 and w is set to 10. 3. Reference samples for deriving templates using the LIC method are proposed. a. In one example, the LIC parameters of the left template and the upper template are the same as the LIC parameters of the current block. b. In one example, the LIC parameters of the left template are derived as the LIC parameters of the current block, which uses its original motion vector plus the motion vector offset (-Wt, 0) as the motion vector of the current block. c. In one example, the LIC parameters of the upper template are derived as the LIC parameters of the current block, which uses its original motion vector plus the motion vector offset (0, -Ht) as the motion vector of the current block. d. Alternatively or additionally, if the local illumination compensation (LIC) flag of the merge candidate is true, the above bullet points may be applied. 4. A reference sample for deriving a template or sub-template using the OBMC method is proposed. In the following discussion, "template" may refer to a template or a sub-template. a. In one example, in order to derive the reference samples of the upper template, the motion information of the sub-block in the first row of the current block and its adjacent neighboring sub-blocks above it is used. The reference samples of all sub-templates constitute the reference samples of the upper template. Fig.26 Shown in. Fig.26 An example graph 2600 showing a reference sample of a template and a template of a block with OBMC is shown. b. In one example, to derive the reference samples of the left template, the motion information of the sub-block in the first column of the current block and its adjacent sub-blocks to the left is used. And the reference samples of all sub-templates constitute the reference samples of the left template. An example is Fig.26 shown. c. In one example, the sub-block size is 4x4. d. The reference sample of the sub-template based on the motion vector of the neighboring sub-block is represented as P N , where N represents the index of the adjacent upper and left sub-blocks, and the reference sample of the sub-template based on the motion vector of the sub-block of the current block is represented by P C . For P generated based on the motion vectors of vertically (horizontally) adjacent sub-blocks N , P N The samples in the same row (column) of P are added to P with the same weighting factor. C . i. The reference sample (P) of the sub-template can be derived as P = W N *P N +W C *P C ii. In one example, if the height of the upper template or the width of the left template is greater than or equal to 4, the weight factors {1 / 4, 1 / 8, 1 / 16, 1 / 32} are used for P N The {first, second, third, fourth} rows (columns) of P, and the weight factors {3 / 4, 7 / 8, 15 / 16, 31 / 32} are used for P C The {first, second, third, fourth} rows (columns) of . iii. In one example, if the height of the upper template or the width of the left template is greater than or equal to 2, the weighting factor {1 / 4, 1 / 8} is used for P N The {first, second} rows (columns) of P, weighting factors {3 / 4,7 / 8} are used for P C The {first, second} row (column) of . iv. In one example, if the height of the template above or the width of the template to the left is greater than or equal to 1, the weighting factor {1 / 4} is used for P N The first row (column) of , the weighting factor {3 / 4} is used for P C The first row (column) of . e. If merge candidates are assigned with OBMC enabled, the above bullets may apply. 5. In one example, if the merged candidate uses multi-hypothesis prediction, the reference sample of the template can be derived using the multi-hypothesis prediction method. 6. A template may include samples of (multiple) specific color components. a. In one example, the template includes samples of the luma component only. b. Alternatively, the template only includes samples of any component, such as Cb / Cr / R / G / B. 7. Whether and / or how to reorder motion candidates may depend on the category of the motion candidates. a. In one example, only neighboring spatial and temporal motion candidates may be reordered. b. In one example, only neighboring spatial, STMVP and temporal motion candidates may be reordered. c. In one example, only adjacent spatial, STMVP, temporal, and non-adjacent spatial motion candidates may be reordered. d. In one example, only adjacent spatial, STMVP, temporal, non-adjacent spatial, and HMVP motion candidates may be reordered. e. In one example, only adjacent spatial, STMVP, temporal, non-adjacent spatial HMVP and pairwise average motion candidates may be reordered. f. In one example, only neighboring spatial, temporal, HMVP and pairwise average motion candidates may be reordered. g. In one example, only neighboring spatial, temporal and HMVP motion candidates may be reordered. h. In one example, only neighboring spatial motion candidates may be reordered. i. In one example, motion candidates based on unidirectional prediction sub-blocks are not reordered. j. In one example, SbTMVP candidates are not reordered. k. In one example, inherited affine motion candidates are not reordered. 1. In one example, the constructed affine motion candidates are not reordered. m. In one example, zero-filled affine motion candidates are not reordered. n. In one example, only the top N motion candidates may be reordered. i. In one example, N is set equal to 5. 8. In one example, the motion candidates may be divided into multiple subgroups. Whether and / or how to reorder the motion candidates may depend on the subgroups of the motion candidates. a. In one example, only the first subgroup may be reordered. b. In one example, the last subgroup cannot be reordered. c. In one example, the last subgroup cannot be reordered. However, if the last subgroup is also the first subgroup, it can be reordered. d. Different subgroups can be reordered individually. e. Two candidates in different subgroups cannot be compared and / or re-ranked. f. If the first subgroup is before the second subgroup, then after reordering, the first candidate in the first subgroup needs to be placed before the second candidate in the second subgroup. 9. In one example, motion candidates not included in the reordering process may be processed in a specified manner. a. In one example, for candidates that are not reordered, they are arranged in the merge candidate list according to the original order. b. In one example, candidates that are not reordered may be placed after candidates to be reordered. c. In one example, candidates that are not reordered may be placed before candidates to be reordered. 10. Whether to apply adaptive merge candidate list reordering can depend on the message transmitted by signal in VPS / SPS / PPS / sequence header / picture header / slice header / CTU / CU. It can also be based on the area of ​​the signal. For example, the picture is divided into groups of CTU / CU evenly or unevenly, and a flag is encoded and decoded for each group to indicate whether to apply merge candidate list reordering. 2.14. Cost function used in codec data refinement in image / video codecs The term "block" may refer to a codec tree block (CTB), a codec tree unit (CTU), a codec block (CB), a CU, a PU, a TU, a PB, a TB, or a video processing unit including multiple samples / pixels, and a block may be rectangular or non-rectangular. In the present disclosure, the phrase "motion candidate" may refer to a merged motion candidate in a regular / extended merge list indicated by a merge candidate index, or an AMVP motion candidate in a regular / extended AMVP list indicated by an AMVP candidate index, or an AMVP motion candidate, or a merged motion candidate. In this disclosure, a motion candidate is referred to as "refined" if its motion information is modified based on information signaled from an encoder or derived at a decoder. For example, a motion vector may be refined by DMVR, FRUC, TM Merge, TMAMVP, TM GPM, TM CIIP, TM Affine, MMVD, GMVD, Affine MMVD, BDOF, etc. In the present disclosure, the phrase "codec data refinement" may refer to a refinement process to derive or refine a signal / decoded / derived prediction mode, prediction direction, or signal / decoded / derived motion information, prediction and / or reconstruction samples for a block. In one example, the refinement process may include motion candidate reordering. In the following discussion, a "template-based codec" block may refer to a block that uses a template matching-based method to derive or refine codec information during the encoding / decoding process, such as template matching-based motion derivation, template matching-based motion list reconstruction, LIC, symbol prediction, template matching-based block vector (e.g., used in IBC mode) derivation, DIMD, template matching-based non-inter-frame (e.g., intra-frame) prediction, etc. The template-based codec method can be combined with any other codec tools, such as MMVD, CIIP, GPM, FRUC, affine, BDOF, DMVR, OBMC, etc. In another example, a "template-based coding" block may also refer to a block that uses (adjacent or non-adjacent) neighboring reconstructed samples to derive or refine its decoding information based on a certain rule, such as the DIMD method in 2.27 and the TIMD method in 2.29. In the following discussion, a "bilateral codec-based" block may refer to a block that uses a bilateral matching-based method to derive or refine codec information during the encoding / decoding process, such as bilateral matching-based motion derivation, bilateral matching-based motion list reconstruction, etc. The bilateral codec-based method can be combined with any other codec tools, such as MMVD, CIIP, GPM, FRUC, Affine, DMVR, etc. W and H are the width and height of the current block (e.g., luminance block). W*H is the size of the current block (e.g., luminance block) In the following discussion, Shift(x,s) is defined as Shift(x,s)=(x+offset)>>s, where the offset (offset) is an integer, such as offset=0 or offset=1<<(s-1) or offset=(1<<(s-1))-1. In another example, offset depends on x. For example, offset = (x<0?(1<<(s-1)):((1<<(s-1)-1). 1. In addition to error measurement, a regular term is proposed to be added during the cost calculation process. a) In one example, the cost is defined as: E + W * RI, where E represents the output of the error function, and W is the weight applied to the regular term represented by RI. i. In one example, to process template-based encoding / decoding blocks / bilateral-based encoding / decoding blocks, the cost function is set as: E + W * RI, where E can be SAD / MRSAD / SATD or others, RI is the true motion vector / estimated bits of motion vector difference, and W is the weight, for example, it can depend on QP / temporal layer, etc. ii. Alternatively, the cost can be defined as: w0 * E + W1 * RI, where E represents the output of the error function, W1 is the weight applied to the regular term represented by RI, and w0 is the weight applied to the output of the error function. (i) Alternatively, additionally, W1 can be set to 0. b) In one example, the regular term can be multiplied by a weight rate. i. In one example, the weight is obtained immediately. ii. In one example, the weight is set to lambda used in the full RDO process. iii. In one example, the weight is set to the square root of lambda used in the full RDO process. c) In one example, the cost is calculated as E + Shift(W * RI, s), where s and W are integers. i. Alternatively, the cost is calculated as Shift((E << s) + W * RI, s), where s and W are integers. 2. An error function different from SAD / MR - SAD (sum of absolute differences with mean removed) is proposed to process template-based encoding / decoding blocks / bilateral-based encoding / decoding blocks. a) In one example, the error function can be i. SATD, ii. MR - SATD, iii. Gradient information, iv. SSE / SSD, v. MR - SSE / MR - SSD, vi. Weighted SAD / Weighted MR - SAD, vii. Weighted SATD / Weighted MR - SATD, viii. Weighted SSD / Weighted MR - SSD, ix. Weighted SSE / Weighted MR - SSE. b) Alternatively, in addition, a method of adaptively selecting an error function among different cost functions such as the above-mentioned error function and SAD / MR-SAD is proposed. i. The choice can be determined instantly. 3. When using an error function based on MR-X (e.g., X is SATD, SAD, SSE) (e.g., MR-SAD / MR-SATD, etc.), the following can be further applied: a) In one example, the mean may be calculated taking into account all samples in the block to be compared. b) In one example, the mean may be calculated using a portion of the samples in the block to be considered for comparison. c) In one example, the mean and the X-function may depend on the same samples in a block. i. In one example, the mean and X-function may be calculated using all samples in a block. ii. In one example, a portion of the samples in a block may be used to calculate the mean and the X-function. d) In one example, the mean and the X-function may depend on at least one different sample in the block. i. In one example, the mean can be calculated using all samples, while the X function can depend on some of the samples in the block. ii. In one example, the mean can be calculated using a portion of the samples, while the X function can depend on all samples in the block. 4. The template / bilateral matching cost can be calculated by applying the cost factor to the error cost function. a) In one example, it is proposed to bias motion candidates towards the front during the template / bilateral matching based re-ranking process. i. In one example, the motion candidate in the i-th position is assigned a cost factor that is smaller than the cost factor of the motion candidate in the (i+1)-th position. ii. In one example, the motion candidates in the i-th group (eg, involving M motion candidates) are assigned a cost factor that is smaller than the cost factor of the motion candidates in the (i+1)-th group (eg, involving N motion candidates). (i) In one example, M may be equal to N. For example, M=N=2. (ii) In one example, M may not be equal to N. For example, M=2, N=3. b) In one example, it is proposed to prefer a search MV that is closer to the original MV during the template / bilateral matching based refinement process. i. In one example, each search region is assigned a cost factor, which may be determined by the distance between each search MV in the search region and the start MV (eg, in integer pixel precision MV increments). ii. In one example, each search region is assigned a cost factor, which may be determined by the distance between a center search MV and a start MV in the search region (eg, in integer pixel precision MV increments). iii. In one example, each search MV is assigned a cost factor, which may be determined by the distance between each search MV and the start MV (eg, in integer pixel precision increments of the MV). 5. The above method can be applied to any codec data refinement process, for example, for blocks based on template codecs, for blocks based on bilateral codecs (such as DMVR in VVC). 6. Template matching cost measurement may be different for different template matching refinement methods. a. In one example, the template matching refinement method may be a motion candidate re-ranking based on template matching. b. In one example, the template matching refinement method may be a motion derivation based on template matching. i. In one example, the refinement method may be TM AMVP, TM Merge, and / or FRUC. c. In one example, the template matching refinement method may be template matching based motion refinement. ii. In one example, the refinement method may be TM GPM, TM CIIP and / or TM Affine. d. In one example, the template matching refinement method may be based on template matching block vector derivation. e. In one example, the template matching refinement method may be intra-mode derivation based on template matching. iii. In one example, the thinning method may be DIMD and / or TIMD. f. In one example, the template matching cost measure may be calculated based on the sum of absolute differences (SAD) between the current template and the reference template. g. In one example, the template matching cost measure may be calculated based on the mean removed SAD between the current template and the reference template. h. In one example, SAD and mean removed SAD (MR-SAD) can be selectively utilized according to the size of the current block. i. In one example, mean removal SAD is used for blocks of size greater than M, and SAD is used for blocks of size less than or equal to M. (i) In one example, M is 64. i. In one example, SAD and mean removed SAD (MR-SAD) can be selectively utilized according to the LIC flag of the current block. i. In one example, if the LIC flag of the current block is false, the template matching cost measure may be SAD. ii. In one example, if the LIC flag of the current block is true, the template matching cost measure may be MR-SAD. j. In one example, the template matching cost measure may be calculated based on the sum of absolute transformed differences (SATD) between the current template and the reference template. k. In one example, the template matching cost measure may be calculated based on the mean-removed SATD between the current template and the reference template. 1. In one example, SATD and mean-removed SATD (MR-SATD) can be selectively utilized according to the size of the current block. i. In one example, mean removal SATD is used for blocks of size greater than M, and SATD is used for blocks of size less than or equal to M. (i) In one example, M is 64. m. In one example, SATD and mean removal SATD (MR-SATD) can be selectively utilized according to the LIC flag of the current block. i. In one example, if the LIC flag of the current block is false, the template matching cost measure can be SATD. ii. In one example, if the LIC flag of the current block is true, the template matching cost measure can be MR-SATD. n. In one example, the template matching cost measure may be calculated based on the sum of squared differences (SSD) between the current template and the reference template. o. In one example, the template matching cost measure may be calculated based on the mean removed SSD between the current template and the reference template. p. In one example, SSD and mean removed SSD (MR-SSD) can be selectively utilized according to the size of the current block. i. In one example, mean removal SSD is used for blocks of size greater than M, and SSD is used for blocks of size less than or equal to M. (i) In one example, M is 64. q. In one example, the template matching cost measurement can be weighted SAD / weighted MR-SAD / selective weighted MR-SAD and SAD / weighted SATD / weighted MR-SATD / selective weighted MR-SATD and SATD / weighted SSD / weighted MR-SSD / selective weighted MR-SSD and SSD between the current template and the reference template. i. In one example, weighting means applying different weights to each sample based on its row and column index in the template block when calculating the distortion between the current template and the reference template. ii. In one example, weighting means applying different weights to each sample based on its position in the template block when calculating the distortion between the current template and the reference template. iii. In one example, weighting means applying different weights to each sample according to its distance to the current block when calculating the distortion between the current template and the reference template. r. In one example, the template matching cost may be calculated as follows: tplCost = w1*mvDistanceCost + w2*distortionCost. i. In one example, distortionCost may be the weighted SAD / weighted MR-SAD / weighted SATD / weighted MR-SATD / weighted SSD / weighted MR-SSD / SAD / MR-SAD / SATD / MR-SATD / SSD / MR-SSD between the current template and the reference template. ii. In one example, mvDistanceCost may be the sum of the absolute mv differences between the search point and the start point in the horizontal direction and the vertical direction. iii. In one example, w1 and w2 may be predetermined, or signaled, or derived from decoded information. (i) In one example, w1 is a weighting factor set to 4 and w2 is a weighting factor set to 1 s. In addition to the SAD calculated in (f), the cost may also consider the continuity between the reference template and the reconstructed samples adjacent to or non-adjacent to the current template (Boundary_SAD). For example, consider the reconstructed samples adjacent to the left and / or above the current sample. i. In one example, the cost can be calculated based on SAD and Boundary_SAD (i) In one example, the cost may be calculated as (SAD + w*Boundary_SAD). w may be predetermined, signaled, or derived from decoded information. 7. The bilateral matching cost measurement can be different for different bilateral matching refinement methods. a) In one example, the bilateral matching refinement method may be a motion candidate re-ranking based on bilateral matching. b) In one example, the bilateral matching refinement method may be motion derivation based on bilateral matching. i. In one example, the refinement method may be BM merging and / or FRUC. c) In one example, the bilateral matching refinement method may be motion refinement based on bilateral matching. i. In one example, the refinement method can be BM GPM, BM CIIP and / or BM Affine. d) In one example, the bilateral matching refinement method may be a block vector derivation based on bilateral matching. e) In one example, the bilateral matching refinement method may be an intra-mode derivation based on bilateral matching. f) In one example, the bilateral matching cost measure may be calculated based on the sum of absolute differences (SAD) between two reference blocks / sub-blocks. g) In one example, the bilateral matching cost measure may be calculated based on the mean removed SAD between two reference blocks / sub-blocks. h) In one example, SAD and mean removed SAD (MR-SAD) can be selectively utilized according to the size of the current block / sub-block. i. In one example, mean removed SAD is used for blocks / sub-blocks with size greater than M, and SAD is used for blocks / sub-blocks with size less than or equal to M. (i) In one example, M is 64. i) In one example, SAD and mean removed SAD (MR-SAD) can be selectively utilized according to the LIC flag of the current block. i. In one example, if the LIC flag of the current block is false, the bilateral matching cost measure can be SAD. ii. In one example, if the LIC flag of the current block is true, the bilateral matching cost measure may be MR-SAD. j) In one example, the bilateral matching cost measure may be calculated based on the Sum of Absolute Transformed Difference (SATD) between two reference blocks / sub-blocks. k) In one example, the bilateral matching cost measure may be calculated based on the mean-removed SATD between two reference blocks / sub-blocks. l) In one example, SATD and mean removed SATD (MR-SATD) can be selectively utilized according to the size of the current block / sub-block. i. In one example, mean removal SATD is used for blocks / sub-blocks with size greater than M, and SATD is used for blocks / sub-blocks with size less than or equal to M. (i) In one example, M is 64. m) In one example, SATD and mean removed SATD (MR-SATD) can be selectively utilized according to the LIC flag of the current block. i. In one example, if the LIC flag of the current block is false, the bilateral matching cost measure can be SATD. ii. In one example, if the LIC flag of the current block is true, the bilateral matching cost measure can be MR-SATD. n) In one example, the bilateral matching cost measure may be calculated based on the sum of squared differences (SSD) between two reference blocks / sub-blocks. o) In one example, the bilateral matching cost measure may be calculated based on the mean removed SSD between two reference blocks / sub-blocks. p) In one example, SSD and mean removed SSD (MR-SSD) can be selectively utilized according to the size of the current block / sub-block. i. In one example, mean removal SSD is used for blocks / sub-blocks with size greater than M, and SSD is used for blocks / sub-blocks with size less than or equal to M. (i) In one example, M is 64. q) In one example, SSD and mean removed SSD (MR-SSD) can be selectively utilized according to the LIC flag of the current block. i. In one example, if the LIC flag of the current block is false, the bilateral matching cost measure can be SSD. ii. In one example, if the LIC flag of the current block is true, the bilateral matching cost measure can be MR-SSD. r) In one instance, the bilateral matching cost measure can be weighted SAD / weighted MR-SAD / selective weighted MR-SAD and SAD / weighted SATD / weighted MR-SATD / selective weighted MR-SATD and SATD / weighted SSD / weighted MR-SSD / selective weighted MR-SSD and SSD between two reference blocks / sub-blocks. i. In one example, weighting means applying different weights to each sample based on its row and column index in the reference block / sub-block when calculating the distortion between two reference blocks / sub-blocks. ii. In one example, weighting means that when calculating the distortion between two reference blocks / sub-blocks, different weights are applied to each sample based on its position in the reference block / sub-block. iii. In one example, weighting means that when calculating the distortion between two reference blocks / sub-blocks, different weights are applied to each sample according to its distance to the center position of the reference block / sub-block. s) In one example, if MR-SAD / MR-SATD / MR-SSD is used for bilateral matching cost measurement, LIC may not be used when deriving reference blocks / sub-blocks. t) In one example, the bilateral matching cost can be calculated as follows: bilCost = w1*mvDistanceCost + w2*distortionCost. i. In one example, distortionCost can be the weighted SAD / weighted MR-SAD / weighted SATD / weighted MR-SATD / weighted SSD / weighted MR-SSD / SAD / MR-SAD / SATD / MR-SATD / SSD / MR-SSD between two reference blocks / sub-blocks. ii. In one example, mvDistanceCost may be the sum of the absolute mv differences between the search point and the start point in the horizontal and vertical directions. iii. In one example, w1 and w2 may be predetermined, or signaled, or derived from decoded information. (i) In one example, w1 is a weighting factor set to 4, and w2 is a weighting factor set to 1. 8. The bilateral matching cost or template matching cost can be calculated based on the prediction / reference samples modified by the function. a) In one example, the prediction / reference samples may be filtered before being used to calculate the bilateral matching cost or the template matching cost. b) In one example, the prediction / reference sample S can be modified to a*S+b before being used to calculate the bilateral matching cost or the template matching cost. c) In one example, the modification may depend on the codec mode of the block, such as whether the block is LIC coded or BCW coded. 2.15. Use of multiple cost functions in image / video codec data refinement The codec term "block" can refer to a codec tree block (CTB), a codec tree unit (CTU), a codec block (CB), CU, PU, ​​TU, PB, TB, or a video processing unit including multiple samples / pixels. The block can be rectangular or non-rectangular. In the present disclosure, the phrase "motion candidate" may refer to a merged motion candidate in a regular / extended merge list indicated by a merge candidate index, or an AMVP motion candidate in a regular / extended AMVP list indicated by an AMVP candidate index, or an AMVP motion candidate, or a merged motion candidate. In this disclosure, a motion candidate is referred to as "refined" if its motion information is modified based on information signaled from an encoder or derived at a decoder. For example, a motion vector may be refined by DMVR, FRUC, TM Merge, TMAMVP, TM GPM, TM CIIP, TM Affine, MMVD, GMVD, Affine MMVD, BDOF, etc. In the present disclosure, the phrase "codec data refinement" may refer to a refinement process to derive or refine a signaled / decoded / derived prediction mode, prediction direction, or signaled / decoded / derived motion information, prediction, and / or reconstructed samples for a block. In one example, the refinement process may include motion candidate reordering. In the following discussion, a "template-based" block may refer to a block that uses a template matching-based method to derive or refine codec information during the encoding / decoding process, such as motion derivation based on template matching, motion list reconstruction based on template matching, LIC, symbol prediction, block vector derivation based on template matching (e.g., used in IBC mode), DIMD, non-inter-frame (e.g., intra-frame) prediction based on template matching, etc. The template-based codec method can be combined with any other codec tools, such as MMVD, CIIP, GPM, FRUC, Affine, BDOF, DMVR, OBMC, etc. In another example, a "template-based" block may also refer to a block that uses (adjacent or non-adjacent) neighboring reconstructed samples based on a certain rule to derive or refine its decoding information, such as the DIMD method in 2.27 and the TIMD method in 2.29. In the following discussion, a "bilateral-based" block may refer to a block that uses a bilateral matching-based method to derive or refine codec information during the encoding / decoding process, such as motion derivation based on bilateral matching, motion list reconstruction based on bilateral matching, etc. The bilateral codec-based approach can be combined with any other codec tools, such as MMVD, CIIP, GPM, FRUC, Affine, DMVR, etc. W and H are the width and height of the current block (e.g., luminance block). W*H is the size of the current block (e.g., luminance block) 1. The cost definition may rely on the output of multiple error functions (eg, distortion measurement methods) regarding the error / difference between two samples / blocks to be estimated in a codec data refinement process of the current block. a) In one example, the error function may be: i.SAD, ii.SATD, iii.MR-SAD, iv.MR-SATD, v. Gradient information, vi.SSE / SSD, vii.MR-SSE / MR-SSD, viii. Weighted SAD / weighted MR-SAD, ix. Weighted SATD / weighted MR-SATD, x. Weighted SSD / weighted MR-SSD, xi. Weighted SSE / Weighted MR-SSE. b) In one example, the error function may be performed at a block level or a sub-block level. i. Alternatively, furthermore, the error function may be different for the two sub-blocks. ii. Alternatively, in addition, the final output of the estimated error of the block can be based on the output of the sub-blocks, for example, the sum of the outputs of the error function applied to each sub-block. 2. When the cost definition depends on the output of multiple functions, the following items may further apply: a) In one example, the cost function may rely on a linear weighted sum of multiple error functions. b) In one example, the cost function may rely on a non-linear weighted sum of multiple error functions. c) In one example, the cost function may further depend on the estimated bits for the side information. d) In one example, the cost function may be defined as: Where R represents the estimated bits for the side information, W i and E i denote the weight applied to the i-th error function and the output of the i-th error function, respectively. 3. Multiple refinement processes may be applied to one block, wherein at least two or more different cost functions are applied to at least two refinement processes. a) In one example, a first refinement process may be invoked using a first cost function. Based on the output of the first refinement process, a second cost function is further applied to a second refinement process. b) The above method can be applied to blocks based on template coding. 4. Whether to use multiple refinement processes and / or how to select one or more error functions and / or how to define the cost function and / or which samples to be involved in the error function may depend on the decoding information of the current block and / or the decoding information of neighboring (adjacent or non-adjacent) blocks of the current block. a) In one example, how to select one or more error functions and / or how to define the cost function may depend on the codec tools applied to the current block and / or the neighboring blocks of the current block. i. In one example, the codec is LIC. (i) In one example, SSD and mean removed SSD (MR-SSD) can be selectively utilized according to the LIC flag of the current block. a) In one example, if the LIC flag of the current block is false, the template matching cost measure can be SSD. b) In one example, if the LIC flag of the current block is true, the template matching cost measure can be MR-SSD. (ii) In one example, if MR-SAD / MR-SATD / MR-SSD is used for the template matching cost measurement, the linear function used in the LIC process may not be used when deriving the reference template. (iii) In one example, if MR-SAD / MR-SATD / MR-SSD is used for the bilateral matching cost measurement, the linear function used in the LIC process may not be used when deriving the reference block. b) In one example, it may depend on block size (eg, W, H), temporal layer, low latency check flag, etc. c) In one example, it may depend on whether the motion information of the current block is similar / identical to the motion information of the neighboring blocks. d) In one example, it may rely on a reference picture list and / or reference picture information. i. In one example, for list X, a first error function (eg, SAD / SSE) may be used, and for list Y (Y=1-X), a second error function (eg, MR-SAD / MR-SSE) may be used. ii. Alternatively, furthermore, the final cost may be based on the cost of each reference picture list. 5. The above method can be applied to any codec data refinement process, for example, for blocks based on template codecs, for blocks based on bilateral codecs (eg, DMVR in VVC). 2.16. Samples for image / video codec data refinement The term "block" may refer to a codec tree block (CTB), a codec tree unit (CTU), a codec block (CB), a CU, a PU, a TU, a PB, a TB, or a video processing unit including multiple samples / pixels, and a block may be rectangular or non-rectangular. In the present disclosure, the phrase "motion candidate" may refer to a merged motion candidate in a regular / extended merge list indicated by a merge candidate index, or an AMVP motion candidate in a regular / extended AMVP list indicated by an AMVP candidate index, or an AMVP motion candidate, or a merged motion candidate. In this disclosure, a motion candidate is referred to as "refined" if its motion information is modified based on information signaled from an encoder or derived at a decoder. For example, a motion vector may be refined by DMVR, FRUC, TM Merge, TMAMVP, TM GPM, TM CIIP, TM Affine, MMVD, GMVD, Affine MMVD, BDOF, etc. In the present disclosure, the phrase "codec data refinement" may refer to a refinement process to derive or refine a signaled / decoded / derived prediction mode, prediction direction, or signaled / decoded / derived motion information, prediction, and / or reconstructed samples for a block. In one example, the refinement process may include motion candidate reordering. In the following discussion, a "template-based" block may refer to a block that uses a template matching-based method to derive or refine codec information during the encoding / decoding process, such as motion derivation based on template matching, motion list reconstruction based on template matching, LIC, symbol prediction, block vector derivation based on template matching (e.g., used in IBC mode), DIMD, non-inter-frame (e.g., intra-frame) prediction based on template matching, etc. The template-based codec method can be combined with any other codec tools, such as MMVD, CIIP, GPM, FRUC, Affine, BDOF, DMVR, OBMC, etc. In another example, a "template-based" block may also refer to a block that uses (adjacent or non-adjacent) neighboring reconstructed samples based on a certain rule to derive or refine its decoding information, such as the DIMD method in 2.27 and the TIMD method in 2.29. In the following discussion, a "bilateral-based" block may refer to a block that uses a bilateral matching-based method to derive or refine codec information during the encoding / decoding process, such as motion derivation based on bilateral matching, motion list reconstruction based on bilateral matching, etc. The bilateral codec-based approach can be combined with any other codec tools, such as MMVD, CIIP, GPM, FRUC, Affine, DMVR, etc. W and H are the width and height of the current block (e.g., luminance block). W*H is the size of the current block (e.g., luminance block) 1. Error / cost estimation during codec data refinement may rely on reference samples corresponding to the current block (eg, the reference block used in bilateral matching) and reference samples corresponding to the template of the current block. a) Alternatively, it may rely on reference samples corresponding to the current block and samples in the current block template. b) In one example, the template may be neighboring samples (adjacent or non-adjacent) of the current block. 2. Multiple refinement processes can be applied to one block, and different templates are applied to at least two refinement processes. a) In one example, a first refinement process may be invoked using a first template. Based on the output of the first refinement process, a second template is further used in a second refinement process. b) In one example, the first template may cover more samples than the second template. c) In one example, the first template and the second template may contain at least one different sample. d) In one example, the first refinement process and the second refinement process may use different cost / error functions. 3. Whether to use multiple refinement processes, and / or how to select one or more error functions and / or how to define the cost function and / or which samples' decoding information is involved in the error function may depend on the current block and / or neighboring (neighboring or non-neighboring) blocks. a) In one example, how to select one or more error functions and / or how to define the cost function may depend on the codec tools applied to the current block and / or neighboring blocks. i. In one example, the codec is LIC. (i) In one example, SSD and mean-removed SSD (MR-SSD) may be selectively used according to the LIC flag of the current block. a) In one example, if the LIC flag of the current block is false, the template matching cost measure can be SSD. b) In one example, if the LIC flag of the current block is true, the template matching cost measure can be MR-SSD. (ii) In one example, if MR-SAD / MR-SATD / MR-SSD is used for the template matching cost measurement, the linear function used in the LIC process may not be used when deriving the reference template. (iii) In one example, if MR-SAD / MR-SATD / MR-SSD is used for the bilateral matching cost measurement, the linear function used in the LIC process may not be used when deriving the reference block. b) In one example, it may depend on block dimensions (eg, W, H), temporal layers, low latency check flags, etc. c) In one example, it may depend on whether the motion information of the current block and the neighboring blocks are similar / identical. d) In one example, a reference picture list and / or reference picture information may be relied upon. i. In one example, for list X, a first error function (eg, SAD / SSE) may be used, and for list Y (Y=1-X), a second error function (eg, MR-SAD / MR-SSE) may be used. ii. Alternatively, furthermore, the final cost may be based on the cost of each reference picture list. 4. In one example, LIC may be enabled for reference list X and disabled for reference list Y. a) In one example, the final prediction of the current block may be a weighted average of the LIC prediction from reference list X and the regular prediction from reference list Y. 5. The above method can be applied to any codec data refinement process, for example, for blocks based on template codecs, for blocks based on bilateral codecs (such as DMVR in VVC). 2.17. Adaptive Motion Candidate List It should be noted that the terms mentioned below are not limited to specific terms defined in existing standards. Any changes in codecs also apply. For example, the term "GPM" is used to represent any codec that derives two sets of motion information and uses the derived information and the partitioning pattern to obtain the final prediction. For example, TPM is also considered a GPM. It should be noted that the proposed method can be applied to the merge candidate list construction process for inter-frame codec blocks (e.g., translational motion), affine codec blocks, TM codec blocks, or IBC codec blocks; or other motion candidate list construction processes (e.g., ordinary AMVP list; affine AMVP list; IBC AMVP list; HMVP table). Cost functions other than template matching cost are also applicable for motion candidate re-ranking. W and H are the width and height of the current block (eg, luma block). 1. The template / bilateral matching cost C can be compared with another template matching cost before being calculated as f(C). a. In one example, f(C) = w*C, where w represents the cost factor. b. In one example, f(C) = w*C + u. c. In one example, f(C)=Shift((w*C), s). d. In one example, w and / or u and / or s are integers. e. In one example, the first template matching cost of the first motion candidate may be multiplied by a cost factor before being compared with the second template matching cost of the second motion candidate. f. In one example, the proposed cost factor for a motion candidate may depend on the position before the re-ranked candidate. i. In one example, a motion candidate at the i-th position may be assigned a cost factor that is smaller than a cost factor of a motion candidate at the j-th position, where j>i, eg, j=i+1. (i) In one example, the cost factor of the motion candidate at the i-th position is 4, and the cost factor of the motion candidate at the j-th position is 5. (ii) In one example, the cost factor of the motion candidate at the i-th position is 1, and the cost factor of the motion candidate at the j-th position is 5. ii. In one example, a motion candidate at the i-th position may be assigned a cost factor greater than a cost factor of a motion candidate at the j-th position, where j>i, eg, j=i+1. iii. In one example, motion candidates in the pth group (eg, containing M motion candidates) may be assigned a cost factor smaller than the cost factor of motion candidates in the qth group (eg, containing N motion candidates), where q>p, eg, q=p+1. (i) Alternatively, the motion candidates in the pth group (eg, containing M motion candidates) may be assigned a cost factor greater than the cost factor of the motion candidates in the qth group (eg, containing N motion candidates), where q>p, eg, q=p+1. (ii) In one example, M may be equal to N. For example, M=N=2. (iii) In one example, M may not be equal to N. For example, M=2, N=3. (iv) In one example, the cost factor of the motion candidates of the pth group is 4, and the cost factor of the motion candidates of the qth group is 5. (v) In one example, the cost factor of the motion candidates of the pth group is 1, and the cost factor of the motion candidates of the qth group is 5. iv. In one example, the cost factor may not be applied to sub-block motion candidates. v. In one example, the cost factor may not be applied to affine motion candidates. vi. In one example, the cost factor may not be applied to the SbTMVP motion candidates. g. In one example, the cost factors of motion candidates in a group / position may be adaptive. i. In one example, the cost factor of a motion candidate in a group / position may depend on the codec mode of the neighbors of the codec block. (i) In one example, the cost factor of a SbTMVP merge candidate may depend on the number of neighboring affine codec blocks. (ii) In one example, a neighbor codec block may include five spatial neighboring blocks (e.g. Figure 4 ) and / or temporal neighboring blocks (as shown Figure 7 at least one of the above (shown). (iii) In one example, when the spatial neighbor affine codec block (e.g. Figure 4 When the number of spatial neighbor affine codec blocks (such as Figure 4 ) is 1, the cost factor of the SbTMVP merge candidate can be 0.5; when the spatial neighbor affine codec block (such as Figure 4 ) is 2, the cost factor of the SbTMVP merge candidate may be 0.8; otherwise, the cost factor of the SbTMVP merge candidate may be 1 (which means remains unchanged). (iv) In one example, when the spatial neighbor affine codec block (e.g. Figure 4 When the number of spatial neighbor affine codec blocks (such as Figure 4 ) is 1, the cost factor of the SbTMVP merge candidate can be 0.5; when the spatial neighbor affine codec block (such as Figure 4 When the number of SbTMVP merge candidates is greater than or equal to 2, the cost factor of the SbTMVP merge candidate can be 0.8. (v) In one example, when the spatial neighbor affine codec block (e.g. Figure 4 ) is 0, the cost factor of the SbTMVP merge candidate can be 2; when the spatial neighbor affine codec block (such as Figure 4 ) is 1, the cost factor of the SbTMVP merge candidate can be 5; when the spatial neighbor affine codec block (such as Figure 4 ) is 2, the cost factor of the SbTMVP merge candidate may be 8; otherwise, the cost factor of the SbTMVP merge candidate may be 10 (wherein the cost factor of the affine merge candidate is 10). (vi) In one example, when the spatial neighbor affine codec block (e.g. Figure 4 ) is 0, the cost factor of the SbTMVP merge candidate can be 2; when the spatial neighbor affine codec block (such as Figure 4 ) is 1, the cost factor of the SbTMVP merge candidate can be 5; when the spatial neighbor affine codec block (such as Figure 4 When the number of merge candidates (as shown) is greater than or equal to 2 (where the cost factor of the affine merge candidate is 10), the cost factor of the SbTMVP merge candidate can be 8. 2. The subgroup size may be different for different encoding and decoding modes. a. The codec mode can include normal / sub-block / TM merge mode. i. For normal merge mode, the subgroup size may be K1 (eg, K1=5). ii. For the sub-block merging mode, the sub-group size may be K2 (eg, K2=3). iii. For TM merging mode, the subgroup size may be K3 (eg, K3=3). b. The subgroup size may be greater than or equal to the maximum number of subblock merging candidates defined in the sps / picture / slice header for the subblock merging mode (which means reordering the entire list together). c. The subgroup size may be greater than or equal to the maximum number of TM merge candidates defined in the sps / picture / slice header for TM merge mode (which means reordering the entire list together). d. The subgroup size for a codec mode may depend on the maximum number of motion candidates in the codec mode. e. The subgroup size for subblock merging mode can be adaptively dependent on the number of neighboring affine codec blocks. i. In one example, a neighbor codec block may include five spatial neighboring blocks (e.g. Figure 4 ) and / or temporal neighboring blocks (as shown Figure 7 at least one of the above (shown). ii. In one example, when the spatial neighbor affine codec block (such as Figure 4 ) is 0 or 1, the subgroup size of the sub-block merging mode can be 3; when the spatial domain neighbor affine codec block (such as Figure 4 When the number of sub-blocks (as shown) is greater than 1, the sub-group size of the sub-block merging mode may be 5; 3. The template size can be different for different encoding and decoding modes. a. The codec mode can include normal / sub-block / TM merge mode. i. For normal merge mode, the template size may be K1 (eg, K1=1). ii. For the sub-block merging mode, the template size may be K2 (eg, K2=1, 2, or 4). iii. For TM merge mode, the template size may be K3 (eg, K3=1). 4. Whether and / or how to reorder motion candidates may depend on the codec mode of the neighboring codec blocks. a. In one example, a neighbor codec block may include five spatial neighboring blocks (e.g. Figure 4 ) and / or temporal neighboring blocks (as shown Figure 7 at least one of the above (shown). b. When there is a regular merge mode (such as Figure 4 When the number of spatial neighbor codec blocks (as shown) is greater than or equal to K (for example, K=1), the candidates can be merged by reordering rules. c. When there is a sub-block merging mode (such as Figure 4 When the number of spatial neighbor codec blocks (as shown in FIG. 1 ) is greater than or equal to K (eg, K=1), the sub-block merging candidates may be reordered. d. When there is an affine merge mode (such as Figure 4 When the number of spatial neighbor codec blocks (as shown) is greater than or equal to K (for example, K=1), the affine merging candidates can be reordered. e. When there is an affine merge mode (such as Figure 4 When the number of spatial neighbor codec blocks (as shown) is greater than or equal to K (for example, K=1, 2 or 3), the SbTMVP merge candidates can be reordered. f. When there is a TM merge mode (such as Figure 4 When the number of spatial neighbor codec blocks (as shown in FIG. 1 ) is greater than or equal to K (eg, K=1), the TM merging candidates may be reordered. 5. The HMVP motion candidates in the HMVP table may be reordered based on template / bilateral matching, etc. a. In one example, HMVP candidates are assigned motion costs, and the HMVP candidates are adaptively reordered in descending order of the costs of the HMVP candidates. i. In one example, the HMVP candidate cost may be a template matching cost. b. In one example, HMVP motion candidates can be reordered before encoding and decoding blocks. i. In one example, HMVP motion candidates may be reordered before coding inter-codec. c. In one example, HMVP motion candidates may be reordered in different ways according to the coding information of the current block and / or neighboring blocks. General claims 6. Whether and / or how to apply the method disclosed above can be transmitted by a signal at the sequence level / picture group level / picture level / slice level / tile group level, such as in a sequence header / picture header / SPS / VPS / DPS / DCI / PPS / APS / slice header / tile group header. 7. Whether and / or how the methods disclosed above can be applied to signal transmission in PB / TB / CB / PU / TU / CU / VPDU / CTU / CTU row / slice / tile / sub-picture / other kinds of areas containing more than one sample or pixel. 8. Whether and / or how to apply the methods disclosed above may depend on the coded information, such as codec mode, block size, color format, single / dual tree partitioning, color component, slice / picture type. 2.18. Adaptive GPM Candidate List The term "block" may refer to a codec tree block (CTB), a codec tree unit (CTU), a codec block (CB), a CU, a PU, a TU, a PB, a TB, or a video processing unit including multiple samples / pixels, and a block may be rectangular or non-rectangular. It should be noted that the terms mentioned below are not limited to specific terms defined in existing standards. Any changes in codecs are also applicable. For example, the term "GPM" is used to represent any codec that derives two or more sets of motion information and uses the derived motion information and partitioning mode / weighting mask to obtain the final prediction. For example, TPM is also considered a GPM. It should be noted that the proposed method can be applied to the merge candidate list construction process for inter-frame codec blocks (e.g., translational motion), affine codec blocks, TM codec blocks, GPM codec blocks, or IBC codec blocks; or other motion candidate list construction processes (e.g., ordinary AMVP list; affine AMVP list; IBC AMVP list; HMVP table). Cost functions other than template matching cost are also applicable to motion candidate reordering. In the following, a template is a set of reconstructed / predicted samples of adjacent current blocks or non-adjacent current blocks. The reference samples of a template (i.e., a reference template) are the mapping of the template in a reference picture, which depends on the motion information of the current block. "Top template" refers to a template constructed by a set of reconstructed / predicted samples of adjacent current blocks or non-adjacent current blocks and their reference templates. "Left template" indicates a template constructed by a set of reconstructed / predicted samples of adjacent current blocks or non-adjacent current blocks and their reference templates. "Top template and left template" includes the top template and the left template. In the following, in one example, a GPM candidate list is referred to as an OGPM list, where the GPM candidates are directly derived from a rule merge list (before motion refinement based on or not based on template matching); a refined GPM candidate list, where the GPM candidates are refined by a first refinement method (e.g., template matching using an upper template), is referred to as an AGPM list; a refined GPM candidate list, where the GPM candidates are refined by a second refinement method (e.g., template matching using a left template), is referred to as an LGPM list; and a refined GPM candidate list, where the GPM candidates are refined by a third refinement method (e.g., template matching using left and upper templates), is referred to as a LAGPM list. W and H are the width and height of the current block (eg, luma block). 1. As proposed for a GPM codec block, the decoded candidate index may correspond to a candidate with a different candidate index in the candidate list for the GPM codec block. a. Alternatively, furthermore, the candidate list constructed for the GPM codec block may be reordered before being used, and the coded index corresponds to the reordered candidate list. b. Alternatively, furthermore, for the first type of GPM codec blocks, the candidate list may be reordered, while for the second type of GPM codec blocks, the candidate list may not be reordered. i. In one example, the first type is based on a template GPM codec block. ii. In one example, the second type is based on the GPM codec block of MMVD (eg, GMVD). c. Alternatively, furthermore, for the first type of GPM codec blocks, the candidate list may be reordered using a first rule, and for the second type of GPM codec blocks, the candidate list may be reordered using a second rule. d. The reordering method of GPM codec blocks can be the same as that of non-GPM codec blocks. i. The reordering method of GPM codec blocks can be different from that of non-GPM codec blocks. 2. For the proposed GPM codec block, the decoded candidate index may correspond to a candidate from the refined candidate list for the GPM codec block. a. Alternatively, furthermore, the candidate list constructed for the GPM codec block may first be refined before being used, and the decoded index corresponds to the refined candidate list. b. Alternatively, furthermore, for the first type of GPM codec blocks, the candidate list may be refined, while for the second type of GPM codec blocks, the candidate list may not be refined. i. In one example, the first type is based on a template GPM codec block. ii. In one example, the second type is based on the GPM codec block of MMVD (eg, GMVD). c. Alternatively, furthermore, for a first type of GPM codec blocks, the candidate list may be refined using a first rule, and for a second type of GPM codec blocks, the candidate list may be refined using a second rule. d. The improvement method of the GPM codec block can be the same as that of the non-GPM codec block. i. The refinement method of GPM codec blocks can be different from that of non-GPM codec blocks. 3. In one example, GPM candidates can be divided into subgroups. Whether and / or how to GPM candidate reordering may depend on subgroups of GPM candidates. a. In one example, only the first subgroup may be reordered. b. In one example, eventually one subgroup cannot be reordered. c. In one example, the last subgroup cannot be reordered. However, if the last subgroup is also the first subgroup, it can be reordered. d. Different subgroups can be reordered individually. e. Two candidates in different subgroups cannot be compared and / or reordered. f. If the first subgroup precedes the second subgroup, then after reordering, the first candidate in the first subgroup must be placed before the second candidate in the second subgroup. 4. In one example, GPM candidates that are not included in the reordering process may be processed in a specified manner. a. In one example, candidates that do not need to be reordered are arranged in the merge candidate list according to the original order. b. In one example, candidates that are not to be reordered may be placed after candidates that are to be reordered. c. In one example, candidates that are not to be reordered may be placed before candidates that are to be reordered. 5. The list of GPM candidates to be reordered can be found at Case 1: A first candidate list prepared for two GPM partitions is used to derive a separate GPM candidate list for each GPM partition. Case 2: Second GPM candidate list for each GPM partition Usually the second GPM candidate comes from the first candidate list. a. In one example, in case 1, the reordering method may be the same as that used for a conventional merge candidate list. b. In one example, in case 1, if the corresponding candidate is bi-predicted, the template matching method in the re-ranking method can be performed in a bi-predictive manner. c. In one example, in case 2, the template matching method in the reordering method cannot be performed in a bidirectional predictive manner. d. In one example, in case 2, the reordering method may be the same for all GPM partitions. e. In one example, in case 2, the reordering method may be different for different GPM partitions. 6. In the above example, the GPM codec block may be a GPM codec block with a merge mode or a GPM codec block with an AMVP mode. a. Alternatively, furthermore, the above merge candidates may be replaced by AMVP candidates. General claims 7. Whether and / or how to apply the method disclosed above can be transmitted by a signal at the sequence level / picture group level / picture level / slice level / tile group level, such as in a sequence header / picture header / SPS / VPS / DPS / DCI / PPS / APS / slice header / tile group header. 8. Whether and / or how the methods disclosed above can be applied to signal transmission in PB / TB / CB / PU / TU / CU / VPDU / CTU / CTU row / slice / tile / sub-picture / other kinds of areas containing more than one sample or pixel. 9. Whether and / or how to apply the methods disclosed above may depend on the coded information, such as codec mode, block size, color format, single / dual tree partitioning, color component, slice / picture type. 2.19. Adaptive GPM Candidate List The term "block" may refer to a codec tree block (CTB), a codec tree unit (CTU), a codec block (CB), a CU, a PU, a TU, a PB, a TB, or a video processing unit including multiple samples / pixels, and a block may be rectangular or non-rectangular. It should be noted that the terms mentioned below are not limited to specific terms defined in existing standards. Any changes in codecs are also applicable. For example, the term "GPM" is used to represent any codec that derives two or more sets of motion information and uses the derived motion information and partitioning mode / weighting mask to obtain the final prediction. For example, TPM is also considered a GPM. It should be noted that the proposed method can be applied to the merge candidate list construction process for inter-frame codec blocks (e.g., translational motion), affine codec blocks, TM codec blocks, GPM codec blocks, or IBC codec blocks; or other motion candidate list construction processes (e.g., ordinary AMVP list; affine AMVP list; IBC AMVP list; HMVP table). Cost functions other than template matching cost are also applicable to motion candidate reordering. In the following, a template is a set of reconstructed / predicted samples of adjacent current blocks or non-adjacent current blocks. The reference samples of a template (i.e., a reference template) are the mapping of the template in a reference picture, which depends on the motion information of the current block. "Top template" refers to a template constructed by a set of reconstructed / predicted samples of adjacent current blocks or non-adjacent current blocks and their reference templates. "Left template" indicates a template constructed by a set of reconstructed / predicted samples of adjacent current blocks or non-adjacent current blocks and their reference templates. "Top template and left template" includes the top template and the left template. In the following, in one example, a GPM candidate list is referred to as an OGPM list, where the GPM candidates are directly derived from a rule merge list (before motion refinement based on or not based on template matching); a refined GPM candidate list, where the GPM candidates are refined by a first refinement method (e.g., template matching using an upper template), is referred to as an AGPM list; a refined GPM candidate list, where the GPM candidates are refined by a second refinement method (e.g., template matching using a left template), is referred to as an LGPM list; and a refined GPM candidate list, where the GPM candidates are refined by a third refinement method (e.g., template matching using left and upper templates), is referred to as a LAGPM list. Regarding the types of GPM candidates in the original GPM candidate list, the GPM candidates derived in the first step of the GPM candidate list construction process in Section 2.29 are called GPM parity-based candidates; the GPM candidates derived in the second step of the GPM candidate list construction process in Section 2.29 are called anti-parity-based GPM candidates; and the GPM candidates derived in the third step of the GPM candidate list construction process in Section 2.29 are called "GPM-filling candidates." W and H are the width and height of the current block (eg, luma block). 1. In one example, if the codec mode is GPM, the merge candidates may be reordered. a. In one example, merge candidates in the OGPM list may be reordered. i. In one example, at least two merge candidates in the OGPM list may be reordered. ii. In one example, at least one type of template may be used for OGPM list reordering. iii. Alternatively, merge candidates in the OGPM list cannot be reordered. iv. In one example, the first type of template may include only neighboring samples to the left of the current block. v. In one example, the second type of template may include only neighboring samples above the current block. vi. In one example, the third type of template may include neighboring samples to the left and above the current block. vii. The reordering process can be called after the parsing process but before the MV reconstruction process. b. In one example, merge candidates in the AGPM list may be reordered. i. In one example, at least two merge candidates in the AGPM list may be reordered. ii. In one example, at least one type of template may be used for AGPM list reordering. iii. In one example, the first type of template may include only neighboring samples above the current block. iv. In one example, the second type of template may include neighboring samples to the left and above the current block. c. In one example, the merge candidates in the LGPM list may be reordered. i. In one example, at least two merge candidates in the LGPM list may be reordered. ii. In one example, at least one type of template may be used for LGPM list reordering. iii. In one example, the first type of template may include only neighboring samples to the left of the current block. iv. In one example, the second type of template may include neighboring samples to the left and above the current block. d. In one example, merge candidates in the LAGPM list may be reordered. i. In one example, at least two merge candidates in the LAGPM list may be reordered. ii. In one example, at least one type of template may be used for LAGPM list reordering. iii. In one example, the first type of template may include only neighboring samples to the left of the current block. iv. In one example, the second type of template may include only neighboring samples above the current block. v. In one example, the third type of template may include neighboring samples to the left and above the current block. e. In one example, whether and / or how to reorder merge candidates in the GPM list may depend on codec information. i. In one example, whether to reorder merge candidates in the GPM list may depend on whether template matching based motion refinement is applied to a GPM partition or two GPM partitions (ie, GPM-encoded CUs). (i) For example, if the motion of a GPM partition or two GPM partitions (ie, GPM-encoded CUs) is not based on template matching refinement (eg, template matching flag equals false), the corresponding GPM lists may not be reordered. a) For example, if the GPM partition is encoded using merge candidates in the OGPM list (eg, no motion refinement is applied), the merge candidates in the OGPM list may not be reordered. (ii) For example, if a GPM partition or the motion of two GPM partitions (ie, a GPM-encoded CU) is refined based on template matching (eg, template matching flag equals true), the corresponding GPM lists may be reordered. a) For example, if the GPM partition is encoded using the merge candidates in the AGPM list (eg, a template matching motion refinement method using an upper template is applied), the merge candidates in the AGPM list may be reordered. b) For example, if the GPM partition is encoded using the merge candidates in the LGPM list (eg, a template matching motion refinement method using a left template is applied), the merge candidates in the LGPM list may be reordered. c) For example, if the GPM partition is encoded using the merge candidates in the LAGPM list (eg, a template matching motion refinement method using the left template and the upper template is applied), the merge candidates in the LAGPM list may be reordered. ii. In one example, how to reorder the merge candidates in the GPM list may depend on the GPM partition information (eg, partition mode, partition squareness, partition distance, etc.). (i) For example, the above template can be used to reorder the merge candidates when the current GPM partition is partitioned by the first partitioning dimension (or partitioning mode, or partitioning distance, etc.). (ii) For example, the left template can be used to reorder the merge candidates when the current GPM partition is partitioned by the second partitioning dimension (or partitioning mode, or partitioning distance, etc.). (iii) For example, the left template and the upper template may be used to reorder the merge candidates when the current GPM partition is partitioned by a third partitioning dimension (or partitioning pattern, or partitioning distance, etc.). (iv) For example, a template type corresponding to the first division squareness / the second division squareness / the third division squareness (or the division pattern, or the division distance, etc.) may be specified. (v) For example, at least one lookup table (i.e., mapping table) is used to map which specified partitioning dimension (or partitioning mode, or partitioning distance, etc.) corresponds to which type of template (e.g., upper template, left template, or upper template and left template). f. In one example, merge candidates in the OGPM list may not be reordered, and merge candidates in the AGPM list and / or the LGPM list and / or the LAGPM list may be reordered. 2. Merge candidates can be adaptively reordered in the final GPM candidate list according to one or some criteria. a. In one example, the GPM candidate list can be i.OGPM list, ii.AGPM List, iii.LGPM list, iv.LAGPM list. b. GPM candidates can be divided into several subgroups. i. For example, the number of GPM candidates in a sub-group (such as X=3 or 5 or any other integer value) may be predetermined. c. In one example, part or all of the current GPM candidate list construction process is called first, and then the candidates in the GPM list are reordered. i. Alternatively, the candidates in the first subgroup may be reordered and added before the candidates in the second subgroup, wherein the first subgroup is added before the second subgroup. ii. The construction process can include pruning methods. d. In one example, merge candidates may be adaptively reordered before being retrieved. i. In one example, the process of adaptively arranging merge candidates may be processed before obtaining merge candidates to be used in the motion compensation process. e. The criterion can be based on template matching cost. i. In one example, the cost function between the current template and the reference template can be (i) SAD / MR-SAD, (ii) SATD / MR-SATD, (iii) SSD / MR-SSD, (iv) SSE / MR-SSE, (v) Weighted SAD / Weighted MR-SAD, (vi) Weighted SATD / weighted MR-SATD, (vii) Weighted SSD / Weighted MR-SSD, (viii) Weighted SSE / weighted MR-SSE, (ix) Gradient information. 3. When deriving two motions for two geometric partitions, the process can be a. In one example, if TM is not applied to a partition, the motion can be derived based on the signaling of the merge index from the OGPM list / reordered OGPM list. b. In one example, if TM is applied to a partition, motion can be derived based on the signaled merge index from the AGPM list / reordered AGPM list or LGPM list / reordered LGPM list or LAGPM list / reordered LAGPM list, which depends on the partition dimension and the partition index. i. In one example, if the partition squareness is X (eg, 0), then for one partition, the AGPM list / reordered AGPM list will be used; for the second partition, the LAGPM list / reordered LAGPM list will be used. c. In one example, if TM is applied to a partition, the motion can be derived from the signaling of the merge index from the AGPM list / reordered AGPM list. d. In one example, if TM is applied to a partition, the motion can be derived based on the signaled merge index from the LGPM list / reordered LGPM list. e. In one example, if TM is applied to one partition, the motion can be derived based on the signaled merge index from the LAGPM list / reordered LAGPM list. 4. Whether and / or how to reorder GPM candidates may depend on the category of the GPM candidates. a. In one example, candidates based only on GPM parity may be reordered. b. In one example, candidates based only on GPM parity and GPM anti-parity may be reordered. c. In one example, candidates for GPM population may not be reordered. d. In one example, two candidates in different GPM lists cannot be compared and / or reordered. e. In one example, only the top N GPM candidates may be reordered. i. In one example, N is set equal to 5. 5. In the above example, the GPM codec block may be a GPM codec block with a merge mode or a GPM codec block with an AMVP mode. a. Alternatively, furthermore, the above merge candidates may be replaced by AMVP candidates. General claims 6. Whether and / or how to apply the method disclosed above can be transmitted by a signal at the sequence level / picture group level / picture level / slice level / tile group level, such as in a sequence header / picture header / SPS / VPS / DPS / DCI / PPS / APS / slice header / tile group header. 7. Whether and / or how the methods disclosed above can be applied to signal transmission in PB / TB / CB / PU / TU / CU / VPDU / CTU / CTU row / slice / tile / sub-picture / other kinds of areas containing more than one sample or pixel. 8. Whether and / or how to apply the methods disclosed above may depend on the coded information, such as codec mode, block size, color format, single / dual tree partitioning, color component, slice / picture type. 2.20. Hash-based motion estimation for screen content encoding and decoding The VTM reference software uses hash-based motion estimation to handle the sometimes large and irregular motion in screen content. For each reference picture, a hash table corresponding to block sizes from 4x4 to 64x64 is generated using a bottom-up approach as shown below: – For each 2x2 block, a block hash value is calculated directly from the original sample values ​​(using luma samples if 4:2:0 chroma format, and using luma and chroma sample values ​​if 4:4:4 chroma format). A cyclic redundancy check (CRC) value is used as the hash value. – For 4x4, 8x8, 16x16, 32x32, and 64x64 blocks, the hash value of the current block is the CRC value calculated based on the CRC values ​​of its four sub-blocks. In order to enable efficient search for the matched blocks, an inverted index structure is used, in which hash values ​​are used to index into the table, and the table entries cover all blocks with the same hash value as the corresponding table index. The blocks corresponding to a given table index are stored as a linked list. Two CRC values ​​are calculated for each block, one is a 16-bit hash and the other is a 24-bit hash. The two hash values ​​are calculated in a similar manner, but different CRC truncation polynomials are used. The first 16-bit CRC value is used as an inverted index. The second 24-bit hash value is stored with the block to resolve hash conflicts in the case of finding multiple matching blocks. In order to reduce the length of the hash table, the hash values ​​of all "simple" blocks (defined as blocks with only one sample value in each row or column) are excluded from the hash table. In motion estimation, if the current block is a square block (except for 128x128 blocks), its hash value is calculated. Then, the encoder queries the corresponding hash table. If a hash match is found, the matched block is used as a reference. If the current block is a rectangular block of size NxM (assuming M>N without loss of generality), it is divided into several non-overlapping square sub-blocks of size NxN. Fig. 27 An example diagram 2700 showing motion estimation for a rectangular block with square sub-block hash values ​​is shown. Fig. 27 An example is shown in . The encoder will find the first non-simple square sub-block and calculate its hash value. The encoder looks up the hash value of this NxN square sub-block on the hash table corresponding to the NxN block size. One or more matched reference blocks are considered as reference block candidates. For each matched reference block candidate, the encoder will continue to check the remaining square sub-blocks (i.e. Fig. 27Whether the hash value of the square sub-block (the white area behind the first non-simple square sub-block depicted in ) is equal to the hash value of the square sub-blocks adjacent to the reference block candidate. If the hash values ​​of all square sub-blocks match, the reference block candidate will be regarded as a valid reference block. For inter codecs, a hash-based motion search is performed before testing all codec modes. In addition, the encoder will reuse the MV of the hash mode as a starting point candidate in the normal motion estimation process. If a hash-based motion vector exists, which indicates that the block most likely covers the screen content, fractional motion estimation is skipped. To speed up the encoder, the skip and merge parts of the ETM_MERGE_SKIP, ETM_AFFINE and ETM_MERGE_GPM modes and the codec modes outside of the finer granularity block partitioning are skipped if all of the following conditions are met: – Current block sizes are 64x64, 128x64, or 64x128. – Find the same reference block in the reference image. – The QP of the reference picture is not greater than that of the current picture. 2.21. Chroma Scaling and Luma Mapping (LMCS) In VVC, a codec tool called Chroma Scaling and Luma Mapping (LMCS) is added as a new processing block before loop filtering. LMCS has two main components: 1) In-loop mapping of luma components based on an adaptive piecewise linear model; 2) For chroma components, luma-dependent chroma residual scaling is applied. Fig.28 An example graph 2800 of an example luma mapping with a chroma scaling architecture 2800 is shown. Fig.28 The LMCS architecture is shown from the decoder perspective. Fig.28 The light blue shaded blocks in indicate where processing is applied in the mapped domain; these include inverse quantization, inverse transform, luma intra prediction, and the addition of the luma prediction to the luma residual. Fig.28 The non-shaded blocks in indicate where processing is applied in the original (i.e., non-mapped) domain; these include loop filtering such as deblocking, ALF and SAO, motion compensated prediction, chroma intra prediction, chroma prediction added together with chroma residual, and storing the decoded picture as a reference picture. Fig.28 The light yellow shaded blocks in the figure are new LMCS functional blocks, which contain the forward and reverse mapping of the luma signal and the luma-dependent chroma scaling process. Like most other tools in VVC, LMCS can be enabled / disabled at the sequence level using the SPS flag. 2.21.1 Brightness Mapping with a Piecewise Linear Model The in-loop mapping of the luminance component adjusts the dynamic range of the input signal by redistributing the codewords within the dynamic range to improve compression efficiency. Luminance mapping uses a forward mapping function FwdMap and a corresponding inverse mapping function InvMap. The FwdMap function uses a piecewise linear model signal transmission with 16 equal segments. The InvMap function does not require signal transmission, but is derived from the FwdMap function. The luminance mapping model is signaled in the adaptive parameter set (APS) syntax structure with aps_params_type set to 1 (LMCS_APS). A maximum of 4 LMCS APSs can be used in a coded video sequence. Only 1 LMCS APS can be used for a picture. The luminance mapping model is signaled using a piecewise linear model. The piecewise linear model divides the dynamic range of the input signal into 16 equal segments, and for each segment, its linear mapping parameters are represented by the number of codewords assigned to the segment. Take 10-bit input as an example. Each of the 16 segments will be assigned 64 codewords by default. The signaled number of codewords is used to calculate the scaling factor and adjust the mapping function accordingly for the segment. At the slice level, the LMCS enable flag is signaled to indicate whether the LMCS process is enabled. Fig.28 If LMCS is enabled for the current slice, aps_id is signaled in the slice header to identify the APS that carries the luma mapping parameters. Each i-th fragment of the FwdMap piecewise linear model, i=0...15, is defined by two input pivot points InputPivot[] and two output (mapping) pivot points MappdPivot[]. InputPivot[] and MappdPivot[] are calculated as follows (assuming 10-bit video): 1)OrgCW=64. 2) For i=0:16, InputPivot[i]=i*OrgCW. 3) For i=0:16, the calculation of MappdPivot[i] is as follows: MappdPivot[0] = 0; For (i=0; i<16; i++) MappdPivot[i+1]=MappdPivot[i]+signalingCW[i] The signaling CW[i] is the signaling number of the codeword for the i-th segment. like Fig.28As shown, for the inter-frame codec block, motion compensation prediction is performed in the mapping domain. In other words, after the motion compensation prediction block is calculated based on the reference signal in the DPB, the FwdMap function is applied to map the luminance prediction block in the original domain to the mapping domain, i.e., Y′ pred =FwdMap(Y pred ). For intra-frame coded blocks, the FwdMap function is not applied because intra-frame prediction is performed in the mapping domain. r Afterwards, the InvMap function is applied to convert the reconstructed brightness values ​​in the mapped domain back to the reconstructed brightness values ​​in the original domain The InvMap function is applied to both intra-coded and inter-coded luminance blocks. The brightness mapping process (forward mapping and / or inverse mapping) can be implemented using a lookup table (LUT) or using dynamic calculation. If LUT is used, FwdMapLUT and InvMapLUT can be pre-calculated and pre-stored to be used at the tile group level, and the forward mapping and inverse mapping can be simply implemented as FwdMap(Y pred )=FwdMapLUT[Y pred ] and InvMap(Y r )=InvMapLUT[Y r ]. Alternatively, dynamic calculation can be used. Take the forward mapping function FwdMap as an example. In order to find out the fragment to which the brightness sample belongs, the sample value is shifted right by 6 bits (corresponding to 16 equal fragments). Then, the linear model parameters of the fragment are retrieved and applied dynamically to calculate the mapped brightness value. Let i be the fragment index, a1 and a2 are InputPivot[i] and InputPivot[i+1] respectively, and b1 and b2 are MappdPivot[i] and MappdPivot[i+1] respectively. The calculation of the FwdMap function is as follows: FwdMap(Y pred )=((b2-b1) / (a2--a1))*(Y pred-a1)+b1InvMap function can be calculated on the fly in a similar way. Typically, the fragment sizes in the mapping domain are not equal, so the most straightforward inverse mapping process requires comparison to find out which fragment the current sample value belongs to. This comparison increases the complexity of the decoder. Therefore, VVC imposes a two-stream constraint on the value of the output pivot point MappdPivot[i] as shown below. Assume that the range of the mapping domain (for 10-bit video, this range is [0,1023]) is divided into 32 equal fragments. If MappdPivot[i] is not a multiple of 32, then MappdPivot[i+1] and MappdPivot[i] cannot belong to the same fragment among the 32 equally sized fragments, that is: MappdPivot[i+1]>>(BitDepthY-5) is not equal to MappdPivot[i]>>(BitDepthY-5). Due to this bitstream constraint, the InvMap function can also be implemented using a simple right shift of 5 bits (corresponding to 32 equally sized fragments) to find out which fragment a sample value belongs to. 2.21.2 Luma-dependent Chroma Residual Scaling Chroma residual scaling aims to compensate for the interaction between luma signals and their corresponding chroma signals. Whether chroma residual scaling is enabled is also signaled at the slice level. If luma mapping is enabled, an additional flag is signaled to indicate whether luma-dependent chroma residual scaling is enabled. When luma mapping is not used, luma-dependent chroma residual scaling is disabled. In addition, luma-dependent chroma residual scaling is always disabled for chroma blocks with area less than or equal to 4. Chroma residual scaling depends on the average of the reconstructed neighboring luma samples at the top and / or left of the current VPDU. If the current CU is Inter-128x128, Inter-128x64, and Inter-64x128, the chroma residual scaling factor derived for the CU associated with the first VPDU is used for all chroma transform blocks in this CU. avgYr represents the average of the reconstructed neighboring luma samples (see Fig.28 ). ScaleInv The value of is calculated as follows: 1) According to the InvMap function, find the index Y of the piecewise linear model to which avgYr belongs Idx . 2) C ScaleInv =cScaleInv[Y Idx ], where cScaleInv[] is a pre-computed 16-segment LUT based on the value of SignalledCW[i] and the offset values ​​in APS used for the chroma residual scaling process. Unlike brightness mapping, C ScaleInv is a constant value for the entire chrominance block. ScaleInv , the chroma residual is scaled as follows: Encoder side: CResScale =C Res *C Scale =C Res / C ScaleInv Decoder side: C Res =C ResScale / C Scale =C ResScale *C ScaleInv 2.21.3 Encoder-side LMCS parameter estimation A non-normative reference implementation is provided in the VTM encoder to estimate the LMCS model parameters. Since the VTM anchor handles SDR, HDR PQ and HDR HLG differently, the reference algorithm in VTM13 is designed differently for SDR, HDR PQ and HDR HLG sequences. For SDR and HDR HLG sequences, the encoder algorithm is based on local luminance variance and optimized for the PSNR metric. For HDR PQ sequences, the encoder algorithm is based on luminance values ​​and optimized for the wPSNR (weighted PSNR) metric. 2.22. About pairwise merge candidates Fig.29 An example diagram 2900 is shown showing example pairs in merge candidate reordering ARMC-TM and additional pairwise candidates after reordering. In this context, pairwise candidates are removed from conventional merge candidate derivation. When merge candidates are not repeated, merge candidates are added during adaptive reordering of merge candidates using template matching (ARMC-TM), such as Fig.29 As shown in Figure 2. Pairs are constructed using the two first re-ranked candidates. The number of re-ranked candidates is the same as in the ECM2.0 implementation and the number of template matching costs calculated. When the reference frames of the first and second re-ranked candidates are the same, the paired candidates are also restricted to using only the average candidate. Furthermore, if the created pairs are not duplicates, each merge candidate in the unreordered subgroup will be replaced by the pair between the first candidate and that candidate. 2.23. IBC Movement Candidates The following detailed implementation examples should be considered as examples to explain the general concept. These embodiments should not be interpreted narrowly. In addition, these embodiments can be combined in any way. The term "block" may refer to a codec tree block (CTB), a codec tree unit (CTU), a codec block (CB), a CU, a PU, a TU, a PB, a TB, or a video processing unit including multiple samples / pixels, and a block may be rectangular or non-rectangular. For IBC-coded blocks, a block vector (BV) is used to indicate the displacement from the current block to a reference block that has been reconstructed in the current picture. W and H are the width and height of the current block (eg, luma block). The non-adjacent spatial candidates of the current codec block are the adjacent spatial candidates of the virtual block in the i-th search round (such as Fig. 9 As shown in FIG. 1 ). The calculation formulas for the width and height of the virtual block used for the i-th round search are: newWidth = i×2×gridX+W, newHeight = i×2×gridY+H. Obviously, if the search round i is 0, the virtual block is the current block. In the following, BV predictors are also BV candidates. Skip mode is also merge mode. BV candidates can be divided into several groups according to certain criteria. Each group is called a subgroup. For example, we can take adjacent spatial and temporal BV candidates as the first subgroup and the remaining BV candidates as the second subgroup; in another example, we can also take the first N (N≥2) BV candidates as the first subgroup, the following M (M≥2) BV candidates as the second subgroup, and the remaining BV as the candidates of the third subgroup. About the use of BV candidates 1. If a BV candidate (eg, a BV search point or a BV predictor) is invalid, the BV candidate is prohibited from being used in the encoding and decoding process of the block. a. In one example, only when a BV candidate is valid can it be used in the encoding / decoding process of a block. i. For example, only when a BV candidate is valid, it can be used for BV search or BV prediction. b. In one example, whether to use a BV candidate in the encoding / decoding process of a block may depend on a validity check of the BV candidate. i. In one example, before inserting a new BV candidate into the BV candidate list, a validity check of the BV candidate needs to be performed. c. Only when a BV candidate is valid can it be inserted into the IBC candidate list. i. In one example, the IBC candidate list may be an IBC merge candidate list. ii. In one example, the IBC candidate list may be an IBC AMVP candidate list. iii. In one example, the IBC candidate list may be an IBC template matching candidate list. iv. In one example, the IBC candidate list may be an intra template matching candidate list. v. In one example, the IBC candidate list may be an IBC MMVD candidate list. vi. In one example, the IBC candidate list may be an IBC GPM candidate list. vii. In one example, the IBC candidate list may be an IBC TPM candidate list. viii. In one example, the IBC candidate list can be any list related to IBC-encoded blocks, ie, the same procedure is adopted. (i) Optionally, whether to allow invalid BV candidates to be added to the IBC candidate list may depend on decoding information (eg, IBC mode). d. Only when a BV candidate is valid, it can be used for hash-based search for IBC. e. Only when a BV candidate is valid, it can be used for block matching based IBC local search. f. Only when a BV candidate is valid, it can be used for intra template matching. g. In addition, optionally, the above-mentioned BV candidates may be those BV candidates from specific neighboring blocks (eg, adjacent or non-adjacent) or the HMVP table, or some virtual candidates generated from these BV candidates. h. In addition, optionally, the above BV candidates may exclude some default candidates (eg, a default zero vector). i. In addition, optionally, when a BV candidate is marked as invalid, a virtual candidate derived from the invalid BV candidate may be used instead. i. In one example, virtual candidates may be derived by adding an offset to an invalid BV candidate. ii. In one example, virtual candidates may be derived by applying a pruning function to invalid BV candidates. Verification Checks for BV Candidates 2. In one example, a BV candidate may be determined to be valid when it satisfies one of the following conditions or a combination of at least one of the following conditions. a. The corresponding reference block has been reconstructed in the current image. b. The corresponding reference block is located in the same CTU row as the current block. c. The corresponding reference block is located in the same tile / sub-picture as the current block. d. The corresponding reference block is located in the same stripe as the current block. e. The BV candidate satisfies the block vector constraints (such as those described in 2.4.2 and 2.4.3). f. The BV candidate satisfies the IBC virtual buffer conditions (such as those described in 2.4.4). 3. In one example, when a BV candidate violates one of the conditions in item 2 or a combination of at least one of the conditions, it may be determined to be invalid. About BV Candidate List 4. BV candidates can be derived / obtained from non-neighboring blocks. a. In one example, the distance between the non-adjacent spatial candidate and the current codec block can be based on the width and height of the current codec block (e.g. Fig. 9 or Fig.10 , gridX=W, gridY=H). i. Alternatively, the distance between the non-adjacent spatial candidates and the current codec block may be a multiple of a constant value. (i) For example, the multiplication factor depends on the search round index (eg, the multiplication factor for the i-th search round is i), and gridX=M, gridY=N (M and N are constant values). b. In one example, non-adjacent spatial candidates may be inserted after the TMVP candidate. i. Alternatively, non-adjacent spatial candidates may be inserted after adjacent spatial candidates and before TMVP candidates. 5. BV candidates can come from spatial neighboring blocks, temporal neighboring blocks, HMVP, pairwise and / or STMVP candidates. Fig. 30A An example graph 3000 showing spatial candidate locations is shown. Fig. 30B An example graph 3050 showing temporal candidate locations is shown. a. In one example, a spatial candidate may consist of adjacent and / or non-adjacent spatial candidates. i. In one example, the adjacent spatial domain candidates may be composed of left and / or above and / or above right and / or below left and / or above left spatial domain candidates (example in Fig. 30A ). b. In one example, for a TMVP candidate, such as Fig. 30B ), the position for the temporal candidate is selected between candidates C0 and C1. If the CU at position C0 is not available, is intra-coded, is outside the current row of the CTU or its BV is invalid for the current block, position C1 is used. Otherwise, position C0 is used for the derivation of the TMVP candidate. i. Alternatively, for TMVP candidates, you can use Fig. 30B Both candidates C0 and C1 are shown. (i) For example, the order is C0->C1. (ii) Alternatively, the order is C1->C0. c. In one example, for paired candidates, paired average candidates are generated by averaging predetermined candidate pairs in an existing BV candidate list, and the predetermined pairs are defined as {(0,1), (0,2), (1,2), (0,3), (1,3), (2,3)}, where the number represents the index of the BV candidate pair in the BV candidate list. i. In one example, the number of pairwise candidates is P. P is an integer from 0 to 6. ii. In one example, pairwise candidates may be inserted after the HMVP. d. In one example, the STMVP candidate is generated by averaging the predetermined E spatial domain BV candidate and the predetermined G temporal domain BV candidate. i. In one example, E is less than or equal to the number of spatial candidates (F) inserted into the current BV candidate list before STMVP. ii. In one example, the predetermined E spatial domain BV candidate may be the first E spatial domain BV candidate among the F spatial domain candidates that is inserted into the current BV candidates before the STMVP. (i) Alternatively, the predetermined E airspace BV candidate may be a selected E airspace BV candidate among the F airspace candidates that is inserted into the current BV candidate list before the STMVP. iii. In one example, E is 2 and G is 1. iv. In one example, the STMVP may be inserted before the top left spatial BV candidate. v. In one example, STMVP can be inserted after the pairwise candidates. e. In one example, the order of BV candidate insertion is adjacent space->HMVP->paired. f. In one example, the BV candidate insertion order is adjacent spatial domain -> temporal domain -> HMVP -> paired. g. In one example, the BV candidate insertion order is adjacent spatial domain -> temporal domain -> non-adjacent spatial domain -> HMVP -> in pairs. h. In one example, the order of BV candidate insertion is adjacent spatial domain > non-adjacent spatial domain > HMVP -> paired. i. In one example, the BV candidate insertion order is adjacent spatial domain (STMVP is inserted before the spatial domain BV candidate) -> temporal domain -> non-adjacent spatial domain -> HMVP -> paired. 6. The BV candidate list may also consist of BV candidates that have been pruned. a. In one example, if a BV candidate is invalid in the sense of the 3rd bullet point, it can be converted to a valid BV according to the given rules and then inserted into the BV candidate list. i. In one example, a BV candidate may be converted to the closest valid BV candidate. ii. In one example, a BV candidate may be converted from a predetermined set of BV candidates to the closest valid BV candidate. b. In one example, if a non-zero BV candidate is invalid, it can be pruned to the closest valid BV and then inserted into the BV candidate list. c. In one example, if a non-zero BV candidate is invalid, the closest valid BV can be pruned from the predetermined BV candidate set and then inserted into the BV candidate list. i. In one example, the predetermined BV candidate set may consist of (-m*W, 0), (0, -n*H), (-m*H, 0), (0, -n*W). Both m and n are positive values. d. In one example, the pruned BV candidate may be inserted after the un-pruned BV candidate. 7. The BV candidate list can be used as the IBC merge / AMVP candidate list. a. Alternatively, the BV candidate list can be used to derive the IBC merge / AMVP candidate list. i. In one example, for the IBC merge mode, the first R entries of the BV candidate list will be used to construct the IBC merge candidate list; for the IBC AMVP mode, the first S entries of the BV candidate list will be used to construct the IBC AMVP candidate list. (i) In one example, R is 6 and S is 2. 8. In one example, sub-block based temporal block vector prediction (SbTBVP) may be supported as a BV candidate or BV prediction mode. Fig.31 An example diagram 3100 is shown, which shows the derivation of a sub-CUbv motion field from the corresponding collocated sub-CU by applying the motion offset of the spatial neighbors. a. Similar to SbTMVP, SbTBVP uses the BV motion field in the co-located picture to improve block vector prediction and IBC merge mode for CU in the current picture. TMVP uses the same co-located picture as used for SbTBVP. b. In one example, SbTBVP applies motion shifting before obtaining temporal BV information from the co-located picture, where the motion shifting is obtained from the motion vector of one of the spatial neighboring blocks of the current CU (example in Fig.31 ). i. In one example, if A1 has a motion vector that uses the co-located picture as its reference picture, then that motion vector is selected as the motion shift to be applied. If such motion is not identified, then the motion shift is set to (0,0). ii. In one example, other spatial candidate positions (eg, A0, B0, B1, B2) may be used to derive motion shifts. (i) In one example, the checking order may be A1->B1->B0->A0->B2. (ii) In one example, the checking order may be B1->A1->B0->A0->B2. (iii) In one example, the checking order may be A0->A1->B0->B1->B2. c. In one example, after deriving the motion shift, for each sub-CU, the BV information of its corresponding block (the smallest motion grid covering the center sample) in the same-position picture is used to derive the BV information of the sub-CU ( Fig.31 ), assuming that the motion shift is set to the motion of block A1. About re-ranking of BV candidate list 9. First, the initial BV candidate list is derived, and then the reordering / refinement process is performed. And the reordering / refinement list is used in the encoding / decoding process of the block. a. In one example, the IBC candidate list may be an IBC merge candidate list. b. In one example, the IBC candidate list may be an IBC AMVP candidate list. c. In one example, the IBC candidate list may be an IBC template matching candidate list. d. In one example, the IBC candidate list may be an intra template matching candidate list. e. In one example, the IBC candidate list may be an IBV MMVD candidate list. f. In one example, the IBC candidate list may be an IBC GPM candidate list. g. In one example, the IBC candidate list may be an IBC TPM candidate list. h. Alternatively, the re-ranking / refinement process may not be applied to the IBC AMVP candidate list. i. Alternatively, the re-ranking / refinement process may not be applied to the IBC GPM candidate list. j. Alternatively, the re-ranking / refinement process may not be applied to the IBC TPM candidate list. k. In one example, the reordering / thinning process can be used with all kinds of IBC codecs, ie, the same process is applied. i. Alternatively, whether to apply the reordering / refinement process may depend on decoding information (eg, IBC mode). (i) In one example, applied to IBC Merge / Skip mode, but not to IBC AMVP mode. 10. BV candidates can be adaptively reordered in the final BV candidate list according to one or some criteria. a. In one example, part or all of the current BV candidate list building process is called first, followed by a reordering of the candidates in the list. i. Alternatively, the candidates in the first subgroup may be reordered and they should be added before those in the second subgroup, where the first subgroup is added before the second subgroup. b. In one example, all BV candidates in the BV candidate list may be reordered together (ie, without subgroups). i. Alternatively, some of the BV candidates in the BV candidate list may be reordered together (ie, without subgroups). (i) The BV candidates to be reordered may be selected based on codec information or signaling. c. In one example, the criterion may be based on template matching cost. i. In one example, the cost function between the current template and the reference template can be (i) SAD / MR-SAD, (ii) SATD / MR-SATD, (iii) SSD / MR-SSD, (iv) SSE / MR-SSE, (v) Weighted SAD / Weighted MR-SAD, (vi) Weighted SATD / Weighted MR-SATD, (vii) Weighted SSD / Weighted MR-SSD, (viii) Weighted SSE / Weighted MR-SSE, (ix) Gradient information. ii. In one example, if LMCS is enabled, the current template and the reference template may consist of samples in the mapped domain. (i) Alternatively, the current template and the reference template may consist of samples in the original domain. iii. In one example, the BV candidates in each subgroup may be reordered in ascending order according to the cost value based on template matching. iv. In one example, if only the upper template is available for the current block, the template matching reordering can only use the upper template. v. In one example, if only the left template is available for the current block, the template matching reordering can only use the left template. vi. In one example, if both the upper template and the left template are available for the current block, the template matching reordering may use the left template, the upper template, or the upper template and the left template. vii. In one example, the template matching process may include a component, such as brightness. (i) Alternatively, the template matching procedure may include multiple components, such as luminance and chrominance. a) In one example, the total template matching cost may be calculated as a weighted sum of the template matching costs on different color components. viii. In one example, the width of the left template and / or the height of the upper template may be fixed. (i) In one example, the width of the left template may be 1. (ii) In one example, the height of the upper template may be 1. ix. In one example, the BV used to locate the reference template can be cropped. Fig.32 A shows an example diagram 3200 showing that the reference template is outside the current picture. Fig.32 B shows an example diagram 3250 showing clipping of the BV so that the reference template is located inside the current picture. (i) In one example, if the reference template is outside the current image (e.g. Fig.33A As shown in FIG. 1 , the BV used to locate the reference template can be clipped so that the reference template is located within the current picture. The clipping operation does not change the BV of the corresponding BV candidate. Fig.33B An example is shown in FIG. x. In one example, if the reference template is outside the current picture (as shown in Fig. 33A), the corresponding BV candidates can still be reordered. (i) In one example, if the reference template is outside the current picture, it can be filled from the closest sample within the current picture. xi. In one example, the reference template should have been reconstructed within the current picture. (i) In one example, if the reference template is not reconstructed or is outside the current picture, the corresponding BV candidates may not be reordered. xii. In one example, the samples contained in the reference template should be available (eg, being reconstructed and within the same stripe / tile / IBC virtual buffer). (i) In addition, if some or all of the samples included in the reference template are unavailable, the above-mentioned method of processing a reference template outside the current image may be used. d. In one example, whether and / or how to reorder BV candidates may depend on the category of the BV candidates. e. In one example, the BV candidates to be reordered may be BV candidates in the final BV candidate list. i. Alternatively, the BV candidates to be reordered may be some / all of the neighboring spatial domain BV candidates, even though they may not be included in the final BV candidate list. ii. Alternatively, the BV candidates to be reordered may be some / all non-adjacent spatial domain BV candidates, even though they may not be included in the final BV candidate list. iii. Alternatively, the BV candidates to be reordered may be some / all HMVP BV candidates, even though they may not be included in the final BV candidate list. iv. Alternatively, the BV candidates to be re-ranked may be some / all pairwise average BV candidates, even though they may not be included in the final BV candidate list. v. Alternatively, the BV candidates to be reordered may be some / all STMVPBV candidates, even though they may not be included in the final BV candidate list. 11. The template shape can be adaptive. a. In one example, the template may include only neighboring samples to the left of the current block. b. In one example, the template may only include neighboring samples above the current block. c. In one example, the template shape is selected according to the CU shape. d. In one example, the width of the left template is selected according to the CU height. i. For example, if H<=M, the size of the left template is w1xH; otherwise, the size of the left template is w2xH. e. In one example, M, w1, and w2 are set equal to 8, 1, and 2, respectively. f. In one example, the height of the upper template is selected according to the width of the CU. i. For example, if W<=N, the size of the upper template is Wxh1; otherwise, the size of the upper template is Wxh2. (i) In one example, N, h1, and h2 are set equal to 8, 1, and 2, respectively. g. In one example, the width of the left template is selected according to the CU width. i. For example, if W<=N, the size of the left template is w1xH; otherwise, the size of the left template is w2xH. (i) In one example, N, w1, and w2 are set equal to 8, 1, and 2, respectively. h. In one example, the height of the upper template is selected according to the height of the CU. i. For example, if H<=M, the size of the upper template is Wxh1; otherwise, the size of the upper template is Wxh2. (i) In one example, M, h1, and h2 are set equal to 8, 1, and 2, respectively. i. In one example, samples of the template and reference samples of the template samples may be sub-sampled or down-sampled before being used to calculate the cost. i. Whether and / or how to perform subsampling may depend on the CU size. ii. In one example, no subsampling is performed on the short edge of the CU. 12. In one example, the cost disclosed in item 0 may be derived for a first BV candidate, which may or may not be placed in the BV candidate list. a. In one example, whether to put the first BV candidate into the BV candidate list may depend on the cost derived for the first BV candidate. b. In one example, whether to place a first BV candidate into the BV candidate list may depend on a comparison between a first cost derived for the first BV candidate and a second cost derived for a second BV candidate, which may or may not be placed into the BV candidate list. 13. In one example, for intra TMP, the L-shaped template can be replaced with an upper left template that excludes the upper left portion. Fig.32 An example diagram 3200 of intra-frame template matching is shown. a. In one example, if only the upper template is available for the current block, the intra TMP can only use the upper template. b. In one example, if only the left template is available for the current block, the intra-frame TMP can only use the left template. c. In one example, if both the above template and the left template are available for the current block, the intra TMP may use the left template, the above template, or both the above template and the left template. 2.24. Template matching based merge candidate list construction (TM-MCLC) In ECM, non-neighboring and HMVP candidates are put into the merged candidate list after neighboring and TMVP candidates based on a predefined traversal order. For TM-MCLC, non-neighboring and HMVP candidates are still put into the merged candidate list after neighboring and TMVP candidates, but in ascending order of template matching cost. Fig.34An example diagram 3400 showing the non-adjacent positions used is shown. More specifically, all available non-adjacent MVPs and HMVPs are collected in a group. The TM cost associated with each candidate in the group is then derived in a manner similar to ARMC. Subsequently, all candidates in the group are sorted in ascending order according to the corresponding TM cost. Finally, the non-adjacent and HMVP candidates are placed in a merge candidate list in ascending order of template matching cost. In this proposal, the non-adjacent MVP in the ECM software is further extended to more spatial and non-adjacent temporal locations, such as Fig.34 In addition to the 18 positions of non-adjacent spatial MVPs in ECM-2.0, an additional 32 spatial positions and 12 non-adjacent temporal positions are introduced, where the non-adjacent temporal MVP positions are located in the same reference frame as the adjacent TMVPs. To reduce complexity, the TM merge mode uses integer precision, while the normal merge mode uses a 2-tap bilinear filter to generate the reference template. 2.25. Modifications to IBC Merge / AMVP List Construction Fig.35 An example diagram 3500 showing spatial candidates for an IBC merge / AMVP candidate list is shown. Fig.36 An example diagram 3600 showing templates and reference samples of templates is shown. In this article, the following changes are made to the IBC merge / AMVP list construction: 1) An IBC merge / AMVP candidate can be inserted into the IBC merge / AMVP candidate list only if it is valid. 2) The upper right, lower left and upper left spatial domain candidates (such as Fig.35 as shown) and a pair of average candidates are added to the IBC merge / AMVP candidate list. 3) ARMC-TM is extended to the IBC merge list, called adaptive re-ranking of merge candidates with IBC template matching (ARMC-TM-IBC). The template and the reference sample of the template are as follows Fig.36 shown. 2.26. ARMC based on MV candidate types An ARMC based on MV candidate types is proposed to reorder the merge candidates in the candidate type according to the TM cost value (the same as the ARMCTM cost used in ECM2.0), such as TMVP, NA-MVP, etc. Then, when building the merge candidate list, M candidates are selected from the reordered candidates in the candidate type, where M varies according to the candidate type (for example, M=1 for TMVP type and M=9 for NA-MVP type). Some candidate types add more merge candidates to perform reordering and selection. Fig.37An example diagram 3700 showing example locations where TMVP is used is shown. Fig.37 The TMVP candidates derived from the different co-location positions shown enable the various prediction reference lists to be re-ranked together. More specifically, the TMVP candidates evaluated are as follows: TMVP0: List 0-MV from {RB, C}; List 1-MV from {RB, C} TMVP1: List 0-MV from {RBR, RBL}; List 1-MV from {RBR, RBL} TMVP2: List 0-MV from {RB,C} (without TMVP0 from List 1) TMVP3: List 1-MV from {RB,C} (without TMVP0 from List 0) TMVP4: List 0-MV from {RBR, RBL} (without TMVP1 from List 1) TMVP5: List 1-MV from {RBR, RBL} (without TMVP1 from List 0) Where {Pos1, Pos2} represents the order of MV availability check, and only the first available position is selected. Finally, 1 TMVP candidate is selected from these TMVP candidates based on the TM cost value. Some candidate types may not be reordered by the proposed ARMC. 2.27. Adaptive Decoder-side Motion Vector Refinement (ADMVR) In ECM-2.0, if the selected merge candidate satisfies the decoder-side motion vector refinement (DMVR) condition, a multi-pass DMVR method is applied in the regular merge mode. In the first pass, bilateral matching (BM) is applied to the codec block. In the second pass, BM is applied to each 16x16 sub-block within the codec block. In the third pass, the MV in each 8x8 sub-block is refined by applying bidirectional optical flow (BDOF). The adaptive decoder-side motion vector refinement method consists of two new merge modes, which introduces refined MVs only in one direction (L0 or L1) of bidirectional prediction of merge candidates that meet the DMVR conditions. A multi-pass DMVR process is applied to the selected merge candidates to refine the motion vectors, but MVD0 or MVD1 is set to zero in the first pass (i.e. PU level) DMVR. As with the regular merge mode, the merge candidates of the proposed merge mode are derived from spatially adjacent coded blocks, TMVP, non-adjacent blocks, HMVP, and paired candidates. The difference is that only those that meet the DMVR condition will be added to the candidate list. The two proposed merge modes use the same merge candidate list (i.e., ADMVR merge list), and the merge index is encoded and decoded as in the regular merge mode. 2.28. Affine model inheritance based on history parameters A history parameter table (HPT) is created. An entry in the HPT stores a set of affine parameters: a, b, c, and d, each represented by a 16-bit signed integer. Entries in the HPT are classified by reference list and reference index. Each reference list in the HPT supports five reference indexes. Formulaically, the category of the HPT (denoted as HPTCat) is calculated as HPTCat(RefList, RefIdx) = 5 × RefList + min(RefIdx, 4), where RefList and RefIdx represent the reference picture list (0 or 1) and the reference index, respectively. Each category can store up to two entries, making a total of twenty entries in the HPT. At the beginning of each CTU row, the number of entries in each category is initialized to zero. Using the reference list RefList cur and RefIdx cur After decoding the affine-coded CU, the affine parameters are used to update the category HPTCat (RefList cur RefIdx cur ) in a manner similar to the HMVP table update. Fig.38 An example diagram 3800 showing the generation of HAPC is shown. The candidate (HAPC) based on the historical affine parameters is obtained from Fig.38 The MV of the neighboring 4×4 block is used as the base MV. In a formal way, the MV of the current block at position (x, y) is calculated as follows: Where (mvhbase, mvvbase) represents the MV of the neighboring 4×4 blocks, and (xbase, ybase) represents the value of the center-position neighboring 4×4 blocks. (x, y) can be the top-left, top-right, and bottom-left corners of the current block to obtain the corner position MV (CPMV) of the current block, or it can be the center of the current block to obtain the normal MV of the current block. Fig.38An example of how to derive a HAPC from block A0 is shown. The affine parameters {a0, b0, c0, d0} are taken directly from an item of class HPTIdx(RefListA0, refIdx0A0) in the HPT. The affine parameters from the HPT, with the center position of A0 as the reference position and the MV of block A0 as the reference MV, are used together to derive the CPMV for affine merge HAPC or affine AMVPHAPC. They can also be used to derive the MV located at the center of the current block as a regular merge candidate. A HAPC can be put into a sub-block based merge candidate list, an affine AMVP candidate list, or a regular merge candidate list. In response to the introduction of the new HAPC, the size of the sub-block based merge candidate list is increased from 5 to 9. 2.29. Non-adjacent spatial neighbors for affine merge mode Fig.39 An example diagram 3900 showing spatial neighbors for deriving affine merge candidates is shown. Similar to the enhanced conventional merge mode, this paper proposes affine merge using non-adjacent spatial neighbors (NSAM). The mode for obtaining non-adjacent spatial neighbors is as follows: Fig.39 As shown in Figure 2, similar to the existing non-adjacent conventional merge candidates, the distance between the non-adjacent spatial neighbors in NSAM and the current codec block is also defined based on the width and height of the current CU. Fig.39 The motion information of non-adjacent spatial neighbors in is used to generate additional inherited and constructed affine merge candidates. Specifically, for inherited candidates, the same derivation process of inherited affine merge candidates in VVC remains unchanged, except that CPMV is inherited from non-adjacent spatial neighbors. Non-adjacent spatial neighbors are checked based on their distance to the current block (i.e., from near to far). At a certain distance, only the first available neighbor (i.e., encoded and decoded using affine mode) on each side of the current block (e.g., left and above) is included for inherited candidate derivation. Fig.39 As shown by the dashed arrows in (a), the order of checking the left and upper neighbors is from bottom to top and from right to left, respectively. Fig.39 As shown in (b), the position of the upper left non-adjacent spatial neighbor is first determined independently; then, the position of the upper left neighbor can be determined accordingly, which can form a rectangular virtual block together with the upper left non-adjacent neighbor. Fig.40 An example diagram 4000 showing affine merge candidates constructed from non-adjacent neighbors is shown. Then, as Fig.40 As shown, the motion information of three non-adjacent neighbors is used to form the CPMV of the virtual blocks located at the upper left (A), upper right (B), and lower left (C), which are finally projected to the current CU to generate the corresponding construction candidates. Non-adjacent spatial merge candidates are inserted into the affine merge candidate list in the following order: 1.SbTMVP candidate (if valid). 2. Inherited from adjacent neighbors. 3. Inherit from non-adjacent neighbors. 4. Constructed by adjacent neighbors. 5. Constructed from non-adjacent neighbors. 6. Zero MV. Due to the inclusion of additional candidates generated by NSAM, the size of the affine merge candidate list is increased from 5 to 15 in this paper. 2.30. Affine Motion Compensated Prediction In HEVC, only the translational motion model is applied for motion compensated prediction (MCP). In the real world, there are many kinds of motion, such as zooming in / out, rotation, perspective motion, and other irregular motions. In VVC, block-based affine transformation motion compensated prediction is applied. Fig.41 An exemplary control point based affine motion model 4110 (4 parameters) and an exemplary control point based affine motion model 4120 (6 parameters) are shown. Fig.41 As shown, the affine motion field of a block is described by the motion information of two control points (4 parameters) or three control point motion vectors (6 parameters). For the 4-parameter affine motion model, the motion vector at the sample position (x, y) in the block is derived as: For the 6-parameter affine motion model, the motion vector at the sample position (x, y) in the block is derived as: Where (mv 0x ,mv 0y ) is the motion vector of the upper left control point, (mv 1x ,mv 1y ) is the motion vector of the upper right corner control point of the motion vector, (mv 2x ,mv 2y ) is the motion vector of the lower left control point. Fig.42 An example diagram 4200 showing an example affine MVF for each sub-block is shown. To simplify motion compensated prediction, block-based affine transform prediction is applied. To derive the motion vector for each 4×4 luma sub-block, as Fig.42 As shown, the motion vector of the center sample of each subblock is calculated according to the above equation and rounded to 1 / 16 fractional accuracy. The motion compensated interpolation filter is then applied to generate a prediction of each subblock with the derived motion vector. The subblock size of the chroma component is also set to 4×4. The MV of the 4×4 chroma subblock is calculated as the average of the MVs of the upper left and lower right luminance subblocks in the same 8x8 luminance region. Like translational motion inter prediction, there are two affine motion inter prediction modes: affine merge mode and affine AMVP mode. 2.30.1 Affine Merge Prediction AF_MERGE mode can be applied to CUs with width and height greater than or equal to 8. In this mode, the CPMV of the current CU is generated based on the motion information of the spatially adjacent CUs. There can be up to five CPMVP candidates, and the candidate to be used for the current CU is indicated by signaling the index. The following three types of CPVM candidates are used to form the affine merge candidate list: – inherited affine merge candidates inferred from the CPMV of neighboring CUs, – Construct affine merged candidate CPMVPs, which are derived using the translation MVs of neighboring CUs, – Zero MV. In VVC, there are at most two inherited affine candidates, which are derived from the affine motion models of neighboring blocks, one from the left neighboring CU and one from the above neighboring CU. Fig.43 An example diagram 4300 showing example locations of inherited affine motion predictors is shown. Candidate blocks such as Fig.43 As shown. For the left predictor, the scan order is A0->A1, and for the upper predictor, the scan order is B0->B1->B2. Only the first inherited candidate on both sides is selected. No pruning check is performed between two inherited candidates. When a neighboring affine CU is identified, its control point motion vectors are used to derive the CPMVP candidate in the affine merge list of the current CU. Fig.44 An example diagram 4400 showing example control point motion vector inheritance is shown. Fig.44 As shown, if the adjacent lower left block A is encoded and decoded in affine mode, the motion vectors v2, v3 and v4 of the upper left corner, upper right corner and lower left corner of the CU containing block A are obtained. When block A is encoded and decoded using a 4-parameter affine model, two CPMVs of the current CU are calculated based on v2 and v3. If block A is encoded and decoded using a 6-parameter affine model, three CPMVs of the current CU are calculated based on v2, v3 and v4. Constructing affine candidates means constructing candidates by combining the neighboring translation motion information of each control point. Fig.45 An example diagram 4500 is shown, which shows example positions of candidate positions for constructed affine merge patterns. The motion information of the control points is obtained from Fig.45 The specified spatial and temporal neighbors are derived as shown. CPMV k(k=1,2,3,4) indicates the kth control point. For CPMV1, check B2->B3->A2 blocks and use the MV of the first available block. For CPMV2, check B1->B0 blocks, and for CPMV3, check A1->A0 blocks. For TMVP, if valid, use as CPMV4. After obtaining the MVs of the four control points, construct the affine merge candidate based on this motion information. Use the following control point MV combinations in order: The combination of 3 CPMVs {CPMV1,CPMV2,CPMV3},{CPMV1,CPMV2,CPMV4},{CPMV1,CPMV3,CPMV4},{CPMV2,CPMV3,CPMV4},{CPMV1,CPMV2},{CPMV1,CPMV3} constructs a 6-parameter affine merge candidate, and the combination of 2 CPMVs constructs a 4-parameter affine merge candidate. To avoid the motion scaling process, if the reference index of the control point is different, the relevant combination of the control point MV is discarded. After checking inherited affine merge candidates and constructed affine merge candidates, if the list is still not full, a zero MV is inserted to the end of the list. 2.30.2 Affine AMVP Prediction Affine AMVP mode can be applied to CUs with width and height both greater than or equal to 16. An affine flag in the CU level is signaled in the bitstream to indicate whether to use affine AMVP mode, and another flag is signaled to indicate whether it is 4-parameter affine or 6-parameter affine. In this mode, the difference between the CPMV of the current CU and its predicted sub-CPMVP is signaled in the bitstream. The affine AVMP candidate list size is 2, which is generated in order using the following four types of CPVM candidates: – inherited affine AMVP candidates inferred from the CPMV of neighboring CUs, – Construct an affine AMVP candidate CPMVP, which is derived using the translation MV of the neighboring CU, – translation MV from neighboring CU, – Zero MV. The order in which inherited affine AMVP candidates are checked is the same as the order in which inherited affine merge candidates are checked. The only difference is that for AVMP candidates, only affine CUs with the same reference picture as in the current block are considered. When inserting inherited affine motion predictors into the candidate list, no pruning process is applied. The constructed AMVP candidates are Fig.45Derived from the specified spatial neighbors shown. Use the same check order as in the affine merge candidate construction. In addition, the reference picture index of the neighboring blocks is also checked. Use the first block in the check order, that is, the first block that is inter-coded and has the same reference image as the current CU. Only one When the current CU is coded and decoded in 4-parameter affine mode, and both mv0 and mv1 are valid, they are added to the affine AMVP list as a candidate. When the current CU is coded and decoded in 6-parameter affine mode and all three CPMVs are valid, they are added to the affine AMVP list as a candidate. Otherwise, the constructed AMVP candidate will be set to invalid. If the affine AMVP list candidate is still less than 2 after inserting a valid inherited affine AMVP candidate and constructing an AMVP candidate, mv0, mv1, and mv2 will be added in order as translation MVs to predict all control point MVs of the current CU (when valid). Finally, if the affine AMVP list is still not full, the list is filled with zero MVs. 2.30.3 Affine Motion Information Storage In VVC, the CPMV of an affine CU is stored in a separate buffer. The stored CPMV is only used to generate inherited CPMVP for the most recently encoded CU in affine merge mode and affine AMVP mode. The sub-block MV derived from the CPMV is used for motion compensation, merging of translation MVs / MV derivation of AMVP lists, and deblocking. To avoid image row buffering with additional CPMV, the affine motion data inheritance of the CU from the above CTU is handled differently from the inheritance from the normal neighboring CU. If the candidate CU for affine motion data inheritance is located in the above CTU row, the lower left and lower right sub-block MVs in the buffer row are used instead of the CPMVs for affine MVP derivation. In this way, the CPMVs are only stored in the local buffer. If the candidate CU is 6-parameter affine coded, the affine model degenerates to a 4-parameter model. Fig.46 An example diagram 4600 showing the use of motion vectors for the proposed combination method is shown. Fig.46 As shown, along the top CTU boundary, the bottom left and bottom right sub-block motion vectors of the CU are used for affine inheritance of the CU in the bottom CTU. 2.30.4 Refinement of optical flow prediction for affine mode Compared with pixel-based motion compensation, sub-block-based affine motion compensation can save memory access bandwidth and reduce computational complexity at the expense of prediction accuracy loss. In order to achieve finer-grained motion compensation, optical flow prediction refinement (PROF) is used to refine sub-block-based affine motion compensation predictions without increasing the memory access bandwidth for motion compensation. In VVC, after performing sub-block-based affine motion compensation, the brightness prediction samples are refined by adding the differences derived from the optical flow equation. PROF is described as the following four steps: Step 1) Perform sub-block based affine motion compensation to generate sub-block prediction I(i, j). Step 2) Use a 3-tap filter [-1, 0, 1] to calculate the spatial gradient g of the sub-block prediction at each sample position x (i, j) and g y (i,j). The gradient calculation is exactly the same as that in BDOF. g x (i,j)=(I(i+1,j)>>shift1)-(I(i-1,j)>>shift1) g y (i,j)=(I(i,j+1)>>shift1)-(I(i,j-1)>>shift1) shift1 is used to control the accuracy of the gradient. The sub-block (i.e. 4x4) prediction is extended by one sample on each side for gradient calculation. To avoid extra memory bandwidth and extra interpolation calculations, those extended samples on the extended boundary are copied from the nearest integer pixel position in the reference picture. Step 3) Calculate the brightness prediction refinement by following the optical flow equation. ΔI(i, j) = g x (i, j)*Δu x (i, j)+g y (i, j)*Δu y (i, j) where Δv(i, j) is the difference between the sample MV calculated for the sample position (i, j) (denoted by v(i, j)) and the sub-block MV of the sub-block to which the sample (i, j) belongs, as Fig.47 shown. Fig.47 An example diagram 4700 showing an example sub-block MV VSB and pixels Δv(i,j) (arrows) is shown. Δv(i,j) is quantized in units of 1 / 32 luma sample precision. Since the affine model parameters and the sample positions relative to the sub-block center do not change between sub-blocks, Δv(i, j) can be calculated for the first sub-block and reused for other sub-blocks of the same CU. Let dx(i, j) and dy(i, j) be the distance from the sample position (i, j) to the sub-block center (x SB ,y SB ), Δv(x, y) can be derived from the following equations, To maintain accuracy, the sub-block (x SB ,y SB ) is calculated as ((W SB -1) / 2,(H SB-1) / 2), where W SB and H SB are the sub-block width and height respectively. For the 4-parameter affine model, For the 6-parameter affine model, Where (v 0x , v 0y )、(v 1x , v 1y )、(v 2x , v 2y ) are the upper left, upper right and lower left control point motion vectors, w and h are the width and height of the CU. Step 4) Finally, the luma prediction refinement ΔI(i, j) is added to the sub-block prediction I(i, j). The final prediction I' is generated by the following equation. I′(i,j)=I(i,j)+ΔI(i,j) For affine-encoded CUs, PROF is not applicable in the following two cases: 1) All control point MVs are the same, indicating that the CU has only translational motion; 2) Affine motion parameters are greater than the specified limit, because the sub-block-based affine MC is downgraded to the CU-based MC to avoid large memory access bandwidth requirements. A fast coding method is used to reduce the coding complexity of PROF affine motion estimation. PROF is not applied in the affine motion estimation stage in the following two cases: a) If the CU is not a root block and its parent block does not select the affine mode as its best mode, PROF is not applied because the current CU has a low probability of selecting the affine mode as the best mode; b) If the sizes of the four affine parameters (C, D, E, F) are all less than the predefined threshold and the current picture is not a low-latency picture, PROF is not applied because the improvement introduced by PROF in this case is small. In this way, the affine motion estimation of PROF can be accelerated. 2.31. Construction of motion candidate list The following detailed implementation examples should be considered as examples to explain the general concept. These embodiments should not be interpreted narrowly. In addition, these embodiments can be combined in any way. The term "block" may refer to a codec tree block (CTB), a codec tree unit (CTU), a codec block (CB), a CU, a PU, a TU, a PB, a TB, or a video processing unit including multiple samples / pixels, and a block may be rectangular or non-rectangular. For IBC-coded blocks, a block vector (BV) is used to indicate the displacement from the current block to a reference block that has been reconstructed in the current picture. W and H are the width and height of the current block (eg, luma block). The non-adjacent spatial candidates of the current codec block are the adjacent spatial candidates of the virtual block in the i-th search round (e.g. Fig. 9 As shown in FIG. 1 ). The calculation formulas for the width and height of the virtual block in the i-th search round are: newWidth = i×2×gridX+W, newHeight = i×2×gridY+H. Obviously, if the search round i is 0, the virtual block is the current block. Hereinafter, the BV predictor is also a BV candidate. The skip mode is also a merge mode. The motion candidates can be divided into several groups according to certain criteria. Each group is called a subgroup. For example, we can take continuous spatial and temporal motion candidates as the first subgroup and the remaining motion candidates as the second subgroup; in another example, we can also take the first N (N≥2) candidate motions as the first subgroup, the following M (M≥2) candidate motions as the second subgroup, and the remaining motion candidates as the third subgroup. For example, the motion candidates can be divided into several groups according to the motion candidate type. A BV candidate can be determined to be valid when it satisfies one of the following conditions or a combination of at least one condition. 1) The corresponding reference block has been reconstructed inside the current image. 2) The corresponding reference block and the current block are located in the same CTU row. 3) The corresponding reference block is in the same tile / sub-image as the current block. 4) The corresponding reference block and the current block are located in the same stripe. 5) The BV candidate satisfies the block vector constraints (eg, as described in 2.4.2 and 2.4.3). 6) The BV candidate satisfies the IBC virtual buffer conditions (such as those described in 2.4.4). When a BV candidate violates one of the above conditions or a combination of at least one of the conditions, it may be determined to be invalid. If the CU at a certain temporal position is invalid or intra-coded or located outside the current CTU row or its BV (if any) is invalid for the current block, then the temporal position is considered invalid; otherwise, the temporal position is considered valid. Note that the proposed method can be applied to the candidate list construction process of merging inter-coded blocks (e.g., translational motion), affine-coded blocks, TM-coded blocks, GPM-coded blocks, ADMVR-coded blocks, or IBC-coded blocks; or other motion candidate list construction processes (e.g., ordinary AMVP list; affine AMVP list; TMAMVP list; IBCAMVP list; HMVP table). Note that the proposed method can be applied to the regular merge / AMVP candidate list after the DMVR / multi-pass DMVR process. Note that the proposed method can be applied to the ADMVR merge candidate list after the DMVR / multi-pass DMVR process. Note that the proposed method can be applied to TM merging / AMVP candidate list after block-based bilateral matching refinement and / or template matching refinement and / or sub-block-based bilateral matching refinement. In the following, a motion candidate that is a repeat of another motion candidate may be defined as 1) Both candidate motions are exactly the same in terms of reference direction, reference image index, affine model (for affine motion), and motion vector or block vector or partial or full CPMV depending on the motion type (inter or IBC or affine). or 2) The two motion candidates are exactly the same in reference direction, reference image index, affine model (for affine motion). However, the difference of motion vector or block vector or part or all CPMV of the two motion candidates according to the motion type (inter or IBC or affine) can be within the threshold. The difference of a motion vector may be defined as the absolute horizontal and / or vertical component of the difference of motion vectors of two motion candidates in reference list 0 and / or reference list 1. The difference of the block vectors may be defined as the absolute horizontal and / or vertical components of the difference of the block vectors of the two motion candidates in reference list 0. A CPMV difference may be defined as the absolute horizontal and / or vertical component of the motion vector difference of the CPMV of two motion candidates in reference list 0 and / or reference list 1. The threshold can be adaptively set according to the coding mode of the current block (e.g., TM merging, affine merging, regular merging, etc.) and / or the current block size and / or the candidate type (e.g., adjacent spatial domain, adjacent temporal domain, non-adjacent spatial domain, non-adjacent temporal domain, HMVP, paired or STMVP, etc.) and / or QP, and / or the reference index or reference POC of the current block, and / or other coding and decoding information of the current block. About pairwise average motion candidates 1. In one example, a pair-wise average motion candidate may be generated by averaging a pair of predefined re-ordered candidates in an existing motion candidate list. a. In one example, the reordering process may be to adaptively reorder the motion candidates using template matching. b. In one example, the re-ranking process may be to adaptively re-rank the motion candidates using bilateral matching. c. In one example, the predefined pairs can be defined as a set of {(0,1), (0,2), (1,2), (0,3), (1,3), (2,3)}, where the number represents the motion candidate index in the motion candidate list. d. In one example, before adding a pair-wise average motion candidate to the motion candidate list, some or all of the following conditions need to be met. i. Pairwise mean motion (eg, BV) candidates are valid. ii. The pairwise average motion candidate is not a duplicate of any previous motion candidate in the existing motion candidate list. iii. A pruning method for the pairwise average motion candidates with an adaptive threshold can be applied. (i) In one example, the threshold may be based on the current block size. (ii) In one example, the threshold may be based on the codec mode of the current block. (iii) In one example, the threshold may be based on QP. (iv) In one example, the threshold value may be based on one of the above conditions or a combination of at least one of the above conditions. (v) In one example, the threshold may be based on a reference index or a reference POC of the current block. e. In one example, the number of pairwise average motion candidates may be predefined. i. In one example, the number of pairwise average motion candidates may be N (eg, N=1). ii. In one example, the number of paired average motion candidates can be calculated by subtracting the number of existing motion candidates before adding paired average motion candidates from the motion candidate list size. That is, there is no limit on the number of paired average motion candidates, so paired average motion candidates can be added until the motion candidate list is full. f. In one example, some or all of the pairwise average motion candidates may be re-ordered. i. Optionally, pairwise average motion candidates may not be re-ordered. ii. In one example, the pair-wise average motion candidates may be reordered with some or all of the previous motion candidates in the same motion candidate list. g. In one example, unpaired motion candidates may be replaced by paired motion candidates derived from a motion candidate list. i. In one example, if some motion candidates are not reordered (ie, a non-reordered subgroup), each motion candidate in the non-reordered subgroup may be replaced by a paired motion candidate constructed by the first reordered candidate and the candidate, if the created pair is not repeated. ii. In one example, if some motion candidates are not reordered (ie, the subgroup is not reordered), Then some of the motion candidates in the non-reordered subgroup may be replaced by pairs of motion candidates constructed from the first reordered candidate and the original motion candidate if the created pairs are not repeated. iii. In one example, if some motion candidates are not reordered (i.e., a non-reordered subgroup), in the last reordered position, if the paired motion candidates are not duplicated and / or valid, the paired motion candidates can be constructed using the first two reordered motion candidates, added to the last reordered position in the motion candidate list, and the last motion candidate in the motion candidate list is removed from the list. (i) In one example, pairwise average motion candidates may not be reordered. (ii) In one example, the pair-wise average motion candidates may be re-ordered together with some or all of the previous motion candidates in the re-ordered sub-group. h. In one example, the motion candidate list can be i. In one example, the motion candidate list may be a TM merge / AMVP list after block-based bilateral matching refinement and / or template matching refinement and / or sub-block-based bilateral matching refinement. ii. In one example, the motion candidate list may be a regular merge / AMVP list after DMVR / multi-pass DMVR processing. iii. In one example, the motion candidate list may be an ADMVR merged list after DMVR / multi-pass DMVR processing. iv. In one example, the motion candidate list may be a GPM merge list (eg, AGPMList, LGPMList, or LAGPMList) after template matching refinement processing. v. In one example, the motion candidate list may be a regular merge / AMVP list. vi. In one example, the motion candidate list may be a TM merge / AMVP list. vii. In one example, the motion candidate list may be an affine merge / AMVP list. viii. In one example, the motion candidate list may be an IBC merge / AMVP list. ix. In one example, the motion candidate list may be an ADMVR merge list x. In one example, the motion candidate list may be a GPM merge list. xi. In one example, the motion candidate list may be a TPM merge list. xii. In one example, the motion candidate list may be an MMVD merge list. xiii. In one example, the motion candidate list may be an IBC template matching candidate list. xiv. In one example, the motion candidate list may be an intra template matching candidate list. xv. In one example, the motion candidate list may be an IBC MMVD candidate list. xvi. In one example, the motion candidate list may be an IBC GPM candidate list. xvii. In one example, the motion candidate list may be an IBC TPM candidate list. xviii. In one example, the motion candidate list may be any other motion candidate list. Sports Candidate List 2. The motion candidate list may consist of spatial candidates, and / or temporal candidates, and / or HMVP candidates, and / or paired candidates, and / or STMVP candidates. a. In one example, a spatial candidate may consist of adjacent and / or non-adjacent spatial candidates. i. In one example, the adjacent spatial domain candidates may be composed of left and / or above and / or upper right and / or lower left and / or upper left spatial domain candidates (for example, Fig. 30A shown). ii. In one example, the distance between the non-adjacent spatial candidate and the current codec block can be based on the width and height of the current codec block (e.g., Fig. 9 , Fig.10 or Fig.34 , gridX=W, gridY=H). (i) Optionally, the distance between the non-adjacent spatial candidates and the current codec block may be a multiple of a fixed value. a) For example, the multiplication factor depends on the search round index (eg, the multiplication factor for the i-th search round is i) and gridX=M, gridY=N (M and N are constant values). b. In one example, the time domain candidates may be composed of adjacent and / or non-adjacent time domain candidates. i. In one example, for adjacent time domain candidates, Fig.37 The positions for the adjacent time domain candidates are selected among the RBs and / or Cs and / or RBRs and / or RBLs in . (i) In one example, four neighboring time domain candidates (eg, RB and C and RBR and RBL) may be used. a) In one example, the order may be RB->C->RBR->RBL. (ii) In one example, two adjacent time-domain candidates may be used. a) In one example, one neighboring time domain candidate may be selected between RB and C. If RB is invalid, C will be used; otherwise, RB will be used. And other neighboring time domain candidates may be selected between RBR and RBL. If RBR is invalid, RBL will be used; otherwise, RBR will be used. (iii) In one example, N adjacent time domain candidates may be used. a) In one example, the first N valid positions can be used to derive N neighboring time domain candidates. b) In one example, N may be an integer from 0 to 4. (iv) In one example, a neighboring time domain candidate may be selected between RB and C. If RB is invalid, C will be used; otherwise, RB will be used. (v) In one example, a neighboring time domain candidate may be selected between RBR and RBL. If RBR is invalid, RBL is used; otherwise, RBR is used. ii. In one example, the distance between the non-adjacent time domain candidate and the current codec block can be based on the width and height of the current codec block (e.g., Fig.34 , gridX=W, gridY=H). (i) Optionally, the distance between the non-adjacent time domain candidates and the current coding block may be a multiple of a fixed value. a) For example, the multiplication factor depends on the search round index (eg, the multiplication factor for the i-th search round is i) and gridX=M, gridY=N (M and N are constant values). c. In one example, for paired candidates, they can be generated by averaging predefined candidate pairs in existing motion candidate lists. i. In one example, the predefined pairs may be defined as {(0,1), (0,2), (1,2), (0,3), (1,3), (2,3)}, where the numbers represent the motion candidate index of the motion candidate list. ii. In one example, the number of pairwise average motion candidates may be predefined. (i) In one example, the number of pairwise average motion candidates may be N (eg, N=1). (ii) In one example, the number of pairwise average motion candidates may be calculated by subtracting the number of existing motion candidates before adding pairwise average motion candidates from the motion candidate list size. That is, pairwise average motion candidates may be added until the motion candidate list is full. iii. In one example, the pairwise candidates may be inserted after the HMVP. iv. In one example, for affine pairwise candidates, an average operation may be performed on each CPMV. (i) In one example, if two affine candidates from which an affine pair candidate is derived have different affine models, the combination may be discarded. (ii) In one example, if two affine candidates from which affine pair candidates are derived have different affine models, the 6-parameter affine model can be degraded to a 4-parameter affine model. d. In one example, for the STMVP candidate, it can be generated by averaging the predefined E spatial motion candidates and the predefined G temporal motion candidates. i. In one example, E may be less than or equal to the number of spatial candidates (F) inserted into the current motion candidate list before STMVP. ii. In one example, the predefined E spatial domain motion candidate may be the first E spatial domain motion candidate in the F spatial domain candidates inserted into the current motion candidate list before the STMVP. (i) Of course, the predefined E airspace motion candidate can be the E airspace motion candidate selected from the F airspace candidates inserted into the current motion candidate list before STMVP. iii. In one example, E is 2 and G is 1. iv. In one example, STMVP may be inserted before the top left spatial motion candidate. v. In one example, STMVP can be inserted after the pairwise candidates. vi.STMVP candidates may participate in the re-ranking process. vii. Optionally, STMVP candidates may not participate in the re-ranking process. e. In one example, motion candidates may include i. In one example, motion candidates may consist of adjacent spatial regions and / or HMVPs and / or pairs. ii. In one example, motion candidates may consist of adjacent spatial and / or adjacent temporal and / or HMVPs and / or pairs. iii. In one example, motion candidates may consist of adjacent spatial domains and / or adjacent temporal domains and / or non-adjacent spatial domains and / or HMVPs and / or pairs. iv. In one example, the motion candidates may consist of adjacent spatial domains and / or adjacent temporal domains and / or non-adjacent spatial domains and / or non-adjacent temporal domains and / or HMVPs and / or pairs. v. In one example, motion candidates may consist of adjacent spatial domains and / or non-adjacent spatial domains and / or HMVPs and / or pairs. vi. In one example, the motion candidates may consist of adjacent spatial (STMVP may be inserted before the upper left spatial motion candidate) and / or adjacent temporal and / or non-adjacent spatial and / or HMVP and / or in pairs. vii. In one example, the motion candidates may be composed of adjacent spatial (STMVP may be inserted before the upper left spatial motion candidate) and / or adjacent temporal and / or non-adjacent spatial and / or non-adjacent temporal and / or HMVP and / or in pairs. viii. In one example, the motion candidates may be inserted in order according to the motion candidate type. ix. In one example, a motion candidate may be inserted into a motion candidate list only if it is not a duplicate of any previous motion candidate in the existing motion candidate list. (i) Optionally, a motion candidate may be inserted into a motion candidate list only if it is not a duplicate of some of the previous motion candidates in the existing motion candidate list. x. In one example, any other motion candidate type combination or any other motion candidate (type) insertion order is also supported. f. In one example, the motion candidate list may be: i. In one example, the motion candidate list may be a TM merge / AMVP list after block-based bilateral matching refinement and / or template matching refinement and / or sub-block-based bilateral matching refinement. ii. In one example, the motion candidate list may be a regular merge / AMVP list after DMVR / multi-pass DMVR processing. iii. In one example, the motion candidate list may be an ADMVR merged list after DMVR / multi-pass DMVR processing. iv. In one example, the motion candidate list may be a GPM merge list (eg, AGPMList, LGPMList, or LAGPMList) after template matching refinement processing. v. In one example, the motion candidate list may be a regular merge / AMVP list. vi. In one example, the motion candidate list may be a TM merge / AMVP list. vii. In one example, the motion candidate list may be an affine merge / AMVP list. viii. In one example, the motion candidate list may be an IBC merge / AMVP list. ix. In one example, the motion candidate list may be an ADMVR merge list x. In one example, the motion candidate list may be a GPM merge list. xi. In one example, the motion candidate list may be a TPM merge list. xii. In one example, the motion candidate list may be an MMVD merge list. xiii. In one example, the motion candidate list may be an IBC template matching candidate list. xiv. In one example, the motion candidate list may be an intra template matching candidate list. xv. In one example, the motion candidate list may be an IBC MMVD candidate list. xvi. In one example, the motion candidate list may be an IBC GPM candidate list. xvii. In one example, the motion candidate list may be an IBC TPM candidate list. xviii. In one example, the motion candidate list may be any other motion candidate list. 2.32. Reordering of motion candidate list The following detailed implementation examples should be considered as examples to explain the general concept. These embodiments should not be interpreted narrowly. In addition, these embodiments can be combined in any way. The term "block" may refer to a codec tree block (CTB), a codec tree unit (CTU), a codec block (CB), a CU, a PU, a TU, a PB, a TB, or a video processing unit including a plurality of samples / pixels. Blocks can be rectangular or non-rectangular. For IBC-encoded blocks, the block vector (BV) is used to indicate the displacement from the current block to the reference block. The reference block has been reconstructed in the current picture. W and H are the width and height of the current block (eg, luma block). The non-adjacent spatial candidates of the current codec block are the adjacent spatial candidates of the virtual block in the i-th search round (e.g. Fig. 9 As shown in FIG. 1 ). The calculation formulas for the width and height of the virtual block in the i-th search round are: newWidth = i×2×gridX+W, newHeight = i×2×gridY+H. Obviously, if the search round is 0, the virtual block is the current block. In the following, BV predictor is also BV candidate. Skip mode is also merge mode. The motion candidates can be divided into several groups according to certain criteria. Each group is called a subgroup. For example, we can take the continuous spatial and temporal motion candidates as the first subgroup and the remaining motion candidates as the second subgroup; in another example, we can also take the first N (N≥2) candidate motions as the first subgroup, the following M (M≥2) candidate motions as the second subgroup, and the remaining motion candidates as the third subgroup. For example, the motion candidates can be divided into several groups according to the motion candidate type. A BV candidate can be determined to be valid when it satisfies one of the following conditions or a combination of at least one condition. 1) The corresponding reference block has been reconstructed within the current image. 2) The corresponding reference block and the current block are located in the same CTU row. 3) The corresponding reference block is in the same tile / sub-image as the current block. 4) The corresponding reference block and the current block are located in the same stripe. 5) The BV candidate satisfies the block vector constraints (e.g., as described in Sections 2.4.2 and 2.4.3). 6) The BV candidate satisfies the IBC virtual buffer conditions (such as those described in Section 2.4.4). When a BV candidate violates one of the above conditions or a combination of at least one of the conditions, it may be determined to be invalid. If the CU at a certain temporal position is invalid or intra-coded or located outside the current CTU row or its BV (if any) is invalid for the current block, then the temporal position is considered invalid; otherwise, the temporal position is considered valid. Note that the proposed method can be applied to the candidate list construction process of merging inter-coded blocks (e.g., translational motion), affine-coded blocks, TM-coded blocks, GPM-coded blocks, ADMVR-coded blocks, or IBC-coded blocks; or other motion candidate list construction processes (e.g., ordinary AMVP list; affine AMVP list; TMAMVP list; IBCAMVP list; HMVP table). Note that the proposed method can be applied to the regular merge / AMVP candidate list after the DMVR / multi-pass DMVR process. Note that the proposed method can be applied to the ADMVR merge candidate list after the DMVR / multi-pass DMVR process. Note that the proposed method can be applied to TM merging / AMVP candidate list after block-based bilateral matching refinement and / or template matching refinement and / or sub-block-based bilateral matching refinement. In the following, a motion candidate that is a repeat of another motion candidate may be defined as 1) Both candidate motions are exactly the same in terms of reference direction, reference image index, affine model (for affine motion), and motion vector or block vector or partial or full CPMV depending on the motion type (inter or IBC or affine). or 2) The two motion candidates are exactly the same in reference direction, reference image index, affine model (for affine motion). However, the difference of motion vector or block vector or part or all CPMV of the two motion candidates according to the motion type (inter or IBC or affine) can be within the threshold. The difference of a motion vector may be defined as the absolute horizontal and / or vertical component of the difference of motion vectors of two motion candidates in reference list 0 and / or reference list 1. The difference of the block vectors may be defined as the absolute horizontal and / or vertical components of the difference of the block vectors of the two motion candidates in reference list 0. A CPMV difference may be defined as the absolute horizontal and / or vertical component of the motion vector difference of the CPMV of two motion candidates in reference list 0 and / or reference list 1. The threshold can be adaptively set according to the coding mode of the current block (e.g., TM merging, affine merging, regular merging, etc.) and / or the current block size and / or the candidate type (e.g., adjacent spatial domain, adjacent temporal domain, non-adjacent spatial domain, non-adjacent temporal domain, HMVP, paired or STMVP, etc.), and / or QP, and / or the reference index or reference POC of the current block, and / or other coding and decoding information of the current block. About reordering of motion candidate list 1. In one example, when building the motion candidate list, a re-ranking / refinement process may be performed. a. In one example, ARMC based on motion candidate type can be used to reorder motion candidates within a candidate type according to one or some criteria. i. In one example, when constructing a motion candidate list, M candidates of a specific candidate type may be selected from the reordered candidates of that candidate type, where M may vary according to a combination of candidate types and / or a coding mode of the current block. ii. In one example, the candidate type may be an adjacent spatial domain candidate. For example, M is 4. iii. In one example, the candidate type may be an adjacent time domain candidate. For example, M is 1. iv. In one example, the candidate type may be a non-adjacent spatial domain candidate. For example, M is 3. v. In one example, the candidate type may be a non-adjacent time domain candidate. For example, M is 1. vi. In one example, the candidate type may be an HMVP candidate. For example, M is 2. vii. In one example, the candidate type can be a pairwise average candidate. For example, M is 2. viii. In one example, the candidate type may be a STMVP candidate. For example, M is 1. ix. In one example, the candidate type may be an affine candidate of adjacent inheritance. x. In one example, the candidate type may be an affine candidate of neighbor construction. xi. In one example, the candidate type may be an affine candidate of non-adjacent inheritance. xii. In one example, the candidate type may be a non-adjacent constructed affine candidate. xiii. In one example, the candidate type may be a history-based affine candidate. xiv. In one example, multiple motion candidate types (ie, candidate type combinations) may be reordered together. (i) In one example, when constructing a motion candidate list, M candidates of any specific motion candidate type may be selected from the reordered candidates in the candidate type combination, where M may vary depending on the combination of candidate types and / or the encoding and decoding mode of the current block. (ii) In one example, neighboring spatial candidates and / or HMVP candidates and / or pairwise average candidates may be reordered together. (iii) In one example, the adjacent spatial domain candidates and the adjacent temporal domain candidates may be reordered together. (iv) In one example, non-adjacent spatial candidates and non-adjacent temporal candidates may be reordered together. (v) In one example, non-adjacent spatial candidates and / or non-adjacent temporal candidates and / or HMVP candidates may be reordered together. (vi) In one example, non-adjacent spatial candidates and / or non-adjacent temporal candidates and / or HMVP candidates and / or pairwise average candidates may be reordered together. (vii) In one example, adjacent inherited affine candidates and adjacent constructed affine candidates may be reordered together. (viii) In one example, non-adjacent inherited affine candidates and non-adjacent constructed affine candidates may be reordered together. (ix) In one example, non-adjacent inherited affine candidates and / or non-adjacent constructed affine candidates and / or Or HMVP affine candidates can be re-ordered together. (x) In one example, non-adjacent inherited affine candidates and / or non-adjacent constructed affine candidates and / or HMVP affine candidates and / or pairwise average affine candidates may be re-ordered together. (xi) In one example, adjacent inherited affine candidates and non-adjacent inherited affine candidates may be reordered together. (xii) In one example, adjacent constructed affine candidates and non-adjacent constructed affine candidates may be reordered together. b. In one example, the motion candidate list can be i. In one example, the motion candidate list may be a TM merge / AMVP list after block-based bilateral matching refinement and / or template matching refinement and / or sub-block-based bilateral matching refinement. ii. In one example, the motion candidate list may be a regular merge / AMVP list after DMVR / multi-pass DMVR processing. iii. In one example, the motion candidate list may be an ADMVR merged list after DMVR / multi-pass DMVR processing. iv. In one example, the motion candidate list may be a GPM merge list (eg, AGPMList, LGPMList, or LAGPMList) after template matching refinement processing. v. In one example, the motion candidate list may be a regular merge / AMVP list. vi. In one example, the motion candidate list may be a TM merge / AMVP list. vii. In one example, the motion candidate list may be an affine merge / AMVP list. viii. In one example, the motion candidate list may be an IBC merge / AMVP list. ix. In one example, the motion candidate list may be an ADMVR merge list. x. In one example, the motion candidate list may be a GPM merge list. xi. In one example, the motion candidate list may be a TPM merge list. xii. In one example, the motion candidate list may be an MMVD merge list. xiii. In one example, the motion candidate list may be an IBC template matching candidate list. xiv. In one example, the motion candidate list may be an intra template matching candidate list. xv. In one example, the motion candidate list may be an IBC MMVD candidate list. xvi. In one example, the motion candidate list may be an IBC GPM candidate list. xvii. In one example, the motion candidate list may be an IBC TPM candidate list. xviii. In one example, the motion candidate list may be any other motion candidate list. 2. In one example, an initial motion candidate list may be derived first, followed by a reordering / refining process according to one or some criteria, and the reordered / refined list is used in the encoding / decoding process of the block. a. In one example, the motion candidate list may be: i. In one example, the motion candidate list may be a TM merge / AMVP list after block-based bilateral matching refinement and / or template matching refinement and / or sub-block-based bilateral matching refinement. ii. In one example, the motion candidate list may be a regular merge / AMVP list after DMVR / multi-pass DMVR processing. iii. In one example, the motion candidate list may be an ADMVR merged list after DMVR / multi-pass DMVR processing. iv. In one example, the motion candidate list may be a GPM merge list (eg, AGPMList, LGPMList, or LAGPMList) after template matching refinement processing. v. In one example, the motion candidate list may be a regular merge / AMVP list. vi. In one example, the motion candidate list may be a TM merge / AMVP list. vii. In one example, the motion candidate list may be an affine merge / AMVP list. viii. In one example, the motion candidate list may be an IBC merge / AMVP list. ix. In one example, the motion candidate list may be an ADMVR merge list x. In one example, the motion candidate list may be a GPM merge list. xi. In one example, the motion candidate list may be a TPM merge list. xii. In one example, the motion candidate list may be an MMVD merge list. xiii. In one example, the motion candidate list may be an IBC template matching candidate list. xiv. In one example, the motion candidate list may be an intra template matching candidate list. xv. In one example, the motion candidate list may be an IBC MMVD candidate list. xvi. In one example, the motion candidate list may be an IBC GPM candidate list. xvii. In one example, the motion candidate list may be an IBC TPM candidate list. xviii. In one example, the motion candidate list may be any other motion candidate list. Movement Candidate List Re-ranking Criteria 3. In one example, the re-ranking / refinement process can be based on template matching cost. a. In practice, the re-ranking / refinement process may be based on a bilateral matching cost. b. In one example, the cost function between the current template and the reference template can be i.SAD / MR-SAD, ii.SATD / MR-SATD, iii.SSD / MR-SSD, iv.SSE / MR-SSE, v. Weighted SAD / Weighted MR-SAD, vi. Weighted SATD / weighted MR-SATD, vii. Weighted SSD / Weighted MR-SSD, viii. Weighted SSE / weighted MR-SSE, ix. Gradient information. c. In one example, if LMCS is enabled, the current template and the reference template may consist of samples in the mapped domain. i. Optionally, the current template and the reference template may consist of samples in the original domain. d. In one example, the motion candidates may be divided into several subgroups, and the motion candidates in each subgroup may be reordered in ascending order according to the cost value based on template / bilateral matching. e. In one example, the samples of the template may be selected based on rules. i. In one example, if only the upper template is valid for the current block, template matching reordering can only use the upper template. ii. In one example, if only the left template is valid for the current block, the template matching reordering can only use the left template. iii. In one example, if both the upper template and the left template are valid for the current block, the template matching reordering may use the left template, the upper template, or both the upper template and the left template. f. In one example, the template / bilateral matching process may include a component, such as brightness. i. Optionally, the template / bilateral matching procedure can include multiple components, such as luminance and chrominance. (i) In one example, the total template / bilateral matching cost may be calculated as a weighted sum of the template / bilateral matching costs over different color components. g. In one example, the width of the left template and / or the height of the upper template may be fixed. i. In one example, the width of the left template can be 1. ii. In one example, the height of the upper template can be 1. h. In one example, for a BV candidate, some operations need to be performed. i. The reordering / processing methods for MV candidates and BV candidates may be different. ii. In one example, the BV used to locate the reference template may be cropped. (i) In one example, if the reference template is outside the current image (e.g. Fig.33A As shown in FIG. 1 , the BV used to locate the reference template can be cropped so that the reference template is located within the current picture. The cropping operation does not change the BV of the corresponding BV candidate. Fig.33B An example is shown in FIG. iii. In one example, if the reference template is outside the current image (eg Fig.33A As shown), the corresponding BV candidates can still be reordered. (i) In one example, if the reference template is outside the current image, it can be filled from the nearest sample in the current image. iv. In one example, the reference template should have been reconstructed within the current picture. (i) In one example, if the reference template is not reconstructed or is outside the current picture, the corresponding BV candidates may not be reordered. v. In one example, the samples contained in the reference template should be valid (eg, being reconstructed and within the same stripe / tile / IBC virtual buffer). (i) In addition, if some or all of the samples included in the reference template are unavailable, the above-mentioned method of processing a reference template outside the current image may be used. 4. Template shape and / or size can be adaptive. a. In one example, the template may include only neighboring samples to the left of the current block. b. In one example, the template may only include neighboring samples above the current block. c. In one example, the template shape and / or size may be selected according to the CU shape and / or size. d. In one example, the width of the left template can be selected according to the CU height. i. For example, if H<=M, the size of the left template is w1xH; otherwise, the size of the left template is w2xH. (i) In one example, M, w1, and w2 are set equal to 8, 1, and 2, respectively. e. In one example, the height of the upper template can be selected according to the width of the CU. i. For example, if W<=N, the size of the upper template is Wxh1; otherwise, the size of the upper template is Wxh2. (i) In one example, N, h1, and h2 are set equal to 8, 1, and 2, respectively. f. In one example, the width of the left template may be selected according to the CU width. i. For example, if W<=N, the size of the left template is w1xH; otherwise, the size of the left template is w2xH. (i) In one example, N, w1, and w2 are set equal to 8, 1, and 2, respectively. g. In one example, the height of the upper template can be selected according to the height of the CU. i. For example, if H<=M, the size of the upper template is Wxh1; otherwise, the size of the upper template is Wxh2. (i) In one example, M, h1, and h2 are set equal to 8, 1, and 2, respectively. h. In one example, samples of the template and reference samples of the template samples may be subsampled or downsampled before being used to calculate the cost. i. Whether and / or how to perform subsampling may depend on the CU size. ii. In one example, subsampling may not be performed on the short edge of the CU. 5. In one example, the reordering / thinning process can be based on a block boundary discontinuity metric. Fig.48 An example diagram 4800 showing example neighboring reconstructed blocks (grey) and a current predicted block (white) is shown. a. In one example, a boundary discontinuity metric (eg, disCost) may be defined as: Where P is the prediction of the current block and R is the reconstructed samples in the neighboring blocks. Fig.48 An example is shown. i. In one example, w1 is 0, w3 is 0, w2 is 1, and w4 is 1. ii. In one example, w1=w2=w3=w4=1. 6. In one example, the reordering / refinement process can be based on a block boundary discontinuity metric (eg, disCost) and / or a template matching cost (eg, tmCost). a. In one example, the reordering / refinement process can be based on a weighted average of a block boundary discontinuity metric (e.g., disCost) and a template matching cost (e.g., tmCost), which can be calculated as w dis ×disCostt+w tm ×tmCost. i. In one example, w dis and w tm It can be predefined, or sent by signal, or derived from codec information. ii. In one example, w dis May depend on QP. (i) In one example, for higher QP, w dis Can be set to a smaller value. b. In one example, a flag may be signaled to indicate whether to use the block boundary discontinuity metric or the template matching cost. c. In one example, whether to use the block boundary discontinuity metric or the template matching cost may depend on the encoding / decoding information. i. In one example, for QPs greater than a predefined threshold, a template matching cost may be used. 2.33. Improvements to Adaptive Reordering of Merge Candidate Lists (ARMC) This paper introduces some modifications to the Merge Candidate List Adaptive Reordering (ARMC) process. The proposed modifications are applied after the first reordering pass. This change consists of reordering the motion predictor list based on the distortion calculated during the first sorting to generate a final list for a more diverse merge candidate list. 2.33.1 Diversity Re-ranking To create diversity in the merge candidate list, the proposed method looks for candidates that are too redundant in the rate-distortion (RD) sense. In the proposed algorithm, a candidate is considered redundant if the cost difference between it and its predecessor is below the lambda value, such as |D1-D2|<λ, where D1 and D2 are the costs obtained during the first ARMC sorting and λ is the Lagrangian parameter used in the encoder-side RD criterion. The proposed algorithm is defined as follows: -Determine the minimum cost difference between a candidate and its predecessor among all candidates in the list. - If the minimum cost difference is greater than or equal to λ, the list is considered diverse enough and the re-ranking stops. - If the minimum cost difference is less than λ, the candidate is considered redundant and is moved to another position in the list. This further position is the first position where the candidate has sufficient diversity compared to its predecessor. - The algorithm stops after a finite number of iterations (if the minimum cost difference is not less than λ). This algorithm works for the regular, TM, BM, and affine merge modes of ECM-5.0. Similar algorithms work for the merged MMVD and signed MVD prediction methods, which also use ARMC for reordering. For low-latency configurations, the value of λ is set equal to the rate-distortion criterion λ used to select the best merge candidate at the encoder side, and for random access configurations, it is set equal to the value λ corresponding to another QP. For QP offsets not present in the SPS, a set of λ values ​​corresponding to each signal QP offset is provided in the SPS or Slice Header. 2.33.2 Modification of Merge Mode The following modifications have been applied to the regular merge list: - Use the same motion vector threshold for temporal, non-neighboring and other candidates. - ARMC reordering is not performed on subgroups. - Add up to 4 pairwise candidates after the first ARMC reordering for regular, TM and BM merge modes. 2.34. Combination of CIIP with TIMD and TM In CIIP mode, prediction samples are generated by weighting the inter prediction signal predicted using CIIP-TM merged candidate predictions and the intra prediction signal predicted using TIMD derived intra prediction modes. This method is only applicable to codec blocks with an area less than or equal to 1024. The TIMD derivation method is used to derive the intra prediction mode in CIIP. Specifically, the intra prediction mode with the smallest SATD value in the TIMD mode list is selected and mapped to one of the 67 conventional intra prediction modes. Furthermore, it is proposed that if the derived intra prediction mode is an angular mode, the weights of the two tests (wIntra, wInter) are modified. Fig.49A and Fig.49B The angle mode division method is shown respectively. For the nearly horizontal mode (2 <= angle mode index < 34), the current block 4900 is divided vertically, such as Fig.49A As shown; for the near vertical mode (34 <= angle mode index <= 66), the current block 4950 is divided horizontally, such as Fig.49B shown. The (wIntra, wInter) of different sub-blocks are shown in Table 2. Table 2 Modified weights for angle mode Sub-block index (wIntra,wInter) 0 (6,2) 1 (5,3) 2 (3,5) 3 (2,6) Using CIIP-TM, a CIIP-TM merge candidate list is constructed for the CIIP-TM mode. The merge candidates are refined by template matching. The ARMC method also reorders the CIIP-TM merge candidates into regular merge candidates. The maximum number of CIIP-TM merge candidates is equal to two. 2.35. Multiple Hypothesis Prediction (MHP) In the multi-hypothesis inter-frame prediction mode, in addition to the traditional bidirectional prediction signal, one or more additional motion compensation prediction signals are sent. The final overall prediction signal is obtained by weighted superposition of each sample. bi and the first additional inter-frame prediction signal / hypothesis h3, the final prediction signal p3 can be obtained as follows: p3=(1-α)P bi +αh3 The weighting factors are specified by a new syntax element add_hyp_weight_idx according to the following mapping: add_hyp_weight_idx α 0 1 / 4 1 -1 / 8 Similar to the above, multiple additional prediction signals may be used. The final overall prediction signal is iteratively accumulated with each additional prediction signal. p ｎ+1 =(1-α n+1 ) n +α ｎ+1 h n+1 The final overall prediction signal is the last p n (i.e. the p with the largest index n n ). In this EE, at most two additional prediction signals can be used (i.e., n is limited to 2). The motion parameters for each additional prediction hypothesis can be signaled either explicitly by specifying the reference index, motion vector predictor index, and motion vector difference, or implicitly by specifying the merge index. A separate multi-hypothesis merge flag distinguishes these two signaling modes. For inter-AMVP mode, MHP is applied only when unequal weights in BCW are selected in bi-prediction mode. A combination of MHP and BDOF is possible, but BDOF is only applicable to the bi-prediction signal part of the prediction signal (ie the ordinary first two hypotheses). 2.36. Bidirectional matching AMVP merge mode The bidirectional predictor consists of an AMVP predictor in one direction and a merged predictor in the other direction. This mode can be enabled for a codec block when the selected merged predictor and AMVP predictor meet the DMVR condition, where there is at least one reference picture from the past and one reference picture from the future relative to the current picture, and the two reference pictures are at the same distance to the current picture, and bidirectional matching MV refinement is applied to the merged MV candidate and the AMVP MVP as a starting point. Otherwise, if the template matching function is enabled, the template matching MV refinement is applied to the merged predictor or the AMVP predictor with a higher template matching cost. The AMVP part of the pattern is signaled as regular unidirectional AMVP, i.e. the reference index and MVD are signaled, and if template matching is used, it has a derived MVP index, or the MVP index is signaled when template matching is disabled. For AMVP direction LX, X can be 0 or 1, and the merge part of the other direction (1–LX) is implicitly derived by minimizing the bilateral matching cost between the AMVP predictor and the merge predictor, that is, for a pair of AMVP and merge motion vectors. For each merge candidate in the merge candidate list with a motion vector in the other direction (1–LX), the bilateral matching cost is calculated using the merge candidate MV and the AMVP MV. The merge candidate with the minimum cost is selected. The bilateral matching refinement is applied to the codec block starting from the selected merge candidate MV and AMVP MV. The third pass of multi-pass DMVR is the 8x8 sub-PUBDOF refinement of multi-pass DMVR, which is enabled to the AMVP merge mode codec block. The mode is indicated by a flag, and if the mode is enabled, the AMVP direction LX is also indicated by a flag. When the current block uses bilateral matching (BM) AMVP merge mode and template matching is enabled, no MVD signal is emitted. An additional pair of AMVP merge MVPs is introduced. The merge candidate list is sorted in ascending order based on the BM cost. An index (0 or 1) is emitted to indicate which merge candidate in the sorted merge candidate list to use. When there is only one candidate in the merge candidate list, the AMVP MVP and merge MVP pairs without bilateral matching MV refinement are filled in. 2.37. Geometric Partition Mode (GPM) with Merged Motion Vector Difference (MMVD) GPM in VVC is extended by applying motion vector refinement on top of the existing GPM unidirectional MV. First a flag is emitted for the GPM CU to specify whether to use this mode. If this mode is used, each geometry partition of the GPM CU can further decide whether to emit an MVD signal. If MVD is signaled for a geometry partition, after selecting a GPM merge candidate, the motion of the partition will be further refined by the signaled MVD information. All other procedures are the same as for GPM. MVD is signaled in the form of distance and direction pairs, similar to MMVD. GPM with MMVD (GPM-MMVD) involves 9 candidate distances (1 / 4 pixel, 1 / 2 pixel, 1 pixel, 2 pixels, 3 pixels, 4 pixels, 6 pixels, 8 pixels, 16 pixels) and 8 candidate directions (4 horizontal / vertical directions and 4 diagonal directions). In addition, when pic_fpel_mmvd_enabled_flag is equal to 1, MVD is left-shifted by 2 like MMVD. 2.38. Geometric Partitioning Pattern (GPM) with Template Matching (TM) Template matching is applied to GPM. When GPM mode is enabled for a CU, a CU-level flag is emitted to indicate whether TM is applied to two geometry partitions. The motion information of each geometry partition is refined using TM. When TM is selected, a template is constructed using left, top, or left and top neighboring samples depending on the partition angle, as shown in Table 3. The motion is then refined by minimizing the difference between the current template and the template in the reference picture using the same search pattern as in merge mode (half-pixel interpolation filter disabled). Table 3. Templates for the first and second geometric partitions, where A means using the upper sample, L means using the left sample, and L+A means using both the left and upper samples. Partition Angle 0 2 3 4 5 8 11 12 13 14 First Division A A A A L+A L+A L+A L+A A A Second Division L+A L+A L+A L L L L L+A L+A L+A Partition Angle 16 18 19 20 21 24 27 28 29 30 First Division A A A A L+A L+A L+A L+A A A Second Division L+A L+A L+A L L L L L+A L+A L+A The GPM candidate list is constructed as follows: 1. Interleaved list-0MV candidates and list-1MV candidates are directly from the regular merge candidate list, where list-0MV candidates have higher priority than list-1MV candidates. An adaptive threshold pruning method based on the current CU size is used to remove redundant MV candidates. 2. Interleaved list 1 MV candidates and list 0 MV candidates are further directly from the regular merge candidate list, where list 1 MV candidates have higher priority than list 0 MV candidates. The same adaptive threshold pruning method is also used to remove redundant MV candidates. 3. Fill in zero MV candidates until the GPM candidate list is full. GPM-MMVD and GPM-TM are dedicated to one GPM CU. This is achieved by sending the GPM-MMVD syntax first. When both GPM-MMVD control flags are equal to false (i.e., GPM-MMVD is disabled for both GPM partitions), a GPM-TM flag signal is issued to indicate whether template matching is applied to both GPM partitions. Otherwise (at least one GPM-MMVD flag is equal to true), the value of the GPM-TM flag is inferred to be false. 2.39. GPM with inter-frame and intra-frame prediction In GPM with inter and intra prediction, the final prediction samples are generated by weighting the inter prediction samples and intra prediction samples for each GPM separation region. The inter prediction samples are derived from the inter GPM, while the intra prediction samples are derived from the intra prediction mode (IPM) candidate list and the index sent from the encoder. The IPM candidate list size is predefined to be 3. FIG. 50A to FIG. 50D A GPM with inter-frame and intra-frame prediction is shown. FIG. 50A to FIG. 50C Example graphs 5000, 5020, and 5040, respectively, illustrate available IPM candidates. Fig.50D An example diagram 5060 of a GPM with intra and intra prediction is shown. The available IPM candidates are FIG. 50A to FIG. 50C The angle mode shown is parallel to the GPM block boundary (parallel mode), the angle mode perpendicular to the GPM block boundary (perpendicular mode), and the planar mode. Fig.50D The GPM shown with intra and intra prediction is restricted to reduce the signaling overhead of IPM and avoid increasing the size of intra prediction circuitry on the hardware decoder. In addition, direct motion vector and IPM storage on GPM mixed regions is introduced to further improve coding performance. In the IPM derivation based on DIMD and neighboring modes, the parallel modes are registered first. Therefore, if there are no identical IPM candidates in the list, up to two IPM candidates derived from the decoder-side intra mode derivation (DIMD) method and / or neighboring blocks can be registered. As for the neighboring mode derivation, there are up to 5 positions of available neighboring blocks, but they are limited by the GPM block boundary angle (as shown in Table 4), which have been used for GPM with template matching (GPM-TM). Table 4. Positions of available neighboring blocks for IPM candidate derivation based on GPM block boundary angle. A and L represent the above and left side of the prediction block. GPM angle 0 2 3 4 5 8 11 12 13 14 First Division A A A A L+A L+A L+A L+A A A Second Division L+A L+A L+A L L L L L+A L+A L+A Partition Angle 16 18 19 20 21 24 27 28 29 30 First Division A A A A L+A L+A L+A L+A A A Second Division L+A L+A L+A L L L L L+A L+A L+A GPM-intra can be combined with GPM with merged motion vector difference (GPM-MMVD). TIMD is used for IPM candidates of GPM-intra to further improve encoding performance. Parallel mode can be registered first, and then TIMD, DIMD, and IPM candidates of neighboring blocks can be registered. 2.40. Reordering of GPM partition patterns based on template matching In template matching based GPM partition mode reordering, given the motion information of the current GPM block, the corresponding TM cost value of the GPM partition mode is calculated. Then, all GPM partition modes are reordered in ascending order according to the TM cost value. Instead of sending the GPM partition mode, an index is sent using the Golomb-Rice code to indicate the exact position of the GPM partition mode in the reorder list. The GPM partition pattern reordering method is a two-step process performed after generating the corresponding reference templates for the two GPM partitions in the coding unit as follows: Extend the GPM partition edge into the reference templates of the two GPM partitions to obtain 64 reference templates, and calculate the corresponding TM cost for each of the 64 reference templates; Re-sort the GPM partitioning patterns in ascending order according to their TM cost values ​​and mark the best 32 as usable partitioning patterns. Fig.51 An example diagram 5100 showing the top edge of a display template is shown. Fig.51 As shown, the edge on the template extends from the edge of the current CU, but the template area on the edge does not use the GPM blending process. After reordering in ascending order using TM cost, an index signal is issued. MVD symbol prediction In this method, possible MVD symbol combinations are sorted according to the template matching cost, and the index corresponding to the real MVD symbol is derived and context encoded. At the decoder side, the MVD symbol is derived as follows: 1. Analyze the size of the MVD components. 2. Parse the MVD symbol prediction index of the context encoding. 3. Construct MV candidates by creating combinations between possible symbols and absolute MVD values ​​and add them to the MV predictor. 4. Derive the MVD symbol prediction cost for each derived MV and sort them according to the template matching cost. 5. Use the MVD symbol prediction index to select the true MVD symbol. MVD sign prediction is applicable to Inter-AMVP, Affine AMVP, MMVD, and Affine MMVD modes. 3. Question The current design of the sports candidate list can be further improved. More motion candidate positions and / or more motion candidate types may be used to improve encoding and decoding efficiency. 4. Specific plan The following detailed embodiments should be considered as examples to explain the general concept. These embodiments should not be interpreted narrowly. In addition, these embodiments can be combined in any way. The term "block" may refer to a codec tree block (CTB), a codec tree unit (CTU), a codec block (CB), a CU, a PU, a TU, a PB, a TB, or a video processing unit including multiple samples / pixels, and a block may be rectangular or non-rectangular. W and H are the width and...

Claims

1. A method for video processing, comprising: determining, for a conversion between a current video block of a video and a bitstream of the video, a sign prediction of a block vector difference (BVD) of the current video block; determining the BVD based on the sign prediction of the BVD; as well as The converting is performed based on the BVD.

2. The method of claim 1, wherein the symbol prediction based on the BVD is applied to at least one of the following: Intra-block copy (IBC) Advanced Motion Vector Prediction (AMVP) mode; IBC merge mode with motion vector differencing (MMVD); or IBC merge mode with BVD (MBVD).

3. The method of claim 1 or 2, wherein determining the sign prediction of the BVD comprises: determining a plurality of costs associated with a plurality of candidate symbol predictions for the BVD; as well as The sign prediction of the BVD is determined based on the plurality of costs.

4. The method of claim 3, wherein a cost in the plurality of costs is based on at least one of the following metrics for a corresponding candidate symbol prediction in the plurality of candidate symbol predictions: Template matching cost, bilateral matching costs, or Block boundary discontinuity measure. 5 . The method of claim 4 , wherein the cost is determined based on a weighted average of at least two of: the template matching cost, the bilateral matching cost, and the block boundary discontinuity metric.

6. The method according to any one of claims 3-5, wherein a candidate symbol prediction among the plurality of candidate symbol predictions comprises a combination of a first symbol prediction component of a first component of the BVD and a second symbol prediction component of a second component of the BVD.

7. The method of any one of claims 3-6, wherein determining the plurality of costs associated with the plurality of candidate symbol predictions comprises: determining a magnitude of the BVD; determining a plurality of BV candidates based on the magnitude of the BVD, the plurality of candidate symbol predictions, and a block vector (BV) prediction for the current video block; as well as The plurality of costs for the plurality of BV candidates are determined.

8. The method of claim 7, wherein the magnitude of the BVD comprises a first magnitude of a first component of the BVD and a second magnitude of a second component of the BVD, and For a candidate symbol prediction among the plurality of candidate symbol predictions, a corresponding BV candidate among the plurality of BV candidates is determined based on the first magnitude, the second magnitude, the first component of the candidate symbol prediction, the second component of the candidate symbol prediction, and the BV prediction.

9. The method of claim 8, wherein the corresponding BV candidate is determined by: determining a first BVD component based on the first component of the candidate symbol prediction and the first magnitude; determining a second BVD component based on the second component of the candidate symbol prediction and the second magnitude; as well as A sum of a combination of the first BVD component and the second BVD component and the BV prediction is determined as the corresponding BV candidate.

10. The method of any one of claims 3-9, wherein determining the sign prediction of the BVD based on the plurality of costs comprises: determining a plurality of indices of the plurality of candidate symbol predictions based on the ranking of the plurality of costs; determining a target index among the plurality of indices based on a comparison between a true BVD symbol and a plurality of ranked candidate symbol predictions, the target index being associated with the true BVD symbol; as well as The true BVD symbol is determined based on the target index. The method according to claim 10 , wherein the target index is encoded and decoded by context coding.

12. A method for video processing, comprising: determining, for conversion between a current video block of a video and a bitstream of the video, a cost associated with the current video block based on at least one of: a template matching cost associated with the current video block, a bilateral matching cost associated with the current video block, or a block boundary discontinuity metric associated with the current video block; performing processing on the current video block based on the cost; and The converting is performed based on the processing.

13. The method of claim 12, wherein the cost comprises a weighted average of at least two of: the template matching cost, the bilateral matching cost, or the block boundary discontinuity measure.

14. The method according to claim 12 or 13, wherein the processing comprises at least one of the following: reordering process, or Refinement process.

15. The method of any one of claims 12-14, wherein performing the processing on the current video block comprises: A reordering process is performed on a plurality of candidates of at least one of: a motion candidate list of the current video block, or an intra candidate list of the current video block.

16. The method of claim 15, wherein determining the cost associated with the current video block comprises: determining a plurality of costs for the plurality of candidates of at least one of: a motion candidate list or an intra candidate list; as well as The reordering process includes: The reordering process is performed based on the plurality of costs.

17. The method of claim 15 or 16, wherein the reordering process comprises an adaptive reordering with merge candidates (ARMC) process.

18. The method according to any one of claims 15 to 17, wherein performing the reordering process based on the plurality of costs comprises: determining a first cost for a first candidate among the plurality of candidates and a second cost for a second candidate among the plurality of candidates; as well as The reordering process is performed based on the first and second costs.

19. The method of claim 18, wherein the first candidate is a redundant candidate if a cost difference between the first cost and the second cost is less than or equal to a threshold difference, the cost difference representing a difference between the first candidate and the second candidate.

20. The method of claim 19, wherein the threshold difference comprises a Lagrangian parameter associated with a rate-distortion (RD) process.

21. The method according to any one of claims 16 to 20, wherein the corresponding cost of a candidate among the plurality of candidates is determined based on at least one of the following: a template matching cost of the candidate, a bilateral matching cost of the candidate, or a block boundary discontinuity metric of the candidate.

22. The method of claim 21, wherein the corresponding cost comprises a weighted average of at least two of: a template matching cost of the candidate, a bilateral matching cost of the candidate, or a block boundary discontinuity metric of the candidate.

23. The method of any one of claims 16-22, wherein the plurality of costs determined in an adaptive re-order merge candidate (ARMC) process is determined based on a category of at least one of: a motion candidate list or an intra candidate list.

24. The method of claim 18, wherein the first candidate is a redundant candidate if a weighted cost difference between the first cost and the second cost is less than or equal to a weighted threshold difference.

25. The method of claim 24, wherein the first weight of the first cost is based on a first candidate position of the first candidate, and / or A second weight of the second cost is based on a second candidate position of the second candidate.

26. The method of claim 24 or 25, wherein the third weight of the weighted threshold difference is based on a quantization parameter (QP).

27. The method of claim 24, wherein at least one of a first weight of the first cost, a second weight of the second cost, or a third weight of the weighted threshold is based on a category of at least one of: the motion candidate list or the intra candidate list.

28. The method of any one of claims 15-27, wherein a threshold difference for the reordering process is based on a configuration of the transformation.

29. The method of claim 28, wherein the configuration comprises at least one of: Full Intraframe (AI) configuration, Low latency B (LB) configuration, Low latency P (LP) configuration, Random Access (RA) configuration.

30. The method of any one of claims 15-29, wherein a threshold difference for the re-ranking process is based on a category of at least one of: the motion candidate list or the intra candidate list.

31. The method according to any one of claims 15-30, wherein a threshold difference for the reordering process is predefined.

32. The method of claim 31, wherein the threshold difference of the reordering process is predefined as a Lagrangian parameter associated with a rate-distortion (RD) process, or a square root of the Lagrangian parameter.

33. The method of any one of claims 15-32, wherein the threshold difference of the reordering process is based on a plurality of costs of the plurality of candidates, the plurality of costs being determined during a Merge Candidate Adaptive Reordering (ARMC) process.

34. The method of claim 33, wherein the threshold difference comprises a Lagrangian parameter associated with a rate-distortion (RD) process if the plurality of costs are determined based on a metric associated with a squared error, the metric comprising at least one of: Sum of squared differences (SSD), Sum of Squared Error (SSE), Mean removed sum of squared differences (MR-SSD), Mean removed sum of squared errors (MR-SSE), Weighted SSD, Weighted SSE, Weighted MR-SSD, or Weighted MR-SSE.

35. The method of claim 33, wherein the threshold difference comprises a square root of a Lagrangian parameter associated with a rate-distortion (RD) process if the plurality of costs are determined based on a metric associated with an absolute error, the metric comprising at least one of: Sum of Absolute Difference (SAD), Sum of Absolute Transform Difference (SATD), Mean removed sum of absolute differences (MR-SAD), Mean removed sum of absolute transformed differences (MR-SATD), Weighted SAD, Weighted SATD, Weighted MR-SAD, or Weighted MR-SATD.

36. The method according to any one of claims 15 to 35, further comprising: A threshold difference for the reordering process is determined based on at least one of: temporal information associated with the current video block, spatial information associated with the current video block, or codec information associated with the current video block.

37. The method of claim 36, wherein the codec information comprises a codec mode.

38. The method according to any one of claims 15 to 37, further comprising: A threshold difference for the reordering process is determined based on codec information of neighboring blocks of the current video block.

39. The method according to claim 38, wherein the codec information of the neighboring blocks comprises a codec mode of the neighboring blocks.

40. The method according to claim 38 or 39, wherein the neighboring blocks include at least one of the following: The spatial neighboring blocks of the current video block, or The temporal neighboring blocks of the current video block.

41. The method of claim 40, wherein the spatially neighboring blocks include at least one of the following: The left neighboring block on the left side of the lower left sample of the current video block, a lower left neighboring block below the left neighboring block, The upper neighboring block above the upper right sample of the current video block, the upper right adjacent block to the right of the upper adjacent block, or The upper left neighboring block above and to the left of the upper left sample of the current video block.

42. The method according to claim 40 or 41, wherein the temporal neighboring blocks include at least one of the following: a first temporal neighboring block located at the center of the co-located video block of the current video block, or A second time-domain neighboring block is located at a lower right position of the co-located video block of the current video block.

43. The method according to any one of claims 15-42, further comprising: A threshold difference for the reordering process is determined based on a codec parameter of the current video block.

44. The method according to claim 43, wherein the codec parameters include at least one of the following: the block size of the current video block, the temporal layer of the current video block, the quantization parameter (QP) of the current video block, the picture resolution associated with the current video block, or A slice type associated with the current video block.

45. The method of any one of claims 28-44, wherein the threshold difference is included in the bitstream.

46. ​​The method of claim 45, wherein the threshold difference is comprised of at least one of: Sequence level, picture group level, picture level, strip level, tile level, tile group level.

47. The method of claim 45 or 46, wherein the threshold difference is comprised of at least one of: Sequence header, picture header, sequence parameter set (SPS), video parameter set (VPS), decoding parameter set (DPS), decoding capability information (DCI), picture parameter set (PPS), adaptation parameter set (APS), slice header or tile group header.

48. The method of any one of claims 45-47, wherein the threshold difference is indicated in a region containing more than one sample or pixel.

49. The method of claim 48, wherein the region comprises at least one of a prediction block (PB), a transform block (TB), a codec block (CB), a prediction unit (PU), a transform unit (TU), a codec unit (CU), a virtual pipeline data unit (VPDU), a codec tree unit (CTU), a CTU row, a slice, a tile, and a sub-picture.

50. The method according to any one of claims 15-49, wherein the motion candidate list comprises one of the following: Template Match (TM) merge list after at least one of the following processes: block-based bilateral match refinement, template match refinement, or sub-block-based bilateral match refinement, A TM Advanced Motion Vector Prediction (AMVP) list after at least one of the following processes: block-based bilateral matching refinement, template matching refinement, or sub-block-based bilateral matching refinement, A regular merge list after at least one of the following processes: decoder-side motion vector refinement (DMVR) or a multi-pass DMVR process, Regular AMVP listing after at least one of the following processes: DMVR or multi-pass DMVR process, Advanced DMVR (ADMVR) merged list after at least one of the following processes: DMVR or multi-pass DMVR process, The merged list of geometric partitioning patterns (GPMs) after the template matching refinement process, Regular merged list or regular AMVP list, TM Merged List or TM AMVP List, Affine merged list or affine AMVP list, Intra Block Copy (IBC) merge list or IBC AMVP list, ADMVR merge list, GPM merge list, Triangle Partitioning Mode (TPM) merge list, Merge mode with motion vector differences (MMVD) merge list, MMVD candidate list, MMVD base candidate list, IBC template matching candidate list, Intra-frame template matching candidate list, IBC MMVD Candidate List, IBC merge mode with block vector differences (MBVD) candidate list, MBVD base candidate list, IBC GPM Candidate List, IBC TPM Candidate List, GPM MMVD Foundation Candidate List, Affine MMVD basis candidate list, GPM MMVD Candidate List, Affine MMVD candidate list, CIIP-TM Merger Candidate List, CIIP Merger Candidate List, For the motion candidate list of multiple hypothesis prediction (MHP), Motion candidate list for bilateral matching AMVP merge mode, The list of motion candidates associated with the GPM partitioning mode, a motion candidate list associated with sign prediction of motion vector differences (MVD) for inter-AMVP and / or affine AMVP and / or MMVD and / or affine MMVD modes, or Motion candidate lists associated with sign prediction of block vector differences (BVD) for IBC AMVP and / or IBC MMVD / MBVD modes.

51. The method of claim 50, wherein the GPM merge list after the template matching refinement process comprises at least one of the following: AGPMList, LGPMList, or LAGPMList.

52. The method of any one of claims 1-51, wherein the converting comprises encoding the current video block into the bitstream.

53. The method of any one of claims 1-51, wherein the converting comprises decoding the current video block from the bitstream.

54. An apparatus for video processing, comprising a processor and a non-transitory memory having instructions stored thereon, wherein: When the instructions are executed by the processor, the processor is caused to perform the method according to any one of claims 1 to 53.

55. A transient computer readable storage medium storing instructions that cause a processor to perform the method of any one of claims 1 to 53.

56. A transient computer-readable storage medium storing a bitstream of a video generated by a method performed by a video processing device, wherein the method comprises: determining a sign prediction of a block vector difference (BVD) for a current video block of the video; determining the BVD based at least on the sign prediction of the BVD; as well as The bitstream is generated based on the BVD.

57. A method for storing a bitstream of a video, comprising: determining a sign prediction of a block vector difference (BVD) for a current video block of the video; determining the BVD based at least on the sign prediction of the BVD; generating the bitstream based on the BVD; as well as The bitstream is stored in a transitory computer-readable recording medium.

58. A transient computer-readable recording medium storing a bitstream of a video generated by a method performed by a video processing device, wherein the method comprises: determining a cost associated with a current video block of the video based on at least one of: a template matching cost associated with the current video block, a bilateral matching cost associated with the current video block, or a block boundary discontinuity metric associated with the current video block; performing processing on the current video block based on the cost; as well as The bitstream is generated based on the processing.

59. A method for storing a bitstream of a video, comprising: determining a cost associated with a current video block of the video based on at least one of: a template matching cost associated with the current video block, a bilateral matching cost associated with the current video block, or a block boundary discontinuity metric associated with the current video block; performing processing on the current video block based on the cost; generating the bitstream based on the processing; as well as The bitstream is stored in a transitory computer-readable recording medium.

Images