Method and device for video processing and medium

Abstract

Embodiments of the present disclosure provide a solution for video processing. A method for video processing is presented. The method includes determining, for a conversion between a current video block of a video and a code stream of the video, at least one history-based motion vector prediction (HMVP) table of the current video block based on at least one of: an MV difference threshold for comparing at least two motion vectors (MVs) of at least two HMVP candidates, a codec mode of the current video block, or a reference picture list index associated with the current video block; and performing a conversion based on the at least one HMVP table.

Description

Technical Field

[0001] Embodiments of the present disclosure relate generally to video coding techniques, and more particularly, to history-based motion vector prediction (HMVP) table and motion candidate list construction. Background Art

[0002] Nowadays, digital video capabilities are being applied to all aspects of people's lives. Various types of video compression technologies have been proposed for video encoding / decoding, such as MPEG-2, MPEG-4, ITU-TH.263, ITU-TH.264 / MPEG-4 Part 10 Advanced Video Codec (AVC), ITU-TH.265 High Efficiency Video Codec (HEVC) standard, and Versatile Video Codec (VVC) standard. However, the encoding and decoding efficiency of conventional video encoding and decoding technologies is usually very low, which is undesirable. Summary of the invention

[0003] Embodiments of the present disclosure provide a solution for video processing.

[0004] In a first aspect, a method for video processing is proposed. The method includes: for conversion between a current video block of a video and a bitstream of the video, determining at least one history-based motion vector prediction (HMVP) table of the current video block based on at least one of the following: an MV difference threshold for comparing at least two motion vectors (MVs) of at least two HMVP candidates, a codec mode of the current video block, or a reference picture list index associated with the current video block; and performing conversion based on at least one HMVP table. The method according to the first aspect of the present disclosure determines the HMVP table based on the MV difference threshold, the codec mode, or the reference picture list index, thereby improving the codec efficiency and codec effectiveness.

[0005] In a second aspect, another method for video processing is proposed. The method includes: for conversion between a current video block of a video and a bitstream of the video, determining whether to add the HMVP candidate to a motion candidate list based on at least one comparison between an HMVP candidate from a history-based motion vector prediction (HMVP) table and at least one candidate in a motion candidate list of the current video block; and performing conversion based on the determination. The method according to the second aspect of the present disclosure adds an HMVP candidate from the HMVP table to the motion candidate list based on a comparison between the HMVP candidate and the candidate in the motion candidate list, thereby improving encoding and decoding efficiency and encoding and decoding effectiveness.

[0006] In a third aspect, another method for video processing is proposed. The method includes: for the conversion between the current video block of the video and the bitstream of the video, based on the maximum number of motion candidates in the first candidate list of the current video block, determining the first candidate list; determining the motion candidate list of the current video block based on the first candidate list, the motion candidate list including at least one item in the intra-block copy (IBC) merge candidate list or the IBC advanced motion vector prediction (AMVP) candidate list; and performing conversion based on the motion candidate list. The method according to the third aspect of the present disclosure determines the IBC merge candidate list or the IBC AMVP candidate list based on the candidate list with the maximum number of candidates, thereby improving the coding efficiency and coding effectiveness.

[0007] In a fourth aspect, another method for video processing is proposed. The method includes: for conversion between a current video block of a video and a bitstream of the video, determining at least one maximum number of candidates of at least one candidate type in a first candidate list of the current video block; determining a first candidate list based on at least one maximum number of candidates of at least one candidate type; and performing conversion based on the first candidate list. The method according to the fourth aspect of the present disclosure determines the first candidate list based on the maximum number of candidates of at least one candidate type, thereby improving encoding and decoding efficiency and encoding and decoding effectiveness.

[0008] In a fifth aspect, another method for video processing is proposed. The method includes: for the conversion between the current video block of the video and the bitstream of the video, determining multiple candidates for the current video block, the multiple candidates are at least one candidate type; based on at least one candidate type, reordering the multiple candidates by performing an adaptive reordering (ARMC) process of merging candidates on the multiple candidates; determining a motion candidate list based on the reordered multiple candidates and at least one candidate type; and performing conversion based on the motion candidate list. The method according to the fifth aspect of the present disclosure determines a motion candidate list based on the ARMC reordered candidates of at least one candidate type, thereby improving the coding efficiency and coding effectiveness.

[0009] In a sixth aspect, another method for video processing is proposed. The method includes: for the conversion between the current video block of the video and the bitstream of the video, determining the IBC AMVP candidate list without reordering the candidates in the intra-block copy (IBC) advanced motion vector prediction (AMVP) candidate list of the current video block; and performing the conversion based on the IBC AMVP candidate list. The method according to the sixth aspect of the present disclosure determines the IBC AMVP candidate list without reordering the candidates in the IBC AMVP candidate list, thereby improving the coding efficiency and coding effectiveness.

[0010] In the seventh aspect, another method for video processing is proposed. The method includes: for the conversion between the current video block of the video and the code stream of the video, determining a first motion candidate list corresponding to the current video block of the first coding mode and a second motion candidate list corresponding to the current video block of the second coding mode of the current video block; and performing the conversion based on the first motion candidate list and the second motion candidate list, wherein the determination of the first motion candidate list is different from the determination of the second motion candidate list. The method according to the seventh aspect of the present disclosure determines different motion candidate lists corresponding to different coding modes differently, thereby improving coding efficiency and coding effectiveness.

[0011] In an eighth aspect, a device for video processing is provided. The device includes a processor and a non-volatile memory having instructions thereon. The instructions, when executed by the processor, cause the processor to perform a method according to the first, second, third, fourth, fifth, sixth or seventh aspect of the present disclosure.

[0012] In a ninth aspect, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium stores instructions for causing a processor to execute a method according to the first, second, third, fourth, fifth, sixth or seventh aspect of the present disclosure.

[0013] In a tenth aspect, another non-transitory computer-readable recording medium is provided. The non-transitory computer-readable recording medium stores a code stream of a video generated by a method performed by a video processing device. The method includes: determining at least one history-based motion vector prediction (HMVP) table of a current video block of the video based on at least one of the following: an MV difference threshold for comparing at least two motion vectors (MVs) of at least two HMVP candidates, a codec mode of the current video block, or a reference picture list index associated with the current video block; and generating a code stream based on at least one HMVP table.

[0014] In an eleventh aspect, a method for storing a bitstream of a video is provided. The method includes: determining at least one history-based motion vector prediction (HMVP) table of a current video block of the video based on at least one of the following: an MV difference threshold for comparing at least two motion vectors (MVs) of at least two HMVP candidates, a codec mode of the current video block, or a reference picture list index associated with the current video block; generating a bitstream based on the at least one HMVP table; and storing the bitstream in a non-transitory computer-readable recording medium.

[0015] In a twelfth aspect, another non-transitory computer-readable recording medium is provided. The non-transitory computer-readable recording medium stores a bitstream of a video generated by a method performed by a video processing device. The method includes: determining whether to add an HMVP candidate to a motion candidate list based on at least one comparison between an HMVP candidate from a history-based motion vector prediction (HMVP) table and at least one candidate in a motion candidate list of a current video block of the video; and generating a bitstream based on the determination.

[0016] In a thirteenth aspect, a method for storing a bitstream of a video is provided. The method includes: determining whether to add a HMVP candidate to a motion candidate list based on at least one comparison between a HMVP candidate from a history-based motion vector prediction (HMVP) table and at least one candidate in a motion candidate list of a current video block of the video; generating a bitstream based on the determination; and storing the bitstream in a non-transitory computer-readable recording medium.

[0017] In a fourteenth aspect, another non-transitory computer-readable recording medium is provided. The non-transitory computer-readable recording medium stores a code stream of a video generated by a method performed by a video processing device. The method includes: determining a first candidate list based on the maximum number of motion candidates in a first candidate list of a current video block of the video; determining a motion candidate list of the current video block based on the first candidate list, the motion candidate list including at least one of an intra-block copy (IBC) merge candidate list or an IBC advanced motion vector prediction (AMVP) candidate list; and generating a code stream based on the motion candidate list.

[0018] In a fifteenth aspect, a method for storing a bitstream of a video is provided. The method includes: determining a first candidate list based on the maximum number of motion candidates in the first candidate list of the current video block of the video; determining a motion candidate list of the current video block based on the first candidate list, the motion candidate list including at least one of an intra-block copy (IBC) merge candidate list or an IBC advanced motion vector prediction (AMVP) candidate list; generating a bitstream based on the motion candidate list; and storing the bitstream in a non-transitory computer-readable recording medium.

[0019] In a sixteenth aspect, another non-transitory computer-readable recording medium is provided. The non-transitory computer-readable recording medium stores a bitstream of a video generated by a method performed by a video processing device. The method includes: determining at least one maximum number of candidates of at least one candidate type in a first candidate list of a current video block of the video; determining a first candidate list based on at least one maximum number of candidates of at least one candidate type; and generating a bitstream based on the first candidate list.

[0020] In a seventeenth aspect, a method for storing a bitstream of a video is provided. The method includes: determining at least one maximum number of candidates of at least one candidate type in a first candidate list of a current video block of the video; determining the first candidate list based on at least one maximum number of candidates of at least one candidate type; generating a bitstream based on the first candidate list; and storing the bitstream in a non-transitory computer-readable recording medium.

[0021] In an eighteenth aspect, another non-transitory computer-readable recording medium is provided. The non-transitory computer-readable recording medium stores a bitstream of a video generated by a method performed by a video processing device. The method includes: determining multiple candidates for a current video block of the video, the multiple candidates being at least one candidate type; reordering the multiple candidates by performing an adaptive reordering of merged candidates (ARMC) process on the multiple candidates based on the at least one candidate type; determining a motion candidate list based on the reordered multiple candidates and at least one candidate type; and generating a bitstream based on the motion candidate list.

[0022] In a nineteenth aspect, a method for storing a bitstream of a video is provided. The method includes: determining multiple candidates for a current video block of the video, the multiple candidates being at least one candidate type; reordering the multiple candidates by performing an adaptive reordering of merged candidates (ARMC) process on the multiple candidates based on the at least one candidate type; determining a motion candidate list based on the reordered multiple candidates and the at least one candidate type; generating a bitstream based on the motion candidate list; and storing the bitstream in a non-transitory computer-readable recording medium.

[0023] In a twentieth aspect, another non-transitory computer-readable recording medium is provided. The non-transitory computer-readable recording medium stores a bitstream of a video generated by a method performed by a video processing device. The method includes: determining an intra-block copy (IBC) advanced motion vector prediction (AMVP) candidate list of a current video block of the video without reordering the candidates in the IBC AMVP candidate list; and generating a bitstream based on the IBC AMVP candidate list.

[0024] In a twenty-first aspect, a method for storing a bitstream of a video is provided. The method includes: determining an intra-block copy (IBC) advanced motion vector prediction (AMVP) candidate list of a current video block of the video without reordering the candidates in the IBC AMVP candidate list; generating a bitstream based on the IBC AMVP candidate list; and storing the bitstream in a non-transitory computer-readable recording medium.

[0025] In a twenty-second aspect, another non-transitory computer-readable recording medium is provided. The non-transitory computer-readable recording medium stores a code stream of a video generated by a method performed by a video processing device. The method includes: determining a first motion candidate list of a current video block of a video corresponding to a first codec mode and a second motion candidate list of a current video block corresponding to a second codec mode of the current video block; and generating a code stream based on the first motion candidate list and the second motion candidate list, wherein the determination of the first motion candidate list is different from the determination of the second motion candidate list.

[0026] In a twenty-third aspect, a method for storing a bitstream of a video is provided. The method includes: determining a first motion candidate list of a current video block of a video corresponding to a first codec mode and a second motion candidate list of a current video block corresponding to a second codec mode of the current video block; generating a bitstream based on the first motion candidate list and the second motion candidate list; and storing the bitstream in a non-transitory computer-readable recording medium, wherein the determination of the first motion candidate list is different from the determination of the second motion candidate list.

[0027] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become more apparent through the following detailed description with reference to the accompanying drawings. In the exemplary embodiments of the present disclosure, the same reference numerals generally refer to the same components.

[0029] Figure 1 A block diagram showing an example video encoding and decoding system according to some embodiments of the present disclosure is shown;

[0030] Figure 2 A block diagram of an example video encoder according to some embodiments of the present disclosure is shown;

[0031] Figure 3 A block diagram of an example video decoder according to some embodiments of the present disclosure is shown;

[0032] Figure 4 An example diagram showing example locations of spatial merge candidates is shown;

[0033] Figure 5 shows an example diagram illustrating candidate pairs considered for redundancy checking of spatial merging candidates;

[0034] Figure 6An example diagram illustrating example motion vector scaling for temporal merging candidates is shown;

[0035] Figure 7 An example diagram showing candidate positions for time domain merging candidates C0 and C1 is shown;

[0036] Figure 8 An example diagram showing VVC spatial neighboring blocks of a current block is shown;

[0037] Fig. 9 An example virtual block in the i-th round of search is shown;

[0038] Fig.10 An example diagram showing spatial neighboring blocks used to derive spatial merge candidates is shown;

[0039] Fig.11A and Fig. 11B The SbTMVP process in VVC is shown;

[0040] Figures 12A-12D The current CTU processing order and the available samples in the current CTU and the left CTU are shown;

[0041] Fig.13 shows the neighboring samples used to calculate the SAD;

[0042] Fig.14 shows neighboring samples used to calculate SAD for sub-CU level motion information;

[0043] Fig.15 An example diagram showing the sorting process is shown;

[0044] Fig.16 An example diagram illustrating a reordering process in an encoder is shown;

[0045] Fig.17 An example diagram illustrating a reordering process in a decoder is shown;

[0046] Fig.18 shows an example diagram illustrating the execution of template matching on a search area around an initial MV;

[0047] Fig.19 An example graph demonstrating template matching prediction is shown;

[0048] Fig. 20 An example diagram showing the intra-frame template matching search area used is shown;

[0049] Fig.21 shows an example diagram showing a template used in TIMD and its reference sample;

[0050] Fig. 22An example diagram showing a display template and a reference sample of the template is shown;

[0051] Fig.23 An example diagram showing templates in reference list 0 and reference list 1 and reference samples of the templates is shown;

[0052] Fig.24 An example diagram showing a reference sample of a template and a template for a block with sub-block motion using motion information of a sub-block of a current block is shown;

[0053] Fig.25 An example diagram showing a template and reference samples of the template for a block with sub-block motion using motion information of each sub-template is shown;

[0054] Fig.26 An example diagram showing a template for a block with OBMC and a reference sample of the template is shown;

[0055] Fig. 27 An example diagram showing motion estimation of a rectangular block with hash values ​​of square sub-blocks is shown;

[0056] Fig.28 An example luminance mapping with a chroma scaling architecture is shown;

[0057] Fig.29 shows example pairs of candidates in the merged reordered ARMC-TM and additional pairwise candidates after reordering;

[0058] Fig. 30A An example diagram showing candidate locations for spatial domain candidates is shown;

[0059] Fig. 30B An example diagram showing candidate positions for time domain candidates is shown;

[0060] Fig.31 An example diagram showing the derivation of a sub-CUbv motion field from a corresponding co-located sub-CU by applying a motion shift from a spatial neighbor is shown;

[0061] Fig.32 An example diagram illustrating example intra-frame template matching is shown;

[0062] Fig.33A An example diagram showing a reference template outside the current image is shown;

[0063] Fig.33B An example diagram showing cropping of the BV so that the reference template is located inside the current image is shown;

[0064] Fig.34 The non-adjacent positions used are shown;

[0065] Fig.35 An example diagram showing spatial candidates used for IBC merge / AMVP candidate list is shown;

[0066] Fig.36 An example diagram showing a display template and a reference sample of the template is shown;

[0067] Fig.37 An example location used by the TMVP is shown;

[0068] Fig.38 An example of generating a HAPC is shown;

[0069] Fig.39 The spatial neighbors used to derive affine merge candidates are shown;

[0070] Fig.40 An example of non-adjacent neighbors from a constructed affine merge candidate is shown;

[0071] Fig.41 An affine motion model based on example control points is shown;

[0072] Fig.42 An example affine MVF for each sub-block is shown;

[0073] Fig.43 Example positions of inherited affine motion predictors are shown;

[0074] Fig.44 An example control point motion vector inheritance is shown;

[0075] Fig.45 Example positions of candidate positions for a constructed affine merge pattern are shown;

[0076] Fig.46 An example illustration of the use of motion vectors of the proposed combination method is shown;

[0077] Fig.47 An example sub-block MV VSB and a pixel Δv(i,j) are shown (arrows);

[0078] Fig.48 Example neighboring reconstructed blocks and a current prediction block are shown;

[0079] Fig.49 Example neighboring reconstructed blocks and a current prediction block are shown;

[0080] Fig.50A An example reference template outside the valid IBC reference area is shown;

[0081] Fig.50BAn example is shown showing positioning of a reference template by BV relative to a current block instead of a corresponding BV candidate of a current template;

[0082] Fig.51 A flowchart of a method for video processing according to some embodiments of the present disclosure is shown;

[0083] Fig.52 Another flow chart of a method for video processing according to some embodiments of the present disclosure is shown;

[0084] Fig.53 Another flow chart of a method for video processing according to some embodiments of the present disclosure is shown;

[0085] Fig.54 Another flow chart of a method for video processing according to some embodiments of the present disclosure is shown;

[0086] Fig.55 Another flow chart of a method for video processing according to some embodiments of the present disclosure is shown;

[0087] Fig.56 Another flow chart of a method for video processing according to some embodiments of the present disclosure is shown;

[0088] Fig.57 Another flow chart showing a method for video processing according to some embodiments of the present disclosure; and

[0089] Fig.58 A block diagram of a computing device is shown in which various embodiments of the present disclosure may be implemented.

[0090] Throughout the drawings, same or similar reference numbers generally refer to same or similar elements. DETAILED DESCRIPTION

[0091] The principle of the present disclosure will now be described with reference to some embodiments. It should be understood that these embodiments are described only for the purpose of illustrating and helping those skilled in the art to understand and implement the present disclosure, without implying any limitation on the scope of the present disclosure. In addition to the methods described below, the disclosure described herein can also be implemented in various ways.

[0092] In the following description and claims, unless defined otherwise, all scientific and technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present disclosure belongs.

[0093] References in this disclosure to "one embodiment," "an embodiment," "an example embodiment," etc. indicate that the described embodiment may include a particular feature, structure, or characteristic, but not every embodiment must include the particular feature, structure, or characteristic. Furthermore, these phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in conjunction with an example embodiment, whether or not explicitly described, it is considered to be within the knowledge of those skilled in the art to affect such feature, structure, or characteristic in relation to other embodiments.

[0094] It should be understood that although the terms "first" and "second" etc. can be used to describe various elements, these elements should not be limited to these terms. These terms are only used to distinguish one element from another element. For example, a first element can be referred to as a second element, and similarly, a second element can be referred to as a first element without departing from the scope of the exemplary embodiments. As used herein, the term "and / or" includes any and all combinations of one or more of the listed terms.

[0095] The terms used herein are only used for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments. As used herein, the singular forms "a", "an", and "the" are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms "including", "comprising", and / or "having" when used herein indicate the presence of the features, elements, and / or components, etc., but do not exclude the presence or addition of one or more other features, elements, components, and / or combinations thereof. Example Environment

[0096] Figure 1 1 is a block diagram illustrating an example video codec system 100 that may utilize the techniques of the present disclosure. As shown, the video codec system 100 may include a source device 110 and a destination device 120. The source device 110 may also be referred to as a video encoding device, and the destination device 120 may also be referred to as a video decoding device. In operation, the source device 110 may be configured to generate encoded video data, and the destination device 120 may be configured to decode the encoded video data generated by the source device 110. The source device 110 may include a video source 112, a video encoder 114, and an input / output (I / O) interface 116.

[0097] Video source 112 may include a source such as a video capture device. Examples of a video capture device include, but are not limited to, an interface that receives video data from a video content provider, a computer graphics system for generating video data, and / or combinations thereof.

[0098] The video data may include one or more pictures. The video encoder 114 encodes the video data from the video source 112 to generate a code stream. The code stream may include a bit sequence that forms a coded representation of the video data. The code stream may include coded pictures and associated data. The coded picture is a coded representation of the picture. The associated data may include a sequence parameter set, a picture parameter set, and other grammatical structures. The I / O interface 116 may include a modulator / demodulator and / or a transmitter. The coded video data may be directly transmitted to the destination device 120 via the network 130A via the I / O interface 116. The coded video data may also be stored on a storage medium / server 130B for access by the destination device 120.

[0099] The destination device 120 may include an I / O interface 126, a video decoder 124, and a display device 122. The I / O interface 126 may include a receiver and / or a modem. The I / O interface 126 may obtain encoded video data from the source device 110 or the storage medium / server 130B. The video decoder 124 may decode the encoded video data. The display device 122 may display the decoded video data to the user. The display device 122 may be integrated with the destination device 120, or may be outside the destination device 120, which is configured to be connected to an external display device interface.

[0100] The video encoder 114 and the video decoder 124 may operate according to a video compression standard, such as the High Efficiency Video Codec (HEVC) standard, the Versatile Video Codec (VVC) standard, and other existing and / or further standards.

[0101] Figure 2 is a block diagram showing an example of a video encoder 200 according to some embodiments of the present disclosure, which may be Figure 1 An example of a video encoder 114 in the system 100 is shown.

[0102] Video encoder 200 may be configured to implement any or all of the techniques of this disclosure. Figure 2 In the example of , video encoder 200 includes multiple functional components. The techniques described in this disclosure can be shared between the various components of video encoder 200. In some examples, a processor can be configured to perform any or all of the techniques described in this disclosure.

[0103] In some embodiments, the video encoder 200 may include a partitioning unit 201, a prediction unit 202, a residual generation unit 207, a transformation unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transformation unit 211, a reconstruction unit 212, a buffer 213 and an entropy coding and decoding unit 214, and the prediction unit 202 may include a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205 and an intra-frame prediction unit 206.

[0104] In other examples, the video encoder 200 may include more, fewer, or different functional components. In one example, the prediction unit 202 may include an intra-block copy (IBC) unit. The IBC unit may perform prediction in an IBC mode, where at least one reference picture is a picture where the current video block is located.

[0105] Furthermore, although some components (such as the motion estimation unit 204 and the motion compensation unit 205) may be integrated, for the purpose of explanation, these components are described in detail below. Figure 2 are shown separately in the example.

[0106] The partitioning unit 201 may partition a picture into one or more video blocks. The video encoder 200 and the video decoder 300 may support various video block sizes.

[0107] The mode selection unit 203 may select one of a plurality of codec modes (intra-frame coding or inter-frame coding), for example, based on the error result, and provide the generated intra-frame codec block or inter-frame codec block to the residual generation unit 207 to generate residual block data, and to the reconstruction unit 212 to reconstruct the codec block for use as a reference picture. In some examples, the mode selection unit 203 may select a combination of intra-frame and inter-frame prediction (CIIP) modes, where the prediction is based on an inter-frame prediction signal and an intra-frame prediction signal. In the case of inter-frame prediction, the mode selection unit 203 may also select a resolution for the motion vector (e.g., sub-pixel precision or integer pixel precision) for the block.

[0108] To perform inter-frame prediction on the current video block, the motion estimation unit 204 may generate motion information for the current video block by comparing the current video block with one or more reference frames from the buffer 213. The motion compensation unit 205 may determine a predicted video block for the current video block based on the motion information and decoded samples of pictures from the buffer 213 other than the picture associated with the current video block.

[0109] The motion estimation unit 204 and the motion compensation unit 205 may perform different operations on the current video block, for example, depending on whether the current video block is in an I slice, a P slice, or a B slice. As used herein, an "I slice" may refer to a portion of a picture consisting of macroblocks, all of which are based on macroblocks within the same picture. Furthermore, as used herein, in some aspects, a "P slice" and a "B slice" may refer to a portion of a picture consisting of macroblocks that are independent of macroblocks in the same picture.

[0110] In some examples, the motion estimation unit 204 may perform unidirectional prediction on the current video block, and the motion estimation unit 204 may search the reference pictures of list 0 or list 1 to find the reference video block for the current video block. The motion estimation unit 204 may then generate a reference index and a motion vector, the reference index indicating the reference picture in list 0 or list 1 containing the reference video block, and the motion vector indicating the spatial displacement between the current video block and the reference video block. The motion estimation unit 204 may output the reference index, the prediction direction indicator, and the motion vector as the motion information of the current video block. The motion compensation unit 205 may generate a predicted video block for the current video block based on the reference video block indicated by the motion information of the current video block.

[0111] Alternatively, in other examples, the motion estimation unit 204 may perform bidirectional prediction on the current video block. The motion estimation unit 204 may search the reference pictures in list 0 for a reference video block for the current video block, and may also search the reference pictures in list 1 for another reference video block for the current video block. The motion estimation unit 204 may then generate a plurality of reference indexes and a plurality of motion vectors, the plurality of reference indexes indicating a plurality of reference pictures in list 0 and list 1 containing a plurality of reference video blocks, and the plurality of motion vectors indicating a plurality of spatial displacements between the plurality of reference video blocks and the current video block. The motion estimation unit 204 may output the plurality of reference indexes and the plurality of motion vectors of the current video block as motion information of the current video block. The motion compensation unit 205 may generate a predicted video block for the current video block based on the plurality of reference video blocks indicated by the motion information of the current video block.

[0112] In some examples, motion estimation unit 204 may output a complete set of motion information for use in a decoding process by a decoder. Alternatively, in some embodiments, motion estimation unit 204 may signal motion information of a current video block with reference to motion information of another video block. For example, motion estimation unit 204 may determine that motion information of a current video block is sufficiently similar to motion information of a neighboring video block.

[0113] In one example, motion estimation unit 204 may indicate to video decoder 300 a value in a syntax structure associated with the current video block that indicates that the current video block has the same motion information as another video block.

[0114] In another example, the motion estimation unit 204 may identify another video block and a motion vector difference (MVD) in a syntax structure associated with the current video block. The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the indicated video block. The video decoder 300 may use the motion vector of the indicated video block and the motion vector difference to determine the motion vector of the current video block.

[0115] As discussed above, the video encoder 200 may signal motion vectors in a predictive manner.Two examples of prediction signaling techniques that may be implemented by the video encoder 200 include Advanced Motion Vector Prediction (AMVP) and merge mode signaling.

[0116] The intra prediction unit 206 may perform intra prediction on the current video block. When the intra prediction unit 206 performs intra prediction on the current video block, the intra prediction unit 206 may generate prediction data for the current video block based on decoded samples of other video blocks in the same picture. The prediction data for the current video block may include a prediction video block and various syntax elements.

[0117] The residual generation unit 207 may generate residual data for the current video block by subtracting (e.g., indicated by a minus sign) the prediction video block(s) of the current video block from the current video block. The residual data of the current video block may include residual video blocks corresponding to different sample portions of samples in the current video block.

[0118] In other examples, such as in skip mode, there may be no residual data for the current video block, and the residual generation unit 207 may not perform a subtraction operation.

[0119] Transform processing unit 208 may generate one or more transform coefficient video blocks for a current video block by applying one or more transforms to the residual video block associated with the current video block.

[0120] After transform processing unit 208 generates a transform coefficient video block associated with the current video block, quantization unit 209 may quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values ​​associated with the current video block.

[0121] The inverse quantization unit 210 and the inverse transform unit 211 may apply inverse quantization and inverse transform to the transform coefficient video block, respectively, to reconstruct a residual video block from the transform coefficient video block. The reconstruction unit 212 may add the reconstructed residual video block to corresponding samples from one or more prediction video blocks generated by the prediction unit 202 to generate a reconstructed video block associated with the current video block for storage in the buffer 213.

[0122] After reconstruction unit 212 reconstructs the video block, a loop filtering operation may be performed to reduce video blocking artifacts in the video block.

[0123] The entropy coding unit 214 may receive data from other functional components of the video encoder 200. When the entropy coding unit 214 receives data, the entropy coding unit 214 may perform one or more entropy coding operations to generate entropy coded data and output a code stream including the entropy coded data.

[0124] Figure 3 is a block diagram showing an example of a video decoder 300 according to some embodiments of the present disclosure, which may be Figure 1 An example of a video decoder 124 in the system 100 is shown.

[0125] Video decoder 300 may be configured to perform any or all of the techniques of this disclosure. Figure 3 In the example of , video decoder 300 includes multiple functional components. The techniques described in this disclosure can be shared between the various components of video decoder 300. In some examples, a processor can be configured to perform any or all of the techniques described in this disclosure.

[0126] exist Figure 3 In the example of , the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, and a reconstruction unit 306 and a buffer 307. In some examples, the video decoder 300 can perform a decoding process that is generally opposite to the encoding process described with respect to the video encoder 200.

[0127] The entropy decoding unit 301 can retrieve the encoded code stream. The encoded code stream may include entropy encoded video data (e.g., encoded video data blocks). The entropy decoding unit 301 can decode the entropy encoded video data, and the motion compensation unit 302 can determine motion information from the entropy decoded video data, the motion information including motion vectors, motion vector precision, reference picture list indexes, and other motion information. The motion compensation unit 302 can determine the information, for example, by performing AMVP and merge mode. AMVP is used, including deriving several most likely candidates based on data and reference pictures of adjacent PBs. The motion information typically includes horizontal and vertical motion vector displacement values, one or two reference picture indexes, and in the case of prediction areas in B strips, also includes an identification of which reference picture list is associated with each index. As used herein, in some aspects, "merge mode" may refer to deriving motion information from blocks that are adjacent in space or time.

[0128] The motion compensation unit 302 may generate motion compensated blocks, possibly performing interpolation based on interpolation filters.Identifiers for the interpolation filters used with sub-pixel precision may be included in the syntax elements.

[0129] The motion compensation unit 302 may calculate interpolated values ​​for sub-integer pixels of a reference block using interpolation filters used by the video encoder 200 during encoding of the video block. The motion compensation unit 302 may determine the interpolation filters used by the video encoder 200 based on received syntax information, and the motion compensation unit 302 may use the interpolation filters to generate a prediction block.

[0130] The motion compensation unit 302 may use at least part of the syntax information to determine the size of blocks used to encode (multiple) frames and / or (multiple) slices of the encoded video sequence, partition information describing how each macroblock of a picture of the encoded video sequence is partitioned, a mode indicating how each partition is encoded, one or more reference frames (and reference frame lists) for each inter-frame coding block, and other information for decoding the encoded video sequence. As used herein, in some aspects, a "slice" may refer to a data structure that can be decoded independently of other slices of the same picture in terms of entropy coding, signal prediction, and residual signal reconstruction. A slice may be an entire picture, or it may be a region of a picture.

[0131] The intra prediction unit 303 may use, for example, an intra prediction mode received in the bitstream to form a prediction block from spatially adjacent blocks. The inverse quantization unit 304 inversely quantizes (i.e., dequantizes) the quantized video block coefficients provided in the bitstream and decoded by the entropy decoding unit 301. The inverse transform unit 305 applies an inverse transform.

[0132] The reconstruction unit 306 may obtain the decoded block, for example, by adding the residual block to the corresponding prediction block generated by the motion compensation unit 302 or the intra prediction unit 303. If necessary, a deblocking filter may also be applied to filter the decoded block to remove blocking artifacts. The decoded video block is then stored in a buffer 307, which provides reference blocks for subsequent motion compensation / intra prediction, and the buffer 307 also generates the decoded video for presentation on a display device.

[0133] Some example embodiments of the present disclosure will be described in detail below. It should be noted that the section titles used in this document are for ease of understanding, and the embodiments disclosed in the section are not limited to that section. In addition, although some embodiments are described with reference to general video codecs or other specific video codecs, the disclosed technology is also applicable to other video coding and decoding technologies. In addition, although some embodiments describe the video encoding steps in detail, it should be understood that the corresponding decoding steps of canceling the encoding will be implemented by the decoder. In addition, the term video processing includes video coding or compression, video decoding or decompression, and video transcoding, in which video pixels are represented from one compression format to another compression format or at different compression bit rates. 1. Brief Overview The present disclosure relates to video coding technology. Specifically, it is about IBC prediction and related technologies in video coding. It can be applied to existing video coding standards such as HEVC, VVC, etc. It can also be applied to future video coding standards or video codecs. 2. Introduction Video codec standards have evolved primarily through the development of the well-known ITU-T and ISO / IEC standards. ITU-T developed H.261 and H.263, ISO / IEC developed MPEG-1 and MPEG-4 Visual, and the two organizations jointly developed H.262 / MPEG-2 Video and H.264 / MPEG-4 Advanced Video Codec (AVC) and H.265 / HEVC standards. Since H.262, video codec standards have been based on a hybrid video codec structure, in which temporal prediction plus transform codec is used. In order to explore future video codec technologies beyond HEVC, VCEG and MPEG jointly founded the Joint Video Exploration Team (JVET) in 2015. JVET meetings are held simultaneously every quarter, and the new video codec standard was officially named Versatile Video Codec (VVC) at the JVET meeting in April 2018, and the first version of the VVC Test Model (VTM) was released at that time. The VVC working draft and the test model VTM are updated after each meeting. The VVC project achieved technical completion (FDIS) at the July 2020 meeting. 2.1. Extended Merge Prediction In VVC, the merge candidate list is constructed by including the following five types of candidates in order: 1) Spatial MVP from the spatial neighbor CU, 2) Time domain MVP from the same CU, 3) History-based MVP from FIFO table, 4) Paired average MVP, 5) Zero MV. The size of the merge list is signaled in the sequence parameter set header, and the maximum allowed size of the merge list is 6. For each CU code in merge mode, the index of the best merge candidate is encoded using truncated unary binarization (TU). The first bin of the merge index uses context encoding and decoding, while bypass encoding and decoding is used for the other bins. The derivation process of merge candidates for each category is provided in this section. As in HEVC, VVC also supports parallel derivation of merge candidate lists for all CUs within a certain size region. 2.1.1. Spatial Candidate Derivation The derivation of spatial merge candidates in VVC is the same as that in HEVC, except that the positions of the first two merge candidates are swapped. Figure 4 A schematic diagram 400 showing example locations of spatial merging candidates is shown. Figure 4 Among the candidates at the positions shown, up to four merge candidates are selected. The derivation order is B1, A1, B0, A0, and B2. Position B2 is considered only when one or more CUs at positions B0, A0, B1, and A1 are not available (for example, because it belongs to another slice or tile) or is intra-coded. After the candidate at position A1 is added, a redundancy check is performed on the addition of the remaining candidates, which ensures that candidates with the same motion information are excluded from the list, thereby improving coding efficiency. Figure 5 A schematic diagram 500 showing candidate pairs considered for redundancy check of spatial merging candidates is shown. In order to reduce computational complexity, not all possible candidate pairs are considered in the redundancy check mentioned. Instead, only Figure 5 The pairs are linked by arrows in , and candidates are added to the list only if the corresponding candidates used for redundancy checking do not have the same motion information. 2.2.2. Time Domain Candidate Derivation In this step, only one candidate is added to the list. In particular, in the derivation of this temporal merge candidate, the scaled motion vector is derived based on the co-located CU belonging to the co-located reference picture. The reference picture list to be used for deriving the co-located CU is explicitly signaled in the slice header. Figure 6 An example diagram 600 showing example motion vector scaling for a temporal merge candidate is shown. Figure 6 As shown by the dotted line in , the scaled motion vector for the temporal merge candidate is obtained, which is scaled from the motion vector of the co-located CU using the POC distances tb and td, where tb is defined as the POC difference between the reference picture of the current picture and the current picture, and td is defined as the POC difference between the reference picture of the co-located picture and the co-located picture. The reference picture index of the temporal merge candidate is set equal to zero. Figure 7 A schematic diagram 700 showing candidate positions for time domain merging candidates C0 and C1 is shown. Figure 7 As shown, the position of the temporal candidate is selected between candidates C0 and C1. If the CU at position C0 is not available, intra-coded, or outside the current row of the CTU, position C1 is used. Otherwise, position C0 is used in the derivation of the temporal merge candidate. 2.1.3. History-based Merge Candidate Derivation History-based MVP (HMVP) merge candidates are added to the merge list after spatial MVP and TMVP. In this method, the motion information of the previous codec block is stored in a table and used as the MVP of the current CU. A table with multiple HMVP candidates is maintained during the encoding / decoding process. The table is reset (cleared) when a new CTU row is encountered. Whenever there is a non-sub-block inter-coded CU, the associated motion information is added as a new HMVP candidate to the last entry of the table. The HMVP list size S is set to 6, which indicates that up to 6 history-based MVP (HMVP) candidates can be added to the table. When a new motion candidate is inserted into the table, a constrained first-in, first-out (FIFO) rule is used, where a redundancy check is first applied to find if there is an identical HMVP in the table. If found, the identical HMVP is removed from the table, and then all subsequent HMVP candidates are moved forward. HMVP candidates can be used in the merge candidate list construction process. Check the nearest HMVP candidates in the table in order and insert them into the candidate list after the TMVP candidate. Apply redundancy check to spatial merge candidates or temporal merge candidates for HMVP candidates. To reduce the number of redundant checking operations, the following simplifications are introduced: 1. The number of HMPV candidates for merge list generation is set to (N<=4)?M:(8-N), where N indicates the number of existing candidates in the merge list and M indicates the number of available HMVP candidates in the table. 2. Once the total number of available merge candidates reaches the maximum allowed merge candidate minus 1, the merge candidate list construction process from HMVP is terminated. 2.1.4. Pairwise Average Merging Candidate Derivation The pairwise average candidate is generated by averaging the predefined candidate pairs in the existing merge candidate list, and the predefined pairs are defined as {(0, 1), (0, 2), (1, 2), (0, 3), (3, 1), (2, 3)}, where the number represents the merge index of the merge candidate list. The averaged motion vector is calculated separately for each reference list. If two motion vectors are available in one list, they are averaged even if they point to different reference pictures; if only one motion vector is available, it is used directly; if no motion vector is available, this list is kept invalid. When the merge list is not full after adding pairwise average merge candidates, zero MVPs will be inserted at the end until the maximum number of merge candidates is encountered. 2.2. New Merge Candidates 2.2.1 Non-adjacent merging candidate export Figure 8 An example diagram 800 showing VVC spatial neighboring blocks of a current block is shown. In VVC, Figure 8 The five spatial neighboring blocks and one temporal neighboring block shown are used to derive merge candidates. It is proposed to derive additional merge candidates from positions non-adjacent to the current block using the same style as in VVC. To achieve this, for each search round i, a virtual block is generated based on the current block as follows: First, the relative position of the virtual block to the current block is calculated using the following formula: Offsetx=-i×gridX, Offsety=-i×gridY Where Offsetx and Offsety represent the offset of the upper left corner of the virtual block relative to the upper left corner of the current block, and gridX and gridY are the width and height of the search grid. Second, the width and height of the virtual block are calculated using the following formula: newWidth = i×2×gridX+currWidth newHeight = i×2×gridY+currHeight. currWidth and currHeight are the width and height of the current block. newWidth and newHeight are the width and height of the new virtual block. gridX and gridY are currently set to currWidth and currHeight respectively. Fig. 9A schematic diagram 900 showing virtual blocks in the i-th round of searching is shown. Fig. 9 The relationship between the virtual block and the current block is shown. After the virtual block is generated, block A i , B i , C i , D i and E i can be considered as VVC spatial neighbors of the virtual block, and their positions are obtained using the same pattern as in VVC. Obviously, if the search round i is 0, the virtual block is the current block. In this case, block A i , B i , C i , D i and E i It is the spatial neighboring block used in VVC merge mode. When constructing the merge candidate list, pruning is performed to ensure that each element in the merge candidate list is unique. The maximum search round is set to 1, which means that five non-adjacent spatial neighbor blocks are used. Non-adjacent spatial merging candidates are inserted into the merging list after the temporal merging candidates in the order of B1->A1->C1->D1->E1. 2.2.2 Non-adjacent airspace candidates Non-adjacent spatial merging candidates are inserted after TMVP in the regular merging candidate list. Fig.10 A schematic diagram 1000 is shown showing spatial neighboring blocks used to derive spatial merge candidates. The pattern of spatial merge candidates is as follows: Fig.10 As shown in Figure 2. The distance between non-adjacent spatial candidates and the current codec block is based on the width and height of the current codec block. The line buffer limit is not applied. 2.2.3STMVP It is proposed to use three spatial merging candidates and one temporal merging candidate to derive an average candidate as the STMVP candidate. The STMVP is inserted before the upper left spatial merging candidate. The STMVP candidate is pruned along with all previous merge candidates in the merge list. For spatial candidates, the first three candidates in the current merged candidate list are used. For the temporal candidates, the same positions as the VTM / HEVC co-location positions are used. For spatial candidates, the first, second, and third candidates inserted before STMVP in the current merge candidate list are denoted as F, S, and T. The temporal candidate having the same position as the VTM / HEVC co-location position used in TMVP is denoted as Col. The motion vector of the STMVP candidate in prediction direction X (denoted as mvLX) is derived as follows: 1) If the reference indices of the four merge candidates are all valid and equal to zero in the prediction direction X (X=0 or 1), mvLX=(mvLX_F+mvLX_S+mvLX_T+mvLX_Col)>>2 2) If the reference indexes of three of the four merge candidates are valid and in the prediction direction X(X =0 or 1) is equal to zero, mvLX=(mvLX_F×3+mvLX_S×3+mvLX_Col×2)>>3 or mvLX=(mvLX_F×3+mvLX_T×3+mvLX_Col×2)>>3 or mvLX=(mvLX_S×3+mvLX_T×3+mvLX_Col×2)>>3. 3) If the reference indexes of two of the four merge candidates are valid and in the prediction direction X (X =0 or 1) is equal to zero, mvLX=(mvLX_F+mvLX_Col)>>1 or mvLX=(mvLX_S+mvLX_Col)>>1 or mvLX=(mvLX_T+mvLX_Col)>>1. NOTE: If time domain candidates are not available, STMVP mode is turned off. 2.2.4 Merge List Size If both non-adjacent merge candidates and STMVP merge candidates are considered, the size of the merge list is signaled in the sequence parameter set header and the maximum allowed size of the merge list is increased (eg, 8). 2.3. Sub-block based temporal motion vector prediction (SbTMVP) VVC supports the sub-block-based temporal motion vector prediction (SbTMVP) method. Similar to the temporal motion vector prediction (TMVP) in HEVC, SbTMVP uses the motion field in the co-located picture to improve the motion vector prediction and merge mode of the CU in the current picture. The same co-located picture used by TMVP is used for SbTMVP. SbTMVP differs from TMVP in the following two main aspects: –TMVP predicts motion at CU level, but SbTMVP predicts motion at sub-CU level; – While TMVP pre-fetches the temporal motion vector from the co-located block in the co-located picture (the co-located block is the bottom-right or center block relative to the current CU), SbTMVP applies motion shifting before pre-fetching the temporal motion information from the co-located picture, where the motion shifting is obtained from the motion vector of one of the spatial neighboring blocks from the current CU. SbTMVP process Fig.11A and Fig. 11B As shown in Figure 2, SbTMVP predicts the motion vector of the sub-CU within the current CU in two steps. Fig.11A shows the spatial neighboring blocks used by SbTMVP. In the first step, check Fig.11A The spatial neighbor A1 in . If A1 has a motion vector that uses the co-located picture as its reference picture, then that motion vector is selected as the motion offset to be applied. If no such motion is identified, then the motion shift is set to (0, 0). Fig. 11B A schematic diagram of driving the sub-CU motion field by applying motion shifts from spatial neighbors and scaling motion information from corresponding co-located sub-CUs is shown. In the second step, the motion shift identified in step 1 is applied (i.e., added to the coordinates of the current block) to Fig. 11B The co-located picture shown obtains motion information (motion vector and reference index) at the sub-CU level. Fig. 11B The example in assumes that the motion shift is set to the motion of block A1. Then, for each sub-CU, the motion information of its corresponding block in the co-located picture (the minimum motion grid covering the center sample) is used to derive the motion information for the sub-CU. After the motion information of the co-located sub-CU is identified, it is converted into a reference index and motion vector of the current sub-CU in a manner similar to the TMVP process of HEVC, where temporal motion scaling is applied to align the reference picture of the temporal motion vector with the reference picture of the current CU. In VVC, a combined sub-block based merge list containing both SbTMVP candidates and affine merge candidates is used for signaling of sub-block based merge mode. The SbTMVP mode is enabled / disabled by a sequence parameter set (SPS) flag. If the SbTMVP mode is enabled, the SbTMVP predictor is added as the first entry in the list of sub-block based merge candidates, followed by the affine merge candidates. The size of the sub-block based merge list is signaled in the SPS, and the maximum allowed size of the sub-block based merge list in VVC is 5. The sub-CU size used in SbTMVP is fixed to 8x8, and like the affine merge mode, the SbTMVP mode is only applicable to CUs whose width and height are both greater than or equal to 8. The encoding logic of the additional SbTMVP merge candidate is the same as that of other merge candidates, ie, for each CU in a P or B slice, an additional RD check is performed to decide whether to use the SbTMVP candidate. 2.4. Intra-block copy (IBC) Intra-block copying (IBC) is a tool adopted in the HEVC extension on SCC. As we all know, it significantly improves the coding efficiency of screen content materials. Since the IBC mode is implemented as a block-level coding mode, block matching (BM) is performed at the encoder to find the best block vector (or motion vector) for each CU. Here, the block vector is used to indicate the displacement from the current block to the reference block that has been reconstructed inside the current picture. The luminance block vector of the CU encoded and decoded by IBC is in integer precision. The chrominance block vector is also rounded to integer precision. When combined with AMVR, the IBC mode can switch between 1-pixel motion vector precision and 4-pixel motion vector precision. The CU encoded and decoded by IBC is regarded as a third prediction mode different from the intra or inter prediction mode. The IBC mode is applicable to CUs with width and height less than or equal to 64 luminance samples. On the encoder side, hash-based motion estimation is performed for IBC. The encoder performs RD check on blocks with width or height no greater than 16 luma samples. For non-merge mode, block vector search is first performed using hash-based search. If the hash search does not return a valid candidate, a local search based on block matching will be performed. In hash-based search, the hash key matching (32-bit CRC) between the current block and the reference blocks is extended to all allowed block sizes. The hash key calculation for each position in the current picture is based on 4x4 sub-blocks. For larger-sized current blocks, the hash key is determined to match the hash key of the reference block when all hash keys of all 4×4 sub-blocks match the hash keys in the corresponding reference positions. If the hash keys of multiple reference blocks are found to match the hash key of the current block, the block vector cost of each matching reference is calculated, and the one with the smallest cost is selected. In the block matching search, the search range is set to cover both the previous CTU and the current CTU. At CU level, the IBC mode is signaled using a flag, and it can be signaled in IBC AMVP mode or IBC Skip / Merge mode as follows: - IBC skip / merge mode: The merge candidate index is used to indicate which block vectors from a list of neighboring candidate IBC-coded blocks are used to predict the current block. The merge list consists of spatial, HMVP and pairwise candidates. – IBC AMVP mode: Block vector differences are encoded in the same way as motion vector differences. The block vector prediction method uses two candidates as predictors, one from the left neighbor and one from the top neighbor (if encoded by IBC). When either neighbor is not available, the default block vector will be used as the predictor. A flag is signaled to indicate the block vector predictor index. 2.4.1 Simplification of IBC vector prediction The BV predictors in IBC for merge mode and AMVP mode will share a common predictor list consisting of the following elements: ·2 adjacent airspace locations (in Figure 4 A0, B0 in 5 HMVP entries, The default is zero vector. For merge mode, up to the first 6 entries of the list will be used; for AMVP mode, the first 2 entries of the list will be used. And the list meets the shared merge list area requirements (sharing the same list within SMR). 2.4.2 IBC Reference Area To reduce memory consumption and decoder complexity, IBC in VVC only allows the reconstruction of a predefined area, which includes the area of ​​the current CTU and some areas of the left CTU. Figures 12A-12D Example diagrams 1210 , 1230 , 1250 , and 1270 are shown illustrating the current CTU processing order and the available samples in the current CTU and the left CTU. Figures 12A-12D The reference area of ​​the IBC mode is shown, where each block represents a 64x64 luma sample unit. Depending on the location of the current codec CU location within the current CTU, the following applies: – If the current block falls into the upper left 64x64 block of the current CTU, in addition to the samples already reconstructed in the current CTU, the CPR mode can also be used to refer to the reference samples in the lower right 64x64 block of the left CTU. The current block can also use the CPR mode to refer to the reference samples in the lower left 64x64 block of the left CTU and the reference samples in the upper right 64x64 block of the left CTU. – If the current block falls into the upper right 64x64 block of the current CTU, in addition to the samples already reconstructed in the current CTU, if the luma position (0,64) relative to the current CTU has not been reconstructed, the current block can also use the CPR mode to refer to the reference samples in the lower left 64x64 block and the lower right 64x64 block of the left CTU; otherwise, the current block can also refer to the reference samples in the lower right 64x64 block of the left CTU. – If the current block falls into the lower left 64x64 block of the current CTU, in addition to the samples that have been reconstructed in the current CTU, if the luma position (64,0) relative to the current CTU has not been reconstructed, the current block can also use the CPR mode to refer to the reference samples in the upper right 64x64 block and the lower right 64x64 block of the left CTU. Otherwise, the current block can also use the CPR mode to refer to the reference samples in the lower right 64x64 block of the left CTU. – If the current block falls into the lower right 64x64 block of the current CTU, it can only refer to the already reconstructed samples in the current CTU using the CPR mode. This restriction allows the IBC mode to be implemented using local on-chip memory for hardware implementation. 2.4.3 Interaction between IBC and other encoding and decoding tools The interaction between IBC mode and other inter-frame codec tools in VVC, such as paired merge candidates, history-based motion vector predictor (HMVP), combined intra / inter prediction mode (CIIP), merge mode with motion vector difference (MMVD), and geometric partitioning mode (GPM), is as follows: – IBC can be used with pairwise merge candidates and HMVP. A new pairwise IBC merge candidate can be generated by averaging two IBC merge candidates. For HMVP, IBC motion is inserted into the history buffer for future reference. – IBC cannot be used in conjunction with the following interframe tools: Affine Motion, CIIP, MMVD, and GPM. – When DUAL_TREE partitioning is used, IBC is not allowed for chroma codec blocks. Unlike the HEVC screen content codec extension, the current picture is no longer included as one of the reference pictures in reference picture list 0 for IBC prediction. The derivation process of motion vectors for IBC mode excludes all neighboring blocks in inter mode and vice versa. The following IBC design aspects are applied: – IBC shares the same process as in regular MV merging, including the use of pairwise merging candidates and history-based motion predictors, but TMVP and zero vectors are not allowed since they are invalid for IBC mode. – Separate HMVP buffers (5 candidates each) are used for regular MV and IBC. – Block vector constraints are implemented as codestream consistency constraints, the encoder needs to ensure that there are no invalid vectors in the codestream, and if the merge candidate is invalid (out of range or 0), the merge is not used. Such codestream consistency constraints are represented by virtual buffers as described below. – For deblocking, IBC is handled as inter mode. – If the current block is encoded using IBC prediction mode, AMVR does not use quarter-pels; instead, AMVR is signaled to only indicate whether the MV is inter-pel or 4 integer-pel (4 integer-pel). - The number of IBC merge candidates may be signaled in the slice header separately from the number of normal, sub-block and geometric merge candidates. The virtual buffer concept is used to describe the allowable reference area and valid block vectors for the IBC prediction mode. Denoting the CTU size as ctbSize, the virtual buffer ibcBuf has a width wIbcBuf=128x128 / ctbSize and a height hIbcBuf=ctbSize. For example, for a CTU size of 128x128, the size of ibcBuf is also 128x128; for a CTU size of 64x64, the size of ibcBuf is 256x64; and for a CTU size of 32x32, the size of ibcBuf is 512x32. The size of the VPDU is min(ctbSize, 64) in each dimension. v =min(ctbSize, 64). The virtual IBC buffer ibcBuf is maintained as follows. – At the beginning of decoding each CTU line, flush the entire ibcBuf with an invalid value of -1. – Start decoding the VPDU (xVPDU, yVPDU) relative to the upper left corner of the picture, and set ibcBuf[x][y] = -1, where x = xVPDU% wIbcBuf, ..., xVPDU% wIbcBuf+W v -1; y = yVPDU% ctbSize,…,yVPDU%ctbSize+W v -1. – After decoding the CU containing the top left corner (x, y) relative to the picture, set –ibcBuf[x%wIbcBuf][y%ctbSize]=recSample[x][y]. For a block covering coordinates (x, y), if the following is true for the block vector bv = (bv[0], bv[1]), then it is valid; otherwise, it is invalid: ibcBuf[(x+bv[0])%wIbcBuf][(y+bv[1])%ctbSize] should not be equal to -1. 2.4.4IBC Virtual Buffer Test The luma block vector bvL (a luma block vector in 1 / 16 fractional sample precision) shall obey the following constraints: –CtbSizeY is greater than or equal to ((yCb+(bvL[1]>>4))&(CtbSizeY-1))+cbHeight. – For x = xCb ... xCb + cbWidth - 1 and y = yCb ... yCb + cbHeight - 1, IbcVirBuf[0][(x + (bvL[0]>>4))&(IbcBufWidthY-1)][(y+(bvL[1]>>4))&(CtbSizeY-1)] Should not be equal to -1. Otherwise, bvL is considered to be an invalid bv. Samples are processed in CTB units. For each luma CTB, the array size is CtbSizeY in samples in both width and height. – (xCb, yCb) is the luminance position of the top left sample of the current luminance codec block relative to the top left luminance sample of the current picture, –cbWidth specifies the width of the current codec block in luma samples. –cbHeight specifies the height of the current codec block in luma samples. 2.5. Adaptive Merging Candidate Reordering Based on Template Matching In order to improve the encoding and decoding efficiency, after constructing the merge candidate list, the order of each merge candidate is adjusted according to the template matching cost. The merge candidates are arranged in the list according to the template matching cost in ascending order. It is operated in the form of subgroups. Fig.13 A schematic diagram 1300 showing neighboring samples used to calculate the SAD is shown. Fig.14 A schematic diagram 1400 showing neighboring samples used to calculate SAD for sub-CU level motion information is shown. The template matching cost is measured by the SAD (sum of absolute differences) between the neighboring samples of the current CU and its corresponding reference samples. If the merge candidate includes motion information for bidirectional prediction, then Fig.13 As shown, the corresponding reference sample is the average of the corresponding reference sample in reference list 0 and the corresponding reference sample in reference list 1. If the merge candidate contains motion information at the sub-CU level, then Fig.14 As shown, the corresponding reference sample is composed of neighboring samples of the corresponding reference sub-block. Fig.15 An example diagram 1500 illustrating the sorting process is shown. Fig.15As shown, the sorting process is operated in the form of subgroups. The first three merge candidates are sorted together. The next three merge candidates are sorted together. The template size (width of the left template or height of the upper template) is 1. The subgroup size is 3. 2.6. Adaptive Merge Candidate List Assume that the number of merging candidates is 8. The first 5 merging candidates are used as the first subgroup, and the following 3 merging candidates are used as the second subgroup (ie, the last subgroup). Fig.16 An example diagram 1600 illustrating the reordering process in an encoder is shown. For an encoder, such as Fig.16 As shown, after the merge candidate list is constructed at block 1602 , some merge candidates are adaptively reordered in ascending order of merge candidate cost. More specifically, at block 1604, the template matching costs of the merge candidates in all subgroups except the last subgroup are calculated; then at block 1606, the merge candidates in their own subgroups except the last subgroup are reordered; finally, at block 1608, the final merge candidate list will be obtained. Fig.17 An example diagram 1700 illustrating the reordering process in a decoder is shown. For the decoder, after the merge candidate list is constructed, as Fig.17 As shown, some / no merge candidates are adaptively reordered in ascending order of merge candidate cost. Fig.17 In the example, the subgroup in which the selected (signal) merging candidate is located is called the selected subgroup. More specifically, at block 1702, if the selected merge candidate is located in the last subgroup, then at block 1704, the merge candidate list construction process is terminated after the selected merge candidate is derived, and at block 1706, no reordering is performed and the merge candidate list is not changed; otherwise, the execution process is as follows: At block 1708, after all merge candidates in the selected subgroup are derived, the merge candidate list construction process is terminated; at block 1710, the template matching cost of the merge candidates in the selected subgroup is calculated; at block 1712, the merge candidates in the selected subgroup are reordered; finally, at block 1714, a new merge candidate list will be obtained. For both the encoder and the decoder, the template matching cost is derived as a function of T and RT, where T is the set of samples in the template and RT is the set of reference samples for the template. When deriving the reference samples of the template of the merge candidate, the motion vector of the merge candidate is rounded to integer pixel precision. It can also use an 8-tap or 12-tap luma interpolation filter. The reference samples of the template (RT) for bidirectional prediction are derived by weighted averaging the reference samples of the template in reference list 0 (RT0) and the reference samples of the template in reference list 1 (RT1) as follows. RT=((8-w)*RT0+w*RT1+4)>>3 The weight of the reference template in reference list 0 (8-w) and the weight of the reference template in reference list 1 (w) are determined by the BCW index of the merge candidate. BCW indexes equal to {0, 1, 2, 3, 4} correspond to w equal to {-2, 3, 4, 5, 10} respectively. If the local illumination compensation (LIC) flag of the merge candidate is true, the reference samples of the template are derived using the LIC method. The template matching cost is calculated based on the sum of absolute differences (SAD) between T and RT. The template size is 1. This means that the width of the left template and / or the height of the template above is 1. If the codec mode is MMVD, the merge candidates used to derive the base merge candidate are not reordered. If the coding mode is GPM, the merge candidates used to derive the unidirectional prediction candidate list are not reordered. 2.7. Template Matching (TM) Template Matching (TM) is a decoder-side MV derivation method used to refine the motion information of the current CU by finding the closest match between a template in the current picture (i.e., the top and / or left neighboring blocks of the current CU) and a block in a reference picture (i.e., the same size as the template). Fig.18 An example diagram 1800 illustrating the execution of template matching on a search area around an initial MV is shown. Fig.18 As shown, in the [-8, +8] pixel search range, a better MV is searched around the initial motion of the current CU. This paper adopts template matching and makes two modifications: the search step size is determined based on the AMVR mode, and the TM can be cascaded using the bilateral matching process in the merge mode. In AMVP mode, the MVP candidate is determined based on the template matching error by selecting the one that achieves the minimum difference between the current block template and the reference block template, and then TM performs MV refinement only on this specific MVP candidate. TM refines the MVP candidate by using an iterative diamond search, starting with full-pixel MVD accuracy (or 4 pixels for 4-pixel AMVR mode) within the [-8, +8] pixel search range. The AMVP candidate can be further refined by using a cross search with full-pixel MVD accuracy (or 4 pixels for 4-pixel AMVR mode), and then using half-pixels and quarter-pixels in turn according to the AMVR mode specified in Table 1. This search process ensures that the MVP candidate still maintains the same MV accuracy as indicated by the AMVR mode after the TM process. Table 1. Search patterns of AMVR and merging patterns using AMVR In merge mode, a similar search method is applied to the merge candidates indicated by the merge index. As shown in Table 1, TM can be performed all the way to 1 / 8 pixel MVD accuracy, or skip those accuracies exceeding half-pixel MVD accuracy, depending on whether an alternative interpolation filter is used according to the merged motion information (i.e., used when AMVR is half-pixel mode). In addition, when TM mode is enabled, template matching can work as an independent process between a block-based bilateral matching (BM) method and a sub-block-based bilateral matching method, or as an additional MV refinement process, depending on whether BM can be enabled according to its enabling condition check. On the encoder side, the TM merge mode will perform MV refinement for each merge candidate. 2.8. Intra-frame template matching Template Matching Prediction (TMP) is a special intra prediction mode that copies the best prediction block from the reconstructed part of the current frame, whose L-shaped template matches the current template. Fig.19 An example diagram 1900 showing template matching prediction is shown. For a predefined search range, the encoder searches for the template that is most similar to the current template in the reconstructed part of the current frame and uses the corresponding block as the prediction block. The encoder then signals the use of this mode and the inverse operation is performed on the decoder side. It is a codec tool mainly applicable for encoding and decoding of screen content. A prediction signal is generated at the decoder side by matching the L-shaped causal neighbors of the current block with another block in a predefined search area. Fig. 20 An example diagram 2000 showing the intra-frame template matching search area used is shown. Specifically, the search range is divided into three areas: R1: In the current CTU, R2: upper left corner outside the current CTU, R3: Above the current CTU, R4: On the left side of the current CTU. In each region, the decoder searches for the template with the minimum SAD relative to the current region and uses its corresponding block as the prediction block. The sizes of all regions (SearchRange_w, SearchRange_h) are set proportionally to the block size (BlkW, BlkH) so that there is a fixed number of SAD comparisons per pixel. That is: SearchRange_w = a*BlkW, SearchRange_h=a*BlkH. where "a" is a constant that controls the gain / complexity tradeoff. 2.9. Template-based Intra-mode export using MPM Fig.21 An example diagram 2100 showing templates used in TIMD and their reference samples is shown. TIMD mode is derived from MPM using neighboring templates. TIMD mode is used as an additional intra prediction method for a CU. Fig.21 As shown in , using the reference samples of the template for each candidate mode, the prediction samples of the template are generated. The cost is calculated as the sum of absolute transform differences (SATD) between the predicted and reconstructed samples of the template. The intra prediction mode with the minimum cost is selected as the TIMD mode and is used for intra prediction of the CU. 2.9.1 TIMD Mode Export For each intra prediction mode in the MPM, the SATD between the prediction and reconstructed samples of the template is calculated. The intra prediction mode with the smallest SATD is selected as the TIMD mode and is used for intra prediction of the current CU. Position-dependent intra prediction combination (PDPC) and gradient PDPC are supported in the derivation of TIMD mode. 2.9.2TIMD Signaling A flag is signaled in the sequence parameter set (SPS) to enable / disable TIMD. When the flag is true, a CU level flag is signaled to indicate whether TIMD is used for the CU. The TIMD flag is signaled after the MIP flag. If the TIMD flag is equal to true, the remaining syntax elements related to the luma intra prediction mode are skipped. 2.9.3 Interaction with new codecs in ECM-1.0 When the DIMD flag or the MIP flag is equal to true, then the TIMD flag is not signaled and is set equal to false. TIMD is allowed to be combined with ISP and MRL. When TIMD is combined with ISP or MRL and the TIMD flag is equal to true, the derived TIMD mode is used as the intra prediction mode of ISP or MRL. When the auxiliary MPM is enabled, both the primary MPM and the auxiliary MPM are used to derive the TIMD mode. The 6-tap interpolation filter is not used for the TIMD mode export. 2.9.4 Modification of MPM list construction in TIMD mode export During the construction of the MPM list, the intra prediction mode of the neighboring blocks is derived as a plane when they are inter-coded. In order to improve the accuracy of the MPM list, when the neighboring blocks are inter-coded, the propagated intra prediction mode is derived using the motion vector and the reference picture and is used in the construction of the MPM list. 2.10. Adaptive Merge Candidate List In the following, a template is a set of reconstructed samples adjacent to or non-adjacent to a current block. According to the same motion information of the current block, the reference samples of the template are derived. For example, the reference samples of the template are a mapping of the template depending on the motion information. In this case, the reference samples of the template are located by the motion vector of the motion information in the reference picture indicated by the reference index of the motion information. Fig. 22 An example diagram 2200 showing templates and reference samples of templates is shown. Fig. 22 An example is shown, where RT represents a reference sample of template T. When the merge candidate utilizes bidirectional prediction, the reference sample of the template of the merge candidate is represented by RT, and RT may be generated according to RT0 derived from the reference picture in reference picture list 0 and RT1 derived from the reference picture in reference picture list 1. In one example, RT0 includes a set of reference samples on the reference picture of the current block indicated by the reference index of the merge candidate that references the reference picture in reference list 0, wherein the MV of the merge candidate references reference list 0), and in one example, RT1 includes a set of reference samples on the reference picture of the current block indicated by the reference index of the merge candidate that references the reference picture in reference list 1, wherein the MV of the merge candidate references reference list 1). Fig.23 An example diagram 2300 showing templates in reference list 0 and reference list 1 and reference samples of the templates is shown. In one example, the reference samples (RT) of the templates used for bidirectional prediction are derived by taking an equally weighted average of the reference samples (RT0) of the templates in reference list 0 and the reference samples (RT1) of the templates in reference list 1. An example is as follows: RT=(RT0+RT1+1)>>1 In one example, the reference sample (RT) of the template used for bidirectional prediction bi-pred ) is derived by taking a weighted average of the reference samples of the template in reference list 0 (RT0) and the reference samples of the template in reference list 1 (RT1). An example is as follows: RT=((2 N -w)*RT0+w*RT1+2 N-1 )>>N, for example, N=3. In one example, the weight of the reference template in reference list 0 such as (8-w) and the weight of the reference template in reference list 1 such as (w) may be determined by the BCW index of the merge candidate. The merge candidates can be divided into several groups according to some criteria. Each group is called a subgroup. For example, adjacent spatial and temporal merge candidates can be used as the first subgroup, and the remaining merge candidates can be used as the second subgroup. In another example, the first N (N≥2) merge candidates can also be used as the first subgroup, the following M (M≥2) merge candidates can be used as the second subgroup, and the remaining merge candidates can be used as the third subgroup. Note that the proposed method can be applied to the merge candidate list construction process for inter-frame encoded blocks (e.g., translational motion), affine-encoded blocks; or other motion candidate list construction processes (e.g., AMVP list; IBC AMVP list; IBC merge list). W and H are the width and height of the current block (e.g., luminance block). The following description takes the merge candidate list construction process as an example: 1. The merging candidates can be adaptively rearranged in the final merging candidate list according to one or some criteria. a. In one example, firstly, part or all of the process of building the current merge candidate list is called, and then the candidates in the list are reordered. i. Alternatively, the candidates in the first subgroup may be reordered and they should be added before those in the second subgroup, where the first subgroup is added before the second subgroup. (i) In one example, multiple merge candidates of a first category may be first derived and then reordered within the first category; then merge candidates from the second category may be determined based on the reordered candidates in the first category (eg, how to apply pruning). ii. Alternatively, the first merge candidate in the first category may be compared with the second merge candidate in the second category to determine the order of the first merge candidate or the second merge candidate in the final merge candidate list. b. In one example, merge candidates may be adaptively rearranged before retrieval. i. In one example, the process of adaptively arranging merge candidates may be processed before obtaining merge candidates to be used in the motion compensation process. c. In one example, if the width of the current block is greater than the height of the current block, the above candidate is added before the left candidate. d. In one example, if the width of the current block is smaller than the height of the current block, the above candidate is added after the left candidate. e. Whether the merge candidates are adaptively rearranged may depend on the selected merge candidate or the selected merge candidate index. i. In one example, if the selected merge candidate is in the last sub-group, the merge candidates are not adaptively rearranged. f. In one example, costs are assigned to merge candidates, and the merge candidates are adaptively reordered in ascending order of their costs. i. In one example, the cost of merging candidates may be a template matching cost. ii. In one example, the template is a set of reconstructed samples that are adjacent or non-adjacent to the current block. iii. The template matching cost is derived as a function of T and RT, where T is the set of samples in the template and RT is the set of reference samples for the template. (i) How to obtain the reference sample of the template of the merge candidate may depend on the motion information of the merge candidate a) In one example, when deriving reference samples for a template, the motion vector of the merge candidate is rounded to integer pixel precision, where the integer motion vector may be its nearest integer motion vector. b) In one example, when deriving the reference samples of the template, an N-tap interpolation filter is used to obtain the reference samples of the template at the sub-pixel position. For example, N can be 2, 4, 6 or 8. c) In one example, when deriving reference samples for a template, the motion vectors of the merge candidates may be scaled to a given reference picture (eg, for each reference picture list, if available). d) For example, Fig. 22 As shown, a reference sample of the template of the merge candidate is obtained on the reference picture of the current block indicated by the reference index of the merge candidate, which has the MV of the merge candidate or the modified MV (eg, according to items a)-b)). e) For example, when the merge candidate utilizes bidirectional prediction, the reference sample of the template of the merge candidate is represented by RT, and RT can be generated according to RT0 derived from the reference picture in reference picture list 0 and RT1 derived from the reference picture in reference picture list 1. [1] In one example, RT0 includes a set of reference samples on a reference picture of the current block indicated by a reference index of a merge candidate that references a reference picture in reference list 0, where the MV of the merge candidate references reference list 0), [2] In one example, RT1 includes a set of reference samples on a reference picture of the current block indicated by a reference index of a merge candidate that references a reference picture in reference list 1, where the MV of the merge candidate references reference list 1). [3] For example Fig.23 shown. f) In one example, the reference samples (RT) of the templates used for bidirectional prediction are derived by taking an equally weighted average of the reference samples (RT0) of the templates in reference list 0 and the reference samples (RT1) of the templates in reference list 1. An example is as follows: RT=(RT0+RT1+1)>>1 g) In one example, the reference sample (RT) of the template used for bidirectional prediction bi-pred ) is derived by taking a weighted average of the reference samples of the template in reference list 0 (RT0) and the reference samples of the template in reference list 1 (RT1). An example is as follows: RT=((2 N -w)*RT0+w*RT1+2 N-1 )>>N, for example, N=3. h) In one example, the weight of the reference template in reference list 0 such as (8-w) and the weight of the reference template in reference list 1 such as (w) may be determined by the BCW index of the merge candidate. [1] In one example, the BCW index is equal to 0 and w is set equal to -2. [2] In one example, the BCW index is equal to 1 and w is set equal to 3. [3] In one example, the BCW index is equal to 2 and w is set equal to 4. [4] In one example, the BCW index is equal to 3 and w is set equal to 5. [5] In one example, the BCW index is equal to 4 and w is set equal to 10. i) In one example, if the local illumination compensation (LIC) flag of the merge candidate is true, the reference sample of the template is derived using the LIC method. (ii) The cost can be calculated based on the sum of absolute differences (SAD) between T and RT. a) Alternatively, the cost can be calculated based on the sum of absolute transformed differences (SATD) of T and RT. b) Alternatively, the cost can be calculated based on the sum of squared differences (SSD) of T and RT. c) Alternatively, the cost can be calculated based on weighted SAD / weighted SATD / weighted SSD. (iii) In addition to the SAD calculated in (ii), the cost can also consider the continuity between RT and the reconstructed samples adjacent or non-adjacent to T (Boundary_SAD). Reconstructed samples of adjacent or non-adjacent neighboring T and / or above are considered. a) In one example, the cost may be calculated based on SAD and Boundary_SAD. [1] In one example, the cost can be calculated as (SAD+w*Boundary_SAD). w may be predefined, signaled, or derived from decoded information. 2. Whether and / or how to reorder merge candidates may depend on the category of the merge candidates. a. In one example, only adjacent spatial and temporal merging candidates may be reordered. b. In one example, only adjacent spatial merging candidates, STMVP merging candidates and temporal merging candidates may be reordered. c. In one example, only adjacent spatial merging candidates, STMVP merging candidates, temporal merging candidates and non-adjacent spatial merging candidates may be reordered. d. In one example, only adjacent spatial merging candidates, STMVP merging candidates, temporal merging candidates, non-adjacent spatial merging candidates and HMVP merging candidates may be reordered. e. In one example, only adjacent spatial merging candidates, STMVP merging candidates, temporal merging candidates, non-adjacent spatial merging candidates, HMVP merging candidates and pairwise average merging candidates may be reordered. f. In one example, only adjacent spatial merging candidates, temporal merging candidates, HMVP merging candidates and pairwise average merging candidates may be reordered. g. In one example, only adjacent spatial merging candidates, temporal merging candidates and HMVP merging candidates may be reordered. h. In one example, only adjacent spatial merging candidates may be reordered. i. In one example, only the first subgroup can be reordered. j. In one example, the last subgroup cannot be reordered. k. In one example, only the top N merge candidates may be reordered. i. In one example, N is set equal to 5. 1. In one example, for candidates that are not reordered, they will be arranged in the merge candidate list according to the original order. m. In one example, candidates that are not to be reordered may be placed after candidates to be reordered. n. In one example, candidates that are not to be reordered may be placed before candidates to be reordered. o. In one example, some combinations of the above items (a-k) can be reordered. p. Different subgroups can be reordered individually. q. Two candidates in different subgroups cannot be compared and / or reordered. r. If the first subgroup is before the second subgroup, then after reordering, the first candidate in the first subgroup needs to be placed before the second candidate in the second subgroup. 3. Whether and / or how to reorder merge candidates may depend on the codec mode. a. In one example, if the coding mode is the normal merge mode, the merge candidates may be reordered. b. In one example, if the codec mode is MMVD, the merge candidates from which the base merge candidate is derived are not reordered. i. Alternatively, the reordering method may be different for MMVD mode and other merging modes. c. In one example, if the coding mode is CIIP, the merge candidate for combining with intra prediction is based on the reordered merge candidate. i. Alternatively, the reordering method may be different for CIIP mode and other merge modes. d. In one example, if the coding mode is GPM, the merge candidates used to derive the unidirectional prediction candidate list are not reordered. i. Alternatively, the reordering method may be different for GPM mode and other merge modes. e. In one example, if the coding mode is the triangulation mode, the merge candidates used to derive the unidirectional prediction candidate list are not reordered. i. Alternatively, the reordering method may be different for triangle mode and other merging modes. f. In one example, if the coding mode is a sub-block based merge mode, merge candidates based on part or all of the sub-blocks are reordered. i. Alternatively, the reordering method may be different for the sub-block based merge mode and other merge modes. ii. In one example, merging candidates based on unidirectional predicted sub-blocks are not reordered. iii. In one example, SbTMVP candidates are not reordered. iv. In one example, the constructed affine candidates are not reordered. v. In one example, zero-filled affine candidates are not reordered. 4. Whether and / or how to reorder merging candidates may depend on the available number of adjacent spatial and / or STMVP and / or temporal merging candidates. 5. Whether merge candidates need to be reordered may depend on decoded information (eg, the width and / or height of the CU). a. In one example, if the height is greater than or equal to M, the width is greater than or equal to N, and the width*height is greater than or equal to R, then the merge candidates may be reordered. i. In one example, M, N, and R are set equal to 8, 8, and 128. ii. In one example, M, N, and R are set equal to 16, 16, and 512. b. In one example, if the height is greater than or equal to M and the width is greater than or equal to N, the merge candidates may be reordered. i. In one example, M and N are set equal to 8 and 8. ii. In one example, M and N are set equal to 16 and 16. 6. Subgroup size can be adaptive. a. In one example, the subgroup size is determined based on the available number of adjacent spatial and / or STMVP and / or temporal merging candidates denoted as N. i. In one example, if N is less than M and greater than Q, the subgroup size is set to N; ii. In one example, if N is less than or equal to Q, no reordering is performed; iii. In one example, if N is greater than or equal to M, the subgroup size is set to M. iv. In one example, M and Q are set equal to 5 and 1 respectively. (i) Alternatively, M and / or Q may be predefined, signaled, or derived from decoded information. b. In one example, the subgroup size is decided according to the available number of adjacent spatial and temporal merging candidates denoted as N. i. In one example, if N is less than M and greater than Q, the subgroup size is set to N; ii. In one example, if N is less than or equal to Q, no reordering is performed; iii. In one example, if N is greater than or equal to M, the subgroup size is set to M. iv. In one example, M and Q are set equal to 5 and 1 respectively. 7. The template shape can be adaptive. a. In one example, the template may include only neighboring samples to the left of the current block. b. In one example, the template may only include neighboring samples above the current block. c. In one example, the template shape is selected according to the CU shape. d. In one example, the width of the left template is selected according to the CU height. i. For example, if H <= M, the left template size is w1xH; otherwise, the left template size is wxH e. In one example, M, w1, and w2 are set equal to 8, 1, and 2, respectively. f. In one example, the height of the upper template is selected according to the CU width. i. For example, if W<=N, the size of the upper template is Wxh1; otherwise, the size of the upper template is Wxh2. (i) In one example, N, h1, and h2 are set equal to 8, 1, and 2, respectively. g. In one example, the width of the left template is selected according to the CU width. i. For example, if W<=N, the size of the left template is w1xH; otherwise, the size of the left template is w2xH. (i) In one example, N, w1, and w2 are set equal to 8, 1, and 2, respectively. h. In one example, the height of the upper template is selected according to the CU height. i. For example, if H<=M, the size of the upper template is Wxh1; otherwise, the size of the upper template is Wxh2. (i) In one example, M, h1, and h2 are set equal to 8, 1, and 2, respectively. i. In one example, samples of the template and reference samples of the template samples may be downsampled or down-sampled before being used to calculate the cost. i. Whether and / or how to downsample may depend on the CU size. ii. In one example, no downsampling is performed on the short edge of the CU. 8. In the above examples, the merge candidate is one that is included in the final merge candidate list (eg, after pruning). a. Alternatively, the merge candidate is one derived from a given spatial or temporal block or HMVP list, or derived in other ways, even though it may not be included in the final merge candidate list. 9. Templates can include samples of specific color components. a. In one example, the template includes samples of the luma component only. 10. Whether to apply adaptive merge candidate list reordering can depend on the VPS / SPS / PPS / sequence header / picture header / A message transmitted by signal in the slice header / CTU / CU / TU / PU. It can also be region based on the signal. For example, the picture is divided into groups of CTUs / CUs evenly or unevenly, and a flag is encoded for each group to indicate whether the merge candidate list reordering is applied. 2.11. Adaptive Motion Candidate List 1. The motion candidates in the motion candidate list of the block may be adaptively rearranged according to one or some criteria to derive a reordered motion candidate list, and the block is encoded / decoded according to the reordered motion candidate list. a. The motion candidates in the motion candidate list of a block may be adaptively rearranged according to one or some criteria to derive a reordered motion candidate list, which is not a conventional merge candidate list. b. In one example, whether and / or how to reorder motion candidates may depend on the codec mode (e.g., affine merge, affine AMVP, regular merge, regular AMVP, GPM, TPM, MMVD, TM merge, CIIP, GMVD, affine MMVD). c. In one example, whether and / or how to reorder motion candidates may depend on the category of the motion candidates (eg, spatial, temporal, STMVP, HMVP, pairwise, SbTMVP, constructed affine, inherited affine). d. In one example, the motion candidate list may be an AMVP candidate list. e. In one example, the motion candidate list may be a merge candidate list. f. In one example, the motion candidate list may be an affine merge candidate list. g. In one example, the motion candidate list may be a sub-block based merge candidate list. h. In one example, the motion candidate list may be a GPM merge candidate list. i. In one example, the motion candidate list may be a TPM merge candidate list. j. In one example, the motion candidate list may be a TM merge candidate list. k. In one example, the motion candidate list may be a candidate list of an MMVD codec block. 1. In one example, the motion candidate list may be a candidate list of a DMVR codec block. 2. How to adaptively rearrange the motion candidates in the motion candidate list may depend on decoded information, for example, the category of the motion candidate, the category of the motion candidate list, and the encoding and decoding tool. a. In one example, for different motion candidate lists, different criteria may be used to re-rank the motion candidate lists. i. In one example, the criteria may include how to select a template. ii. In one example, the criteria may include how to calculate the template cost. iii. In one example, the criteria may include how many candidates in the candidate list and / or how many subgroups need to be reordered. b. In one example, motion candidates in a motion candidate list are first adaptively rearranged to construct a fully rearranged candidate list or a partially rearranged candidate list, and then at least one motion candidate indicated by at least one index is retrieved from the rearranged candidate list to derive final motion information to be used by the current block. c. In one example, refinements are first adaptively rearranged (e.g., using TM for TM codec blocks; The method further comprises: adding the motion candidates before the MVD for the MMVD coded block to construct a completely rearranged candidate list or a partially rearranged candidate list. Then, at least one motion candidate indicated by at least one index is retrieved from the rearranged candidate list and refined (e.g., using the TM for the TM coded block; The MMVD-encoded block is applied to the retrieved motion candidates to derive the final motion information of the current block. d. In one example, refinement is applied to at least one motion candidate in the motion candidate list (e.g., using TM for TM codec blocks; adding MVD for MMVD codec blocks), and then they are adaptively rearranged to construct a fully rearranged candidate list or a partially rearranged candidate list, and then at least one motion candidate indicated by at least one index is retrieved from the rearranged candidate list to derive final motion information without any further refinement for the current block. 3. In one example, new MERGE / AMVP motion candidates may be generated based on the candidate re-ranking. i. For example, the candidate L0 motion and L1 motion can be reordered separately. ii. For example, a new bi-prediction merging candidate may be constructed by combining one from the reordered L0 motion and another from the reordered L1 motion. iii. For example, new uni-directional prediction merging candidates can be generated by reordering L0 or L1 motion. 2.12. Adaptive Motion Candidate List For sub-block motion prediction, if the sub-block size is Wsub*Hsub, the height of the upper template is Ht, and the width of the left template is Wt, the upper template can be regarded as a composition of several sub-templates with a size of Wsub*Ht, and the left template can be regarded as a composition of several sub-templates with a size of Wt*Hsub. After deriving the reference sample of each sub-template in a similar manner as described above, the reference sample of the template is derived. Fig.24 An example diagram 2400 showing reference samples of templates and templates for blocks with sub-block motion using motion information of sub-blocks of a current block is shown. Fig.25 An example diagram 2500 showing reference samples of templates and templates for blocks with sub-block motion using motion information for each sub-template is shown. It should be noted that the terms mentioned below are not limited to specific terms defined in existing standards. Any changes in codecs also apply. For example, the term "GPM" is used to represent any codec that derives two sets of motion information and uses the derived information and the segmentation pattern to obtain the final prediction. For example, TPM is also considered a GPM. Note that the proposed method can be applied to the merge candidate list construction process of inter-coded blocks (e.g., translational motion), affine-coded blocks, or IBC-coded blocks; or other motion candidate list construction processes (e.g., ordinary AMVP list; affine AMVP list; IBC AMVP list). W and H are the width and height of the current block (eg, luma block). 1. In one example, if the coding mode is TM merging, some or all TM merging candidates may be reordered. a. In one example, if the coding mode is TM merging, part or all of the original TM merging candidates may be reordered before the TM refinement process. b. Alternatively, if the coding mode is TM merging, some or all of the refined TM merging candidates may be reordered after the TM refinement process. c. Alternatively, if the coding mode is TM merging, the TM merging candidates may not be reordered. d. Alternatively, the reordering method may be different for TM merge mode and other merge modes. 2. In one example, if the coding mode is a sub-block based merge mode, merge candidates based on part or all of the sub-blocks may be reordered. a. Alternatively, the reordering method may be different for the sub-block based merge mode and other merge modes. b. In one example, a template may be divided into sub-templates. Each sub-template may have individual motion information. i. In one example, a cost for reordering candidates may be derived based on the cost of each sub-template. For example, the cost for the reordering candidate may be calculated as the sum of the costs of all sub-templates.For example, the cost of a sub-template may be calculated as SAD, SATD, SSD, or any other distortion measure between a sub-template and its corresponding reference sub-template. c. In one example, to derive reference samples of the sub-template, the motion information of the sub-blocks in the first row and first column of the current block may be used. i. In one example, the motion information of a sub-template can be derived (eg, copied) from its neighboring sub-block in the current block. Fig.24 shown. d. In one example, in order to derive a reference sample of a sub-template, the motion information of the sub-template may be derived without referring to the motion information of the sub-block in the current block. Fig.25 shown. i. In one example, the motion information of each sub-template is calculated based on the affine model of the current block. (i) In one example, the motion vector of the center sample of each sub-block containing the sub-template calculated according to the affine model of the current block is regarded as the motion vector of the sub-template. (ii) In one example, the motion vector of the center sample of each sub-template calculated according to the affine model of the current block is regarded as the motion vector of the sub-template. (iii) For the 4-parameter affine motion model, the motion vector at the sample position (x, y) in the block is derived as: (iv) For the 6-parameter affine motion model, the motion vector at the sample position (x, y) in the block is derived as: (v) For (iii) and (iv), the coordinates of the upper left corner, upper right corner, and lower left corner of the current block are (0, 0), (W, 0), and (0, H), and the motion vectors of the upper left corner, upper right corner, and lower left corner of the current block are (mv 0x , mv 0y )、(mv 1x , mv 1y ) and (mv 2x , mv 2y ). (vi) In one example, the coordinates (x, y) in the above equation can be set equal to the position in the template or the position of the sub-template. For example, the coordinates (x, y) can be set equal to the center position of the sub-template. e. In one example, this scheme can be applied to affine merge candidates. f. In one example, this scheme can be applied to affine AMVP candidates. g. In one example, this scheme can be applied to SbTMVP merge candidates. h. In one example, this scheme can be applied to GPM merge candidates. i. In one example, the scheme can be applied to TPM merge candidates. j. In one example, this solution can be applied to TM to refine merge candidates. k. In one example, this solution can be applied to DMVR to refine merge candidates. 1. In one example, this solution can be applied to MULTI_PASS_DMVR to refine the merge candidates. 3. In one example, if the codec mode is MMVD, the merge candidates from which the base merge candidate is derived may be reordered. a. In one example, a reordering process may be applied to merge candidates before they are refined by a signaled or derived MVD. b. For example, the reordering method may be different for MMVD mode and other merge modes. 4. In one example, if the coding mode is MMVD, the merge candidates after MMVD refinement may be reordered. a. In one example, the reordering process can be applied to the merge candidates after they are refined by a signaled or derived MVD. b. For example, the reordering method may be different for MMVD mode and other merge modes. 5. In one example, if the coding mode is affine MMVD, the merge candidates from which the base merge candidate is derived may be reordered. a. In one example, a reordering process may be applied to affine merge candidates before they are refined by a signaled or derived MVD. b. For example, the reordering method can be different for the affine MMVD mode and other merge modes. 6. In one example, if the coding mode is affine MMVD, the merge candidates after affine MMVD refinement may be reordered. a. In one example, the reordering process can be applied to affine merge candidates after they are refined by a signaled or derived MVD. b. For example, the reordering method can be different for the affine MMVD mode and other merge modes. 7. In one example, if the codec mode is GMVD, the merge candidates from which the base merge candidate is derived may be reordered. a. In one example, a reordering process may be applied to merge candidates before they are refined by a signaled or derived MVD. b. For example, the reordering method may be different for GMVD mode and other merge modes. 8. In one example, if the codec mode is GMVD, the merge candidates after GMVD refinement may be reordered. a. In one example, the reordering process can be applied to the merge candidates after they are refined by a signaled or derived MVD. b. For example, the reordering method may be different for GMVD mode and other merge modes. 9. In one example, if the codec mode is GPM, the merge candidates may be reordered. a. In one example, a reordering process may be applied to the original merge candidates before they are used to derive the per-partition GPM candidate list (also referred to as the uni-directional prediction candidate list of the GPM). b. In one example, if the coding mode is GPM, the merge candidates in the unidirectional prediction candidate list may be reordered. c. In one example, the GPM unidirectional prediction candidate list can be constructed based on the reordering. i. In one example, a candidate with bi-directional prediction (also referred to as a bi-directional prediction candidate) may be separated into two uni-directional prediction candidates. (i) If the number of original merging candidates is M, at most 2M unidirectional prediction candidates can be separated from them. ii. In one example, unidirectional prediction candidates separated from bidirectional prediction candidates may be placed into an initial unidirectional prediction candidate list. iii. In one example, candidates in the initial unidirectional prediction candidate list may be reordered using template matching cost. iv. In one example, the first N unidirectional prediction candidates with smaller template matching costs can be used as the final GPM unidirectional prediction candidates. For example, N is equal to M. d. In one example, after deriving the GPM unidirectional prediction candidate list, a bidirectional prediction list for the combination of partition 0 and partition 1 is constructed, and then the bidirectional prediction list is reordered. i. In one example, if the number of GPM unidirectional prediction candidates is M, the number of combined bidirectional prediction candidates is M*(M-1). e. Alternatively, the reordering method may be different for GPM mode and other merge modes. 2.13. Adaptive Motion Candidate List It should be noted that the terms mentioned below are not limited to specific terms defined in existing standards. Any changes in codecs also apply. For example, the term "GPM" is used to represent any codec that derives two sets of motion information and uses the derived information and the partitioning pattern to obtain the final prediction. For example, TPM is also considered a GPM. Note that the proposed method can be applied to the merge candidate list construction process of inter-coded blocks (e.g., translational motion), affine-coded blocks, or IBC-coded blocks; or other motion candidate list construction processes (e.g., ordinary AMVP list; affine AMVP list; IBC AMVP list). W and H are the width and height of the current block (eg, luma block). 1. The reference sample (RT) of the template or sub-template used for bidirectional prediction is derived by taking an equally weighted average of the reference sample (RT0) of the template or sub-template in reference list 0 and the reference sample (RT1) of the template or sub-template in reference list 1. An example is as follows: RT(x,y)=(RT0(x,y)+RT1(x,y)+1)>>1 2. The reference sample (RT) of the template or sub-template used for bidirectional prediction is derived by weighted averaging the reference sample (RT0) of the template or sub-template in reference list 0 and the reference sample (RT1) of the template or sub-template in reference list 1. a. An example is as follows: RT(x,y)=((2 N -w)*RT0(x,y)+w*RT1(x,y)+2 N-1 )>>N, for example, N =3. b. The weights may be determined by the BCW index, or may be derived on the fly or predefined or determined by the weights used in weighted prediction. c. In one example, the weight of the reference template in reference list 0 such as (8-w) and the weight of the reference template in reference list 1 such as (w) may be determined by the BCW index of the merge candidate. i. In one example, the BCW index is equal to 0 and w is set to -2. ii. In one example, the BCW index is equal to 1 and w is set to 3. iii. In one example, the BCW index is equal to 2 and w is set to 4. iv. In one example, BCW index equals 3 and w is set to 5. v. In one example, the BCW index is equal to 4 and w is set to -10. 3. Reference samples of the proposed template can be derived using the LIC method. a. In one example, the LIC parameters of both the left template and the upper template are the same as the LIC parameters of the current block. b. In one example, the LIC parameters of the left template are derived as the LIC parameters of the current block, and the current block uses its original motion vector plus the motion vector offset (-Wt, 0) as the motion vector of the current block. c. In one example, the LIC parameters of the upper template are derived as the LIC parameters of the current block, and the current block uses its original motion vector plus the motion vector offset (0, -Ht) as the motion vector of the current block. d. Alternatively or additionally, if the local illumination compensation (LIC) flag of the merge candidate is true, the above items may be applied. 4. It is proposed that the reference samples of the template or sub-template can be derived using the OBMC method. In the following discussion, "Template" can refer to a template or a sub-template. Fig.26 An example diagram 2600 showing templates and reference samples of the templates for a block with OBMC is shown. a. In one example, in order to derive the reference samples of the upper template, the motion information of the sub-block in the first row of the current block and its adjacent neighboring sub-blocks above it is used. And the reference samples of all sub-templates constitute A sample of the template above. Fig.26 shown. b. In one example, to derive the reference sample of the left template, use the sub- The motion information of the block and its adjacent sub-blocks on the left. And the reference samples of all sub-templates constitute Reference sample of the template on the left. An example is Fig.26 shown. c. In one example, the sub-block size is 4x4. d. The reference sample of the sub-template based on the motion vector of the neighboring sub-block is denoted as P N , where N represents the index of the adjacent upper and left sub-blocks, and the sub-template based on the motion vector of the sub-block of the current block The reference sample is denoted as P C For P generated based on the motion vectors of vertically (horizontally) adjacent sub-blocks N , P NThe samples in the same row (column) of P are added to P with the same weighting factor. C . i. The reference sample (P) of the sub-template can be derived as P = W N *P N +W C *P C . ii. In one example, if the height of the upper template or the width of the left template is greater than or equal to 4, then the weight factors {1 / 4, 1 / 8, 1 / 16, 1 / 32} are used for P N {first, second, third, fourth} row (column), and weight factors {3 / 4, 7 / 8, 15 / 16, 31 / 32} are used for P C of {first, second, third, fourth} row(column). iii. In one example, if the height of the template above or the width of the template to the left is greater than or equal to 2, then the weighting factors {1 / 4,1 / 8} are used for P N The {first, second} rows (columns) of P, and the weighting factors {3 / 4, 7 / 8} are used for P C The {first, second} row (column) of . iv. In one example, if the height of the template above or the width of the template to the left is greater than or equal to 1, then the weighting factor {1 / 4} is used for P N The first row (column) of , and the weighting factor {3 / 4} Used for P C The first row (column) of . e. If merge candidates are allocated with OBMC enabled, the above items may be applied. 5. In one example, if the merged candidate uses multi-hypothesis prediction, the reference sample of the template can be derived using the multi-hypothesis prediction method. 6. A template may include samples of (multiple) specific color components. a. In one example, the template includes samples of the luma component only. b. Alternatively, the template only includes samples of any component, such as Cb / Cr / R / G / B. 7. Whether and / or how to reorder motion candidates may depend on the category of the motion candidates. a. In one example, only adjacent spatial and temporal motion candidates can be reordered. b. In one example, only adjacent spatial, STMVP and temporal motion candidates may be reordered. c. In one example, only adjacent spatial, STMVP, temporal motion candidates and non-adjacent spatial motion candidates may be reordered. d. In one example, only adjacent spatial domains, STMVP, temporal motion candidates, non-adjacent spatial domains, and HMVP motion candidates may be re-ordered. e. In one example, only adjacent spatial, STMVP, temporal motion candidates, non-adjacent spatial, HMVP, and pairwise average motion candidates may be reordered. f. In one example, only neighboring spatial, temporal, HMVP and pairwise average motion candidates may be reordered. g. In one example, only adjacent spatial, temporal and HMVP motion candidates may be reordered. h. In one example, only adjacent spatial motion candidates may be reordered. i. In one example, motion candidates based on unidirectional prediction sub-blocks are not reordered. j. In one example, SbTMVP candidates are not reordered. k. In one example, inherited affine motion candidates are not reordered. 1. In one example, the reconstructed affine motion candidates are not reordered. m. In one example, zero-filled affine motion candidates are not reordered. n. In one example, only the top N motion candidates may be reordered. i. In one example, N is set equal to 5. 8. In one example, the motion candidates may be divided into multiple subgroups. Whether and / or how to reorder the motion candidates may depend on the subgroups of the motion candidates. a. In one example, only the first subgroup can be reordered. b. In one example, the last subgroup may not be reordered. c. In one example, the last subgroup may not be reordered. However, if the last subgroup is also the first subgroup, it may be reordered. d. Different subgroups can be reordered individually. e. Two candidates in different subgroups cannot be compared and / or re-ranked. f. If the first subgroup precedes the second subgroup, then after reordering the first candidate in the first subgroup must be placed before the second candidate in the second subgroup. 9. In one example, motion candidates not included in the reordering process may be processed in a specified manner. a. In one example, for candidates that are not reordered, they will be arranged in the merge candidate list according to the original order. b. In one example, candidates that are not to be reordered may be placed after candidates to be reordered. c. In one example, candidates that are not to be reordered may be placed before candidates to be reordered. 10. Whether to apply adaptive merge candidate list reordering may depend on a message transmitted by signal in VPS / SPS / PPS / sequence header / picture header / slice header / CTU / CU / TU / PU. It may also be based on the region of the signal. For example, the picture is evenly or unevenly divided into groups of CTU / CU, and a flag is encoded for each group to indicate whether the merge candidate list reordering is applied. 2.14. Cost Functions Utilized in Codec Data Refinement in Image / Video Codecs The term "block" may refer to a codec tree block (CTB), a codec tree unit (CTU), a codec block (CB), a CU, a PU, a TU, a PB, a TB, or a video processing unit including a plurality of samples / pixels. A block may be rectangular or non-rectangular. In the present disclosure, the phrase "motion candidate" may refer to a merged motion candidate in a regular / extended merge list indicated by a merge candidate index, or an AMVP motion candidate in a regular / extended AMVP list indicated by an AMVP candidate index, or an AMVP motion candidate, or a merged motion candidate. In this disclosure, a motion candidate is referred to as "refined" if its motion information is modified based on information transmitted by signal from the encoder or derived at the decoder. For example, the motion vector can be refined by DMVR, FRUC, TM Merge, TM AMVP, TM GPM, TM CIIP, TM Affine, MMVD, GMVD, Affine MMVD, BDOF, etc. In the present disclosure, the phrase "codec data refinement" may refer to a refinement process to derive or refine a signaled / decoded / derived prediction mode, prediction direction, or signaled / decoded / derived motion information, prediction, and / or reconstructed samples for a block. In one example, the refinement process may include motion candidate reordering. In the following discussion, a "template-based" block may refer to a block that uses a template matching-based method to derive or refine the coded information during the encoding / decoding process, such as template matching-based motion derivation, template matching-based motion list reconstruction, LIC, symbol prediction, template matching-based block vector (e.g., used in IBC mode) derivation, DIMD, template matching-based non-inter (e.g., intra) prediction, etc. The template-based coding method may be combined with any other coding tools, such as MMVD, CIIP, GPM, FRUC, Affine, BDOF, DMVR, OBMC, etc. In another example, a "template-based" block may also refer to a block that uses neighboring reconstructed samples (adjacent or non-adjacent) to derive or refine its decoded information based on specific rules, such as the DIMD method in Section 2.27 and the TIMD method in Section 2.29). In the following discussion, a "bilateral codec-based" block may refer to a block that uses a bilateral matching-based method to derive or refine the coded information during the encoding / decoding process, such as bilateral matching-based motion derivation, bilateral matching-based motion list reconstruction, etc. The bilateral codec-based method can be combined with any other codec tools, such as MMVD, CIIP, GPM, FRUC, Affine, DMVR, etc. W and H are the width and height of the current block (eg, luma block). W*H is the size of the current block (eg, luma block). In the following discussion, Shift(x,s) is defined as Shift(x,s)=(x+offset)>>s, where offset is an integer, such as offset=0 or offset=1<<(s-1) or offset=(1<<(s-1))-1. In another example, offset depends on x. For example, offset = (x<0?(1<<(s-1)):((1<<(s-1)-1). 1. In addition to error measurement, it is proposed to add conventional items to the cost calculation process. a) In one example, the cost is defined as: E+W*RI, where E represents the output of the error function and W is the weight applied to the regular term represented by RI. i. In one example, to process a template-based codec block / a bilateral-based codec block, the cost function is set to: E+W*RI, where E can be SAD / MRSAD / SATD Or otherwise, RI is the estimated bit for the motion vector / motion vector difference, and W is the weight, which may depend on QP / temporal layer, etc., for example. ii. Alternatively, the cost can be defined as: w0*E + W1*RI, where E represents the output of the error function, W1 is the weight applied to the regular term represented by RI, and w0 is the weight applied to the output of the error function. (i) Alternatively or additionally, W1 can be set to 0. b) In one example, the regular term can be multiplied by a weight rate. i. In one example, the weight is derived on the fly. ii. In one example, the weight is set to lambda used in the full RDO process. iii. In one example, the weight is set to the square root of lambda used in the full RDO process. c) In one example, the cost is calculated as E + Shift(W*RI, s), where s and W are integers. i. Alternatively, the cost is calculated as Shift((E << s) + W*RI, s), where s and W are integers. 2. An error function different from SAD / MR-SAD (Sum of Absolute Differences with Mean Removed) is proposed to process blocks based on template coding / decoding and blocks based on bilateral coding / decoding. a) In one example, the error function can be: i. SATD, ii. MR-SATD, iii. Gradient information, iv. SSE / SSD, v. MR-SSE / MR-SSD, vi. Weighted SAD / Weighted MR-SAD, vii. Weighted SATD / Weighted MR-SATD, viii. Weighted SSD / Weighted MR-SSD, ix. Weighted SSE / Weighted MR-SSE. b) Alternatively or additionally, it is proposed to adaptively select the error function among different cost functions such as the above error functions and SAD / MR-SAD. i. The selection can be determined on the fly. 3. When using an error function based on MR-X (e.g., X is SATD, SAD, SSE) (such as MR-SAD / MR-SATD, etc.), the following can be further applied: a) In one example, the mean can be calculated taking into account all samples in the blocks to be compared. b) In one example, the mean may be calculated using a portion of the samples in the block being considered for comparison. c) In one example, the mean and the X-function may depend on the same samples in a block. i. In one example, the mean and X functions can be calculated using all samples in the block. ii. In one example, the mean and the X function can be calculated using a portion of the samples in the block. d) In one example, the mean and the X-function may depend on at least one different sample in the block. i. In one example, the mean may be calculated using all samples, while the X function may depend on a portion of the samples in the block. ii. In one example, the mean may be calculated using a portion of the samples, while the X function may depend on all samples in the block. 4. The template / bilateral matching cost can be calculated by applying the cost factor to the error cost function. a) In one example, it is proposed to bias motion candidates towards the front during the template / bilateral matching based re-ranking process. i. In one example, the motion candidate in the i-th position is assigned a smaller cost factor than the cost factor of the motion candidate in the (i+1)-th position. ii. In one example, motion candidates in the i-th group (eg, involving M motion candidates) are assigned a smaller cost factor than a cost factor of motion candidates in the (i+1)-th group (eg, involving N motion candidates). (i) In one example, M may be equal to N. For example, M=N=2. (ii) In one example, M may not be equal to N. For example, M=2, N=3. b) In one example, it is proposed that the refinement process based on template / bilateral matching tends to be closer to the original Search MV for MV. i. In one example, each search region is assigned a cost factor, which may be determined by the distance (eg, integer pixel precision delta MV) between each search MV in the search region and the start MV. ii. In one example, each search region is assigned a cost factor, which may be determined by the distance between a center search MV and a start MV in the search region (eg, a delta MV in integer pixel precision). iii. In one example, each search MV is assigned a cost factor, which may be determined by the distance between each search MV and the start MV (eg, a delta MV in integer pixel precision). 5. The above method can be applied to any codec data refinement process, for example, for blocks based on template codecs, for blocks based on bilateral codecs (eg, DMVR in VVC). 6. For different template matching refinement methods, the template matching cost measure can be different. a. In one example, the template matching refinement method may be a motion candidate reordering based on template matching. b. In one example, the template matching refinement method may be a motion derivation based on template matching. i. In one example, the refinement method may be TM AMVP, TM Merge, and / or FRUC. c. In one example, the template matching refinement method may be template matching based motion refinement. ii. In one example, the refinement method may be TM GPM, TM CIIP and / or TM Affine. d. In one example, the template matching refinement method may be a block vector derivation based on template matching. e. In one example, the template matching refinement method may be intra-mode derivation based on template matching. iii. In one example, the thinning method may be DIMD and / or TIMD. f. In one example, the template matching cost measure may be calculated based on the sum of absolute differences (SAD) between the current template and the reference template. g. In one example, the template matching cost measure may be calculated based on the mean-subtracted SAD between the current template and the reference template. h. In one example, SAD and mean-removed SAD (MR-SAD) can be selectively utilized according to the size of the current block. i. In one example, mean removed SAD is used for blocks of size greater than M, and SAD is used for blocks of size less than or equal to M. i. (i) In one example, M is 64. i. In one example, SAD and mean-removed SAD (MR-SAD) can be selectively utilized according to the LIC flag of the current block. i. In one example, if the LIC flag of the current block is false, the template matching cost measure may be SAD. ii. In one example, if the LIC flag of the current block is true, the template matching cost measure may be MR-SAD. j. In one example, the template matching cost measure may be calculated based on the sum of absolute transformed differences (SATD) between the current template and the reference template. k. In one example, the template matching cost measure may be calculated based on the mean-removed SATD between the current template and the reference template. 1. In one example, SATD and mean-removed SATD (MR-SATD) can be selectively utilized according to the size of the current block. i. In one example, SATD with mean removal is used for blocks of size greater than M, and SATD is used for blocks of size less than or equal to M. (i) In one example, M is 64. m. In one example, SATD and mean-removed SATD (MR-SATD) can be selectively utilized according to the LIC flag of the current block. i. In one example, if the LIC flag of the current block is false, the template matching cost measure can be SATD. ii. In one example, if the LIC flag of the current block is true, the template matching cost measure may be MR-SATD. n. In one example, the template matching cost measure may be calculated based on the sum of squared differences (SSD) between the current template and the reference template. o. In one example, the template matching cost measure may be calculated based on a mean-subtracted SSD between the current template and a reference template. p. In one example, SSD and mean removed SSD (MR-SSD) can be selectively utilized according to the size of the current block. i. In one example, mean removed SSD is used for blocks of size greater than M, and SSD is used for blocks of size less than or equal to M. (i) In one example, M is 64. q. In one example, the template matching cost measure can be a weighted sum between the current template and the reference template SAD / weighted MR-SAD / selective weighted MR-SAD and SAD / weighted SATD / weighted MR- SATD / selectively weighted MR-SATD and SATD / weighted SSD / weighted MR-SSD / selectively weighted MR-SSD and SSD. i. In one example, weighting means that when calculating the distortion between the current template and the reference template, Apply different weights to each sample based on its row and column index in the template block. ii. In one example, weighting means that when calculating the distortion between the current template and the reference template, Different weights are applied to each sample based on its position in the template patch. iii. In one example, weighting means calculating the distortion between the current template and the reference template When , different weights are applied to each sample based on the distance from each sample to the current block. r. In one example, the template matching cost can be calculated as follows: tplCost = w1*mvDistanceCost+w2*distortionCost. i. In one example, distortionCost can be the weighted sum between the current template and the reference template. SAD / weighted MR-SAD / weighted SATD / weighted MR-SATD / weighted SSD / weighted MR- SSD / SAD / MR-SAD / SATD / MR-SATD / SSD / MR-SSD. ii. In one example, mvDistanceCost may be the sum of the absolute mv differences between the search point and the start point in the horizontal and vertical directions. iii. In one example, w1 and w2 may be predetermined, or signaled, or derived from decoded information. (i) In one example, w1 is a weighting factor set to 4 and w2 is a weighting factor set to 1 s. In addition to the SAD calculated in (f), the cost may also consider the continuity between the reference template and the reconstructed samples adjacent or non-adjacent to the current template (Boundary_SAD). For example, the reconstructed samples adjacent or non-adjacent to the current sample on the left and / or above are considered. i. In one example, the cost can be calculated based on SAD and Boundary_SAD (i) In one example, the cost may be calculated as (SAD+w*Boundary_SAD). w may be predetermined, signaled, or derived from decoded information. 7. The bilateral matching cost measure may be different for different bilateral matching refinement methods. a) In one example, the bilateral matching refinement method may be a motion candidate re-ranking based on bilateral matching. b) In one example, the bilateral matching refinement method may be motion derivation based on bilateral matching. i. In one example, the refinement method may be BM merging and / or FRUC. c) In one example, the bilateral matching refinement method may be motion refinement based on bilateral matching. i. In one example, the refinement method may be BM GPM, BM CIIP and / or BM Affine. d) In one example, the bilateral matching refinement method may be a block vector derivation based on bilateral matching. e) In one example, the bilateral matching refinement method may be an intra-mode derivation based on bilateral matching. f) In one example, the bilateral matching cost measure may be calculated based on the sum of absolute differences (SAD) between two reference blocks / sub-blocks. g) In one example, the bilateral matching cost measure may be calculated based on the mean-removed SAD between two reference blocks / sub-blocks. h) In one example, SAD and mean-removed SAD (MR-SAD) can be selectively utilized according to the size of the current block / sub-block. i. In one example, mean removed SAD is used for blocks / sub-blocks of size greater than M, and SAD is used for blocks / sub-blocks of size less than or equal to M. i. (i) In one example, M is 64. i) In one example, SAD and mean-removed SAD (MR-SAD) can be selectively utilized according to the LIC flag of the current block. i. In one example, if the LIC flag of the current block is false, the bilateral matching cost measure can be SAD. ii. In one example, if the LIC flag of the current block is true, the bilateral matching cost measure may be MR-SAD. j) In one example, the bilateral matching cost measure may be calculated based on the sum of absolute transformed differences (SATD) between two reference blocks / sub-blocks. k) In one example, the bilateral matching cost measure may be calculated based on the mean-removed SATD between two reference blocks / sub-blocks. l) In one example, SATD and mean-removed SATD (MR-SATD) can be based on the current block / The size of the sub-blocks is selectively utilized. i. In one example, SATD with mean removal is used for blocks / sub-blocks with size greater than M, and SATD is used for blocks / sub-blocks with size less than or equal to M. (i) In one example, M is 64. m) In one example, SATD and mean-removed SATD (MR-SATD) can be selectively utilized according to the LIC flag of the current block. i. In one example, if the LIC flag of the current block is false, the bilateral matching cost measure can be SATD. ii. In one example, if the LIC flag of the current block is true, the bilateral matching cost measure can be MR-SATD. n) In one example, the bilateral matching cost measure may be calculated based on the sum of squared differences (SSD) between two reference blocks / sub-blocks. o) In one example, the bilateral matching cost measure may be calculated based on the mean-removed SSD between two reference blocks / sub-blocks. p) In one example, SSD and mean removed SSD (MR-SSD) may be selectively utilized according to the size of the current block / sub-block. i. In one example, mean removed SSD is used for blocks / sub-blocks of size greater than M, and SSD is used for blocks / sub-blocks of size less than or equal to M. (i) In one example, M is 64. q) In one example, SSD and mean removed SSD (MR-SSD) can be selectively utilized according to the LIC flag of the current block. i. In one example, if the LIC flag of the current block is false, the bilateral matching cost measure can be SSD. ii. In one example, if the LIC flag of the current block is true, the bilateral matching cost measure can be MR-SSD. r) In one example, the bilateral matching cost measure can be the weighted SAD / Weighted MR-SAD / Selective weighted MR-SAD and SAD / Weighted SATD / Weighted MR-SATD / Selective weighted MR-SATD and SATD / Weighted SSD / Weighted MR-SSD / Selective weighted MR-SSD and SSD. i. In one example, weighting means applying different weights to each sample based on its row and column index in the reference block / sub-block when calculating the distortion between two reference blocks / sub-blocks. ii. In one example, weighting means that when calculating the distortion between two reference blocks / sub-blocks, different weights are applied to each sample based on its position in the reference block / sub-block. iii. In one example, weighting means that when calculating the distortion between two reference blocks / sub-blocks, different weights are applied to each sample based on the distance of each sample to the center position of the reference block / sub-block. s) In one example, if MR-SAD / MR-SATD / MR-SSD is used for bilateral matching cost measurement, LIC may not be used when deriving reference blocks / sub-blocks. t) In one example, the bilateral matching cost can be calculated as follows: bilCost = w1*mvDistanceCost+w2*distortionCost. i. In one example, distortionCost can be the weighted SAD between two reference blocks / sub-blocks / Weighted MR-SAD / Weighted SATD / Weighted MR-SATD / Weighted SSD / Weighted MR- SSD / SAD / MR-SAD / SATD / MR-SATD / SSD / MR-SSD. ii. In one example, mvDistanceCost may be the sum of the absolute mv differences between the search point and the start point in the horizontal and vertical directions. iii. In one example, w1 and w2 may be predetermined, or transmitted by a signal, or derived from decoded information. (i) In one example, w1 is a weighting factor set to 4, and w2 is a weighting factor set to 1. 8. The bilateral matching cost or template matching cost can be calculated based on the prediction / reference samples modified by the function. a) In one example, the prediction / reference samples may be filtered before being used to calculate the bilateral matching cost or the template matching cost. b) In one example, the prediction / reference sample S may be modified to a*S+b before being used to calculate the bilateral matching cost or the template matching cost. c) In one example, the modification may depend on the codec mode of the block, such as whether the block is LIC coded or BCW coded. 2.15. Use of multiple cost functions in codec data refinement in image / video codecs The term "block" may refer to a codec tree block (CTB), a codec tree unit (CTU), a codec block (CB), a CU, a PU, a TU, a PB, a TB, or a video processing unit including a plurality of samples / pixels. A block may be rectangular or non-rectangular. In the present disclosure, the phrase "motion candidate" may refer to a merged motion candidate in a regular / extended merge list indicated by a merge candidate index, or an AMVP motion candidate in a regular / extended AMVP list indicated by an AMVP candidate index, or an AMVP motion candidate, or a merged motion candidate. In this disclosure, a motion candidate is referred to as "refined" if its motion information is modified based on information transmitted by signal from the encoder or derived at the decoder. For example, the motion vector can be refined by DMVR, FRUC, TM Merge, TM AMVP, TM GPM, TM CIIP, TM Affine, MMVD, GMVD, Affine MMVD, BDOF, etc. In the present disclosure, the phrase "codec data refinement" may refer to a refinement process to derive or refine a signaled / decoded / derived prediction mode, prediction direction, or signaled / decoded / derived motion information, prediction and / or reconstructed samples for a block. In one example, the refinement process may include motion candidate reordering. In the following discussion, a "template-based" block may refer to a block that uses a template matching-based method to derive or refine the coded information during the encoding / decoding process, such as template matching-based motion derivation, template matching-based motion list reconstruction, LIC, symbol prediction, template matching-based block vector (e.g., used in IBC mode) derivation, DIMD, template matching-based non-inter (e.g., intra) prediction, etc. The template matching-based method may be combined with any other codec tools, such as MMVD, CIIP, GMP, FRUC, Affine, BDOF, DMVR, OBMC, etc. In another example, a "template-based" block may also refer to a block that uses (adjacent or non-adjacent) neighboring reconstructed samples based on specific rules to derive or refine its decoded information, such as the DIMD method in Section 2.27 and the TIMD method in Section 2.29. In the following discussion, a "bilateral codec-based" block may refer to a block that uses a bilateral matching-based method to derive or refine the coded information during the encoding / decoding process, such as bilateral matching-based motion derivation, bilateral matching-based motion list reconstruction, etc. The bilateral codec-based method can be combined with any other codec tools, such as MMVD, CIIP, GPM, FRUC, Affine, DMVR, etc. W and H are the width and height of the current block (eg, luma block). W*H is the size of the current block (eg, luma block). 1. The cost definition may rely on the output of multiple error functions (eg, distortion measurement methods) regarding the error / difference between two samples / blocks to be evaluated in a codec data refinement process of the current block. a) In one example, the error function may be: i.SAD, ii.SATD, iii.MR-SAD, iv.MR-SATD, v. Gradient information, vi.SSE / SSD, vii.MR-SSE / MR-SSD, viii. Weighted SAD / weighted MR-SAD, ix. Weighted SATD / weighted MR-SATD, x. Weighted SSD / weighted MR-SSD, xi. Weighted SSE / Weighted MR-SSE. b) In one example, the error function may be performed at a block level or a sub-block level. i. Alternatively, additionally, the error function may be different for the two sub-blocks. ii. Alternatively, additionally, the final output of the estimated error for the block may be based on the outputs of the sub-blocks, for example, the sum of the outputs of the error function applied to each sub-block. 2. When the cost definition depends on the output of multiple functions, the following items may further apply: a) In one example, the cost function may rely on a linear weighted sum of multiple error functions. b) In one example, the cost function may rely on a non-linear weighted sum of multiple error functions. c) In one example, the cost function may further depend on estimated bits for the side information. d) In one example, the cost function may be defined as: Where R represents the estimated bits for the side information, W i and E i denote the weight applied to the i-th error function and the output of the i-th error function, respectively. 3. Multiple refinement processes may be applied to one block, wherein at least more than two different cost functions are applied to at least two refinement processes. a) In one example, a first refinement process may be invoked using a first cost function. Based on the output of the first refinement process, a second cost function is further applied to a second refinement process. b) The above method can be applied to blocks based on template coding. 4. Whether to use multiple refinement processes and / or how to select one or more error functions and / or how to define the cost function and / or which samples to involve in the error function may depend on the decoding information of the current block and / or the decoding information of neighboring (adjacent or non-adjacent) blocks of the current block. a) In one example, how to select one or more error functions and / or how to define the cost function may depend on the codec tools applied to the current block and / or neighboring blocks of the current block. i. In one example, the codec is LIC. (i) In one example, SSD and mean removed SSD (MR-SSD) can be selectively utilized according to the LIC flag of the current block. a) In one example, if the LIC flag of the current block is false, the template matching cost measure can be SSD. b) In one example, if the LIC flag of the current block is true, the template matching cost measure can be MR-SSD. (ii) In one example, if MR-SAD / MR-SATD / MR-SSD is used for the template matching cost measurement, the linear function used in the LIC process when deriving the reference template may not be used. (iii) In one example, if MR-SAD / MR-SATD / MR-SSD is used for the bilateral matching cost measurement, the linear function used in the LIC process when deriving the reference block may not be used. b) In one example, it may depend on block size, temporal layer, low latency check flag, etc. c) In one example, it may depend on whether the motion information of the current block is similar / identical to the motion information of the neighboring blocks. d) In one example, it may depend on a reference picture list and / or reference picture information. i. In one example, for list X, a first error function (e.g., SAD / SSE) may be used, and for list Y (Y=1-X), a second error function (e.g., MR- SAD / MR-SSE) can be used. ii. Alternatively, additionally, the final cost may be based on the cost of each reference picture list. 5. The above method can be applied to any codec data refinement process, for example, for blocks based on template codecs, for blocks based on bilateral codecs (eg, DMVR in VVC). 2.16. Samples used in codec data refinement for image / video codecs The term "block" may refer to a codec tree block (CTB), a codec tree unit (CTU), a codec block (CB), a CU, a PU, a TU, a PB, a TB, or a video processing unit including a plurality of samples / pixels. A block may be rectangular or non-rectangular. In the present disclosure, the phrase "motion candidate" may refer to a merged motion candidate in a regular / extended merge list indicated by a merge candidate index, or an AMVP motion candidate in a regular / extended AMVP list indicated by an AMVP candidate index, or an AMVP motion candidate, or a merged motion candidate. In this disclosure, a motion candidate is referred to as "refined" if its motion information is modified according to information signaled from an encoder or derived at a decoder. For example, a motion vector may be refined by DMVR, FRUC, TM Merge, TM AMVP, TM GPM, TM CIIP, TM Affine, MMVD, GMVD, Affine MMVD, BDOF, etc. In the present disclosure, the phrase "codec data refinement" may refer to a refinement process to derive or refine a signaled / decoded / derived prediction mode, prediction direction, or signaled / decoded / derived motion information, prediction, and / or reconstructed samples for a block. In one example, the refinement process may include motion candidate reordering. In the following discussion, a "template-based" block may refer to a block that uses a template matching-based method to derive or refine the coded information during the encoding / decoding process, such as template matching-based motion derivation, template matching-based motion list reconstruction, LIC, symbol prediction, template matching-based block vector (e.g., used in IBC mode) derivation, DIMD, template matching-based non-inter (e.g., intra) prediction, etc. The template matching-based method may be combined with any other codec tools, such as MMVD, CIIP, GMP, FRUC, Affine, BDOF, DMVR, OBMC, etc. In another example, a "template-based" block may also refer to a block that uses (adjacent or non-adjacent) neighboring reconstructed samples based on specific rules to derive or refine its decoded information, such as the DIMD method in 2.27 and the TIMD method in 2.29. In the following discussion, a "bilateral codec-based" block may refer to a block that uses a bilateral matching-based method to derive or refine the coded information during the encoding / decoding process, such as bilateral matching-based motion derivation, bilateral matching-based motion list reconstruction, etc. The bilateral codec-based method can be combined with any other codec tools, such as MMVD, CIIP, GPM, FRUC, Affine, DMVR, etc. W and H are the width and height of the current block (eg, luma block). W*H is the size of the current block (eg, luma block). 1. The error / cost estimation in the codec data refinement process may depend on both the reference samples corresponding to the current block (eg, the reference block used in bilateral matching) and the reference samples corresponding to the template of the current block. a) Alternatively, it may depend on both the reference samples corresponding to the current block and the samples in the template of the current block. b) In one example, the template may be neighboring samples (adjacent or non-adjacent) of the current block. 2. Multiple refinement processes may be applied to one block with different templates applied to at least two refinement processes. a) In one example, a first refinement process may involve a first template. Based on the output of the first refinement process, a second template is further utilized in a second refinement process. b) In one example, the first template may contain more samples than the second template. c) In one example, the first template and the second template may contain at least one different sample. d) In one example, the first refinement process and the second refinement process may use different cost / error functions. 3. Whether to use multiple refinement processes, and / or how to select one or more error functions and / or how to define the cost function and / or which samples are to be involved in the error function may depend on the decoding information of the current block and / or the decoding information of neighboring blocks (adjacent or non-adjacent). a) In one example, how to select one or more error functions and / or how to define the cost function may depend on the codec tools applied to the current block and / or neighboring blocks. i. In one example, the codec is LIC. (i) In one example, SSD and mean removed SSD (MR-SSD) may be selectively used according to the LIC flag of the current block. a) In one example, if the LIC flag of the current block is false, the template matching cost measure may be SSD. b) In one example, if the LIC flag of the current block is true, the template matching cost measure may be MR-SSD. (ii) In one example, if MR-SAD / MR-SATD / MR-SSD is used for the template matching cost measurement, the linear function used in the LIC process when deriving the reference template may not be used. (iii) In one example, if MR-SAD / MR-SATD / MR-SSD is used for the bilateral matching cost measurement, the linear function used in the LIC process when deriving the reference template may not be used. b) In one example, it may depend on block size (e.g., W, H), temporal layer, low latency check flag, etc. c) In one example, it may depend on whether the motion information of the current block and the neighboring blocks are similar / identical. d) In one example, it may depend on a reference picture list and / or reference picture information. i. In one example, for list X, a first error function (e.g., SAD / SSE) may be used, and for list Y (Y=1-X), a second error function (e.g., MR-SAD / MR-SSE) Can be used. ii. Alternatively or additionally, the final cost may be based on the cost of each reference picture list. 4. In one example, LIC may be enabled for reference list X and disabled for reference list Y. a) In one example, the final prediction of the current block may be a weighted average of the LIC prediction from reference list X and the regular prediction from reference list Y. 5. The above method can be applied to any codec data refinement process, for example, for blocks based on template codecs, for blocks based on bilateral codecs (eg, DMVR in VVC). 2.17. Adaptive Motion Candidate List It should be noted that the terms mentioned below are not limited to the specific terms defined in existing standards. Any variation of the codec tool is also applicable. For example, the term "GPM" is used to represent any codec tool that derives two sets of motion information and uses the derived information and the partitioning mode to obtain the final prediction. For example, TPM is also considered to be GPM. Note that the proposed method can be applied to the merge candidate list construction process for inter-frame codec blocks (e.g., transform motion), affine codec blocks, TM codec blocks, or IBC codec blocks, or other motion candidate list construction processes (e.g., ordinary AMVP list, affine AMVP list, IBC AMVP list, HMVP table). Cost functions other than template matching cost are also applicable for motion candidate re-ranking. W and H are the width and height of the current block (eg, luma block). 1. The template / bilateral matching cost C can be calculated as f(C) before being used to compare with another template matching cost. a. In one example, f(C) = w*C, where w is represented as the cost factor. b. In one example, f(C) = w*C + u. c. In one example, f(C)=Shift((w*C),s). d. In one example, w and / or u and / or s are integers. e. In one example, a first template matching cost for a first motion candidate may be multiplied by a cost factor before being compared to a second template matching cost for a second motion candidate. f. In one example, it is proposed that the cost factor for a motion candidate may depend on the position of the candidate before re-ranking. i. In one example, a motion candidate at the i-th position may be assigned a cost factor that is smaller than a cost factor of a motion candidate at the j-th position, where j>i, eg, j=i+1. (i) In one example, the cost factor of the motion candidate at the i-th position is 4, and the cost factor of the motion candidate at the j-th position is 5. (ii) In one example, the cost factor of the motion candidate at the i-th position is 1, and the cost factor of the motion candidate at the j-th position is 5. ii. In one example, a motion candidate at the i-th position may be assigned a cost factor greater than a cost factor of a motion candidate at the j-th position, where j>i, e.g., j=i+1. iii. In one example, a motion candidate in a p-th group (e.g., including M motion candidates) may be assigned a cost factor less than a cost factor of a motion candidate in a q-th group (e.g., including N motion candidates), where q>p, e.g., q=p+1. (i) Alternatively, motion candidates in the pth group (eg, including M motion candidates) may be assigned a cost factor greater than a cost factor of motion candidates in the qth group (eg, including N motion candidates), where q>p, eg, q=p+1. (ii) In one example, M may be equal to N. For example, M=N=2. (iii) In one example, M may not be equal to N. For example, M=2, N=3. (iv) In one example, the cost factor of the motion candidates of the pth group is 4, and the cost factor of the motion candidates of the qth group is 5. (v) In one example, the cost factor of the motion candidates of the pth group is 1, and the cost factor of the motion candidates of the qth group is 5. iv. In one example, the cost factor may not be applied to sub-block motion candidates. v. In one example, the cost factor may not be applied to affine motion candidates. vi. In one example, the cost factor may not be applied to SbTMVP motion candidates. g. In one example, the cost factor for motion candidates in one group / position may be adaptive. i. In one example, the cost factor of a motion candidate in a group / position may depend on the codec mode of the neighboring codec blocks. (i) In one example, the cost factor of the SbTMVP merge candidate may depend on the number of neighboring affine codec blocks. (ii) In one example, the neighboring codec block may include five spatial neighboring blocks: (like Figure 4 ) and / or (multiple) temporal neighboring blocks (such as Figure 7 at least one of the above (shown). (iii) In one example, when the spatially adjacent affine codec block (e.g. Figure 4 When the number of affine codec blocks (such as Figure 4 ) is 1, the cost factor of the SbTMVP merge candidate can be 0.5; when the spatial domain adjacent affine codec block (such as Figure 4 When the number of (shown) is 2, The cost factor of the SbTMVP merge candidate can be 0.8; otherwise, The cost factor of the SbTMVP merge candidate can be 1 (which means remain unchanged). (iv) In one example, when the spatially adjacent affine codec block ( Figure 4 When the number of affine codec blocks (shown in FIG. 1 ) is 0, the cost factor of the SbTMVP merge candidate can be 0.2; when the number of spatially adjacent affine codec blocks ( Figure 4 The number of When the cost factor of the SbTMVP merge candidate can be 0.5; when the spatial domain is adjacent to the affine codec block ( Figure 4 When the number of (shown) is greater than or equal to 2, The cost factor of the SbTMVP merge candidate may be 0.8. (v) In one example, when the spatially adjacent affine codec block (e.g. Figure 4 When the number of affine codec blocks (such as Figure 4 ) is 1, the cost factor of the SbTMVP merge candidate can be 5; when the spatial domain adjacent affine codec block (such as Figure 4 When the number of (shown) is 2, The cost factor of the SbTMVP merge candidate can be 8; otherwise, the SbTMVP The cost factor of a merge candidate may be 10 (where the cost factor of an affine merge candidate is 10). (vi) In one example, when the spatially adjacent affine codec block (e.g. Figure 4 When the number of affine codec blocks (such as Figure 4 ) is 1, the cost factor of the SbTMVP merge candidate can be 5; when the spatial domain adjacent affine codec block (such as Figure 4 The number of When , the cost factor of the SbTMVP merge candidate can be 8 (where the cost factor of the affine merge candidate is 10). 2. The subgroup sizes of different codec modes can be different. a. The coding mode may include normal / sub-block / TM merge mode. i. For normal merge mode, the subgroup size may be K1 (eg, K1=5). ii. For the sub-block merging mode, the sub-group size may be K2 (eg, K2=3). iii. For TM merging mode, the subgroup size may be K3 (eg, K3=3). b. The subgroup size may be greater than or equal to the maximum number of subblock merging candidates defined in the sps / picture / slice header for the subblock merging mode (which means reordering the whole list together). c. The subgroup size may be greater than or equal to the maximum number of TM merge candidates defined in the sps / picture / slice header for TM merge mode (which means reordering the whole list together). d. The subgroup size for a codec mode may depend on the maximum number of motion candidates in the codec mode. e. The subgroup size for the subblock merging mode can be adaptively dependent on the number of neighboring affine coded blocks. i. In one example, the adjacent codec block may include five spatial adjacent blocks (such as Figure 4 ) and / or (multiple) temporal neighboring blocks (such as Figure 7 at least one of the above (shown). ii. In one example, when the spatial domain is adjacent to the affine codec block (such as Figure 4 ) is 0 or 1, the subgroup size for the subblock merging mode can be 3; when the spatial domain is adjacent to the affine codec block (such as Figure 4 When the number of sub-blocks (as shown) is greater than 1, the sub-group size of the sub-block merging mode may be 5; 3. The template size can be different for different codec modes. a. The coding mode may include normal / sub-block / TM merge mode. i. For the normal merge mode, the template size may be K1 (eg, K1=1). ii. For the sub-block merging mode, the template size may be K2 (eg, K2=1, 2, or 4). iii. For TM merge mode, the template size may be K3 (eg, K3=1). 4. Whether and / or how to reorder motion candidates may depend on the coding modes of neighboring coded blocks. a. In one example, the adjacent coded blocks may include five spatially adjacent blocks (eg Figure 4 ) and / or (multiple) temporal neighboring blocks (such as Figure 7 at least one of the above (shown). b. When using the spatially adjacent codec blocks of the conventional merge mode (such as Figure 4 When the number of merging candidates (as shown) is greater than or equal to K (eg, K=1), the conventional merge candidates may be reordered. c. When using the spatial domain adjacent codec block of the sub-block merging mode (such as Figure 4 ), when the number of sub-block merging candidates is greater than or equal to K (eg, K=1), the sub-block merging candidates may be reordered. d. When using the spatially adjacent codec blocks of the affine merge mode (such as Figure 4 ) is greater than or equal to K (eg, K=1), the affine merge candidates may be reordered. e. When using the affine merge mode, the spatial domain of the adjacent codec blocks (such as Figure 4 When the number of SbTMVP merge candidates (as shown) is greater than or equal to K (eg, K=1, 2, or 3), the SbTMVP merge candidates may be reordered. f. When using the spatial domain adjacent codec block of the TM merge mode (such as Figure 4 When the number of TM merge candidates (as shown) is greater than or equal to K (eg, K=1), the TM merge candidates may be reordered. 5. HMVP motion candidates in the HMVP table can be reordered based on template / bilateral matching, etc. a. In one example, costs are assigned to HMVP motion candidates, and the HMVP candidates are adaptively reordered in descending order of HMVP candidate cost. i. In one example, the cost of the HMVP candidate may be a template matching cost. b. In one example, HMVP motion candidates can be reordered before encoding and decoding blocks. i. In one example, HMVP motion candidates may be reordered before encoding inter-coded blocks. c. In one example, HMVP motion candidates may be reordered in different ways according to the coding information of the current block and / or neighboring blocks. General rights 6. Whether and / or how to apply the method disclosed above can be transmitted by a signal at the sequence level / picture group level / picture level / slice level / tile group level, such as in a sequence header / picture header / SPS / VPS / DPS / DCI / PPS / APS / slice header / tile group header. 7. Whether and / or how to apply the methods disclosed above may be signaled at PB / TB / CB / PU / TU / CU / VPDU / CTU / CTU row / slice / tile / sub-picture / other kind of region containing more than one sample or pixel. 8. Whether and / or how to apply the methods disclosed above may depend on the coded information, such as codec mode, block size, color format, single / dual tree partitioning, color component, slice / picture type. 2.18. Adaptive GPM Candidate List The term "block" may refer to a codec tree block (CTB), a codec tree unit (CTU), a codec block (CB), a CU, a PU, a TU, a PB, a TB, or a video processing unit including a plurality of samples / pixels. A block may be rectangular or non-rectangular. It should be noted that the terms mentioned below are not limited to specific terms defined in existing standards. Any changes in codecs also apply. For example, the term "GPM" is used to represent any codec that derives two or more sets of motion information and uses the derived motion information and partitioning patterns / weighted masks to obtain the final prediction. For example, TPM is also considered a GPM. Note that the proposed method can be applied to the merge candidate list construction process for inter-frame codec blocks (e.g., transform motion), affine codec blocks, TM codec blocks, GPM codec blocks or IBC codec blocks, or other motion candidate list construction processes (e.g., ordinary AMVP list; affine AMVP list; IBC AMVP list; HMVP table). Cost functions other than template matching cost are also applicable for motion candidate re-ranking. In the following, a template is a set of adjacent or non-adjacent reconstructed / predicted samples of the current block. The reference samples of the template (i.e., the reference template) are the mapping of the template in the reference picture, which depends on the motion information of the current block. "Template above" means a template constructed by a set of adjacent or non-adjacent reconstructed / predicted samples above the current block and its reference template. "Template on the left" means a template constructed by a set of adjacent or non-adjacent reconstructed / predicted samples on the left side of the current block and its reference template. "Template above and on the left" includes the template above and the template on the left. In the following, in one example, a GPM candidate list is referred to as OGPMList, where the GPM candidates are directly derived from a rule merge list (before or without motion refinement based on template matching); a refined GPM candidate list is referred to as AGPMList, where the GPM candidates are refined by a first refinement method (e.g., template matching using an upper template); a refined GPM candidate list is referred to as LGPMList, where the GPM candidates are refined by a second refinement method (e.g., template matching using a left template); and a refined GPM candidate list is referred to as LAGPMList, where the GPM candidates are refined by a third refinement method (e.g., template matching using left and upper templates). W and H are the width and height of the current block (eg, luma block). 1. It is proposed that for a GPM-coded block, a decoded candidate index may correspond to a candidate with a different candidate index in a candidate list for the GPM-coded block. a. Alternatively, additionally, the candidate list constructed for the GPM-coded block may be reordered before being used, and the coded index corresponds to the reordered candidate list. b. Alternatively, additionally, for the first type of GPM-coded blocks, the candidate list may be reordered, while for the second type of GPM-coded blocks, the candidate list may not be reordered. i. In one example, the first type is a template-based GPM-encoded block. ii. In one example, the second type is a GPM-coded block based on MMVD (eg, GMVD). c. Alternatively, additionally, for a first type of GPM-coded blocks, the candidate list may be reordered using a first rule, and for a second type of GPM-coded blocks, the candidate list may be reordered using a second rule. d. In one example, the reordering method for blocks decoded with GPM can be the same as that for blocks not decoded with GPM. The reordering method of blocks for GPM codec is the same. i. The reordering method for blocks encoded with GPM can be different from that for blocks not encoded with GPM. Block reordering method for codec. 2. It is proposed that for a GPM-encoded block, the decoded candidate index may correspond to the candidate index from the GPM-encoded block. The candidates in the refinement candidate list of the coded block. a. Alternatively, additionally, the candidate list constructed for the GPM-coded block may first be refined before being used, and the decoded index corresponds to the refined candidate list. b. Alternatively, additionally, for the first type of GPM-coded blocks, the candidate list may be refined, while for the second type of GPM-coded blocks, the candidate list may not be refined. i. In one example, the first type is a template-based GPM-encoded block. ii. In one example, the second type is a GPM-coded block based on MMVD (eg, GMVD). c. Alternatively, additionally, for a first type of GPM-coded blocks, the candidate list may be refined with a first rule, and for a second type of GPM-coded blocks, the candidate list may be refined with a second rule. d. In one example, the refinement method for a block decoded with GPM can be the same as that for a block not decoded with GPM. The block refinement method for GPM codec is the same. i. The refinement method for GPM-coded blocks may be different from the refinement method for non-GPM-coded blocks. 3. In one example, the GPM candidates may be divided into sub-groups, and whether and / or how to reorder the GPM candidates may depend on the sub-groups of the GPM candidates. a. In one example, only the first subgroup can be reordered. b. In one example, the last subgroup cannot be reordered. c. In one example, the last subgroup cannot be reordered. However, if the last subgroup is also the first subgroup, it can be reordered. d. Different subgroups can be reordered individually. e. Two candidates in different subgroups cannot be compared and / or reordered. f. If the first subgroup precedes the second subgroup, then after reordering, the first candidate in the first subgroup must be placed before the second candidate in the second subgroup. 4. In one example, GPM candidates that are not included in the reordering process may be processed in a specified manner. a. In one example, for candidates that do not need to be reordered, they are arranged in the merge candidate list according to the original order. b. In one example, the candidates that are not to be reordered may be placed after the candidates that are to be reordered. c. In one example, candidates that are not to be reordered may be placed before candidates that are to be reordered. 5. The GPM candidate list to be reordered can be referenced Case 1: The first candidate lists prepared for two GPM partitions are used to derive separate GPM candidate lists for each GPM partition. Case 2: Second GPM candidate list used for each GPM partition. Usually the second GPM candidates are derived from the first candidate list. a. In one example, in case 1, the reordering method may be the same as the reordering method used for a conventional merge candidate list. b. In one example, in case 1, if the corresponding candidate is bidirectionally predicted, the template matching method in the reordering method can be performed in a bidirectionally predicted manner. c. In one example, in case 2, the template matching method in the reordering method cannot be performed in a bidirectional predictive manner. d. In one example, in case 2, the reordering method can be the same for all GPM partitions. e. In one example, in case 2, the reordering method may be different for different GPM partitions. 6. In the above example, the GPM-encoded block may be a GPM-encoded block with a merge mode or a GPM-encoded block with an AMVP mode. a. Alternatively, additionally, the above merge candidates may be replaced by AMVP candidates. General rights 7. Whether and / or how to apply the method disclosed above can be transmitted by signal at sequence level / picture group level / picture level / slice level / tile group level, such as in sequence header / picture header / SPS / VPS / DPS / DCI / PPS / APS / In the strip header / tile group header. 8. Whether and / or how to apply the methods disclosed above can be transmitted by signals in PB / TB / CB / PU / TU / CU / VPDU / CTU / CTU row / slice / tile / sub-picture / other kinds of regions containing more than one sample or pixel. 9. Whether and / or how to apply the method disclosed above may depend on the coded information, such as codec mode, block size, GPM partition information, color format, single / dual tree partition, color component, slice / picture type. 2.19. Adaptive GPM Candidate List The term "block" may refer to a codec tree block (CTB), a codec tree unit (CTU), a codec block (CB), a CU, a PU, a TU, a PB, a TB, or a video processing unit including a plurality of samples / pixels. A block may be rectangular or non-rectangular. It should be noted that the terms mentioned below are not limited to specific terms defined in existing standards. Any changes in codecs also apply. For example, the term "GPM" is used to represent any codec that derives two or more sets of motion information and uses the derived motion information and partitioning mode / weighting mask to obtain the final prediction. For example, TPM is also considered a GPM. Note that the proposed method can be applied to the merge candidate list construction process for inter-coded blocks (e.g., translational motion), affine-coded blocks, TM-coded blocks, GPM-coded blocks, or IBC-coded blocks; or other motion candidate list construction processes (e.g., ordinary AMVP list; affine AMVP list; IBC AMVP list; HMVP table). Cost functions other than template matching cost are also applicable for motion candidate re-ranking. In the following, a template is a set of adjacent or non-adjacent reconstructed / predicted samples adjacent to the current block. The reference samples of the template (i.e., the reference template) are the mapping of the template in the reference picture, which depends on the motion information of the current block. "Template above" means a template constructed by a set of adjacent or non-adjacent reconstructed / predicted samples adjacent to the current block and its reference template above. "Template on the left" means a template constructed by a set of adjacent or non-adjacent reconstructed / predicted samples adjacent to the left of the current block and its reference template. "Template above and left" includes the template above and the template on the left. In the following, in one example, a GPM candidate list is referred to as OGPMList, where the GPM candidates are directly derived from a rule merge list (before or without motion refinement based on template matching); a refined GPM candidate list is referred to as AGPMList, where the GPM candidates are refined by a first refinement method (e.g., template matching using an upper template); a refined GPM candidate list is referred to as LGPMList, where the GPM candidates are refined by a second refinement method (e.g., template matching using a left template); and a refined GPM candidate list is referred to as LAGPMList, where the GPM candidates are refined by a third refinement method (e.g., template matching using left and upper templates). Regarding the types of GPM candidates in the original GPM candidate list, the GPM candidates derived in the first step of the GPM candidate list construction process in Section 2.29 are called GPM parity-based candidates; the GPM candidates derived in the second step of the GPM candidate list construction process in Section 2.29 are called GPM anti-parity-based candidates; and the GPM candidates derived in the third step of the GPM candidate list construction process in Section 2.29 are called filled GPM candidates. W and H are the width and height of the current block (eg, luma block). 1. In one example, if the codec mode is GPM, the merge candidates may be reordered. a. In one example, merge candidates in OGPMList may be reordered. i. In one example, at least two merge candidates in the OGPMList may be reordered. ii. In one example, at least one type of template may be used for OGPMList reordering. iii. Alternatively, merge candidates in OGPMList cannot be reordered. iv. In one example, the first type of template may include only left neighboring samples of the current block. v. In one example, the second type of template may include only the upper neighboring samples of the current block. vi. In one example, the third type of template may include neighboring samples above and to the left of the current block. vii. The reordering process can be called after the parsing process but before the MV reconstruction process. b. In one example, the merge candidates in AGPMList may be reordered. i. In one example, at least two merge candidates in the AGPMList may be reordered. ii. In one example, at least one type of template may be used for AGPMList reordering. iii. In one example, the first type of template may include only neighboring samples above the current block. iv. In one example, the second type of template may include neighboring samples to the left and above the current block. c. In one example, the merge candidates in LGPMList may be reordered. i. In one example, at least two merge candidates in LGPMList may be reordered. ii. In one example, at least one type of template may be used for LGPMList reordering. iii. In one example, the first type of template may include only neighboring samples to the left of the current block. iv. In one example, the second type of template may include neighboring samples to the left and above the current block. d. In one example, the merge candidates in LAGPMList may be reordered. i. In one example, at least two merge candidates in LAGPMList may be reordered. ii. In one example, at least one type of template may be used for LAGPMList reordering. iii. In one example, the first type of template may include only neighboring samples to the left of the current block. iv. In one example, the second type of template may include only neighboring samples above the current block. v. In one example, the third type of template may include neighboring samples to the left and above the current block. e. In one example, whether and / or how to reorder merge candidates in the GPM list may depend on codec information. i. In one example, whether to reorder merge candidates in the GPM list may depend on whether template matching based motion refinement is applied to one GPM partition or two GPM partitions (ie, GPM-encoded CUs). (i) For example, if the motion of one GPM partition or two GPM partitions (ie, GPM-encoded CUs) is not refined based on template matching (eg, template matching flag equals false), the corresponding GPM lists may not be reordered. a) For example, if the GPM partition is encoded using the merge candidates in OGPMList (eg, no motion refinement is applied), the merge candidates in OGPMList may not be reordered. (ii) For example, if the movement of a GPM division or two GPM divisions (ie, a CU encoded by a GPM) is refined based on template matching (eg, the template matching flag is equal to true), the corresponding GPM list may be reordered. a) For example, if the GPM partition is encoded using the merge candidate in AGPMList (e.g., the template matching motion refinement method using the upper template is applied), then The merge candidates in AGPMList can be reordered. b) For example, if the GPM partition is encoded using the merge candidate in LGPMList (eg, a template matching motion refinement method using the left template is applied), then The merge candidates in LGPMList can be reordered. c) For example, if the GPM partition is using LAGPMList If the merge candidates in LAGPMList are encoded and decoded (for example, a template matching motion refinement method using the left template and the upper template is applied), the merge candidates in LAGPMList can be reordered. ii. In one example, how to reorder the merge candidates in the GPM list may depend on the GPM partition information (eg, partition mode, partition angle, partition distance, etc.). (i) For example, the above template may be used for merging candidate reordering when the current GPM partition is partitioned by the first partition angle (or partition mode, or partition distance, etc.). (ii) For example, the left template may be used for merging candidate reordering when the current GPM partition is partitioned by a second partition angle (or partition mode, or partition distance, etc.). (iii) For example, the template on the left and the template above can be used for merging candidate reordering when the current GPM partition is divided by a third partition angle (or partition mode, or partition distance, etc.). (iv) For example, a template type corresponding to the first / second / third division angle (or division mode, or division distance, etc.) may be specified. (v) For example, at least one lookup table (i.e., mapping table) is used to map what specified division angle (or division mode, or division distance, etc.) corresponds to what type of template (e.g., upper template, left template, or upper and left templates. ) f. In one example, merge candidates in OGPMList may not be reordered, and merge candidates in AGPMList and / or LGPMList and / or LAGPMList may be reordered. 2. Merge candidates can be adaptively rearranged in the final GPM candidate list according to one or some criteria. a. In one example, the GPM candidate list can be i.OGPMList, ii.AGPMList, iii.LGPMLIst, iv.LAGPMList. b. GPM candidates can be divided into several subgroups. i. For example, the number of GPM candidates in a sub-group (such as X=3 or 5 or any other integer value) may be predefined. c. In one example, part or all of the current GPM candidate list construction process is called first, followed by a reordering of the candidates in the GPM list. i. Alternatively, the candidates in the first subgroup may be reordered and they should be added before those in the second subgroup, where the first subgroup is added before the second subgroup. ii. The construction process may include pruning methods. d. In one example, merge candidates may be adaptively rearranged before retrieval. i. In one example, the process of ranking merge candidates can be adaptively processed before obtaining merge candidates to be used in the motion compensation process. e. The criterion can be based on template matching cost. i. In one example, the cost function between the current template and the reference template can be (i) SAD / MR-SAD, (ii) SATD / MR-SATD, (iii) SSD / MR-SSD, (iv) SSE / MR-SSE, (v) Weighted SAD / Weighted MR-SAD, (vi) Weighted SATD / weighted MR-SATD, (vii) Weighted SSD / Weighted MR-SSD, (viii) Weighted SSE / weighted MR-SSE, (ix) Gradient information. 3. When deriving two motions of two geometric partitions, the process can be: a. In one example, if TM is not applied to a partition, the motion may be derived based on the signaled merge index from OGPMList / reordered OGPMList. b. In one example, if TM is applied to one partition, motion can be derived based on the signaled merge index from AGPMList / reordered AGPMList or LGPMList / reordered LGPMLIst or LAGPMList / reordered LAGPMLIst, depending on the partition angle and partition index. i. In one example, if the partition angle is X (eg, 0), for the first partition, AGPMList / reordered AGPMList will be used; for the second partition, LAGPMList / reordered LAGPMLIst will be used. c. In one example, if TM is applied to one partition, the motion can be derived based on the signaled merge index from the AGPMList / reordered AGPMList. d. In one example, if TM is applied to a partition, the motion can be derived based on the signaled merge index from LGPMList / reordered LGPMLIst. e. In one example, if TM is applied to a partition, the motion can be derived based on the signaled merge index from LAGPMList / reordered LAGPMLIst. 4. Whether and / or how to reorder GPM candidates may depend on the category of the GPM candidates. a. In one example, only candidates based on GPM parity may be reordered. b. In one example, only GPM parity-based and GPM anti-parity-based candidates may be reordered. c. In one example, filled GPM candidates may not be reordered. d. In one example, two candidates in different GPM lists cannot be compared and / or reordered. e. In one example, only the top N GPM candidates may be reordered. i. In one example, N is set equal to 5. 5. In the above example, the GPM-encoded block may be a GPM-encoded block with a merge mode or a GPM-encoded block with an AMVP mode. a. Alternatively, additionally, the above merge candidates may be replaced by AMVP candidates. General rights 6. Whether and / or how to apply the above disclosed method can be applied at sequence level / picture group level / picture level / The slice level / tile group level is signaled, such as in the sequence header / picture header / SPS / VPS / DPS / DCI / PPS / APS / Slice Header / Tile Group Header. 7. Whether and / or how to apply the methods disclosed above can be transmitted by signals in PB / TB / CB / PU / TU / CU / VPDU / CTU / CTU row / slice / tile / sub-picture / other kinds of regions containing more than one sample or pixel. 8. Whether and / or how to apply the method disclosed above may depend on the encoded information, such as the encoding and decoding mode, block size, GPM partition information, color format, single / dual tree partition, color component, stripe / Image type. 2.20. Hash-based motion estimation for screen content encoding and decoding The VTM reference software uses hash-based motion estimation to handle the sometimes large and irregular motion in screen content. For each reference picture, a hash table corresponding to 4x4 to 64x64 block sizes is generated using a bottom-up approach as follows: – For each 2x2 block, a block hash value is calculated directly from the original sample values ​​(luminance samples are used if 4:2:0 chroma format, and both luma and chroma sample values ​​are used if 4:4:4 chroma format). A cyclic redundancy check (CRC) value is used as the hash value. – For 4x4, 8x8, 16x16, 32x32, and 64x64 blocks, the hash value of the current block is the CRC value calculated based on the CRC values ​​of its four sub-blocks. To enable efficient search for matching blocks, an inverted index structure is used, where hash values ​​are used as indexes into the table and the table entries contain all blocks with the same hash value as the corresponding table index. Blocks corresponding to a given table index are stored as a linked list. Two CRC values ​​are calculated for each block, one 16-bit hash and the other 24-bit hash. The two hash values ​​are calculated in a similar manner but using different CRC truncation polynomials. The first 16-bit CRC value is used as an inverted index. In the event that more than one matching block is found, the second 24-bit hash value is stored with the block to resolve hash conflicts. In order to reduce the length of the hash table, the hash values ​​of all "simple" blocks (defined as blocks with only one sample value in each row or column) are excluded from the hash table. In motion estimation, if the current block is a square block (except for 128x128 blocks), its hash value is calculated. Then, the encoder queries the corresponding hash table. If a hash match is found, the matching block is used as a reference. If the current block is a rectangular block of size NxM (and without loss of generality assume M>N), it will be divided into several non-overlapping square sub-blocks of size NxN. Fig. 27 An example diagram 2700 showing motion estimation of a rectangular block with hash values ​​of square sub-blocks is shown. Fig. 27 The encoder will find the first non-simple square sub-block and calculate its hash value. The encoder looks up the hash value of this NxN square sub-block on the hash table corresponding to the NxN block size. One or more matching reference blocks are considered as reference block candidates. For each matching reference block candidate, the encoder will continue to check the remaining square sub-blocks (i.e., in Fig. 27 The white area after the first non-simple square sub-block depicted in is equal to the hash value of the square sub-blocks adjacent to the reference block candidate. If the hash values ​​of all square sub-blocks match, the reference block candidate is considered a valid reference block. For inter codecs, a hash-based motion search is performed before testing all codec modes. Additionally, the encoder will reuse the MV of the hash mode as a starting point candidate in the normal motion estimation process. If there is a hash-based motion vector that indicates that the block most likely contains screen content, then fractional motion estimation is skipped. To speed up the encoder, codec modes other than the skip and merge parts of ETM_MERGE_SKIP, ETM_AFFINE, and ETM_MERGE_GPM modes and finer-grained block partitioning are skipped if all of the following conditions are met: – Current block sizes are 64x64, 128x64, or 64x128. – The same reference block is found in the reference picture. – The QP of the reference picture is not greater than the QP of the current picture. 2.21. Luminance Mapping with Chroma Scaling (LMCS) In VVC, a codec tool called "luminance mapping with chroma scaling (LMCS)" is added as a new processing block before loop filtering. LMCS has two main components: 1) in-loop mapping of luma components based on an adaptive piecewise linear model; 2) for chroma components, luma-dependent chroma residual scaling is applied. Fig.28 An example diagram 2800 illustrating luma mapping with a chroma scaling architecture is shown. Fig.28 shows the LMCS architecture from the decoder perspective. Fig.28 The light blue shaded blocks in represent where processing is applied in the mapped domain; and these include inverse quantization, inverse transform, luma intra prediction, and adding the luma prediction to the luma residual. Fig.28 The unshaded blocks in indicate where processing is applied in the original (i.e., unmapped) domain; and these include loop filters such as deblocking, ALF and SAO, motion compensated prediction, chroma intra prediction, adding chroma prediction to chroma residual, and storing the decoded picture as a reference picture. Fig.28 The light yellow shaded blocks in the are the new LMCS functional blocks, including the forward and reverse mapping of the luma signal, and the luma-dependent chroma scaling process. Like most other tools in VVC, LMCS can be enabled / disabled at the sequence level using the SPS flag. 2.21.1 Brightness Mapping with a Piecewise Linear Model The in-loop mapping of the luma component adjusts the dynamic range of the input signal by redistributing codewords within the dynamic range to improve compression efficiency. Luma mapping uses a forward mapping function FwdMap and a corresponding inverse mapping function InvMap. The FwdMap function is signaled using a piecewise linear model with 16 equal segments. The InvMap function does not need to be signaled but is derived from the FwdMap function. The luminance mapping model is signaled in the adaptive parameter set (APS) syntax structure with aps_params_type set to equal 1 (LMCS_APS). A maximum of 4 LMCS APSs can be used in a coded video sequence. Only 1 LMCS APS can be used for a picture. The luminance mapping model is signaled using a piecewise linear model. The piecewise linear model divides the dynamic range of the input signal into 16 equal parts, and for each part, its linear mapping parameters are represented by the number of codewords assigned to the part. Take 10-bit input as an example. By default, each of the 16 fragments is assigned 64 codewords. The number of codewords transmitted by the signal is used to calculate the scaling factor and adjust the mapping function of the fragment accordingly. At the slice level, the LMCS enable flag is signaled to indicate if Fig.28 Whether the LMCS process shown is applied to the current slice. If LMCS is enabled for the current slice, aps_id is signaled in the slice header to identify the APS that carries the luma mapping parameters. Each i-th fragment, i=0...15 of the FwdMap piecewise linear model is defined by two input pivot points InputPivot[] and two output (mapped) pivot points MappedPivot[]. InputPivot[] and MappedPivot[] are calculated as follows (assuming 10-bit video): 1)OrgCW=64 2) For i = 0:16, InputPivot[i] = i * OrgCW 3) For i = 0:16, MappedPivot[i] is calculated as follows: MappedPivot[0] = 0; for(i=0;i<16;i++) MappedPivot[i+1]=MappedPivot[i]+SignalledCW[i] where SignalledCW[i] is the number of codewords signaled for the i-th segment. like Fig.28 As shown, for the inter-frame coded block, motion compensation prediction is performed in the mapping domain. In other words, the motion compensation prediction block Y is calculated based on the reference signal in the DPB. pred Finally, the FwdMap function is applied to map the brightness prediction block in the original domain to the mapping domain, Y′ pred =FwdMap(Y pred). For intra-coded blocks, the FwdMap function is not applied because intra prediction is performed in the mapping domain. Calculate the reconstructed block Y r Finally, the InvMap function is applied to convert the reconstructed brightness value in the mapping domain back to the reconstructed brightness value in the original domain. The InvMap function is applicable to both intra-coded and inter-coded luma blocks. The brightness mapping process (forward and / or inverse mapping) can be implemented using a lookup table (LUT) or using on-the-fly calculations. If LUTs are used, then FwdMapLUT and InvMapLUT can be pre-calculated and pre-stored for use at the tile group level, and forward and inverse mapping can be simply implemented as FwdMap(Y pred )=FwdMapLUT[Y pred ] and InvMap(Y r )=InvMapLUT[Y r ]. Alternatively, immediate calculation can be used. Take the forward mapping function FwdMap as an example. To find out which fragment a brightness sample belongs to, the sample value is shifted right by 6 bits (equivalent to 16 equal fragments). Then, the linear model parameters of the fragment are retrieved and applied immediately to calculate the mapped brightness value. Let i be the fragment index, a1 and a2 be InputPivot[i] and InputPivot[i+1] respectively, and b1 and b2 be MappedPivot[i] and MappedPivot[i+1] respectively. The calculation of the FwdMap function is as follows: FwdMap(Y pred )=((b2-b1) / (a2-a1))*(Y pred -a1)+b1 The InvMap function can be calculated on the fly in a similar way. Typically, the fragments in the mapping domain are of unequal size, so the most straightforward inverse mapping process will require comparisons in order to find out which fragment the current sample value belongs to. This comparison increases the complexity of the decoder. Therefore, VVC imposes a bitstream constraint on the value of the output pivot point MappedPivot[i] as shown below. Assume that the range of the mapping domain (for 10-bit video, the range is [0,1023]) is divided into 32 equal fragments. If MappedPivot[i] is not a multiple of 32, then MappedPivot[i+1] and MappedPivot[i] cannot belong to the same fragment among the 32 fragments of the same size, that is, MappedPivot[i+1]>>(BitDepth Y -5) must not be equal to MappedPivot[i]>>(BitDepth YDue to this bitrate constraint, the InvMap function can also be performed using a simple right shift of 5 bits (corresponding to 32 equally sized fragments) to find out which fragment the sample value belongs to. 2.21.2 Luma-Dependent Chroma Residual Scaling Chroma residual scaling is intended to compensate for the interaction between luma signals and their corresponding chroma signals. Whether chroma residual scaling is enabled is also signaled at the slice level. If luma mapping is enabled, an additional flag is emitted to indicate whether luma-dependent chroma residual scaling is enabled. When luma mapping is not used, luma-dependent chroma residual scaling is disabled. Additionally, luma-dependent chroma residual scaling is always disabled for chroma blocks with an area less than or equal to 4. The chroma residual scaling depends on the average of the neighboring luma samples reconstructed on top and / or to the left of the current VPDU. If the current CU is Inter 128x128, Inter 128x64, and Inter 64x128, the chroma residual scaling factor derived for the CU associated with the first VPDU is used for all chroma transform blocks in that CU. Denote avgYR as the average of the neighboring luma samples reconstructed (see Fig.28 ). C ScaleInv The value of is calculated as follows: 1) Based on the InvMap function, find the index Y of the piecewise linear model to which avgYr belongs Idx . 2) C ScaleInv =cScaleInv[Y Idx ], where cScaleInv[] is a 16-segment LUT pre-computed based on the value of SignalledCW[i] and the offset value for the chroma residual scaling process signaled in APS. Unlike brightness mapping performed based on samples, C ScaleInv Is a constant for the entire chrominance block. ScaleInv , chroma residual scaling is applied as follows: Encoder side: C ResScale =C Res *C Scale =C Res / C SCcaleInv Decoder side: C Res =C ResScale / C Scale =C ResScale *C ScaleInv 2.21.3 Encoder-side LMCS parameter estimation A non-standard reference implementation is provided in the VTM encoder to estimate the LMCS model parameters. Because the VTM anchor handles SDR, HDR PQ and HDR HLG differently, the reference algorithm in VTM13 is designed differently for SDR sequences, HDR PQ sequences and HDR HLG sequences. For SDR sequences and HDR HLG sequences, the encoder algorithm is optimized based on local brightness changes and for the PSNR metric. For HDR PQ sequences, the encoder algorithm is optimized based on brightness values ​​and for the wPSNR (weighted PSNR) metric. 2.22. About pairwise merge candidates Fig.29 An example diagram 2900 showing example pairs in the merge candidate reordering ARMC-TM and additional pairwise candidates after reordering is shown. In this contribution, the pairwise candidate is removed from the conventional merge candidate derivation. When it is not a duplicate, such as Fig.29 As shown, it is added during adaptive reordering of merging candidates with template matching (ARMC-TM). Pairs are built using the two first reordered candidates. The number of reordered candidates is the same as in the ECM2.0 implementation, as well as the number of calculated template matching costs. When the reference frame of the first reordered candidate is the same as the reference frame of the second reordered candidate, the paired candidates are also restricted to using only the average candidate. Furthermore, if the created pair is not a duplicate, each merge candidate in the non-reordered subgroup is replaced by a pair between the first candidate and the candidate. 2.23. IBC Movement Candidates The following detailed embodiments should be considered as examples to explain the general concept. These embodiments should not be interpreted in a narrow manner. In addition, these embodiments can be combined in any way. The term "block" may refer to a codec tree block (CTB), a codec tree unit (CTU), a codec block (CB), a CU, a PU, a TU, a PB, a TB, or a video processing unit including multiple samples / pixels. A block may be rectangular or non-rectangular. For IBC-coded blocks, a block vector (BV) is used to indicate the displacement from the current block to a reference block that has been reconstructed inside the current picture. W and H are the width and height of the current block (eg, luma block). The non-adjacent spatial candidates of the current codec block are the adjacent spatial candidates of the virtual block in the i-th search round (such as Fig. 9As shown in FIG. 1 ). For the i-th search round, the width and height of the virtual block are calculated by the following formulas: newWidth = i×2×gridX+W, newHeight = i×2×gridY+H. Obviously, if the search round i is 0, the virtual block is the current block. In the following, BV predictor is also BV candidate. Skip mode is also merge mode. BV candidates can be divided into several groups according to some criteria. Each group is called a subgroup. For example, adjacent spatial and temporal BV candidates can also be used as the first subgroup, and the remaining BV candidates can be used as the second subgroup; in another example, the first N (N≥2) BV candidates can also be used as the first subgroup, the last M (M≥2) BV candidates can be used as the second subgroup, and the remaining BV candidates can be used as the third subgroup. About the use of BV candidates 1. If a BV candidate is invalid, the BV candidate (eg, BV search point or BV predictor) is not allowed to be used in the encoding / decoding process of a block. a. In one example, only if a BV candidate is valid, it can be used in the encoding / decoding process of a block. i. For example, only if a BV candidate is valid, it can be used for BV search or BV prediction. b. In one example, whether to use a BV candidate in the encoding / decoding process of a block may depend on a validation check of the BV candidate. i. In one example, before inserting a new BV candidate into the BV candidate list, a validation check of the BV candidate needs to be performed. c. Only if a BV candidate is valid can it be inserted into the IBC candidate list. i. In one example, the IBC candidate list may be an IBC merge candidate list. ii. In one example, the IBC candidate list may be an IBC AMVP candidate list. iii. In one example, the IBC candidate list may be an IBC template matching candidate list. iv. In one example, the IBC candidate list may be an intra template matching candidate list. v. In one example, the IBC candidate list may be an IBC MMVD candidate list. vi. In one example, the IBC candidate list may be an IBC GPM candidate list. vii. In one example, the IBC candidate list may be an IBC TPM candidate list. viii. In one example, the IBC candidate list may be any list related to IBC coded blocks, ie the same process is applied. (i) Alternatively, whether to allow invalid BV candidates to be added to the IBC candidate list may depend on decoded information (eg, IBC mode). d. Only if a BV candidate is valid, it can be used for hash-based search for IBC. e. Only if a BV candidate is valid, it can be used for block matching based local search for IBC. f. Only if the BV candidate is valid, it can be used for intra template matching. g. Alternatively, additionally, the above-mentioned BV candidates may be those from specific neighboring blocks (eg, adjacent or non-adjacent) or the HMVP table or some virtual candidates generated from these BV candidates. h. Alternatively, additionally, the above BV candidates may exclude some default candidates (eg, default zero vector). i. Alternatively, additionally, when a BV candidate is marked as invalid, a virtual candidate derived from the invalid BV candidate may be used instead. i. In one example, virtual candidates may be derived by adding offsets to invalid BV candidates. ii. In one example, virtual candidates may be derived by applying a pruning function to invalid BV candidates. Verification checks on BV candidates 2. In one example, a BV candidate may be determined to be valid when the BV candidate satisfies one or a combination of at least one of the following conditions. a. The corresponding reference block has been reconstructed inside the current image. b. The corresponding reference block is located in the same CTU row as the current block. c. The corresponding reference block is located in the same tile / sub-picture as the current block. d. The corresponding reference block is located in the same stripe as the current block. e. The BV candidate satisfies the block vector constraints (eg, which are described in 2.4.2 and 2.4.3). f. The BV candidate satisfies the IBC virtual buffer conditions (eg, which are described in 2.4.4). 3. In one example, when a BV candidate violates one or a combination of at least one of the conditions in item 2 above, the BV candidate may be determined to be invalid. About BV Candidate List 4. BV candidates can be derived / obtained from non-neighboring blocks. a. In one example, the distance between the non-adjacent spatial candidate and the current codec block can be based on the width and height of the current codec block (e.g., Fig. 9 or Fig.10 , gridX=W, gridY=H). i. Alternatively, the distance between the non-adjacent spatial candidates and the current coding block may be a multiple of a constant value. (i) For example, the multiplication factor depends on the search round index (eg, the multiplication factor is i for the i-th search round) and gridX=M, gridY=N (M and N are constant values). b. In one example, non-adjacent spatial candidates may be inserted after the TMVP candidate. i. Alternatively, non-adjacent spatial candidates may be inserted after adjacent spatial candidates and before TMVP candidates. 5. BV candidates can come from spatial neighboring blocks, temporal neighboring blocks, HMVP, paired, and / or STMVP candidates. Fig. 30A An example diagram 3000 showing candidate locations for spatial domain candidates is shown. Fig. 30B An example diagram 3050 showing candidate locations for time domain candidates is shown. a. In one example, a spatial candidate may consist of adjacent and / or non-adjacent spatial candidates. i. In one example, the adjacent spatial domain candidates may be from the left and / or above and / or upper right and / or lower left and / or Or the upper left spatial candidate composition (example in Fig. 30A ). b. In one example, Fig. 30B As shown, for TMVP candidates, the position for the temporal candidate is selected between candidates C0 and C1. If the CU at position C0 is unavailable, intra-coded, outside the current row of the CTU, or its BV is invalid for the current block, then position C1 is used. Otherwise, position C0 is used for the derivation of TMVP candidates. i. Or, for TMVP candidates, such as Fig. 30B Both candidates C0 and C1 depicted in can be used. (i) For example, the order is C0->C1. (ii) Alternatively, the order is C1->C0. c. In one example, for paired candidates, the paired average candidates are generated by averaging predefined pairs of candidates in the existing BV candidate list, and the predefined pairs are defined as {(0,1), (0,2), (1,2), (0,3), (1,3), (2,3)}, where the number represents the BV candidate index of the BV candidate list. i. In one example, the number of pairwise candidates is P. P is an integer from 0 to 6. ii. In one example, pairwise candidates may be inserted after HMVP. d. In one example, for the STMVP candidate, it is generated by averaging the predefined E spatial domain BV candidates and the predefined G temporal domain BV candidates. i. In one example, E is less than or equal to the number of spatial candidates (F) inserted into the current BV candidate list before STMVP. ii. In one example, the predefined E spatial domain BV candidates may be the first E spatial domain BV candidates among the F spatial domain candidates inserted into the current BV candidate list before STMVP. (i) Alternatively, the predefined E spatial BV candidates may be the selected E spatial BV candidates from among the F spatial candidates inserted into the current BV candidate list before STMVP. iii. In one example, E is 2 and G is 1. iv. In one example, the STMVP may be inserted before the top left spatial BV candidate. v. In one example, STMVP can be inserted after the pairwise candidates. e. In one example, the BV candidate insertion order is adjacent airspace -> HMVP -> pairs. f. In one example, the BV candidate insertion order is adjacent spatial domain -> temporal domain -> HMVP -> in pairs. g. In one example, the order of BV candidate insertion is adjacent spatial domain -> temporal domain -> non-adjacent spatial domain -> HMVP -> In pairs. h. In one example, the BV candidate insertion order is adjacent spatial domains -> non-adjacent spatial domains -> HMVP -> pairs. i. In one example, the BV candidate insertion order is adjacent spatial domain (STMVP is inserted before the upper left spatial domain BV candidate) -> temporal domain -> non-adjacent spatial domain -> HMVP -> paired. 6. The BV candidate list can also consist of pruned BV candidates. a. In one example, if a BV candidate is invalid in the sense of the third item above, it can be converted into a valid BV following the given rules and then inserted into the BV candidate list. i. In one example, a BV candidate may be converted to the nearest valid BV candidate. ii. In one example, a BV candidate may be converted to the closest valid BV candidate from a predefined set of BV candidates. b. In one example, if a non-zero BV candidate is invalid, it may be pruned to the nearest valid BV and then inserted into the BV candidate list. c. In one example, if a non-zero BV candidate is invalid, it may be pruned to the closest valid BV from a predefined BV candidate set and then inserted into the BV candidate list. i. In one example, the predefined BV candidate set can be composed of (-m*W, 0), (0, -n*H), (-m*H, 0), (0, -n*W). m and n are positive values. d. In one example, the cropped BV candidate may be inserted after the non-cropped BV candidate. 7. The BV candidate list can be used as the IBC merge / AMVP candidate list. a. Alternatively, the BV candidate list can be used to derive the IBC merge / AMVP candidate list. i. In one example, for the IBC merge mode, the first R entries of the BV candidate list will be used to construct the IBC merge candidate list; for the IBC AMVP mode, the first S entries of the BV candidate list will be used to construct the IBC AMVP candidate list. (i) In one example, R is 6 and S is 2. 8. In one example, sub-block based temporal block vector prediction (SbTMVP) may be supported as a BV candidate or BV prediction mode. Fig.31 An example diagram 3100 is shown demonstrating the derivation of a sub-CUbv motion field from a corresponding co-located sub-CU by applying a motion shift from a spatial neighbor. a. Similar to SbTMVP, SbTMVP uses the BV motion field in the same image to improve the current image for Block vector prediction and IBC merge mode for CU. The same co-located pictures used by TMVP are used for SbTMVP. b. In one example, SbTMVP applies motion shifting before extracting temporal BV information from the co-located picture, where the motion shifting is obtained from the motion vector of one of the spatial neighboring blocks of the current CU (example in Fig.31 ). i. In one example, if A1 has a motion vector that uses the co-located picture as its reference picture, This motion vector is then selected as the motion shift to be applied. If no such motion is identified, Then the motion shift is set to (0,0). ii. In one example, other spatial candidate positions (eg, A0, B0, B1, B2) may be used to derive motion shifts. (i) In one example, the checking order may be A1->B1->B0->A0->B2. (ii) In one example, the checking order may be B1->A1->B0->A0->B2. (iii) In one example, the checking order may be A0->A1->B0->B1->B2. c. In one example, after deriving the motion shift, for each sub-CU, the BV information of its corresponding block (the smallest motion grid covering the center sample) in the co-located picture is used to derive the BV information for the sub-CU ( Fig.31 Example in ) assuming that the motion shift is set to the motion of block A1). About re-ranking of BV candidate list 9. An initial BV candidate list may be derived first, followed by a reordering / refinement process. And the reordering / refinement list is utilized during the encoding / decoding process of the block. a. In one example, the BV candidate list may be an IBC merge candidate list. b. In one example, the BV candidate list may be an IBC AMVP candidate list. c. In one example, the BV candidate list may be an IBC template matching candidate list. d. In one example, the BV candidate list may be an intra template matching candidate list. e. In one example, the BV candidate list may be an IBC MMVD candidate list. f. In one example, the BV candidate list may be an IBC GPM candidate list. g. In one example, the BV candidate list may be an IBC TPM candidate list. h. Alternatively, the re-ranking / refinement process may not be applied to the IBC AMVP candidate list. i. Alternatively, the re-ranking / refinement process may not be applied to the IBC GPM candidate list. j. Alternatively, the re-ordering / refinement process may not be applied to the IBC TPM candidate list. k. In one example, the reordering / refining process may be applied to all types of IBC-coded blocks, ie, the same process is applied. i. Alternatively, whether to apply the reordering / refinement process may depend on decoding information (eg, IBC mode). (i) In one example, it is applied to IBC Merge / Skip mode, but not to IBC AMVP mode. 10. BV candidates can be adaptively rearranged in the final BV candidate list according to one or some criteria. a. In one example, part or all of the current BV candidate list building process is called first, followed by the reordering of the candidates in the list. i. Alternatively, the candidates in the first subgroup may be reordered and they should be added before those in the second subgroup, where the first subgroup is added before the second subgroup. b. In one example, all BV candidates in the BV candidate list may be reordered together (ie, no subgroups). i. Alternatively, some of the BV candidates in the BV candidate list may be reordered together (ie, without subgroups). (i) The BV candidates to be reordered may be selected based on codec information or signaling. c. In one example, the criteria may be based on template matching cost. i. In one example, the cost function between the current template and the reference template may be: (i) SAD / MR-SAD, (ii) SATD / MR-SATD, (iii) SSD / MR-SSD, (iv) SSE / MR-SSE, (v) Weighted SAD / Weighted MR-SAD, (vi) Weighted SATD / weighted MR-SATD, (vii) Weighted SSD / Weighted MR-SSD, (viii) Weighted SSE / weighted MR-SSE, (ix) Gradient information. ii. In one example, if LMCS is enabled, the current template and the reference template may consist of samples in the mapped domain. (i) Alternatively, the current template and the reference template may consist of samples in the original domain. iii. In one example, the BV candidates in each subgroup may be reordered in ascending order according to the cost value based on template matching. iv. In one example, if only the upper template is valid for the current block, the template matching reordering can only use the upper template. v. In one example, if only the left template is valid for the current block, the template matching reordering can only use the left template. vi. In one example, if both the above template and the left template are available for the current block, the template matching reordering may use the left template, the above template, or both the above and left templates. vii. In one example, the template matching process may include a component, such as brightness. (i) Alternatively, the template matching process may include multiple components, such as luminance and chrominance. a) In one example, the total template matching cost may be calculated as a weighted sum of the template matching costs on different color components. viii. In one example, the width of the left template and / or the height of the upper template may be fixed. (i) In one example, the width of the left template may be 1. (ii) In one example, the height of the upper template may be 1. ix. In one example, the BV used to locate the reference template can be cropped. Fig.33A An example diagram 3300 showing a reference template outside of the current picture is shown. Fig.33B An example diagram 3350 is shown demonstrating cropping the BV so that the reference template is inside the current picture. (i) In one example, if the reference template is outside the current image (e.g. Fig.33A As shown in the figure), the BV used to locate the reference template can be cropped so that the reference template is located inside the current picture. The cropping operation will not change the BV of the corresponding BV candidate. Fig.33B is shown in . x. In one example, if the reference template is outside the current image (such as Fig.33A As shown), the corresponding BV candidates can still be reordered. (i) In one example, if the reference template is outside the current picture, it can be filled by the nearest sample from within the current picture. xi. In one example, the reference template should have been reconstructed inside the current picture. (i) In one example, if the reference template is not reconstructed or is outside the current picture, the corresponding BV candidates may not be reordered. xii. In one example, the samples included in the reference template should be available (eg, reconstructed and within the same stripe / tile / IBC virtual buffer). (i) Alternatively, additionally, if some or all samples contained in the reference template are not available, the above method of processing a reference template outside the current picture may be applied. d. In one example, whether and / or how to reorder BV candidates may depend on the category of the BV candidates. e. In one example, the BV candidates to be reordered may be BV candidates in the final BV candidate list. i. Alternatively, the BV candidates to be reordered may be some / all of the adjacent spatial BV candidates, Even though it may not be included in the final BV candidate list. ii. Alternatively, the BV candidates to be reordered may be some / all of the non-adjacent spatial BV candidates, Even though it may not be included in the final BV candidate list. iii. Alternatively, the BV candidate to be reordered may be part / all of the HMVP BV candidate, even though it may not be included in the final BV candidate list. iv. Alternatively, the BV candidate to be re-ranked may be part / all of the pairwise average BV candidate, even though it may not be included in the final BV candidate list. v. Alternatively, the BV candidate to be reordered may be a partial / full STMVP BV candidate, even though it may not be included in the final BV candidate list. 11. The template shape can be adaptive. a. In one example, the template may include only neighboring samples to the left of the current block. b. In one example, the template may only include neighboring samples above the current block. c. In one example, the template shape is selected according to the CU shape. d. In one example, the width of the left template is selected according to the CU height. i. For example, if H<=M, the size of the left template is w1xH; otherwise, the size of the left template is w2xH. e. In one example, M, w1, and w2 are set equal to 8, 1, and 2, respectively. f. In one example, the height of the upper template is selected according to the CU width. i. For example, if W<=N, the size of the upper template is Wxh1; otherwise, the size of the upper template is Wxh2. (i) In one example, N, h1, and h2 are set equal to 8, 1, and 2, respectively. g. In one example, the width of the left template is selected according to the CU width. i. For example, if W<=N, the size of the left template is w1xH; otherwise, the size of the left template is w2xH. (i) In one example, N, w1, and w2 are set equal to 8, 1, and 2, respectively. h. In one example, the height of the upper template is selected according to the CU height. i. For example, if H<=M, the size of the upper template is Wxh1; otherwise, the size of the upper template is Wxh2. (i) In one example, M, h1, and h2 are set equal to 8, 1, and 2, respectively. i. In one example, samples of the template and reference samples of the template samples may be downsampled or down-sampled before being used to calculate the cost. i. Whether and / or how downsampling is performed may depend on the CU size. ii. In one example, no downsampling is performed on the short edge of the CU. 12. In one example, the costs disclosed in item 10 above may be derived for a first BV candidate, which may or may not be placed in the BV candidate list. a. In one example, whether to place the first BV candidate into the BV candidate list may depend on the cost derived for the first BV candidate. b. In one example, whether to place a first BV candidate into a BV candidate list may depend on a comparison between a first cost derived for the first BV candidate and a second cost derived for a second BV candidate, and the second BV candidate may or may not be placed into the BV candidate list. 13. In one example, for intra TMP, the L-shaped template can be replaced with the upper and left templates excluding the upper left portion. Fig.32 An example diagram 3200 illustrating example intra-frame template matching is shown. a. In one example, if only an upper template is available for the current block, the intra TMP can only use the upper template. b. In one example, if only the left template is available for the current block, the intra TMP can only use the left template. c. In one example, if both the above template and the left template are available for the current block, the intra TMP may use the left template, the above template, or both the above template and the left template. 2.24. Template matching based merge candidate list construction (TM-MCLC) In ECM, non-neighboring candidates and HMVP candidates are placed in the merged candidate list based on a predefined traversal order after neighboring candidates and TMVP candidates. With TM-MCLC, non-neighboring candidates and HMVP candidates are still placed in the merged candidate list after neighboring candidates and TMVP candidates, but in ascending order of template matching cost. Fig.34An example diagram 3400 showing the non-adjacent positions used is shown. More specifically, all available non-adjacent MVPs and HMVPs are collected in groups. The TM cost associated with each candidate in the group is then derived in a manner similar to ARMC. Subsequently, all candidates in the group are sorted in ascending order with respect to the corresponding TM cost. Finally, the non-adjacent candidates and HMVP candidates are placed in a merge candidate list in ascending order of template matching cost. In this proposal, if Fig.34 As shown, the non-adjacent MVP in the ECM software is further extended with more spatial positions and non-adjacent temporal positions. In addition to the 18 positions for non-adjacent spatial MVP in ECM-2.0, an additional 32 spatial positions and 12 non-adjacent temporal positions are introduced, where the non-adjacent temporal MVP positions are located in the same reference frame as the adjacent TMVP. To reduce complexity, integer precision is used for the TM merge mode, while a 2-tap bilinear filter is used to generate the reference template for the normal merge mode. 2.25. Modifications to IBC Merge / AMVP List Construction Fig.35 An example diagram 3500 showing spatial candidates used for IBC merge / AMVP candidate list is shown. Fig.36 An example diagram 3600 showing a presentation template and a reference sample of the template is shown. In this contribution, the IBC merge / AMVP list construction is modified with the following changes: 1) Only if the IBC merge / AMVP candidate is valid, it can be inserted into the IBC merge / AMVP candidate list. 2) Candidates for the upper right, lower left, and upper left airspaces (such as Fig.35 As shown) and a pairwise average candidate can be added to the IBC merge / AMVP candidate list. 3) ARMC-TM is extended to the IBC merge list, which is called adaptive reordering of merge candidates with template matching for IBC (ARMC-TM-IBC). The template and the reference sample of the template are in Fig.36 is shown in . 2.26. ARMC based on MV candidate types MV candidate type based ARMC is proposed to reorder the merge candidates by candidate type (e.g., TMVP, NA-MVP, etc.) based on TM cost value (same as ARMC TM cost used in ECM2.0). Then, when building the merge candidate list, M candidates are selected from the reordered candidates by candidate type, where M varies according to the candidate type (e.g., M=1 for TMVP type, M=9 for NA-MVP type). Some candidate types add more merge candidates to perform reordering and selection. Fig.37 An example diagram 3700 showing example locations used by TMVP is shown. Fig.37 The TMVP candidates derived from the different co-location positions shown enable the various prediction reference lists to be reordered together. More specifically, the evaluated TMVP candidates are as follows: TMVP0: List 0 - MVs from {RB, C}; List 1 - MVs from {RB, C}, TMVP1: List 0 - MVs from {RBR, RBL}; List 1 - MVs from {RBR, RBL}, TMVP2: List 0 - MVs from {RB, C} (without TMVP0 from List 1), TMVP3: List 1 - MVs from {RB, C} (without TMVP0 from List 0), TMVP4: List 0 - MVs from {RBR, RBL} (without TMVP1 from List 1), • TMVP5: List 1 - MVs from {RBR, RBL} (without TMVP1 for List 0). Where {Pos1, Pos2} indicates the order of MV availability check, and only the first available position is selected. Finally, 1 TMVP candidate is selected from these TMVP candidates based on TM cost value. Some candidate types may not be reordered by the proposed ARMC. 2.27. Adaptive Decoder-side Motion Vector Refinement (ADMVR) In ECM-2.0, a multi-pass decoder-side motion vector refinement (DMVR) method is applied in regular merge mode if the selected merge candidate satisfies the DMVR condition. In the first pass, bilateral matching (BM) is applied to the codec block. In the second pass, BM is applied to each 16x16 sub-block within the codec block. In the third pass, the MV in each 8x8 sub-block is refined by applying bidirectional optical flow (BDOF). The adaptive decoder-side motion vector refinement method consists of two new merge modes that are introduced to refine the MV in only one direction (L0 or L1) of the bidirectional prediction of the merge candidate that meets the DMVR condition. A multi-pass DMVR process is applied to refine the motion vector for the selected merge candidate, however, in the first pass (i.e., PU level) DMVR, MVD0 or MVD1 is set to zero. Similar to the conventional merge mode, the merge candidates for the proposed merge mode are derived from spatially adjacent coded blocks, TMVP, non-adjacent blocks, HMVP, and paired candidates. The difference is that only those that satisfy the DMVR condition are added to the candidate list. The same merge candidate list (i.e., ADMVR merge list) is used by both proposed merge modes, and the merge index is encoded and decoded in the conventional merge mode. 2.28. Affine model inheritance based on history parameters A History Parameter Table (HPT) is created. The entries of the HPT store affine parameter sets: a, b, c, and d, each of which is represented by a 16-bit signed integer. The entries in the HPT are classified by reference list and reference index. Five reference indexes are supported for each reference list in the HPT. Formulaically, the classification of the HPT (denoted as HPTCat) is calculated as: HPTCat(RefList, RefIdx) = 5×RefList+min(RefIdx, 4), where RefList and RefIdx represent the reference picture list (0 or 1) and the reference index, respectively. For each category, up to two entries can be stored, resulting in a total of 20 entries in the HPT. At the beginning of each CTU row, the number of entries for each category is initialized to zero. When decoding a CTU with a reference list RefList cur and RefIdx cur After the affine-encoded CU is decoded, the affine parameters are used to update the classification HPTCat (RefList cur ,RefIdx cur ) in the . Fig.38 An example diagram 3800 showing the generation of a HAPC is shown. The candidate (HAPC) based on the historical affine parameters is obtained from Fig.38 The affine parameter set in the neighboring 4x4 block denoted as A0, A1, B0, B1 or B2 in the HPT and the corresponding entry stored in the HPT is derived. The MV of the neighboring 4x4 block is used as the base MV. In a formal way, the MV of the current block at position (x, y) is calculated as: Where (mv hbase , mv v base ) represents the MV of the adjacent 4x4 block, (x base ,y base ) represents the center position of the neighboring 4x4 blocks. (x, y) can be the top left, top right, and bottom left corners of the current block to obtain the corner position MV (CPMV) for the current block, or it can be the center of the current block to obtain the normal MV for the current block. Fig.38 An example of how to derive HAPC from block A0 is shown. The affine parameters {a0, b0, c0, d0} are derived from the classification HPTIdx(RefList A0 , refIdx0 A0 ) is directly obtained. The affine parameters from the HPT (with the center position A0 as the base position and the MV of block A0 as the base MV) are used together to derive the CPMV for affine merge HAPC or affine AMVP HAPC. They can also be used to derive the MV located at the center of the current block as a regular merge candidate. HAPC can be placed in the sub-block based merge candidate list, the affine AMVP candidate list, or the regular merge candidate list. In response to the introduction of the new HAPC, the size of the sub-block based merge candidate list is increased from 5 to 9. 2.29. Non-adjacent spatial neighbors for affine merge mode Fig.39 An example diagram 3900 showing spatial neighbors for deriving affine merge candidates is shown. Similar to the enhanced conventional merge mode, this contribution proposes to use non-adjacent spatial neighbors (NSAM) for affine merge. The mode for obtaining non-adjacent spatial neighbors is Fig.39 As shown in . Similar to the existing non-adjacent conventional merge candidates, the distance between the non-adjacent spatial neighbors and the current coding block in NSAM is also defined based on the width and height of the current CU. Fig.39 The motion information of non-adjacent spatial neighbors in is utilized to generate additional inherited and constructed affine merge candidates. Specifically, for inherited candidates, the same derivation process of inherited affine merge candidates in VVC remains unchanged, except that CPMV is inherited from non-adjacent spatial neighbors. Non-adjacent spatial neighbors are checked based on their distance from the current block (i.e., from near to far). At a certain distance, only the first available neighbor (encoded in affine mode) from each side of the current block (e.g., left and above) is included for inherited candidate derivation. Fig.39 As shown by the red dashed arrows in (a), the order of checking the neighbors on the left and above is from bottom to top and from right to left, respectively. Fig.39As shown in (b), the position of a left and upper non-adjacent spatial neighbor is first determined independently; then, the position of the upper left neighbor can be determined accordingly, which can form a rectangular virtual block together with the left and upper non-adjacent neighbors. Fig.40 An example 4000 showing non-adjacent neighbors from constructed affine merge candidates is shown. Then, as Fig.40 As shown, motion information of three non-adjacent neighbors is used to form CPMVs at the upper left (A), upper right (B), and lower left (C) of the virtual block, which are finally projected to the current CU to generate corresponding constructed candidates. Non-adjacent spatial merge candidates are inserted into the affine merge candidate list in the following order: 1.SbTMVP candidate, if available, 2. Inherited from adjacent neighbors, 3. Inherited from non-adjacent neighbors, 4. Built from adjacent neighbors, 5. Constructed from non-adjacent neighbors, 6. Zero MV. Due to the inclusion of additional candidates generated by NSAM, the size of the affine merge candidate list is increased from 5 to 15 in this contribution. 2.30. Affine Motion Compensated Prediction In HEVC, only the translational motion model is applied to motion compensated prediction (MCP). In the real world, there are many kinds of motions, such as zooming in / out, rotation, perspective motion, and other irregular motions. In VVC, block-based affine transformation motion compensated prediction is applied. Fig.41 An affine motion model 4110 (4 parameters) based on an example control point and an affine motion model 4120 (6 parameters) based on an example control point are shown. Fig.41 As shown, the affine motion field of a block is described by the motion information of two control points (4 parameters) or three control point motion vectors (6 parameters). For the 4-parameter affine motion model, the motion vector at the sample position (x, y) in the block is derived as: For the 6-parameter affine motion model, the motion vector at a sample position (x, y) in the block is derived as: Where (mv 0x ,mv 0y ) is the motion vector of the upper left control point, (mv 1x ,mv 1y ) is the motion vector of the upper right control point, and (mv 2x ,mv 2y) is the motion vector of the lower left control point. Fig.42 An example diagram 4200 showing an example affine MVF for each sub-block is shown. To simplify motion compensated prediction, block-based affine transform prediction is applied. To derive the motion vector for each 4×4 luminance sub-block, the motion vector of the center sample of each sub-block is calculated according to the above equation (e.g. Fig.42 ) and rounded to 1 / 16 fractional precision. A motion compensated interpolation filter is then applied to generate a prediction for each subblock with a derived motion vector. The subblock size of the chrominance component is also set to 4×4. The MV of the 4×4 chrominance subblock is calculated as the average of the MV of the upper left luminance subblock and the MV of the lower right luminance subblock in the same 8×8 luminance region. Like translational motion inter prediction, there are two affine motion inter prediction modes: affine merge mode and affine AMVP mode. 2.30.1 Affine Merge Prediction AF_MERGE mode can be applied to CUs with width and height greater than or equal to 8. In this mode, the CPMV of the current CU is generated based on the motion information of the spatially neighboring CUs. There can be up to five CPMVP candidates, and the one to be used for the current CU is indicated by a signal transmission index. The following three types of CPVM candidates are used to form the affine merge candidate list: – Inherited affine merge candidates inferred from the CPMV of neighboring CUs; – The constructed affine merged candidate CPMVP derived using the translation MV of the neighboring CU; – Zero MV. In VVC, there are at most two inherited affine candidates, which are derived from the affine motion models of neighboring blocks, one from the left neighboring CU and one from the above neighboring CU. Fig.43 An example diagram 4300 showing example positions of inherited affine motion predictors is shown. Candidate blocks such as Fig.43 As shown. For the predictor on the left, the scan order is A0->A1, and for the predictor above, the scan order is B0->B1->B2. Only the first inherited candidate from each side is selected. No pruning check is performed between two inherited candidates. When a neighboring affine CU is identified, its control point motion vectors are used to derive the CPMVP candidate in the affine merge list of the current CU. Fig.44 An example diagram 4400 showing example control point motion vector inheritance is shown. Fig.44As shown, if the adjacent lower left block A is encoded and decoded in affine mode, the motion vectors v2, v3 and v4 of the upper left corner, upper right corner and lower left corner of the CU containing block A are obtained. When block A is encoded and decoded with a 4-parameter affine model, the two CPMVs of the current CU are calculated based on v2 and v3. When block A is encoded and decoded with a 6-parameter affine model, the three CPMVs of the current CU are calculated based on v2, v3 and v4. The constructed affine candidates are constructed by combining the neighboring translation motion information of each control point. Fig.45 An example diagram 4500 showing example positions of candidate positions for a constructed affine merge pattern is shown. The motion information for the control points is obtained from Fig.45 The specified spatial and temporal neighbors shown in are exported. CPMV k (k=1,2,3,4) represents the kth control point. For CPMV1, B2->B3->A2 blocks are checked and the MV of the first available block is used. For CPMV2, B1->B0 blocks are checked, and for CPMV3, A1->A0 blocks are checked. TMVP is used as CPMV4 if it is available. After obtaining the MVs of the four control points, an affine merge candidate is constructed based on the motion information. The following combinations of control point MVs are used to construct in order: {CPMV1, CPMV2, CPMV3}, {CPMV1, CPMV2, CPMV4}, {CPMV1, CPMV3, CPMV4}, {CPMV2, CPMV3, CPMV4}, {CPMV1, CPMV2}, {CPMV1, CPMV3}. The combination of 3 CPMVs constructs a 6-parameter affine merge candidate, and the combination of 2 CPMVs constructs a 4-parameter affine merge candidate. To avoid the motion scaling process, the relevant combination of control point MVs is discarded if the reference indexes of the control points are different. After the inherited affine merge candidates and the constructed affine merge candidates are checked, if the list is still not full, a zero MV is inserted at the end of the list. 2.30.2 Affine AMVP Prediction Affine AMVP mode can be applied to CUs with width and height greater than or equal to 16. A CU-level affine flag is signaled in the bitstream to indicate whether the affine AMVP mode is used, and then another flag is signaled to indicate whether it is 4-parameter affine or 6-parameter affine. In this mode, the difference between the CPMV of the current CU and its predicted sub-CPMVP is signaled in the bitstream. The affine AVMP candidate list size is 2 and is generated by using the following four types of CPVM candidates in order: – Inherited affine AMVP candidates inferred from the CPMV of neighboring CUs; – The constructed affine AMVP candidate CPMVP derived using the translation MV of the neighboring CU; – Translational MV from neighboring CU; – Zero MV. The order in which inherited affine AMVP candidates are checked is the same as that of inherited affine merge candidates. The only difference is that for AVMP candidates, only affine CUs with the same reference picture as in the current block are considered. When an inherited affine motion predictor is inserted into the candidate list, the pruning process is not applied. The AMVP candidates constructed from Fig.45 The specified spatial neighbors are derived as shown. The same check order as done in the affine merge candidate construction is used. In addition, the reference picture index of the neighboring blocks is also checked. The first block in the check order is used, which is inter-coded and has the same reference picture as in the current CU. There is only one. When the current CU is encoded and decoded using a 4-parameter affine mode and both mv0 and mv1 are available, they are added as a candidate in the affine AMVP list. When the current CU is encoded and decoded using a 6-parameter affine mode and all three CPMVs are available, they are added as a candidate in the affine AMVP list. Otherwise, the constructed AMVP candidate is set to unavailable. If the affine AMVP list candidate is still less than 2 after the valid inherited affine AMVP candidate and the constructed AMVP candidate are inserted, mn0, mn1 and mn2 will be added as translation MVs in order to predict all control point MVs of the current CU when available. Finally, if the affine AMVP list is still not full, the list is filled with zero MVs. 2.30.3. Affine motion information storage In VVC, the CPMV of an affine CU is stored in a separate buffer. The stored CPMV is only used to generate the inherited CPMV in affine merge mode and the inherited CPMV in affine AMVP mode for the most recently encoded CU. The sub-block MV derived from the CPMV is used for motion compensation, merging of translation MVs / MV derivation of AMVP lists, and deblocking. To avoid picture line buffering for additional CPMV, affine motion data inheritance from CUs above the CTU is handled differently than inheritance from normal neighboring CUs. If the candidate CU for affine motion data inheritance is above a CTU line, the bottom left and bottom right sub-block MVs in the line buffer are used for affine MVP derivation instead of CPMVs. In this way, CPMVs are only stored in the local buffer. If the candidate CU is 6-parameter affine coded, the affine model is downgraded to a 4-parameter model. Fig.46 An example diagram 4600 illustrating the use of motion vectors for the proposed combination method is shown. Fig.46 As shown, along the top CTU boundary, the bottom left and bottom right sub-block motion vectors of the CU are used for affine inheritance of the CU in the bottom CTU. 2.30.4 Prediction Refinement Using Optical Flow for Affine Mode Compared with pixel-based motion compensation, sub-block based affine motion compensation can save memory access bandwidth and reduce computational complexity, but at the expense of prediction accuracy. In order to achieve finer motion compensation granularity, prediction refinement using optical flow (PROF) is used to refine sub-block based affine motion compensation predictions without increasing the memory access bandwidth for motion compensation. In VVC, after sub-block based affine motion compensation is performed, the brightness prediction samples are refined by adding the differences derived from the optical flow equation. PROF is described as the following four steps: Step 1) Sub-block based affine motion compensation is performed to generate a sub-block prediction I(i,j). Step 2) Using a 3-tap filter [-1, 0, 1], the spatial gradient g of the sub-block prediction x (i,j) and g y (i,j) is calculated at each sample position. The gradient calculation is exactly the same as the gradient calculation in BDOF. g x (i,j)=(I(i+1,j)>>shift1)-(I(i-1,j)>>shift1) g y (i,j)=(I(i,j+1)>>shift1)-(I(i,j-1)>>shift1) Shift1 is used to control the precision of the gradient. The sub-block (i.e. 4x4) prediction is extended by one sample on each side of the gradient calculation. To avoid extra memory bandwidth and extra interpolation calculations, those extended samples on the extended boundary are copied from the nearest integer pixel position in the reference picture. Step 3) The brightness prediction refinement is calculated by the following optical flow equation. ΔI(i,j)=g x (i,j)*Δv x (i,j)+g y (i,j)*Δv y (i,j) Among them Fig.47 As shown, Δv(i,j) is the difference between the sample MV (denoted as v(i,j)) calculated for the sample position (i,j) and the sub-block MV of the sub-block to which the sample (i,j) belongs. Fig.47 An example diagram 4700 showing an example sub-block MV VSB and pixels Δv(i,j) (arrows) is shown. Δv(i,j) is quantized in units of 1 / 32 luma sample precision. Since the affine model parameters and the sample positions relative to the sub-block center do not change from sub-block to sub-block, Δv(i,j) can be calculated for the first sub-block and reused for other sub-blocks in the same CU. Let dx(i,j) and dy(i,j) be the distance from the sample position (i,j) to the sub-block center (x SB ,y SB )’s horizontal and vertical offsets, Δv(x,y) can be derived from the following equations: To maintain accuracy, the sub-block (x SB ,y SB ) is calculated as ((W SB -1) / 2,(H SB -1) / 2), where W SB and H SB are the width and height of the sub-block respectively. For the 4-parameter affine model, For the 6-parameter affine model, Where (v 0x ,v 0y ), (v 1x ,v 1y ), (v 2x ,v 2y ) are the control point motion vectors of the upper left, upper right and lower left, and w and h are the width and height of the CU. Step 4) Finally, the luma prediction refinement ΔI(i,j) is added to the sub-block prediction I(i,j). The final prediction I' is generated as follows. I′(i,j)=I(i,j)+ΔI(i,j) PROF is not applicable to affine codec CUs in two cases: 1) all control point MVs are the same, which indicates that the CU has only translational motion; 2) the affine motion parameters are larger than the specified limit, because the sub-block based affine MC is downgraded to CU based MC to avoid large memory access bandwidth requirements. A fast codec method is applied to reduce the codec complexity of affine motion estimation using PROF. PROF is not applied to the affine motion estimation stage in the following two cases: a) If the CU is not a root block and the parent block of the CU does not select the affine mode as its best mode, PROF is not applied because the possibility that the current CU selects the affine mode as the best mode is low; b) If the magnitudes of the four affine parameters (C, D, E, F) are all less than a predefined threshold and the current picture is not a low-latency picture, PROF is not applied because the improvement introduced by PROF for this case is small. In this way, affine motion estimation using PROF can be accelerated. 2.31. Construction of motion candidate list The following detailed embodiments should be considered as examples to explain the general concept. These embodiments should not be interpreted in a narrow manner. In addition, these embodiments can be combined in any way. The term "block" may refer to a codec tree block (CTB), a codec tree unit (CTU), a codec block (CB), a CU, a PU, a TU, a PB, a TB, or a video processing unit including multiple samples / pixels. A block may be rectangular or non-rectangular. For IBC-coded blocks, a block vector (BV) is used to indicate the displacement from the current block to a reference block that has been reconstructed inside the current picture. W and H are the width and height of the current block (eg, luma block). The non-adjacent spatial candidates of the current codec block are the adjacent spatial candidates of the virtual block in the i-th search round (such as Fig. 9 As shown in FIG. 1 ). For the i-th search round, the width and height of the virtual block are calculated by the following formulas: newWidth = i×2×gridX+W, newHeight = i×2×gridY+H. Obviously, if the search round i is 0, the virtual block is the current block. In the following, BV predictor is also BV candidate. Skip mode is also merge mode. According to some criteria, motion candidates can be divided into several groups. Each group is called a subgroup. For example, adjacent spatial and temporal motion candidates can be used as the first subgroup, and the remaining motion candidates can be used as the second subgroup. In another example, the first N (N≥2) motion candidates can also be regarded as the first subgroup, the subsequent M (M≥2) motion candidates can be regarded as the second subgroup, and the remaining motion candidates can be regarded as the third subgroup. For example, motion candidates can be divided into several groups according to motion candidate types. A BV candidate may be determined to be valid when it satisfies one or a combination of at least one of the following conditions. 1) The corresponding reference block has been reconstructed inside the current image. 2) The corresponding reference block is located in the same CTU row as the current block. 3) The corresponding reference block is located in the same tile / sub-picture as the current block. 4) The corresponding reference block is located in the same slice as the current block. 5) The BV candidate satisfies the block vector constraints (eg, as described in 2.4.2 and 2.4.3). 6) The BV candidate satisfies the IBC virtual buffer conditions (e.g., described in 2.4.4). When a BV candidate violates one or a combination of at least one of the above conditions, the BV candidate may be determined to be invalid. If the CU at the temporal position is not available or intra-coded or outside the current row of the CTU or its BV (if any) is invalid for the current block, the temporal position is considered invalid; otherwise, the temporal position is considered valid. Note that the proposed method can be applied to the merge candidate list construction process for inter-frame codec blocks (e.g., translational motion), affine codec blocks, TM codec blocks, GPM codec blocks, ADMVR codec blocks, or IBC codec blocks; or other motion candidate list construction processes (e.g., ordinary AMVP list; affine AMVP list; TM AMVP list; IBC AMVP list; HMVP table). Note that the proposed method can be applied to the regular merge / AMVP candidate list after the DMVR / multi-pass DMVR process. Note that the proposed method can be applied to the ADMVR merge candidate list after the DMVR / multi-pass DMVR process. Note that the proposed method can be applied to TM merging / AMVP candidate list after block-based bilateral matching refinement and / or template matching refinement and / or sub-block-based bilateral matching refinement. In the following, a motion candidate is a repetition of another motion candidate and can be defined as: 1) Two motion candidates are identical in reference direction, reference picture index, affine model (for affine motion) and motion vector or block vector or part or all CPMV depending on the motion type (inter or IBC or affine). or 2) The two motion candidates are exactly the same in reference direction, reference picture index, affine model (for affine motion), but the difference of motion vector or block vector or part or all CPMV of the two motion candidates according to the motion type (inter frame or IBC or affine) can be within the threshold. The difference of the motion vector may be defined as the absolute horizontal component and / or vertical component of the difference of the motion vectors of two motion candidates in reference list 0 and / or reference list 1. The difference of the block vector may be defined as the absolute horizontal component and / or vertical component of the difference of the block vectors of the two motion candidates in reference list 0. A CPMV difference may be defined as an absolute horizontal component and / or vertical component of the motion vector difference of the CPMV of two motion candidates in reference list 0 and / or reference list 1. The threshold can be adaptively set according to the coding mode of the current block (e.g., TM merging, affine merging, conventional merging, etc.) and / or the current block size and / or the candidate type (e.g., adjacent spatial domain, adjacent temporal domain, non-adjacent spatial domain, non-adjacent temporal domain, HMVP, paired or STMVP, etc.) and / or QP and / or the reference index or reference POC of the current block and / or other coding information of the current block. About pairwise average motion candidates 1. In one example, a pairwise average motion candidate may be generated by averaging a predefined pair of reordered candidates in an existing motion candidate list. a. In one example, the re-ranking process may be an adaptive re-ranking of motion candidates with template matching. b. In one example, the re-ranking process may be an adaptive re-ranking of motion candidates with bilateral matching. c. In one example, the predefined pairs can be {(0,1), (0,2), (1,2), (0,3), (1,3), (2,3)}, where the number represents the motion candidate index in the motion candidate list. d. In one example, before adding a pair-wise average motion candidate to the motion candidate list, some or all of the following conditions need to be met. i. Pairwise mean motion (eg, BV) candidates are valid. ii. The pairwise average motion candidate is not a duplicate of any previous motion candidate in the existing motion candidate list. iii. A pruning method for pairwise average motion candidates with adaptive thresholding can be applied. (i) In one example, the threshold may be based on the current block size. (ii) In one example, the threshold may be based on the codec mode of the current block. (iii) In one example, the threshold may be based on QP. (iv) In one example, the threshold may be based on a combination of one or at least one of the above conditions. (v) In one example, the threshold may be based on a reference index or a reference POC of the current block. e. In one example, the number of pairwise average motion candidates may be predefined. i. In one example, the number of pairwise average motion candidates may be N (eg, N=1). ii. In one example, the number of pairwise average motion candidates can be calculated by subtracting the number of existing motion candidates from the motion candidate list size before adding the pairwise average motion candidates. That is, there is no constraint on the number of pairwise average motion candidates, so pairwise average motion candidates can be added until the motion candidate list is full. f. In one example, some or all of the pairwise average motion candidates may be reordered. i. Alternatively, the pairwise average motion candidates may not be reordered. ii. In one example, the pair-wise average motion candidates may be reordered with part or all of the previous motion candidates in the same motion candidate list. g. In one example, unpaired motion candidates may be replaced by paired motion candidates derived from a motion candidate list. i. In one example, if the partial motion candidates are not reordered (i.e., a non-reordered subgroup), each motion candidate in the non-reordered subgroup may be replaced by a paired motion candidate constructed by the first reordered candidate and the candidate, if the created pair is not a duplicate. ii. In one example, if the partial motion candidates are not reordered (i.e., a non-reordered subgroup), the partial motion candidates in the non-reordered subgroup may be replaced by a paired motion candidate constructed by the first reordered candidate and the original candidate, if the created pair is not a duplicate. iii. In one example, if the partial motion candidates are not reordered (i.e., a non-reordered subgroup), in the last reordered position, a paired motion candidate may be constructed using the first two reordered motion candidates, and if the paired motion candidate is not a duplicate and / or valid, it is added to the last reordered position of the motion candidate list, and the last motion candidate in the motion candidate list is removed from the list. (i) In one example, pairwise average motion candidates may not be reordered. (ii) In one example, the pair-wise average motion candidates may be reordered together with some or all of the previous motion candidates in the reordered subgroup. h. In one example, the motion candidate list may be: i. In one example, the motion candidate list may be a TM merge / AMVP list after block-based bilateral matching refinement and / or template matching refinement and / or sub-block-based bilateral matching refinement. ii. In one example, the motion candidate list may be a regular merge / AMVP list after DMVR / multi-pass DMVR process. iii. In one example, the motion candidate list may be an ADMVR merged list after the DMVR / multi-pass DMVR process. iv. In one example, the motion candidate list can be the GPM after the template matching refinement process A merged list (for example, AGPMList, LGPMList, or LAGPMList). v. In one example, the motion candidate list may be a regular merge / AMVP list. vi. In one example, the motion candidate list may be a TM merge / AMVP list. vii. In one example, the motion candidate list may be an affine merge / AMVP list. viii. In one example, the motion candidate list may be an IBC merge / AMVP list. ix. In one example, the motion candidate list may be an ADMVR merge list. x. In one example, the motion candidate list may be a GPM merge list. xi. In one example, the motion candidate list may be a TPM merge list. xii. In one example, the motion candidate list may be an MMVD merge list. xiii. In one example, the motion candidate list may be an IBC template matching candidate list. xiv. In one example, the motion candidate list may be an intra template matching candidate list. xv. In one example, the motion candidate list may be an IBC MMVD candidate list. xvi. In one example, the motion candidate list may be an IBC GPM candidate list. xvii. In one example, the motion candidate list may be an IBC TPM candidate list. xviii. In one example, the motion candidate list may be any other motion candidate list. About the Sports Candidate List 2. The motion candidate list may consist of spatial candidates, and / or temporal candidates, and / or HMVP candidates, and / or paired candidates, and / or STMVP candidates. a. In one example, a spatial candidate may consist of adjacent and / or non-adjacent spatial candidates. i. In one example, the adjacent spatial domain candidates may be composed of spatial domain candidates on the left and / or above and / or upper right and / or lower left and / or upper left (example in Fig. 30A ). ii. In one example, the distance between the non-neighboring spatial candidate and the current codec block can be based on the width and height of the current codec block (e.g., Fig. 9 , Fig.10 or Fig.34 , gridX=W,gridY =H). (i) Alternatively, the distance between the non-adjacent spatial candidates and the current codec block may be a multiple of a constant value. a) For example, the multiplication factor depends on the search round index (e.g., the multiplication factor is i for the i-th search round) and gridX=M, gridY=N (M and N are constant values). b. In one example, the time domain candidates may consist of adjacent and / or non-adjacent time domain candidates. i. In one example, for adjacent time domain candidates, the position of the adjacent time domain candidates can be Fig.37 RB and / or C and / or RBR and / or RBL are selected. (i) In one example, four adjacent time domain candidates (eg, RB and C and RBR and RBL) may be used. a) In one example, the order may be RB->C->RBR->RBL. (ii) In one example, two adjacent time-domain candidates may be used. a) In one example, one neighboring time domain candidate may be selected between RB and C. If RB is invalid, C is used; otherwise, RB is used. And another adjacent time domain candidate can be selected between RBR and RBL. If RBR is invalid, RBL is used; otherwise, RBR is used. (iii) In one example, N adjacent time-domain candidates may be used. a) In one example, the first N valid positions can be used to derive N adjacent time-domain candidates. b) In one example, N may be an integer from 0 to 4. (iv) In one example, a neighboring time domain candidate may be selected between RB and C. If RB is invalid, C is used; otherwise, RB is used. (v) In one example, a neighboring temporal candidate may be selected between RBR and RBL. If RBR is invalid, RBL will be used; otherwise, RBR will be used. ii. In one example, the distance between the non-adjacent time domain candidate and the current codec block may be based on the width and height of the current codec block (eg, Fig.34 , gridX=W, gridY=H). (i) Alternatively, the distance between the non-adjacent time domain candidates and the current coding block may be a multiple of a constant value. a) For example, the multiplication factor depends on the search round index (e.g., the multiplication factor is i for the i-th search round) and gridX=M, gridY=N (M and N are constant values). c. In one example, for paired candidates, it can be determined by pre-determining the candidate in the existing motion candidate list. It is generated by averaging the pairs of meanings. i. In one example, the predefined pairs may be defined as {(0,1), (0,2), (1,2), (0,3), (1,3), (2,3)}, where the number represents the motion candidate index in the motion candidate list. ii. In one example, the number of pairwise average motion candidates may be predefined. (i) In one example, the number of pairwise average motion candidates may be N (eg, N=1). (ii) In one example, the number of pairwise average motion candidates may be calculated by subtracting the number of existing motion candidates from the motion candidate list size before adding the pairwise average motion candidates. That is, the pairwise average motion candidates may be added until the motion candidate list is full. iii. In one example, pairwise candidates may be inserted after HMVP. iv. In one example, for affine pairwise candidates, an averaging operation may be performed for each CPMV. (i) In one example, if two affine candidates from which an affine pair candidate is derived have different affine models, the combination may be discarded. (ii) In one example, if the two affine candidates from which the affine pair candidates are derived have different affine models, the 6-parameter affine model can be downgraded to a 4-parameter affine model. d. In one example, for the STMVP candidate, it can be obtained by performing a predefined E spatial motion candidate and The predefined G temporal motion candidates are generated by averaging. i. In one example, E may be less than or equal to the number of spatial candidates (F) inserted into the current motion candidate list before STMVP. ii. In one example, the predefined E spatial motion candidates may be the first E spatial motion candidates among the F spatial candidates inserted into the current motion candidate list before STMVP. (i) Alternatively, the predefined E spatial motion candidates may be E spatial motion candidates selected from among the F spatial candidates inserted into the current motion candidate list before STMVP. iii. In one example, E is 2 and G is 1. iv. In one example, the STMVP may be inserted before the top left spatial motion candidate. v. In one example, STMVP can be inserted after the pairwise candidates. vi.STMVP candidates can be involved in the re-ranking process. vii. Alternatively, STMVP candidates may not be involved in the re-ordering process. e. In one example, motion candidates may include i. In one example, motion candidates may consist of adjacent spatial domains and / or HMVPs and / or pairs. ii. In one example, motion candidates may consist of adjacent spatial domains and / or adjacent temporal domains and / or HMVPs and / or pairs. iii. In one example, motion candidates may consist of adjacent spatial domains and / or adjacent temporal domains and / or non-adjacent spatial domains and / or HMVPs and / or pairs. iv. In one example, the motion candidates may consist of adjacent spatial domains and / or adjacent temporal domains and / or non-adjacent spatial domains and / or non-adjacent temporal domains and / or HMVPs and / or pairs. v. In one example, motion candidates may be composed of adjacent spatial domains and / or non-adjacent spatial domains and / or HMVP and / or in pairs. vi. In one example, the motion candidates may consist of adjacent spatial (STMVP may be inserted before the upper left spatial motion candidate) and / or adjacent temporal and / or non-adjacent spatial and / or HMVP and / or in pairs. vii. In one example, the motion candidates may consist of adjacent spatial (STMVP may be inserted before the upper left spatial motion candidate) and / or adjacent temporal and / or non-adjacent spatial and / or non-adjacent temporal and / or HMVP and / or in pairs. viii. In one example, motion candidates may be inserted sequentially according to motion candidate types. ix. In one example, a motion candidate may be inserted into a motion candidate list only if it is not a duplicate of any previous motion candidate in an existing motion candidate list. (i) Alternatively, a motion candidate may be inserted into a motion candidate list only if it is not a duplicate of some previous motion candidates in the existing motion candidate list. x. In one example, any other motion candidate type combination or any other motion candidate (type) Insertion order is also supported. f. In one example, the motion candidate list may be: i. In one example, the motion candidate list may be a TM merge / AMVP list after block-based bilateral matching refinement and / or template matching refinement and / or sub-block-based bilateral matching refinement. ii. In one example, the motion candidate list may be a regular merge / AMVP list after DMVR / multi-pass DMVR process. iii. In one example, the motion candidate list may be an ADMVR merged list after the DMVR / multi-pass DMVR process. iv. In one example, the motion candidate list may be a GPM merge list (eg, AGPMList, LGPMList, or LAGPMList) after a template matching refinement process. v. In one example, the motion candidate list may be a regular merge / AMVP list. vi. In one example, the motion candidate list may be a TM merge / AMVP list. vii. In one example, the motion candidate list may be an affine merge / AMVP list. viii. In one example, the motion candidate list may be an IBC merge / AMVP list. ix. In one example, the motion candidate list may be an ADMVR merge list. x. In one example, the motion candidate list may be a GPM merge list. xi. In one example, the motion candidate list may be a TPM merge list. xii. In one example, the motion candidate list may be an MMVD merge list. xiii. In one example, the motion candidate list may be an IBC template matching candidate list. xiv. In one example, the motion candidate list may be an intra template matching candidate list. xv. In one example, the motion candidate list may be an IBC MMVD candidate list. xvi. In one example, the motion candidate list may be an IBC GPM candidate list. xvii. In one example, the motion candidate list may be an IBC TPM candidate list. xviii. In one example, the motion candidate list may be any other motion candidate list. 2.32. Reordering of motion candidate list The following detailed embodiments should be considered as examples to explain the general concept. These embodiments should not be interpreted in a narrow manner. In addition, these embodiments can be combined in any way. The term "block" may refer to a codec tree block (CTB), a codec tree unit (CTU), a codec block (CB), a CU, a PU, a TU, a PB, a TB, or a video processing unit including multiple samples / pixels. A block may be rectangular or non-rectangular. For IBC-coded blocks, a block vector (BV) is used to indicate the displacement from the current block to a reference block that has been reconstructed inside the current picture. W and H are the width and height of the current block (eg, luma block). The non-adjacent spatial candidates of the current codec block are the adjacent spatial candidates of the virtual block in the i-th search round (such as Fig. 9 As shown in FIG. 1 ). For the i-th search round, the width and height of the virtual block are calculated by the following formulas: newWidth = i×2×gridX+W, newHeight = i×2×gridY+H. Obviously, if the search round i is 0, the virtual block is the current block. In the following, BV predictor is also BV candidate. Skip mode is also merge mode. According to some criteria, motion candidates can be divided into several groups. Each group is called a subgroup. For example, adjacent spatial and temporal motion candidates can be used as the first subgroup, and the remaining motion candidates can be used as the second subgroup. In another example, the first N (N≥2) motion candidates can also be regarded as the first subgroup, the subsequent M (M≥2) motion candidates can be regarded as the second subgroup, and the remaining motion candidates can be regarded as the third subgroup. For example, motion candidates can be divided into several groups according to motion candidate types. A BV candidate may be determined to be valid when it satisfies one or a combination of at least one of the following conditions. 1) The corresponding reference block has been reconstructed inside the current image. 2) The corresponding reference block is located in the same CTU row as the current block. 3) The corresponding reference block is located in the same tile / sub-picture as the current block. 4) The corresponding reference block is located in the same slice as the current block. 5) The BV candidate satisfies the block vector constraints (eg, as described in 2.4.2 and 2.4.3). 6) The BV candidate satisfies the IBC virtual buffer conditions (e.g., described in 2.4.4). When a BV candidate violates one or a combination of at least one of the above conditions, the BV candidate may be determined to be invalid. If the CU at the temporal position is not available or intra-coded or outside the current row of the CTU or its BV (if any) is invalid for the current block, the temporal position is considered invalid; otherwise, the temporal position is considered valid. Note that the proposed method can be applied to the merge candidate list construction process for inter-frame codec blocks (e.g., translational motion), affine codec blocks, TM codec blocks, GPM codec blocks, ADMVR codec blocks, or IBC codec blocks; or other motion candidate list construction processes (e.g., ordinary AMVP list; affine AMVP list; TM AMVP list; IBC AMVP list; HMVP table). Note that the proposed method can be applied to the regular merge / AMVP candidate list after the DMVR / multi-pass DMVR process. Note that the proposed method can be applied to the ADMVR merge candidate list after the DMVR / multi-pass DMVR process. Note that the proposed method can be applied to TM merging / AMVP candidate list after block-based bilateral matching refinement and / or template matching refinement and / or sub-block-based bilateral matching refinement. In the following, a motion candidate is a repetition of another motion candidate and can be defined as: 1) Two motion candidates are identical in reference direction, reference picture index, affine model (for affine motion) and motion vector or block vector or part or all CPMV depending on the motion type (inter or IBC or affine). or 2) The two motion candidates are exactly the same in reference direction, reference picture index, affine model (for affine motion), but the difference of motion vector or block vector or part or all CPMV of the two motion candidates according to the motion type (inter frame or IBC or affine) can be within the threshold. The difference of the motion vector may be defined as the absolute horizontal component and / or vertical component of the difference of the motion vectors of two motion candidates in reference list 0 and / or reference list 1. The difference of the block vector may be defined as the absolute horizontal component and / or vertical component of the difference of the block vectors of the two motion candidates in reference list 0. A CPMV difference may be defined as an absolute horizontal component and / or vertical component of the motion vector difference of the CPMV of two motion candidates in reference list 0 and / or reference list 1. The threshold can be adaptively set according to the coding mode of the current block (e.g., TM merging, affine merging, conventional merging, etc.) and / or the current block size and / or the candidate type (e.g., adjacent spatial domain, adjacent temporal domain, non-adjacent spatial domain, non-adjacent temporal domain, HMVP, paired or STMVP, etc.) and / or QP and / or the reference index or reference POC of the current block and / or other coding information of the current block. About reordering of motion candidate list 1. In one example, when building a motion candidate list, a re-ranking / refinement process may be performed. a. In one example, ARMC based on motion candidate type can be used to reorder motion candidates within a candidate type according to one or some criteria. i. In one example, when constructing a motion candidate list, M candidates of a specific candidate type may be selected from the reordered candidates in the candidate type, where M may vary according to the candidate type and / or the coding mode of the current block. ii. In one example, the candidate type may be an adjacent spatial domain candidate. For example, M is 4. iii. In one example, the candidate type may be an adjacent time domain candidate. For example, M is 1. iv. In one example, the candidate type may be a non-adjacent spatial candidate. For example, M is 3. v. In one example, the candidate type may be a non-adjacent time domain candidate. For example, M is 1. vi. In one example, the candidate type may be an HMVP candidate. For example, M is 2. vii. In one example, the candidate type can be a pairwise average candidate. For example, M is 2. viii. In one example, the candidate type may be a STMVP candidate. For example, M is 1. ix. In one example, the candidate type may be an adjacent inherited affine candidate. x. In one example, the candidate type may be a neighboring constructed affine candidate. xi. In one example, the candidate type may be a non-adjacent inherited affine candidate. xii. In one example, the candidate type may be a non-adjacent constructed affine candidate. xiii. In one example, the candidate type may be a history-based affine candidate. xiv. In one example, multiple motion candidate types (ie, candidate type combinations) may be reordered together. (i) In one example, when constructing a motion candidate list, M candidates of any particular motion candidate type may be selected from the reordered candidates in the candidate type combination, where M may vary depending on the candidate type combination and / or the codec mode of the current block. (ii) In one example, adjacent spatial candidates and / or HMVP candidates and / or pairwise average candidates may be reordered together. (iii) In one example, adjacent spatial domain candidates and adjacent temporal domain candidates may be reordered together. (iv) In one example, non-adjacent spatial domain candidates and non-adjacent temporal domain candidates may be reordered together. (v) In one example, non-adjacent spatial candidates and / or non-adjacent temporal candidates and / or HMVP Candidates can be reordered together. (vi) In one example, non-adjacent spatial candidates and / or non-adjacent temporal candidates and / or HMVP Candidates and / or pair-wise average candidates may be reordered together. (vii) In one example, adjacent inherited affine candidates and adjacent constructed affine candidates may be reordered together. (viii) In one example, non-adjacent inherited affine candidates and non-adjacent constructed affine candidates may be reordered together. (ix) In one example, non-adjacent inherited affine candidates and / or non-adjacent constructed affine candidates and / or HMVP affine candidates may be reordered together. (x) In one example, non-adjacent inherited affine candidates and / or non-adjacent constructed affine candidates and / or HMVP affine candidates and / or pairwise average affine candidates may be reordered together. (xi) In one example, adjacent inherited affine candidates and non-adjacent inherited affine candidates may be reordered together. (xii) In one example, adjacent constructed affine candidates and non-adjacent constructed affine candidates may be reordered together. b. In one example, the motion candidate list may be: i. In one example, the motion candidate list may be a TM merge / AMVP list after block-based bilateral matching refinement and / or template matching refinement and / or sub-block-based bilateral matching refinement. ii. In one example, the motion candidate list may be a regular merge / AMVP list after DMVR / multi-pass DMVR process. iii. In one example, the motion candidate list may be an ADMVR merged list after the DMVR / multi-pass DMVR process. iv. In one example, the motion candidate list may be a GPM merge list (eg, AGPMList, LGPMList, or LAGPMList) after a template matching refinement process. v. In one example, the motion candidate list may be a regular merge / AMVP list. vi. In one example, the motion candidate list may be a TM merge / AMVP list. vii. In one example, the motion candidate list may be an affine merge / AMVP list. viii. In one example, the motion candidate list may be an IBC merge / AMVP list. ix. In one example, the motion candidate list may be an ADMVR merge list. x. In one example, the motion candidate list may be a GPM merge list. xi. In one example, the motion candidate list may be a TPM merge list. xii. In one example, the motion candidate list may be an MMVD merge list. xiii. In one example, the motion candidate list may be an IBC template matching candidate list. xiv. In one example, the motion candidate list may be an intra template matching candidate list. xv. In one example, the motion candidate list may be an IBC MMVD candidate list. xvi. In one example, the motion candidate list may be an IBC GPM candidate list. xvii. In one example, the motion candidate list may be an IBC TPM candidate list. xviii. In one example, the motion candidate list may be any other motion candidate list. 2. In one example, an initial motion candidate list may be derived first, and then a reordering / refining process is performed according to one or some criteria. And the reordered / refined list is utilized during the encoding / decoding process of the block. a. In one example, the motion candidate list may be: i. In one example, the motion candidate list may be a TM merge / AMVP list after block-based bilateral matching refinement and / or template matching refinement and / or sub-block-based bilateral matching refinement. ii. In one example, the motion candidate list may be a regular merge / AMVP list after DMVR / multi-pass DMVR process. iii. In one example, the motion candidate list may be an ADMVR merged list after the DMVR / multi-pass DMVR process. iv. In one example, the motion candidate list can be the GPM after the template matching refinement process A merged list (for example, AGPMList, LGPMList, or LAGPMList). v. In one example, the motion candidate list may be a regular merge / AMVP list. vi. In one example, the motion candidate list may be a TM merge / AMVP list. vii. In one example, the motion candidate list may be an affine merge / AMVP list. viii. In one example, the motion candidate list may be an IBC merge / AMVP list. ix. In one example, the motion candidate list may be an ADMVR merge list. x. In one example, the motion candidate list may be a GPM merge list. xi. In one example, the motion candidate list may be a TPM merge list. xii. In one example, the motion candidate list may be an MMVD merge list. xiii. In one example, the motion candidate list may be an IBC template matching candidate list. xiv. In one example, the motion candidate list may be an intra template matching candidate list. xv. In one example, the motion candidate list may be an IBC MMVD candidate list. xvi. In one example, the motion candidate list may be an IBC GPM candidate list. xvii. In one example, the motion candidate list may be an IBC TPM candidate list. xviii. In one example, the motion candidate list may be any other motion candidate list. Regarding the re-ranking criteria of the motion candidate list 3. In one example, the re-ranking / refinement process can be based on template matching cost. a. Alternatively, the re-ranking / refinement process can be based on the bilateral matching cost. b. In one example, the cost function between the current template and the reference template can be: i.SAD / MR-SAD, ii.SATD / MR-SATD, iii.SSD / MR-SSD, iv.SSE / MR-SSE, v. Weighted SAD / Weighted MR-SAD, vi. Weighted SATD / weighted MR-SATD, vii. Weighted SSD / weighted MR-SSD, viii. Weighted SSE / weighted MR-SSE, ix. Gradient information. c. In one example, if LMCS is enabled, the current template and the reference template may consist of samples in the mapped domain. i. Alternatively, the current template and the reference template may consist of samples in the original domain. d. In one example, the motion candidates may be divided into several subgroups, and the motion candidates in each subgroup may be reordered in ascending order according to the cost value based on the template / bilateral matching. e. In one example, samples of a template may be selected based on a rule. i. In one example, if only the above template is available for the current block, template matching reordering can only use the above template. ii. In one example, if only the left template is available for the current block, the template matching reordering can only use the left template. iii. In one example, if both the above template and the left template are available for the current block, the template matching reordering may use the left template, the above template, or both the above and left templates. f. In one example, the template / bilateral matching process may include one component, such as brightness. i. Alternatively, the template / bilateral matching process may include multiple components, such as luminance and chrominance. (i) In one example, the total template / bilateral matching cost may be calculated as a weighted sum of the template / bilateral matching costs over different color components. g. In one example, the width of the left template and / or the height of the upper template may be fixed. i. In one example, the width of the left template can be 1. ii. In one example, the height of the upper template can be 1. h. In one example, for a BV candidate, some operations need to be performed. i. The reordering / processing methods may be different for MV candidates and BV candidates. ii. In one example, the BV used to locate the reference template may be cropped. (i) In one example, if the reference template is outside the current image (e.g. Fig.33AAs shown in the figure), the BV used to locate the reference template can be cropped so that the reference template is located inside the current picture. The cropping operation will not change the BV of the corresponding BV candidate. Fig.33B Shown in. iii. In one example, if the reference template is outside the current image (such as Fig.33A As shown), the corresponding BV candidates can still be reordered. (i) In one example, if the reference template is outside the current picture, it can be filled by the nearest sample from within the current picture. iv. In one example, the reference template should have been reconstructed inside the current picture. (i) In one example, if the reference template is not reconstructed or is outside the current picture, the corresponding BV candidates may not be reordered. v. In one example, the samples included in the reference template should be available (eg, reconstructed and within the same stripe / tile / IBC virtual buffer). (i) Alternatively, additionally, if some or all samples contained in the reference template are not available, the above method of handling a reference template outside the current picture may be applied. 4. Template shape and / or size can be adaptive. a. In one example, the template may include only neighboring samples to the left of the current block. b. In one example, the template may only include neighboring samples above the current block. c. In one example, the template shape and / or size may be selected according to the CU shape and / or size. d. In one example, the width of the left template can be selected according to the CU height. i. For example, if H<=M, the size of the left template is w1xH; otherwise, the size of the left template is w2xH. (i) In one example, M, w1, and w2 are set equal to 8, 1, and 2, respectively. e. In one example, the height of the upper template can be selected according to the CU width. i. For example, if W<=N, the size of the upper template is Wxh1; otherwise, the size of the upper template is Wxh2. (i) In one example, N, h1, and h2 are set equal to 8, 1, and 2, respectively. f. In one example, the width of the left template can be selected according to the CU width. i. For example, if W<=N, the size of the left template is w1xH; otherwise, the size of the left template is w2xH. (i) In one example, N, w1, and w2 are set equal to 8, 1, and 2, respectively. g. In one example, the height of the upper template can be selected according to the CU height. i. For example, if H<=M, the size of the upper template is Wxh1; otherwise, the size of the upper template is Wxh2. (i) In one example, M, h1, and h2 are set equal to 8, 1, and 2, respectively. h. In one example, samples of the template and reference samples of the template samples may be downsampled or down-sampled before being used to calculate the cost. i. Whether and / or how downsampling is performed may depend on the CU size. ii. In one example, no downsampling is performed on the short edge of the CU. 5. In one example, the reordering / thinning process can be based on block boundary discontinuity measurements. Fig.48 An example diagram 4800 showing example neighboring reconstructed blocks (grey) and a current prediction block (white) is shown. a. In one example, a boundary discontinuity measure (e.g., disCost) may be defined as: Where P is the prediction of the current block and R is the reconstructed samples in the neighboring blocks. Fig.48 Shown in. i. In one example, w1 is 0 and w3 is 0. w2 is 1 and w4 is 1. ii. In one example, w1=w2=w3=w4=1. 6. In one example, the reordering / refinement process can be based on a block boundary discontinuity measure (eg, disCost) and / or a template matching cost (eg, tmCost). a. In one example, the reordering / refinement process can be based on a block boundary discontinuity measure (e.g., disCost) and the weighted average of the template matching cost (e.g., tmCost), which can be calculated as w dis ×disCost+w tm ×tmCost. i. In one example, w dis and w tm It can be predefined or transmitted by signal or derived from codec information. ii. In one example, w dis May depend on QP. (i) In one example, for higher QP, w disCan be set to a smaller value. b. In one example, a flag may be signaled to indicate whether block boundary discontinuity measurement or template matching cost is used. c. In one example, whether block boundary discontinuity measurement or template matching cost is used may depend on encoding / decoding information. i. In one example, for QPs greater than a predefined threshold, the template matching cost may be used. 2.33. Bilateral matching AMVP merge mode The bidirectional predictor consists of an AMVP predictor in one direction and a merged predictor in the other direction. This mode can be enabled for a codec block when the selected merged predictor and AMVP predictor meet the DMVR condition, where there is at least one reference picture from the past and one reference picture from the future relative to the current picture, and the distances from both reference pictures to the current picture are the same, and bilateral matching MV refinement is applied to the merged MV candidate and AMVP MVP as a starting point. Otherwise, if the template matching function is enabled, template matching MV refinement is applied to the merged predictor or AMVP predictor with a higher template matching cost. The AMVP part of this mode is signaled like regular unidirectional AMVP, ie, the reference index and MVD are signaled, and it has a derived MVP index if template matching is used, or the MVP index is signaled when template matching is disabled. For AMVP direction LX, X can be 0 or 1, and the merge part in the other direction (1-LX) is implicitly derived by minimizing the bilateral matching cost between the AMVP predictor and the merge predictor (i.e., for a pair of AMVP motion vectors and merge motion vectors). For each merge candidate in the merge candidate list with a motion vector in the other direction (1-LX), the bilateral matching cost is calculated using the merge candidate MV and the AMVP MV. The merge candidate with the minimum cost is selected. Bilateral matching refinement is applied to the codec block with the selected merge candidate MV and AMVP MV as starting points. The third pass in multi-pass DMVR is enabled for AMVP merge mode coded blocks, which is an 8x8 sub-PU BDOF refinement of multi-pass DMVR. The mode is indicated by a flag, and if the mode is enabled the AMVP direction LX is further indicated by a flag. 2.34. Construction of motion candidate list The following detailed embodiments should be considered as examples to explain the general concept. These embodiments should not be interpreted in a narrow manner. In addition, these embodiments can be combined in any way. The term "block" may refer to a codec tree block (CTB), a codec tree unit (CTU), a codec block (CB), a CU, a PU, a TU, a PB, a TB, or a video processing unit including multiple samples / pixels. A block may be rectangular or non-rectangular. W and H are the width and height of the current block (eg, luma block). For IBC-coded blocks, a block vector (BV) is used to indicate the displacement from the current block to a reference block that has been reconstructed inside the current picture. In the following, BV predictor is also BV candidate. Skip mode is also merge mode. When a BV candidate satisfies one or a combination of at least one of the following conditions, the BV candidate may be determined to be valid. 1) The corresponding reference block has been reconstructed inside the current image. 2) The corresponding reference block is located in the same CTU row as the current block. 3) The corresponding reference block is located in the same tile / sub-picture as the current block. 4) The corresponding reference block is located in the same slice as the current block. 5) The BV candidate satisfies the block vector constraints (eg, which are described in 2.4.2 and 2.4.3). 6) The BV candidate satisfies the IBC virtual buffer conditions (eg, which are described in 2.4.4). When a BV candidate violates one or a combination of at least one of the above conditions, the BV candidate may be determined to be invalid. The non-adjacent spatial candidates of the current codec block are the adjacent spatial candidates of the virtual block in the i-th search round (such as Fig. 9 As shown in FIG. 1 ). For the i-th search round, the width and height of the virtual block are calculated by the following formulas: newWidth = i×2×gridX+W, newHeight = i×2×gridY+H. Obviously, if the search round i is 0, the virtual block is the current block. If the CU at the te...

Claims

1. A method for video processing, comprising: For conversion between a current video block of a video and a bitstream of the video, determining at least one history-based motion vector prediction (HMVP) table of the current video block based on at least one of the following: a MV difference threshold for comparing at least two motion vectors (MVs) of at least two HMVP candidates, the codec mode of the current video block, or an index to a reference picture list associated with the current video block; as well as The converting is performed based on the at least one HMVP table. 2 . The method according to claim 1 , wherein the at least one HMVP table comprises a plurality of HMVP tables, and the plurality of HMVP tables are determined for different coding and decoding modes. The method according to claim 2 , wherein the table sizes of the plurality of HMVP tables are different.

4. The method according to any one of claims 1-3, wherein a single HMVP table is associated with multiple codec modes. 5 . The method according to claim 1 , wherein the at least one HMVP table comprises more than two HMVP tables.

6. The method according to any one of claims 1 to 5, wherein determining the at least one HMVP table comprises: determining whether a difference between a first motion candidate in a target HMVP table to be added to the at least one HMVP table and a second motion candidate in the target HMVP table is less than the MV difference threshold; as well as The target HMVP table is updated based on the determination.

7. The method of claim 6, wherein updating the target HMVP table based on the determination comprises: If it is determined that the difference is less than the MV difference threshold, adding the first motion candidate at the beginning or end of the target HMVP table; as well as The second motion candidate is removed from the target HMVP table. 8 . The method of claim 6 , wherein if it is determined that the difference is less than the MV difference threshold, the first motion candidate is not added to the target HMVP table.

9. The method according to any one of claims 6 to 8, wherein the difference comprises one of the following: a first difference between a first MV of the first motion candidate and a second MV of the second motion candidate, a second difference between a first block vector (BV) of the first motion candidate and a second BV of the second motion candidate, a third difference between a first set of angular position MVs (CPMVs) of the first motion candidate and a second set of CPMVs of the second motion candidate, or A fourth difference between the portion of CPMVs in the first set of CPMVs and the portion of CPMVs in the second set of CPMVs.

10. The method of claim 9, wherein determining the difference between the first motion candidate and the second motion candidate comprises: The difference is determined from among the first difference, the second difference, the third difference, and the fourth difference based on the motion types of the first motion candidate and the second motion candidate.

11. The method according to claim 10, wherein the motion type comprises one of the following: Inter-frame motion candidate type, Intra Block Copy (IBC) motion candidate type, or Affine motion candidate type.

12. The method according to claim 10 or claim 11, further comprising: The MV difference threshold is determined based on the motion type.

13. The method according to any one of claims 6 to 12, further comprising: The MV difference threshold is determined based on at least one of the following: the codec mode associated with the target HMVP table, or The accuracy of the MVs of the first motion candidate and the second motion candidate.

14. The method according to claim 13, wherein the codec mode comprises one of the following: Template Matching (TM) Merge Mode, Affine merge mode, Regular merge mode, Intra Block Copy (IBC) merge mode, or Adaptive decoder side motion vector refinement (ADMVR) mode. The method of claim 13 or claim 14 , wherein the precision of the MV indicates fractional digits of the MV.

16. The method of any one of claims 1-15, wherein the at least one HMVP table comprises an Adaptive Decoder Side Motion Vector Refinement (ADMVR) HMVP table, Wherein determining the at least one table comprises: If it is determined that the first motion candidate satisfies conditions for decoder-side motion vector refinement (DMVR), the first motion candidate is added to the ADMVR HMVP table.

17. The method of claim 16, wherein the condition for DMVR comprises at least one of the following: The first motion candidate comprises a bi-directional prediction motion, The first reference picture of the first motion candidate is in the past relative to the current picture, and the second reference picture of the first motion candidate is in the future relative to the current picture, A first distance between the first reference picture and the current picture is the same as a second distance between the second reference picture and the current picture, The first reference picture and the second reference picture are short-term reference pictures, or The first reference picture and the second reference picture are non-scaled reference pictures. The method of claim 17 , wherein the first distance or the second distance comprises a picture order count (POC) difference.

19. The method of any one of claims 1-18, wherein the at least one HMVP table comprises a target HMVP table corresponding to a target reference picture list.

20. The method according to claim 19, wherein the target HMVP table is determined by: Determining, based on the bidirectional motion candidates, a unidirectional motion candidate corresponding to the target reference picture list; and The unidirectional motion candidate is added to the target HMVP table.

21. The method of claim 20, wherein determining the unidirectional motion candidate comprises: Determine a first MV of the bidirectional motion candidate as a second MV of the unidirectional motion candidate, wherein the first MV is associated with the target reference picture list; as well as A first reference picture index of the bidirectional motion candidate is determined as a second reference picture index of the unidirectional motion candidate, the first reference picture index being associated with the target reference picture list.

22. The method of claim 20 or claim 21, wherein determining the unidirectional motion candidate comprises: If it is determined that the third MV of the bidirectional motion candidate is associated with another reference picture list different from the target reference picture list, adding a zero MV as a fourth MV of the unidirectional motion candidate; as well as Negative one is determined as the reference picture index of the uni-directional motion candidate.

23. The method according to any one of claims 19 to 22, wherein the target HMVP table is determined by at least one of the following: If it is determined that the first uni-directional motion candidate includes valid motion information corresponding to the target reference picture list, adding the first uni-directional motion candidate to the target HMVP table; or If it is determined that the second uni-directional motion candidate includes valid motion information corresponding to another reference picture list different from the target reference picture list, the second uni-directional motion candidate is not added to the target HMVP table.

24. The method according to any one of claims 19 to 23, further comprising: In bilateral matching advanced motion vector prediction (AMVP) merge mode, the target HMVP table corresponding to the target reference picture list is used.

25. A method for video processing, comprising: For a conversion between a current video block of a video and a bitstream of the video, determining whether to add a history-based motion vector prediction (HMVP) candidate from a HMVP table to at least one candidate in a motion candidate list of the current video block based on at least one comparison between the HMVP candidate and the HMVP candidate in the motion candidate list; as well as The converting is performed based on the determination.

26. The method of claim 25, wherein the determining is based on a codec mode associated with the HMVP candidate.

27. The method according to claim 25 or claim 26, wherein the at least one candidate comprises a portion of the candidates in the motion candidate list based on the ranking of the candidates in the motion candidate list, or The at least one candidate includes all candidates in the motion candidate list.

28. The method of any one of claims 25-27, wherein the at least one comparison comprises comparing the HMVP candidate to the at least one candidate based on at least one motion vector (MV) difference threshold.

29. The method of claim 28, wherein the at least one comparison comprises at least one of: If it is determined that a first MV difference threshold among the at least one MV difference threshold is less than a second MV difference threshold, comparing the HMVP candidate with all candidates in the motion candidate list, the first MV difference threshold is used to determine the HMVP table, and the second MV difference threshold is used to determine the motion candidate list; or If it is determined that the first MV difference threshold is greater than or equal to the second MV difference threshold, the HMVP candidate is compared with some of the candidates in the motion candidate list based on the ranking of the candidates in the motion candidate list.

30. The method of claim 27 or claim 29, wherein the number of candidates in the partial candidates is based on the total number of candidates in the motion candidate list before adding the HMVP candidate.

31. The method of any one of claims 28-30, wherein a first MV difference threshold for a first comparison in the at least one comparison is the same as a second MV difference threshold for a second comparison in the at least one comparison.

32. The method of any one of claims 28-30, wherein a first MV difference threshold for a first comparison in the at least one comparison is different from a second MV difference threshold for a second comparison in the at least one comparison.

33. The method of claim 32, wherein the first MV difference threshold is used to compare the HMVP candidate with a portion of the candidates in the motion candidate list, and the second MV difference threshold is used to compare the HMVP candidate with the remaining candidates in the motion candidate list.

34. The method of claim 32 or claim 33, wherein the first MV difference threshold is less than the second MV difference threshold.

35. The method according to any one of claims 28 to 34, wherein a third MV difference threshold for comparison between two HMVP candidates is different from at least one of: a fourth MV difference threshold for comparison between the HMVP candidate and another motion candidate of a motion type different from the HMVP, or A fifth MV difference threshold for comparison between motion candidates of at least one motion type different from the HMVP.

36. The method of claim 35, wherein the third MV difference threshold is greater than at least one of the fourth MV difference threshold or the fifth MV difference threshold, or The third MV difference threshold is smaller than at least one of the fourth MV difference threshold or the fifth MV difference threshold.

37. The method according to any one of claims 28 to 38, further comprising: The at least one MV difference threshold is determined based on at least one of the following: the encoding and decoding mode of the current video block, the block size of the current video block, the candidate type of a candidate in the motion candidate list, or The accuracy of MV.

38. The method of claim 37, wherein the codec mode comprises at least one of the following: Template Matching (TM) Merge Mode, Affine merge mode, Regular merge mode, Intra Block Copy (IBC) merge mode, or Adaptive decoder side motion vector refinement (ADMVR) mode.

39. The method of claim 37 or claim 38, wherein the precision of the MV indicates fractional digits of the MV.

40. The method according to any one of claims 37-39, wherein the candidate types include at least one of the following: Adjacent airspace candidate type, Adjacent time domain candidate type, Non-adjacent airspace candidate types, Non-adjacent time domain candidate type, HMVP candidate type, Paired candidate types, or Spatial-Temporal Motion Vector Prediction (STMVP) candidate type.

41. The method of any one of claims 28-38, wherein the at least one MV difference threshold is fixed.

42. The method according to any one of claims 28 to 41, further comprising: The at least one MV difference threshold is included in the bitstream.

43. The method of any one of claims 25-42, wherein the at least one candidate is located at at least one position of the current video block.

44. The method of claim 43, wherein the at least one location comprises at least one of: Neighbors in the airspace above, The left airspace neighbor, The upper right airspace neighbor, The lower left airspace neighbor, or Upper left airspace neighbor.

45. A method for video processing, comprising: For conversion between a current video block of a video and a bitstream of the video, determining the first candidate list based on a maximum number of motion candidates in the first candidate list of the current video block; Determine a motion candidate list for the current video block based on the first candidate list, the motion candidate list comprising at least one of an intra block copy (IBC) merge candidate list or an IBC advanced motion vector prediction (AMVP) candidate list; as well as The converting is performed based on the motion candidate list.

46. ​​The method of claim 45, wherein the candidates in the first candidate list are unique.

47. A method according to claim 45 or claim 46, wherein the candidates in the first candidate list are completely pruned.

48. The method according to any one of claims 45-47, further comprising: Reorder the candidates in the first candidate list.

49. The method of claim 48, wherein determining the motion candidate list comprises: Based on the ranking of the first candidate list, a first number of candidates in the first candidate list are added to the motion candidate list.

50. The method of claim 49, wherein the first number is 6.

51. A method according to claim 49 or claim 50, wherein the order of the first number of candidates is maintained while the first number of candidates are added to the motion candidate list.

52. The method of any one of claims 45-51, wherein the motion candidate list comprises one of the following: Regular merge candidate list, AMVP Candidate List, or Affine merge candidate list.

53. The method according to any one of claims 45-52, wherein the maximum number of the motion candidates in the first candidate list is 20.

54. A method for video processing, comprising: For conversion between a current video block of a video and a bitstream of the video, determining at least one maximum number of candidates of at least one candidate type in a first candidate list of the current video block; determining the first candidate list based on the at least one maximum number of candidates of the at least one candidate type; as well as The converting is performed based on the first candidate list.

55. The method of claim 54, wherein the at least one maximum number comprises one of: a first maximum number of history-based motion vector prediction (HMVP) candidates, or HMVP table size.

56. The method of claim 55, wherein the first maximum number or the HMVP table size is 25.

57. The method of any one of claims 54-56, wherein the at least one maximum number comprises a second maximum number of history-based motion vector prediction (HMVP) candidates, the HMVP candidate being unique.

58. The method of claim 57, wherein the second maximum number is 15.

59. The method of any one of claims 54-58, wherein the at least one maximum number comprises a third maximum number of adjacent spatial domain candidates, the adjacent spatial domain candidates being unique.

60. The method of claim 59, wherein the third maximum number is 4.

61. The method of any one of claims 54-60, wherein the at least one maximum number comprises a fourth maximum number of pairwise average candidates, the pairwise average candidates being unique.

62. The method of claim 61, wherein the fourth maximum number is 1.

63. A method according to any one of claims 54-62, wherein a fifth number of positions for pair-wise candidates are retained in the first candidate list.

64. The method of claim 63, wherein the fifth number is 1.

65. The method according to any one of claims 54-64, wherein the maximum number of motion candidates in the first candidate list is 20.

66. The method according to any one of claims 54-65, further comprising: reordering at least one candidate in the first candidate list; as well as Based on the reordering, the motion candidate list is determined by adding a target number of candidates from the first candidate list to a motion candidate list.

67. The method of claim 66, wherein the at least one candidate comprises one of: All candidates in the first candidate list; or Some of the candidates in the first candidate list.

68. A method according to claim 66 or claim 67, wherein the target number is 6.

69. A method for video processing, comprising: For conversion between a current video block of a video and a bitstream of the video, determine a plurality of candidates for the current video block, wherein the plurality of candidates are of at least one candidate type; reordering the plurality of candidates based on the at least one candidate type by performing an adaptive reordering of merged candidates (ARMC) process on the plurality of candidates; determining a motion candidate list based on the reordered plurality of candidates and the at least one candidate type; as well as The converting is performed based on the motion candidate list.

70. The method of claim 69, wherein determining the motion candidate list comprises: Based on costs of candidates of a first candidate type among the at least one candidate type, a first number of reordered candidates of the first candidate type are added to the motion candidate list.

71. The method of claim 70, further comprising: The first number is determined based on at least one of the following: the first candidate type, or The encoding and decoding mode of the current video block.

72. The method of claim 70 or claim 71, wherein the first candidate type is an adjacent spatial domain candidate type and the first number is 4.

73. The method of claim 70 or claim 71, wherein the first candidate type is a history-based motion vector prediction (HMVP) candidate type and the first number is 15.

74. The method of claim 70 or claim 71, wherein the first candidate type is a pairwise average candidate type and the first number is 1.

75. The method of any one of claims 69-74, wherein reordering the plurality of candidates comprises: The plurality of candidates of a set of candidate types are reordered together.

76. The method of claim 75, wherein the set of candidate types includes at least one of: Adjacent airspace candidate type, History-based Motion Vector Prediction (HMVP) candidate type, or Pairwise average candidate types.

77. The method of claim 75 or claim 76, wherein determining the motion candidate list comprises: A second number of reordered candidates of the set of candidate types are added to the motion candidate list based on the costs of the candidates of the set of candidate types.

78. The method of claim 77, further comprising: The second number is determined based on at least one of the following: the set of candidate types, or The encoding and decoding mode of the current video block.

79. A method according to claim 77 or claim 78, wherein the second number is 6.

80. The method of any one of claims 69-79, wherein determining the motion candidate list comprises: determining a third number of history-based motion vector prediction (HMVP) candidates from the reordered candidates; reordering a group of candidates, the group of candidates comprising the third number of HMVP candidates and at least one of: a group of adjacent spatial domain candidates or a group of paired candidates; as well as A fourth number of the reordered set of candidates is added to the motion candidate list.

81. The method of any one of claims 69-79, wherein determining the motion candidate list comprises: determining a fifth number of HMVP candidates from a history-based motion vector prediction (HMVP) table based on a first motion vector (MV) difference threshold; determining a sixth number of HMVP candidates from the fifth number of HMVP candidates by reordering the fifth number of HMVP candidates; A group of motion candidates is determined based on the second MV difference threshold, the group of motion candidates comprising at least one of the following: the sixth number of HMVP candidates, A set of adjacent spatial candidates, or A group of paired candidates; as well as A seventh number of motion candidates is determined from the set of motion candidates by sorting the set of motion candidates.

82. The method of claim 81, wherein the first MV difference threshold is the same as the second MV difference threshold.

83. The method of claim 81 or claim 82, further comprising: The first MV difference threshold is determined based on a block size of the current video block.

84. The method of claim 83, wherein the block size comprises at least one of: the block width of the current video block, the block height of the current video block, or The product of the block width and the block height.

85. The method of claim 81 or claim 82, further comprising: The first MV difference threshold is determined based on at least one of the following: the encoding / decoding mode of the current video block, the block size of the current video block, or The accuracy of MV.

86. The method of claim 85, wherein the codec mode comprises one of the following: Template Matching (TM) Merge Mode, Affine merge mode, Regular merge mode, Intra Block Copy (IBC) merge mode, or Adaptive decoder side motion vector refinement (ADMVR) mode.

87. The method of claim 85 or claim 86, wherein the precision of the MV indicates fractional digits of the MV.

88. The method of any one of claims 69-87, wherein if the candidates for the current video block are reordered multiple times, the reordering criteria of the candidates are reused for the multiple reorderings.

89. The method of claim 88, wherein the re-ranking criteria comprises a template matching cost of the candidates.

90. A method for video processing, comprising: For conversion between a current video block of a video and a bitstream of the video, determining the IBC AMVP candidate list without reordering candidates in the intra block copy (IBC) advanced motion vector prediction (AMVP) candidate list of the current video block; as well as The converting is performed based on the IBC AMVP candidate list.

91. The method of claim 90, wherein determining the IBC AMVP candidate list comprises: Determining an IBC merge candidate list for the current video block; as well as The IBC AMVP candidate list is determined based on the IBC merge candidate list without reordering the candidates in the IBC merge candidate list.

92. A method for video processing, comprising: For conversion between a current video block of a video and a bitstream of the video, determine a first motion candidate list of the current video block corresponding to a first coding mode and a second motion candidate list of the current video block corresponding to a second coding mode of the current video block; as well as performing the conversion based on the first motion candidate list and the second motion candidate list, The determination of the first motion candidate list is different from the determination of the second motion candidate list.

93. The method of claim 92, wherein a first motion vector (MV) difference threshold for the first motion candidate list is different from a second MV difference threshold for the second motion candidate list.

94. The method of claim 92 or claim 93, wherein a first motion vector (MV) difference threshold for a first motion candidate type is different from a second MV difference threshold for a second motion candidate type.

95. The method of any one of claims 92-94, wherein a first motion vector (MV) difference threshold for a first block size is different from a second MV difference threshold for a second block size.

96. The method of any one of claims 92-94, wherein a first MV difference threshold for a first motion vector (MV) having a first fractional digit is different from a second MV difference threshold for a second MV having a second fractional digit.

97. The method according to any one of claims 1-96, wherein the converting comprises encoding the current video block into the bitstream.

98. The method of any one of claims 1-96, wherein the converting comprises decoding the current video block from the codestream.

99. A device for processing video data, comprising a processor and a non-volatile memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform a method according to any one of claims 1-98.

100. A non-transitory computer-readable storage medium storing instructions for causing a processor to execute the method according to any one of claims 1-98.

101. A non-transitory computer-readable recording medium storing a code stream of a video generated by a method performed by a video processing device, wherein the method comprises: determining at least one history-based motion vector prediction (HMVP) table for a current video block of the video based on at least one of: a MV difference threshold for comparing at least two motion vectors (MVs) of at least two HMVP candidates, the codec mode of the current video block, or an index to a reference picture list associated with the current video block; as well as The code stream is generated based on the at least one HMVP table.

102. A method for storing a video code stream, comprising: determining at least one history-based motion vector prediction (HMVP) table for a current video block of the video based on at least one of: a MV difference threshold for comparing at least two motion vectors (MVs) of at least two HMVP candidates, the codec mode of the current video block, or an index to a reference picture list associated with the current video block; Generate the bitstream based on the at least one HMVP table; as well as The code stream is stored in a non-transitory computer-readable recording medium.

103. A non-transitory computer-readable recording medium storing a code stream of a video generated by a method performed by a video processing device, wherein the method comprises: determining whether to add a history-based motion vector prediction (HMVP) candidate to a motion candidate list for a current video block of the video based on at least one comparison between the HMVP candidate from a HMVP table and at least one candidate in the motion candidate list; as well as The code stream is generated based on the determination.

104. A method for storing a video code stream, comprising: determining whether to add a history-based motion vector prediction (HMVP) candidate to a motion candidate list for a current video block of the video based on at least one comparison between the HMVP candidate from a HMVP table and at least one candidate in the motion candidate list; generating the code stream based on the determination; as well as The code stream is stored in a non-transitory computer-readable recording medium.

105. A non-transitory computer-readable recording medium storing a code stream of a video generated by a method performed by a video processing device, wherein the method comprises: Determine the first candidate list based on a maximum number of motion candidates in the first candidate list for the current video block of the video; Determine a motion candidate list for the current video block based on the first candidate list, the motion candidate list comprising at least one of an intra block copy (IBC) merge candidate list or an IBC advanced motion vector prediction (AMVP) candidate list; as well as The code stream is generated based on the motion candidate list.

106. A method for storing a video code stream, comprising: Determine the first candidate list based on a maximum number of motion candidates in the first candidate list for the current video block of the video; Determine a motion candidate list for the current video block based on the first candidate list, the motion candidate list comprising at least one of an intra block copy (IBC) merge candidate list or an IBC advanced motion vector prediction (AMVP) candidate list; generating the bitstream based on the motion candidate list; as well as The code stream is stored in a non-transitory computer-readable recording medium.

107. A non-transitory computer-readable recording medium storing a code stream of a video generated by a method performed by a video processing device, wherein the method comprises: determining at least one maximum number of candidates of at least one candidate type in a first candidate list for a current video block of the video; determining the first candidate list based on the at least one maximum number of candidates of the at least one candidate type; as well as The code stream is generated based on the first candidate list.

108. A method for storing a video code stream, comprising: determining at least one maximum number of candidates of at least one candidate type in a first candidate list for a current video block of the video; determining the first candidate list based on the at least one maximum number of candidates of the at least one candidate type; generating the bitstream based on the first candidate list; as well as The code stream is stored in a non-transitory computer-readable recording medium.

109. A non-transitory computer-readable recording medium storing a code stream of a video generated by a method performed by a video processing device, wherein the method comprises: Determine a plurality of candidates for a current video block of the video, the plurality of candidates being of at least one candidate type; reordering the plurality of candidates based on the at least one candidate type by performing an adaptive reordering of merged candidates (ARMC) process on the plurality of candidates; determining a motion candidate list based on the reordered plurality of candidates and the at least one candidate type; as well as The code stream is generated based on the motion candidate list.

110. A method for storing a video code stream, comprising: Determine a plurality of candidates for a current video block of the video, the plurality of candidates being of at least one candidate type; reordering the plurality of candidates based on the at least one candidate type by performing an adaptive reordering of merged candidates (ARMC) process on the plurality of candidates; determining a motion candidate list based on the reordered plurality of candidates and the at least one candidate type; generating the bitstream based on the motion candidate list; as well as The code stream is stored in a non-transitory computer-readable recording medium.

111. A non-transitory computer-readable recording medium storing a code stream of a video generated by a method performed by a video processing device, wherein the method comprises: determining an intra block copy (IBC) advanced motion vector prediction (AMVP) candidate list for a current video block of the video without reordering candidates in the IBC AMVP candidate list; as well as The codestream is generated based on the IBC AMVP candidate list.

112. A method for storing a video code stream, comprising: determining an intra block copy (IBC) advanced motion vector prediction (AMVP) candidate list for a current video block of the video without reordering candidates in the IBC AMVP candidate list; Generate the bitstream based on the IBC AMVP candidate list; as well as The code stream is stored in a non-transitory computer-readable recording medium.

113. A non-transitory computer-readable recording medium storing a code stream of a video generated by a method performed by a video processing device, wherein the method comprises: Determining a first motion candidate list for a current video block of the video corresponding to a first codec mode and a second motion candidate list for the current video block corresponding to a second codec mode of the current video block; as well as generating the code stream based on the first motion candidate list and the second motion candidate list, The determination of the first motion candidate list is different from the determination of the second motion candidate list.

114. A method for storing a video code stream, comprising: Determining a first motion candidate list for a current video block of the video corresponding to a first codec mode and a second motion candidate list for the current video block corresponding to a second codec mode of the current video block; Generate the code stream based on the first motion candidate list and the second motion candidate list; as well as storing the code stream in a non-transitory computer-readable recording medium, The determination of the first motion candidate list is different from the determination of the second motion candidate list.

Images