Method, apparatus and medium for video processing

Abstract

The embodiments of the present disclosure provide a solution for video processing. A video processing method is proposed, which includes: applying a processing procedure to at least one motion candidate in a motion candidate list during conversion between a target block of a video and a bitstream of the target block; after the processing procedure, reordering the motion candidate list by applying a first reordering process to the motion candidate list; and performing the conversion based on the reordered motion candidate list.

Description

[Technical Field]

[0001] FIELD Embodiments of the present disclosure relate generally to video coding techniques, and more particularly to reordering a motion candidate list with refined motion information. [Background technology]

[0002] Nowadays, digital video capabilities are applied to various aspects of people's lives. Several types of video compression technologies have been proposed for video encoding / decoding, including MPEG-2, MPEG-4, ITU-TH.263, ITU-TH.264 / MPEG-4 Part 10 Advanced Video Coding (AVC), ITU-TH.265 High Efficiency Video Coding (HEVC) standard, and versatile video coding (VVC) standard. However, the coding efficiency of video coding technologies is generally expected to be further improved.

[0003] Embodiments of the present disclosure provide a solution for video processing.

[0004] In a first aspect, a video processing method is proposed. The method includes applying a processing procedure to at least one motion candidate in a motion candidate list during conversion between a target block of a video and a bitstream of the target block; after the processing procedure, reordering the motion candidate list by applying a first reordering process to the motion candidate list; and performing the conversion based on the reordered motion candidate list. Compared with conventional techniques, a higher coding gain can be achieved. Furthermore, coding efficiency can be improved.

[0005] In a second aspect, another video processing method is proposed. The method includes, during conversion between a target block of a video and a bitstream of the target block, applying a reordering process using refined motion information to the target block coded in an inter mode with decoder-side motion vector refinement (DMVR) to reorder at least one of a first motion candidate list or a second motion candidate list, where a first set of motion candidates in the first motion candidate list are refined in one direction and a second set of motion candidates in the second motion candidate list are refined in another direction, and performing the conversion based on at least one of the reordered first motion candidate list or the reordered second motion candidate list. Compared with conventional techniques, a higher coding gain can be achieved. Furthermore, coding efficiency can be improved.

[0006] In a third aspect, another video processing method is proposed. The method includes, during conversion between a target block of a video and a bitstream of the target block, updating at least one motion candidate list for adaptive decoder-side motion vector refinement (DMVR) by removing a set of redundant motion candidates from the at least one motion candidate list, and performing the conversion based on the at least one updated motion candidate list. Compared with conventional techniques, a higher coding gain can be achieved. Furthermore, coding efficiency can be improved.

[0007] In a fourth aspect, another video processing method is proposed. The method includes, during conversion between a target block of a video and a bitstream of the target block, generating a first number of motion candidates for a motion candidate list associated with the target block, where a second number of motion candidates is allowed for the motion candidate list, the first number being greater than the second number, and performing the conversion based on the motion candidate list. Compared with conventional techniques, a higher coding gain can be achieved. Furthermore, coding efficiency can be improved.

[0008] In a fifth aspect, another video processing method is proposed. The method includes, during conversion between a target block of a video and a bitstream of the target block, generating a cost value in a motion candidate list refinement process for the target block, applying a reordering process to the motion candidate list using the cost value, and performing the conversion based on the motion candidate list. Compared with conventional techniques, a higher coding gain can be achieved. Furthermore, coding efficiency can be improved.

[0009] In a sixth aspect, another video processing method is proposed. The method includes, during conversion between a target block of a video and a bitstream of the target block, sorting a set of merge candidates for the target block according to a set of merge types, constructing an adaptive decoder-side motion vector refinement (DMVR) merge candidate list using the first N merge candidates of each of the set of merge types, where N is an integer, and performing the conversion based on the constructed adaptive DMVR merge candidate list. Compared with conventional techniques, a higher coding gain can be achieved. Furthermore, coding efficiency can be improved.

[0010] In a seventh aspect, there is proposed an apparatus for processing video data, said apparatus comprising a processor and a non-transitory memory with instructions that, when executed by the processor, cause the processor to perform a method according to any of the first, second, third, fourth, fifth or sixth aspects.

[0011] In an eighth aspect, a non-transitory computer-readable storage medium is proposed, the non-transitory computer-readable storage medium storing instructions for causing a processor to perform a method according to any of the first, second, third, fourth, fifth or sixth aspects.

[0012] In a ninth aspect, a non-transitory computer-readable recording medium is proposed, which stores a video bitstream generated by a method executed by a video processing device, the method including: applying a processing procedure to at least one motion candidate in a motion candidate list for the target block; after the processing procedure, reordering the motion candidate list by applying a first reordering process to the motion candidate list; and generating a bitstream for the target block based on the reordered motion candidate list.

[0013] In a tenth aspect, a method for storing a video bitstream includes the steps of applying a processing procedure to at least one motion candidate in a motion candidate list for the target block, sorting the motion candidate list by applying a first sorting process to the motion candidate list after the processing procedure, generating a bitstream for the target block based on the sorted motion candidate list, and storing the bitstream on a non-transitory computer-readable recording medium.

[0014] In an eleventh aspect, another non-transitory computer-readable recording medium is proposed, which stores a video bitstream generated by a method executed by a video processing device, the method including: applying a reordering process using refined motion information to a target block of video coded in an inter mode with decoder-side motion vector refinement (DMVR) to reorder at least one of a first motion candidate list or a second motion candidate list, wherein a first set of motion candidates in the first motion candidate list are refined in one direction and a second set of motion candidates in the second motion candidate list are refined in another direction; and generating a bitstream for the target block based on at least one of the reordered first motion candidate list or the reordered second motion candidate list.

[0015] In a twelfth aspect, a method for storing a video bitstream includes applying a reordering process using refined motion information to a target block of video coded in inter mode with decoder-side motion vector refinement (DMVR) to reorder at least one of a first motion candidate list or a second motion candidate list, wherein a first set of motion candidates in the first motion candidate list are refined in one direction and a second set of motion candidates in the second motion candidate list are refined in another direction; generating a bitstream for the target block based on at least one of the reordered first motion candidate list or the reordered second motion candidate list; and storing the bitstream on a non-transitory computer-readable recording medium.

[0016] In a thirteenth aspect, another non-transitory computer-readable recording medium is proposed, the non-transitory computer-readable recording medium storing a bitstream of video generated by a method executed by a video processing device, the method including: updating at least one motion candidate list for adaptive decoder-side motion vector refinement (DMVR) of a target block of the video by removing a set of redundant motion candidates from the at least one motion candidate list; and generating a bitstream of the target block based on the at least one updated motion candidate list.

[0017] In a fourteenth aspect, a method for storing a bitstream of a video includes the steps of updating at least one motion candidate list for an adaptive decoder-side motion vector refinement (DMVR) of a target block of the video by removing a set of redundant motion candidates from the at least one motion candidate list, generating a bitstream of the target block based on the at least one updated motion candidate list, and storing the bitstream in a non-transitory computer-readable recording medium.

[0018] In a fifteenth aspect, another non-transitory computer-readable recording medium is proposed, the non-transitory computer-readable recording medium storing a bitstream of video generated by a method executed by a video processing device, the method including generating a first number of motion candidates for a motion candidate list associated with a target block of the video, where a second number of motion candidates is allowed for the motion candidate list and the first number is greater than the second number, and generating a bitstream of the target block based on the motion candidate list.

[0019] In a sixteenth aspect, a method for storing a bitstream of a video includes the steps of generating a first number of motion candidates for a motion candidate list associated with a target block of the video, wherein a second number of motion candidates is allowed for the motion candidate list, the first number being greater than the second number; generating a bitstream of the target block based on the motion candidate list; and storing the bitstream in a non-transitory computer-readable recording medium.

[0020] In a seventeenth aspect, another non-transitory computer-readable recording medium is proposed, which stores a bitstream of a video generated by a method executed by a video processing device, the method including generating cost values ​​in a motion candidate list refinement process for a target block of the video, applying a reordering process to the motion candidate list using the cost values, and generating a bitstream of the target block based on the motion candidate list.

[0021] In an 18th aspect, a method for storing a bitstream of a video includes steps of generating cost values ​​in a motion candidate list refinement process for a target block of the video, applying a reordering process to the motion candidate list using the cost values, generating a bitstream of the target block based on the motion candidate list, and storing the bitstream in a non-transitory computer-readable recording medium.

[0022] In a nineteenth aspect, another non-transitory computer-readable storage medium is proposed, the non-transitory computer-readable storage medium storing a bitstream of video generated by a method executed by a video processing device, the method including: sorting a set of merge candidates for a target block of the video according to a set of merge types; constructing an adaptive decoder-side motion vector refinement (DMVR) merge candidate list using a first N merge candidates of each of the set of merge types, where N is an integer; and generating a bitstream for the target block based on the constructed adaptive DMVR merge candidate list.

[0023] In a twentieth aspect, a method for storing a bitstream of a video includes the steps of sorting a set of merge candidates for a target block of the video according to a set of merge types; constructing an adaptive decoder-side motion vector refinement (DMVR) merge candidate list using the first N merge candidates of each of the set of merge types, where N is an integer; generating a bitstream for the target block based on the constructed adaptive DMVR merge candidate list; and storing the bitstream on a non-transitory computer-readable recording medium.

[0024] This description is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. It is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. [Brief explanation of the drawings]

[0025] These and other objects, features, and advantages of exemplary embodiments of the present disclosure will become more apparent through the following detailed description taken in conjunction with the accompanying drawings, in which like reference numerals generally refer to like components.

[0026] [Figure 1] 1 shows a block diagram illustrating an example video coding system, in accordance with some embodiments of the present disclosure.

[0027] [Figure 2] 1 shows a block diagram illustrating a first exemplary video encoder, in accordance with some embodiments of the present disclosure.

[0028] [Figure 3] 1 shows a block diagram illustrating an exemplary video decoder, according to some embodiments of the present disclosure.

[0029] [Figure 4] 1 shows a schematic diagram of an example encoder block diagram.

[0030] [Figure 5] 1 shows a schematic diagram of the 67 intra prediction modes.

[0031] [Figure 6] 1 shows a schematic diagram of a reference sample for wide-angle intra prediction.

[0032] [Figure 7] 1 shows a schematic diagram of the discontinuity problem for orientations greater than 45°.

[0033] [Figure 8] A schematic diagram of the MMVD search points is shown.

[0034] [Figure 9] 1 shows a schematic diagram of an illustration of a symmetric MVD mode.

[0035] [Figure 10] Indicates the extended CU area used in BDOF.

[0036] [Figure 11] 1 shows a control point-based affine motion model.

[0037] [Figure 12] Affine MVF for each subblock is shown.

[0038] [Figure 13] Indicates the position of the inherited affine motion predictor.

[0039] [Figure 14] Indicates control point motion vector inheritance.

[0040] [Figure 15] The locations of the candidate positions for the constructed affine merge mode are shown.

[0041] [Figure 16] 1 shows an illustration of the motion vector usage for the proposed combining method.

[0042] [Figure 17] The sub-blocks MV VSB and pixels Δv(i,j) are shown.

[0043] [Figure 18a] 1 illustrates the SbTMVP process in VVC, showing the spatial neighboring blocks used by ATVMP. [Figure 18b] We demonstrate the SbTMVP process in VVC, deriving sub-CU motion fields by applying motion shifts from spatial neighbors and scaling motion information from the corresponding co-located sub-CUs.

[0044] [Figure 19] 1 shows local illumination compensation.

[0045] [Figure 20] Indicates no subsampling of the short side.

[0046] [Figure 21]10 shows the decoding side motion vector refinement.

[0047] [Figure 22] The diamond region within the search area is shown.

[0048] [Figure 23] 10 shows the location of spatial merge candidates.

[0049] [Figure 24] 10 shows candidate pairs considered in a spatial merge candidate redundancy check.

[0050] [Figure 25] 10 shows an illustration of motion vector scaling for temporal merge candidates.

[0051] [Figure 26] The candidate positions of the temporal merge candidates C0 and C1 are shown.

[0052] [Figure 27] Indicates the VVC spatial neighboring blocks of the current block.

[0053] [Figure 28] 1 shows an illustration of a virtual block in the i-th search round.

[0054] [Figure 29] An example of GPM splits grouped at the same angle is shown.

[0055] [Figure 30] 10 illustrates uni-predictive MV selection for geometric partitioning mode.

[0056] [Figure 31] 10 shows an example of generating bending weight w_0 using the geometric division mode.

[0057] [Figure 32]10 shows spatial neighboring blocks used to derive spatial merge candidates.

[0058] [Figure 33] 1 shows the execution of template matching in a search region around the initial MV.

[0059] [Figure 34] 1 shows an illustration of the sub-blocks to which OBMC is applied.

[0060] [Figure 35] Indicates the SBT location, type, and conversion type.

[0061] [Figure 36] Indicates the adjacent samples used to calculate the SAD.

[0062] [Figure 37] Indicates the neighboring samples used to calculate the SAD for sub-CU level motion information.

[0063] [Figure 38] Illustrates the sorting process.

[0064] [Figure 39] 1 illustrates the reordering process in the encoder.

[0065] [Figure 40] 1 illustrates the reordering process in the decoder.

[0066] [Figure 41] 1 shows a flowchart of a method according to an embodiment of the present disclosure.

[0067] [Figure 42] 1 shows a flowchart of a method according to an embodiment of the present disclosure.

[0068] [Figure 43]1 shows a flowchart of a method according to an embodiment of the present disclosure.

[0069] [Figure 44] 1 shows a flowchart of a method according to an embodiment of the present disclosure.

[0070] [Figure 45] 1 shows a flowchart of a method according to an embodiment of the present disclosure.

[0071] [Figure 46] 1 shows a flowchart of a method according to an embodiment of the present disclosure.

[0072] [Figure 47] FIG. 1 illustrates a block diagram of a computing device capable of implementing various embodiments of the present disclosure.

[0073] Throughout the drawings, the same or similar reference numbers generally refer to the same or similar elements. DETAILED DESCRIPTION OF THE INVENTION

[0074] Next, the principles of the present disclosure will be explained with reference to some embodiments. It should be understood that these embodiments are provided for illustrative purposes only, to help those skilled in the art understand and embody the present disclosure, and do not imply any limitations on the scope of the present disclosure. The disclosure described herein can be embodied in various ways other than those described below.

[0075] In the following description and claims, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

[0076] References in this disclosure to "one embodiment," "one embodiment," "exemplary embodiment," etc. indicate that the described embodiment may include a particular feature, structure, or characteristic, but not all embodiments necessarily include the particular feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in connection with an exemplary embodiment, it is noted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments, whether or not explicitly stated.

[0077] Terms such as "first" and "second" may be used herein to describe various elements, but it should be understood that these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, a first element could be referred to as a second element, and similarly, a second element could be referred to as a first element, without departing from the scope of the exemplary embodiments. As used herein, the term "and / or" includes any and all combinations of one or more of the listed terms.

[0078] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit example embodiments. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly dictates otherwise. It will be further understood that the terms "comprise," "comprise," "have," "have," "contain," and / or "comprise," when used herein, specify the presence of stated features, elements, and / or components, etc., but do not exclude the presence or addition of one or more other features, elements, components, and / or combinations thereof. Example Environment

[0079] 1 is a block diagram illustrating an example video coding system 100 that may utilize the techniques of this disclosure. As shown, video coding system 100 may include a source device 110 and a destination device 120. Source device 110 may also be referred to as a video encoding device, and destination device 120 may also be referred to as a video decoding device. In operation, source device 110 may be configured to generate encoded video data, and destination device 120 may be configured to decode the encoded video data generated by source device 110. Source device 110 may include a video source 112, a video encoder 114, and an input / output (I / O) interface 116.

[0080] Video source 112 may include sources such as a video capture device. Examples of a video capture device include, but are not limited to, an interface that receives video data from a video content provider, a computer graphics system that generates video data, and / or combinations thereof.

[0081] The video data may include one or more images. The video encoder 114 encodes the video data from the video source 112 to generate a bitstream. The bitstream may include a series of bits forming a coded representation of the video data. The bitstream may include coded images and associated data. A coded image is a coded representation of an image. The associated data may include sequence parameter sets, image parameter sets, and other syntax structures. The I / O interface 116 may include a modulator / demodulator and / or transmitter. The encoded video data may be transmitted directly to the destination device 120 over the network 130A via the I / O interface 116. The encoded video data may be stored on a storage medium / server 130B for access by the destination device 120.

[0082] Destination device 120 may include an I / O interface 126, a video decoder 124, and a display device 122. I / O interface 126 may include a receiver and / or a modem. I / O interface 126 may obtain encoded video data from source device 110 or storage medium / server 130B. Video decoder 124 may decode the encoded video data. Display device 122 may display the decoded video data to a user. Display device 122 may be integrated with destination device 120 or may be external to destination device 120 configured to interface with an external display device.

[0083] Video encoder 114 and video decoder 124 may operate according to a video compression standard such as the High Efficiency Video Coding (HEVC) standard, the Versatile Video Coding (VVC) standard, and other current and / or future standards.

[0084] FIG. 2 is a block diagram illustrating an example of a video encoder 200, which may be an example of the video encoder 114 in the system 100 shown in FIG. 1, according to some embodiments of the disclosure.

[0085] Video encoder 200 may be configured to embody any or all of the techniques of this disclosure. In the example of FIG. 2, video encoder 200 includes multiple functional components. The techniques described in this disclosure may be shared among various components of video encoder 200. In some examples, a processor may be configured to perform any or all of the techniques described in this disclosure.

[0086] In some embodiments, the video encoder 200 may include a division unit 201, a prediction unit 202, which may include a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205, and an intra prediction unit 206, a residual generation unit 207, a transform unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transform unit 211, a reconstruction unit 212, a buffer 213, and an entropy encoding unit 214.

[0087] In other examples, video encoder 200 may include more, fewer, or different functional components. In one example, prediction unit 202 may include an intra block copy (IBC) unit. The IBC unit may perform prediction in an IBC mode, where at least one reference image is the image in which the current video block is located.

[0088] Furthermore, some components, such as the motion estimation unit 204 and the motion compensation unit 205, may be integrated but are depicted separately in the example of FIG. 2 for illustrative purposes.

[0089] The division unit 201 may divide an image into one or more video blocks. The video encoder 200 and the video decoder 300 may support a variety of video block sizes.

[0090] The mode select unit 203 may, for example, select one of intra or inter coding modes based on an error result, provide the resulting intra-coded or inter-coded block to the residual generation unit 207 to generate residual block data, and provide the resulting intra-coded or inter-coded block to the reconstruction unit 212 to reconstruct the encoded block and use it as a reference image. In some examples, the mode select unit 203 may select a combined intra- and inter-prediction (CIIP) mode in which prediction is based on an inter-prediction signal and an intra-prediction signal. In the case of inter-prediction, the mode select unit 203 may select the resolution of the motion vector of the block (e.g., sub-pixel or integer-pixel accuracy).

[0091] To perform inter prediction on the current video block, motion estimation unit 204 may generate motion information for the current video block by comparing one or more reference frames from buffer 213 with the current video block. Motion compensation unit 205 may determine a prediction video block for the current video block based on the motion information and decoded samples of pictures from buffer 213 other than the picture associated with the current video block.

[0092] Motion estimation unit 204 and motion compensation unit 205 may perform different operations on a current video block depending, for example, on whether the current video block is in an I slice, a P slice, or a B slice. As used herein, an "I slice" may refer to a portion of an image composed of macroblocks, all of which are based on macroblocks within the same image. Additionally, as used herein, in some aspects, a "P slice" and a "B slice" may refer to a portion of an image composed of macroblocks that are independent of macroblocks within the same image.

[0093] In some examples, motion estimation unit 204 may perform unidirectional prediction on the current video block, and motion estimation unit 204 may look up a reference image in list 0 or list 1 for a reference video block of the current video block. Motion estimation unit 204 may then generate a reference index indicating a reference image in list 0 or list 1 that contains the reference video block, and a motion vector indicating a spatial displacement between the current video block and the reference video block. Motion estimation unit 204 may output the reference index, the prediction direction indicator, and the motion vector as motion information for the current video block. Motion compensation unit 205 may generate a predictive video block for the current video block based on the reference video block indicated by the motion information of the current video block.

[0094] Alternatively, in other examples, motion estimation unit 204 may perform bidirectional prediction on the current video block. Motion estimation unit 204 may look up a reference image in list 0 for a reference video block of the current video block, or look up a reference image in list 1 for another reference video block of the current video block. Motion estimation unit 204 may then generate a reference index indicating the reference image in list 0 or list 1 that contains the reference video block, and a motion vector indicating the spatial displacement between the reference video block and the current video block. Motion estimation unit 204 may output the reference index and the motion vector for the current video block as motion information for the current video block. Motion compensation unit 205 may generate a predictive video block for the current video block based on the reference video block indicated by the motion information of the current video block.

[0095] In some examples, motion estimation unit 204 may output a full set of motion information for the decoder's decoding process. Alternatively, in some embodiments, motion estimation unit 204 may signal the motion information of the current video block by reference to the motion information of another video block. For example, motion estimation unit 204 may determine that the motion information of the current video block is sufficiently similar to the motion information of a neighboring video block.

[0096] In one example, motion estimation unit 204 may indicate, in a syntax structure associated with the current video block, a value that indicates to video decoder 300 that the current video block has the same motion information as another video block.

[0097] In another example, motion estimation unit 204 may identify another video block and a motion vector differential (MVD) in a syntax structure associated with the current video block. The motion vector differential indicates the difference between the motion vector of the current video block and the motion vector of the indicated video block. Video decoder 300 may determine the motion vector of the current video block using the motion vector and the motion vector differential of the indicated video block.

[0098] As discussed above, video encoder 200 may predictively signal motion vectors. Two examples of predictive signaling techniques that may be implemented by video encoder 200 include advanced motion vector prediction (AMVP) and merge mode signaling.

[0099] Intra prediction unit 206 may perform intra prediction on the current video block. When intra prediction unit 206 performs intra prediction on the current video block, intra prediction unit 206 may generate predictive data for the current video block based on decoded samples of other video blocks within the same image. The predictive data for the current video block may include a predicted video block and various syntax elements.

[0100] Residual generation unit 207 may generate residual data for the current video block by subtracting (e.g., as indicated by a minus sign) the prediction video block of the current video block from the current video block. The residual data for the current video block may include residual video blocks that correspond to different sample components of the samples in the current video block.

[0101] In other examples, such as in skip mode, residual data for the current video block may not be present, and residual generation unit 207 may not perform the subtraction operation.

[0102] Transform processing unit 208 may generate one or more transform coefficient video blocks for the current video block by applying one or more transforms to the residual video block associated with the current video block.

[0103] After transform processing unit 208 generates the transform coefficient video block associated with the current video block, quantization unit 209 may quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values ​​associated with the current video block.

[0104] Inverse quantization unit 210 and inverse transform unit 211 may apply inverse quantization and inverse transform, respectively, to the transform coefficient video block to reconstruct a residual video block from the transform coefficient video block. Reconstruction unit 212 may add the reconstructed residual video block to corresponding samples from one or more prediction video blocks generated by prediction unit 202 to generate a reconstructed video block associated with the current video block for storage in buffer 213.

[0105] After reconstruction unit 212 reconstructs the video blocks, a loop filtering operation may be performed to reduce video blocking artifacts in the video blocks.

[0106] Entropy encoding unit 214 may receive data from other functional components of video encoder 200. Once entropy encoding unit 214 receives the data, entropy encoding unit 214 may perform one or more entropy encoding operations to generate entropy encoded data and output a bitstream including the entropy encoded data.

[0107] FIG. 3 is a block diagram illustrating an example of a video decoder 300, which may be an example of the video decoder 124 in the system 100 shown in FIG. 1, according to some embodiments of this disclosure.

[0108] Video decoder 300 may be configured to perform any or all of the techniques of this disclosure. In the example of FIG. 3, video decoder 300 includes multiple functional components. The techniques described in this disclosure may be shared among various components of video decoder 300. In some examples, a processor may be configured to perform any or all of the techniques described in this disclosure.

[0109] 3, video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, a reconstruction unit 306, and a buffer 307. Video decoder 300 may, in some examples, perform a decoding path that is generally inverse to the encoding path described with respect to video encoder 200.

[0110] The entropy decoding unit 301 may retrieve an encoded bitstream. The encoded bitstream may include entropy-coded video data (e.g., encoded blocks of video data). The entropy decoding unit 301 may decode the entropy-coded video data, and from the entropy-decoded video data, the motion compensation unit 302 may determine motion information including motion vectors, motion vector precision, reference picture list indexes, and other motion information. The motion compensation unit 302 may determine such information by, for example, performing AMVP and merge mode. AMVP is used and includes deriving several most likely candidates based on data from neighboring PBs and reference pictures. The motion information typically includes horizontal and vertical motion vector displacement values, one or two reference picture indexes, and, for prediction regions in B slices, identification of which reference picture list is associated with each index. As used herein, in some aspects, "merge mode" may refer to deriving motion information from spatially or temporally neighboring blocks.

[0111] The motion compensation unit 302 may generate the motion-compensated block, possibly performing interpolation based on an interpolation filter. An identifier of the interpolation filter used with sub-pixel precision may be included in the syntax element.

[0112] Motion compensation unit 302 may calculate interpolated values ​​for sub-integer pixels of the reference block using interpolation filters used by video encoder 200 during encoding of the video block. Motion compensation unit 302 may determine the interpolation filters used by video encoder 200 according to received syntax information and generate the predictive block using the interpolation filters.

[0113] The motion compensation unit 302 may use at least a portion of the syntax information to determine the size of blocks used to encode frames and / or slices of the encoded video sequence, partition information describing how each macroblock of the images of the encoded video sequence is divided, a mode indicating how each partition is encoded, one or more reference frames (and reference frame lists) for each inter-encoded block, and other information for decoding the encoded video sequence. As used herein, in some aspects, a "slice" may refer to a data structure that can be decoded independently from other slices of the same image with respect to entropy coding, signal prediction, and residual signal reconstruction. A slice can be either an entire image or a region of an image.

[0114] The intra prediction unit 303 may form a prediction block from spatially adjacent blocks, e.g., using an intra prediction mode received in the bitstream. The inverse quantization unit 304 inverse quantizes, or dequantizes, the quantized video block coefficients provided in the bitstream and decoded by the entropy decoding unit 301. The inverse transform unit 305 applies an inverse transform.

[0115] Reconstruction unit 306 may obtain decoded blocks, for example, by adding residual blocks and corresponding prediction blocks generated by motion compensation unit 302 or intra prediction unit 303. Optionally, a deblocking filter may be applied to filter the decoded blocks to remove block artifacts. The decoded video blocks are then stored in buffer 307, which provides reference blocks for subsequent motion compensation / intra prediction and also generates decoded video for presentation on a display device.

[0116] Several exemplary embodiments of the present disclosure will be described in detail below. While section headings are used herein for ease of understanding, it should be understood that they do not limit the embodiments disclosed in a section to that section alone. Furthermore, while certain embodiments are described with reference to versatile video coding or other specific video codecs, the disclosed techniques are applicable to other video coding techniques. Furthermore, while some embodiments describe video coding steps in detail, it will be understood that the corresponding decoding steps that undo the coding are performed by a decoder. Furthermore, the term video processing encompasses video coding or compression, video decoding or decompression, and video transcoding, which represents video pixels from one compressed format to another compressed format or at a different compressed bit rate. 1. Overview This disclosure relates to video coding techniques. Specifically, it relates to a motion candidate list for inter-coding that uses refined motion information to reorder a motion candidate list, whether and / or how the motion candidate list reordering process is applied, and other coding tools in image / video coding. This may be applied to existing video coding standards such as HEVC and Versatile Video Coding (VVC), and may also be applied to future video coding standards or video codecs. 2.Background Video coding standards have evolved primarily through the development of well-known ITU-T and ISO / IEC standards. ITU-T developed H.261 and H.263, while ISO / IEC developed MPEG-1 and MPEG-4 Visual. These two organizations jointly developed H.262 / MPEG-2 Video, H.264 / MPEG-4 Advanced Video Coding (AVC), and H.265 / HEVC. Since H.262, video coding standards have been based on a hybrid video coding architecture that utilizes temporal prediction and transform coding. To develop future coding technologies beyond HEVC, the Joint Video Exploration Team (JVET) was jointly established by VCEG and MPEG in 2015. Since then, many new methods have been adopted by the JVET and incorporated into reference software called the Joint Exploration Model (JEM). In April 2018, a Joint Video Expert Team (JVET) was created between VCEG (Q6 / 16) and ISO / IEC JTC1SC29 / WG11 (MPEG) to work on the VVC standard, which aims to achieve a 50% bitrate reduction compared to HEVC. The latest version of the VVC standard draft, Versatile Video Coding (Draft 10), can be found at: http: / / phenix.it-sudparis.eu / jvet / doc_end_user / documents / 20_Teleconference / wg11 / JVET-T2001-v1.zip The latest reference software for VVC, called VTM, can be found at: https: / / vcgit.hhi.fraunhofer.de / jvet / VVCSoftware_VTM / - / tags / VTM-11.0. 2.1. Common video codec coding flow Figure 4 shows an example of a VVC encoder block diagram, which includes three in-loop filtering blocks: a deblocking filter (DF), a sample adaptive offset (SAO), and an ALF. Unlike DF, which uses a predefined filter, SAO and ALF utilize the original samples of the current image, add an offset, and apply a finite impulse response (FIR) filter to reduce the mean squared error between the original and reconstructed samples, respectively, with coded side information signaling the offset and filter coefficients. The ALF is the final processing stage for each image and can be seen as a tool that attempts to capture and repair artifacts created in previous stages. 2. Intra-mode coding with 67 intra-prediction modes To capture arbitrary edge directions present in natural video, the number of directional intra modes is expanded from 33 as used in HEVC to 65 as shown in Figure 5, while the planar and DC modes remain the same. These dense directional intra prediction modes apply to all block sizes and to both luma and chroma intra prediction. In HEVC, all intra-coded blocks are square, with the length of each side being a power of two. Therefore, no division operations are required to generate the intra predictor using DC mode. In VVC, blocks can be rectangular, which typically requires the use of division operations for each block. To avoid the division operations for DC prediction, only the long side is used to calculate the average for non-square blocks. Wide-angle Intra Prediction VVC defines 67 modes, but the exact prediction direction for a given intra-prediction mode index further depends on the block shape. Conventional angular intra-prediction directions are defined from 45 degrees to -135 degrees clockwise. In VVC, for non-square blocks, some of the conventional angular intra-prediction modes are adaptively replaced with wide-angle intra-prediction modes. The replaced modes are signaled using the original mode index and are remapped to the wide-angle mode index after analysis. The total number of intra-prediction modes remains unchanged, i.e., 67, and the intra-mode coding method remains unchanged. To support these prediction directions, a top reference of length 2W+1 and a left reference of length 2H+1 are defined as shown in FIG. The number of modes replaced by the wide-angle direction mode depends on the aspect ratio of the block. The replaced intra prediction modes are shown in Table 1. [Table 1] Figure 7 shows a block diagram of discontinuities for directions greater than 45 degrees. As shown in diagram 700 of Figure 7, in wide-angle intra prediction, two vertically adjacent prediction samples may use two non-adjacent reference samples. Therefore, a low-pass reference sample filter and side smoothing are applied to wide-angle prediction to mitigate the adverse effects of an increased gap Δpα. If the wide-angle mode represents a non-fractional offset, there are eight wide-angle modes that satisfy this condition: [-14, -12, -10, -6, 72, 76, 78, 80]. When a block is predicted in these modes, samples in the reference buffer are directly copied without applying interpolation. This change reduces the number of samples required for smoothing. Furthermore, it aligns the design of non-fractional modes with conventional prediction modes and wide-angle modes. VVC supports not only 4:2:0, but also 4:2:2 and 4:4:4 chroma formats. The chroma derivative mode (DM) derivation table for 4:2:2 chroma format is originally ported from HEVC, which expands the number of entries from 35 to 67 to accommodate the expansion of intra prediction modes. Because the HEVC specification does not support prediction angles below -135 degrees and above 45 degrees, luma intra prediction modes ranging from 2 to 5 are mapped to 2. Therefore, the chroma DM derivation table for 4:2:2 chroma format is updated by replacing some values ​​in the mapping table entries to more accurately convert the prediction angles of chroma blocks. 2.3. Inter Prediction For each inter-predicted CU, the motion parameters consist of a motion vector, a reference picture index, a reference picture list usage index, and additional information required by VVC's new coding features used to generate inter-predicted samples. Motion parameters can be signaled explicitly or implicitly. When a CU is coded in skip mode, it is associated with one PU and has no significant residual coefficients, coded motion vector deltas, or reference picture indexes. A merge mode is specified, whereby the motion parameters of the current CU are obtained from neighboring CUs, including spatial and temporal candidates and the additional schedule introduced in VVC. Merge mode can be applied to any inter-predicted CU, not just skip mode. An alternative to merge mode is explicit transmission of motion parameters, where the motion vector, the corresponding reference picture index for each reference picture list, a reference picture list usage flag, and other necessary information are explicitly signaled for each CU. 2.4. Intra-Block Copy (IBC) Intra Block Copy (IBC) is a tool adopted in the HEVC extension of SCC. It is well known that it significantly improves the coding efficiency of screen content material. Since IBC mode is implemented as a block-level coding mode, block matching (BM) is implemented in the encoder to find the optimal block vector (or motion vector) for each CU. Here, the block vector is used to indicate the displacement from the current block to a reference block already reconstructed within the current image. The luma block vectors of IBC-coded CUs are integer-precision. Chroma block vectors are also rounded to integer precision. When combined with AMVR, IBC mode can switch between 1-pel and 4-pel motion vector precision. IBC-coded CUs are treated as a third prediction mode other than intra or inter prediction modes. IBC mode is applicable to CUs with both width and height of 64 luma samples or less. On the encoder side, hash-based motion estimation is performed for IBC. The encoder performs an RD check for blocks whose width or height is less than or equal to 16 luma samples. In non-merge mode, a block vector search is first performed using a hash-based search. If the hash search does not return a valid candidate, a block-matching-based local search is performed. In hash-based search, hash key matching (32-bit CRC) between the current block and reference block is extended to all allowable block sizes. The calculation of the hash key for each location in the current image is based on 4x4 subblocks. For larger-sized current blocks, if all hash keys of all 4x4 subblocks match the hash key of the corresponding reference location, the hash key is deemed to match the hash key of the reference block. If hash keys of multiple reference blocks are found to match the hash key of the current block, the block vector cost of each matching reference is calculated and the one with the smallest cost is selected. In block matching search, the search range is set to cover both the previous CTU and the current CTU. At the CU level, the IBC mode is signaled by a flag and can be signaled as IBC AMVP mode or IBC skip / merge mode as follows: IBC skip / merge mode: The merge candidate index is used to indicate which block vectors in the list from neighboring candidate IBC coded blocks are used to predict the current block. The merge list consists of spatial, HMVP, and pairwise candidates. - IBC AMVP mode: Block vector differences are coded in the same way as motion vector differences. The block vector prediction method uses two candidates as predictors: one from the left neighbor and one from the upper neighbor (in case of IBC coding). If either neighbor is unavailable, a default block vector is used as the predictor. A flag is signaled indicating the index of the block vector predictor. 2.5. Merge Mode with MVD (MMVD) In addition to the merge mode in which implicitly derived motion information is directly used to generate the predicted samples of the current CU, a merge mode based on motion vector difference (MMVD) is introduced in VVC. The MMVD flag is signaled immediately after sending the normal merge flag to specify whether the MMVD mode is used for the CU. In MMVD, after a merge candidate is selected, it is further refined by signaled MVD information. The additional information includes a merge candidate flag, an index specifying the magnitude of motion, and an index indicating the direction of motion. In MMVD mode, one of the first two candidates in the merge list is selected to be used as the MV basis. The MMVD candidate flag is signaled to specify whether to use the first or second merge candidate. The distance index specifies motion magnitude information and indicates a predefined offset from the starting point. Figure 8 shows the MMVD search points. As shown in Figure 8, the offset is added to either the horizontal or vertical component of the starting MV. The relationship between the distance index and the predefined offset is specified in Table 2-2. [Table 2] The direction index represents the direction of the MVD relative to the starting point. The direction index can represent four directions as shown in Table 2-3. It should be noted that the meaning of the MVD code can change depending on the information of the starting MV. If the starting MV is a uni-predictive or bi-predictive MV and both lists point to the same side of the current picture (i.e., the POCs of the two references are both greater than the POC of the current picture or both less than the POC of the current picture), the code in Table 2-3 specifies the sign of the MV offset added to the starting MV. If the starting MV is a bi-predictive MV and the two MVs point to different sides of the current picture (i.e., the POC of one reference is greater than the POC of the current picture and the POC of the other reference is less than the POC of the current picture), and the difference in POC of list0 is greater than the difference in POC of list1, the code in Table 2-3 specifies the sign of the MV offset added to the list0 MV component of the starting MV, and the sign for the list1 MV has the opposite value. Otherwise, if the difference in POC in list 1 is greater than in list 0, the sign in Table 2-3 specifies the sign of the MV offset that is added to the list1 MV component of the start MV, and the sign for the list0 MV has the opposite value. The MVD is scaled according to the POC difference in each direction. If the POC difference in both lists is the same, no scaling is required. Otherwise, if the POC difference in list 0 is larger than the POC difference in list 1, the MVD for list 1 is scaled by defining the L10 POC difference as td and the L1 POC difference as tb, as shown in Figure 9. If the POC difference in L1 is larger than L0, the MVD for list 0 is scaled in the same way. If the starting MV is uni-predictive, the MVD is added to the available MV. [Table 3] 2.6. Symmetric MVD Coding In VVC, in addition to the usual uni- and bi-predictive mode MVD signaling, a symmetric MVD mode for bi-predictive MVD signaling is applied. In the symmetric MVD mode, motion information including reference image indices for both list-0 and list-1 and the MVD for list-1 are derived without being signaled. The decoding process of the symmetric MVD mode is as follows: 1. At the slice level, the variables BiDirPredFlag, RefIdxSymL0 and RefIdxSymL1 are derived as follows: - If mvd_l1_zero_flag is 1, BiDirPredFlag is set equal to 0. Otherwise, if the closest reference picture in list-0 and the closest reference picture in list-1 form a pair of pre-post or post-pre reference pictures, BiDirPredFlag is set to 1 and both reference pictures in list-0 and list-1 are short-term reference pictures. Otherwise, BiDirPredFlag is set to 0. 2. At the CU level, if the CU is bi-predictively coded and BiDirPredFlag is equal to 1, then a symmetric mode flag is explicitly signaled indicating whether symmetric mode is used or not. If the symmetric mode flag is true, only mvp_l0_flag, mvp_l1_flag, and MVD0 are explicitly signaled. The reference indices for list-0 and list-1 are set equal to the reference picture pairs, respectively. MVD1 is set equal to (-MVD0). The final motion vector is given by the following equation:

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0117] TECHNICAL FIELD Embodiments of the present disclosure relate to motion candidate list construction.

[0118] As used herein, the term "block" may refer to a coding tree block (CTB), coding tree unit (CTU), coding block (CB), CU, PU, ​​TU, PB, TB, or video processing unit containing multiple samples / pixels. A block may be rectangular or non-rectangular. As used herein, the term "optical flow" may refer to a motion vector.

[0119] 41 shows a flowchart of a method 4100 of video processing according to some embodiments of the present disclosure. The method 4100 may be implemented during conversion between a target block and a bitstream of the target block.

[0120] As shown in Figure 41, in block 4110, a processing procedure is applied to at least one motion candidate in the motion candidate list during conversion between the target block of the video and the bitstream of the target block. In some embodiments, the processing procedure includes at least one of a retouching process, a motion refinement process, or a second reordering process.

[0121] In block 4120, the motion candidate list is reordered by applying a first reordering process to the motion candidate list after the processing procedure. In one example, before performing the first reordering procedure on the motion candidate list, at least one motion candidate in the list may be modified or refined.

[0122] In block 4130, a conversion is performed based on the reordered motion candidate list. In some embodiments, the conversion may include encoding the target block into a bitstream. In some embodiments, the conversion may include decoding the target block from the bitstream. Compared to conventional techniques, the accuracy and continuity of the optical flow motion field are improved. Furthermore, coding efficiency can be improved.

[0123] In some embodiments, the second reordering process is applied to the motion candidate list before the first reordering process. In some embodiments, the motion candidate list is reordered using original motion information during construction of the motion candidate list in the second reordering process.

[0124] In some embodiments, if at least one motion candidate in the motion candidate list is modified by a motion refinement process for the target block, the motion candidate list is reordered using refined motion information. In some embodiments, the motion candidate list is not reordered before the first reordering process is applied.

[0125] In some embodiments, the motion refinement process includes at least one of template matching-based motion refinement or bilateral matching-based motion refinement. In one example, the motion refinement process may refer to template matching-based motion refinement, such as template matching, and / or bilateral matching-based motion refinement, such as DMVR or multi-path DMVR.

[0126] In some embodiments, a simplified version of the motion refinement process is applied, for example, when template matching is used as the motion refinement process, the template size is set to 1 or 2.

[0127] In some embodiments, the processing procedure is a motion refinement process, and the refined motion information used in the first reordering process is the final motion information of the motion refinement process. In some embodiments, if the motion refinement process includes multiple steps, the motion information after the last step in the motion refinement process is used in the first reordering process. In some embodiments, the multiple steps refine the motion information with motion vectors of different resolutions. For example, integer resolution motion vectors are refined to become fractional resolution motion vectors. In one example, the refinement may be from integer resolution motion vectors to fractional resolution motion vectors. In some embodiments, fractional resolution motion vectors are refined to become fractional resolution motion vectors. In another example, the refinement may be from fractional resolution motion vectors to fractional resolution motion vectors. In some embodiments, the different resolutions include at least one of 16 pels, 8 pels, 4 pels, 2 pels, 1 pel, ½ pel, ¼ pel, ⅛ pel, 1 / 16 pel, 1 / 32 pel, or 1 / 64 pel.

[0128] In some embodiments, the multiple steps include different search shapes for refining the motion vector in template matching. In one example, the multiple steps may refer to different search shapes for refining the motion vector, such as diamonds and / or crosses in template matching.

[0129] In some embodiments, motion information after a diamond-shaped search step of the template matching is used in the first reordering process. In some embodiments, the resolution of the motion vectors used depends on a syntax element indicating the resolution of the motion vectors of the motion candidates. In some embodiments, if early termination is used in the motion refinement process, motion information output associated with the early termination is used in the first reordering process.

[0130] In some embodiments, multiple motion refinement processes are applied to the target block, the multiple motion refinement processes are applied in sequence, and the motion information derived after the last motion refinement process is used in the first reordering process. In one example, when multiple motion refinement processes are involved and processed one by one, the motion information derived after the last motion refinement process may be used in the first reordering process.

[0131] In some embodiments, when the plurality of motion refinement processes includes a first motion refinement process and a second motion refinement process, motion information of the second motion refinement process is used in the first reordering process.

[0132] In some embodiments, the first motion refinement process includes at least one of a template matching method or a bilateral matching method. In one example, the first motion refinement process may refer to a template matching method such as TM or a bilateral matching method such as DMVR / multi-path DMVR.

[0133] In some embodiments, the second motion refinement process includes at least one of a template matching method or a bilateral matching method. In one example, the second motion refinement process may refer to a template matching method such as TM, or a bilateral matching method such as DMVR / multi-path DMVR.

[0134] In some embodiments, the processing procedure includes a motion refinement process, and the refined motion information used in the first reordering process is intermediate motion information of the motion refinement process. In some embodiments, the motion refinement process includes multiple steps, and the motion information before the last step in the motion refinement process is used in the first reordering process. In some embodiments, the motion information of the first step in the motion refinement process is used in the first reordering process. In some embodiments, the motion information at integer resolution of the motion vectors is used in the first reordering process. In another example, motion information at integer resolution of the motion vectors (e.g., 16 pels, 8 pels, 4 pels, 2 pels, 1 pel) may be used.

[0135] In some embodiments, when multiple motion refinement processes are applied in sequence, the motion derived from the motion refinement process before the last motion refinement process is used in said first reordering process. In one example, when multiple motion refinement processes are involved and processed one by one, motion information derived from any motion refinement process before the last motion refinement process may be used in the first reordering process.

[0136] In some embodiments, when the plurality of motion refinement processes includes a first motion refinement process and a second motion refinement process, motion information of the first motion refinement process is used in the first reordering process.

[0137] In some embodiments, the first motion refinement process includes at least one of a template matching method or a bilateral matching method. In one example, the first motion refinement process may refer to a template matching method such as TM or a bilateral matching method such as DMVR / multi-path DMVR.

[0138] In some embodiments, the second motion refinement process includes at least one of a template matching method or a bilateral matching method. In one example, the second motion refinement process may refer to a template matching method such as TM, or a bilateral matching method such as DMVR / multi-path DMVR.

[0139] In some embodiments, multiple motion refinement processes are applied in sequence, with the final motion refinement process comprising multiple steps, in which case, in some embodiments, motion information derived from a step prior to the final step in the final motion refinement process is used in said first reordering process.

[0140] In some embodiments, the processing procedure includes a motion refinement process, which includes template matching (TM) and multi-pass decoder-side motion vector refinement (DMVR), and the TM is between the first-pass DMVR and the second-pass DMVR. In one example, when the motion refinement process includes a TM and a multi-pass DMVR, and the TM is processed between the first-pass DMVR (e.g., block-based bilateral matching MV refinement) and the second-pass DMVR (e.g., sub-block-based bilateral matching MV refinement), the motion information after the TM may be used in the first reordering process.

[0141] In some embodiments, the post-TM motion information is used in the first reordering process. In some embodiments, the post-first-pass DMVR motion information is used in the first reordering process. In some embodiments, the post-second-pass DMVR motion information is used in the first reordering process. In some embodiments, if refined motion information is used in the first reordering process, the early termination method used to skip the second-pass DMVR is not applied.

[0142] In some embodiments, if the refined motion information is sub-block-based motion information, the refined motion information of one or more sub-blocks is used in the first reordering process. In some embodiments, the refined motion information of one of the one or more sub-blocks is used in the first reordering process. In some embodiments, the refined motion information of at least one of the top-left sub-block, one of the sub-blocks located at the left block boundary, or one of the sub-blocks located at the top block boundary is used in the first reordering process.

[0143] In some embodiments, refined motion information of multiple sub-blocks of the one or more sub-blocks is used in the first reordering process, in some embodiments, motion information of all sub-blocks is averaged and used in the first reordering process.

[0144] In some embodiments, the motion information of all sub-blocks is weighted-averaged. In some embodiments, a set of weighting parameters depends on at least one of a block dimension, a block size, a sub-block dimension, a sub-block size, or a distance between a sub-block and a template used in the first reordering process. In some embodiments, at least one of motion information of a sub-block at the top boundary and motion information of a sub-block at the left boundary is used in the first reordering process.

[0145] In some embodiments, the motion candidates are divided into a first number of subgroups, and the first reordering process is applied within each subgroup, the first number being greater than 1. In one example, the motion candidates may be divided into X subgroups, and a reordering process may be applied within each subgroup, where X is greater than 1.

[0146] In some embodiments, the number of motion candidates in each subgroup is the same, for example, the number of motion candidates is one of 2, 3, 4, 5, 6, 7, 8, 9, and 10. In one example, the size of the subgroup (i.e., the number of motion candidates in a subgroup, Ns) may be the same for all subgroups, such as Ns=2, 3, 4, 5, 6, 7, 8, 9, 10, etc.

[0147] In some embodiments, the size of at least one subgroup is different from the size of the other subgroups. In some embodiments, the size of a first subgroup is larger than the size of the other subgroups. In some embodiments, the size of the first subgroup is smaller than the size of the other subgroups. In some embodiments, the size of the subgroups is predefined.

[0148] In some embodiments, the motion candidates are divided into the first number of subgroups based on motion candidate type. In one example, the motion candidates may be divided into multiple subgroups according to motion candidate type (e.g., contiguous spatial, temporal, non-contiguous spatial, HMVP).

[0149] In some embodiments, the size of the subgroups is determined based on the number of types of motion candidates available. In one example, the size of the subgroups may be determined as a function of the number of types of motion candidates available (e.g., adjacent spatial and / or temporal motion candidates).

[0150] In some embodiments, whether and / or how to apply the first reordering process using refined motion information is indicated in the bitstream. In some embodiments, whether and / or how to apply the first reordering process using refined motion information is determined dynamically. In some embodiments, whether and / or how to apply the first reordering process using refined motion information depends on coding information. In some embodiments, the first reordering process using refined motion information is applied to the target block, and the size of the target block is greater than a first threshold. In some embodiments, the first reordering process using refined motion information is applied to the target block, and the size of the target block is less than a second threshold. In some embodiments, the first threshold is one of 8, 16, 32, 64, or 128. In some embodiments, the second threshold is one of 64, 128, 256, 512, 1024, or 2048.

[0151] In some embodiments, the target block refers to one of a color component, a sub-image, a slice, a tile, a coding tree unit (CTU), a CTU row, a group of CTUs, a coding unit (CU), a prediction unit (PU), a transform unit (TU), a coding tree block (CTB), a coding block (CB), a prediction block (PB), a transform block (TB), a block, a sub-block of a block, a sub-region within a block, or a region containing multiple samples or pixels.

[0152] In some embodiments, after the processing procedure, an indication of whether and / or how to reorder the motion candidate list by applying the first reordering process to the motion candidate list is indicated at one of the sequence level, image group level, image level, slice level, or tile group level.

[0153] In some embodiments, after the processing procedure, an indication of whether and / or how to reorder the motion candidate list by applying the first reordering process to the motion candidate list is indicated in one of a sequence header, a picture header, a sequence parameter set (SPS), a video parameter set (VPS), a dependency parameter set (DPS), a decoding capability information (DCI), a picture parameter set (PPS), an adaptation parameter set (APS), a slice header, or a tile group header.

[0154] In some embodiments, after the processing procedure, an indication of whether and / or how to reorder the motion candidate list by applying the first reordering process to the motion candidate list is included in one of a prediction block (PB), a transform block (TB), a coding block (CB), a prediction unit (PU), a transform unit (TU), a coding unit (CU), a virtual pipeline data unit (VPDU), a coding tree unit (CTU), a CTU row, a slice, a tile, a sub-image, or a region containing multiple samples or pixels.

[0155] In some embodiments, after the processing procedure, whether and / or how to reorder the motion candidate list by applying the first reordering process to the motion candidate list may be determined based on coding information of the target block, which may include at least one of block size, color format, single and / or dual tree partitioning, color component, slice type, or image type.

[0156] In some embodiments, a non-transitory computer-readable recording medium stores a video bitstream generated by a method executed by a video processing device, the method including applying a processing procedure to at least one motion candidate in a motion candidate list for the target block, reordering the motion candidate list by applying a first reordering process to the motion candidate list after the processing procedure, and generating a bitstream for the target block based on the reordered motion candidate list.

[0157] In some embodiments, a method for storing a video bitstream includes applying a processing procedure to at least one motion candidate in a motion candidate list for the target block; after the processing procedure, reordering the motion candidate list by applying a first reordering process to the motion candidate list; generating a bitstream for the target block based on the reordered motion candidate list; and storing the bitstream on a non-transitory computer-readable recording medium.

[0158] 42 shows a flowchart of a method 4200 of video processing according to some embodiments of the present disclosure. The method 4200 may be implemented during conversion between a target block and a bitstream of the target block.

[0159] As shown in Figure 42, in block 4210, during conversion between a target block of video and a bitstream of the target block, a reordering process using refined motion information is applied to the target block coded in inter mode with decoder-side motion vector refinement (DMVR) to reorder at least one of a first motion candidate list or a second motion candidate list, where a first set of motion candidates in the first motion candidate list are refined in one direction and a second set of motion candidates in the second motion candidate list are refined in another direction. In one example, when the reordering process using refined motion information is applied to inter mode with one-sided DMVR, adaptive DMVR, the reordering process using refined motion information may be applied to reorder the first motion candidate list and / or the second motion candidate list, where the motion candidates in the first motion candidate list are refined in one side (direction) and the motion candidates in the second motion candidate list are refined in the other side (direction).

[0160] In block 4220, the conversion is performed based on at least one of the reordered first motion candidate list or the reordered second motion candidate list. In some embodiments, the conversion may include encoding the target block into the bitstream. In some embodiments, the conversion may include decoding the target block from the bitstream. Compared to conventional techniques, the accuracy and continuity of the optical flow motion field are improved. Furthermore, coding efficiency can be improved.

[0161] In some embodiments, the DMVR comprises at least one of a one-sided DMVR or an adaptive DMVR. In some embodiments, the one-sided DMVR is a DMVR method in which motion information is refined in one direction at the DMVR step. In one example, the one-sided DMVR may refer to a DMVR method in which motion information (e.g., motion vectors) is refined in one direction at the DMVR step.

[0162] In some embodiments, the motion information is refined unidirectionally in a first pass of a multi-path DMVR. In some embodiments, the motion information is refined unidirectionally in one or more of a first pass of a multi-path DMVR or a second pass of the multi-path DMVR.

[0163] In some embodiments, the inter mode is adaptive DMVR. In some embodiments, the reordering process is applied to reorder the first motion candidate list and the second motion candidate list separately. In some embodiments, the first motion candidate list and the second motion candidate list are combined. In this case, the reordering process is used to reorder the combined first motion candidate list and the second motion candidate list. In some embodiments, the refined motion information is block-based motion vectors (MVs) of a first pass in multi-pass DMVR.

[0164] In some embodiments, the target block refers to one of a color component, a sub-image, a slice, a tile, a coding tree unit (CTU), a CTU row, a group of CTUs, a coding unit (CU), a prediction unit (PU), a transform unit (TU), a coding tree block (CTB), a coding block (CB), a prediction block (PB), a transform block (TB), a block, a sub-block of a block, a sub-region within a block, or a region containing multiple samples or pixels.

[0165] In some embodiments, the indication of whether and / or how to apply the reordering process using the refined motion information to the target block coded in inter mode with the DMVR is indicated at one of the sequence level, group of pictures level, picture level, slice level, or tile group level.

[0166] In some embodiments, an indication of whether and / or how to apply the reordering process using the refined motion information to the target block coded in inter mode with the DMVR is indicated in one of a sequence header, a picture header, a sequence parameter set (SPS), a video parameter set (VPS), a dependency parameter set (DPS), a decoding capability information (DCI), a picture parameter set (PPS), an adaptation parameter set (APS), a slice header, or a tile group header.

[0167] In some embodiments, an indication of whether and / or how to apply the reordering process using the refined motion information to the target block coded in inter mode with the DMVR is included in one of a prediction block (PB), a transform block (TB), a coding block (CB), a prediction unit (PU), a transform unit (TU), a coding unit (CU), a virtual pipeline data unit (VPDU), a coding tree unit (CTU), a CTU row, a slice, a tile, a sub-picture, or a region containing multiple samples or pixels.

[0168] In some embodiments, whether and / or how to apply the reordering process using the refined motion information to the target block coded in the DMVR inter mode may be determined based on coding information of the target block, which may include at least one of block size, color format, single and / or dual tree partitioning, color component, slice type, or picture type.

[0169] In some embodiments, a non-transitory computer-readable recording medium stores a video bitstream generated by a method executed by a video processing device, the method including applying a reordering process using refined motion information to a target block of video coded in inter mode with decoder-side motion vector refinement (DMVR) to reorder at least one of a first motion candidate list or a second motion candidate list, wherein a first set of motion candidates in the first motion candidate list are refined in one direction and a second set of motion candidates in the second motion candidate list are refined in another direction, and generating a bitstream for the target block based on at least one of the reordered first motion candidate list or the reordered second motion candidate list.

[0170] In some embodiments, a method for storing a video bitstream includes applying a reordering process using refined motion information to a target block of video coded in inter mode with decoder-side motion vector refinement (DMVR) to reorder at least one of a first motion candidate list or a second motion candidate list, wherein a first set of motion candidates in the first motion candidate list are refined in one direction and a second set of motion candidates in the second motion candidate list are refined in another direction; generating a bitstream for the target block based on at least one of the reordered first motion candidate list or the reordered second motion candidate list; and storing the bitstream on a non-transitory computer-readable recording medium.

[0171] 43 shows a flowchart of a method 4300 of video processing according to some embodiments of the present disclosure. The method 4300 may be implemented during conversion between a target block and a bitstream of the target block.

[0172] As shown in FIG. 43, in block 4310, during conversion between a target block of video and a bitstream of the target block, at least one motion candidate list for adaptive decoder-side motion vector refinement (DMVR) is updated by removing a set of redundant motion candidates from the at least one motion candidate list.

[0173] In block 4320, the conversion is performed based on the at least one updated motion candidate list. In some embodiments, the conversion may include encoding the target block into the bitstream. In some embodiments, the conversion may include decoding the target block from the bitstream. Compared to conventional techniques, the accuracy and continuity of the optical flow motion field are improved. Furthermore, coding efficiency can be improved.

[0174] In some embodiments, the set of redundant motion candidates includes a motion candidate in a first motion candidate list that has the same motion information as one of the motion candidates in a second motion candidate list. In some embodiments, the first motion candidate list is a first direction motion candidate list and the second motion candidate list is a second direction motion candidate list. In some embodiments, the first motion candidate list is a second direction motion candidate list and the second motion candidate list is a first direction motion candidate list. In some embodiments, the set of redundant motion candidates includes motion candidates after a first pass DMVR.

[0175] In some embodiments, the target block refers to one of a color component, a sub-image, a slice, a tile, a coding tree unit (CTU), a CTU row, a group of CTUs, a coding unit (CU), a prediction unit (PU), a transform unit (TU), a coding tree block (CTB), a coding block (CB), a prediction block (PB), a transform block (TB), a block, a sub-block of a block, a sub-region within a block, or a region containing multiple samples or pixels.

[0176] In some embodiments, the instruction as to whether and / or how to remove the set of redundant motion candidates from at least one motion candidate list for the adaptive DMVR is indicated at one of the sequence level, the group of pictures level, the picture level, the slice level, or the tile group level.

[0177] In some embodiments, an indication of whether and / or how to remove the set of redundant motion candidates from at least one motion candidate list for the adaptive DMVR is indicated in one of a sequence header, a picture header, a sequence parameter set (SPS), a video parameter set (VPS), a dependency parameter set (DPS), a decoding capability information (DCI), a picture parameter set (PPS), an adaptation parameter set (APS), a slice header, or a tile group header.

[0178] In some embodiments, instructions for whether and / or how to remove the set of redundant motion candidates from at least one motion candidate list for the adaptive DMVR are included in one of a prediction block (PB), a transform block (TB), a coding block (CB), a prediction unit (PU), a transform unit (TU), a coding unit (CU), a virtual pipeline data unit (VPDU), a coding tree unit (CTU), a CTU row, a slice, a tile, a sub-image, or a region containing multiple samples or pixels.

[0179] In some embodiments, whether and / or how to remove the set of redundant motion candidates from at least one motion candidate list for the adaptive DMVR may be determined based on coding information of the target block, which may include at least one of block size, color format, single and / or dual tree partitioning, color component, slice type, or picture type.

[0180] In some embodiments, a non-transitory computer-readable recording medium stores a bitstream of video generated by a method executed by a video processing device, the method including updating at least one motion candidate list for an adaptive decoder-side motion vector refinement (DMVR) of a target block of the video by removing a set of redundant motion candidates from the at least one motion candidate list, and generating a bitstream of the target block based on the at least one updated motion candidate list.

[0181] In some embodiments, a method for storing a bitstream of a video includes updating at least one motion candidate list for an adaptive decoder-side motion vector refinement (DMVR) of a target block of the video by removing a set of redundant motion candidates from the at least one motion candidate list, generating a bitstream of the target block based on the at least one updated motion candidate list, and storing the bitstream on a non-transitory computer-readable recording medium.

[0182] 44 shows a flowchart of a video processing method 4400 according to some embodiments of the present disclosure. The method 4400 may be implemented during conversion between a target block and a bitstream of the target block.

[0183] As shown in Figure 44, in block 4410, during conversion between a target block of video and a bitstream of the target block, a first number of motion candidates for a motion candidate list associated with the target block is generated. A second number of motion candidates is allowed for the motion candidate list. The first number is greater than the second number. For example, N motion candidates may be generated for the motion candidate list, where the maximum allowed number of motion candidates is M, and N>M.

[0184] In block 4420, a conversion is performed based on the motion candidate list. In some embodiments, the conversion may include encoding the target block into the bitstream. In some embodiments, the conversion may include decoding the target block from the bitstream. Compared to conventional techniques, the accuracy and continuity of the optical flow motion field are improved. Furthermore, coding efficiency can be improved.

[0185] In some embodiments, the second number of motion candidates are selected to be used for prediction or reconstruction, for example, an ordered set of M selected motion candidates are used for prediction / reconstruction.

[0186] In some embodiments, the second number of motion candidates are selected by sorting the first number of motion candidates, where the selected second number of motion candidates are the first M motion candidates of the sorted first number of motion candidates, where M is the second number. For example, the ordered M motion candidates are selected by sorting N motion candidates, where the selected M motion candidates are the first M (e.g., lowest cost) of the sorted N motion candidates.

[0187] In some embodiments, the first number is predefined. In some embodiments, the first number is indicated in the bitstream. In some embodiments, the first number is dynamically determined. In some embodiments, the first number may vary for each coding mode. In some embodiments, the first number is one of 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, or 20.

[0188] In some embodiments, motion information derived from the motion candidate list is used in the reordering process. In some embodiments, a reordering process using refined motion information is applied to the motion candidate list.

[0189] In some embodiments, the target block refers to one of a color component, a sub-image, a slice, a tile, a coding tree unit (CTU), a CTU row, a group of CTUs, a coding unit (CU), a prediction unit (PU), a transform unit (TU), a coding tree block (CTB), a coding block (CB), a prediction block (PB), a transform block (TB), a block, a sub-block of a block, a sub-region within a block, or a region containing multiple samples or pixels.

[0190] In some embodiments, the instruction on whether and / or how to generate the first number of motion candidates for the motion candidate list associated with the target block is indicated at one of the sequence level, group of images level, image level, slice level, or tile group level.

[0191] In some embodiments, an indication of whether and / or how to generate the first number of motion candidates for the motion candidate list associated with the target block is indicated in one of a sequence header, a picture header, a sequence parameter set (SPS), a video parameter set (VPS), a dependency parameter set (DPS), a decoding capability information (DCI), a picture parameter set (PPS), an adaptation parameter set (APS), a slice header, or a tile group header.

[0192] In some embodiments, the instructions for whether and / or how to generate the first number of motion candidates for the motion candidate list associated with the target block are included in one of a prediction block (PB), a transform block (TB), a coding block (CB), a prediction unit (PU), a transform unit (TU), a coding unit (CU), a virtual pipeline data unit (VPDU), a coding tree unit (CTU), a CTU row, a slice, a tile, a sub-image, or a region containing multiple samples or pixels.

[0193] In some embodiments, whether and / or how to generate the first number of motion candidates for the motion candidate list associated with the target block may be determined based on coding information of the target block, which may include at least one of block size, color format, single and / or dual tree partitioning, color component, slice type, or image type.

[0194] In some embodiments, a non-transitory computer-readable recording medium stores a bitstream of a video generated by a method executed by a video processing device, the method including generating a first number of motion candidates for a motion candidate list associated with a target block of the video, wherein a second number of motion candidates is allowed for the motion candidate list, the first number being greater than the second number, and generating a bitstream for the target block based on the motion candidate list.

[0195] In some embodiments, a method for storing a bitstream of a video includes generating a first number of motion candidates for a motion candidate list associated with a target block of the video, wherein a second number of motion candidates is allowed for the motion candidate list and the first number is greater than the second number; generating a bitstream for the target block based on the motion candidate list; and storing the bitstream on a non-transitory computer-readable recording medium.

[0196] 45 shows a flowchart of a method 4500 of video processing according to some embodiments of the present disclosure. The method 4500 may be implemented during conversion between a target block and a bitstream of the target block.

[0197] As shown in Figure 45, block 4510 generates cost values ​​in a motion candidate list refinement process for the target block during conversion between a target block of video and a bitstream of the target block. In some embodiments, the refinement process includes at least one of template matching (TM), decoder-side motion vector refinement (DMVR), or multi-pass DMVR.

[0198] In block 4520, a reordering process is applied to the motion candidate list using the cost values. In one example, cost values ​​generated in a refinement process such as TM or DMVR / multi-pass DMVR may be reused in a reordering process after the refinement process.

[0199] In block 4530, a conversion is performed based on the motion candidate list. In some embodiments, the conversion may include encoding the target block into the bitstream. In some embodiments, the conversion may include decoding the target block from the bitstream. Compared to conventional techniques, the accuracy and continuity of the optical flow motion field are improved. Furthermore, coding efficiency can be improved.

[0200] In some embodiments, the final cost value generated in the refinement process for the motion candidate is saved. In one example, the final (or minimum or maximum) cost value generated in the refinement process for the motion candidate may be saved.

[0201] In some embodiments, in the reordering process, a first final cost value generated in a first refinement process for a first motion candidate is compared with a second final cost value generated in a second refinement process for a second motion candidate. In one example, the final (or minimum or maximum) cost value generated in the refinement process for a first motion candidate may be compared with the final (or minimum or maximum) cost value generated in the refinement process for a second motion candidate in the reordering process.

[0202] In some embodiments, the target block refers to one of a color component, a sub-image, a slice, a tile, a coding tree unit (CTU), a CTU row, a group of CTUs, a coding unit (CU), a prediction unit (PU), a transform unit (TU), a coding tree block (CTB), a coding block (CB), a prediction block (PB), a transform block (TB), a block, a sub-block of a block, a sub-region within a block, or a region containing multiple samples or pixels.

[0203] In some embodiments, the indication of whether and / or how to apply the reordering process to the motion candidate list using the cost values ​​is indicated at one of the sequence level, group of pictures level, picture level, slice level, or tile group level.

[0204] In some embodiments, an indication of whether and / or how to apply the reordering process to the motion candidate list using the cost values ​​is indicated in one of a sequence header, a picture header, a sequence parameter set (SPS), a video parameter set (VPS), a dependency parameter set (DPS), a decoding capability information (DCI), a picture parameter set (PPS), an adaptation parameter set (APS), a slice header, or a tile group header.

[0205] In some embodiments, an indication of whether and / or how to apply the reordering process to the motion candidate list using the cost values ​​is included in one of a prediction block (PB), a transform block (TB), a coding block (CB), a prediction unit (PU), a transform unit (TU), a coding unit (CU), a virtual pipeline data unit (VPDU), a coding tree unit (CTU), a CTU row, a slice, a tile, a sub-image, or a region containing multiple samples or pixels.

[0206] In some embodiments, whether and / or how to apply the reordering process to the motion candidate list using the cost values ​​may be determined based on coding information of the target block, which may include at least one of block size, color format, single and / or dual tree partitioning, color component, slice type, or image type.

[0207] In some embodiments, a non-transitory computer-readable recording medium stores a bitstream of video generated by a method executed by a video processing device, the method may include generating cost values ​​in a motion candidate list refinement process for a target block of the video, applying a reordering process to the motion candidate list using the cost values, and generating a bitstream for the target block based on the motion candidate list.

[0208] In some embodiments, a method for storing a bitstream of a video includes generating cost values ​​in a motion candidate list refinement process for a target block of the video, applying a reordering process to the motion candidate list using the cost values, generating a bitstream for the target block based on the motion candidate list, and storing the bitstream on a non-transitory computer-readable recording medium.

[0209] 46 shows a flowchart of a method 4600 of video processing according to some embodiments of the present disclosure. The method 4600 may be implemented during conversion between a target block and a bitstream of the target block.

[0210] As shown in Figure 46, block 4610 sorts a set of merge candidates for the target block according to a set of merge types during conversion between a target block of video and a bitstream of the target block.

[0211] In block 4620, the first N merge candidates of each of the set of merge types are used to construct an adaptive decoder-side motion vector refinement (DMVR) merge candidate list, where N is an integer. In one example, the merge candidates may be sorted according to merge type, and the first X merge candidates of each merge type are used to construct the adaptive DMVR merge candidate list.

[0212] In block 4630, the conversion is performed based on the constructed adaptive DMVR merge candidate list. In some embodiments, the conversion may include encoding the target block into the bitstream. In some embodiments, the conversion may include decoding the target block from the bitstream. Compared to conventional techniques, the accuracy and continuity of the optical flow motion field are improved. Furthermore, coding efficiency can be improved.

[0213] In some embodiments, the set of merge types is at least one of adjacent spatial, temporal, non-adjacent spatial, history-based motion vector prediction (HMVP), or pairwise. In one example, the merge types may refer to adjacent spatial, and / or temporal, and / or non-adjacent spatial, and / or HMVP, and / or pairwise.

[0214] In some embodiments, the value of N is different for different merge types. In some embodiments, for temporal merge candidates, N is set equal to 1 or 2. In some embodiments, for spatial merge candidates, N is set equal to one of 1, or 2, or 3, or 4, or 5, or 6, or 7, or 8, or 9, or 10. In some embodiments, for non-adjacent spatial merge candidates, N is set equal to one of 1, or 2, or 3, or 4, or 5, or 6, or 7, or 8, or 9, or 10. In some embodiments, for HMVP merge candidates, N is set equal to one of 1, or 2, or 3, or 4, or 5, or 6, or 7, or 8, or 9, or 10. In some embodiments, for pairwise merge candidates, N is set equal to one of 1, or 2, or 3, or 4, or 5, or 6, or 7, or 8, or 9, or 10.

[0215] In some embodiments, the target block refers to one of a color component, a sub-image, a slice, a tile, a coding tree unit (CTU), a CTU row, a group of CTUs, a coding unit (CU), a prediction unit (PU), a transform unit (TU), a coding tree block (CTB), a coding block (CB), a prediction block (PB), a transform block (TB), a block, a sub-block of a block, a sub-region within a block, or a region containing multiple samples or pixels.

[0216] In some embodiments, the indication of whether and / or how to construct the adaptive DMVR merge candidate list using the first N merge candidates of each of the set of merge types is indicated at one of the sequence level, group of pictures level, image level, slice level, or tile group level.

[0217] In some embodiments, an indication of whether and / or how to construct the adaptive DMVR merge candidate list using the first N merge candidates of each of the set of merge types is indicated in one of a sequence header, a picture header, a sequence parameter set (SPS), a video parameter set (VPS), a dependency parameter set (DPS), a decoding capability information (DCI), a picture parameter set (PPS), an adaptation parameter set (APS), a slice header, or a tile group header.

[0218] In some embodiments, instructions for whether and / or how to construct the adaptive DMVR merge candidate list using the first N merge candidates of each of the set of merge types are included in one of a prediction block (PB), a transform block (TB), a coding block (CB), a prediction unit (PU), a transform unit (TU), a coding unit (CU), a virtual pipeline data unit (VPDU), a coding tree unit (CTU), a CTU row, a slice, a tile, a sub-image, or a region containing multiple samples or pixels.

[0219] In some embodiments, whether and / or how to construct the adaptive DMVR merge candidate list using the first N merge candidates of each of the set of merge types may be determined based on coding information of the target block, which may include at least one of block size, color format, single and / or dual tree partitioning, color component, slice type, or image type.

[0220] In some embodiments, a non-transitory computer-readable recording medium stores a bitstream of a video generated by a method executed by a video processing device, the method may include sorting a set of merge candidates for a target block of the video according to a set of merge types, constructing an adaptive decoder-side motion vector refinement (DMVR) merge candidate list using a first N merge candidates of each of the set of merge types, where N is an integer, and generating a bitstream for the target block based on the constructed adaptive DMVR merge candidate list.

[0221] In some embodiments, a method for storing a bitstream of a video includes the steps of: sorting a set of merge candidates for a target block of the video according to a set of merge types; constructing an adaptive decoder-side motion vector refinement (DMVR) merge candidate list using the first N merge candidates of each of the set of merge types, where N is an integer; generating a bitstream for the target block based on the constructed adaptive DMVR merge candidate list; and storing the bitstream on a non-transitory computer-readable recording medium.

[0222] Implementations of the present disclosure may be described in light of the following clauses, the features of which may be combined in any reasonable manner.

[0223] Clause 1. A video processing method, comprising the steps of: applying a processing procedure to at least one motion candidate in a motion candidate list during conversion between a target block of a video and a bitstream of the target block; after the processing procedure, reordering the motion candidate list by applying a first reordering process to the motion candidate list; and performing the conversion based on the reordered motion candidate list.

[0224] Clause 2. The method of clause 1, wherein the processing procedure includes at least one of a correction process, a motion refinement process, or a second reordering process.

[0225] Clause 3. The method of clause 2, wherein the second reordering process is applied to the motion candidate list before the first reordering process.

[0226] Clause 4. The method of clause 3, wherein the motion candidate list is sorted using original motion information during construction of the motion candidate list in the second sorting process.

[0227] Clause 5. The method of clause 1, wherein if at least one motion candidate in the motion candidate list is modified by a motion refinement process for the target block, the motion candidate list is sorted using refined motion information.

[0228] Clause 6. The method of clause 1, wherein the motion candidate list is unsorted before the first sorting process is applied.

[0229] Clause 7. The method of clause 2, wherein the motion refinement process includes at least one of template matching based motion refinement or bilateral matching based motion refinement.

[0230] Clause 8. The method of clause 2, wherein a simplified version of the motion refinement process is applied.

[0231] Clause 9. The method of clause 8, wherein if template matching is used as the motion refinement process, the template size is set to 1 or 2.

[0232] Clause 10. The method of clause 1, wherein the processing procedure is a motion refinement process, and wherein the refined motion information used in the first reordering process is the final motion information of the motion refinement process.

[0233] Clause 11. The method of clause 10, wherein if the motion refinement process comprises multiple steps, motion information after a last step in the motion refinement process is used in the first reordering process.

[0234] Clause 12. The method of clause 11, wherein the multiple steps refine the motion information with motion vectors of different resolutions.

[0235] Clause 13. The method of clause 12, wherein integer resolution motion vectors are refined to become fractional resolution motion vectors.

[0236] Clause 14. The method of clause 12, wherein fractional resolution motion vectors are refined to become fractional resolution motion vectors.

[0237] Clause 15. The method of clause 12, wherein the different resolutions include at least one of 16 pels, 8 pels, 4 pels, 2 pels, 1 pel, ½ pel, ¼ pel, ⅛ pel, 1 / 16 pel, 1 / 32 pel, or 1 / 64 pel.

[0238] Clause 16. The method of clause 11, wherein the multiple steps include different search shapes for refining the motion vector in template matching.

[0239] Clause 17. The method of clause 16, wherein motion information after a diamond-shaped search step of said template matching is used in said first sorting process.

[0240] Clause 18. The method of clause 11, wherein the resolution of the motion vectors used depends on a syntax element indicating the resolution of the motion vectors of the motion candidates.

[0241] Clause 19. The method of clause 11, wherein if early termination is used in the motion refinement process, motion information output associated with said early termination is used in the first reordering process.

[0242] Clause 20. The method described in clause 1, wherein multiple motion refinement processes are applied to the target block, the multiple motion refinement processes being applied in sequence, and wherein motion information derived after the last motion refinement process is used in the first reordering process.

[0243] Clause 21. The method of clause 20, wherein if the plurality of motion refinement processes includes a first motion refinement process and a second motion refinement process, motion information of the second motion refinement process is used in the first reordering process.

[0244] Clause 22. The method of clause 21, wherein the first motion refinement process includes at least one of a template matching method or a bilateral matching method.

[0245] Clause 23. The method of clause 21, wherein the second motion refinement process includes at least one of a template matching method or a bilateral matching method.

[0246] Clause 24. The method according to clause 1, wherein the processing procedure includes a motion refinement process, and the refined motion information used in the first reordering process is intermediate motion information of the motion refinement process.

[0247] Clause 25. The method of clause 24, wherein the motion refinement process includes multiple steps, and motion information prior to a last step in the motion refinement process is used in the first reordering process.

[0248] Clause 26. The method of clause 24, wherein motion information from a first step in the motion refinement process is used in the first reordering process.

[0249] Clause 27. The method of clause 24, wherein motion information at integer resolution of motion vectors is used in the first reordering process.

[0250] Clause 28. The method of clause 1, wherein if multiple motion refinement processes are applied in sequence, the motion derived from the motion refinement process preceding the last motion refinement process is used in said first reordering process.

[0251] Clause 29. The method of clause 28, wherein if the plurality of motion refinement processes includes a first motion refinement process and a second motion refinement process, motion information of the first motion refinement process is used in the first reordering process.

[0252] Clause 30. The method of clause 29, wherein the first motion refinement process includes at least one of a template matching method or a bilateral matching method.

[0253] Clause 31. The method of clause 29, wherein the second motion refinement process includes at least one of a template matching method or a bilateral matching method.

[0254] Clause 32. The method of clause 1, wherein multiple motion refinement processes are applied in sequence, the final motion refinement process comprising multiple steps, and wherein motion information derived from a step prior to the final step in the final motion refinement process is used in the first reordering process.

[0255] Clause 33. The method of clause 1, wherein the processing procedure includes a motion refinement process, the motion refinement process including template matching (TM) and multi-pass decoder-side motion vector refinement (DMVR), and the TM is between a first-pass DMVR and a second-pass DMVR.

[0256] Clause 34. The method described in Clause 33, wherein the motion information after the TM is used in the first reordering process, or the motion information after the first pass DMVR is used in the first reordering process, or the motion information after the second pass DMVR is used in the first reordering process.

[0257] Clause 35. The method of clause 33, wherein if refined motion information is used in the first reordering process, the early termination method used to skip the second pass DMVR is not applied.

[0258] Clause 36. The method of clause 1, wherein if the refined motion information is sub-block-based motion information, the refined motion information of one or more sub-blocks is used in the first reordering process.

[0259] Clause 37. The method of clause 36, wherein refined motion information of one of the one or more sub-blocks is used in the first reordering process.

[0260] Clause 38. The method of clause 36, wherein refined motion information of at least one of the top left sub-block, one of the sub-blocks located at the left block boundary, or one of the sub-blocks located at the top block boundary is used in the first reordering process.

[0261] Clause 39. The method of clause 36, wherein refined motion information of a plurality of sub-blocks of the one or more sub-blocks is used in the first reordering process.

[0262] Clause 40. The method of clause 36, wherein motion information of all sub-blocks is averaged and used in the first reordering process.

[0263] Clause 41. The method of clause 40, wherein the motion information of all sub-blocks is weighted averaged, wherein the set of weighting parameters depends on at least one of the block dimension, block size, sub-block dimension, sub-block size, or the distance between the sub-block and the template used in the first sorting process.

[0264] Clause 42. The method of clause 40, wherein at least one of motion information of sub-blocks at the top boundary and motion information of sub-blocks at the left boundary is used in the first reordering process.

[0265] Clause 43. The method of clause 1, wherein the motion candidates are divided into a first number of subgroups, and the first sorting process is applied within each subgroup, wherein the first number is greater than one.

[0266] Clause 44. The method of clause 43, wherein the number of motion candidates in each subgroup is the same.

[0267] Clause 45. The method of clause 44, wherein the number of motion candidates is one of 2, 3, 4, 5, 6, 7, 8, 9, and 10.

[0268] Clause 46. The method of clause 43, wherein the size of at least one subgroup is different from the size of the other subgroups.

[0269] Clause 47. The method of clause 43, wherein the size of the first subgroup is greater than the size of the other subgroups, or the size of the first subgroup is smaller than the size of the other subgroups.

[0270] Clause 48. The method of clause 43, wherein the size of the subgroups is predefined.

[0271] Clause 49. The method of clause 43, wherein the motion candidates are divided into the first number of subgroups based on motion candidate type.

[0272] Clause 50. The method of clause 43, wherein the size of the subgroup is determined based on the number of types of motion candidates available.

[0273] Clause 51. The method of clause 1, wherein whether and / or how to apply the first reordering process using refined motion information is indicated in the bitstream, or whether and / or how to apply the first reordering process using refined motion information is dynamically determined, or whether and / or how to apply the first reordering process using refined motion information depends on coding information.

[0274] Clause 52. The method of clause 51, wherein the first reordering process using the refined motion information is applied to the target block and the size of the target block is greater than a first threshold, or the first reordering process using the refined motion information is applied to the target block and the size of the target block is less than a second threshold.

[0275] Clause 53. The method of clause 52, wherein the first threshold is one of 8, 16, 32, 64, or 128, and the second threshold is one of 64, 128, 256, 512, 1024, or 2048.

[0276] Clause 54. The method of any one of clauses 1 to 53, wherein the target block refers to one of a color component, a sub-image, a slice, a tile, a coding tree unit (CTU), a CTU row, a group of CTUs, a coding unit (CU), a prediction unit (PU), a transform unit (TU), a coding tree block (CTB), a coding block (CB), a prediction block (PB), a transform block (TB), a block, a sub-block of a block, a sub-region within a block, or a region comprising multiple samples or pixels.

[0277] Clause 55. A method according to any one of clauses 1 to 53, wherein after the processing procedure, an instruction as to whether and / or how to reorder the motion candidate list by applying the first reordering process to the motion candidate list is indicated at one of the sequence level, the image group level, the image level, the slice level, or the tile group level.

[0278] Clause 56. A method according to any one of clauses 1 to 53, wherein after the processing procedure, an indication of whether and / or how to reorder the motion candidate list by applying the first reordering process to the motion candidate list is indicated in one of a sequence header, a picture header, a sequence parameter set (SPS), a video parameter set (VPS), a dependency parameter set (DPS), a decoding capability information (DCI), a picture parameter set (PPS), an adaptation parameter set (APS), a slice header, or a tile group header.

[0279] Clause 57. A method according to any one of clauses 1 to 53, wherein after the processing procedure, an instruction as to whether and / or how to reorder the motion candidate list by applying the first reordering process to the motion candidate list is included in one of a prediction block (PB), a transform block (TB), a coding block (CB), a prediction unit (PU), a transform unit (TU), a coding unit (CU), a virtual pipeline data unit (VPDU), a coding tree unit (CTU), a CTU row, a slice, a tile, a sub-image, or a region comprising a plurality of samples or pixels.

[0280] Clause 58. A method according to any one of clauses 1 to 53, further comprising a step of determining whether and / or how to reorder the motion candidate list by applying the first reordering process to the motion candidate list after the processing procedure based on coding information of the target block, wherein the coding information includes at least one of block size, color format, single and / or dual tree division, color component, slice type, or image type.

[0281] Clause 59. A video processing method, comprising: during conversion between a target block of a video and a bitstream of the target block, applying a reordering process using refined motion information to the target block coded in inter mode with decoder-side motion vector refinement (DMVR) to reorder at least one of a first motion candidate list or a second motion candidate list, wherein a first set of motion candidates in the first motion candidate list are refined in one direction and a second set of motion candidates in the second motion candidate list are refined in another direction; and performing the conversion based on at least one of the reordered first motion candidate list or the reordered second motion candidate list.

[0282] Clause 60. The method of clause 59, wherein the DMVR comprises at least one of a one-sided DMVR or an adaptive DMVR.

[0283] Clause 61. The method according to clause 60, wherein the one-sided DMVR is a DMVR method in which motion information is refined in one direction in the DMVR step.

[0284] Clause 62. The method of clause 61, wherein the motion information is refined unidirectionally in a first pass of a multi-pass DMVR.

[0285] Clause 63. The method of clause 61, wherein the motion information is refined unidirectionally in one or more of a first pass of a multi-path DMVR or a second pass of the multi-path DMVR.

[0286] Clause 64. The method of clause 59, wherein the inter-mode is an adaptive DMVR.

[0287] Clause 65. The method of clause 59, wherein the reordering process is applied to reorder the first motion candidate list and the second motion candidate list separately.

[0288] Clause 66. The method described in Clause 59, wherein the first motion candidate list and the second motion candidate list are combined, and wherein the sorting process is used to sort the combined first motion candidate list and the second motion candidate list.

[0289] Clause 67. The method of clause 59, wherein the refined motion information is a block-based motion vector (MV) of a first pass in a multi-pass DMVR.

[0290] Clause 68. The method of any one of clauses 59 to 67, wherein the target block refers to one of a color component, a sub-image, a slice, a tile, a coding tree unit (CTU), a CTU row, a group of CTUs, a coding unit (CU), a prediction unit (PU), a transform unit (TU), a coding tree block (CTB), a coding block (CB), a prediction block (PB), a transform block (TB), a block, a sub-block of a block, a sub-region within a block, or a region comprising multiple samples or pixels.

[0291] Clause 69. A method according to any one of clauses 59 to 67, wherein the instruction as to whether and / or how to apply the reordering process using the refined motion information to the target block coded in inter mode by the DMVR is indicated at one of the sequence level, group of pictures level, picture level, slice level, or tile group level.

[0292] Clause 70. A method according to any one of clauses 59 to 67, wherein an indication of whether and / or how to apply the reordering process using the refined motion information to the target block coded in inter mode by the DMVR is indicated in one of a sequence header, a picture header, a sequence parameter set (SPS), a video parameter set (VPS), a dependency parameter set (DPS), a decoding capability information (DCI), a picture parameter set (PPS), an adaptation parameter set (APS), a slice header, or a tile group header.

[0293] Clause 71. A method according to any one of clauses 59 to 67, wherein the instruction as to whether and / or how to apply the reordering process using the refined motion information to the target block coded in inter mode by the DMVR is included in one of a prediction block (PB), a transform block (TB), a coding block (CB), a prediction unit (PU), a transform unit (TU), a coding unit (CU), a virtual pipeline data unit (VPDU), a coding tree unit (CTU), a CTU row, a slice, a tile, a sub-image, or a region comprising multiple samples or pixels.

[0294] Clause 72. A method according to any one of clauses 59 to 67, further comprising a step of determining whether and / or how to apply the reordering process using the refined motion information to the target block coded in inter mode by the DMVR based on coding information of the target block, the coding information including at least one of block size, color format, single and / or dual tree division, color component, slice type, or image type.

[0295] Clause 73. A video processing method, comprising: during conversion between a target block of a video and a bitstream of the target block, updating at least one motion candidate list for adaptive decoder-side motion vector refinement (DMVR) by removing a set of redundant motion candidates from the at least one motion candidate list; and performing the conversion based on the at least one updated motion candidate list.

[0296] Clause 74. The method of clause 73, wherein the set of redundant motion candidates includes motion candidates in the first motion candidate list that have the same motion information as one of the motion candidates in the second motion candidate list.

[0297] Clause 75. The method of clause 74, wherein the first motion candidate list is a first direction motion candidate list and the second motion candidate list is a second direction motion candidate list.

[0298] Clause 76. The method of clause 74, wherein the first motion candidate list is a second direction motion candidate list and the second motion candidate list is a first direction motion candidate list.

[0299] Clause 77. The method of clause 73, wherein the set of redundant motion candidates includes motion candidates after a first pass DMVR.

[0300] Clause 78. The method of any one of clauses 73 to 77, wherein the target block refers to one of a color component, a sub-image, a slice, a tile, a coding tree unit (CTU), a CTU row, a group of CTUs, a coding unit (CU), a prediction unit (PU), a transform unit (TU), a coding tree block (CTB), a coding block (CB), a prediction block (PB), a transform block (TB), a block, a sub-block of a block, a sub-region within a block, or a region comprising multiple samples or pixels.

[0301] Clause 79. A method according to any one of clauses 73 to 77, wherein the instruction as to whether and / or how to remove the set of redundant motion candidates from at least one motion candidate list for the adaptive DMVR is indicated at one of the sequence level, group of pictures level, image level, slice level, or tile group level.

[0302] Clause 80. A method according to any one of clauses 73 to 77, wherein an instruction as to whether and / or how to remove the set of redundant motion candidates from at least one motion candidate list for the adaptive DMVR is indicated in one of a sequence header, a picture header, a sequence parameter set (SPS), a video parameter set (VPS), a dependency parameter set (DPS), a decoding capability information (DCI), a picture parameter set (PPS), an adaptation parameter set (APS), a slice header, or a tile group header.

[0303] Clause 81. The method of any one of clauses 73 to 77, wherein the instructions for whether and / or how to remove the set of redundant motion candidates from at least one motion candidate list for the adaptive DMVR are included in one of a prediction block (PB), a transform block (TB), a coding block (CB), a prediction unit (PU), a transform unit (TU), a coding unit (CU), a virtual pipeline data unit (VPDU), a coding tree unit (CTU), a CTU row, a slice, a tile, a sub-image, or a region containing multiple samples or pixels.

[0304] Clause 82. The method of any one of clauses 73 to 77, further comprising determining whether and / or how to remove the set of redundant motion candidates from at least one motion candidate list for the adaptive DMVR based on coding information of the target block, the coding information including at least one of block size, color format, single and / or dual tree partitioning, color component, slice type, or image type.

[0305] Clause 83. A video processing method, comprising: during conversion between a target block of a video and a bitstream of the target block, generating a first number of motion candidates for a motion candidate list associated with the target block, wherein a second number of motion candidates is allowed for the motion candidate list, the first number being greater than the second number; and performing the conversion based on the motion candidate list.

[0306] Clause 84. The method of clause 83, wherein the second number of motion candidates are selected and used for prediction or reconstruction.

[0307] Clause 85. The method described in Clause 83, wherein the second number of motion candidates is selected by sorting the first number of motion candidates, wherein the selected second number of motion candidates are the first M motion candidates of the first number of motion candidates after the sorting, and M is the second number.

[0308] Clause 86. The method of clause 83, wherein the first number is predefined, or the first number is indicated in the bitstream, or the first number is dynamically determined.

[0309] Clause 87. The method of clause 83, wherein the first number may be different for each coding mode.

[0310] Clause 88. The method of clause 83, wherein the first number is one of 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, or 20.

[0311] Clause 89. The method of clause 83, wherein motion information derived from the motion candidate list is used in a reordering process.

[0312] Clause 90. The method of any one of clauses 83 to 89, further comprising applying a reordering process using refined motion information to said motion candidate list.

[0313] Clause 91. The method of any one of clauses 83 to 90, wherein the target block refers to one of a color component, a sub-image, a slice, a tile, a coding tree unit (CTU), a CTU row, a group of CTUs, a coding unit (CU), a prediction unit (PU), a transform unit (TU), a coding tree block (CTB), a coding block (CB), a prediction block (PB), a transform block (TB), a block, a sub-block of a block, a sub-region within a block, or a region comprising multiple samples or pixels.

[0314] Clause 92. A method according to any one of clauses 83 to 90, wherein the instruction as to whether and / or how to generate the first number of motion candidates for the motion candidate list associated with the target block is indicated at one of the sequence level, group of pictures level, image level, slice level, or tile group level.

[0315] Clause 93. A method according to any one of clauses 83 to 90, wherein the instruction as to whether and / or how to generate the first number of motion candidates for the motion candidate list associated with the target block is indicated in one of a sequence header, a picture header, a sequence parameter set (SPS), a video parameter set (VPS), a dependency parameter set (DPS), a decoding capability information (DCI), a picture parameter set (PPS), an adaptation parameter set (APS), a slice header, or a tile group header.

[0316] Clause 94. A method according to any one of clauses 83 to 90, wherein the instructions on whether and / or how to generate the first number of motion candidates for the motion candidate list associated with the target block are included in one of a prediction block (PB), a transform block (TB), a coding block (CB), a prediction unit (PU), a transform unit (TU), a coding unit (CU), a virtual pipeline data unit (VPDU), a coding tree unit (CTU), a CTU row, a slice, a tile, a sub-image, or a region containing multiple samples or pixels.

[0317] Clause 95. The method of any one of clauses 83 to 90, further comprising determining whether and / or how to generate the first number of motion candidates for the motion candidate list associated with the target block based on coding information of the target block, the coding information including at least one of block size, color format, single and / or dual tree partitioning, color component, slice type, or image type.

[0318] Clause 96. A video processing method, during conversion between a target block of a video and a bitstream of said target block, comprising the steps of generating cost values ​​in a motion candidate list refinement process for said target block, applying a reordering process to said motion candidate list using said cost values, and performing said conversion based on said motion candidate list.

[0319] Clause 97. The method of clause 96, wherein the refinement process includes at least one of template matching (TM), decoder-side motion vector refinement (DMVR), or multi-pass DMVR.

[0320] Clause 98. The method of clause 96, wherein final cost values ​​generated in the refinement process for the motion candidates are saved.

[0321] Clause 99. The method of clause 96, wherein in the reordering process, a first final cost value generated in a first refinement process for a first motion candidate is compared with a second final cost value generated in a second refinement process for a second motion candidate.

[0322] Clause 100. The method of any one of clauses 96 to 99, wherein the target block refers to one of a color component, a sub-image, a slice, a tile, a coding tree unit (CTU), a CTU row, a group of CTUs, a coding unit (CU), a prediction unit (PU), a transform unit (TU), a coding tree block (CTB), a coding block (CB), a prediction block (PB), a transform block (TB), a block, a sub-block of a block, a sub-region within a block, or a region comprising multiple samples or pixels.

[0323] Clause 101. A method according to any one of clauses 96 to 99, wherein the instruction as to whether and / or how to apply the reordering process to the motion candidate list using the cost values ​​is indicated at one of the sequence level, group of images level, image level, slice level, or tile group level.

[0324] Clause 102. A method according to any one of clauses 96 to 99, wherein an indication of whether and / or how to apply the reordering process to the motion candidate list using the cost values ​​is indicated in one of a sequence header, a picture header, a sequence parameter set (SPS), a video parameter set (VPS), a dependency parameter set (DPS), a decoding capability information (DCI), a picture parameter set (PPS), an adaptation parameter set (APS), a slice header, or a tile group header.

[0325] Clause 103. The method of any one of clauses 96 to 99, wherein the instructions for whether and / or how to apply the reordering process to the motion candidate list using the cost values ​​are included in one of a prediction block (PB), a transform block (TB), a coding block (CB), a prediction unit (PU), a transform unit (TU), a coding unit (CU), a virtual pipeline data unit (VPDU), a coding tree unit (CTU), a CTU row, a slice, a tile, a sub-image, or a region containing multiple samples or pixels.

[0326] Clause 104. The method of any one of clauses 96 to 99, further comprising a step of determining whether and / or how to apply the reordering process to the motion candidate list using the cost value based on coding information of the target block, the coding information including at least one of block size, color format, single and / or dual tree partitioning, color component, slice type, or image type.

[0327] Clause 105. A video processing method, comprising: during conversion between a target block of a video and a bitstream of the target block, sorting a set of merge candidates for the target block according to a set of merge types; constructing an adaptive decoder-side motion vector refinement (DMVR) merge candidate list using the first N merge candidates of each of the set of merge types, where N is an integer; and performing the conversion based on the constructed adaptive DMVR merge candidate list.

[0328] Clause 106. The method of clause 105, wherein the set of merge types is at least one of adjacent spatial, temporal, non-adjacent spatial, history-based motion vector prediction (HMVP), or pairwise.

[0329] Clause 107. The method of clause 105, wherein the value of N is different for different merge types.

[0330] Clause 108. The method of clause 105, wherein N is set equal to 1 or 2 for temporal merge candidates.

[0331] Clause 109. The method of clause 105, wherein for spatial merge candidates, N is set equal to one of 1, or 2, or 3, or 4, or 5, or 6, or 7, or 8, or 9, or 10.

[0332] Clause 110. The method of clause 105, wherein for non-adjacent spatial merge candidates, N is set equal to one of 1, or 2, or 3, or 4, or 5, or 6, or 7, or 8, or 9, or 10.

[0333] Clause 111. The method of clause 105, wherein for HMVP merge candidates, N is set equal to one of 1, or 2, or 3, or 4, or 5, or 6, or 7, or 8, or 9, or 10.

[0334] Clause 112. The method of Clause 105, wherein for pairwise merge candidates, N is set equal to one of 1, or 2, or 3, or 4, or 5, or 6, or 7, or 8, or 9, or 10.

[0335] Clause 113. The method of any one of clauses 105 to 112, wherein the target block refers to one of a color component, a sub-image, a slice, a tile, a coding tree unit (CTU), a CTU row, a group of CTUs, a coding unit (CU), a prediction unit (PU), a transform unit (TU), a coding tree block (CTB), a coding block (CB), a prediction block (PB), a transform block (TB), a block, a sub-block of a block, a sub-region within a block, or a region comprising multiple samples or pixels.

[0336] Clause 114. The method of any one of clauses 105 to 112, wherein the instruction as to whether and / or how to construct the adaptive DMVR merge candidate list using the first N merge candidates of each of the set of merge types is indicated at one of the sequence level, picture group level, picture level, slice level, or tile group level.

[0337] Clause 115. The method of any one of clauses 105 to 112, wherein an instruction as to whether and / or how to construct the adaptive DMVR merge candidate list using the first N merge candidates of each of the set of merge types is indicated in one of a sequence header, a picture header, a sequence parameter set (SPS), a video parameter set (VPS), a dependency parameter set (DPS), a decoding capability information (DCI), a picture parameter set (PPS), an adaptation parameter set (APS), a slice header, or a tile group header.

[0338] Clause 116. The method of any one of clauses 105 to 112, wherein the instructions for whether and / or how to construct the adaptive DMVR merge candidate list using the first N merge candidates of each of the set of merge types are included in one of a prediction block (PB), a transform block (TB), a coding block (CB), a prediction unit (PU), a transform unit (TU), a coding unit (CU), a virtual pipeline data unit (VPDU), a coding tree unit (CTU), a CTU row, a slice, a tile, a sub-image, or a region containing multiple samples or pixels.

[0339] Clause 117. The method of any one of clauses 105 to 112, further comprising determining whether and / or how to construct the adaptive DMVR merge candidate list using the first N merge candidates of each of the set of merge types based on coding information of the target block, wherein the coding information includes at least one of block size, color format, single and / or dual tree partitioning, color component, slice type, or image type.

[0340] Clause 118. The method of any one of clauses 1 to 117, wherein the converting comprises encoding the target block into the bitstream.

[0341] Clause 119. The method of any one of clauses 1 to 117, wherein the converting comprises decoding the target block from the bitstream.

[0342] Clause 120. Apparatus for processing video data comprising a processor and a non-transitory memory comprising instructions which, when executed by the processor, cause the processor to perform a method according to any one of clauses 1 to 119.

[0343] Clause 121. A non-transitory computer-readable storage medium storing instructions that cause a processor to perform the method of any one of clauses 1 to 119.

[0344] Clause 122. A non-transitory computer-readable recording medium storing a video bitstream generated by a method executed by a video processing device, the method comprising: applying a processing procedure to at least one motion candidate in a motion candidate list for the target block; after the processing procedure, reordering the motion candidate list by applying a first reordering process to the motion candidate list; and generating a bitstream for the target block based on the reordered motion candidate list.

[0345] Clause 123. A method for storing a video bitstream, comprising: applying a processing procedure to at least one motion candidate in a motion candidate list for the target block; after the processing procedure, reordering the motion candidate list by applying a first reordering process to the motion candidate list; generating a bitstream for the target block based on the reordered motion candidate list; and storing the bitstream on a non-transitory computer-readable recording medium.

[0346] Clause 124. A non-transitory computer-readable recording medium storing a video bitstream generated by a method executed by a video processing device, the method comprising: applying a reordering process using refined motion information to a target block of video coded in inter mode with decoder-side motion vector refinement (DMVR) to reorder at least one of a first motion candidate list or a second motion candidate list, wherein a first set of motion candidates in the first motion candidate list are refined in one direction and a second set of motion candidates in the second motion candidate list are refined in another direction; and generating a bitstream of the target block based on at least one of the reordered first motion candidate list or the reordered second motion candidate list.

[0347] Clause 125. A method for storing a video bitstream, comprising: applying a reordering process using refined motion information to a target block of video coded in inter mode with decoder-side motion vector refinement (DMVR) to reorder at least one of a first motion candidate list or a second motion candidate list, wherein a first set of motion candidates in the first motion candidate list are refined in one direction and a second set of motion candidates in the second motion candidate list are refined in another direction; generating a bitstream for the target block based on at least one of the reordered first motion candidate list or the reordered second motion candidate list; and storing the bitstream on a non-transitory computer-readable recording medium.

[0348] Clause 126. A non-transitory computer-readable recording medium storing a bitstream of video generated by a method executed by a video processing device, the method comprising: updating at least one motion candidate list for adaptive decoder-side motion vector refinement (DMVR) of a target block of the video by removing a set of redundant motion candidates from the at least one motion candidate list; and generating a bitstream of the target block based on the at least one updated motion candidate list.

[0349] Clause 127. A method for storing a bitstream of a video, comprising: updating at least one motion candidate list for an adaptive decoder-side motion vector refinement (DMVR) of a target block of the video by removing a set of redundant motion candidates from the at least one motion candidate list; generating a bitstream of the target block based on the at least one updated motion candidate list; and storing the bitstream in a non-transitory computer-readable recording medium.

[0350] Clause 128. A non-transitory computer-readable recording medium storing a bitstream of video generated by a method executed by a video processing device, the method including the steps of generating a first number of motion candidates for a motion candidate list associated with a target block of the video, wherein a second number of motion candidates is allowed for the motion candidate list, the first number being greater than the second number, and generating a bitstream of the target block based on the motion candidate list.

[0351] Clause 129. A method for storing a bitstream of a video, the method comprising: generating a first number of motion candidates for a motion candidate list associated with a target block of the video, wherein a second number of motion candidates is allowed for the motion candidate list, the first number being greater than the second number; generating a bitstream for the target block based on the motion candidate list; and storing the bitstream on a non-transitory computer-readable recording medium.

[0352] Clause 130. A non-transitory computer-readable recording medium storing a bitstream of video generated by a method executed by a video processing device, the method including the steps of generating cost values ​​in a motion candidate list refinement process for a target block of the video, applying a reordering process to the motion candidate list using the cost values, and generating a bitstream for the target block based on the motion candidate list.

[0353] Clause 131. A method for storing a bitstream of a video, comprising the steps of generating cost values ​​in a motion candidate list refinement process for a target block of the video, applying a reordering process to the motion candidate list using the cost values, generating a bitstream for the target block based on the motion candidate list, and storing the bitstream in a non-transitory computer-readable recording medium.

[0354] Clause 132. A non-transitory computer-readable recording medium storing a bitstream of a video generated by a method executed by a video processing device, the method comprising: sorting a set of merge candidates for a target block of the video according to a set of merge types; constructing an adaptive decoder-side motion vector refinement (DMVR) merge candidate list using a first N merge candidates of each of the set of merge types, where N is an integer; and generating a bitstream for the target block based on the constructed adaptive DMVR merge candidate list.

[0355] Clause 133. A method for storing a bitstream of a video, comprising: sorting a set of merge candidates for a target block of the video according to a set of merge types; constructing an adaptive decoder-side motion vector refinement (DMVR) merge candidate list using the first N merge candidates of each of the set of merge types, where N is an integer; generating a bitstream for the target block based on the constructed adaptive DMVR merge candidate list; and storing the bitstream on a non-transitory computer-readable recording medium. Exemplary Devices

[0356] 47 shows a block diagram of a computing device 4700 capable of implementing various embodiments of the present disclosure. The computing device 4700 may be embodied as or included in a source device 110 (or a video encoder 114 or 200) or a destination device 120 (or a video decoder 124 or 300).

[0357] It will be understood that the computing device 4700 shown in FIG. 47 is for illustrative purposes only and is not intended to limit in any way the functionality and scope of the embodiments of the present disclosure.

[0358] 47, the computing device 4700 includes a general-purpose computing device 4700. The computing device 4700 may include at least one or more processors or processing units 4710, a memory 4720, a storage unit 4730, one or more communication units 4740, one or more input devices 4750, and one or more output devices 4760.

[0359] In some embodiments, computing device 4700 may be embodied as any user terminal or server terminal having computing capabilities. The server terminal may be a server provided by a service provider, a large-scale computing device, or the like. The user terminal may be any type of mobile, fixed, or portable terminal, including, for example, a mobile phone, a station, a unit, a device, a multimedia computer, a multimedia tablet, an Internet node, a communicator, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio receiver, an electronic book device, a gaming device, or any combination thereof (including accessories and peripherals of these devices, or any combination thereof). It is contemplated that computing device 4700 may support any type of interface to a user (e.g., “wearable” circuitry, etc.).

[0360] The processing unit 4710 may be a physical or virtual processor and may implement various processes based on programs stored in memory 4720. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to increase the parallel processing capabilities of the computing device 4700. The processing unit 4710 may also be referred to as a central processing unit (CPU), a microprocessor, a controller, or a microcontroller.

[0361] Computing device 4700 typically includes a variety of computer storage media. Such media may be any media accessible by computing device 4700, including, but not limited to, volatile and nonvolatile media, or removable and non-removable media. Memory 4720 may be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or any combination thereof. Storage unit 4730 may be any removable or non-removable media, including machine-readable media such as memory, flash memory drives, magnetic disks, or other media, that can be used to store information and / or data and that can be accessed by computing device 4700.

[0362] The computing device 4700 may further include additional removable / non-removable, volatile / non-volatile memory media. Although not shown in Figure 47, it is possible to provide a magnetic disk drive that reads from and writes to a removable non-volatile magnetic disk and an optical disk drive that reads from and writes to a removable non-volatile optical disk. In such cases, each drive may be connected to a bus (not shown) via one or more data medium interfaces.

[0363] The communications unit 4740 communicates with additional computing devices via a communications medium. Furthermore, the functionality of the components within the computing device 4700 may be embodied by a single computing cluster or multiple computing machines that can communicate via communications connections. Thus, the computing device 4700 can operate in a networked environment using logical connections with one or more other servers, networked personal computers (PCs), or additional general network nodes.

[0364] The input device(s) 4750 may be one or more of a variety of input devices such as a mouse, keyboard, tracking ball, audio input device, etc. The output device(s) 4760 may be one or more of a variety of output devices such as a display, loudspeaker, printer, etc. The communication unit 4740 enables the computing device 4700 to further communicate with one or more external devices (not shown), such as storage devices and display devices, one or more devices that allow a user to interact with the computing device 4700, or any device (e.g., network card, modem, etc.) that allows the computing device 4700 to communicate with one or more other computing devices, as needed. Such communication may be performed via input / output (I / O) interfaces (not shown).

[0365] In some embodiments, instead of being integrated into a single device, some or all of the components of computing device 4700 may be located in a cloud computing architecture. In a cloud computing architecture, components may be provided remotely and work together to perform the functions described in this disclosure. In some embodiments, cloud computing provides computing, software, data access, and storage services without requiring end users to be aware of the physical location or configuration of the systems or hardware providing these services. In various embodiments, cloud computing provides services over a wide area network (e.g., the Internet) using appropriate protocols. For example, a cloud computing provider may provide applications over a wide area network that can be accessed through a web browser or other computing component. Software or components of a cloud computing architecture and corresponding data may be stored on servers in remote locations. Computing resources in a cloud computing environment may be consolidated or distributed at remote data center locations. A cloud computing infrastructure may act as a single access point for users but provide services through a shared data center. Thus, a cloud computing architecture may be used to provide the components and functions described herein from a service provider in a remote location. Alternatively, they may be provided from traditional servers or installed directly or otherwise on client devices.

[0366] The computing device 4700 may be used to implement video encoding / decoding in embodiments of the present disclosure. The memory 4720 may include one or more video encoding modules 4725 having one or more program instructions. These modules are accessible and executable by the processing unit 4710 to perform the functions of various embodiments described herein.

[0367] In an example embodiment that performs video encoding, input device 4750 may receive video data to be encoded as input 4770. The video data may be processed, for example, by video coding module 4725 to generate an encoded bitstream. The encoded bitstream may be provided as output 4780 via output device 4760.

[0368] In an example embodiment that performs video decoding, input device 4750 may receive an encoded bitstream as input 4770. The encoded bitstream may be processed, for example, by video coding module 4725 to generate decoded video data. The decoded video data may be provided as output 4780 via output device 4760.

[0369] While the present disclosure has been particularly shown and described with reference to preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the present application as defined by the appended claims. Such variations are intended to be within the scope of the present application. Accordingly, the foregoing description of the embodiments of the present application is not intended to be limiting.

Claims

1. 1. A video processing method comprising: applying a procedure to at least one motion candidate in a motion candidate list during conversion between a target block of a video and a bitstream of the video; after said processing procedure, reordering said motion candidate list by applying a first reordering process to said motion candidate list; performing the conversion based on the sorted motion candidate list; Including, A method wherein if at least one motion candidate in the motion candidate list is modified by a motion refinement process for the target block, the motion candidate list is reordered using refined motion information, or the motion candidate list is not reordered before the first reordering process is applied.

2. The processing procedure is as follows: Correction process, Movement elaboration process, or Second Sorting Process The method of claim 1 , comprising at least one of:

3. The method of claim 2 , wherein the second reordering process is applied to the motion candidate list before the first reordering process.

4. The method of claim 3 , wherein the motion candidate list is sorted using original motion information during construction of the motion candidate list in the second sorting process.

5. The motion refinement process comprises: template matching based motion refinement, or The method of claim 2 , including at least one of bilateral matching based motion refinement.

6. 3. The method of claim 2, wherein a simplified version of the motion refinement process is applied, and when template matching is used as the motion refinement process, the simplified version of the motion refinement process includes setting a template size in the template matching to 1 or 2.

7. the procedure is a motion refinement process; The method of claim 1 , wherein the refined motion information used in the first reordering process is the final motion information of the motion refinement process.

8. The method of claim 7 , wherein if the motion refinement process includes multiple steps, the motion information after the last step in the motion refinement process is used in the first reordering process.

9. The method of claim 8 , wherein the steps refine the motion information with motion vectors of different resolutions.

10. Integer resolution motion vectors are refined to become fractional resolution motion vectors, or The fractional resolution motion vectors are refined to become fractional resolution motion vectors, or 10. The method of claim 9, wherein the different resolutions include at least one of 16 pels, 8 pels, 4 pels, 2 pels, 1 pel, ½ pel, ¼ pel, ⅛ pel, 1 / 16 pel, 1 / 32 pel, or 1 / 64 pel.

11. The method of claim 8 , wherein the steps include different search shapes for refining the motion vectors in template matching.

12. The method of claim 11 , wherein motion information after a diamond-shaped search step of the template matching is used in the first sorting process.

13. The resolution of the motion vectors used depends on the syntax element indicating the resolution of the motion vectors of the motion candidates, or The method of claim 8 , wherein if early termination is used in the motion refinement process, motion information output associated with the early termination is used in the first reordering process.

14. A plurality of motion refinement processes are applied to the target block, the plurality of motion refinement processes being applied in sequence; The method of claim 1 , wherein motion information derived after a final motion refinement process is used in the first reordering process.

15. the conversion comprises encoding the target block into the bitstream, or The method of claim 1 , wherein the conversion comprises decoding the target block from the bitstream.

16. 1. An apparatus for processing video data comprising a processor and a non-transitory memory with instructions, Apparatus, wherein the instructions, when executed by the processor, cause the processor to perform the method of any one of claims 1 to 15.

17. A non-transitory computer readable storage medium storing instructions that cause a processor to perform the method of any one of claims 1 to 15.

18. 1. A method for storing a video bitstream, comprising: applying a procedure to at least one motion candidate in a motion candidate list for the target block; after said processing procedure, reordering said motion candidate list by applying a first reordering process to said motion candidate list; generating the bitstream based on the reordered motion candidate list; storing the bitstream on a non-transitory computer-readable recording medium; A method wherein if at least one motion candidate in the motion candidate list is modified by a motion refinement process for the target block, the motion candidate list is reordered using refined motion information, or the motion candidate list is not reordered before the first reordering process is applied.

Images