Method and device for video processing and medium

Abstract

Embodiments of the present disclosure provide a solution for video processing. A method for video processing is presented. The method includes determining, for a conversion between a video unit of a video and a bitstream of the video, that the video unit is coded and decoded by a motion pair from the same direction relative to a current picture, where the video unit is applied with an advanced motion vector prediction (AMVP) merge mode and the video unit belongs to a low latency picture; and performing a conversion based on the motion pair.

Description

Technical Field

[0001] Embodiments of the present disclosure generally relate to video processing techniques, and more specifically, to local illumination compensation (LIC), advanced motion vector prediction (AMVP) merging (MERGE), decoder-side motion vector refinement (DMVR), out of bounds (OOB), and bidirectional prediction (BCW) with codec unit (CU) level weights in image / video codecs. Background Art

[0002] Nowadays, digital video capabilities are being applied to all aspects of people's lives. Various types of video compression technologies, such as MPEG-2, MPEG-4, ITU-TH.263, ITU-TH.264 / MPEG-4 Part 10 Advanced Video Codec (AVC), ITU-TH.265 High Efficiency Video Codec (HEVC) standard, and Versatile Video Codec (VVC) standard have been proposed for video encoding / decoding. However, the encoding and decoding efficiency of video encoding and decoding technologies is generally expected to be further improved. Summary of the invention

[0003] Embodiments of the present disclosure provide a solution for video processing.

[0004] In a first aspect, a method for video processing is proposed. The method includes: for conversion between a video unit of a video and a bitstream of the video, determining that the video unit is encoded and decoded by a motion pair from the same direction relative to a current picture, wherein the video unit is applied with an advanced motion vector prediction (AMVP) merge mode, and the video unit belongs to a low-latency picture; and performing conversion based on the motion pair. Compared with conventional techniques, the method according to the first aspect of the present disclosure improves encoding and decoding efficiency.

[0005] In a second aspect, another method for video processing is proposed. The method includes: for conversion between a video unit of a video and a bitstream of the video, applying decoder-side motion vector refinement (DMVR) or at least one of a variant of DMVR to the video unit to refine the motion vector of the video unit, wherein the video unit is applied with one of the following: affine advanced motion vector prediction (AMVP) mode, sub-block-based temporal motion vector prediction (sbTMVP) mode or affine merge mode; and performing conversion based on the refined motion vector. Compared with conventional techniques, the method according to the first aspect of the present disclosure improves encoding and decoding efficiency.

[0006] In a third aspect, another method for video processing is proposed. The method includes: during conversion between a video unit of a video and a bitstream of the video, deriving at least one LIC model parameter of a local illumination compensation (LIC) model for the video unit based on a nonlinear model, wherein the coded video unit is a block coded by LIC; updating the LIC model by adjusting at least one LIC model parameter; and performing conversion based on the updated LIC model. Compared with conventional techniques, the method according to the first aspect of the present disclosure improves coding efficiency.

[0007] In a fourth aspect, another method for video processing is proposed. The method includes: for conversion between a video unit of a video and a bitstream of the video, applying a non-average weighting method to the video unit, wherein the video unit is applied with an advanced motion vector prediction (AMVP) merge mode; and performing conversion based on the weighted video unit. Compared with conventional techniques, the method according to the first aspect of the present disclosure improves encoding and decoding efficiency.

[0008] In a fifth aspect, a device for video processing is provided. The device includes a processor and a non-volatile memory having instructions thereon. The instructions, when executed by the processor, cause the processor to perform a method according to any one of the first aspect, the second aspect, the third aspect, or the fourth aspect of the present disclosure.

[0009] In a sixth aspect, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores a method for causing a processor to execute any one of the first aspect, the second aspect, the third aspect, or the fourth aspect of the present disclosure.

[0010] In a seventh aspect, another non-transitory computer-readable recording medium is provided. The non-transitory computer-readable recording medium stores a bitstream of a video generated by a method performed by an apparatus for video processing. The method includes: determining that a video unit of a video is encoded and decoded by motion pairs from the same direction relative to a current picture, wherein the video unit is applied with an advanced motion vector prediction (AMVP) merge mode, and the video unit belongs to a low-latency picture; and generating a bitstream of the video unit based on the motion pairs.

[0011] In an eighth aspect, a method for storing a bitstream of a video is provided. The method includes: determining that a video unit of the video is encoded and decoded by a motion pair from the same direction relative to a current picture, wherein the video unit is applied with an advanced motion vector prediction (AMVP) merge mode, and the video unit belongs to a low-latency picture; generating a bitstream of the video unit based on the motion pair; and storing the bitstream in a non-transitory computer-readable recording medium.

[0012] In a ninth aspect, another non-transitory computer-readable recording medium is provided. The non-transitory computer-readable recording medium stores a bitstream of a video generated by a method performed by an apparatus for video processing. The method includes: applying decoder-side motion vector refinement (DMVR) or at least one of a variant of DMVR to a video unit of the video to refine the motion vector of the video unit, wherein the video unit is applied with one of the following: an affine advanced motion vector prediction (AMVP) mode, a sub-block-based temporal motion vector prediction (sbTMVP) mode, or an affine merge mode; and generating a bitstream of the video unit based on the refined motion vector.

[0013] In a tenth aspect, a method for storing a bitstream of a video is provided. The method includes: applying decoder-side motion vector refinement (DMVR) or at least one of a variant of DMVR to a video unit of the video to refine a motion vector of the video unit, wherein the video unit is applied with one of the following: an affine advanced motion vector prediction (AMVP) mode, a sub-block-based temporal motion vector prediction (sbTMVP) mode, or an affine merge mode; generating a bitstream of the video unit based on the refined motion vector; and storing the bitstream in a non-transitory computer-readable recording medium.

[0014] In an eleventh aspect, another non-transitory computer-readable recording medium is provided. The non-transitory computer-readable recording medium stores a bitstream of a video generated by a method performed by an apparatus for video processing. The method includes: deriving at least one LIC model parameter of a local illumination compensation (LIC) model for a video unit of the video based on a nonlinear model, wherein the coded video unit is a LIC coded block; updating the LIC model by adjusting at least one LIC model parameter; and generating a bitstream of the video unit based on the updated LIC model.

[0015] In a twelfth aspect, a method for storing a bitstream of a video is provided, the method comprising: deriving at least one LIC model parameter of a local illumination compensation (LIC) model for a video unit of the video based on a nonlinear model, wherein the coded video unit is a LIC coded block; updating the LIC model by adjusting at least one LIC model parameter; generating a bitstream of the video unit based on the updated LIC model; and storing the bitstream in a non-transitory computer-readable recording medium.

[0016] In a thirteenth aspect, another non-transitory computer-readable recording medium is provided. The non-transitory computer-readable recording medium stores a bitstream of a video generated by a method performed by an apparatus for video processing. The method includes: applying a non-average weighting method to a video unit of the video, wherein the video unit is applied with an advanced motion vector prediction (AMVP) merge mode; and generating a bitstream of the video unit based on the weighted video unit.

[0017] In a fourteenth aspect, a method for storing a bitstream of a video is provided. The method includes: applying a non-average weighting method to a video unit of the video, wherein the video unit is applied with an advanced motion vector prediction (AMVP) merge mode; generating a bitstream of the video unit based on the weighted video unit; and storing the bitstream in a non-transitory computer-readable recording medium.

[0018] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become more apparent through the following detailed description with reference to the accompanying drawings. In the exemplary embodiments of the present disclosure, the same reference numerals generally refer to the same components.

[0020] Figure 1 illustrates a block diagram illustrating an example video encoding and decoding system according to some embodiments of the present disclosure;

[0021] Figure 2 illustrates a block diagram illustrating a first example video encoder according to some embodiments of the present disclosure;

[0022] Figure 3 illustrates a block diagram illustrating an example video decoder according to some embodiments of the present disclosure;

[0023] Figure 4 The positions of the spatial merging candidates are illustrated;

[0024] Figure 5 illustrates candidate pairs considered for redundancy check of spatial merging candidates;

[0025] Figure 6 is an illustration of motion vector scaling for temporal merging candidates;

[0026] Figure 7 The temporal merging candidate C is shown 0 and C 1 Candidate location of

[0027] Figure 8 The MMVD search point is shown;

[0028] Fig. 9 shows the extended CU area used in BDOF;

[0029] Fig.10 It is a diagram for the symmetric MVD mode;

[0030] Fig.11 An affine motion model based on control points is shown;

[0031] Fig.12 The affine MVF of each sub-block is shown;

[0032] Fig.13 The position of the inherited affine motion predictor is shown;

[0033] Fig.14 Control point motion vector inheritance is shown;

[0034] Fig.15 The positions of candidate positions for the constructed affine merge pattern are shown;

[0035] Fig.16 is a diagram of the use of motion vectors for the proposed combination method;

[0036] Fig.17 Sub-block MV VSB and pixel Δv(i,j) are shown;

[0037] Fig.18a and Fig.18b The SbTMVP process in VVC is shown in the figure, where Fig.18a illustrates the spatial neighborhood blocks used by ATVMP, and Fig.18b illustrates the derivation of a sub-CU motion field by applying motion shifts from spatial neighbors and scaling motion information from corresponding co-located sub-CUs;

[0038] Fig.19 The extended CU area used in BDOF is shown.

[0039] Fig. 20 Decoding side motion vector refinement is shown;

[0040] Fig.21 The upper and left neighboring blocks used in the derivation of CIIP weights are shown;

[0041] Fig. 22 An example of a GPM split grouped by the same angle is shown;

[0042] Fig.23shows the unidirectional prediction MV selection for geometric partitioning mode;

[0043] Fig.24 Illustration of the warp weight w using the geometry partitioning pattern 0 Example generation;

[0044] Fig.25 The spatial neighboring blocks used to derive spatial merge candidates are shown;

[0045] Fig.26 shows the template matching performed on the search area around the initial MV;

[0046] Fig. 27 A diamond-shaped area in the search area is shown;

[0047] Fig.28 The frequency response of the interpolation filter and the VVC interpolation filter at a half-pixel phase is shown;

[0048] Fig.29 A template and a reference sample of the template in a reference picture are shown;

[0049] Fig.30 A template and a reference sample of the template for a block with sub-block motion using motion information of a sub-block of a current block are shown;

[0050] Fig.31 The diagram shows the bidirectional MC block in the current ECM;

[0051] Fig.32 A flowchart of a method for video processing according to an embodiment of the present disclosure is illustrated;

[0052] Fig.33 A flowchart of a method for video processing according to an embodiment of the present disclosure is illustrated;

[0053] Fig.34 A flowchart of a method for video processing according to an embodiment of the present disclosure is illustrated;

[0054] Fig.35 A flowchart of a method for video processing according to an embodiment of the present disclosure is illustrated; and

[0055] Fig.36 Illustrated is a block diagram of a computing device in which various embodiments of the present disclosure may be implemented.

[0056] Throughout the drawings, same or similar reference numbers generally refer to same or similar elements. DETAILED DESCRIPTION

[0057] The principle of the present disclosure will now be described with reference to some embodiments. It should be understood that these embodiments are described for illustrative purposes only, and help those skilled in the art to understand and implement the present disclosure, without implying any limitation on the scope of the present disclosure. The disclosure described herein can be implemented in various ways except as follows.

[0058] In the following description and claims, unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

[0059] References in this disclosure to "one embodiment," "an embodiment," "an example embodiment," etc. indicate that the described embodiment may include a particular feature, structure, or characteristic, but it is not required that every embodiment include the particular feature, structure, or characteristic. Moreover, such phrases do not necessarily refer to the same embodiment. Further, when a particular feature, structure, or characteristic is described with respect to an example embodiment, it is considered within the knowledge of those skilled in the art to affect such feature, structure, or characteristic with respect to other embodiments, whether or not explicitly described.

[0060] It should be understood that although the terms "first" and "second" etc. can be used in this article to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, the first element can be called the second element, and similarly, the second element can be called the first element without departing from the scope of the exemplary embodiment. As used herein, the term "and / or" includes any and all combinations of one or more of the listed items.

[0061] The terms used herein are only used for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments. As used herein, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are also intended to include the plural forms. It should also be understood that the terms "include", "comprises", "has", "has", "includes", and / or "comprising" when used herein specify the presence of stated features, elements, and / or components, etc., but do not exclude the presence or addition of one or more other features, elements, components, and / or combinations thereof. Example Environment

[0062] Figure 11 is a block diagram illustrating an example video codec system 100 that may utilize the techniques of the present disclosure. As shown, the video codec system 100 may include a source device 110 and a destination device 120. The source device 110 may also be referred to as a video encoding device, and the destination device 120 may also be referred to as a video decoding device. In operation, the source device 110 may be configured to generate encoded video data, and the destination device 120 may be configured to decode the encoded video data generated by the source device 110. The source device 110 may include a video source 112, a video encoder 114, and an input / output (I / O) interface 116.

[0063] The video source 112 may include a source such as a video capture device. Examples of a video capture device include, but are not limited to, an interface for receiving video data from a video content provider, a computer graphics system for generating video data, and / or a combination thereof.

[0064] The video data may include one or more pictures. The video encoder 114 encodes the video data from the video source 112 to generate a bitstream. The bitstream may include a sequence of bits that form a coded representation of the video data. The bitstream may include coded pictures and related data. The coded pictures are coded representations of pictures. Related data may include sequence parameter sets, picture parameter sets and other grammatical structures. The I / O interface 116 may include a modulator / demodulator and / or a transmitter. The coded video data may be directly sent to the destination device 120 via the network 130A via the I / O interface 116. The coded video data may also be stored on a storage medium / server 130B for access by the destination device 120.

[0065] The destination device 120 may include an I / O interface 126, a video decoder 124, and a display device 122. The I / O interface 126 may include a receiver and / or a modem. The I / O interface 126 may obtain encoded video data from the source device 110 or the storage medium / server 130B. The video decoder 124 may decode the encoded video data. The display device 122 may display the decoded video data to the user. The display device 122 may be integrated with the destination device 120, or may be outside the destination device 120, which is configured to interface with an external display device.

[0066] The video encoder 114 and the video decoder 124 may operate according to a video compression standard such as the High Efficiency Video Codec (HEVC) standard, the Versatile Video Codec (VVC) standard, and other current and / or future standards.

[0067] Figure 2is a block diagram illustrating an example of a video encoder 200 according to some embodiments of the present disclosure, which may be Figure 1 An example of a video encoder 114 in the system 100 is illustrated.

[0068] Video encoder 200 may be configured to implement any or all of the techniques of this disclosure. Figure 2 In the example of , video encoder 200 includes multiple functional components. The techniques described in this disclosure can be shared between various components of video encoder 200. In some examples, a processor can be configured to perform any or all of the techniques described in this disclosure.

[0069] In some embodiments, the video encoder 200 may include a partitioning unit 201, a prediction unit 202, a residual generation unit 207, a transformation unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transformation unit 211, a reconstruction unit 212, a buffer 213 and an entropy coding unit 214, and the prediction unit 202 may include a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205 and an intra-frame prediction unit 206.

[0070] In other examples, the video encoder 200 may include more, fewer, or different functional components. In an example, the prediction unit 202 may include an intra-block copy (IBC) unit. The IBC unit may perform prediction in an IBC mode in which at least one reference picture is a picture in which the current video block is located.

[0071] Furthermore, although some components, such as the motion estimation unit 204 and the motion compensation unit 205, may be integrated, for purposes of illustration, these components are described in detail in the following sections. Figure 2 are represented separately in the example.

[0072] The partitioning unit 201 may partition a picture into one or more video blocks. The video encoder 200 and the video decoder 300 may support various video block sizes.

[0073] The mode selection unit 203 may select one of a plurality of codec modes (intra-frame codec or inter-frame codec), for example, based on the error result, and provide the resulting intra-frame codec block or inter-frame codec block to the residual generation unit 207 to generate residual block data, and to the reconstruction unit 212 to reconstruct the codec block for use as a reference picture. In some examples, the mode selection unit 203 may select a combination of intra-frame and inter-frame prediction (CIIP) modes, where the prediction is based on an inter-frame prediction signal and an intra-frame prediction signal. In the case of inter-frame prediction, the mode selection unit 203 may also select a resolution of motion vectors for the block (e.g., sub-pixel precision or integer pixel precision).

[0074] To perform inter-frame prediction on the current video block, the motion estimation unit 204 may generate motion information of the current video block by comparing the current video block with one or more reference frames from the buffer 213. The motion compensation unit 205 may determine a predicted video block of the current video block based on the motion information and decoded samples of pictures from the buffer 213 other than the picture associated with the current video block.

[0075] For example, depending on whether the current video block is in an I-slice, a P-slice, or a B-slice, the motion estimation unit 204 and the motion compensation unit 205 may perform different operations for the current video block. As used herein, an "I-slice" may refer to a portion of a picture consisting of macroblocks, all of which are based on macroblocks within the same picture. Furthermore, as used herein, in some aspects, a "P-slice" and a "B-slice" may refer to a portion of a picture consisting of macroblocks that are not dependent on macroblocks in the same picture.

[0076] In some examples, the motion estimation unit 204 may perform unidirectional prediction for the current video block, and the motion estimation unit 204 may search the reference picture of list 0 or list 1 to obtain the reference video block of the current video block. The motion estimation unit 204 may then generate a reference index and a motion vector, the reference index indicating the reference picture in list 0 or list 1 containing the reference video block, and the motion vector indicating the spatial displacement between the current video block and the reference video block. The motion estimation unit 204 may output the reference index, the prediction direction indicator, and the motion vector as the motion information of the current video block. The motion compensation unit 205 may generate a predicted video block of the current video block based on the reference video block indicated by the motion information of the current video block.

[0077] Alternatively, in other examples, the motion estimation unit 204 may perform bidirectional prediction for the current video block. The motion estimation unit 204 may search the reference pictures in list 0 to obtain a reference video block of the current video block, and may also search the reference pictures in list 1 to obtain another reference video block of the current video block. The motion estimation unit 204 may then generate reference indexes and motion vectors, which indicate the reference pictures in list 0 and list 1 that contain the reference video blocks, and which indicate the spatial displacement between the reference video blocks and the current video block. The motion estimation unit 204 may output the reference index and motion vector of the current video block as the motion information of the current video block. The motion compensation unit 205 may generate a predicted video block of the current video block based on the reference video block indicated by the motion information of the current video block.

[0078] In some examples, motion estimation unit 204 may output a complete set of motion information for use in a decoding process by a decoder. Alternatively, in some embodiments, motion estimation unit 204 may signal motion information for a current video block with reference to motion information of another video block. For example, motion estimation unit 204 may determine that the motion information of the current video block is sufficiently similar to the motion information of a neighboring video block.

[0079] In one example, motion estimation unit 204 may indicate a value in a syntax structure associated with the current video block that indicates to video decoder 300 that the current video block has the same motion information as another video block.

[0080] In another example, the motion estimation unit 204 may identify another video block and a motion vector difference (MVD) in a syntax structure associated with the current video block. The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the indicated video block. The video decoder 300 may use the motion vector of the indicated video block and the motion vector difference to determine the motion vector of the current video block.

[0081] As discussed above, the video encoder 200 may predictively signal motion vectors.Two examples of predictive signaling techniques that may be implemented by the video encoder 200 include Advanced Motion Vector Prediction (AMVP) and merge mode signaling.

[0082] The intra prediction unit 206 may perform intra prediction on the current video block. When the intra prediction unit 206 performs intra prediction on the current video block, the intra prediction unit 206 may generate prediction data for the current video block based on decoded samples of other video blocks in the same picture. The prediction data for the current video block may include a prediction video block and various syntax elements.

[0083] The residual generation unit 207 may generate residual data for the current video block by subtracting (e.g., indicated by a minus sign) the prediction video block(s) of the current video block from the current video block. The residual data for the current video block may include residual video blocks corresponding to different sample components of the samples in the current video block.

[0084] In other examples, such as in skip mode, there may be no residual data for the current video block, and the residual generation unit 207 may not perform a subtraction operation.

[0085] Transform processing unit 208 may generate one or more transform coefficient video blocks for a current video block by applying one or more transforms to the residual video block associated with the current video block.

[0086] After transform processing unit 208 generates a transform coefficient video block associated with the current video block, quantization unit 209 may quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values ​​associated with the current video block.

[0087] Inverse quantization unit 210 and inverse transform unit 211 may apply inverse quantization and inverse transform to the transform coefficient video block, respectively, to reconstruct a residual video block from the transform coefficient video block. Reconstruction unit 212 may add the reconstructed residual video block to corresponding samples from one or more prediction video blocks generated by prediction unit 202 to generate a reconstructed video block associated with the current video block for storage in buffer 213.

[0088] After the reconstruction unit 212 reconstructs the video block, a loop filtering operation may be performed to reduce video block artifacts in the video block.

[0089] The entropy coding unit 214 may receive data from other functional components of the video encoder 200. When the entropy coding unit 214 receives the data, the entropy coding unit 214 may perform one or more entropy coding operations to generate entropy-coded data and output a bitstream including the entropy-coded data.

[0090] Figure 3 is a block diagram illustrating an example of a video decoder 300 according to some embodiments of the present disclosure, which may be Figure 1 An example of a video decoder 124 in the system 100 is illustrated.

[0091] Video decoder 300 may be configured to perform any or all of the techniques of this disclosure. Figure 3 In the example of , video decoder 300 includes multiple functional components. The techniques described in this disclosure can be shared between various components of video decoder 300. In some examples, a processor can be configured to perform any or all of the techniques described in this disclosure.

[0092] exist Figure 3 In the example of , the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, a reconstruction unit 306 and a buffer 307. In some examples, the video decoder 300 can perform a decoding stage that is generally reciprocal to the encoding stage described with respect to the video encoder 200.

[0093] The entropy decoding unit 301 may retrieve an encoded bitstream. The encoded bitstream may include entropy-encoded video data (e.g., a coded block of video data). The entropy decoding unit 301 may decode the entropy-encoded video data, and from the entropy-decoded video data, the motion compensation unit 302 may determine motion information, including motion vectors, motion vector precision, reference picture list indexes, and other motion information. The motion compensation unit 302 may determine such information, for example, by performing AMVP and merge mode. AMVP is used, including deriving several most likely candidates based on data and reference pictures from adjacent PBs. The motion information typically includes horizontal and vertical motion vector displacement values, one or two reference picture indexes, and in the case of a prediction region in a B stripe, an identification of which reference picture list is associated with each index. As used herein, in some aspects, "merge mode" may refer to deriving motion information from adjacent blocks in the spatial or temporal domain.

[0094] The motion compensation unit 302 may generate a motion compensated block, possibly performing interpolation based on an interpolation filter. An identifier of an interpolation filter to be used with sub-pixel precision may be included in a syntax element.

[0095] Motion compensation unit 302 may calculate interpolated values ​​of sub-integer pixels of a reference block using interpolation filters as used during encoding of the video block by video encoder 200. Motion compensation unit 302 may determine the interpolation filters used by video encoder 200 from received syntax information and use these interpolation filters to produce a prediction block.

[0096] The motion compensation unit 302 may use at least a portion of the syntax information to determine the size of blocks used to encode (one or more) frames and / or (one or more) slices of the coded video sequence, partition information describing how each macroblock of a picture of the coded video sequence is partitioned, a mode indicating how each partition is encoded, one or more reference frames (and reference frame lists) for each inter-frame coding block, and other information used to decode the coded video sequence. As used herein, in some aspects, a "slice" may refer to a data structure that can be decoded independently of other slices of the same picture in terms of entropy coding, signal prediction, and residual signal reconstruction. A slice may be an entire picture or a region of a picture.

[0097] The intra prediction unit 303 may form a prediction block from spatially neighboring blocks using, for example, an intra prediction mode received in the bitstream. The inverse quantization unit 304 inverse quantizes (i.e., dequantizes) the quantized video block coefficients provided in the bitstream and decoded by the entropy decoding unit 301. The inverse transform unit 305 applies an inverse transform.

[0098] The reconstruction unit 306 may obtain a decoded block, for example, by adding the residual block to the corresponding prediction block generated by the motion compensation unit 302 or the intra prediction unit 303. Optionally, a deblocking filter may also be applied to filter the decoded block to remove blocking artifacts. The decoded video block is then stored in a buffer 307, which provides reference blocks for subsequent motion compensation / intra prediction and also generates a decoded video for presentation on a display device.

[0099] Some exemplary embodiments of the present disclosure will be described in detail below. It should be noted that the section titles are used in this document to facilitate ease of understanding, and the embodiments disclosed in the section are not limited to that section. In addition, although certain embodiments are described with reference to general video codecs or other specific video codecs (codecs), the disclosed technology is also applicable to other video codec technologies. In addition, although some embodiments describe the video coding and decoding steps in detail, it should be understood that the corresponding decoding steps of the cancellation codec will be implemented by the decoder. In addition, the term video processing includes video coding or compression, video decoding or decompression, and video transcoding, in which video pixels are represented from one compression format to another compression format or at different compression bit rates. 1. Brief Overview The present disclosure relates to video coding technology. Specifically, the present disclosure is about enhanced functions based on DMVR / BDOF in image / video coding. The present disclosure can be applied to existing video coding standards such as HEVC, VVC, etc. The present disclosure can also be applied to future video coding standards or video codecs. 2. Introduction Video codec standards have evolved primarily through the development of the well-known ITU-T and ISO / IEC standards. ITU-T developed H.261 and H.263, ISO / IEC developed MPEG-1 and MPEG-4 Visual, and the two organizations jointly developed H.262 / MPEG-2 Video and H.264 / MPEG-4 Advanced Video Codec (AVC) and H.265 / HEVC standards. Since H.262, video codec standards have been based on a hybrid video codec structure in which temporal prediction plus transform codec is utilized. In order to explore future video codec technologies beyond HEVC, VCEG and MPEG jointly founded the Joint Video Exploration Team (JVET) in 2015. JVET meetings are held quarterly at the same time, and the new video codec standard was officially named the Versatile Video Codec (VVC) at the JVET meeting in April 2018, and the first version of the VVC Test Model (VTM) was released at that time. The VVC working draft and the test model VTM are updated after each meeting. The VVC project achieved technical completion (FDIS) at the July 2020 meeting. 2.1. Existing inter-frame prediction codec tools For each inter-predicted CU, motion parameters including motion vector, reference picture index and reference picture list usage index and additional information required by the new coding features of VVC will be used for inter-predicted sample generation. Motion parameters can be signaled explicitly or implicitly. When a CU is encoded and decoded in skip mode, the CU is associated with one PU and has no significant residual coefficients, no coded motion vector increments or reference picture indexes. A merge mode is specified, whereby the motion parameters of the current CU are obtained from neighboring CUs (including spatial candidates and temporal candidates) and the additional temporal table introduced in VVC. Merge mode can be applied to any inter-predicted CU, not just for skip mode. An alternative to merge mode is explicit transmission of motion parameters, where motion vectors, corresponding reference picture indexes for each reference picture list and reference picture list usage flags, as well as other required information, are explicitly signaled per CU. In addition to the inter-frame coding features in HEVC, VVC also includes many new and refined inter-frame prediction coding tools listed below: – Extended combined forecast; – Merge mode with MVD (MMVD); – Symmetric MVD (SMVD) signaling; – Affine motion compensated prediction; – Sub-block based temporal motion vector prediction (SbTMVP); – Adaptive Motion Vector Resolution (AMVR); – Motion field storage: 1 / 16th luminance sample MV storage and 8x8 motion field compression; – Bidirectional prediction with CU level weights (BCW); – Bidirectional Optical Flow (BDOF); – Decoder-side motion vector refinement (DMVR); – Geometric Partitioning Mode (GPM); – Combined intra and inter prediction (CIIP). Details on those inter prediction methods specified in VVC are provided below. 2.1.1. Extended Merge Prediction In VVC, the merge candidate list is constructed by including the following five types of candidates in order: 1) Airspace MVP from airspace neighboring CUs; 2) Temporal MVP from the co-located CU; 3) History-based MVP from FIFO table; 4) Paired average MVP; 5) Zero MV. The size of the merge list is signaled in the sequence parameter set header, and the maximum allowed size of the merge list is 6. For each CU code in merge mode, the index of the best merge candidate is encoded using truncated unary binarization (TU). The first bin of the merge index is coded with context, while bypass coding is used for other bins. The derivation process of merge candidates for each category is provided in this section. As done in HEVC, VVC also supports parallel derivation of merge candidate lists for all CUs in a region of a certain size. 2.1.1.1. Spatial Candidate Derivation The derivation of spatial merge candidates in VVC is the same as that in HEVC, except that the positions of the first two merge candidates are swapped. Figure 4 is a schematic diagram 400 illustrating the locations of spatial merging candidates. Figure 4 Select a maximum of four merge candidates from the candidates for the depicted position. The derivation order is B 0 , A 0 , B 1 , A 1 and B 2 Only when position B 0 , A 0 , B 1 and A 1 Position B is considered only when one or more CUs are not available (for example, because they belong to another slice or tile) or are intra-coded. 2 . Added position A1 After the candidates at, the addition of the remaining candidates undergoes a redundancy check, which ensures that candidates with the same motion information are excluded from the list, so that the encoding and decoding efficiency is improved. In order to reduce the computational complexity, not all possible candidate pairs are considered in the mentioned redundancy check. Figure 5 is a diagram 500 illustrating candidate pairs considered for redundancy checking of spatial merging candidates. Figure 5 The pairs linked with arrows in , and the candidate is added to the list only if the corresponding candidate for redundancy check does not have the same motion information. 2.1.1.2. Temporal candidate extraction In this step, only one candidate is added to the list. In particular, in the derivation of this temporal merge candidate, the scaled motion vector is derived based on the co-located CU belonging to the co-located reference picture. The reference picture list to be used for deriving the co-located CU is explicitly signaled in the slice header. The scaled motion vector of the temporal merge candidate is Figure 6 The scaled motion vector is obtained as illustrated by the dashed line in the diagram 600 of , which is scaled from the motion vector of the co-located CU using the POC distances tb and td, where tb is defined as the POC difference between the reference picture of the current picture and the current picture and td is defined as the POC difference between the reference picture of the co-located picture and the co-located picture. The reference picture index of the temporal merge candidate is set equal to zero. Figure 7 is a diagram illustrating the time domain merging candidate C 0 and C 1 Schematic diagram 700 of candidate locations. Figure 7 As depicted, the position of the temporal candidate is within the candidate C 0 With C 1 If position C 0 If the CU at position C is not available, is intra-coded, or is outside the current row of the CTU, then position C is used. 1 Otherwise, position C is used in the derivation of the temporal merge candidate 0 . 2.1.1.3. History-based Merge Candidate Derivation History-based MVP (HMVP) merge candidates are added to the merge list after spatial MVP and TMVP. In this method, the motion information of previously coded blocks is stored in a table and used as the MVP of the current CU. A table with multiple HMVP candidates is maintained during the encoding / decoding process. The table is reset (cleared) when a new CTU row is encountered. Whenever there is a non-sub-block inter-coded CU, the relevant motion information is added to the last entry of the table as a new HMVP candidate. The HMVP table size S is set to 6, which indicates that up to 6 history-based MVP (HMVP) candidates can be added to the table. When a new motion candidate is inserted into the table, a restricted first-in-first-out (FIFO) rule is used, where a redundancy check is first applied to find if there is an identical HMVP in the table. If found, the identical HMVP is removed from the table and all subsequent HMVP candidates are moved forward. HMVP candidates can be used in the merge candidate list construction process. The latest few HMVP candidates in the table are checked in order and inserted into the candidate list after the TMVP candidate. Redundancy check is applied to HMVP candidates to spatial or temporal merge candidates. To reduce the number of redundant checking operations, the following simplifications are introduced: 1. The number of HMPV candidates for merge list generation is set to (N<=4)?M:(8-N), where N indicates the number of existing candidates in the merge list and M indicates the number of available HMVP candidates in the table. 2. Once the total number of available merge candidates reaches the maximum allowed merge candidate minus 1, the merge candidate list construction process from HMVP is terminated. 2.1.1.4. Pairwise average merge candidate derivation The pairwise average candidate is generated by averaging predefined pairs of candidates in the existing merge candidate list, and the predefined pairs are defined as {(0,1), (0,2), (1,2), (0,3), (1,3), (2,3)}, where the number represents the merge index of the merge candidate list. The average motion vector is calculated separately for each reference list. If two motion vectors are available in one list, they are averaged even when they point to different reference pictures; if only one motion vector is available, it is used directly; if no motion vector is available, this list is kept invalid. When the merge list is not full after adding pairwise average merge candidates, zero MVPs are inserted at the end until the maximum number of merge candidates is encountered. 2.1.1.5. Merge estimated regions The merge estimation region (MER) allows the merge candidate list to be derived independently for a CU in the same merge estimation region (MER). For the generation of the merge candidate list for the current CU, candidate blocks in the same MER as the current CU are not included. In addition, the update process for the history-based motion vector predictor candidate list is only updated when (xCb+cbWidth)>>Log2ParMrgLevel is greater than xCb>>Log2ParMrgLevel and (yCb+cbHeight)>>Log2parMrglevel is greater than (yCb>>Log2ParMrgLevel) and when (xCb, yCb) is the top left luma sample position of the current CU in the picture and (cbWidth, cbHeight) is the CU size. The MER size is selected at the encoder side and signaled in the sequence parameter set as log2_parallel_merge_level_minus2. 2.1.2. Merge Mode with MVD (MMVD) In addition to the merge mode where the implicitly derived motion information is directly used for prediction sample generation of the current CU, a merge mode with motion vector difference (MMVD) is introduced in VVC. The MMVD flag is signaled just after the skip flag and merge flag are sent to specify whether the MMVD mode is used for the CU. In MMVD, after a merge candidate is selected, it is further refined by the MVD information transmitted with the signal. Further information includes a merge candidate flag, an index to specify the magnitude of motion, and an index indicating the direction of motion. In MMVD mode, one of the first two candidates in the merge list is selected to be used as the MV basis. The MMVD candidate flag is transmitted with the signal to specify which candidate is used. The distance index specifies the motion magnitude information and indicates a predefined offset from the starting point. Figure 8 As shown, the offset is added to the horizontal component or the vertical component of the starting MV. The relationship between the distance index and the predefined offset is specified in Table 1. Table 1 – Relationship between distance index and predefined offsets The direction index indicates the direction of the MVD relative to the starting point. The direction index can represent four directions as shown in Table 2. Note that the meaning of the MVD symbol can change according to the information of the starting MV. When the starting MV is a non-predicted MV or a bidirectionally predicted MV in which the two lists point to the same side of the current picture (that is, the POCs of the two references are both greater than the POC of the current picture, or both are less than the POC of the current picture), the symbol in Table 1 specifies the sign of the MV offset added to the starting MV. When the starting MV is a bidirectionally predicted MV in which the two MVs point to different sides of the current picture (that is, the POC of one reference is greater than the POC of the current picture, and the POC of the other reference is less than the POC of the current picture), the symbol in Table 2 specifies the sign of the MV offset added to the list0 MV component of the starting MV and the sign of the list1 MV has an opposite value. Table 2 – Sign of MV offsets specified by direction index Direction Index 00 01 10 11 x-axis + - N / A N / A y-axis N / A N / A + - 2.1.2.1. Bidirectional Prediction with CU-Level Weights (BCW) In HEVC, the bidirectional prediction signal is generated by averaging two prediction signals obtained from two different reference pictures and / or using two different motion vectors. In VVC, the bidirectional prediction mode is extended beyond simple averaging to allow weighted averaging of the two prediction signals: P bi-pred =((8-w)*P 0 +w*P 1 +4)>>3 (2-1). Five weights are allowed in weighted average bidirectional prediction, w∈{-2,3,4,5,10}. For each bidirectionally predicted CU, the weight w is determined in one of two ways: 1) For non-merged CUs, the weight index is signaled after the motion vector difference; 2) For merged CUs, the weight index is inferred from neighboring blocks based on the merge candidate index. BCW is only applicable to CUs with 256 or more luma samples (i.e., CU width times CU height is greater than or equal to 256). For low-latency pictures, all 5 weights are used. For non-low-latency pictures, only 3 weights (w∈{3,4,5}) are used. – At the encoder, fast search algorithms are applied to find the weight index without significantly increasing the encoder complexity. These algorithms are summarized below. When combined with AMVR, unequal weights are only conditionally checked for 1-pixel and 4-pixel motion vector accuracy if the current picture is a low-latency picture. - When combined with affine, affine ME will be performed for unequal weights if and only if the affine mode is selected as the current best mode. – When the two reference pictures in bidirectional prediction are the same, unequal weights are only checked conditionally. – Do not search for unequal weights when certain conditions are met, which depend on the POC distance between the current picture and its reference pictures, the codec QP, and the temporal level. The BCW weight index is encoded using one context encoded bit followed by a bypass encoded bit. The first context encoded bit indicates whether equal weights are used; and if unequal weights are used, an additional bit is signaled using bypass codec to indicate which unequal weights are used. Weighted prediction (WP) is a codec tool supported by the H.264 / AVC and HEVC standards to efficiently encode and decode video content in fading situations. Support for WP has also been added to the VVC standard. WP allows weighting parameters (weights and offsets) to be signaled for each reference picture in each of the reference picture lists L0 and L1. Then, during motion compensation, the corresponding (one or more) weights and (one or more) offsets of the reference picture(s) are applied. WP and BCW are designed for different types of video content. In order to avoid the interaction between WP and BCW (which will complicate the VVC decoder design), if the CU uses WP, the BCW weight index is not signaled, and w is inferred to be 4 (i.e., equal weights are applied). For the merged CU, the weight index is inferred from the neighboring blocks based on the merge candidate index. This can be applied to both the normal merge mode and the inherited affine merge mode. For the constructed affine merge mode, affine motion information is constructed based on the motion information of up to 3 blocks. The BCW index of the CU using the constructed affine merge mode is simply set equal to the BCW index of the first control point MV. In VVC, CIIP and BCW cannot be applied jointly to a CU. When encoding and decoding a CU in CIIP mode, the BCW index of the current CU is set to 2, for example, equal weight. 2.1.2.2. Bidirectional Optical Flow (BDOF) The Bidirectional Optical Flow (BDOF) tool is included in VVC. BDOF (formerly known as BIO) is included in JEM. Compared to the JEM version, BDOF in VVC is a simpler version that requires much less computation, especially in terms of the number of multiplications and multiplier size. BDOF is used to refine the bidirectional prediction signal of a CU at the 4×4 sub-block level. If BDOF meets all the following conditions, it is applicable to a CU: – The CU is encoded and decoded using the “true” bidirectional prediction mode, that is, one of the two reference pictures is before the current picture in the display order, and the other reference picture is after the current picture in the display order; – The distances from the two reference pictures to the current picture (i.e., POC differences) are the same; – Both reference images are short-term reference images; –CU does not use affine mode or SbTMVP merge mode for encoding and decoding; – CU has more than 64 luminance samples; – CU height and CU width are both greater than or equal to 8 luma samples; – BCW weight index indicates equal weight; – WP is not enabled for the current CU; –CIIP mode is not used for the current CU. BDOF is only applicable to the luma component. As its name indicates, BDOF mode is based on the concept of optical flow, which assumes that the motion of objects is smooth. For each 4×4 sub-block, motion refinement (v x ,v y ) is calculated by minimizing the difference between the L0 prediction samples and the L1 prediction samples. Motion refinement is then used to adjust the bi-prediction sample values ​​in the 4x4 sub-block. The following steps are applied in the BDOF process. First, the horizontal and vertical gradients of the two prediction signals are calculated by directly computing the difference between two neighboring samples: and Right now Among them I (k) (i, j) is the sample value at coordinate (i, j) of the prediction signal in list k (k=0, 1), and shift1 is calculated based on the luma bit depth bitDepth as shift1=max(6, bitDepth-6). Then, the gradient S 1 ,S 2 ,S 3 ,S 5 and S 6 The autocorrelation and cross-correlation of S 1 =∑ (i,j)∈Ω Abs(ψ x (i,j)),S 3 =∑ (i,j)∈Ω θ(i,j)·Sign(ψ x (i,j)) S 2 =∑ (i,j)∈Ω ψ x (i,j)·Sign(ψ y (i,j)) (2-3) S 5 =∑(i,j)∈Ω Abs(ψ y (i,j))S 6 =∑ (i,j)∈Ω θ(i,j)·Sign(ψ y (i,j)) in θ(i,j)=(I (1) (i,j)>>n b )-(I (0) (i,j)>>n b ) where Ω is a 6×6 window around the 4×4 sub-block, and n a and n b The values ​​of are set equal to min(1, bitDepth-11) and min(4, bitDepth-8) respectively. Then, the motion refinement (v) is derived using the cross-correlation and autocorrelation terms as follows: x ,v y ): in th′ BIO =2 max(5,BD-7) . is the round-down (floor) function, and Based on the motion refinement and gradients, the following adjustments are computed for each sample in the 4×4 sub-block: Finally, the BDOF samples of the CU are calculated by adjusting the bidirectional prediction samples as follows: pred BDOF (x,y)=(I (0) (x,y)+I (1) (x,y)+b(x,y)+o offset )>>shift (2-7). These values ​​are chosen so that the multipliers in the BDOF process do not exceed 15 bits and the maximum bit width of the intermediate parameters in the BDOF process is kept within 32 bits. In order to derive the gradient value, some prediction samples I in the list k (k = 0, 1) outside the current CU boundary (k) (i,j) needs to be generated. Fig. 9 FIG. 2 is a schematic diagram showing an extended CU region used in BDOF. Fig. 9As depicted in the schematic diagram 900 of FIG. 1 , BDOF in VVC uses an extended row / column around the boundary of the CU. In order to control the computational complexity of generating prediction samples outside the boundary, the extended area (in Fig. 9 The prediction samples in 910 are generated by directly taking the reference samples at nearby integer positions without interpolation (using the floor() operation on the coordinates), and the normal 8-head motion compensation interpolation filter is used to generate the CU (in Fig. 9 The predicted samples within the CU boundary are represented as 920 in FIG. These extended sample values ​​are only used in the gradient calculation. For the remaining steps in the BDOF process, if any samples and gradient values ​​outside the CU boundary are needed, these sample values ​​and gradient values ​​are filled (i.e., repeated) from their nearest neighbors. When the width and / or height of a CU is greater than 16 luma samples, it will be split into sub-blocks with a width and / or height equal to 16 luma samples, and the sub-block boundaries are regarded as CU boundaries in the BDOF process. The maximum unit size for the BDOF process is limited to 16×16. For each sub-block, the BDOF process can be skipped. When the SAD between the initial L0 prediction sample and the L1 prediction sample is less than the threshold, the BDOF process is not applied to the sub-block. The threshold is set equal to (8*W*(H>>1)), where W indicates the sub-block width and H indicates the sub-block height. In order to avoid the additional complexity of the SAD calculation, the SAD between the initial L0 prediction sample and the L1 prediction sample calculated in the DVMR process is reused here. If BCW is enabled for the current block, that is, the BCW weight index indicates unequal weights, bidirectional optical flow is disabled. Similarly, if WP is enabled for the current block, that is, luma_weight_lx_flag is 1 for either of the two reference pictures, BDOF is also disabled. BDOF is also disabled when the CU is encoded or decoded in symmetric MVD mode or CIIP mode. 2.1.2.3. Symmetric MVD Codec (SMVD) In VVC, in addition to normal unidirectional prediction and bidirectional prediction mode MVD signaling, symmetric MVD mode for bidirectional prediction MVD signaling is also applied. In symmetric MVD mode, the motion information including reference picture indexes of both list-0 and list-1 and MVD of list-1 is not signaled but derived. The decoding process of the symmetric MVD mode is as follows: 1) At the stripe level, the variables BiDirPredFlag, RefIdxSymL0, and RefIdxSymL1 are derived as follows: – If mvd_l1_zero_flag is 1, BiDirPredFlag is set equal to 0. – Otherwise, if the nearest reference picture in list-0 and the nearest reference picture in list-1 form a preceding pair of reference pictures or a following pair of reference pictures, BiDirPredFlag is set to 1, and both the list-0 reference picture and the list-1 reference picture are short-term reference pictures. Otherwise, BiDirPredFlag is set to 0. 2) At the CU level, if the CU is bidirectionally predicted and BiDirPredFlag is equal to 1, the symmetric mode flag indicating whether the symmetric mode is used is explicitly signaled. When the symmetric mode flag is true, only mvp_l0_flag, mvp_l1_flag and MVD0 are explicitly signaled. The reference indexes of list-0 and list-1 are set equal to the reference picture pair, respectively. MVD1 is set equal to (-MVD0). The final motion vector is shown in the formula below. Fig.10 is an illustration for the symmetric MVD mode. In the encoder, symmetric MVD motion estimation starts with an initial MV evaluation. A set of initial MV candidates includes MVs obtained from unidirectional prediction search, MVs obtained from bidirectional prediction search, and MVs from the AMVP list. The one with the lowest rate-distortion cost is selected as the initial MV for the symmetric MVD motion search. 2.1.3. Affine Motion Compensated Prediction In HEVC, only the translational motion model is applied to motion compensated prediction (MCP). However, in the real world, there are many kinds of motion, such as zooming in / out, rotation, perspective motion, and other irregular motions. In VVC, block-based affine transformation motion compensated prediction is applied. Fig.11 As shown, the affine motion field of a block is described by motion information of two control point motion vectors (4 parameters) or three control point motion vectors (6 parameters). for Fig.11 Using the 4-parameter affine motion model 1110 in , the motion vector at the sample position (x, y) in the block is derived as: for Fig.11 Using the 6-parameter affine motion model 1120 in , the motion vector at the sample position (x, y) in the block is derived as: Where (mv 0x ,mv 0y ) is the motion vector of the upper left control point, (mv 1x,mv 1y ) is the motion vector of the upper right control point, and (mv 2x ,mv 2y ) is the motion vector of the lower left control point. To simplify motion compensated prediction, block-based affine transform prediction is applied. Fig.12 A schematic diagram 1200 of the affine MVF for each sub-block is shown. To derive the motion vector for each 4×4 luminance sub-block, as Fig.12 As shown, the motion vector of the center sample of each subblock is calculated according to the above equation and rounded to 1 / 16 fractional precision. Then, the motion compensated interpolation filter is applied to generate a prediction for each subblock with the derived motion vector. The subblock size of the chroma component is also set to 4×4. The MV of a 4×4 chroma subblock is calculated as the average of the MVs of the four corresponding 4×4 luminance subblocks. As done for translational motion inter prediction, there are also two affine motion inter prediction modes: affine merge mode and affine AMVP mode. 2.1.3.1. Affine Merge Prediction AF_MERGE mode can be applied to CUs with width and height greater than or equal to 8. In this mode, the CPMV of the current CU is generated based on the motion information of the spatially adjacent CUs. There can be up to five CPMVP candidates, and the index is signaled to indicate the CPMVP to be used for the current CU. The following three types of CPVM candidates are used to form the affine merge candidate list: - Inherited affine merge candidates extrapolated from the CPMV of neighboring CUs; - Affine merged candidate CPMVP constructed using the translation MV of the neighboring CU; -Zero MV. In VVC, there are at most two inherited affine candidates derived from the affine motion models of neighboring blocks, one from the left neighboring CU and one from the above neighboring CU. Fig.13 Schematic diagram 1300 illustrating the positions of inherited affine motion predictors. Candidate blocks such as Fig.13 As shown. For the left predictor, the scan order is A0->A1, while for the upper predictor, the scan order is B0->B1->B2. Only the first inheritance candidate from each side is selected. No pruning check is performed between two inheritance candidates. When a neighboring affine CU is identified, its control point motion vector is used to derive the CPMVP candidate in the affine merge list of the current CU. Fig.14 A schematic diagram 1400 of control point motion vector inheritance is shown. Fig.14As shown, if the adjacent lower left block A 1410 is encoded and decoded in the affine mode, the motion vectors v of the upper left corner, upper right corner and lower left corner of the CU 1420 containing the block A 1410 are obtained. 2 , v 3 and v 4 When encoding and decoding block A 1410 using a 4-parameter affine model, according to v 2 and v 3 Calculate the two CPMVs of the current CU. In the case of encoding and decoding block A using a 6-parameter affine model, according to v 2 , v 3 and v 4 Calculate the three CPMVs of the current CU. The constructed affine candidate means that the candidate is constructed by combining the neighboring translational motion information of each control point. The motion information of the control point is obtained from Fig.15 The specified spatial neighbors and specified temporal neighbors shown are derived. Fig.15 A schematic diagram 1500 is shown illustrating the positions of candidate positions for a constructed affine merge pattern. k (k=1, 2, 3, 4) represents the kth control point. 1 , B2->B3->A2 blocks are checked and the MV of the first available block is used. For CPMV 2 , the B1->B0 block is checked, and for CPMV 3 , A1->A0 block is checked. For TMVP, if available, use TMVP as CPMV 4 . After the MVs of the four control points are obtained, an affine merge candidate is constructed based on the motion information. The following combinations of control point MVs are used to construct in order: {CPMV 1 ,CPMV 2 ,CPMV 3}、{CPMV 1 ,CPMV 2 ,CPMV 4}、{CPMV 1 ,CPMV 3 ,CPMV 4}、{CPMV 2 ,CPMV 3 ,CPMV 4}、{CPMV 1 ,CPMV 2}、{CPMV 1 ,CPMV 3}. A combination of 3 CPMVs constructs a 6-parameter affine merge candidate, and a combination of 2 CPMVs constructs a 4-parameter affine merge candidate. To avoid the motion scaling process, the relevant combination of control point MVs is discarded if the reference index of the control points is different. After the inherited affine merge candidates and the constructed affine merge candidates are checked, if the list is still not full, a zero MV is inserted at the end of the list. 2.1.3.2. Affine AMVP prediction Affine AMVP mode can be applied to CUs with width and height both greater than or equal to 16. An affine flag at the CU level is signaled in the bitstream to indicate whether the affine AMVP mode is used, and then another flag is signaled to indicate whether it is 4-parameter affine or 6-parameter affine. In this mode, the difference between the CPMVP of the current CU and its predicted sub-CPMVP is signaled in the bitstream. The affine AVMP candidate list size is 2, and it is generated by using the following four types of CPVM candidates in order: - Inherited affine AMVP candidates extrapolated from the CPMV of neighboring CUs; - A constructed affine AMVP candidate CPMVP derived using the translation MV of the neighboring CU; -Translated MV from neighboring CU; -Zero MV. The order of checking inherited affine AMVP candidates is the same as that of checking inherited affine merge candidates. The only difference is that for AVMP candidates, only affine CUs with the same reference picture as in the current block are considered. The pruning process is not applied when inserting inherited affine motion predictors into the candidate list. The constructed AMVP candidates are Fig.15 The same check order as done in the affine merge candidate construction is used. In addition, the reference picture index of the neighboring blocks is also checked. The first block in the check order that is inter-coded and has the same reference picture as in the current CU is used. There is only one. When the current CU is coded and decoded with 4 parameter affine mode and mv 0 and mv 1 When all three CPMVs are available, they are added as a candidate in the affine AMVP list. When the current CU is encoded and decoded with 6-parameter affine mode and all three CPMVs are available, they are added as a candidate in the affine AMVP list. Otherwise, the constructed AMVP candidate is set to unavailable. If after the valid inherited affine AMVP candidates and constructed AMVP candidates are checked, the affine AMVP list of candidates is still less than 2, then mv 0 、mv 1and mv 2 Added as translation MVs in order to predict all control point MVs of the current CU when available. Finally, if the affine AMVP list is still not full, zero MVs are used to fill the affine AMVP list. 2.1.3.3. Affine motion information storage In VVC, the CPMV of an affine CU is stored in a separate buffer. The stored CPMV is only used to generate inherited CPMVs for the most recently encoded CU in affine merge mode and affine AMVP mode. The sub-block MVs derived from the CPMVs are used for motion compensation, merging of translation MVs / MV derivation of AMVP lists, and deblocking. To avoid picture line buffering for additional CPMV, affine motion data inheritance from CUs from upper CTUs is handled differently than inheritance from normal neighboring CUs. If the candidate CU for affine motion data inheritance is in the upper CTU row, the lower left and lower right sub-block MVs in the line buffer are used for affine MVP derivation instead of the CPMVs. In this way, the CPMVs are stored only in the local buffer. If the candidate CU is decoded by 6-parameter affine codec, the affine model is downgraded to a 4-parameter model. Fig.16 As shown, along the top CTU boundary, the bottom left and bottom right sub-block motion vectors of the CU are used for affine inheritance of the CU in the bottom CTU. 2.1.3.4. Prediction refinement using optical flow for affine mode (PROF) Compared with pixel-based motion compensation, sub-block-based affine motion compensation can save memory access bandwidth and reduce computational complexity, but at the cost of prediction accuracy penalty. In order to achieve finer motion compensation granularity, prediction refinement with optical flow (PROF) is used to refine sub-block-based affine motion compensation prediction without increasing the memory access bandwidth of motion compensation. In VVC, after sub-block-based affine motion compensation is performed, the brightness prediction samples are refined by adding the differences derived from the optical flow equation. PROF is described as the following four steps: Step 1) Sub-block based affine motion compensation is performed to generate a sub-block prediction I(i,j). Step 2) Use a 3-tap filter [-1, 0, 1] to calculate the spatial gradient g of the sub-block prediction at each sample position x (i,j) and g y (i,j). The gradient calculation is exactly the same as that in BDOF. g x (i,j)=(I(i+1,j)>>shift1)-(I(i-1,j))shift1) (2-11) g y(i,j)=(I(i,j+1))shift1)-(I(i,j-1))shift1) (2-12) Shift1 is used to control the accuracy of the gradient. Sub-block (i.e. 4×4) prediction is extended by one sample on each side of the gradient calculation. To avoid additional memory bandwidth and additional interpolation calculations, these extended samples on the extended boundary are copied from the nearest integer pixel position in the reference picture. Step 3) The brightness prediction refinement is calculated by the following optical flow equation. Δi(i,j)=g x (i,j)*Δv x (i,j)+g y (i,j)*Δv y (i,j) (2-13) where Δv(i,j) is the difference between the sample MV calculated for the sample position (i,j) (denoted as v(i,j)) and the sub-block MV of the sub-block to which the sample (i,j) belongs, as Fig.17 Δv(i,j) (shown as arrow 1710) is quantized in units of 1 / 32 luma sample precision. Since the affine model parameters and the sample positions relative to the sub-block center do not change from sub-block to sub-block, Δv(i,j) can be calculated for the first sub-block and reused for other sub-blocks in the same CU. Let dx(i,j) and dy(i,j) be the distance from the sample position (i,j) to the sub-block center (x SB ,y SB ), Δv(x,y) can be derived from the following equations: To maintain accuracy, the sub-block center (x SB ,y SB ) is calculated as ((WSB–1) / 2,(HSB–1) / 2), where W SB and H SB are the sub-block width and height respectively. For the 4-parameter affine model, For the 6-parameter affine model, Where (v 0x ,v 0y )、(v 1x ,v 1y )、(v 2x ,v 2y ) are the upper left, upper right and lower left control point motion vectors, w and h are the width and height of the CU. Step 4) Finally, the luma prediction refinement ΔI(i,j) is added to the sub-block prediction I(i,j) and the final prediction I' is generated as the following equation. I′(i,j)=I(i,j)+ΔI(i,j) PROF is not applicable to affine-coded CUs in two cases: 1) all control point MVs are the same, indicating that the CU has only translational motion; 2) the affine motion parameters are larger than the specified limit because sub-block-based affine MCs are downgraded to CU-based MCs to avoid large memory access bandwidth requirements. A fast coding method is applied to reduce the coding complexity of affine motion estimation using PROF. PROF is not applied in the affine motion estimation stage in the following two cases: a) If this CU is not a root block and its parent block does not select the affine mode as its best mode, PROF is not applied because the possibility that the current CU selects the affine mode as the best mode is low; b) If the magnitudes of the four affine parameters (C, D, E, F) are all less than a predefined threshold and the current picture is not a low-latency picture, PROF is not applied. In this way, affine motion estimation using PROF can be accelerated. 2.1.4. Sub-block-based temporal motion vector prediction (SbTMVP) VVC supports a sub-block-based temporal motion vector prediction (SbTMVP) method. Similar to the temporal motion vector prediction (TMVP) in HEVC, SbTMVP uses the motion field in the co-located picture to improve the motion vector prediction and merge mode for the CU in the current picture. The same co-located picture used by TMVP is used for SbTMVP. SbTMVP differs from TMVP in the following two main aspects: –TMVP predicts motion at the CU level, but SbTMVP predicts motion at the sub-CU level; – Whereas TMVP extracts the temporal motion vector from the co-located block in the co-located picture (the co-located block is the lower right or center block relative to the current CU), SbTMVP applies motion shifting before extracting the temporal motion information from the co-located picture. The motion shift is obtained from the motion vector of one of the spatially neighboring blocks of the current CU. The SbTMVP process is shown in Fig.18a and Fig.18b middle. Fig.18a Schematic diagram 1110 of spatial neighboring blocks used by SbTMVP is illustrated. SbTMVP predicts the motion vector of a sub-CU within the current CU in two steps. In the first step, Fig.18aThe spatial neighbor A1 in is checked. If A1 has a motion vector that uses the co-located picture as its reference picture, then this motion vector is selected as the motion shift to be applied. If no such motion is identified, the motion offset is set to (0,0). Fig.18b A schematic diagram of deriving a sub-CU motion field by applying motion shifts from spatial neighbors and scaling the motion information from the corresponding co-located sub-CU is illustrated. In the second step, the motion shift identified in step 1 is applied (i.e., added to the coordinates of the current block) to obtain sub-CU level motion information (motion vectors and reference indices) from the co-located picture as shown in FIG18 . Fig.18b The example in assumes that the motion shift is set to the motion of block A1. Then, for each sub-CU, the motion information of its corresponding block in the co-located picture (the minimum motion grid covering the center sample) is used to derive the motion information of the sub-CU. After the motion information of the co-located sub-CU is identified, it is converted into the motion vector and reference index of the current sub-CU in a manner similar to the TMVP process of HEVC, where temporal motion scaling is applied to align the reference picture of the temporal motion vector with the reference picture of the current CU. In VVC, a sub-block based merge list containing a combination of both SbTMVP candidates and affine merge candidates is used for signaling of sub-block based merge mode. The SbTMVP mode is enabled / disabled by a sequence parameter set (SPS) flag. If the SbTMVP mode is enabled, the SbTMVP predictor is added as the first entry in the list of sub-block based merge candidates, followed by the affine merge candidates. The size of the sub-block based merge list is signaled in the SPS, and the maximum allowed size of the sub-block based merge list is 5 in VVC. The sub-CU size used in SbTMVP is fixed to 8×8, and as done for affine merge mode, SbTMVP mode is only applicable to CUs whose width and height are both greater than or equal to 8. The encoding logic of the additional SbTMVP merge candidates is the same as that for other merge candidates, that is, for each CU in a P slice or a B slice, an additional RD check is performed to decide whether to use the SbTMVP candidate. 2.1.5. Adaptive Motion Vector Resolution (AMVR) In HEVC, when use_integer_mv_flag is equal to 0 in the slice header, the motion vector difference (MVD) (between the CU's motion vector and the predicted motion vector) is signaled in units of quarter luma samples. In VVC, the CU-level adaptive motion vector resolution (AMVR) scheme is introduced. AMVR allows the MVD of a CU to be encoded and decoded with different precisions. Depending on the mode of the current CU (normal AMVP mode or affine AVMP mode), the MVD of the current CU can be adaptively selected as follows: - Normal AMVP mode: quarter brightness samples, half brightness samples, integer brightness samples, or quad brightness samples. - Affine AMVP mode: quarter luma samples, integer luma samples, or 1 / 16 luma samples. If the current CU has at least one non-zero MVD component, the CU-level MVD resolution indication is conditionally signaled.If all MVD components (ie, both horizontal and vertical MVD for reference list L0 and reference list L1) are zero, then quarter luma sample MVD resolution is inferred. For a CU with at least one non-zero MVD component, a first flag is signaled to indicate whether quarter luma sample MVD precision is used for the CU. If the first flag is 0, no further signaling is required and quarter luma sample MVD precision is used for the current CU. Otherwise, a second flag is signaled to indicate that half luma samples or other MVD precision (integer or four luma samples) are used for normal AMVP CUs. In the case of half luma samples, a 6-tap interpolation filter is used for half luma sample positions instead of the default 8-tap interpolation filter. Otherwise, a third flag is signaled to indicate whether integer luma sample MVD precision or four luma sample MVD precision is used for normal AMVP CUs. In the case of affine AMVP CUs, a second flag is used to indicate whether integer luma sample MVD precision or 1 / 16 luma sample MVD precision is used. In order to ensure that the reconstructed MV has a predetermined precision (quarter luma sample, half luma sample, integer luma sample, or four luma sample), the motion vector predictor for the CU will be rounded to the same precision as the MVD before being added to the MVD. The motion vector predictor is rounded towards zero (that is, a negative motion vector predictor is rounded towards positive infinity, and a positive motion vector predictor is rounded towards negative infinity). The encoder uses the RD check to determine the motion vector resolution of the current CU. In order to avoid always performing four CU-level RD checks for each MVD resolution, in VTM11, the RD check of MVD precision other than quarter luma samples is only conditionally called. For normal AVMP mode, the RD cost of quarter luma sample MVD precision and integer luma sample MV precision is first calculated. Then, the RD cost of integer luma sample MVD precision is compared with the RD cost of quarter luma sample MVD precision to decide whether it is necessary to further check the RD cost of four luma sample MVD precision. When the RD cost of quarter luma sample MVD precision is much smaller than the RD cost of integer luma sample MVD precision, the RD check of four luma sample MVD precision is skipped. Then, if the RD cost of integer luma sample MVD precision is significantly greater than the best RD cost of the previously tested MVD precision, the check of half luma sample MVD precision is skipped. For the affine AMVP mode, if the affine inter mode is not selected after checking the rate-distortion cost of the affine merge / skip mode, merge / skip mode, quarter luma sample MVD accuracy normal AMVP mode, and quarter luma sample MVD accuracy affine AMVP mode, the 1 / 16 luma sample MV accuracy and 1-pixel MV accuracy affine inter modes are not checked. In addition, the affine parameters obtained in the quarter luma sample MV accuracy affine inter mode are used as the starting search points in the 1 / 16 luma sample and quarter luma sample MV accuracy affine inter modes. 2.1.6. Bidirectional Prediction with CU-Level Weights (BCW) In HEVC, a bidirectional prediction signal is generated by averaging two prediction signals obtained from two different reference pictures and / or using two different motion vectors. In VVC, the bidirectional prediction mode is extended beyond simple averaging to allow weighted averaging of the two prediction signals. P bi-pred =((8-w)*P 0 +w*P 1 +4)>>3 (2-18). Five weights, w∈{-2,3,4,5,10}, are allowed in weighted average bidirectional prediction. For each bidirectionally predicted CU, the weight w is determined in one of two ways: 1) for non-merged CUs, the weight index is signaled after the motion vector difference; 2) for merged CUs, the weight index is inferred from neighboring blocks based on the merge candidate index. BCW is only applicable to CUs with 256 or more luma samples (i.e., CU width times CU height is greater than or equal to 256). For low-latency pictures, all 5 weights are used. For non-low-latency pictures, only 3 weights (w∈{3,4,5}) are used. – At the encoder, a fast search algorithm is applied to find the weight index without significantly increasing the encoder complexity. These algorithms are summarized below. For further details, the reader is referred to the VTM software and the document JVET-L0646. When combined with AMVR, if the current picture is a low-latency picture, the unequal weights are only conditionally checked for 1-pixel and 4-pixel motion vector accuracy. - When combined with affine, affine ME will be performed for unequal weights if and only if the affine mode is selected as the current best mode. – When the two reference pictures in bidirectional prediction are the same, unequal weights are only checked conditionally. – Do not search for unequal weights when certain conditions are met, which depend on the POC distance between the current picture and its reference pictures, the codec QP, and the temporal level. The BCW weight index is encoded using one context encoded bit followed by a bypass encoded bit. The first context encoded bit indicates whether equal weights are used; and if unequal weights are used, an additional bit is signaled using bypass codec to indicate which unequal weights are used. Weighted prediction (WP) is a codec tool supported by the H.264 / AVC and HEVC standards to efficiently encode and decode video content in fading situations. Support for WP has also been added to the VVC standard. WP allows weighting parameters (weights and offsets) to be signaled for each reference picture in each of the reference picture lists L0 and L1. Then, during motion compensation, the corresponding (one or more) weights and (one or more) offsets of the reference picture(s) are applied. WP and BCW are designed for different types of video content. In order to avoid the interaction between WP and BCW (which will complicate the VVC decoder design), if the CU uses WP, the BCW weight index is not signaled, and w is inferred to be 4 (i.e., equal weights are applied). For the merged CU, the weight index is inferred from the neighboring blocks based on the merge candidate index. This can be applied to both the normal merge mode and the inherited affine merge mode. For the constructed affine merge mode, affine motion information is constructed based on the motion information of up to 3 blocks. The BCW index of the CU using the constructed affine merge mode is simply set equal to the BCW index of the first control point MV. In VVC, CIIP and BCW cannot be applied jointly to a CU. When encoding and decoding a CU in CIIP mode, the BCW index of the current CU is set to 2, for example, equal weight. 2.1.7. Bidirectional Optical Flow (BDOF) The Bidirectional Optical Flow (BDOF) tool is included in VVC. BDOF (formerly known as BIO) is included in JEM. Compared to the JEM version, BDOF in VVC is a simpler version that requires much less computation, especially in terms of the number of multiplications and multiplier size. BDOF is used to refine the bidirectional prediction signal of a CU at the 4×4 sub-block level. If BDOF meets all the following conditions, it is applicable to a CU: – The CU is encoded and decoded using the “true” bidirectional prediction mode, that is, one of the two reference pictures is before the current picture in the display order, and the other reference picture is after the current picture in the display order; – The distances from the two reference pictures to the current picture (i.e., POC differences) are the same; – Both reference images are short-term reference images; –CU does not use affine mode or SbTMVP merge mode for encoding and decoding; – CU has more than 64 luminance samples; – CU height and CU width are both greater than or equal to 8 luma samples; – BCW weight index indicates equal weight; – WP is not enabled for the current CU; –CIIP mode is not used for the current CU. BDOF is only applicable to the luma component. As its name indicates, BDOF mode is based on the concept of optical flow, which assumes that the motion of objects is smooth. For each 4×4 sub-block, motion refinement (v x ,v y ) is calculated by minimizing the difference between the L0 prediction samples and the L1 prediction samples. Motion refinement is then used to adjust the bi-prediction sample values ​​in the 4x4 sub-block. The following steps are applied in the BDOF process. First, the horizontal and vertical gradients of the two prediction signals are calculated by directly computing the difference between two neighboring samples: and Right now Among them I (k) (i, j) is the sample value at coordinate (i, j) of the prediction signal in list k (k=0, 1), and shift1 is calculated based on the luma bit depth bitDepth as shift1=max(6, bitDepth-6). Then, the gradient S 1 ,S 2 ,S 3 ,S 5 and S 6The autocorrelation and cross-correlation of S 1 =∑ (i,j)∈Ω Abs(ψ x (i,j)),S 3 =∑ (i,j)∈Ω θ(i,j)·Sign(ψ x (i,j)) S 2 =∑ (i,j)∈Ω ψ x (i,j)·Sign(ψ y (i,j)) (2-20) in θ(i,j)=(I (1) (i,j)>>n b )-(I (0) (i,j)>>n b ) where Ω is a 6×6 window around the 4×4 sub-block, and n a and n b The values ​​of are set equal to min(1, bitDepth-11) and min(4, bitDepth-8) respectively. Then, the motion refinement (v) is derived using the cross-correlation and autocorrelation terms as follows: x ,v y ): in th′ BIO =2 max(5,BD-7) . is the round-down (floor) function, and Based on the motion refinement and gradients, the following adjustments are computed for each sample in the 4×4 sub-block: Finally, the BDOF samples of the CU are calculated by adjusting the bidirectional prediction samples as follows: pred BDOF (x,y)=(I (0) (x,y)+I (1) (x,y)+b(x,y)+o offset )>>shift (2-24). These values ​​are chosen so that the multipliers in the BDOF process do not exceed 15 bits and the maximum bit width of the intermediate parameters in the BDOF process is kept within 32 bits. In order to derive the gradient value, some prediction samples I in the list k (k = 0, 1) outside the current CU boundary (k) (i,j) needs to be generated. Fig.19 FIG. 2 is a schematic diagram showing an extended CU region used in BDOF. Fig.19 As depicted in the schematic diagram 1900 of FIG. 1 , the BDOF in VVC uses an extended row / column around the boundary of the CU. In order to control the computational complexity of generating prediction samples outside the boundary, the extended area (in Fig.19 The prediction samples in 1910 are generated by directly taking the reference samples at nearby integer positions without interpolation (using the floor() operation on the coordinates), and the normal 8-head motion compensation interpolation filter is used to generate the CU (in Fig.19 The predicted samples within the CU boundary are represented as 1920 in FIG. These extended sample values ​​are only used in the gradient calculation. For the remaining steps in the BDOF process, if any samples and gradient values ​​outside the CU boundary are needed, these sample values ​​and gradient values ​​are filled (i.e., repeated) from their nearest neighbors. When the width and / or height of a CU is greater than 16 luma samples, it will be split into sub-blocks with a width and / or height equal to 16 luma samples, and the sub-block boundaries are regarded as CU boundaries in the BDOF process. The maximum unit size for the BDOF process is limited to 16×16. For each sub-block, the BDOF process can be skipped. When the SAD between the initial L0 prediction sample and the L1 prediction sample is less than the threshold, the BDOF process is not applied to the sub-block. The threshold is set equal to (8*W*(H>>1)), where W indicates the sub-block width and H indicates the sub-block height. In order to avoid the additional complexity of the SAD calculation, the SAD between the initial L0 prediction sample and the L1 prediction sample calculated in the DVMR process is reused here. If BCW is enabled for the current block, that is, the BCW weight index indicates unequal weights, bidirectional optical flow is disabled. Similarly, if WP is enabled for the current block, that is, luma_weight_lx_flag is 1 for either of the two reference pictures, BDOF is also disabled. BDOF is also disabled when the CU is encoded or decoded in symmetric MVD mode or CIIP mode. 2.1.8. Decoder-side Motion Vector Refinement (DMVR) To improve the accuracy of the MV in merge mode, decoder-side motion vector refinement based on bilateral matching (BM) is applied in VVC. In bidirectional prediction operation, the refined MV is searched around the initial MV in reference picture list L0 and reference picture list L1. The BM method calculates the distortion between two candidate blocks in reference picture list L0 and list L1. Fig. 20FIG. 4 is a schematic diagram showing the refinement of the motion vector at the decoding side. Fig. 20 As shown, based on each MV candidate around the initial MV, the SAD between block 2010 and block 2012 is calculated, where for the current picture 2002, block 2010 is in the reference picture 2001 in list L0 and block 2012 is in the reference picture 2003 in list L1. The MV candidate with the lowest SAD becomes the refined MV and is used to generate a bidirectional prediction signal. In VVC, the application of DMVR is restricted and is only applied to CUs of the following modes and feature codecs: – CU level merge mode with bi-predictive MV; - relative to the current picture, one reference picture is in the past and the other reference picture is in the future; – The distances from the two reference pictures to the current picture (i.e., POC differences) are the same; – Both reference images are short-term reference images; – CU has more than 64 luminance samples; – CU height and CU width are both greater than or equal to 8 luma samples; – BCW weight index indicates equal weight; – WP is not enabled for the current block; –CIIP mode is not used for the current block. The refined MV derived by the DMVR process is used to generate inter prediction samples and is also used in temporal motion vector prediction for future picture coding, whereas the original MV is used in the deblocking process and is also used in spatial motion vector prediction for future CU coding. Additional features of DMVR are mentioned in the following sub-clauses. 2.1.8.1. Search scheme In DVMR, the search point is around the initial MV, and the MV offset obeys the MV difference mirror rule. In other words, any point that passes the DMVR check represented by the candidate MV pair (MV0, MV1) obeys the following two equations: MV0′=MV0+MV_offset (2-25) MV1′=MV1-MV_offset (2-26) Where MV_offset represents the refinement offset between the initial MV and the refined MV in one of the reference pictures. The refinement search range is two integer luma samples from the initial MV. The search includes an integer sample offset search phase and a fractional sample refinement phase. A 25-point full search is applied to the integer sample offset search. The SAD of the initial MV pair is first calculated. If the SAD of the initial MV pair is less than the threshold, the integer sampling stage of DMVR is terminated. Otherwise, the SAD of the remaining 24 points is calculated and checked in raster scan order. The point with the smallest SAD is selected as the output of the integer sample offset search stage. In order to reduce the penalty of uncertainty in DMVR refinement, it is proposed to support the original MV during the DMVR process. The SAD between the reference blocks referenced by the initial MV candidates is reduced by 1 / 4 of the SAD value. The integer sample search is followed by fractional sample refinement. To save computational complexity, fractional sample refinement is derived by using the parameter error surface equation instead of an additional search with SAD comparison. Fractional sample refinement is conditionally called based on the output of the integer sample search stage. When the integer sample search stage terminates with a center with minimum SAD in the first iteration search or the second iteration search, fractional sample refinement is further applied. In the sub-pixel offset estimation based on the parametric error surface, the cost at the center position and the costs at the four neighboring positions from the center are used to fit a 2D parabolic error surface equation of the following form E(x,y)=A(x min ) 2 +B(yy min ) 2 +C (2-27) Where (x min ,y min ) corresponds to the fractional position with the minimum cost, and C corresponds to the minimum cost value. By solving the above equation using the cost values ​​of the five search points, (x min ,y min ) is calculated as: x min =(E(-1,0)-E(1,0)) / (2(E(-1,0)+E(1,0)-2E(0,0))) (2-28) y min =(E(0,-1)-E(0,1)) / (2((E(0,-1)+E(0,1)-2E(0,0))) (2-29). x min and min The value of is automatically limited to between -8 and 8, since all cost values ​​are positive and the minimum is E(0,0). This corresponds to a half-pixel offset with 1 / 16th pixel MV precision in VVC. The calculated fraction (x min ,y min ) is added to the integer distance refinement MV to get the sub-pixel accurate refinement deltaMV. 2.1.8.2. Bilinear interpolation and sample padding In VVC, the resolution of MV is 1 / 16 luminance sample. An 8-tap interpolation filter is used to interpolate samples at fractional positions. In DMVR, the search point is around the initial fractional pixel MV with an integer sample offset, so the samples at these fractional positions need to be interpolated for the DMVR search process. In order to reduce the computational complexity, a bilinear interpolation filter is used to generate fractional samples for the search process in DMVR. Another important effect is that by using a bilinear filter, DVMR does not access more reference samples than the normal motion compensation process in the case of a 2-sample search range. After obtaining the refined MV using the DMVR search process, a normal 8-tap interpolation filter is applied to generate the final prediction. In order not to access more reference samples of the normal MC process, samples will be filled from those available samples, which are not required for the interpolation process based on the original MV, but are required for the interpolation process based on the refined MV. 2.1.8.3. Maximum DMVR processing unit When the width and / or height of a CU is greater than 16 luma samples, it will be further split into sub-blocks with width and / or height equal to 16 luma samples. The maximum unit size for the DMVR search process is limited to 16×16. 2.1.9. Combined Inter and Intra Prediction (CIIP) In VVC, when a CU is encoded and decoded in merge mode, if the CU contains at least 64 luma samples (that is, the CU width multiplied by the CU height is equal to or greater than 64), and if both the CU width and the CU height are less than 128 luma samples, an additional flag is signaled to indicate whether the combined inter- and intra-prediction (CIIP) mode is applicable to the current CU. As its name indicates, CIIP prediction combines the inter-prediction signal with the intra-prediction signal. The inter-prediction signal P in CIIP mode inter is derived using the same inter-prediction process as applied to the normal merge mode; and the intra-prediction signal P intra is derived following the conventional intra prediction process with planar mode. Fig.21 The top and left neighboring blocks used in the CIIP weight derivation are shown. The intra prediction signal and the inter prediction signal are then combined using a weighted average, where the weight values ​​depend on the codec mode of the top and left neighboring blocks (in Fig.21 is calculated as follows: – If the top neighbor is available and intra-coded, set isIntraTop to 1, otherwise set isIntraTop to 0; – If the left neighbor is available and intra-coded, set isIntraLeft to 1, otherwise set isIntraLeft to 0; – If (isIntraLeft+isIntraTop) is equal to 2, wt is set to 3; – Otherwise, if (isIntraLeft+isIntraTop) is equal to 1, wt is set to 2; – Otherwise, set wt to 1. The CIIP forecast is formed as follows: P CIIP =((4-wt)*P inter +wt*P intra +2)>>2 (2-30). 2.1.10. Geometric Partitioning Model (GPM) In VVC, geometric partitioning mode is supported for inter prediction. The geometric partitioning mode is signaled using a CU level flag as a merge mode, where other merge modes include normal merge mode, MMVD mode, CIIP mode, and sub-block merge mode. For each possible CU size w×h=2 excluding 8×64 and 64×8 m ×2 n , where m,n∈{3…6}, a total of 64 partitions are supported through the geometric partitioning mode. Fig. 22 An example of a GPM split with equal angle grouping is shown. When this mode is used, the CU is split into two parts by geometrically positioned straight lines ( Fig. 22 ). The position of the split line is mathematically derived from the angle and offset parameters of the specific partition. Each part of the geometric partition in the CU uses its own motion for inter-frame prediction; each partition only allows unidirectional prediction, that is, each part has one motion vector and one reference index. Unidirectional prediction motion constraints are applied to ensure that only two motion compensated predictions are required per CU, the same as regular bidirectional prediction. If a geometric partitioning mode is used for the current CU, a geometric partitioning index indicating the partitioning mode (angle and offset) of the geometric partitioning and two merge indices (one for each partition) are further signaled. The number of maximum GPM candidate sizes is explicitly signaled in the SPS and the syntax binarization for the GPM merge indices is specified. After predicting each part of the geometric partitioning, a hybrid process with adaptive weights is used to adjust the sample values ​​along the edges of the geometric partitioning. This is the prediction signal for the entire CU and the transform and quantization processes will be applied to the entire CU as in other prediction modes. Finally, the motion field of the CU predicted using the geometric partitioning mode is stored. 2.1.10.1. One-way prediction candidate list construction The unidirectional prediction candidate list is derived directly from the merge candidate list constructed according to the extended merge prediction process. Denote n as the index of the unidirectional prediction motion in the geometric unidirectional prediction candidate list. The LX motion vector of the nth extended merge candidate is used as the nth unidirectional prediction motion vector of the geometric partitioning mode, where X is equal to the parity of n. Fig.23 The unidirectional prediction MV selection for the geometric partitioning mode is shown. These motion vectors are Fig.23 In the case where the corresponding LX motion vector of the n-th extended merge candidate does not exist, the L(1-X) motion vector of the same candidate is used as the unidirectional prediction motion vector of the geometric partitioning mode instead. 2.1.10.2. Blending Along Geometric Partition Edges After predicting each part of the geometric partition using its own motion, blending is applied to the two prediction signals to derive samples around the geometric partition edges. The blending weights for each position of the CU are derived based on the distance between the individual position and the partition edge. The distance from position (x,u) to the partition edge is derived as: where i, j are the angle and offset indices of the geometric partitioning, which depend on the geometric partitioning index transmitted with the signal. x,j and ρ y,j The sign of depends on the angle index i. The weights for each part of the geometric partitioning are derived as follows: wIdxL(x,y)=partIdx? 32+d(x,y):32-d(x,y) w 1 (x,y)=1-w 0 (x,y). partIdx depends on the angle index i. Fig.24 shows the warping weight w using the geometric partitioning pattern 0 The weight w 0 An example of this is shown in Fig.24 middle. 2.1.10.3. Motion Field Storage for Geometry Partitioning Mode Mv1 from the first part of the geometric partitioning, Mv2 from the second part of the geometric partitioning, and a combination Mv of Mv1 and Mv2 are stored in the motion field of the CU coded in the geometric partitioning mode. The type of motion vector stored for each individual position in the motion field is determined as: sType=abs(motionIdx)<32?2:(motionIdx≤0?(1-partIdx):partIdx) where motionIdx is equal to d(4x+2,4y+2). partIdx depends on the angle index i. If sType is equal to 0 or 1, then Mv0 or Mv1 is stored in the corresponding motion field, otherwise if sTType is equal to 2, the combined Mv from Mv0 and Mv2 is stored. The combined Mv is generated using the following process: 1) If Mv1 and Mv2 are from different reference picture lists (one from L0 and the other from L1), then Mv1 and Mv2 are simply combined to form a bi-directional prediction motion vector. 2) Otherwise, if Mv1 and Mv2 are from the same list, only the unidirectional predicted motion Mv2 is stored. 2.1.11. Local Illumination Compensation (LIC) LIC is an inter-frame prediction technique used to model the local illumination variation between the current block and its prediction block as a function of the local illumination variation between the current block template and the reference block template. The parameters of the function can be represented by a scale α and an offset β, which form a linear equation, i.e., α*p[x]+β to compensate for illumination variation, where p[x] is the reference sample pointed to by the MV at position x on the reference picture. Since α and β can be derived based on the current block template and the reference block template, they do not require signaling overhead except for signaling a LIC flag for AMVP mode to indicate the use of LIC. The local illumination compensation proposed in JVET-O0066 is used for unidirectionally predicted inter CUs with the following modifications. Intra-frame neighbor samples can be used in LIC parameter derivation; Disable LIC for blocks with less than 32 luma samples; For both non-sub-block mode and affine mode, LIC parameter derivation is performed based on the template block samples corresponding to the current CU instead of the partial template block samples corresponding to the top-left first 16×16 unit; • Samples of the reference block template are generated by using MC with the block MV without rounding it to integer pixel precision. 2.1.12. Non-adjacent spatial domain candidates Non-adjacent spatial merging candidates as in JVET-L0399 are inserted after the TMVP in the regular merging candidate list. Fig.25 The spatial neighboring blocks used to derive the spatial merge candidates are shown in FIG. Fig.25The distance between non-adjacent spatial candidates and the current codec block is based on the width and height of the current codec block. No line buffer restrictions are applied. 2.1.13. Template Matching (TM) Template Matching (TM) is a decoder-side MV derivation method used to refine the motion information of the current CU by finding the closest match between a template in the current picture (i.e., the top and / or left neighboring blocks of the current CU) and a block in a reference picture (i.e., the same size as the template). Fig.26 2600 is a diagram illustrating template matching performed on a search area around an initial MV. Fig.26 As shown, a better MV is searched around the initial motion of the current CU in the [-8, +8] pixel search range. Template matching previously proposed in JVET-J0021 is adopted in this contribution with two modifications: the search step size is determined based on the adaptive motion vector resolution (AMVR) mode and the TM can be cascaded with the bilateral matching process in merge mode. In AMVP mode, MVP candidates are determined based on template matching error to select the one that achieves the minimum difference between the current block template and the reference block template, and then TM is performed only for this specific MVP candidate for MV refinement. TM refines this MVP candidate starting with full-pixel MVD accuracy (or 4-pixel MVD accuracy for 4-pixel AMVR mode) within the [-8, +8] pixel search range using an iterative diamond search. The AMVP candidate can be further refined by using a cross search with full-pixel MVD accuracy (or 4-pixel MVD accuracy for 4-pixel AMVR mode), and then further refined by using half-pixel and quarter-pixel MVD accuracy depending on the AMVR mode as specified in Table 3. This search process ensures that the MVP candidate still maintains the same MV accuracy as indicated by the AMVR mode after the TM process. Table 3 Search patterns of AMVR and merge patterns with AMVR. In merge mode, a similar search method is applied to the merge candidates indicated by the merge index. As shown in Table 3, TM can be performed until 1 / 8 pixel MVD accuracy or skip those beyond half-pixel MVD accuracy, depending on whether an alternative interpolation filter is used according to the merged motion information (used when AMVR is half-pixel mode). In addition, when TM mode is enabled, template matching can work as an independent process or an additional MV refinement process between a block-based bilateral matching (BM) method and a sub-block-based BM method, depending on whether BM can be enabled according to its enabling condition check. 2.1.14. Multi-pass decoder-side motion vector refinement (mpDMVR) Multiple passes of decoder side motion vector refinement are applied. In the first pass, bilateral matching (BM) is applied to the codec block. In the second pass, BM is applied to each 16×16 sub-block within the codec block. In the third pass, the MVs in each 8×8 sub-block are refined by applying bidirectional optical flow (BDOF). The refined MVs are stored for spatial and temporal motion vector prediction. 2.1.14.1. First pass - Block-based bilateral matching MV refinement In the first pass, refined MVs are derived by applying BM to the codec blocks. Similar to decoder side motion vector refinement (DMVR), in bidirectional prediction operation, refined MVs are searched around two initial MVs (MV0 and MV1) in reference picture lists L0 and L1. Refined MVs (MV0_pass1 and MV1_pass1) are derived around the initial MVs based on the minimum bilateral matching cost between the two reference blocks in L0 and L1. BM performs a local search to derive integer sample precision intDeltaMV. The local search applies a 3×3 square search pattern to loop through the search range [–sHor, sHor] in the horizontal direction and the search range [–sVer, sVer] in the vertical direction, where the values ​​of sHor and sVer are determined by the block dimension and the maximum value of sHor and sVer is 8. The bilateral matching cost is calculated as: bilCost = mvDistanceCost + SadCost. When the block size cbW*cbH is greater than 64, the MRSAD cost function is applied to remove the DC effect of the distortion between the reference blocks. When the bilCost at the center point of the 3×3 search pattern has the minimum cost, the intDeltaMV local search is terminated. Otherwise, the current minimum cost search point becomes the new center point of the 3×3 search pattern and continues to search for the minimum cost until it reaches the end of the search range. The existing fractional sample refinement is further applied to derive the final deltaMV. The refined MV after the first pass is then exported as: MV0_pass1 = MV0 + deltaMV; ·MV1_pass1=MV1–deltaMV. 2.1.14.2. Second pass - sub-block based bilateral matching MV refinement In the second pass, refined MVs are derived by applying BM to 16×16 grid sub-blocks. For each sub-block, the refined MVs are searched around the two MVs (MV0_pass1 and MV1_pass1) obtained at the first pass in the reference picture lists L0 and L1. The refined MVs (MV0_pass2(sbIdx2) and MV1_pass2(sbIdx2)) are derived based on the minimum bilateral matching cost between the two reference sub-blocks in L0 and L1. For each subblock, BM performs a full search to derive integer sample precision intDeltaMV. The full search has a horizontal search range of [-sHor, sHor] and a vertical search range of [-sVer, sVer], where the values ​​of sHor and sVer are determined by the block dimension and the maximum value of sHor and sVer is 8. The bilateral matching cost is calculated by applying the cost factor to the SATD cost between the two reference sub-blocks as: bilCost = satdCost * costFactor. The search area (2*sHor+1)*(2*sVer+1) is divided into Fig. 27 Up to 5 diamond-shaped search regions are shown in FIG. 2700. Each search region is assigned a costFactor, which is determined by the distance (intDeltaMV) between each search point and the starting MV, and each diamond region is processed sequentially starting from the center of the search region. In each region, the search points are processed in a raster scan order starting from the upper left corner to the lower right corner of the region. When the minimum bilCost in the current search region is less than or equal to the threshold of sbW*sbH, the integer pixel full search is terminated, otherwise, the integer pixel full search continues to the next search region until all search points are checked. The existing VVC DMVR fractional sample refinement is further applied to derive the final deltaMV (sbIdx2). The refined MV in the second pass is then derived as: ·MV0_pass2(sbIdx2)=MV0_pass1+deltaMV(sbIdx2), ·MV1_pass2(sbIdx2)=MV1_pass1-deltaMV(sbIdx2). 2.1.14.3. Third pass - sub-block based bidirectional optical flow MV refinement In the third pass, refined MVs are derived by applying BDOF to 8×8 grid sub-blocks. For each 8×8 sub-block, BDOF refinement is applied to derive scaled Vx and Vy without clipping from the refined MV of the parent sub-block of the second pass. The derived bioMv(Vx, Vy) is rounded to 1 / 16 sample precision and clipped between -32 and 32. The refined MVs in the third pass (MV0_pass3(sbIdx3) and MV1_pass3(sbIdx3)) are derived as: ·MV0_pass3(sbIdx3)=MV0_pass2(sbIdx2)+bioMv, ·MV1_pass3(sbIdx3)=MV0_pass2(sbIdx2)–BioMv. 2.1.15.OBMC When OBMC is applied, the top and left boundary pixels of the CU are refined using motion information of neighboring blocks in a weighted prediction as described in JVET-L0101. The conditions under which OBMC should not be applied are as follows: When OBMC is disabled at SPS level; When the current block has intra mode or IBC mode; When the current block applies LIC; When the current luminance block area is less than or equal to 32. Sub-block boundary OBMC is performed by applying the same blending to the top, left, bottom and right sub-block boundary pixels using the motion information of the neighboring sub-blocks. It is enabled for sub-block based codecs: Affine AMVP mode; Affine merge mode and sub-block based temporal motion vector prediction (SbTMVP); Sub-block based bilateral matching. 2.1.16. Sample-based BDOF In sample-based BDOF, instead of deriving the motion refinement (Vx, Vy) on a block basis, it is performed per sample. The codec block is divided into 8×8 sub-blocks. For each sub-block, whether to apply BDOF is determined by checking the SAD between two reference sub-blocks against a threshold. If it is decided to apply BDOF to a sub-block, for each sample in the sub-block, a sliding 5×5 window is used and the existing BDOF process is applied for each sliding window to derive Vx and Vy. The derived motion refinement (Vx, Vy) is applied to adjust the bidirectional prediction sample value of the center sample of the window. 2.1.17. Interpolation The 8-tap interpolation filter used in VVC is replaced with a 12-tap filter. The interpolation filter is derived from a sinc function whose frequency response is cut off at the Nyquist frequency and is clipped by a cosine window function. Table 4 gives the filter coefficients for all 16 phases. Fig.28 The frequency responses of the interpolation filter and the VVC interpolation filter at half-pixel phase are shown. It compares the frequency responses of the interpolation filter and the VVC interpolation filter all at half-pixel phase. Table 4. Filter coefficients for 12-tap interpolation filter 2.1.18. Multiple Hypothesis Prediction (MHP) In the multi-hypothesis inter prediction mode (JVET-M0425), one or more additional motion compensated prediction signals are signaled in addition to the conventional bidirectional prediction signal. The resulting overall prediction signal is obtained by sample-by-sample weighted superposition. bi and the first additional inter-frame prediction signal / hypothesis h 3 , the resulting prediction signal p 3 is obtained as follows: p 3 =(1-α)p bi +αh 3 . The weight factor α is specified by a new syntax element add_hyp_weight_idx according to the following mapping. add_hyp_weight_idx α 0 1 / 4 1 -1 / 8 Similar to above, more than one additional prediction signal may be used.The resulting overall prediction signal is iteratively accumulated with each additional prediction signal. p n+1 =(1-α n+1 ) n +α n+1 h n+1 Get the overall prediction signal as the last p n (i.e., the p with the largest index n n n). Within this EE, at most two additional prediction signals may be used (i.e., n is limited to 2). The motion parameters for each additional prediction hypothesis can be signaled explicitly by specifying the reference index, motion vector predictor index and motion vector difference, or implicitly by specifying the merge index. A separate multi-hypothesis merge flag distinguishes these two signaling modes. For inter-AMVP mode, MHP is applied only when unequal weights in BCW are selected in bi-prediction mode. A combination of MHP and BDOF is possible, however BDOF is only applicable to the bidirectional prediction signal part of the prediction signal (ie the original first two hypotheses). 2.1.19. Adaptive Re-ranking of Merging Candidates by Template Matching (ARMC-TM) The merge candidates are adaptively reordered by Template Matching (TM). The reordering method is applicable to the regular merge mode, Template Matching (TM) merge mode and Affine merge mode (excluding SbTMVP candidates). For TM merge mode, the merge candidates are reordered before the refinement process. After constructing the merge candidate list, the merge candidates are divided into several subgroups. For the normal merge mode and the TM merge mode, the subgroup size is set to 5. For the affine merge mode, the subgroup size is set to 3. The merge candidates in each subgroup are reordered in ascending order according to the cost value based on template matching. For simplicity, the merge candidates in the last subgroup instead of the first subgroup are not reordered. The template matching cost of the merge candidate is measured by the sum of absolute differences (SAD) between the samples of the template of the current block and its corresponding reference samples. The template includes a set of reconstructed samples adjacent to the current block. The reference samples of the template are located by the motion information of the merge candidate. Fig.29 A schematic diagram 2900 is shown of a template and reference samples of the template in reference list 0 and reference list 1. When the merge candidate uses bidirectional prediction, the reference samples of the template of the merge candidate are also Fig.29 When the merge candidate uses bidirectional prediction, the reference sample of the template of the merge candidate is represented by RT, and RT can be derived from the reference picture 2920 in the reference picture list 0. 0 and RT derived from reference picture 2930 in reference picture list 1 1 In one example, RT 0 A set of reference samples on the reference picture 2920 including the current block in the current picture 2910 is indicated by the reference index of the merge candidate of the reference picture in the reference list 0 using the MV of the merge candidate of the reference list 0. In one example, RT 1 A set of reference samples indicated by the MV of the merge candidate of the reference list 1 through the reference index of the merge candidate of the reference picture in the reference list 1 on the reference picture 2930 including the current block. For a sub-block based merge candidate with a sub-block size equal to Wsub×Hsub, the above template includes several sub-templates with a size of Wsub×1, and the left template includes several sub-templates with a size of 1×Hsub. Fig.30A template and a reference sample of the template for a block with sub-block motion using motion information of a sub-block of a current block are shown. Fig.30 As shown, the motion information of the sub-blocks in the first row and first column of the current block is used to derive the reference samples of each sub-template. 2.1.20. Geometric Partitioning Mode (GPM) with Merged Motion Vector Difference (MMVD) GPM in VVC is extended by applying motion vector refinement on top of the existing GPM unidirectional MV. First, a flag is signaled for the GPMCU to specify whether to use this mode. If this mode is used, each geometric partition of the GPM CU can further decide whether to signal MVD. If MVD is signaled for the geometric partition, after selecting the GPM merge candidate, the motion of the partition is further refined by the signaled MVD information. All other processes remain the same as in GPM. The MVD is signaled as a distance and direction pair, similar to that in MMVD. There are nine candidate distances (1 / 4 pixel, 1 / 2 pixel, 1 pixel, 2 pixels, 3 pixels, 4 pixels, 6 pixels, 8 pixels, 16 pixels) and eight candidate directions (four horizontal / vertical directions and four diagonal directions) involved in GPM with MMVD (GPM-MMVD). In addition, when pic_fpel_mmvd_enabled_flag is equal to 1, the MVD is left shifted by 2 as in MMVD. 2.1.21. Geometric Partitioning Pattern (GPM) with Template Matching (TM) Template matching is applied to GPM. When GPM mode is enabled for a CU, a CU-level flag is signaled to indicate whether TM is applied to two geometric partitions. The motion information of each geometric partition is refined using TM. When TM is chosen, the template is constructed using the left, top, or left and top neighboring samples depending on the partition angle, as shown in Table 5. The motion is then refined by minimizing the difference between the current template and the template in the reference picture using the same search pattern of merge mode with half-pixel interpolation filter disabled. Table 5 Templates for the 1st geometric partitioning and the 2nd geometric partitioning, where A means using the upper samples, L means using the left samples, and L+A means using both the left samples and the upper samples. The GPM candidate list is constructed as follows: 1. The interleaved List-0 MV candidates and List-1 MV candidates are derived directly from the regular merge candidate list, where List-0 MV candidates have higher priority than List-1 MV candidates. A pruning method with adaptive threshold based on current CU size is applied to remove redundant MV candidates. 2. Interleaved List-1 MV candidates and List-0 MV candidates are further derived directly from the regular merge candidate list, where List-1 MV candidates have higher priority than List-0 MV candidates. The same pruning method with adaptive threshold is also applied to remove redundant MV candidates. 3. Zero MV candidates are filled until the GPM candidate list is full. GPM-MMVD and GPM-TM are exclusively enabled for one GPM CU. This is done by first signaling the GPM-MMVD syntax. When both GPM-MMVD control flags are equal to false (i.e., GPM-MMVD is disabled for both GPM partitions), the GPM-TM flag is signaled to indicate whether template matching is applied to both GPM partitions. Otherwise (at least one GPM-MMVD flag is equal to true), the value of the GPM-TM flag is inferred to be false. 2.1.22. GPM with inter-frame and intra-frame prediction (GPM inter-intra) With GPM inter-intra, in addition to merging candidates for each non-rectangular split region in the CU to which GPM is applied, predefined intra prediction modes along geometric dividing lines can also be selected. In the proposed method, a flag from the encoder is used to determine whether the intra prediction mode or the inter prediction mode is for each GPM separation region. When it is inter prediction mode, a unidirectional prediction signal is generated from the MV from the merged candidate list. On the other hand, when it is intra prediction mode, the unidirectional prediction signal is generated from neighboring pixels for the intra prediction mode specified by the index from the encoder. The variation of possible intra prediction modes is subject to geometric constraints. Finally, the two unidirectional prediction signals are mixed in the same way as ordinary GPM. 2.1.23. Adaptive decoder-side motion vector refinement (Adaptive DMVR) The adaptive decoder-side motion vector refinement method includes two new merge modes introduced to refine the MV in one direction (L0 or L1) of bidirectional prediction for merge candidates that meet the DMVR conditions. A multi-pass DMVR process is applied to the selected merge candidate to refine the motion vector, however MVD0 or MVD1 is set to zero in the first pass (i.e. PU level) DMVR. Like the regular merge mode, the merge candidates for the proposed merge mode are derived from spatially adjacent coded blocks, TMVP, non-adjacent blocks, HMVP and paired candidates. The difference is that only those merge candidates that satisfy the DMVR condition are added to the candidate list. Both proposed merge modes use the same merge candidate list, and the merge index is encoded and decoded as in the regular merge mode. 2.1.24. Bilateral Matching AMVP-MERGE Mode (AMVP-MERGE) In AMVP merge mode, a bidirectional predictor consists of an AMVP predictor in one direction and a merged predictor in the other direction. The AMVP part of the proposed mode is signaled as regular unidirectional AMVP, i.e., the reference index and MVD are signaled, and it has a derived MVP index if template matching is used (TM_AMVP), or the MVP index is signaled when template matching is disabled. The merge index is not signaled, and the merged predictor is selected from the candidate list with the minimum template or bilateral matching cost. When the selected merged predictor and AMVP predictor meet the DMVR condition, that is, there is at least one reference picture from the past and one reference picture from the future relative to the current picture, and the distances from the two reference pictures to the current picture are the same, bilateral matching MV refinement is applied with the merged MV candidate and AMVP MVP as the starting point. Otherwise, if the template matching function is enabled, template matching MV refinement is applied to the merged predictor or AMVP predictor with a higher template matching cost. The third pass of 8×8 sub-PU BDOF refinement as multi-pass DMVR is enabled for the AMVP merge mode codec block. 2.2. Out-of-bounds problem in bidirectional motion compensation In ECM-3.0, due to the reference sample filling of the reference picture, for inter-frame CU Fig.31 The illustrated reference blocks are valid if they are partially or fully located outside the reference picture. Fig.31 The bidirectional MC block in the current ECM is shown. Fig.31 In ECM, bidirectional motion compensation is performed to generate an inter-frame prediction block for the current block. In this example, the list 0 reference block is partially outside the boundary (OOB), while the list 1 reference block is completely inside the reference picture. However, the OOB portion of the motion compensated block generally provides lower prediction efficiency because the OOB portion is just repeated samples derived from boundary samples within the reference picture. However, in ECM, the lower efficiency of the OOB portion is not considered for inter-frame prediction. 3. Combined inter-frame intra-frame prediction and transform Interaction between Adaptive DMVR and other tools 1. In one example, Adaptive DMVR can be applied to other codecs besides Adaptive DMVR itself. a. For example, adaptive DMVR may refer to a DMVR method that fixes a motion vector in one prediction direction (such as LX) and then refines the motion vector in another direction (such as L(1-X)), where the motion vector is predicted bidirectionally. b. For example, the motion vectors in the merge candidate list can be further refined by adaptive DMVR. c. For example, the motion vectors of CIIP can be further refined by adaptive DMVR. d. For example, the motion vectors of GPM can be further refined by adaptive DMVR. e. For example, the motion vectors of MMVD can be further refined by adaptive DMVR. f. For example, the motion vectors of sub-block merging (eg, affine merging, sub-block TMVP, etc.) can be further refined by adaptive DMVR. g. For example, the motion vectors between AMVP frames can be further refined by adaptive DMVR. h. For example, the motion vectors of SMVD can be further refined by adaptive DMVR. i. For example, the motion vector of a sub-block AMVP (eg, affine AMVP, etc.) may be further refined by adaptive DMVR. j. For example, when adaptive DMVR is applied to other codecs, signaling of the use of adaptive DMVR for a block may be unnecessary. i. For example, if certain conditions are met, adaptive DMVR is applied to a block without signaling. 2. For example, whether to use adaptive DMVR or conventional DMVR (and / or BDMVR) at the video unit level Can be signaled in the bitstream. a. For example, the video unit level can be the level of sequence / picture / slice / tile group / tile / sub-picture / PB / TB / CB / PU / TU / CU / VPDU / CTU / CTU row / other kinds of regions containing more than one sample or pixel. Interaction between AMVP-MERGE and other tools 3. In one example, AMVP-MERGE can be applied to other codecs besides AMVP-MERGE itself. a. For example, AMVP-MERGE may refer to an inter-frame coding method that generates a motion vector based on the LX motion of an AMVP candidate and the L(1-X) motion of a MERGE candidate. b. For example, the motion vector generated by AMVP-MERGE can be used in CIIP. c. For example, the motion vectors generated by AMVP-MERGE can be used in GPM. d. For example, the motion vectors generated by AMVP-MERGE can be used in MMVD. e. For example, the motion vector generated by AMVP-MERGE can be used in sub-block merging (eg, affine merging, sub-block TMVP, etc.). f. For example, the motion vectors generated by AMVP-MERGE can be used in GPM. g. For example, the motion vectors generated by AMVP-MERGE can be used in AMVP inter prediction (eg, as AMVP candidates for conventional AMVP inter coding). h. For example, the motion vector generated by AMVP-MERGE can be used in SMVD. i. For example, when the motion vector generated by AMVP-MERGE is used in other codecs, it can be perceived as a motion candidate. 4. Decoder side MHP data export_V0 1. In one example, a hypothetical prediction may be generated based on virtually constructed motion data that is not identical to any motion data in the original MHP candidate list. a. For example, predictions for additional hypotheses may be generated based on virtually constructed motion data. b. For example, the movement of virtual constructs can be based on the AMVP-MERGE candidate list. i. For example, an AMVP-MERGE candidate list may be generated based on at least one type of AMVP motion candidate information and at least one type of MERGE motion candidate information. ii. For example, the AMVP-MERGE candidate list may be reordered. iii. For example, the motion candidates in the AMVP-MERGE motion candidate list may be refined. c. For example, the motion of the virtual construct can be based on the bidirectional virtual motion candidate. i. For example, bidirectional virtual motion candidates may be generated based on a certain AMVP candidate list. ii. For example, the bidirectional virtual motion candidates may be generated based on one of the following MERGE candidate lists: a regular merge candidate list, an MMVD candidate list, a TM merge list, a GEO merge list, a CIIP merge list, a sub-block merge list, etc. iii. For example, bidirectional virtual motion candidates may be generated based on the reordered motion candidate list. iv. For example, bidirectional virtual motion candidates may be further refined. d. For example, the motion of the virtual construct can be based on a unidirectional virtual motion candidate. i. For example, one-way virtual motion candidates may be generated based on a certain AMVP candidate list. ii. For example, the unidirectional virtual motion candidate may be generated based on one of the following MERGE candidate lists: a regular merge candidate list, an MMVD candidate list, a TM merge list, a GEO merge list, a CIIP merge list, a sub-block merge list, etc. iii. For example, a unidirectional virtual motion candidate may be generated based on the reordered motion candidate list. iv. For example, one-way virtual motion candidates can be further refined. e. For example, predictions of basic assumptions can be generated based on virtually constructed motion data. i. For example, an AMVP-MERGE codec block may be perceived as a base hypothesis, and at least one additional hypothesis may be applied thereto. Finally, a final prediction of the AMVP-MERGE codec block is generated by mixing the base hypothesis and the additional hypothesis together. 5. Question There are several problems in the existing video coding and decoding technology, which will be further improved to obtain higher coding and decoding gain. 1. In ECM-3.0, the existing AMVP-MERGE mode only supports 1 / 4 pixel precision MVD signaling, where AMVR is not supported. This can be further improved. 2. In ECM-3.0, AMVP-MERGE prediction cannot be used as one of the MHP assumptions, which may be suboptimal. 3. In ECM-3.0, the existing AMVP-MERGE mode generates a new prediction candidate list, which contains several bidirectional prediction motion candidates. Such motion candidates can be used in other coding modes to obtain higher coding efficiency. 4. In ECM-3.0, the existing adaptive DMVR mode generates a new prediction candidate list, which only contains bidirectional prediction motion candidates. Such motion candidates can be used in other codec modes to obtain higher codec efficiency. 5. In ECM-3.0, AMVP-MERGE mode uses DMVR and TM to determine (L0, L1) prediction candidate pairs and refine the prediction candidates. However, this codec tool is not controlled by the DMVR / TM / DMVR control flag, which can be further designed. 6. In ECM-3.0, the LIC mode is designed based on the assumption that there is a strong linear correlation between the neighboring samples of the current block and the temporal co-located block. More specifically, the LIC mode uses a least squares model with two parameters (such as a and b) to map the neighboring samples of the current block and the neighboring samples of the temporal co-located block. However, this assumption may not be obvious, especially when the neighbors and the reference are noisy. In such cases, the actual prediction may fail and the process leads to suboptimal coding efficiency. 7. In ECM-3.0, the LIC for AMVP is signaled in association with a video unit, and the LIC for MERGE is inherited from an adjacent video unit. The signaling and determination of the LIC can be changed using a decoder-derived method. 8. In ECM-3.0, for bidirectional prediction, MHP prediction, GPM prediction, etc., a hypothetical prediction block is generated by weighting multiple hypothetical predictions. Due to repeated picture boundary filling, the weighting process is performed anyway, regardless of whether any prediction sample is out of bounds in any hypothetical prediction. However, further consideration can be given to how to deal with out-of-bounds prediction samples. 9. In ECM-5.0, non-average weighting (eg, BCW) is not applied to AMVP-MERGE mode and MHP mode, which may not be efficient. 10. In ECM-5.0, only AMVP-MERGE mode is allowed to have true bidirectional reference pictures (both forward and backward) for a picture, and BCW is not allowed (the weights for the two reference pictures are set to average weighting by default), which may not be efficient. 11. In ECM-5.0, the DMVR-based method is not used for affine AMVP or affine MERGE's sbTMVP and CPMV, which may not be efficient. 6. Embodiments of the present disclosure The following detailed implementation examples should be considered as examples to illustrate the general concept. These embodiments should not be interpreted narrowly. In addition, these embodiments can be combined in any way. The term “video unit” or “codec unit” or “block” may refer to a codec tree block (CTB), a codec tree unit (CTU), a codec block (CB), CU, PU, ​​TU, PB, TB. In the present disclosure, regarding “encoding and decoding blocks in mode N”, “mode N” here can be a prediction mode (for example, MODE_INTRA, MODE_INTER, MODE_PLT, MODE_IBC, etc.) or a coding and decoding technology (for example, AMVP, Merge, SMVD, BDOF, PROF, DMVR, AMVR, TM, Affine, CIIP, GPM, GEO, TPM, MMVD, BCW, HMVP, SbTMVP, etc.). In the present disclosure, "bidirectional DMVR" may refer to conventional DMVR that refines both L0 motion vectors and L1 motion vectors, as described in detail in Section 2.1.14. In addition, "unidirectional DMVR" may refer to a DMVR process that refines only L0 motion vectors or L1 motion vectors, such as the adaptive DMVR described in detail in Section 2.1.23. In the following discussion, LIC parameters may refer to two parameters (such as slope parameter "a" and bias parameter "b") derived based on a linear model that is used to map neighboring samples of a current block and neighboring samples of a temporal co-located block (e.g., the temporal co-located block may be pointed to by a motion vector or a rounded motion vector of the current block). In addition, LIC parameters may be used to estimate prediction values ​​of samples within the current video unit. In the following discussion, the AMVP mode may be a conventional AMVP mode, an affine AMVP mode, and / or an SMVD mode, and / or an AMVP-MERGE mode. Note that the terms mentioned below are not limited to the specific terms defined in existing standards. Any variants of the codec tool are also applicable. 6.1. To deal with the first problem, the following method is proposed: a. Propose to support multiple MVD / MV precisions for AMVP-MERGE mode. i. In one example, the supported precision candidates may be the same as those used for normal AMVP mode, e.g., for the non-affine case, half pixel, 1 / 4 pixel, 1 pixel, 4 pixels are applied; for the affine case, 1 / 16 pixel, 1 / 8 pixel, 1 / 4 pixel are applied. ii. In another example, at least one of the supported precision candidates may be different from the precision candidate used for normal AMVP mode. b. For example, the motion vector difference (eg, MVD) on the AMVP side of the AMVP-MERGE mode may be encoded and decoded with other precisions other than 1 / 4 pixel resolution. i. For example, the MVD value can be 4-pixel accurate. ii. For example, the MVD value can be 1 pixel accurate. iii. For example, the MVD value can be half-pixel accurate. iv. For example, the MVD value can be 1 / 8 pixel accurate. v. For example, the MVD value can be 1 / 16 pixel accurate. c. For example, a second interpolation filter (eg, a Y-tap filter such as Y=6) in addition to the first interpolation filter (eg, an X-tap filter such as X=8 or 12) may also be used for motion compensation. i. For example, when to use the second interpolation filter for an AMVP-MERGE codec block may depend on the MVD prediction and / or the final MV accuracy. ii. For example, when the signaled MVD is 1 / 2 pixel accurate and the final MV is also 1 / 2 pixel accurate, the second interpolation filter may be used. Otherwise, the first interpolation filter may be used. iii. In one example, the second interpolation filter used in the AMVP-MERGE mode may be the same as the second interpolation filter used for the normal AMVP mode (eg, the second interpolation filter for 1 / 2 pixel precision). iv. Alternatively, the second interpolation filter used in the AMVP-MERGE mode may be different from the second interpolation filter used for the normal AMVP mode (eg, an interpolation filter for 1 / 2 pixel precision). d. Alternatively, half-pixel MVD accuracy may not be allowed for AMVP-MERGE mode. e. For example, at least one syntax element (e.g., a flag and / or a parameter index) may be signaled at a block level to indicate which motion vector precision is used to encode / signal the MVD value and / or MV of a video unit encoded in AMVP-MERGE mode. i. In addition, the MVD signaled at any resolution can be converted to internal precision (e.g., 1 / 16 pixel resolution) for subsequent processes such as motion compensation, etc. 6.2. To deal with the second problem, the following method is proposed: a. In one example, the prediction units generated based on the AMVP-MERGE mode can be used as the MHP hypothesis. i. For example, the AMVP-MERGE mode prediction block can be used as the basic assumption of the MHP block. ii. For example, syntax elements / structures related to MHP assumption data may be signaled in the bitstream just after a video unit is identified as a video unit encoded in AMVP-MERGE mode (e.g., whether there are additional assumptions associated with the video unit encoded in AMVP-MERGE mode; if so, whether the MHP additional assumptions based on AMVP or MERGE are used, etc.). iii. For example, the AMVP-MERGE prediction block can be used as an additional hypothesis for the MHP block. iv. For example, additional hypotheses for MHP blocks may be generated based on AMVP-MERGE motion candidates. 1) For example, in this case, syntax elements related to this AMVP-MERGE motion candidate (e.g., which side is AMVP / MERGE encoded or decoded, the reference index of the AMVP side, the MVD value of the AMVP side, and / or the MVP index side of the AMVP) can be signaled in a multi-hypothesis data structure. v. For example, additional assumptions on whether LIC is used for AMVP-MERGE codec may be inherited from the base assumption on the use of LIC. 1) For example, if the base hypothesis is LIC codec, then the additional hypothesis of AMVP-MERGE codec is LIC codec, without signaling the use of LIC for this additional hypothesis. 2) Alternatively, if the base hypothesis is not LIC coded, then the additional hypothesis of AMVP-MERGE codec is also not LIC coded, without signaling the use of LIC for such additional hypothesis. vi. For example, the assumptions (base assumptions and / or additional assumptions) of whether LIC is used for AMVP-MERGE encoding and decoding may depend on the use of LIC on the merging side of the AMVP-MERGE candidate. 1) For example, suppose a certain AMVP-MERGE candidate consists of a uni-merge candidate in one side (L0 or L1) and a uni-AMVP candidate in the other side (L1 or L0), a. For example, the usage of LIC for such an AMVP-MERGE candidate may be inherited from its merge candidate. b. For example, if the merge candidate uses LIC, then LIC can be used for the prediction block of AMVP-MERGE codec. vii. Alternatively, the use of an assumed LIC for the AMVP-MERGE codec may be signaled in the bitstream. 6.3. To deal with the third problem, the following method is proposed: a. For example, the AMVP-MERGE candidate can be used in one or more of the following encoding and decoding modes. i. CIIP mode (and / or its variants, such as conventional CIIP, CIIP-PDPC, CIIP-TM, etc.). ii. MMVD mode (and / or its variants, such as conventional MMVD, affine MMVD, etc.). iii. MHP model (and / or its variants, such as MHP basic assumptions and / or MHP additional assumptions, etc.). iv. GPM mode (and / or its variants, such as conventional GPM, GPM-TM, GPM-MMVD, GPM-Inter-Intra, etc.). b. For example, the AMVP-MERGE candidate may be first refined by a decoder-side motion vector refinement process (eg, TM- or DMVR-based motion vector refinement) and then used for a second encoding / decoding mode (eg, such as listed in the above sub-item). c. For example, an AMVP-MERGE candidate may be inserted into another candidate list. i. For example, AMVP-MERGE candidates can be inserted into the regular merge candidate list. 1) For example, AMVP-MERGE candidates can be used in regular merge mode and / or its variants. 2) For example, AMVP-MERGE candidates may be used in MMVD mode and / or its variants. 3) For example, AMVP-MERGE candidates may be used in CIIP mode and / or its variants. 4) For example, the AMVP-MERGE candidate may be used in the MHP mode and / or its variants. 5) For example, the AMVP-MERGE candidate may be used in the GPM mode and / or its variants. ii. For example, AMVP-MERGE candidates may be inserted into the regular TM merge candidate list. 1) For example, AMVP-MERGE candidates can be used in conventional TM merge mode and / or its variants. iii. Additionally, furthermore, AMVP-MERGE candidates may be inserted into another prediction list after the original candidates of that prediction list. d. For example, the AMVP-MERGE candidates may be reordered based on a decoder-derived method (by TM- or DMVR-based cost evaluation), and then M of them will be selected to be added to a second candidate list (such as a regular merge candidate list, a regular TM merge candidate list). i. Additionally, more than one AMVP-MERGE candidate may be re-ordered together. ii. Alternatively, the first candidate from the first AMVP-MERGE prediction list and the second candidate from the second prediction list may be reordered together. e. For example, once an AMVP-MERGE candidate is used for a codec block, additional syntax elements may be signaled to specify the prediction direction (L0 or L1) of the AMVP portion and / or the reference picture index of the selected AMVP candidate, and / or the motion vector prediction index of the selected AMVP candidate, and / or the motion vector difference associated with the AMVP motion vector predictor. i. Alternatively, the motion vector predictor index on the AMVP side of the AMVP-MERGE candidate may not be signaled (eg, it may be selected by a decoder-side method via a TM or DMVR based cost evaluation). f. Alternatively, once AMVP-MERGE candidates are used, additional syntax element(s) may be signaled to specify the predictor index of the merge candidate. i. Alternatively, the motion vector predictor index of the merge side of the AMVP-MERGE candidate may not be signaled (eg, it may be selected by a decoder-side method via a TM or DMVR based cost evaluation). g. In one example, before generating AMVP-MERGE candidates, the merged portion of the AMVP-MERGE mode may first be refined by a decoder-side motion refinement process (such as TM or DMVR). h. In one example, the AMVP portion of the AMVP-MERGE mode may first be refined by a decoder-side motion vector refinement process (such as TM or DMVR) before generating AMVP-MERGE candidates. 6.4. To deal with the fourth problem, the following method is proposed: a. For example, adaptive DMVR motion candidates can be used in one or more of the following codec modes. i. CIIP mode (and / or its variants, such as conventional CIIP, CIIP-PDPC, CIIP-TM, etc.). ii. MMVD mode (and / or its variants, such as conventional MMVD, affine MMVD, etc.). iii. MHP model (and / or its variants, such as MHP basic assumptions and / or MHP additional assumptions, etc.). iv. GPM mode (and / or its variants, such as conventional GPM, GPM-TM, GPM-MMVD, GPM-Inter-Intra, etc.). v. AMVP mode (and / or its variants, such as regular AMVP, SMVD, AMVP-MERGE, affine AMVP, etc.). b. For example, the adaptive DMVR motion candidates may be first refined by a decoder-side motion vector refinement process (eg, TM- or DMVR-based motion vector refinement) and then used in a second encoding / decoding mode (eg, such as listed in the above sub-item). c. For example, when adaptive DMVR motion candidates are used in AMVP mode, i. A DMVR motion candidate may refer to a motion vector pair containing both an L0 motion vector and an L1 motion vector, and / or both an L0 reference picture index and an L1 reference picture index. ii. It can be used as an MVP candidate. iii. The candidate indices of DMVR motion candidates (instead of L0 and L1 motion vector predictor indices and L0 and L1 reference picture indices) may be signaled in the bitstream for AMVP mode. iv. The reference picture index for one prediction direction (L0 or L1) may be signaled for AMVP mode, and the reference picture index for the other prediction direction (L1 or L0) may be inferred (eg, based on DMVR conditions). d. For example, the adaptive DMVR motion candidate can be inserted into another candidate list. i. For example, adaptive DMVR motion candidates can be inserted into the regular merge candidate list. 1) For example, adaptive DMVR motion candidates can be used in the regular merge mode and / or its variants. 2) For example, adaptive DMVR motion candidates can be used in MMVD mode and / or its variants. 3) For example, adaptive DMVR motion candidates can be used in CIIP mode and / or its variants. 4) For example, adaptive DMVR motion candidates can be used in MHP mode and / or its variants. 5) For example, adaptive DMVR motion candidates can be used in GPM mode and / or its variants. ii. For example, adaptive DMVR motion candidates can be inserted into the regular TM merge candidate list. 1) For example, adaptive DMVR motion candidates can be used in the conventional TM merge mode and / or its variants. iii. Additionally, furthermore, the adaptive DMVR motion candidates may be inserted into another prediction list after the original candidates of that prediction list. e. For example, the adaptive DMVR motion candidates may be reordered based on a decoder-derived method (via TM or DMVR-based cost evaluation), and then M of them will be selected to be added to a second candidate list (such as a regular merge candidate list, a regular TM merge candidate list). i. Additionally, more than one adaptive DMVR motion candidates may be reordered together. ii. Alternatively, the first candidate from the first adaptive DMVR merge list and the second candidate from the second prediction list may be reordered together. 6.5. To deal with the fifth problem, the following method is proposed: a. For example, the enabling / disabling of the first codec tool may be controlled by a second syntax element signaled at a syntax level higher than the codec block level. i. For example, a syntax level higher than the codec block level may indicate sequence level / picture group level / picture level / slice level / tile group level such as in a sequence header / picture header / SPS / VPS / DPS / DCI / PPS / APS / slice header / tile group header. ii. For example, a single codec may be controlled by the second syntax element. iii. For example, more than one codec may be controlled by the second syntax element. iv. For example, the first codec tool may be a prediction mode (such as AMVP-MERGE mode, etc.) utilizing a decoder-side motion derivation method. v. For example, the second syntax element may be a SPS / PPS / PH / SH flag that specifies whether decoder-side motion derivation methods (such as DMVR and TM, etc.) are allowed. vi. For example, the second syntax element may be a SPS / PPS / PH / SH flag that specifies whether decoder-side motion vector refinement (eg, DMVR, etc.) is allowed. vii. For example, the second syntax element may be a SPS / PPS / PH / SH flag that specifies whether decoder-side template matching (such as TM and / or inter-TM, etc.) is allowed. 6.6. Regarding LIC parameter derivation (e.g., as illustrated in the sixth question), assuming that the linear model for the LIC codec block is based on at least two parameters: a slope parameter "a" and a bias parameter "b", and the relationship between the neighboring samples of the current block and the neighboring samples of the time-domain co-located block can be represented by "reconTempNeigh=a*reconCurNeigh+b", where "recon-TempNeigh" represents the reconstructed / predicted value of the neighboring samples of the time-domain co-located block, and "reconCurNeigh" represents the reconstructed / predicted value of the neighboring samples of the current block, the following method is proposed: a. For example, at least one adjustment factor may be applied to adjust at least one LIC parameter used for LIC model derivation. a. For example, the adjustment factor may be signaled / present in the bitstream. b. For example, the adjustment factors may be derived at both the encoder and the decoder. b. For example, at least one syntax element (eg, syntax parameter or index, or variable or offset value or integer) may be signaled at video unit level for calculating at least one LIC parameter of at least one LIC model. a. For example, the video unit level may be PU / CU / block level. i. For example, in addition, the video unit level can be a sequence / picture group / picture / slice / tile group / PB / TB / CB / PU / TU / CU / VPDU / CTU / CTU row / slice / tile / sub-picture level. b. For example, the syntax element(s) may be used to adjust the value of at least one LIC parameter of at least one LIC model. i. For example, syntax element(s) may be used as indicator(s) of adjustment factor(s). ii. Alternatively, syntax element(s) may be used to represent / indicate the value of at least one LIC parameter. iii. For example, an indicator may be signaled to adjust a parameter of a LIC model. c. For example, the derivation of LIC parameters may be based on both decoder-derived methods and signaled syntax elements. d. For example, a syntax element may be an indicator of an integer. i. For example, the value of the syntax element may be in the range of [-N, +N], for example, N=4. ii. For example, LIC parameters can be derived directly based on integers. e. For example, a syntax element may be an indicator of an index. i. For example, based on the first value of the index, the second value may be derived from a (predefined) lookup table for LIC parameter derivation. f. For example, how many syntax elements are signaled may depend on how many linear / LIC models are used for a video unit (eg, codec block). i. For example, if M LIC models are used for a video unit, M syntax elements may be signaled in association with the video unit. c. For example, the predicted sample value of the video unit for LIC coding and decoding may be derived based on the updated model: ValueAfter=a'*ValueBefore+b', where a'=a+Delta, b'=b-Delta*funcD. a. For example, the updated model can be used to estimate / derive the prediction samples inside the current block. i. Alternatively, the updated model can be used to modulate the relationship between neighboring samples of the current block and neighboring samples of the co-located block in the time domain. b. For example, "Delta" can be the slope adjustment / offset value for the LIC model. i. For example, at least one indicator of "Delta" may be signaled in the bitstream. ii. Alternatively, the value of "Delta" may be derived based on decoded information (eg, decoded sample values, decoded prediction modes of neighboring / reference blocks). iii. For example, "Delta" can be an integer. iv. For example, “Delta” may be an integer in the range of [-N, +N], for example, N=4. v. For example, "Delta" can be a number / value / integer / constant / variable derived from an index on a lookup table. c. For example, “funcD” may be calculated by averaging the reconstructed / predicted values ​​of all available / appropriate / possible neighboring samples of the current block (or all available / appropriate / possible neighboring samples of the time-domain co-located block). i. For example, "funcD" may be calculated by averaging neighboring / reference samples from both intra-coded blocks and inter-coded blocks. 1. Alternatively, "funcD" can be calculated by simply averaging neighboring / reference samples from inter-coded blocks. ii. For example, “funcD” may be calculated by averaging all available neighboring / reference samples located to the left and / or above the current block and / or the temporally co-located block. 1. Alternatively, only a portion of such samples may be considered (eg, adjacent to the first top-left M×M unit, such as M=16 or 8 or 4 or 32). iii. For example, (neighboring samples of) the temporal co-located block may be retrieved / pointed to by a block motion vector (or its variants). 1. Alternatively, (neighboring samples of) the temporally co-located block may be retrieved / pointed to by a rounded block motion vector (eg, rounded to integer pixel precision). iv. For example, the averaging process can be handled with (or without) rounding factors. v. Alternatively, the averaging process can be replaced by other functions such as summation. d. For example, the updated model (eg, with adjustments) may be used / enabled for all LIC codec blocks. i. Alternatively, an updated model may be used / enabled for certain LIC codec blocks. 1. For example, “something” may be determined based on available / appropriate / possible neighboring samples (such as both left and top neighboring samples are available, or only left neighboring samples are available, or only top neighboring samples are available, etc.). 2. For example, "some kind" may be determined based on a prediction mode (such as AMVP codec or MERGE codec, unidirectional prediction or bidirectional prediction, etc.). ii. Alternatively, the updated model may be used / allowed only when both the left reference sample and the upper reference sample are available / appropriate / possible for the video unit. iii. Alternatively, the updated model can be used / allowed only when the video unit is predicted unidirectionally. d. The information for adjustment of LIC or CCLM or MM-CCLM can be encoded and decoded in a predictive manner. e. Information for adjustment of LIC or CCLM or MM-CCLM may be encoded or decoded using at least one context model. a. The context model may depend on codec information. b. Alternatively, it can be encoded and decoded in bypass mode. f. For example, the neighboring / reference samples used to derive the LIC model parameters may not be all available / appropriate / possible neighboring / reference samples from the left and top sides adjacent to the coding block and the time domain co-located block. a. For example, it may refer to neighboring / reference samples from both intra-coded blocks and inter-coded blocks. i. Alternatively, it may refer only to neighboring / reference samples from inter-coded blocks. b. For example, it may refer to a neighboring / reference sample located to the left (or above) of the current block and / or the temporally co-located block. i. Alternatively, only a portion of such samples may be considered (eg, adjacent to the first top left M×M unit, such as M=16 or 8 or 4 or 32). c. For example, (neighboring samples of) the temporal co-located block may be retrieved / pointed to by a block motion vector (or its variant). i. Alternatively, (neighboring samples of) the temporally co-located block may be retrieved / pointed to by a rounded block motion vector (eg, rounded to integer pixel precision). g. For example, whether to apply / allow adjustments (eg, updated models) for a video unit may depend on the encoded information. a. For example, the original model (without adjustments) and the updated model (with adjustments) may be used / allowed. i. Alternatively, only updated models will be used / allowed. b. For example, whether adjustment-based LIC model update is allowed (or applied) can be signaled in the bitstream. i. For example, it may be signaled at (at least) one video unit level such as sequence / group of picture / picture / slice / tile group / PB / TB / CB / PU / TU / CU / VPDU / CTU / CTU row / slice / tile / sub-picture level. c. For example, whether to allow (or apply) the adjustment-based LIC model update can be derived at both the encoder side and the decoder side. h. For example, LIC parameters can be derived based on a nonlinear model. a. For example, the LIC parameters can be derived based on predC=a0*((pred R*_predR+b)>>s)+a1*predR+a2*c, where predC represents the predicted sample value of the samples adjacent to the current block, predR represents the predicted sample value of the samples adjacent to the reference block, a0, a1, a2 represent LIC parameters, b and c are integers, and s represents a shift factor which can be an integer. b. For example, the LIC parameters can be derived based on predC=a0*((recR*recR+b)>>s)+a1*recR+a2*c, where recC represents the reconstructed sample value of the samples adjacent to the current block, recR represents the reconstructed sample value of the samples adjacent to the reference block, a0, a1, a2 represent LIC parameters, b and c are integers, and s represents the shift factor which can be an integer. c. For example, the values ​​of a0, a1, a2 can be derived based on the LDL decomposition method. d. For example, neighboring samples of M rows and N columns can be used to derive a0, a1, a2. i. For example, M may be equal to an integer not greater than 6. ii. For example, N may be equal to an integer not greater than 6. iii. For example, the values ​​of M and / N may depend on the CTU / LCU boundary (eg, based on the distance between the current block and the CTU top boundary). e. For example, the value of s may be derived based on the internal bit depth of the encoded samples. f. For example, the values ​​of b and c may be derived based on the median / middle / average / mean of sample values ​​neighboring the current block / reference block. i. For example, b=c. 6.7. Whether and / or how to apply adjustments for CCLM, MM-CCLM or LIC may depend on codec information such as block dimension, codec mode, (transformed) residual, transform, etc. a. For example, if the width and / or height and / or size of the block is less than a threshold, no adjustment may be applied. 6.8. Regarding the signaling and determination of LIC (e.g., as illustrated in the seventh question), the following method is proposed: a. For example, the LIC flag at the video unit level (eg, CU / PU level LIC flag) may not be signaled but derived at both the encoder side and the decoder side. a. For example, the CU / PU level LIC flag for the AMVP codec block may not be signaled. b. For example, the CU / PU level LIC flag for the affine AMVP codec block may not be signaled. c. For example, the CU / PU level LIC flag for the AMVP-MERGE codec block may not be signaled. b. For example, whether to use LIC for a video unit may depend on coded information (eg, a decoder-derived method). a. For example, the LIC flag at the video unit level can be implicitly derived at both the encoder side and the decoder side. i. For example, the CU / PU level LIC flag for non-merging (such as AMVP and / or Affine AMVP) codec blocks may be derived implicitly. ii. For example, the CU / PU level LIC flag for the merge (and / or its variants such as TM, BM, MHP, ADMVR, CIIP, GPM, sbTMVP, affine merge, etc.) codec block may be derived implicitly. iii. For example, implicitly deriving a method that can be derived based on a decoder. iv. For example, implicit derivation can be based on template matching. v. For example, implicit derivation can be based on bilateral matching. b. For example, the decoder-derived cost / error / distortion may be calculated for both the non-LIC case and the LIC case, and the one with the lower cost / error / distortion is determined as the codec method for the video unit. i. For example, the template (and / or bilateral) matching cost may be calculated for LIC coded video units and non-LIC coded video units, respectively. ii. For example, the cost / error / distortion is derived from temporal (co-located) neighboring samples and / or reference samples in a reference picture. iii. For example, the cost / error / distortion is not derived from the current block sample in the current picture. c. For example, the coded information for LIC mode may be neighboring / reference samples from both intra-coded blocks and inter-coded blocks. i. Alternatively, the coded information may be neighboring / reference samples from inter-coded blocks only. d. For example, the coded information for the LIC mode may be all available neighboring / reference samples located to the left and / or above the current block and / or the time-domain co-located block. i. Alternatively, only a portion of such samples may be considered (eg, adjacent to the first top left M×M unit, such as M=16 or 8 or 4 or 32). e. For example, the temporally co-located block may be retrieved / pointed to by a block motion vector (or its variant). i. Alternatively, the temporally co-located block may be retrieved / pointed to by a rounded block motion vector (eg, rounded to integer pixel precision). c. For example, the merge index of the merged block for LIC coding may not be signaled (eg, derived at both the encoder and the decoder). a. For example, the motion (eg, motion vector, reference index, prediction direction, etc.) of the LIC-encoded merged block can be derived at both the encoder side and the decoder side. b. For example, multiple template (and / or bilateral) matching costs / errors / distortions may be calculated for all (or multiple or predefined portions) available / possible / appropriate merge candidates, respectively. The one with the minimum cost / error / distortion is determined as the motion for the video unit. i. For example, the template is constructed from temporal (co-located) neighboring samples and / or reference samples in a reference picture. ii. For example, the template is not constructed from the current block samples in the current picture. iii. For example, the template is constructed with a sample without LIC. iv. For example, the template is constructed with a sample having LIC. d. For example, the best LIC parameter set (such as a and b calculated by a least squares fitting method) can be determined from more than one LIC parameter set. a. For example, more than one LIC parameter set may be applicable to a video unit of the LIC codec. b. For example, which LIC parameter set is used for a video unit can be derived at both the encoder side and the decoder side. i. Alternatively, a syntax element (eg, an index) may be signaled to specify the LIC parameter set for a video unit. c. For example, multiple template (and / or bilateral) matching costs / errors / distortions may be calculated for all appropriate LIC parameter sets, respectively. The one with the minimum cost / error / distortion is determined as the LIC parameter set for the video unit. i. For example, the template is constructed from temporal (co-located) neighboring samples and / or reference samples in a reference picture. ii. For example, the template is not constructed from the current block samples in the current picture. 6.9. Regarding out-of-bounds (OOB) prediction treatment (e.g., as illustrated in the eighth problem), the following method is proposed: a. For example, an OOB handling method may be used to blend the first prediction and the second prediction. a. For example, at least one prediction block / sub-block is a motion compensated prediction block / sub-block. i. For example, the first prediction may be non-inter (eg, intra) prediction, and the second prediction may be motion compensated prediction (such as CIIP mode, etc.). ii. For example, the prediction blocks / sub-blocks can all be motion compensated predicted. b. For example, two predictions may be generated from the same inter prediction direction (eg, MHP mode, GPM mode, etc.). c. For example, two predictions may be generated from different inter prediction directions (eg, bi-directional prediction weighting). d. For example, it can be used in MERGE mode and / or its variant modes (such as BM / ADMVR / CIIP / TM / MMVD / AffinMerge / sbTMVP / GPM / GPM-MMVD / GPM-TM mode, etc.). e. For example, it can be used in AMVP mode and / or its variant modes (such as SMVD / AMVVmerge / AffineAMVP, etc.). f. For example, it may be allowed / used for prediction based on decoder-side motion vector refinement (such as template matching and / or bilateral matching based motion vector refinement, etc.). i. For example, it may be allowed / used for predictions generated by decoder-side motion vector refinement related codec modes. ii. Alternatively, it may not be allowed / used for prediction based on decoder-side motion vector refinement. iii. Alternatively, it may not be allowed / used for prediction generated by decoder-side motion vector refinement related codec modes. g. For example, it may be allowed / used for predictions generated based on some kind of sub-block based prediction method (e.g., DMVR, TM, affine merge, affine AMVP, sbTMVP, OBMC, BDOF, AMVP-merge, GPM, etc.). i. For example, it may be allowed / used for certain kinds of CU / PU based prediction methods (eg, regular merge, CIIP, MMVD, MHP and / or regular AMVP, SMVD, etc.). ii. Alternatively, it may not be allowed / used for predictions generated based on some kind of sub-block based prediction method. 1. For example, it may not coexist with certain kinds of sub-block based prediction methods. iii. Alternatively, it may not be allowed / used for certain kinds of CU / PU based prediction methods. 1. For example, it may not coexist with certain CU / PU based prediction methods. b. For example, OOB may refer to outside reference picture boundaries. a. Alternatively, it may refer to outside the reference sub-picture boundaries. i. For example, the reference sub-picture ID is the same as the current sub-picture. b. Alternatively, it may refer to outside the reference strip boundaries. c. Alternatively, it may refer to outside the reference tile boundaries. i. For example, in the case of MCTS (ie, Motion Constrained Tile Set), motion vectors are constrained to be within tiles with the same coordinates / position in the reference picture. c. For example, the blending weights for OOB samples may be different from those for non-OOB samples inside the boundary. a. For example, the weighting factor for OOB samples of a motion compensated block / sub-block may be set according to a rule, for example, to a certain value (such as zero). b. For example, the weighting factors for motion compensated samples around the butting boundary of OOB samples and non-OOB samples may be set according to a rule, for example, gradually increasing / decreasing the weighting value as it moves away from the butting boundary. c. For example, the blending weights may be calculated based on boundary-based distances. d. For example, the blending weight may be calculated based on the distance based on the boundary between the OOB sample and the non-OOB sample. d. For example, when mixing the first prediction sample with the second prediction sample, if both motion compensated samples are OOB, the final prediction value can be generated without mixing. a. For example, non-OOB samples closer to the boundary can be obtained for final prediction sample generation. b. For example, the final prediction value may be generated based on non-OOB prediction samples within the current mixed block according to a rule (eg, the most recently available non-OOB prediction sample within the current mixed block). c. Alternatively, in this case, the final prediction value for such a sample may be generated by the same averaging / weighted blending as usual (eg, the same behavior as for samples inside the blending boundaries). e. For example, the values ​​of the OOB samples of a motion compensated block / sub-block may be set according to a rule. a. For example, OOB samples may refer to OOB samples just after the (unidirectional) motion compensation process (but before the BDOF and blending / weighting processes). b. For example, the rules may be based on non-OOB motion compensated sample values ​​inside the boundary. i. For example, the non-OOB motion compensated sample values ​​at the first row located inside the boundary may be copied above for the above OOB samples. ii. For example, the non-OOB motion compensated sample values ​​at the first column located inside the boundary may be duplicated on the left for the left OOB samples. iii. For example, the non-OOB motion compensated sample value at the top left corner located inside the boundary may be copied for the top left OOB sample. c. For example, OOB samples may refer to OOB samples after BDOF (if any) and before the mixing / weighting process. d. For example, the rule can be based on the sample value of the non-OOB BDOF refinement inside the boundary. i. For example, the non-OOB BDOF refined sample values ​​at the first row located inside the boundary may be copied above for the above OOB samples. ii. For example, the non-OOB BDOF refined sample values ​​at the first column located inside the boundary may be duplicated on the left for the left OOB samples. iii. For example, the non-OOB BDOF refined sample value at the top left corner located inside the boundary may be copied for the top left OOB sample. f. For example, if the prediction block / sub-block pointed to by the motion vector is OOB, a new prediction block / sub-block may be generated according to a rule. a. For example, a new prediction block / sub-block may be generated based on a zero motion vector (eg, (0,0)). b. For example, the new prediction block / sub-block can be replaced by the co-located block. c. For example, the new prediction block / sub-block may be replaced with the non-OOB prediction block / sub-block closest to the OOB block / sub-block. g. For example, the OOB check may be based on motion vectors prior to decoder-side motion refinement (such as template matching based motion refinement and / or bilateral matching based motion refinement). a. Alternatively, the OOB check can be based on motion vectors after a certain stage of DMVR-based motion refinement (e.g., PU level or 16×16 sub-block level or 8×8 sub-block level). b. Alternatively, the OOB check can be based on motion vectors after all stages of DMVR-based motion refinement (e.g., PU level, 16x16 sub-block level, and 8x8 sub-block level). c. Alternatively, the OOB check can be based on the motion vectors after TM-based motion refinement. d. For example, the blending weights for OOB samples may be determined based on an OOB check. h. For example, if the reference block is partially (or entirely) OOB, the OOB portion of the motion compensated block / sub-block may not be used to generate the final prediction block. i. For example, if the reference block is partially (or entirely) OOB, the original motion compensation block / sub-block may be modified / shifted to another motion compensation block / sub-block. j. For example, if the reference block is partially (or fully) OOB, the original motion vector used to generate the motion compensated block / sub-block may be modified to another motion vector. k. For example, whether the OOB handling method is used for a video unit may be signaled at the video unit level. a. Alternatively, the OOB handling method can be forced for certain kinds of blocks (such as multiple hypothesis, bi-directionally predicted blocks) without video unit level signaling. b. Alternatively, whether to use OOB handling can be implicitly derived based on the encoded information. 6.10 With regard to non-equal weighting (e.g., Question 8 and related questions), the following approach is proposed: a. For example, non-equal weighting (eg, BCW) may be applied to the AMVP-MERGE mode. a. For example, it can be applied to low-latency pictures and / or non-low-latency pictures. b. For example, a weight index in a predefined table containing M possible weighting factors may be determined. i. For example, M=3, or 4, or 5, or 6. ii. For example, depending on the encoding and decoding information of the current picture (eg, whether it is a low-delay picture), M may be equal to different values. c. For example, the weight index may be signaled in the bitstream. d. For example, the weight index may be derived implicitly through a decoder-side derivation method (eg, without signaling). i. For example, the weight index may be determined based on a cost derived based on DMVR / DMVD / mpDMVD / puDMVD (and / or variations thereof). ii. For example, the weight index may be determined based on the TM cost. iii. For example, the TM cost derived template may be equal to the MxN samples neighboring the current block and the reference block. iv. For example, the TM cost derived template may be equal to the MxN samples neighboring the first reference block and the second reference block. v. For example, TM / DMVR costs can be scaled for cost comparison. 1. For example, the TM / DMVR cost derived by average weighting may be scaled (eg, to a smaller cost value). b. For example, non-equal weighting of K entries (such as K>2 or K=2) can be applied to the MHP mode. a. For example, it can be applied to the MHP AMVP mode (eg, generating additional MHP hypotheses based on AMVP) to weight the two hypotheses. b. For example, it can be applied in MHP MERGE mode (eg, generating additional MHP hypotheses based on MERGE) to weight the two hypotheses. c. For example, it can be applied to low-latency pictures and / or non-low-latency pictures. d. For example, a weight index in a predefined table containing M possible weighting factors may be determined. i. For example, M=2, or 3, or 4, or 5, or 6. ii. For example, depending on the encoding and decoding information of the current picture (eg, whether it is a low-delay picture), M may be equal to different values. iii. For example, M may be equal to different values ​​depending on whether it is MHP AMVP or MHP MERGE. e. For example, the weight index may be signaled in the bitstream. f. For example, the weight index may be derived implicitly via a decoder-side derivation method (eg, without signaling). i. For example, the weight index may be determined based on a cost derived based on DMVR / DMVD / mpDMVD / puDMVD (and / or variations thereof). ii. For example, the weight index may be determined based on the TM cost. iii. For example, the TM cost derived template may be equal to the MxN samples neighboring the current block and the reference block. iv. For example, the TM cost derived template may be equal to the MxN samples neighboring the first reference block and the second reference block. v. For example, TM / DMVR costs can be scaled for cost comparison. 1. For example, the TM / DMVR cost derived by average weighting may be scaled (eg, to a smaller cost value). c. For example, a decoder-derived non-average weighting (e.g., BCW) approach may be applied to one or more of the following: a. Non-low latency images b. Low latency images c. MHP AMVP and its variants d. MHP MERGE and its variants e.MMVD and its variants f. AMVP-MERGE and its variants 6.11 Regarding improvements to the AMVP-MERGE model (e.g., question 9 and related questions), the following methods are proposed: a. In one example, AMVP-MERGE mode may be enabled for low-latency pictures. Assuming that a block encoded and decoded by AMVP-MERGE is encoded and decoded by a motion pair (AMVP-MV, MERGE-MV) from the same direction relative to the current picture, a. For example, AMVP-MERGE mode may be enabled for low-delay pictures in a random access configuration. b. For example, AMVP-MERGE mode may be enabled for low latency pictures in a low latency configuration. c. For example, AMVP-MV and MERGE-MV may point to two different reference pictures with different POC values, respectively. d. For example, for AMVP-MERGE list generation, a two-step reference picture check may be applied for video units. i. For example, it may first check the first condition (eg, whether there are available reference pictures from both the forward and backward directions). 1. If it meets the first condition, a true bidirectional AMVP-MERGE node can be made. ii. For example, if it does not meet the first condition, it may further check the second condition (eg, whether there is an available reference picture from the same direction but different POC distance). 1. If it meets the second condition, AMVP-MERGE in the same direction can be performed. e. For example, in the case where there is more than one reference picture having a POC distance (diff-POC-dist) different from the POC distance of AMVR-REF (i.e., the reference picture pointed to by the AMVP-MV of the AMVP-MERGE mode), a specific reference picture among such diff-POT-dist reference pictures may be selected to derive the MERGE-MV. i. For example, it can be based on the distance from the current picture. 1. For example, it is closer to the current picture. 2. For example, it is closer to the AMVP-REF picture. ii. For example, it can be based on the POC distance of the diff-POC-dist reference picture. 1. For example, it has a smaller POC value. 2. For example, it has a larger POC value. f. For example, AMVP-MV (single direction) may be determined based on a coded MVP index and / or a coded reference index. g. For example, MERGE-MV may be determined based on specific rules. i. For example, a specific rule may be based on a predefined position of a MERGE-MV candidate in the MERGE-MV list. 1. For example, the Mth (eg, first) available MERGE-MV candidate. ii. For example, specific rules can be based on TM costs. 1. For example, multiple TM costs regarding multiple MERGE-MV candidates may be calculated, and one with the minimum TM cost may be selected as the final MERGE-MV for the AMVP-MERGE mode. 2. For example, the TM cost may contain costs calculated from both AMVP-MV candidates and MERGE-MV candidates. 3. For example, the TM cost can contain the cost computed only from the MERGE-MV candidates. h. For example, TM-based refinement can be used for AMVR-MERGE motion pairs from the same direction. b. For example, when DMVR is off, the items / sub-items in 6.11 above may apply. c. For example, when RPR is turned on, the items / sub-items in 6.11 above may apply. 6.12 With respect to DMVD applications (e.g., Question 10 and related questions), the following approach is proposed: a. For example, the DMVR / DMVR method and / or its variants may be used to refine the shifted MV of the sbTMVP mode. a. For example, the shifted MV can be bi-directionally predicted. b. For example, the shifted MV may point to two reference pictures from both forward and backward. c. For example, a shifted MV that satisfies the DMVR / DMVD conditions may be generated for sbTMVP. d. For example, the shifted MV can be used to derive a motion vector from the corresponding PU in the reference picture. b. For example, DMVR / DMVR methods and / or their variants may be used to refine the CPMV of an affine AMVP model. a. For example, only PU-level DMVD / DMVR may be used. b. For example, it can be applied only to CPMVs that meet DMVR / DMVD conditions. c. For example, it may be applied only to CPMVs pointing to reference pictures from both forward and backward. d. For example, it can be applied only to CPMV pairs of bi-predictive affine AMVP mode. c. For example, the DMVR / DMVR method and / or its variants can be used to refine the CPMV of the affine MERGE model. Miscellaneous aspects 6.13. For example, is it possible to control deblocking at the CTU (or CU) level? 6.14. For example, the intra-MTS type may be determined based on syntax elements signaled at the PPS level (eg, parameters of a threshold). 6.15. For example, adaptive mixing weights may be used for OBMC codec blocks, where the adaptive mixing weights may be based on coded information other than neighboring reconstructed / predicted samples. a. For example, the adaptive mixing weights can be determined by the temporal coding information in the reference picture. b. For example, adaptive blending weights may be selected based on more than one lookup table. a. For example, weight indices can be derived. b. For example, the weight index may be signaled. General 6.16. Whether and / or how to apply the above disclosed method may be signaled at the sequence level / picture group level / picture level / slice level / tile group level, such as in a sequence header / picture header / SPS / VPS / DPS / DCI / PPS / APS / slice header / tile group header. 6.17. Whether and / or how the above disclosed methods can be applied to be signaled under PB / TB / CB / PU / TU / CU / VPDU / CTU / CTU line / slice / tile / sub-picture / other types of regions containing more than one sample or pixel 6.18. Whether and / or how to apply the above disclosed method may depend on the coded information, such as block size, color format, single / dual tree partitioning, color component, slice / picture type.

[0100] As used herein, the terms "video unit" or "codec unit" or "block" used herein may refer to one or more of the following: a color component, a sub-picture, a slice, a tile, a codec tree unit (CTU), a CTU row, a CTU group, a codec unit (CU), a prediction unit (PU), a transform unit (TU), a codec tree block (CTB), a codec block (CB), a prediction block (PB), a transform block (TB), a block, a sub-block of a block, a sub-region within a block, or a region including more than one sample or pixel.

[0101] In the present disclosure, with respect to “encoding a block in mode N”, the term “mode N” may be a prediction mode (e.g., MODE_INTRA, MODE_INTER, MODE_PLT, MODE_IBC, etc.) or a coding technology (e.g., AMVP, Merge, SMVD, BDOF, PROF, DMVR, AMVR, TM, Affine, CIIP, GPM, MMVD, BCW, HMVP, SbTMVP, etc.).

[0102] In the following discussion, LIC parameters may refer to two parameters (such as slope parameter "a" and bias parameter "b") derived based on a linear model that is used to map neighboring samples of a current block and neighboring samples of a temporal co-located block (e.g., the temporal co-located block may be pointed to by a motion vector or a rounded motion vector of the current block). In addition, LIC parameters may be used to estimate prediction values ​​of samples within the current video unit.

[0103] Fig.32 A flow chart of a method 3200 for video processing according to an embodiment of the present disclosure is illustrated. The method 3200 is implemented during conversion between a video unit and a bitstream of the video unit.

[0104] At block 3210, for conversion between a video unit of a video and a bitstream of the video, it is determined that the video unit is encoded with a motion pair from the same direction relative to a current picture. The video unit has an advanced motion vector prediction (AMVP) merge mode applied, and the video unit belongs to a low-delay picture.

[0105] At block 3220, conversion is performed based on the motion pair. In some embodiments, conversion may include encoding the video unit into a bitstream. Alternatively, conversion may include decoding the video unit from the bitstream. In this way, encoding and decoding efficiency may be improved.

[0106] In some embodiments, the motion pair includes an AMVP motion vector (MV) and a MERGE MV. For example, an AMVP-MERGE coded block may be coded by a motion pair (AMVR-MV, MERGE-MV) from the same direction relative to the current picture.

[0107] In some embodiments, AMVP merge mode is enabled for low-delay pictures in random access configurations. In some embodiments, AMVP merge mode is enabled for low-delay pictures in low-delay configurations.

[0108] In some embodiments, the motion pair includes an AMVP MV and a MERGE MV. The AMVP MV may point to a first reference picture having a first picture order count (POC) value, and the MERGE MV may point to a second reference picture having a second POC value. For example, the AMVP-MV and the MERGE-MV may point to two different reference pictures having different POC values, respectively.

[0109] In some embodiments, method 3200 further includes: applying a two-step reference picture check process to the video unit for generation of the AMVP merge list. In some embodiments, applying the two-step reference picture check process includes: determining whether a first condition is satisfied. In some embodiments, the first condition includes whether there are available reference pictures from both the forward direction and the backward direction. For example, it may first check the first condition (e.g., whether there are available reference pictures from both the forward direction and the backward direction). In some embodiments, if the first condition is satisfied, a true bidirectional AMVP merge mode is performed.

[0110] In some embodiments, method 3200 further includes: if the first condition is not satisfied, determining whether the second condition is satisfied.

[0111] In some embodiments, the second condition includes whether there is an available reference picture from the same direction but with a different POC distance. For example, if it does not meet the first condition, it can further check the second condition (e.g., whether there is an available reference picture from the same direction but with a different POC distance). In some embodiments, if the second condition is met, the same direction AMVP merge mode is performed.

[0112] In some embodiments, the AMVP MV in a motion pair is determined based on at least one of: a decoded motion vector prediction (MVP) index or a decoded reference index. In some embodiments, template matching (TM) based refinement is used for motion pairs from the same direction.

[0113] In some embodiments, if there are multiple reference pictures with different POC distances from the POC distance of the AMVP reference picture, then the target reference picture from the multiple reference pictures is selected to derive the MERGE MV in the motion pair. For example, in the case where there are more than one reference picture with a different POC distance (diff-POC-dist) from the POC distance of the AMVR-REF (i.e., the reference picture pointed to by the AMVP-MV of the AMVP-MERGE mode), then a specific reference picture from such diff-POT-dist reference picture is selected to derive the MERGE MV.

[0114] In some embodiments, the target reference picture is selected based on the distance to the current picture. In some embodiments, the target reference picture is closer to the current picture than the other reference pictures. In some embodiments, the target reference picture is then closer to the AMVP reference picture than the other reference pictures. For example, it can be based on the distance from the current picture. For example, it is closer to the current picture. For example, it is closer to the AMVP-REF picture.

[0115] In some embodiments, the target reference picture is selected based on the POC distance. In some embodiments, among the multiple reference pictures, the target reference picture has a smaller POC value than other reference pictures. In some embodiments, among the multiple reference pictures, the target reference picture has a larger POC value than other reference pictures. For example, it can be based on the POC distance of the diff-POC-dist reference picture. For example, it has a smaller POC value. For example, it has a larger POC value.

[0116] In some embodiments, the MERGE MV in the motion pair is determined based on a rule. In some embodiments, the rule is based on a predefined position of the MERGE MV candidate in the MERGE MV list. In some embodiments, the MERGE MV candidate is the Mth available MERGE MV candidate in the MERGE MV list, where M is an integer.

[0117] In some embodiments, the rule is based on TM cost. In some embodiments, multiple TM costs are determined for multiple MERGE MV candidates, and the MERGE MV candidate with the smallest TM cost is selected as the MERGE MV in the motion pair. In some embodiments, the TM cost includes costs determined for both AMVP MV candidates and MERGE MV candidates. In some embodiments, the TM cost includes costs determined only by MERGE MV candidates.

[0118] In some embodiments, decoder-side motion vector refinement (DMVR) of the video unit is OFF, and / or reference picture resampling (RPR) of the video unit is ON. For example, when DMVR is OFF, reference picture resampling (RPR) may be applied. Fig.32 For example, when RPR is ON, the above embodiment can be applied.

[0119] In some embodiments, an indication of whether and / or how to determine that a video unit is encoded with motion pairs from the same direction relative to a current picture is indicated at one of the following: a sequence level, a group of pictures level, a picture level, a slice level, or a tile group level. In some embodiments, an indication of whether and / or how to determine that a video unit is encoded with motion pairs from the same direction relative to a current picture is indicated in one of the following: a sequence header, a picture header, a sequence parameter set (SPS), a video parameter set (VPS), a dependency parameter set (DPS), decoding capability information (DCI), a picture parameter set (PPS), an adaptation parameter set (APS), a slice header, or a tile group header. In some embodiments, an indication of whether and / or how to determine whether a video unit is encoded with motion from the same direction relative to a current picture is included in one of the following: a prediction block (PB), a transform block (TB), a codec block (CB), a prediction unit (PU), a transform unit (TU), a codec unit (CU), a virtual pipeline data unit (VPDU), a codec tree unit (CTU), a CTU row, a slice, a tile, a sub-picture, or a region containing more than one sample or pixel.

[0120] In some embodiments, method 3200 further includes determining whether and / or how to determine that the video unit is encoded with a motion pair from the same direction relative to the current picture based on encoded information of the video unit. The encoded information includes at least one of the following: block size, color format, single tree and / or dual tree partitioning, color component, slice type, or picture type.

[0121] According to another embodiment of the present disclosure, a non-transitory computer-readable recording medium is provided. The non-transitory computer-readable recording medium stores a bitstream of a video generated by a method performed by a device for video processing. The method includes: determining that a video unit of a video is encoded and decoded by a motion pair from the same direction relative to a current picture, wherein the video unit is applied with an advanced motion vector prediction (AMVP) merge mode, and the video unit belongs to a low-latency picture; and generating a bitstream of the video unit based on the motion pair.

[0122] According to some other embodiments of the present disclosure, a method for storing a bitstream of a video is provided. The method includes: determining that a video unit of the video is encoded and decoded by a motion pair from the same direction relative to a current picture, wherein the video unit is applied with an advanced motion vector prediction (AMVP) merge mode, and the video unit belongs to a low-latency picture; generating a bitstream of the video unit based on the motion pair; and storing the bitstream in a non-transitory computer-readable recording medium.

[0123] Fig.33 A flow chart of a method 3300 for video processing according to an embodiment of the present disclosure is illustrated. The method 3300 is implemented during conversion between a video unit and a bitstream of the video unit.

[0124] At block 3310, during conversion between a video unit of a video and a bitstream of the video, at least one of decoder-side motion vector refinement (DMVR) or a variant of DMVR is applied to the video unit to refine a motion vector of the video unit. The video unit is applied with one of: an affine advanced motion vector prediction (AMVP) mode, a sub-block based temporal motion vector prediction (sbTMVP) mode, or an affine merge mode.

[0125] At block 3320, conversion is performed based on the refined motion vector. In some embodiments, the conversion includes encoding the video unit into a bitstream. Alternatively, the conversion may include decoding the video unit from the bitstream. In this way, encoding and decoding efficiency may be improved.

[0126] In some embodiments, at least one of DMVR or a variant of DMVR is used to refine a shifted motion vector (MV) for the sbTMVP mode. In some embodiments, the shifted MV is bidirectionally predicted. In some embodiments, the shifted MV points to two reference pictures from both forward and backward. In some embodiments, for the sbTMVP mode, a shifted MV that satisfies a DMVR condition or a decoder-side motion vector difference (DMVD) condition is generated. In some embodiments, the shifted MV is used to derive a motion vector from a corresponding prediction unit (PU) in a reference picture.

[0127] In some embodiments, DMVR or at least one of its variants is used to refine the control point motion vectors (CPMV) of the affine merge mode. In some embodiments, DMVR or at least one of its variants is used to refine the control point motion vectors (CPMV) of the affine AMVP mode. In some embodiments, only PU-level DMVD or DMVR is used. In some embodiments, DMVR or at least one of its variants is applied to CPMVs that satisfy the DMVR condition or the DMVD condition. In some embodiments, DMVR or at least one of its variants is applied to CPMVs pointing to reference pictures from both forward and backward. In some embodiments, DMVR or at least one of its variants is applied to CPMV pairs of bidirectionally predicted affine AMVP modes.

[0128] In some embodiments, an indication of whether and / or how to determine that a video unit is encoded with motion pairs from the same direction relative to a current picture is indicated at one of the following: a sequence level, a group of pictures level, a picture level, a slice level, or a tile group level. In some embodiments, an indication of whether and / or how to determine that a video unit is encoded with motion pairs from the same direction relative to a current picture is indicated in one of the following: a sequence header, a picture header, a sequence parameter set (SPS), a video parameter set (VPS), a dependency parameter set (DPS), decoding capability information (DCI), a picture parameter set (PPS), an adaptation parameter set (APS), a slice header, or a tile group header. In some embodiments, an indication of whether and / or how to determine whether a video unit is encoded with motion from the same direction relative to a current picture is included in one of the following: a prediction block (PB), a transform block (TB), a codec block (CB), a prediction unit (PU), a transform unit (TU), a codec unit (CU), a virtual pipeline data unit (VPDU), a codec tree unit (CTU), a CTU row, a slice, a tile, a sub-picture, or a region containing more than one sample or pixel.

[0129] In some embodiments, method 3300 further includes: determining whether and / or how to determine that the video unit is encoded by a motion pair from the same direction relative to the current picture based on the encoded information of the video unit. The encoded information may include at least one of the following: block size, color format, single tree and / or dual tree partitioning, color component, slice type, or picture type.

[0130] According to another embodiment of the present disclosure, a non-transitory computer-readable recording medium is provided. The non-transitory computer-readable recording medium stores a bitstream of a video generated by a method performed by a device for video processing. The method includes: applying decoder-side motion vector refinement (DMVR) or at least one of a variant of DMVR to a video unit of the video to refine the motion vector of the video unit, wherein the video unit is applied with one of the following: affine advanced motion vector prediction (AMVP) mode, sub-block-based temporal motion vector prediction (sbTMVP) mode, or affine merge mode; and generating a bitstream of the video unit based on the refined motion vector.

[0131] According to some other embodiments of the present disclosure, a method for storing a bitstream of a video is provided. The method includes: applying decoder-side motion vector refinement (DMVR) or at least one of a variant of DMVR to a video unit of the video to refine a motion vector of the video unit, wherein the video unit is applied with one of the following: an affine advanced motion vector prediction (AMVP) mode, a sub-block-based temporal motion vector prediction (sbTMVP) mode, or an affine merge mode; generating a bitstream of the video unit based on the refined motion vector; and storing the bitstream in a non-transitory computer-readable recording medium.

[0132] Fig.34 A flow chart of a method 3400 for video processing according to an embodiment of the present disclosure is illustrated. The method 3400 is implemented during conversion between a video unit and a bitstream of the video unit.

[0133] At block 3410, during conversion between a video unit of a video and a bitstream of the video, at least one LIC model parameter of a local illumination compensation (LIC) model for the video unit is determined based on a nonlinear model. The coded video unit is a LIC coded block.

[0134] At block 3420, the LIC model is updated by adjusting at least one LIC model parameter. At block 3430, conversion is performed based on the updated LIC model. In some embodiments, the conversion may include encoding the video unit into a bitstream. Alternatively, the conversion may include decoding the video unit from the bitstream. In this way, encoding and decoding efficiency may be improved.

[0135] In some embodiments, at least one LIC model parameter is derived based on predC=a0*((predR*predR+b)>>s)+a1*predR+a2*c. In this case, predC represents the predicted sample value of the sample adjacent to the current block, predR represents the predicted sample value of the sample adjacent to the reference block, a0, a1 and a2 represent LIC parameters respectively, b and c are integers, and s represents a shift factor which is an integer.

[0136] In some embodiments, at least one LIC model parameter is derived based on predC=a0*((recR*recR+b)>>s)+a1*recR+a2*c. In this case, recC represents the reconstructed sample values ​​of samples adjacent to the current block, recR represents the reconstructed sample values ​​of samples adjacent to the reference block, a0, a1 and a2 represent LIC parameters respectively, b and c are integers, and s represents a shift factor which is an integer.

[0137] In some embodiments, at least one LIC model parameter is derived based on an LDL decomposition method, where L represents a lower triangle and D represents a diagonal with positive diagonal lines. For example, the values ​​of a0, a1, a2 can be derived based on an LDL decomposition method. Alternatively, at least one LIC model parameter is derived based on a coefficient solution method based on Gaussian elimination.

[0138] In some embodiments, M rows and N columns of neighboring samples are used to derive at least one LIC model parameter, where M and N are integers. For example, M rows and N columns of neighboring samples can be used to derive a0, a1, a2. In some embodiments, M is equal to an integer and is not greater than 6. In some embodiments, N is equal to an integer and is not greater than 6.

[0139] In some embodiments, the value of at least M or N depends on a codec tree unit (CTU) boundary or a largest codec unit (LCU) boundary. For example, the value of M and / or N may depend on a CTU / LCU boundary (e.g., based on the distance between the current block and the top boundary of the CTU).

[0140] In some embodiments, the shift factor is derived based on the internal bit depth of the encoded samples.For example, the value of s may be derived based on the internal bit depth of the encoded samples.

[0141] In some embodiments, the values ​​of b and c are derived based on at least one of the following: a median value of sample values ​​adjacent to the current block, a middle value of sample values ​​adjacent to the current block, an average value of sample values ​​adjacent to the current block, a median value of sample values ​​adjacent to the reference block, a middle value of sample values ​​adjacent to the reference block, or an average value of sample values ​​adjacent to the reference block. For example, the values ​​of b and c may be derived based on the median / middle / average / mean of sample values ​​adjacent to the current block / reference block. In some embodiments, b is equal to c.

[0142] In some embodiments, an indication of whether and / or how to derive at least one LIC model parameter based on a non-linear model is indicated at one of the following: sequence level, group of picture level, picture level, slice level, or tile group level. In some embodiments, an indication of whether and / or how to derive at least one LIC model parameter based on a non-linear model is indicated in one of the following: sequence header, picture header, sequence parameter set (SPS), video parameter set (VPS), dependency parameter set (DPS), decoding capability information (DCI), picture parameter set (PPS), adaptation parameter set (APS), slice header, or tile group header. In some embodiments, an indication of whether and / or how to derive at least one LIC model parameter based on a non-linear model is included in one of the following: prediction block (PB), transform block (TB), codec block (CB), prediction unit (PU), transform unit (TU), codec unit (CU), virtual pipeline data unit (VPDU), codec tree unit (CTU), CTU row, slice, tile, sub-picture, or region containing more than one sample or pixel.

[0143] In some embodiments, the method 3400 further comprises: determining whether and / or how to derive at least one LIC model parameter based on a non-linear model based on coded information of the video unit. The coded information may include at least one of the following: block size, color format, single tree and / or dual tree partitioning, color component, slice type, or picture type.

[0144] According to another embodiment of the present disclosure, a non-transitory computer-readable recording medium is provided. The non-transitory computer-readable recording medium stores a bitstream of a video generated by a method performed by a device for video processing. The method includes: deriving at least one LIC model parameter of a local illumination compensation (LIC) model for a video unit of the video based on a nonlinear model, wherein the coded video unit is a block coded by LIC; updating the LIC model by adjusting at least one LIC model parameter; and generating a bitstream of the video unit based on the updated LIC model.

[0145] According to some further embodiments of the present disclosure, a method for storing a bitstream of a video is provided, the method comprising: deriving at least one LIC model parameter of a local illumination compensation (LIC) model for a video unit of the video based on a nonlinear model, wherein the coded video unit is a block coded by LIC; updating the LIC model by adjusting at least one LIC model parameter; generating a bitstream of the video unit based on the updated LIC model; and storing the bitstream in a non-transitory computer-readable recording medium.

[0146] Fig.35A flow chart of a method 3500 for video processing according to an embodiment of the present disclosure is illustrated. The method 3500 is implemented during conversion between a video unit and a bitstream of the video unit.

[0147] At block 3510, for conversion between a video unit of a video and a bitstream of the video, a non-average weighting method is applied to the video unit. The video unit is applied with an advanced motion vector prediction (AMVP) merge mode. For example, non-average weighting (e.g., BCW) can be applied to the AMVP-MERGE mode.

[0148] At block 3520, a conversion is performed based on the weighted video units. In some embodiments, the conversion may include encoding the video units into a bitstream. Alternatively, the conversion may include decoding the video units from the bitstream. In this way, encoding and decoding efficiency may be improved.

[0149] In some embodiments, the non-average weighting method is applied to at least one of the following: a low-delay picture or a non-low-delay picture. In some embodiments, a weight index in a predefined table of M possible weighting factors is determined, where M is an integer. In some embodiments, M is one of the following: 3, 4, 5, or 6. In some embodiments, the value of M depends on the codec information of the current picture. For example, depending on the codec information of the current picture (e.g., a low-delay picture), M may be equal to different values.

[0150] In some embodiments, the weight index is indicated in the bitstream. In some embodiments, the weight index is implicitly derived by a decoder-side derivation method. In some embodiments, the weight index is determined based on a cost derived based on at least one of: decoder-side motion vector refinement (DMVR), decoder-side motion vector difference (DMVD), multi-pass decoder-side motion vector difference (mpDMVD), prediction unit decoder-side motion vector difference (puDMVD), a variant of DMVR, a variant of DMVD, a variant of mpDMVD, or a variant of puDMVd.

[0151] In some embodiments, the weight index is determined based on a template matching (TM) cost. In some embodiments, the template derived for the TM cost is equal to M*N samples adjacent to the current block and the reference block, where M and N are integers. In some embodiments, the template derived for the TM cost is equal to M*N samples adjacent to the first reference block and the second reference block.

[0152] In some embodiments, the TM cost or DMVR cost is scaled for the cost comparison. In some embodiments, the TM cost or DMVR cost derived by averaging the weights is scaled. For example, the TM / DMVR cost derived by averaging the weights can be scaled (e.g., to a smaller cost value).

[0153] In some embodiments, K entries unequally weighted are applied to a multiple hypothesis prediction (MHP) mode, where K is an integer. For example, K entries (such as K>2 or K=2) unequally weighted can be applied to the MHP mode.

[0154] In some embodiments, K-entry non-even weighting is applied to MHP AMVP mode to weight the two hypotheses. For example, it can be applied to MHP AMVP mode (eg, generating additional MHP hypotheses based on AMVP) to weight the two hypotheses.

[0155] Alternatively, K-entry non-even weighting is applied to the MHP MERGE mode to weight the two hypotheses. For example, it can be applied to the MHP MERGE mode (eg, generating additional MHP hypotheses based on MERGE) to weight the two hypotheses.

[0156] In some embodiments, the K entries are non-averagely weighted and are at least one of: a low-delay picture or a non-low-delay picture. In some embodiments, a weight index in a predefined table comprising M weighting factors is determined, where M is an integer. In some embodiments, M is equal to 2, 3, 4, 5, or 6. In some embodiments, the value of M depends on the codec information of the current picture. For example, depending on the codec information of the current picture (e.g., a low-delay picture), M may be equal to different values.

[0157] In some embodiments, the value of M depends on whether the video unit is encoded using MHP AMVP mode or MHP MERGE mode. For example, M may be equal to different values ​​depending on MHP AMVP or MHP MERGE.

[0158] In some embodiments, the weight index is indicated in the bitstream. In some embodiments, the weight index is implicitly derived by a decoder-side derivation method. In some embodiments, the weight index is determined based on a cost derived based on at least one of: decoder-side motion vector refinement (DMVR), decoder-side motion vector difference (DMVD), mpDMVD, puDMVD, a variant of DMVR, a variant of DMVD, a variant of mpDMVD, or a variant of puDMVD.

[0159] In some embodiments, the weight index is determined based on a template matching (TM) cost. In some embodiments, the template derived for the TM cost is equal to M*N samples neighboring the current block and the reference block, where M and N are integers.

[0160] In some embodiments, the template derived for the TM cost is equal to M*N samples adjacent to the first reference block and the second reference block. In some embodiments, the TM cost or DMVR cost is scaled for the cost comparison. In some embodiments, the TM cost or DMVR cost derived by averaging the weights is scaled. For example, the TM / DMVR cost derived by averaging the weights can be scaled (e.g., to a smaller cost value).

[0161] In some embodiments, the decoder-derived non-average weighting method is applied to at least one of: a non-low-latency picture, a low-latency picture, an MHP AMVP mode, a variant of the MHP AMVP mode, an MHP MERGE mode, a variant of the MHP MERGE mode, a merge mode with motion vector difference (MMVD), a variant of MMVD, an AMVP-MERGE mode, or a variant of the AMVP-MERGE mode.

[0162] In some embodiments, the indication of whether and / or how to apply a non-average weighting method to a video unit is indicated at one of: a sequence level, a group of pictures level, a picture level, a slice level, or a tile group level.

[0163] In some embodiments, an indication of whether and / or how a non-average weighting method is applied to a video unit is indicated in one of the following: a sequence header, a picture header, a sequence parameter set (SPS), a video parameter set (VPS), a dependency parameter set (DPS), decoding capability information (DCI), a picture parameter set (PPS), an adaptation parameter set (APS), a slice header, or a tile group header.

[0164] In some embodiments, an indication of whether and / or how a non-average weighting method is applied to a video unit is included in one of the following: a prediction block (PB), a transform block (TB), a codec block (CB), a prediction unit (PU), a transform unit (TU), a codec unit (CU), a virtual pipeline data unit (VPDU), a codec tree unit (CTU), a CTU row, a slice, a tile, a sub-picture, or a region containing more than one sample or pixel.

[0165] In some embodiments, method 3500 further includes determining whether and / or how to apply a non-average weighting method to the video unit based on coded information of the video unit. The coded information may include at least one of the following: block size, color format, single tree and / or dual tree partitioning, color component, slice type, or picture type.

[0166] According to another embodiment of the present disclosure, a non-transitory computer-readable recording medium is provided. The non-transitory computer-readable recording medium stores a bitstream of a video generated by a method performed by a device for video processing. The method includes: applying a non-average weighting method to a video unit of the video, wherein the video unit is applied with an advanced motion vector prediction (AMVP) merge mode; and generating a bitstream of the video unit based on the weighted video unit.

[0167] According to some other embodiments of the present disclosure, a method for storing a bitstream of a video is provided. The method includes: applying a non-average weighting method to a video unit of the video, wherein the video unit is applied with an advanced motion vector prediction (AMVP) merge mode; generating a bitstream of the video unit based on the weighted video unit; and storing the bitstream in a non-transitory computer-readable recording medium.

[0168] Embodiments of the present disclosure may be described according to the following clauses, features of which may be combined in any reasonable way.

[0169] Clause 1. A method of video processing, comprising: for conversion between a video unit of a video and a bitstream of the video, determining that the video unit is encoded and decoded by a motion pair from the same direction relative to a current picture, wherein the video unit is applied with an advanced motion vector prediction (AMVP) merge mode, and the video unit belongs to a low-latency picture; and performing the conversion based on the motion pair.

[0170] Clause 2. The method of clause 1, wherein the motion pair comprises an AMVP motion vector (MV) and a MERGE MV.

[0171] Clause 3. The method of clause 1, wherein the AMVP merge mode is enabled for the low-delay picture in a random access configuration.

[0172] Clause 4. The method of clause 1, wherein the AMVP merge mode is enabled for the low-latency picture in a low-latency configuration.

[0173] Clause 5. The method of clause 1, wherein the motion pair comprises an AMVP MV and a MERGE MV, and wherein the AMVP MV points to a first reference picture having a first picture order count (POC) value and the MERGE MV points to a second reference picture having a second POC value.

[0174] Clause 6. The method of clause 1 further comprising: applying a two-step reference picture checking process to the video unit for generation of an AMVP merge list.

[0175] Clause 7. The method of clause 6, wherein applying the two-step reference picture checking process comprises: determining whether a first condition is satisfied.

[0176] Clause 8. The method of clause 7, wherein the first condition comprises whether there are available reference pictures from both a forward direction and a backward direction.

[0177] Clause 9. The method according to clause 7, wherein if the first condition is met, a true bidirectional AMVP merge mode is performed.

[0178] Clause 10. The method according to clause 7 further comprises: if the first condition is not satisfied, determining whether a second condition is satisfied.

[0179] Clause 11. The method of clause 10, wherein the second condition comprises whether there is an available reference picture from the same direction but with a different POC distance.

[0180] Clause 12. The method according to clause 10, wherein if the second condition is satisfied, a same-direction AMVP merge mode is performed.

[0181] Clause 13. The method of clause 1, wherein the AMVP MV in the motion pair is determined based on at least one of: a coded motion vector prediction (MVP) index or a coded reference index.

[0182] Clause 14. The method of clause 1, wherein template matching (TM) based refinement is used for the motion pairs coming from the same direction.

[0183] Clause 15. The method of clause 1, wherein if there are multiple reference pictures having a different POC distance from the AMVP reference picture, a target reference picture from the multiple reference pictures is selected to derive the merged MV in the motion pair.

[0184] Clause 16. The method of clause 15, wherein the target reference picture is selected based on a distance to a current picture.

[0185] Clause 17. The method of clause 16, wherein the target reference picture is closer to the current picture than other reference pictures.

[0186] Clause 18. The method of clause 16, wherein the target reference picture is then closer to the AMVP reference picture than the other reference pictures.

[0187] Clause 19. The method of clause 15, wherein the target reference picture is selected based on a POC distance.

[0188] Clause 20. The method of clause 19, wherein among the plurality of reference pictures, the target reference picture has a smaller POC value than other reference pictures.

[0189] Clause 21. The method of clause 19, wherein among the plurality of reference pictures, the target reference picture has a larger POC value than other reference pictures.

[0190] Clause 22. The method of clause 1, wherein the MERGE MV in the motion pair is determined based on a rule.

[0191] Clause 23. The method of clause 22, wherein the rule is based on a predefined position of the MERGE MV candidate in the MERGE MV list.

[0192] Clause 24. The method of clause 23, wherein the MERGE MV candidate is the Mth available MERGE MV candidate in the MERGE MV list, where M is an integer.

[0193] Clause 25. The method of clause 22, wherein the rule is based on TM cost.

[0194] Clause 26. The method of clause 25, wherein a plurality of TM costs are determined for a plurality of MERGE MV candidates, and a MERGE MV candidate having a minimum TM cost is selected as the MERGE MV in the motion pair.

[0195] Clause 27. The method of Clause 25, wherein the TM cost comprises costs determined for both AMVP MV candidates and MERGEMV candidates.

[0196] Clause 28. The method of Clause 25, wherein the TM cost comprises a cost determined solely by MERGE MV candidates.

[0197] Clause 29. A method according to any of clauses 1 to 28, wherein decoder-side motion vector refinement (DMVR) of the video unit is turned off, and / or wherein reference picture resampling (RPR) of the video unit is turned on.

[0198] Clause 30. A method according to any one of clauses 1 to 29, wherein an indication of whether and / or how to determine that the video unit is encoded by the motion pair from the same direction relative to the current picture is indicated at one of: a sequence level, a picture group level, a picture level, a slice level, or a tile group level.

[0199] Clause 31. A method according to any one of clauses 1 to 29, wherein an indication of whether and / or how to determine that the video unit is encoded and decoded by the motion pair from the same direction relative to the current picture is indicated in one of the following: a sequence header, a picture header, a sequence parameter set (SPS), a video parameter set (VPS), a dependency parameter set (DPS), decoding capability information (DCI), a picture parameter set (PPS), an adaptation parameter set (APS), a slice header, or a tile group header.

[0200] Clause 32. A method according to any one of clauses 1 to 29, wherein the indication of whether and / or how to determine whether the video unit is encoded and decoded by the motion pair from the same direction relative to the current picture is included in one of the following: a prediction block (PB), a transform block (TB), a codec block (CB), a prediction unit (PU), a transform unit (TU), a codec unit (CU), a virtual pipeline data unit (VPDU), a codec tree unit (CTU), a CTU row, a slice, a tile, a sub-picture, or a region containing more than one sample or pixel.

[0201] Clause 33. The method according to any one of clauses 1 to 29 further includes: determining whether and / or how to determine that the video unit is encoded and decoded by the motion pair from the same direction relative to the current picture based on the encoded and decoded information of the video unit, the encoded and decoded information including at least one of the following: block size, color format, single tree and / or double tree partitioning, color component, slice class, or picture type.

[0202] Clause 34. A method of video processing, comprising: for conversion between a video unit of a video and a bitstream of the video, applying decoder-side motion vector refinement (DMVR) or at least one of its variants to the video unit to refine the motion vector of the video unit, wherein the video unit is applied with one of the following: affine advanced motion vector prediction (AMVP) mode, sub-block based temporal motion vector prediction (sbTMVP) mode or affine merge mode; and performing the conversion based on the refined motion vector.

[0203] Clause 35. The method of clause 34, wherein at least one of the DMVR or a variant of the DMVR is used to refine a shifted motion vector MV for the sbTMVP mode.

[0204] Clause 36. The method of clause 35, wherein the shifted MV is bi-directionally predicted.

[0205] Clause 37. The method of clause 35, wherein the shifted MV points to two reference pictures from both forward and backward.

[0206] Clause 38. The method according to clause 35, wherein for the sbTMVP mode, the shifted MV that satisfies a DMVR condition or a decoder-side motion vector difference (DMVD) condition is generated.

[0207] Clause 39. The method of clause 35, wherein the shifted MV is used to derive a motion vector from a corresponding prediction unit (PU) in a reference picture.

[0208] Clause 40. The method of clause 34, wherein at least one of the DMVR or a variant of the DMVR is used to refine control point motion vectors (CPMVs) of the affine merge mode.

[0209] Clause 41. The method of clause 34, wherein at least one of the DMVR or a variant of the DMVR is used to refine control point motion vectors (CPMVs) of the affine AMVP mode.

[0210] Clause 42. The method of clause 41, wherein only PU-level DMVD or DMVR is used.

[0211] Clause 43. The method of clause 41, wherein at least one of the DMVR or a variation of the DMVR is applied to the CPMV satisfying a DMVR condition or a DMVD condition.

[0212] Clause 44. The method of clause 41, wherein at least one of the DMVR or a variant of the DMVR is applied to the CPMV pointing to reference pictures from both forward and backward directions.

[0213] Clause 45. The method of clause 41, wherein at least one of the DMVR or a variant of the DMVR is applied to the CPMV pair of a bi-predictive affine AMVP mode.

[0214] Clause 46. A method according to any one of clauses 34 to 45, wherein an indication of whether and / or how to determine that the video unit is encoded by the motion pair from the same direction relative to the current picture is indicated at one of: a sequence level, a picture group level, a picture level, a slice level, or a tile group level.

[0215] Clause 47. A method according to any one of clauses 34 to 45, wherein an indication of whether and / or how to determine that the video unit is encoded and decoded by the motion pair from the same direction relative to the current picture is indicated in one of the following: a sequence header, a picture header, a sequence parameter set (SPS), a video parameter set (VPS), a dependency parameter set (DPS), decoding capability information (DCI), a picture parameter set (PPS), an adaptation parameter set (APS), a slice header, or a tile group header.

[0216] Clause 48. A method according to any one of clauses 34 to 45, wherein the indication of whether and / or how to determine whether the video unit is encoded and decoded by the motion pair from the same direction relative to the current picture is included in one of the following: a prediction block (PB), a transform block (TB), a codec block (CB), a prediction unit (PU), a transform unit (TU), a codec unit (CU), a virtual pipeline data unit (VPDU), a codec tree unit (CTU), a CTU row, a slice, a tile, a sub-picture, or a region containing more than one sample or pixel.

[0217] Clause 49. The method according to any one of clauses 34 to 45 further includes: determining whether and / or how to determine that the video unit is encoded and decoded by the motion pair from the same direction relative to the current picture based on the encoded and decoded information of the video unit, the encoded and decoded information including at least one of the following: block size, color format, single tree and / or double tree partitioning, color component, slice type, or picture type.

[0218] Clause 50. A video processing method, comprising: during conversion between a video unit of a video and a bitstream of the video, deriving at least one LIC model parameter of a local illumination compensation (LIC) model for the video unit based on a nonlinear model, wherein the encoded video unit is a LIC encoded block; updating the LIC model by adjusting the at least one LIC model parameter; and performing the conversion based on the updated LIC model.

[0219] Clause 51. A method according to clause 50, wherein the at least one LIC model parameter is derived based on predC=a0*((predR*predR+b)>>s)+a1*predR+a2*c, wherein predC represents the predicted sample value of the sample adjacent to the current block, predR represents the predicted sample value of the sample adjacent to the reference block, a0, a1 and a2 represent LIC parameters respectively, b and c are integers, and s represents a shift factor which is an integer.

[0220] Clause 52. A method according to clause 50, wherein the at least one LIC model parameter is derived based on predC=a0*((recR*recR+b)>>s)+a1*recR+a2*c, where recC represents the reconstructed sample value of the sample adjacent to the current block, recR represents the reconstructed sample value of the sample adjacent to the reference block, a0, a1 and a2 represent LIC parameters respectively, b and c are integers, and s represents a shift factor which is an integer.

[0221] Clause 53. The method of clause 51 or 52, wherein the at least one LIC model parameter is derived based on an LDL decomposition method, or wherein the at least one LIC model parameter is derived based on a coefficient solution method based on Gaussian elimination.

[0222] Clause 54. The method of clause 51 or 52, wherein M rows and N columns of neighboring samples are used to derive the at least one LIC model parameter, wherein M and N are integers.

[0223] Clause 55. The method according to clause 54, wherein M is equal to an integer and is not greater than 6.

[0224] Clause 56. The method of clause 54, wherein N is equal to an integer not greater than 6.

[0225] Clause 57. The method of clause 54, wherein the value of at least M or N depends on a codec tree unit (CTU) boundary or a largest codec unit (LCU) boundary.

[0226] Clause 58. The method of clause 51 or 52, wherein the shift factor is derived based on an internal bit depth of the encoded samples.

[0227] Clause 59. A method according to clause 51 or 52, wherein the values ​​of b and c are derived based on at least one of: the median value of the sample values ​​adjacent to the current block, the middle value of the sample values ​​adjacent to the current block, the average value of the sample values ​​adjacent to the current block, the median value of the sample values ​​adjacent to the reference block, the middle value of the sample values ​​adjacent to the reference block, or the average value of the sample values ​​adjacent to the reference block.

[0228] Clause 60. The method of clause 59, wherein b is equal to c.

[0229] Clause 61. Wherein the indication of whether and / or how to derive the at least one LIC model parameter based on the non-linear model is indicated at one of: a sequence level, a group of pictures level, a picture level, a slice level, or a tile group level.

[0230] Clause 62. A method according to any one of clauses 50 to 60, wherein an indication of whether and / or how to derive the at least one LIC model parameter based on the non-linear model is indicated in one of the following: a sequence header, a picture header, a sequence parameter set (SPS), a video parameter set (VPS), a dependency parameter set (DPS), decoding capability information (DCI), a picture parameter set (PPS), an adaptation parameter set (APS), a slice header, or a tile group header.

[0231] Clause 63. Wherein an indication of whether and / or how to derive the at least one LIC model parameter based on the non-linear model is included in one of the following: a prediction block (PB), a transform block (TB), a codec block (CB), a prediction unit (PU), a transform unit (TU), a codec unit (CU), a virtual pipeline data unit (VPDU), a codec tree unit (CTU), a CTU row, a slice, a tile, a sub-picture, or a region containing more than one sample or pixel.

[0232] Clause 64. The method according to any one of clauses 50 to 60 further comprises: determining whether and / or how to derive the at least one LIC model parameter based on the non-linear model based on coded information of the video unit, wherein the coded information comprises at least one of the following: block size, color format, single tree and / or dual tree partitioning, color component, slice type, or picture type.

[0233] Item 65. A video processing method, comprising: for conversion between a video unit of a video and a bitstream of the video, applying a non-average weighting method to the video unit, wherein the video unit is applied with an advanced motion vector prediction (AMVP) merge mode; and performing the conversion based on the weighted video unit.

[0234] Clause 66. The method of clause 65, wherein the non-average weighting method is applied to at least one of: a low-latency picture or a non-low-latency picture.

[0235] Clause 67. The method of clause 65, wherein a weight index in a predefined table of M possible weighting factors is determined, where M is an integer.

[0236] Clause 68. The method according to clause 67, wherein M is one of the following: 3, 4, 5 or 6.

[0237] Clause 69. The method according to clause 67, wherein the value of M depends on the coding information of the current picture.

[0238] Clause 70. The method of clause 65, wherein a weight index is indicated in the bitstream.

[0239] Clause 71. The method of clause 65, wherein the weight index is implicitly derived by a decoder-side derivation method.

[0240] Clause 72. A method according to clause 71, wherein the weight index is determined based on a cost, and the cost is derived based on at least one of the following: decoder side motion vector refinement (DMVR), decoder side motion vector difference (DMVD), multi-pass decoder side motion vector difference (mpDMVD), prediction unit decoder side motion vector difference (puDMVD), a variant of DMVR, a variant of DMVD, a variant of mpDMVD, or a variant of puDMVd.

[0241] Clause 73. The method of clause 71, wherein the weight index is determined based on a template matching (TM) cost.

[0242] Clause 74. The method of clause 71, wherein the template derived for the TM cost is equal to M*N samples neighboring the current block and the reference block, where M and N are integers.

[0243] Clause 75. The method of clause 71, wherein the template derived for the TM cost is equal to M*N samples neighboring the first reference block and the second reference block.

[0244] Clause 76. The method of clause 71, wherein the TM cost or the DMVR cost is scaled for the cost comparison.

[0245] Clause 77. The method of clause 76, wherein the TM cost or the DMVR cost derived by weighted averaging is scaled.

[0246] Clause 78. The method of clause 64, wherein non-equal weighting of K entries is applied to a multiple hypothesis prediction (MHP) mode, where K is an integer.

[0247] Clause 79. The method of clause 78, wherein the K-entry non-even weighting is applied in MHP AMVP mode to weight two hypotheses, or wherein the K-entry non-even weighting is applied in MHP MERGE mode to weight two hypotheses.

[0248] Clause 80. The method of clause 78, wherein the K entries non-equally weighted are at least one of: low-latency pictures or non-low-latency pictures.

[0249] Clause 81. The method of clause 78, wherein a weight index in a predefined table of M weighting factors is determined, wherein M is an integer.

[0250] Clause 82. The method according to clause 81, wherein M is one of the following: 2, 3, 4, 5 or 6.

[0251] Clause 83. A method according to clause 81, wherein the value of M depends on the codec information of the current picture.

[0252] Clause 84. The method of clause 81, wherein the value of M depends on whether the video unit is encoded using MHP AMVP mode or MHP MERGE mode.

[0253] Clause 85. The method of clause 78, wherein a weight index is indicated in the bitstream.

[0254] Clause 86. The method of clause 78, wherein the weight index is implicitly derived through a decoder-side derivation method.

[0255] Clause 87. A method according to clause 86, wherein the weight index is determined based on a cost, and the cost is derived based on at least one of the following: decoder side motion vector refinement (DMVR), decoder side motion vector difference (DMVD), multi-pass decoder side motion vector difference (mpDMVD), prediction unit decoder side motion vector difference (puDMVD), a variant of DMVR, a variant of DMVD, a variant of mpDMVD, or a variant of puDMVD.

[0256] Clause 88. The method of clause 86, wherein the weight index is determined based on a template matching (TM) cost.

[0257] Clause 89. The method of clause 86, wherein the template derived for the TM cost is equal to M*N samples neighboring the current block and the reference block, where M and N are integers.

[0258] Clause 90. The method of clause 86, wherein the template derived for the TM cost is equal to M*N samples neighboring the first reference block and the second reference block.

[0259] Clause 91. The method of clause 86, wherein the TM cost or the DMVR cost is scaled for the cost comparison.

[0260] Clause 92. The method of clause 91, wherein the TM cost or the DMVR cost derived by weighted averaging is scaled.

[0261] Clause 93. A method according to clause 65, wherein the decoder-derived non-average weighting method is applied to at least one of: a non-low-latency picture, a low-latency picture, an MHP AMVP mode, a variant of the MHP AMVP mode, an MHP MERGE mode, a variant of the MHP MERGE mode, a merge mode with motion vector difference (MMVD), a variant of MMVD, an AMVP-MERGE mode, or a variant of the AMVP-MERGE mode.

[0262] Clause 94. A method according to any one of clauses 65 to 93, wherein an indication of whether and / or how to apply a non-average weighting method to the video unit is indicated at one of the following: sequence level, picture group level, picture level, slice level, or tile group level.

[0263] Clause 95. A method according to any one of clauses 65 to 93, wherein an indication of whether and / or how a non-average weighting method is applied to the video unit is indicated in one of the following: a sequence header, a picture header, a sequence parameter set (SPS), a video parameter set (VPS), a dependency parameter set (DPS), decoding capability information (DCI), a picture parameter set (PPS), an adaptation parameter set (APS), a slice header, or a tile group header.

[0264] Clause 96. A method according to any one of clauses 65 to 93, wherein an indication of whether and / or how to apply a non-average weighting method to the video unit is included in one of the following: a prediction block (PB), a transform block (TB), a codec block (CB), a prediction unit (PU), a transform unit (TU), a codec unit (CU), a virtual pipeline data unit (VPDU), a codec tree unit (CTU), a CTU row, a slice, a tile, a sub-picture, or an area containing more than one sample or pixel.

[0265] Clause 97. The method according to any one of clauses 65 to 93 further includes: determining whether and / or how to apply a non-average weighting method to the video unit based on the encoded and decoded information of the video unit, the encoded and decoded information including at least one of the following: block size, color format, single tree and / or double tree partitioning, color component, slice type, or picture type.

[0266] Clause 98. A method according to any one of clauses 1 to 97, wherein the converting comprises encoding the video unit into the bitstream.

[0267] Clause 99. A method according to any one of clauses 1 to 97, wherein the converting comprises decoding the video unit from the bitstream.

[0268] Clause 100. An apparatus for video processing, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform a method according to any one of claims 1 to 99.

[0269] Clause 101. A non-transitory computer-readable storage medium storing instructions for causing a processor to perform a method according to any one of claims 1 to 99.

[0270] A non-transitory computer-readable recording medium stores a bitstream of a video generated by a method performed by an apparatus for video processing, wherein the method comprises: determining that a video unit of the video is encoded and decoded by motion pairs from the same direction relative to a current picture, wherein the video unit is applied with an advanced motion vector prediction (AMVP) merge mode, and the video unit belongs to a low-latency picture; and generating a bitstream of the video unit based on the motion pairs.

[0271] Item 103. A method for storing a bitstream of a video, comprising: determining that a video unit of the video is encoded and decoded by motion pairs from the same direction relative to a current picture, wherein the video unit is applied with an advanced motion vector prediction (AMVP) merge mode and the video unit belongs to a low-latency picture; generating a bitstream of the video unit based on the motion pairs; and storing the bitstream in a non-transitory computer-readable recording medium.

[0272] Item 104. A non-transitory computer-readable recording medium storing a bitstream of a video generated by a method performed by an apparatus for video processing, wherein the method comprises: applying decoder-side motion vector refinement (DMVR) or at least one of a variant of DMVR to a video unit of the video to refine the motion vector of the video unit, wherein the video unit is applied with one of the following: affine advanced motion vector prediction (AMVP) mode, sub-block based temporal motion vector prediction (sbTMVP) mode, or affine merge mode; and generating a bitstream of the video unit based on the refined motion vector.

[0273] Item 105. A method for storing a bitstream of a video, comprising: applying decoder-side motion vector refinement (DMVR) or at least one of a variant of DMVR to a video unit of the video to refine the motion vector of the video unit, wherein the video unit is applied with one of the following: affine advanced motion vector prediction (AMVP) mode, sub-block based temporal motion vector prediction (sbTMVP) mode, or affine merge mode; generating a bitstream of the video unit based on the refined motion vector; and storing the bitstream in a non-transitory computer-readable recording medium.

[0274] Item 106. A non-transitory computer-readable recording medium storing a bitstream of a video generated by a method performed by an apparatus for video processing, wherein the method comprises: deriving at least one LIC model parameter of a local illumination compensation (LIC) model for a video unit of the video based on a nonlinear model, wherein the encoded and decoded video unit is a LIC encoded and decoded block; updating the LIC model by adjusting the at least one LIC model parameter; and generating a bitstream of the video unit based on the updated LIC model.

[0275] Clause 107. A method for storing a bitstream of a video, comprising: deriving at least one local illumination compensation (LIC) model parameter of a video unit of the video based on a nonlinear model, wherein the encoded video unit is a LIC encoded block; updating the LIC model by adjusting the at least one LIC model parameter; generating a bitstream of the video unit based on the updated LIC model; and storing the bitstream in a non-transitory computer-readable recording medium.

[0276] Item 108. A non-transitory computer-readable recording medium storing a bitstream of a video generated by a method performed by a device for video processing, wherein the method comprises: applying a non-average weighting method to video units of the video, wherein the video units are applied with an advanced motion vector prediction (AMVP) merge mode; and generating a bitstream of the video units based on the weighted video units.

[0277] Item 109. A method for storing a bitstream of a video, comprising: applying a non-average weighting method to video units of the video, wherein the video units are applied with an advanced motion vector prediction (AMVP) merge mode; generating a bitstream of the video units based on the weighted video units; and storing the bitstream in a non-transitory computer-readable recording medium. Example Device

[0278] Fig.36 A block diagram of a computing device 3600 in which various embodiments of the present disclosure may be implemented is illustrated. The computing device 3600 may be implemented as or included in the source device 110 (or video encoder 114 or 200) or the destination device 120 (or video decoder 124 or 300).

[0279] It should be understood that Fig.36 The computing device 3600 shown is for illustrative purposes only and is not intended to in any way imply any limitation on the functionality and scope of the embodiments of the present disclosure.

[0280] like Fig.36 As shown, computing device 3600 includes a general computing device 3600. Computing device 3600 may include at least one or more processors or processing units 3610, memory 3620, storage unit 3630, one or more communication units 3640, one or more input devices 3650, and one or more output devices 3660.

[0281] In some embodiments, computing device 3600 can be implemented as any user terminal or server terminal with computing power. The server terminal can be a server, a large computing device, etc. provided by a service provider. The user terminal can be, for example, any type of mobile terminal, fixed terminal or portable terminal, including a mobile phone, a station, a unit, a device, a multimedia computer, a multimedia tablet, an Internet node, a communicator, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an electronic book device, a gaming device, or any combination thereof, including accessories and peripherals of these devices or any combination thereof. It will be envisioned that computing device 3600 can support any type of interface to the user (such as a "wearable" circuit, etc.).

[0282] Processing unit 3610 may be a physical or virtual processor and may implement various processes based on programs stored in memory 3620. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capabilities of computing device 3600. Processing unit 3610 may also be referred to as a central processing unit (CPU), a microprocessor, a controller, or a microcontroller.

[0283] The computing device 3600 typically includes various computer storage media. Such media can be any media accessible to the computing device 3600, including but not limited to volatile and non-volatile media, or removable and non-removable media. The memory 3620 can be a volatile memory (e.g., a register, a cache, a random access memory (RAM)), a non-volatile memory (such as a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM) or flash memory), or any combination thereof. The storage unit 3630 can be any removable or non-removable medium, and can include a machine-readable medium that can be used to store information and / or data and can be accessed in the computing device 3600, such as a memory, a flash drive, a disk, or another other medium.

[0284] The computing device 3600 may also include additional removable / non-removable, volatile / non-volatile storage media. Fig.36Although not shown in the figure, a magnetic disk drive for reading from and / or writing to a removable and nonvolatile magnetic disk and an optical disk drive for reading from and / or writing to a removable nonvolatile optical disk may be provided. In such a case, each drive may be connected to the bus (not shown) via one or more data media interfaces.

[0285] The communication unit 3640 communicates with another computing device via a communication medium. In addition, the functionality of the components in the computing device 3600 can be implemented by a single computing cluster or multiple computing machines that can communicate via a communication connection. Therefore, the computing device 3600 can work in a networked environment using logical connections to one or more other servers, networked personal computers (PCs), or other general-purpose network nodes.

[0286] Input device 3650 may be one or more of various input devices, such as a mouse, keyboard, trackball, voice input device, etc. Output device 3660 may be one or more of various output devices, such as a display, a loudspeaker, a printer, etc. Computing device 3600 may also communicate with one or more external devices (not shown) such as storage devices and display devices, with the aid of communication unit 3640, where one or more devices enable a user to interact with computing device 3600 or any device that enables computing device 3600 to communicate with one or more other computing devices (such as a network card, a modem, etc.), as desired. Such communication may be performed via an input / output (I / O) interface (not shown).

[0287] In some embodiments, some or all components of the computing device 3600 may also be arranged in a cloud computing architecture, rather than being integrated in a single device. In a cloud computing architecture, components may be provided remotely and work together to implement the functions described in the present disclosure. In some embodiments, cloud computing provides computing, software, data access, and storage services, which do not require end users to understand the physical location or configuration of the system or hardware that provides these services. In various embodiments, cloud computing provides services via a wide area network (such as the Internet) using a suitable protocol. For example, a cloud computing provider provides applications via a wide area network, which can be accessed through a web browser or any other computing component. The software or components of the cloud computing architecture and the corresponding data may be stored on a server at a remote location. The computing resources in a cloud computing environment may be merged or distributed at a location in a remote data center. Cloud computing infrastructure may provide services through a shared data center, but they act as a single access point for users. Therefore, cloud computing architecture may be used to provide components and functions described herein from a service provider at a remote location. Alternatively, they may be provided from a conventional server or directly installed or otherwise installed on a client device.

[0288] The computing device 3600 may be used to implement video encoding / decoding in embodiments of the present invention. The memory 3620 may include one or more video encoding / decoding modules 3625 having one or more program instructions. These modules may be accessed and executed by the processing unit 3610 to perform the functions of various embodiments described herein.

[0289] In an example embodiment performing video encoding, input device 3650 may receive video data as input to be encoded 3670. For example, the video data may be processed by video codec module 3625 to generate an encoded bitstream. The encoded bitstream may be provided as output 3680 via output device 3660.

[0290] In an example embodiment performing video decoding, input device 3650 may receive an encoded bitstream as input 3670. The encoded bitstream may be processed, for example, by video codec module 3625 to generate decoded video data. The decoded video data may be provided as output 3680 via output device 3660.

[0291] Although the present disclosure has been specifically shown and described with reference to the preferred embodiments of the present disclosure, it will be appreciated by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the present application as defined by the appended claims. These changes are intended to be encompassed by the scope of the present application. Therefore, the foregoing description of the embodiments of the present application is not intended to be limiting.

Claims

1. A video processing method, comprising: For conversion between a video unit of a video and a bitstream of the video, determining that the video unit is encoded and decoded with a motion pair from a same direction relative to a current picture, wherein the video unit has an advanced motion vector prediction (AMVP) merge mode applied thereto, and the video unit belongs to a low-delay picture; and The transforming is performed based on the motion pair. 2 . The method of claim 1 , wherein the motion pair comprises an AMVP motion vector (MV) and a merged MV. 3 . The method of claim 1 , wherein the AMVP merge mode is enabled for the low-delay picture in a random access configuration. The method of claim 1 , wherein the AMVP merge mode is enabled for the low-latency picture in a low-latency configuration.

5. The method of claim 1, wherein the motion pair comprises an AMVP MV and a merged MV, and The AMVP MV points to a first reference picture having a first picture order count (POC) value, and the merged MV points to a second reference picture having a second POC value.

6. The method according to claim 1, further comprising: For the generation of the AMVP merge list, a two-step reference picture checking process is applied to the video unit.

7. The method of claim 6, wherein applying the two-step reference picture checking process comprises: It is determined whether the first condition is satisfied. The method of claim 7 , wherein the first condition comprises whether there are available reference pictures from both a forward direction and a backward direction.

9. The method according to claim 7, wherein if the first condition is met, a true bidirectional AMVP merge mode is performed.

10. The method according to claim 7, further comprising: If the first condition is not satisfied, it is determined whether the second condition is satisfied. The method according to claim 10 , wherein the second condition comprises whether there is an available reference picture from the same direction but with a different POC distance. 12 . The method according to claim 10 , wherein if the second condition is satisfied, performing the same-direction AMVP merging mode.

13. The method of claim 1, wherein the AMVP MV in the motion pair is determined based on at least one of: a coded motion vector prediction (MVP) index or a coded reference index.

14. The method of claim 1, wherein Template Matching (TM) based refinement is used for the motion pairs coming from the same direction. 15 . The method of claim 1 , wherein if there are multiple reference pictures having a different POC distance from that of an AMVP reference picture, a target reference picture from among the multiple reference pictures is selected to derive a merged MV in the motion pair. The method of claim 15 , wherein the target reference picture is selected based on a distance to a current picture. The method of claim 16 , wherein the target reference picture is closer to the current picture than other reference pictures.

18. The method of claim 16, wherein the target reference picture is then closer to the AMVP reference picture than other reference pictures. The method of claim 15 , wherein the target reference picture is selected based on a POC distance. 20 . The method of claim 19 , wherein among the plurality of reference pictures, the target reference picture has a smaller POC value than other reference pictures. 21 . The method of claim 19 , wherein among the plurality of reference pictures, the target reference picture has a larger POC value than other reference pictures.

22. The method of claim 1, wherein the merged MV in the motion pair is determined based on a rule.

23. The method of claim 22, wherein the rule is based on a predefined position of a merge MV candidate in a merge MV list. The method of claim 23 , wherein the merge MV candidate is the Mth available merge MV candidate in the merge MV list, where M is an integer.

25. The method of claim 22, wherein the rules are based on TM costs. 26 . The method of claim 25 , wherein a plurality of TM costs are determined for a plurality of merge MV candidates, and a merge MV candidate with a minimum TM cost is selected as the merge MV in the motion pair.

27. The method of claim 25, wherein the TM cost comprises costs determined for both AMVP MV candidates and merged MV candidates.

28. The method of claim 25, wherein the TM cost comprises a cost determined solely by merging MV candidates.

29. The method of any one of claims 1 to 28, wherein decoder-side motion vector refinement (DMVR) of the video unit is turned off, and / or Wherein reference picture resampling (RPR) of the video unit is turned on.

30. The method of any one of claims 1 to 29, wherein the indication of whether and / or how to determine that the video unit is encoded by the motion pair from the same direction relative to the current picture is indicated at one of: Sequence level, Picture group level, Picture level, Stripe level, or Tile group level.

31. The method according to any one of claims 1 to 29, wherein the indication of whether and / or how to determine that the video unit is encoded by the motion pair from the same direction relative to the current picture is indicated in one of: Sequence header, Picture header, Sequence Parameter Set (SPS), Video Parameter Set (VPS), Dependency Parameter Set (DPS), Decoding Capability Information (DCI), Picture Parameter Set (PPS), Adaptive Parameter Set (APS), Strip header, or Tile group header.

32. The method according to any one of claims 1 to 29, wherein the indication of whether and / or how to determine that the video unit is encoded by the motion pair from the same direction relative to the current picture is comprised in one of: Prediction Block (PB), Transform Block (TB), Codec Block (CB), Prediction Unit (PU), Transformation Unit (TU), Codec Unit (CU), Virtual Pipeline Data Unit (VPDU), Codec Tree Unit (CTU), CTU line, Strips, Tile, sub-image, or An area that contains more than one sample or pixel.

33. The method according to any one of claims 1 to 29, further comprising: determining whether and / or how to determine that the video unit is encoded with the motion pair from the same direction relative to the current picture based on encoded information of the video unit, the encoded information comprising at least one of: Block size, Color format, Single-tree and / or dual-tree partitioning, Color components, Strip type, or Image type.

34. A method for video processing, comprising: For conversion between a video unit of a video and a bitstream of the video, applying at least one of decoder-side motion vector refinement (DMVR) or a variant of DMVR to the video unit to refine a motion vector of the video unit, wherein the video unit is applied with one of: an affine advanced motion vector prediction (AMVP) mode, a sub-block based temporal motion vector prediction (sbTMVP) mode, or an affine merge mode; and The converting is performed based on the refined motion vector.

35. The method of claim 34, wherein at least one of the DMVR or a variant of the DMVR is used to refine the shifted motion vector MV for the sbTMVP mode. The method of claim 35 , wherein the shifted MV is bi-directionally predicted.

37. The method of claim 35, wherein the shifted MV points to two reference pictures from both forward and backward.

38. The method of claim 35, wherein for the sbTMVP mode, the shifted MV satisfying a DMVR condition or a decoder-side motion vector difference (DMVD) condition is generated.

39. The method of claim 35, wherein the shifted MV is used to derive a motion vector from a corresponding prediction unit (PU) in a reference picture.

40. The method of claim 34, wherein at least one of the DMVR or a variant of the DMVR is used to refine control point motion vectors (CPMVs) of the affine merge mode.

41. The method of claim 34, wherein at least one of the DMVR or a variation of the DMVR is used to refine control point motion vectors (CPMVs) of the affine AMVP pattern.

42. The method of claim 41, wherein only PU-level DMVD or DMVR is used.

43. The method of claim 41, wherein at least one of the DMVR or a variation of the DMVR is applied to the CPMV satisfying a DMVR condition or a DMVD condition.

44. The method of claim 41, wherein at least one of the DMVR or a variant of the DMVR is applied to the CPMV pointing to reference pictures from both forward and backward.

45. The method of claim 41, wherein at least one of the DMVR or a variant of the DMVR is applied to the CPMV pair of a bi-predictive affine AMVP mode.

46. ​​The method of any one of claims 34 to 45, wherein the indication of whether and / or how to determine that the video unit is encoded with the motion pair from the same direction relative to the current picture is indicated at one of: Sequence level, Picture group level, Picture level, Stripe level, or Tile group level.

47. The method of any one of claims 34 to 45, wherein the indication of whether and / or how to determine that the video unit is encoded by the motion pair from the same direction relative to the current picture is indicated in one of: Sequence header, Picture header, Sequence Parameter Set (SPS), Video Parameter Set (VPS), Dependency Parameter Set (DPS), Decoding Capability Information (DCI), Picture Parameter Set (PPS), Adaptive Parameter Set (APS), Strip header, or Tile group header.

48. The method of any one of claims 34 to 45, wherein the indication of whether and / or how to determine that the video unit is encoded by the motion pair from the same direction relative to the current picture is comprised in one of: Prediction Block (PB), Transform Block (TB), Codec Block (CB), Prediction Unit (PU), Transformation Unit (TU), Codec Unit (CU), Virtual Pipeline Data Unit (VPDU), Codec Tree Unit (CTU), CTU line, Strips, Tile, sub-image, or An area that contains more than one sample or pixel.

49. The method according to any one of claims 34 to 45, further comprising: determining whether and / or how to determine that the video unit is encoded with the motion pair from the same direction relative to the current picture based on encoded information of the video unit, the encoded information comprising at least one of: Block size, Color format, Single-tree and / or dual-tree partitioning, Color components, Strip type, or Image type.

50. A method for video processing, comprising: deriving at least one LIC model parameter of a local illumination compensation (LIC) model for the video unit based on a nonlinear model during conversion between a video unit of a video and a bitstream of the video, wherein the coded video unit is a LIC coded block; updating the LIC model by adjusting the at least one LIC model parameter; and The converting is performed based on the updated LIC model.

51. The method of claim 50, wherein the at least one LIC model parameter is derived based on predC = a0*((predR*predR+b)>>s)+a1*predR+a2*c, Where predC represents the predicted sample value of the sample adjacent to the current block, predR represents the predicted sample value of the sample adjacent to the reference block, a0, a1 and a2 represent LIC parameters respectively, b and c are integers, and s represents a shift factor of an integer.

52. The method of claim 50, wherein the at least one LIC model parameter is derived based on predC = a0*((recR*recR+b)>>s)+a1*recR+a2*c, Wherein recC represents the reconstructed sample value of the sample adjacent to the current block, recR represents the reconstructed sample value of the sample adjacent to the reference block, a0, a1 and a2 represent LIC parameters respectively, b and c are integers, and s represents the shift factor which is an integer.

53. The method of claim 51 or 52, wherein the at least one LIC model parameter is derived based on an LDL decomposition method, or Wherein the at least one LIC model parameter is derived based on a coefficient solution method based on Gaussian elimination.

54. The method of claim 51 or 52, wherein M rows and N columns of neighboring samples are used to derive the at least one LIC model parameter, wherein M and N are integers.

55. The method of claim 54, wherein M is equal to an integer not greater than 6.

56. The method of claim 54, wherein N is equal to an integer not greater than 6.

57. The method of claim 54, wherein a value of at least M or N depends on a codec tree unit (CTU) boundary or a largest codec unit (LCU) boundary.

58. The method of claim 51 or 52, wherein the shift factor is derived based on an internal bit depth of the encoded samples.

59. The method of claim 51 or 52, wherein the values ​​of b and c are derived based on at least one of: the median value of the sample values ​​adjacent to the current block, the middle value of the sample values ​​adjacent to the current block, an average value of the sample values ​​adjacent to the current block, the median value of the sample values ​​adjacent to the reference block, the median value of the sample values ​​adjacent to the reference block, or The average value of the sample values ​​adjacent to the reference block.

60. The method of claim 59, wherein b equals c.

61. The method of any one of claims 50 to 60, wherein an indication of whether and / or how to derive the at least one LIC model parameter based on the nonlinear model is indicated at one of: Sequence level, Picture group level, Picture level, Stripe level, or Tile group level.

62. The method of any one of claims 50 to 60, wherein the indication of whether and / or how to derive the at least one LIC model parameter based on the nonlinear model is indicated in one of: Sequence header, Picture header, Sequence Parameter Set (SPS), Video Parameter Set (VPS), Dependency Parameter Set (DPS), Decoding Capability Information (DCI), Picture Parameter Set (PPS), Adaptive Parameter Set (APS), Strip header, or Tile group header.

63. The method of any one of claims 50 to 60, wherein the indication of whether and / or how to derive the at least one LIC model parameter based on the nonlinear model is comprised in one of: Prediction Block (PB), Transform Block (TB), Codec Block (CB), Prediction Unit (PU), Transformation Unit (TU), Codec Unit (CU), Virtual Pipeline Data Unit (VPDU), Codec Tree Unit (CTU), CTU line, Strips, Tile, sub-image, or An area that contains more than one sample or pixel.

64. The method according to any one of claims 50 to 60, further comprising: determining whether and / or how to derive the at least one LIC model parameter based on the nonlinear model based on the coded information of the video unit, the coded information comprising at least one of the following: Block size, Color format, Single-tree and / or dual-tree partitioning, Color components, Strip type, or Image type.

65. A method of video processing, comprising: For conversion between a video unit of a video and a bitstream of the video, applying a non-average weighting method to the video unit, wherein the video unit has an advanced motion vector prediction (AMVP) merge mode applied thereto; and The converting is performed based on the weighted video units.

66. The method of claim 65, wherein the non-average weighting method is applied to at least one of: a low-latency picture or a non-low-latency picture.

67. The method of claim 65, wherein a weight index from a predefined table of M possible weighting factors is determined, where M is an integer.

68. The method of claim 67, wherein M is one of the following: 3, 4, 5 or 6.

69. The method of claim 67, wherein the value of M depends on the coding information of the current picture.

70. The method of claim 65, wherein a weight index is indicated in the bitstream.

71. The method of claim 65, wherein the weight index is implicitly derived through a decoder-side derivation method.

72. The method of claim 71, wherein the weight index is determined based on a cost, wherein the cost is derived based on at least one of: decoder side motion vector refinement (DMVR), decoder side motion vector difference (DMVD), multi-pass decoder side motion vector difference (mpDMVD), prediction unit decoder side motion vector difference (puDMVD), a variant of DMVR, a variant of DMVD, a variant of mpDMVD, or a variant of puDMVd.

73. The method of claim 71, wherein the weight index is determined based on a template matching (TM) cost.

74. The method of claim 71, wherein the template derived for the TM cost is equal to M*N samples neighboring the current block and the reference block, where M and N are integers.

75. The method of claim 71, wherein a template derived for the TM cost is equal to M*N samples neighboring the first reference block and the second reference block.

76. The method of claim 71, wherein the TM cost or the DMVR cost is scaled for cost comparison.

77. The method of claim 76, wherein the TM cost or the DMVR cost derived by weighted averaging is scaled.

78. The method of claim 64, wherein unequal weighting of K entries is applied in a multiple hypothesis prediction (MHP) mode, where K is an integer.

79. The method of claim 78, wherein the K entries are non-evenly weighted and applied to the MHP AMVP mode to weight the two hypotheses, or The non-even weighting of the K entries is applied in MHP merging mode to weight the two hypotheses.

80. The method of claim 78, wherein the K entries are non-equally weighted to be at least one of: a low-latency picture or a non-low-latency picture.

81. The method of claim 78, wherein a weight index in a predefined table of M weighting factors is determined, where M is an integer.

82. The method of claim 81, wherein M is one of the following: 2, 3, 4, 5 or 6.

83. The method according to claim 81, wherein the value of M depends on the coding information of the current picture.

84. The method of claim 81, wherein the value of M depends on whether the video unit is encoded using MHP AMVP mode or MHP merge mode.

85. The method of claim 78, wherein a weight index is indicated in the bitstream.

86. The method of claim 78, wherein the weight index is implicitly derived through a decoder-side derivation method.

87. The method of claim 86, wherein the weight index is determined based on a cost, wherein the cost is derived based on at least one of: decoder side motion vector refinement (DMVR), decoder side motion vector difference (DMVD), multi-pass decoder side motion vector difference (mpDMVD), prediction unit decoder side motion vector difference (puDMVD), a variant of DMVR, a variant of DMVD, a variant of mpDMVD, or a variant of puDMVD.

88. The method of claim 86, wherein the weight index is determined based on a template matching (TM) cost.

89. The method of claim 86, wherein the template derived for the TM cost is equal to M*N samples neighboring the current block and the reference block, where M and N are integers.

90. The method of claim 86, wherein a template derived for the TM cost is equal to M*N samples neighboring the first reference block and the second reference block.

91. The method of claim 86, wherein the TM cost or the DMVR cost is scaled for cost comparison.

92. The method of claim 91, wherein the TM cost or the DMVR cost derived by weighted averaging is scaled.

93. The method of claim 65, wherein a decoder-derived non-average weighting method is applied to at least one of: Non-low latency images, Low latency images, MHP AMVP mode, A variant of the MHP AMVP model. MHP merge mode, A variant of the MHP merge mode, Merge mode with Motion Vector Difference (MMVD), Variants of MMVD, AMVP merge mode, or A variation of the AMVP merge mode.

94. The method of any one of claims 65 to 93, wherein an indication of whether and / or how to apply a non-average weighting method to the video units is indicated at one of: Sequence level, Picture group level, Picture level, Stripe level, or Tile group level.

95. The method of any one of claims 65 to 93, wherein an indication of whether and / or how to apply a non-average weighting method to the video units is indicated in one of: Sequence header, Picture header, Sequence Parameter Set (SPS), Video Parameter Set (VPS), Dependency Parameter Set (DPS), Decoding Capability Information (DCI), Picture Parameter Set (PPS), Adaptive Parameter Set (APS), Strip header, or Tile group header.

96. The method of any one of claims 65 to 93, wherein an indication of whether and / or how to apply a non-average weighting method to the video units is comprised in one of: Prediction Block (PB), Transform Block (TB), Codec Block (CB), Prediction Unit (PU), Transformation Unit (TU), Codec Unit (CU), Virtual Pipeline Data Unit (VPDU), Codec Tree Unit (CTU), CTU line, Strips, Tile, sub-image, or An area that contains more than one sample or pixel.

97. The method of any one of claims 65 to 93, further comprising: Determining whether and / or how to apply a non-average weighting method to the video unit based on coded information of the video unit, the coded information comprising at least one of the following: Block size, Color format, Single-tree and / or dual-tree partitioning, Color components, Strip type, or Image type.

98. The method of any one of claims 1 to 97, wherein the converting comprises encoding the video unit into the bitstream.

99. The method of any one of claims 1 to 97, wherein the converting comprises decoding the video unit from the bitstream.

100. An apparatus for video processing, comprising a processor and a non-volatile memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform a method according to any one of claims 1 to 99.

101. A non-transitory computer-readable storage medium storing instructions for causing a processor to execute a method according to any one of claims 1 to 99.

102. A non-transitory computer-readable recording medium storing a bit stream of a video generated by a method performed by an apparatus for video processing, wherein the method comprises: determining that a video unit of the video is encoded with a motion pair from a same direction relative to a current picture, wherein the video unit has an advanced motion vector prediction (AMVP) merge mode applied thereto, and the video unit belongs to a low-delay picture; and A bitstream for the video unit is generated based on the motion pairs.

103. A method for storing a bitstream of a video, comprising: determining that a video unit of the video is encoded with a motion pair from a same direction relative to a current picture, wherein the video unit has an advanced motion vector prediction (AMVP) merge mode applied thereto, and the video unit belongs to a low-latency picture; generating a bitstream for the video unit based on the motion pairs; as well as The bit stream is stored in a non-transitory computer-readable recording medium.

104. A non-transitory computer-readable recording medium storing a bit stream of a video generated by a method performed by an apparatus for video processing, wherein the method comprises: applying at least one of decoder-side motion vector refinement (DMVR) or a variant of DMVR to a video unit of the video to refine a motion vector of the video unit, wherein the video unit is applied with one of: an affine advanced motion vector prediction (AMVP) mode, a sub-block based temporal motion vector prediction (sbTMVP) mode, or an affine merge mode; as well as A bitstream for the video unit is generated based on the refined motion vectors.

105. A method for storing a bitstream of a video, comprising: applying at least one of decoder-side motion vector refinement (DMVR) or a variant of DMVR to a video unit of the video to refine a motion vector of the video unit, wherein the video unit is applied with one of: an affine advanced motion vector prediction (AMVP) mode, a sub-block based temporal motion vector prediction (sbTMVP) mode, or an affine merge mode; generating a bitstream for the video unit based on the refined motion vector; as well as The bit stream is stored in a non-transitory computer-readable recording medium.

106. A non-transitory computer-readable recording medium storing a bit stream of a video generated by a method performed by an apparatus for video processing, wherein the method comprises: deriving at least one local illumination compensation (LIC) model parameter of a LIC model for a video unit of the video based on a nonlinear model, wherein the coded video unit is a LIC coded block; updating the LIC model by adjusting the at least one LIC model parameter; and A bitstream for the video unit is generated based on the updated LIC model.

107. A method for storing a bitstream of a video, comprising: deriving at least one local illumination compensation (LIC) model parameter of a LIC model for a video unit of the video based on a nonlinear model, wherein the coded video unit is a LIC coded block; updating the LIC model by adjusting the at least one LIC model parameter; generating a bitstream of the video unit based on the updated LIC model; as well as The bit stream is stored in a non-transitory computer-readable recording medium.

108. A non-transitory computer-readable recording medium storing a bit stream of a video generated by a method performed by an apparatus for video processing, wherein the method comprises: applying a non-average weighting method to a video unit of the video, wherein the video unit has an advanced motion vector prediction (AMVP) merge mode applied thereto; as well as A bitstream for the video unit is generated based on the weighted video units.

109. A method for storing a bitstream of a video, comprising: applying a non-average weighting method to a video unit of the video, wherein the video unit has an advanced motion vector prediction (AMVP) merge mode applied thereto; generating a bitstream for the video unit based on the weighted video unit; as well as The bit stream is stored in a non-transitory computer-readable recording medium.

Images