Method and device for video processing and medium

Abstract

Embodiments of the present disclosure provide a solution for video processing. A method for video processing is presented. The method includes, for a transition between a video unit of a video and a bitstream of the video unit, deriving a prediction or reconstruction of the video unit based on one or more intra template matching prediction (intra TMP) candidates, one or more intra prediction modes (IPMs), one or more reordered intra TMP candidates, or one or more reordered IPMs, and wherein a fusion of an intra template match prediction (intra TMP) mode with a codec tool is applied to the video unit; and performing the conversion based on the prediction or reconstruction of the video unit.

Description

Technical Field

[0001] The embodiments of this disclosure generally relate to video processing techniques, and more specifically, to the derivation of local illumination compensation (LIC) parameters in video encoding and decoding. Background Technology

[0002] Today, digital video capabilities are being applied to all aspects of people's lives. Various video compression technologies have been proposed for video encoding / decoding, such as MPEG-2, MPEG-4, ITU-TH.263, ITU-TH.264 / MPEG-4 Part 10 Advanced Video Codec (AVC), ITU-TH.265 High Efficiency Video Codec (HEVC) standard, and Multi-Functional Video Codec (VVC) standard. However, the encoding and decoding efficiency of video encoding and decoding technologies is generally expected to be further improved. Summary of the Invention

[0003] Embodiments of this disclosure provide a solution for video processing.

[0004] In a first aspect, a method for video processing is proposed. This method includes: converting between video units and bitstreams of video units to obtain a set of local illumination compensation (LIC) parameters for previously encoded / decoded video units; applying the set of LIC parameters to the video units; and performing a conversion based on the set of LIC parameters. In this manner, encoding / decoding performance and efficiency are improved.

[0005] In a second aspect, an apparatus for video processing is provided. The apparatus includes a processor and a non-transitory memory having instructions thereon. When executed by the processor, the instructions cause the processor to perform the method according to the first aspect of this disclosure.

[0006] In a third aspect, a non-transitory computer-readable storage medium is proposed. This non-transitory computer-readable storage medium stores instructions that cause a processor to execute the method according to the first aspect of this disclosure.

[0007] In a fourth aspect, another non-transitory computer-readable recording medium is proposed. This non-transitory computer-readable recording medium stores a bitstream of video generated by a method performed by an apparatus for video processing. The method includes: obtaining a set of local illumination compensation (LIC) parameters for previously encoded / decoded video units stored therein; applying the set of LIC parameters to the video units of the video; and generating a bitstream based on the set of LIC parameters.

[0008] In a fifth aspect, a method for storing a bitstream of video is proposed. The method includes: obtaining a set of local illumination compensation (LIC) parameters for a previously encoded / decoded video unit; applying the set of LIC parameters to the video unit of the video; generating a bitstream based on the set of LIC parameters; and storing the bitstream in a non-transitory computer-readable recording medium.

[0009] This summary aims to present, in a simplified form, the selected concepts further described below in the detailed embodiments. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Attached Figure Description

[0010] The above and other objects, features, and advantages of exemplary embodiments of the present disclosure will become more apparent from the following detailed description with reference to the accompanying drawings. In the exemplary embodiments of the present disclosure, the same reference numerals generally refer to the same components.

[0011] Figure 1 A block diagram of an example video codec system according to some embodiments of the present disclosure is shown; Figure 2 A block diagram of a first example video encoder according to some embodiments of the present disclosure is shown; Figure 3 A block diagram of an example video decoder according to some embodiments of the present disclosure is shown; Figure 4 A schematic diagram showing the locations of the spatial merge candidates is provided. Figure 5 A schematic diagram is shown of the candidate pairs considered in the redundancy check for spatial merge candidates. Figure 6 A schematic diagram of motion vector scaling for temporal Merge candidates is shown; Figure 7 A schematic diagram of the candidate positions for time-domain Merge candidates C0 and C1 is shown; Figure 8 A schematic diagram of the MMVD search point is shown; Figure 9 A schematic diagram of the extended CU region used in BDOF is shown; Figure 10 An illustration of the symmetric MVD mode is shown; Figure 11 An affine motion model based on control points is shown; Figure 12 The affine MVF for each sub-block is shown; Figure 13 The location of the inherited affine motion prediction value is shown; Figure 14 This demonstrates the inheritance of control point motion vectors; Figure 15 The locations of candidate positions for the constructed affine Merge pattern are shown; Figure 16 A diagram illustrating the use of motion vectors for the proposed combination method is shown; Figure 17 The sub-block MV VSB and pixels are shown. (Red arrow); Figure 18A and Figure 18B The SbTMVP procedure in VVC is shown, where Figure 18A The spatial neighbor blocks used by ATMVP are shown, and Figure 18B The paper demonstrates how to derive the motion field of a sub-CU by applying motion shifts from spatial neighbors and scaling motion information from the corresponding co-located sub-CUs. Figure 19 The extended CU region used in BDOF is shown; Figure 20 This shows the refinement of motion vectors on the decoding side; Figure 21 The top neighbor block and left neighbor block used in the CIIP weight derivation are shown; Figure 22 An example of GPM partitioning grouped at the same angle is shown; Figure 23 The unidirectional prediction MV selection for geometric segmentation patterns is shown; Figure 24 An exemplary generation of the bending weight w0 using a geometric segmentation pattern is shown; Figure 25 The spatial neighbor block used to derive spatial merge candidates is shown; Figure 26 This shows that template matching is performed on the search area surrounding the initial MV; Figure 27 The diamond-shaped area in the search region is shown; Figure 28 The frequency response of the interpolation filter and the VVC interpolation filter at the half-pixel phase is shown; Figure 29 Reference sample points of the template are shown in the template and reference image; Figure 30 The template and reference sample points of the template are shown for a block with sub-block motion information using the current block; Figure 31 A flowchart of a method for video processing according to embodiments of the present disclosure is shown; and Figure 32 A block diagram of a computing device in which various embodiments of the present disclosure may be implemented is shown.

[0012] In all accompanying drawings, the same or similar reference numerals usually refer to the same or similar elements. Detailed Implementation

[0013] The principles of this disclosure will now be described with reference to some embodiments. It should be understood that these embodiments are described for illustrative purposes only and to help those skilled in the art understand and implement this disclosure, and do not imply any limitation on the scope of this disclosure. In addition to the methods described below, the disclosure described herein can be implemented in various other ways.

[0014] In the following description and claims, unless otherwise defined, all scientific and technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0015] The terms "an embodiment," "embodiment," "example embodiment," etc., used in this disclosure refer to embodiments that may include specific features, structures, or characteristics, but not every embodiment is required to include that specific feature, structure, or characteristic. Furthermore, these phrases do not necessarily refer to the same embodiment. Additionally, when a specific feature, structure, or characteristic is described in conjunction with an example embodiment, whether explicitly described or not, it is believed that such a feature, structure, or characteristic affecting its relation to other embodiments is within the knowledge of those skilled in the art.

[0016] It should be understood that although the terms “first” and “second”, etc., can be used to describe various elements, these elements should not be limited to these terms. These terms are used only to distinguish one element from another. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element, without departing from the scope of the exemplary embodiments. As used herein, the term “and / or” includes any and all combinations of one or more of the listed terms.

[0017] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments. As used herein, the singular forms “a,” “an,” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms “comprising,” “including,” “having,” “containing,” and / or “comprising” as used herein indicate the presence of the said features, elements, and / or components, but do not exclude the presence or addition of one or more other features, elements, components, and / or combinations thereof.

[0018] Example Environment Figure 1This is a block diagram illustrating an example video encoding / decoding system 100 from which the techniques of this disclosure may be utilized. As shown, the video encoding / decoding system 100 may include a source device 110 and a destination device 120. The source device 110 may also be referred to as a video encoding device, and the destination device 120 may also be referred to as a video decoding device. In operation, the source device 110 may be configured to generate encoded video data, and the destination device 120 may be configured to decode the encoded video data generated by the source device 110. The source device 110 may include a video source 112, a video encoder 114, and an input / output (I / O) interface 116.

[0019] Video source 112 may include sources such as video capture devices. Examples of video capture devices include, but are not limited to, interfaces for receiving video data from video content providers, computer graphics systems for generating video data, and / or combinations thereof.

[0020] Video data may include one or more images. Video encoder 114 encodes the video data from video source 112 to generate a bitstream. The bitstream may include a sequence of bits forming an encoded and decoded representation of the video data. The bitstream may include encoded images and associated data. An encoded image is a coded representation of an image. Associated data may include sequence parameter sets, image parameter sets, and other syntax structures. I / O interface 116 may include a modulator / demodulator and / or a transmitter. Encoded video data can be directly transmitted to destination device 120 via network 130A through I / O interface 116. Encoded video data may also be stored on storage medium / server 130B for access by destination device 120.

[0021] The destination device 120 may include an I / O interface 126, a video decoder 124, and a display device 122. The I / O interface 126 may include a receiver and / or a modem. The I / O interface 126 may obtain encoded video data from the source device 110 or the storage medium / server 130B. The video decoder 124 may decode the encoded video data. The display device 122 may display the decoded video data to a user. The display device 122 may be integrated with the destination device 120, or it may be external to the destination device 120, which is configured to interface with an external display device.

[0022] The video encoder 114 and the video decoder 124 can operate according to video compression standards, such as the High Efficiency Video Codec (HEVC) standard, the Multi-Functional Video Codec (VVC) standard, and other existing and / or further standards.

[0023] Figure 2This is a block diagram illustrating an example of a video encoder 200 according to some embodiments of the present disclosure. The video encoder 200 may be... Figure 1 An example of a video encoder 114 in system 100 is shown.

[0024] The video encoder 200 can be configured to implement any or all of the technologies disclosed herein. Figure 2 In the example, the video encoder 200 includes multiple functional components. The techniques described in this disclosure can be shared among the various components of the video encoder 200. In some examples, the processor can be configured to perform any or all of the techniques described in this disclosure.

[0025] In some embodiments, the video encoder 200 may include a segmentation unit 201, a prediction unit 202, a residual generation unit 207, a transform unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transform unit 211, a reconstruction unit 212, a buffer 213, and an entropy coding unit 214. The prediction unit 202 may include a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205, and an intra-frame prediction unit 206.

[0026] In other examples, the video encoder 200 may include more, fewer, or different functional components. In one example, the prediction unit 202 may include an intra-block copy (IBC) unit. The IBC unit can perform prediction in an IBC mode in which at least one reference picture is the picture in which the current video block is located.

[0027] Furthermore, although some components (such as motion estimation unit 204 and motion compensation unit 205) can be integrated, for interpretable purposes, these components are... Figure 2 The examples are shown separately.

[0028] The segmentation unit 201 can segment an image into one or more video blocks. The video encoder 200 and the video decoder 300 can support various video block sizes.

[0029] The mode selection unit 203 can, for example, select one of several coding modes (intra-frame coding or inter-frame coding) based on the error result, and provide the resulting intra-frame coded block or inter-frame coded block to the residual generation unit 207 to generate residual block data, and to the reconstruction unit 212 to reconstruct the coded block for use as a reference image. In some examples, the mode selection unit 203 can select an intra-frame / inter-frame joint prediction (CIIP) mode, in which prediction is based on inter-frame prediction signals and intra-frame prediction signals. In the case of inter-frame prediction, the mode selection unit 203 can also select a resolution for the block based on the motion vector (e.g., sub-pixel precision or integer pixel precision).

[0030] To perform inter-frame prediction on the current video block, motion estimation unit 204 can generate motion information for the current video block by comparing one or more reference frames from buffer 213 with the current video block. Motion compensation unit 205 can determine the predicted video block for the current video block based on the motion information and decoded samples of images from buffer 213 other than the image associated with the current video block.

[0031] The motion estimation unit 204 and the motion compensation unit 205 can perform different operations on the current video block, for example, depending on whether the current video block is in an I-strip, P-strip, or B-strip. As used herein, an "I-strip" can refer to a portion of an image composed of macroblocks, all of which are based on macroblocks within the same image. Furthermore, as used herein, in some aspects, "P-strip" and "B-strip" can refer to portions of an image composed of macroblocks that do not depend on macroblocks within the same image.

[0032] In some examples, motion estimation unit 204 can perform unidirectional prediction on the current video block, and can search reference images in list 0 or list 1 to find a reference video block for the current video block. Motion estimation unit 204 can then generate a reference index indicating the reference image containing the reference video block in list 0 or list 1, and a motion vector indicating the spatial displacement between the current video block and the reference video block. Motion estimation unit 204 can output the reference index, prediction direction indicator, and motion vector as motion information for the current video block. Motion compensation unit 205 can generate a predicted video block for the current video block based on the reference video block indicated by the motion information of the current video block.

[0033] Alternatively, in other examples, motion estimation unit 204 can perform bidirectional prediction on the current video block. Motion estimation unit 204 can search reference images in list 0 to find a reference video block for the current video block, and can also search reference images in list 1 to find another reference video block for the current video block. Motion estimation unit 204 can then generate multiple reference indices and multiple motion vectors, the multiple reference indices indicating multiple reference images containing reference video blocks in lists 0 and 1, and the multiple motion vectors indicating multiple spatial displacements between the multiple reference video blocks and the current video block. Motion estimation unit 204 can output the multiple reference indices and multiple motion vectors of the current video block as motion information for the current video block. Motion compensation unit 205 can generate a predicted video block for the current video block based on the multiple reference video blocks indicated by the motion information of the current video block.

[0034] In some examples, the motion estimation unit 204 can output a complete set of motion information for use in the decoder's decoding process. Alternatively, in some embodiments, the motion estimation unit 204 can reference the motion information of another video block to transmit the motion information of the current video block via a signal. For example, the motion estimation unit 204 can determine that the motion information of the current video block is sufficiently similar to the motion information of neighboring video blocks.

[0035] In one example, the motion estimation unit 204 may indicate a value to the video decoder 300 in the syntax structure associated with the current video block, which indicates that the current video block has the same motion information as another video block.

[0036] In another example, motion estimation unit 204 can identify another video block and motion vector difference (MVD) in the syntax structure associated with the current video block. The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the indicated video block. Video decoder 300 can use the motion vector of the indicated video block and the motion vector difference to determine the motion vector of the current video block.

[0037] As discussed above, the video encoder 200 can transmit motion vectors via signals in a predictive manner. Two examples of predictive signaling techniques that can be implemented by the video encoder 200 include Advanced Motion Vector Prediction (AMVP) and Merge Pattern Signaling.

[0038] Intra-prediction unit 206 can perform intra-prediction on the current video block. When intra-prediction unit 206 performs intra-prediction on the current video block, it can generate prediction data for the current video block based on decoded samples from other video blocks in the same frame. The prediction data for the current video block can include the predicted video block and various syntax elements.

[0039] The residual generation unit 207 can generate residual data for the current video block by subtracting (e.g., indicated by a minus sign) multiple predicted video blocks from the current video block. The residual data for the current video block may include residual video blocks corresponding to different sample components of the samples in the current video block.

[0040] In other examples, such as in skip mode, there may be no residual data for the current video block, and the residual generation unit 207 may not perform subtraction operations.

[0041] The transform processing unit 208 can generate one or more transform coefficient video blocks for the current video block by applying one or more transforms to the residual video blocks associated with the current video block.

[0042] After the transform processing unit 208 generates a transform coefficient video block associated with the current video block, the quantization unit 209 can quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values ​​associated with the current video block.

[0043] The inverse quantization unit 210 and the inverse transform unit 211 can apply inverse quantization and inverse transform to the transform coefficient video block respectively to reconstruct the residual video block from the transform coefficient video block. The reconstruction unit 212 can add the reconstructed residual video block to the corresponding samples of one or more predicted video blocks generated by the prediction unit 202 to generate a reconstructed video block associated with the current video block for storage in the buffer 213.

[0044] After the video block is reconstructed by reconstruction unit 212, a loop filtering operation can be performed to reduce video block artifacts in the video block.

[0045] Entropy encoding unit 214 can receive data from other functional components of video encoder 200. When entropy encoding unit 214 receives data, it can perform one or more entropy encoding operations to generate entropy-encoded data and output a bitstream including the entropy-encoded data.

[0046] Figure 3 This is a block diagram illustrating an example of a video decoder 300 according to some embodiments of the present disclosure. The video decoder 300 may be... Figure 1 An example of video decoder 124 in system 100 is shown.

[0047] The video decoder 300 can be configured to perform any or all of the technologies disclosed herein. Figure 3 In the example, the video decoder 300 includes multiple functional components. The techniques described in this disclosure can be shared among the various components of the video decoder 300. In some examples, the processor can be configured to perform any or all of the techniques described in this disclosure.

[0048] exist Figure 3 In the example, the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra-frame prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, a reconstruction unit 306, and a buffer 307. In some examples, the video decoder 300 can perform a decoding process that is generally contrasted with the encoding process described with respect to the video encoder 200.

[0049] Entropy decoding unit 301 can retrieve the encoded bitstream. The encoded bitstream may include entropy-encoded video data (e.g., encoded blocks of video data). Entropy decoding unit 301 can decode the entropy-encoded video data, and motion compensation unit 302 can determine motion information, including motion vectors, motion vector precision, reference picture list indices, and other motion information, based on the entropy-decoded video data. Motion compensation unit 302 can determine such information, for example, by performing AMVP and Merge mode. AMVP is used, which includes deriving several most likely candidates based on data from neighboring PBs and reference pictures. Motion information typically includes horizontal motion vector displacement values ​​and vertical motion vector displacement values, one or two reference picture indices, and, in the case of a prediction region in a B-strip, an identifier of which reference picture list is associated with each index. As used herein, in some aspects, "Merge mode" may refer to deriving motion information from spatially or temporally neighboring blocks.

[0050] The motion compensation unit 302 can generate motion compensation blocks and can perform interpolation based on an interpolation filter. Identifiers for interpolation filters used at sub-pixel precision can be included in the syntax elements.

[0051] The motion compensation unit 302 can use the interpolation filter used by the video encoder 200 during the encoding of the video block to calculate the interpolated values ​​for sub-integer pixels of the reference block. The motion compensation unit 302 can determine the interpolation filter used by the video encoder 200 based on the received syntax information, and the motion compensation unit 302 can use the interpolation filter to generate the prediction block.

[0052] Motion compensation unit 302 may use at least some of the syntax information to determine the size of the blocks used to encode (multiple) frames and / or (multiple) stripes of the encoded video sequence, segmentation information describing how each macroblock of the image of the encoded video sequence is segmented, a pattern indicating how each segment is encoded, one or more reference frames (and a list of reference frames) for each inter-frame coded block, and other information for decoding the encoded video sequence. As used herein, in some aspects, a “strip” can refer to a data structure that can be decoded independently of other stripes of the same image in terms of entropy encoding / decoding, signal prediction, and residual signal reconstruction. A strip can be an entire image or a region of an image.

[0053] Intra-prediction unit 303 can use, for example, an intra-prediction mode received in the bitstream to form prediction blocks from spatially adjacent blocks. Dequantization unit 304 dequantizes (i.e., dequantizes) the quantized video block coefficients provided in the bitstream and decoded by entropy decoding unit 301. Inverse transform unit 305 applies an inverse transform.

[0054] The reconstruction unit 306 can obtain the decoded block, for example, by adding the residual block to the corresponding predicted block generated by the motion compensation unit 302 or the intra-frame prediction unit 303. If necessary, a deblocking filter can also be used to filter the decoded block to remove block artifacts. The decoded video block is then stored in a buffer 307, which provides a reference block for subsequent motion compensation / intra-frame prediction and also generates decoded video for presentation on a display device.

[0055] Some exemplary embodiments of this disclosure will be described in detail below. It should be understood that section headings are used in this document for ease of understanding and not to limit the embodiments disclosed in a section to that section only. Furthermore, while some embodiments are described with reference to multi-function video codecs or other specific video codecs, the disclosed techniques are also applicable to other video codec techniques. Additionally, although some embodiments describe video encoding steps in detail, it will be understood that the corresponding decoding steps for decoding will be implemented by the decoder. Furthermore, the term "video processing" includes video encoding or compression, video decoding or decompression, and video transcoding, in which video pixels are represented from one compression format to another or at different compression bitrates.

[0056] 1. Overview This disclosure relates to video codec technology. Specifically, this disclosure pertains to local illumination compensation (also known as LIC) in video codecs. This disclosure can be applied to existing video codec standards such as HEVC, VVC, etc. This disclosure can also be applied to future video codec standards or video codecs.

[0057] 2. Background Video codec standards have primarily evolved through the development of well-known ITU-T and ISO / IEC standards. ITU-T developed the H.261 and H.263 standards, while ISO / IEC developed MPEG-1 and MPEG-4 Vision. The two organizations jointly developed the H.262 / MPEG-2 video standard, the H.264 / MPEG-4 Advanced Video Codec (AVC) standard, and the H.265 / HEVC standard. Starting with H.262, video codec standards are based on a hybrid video codec architecture, utilizing temporal prediction plus transform coding. To explore future video codec technologies beyond HEVC, the Joint Video Exploration Team (JVET) was established in 2015 by VCEG and MPEG. JVET meetings are held quarterly, and the new video codec standard was officially named Multifunctional Video Codec (VVC) at the April 2018 JVET meeting, where the first version of the VVC Test Model (VTM) was also released. The VVC working draft and the VTM test model are updated after each meeting. The VVC project achieved technical completion (FDIS) at a meeting in July 2020.

[0058] 2.1. Existing Inter-Frame Predictive Encoding / Decoding Tools For each inter-frame prediction CU, motion parameters consist of a motion vector, a reference picture index, a reference picture list usage index, and additional information required for generating new encoding / decoding features for the VVC, which will be used for inter-frame prediction sample generation. Motion parameters can be transmitted via signaling in an explicit or implicit manner. When a CU is encoded / decoded in skip mode, the CU is associated with a PU and does not have significant residual coefficients, encoded / decoded motion vector differences, or reference picture indices. A Merge mode is defined, whereby motion parameters for the current CU are obtained from neighboring CUs, including spatial and temporal candidates, as well as additional scheduling introduced in the VVC. The Merge mode can be applied to any inter-frame prediction CU, not just skip mode. An alternative to the Merge mode is explicit transmission of motion parameters, where the motion vector for each reference picture list, the corresponding reference picture index, the reference picture list usage flag, and other required information are explicitly transmitted via signaling for each CU.

[0059] In addition to the inter-frame coding and decoding features in HEVC, VVC also includes many new and improved inter-frame prediction coding and decoding tools, as listed below: – Extended Merge forecast; – Merge pattern with MVD (MMVD); – Symmetric MVD (SMVD) signaling; – Affine motion compensation prediction; – Sub-block-based temporal motion vector prediction (SbTMVP); – Adaptive Motion Vector Resolution (AMVR); – Motion field storage: 1 / 16 luminance sample MV storage and 8x8 motion field compression; – Bidirectional prediction (BCW) with CU-level weights; – Bidirectional optical flow (BDOF); – Decoder-side motion vector refinement (DMVR); – Geometric Partitioning (GPM); – Inter-frame and intra-frame joint prediction (CIIP).

[0060] The following text provides details of the inter-frame prediction methods specified in VVC.

[0061] 2.1.1. Extended Merge Forecast In VVC, the Merge candidate list is constructed by including the following five types of candidates in sequence: 1) Airspace MVP from the adjacent CU; 2) Temporal MVP from the same CU; 3) History-based MVP from FIFO table; 4) Paired average MVP; 5) Zero MV.

[0062] The size of the Merge list is transmitted via signaling in the sequence parameter set header, and the maximum allowed size of the Merge list is 6. For each CU encoded in Merge mode, the index of the best Merge candidate is encoded using rounding univariate binarization (TU). The first bit of the Merge index is encoded using the context, and bypass encoding is used for the remaining bits.

[0063] This section provides the derivation process for various merge candidates. Similar to HEVC, VVC also supports parallel derivation of the merge candidate list for all CUs within a region of a specific size.

[0064] 2.1.1.1. Derivation of Airspace Candidates The derivation of spatial merge candidates in VVC is the same as that in HEVC, except that the positions of the first two merge candidates are swapped. Figure 4At most four merged candidates are selected from the candidates at the indicated positions. The derivation order is B0, A0, B1, A1, and B2. Position B2 is considered only if one or more CUs at positions B0, A0, B1, and A1 are unavailable (e.g., because it belongs to another stripe or slice) or if it is intra-frame encoded / decoded. After adding the candidate at position A1, a redundancy check is performed on the addition of the remaining candidates. This redundancy check ensures that candidates with the same motion information are excluded from the list, thereby improving encoding / decoding efficiency. To reduce computational complexity, not all possible candidate pairs are considered in the mentioned redundancy check. Instead, only... Figure 5 The system uses arrow links to select pairs, and only adds candidates to the list if the corresponding candidates used for redundancy checks do not have the same motion information.

[0065] 2.1.1.2. Time-domain candidate derivation In this step, only one candidate is added to the list. Specifically, in the derivation of this temporal merge candidate, the scaled motion vector is derived based on the co-located CU belonging to the co-located reference image. The list of reference images to be used for the derivation of the co-located CU is explicitly transmitted via signal transmission in the strip header. Figure 6 As shown by the dashed lines, the scaled motion vectors of the temporal merge candidate are obtained by scaling the motion vectors of the co-located CU using the POC distances tb and td, where tb is defined as the POC difference between the current image and the reference image, and td is defined as the POC difference between the co-located reference image and the co-located image. The reference image index of the temporal merge candidate is set to 0.

[0066] like Figure 7 As shown, the position for the temporal candidate is selected between candidate C0 and C1. If the CU at position C0 is unavailable, intra-frame encoded or decoded, or outside the current line of the CTU, position C1 is used. Otherwise, position C0 is used for the derivation of the temporal merge candidate.

[0067] 2.1.1.3. Historical Merge Candidate Derivation Historically based MVP (HMVP) merge candidates are added to the merge list, following the spatial MVP and TMVP. In this method, motion information from previously encoded / decoded blocks is stored in a table and used as the MVP for the current CU. The table with multiple HMVP candidates is maintained during the encoding / decoding process. The table is reset (cleared) when a new CTU row is encountered. Whenever a non-sub-block inter-frame encoding / decoding CU is present, the associated motion information is added to the last entry of the table as a new HMVP candidate.

[0068] HMVP Table Size SSetting it to 6 indicates that a maximum of 6 history-based MVP (HMVP) candidates can be added to the table. When a new motion candidate is inserted into the table, a constrained First-In-First-Out (FIFO) rule is used, where a redundancy check is first applied to find if a duplicate HMVP already exists in the table. If found, the duplicate HMVP is removed from the table, and all subsequent HMVP candidates are moved forward.

[0069] HMVP candidates can be used in the Merge candidate list construction process. The latest HMVP candidates in the table are checked sequentially and inserted into the candidate list after the TMVP candidates. Redundancy checks are applied to HMVP candidates for spatial or temporal Merge candidates.

[0070] To reduce the number of redundant check operations, the following simplifications are introduced: 1. The number of HMPV candidates used in the Merge list generation is set to ( N <= 4) ? M : (8 – N ), where N indicates the number of existing candidates in the Merge list, and M indicates the number of available HMVP candidates in the table.

[0071] 2. Once the total number of available Merge candidates reaches the maximum allowed Merge candidates minus 1, the Merge candidate list building process starting from HMVP is terminated.

[0072] 2.1.1.4. Derivation of Pairwise Average Merge Candidates Pairwise averaging candidates are generated by averaging predefined candidate pairs from an existing Merge candidate list. These predefined pairs are defined as {(0, 1), (0, 2), (1, 2), (0, 3), (1, 3), (2, 3)}, where the numbers represent the Merge indices in the Merge candidate list. The averaged motion vector is calculated separately for each reference list. If two motion vectors are available in a list, they are averaged even if they point to different reference images; if only one motion vector is available, that vector is used directly; if no motion vector is available, the list remains invalid.

[0073] If the Merge list is not full after adding pairwise average Merge candidates, a zero MVP will be inserted at the end until the maximum number of Merge candidates is reached.

[0074] 2.1.1.5. Merge estimation region The Merge Estimation Region (MER) allows for the independent derivation of Merge candidate lists for CUs within the same MER. Candidate blocks located within the same MER as the current CU are not included in the generation of the Merge candidate list for the current CU. Furthermore, the update process for the historical motion vector prediction candidate list is only updated when (xCb + cbWidth) >> Log2ParMrgLevel is greater than xCb >> Log2ParMrgLevel and (yCb + cbHeight) >> Log2ParMrgLevel is greater than (yCb >> Log2ParMrgLevel), where (xCb, yCb) is the top-left brightness sample position of the current CU in the image, and (cbWidth, cbHeight) is the CU size. The MER size is selected on the encoder side and is transmitted via signaling as log2_parallel_merge_level_minus2 in the sequence parameter set.

[0075] 2.1.2. Merge Schema with MVD (MMVD) In addition to the Merge mode (where implicitly derived motion information is directly used for generating prediction samples for the current CU), the Merge mode with motion vector difference (MMVD) is also introduced into VVC. The MMVD flag is transmitted immediately after the skip flag and the Merge flag are sent via signaling to indicate whether the MMVD mode is used for the CU.

[0076] In MMVD, after selecting a Merge candidate, the MVD information transmitted via signals further refines the Merge candidate. This further information includes a Merge candidate flag, an index specifying the motion amplitude, and an index indicating the motion direction. In MMVD mode, one of the top two candidates in the Merge list is selected as the MV basis. The Merge candidate flag is transmitted via signals to specify which one to use.

[0077] The distance index specifies motion amplitude information and indicates a predefined offset from the starting point. For example... Figure 8 As shown, the offset is added to the horizontal or vertical component of the starting MV. The relationship between the distance index and the predefined offset is specified in Table 1.

[0078] Table 1 – Relationship between Distance Index and Predefined Offset

[0079] The direction index indicates the direction of the MVD relative to the starting point. The direction index can represent the four directions shown in Table 2. It is important to note that the meaning of the MVD sign may differ depending on the information of the starting MV. When the starting MV is a unidirectional or bidirectional prediction MV (where both lists point to the same side of the current image, i.e., both reference POCs are greater than or less than the current image's POC), the signs in Table 2 specify the signs added to the MV offset of the starting MV. When the starting MV is a bidirectional prediction MV (where the two MVs point to different sides of the current image, i.e., one reference POC is greater than the current image's POC, and the other reference POC is less than the current image's POC), the signs in Table 2 specify the signs added to the MV offsets of list 0 MV components of the starting MV, and have the opposite values ​​for the signs of list 1 MV components.

[0080] Table 2 – Signs of MV Offsets Defined by Direction Index

[0081] 2.1.2.1. Bidirectional prediction (BCW) with CU-level weights In HEVC, the bidirectional prediction signal is generated by averaging two prediction signals obtained from two different reference images and / or using two different motion vectors. In VVC, the bidirectional prediction mode is extended beyond simple averaging to allow for a weighted average of the two prediction signals.

[0082]

[0083] Five weights are allowed in weighted average two-way forecasting. For each bidirectional prediction CU, the weights w are determined in one of two ways: 1) for non-merge CUs, the weight index is transmitted via signal after the motion vector difference; 2) for merge CUs, the weight index is inferred from neighboring blocks based on the merge candidate index. BCW is applied only to CUs with 256 or more luma samples (i.e., CU width multiplied by CU height is greater than or equal to 256). For low-latency images, all 5 weights are used. For non-low-latency images, only 3 weights are used (w∈{3,4,5}).

[0084] – At the encoder, fast search algorithms are applied to find weight indices without significantly increasing encoder complexity. These algorithms are summarized below. When combined with AMVR, if the current image is a low-latency image, unequal weights are conditionally checked only for 1-pixel and 4-pixel motion vector precision.

[0085] – When combined with an affine pattern, an affine ME will be performed for unequal weights if and only if the affine pattern is selected as the current best pattern.

[0086] – When the two reference images in bidirectional prediction are the same, only conditional checks are performed on unequal weights.

[0087] – When certain conditions are met, unequal weights are not searched, depending on the POC distance between the current image and its reference image, the encoding / decoding QP, and the temporal level.

[0088] The BCW weight index is encoded using a context-coded bit and a subsequent bypass-coded bit. The first context-coded bit indicates whether equal weights are used; if unequal weights are used, the bypass-coded bit is used to signal additional bits to indicate which unequal weights are used.

[0089] Weighted Prediction (WP) is a codec tool supported by the H.264 / AVC and HEVC standards for efficiently encoding and decoding video content with shading. Support for WP has also been added to the VVC standard. WP allows weighting parameters (weights and offsets) to be transmitted via signaling for each reference picture in each of the reference picture lists L0 and L1. Then, during motion compensation, the corresponding weights and offsets of the reference picture(s) are applied. WP and BCW are designed for different types of video content. To avoid interaction between WP and BCW (which would complicate the VVC decoder design), if the CU uses WP, the BCW weight index is not transmitted via signaling, and w is presumed to be 4 (i.e., equal weights are applied). For Merge CUs, the weight index is presumed from neighboring blocks based on the Merge candidate index. This can be applied to normal Merge patterns and inherited affine Merge patterns. For constructed affine Merge patterns, affine motion information is constructed based on the motion information of up to 3 blocks. The BCW index of the CU using the constructed affine Merge pattern is simply set to be equal to the BCW index of the first control point MV.

[0090] In VVC, CIIP and BCW cannot be jointly applied to a CU. When a CU is encoded or decoded in CIIP mode, the BCW index of the current CU is set to 2, for example, with equal weights.

[0091] 2.1.2.2. Bidirectional Optical Flow (BDOF) The Bidirectional Optical Flow (BDOF) tool is included in VVC. BDOF (formerly known as BIO) was included in JEM. Compared to the JEM version, the BDOF in VVC is a simpler version, requiring far fewer computations, especially in terms of the number of multiplications and multiplier size.

[0092] BDOF is used to refine the bidirectional prediction signal of the CU at the 4x4 sub-block level. BDOF is applied to the CU if all of the following conditions are met: - CU is encoded and decoded using a "true" bidirectional prediction mode, that is, one of the two reference images is displayed before the current image in the order of display, and the other is displayed after the current image in the order of display.

[0093] - The distances (i.e., the difference in point of view) from the two reference images to the current image are the same.

[0094] - Both reference images are short-term reference images.

[0095] - CU is not encoded or decoded using affine mode or ATMVP Merge mode.

[0096] - The CU has more than 64 luminance samples.

[0097] - Both the CU height and CU width are greater than or equal to 8 luminance samples.

[0098] - The BCW weight index indicates equal weights.

[0099] - WP is not enabled for the current CU.

[0100] - CIIP mode is not used in the current CU.

[0101] BDOF is applied only to the luminance component. As the name suggests, the BDOF mode is based on the concept of optical flow, which assumes that the motion of the object is smooth. For each 4x4 sub-block, motion refinement is calculated by minimizing the difference between the L0 and L1 predicted samples. Motion refinement is then used to adjust the bidirectional prediction sample values ​​in the 4x4 sub-blocks. The following steps are applied during the BDOF process.

[0102] First, the horizontal and vertical gradients of the two predicted signals. and , It is calculated by directly calculating the difference between two neighboring sample points, i.e.

[0103] in It is a list The coordinates of the predicted signal in ( i, j The sample value at () is calculated as shift1 = max(6, bitDepth-6) based on the luminance bit depth bitDepth.

[0104] Then, the autocorrelation and cross-correlation of the gradients. Calculated as

[0105] in

[0106] Where Ω is the 6x6 window surrounding the 4x4 sub-block, and n a and n b The values ​​are set to min(1, bitDepth – 11) and min(4, bitDepth – 8) respectively.

[0107] Then, use the following formula to refine the motion. It is derived using cross-correlation and autocorrelation terms:

[0108] in . It is a floor function, and .

[0109] Based on motion refinement and gradients, the following adjustments are calculated for each sample point in the 4x4 sub-block:

[0110] Finally, the BDOF samples of CU are calculated by adjusting the bidirectional prediction samples as follows:

[0111] These values ​​are chosen so that the multiplier in the BDOF process does not exceed 15 bits, and the maximum bit width of the intermediate parameters in the BDOF process is kept within 32 bits.

[0112] To derive the gradient values, a list of elements outside the current CU boundary needs to be generated. k ( k Some predicted samples in (=0,1) .like Figure 9As shown, BDOF in VVC uses an extended row / column around the CU boundary. To control the computational complexity of generating prediction samples outside the boundary, prediction samples in the extended region (white area) are generated by directly obtaining reference samples at nearby integer positions (using the floor() operation on the coordinates) without interpolation, and a normal 8-tap motion-compensated interpolation filter is used to generate prediction samples inside the CU (gray area). These extended sample values ​​are only used in gradient calculations. For the remaining steps in the BDOF process, if any samples and gradient values ​​outside the CU boundary are needed, they are filled from their nearest neighbors (i.e., repeated).

[0113] When the width and / or height of a CU (Cubic Unit) is greater than 16 luminance samples, it will be divided into sub-blocks with a width and / or height equal to 16 luminance samples, and the sub-block boundaries will be considered CU boundaries in the BDOF process. The maximum cell size for the BDOF process is limited to 16x16. The BDOF process can be skipped for each sub-block. The BDOF process is not applied to the sub-block when the SAD (Self-Adjustment Aspect) between the initial L0 and L1 predicted samples is less than a threshold. The threshold is set to equal to... , where W indicates the sub-block width and H indicates the sub-block height. To avoid the additional complexity of SAD calculation, the SAD between the initial L0 prediction samples and L1 prediction samples calculated during the DVMR process is reused here.

[0114] Bidirectional optical flow is disabled if BCW is enabled for the current block, meaning the BCW weight index indicates unequal weights. Similarly, BDOF is disabled if WP is enabled for the current block, meaning luma_weight_lx_flag is 1 for either of the two reference images. BDOF is also disabled when the CU is encoded and decoded in symmetric MVD or CIIP mode.

[0115] 2.1.2.3. Symmetric MVD Encoding / Decoding (SMVD) In VVC, in addition to the normal one-way and two-way predictive MVD signaling, a symmetric MVD mode for two-way predictive MVD signaling is also applied. In the symmetric MVD mode, the reference image indices of both List 0 and List 1, as well as the motion information of the MVD in List 1, are not transmitted through signals but are derived.

[0116] The decoding process of the symmetric MVD mode is as follows: 1) At the strip level, the variables BiDirPredFlag, RefIdxSymL0, and RefIdxSymL1 are derived as follows: – If mvd_l1_zero_flag is 1, then BiDirPredFlag is set to 0.

[0117] Otherwise, if the most recent reference image in list 0 and the most recent reference image in list 1 form a pair of forward and backward reference images or a pair of backward and forward reference images, then BiDirPredFlag is set to 1, and both the reference images in list 0 and list 1 are short-term reference images. Otherwise, BiDirPredFlag is set to 0.

[0118] 2) At the CU level, if the CU is bidirectional predictive codec and BiDirPredFlag is equal to 1, then the symmetric mode flag indicating whether to use symmetric mode is explicitly indicated by signal transmission.

[0119] When the symmetry mode flag is true, only mvp_l0_flag, mvp_l1_flag, and MVD0 are explicitly transmitted via signaling. The reference indices of list 0 and list 1 are each set to equal a pair of reference images. MVD1 is set to equal to (-MVD0). The final motion vector is shown in the following formula.

[0120]

[0121] Figure 10 This is a diagram of the symmetric MVD model. In the encoder, symmetric MVD motion estimation begins with an initial MV evaluation. A set of initial MV candidates includes MVs obtained from a one-way prediction search, MVs obtained from a two-way prediction search, and MVs from an AMVP list. The one with the lowest rate-distortion cost is selected as the initial MV for the symmetric MVD motion search.

[0122] 2.1.3. Affine Motion Compensation Prediction In HEVC, only the translational motion model is applied to motion compensation prediction (MCP). In the real world, there are many types of motion, such as zooming in / out, rotation, perspective motion, and other irregular motions. In VVC, block-based affine transformation motion compensation prediction is applied. Figure 11 As shown, the affine motion field of a block is described by motion information from two control points (4 parameters) or three control point motion vectors (6 parameters).

[0123] For a 4-parameter affine motion model, the location of the sample point in the block ( x, y The motion vector at point () is derived as:

[0124] For a 6-parameter affine motion model, the location of the sample points in the block ( x, y The motion vector at point () is derived as:

[0125] in( mv 0x , mv 0y ) is the motion vector of the upper left control point, ( mv 1x , mv 1y ) is the motion vector of the upper right control point, and ( mv 2x , mv 2y ) is the motion vector of the lower left control point.

[0126] To simplify motion compensation prediction, block-based affine transformation prediction is applied. To derive the motion vector for each 4x4 luma sub-block, the motion vector of the center sample point of each sub-block is calculated according to the above equation (e.g., ...). Figure 12 (as shown), and rounded to 1 / 16 fractional precision. Then a motion-compensated interpolation filter is applied to generate a prediction for each sub-block with a derived motion vector. The sub-block size for the chroma component is also set to 4x4. The MV of the 4x4 chroma sub-block is calculated as the average of the MV of the top-left luminance sub-block and the bottom-right luminance sub-block in the corresponding 8x8 luminance region.

[0127] Similar to translational motion inter-frame prediction, there are two affine motion inter-frame prediction modes: affine Merge mode and affine AMVP mode.

[0128] 2.1.3.1. Affine Merge Prediction The AF_MERGE mode can be applied to CUs with a width and height greater than or equal to 8. In this mode, the CPVM of the current CU is generated based on the motion information of spatially neighboring CUs. There can be up to five CPVM candidates, and the one to be used for the current CU is indicated by a signal transmission index. The following three types of CPVM candidates are used to form the affine merge candidate list: – Inherited affine Merge candidates inferred from the CPMV of neighboring CUs – Affine Merge candidate CPMVP constructed using translational MV derivation of neighboring CUs – Zero MV.

[0129] In VVC, there are at most two inherited affine candidates, which are derived from the affine motion model of neighboring blocks: one from the left neighboring CU and one from the upper neighboring CU. Candidate blocks are as follows: Figure 13As shown. For the predicted value on the left, the scanning order is A0->A1, and for the predicted value above, the scanning order is B0->B1->B2. Only the first inherited candidate from each side is selected. No deduplication check is performed between candidates from two inheritances. When a neighboring affine CU is identified, its control point motion vector is used to derive the CPMVP candidate in the affine Merge list of the current CU. As shown, if the neighboring lower left block A is encoded and decoded in affine mode, the motion vectors of the upper left, upper right, and lower left corners of the CU containing block A are obtained. When block A is encoded and decoded using a 4-parameter affine model, the two CPMVs of the current CU are calculated based on v2 and v3. When block A is encoded and decoded using a 6-parameter affine model, the three CPMVs of the current CU are calculated based on... Calculated. Figure 14 The inheritance of control point motion vectors is shown.

[0130] The constructed affine candidate refers to the candidate built by combining the translational motion information of the neighbors of each control point. The motion information of the control points is derived from... Figure 15 The spatial and temporal nearest neighbors shown are derived. CPMV k (k=1, 2, 3, 4) represents the k-th control point. For CPMV1, check the B2->B3->A2 block and use the MV of the first available block. For CPMV2, check the B1->B0 block, and for CPMV3, check the A1->A0 block. If available, the TMVP is used as CPMV4.

[0131] After obtaining the motion signatures (MVs) of the four control points, the affine Merge candidate is constructed based on this motion information. The following combinations of control point MVs are used for sequential construction: {CPMV1, CPMV2, CPMV3}, {CPMV1, CPMV2, CPMV4}, {CPMV1, CPMV3, CPMV4}, {CPMV2, CPMV3, CPMV4}, { CPMV1, CPMV2}, { CPMV1, CPMV3}.

[0132] Combining three CPMVs constructs a 6-parameter affine merge candidate, and combining two CPMVs constructs a 4-parameter affine merge candidate. To avoid motion scaling, combinations of control point MVs are discarded if the reference indices of the control points are different.

[0133] After checking the inherited affine Merge candidates and the constructed affine Merge candidates, if the list is still not full, a zero MV is inserted at the end of the list.

[0134] 2.1.3.2. Affine AMVP Prediction The affine AMVP mode can be applied to CUs with a width and height both greater than or equal to 16. An affine flag at the CU level is signaled in the bitstream to indicate whether the affine AMVP mode is used, and another flag is signaled to indicate whether it is a 4-parameter affine or a 6-parameter affine. In this mode, the difference between the current CU's CPVM and its predicted CPMVP is signaled in the bitstream. The affine AMVP candidate list is of size 2 and is generated by sequentially using the following four types of CPVM candidates: – Inherited affine AMVP candidates inferred from the CPMV of neighboring CUs; – A constructive affine AMVP candidate CPMVP derived using translational MV of neighboring CUs; – Translation MV from the neighboring CU; – Zero MV.

[0135] The checking order for inherited affine AMVP candidates is the same as that for inherited affine Merge candidates. The only difference is that, for AVMP candidates, only affine CUs with the same reference picture as those in the current block are considered. No deduplication is applied when inserting inherited affine motion predictions into the candidate list.

[0136] The constructed AMVP candidate is from Figure 15 The specified spatial nearest neighbor derivation is shown. The same checking order as in the affine Merge candidate construction is used. Additionally, the reference picture index of neighboring blocks is checked. The block that is the first to be inter-coded in the checking order and has the same reference picture as in the current CU is used. There is only one. Encoded in the current CU using a 4-parameter affine mode, and... mv 0 and mv If all three CPMVs are available, they are added as candidates to the affine AMVP list. If the current CU is encoded and decoded using a 6-parameter affine mode, and all three CPMVs are available, they are added as candidates to the affine AMVP list. Otherwise, the constructed AMVP candidates are set to unavailable.

[0137] If, after inserting valid inherited affine AMVP candidates and constructed AMVP candidates, the affine AMVP list still has fewer than 2 candidates, then when available, mv 0、 mv 1 and mv 2 will be added sequentially as translation MVs to predict all control point MVs of the current CU. Finally, if the affine AMVP list is still not full, zero MVs are used to fill the affine AMVP list.

[0138] 2.1.3.3. Affine Motion Information Storage In VVC, the CPMV of an affine CU is stored in a separate cache. The stored CPMV is used only to generate inherited CPMVPs in the affine Merge mode and affine AMVP mode for the most recently encoded / decoded CU. Subblock MVs derived from the CPMV are used for motion compensation, MV derivation of the Merge / AMVP list for translation MVs, and deblocking.

[0139] To avoid image row caching for additional CPMVs, affine motion data inheritance from the CU of the upper CTU is handled differently from inheritance from normal neighboring CUs. If the candidate CU for affine motion data inheritance is in the upper CTU row, the lower left and lower right sub-block MVs in the row cache, instead of the CPMV, are used for affine MVP derivation. Thus, the CPMV is only stored in the local cache. If the candidate CU is a 6-parameter affine codec, the affine model is downgraded to a 4-parameter model. Figure 16 As shown, along the top CTU boundary, the motion vectors of the lower left and lower right sub-blocks of the CU are used for the affine inheritance of the CU in the bottom CTU.

[0140] 2.1.3.4. Prediction refinement using optical flow (PROF) for affine modes Compared to pixel-based motion compensation, sub-block-based affine motion compensation saves memory access bandwidth and reduces computational complexity, at the cost of reduced prediction accuracy. To achieve finer-grained motion compensation, Prediction Refinement (PROF) using optical flow is used to refine sub-block-based affine motion compensation predictions without increasing the memory access bandwidth required for motion compensation. In VVC, after sub-block-based affine motion compensation is performed, the luminance prediction samples are refined by adding the difference derived from the optical flow equation. PROF is described in the following four steps: Step 1) Sub-block-based affine motion compensation is performed to generate sub-block prediction I( i , j ).

[0141] Step 2) Using a 3-tap filter [-1, 0, 1], calculate the spatial gradient of the sub-block prediction at each sample location. The gradient calculation is exactly the same as the gradient calculation in BDOF.

[0142]

[0143] shift 1 is used to control the precision of the gradient. For gradient computation, the sub-block (i.e., 4x4) prediction is expanded by one sample on each side. To avoid additional memory bandwidth and additional interpolation computations, those expanded samples on the expanded boundaries are copied from the nearest integer pixel position in the reference image.

[0144] Step 3) Brightness prediction refinement is calculated using the following optical flow equation.

[0145]

[0146] in It refers to the location of the sample points. The calculated sample MV (by (representation) and sample points The difference between the MVs of the sub-blocks of the same sub-block, such as Figure 17 As shown. It is quantized in units of 1 / 32 brightness sample precision.

[0147] Since the affine model parameters and the sample point positions relative to the sub-block center do not change from one sub-block to another, therefore It can be computed for the first sub-block and reused for other sub-blocks in the same CU. This allows... dx ( i , j )and dy ( i , j ) is from the sample point location ( i , j ) to the center of the sub-block Horizontal and vertical offsets It can be derived from the following equation:

[0148] To maintain accuracy, the center of the sub-block Calculated as ( ( W SB – 1 ) / 2, ( H SB – 1) / 2), where W SB and H SB These are the width and height of the sub-block.

[0149] For a 4-parameter affine model

[0150] For a 6-parameter affine model

[0151] in These are the motion vectors of the top left, top right, and bottom left control points. w and h These are the width and height of the CU.

[0152] Step 4) Finally, refine the brightness prediction. Added to sub-block prediction I (i , j Final prediction I’ It is generated as the following equation.

[0153]

[0154] For a CU that has been affinely encoded and decoded, PROF is not applied in two cases: 1) all control points MV are the same, indicating that the CU only has translational motion; 2) the affine motion parameters are greater than the specified limit, because the sub-block-based affine MC is downgraded to the CU-based MC to avoid large memory access bandwidth requirements.

[0155] Fast encoding / decoding methods are applied to reduce the encoding / decoding complexity of affine motion estimation using PROF. PROF is not applied during the affine motion estimation stage in the following two cases: a) if the CU is not the root block and its parent block does not choose an affine mode as its optimal mode, then PROF is not applied because the probability of the current CU choosing an affine mode as its optimal mode is low; b) if the amplitudes of all four affine parameters (C, D, E, F) are less than a predefined threshold and the current image is not a low-latency image, then PROF is not applied because the improvement introduced by PROF is small in this case. Thus, affine motion estimation using PROF can be accelerated.

[0156] 2.1.4. Sub-block-based temporal motion vector prediction (SbTMVP) VVC supports a sub-block-based temporal motion vector prediction (SbTMVP) method. Similar to temporal motion vector prediction (TMVP) in HEVC, SbTMVP uses the motion field in the co-image to improve motion vector prediction and merge patterns for CUs in the current image. The same co-image used by TMVP is used for SbTVMP. SbTMVP differs from TMVP in two main aspects: – TMVP predicts motion at the CU level, but SbTMVP predicts motion at the sub-CU level; – TMVP obtains temporal motion vectors from co-op blocks in the co-op image (the co-op block is the lower right or center block relative to the current CU), and SbTMVP applies motion shift before obtaining temporal motion information from the co-op image, where the motion shift is obtained from the motion vector of one of the spatial neighboring blocks from the current CU.

[0157] The SbTVMP process is as follows: Figure 18A and 18B As shown. SbTMVP predicts the motion vectors of sub-CUs within the current CU in two steps. In the first step, it checks... Figure 18AThe spatial nearest neighbor A1 in the image is selected. If A1 has a motion vector that uses a co-located image as its reference image, then that motion vector is chosen as the motion shift to be applied. If no such motion is identified, the motion shift is set to (0, 0).

[0158] In the second step, the motion shift identified in step 1 is applied (i.e., added to the coordinates of the current block) to shift from, for example, Figure 18B The corresponding image shown obtains motion information (motion vectors and reference indices) at the sub-CU level. Figure 18B The example assumes that motion shift is set as the motion of block A1. Then, for each sub-CU, the motion information of its corresponding block (the smallest motion grid covering the center sample) in the co-location image is used to derive the motion information for the sub-CU. After the motion information of the co-location sub-CU is identified, it is converted into the motion vector and reference index of the current sub-CU in a manner similar to the TMVP process of HEVC, where temporal motion scaling is applied to align the reference image of the temporal motion vector with the reference image of the current CU.

[0159] In VVC, a combination of sub-block-based Merge lists containing both SbTVMP candidates and affine Merge candidates is used for signaling in sub-block-based Merge mode. SbTVMP mode is enabled / disabled via Sequence Parameter Set (SPS) flags. If SbTVMP mode is enabled, the SbTVMP prediction is added as the first entry in the sub-block-based Merge candidate list, followed by the affine Merge candidate. The size of the sub-block-based Merge list is transmitted via signaling in the SPS, and the maximum allowed size of the sub-block-based Merge list in VVC is 5.

[0160] The sub-CU size used in SbTMVP is fixed at 8x8, and like the affine Merge pattern, the SbTMVP pattern is only applicable to CUs with a width and height greater than or equal to 8.

[0161] The encoding logic for the additional SbTMVP Merge candidate is the same as that for other Merge candidates, that is, for each CU in the P-strip or B-strip, an additional RD check is performed to determine whether to use the SbTMVP candidate.

[0162] 2.1.5. Adaptive Motion Vector Resolution (AMVR) In HEVC, when `use_integer_mv_flag` in the strip header is equal to 0, the motion vector difference (MVD) (between the CU's motion vector and the predicted motion vector) is transmitted through the signal in quarter-luminance samples. In VVC, a CU-level adaptive motion vector resolution (AMVR) scheme is introduced. AMVR allows the CU's MVD to be encoded and decoded with different precisions. Depending on the current CU's mode (normal AMVP mode or affine AVMP mode), the current CU's MVD can be adaptively selected as follows: – Normal AMVP mode: quarter brightness sample, half brightness sample, integer brightness sample or four brightness sample.

[0163] – Affine AMVP mode: quarter luminance sample, integer luminance sample, or 1 / 16 luminance sample.

[0164] If the current CU has at least one non-zero MVD component, the MVD resolution indication at the CU level is conditionally transmitted via signal transmission. If all MVD components (i.e., both the horizontal and vertical MVD of reference list L0 and reference list L1) are zero, the quarter-spot luminance sample MVD resolution is presumed.

[0165] For a CU with at least one non-zero MVD component, a first flag is signaled to indicate whether quarter-luminance sample MVD precision is used for the CU. If the first flag is 0, no further signaling is required, and quarter-luminance sample MVD precision is used for the current CU. Otherwise, a second flag is signaled to indicate that half-luminance sample or other MVD precision (integer or quad-luminance sample) is used for the normal AMVP CU. In the case of half-luminance sample, a 6-tap interpolation filter is used instead of the default 8-tap interpolation filter for the half-luminance sample position. Otherwise, a third flag is signaled to indicate whether integer luminance sample MVD precision or quad-luminance sample MVD precision is used for the normal AMVP CU. In the case of an affine AMVP CU, the second flag is used to indicate whether integer luminance sample MVD precision or 1 / 16 luminance sample MVD precision is used. To ensure that the reconstructed MV has the expected precision (quarter-luminance sample, half-luminance sample, integer luminance sample, or quad-luminance sample), the CU's motion vector prediction value is rounded to the same precision as the MVD before being added to the MVD. The motion vector predictions are rounded to zero (i.e., negative motion vector predictions are rounded to positive infinity, and positive motion vector predictions are rounded to negative infinity).

[0166] The encoder uses RD checks to determine the current CU's motion vector resolution. To avoid always performing four CU-level RD checks for each MVD resolution, in VTM13, RD checks for MVD accuracy other than quarter-luminance samples are only conditionally invoked. For normal AVMP mode, the RD costs for quarter-luminance sample MVD accuracy and integer luminance sample MVD accuracy are first calculated. Then, the RD costs for integer luminance sample MVD accuracy are compared with those for quarter-luminance sample MVD accuracy to determine if further checks of the four-luminance sample MVD accuracy's RD cost are necessary. When the RD cost for quarter-luminance sample MVD accuracy is significantly less than that for integer luminance sample MVD accuracy, the four-luminance sample MVD accuracy RD check is skipped. Then, if the RD cost for integer luminance sample MVD accuracy is significantly greater than the best RD cost of the previously tested MVD accuracy, the half-luminance sample MVD accuracy check is skipped. For affine AMVP mode, if an affine inter-frame mode is not selected after checking the rate-distortion cost of affine Merge / Skip mode, Merge / Skip mode, quarter-lumen sample MVD precision normal AMVP mode, and quarter-lumen sample MVD precision affine AMVP mode, then the 1 / 16-lumen sample MVD precision and 1-pixel MVD precision affine inter-frame modes are not checked. Furthermore, in the 1 / 16-lumen sample and quarter-lumen sample MVD precision affine inter-frame modes, the affine parameters obtained in the quarter-lumen sample MVD precision affine inter-frame mode are used as the starting search point.

[0167] 2.1.6. Bidirectional prediction with CU-level weights (BCW) In HEVC, the bidirectional prediction signal is generated by averaging two prediction signals obtained from two different reference images and / or using two different motion vectors. In VVC, the bidirectional prediction mode is extended beyond simple averaging to allow for a weighted average of the two prediction signals.

[0168]

[0169] Five weights are allowed in weighted average two-way forecasting. For each bidirectional prediction CU, the weights w are determined in one of two ways: 1) for non-merge CUs, the weight index is transmitted via signal after the motion vector difference; 2) for merge CUs, the weight index is inferred from neighboring blocks based on the merge candidate index. BCW is applied only to CUs with 256 or more luma samples (i.e., CU width multiplied by CU height is greater than or equal to 256). For low-latency images, all 5 weights are used. For non-low-latency images, only 3 weights are used (w∈{3,4,5}).

[0170] – At the encoder, fast search algorithms are applied to find the weight indices without significantly increasing encoder complexity. These algorithms are summarized below. For more details, readers can refer to the VTM software and documentation JVET-L0646. When combined with AMVR, if the current image is a low-latency image, the unequal weights are conditionally checked only for 1-pixel and 4-pixel motion vector precision.

[0171] – When combined with an affine pattern, an affine ME will be performed for unequal weights if and only if the affine pattern is selected as the current best pattern.

[0172] – When the two reference images in bidirectional prediction are the same, unequal weights are only conditionally checked.

[0173] – When certain conditions are met, unequal weights are not searched, depending on the POC distance between the current image and its reference image, the encoding / decoding QP, and the temporal level.

[0174] The BCW weight index is encoded using a context-coded bit and a subsequent bypass-coded bit. The first context-coded bit indicates whether equal weights are used; if unequal weights are used, the bypass-coded bit is used to signal additional bits to indicate which unequal weights are used.

[0175] Weighted Prediction (WP) is a codec tool supported by the H.264 / AVC and HEVC standards for efficiently encoding and decoding video content with fading. Support for WP has also been added to the VVC standard. WP allows weighting parameters (weights and offsets) to be transmitted via signaling for each reference picture in each of the reference picture lists L0 and L1. Then, during motion compensation, the corresponding weights and offsets of the reference picture(s) are applied. WP and BCW are designed for different types of video content. To avoid interaction between WP and BCW (which would complicate the VVC decoder design), if the CU uses WP, the BCW weight index is not transmitted via signaling, and w is presumed to be 4 (i.e., equal weights are applied). For MergeCU, the weight index is presumed from neighboring blocks based on the Merge candidate index. This can be applied to normal Merge patterns and inherited affine Merge patterns. For constructed affine Merge patterns, affine motion information is constructed based on the motion information of up to 3 blocks. The BCW index of the CU using the constructed affine Merge pattern is simply set to be equal to the BCW index of the first control point MV.

[0176] In VVC, CIIP and BCW cannot be jointly applied to a CU. When a CU is encoded or decoded in CIIP mode, the BCW index of the current CU is set to 2, for example, with equal weights.

[0177] 2.1.7. Bidirectional Optical Flow (BDOF) The Bidirectional Optical Flow (BDOF) tool is included in VVC. BDOF (formerly known as BIO) was included in JEM. Compared to the JEM version, the BDOF in VVC is a simpler version, requiring far fewer computations, especially in terms of the number of multiplications and multiplier size.

[0178] BDOF is used to refine the bidirectional prediction signal of the CU at the 4x4 sub-block level. BDOF is applied to the CU if all of the following conditions are met: - CU is encoded and decoded using a "true" bidirectional prediction mode, that is, one of the two reference images is displayed before the current image in the order of display, and the other is displayed after the current image in the order of display.

[0179] - The distances (i.e., the difference in point of view) from the two reference images to the current image are the same.

[0180] - Both reference images are short-term reference images.

[0181] - CU is not encoded or decoded using affine mode or SbTMVP Merge mode.

[0182] - The CU has more than 64 luminance samples.

[0183] – Both the CU height and CU width are greater than or equal to 8 luminance samples.

[0184] – The BCW weight index indicates equal weights.

[0185] – WP is not enabled for the current CU.

[0186] – CIIP mode is not used in the current CU.

[0187] BDOF is applied only to the luminance component. As the name suggests, the BDOF mode is based on the concept of optical flow, which assumes that the motion of the object is smooth. For each 4x4 sub-block, motion refinement is calculated by minimizing the difference between the L0 and L1 predicted samples. Motion refinement is then used to adjust the bidirectional prediction sample values ​​in the 4x4 sub-blocks. The following steps are applied during the BDOF process.

[0188] First, the horizontal and vertical gradients of the two predicted signals. It is calculated by directly calculating the difference between two neighboring sample points, i.e.

[0189] in It is a list Coordinates of the predicted signal in The sample value at the location, and shift1 is calculated based on the luminance bit depth bitDepth as shift1 = max(6, bitDepth-6).

[0190] Then, the autocorrelation and cross-correlation of the gradients. Calculated as

[0191] in

[0192] Where Ω is the 6x6 window surrounding the 4x4 sub-block, and n a and n b The values ​​are set to min(1, bitDepth – 11) and min(4, bitDepth – 8) respectively.

[0193] Then, use the following formula to refine the motion. It is derived using cross-correlation and autocorrelation terms:

[0194] in . It is a floor function, and .

[0195] Based on motion refinement and gradients, the following adjustments are calculated for each sample point in the 4x4 sub-block:

[0196] Finally, the BDOF samples of CU are calculated by adjusting the bidirectional prediction samples as follows:

[0197] These values ​​are chosen so that the multiplier in the BDOF process does not exceed 15 bits, and the maximum bit width of the intermediate parameters in the BDOF process is kept within 32 bits.

[0198] To derive the gradient values, a list of elements outside the current CU boundary needs to be generated. Some predicted samples .like Figure 19As shown, BDOF in VVC uses an extended row / column around the CU boundary. To control the computational complexity of generating prediction samples outside the boundary, prediction samples in the extended region (white area) are generated by directly obtaining reference samples at nearby integer positions (using floor() operations on coordinates) without interpolation, and a normal 8-tap motion-compensated interpolation filter is used to generate prediction samples inside the CU (gray area). These extended sample values ​​are only used in gradient calculations. For the remaining steps in the BDOF process, if any samples and gradient values ​​outside the CU boundary are needed, they are filled from their nearest neighbors (i.e., repeated).

[0199] When the width and / or height of a CU (Cubic Unit) is greater than 16 luminance samples, it will be divided into sub-blocks with a width and / or height equal to 16 luminance samples, and the sub-block boundaries will be considered CU boundaries in the BDOF process. The maximum cell size for the BDOF process is limited to 16x16. The BDOF process can be skipped for each sub-block. The BDOF process is not applied to the sub-block when the SAD (Self-Adjustment Aspect) between the initial L0 and L1 predicted samples is less than a threshold. The threshold is set to equal to... , where W indicates the sub-block width and H indicates the sub-block height. To avoid the additional complexity of SAD calculation, the SAD between the initial L0 prediction samples and L1 prediction samples calculated during the DVMR process is reused here.

[0200] Bidirectional optical flow is disabled if BCW is enabled for the current block, meaning the BCW weight index indicates unequal weights. Similarly, BDOF is disabled if WP is enabled for the current block, meaning luma_weight_lx_flag is 1 for either of the two reference images. BDOF is also disabled when the CU is encoded and decoded in symmetric MVD or CIIP mode.

[0201] 2.1.8. Decoder-side Motion Vector Refinement (DMVR) To improve the accuracy of the motion vector refinement (MV) in the Merge mode, a decoder-side motion vector refinement based on bilateral matching is applied in the VVC. In the bidirectional prediction operation, a refined MV is searched around the initial MV in reference image lists L0 and L1. The BM method computes the distortion between two candidate blocks in reference image lists L0 and L1. Figure 20 As shown, the SAD between the red blocks of each MV candidate around the initial MV is calculated. The MV candidate with the lowest SAD becomes the refined MV and is used to generate the bidirectional prediction signal.

[0202] In VVC, DMVR can be applied to CUs that are encoded and decoded using the following modes and features: - CU-level Merge pattern with bidirectional prediction MV.

[0203] - Relative to the current image, one reference image is from the past and the other is from the future.

[0204] - The distances (i.e., the difference in point of view) from the two reference images to the current image are the same.

[0205] - Both reference images are short-term reference images.

[0206] - The CU has more than 64 luminance samples.

[0207] - Both the CU height and CU width are greater than or equal to 8 luminance samples.

[0208] - The BCW weight index indicates equal weights.

[0209] - WP is not enabled for the current block.

[0210] - CIIP mode is not used in the current block.

[0211] The refined motion vector (MV) derived through the DMVR process was used to generate inter-frame prediction samples and also for temporal motion vector prediction in future image encoding and decoding. The original MV was used in the deblocking process and also for spatial motion vector prediction in future CU encoding and decoding.

[0212] Additional features of DMVR are mentioned in the following sub-entries.

[0213] 2.1.8.1. Search Scheme In DVMR, the search point revolves around the initial MV, and the MV offset follows the MV difference mirror rule. In other words, any point examined by DMVR, represented by the candidate MV pair (MV0, MV1), follows the following two equations:

[0214] in MV_offset This represents the refinement offset between the initial MV and the refined MV in one of the reference images. The refinement search range is two integer luminance samples from the initial MV. The search includes an integer sample offset search phase and a fractional sample refinement phase.

[0215] A 25-point full search is applied to the integer sample offset search. The SAD of the initial MV pair is calculated first. If the SAD of the initial MV pair is less than a threshold, the integer sample stage of DMVR is terminated. Otherwise, the SAD of the remaining 24 points is calculated and checked in raster scan order. The point with the smallest SAD is selected as the output of the integer sample offset search stage. To reduce the impact of DMVR refinement uncertainties, a bias towards the original MV is proposed during the DMVR process. The SAD value between the reference blocks of the initial MV candidate reference is reduced by 1 / 4.

[0216] The integer sample search is followed by fractional sample refinement. To save computational complexity, fractional sample refinement is derived using the parameter error surface equation, rather than through an additional search utilizing SAD comparisons. Fractional sample refinement is conditionally invoked based on the output of the integer sample search phase. Fractional sample refinement is further applied when the integer sample search phase terminates in the first or second iteration with the minimum SAD at the center.

[0217] In subpixel offset estimation based on parametric error surfaces, the cost at the center location and the costs at the four nearest neighbor locations are used to fit a two-dimensional parabolic error surface equation of the following form:

[0218] in This corresponds to the score position with the minimum cost, and C corresponds to the minimum cost value. The above equation is solved by using the cost values ​​of the five search points. Calculated as:

[0219] and The value is automatically constrained between -8 and 8 because all cost values ​​are positive, and the minimum value is... This corresponds to a half-pixel offset with 1 / 16 pixel MV precision in VVC. The calculated score... It is added to the integer distance refinement MV to obtain the subpixel precise refinement difference MV.

[0220] 2.1.8.2. Bilinear Interpolation and Sample Filling In VVC, the resolution of the MV is 1 / 16 of a lumen sample. Samples at fractional positions are interpolated using an 8-tap interpolation filter. In DMVR, the search point surrounds the initial fractional pixel MV with an integer sample offset; therefore, for the DMVR search process, samples at those fractional positions need to be interpolated. To reduce computational complexity, a bilinear interpolation filter is used to generate fractional samples for the search process in DMVR. Another important effect of using a bilinear filter is that, utilizing a 2-sample search range, DVMR does not access more reference samples compared to the normal motion compensation process. After obtaining the refined MV through the DMVR search process, a normal 8-tap interpolation filter is applied to generate the final prediction. To avoid accessing more reference samples than the normal MC process, samples that are not needed by the interpolation process based on the original MV but are needed by the interpolation process based on the refined MV are filled from these available samples.

[0221] 2.1.8.3. Maximum DMVR Processing Unit When the width and / or height of a CU is greater than 16 luminance samples, it will be further divided into sub-blocks with a width and / or height equal to 16 luminance samples. The maximum cell size for the DMVR search process is limited to 16x16.

[0222] 2.1.9. Inter-frame and Intra-frame Joint Prediction (CIIP) In VVC, when a CU is encoded and decoded in Merge mode, if the CU contains at least 64 luma samples (i.e., the CU width multiplied by the CU height is equal to or greater than 64), and if both the CU width and CU height are less than 128 luma samples, an additional flag is transmitted via signaling to indicate whether Inter-Frame Intra-Frame Joint Prediction (CIIP) mode is applied to the current CU. As the name suggests, CIIP prediction combines inter-frame prediction signals with intra-frame prediction signals. The inter-frame prediction signal in CIIP mode... P inter The inter-frame prediction process is derived using the same procedure as the regular Merge mode; and the intra-frame prediction signal... P intra The conventional intra-frame prediction process with a planar pattern is derived. Then, a weighted average is used to combine the intra-frame prediction signal and the inter-frame prediction signal, where the weight values ​​are based on the top neighbor block and the left neighbor block (e.g., ...). Figure 21 The encoding / decoding mode (as shown) is calculated as follows: – If the top nearest neighbor is available and is intra-coded, set isIntraTop to 1; otherwise, set isIntraTop to 0. – If the left nearest neighbor is available and is intra-coded, set isIntraLeft to 1; otherwise, set isIntraLeft to 0. – If (isIntraLeft + isIntraTop) equals 2, then wt is set to 3; Otherwise, if (isIntraLeft + isIntraTop) equals 1, then wt is set to 2; Otherwise, set wt to 1.

[0223] The CIIP predictions are formed as follows:

[0224] 2.1.10. Geometric Partitioning (GPM) In VVC, geometric segmentation modes are supported for inter-frame prediction. Geometric segmentation modes are transmitted via signaling using a CU-level flag as a merge mode, where other merge modes include regular merge mode, MMVD mode, CIIP mode, and sub-block merge mode. For each possible CU size... (in Excluding 8x64 and 64x8, the geometric segmentation mode supports a total of 64 segments.

[0225] When using this mode, the CU is divided into two parts by a straight line of geometric positioning ( Figure 22 The position of the dividing line is mathematically derived from the angle and offset parameters of a specific segment. Each part of the geometric segmentation in the CU is predicted inter-frame using its own motion; only unidirectional prediction is allowed for each segment, meaning each part has one motion vector and one reference index. Unidirectional prediction motion constraints are applied to ensure that, as with regular bidirectional prediction, only two motion-compensated predictions are required for each CU.

[0226] If a geometric segmentation pattern is used for the current CU, the geometric segmentation pattern (angle and offset) and two merge indices (one for each segment) are further indicated via signal transmission. The number of maximum GPM candidate dimensions is explicitly transmitted in SPS, and the syntax binarization used for the GPM merge indices is specified. After predicting each part of the geometric segmentation, a blending process with adaptive weights is used to adjust the sample values ​​along the geometric segmentation edges. This is the prediction signal for the entire CU, and the transformation and quantization processes are applied to the entire CU as in other prediction patterns. Finally, the motion field of the CU predicted using the geometric segmentation pattern is stored.

[0227] 2.1.10.1. Construction of One-Way Prediction Candidate List The unidirectional prediction candidate list is directly derived from the Merge candidate list constructed according to the extended Merge prediction process. Let n denote the index of the unidirectional prediction motion in the geometric unidirectional prediction candidate list. The LX motion vector of the nth extended Merge candidate (where X equals the parity of n) is used as the nth unidirectional prediction motion vector for the geometric segmentation pattern. These motion vectors in... Figure 23 The value is marked with "x". If the corresponding LX motion vector of the nth extended Merge candidate does not exist, the L(1-X) motion vector of the same candidate is used as the unidirectional predicted motion vector for the geometric segmentation pattern.

[0228] 2.1.10.2. Blending along geometrically segmented edges After using its own motion to predict each part of the geometric segmentation, mixing is applied to the two predicted signals to derive samples around the geometric segmentation edges. The mixing weights at each location of the CU are derived based on the distance between the individual location and the segmentation edge.

[0229] For location The distance to the segmentation edge is derived as follows:

[0230] in i , j It is an index used for the angle and offset of geometric segmentation, which depends on the geometric segmentation index transmitted via signal. The sign depends on the angle index. i .

[0231] The weights of each part of the geometric segmentation are derived as follows:

[0232] partIdx depends on the angle index i Weight w An example of 0 in Figure 24 It is shown in the middle.

[0233] 2.1.10.3. Motion field storage for geometric segmentation patterns Mv1 from the first part of the geometric segmentation, Mv2 from the second part of the geometric segmentation, and Mv, a combination of Mv1 and Mv2, are stored in the motion field of the CU encoded and decoded by the geometric segmentation pattern.

[0234] The type of motion vector stored for each individual location in the sports field is determined as follows:

[0235] Where motionIdx equals partIdx depends on the angle index. i .

[0236] If sType equals 0 or 1, then Mv0 or Mv1 is stored in the corresponding motion field; otherwise, if sType equals 2, then the combined Mv from Mv0 and Mv2 is stored. The combined Mv is generated using the following process: 1) If Mv1 and Mv2 come from different lists of reference images (one from L0 and the other from L1), then Mv1 and Mv2 are simply combined to form a bidirectional predicted motion vector.

[0237] 2) Otherwise, if Mv1 and Mv2 come from the same list, only the unidirectional predicted motion Mv2 is stored.

[0238] 2.1.11. Local Illumination Compensation (LIC) LIC is an inter-frame prediction technique used to model the local illumination variation between the current block and its predicted block as a function of the local illumination variation between the current block template and the reference block template. The parameters of this function can be scaled... α and offset β This means that they form a linear equation, that is... To compensate for changes in illumination, p[x] is the reference sample point at position x on the reference image, pointed to by MV. Since α and β It can be derived based on the current block template and the reference block template, so there is no signaling overhead for them except for signaling the LIC flag to indicate the use of LIC for AMVP mode.

[0239] The local illumination compensation proposed in JVET-O0066 was used for unidirectional prediction of inter-frame CU, and the following modifications were made.

[0240] • Intra-frame neighbor samples can be used for LIC parameter derivation; • For blocks with fewer than 32 luminance samples, LIC is disabled; • For both non-subblock mode and affine mode, the LIC parameter derivation is based on the template block samples corresponding to the current CU, rather than on the partial template block samples corresponding to the first top-left 16x16 cell that are executed. • The samples of the reference block template are generated by using a MC with block MV without rounding it to integer pixel precision.

[0241] 2.1.12. Non-adjacent airspace candidates For example, in JVET-L0399, non-adjacent airspace merge candidates are inserted after the TMVP in the regular merge candidate list. The style of airspace merge candidates is as follows: Figure 25 The distance between non-adjacent spatial domain candidates and the current codec block is shown in the diagram. The distance between these candidates and the current codec block is based on the width and height of the current codec block. Line buffer limits are not applied.

[0242] 2.1.13. Template Matching I Template matching (TM) is a decoder-side MV derivation method used to refine the motion information of the current CU by finding the closest match between a template in the current image (i.e., the top and / or left neighboring blocks of the current CU) and a block in the reference image (i.e., of the same size as the template). For example... Figure 26 As shown, within the search range of [-8, +8] pixels, a better MV is searched around the initial motion of the current CU. The template matching method in JVET-J0021 is used, with the following modifications: the search step size is determined based on the AMVR mode, and in Merge mode, the TM can be cascaded with the bilateral matching process.

[0243] In AMVP mode, MVP candidates are determined based on template matching error to select the one that minimizes the difference between the current block template and the reference block template. TM is then performed only for that specific MVP candidate for MV refinement. TM refines the MVP candidate using an iterative diamond search, starting with full-pixel MVD precision (or 4 pixels for 4-pixel AMVR mode) within a search range of [-8, +8] pixels. AMVP candidates can be further refined using a cross search with full-pixel MVD precision (or 4 pixels for 4-pixel AMVR mode), followed by half-pixels and quarter-pixels sequentially according to the AMVR mode specified in Table 3. This search process ensures that the MVP candidate maintains the same MV precision as indicated by the AMVR mode after the TM process.

[0244] Table 3. AMVR search patterns and search patterns using AMVR's Merge mode

[0245] In Merge mode, a similar search method is applied to the Merge candidates indicated by the Merge index. As shown in Table 3, TM can proceed up to 1 / 8 pixel MVD precision, or skip those precisions beyond half-pixel MVD precision, depending on whether an alternative interpolation filter is used based on the merged motion information (i.e., used when AMVR is in half-pixel mode). Furthermore, when TM mode is enabled, template matching can operate as a standalone process or as an additional MV refinement process between block-based and sub-block-based bilateral matching (BM) methods, depending on whether BM can be enabled according to its enable condition check.

[0246] 2.1.14. Multi-pass decoder-side motion vector refinement (mpDMVR) Multi-pass decoder-side motion vector refinement is applied. In the first pass, bilateral matching (BM) is applied to the codec block. In the second pass, BM is applied to each 16x16 sub-block within the codec block. In the third pass, the motion vector (MV) in each 8x8 sub-block is refined by applying bidirectional optical flow (BDOF). The refined MV is stored for both spatial and temporal motion vector predictions.

[0247] 2.1.14.1. First pass – Block-based bilateral matching MV refinement In the first pass, the refined MV is derived by applying BM to the codec block. Similar to decoder-side motion vector refinement (DMVR), in the bidirectional prediction operation, the refined MV is searched around the two initial MVs (MV0 and MV1) in the reference picture lists L0 and L1. The refined MVs (MV0_pass1 and MV1_pass1) are derived around the initial MVs based on the minimum bilateral matching cost between the two reference blocks in L0 and L1.

[0248] BM performs a local search to derive the integer sample precision intDeltaMV. The local search applies a 3x3 square search pattern to iterate through the search range [-sHor, sHor] in the horizontal direction and the search range [-sVer, sVer] in the vertical direction, where the values ​​of sHor and sVer are determined by the block dimension, and the maximum value of sHor and sVer is 8.

[0249] The cost of a two-sided match is calculated as: bilCost = mvDistanceCost + sadCost. When the block size... When the value is greater than 64, the MRSAD cost function is applied to eliminate the DC effect of distortion between reference blocks. The intDeltaMV local search terminates when the bilCost at the center point of the 3x3 search pattern has the minimum cost. Otherwise, the current minimum cost search point becomes the new center point of the 3x3 search pattern, and the search for the minimum cost continues until the end of the search range is reached.

[0250] The existing fractional sample refinement is further applied to derive the final deltaMV. Then, the refined MV after the first pass is derived as follows: · MV0_pass1 = MV0 + deltaMV; · MV1_pass1 = MV1 – deltaMV.

[0251] 2.1.14.2. Second pass – Sub-block-based bilateral matching MV refinement In the second pass, the refined MV is derived by applying BM to 16x16 grid sub-blocks. For each sub-block, a refined MV is searched around the two MVs (MV0_pass1 and MV1_pass1) obtained in the first pass within the reference image lists L0 and L1. The refined MVs (MV0_pass2(sbIdx2) and MV1_pass2(sbIdx2)) are derived based on the minimum bilateral matching cost between the two reference sub-blocks in L0 and L1.

[0252] For each subblock, BM performs a full search to derive the integer sample precision intDeltaMV. The full search has a search range of [-sHor, sHor] in the horizontal direction and a search range of [-sVer, sVer] in the vertical direction, where the values ​​of sHor and sVer are determined by the block dimension, and the maximum value of sHor and sVer is 8.

[0253] The bilateral matching cost is calculated by applying a cost factor to the SATD cost between the two reference sub-blocks, as follows: Search area Classified as Figure 27 A maximum of 5 diamond-shaped search regions are shown. Each search region is assigned a costFactor, which is determined by the distance (intDeltaMV) between each search point and the starting MV, and each diamond region is processed sequentially starting from the center of the search region. Within each region, search points are processed in raster scan order, starting from the top left corner and ending at the bottom right corner. The minimum bilCost within the current search region is less than a threshold (which is equal to...). If the search fails, terminate the full pixel search; otherwise, continue to the next search area until all search points have been checked.

[0254] The existing VVC DMVR fractional sample refinement is further applied to derive the final deltaMV(sbIdx2). Then, the refined MV for the second pass is derived as follows: ● MV0_pass2(sbIdx2) = MV0_pass1 + deltaMV(sbIdx2); ● MV1_pass2(sbIdx2) = MV1_pass1 – deltaMV(sbIdx2).

[0255] 2.1.14.3. Third pass – Sub-block based bidirectional optical flow MV refinement In the third pass, the refined MV is derived by applying BDOF to the 8x8 grid subblocks. For each 8x8 subblock, BDOF refinement is applied starting from the refined MV of the parent subblock in the second pass to derive scaled Vx and Vy without clipping. The derived bioMv(Vx, Vy) is rounded to 1 / 16 sample precision and clipped between -32 and 32.

[0256] The refined MVs (MV0_pass3(sbIdx3) and MV1_pass3(sbIdx3)) of the third pass are derived as follows: ● MV0_pass3(sbIdx3) = MV0_pass2(sbIdx2) + bioMv; ● MV1_pass3(sbIdx3) = MV0_pass2(sbIdx2) – bioMv.

[0257] 2.1.15. OBMC When OBMC is applied, the top and left boundary pixels of the CU are refined using motion information from neighboring blocks with weighted prediction, as described in JVET-L0101.

[0258] The following conditions should not be used for OBMC: • When OBMC is disabled at the SPS level.

[0259] • When the current block has intra-frame mode or IBC mode.

[0260] • When applying LIC to the current block.

[0261] • When the current luminance block area is less than or equal to 32.

[0262] Sub-block boundary OBMC is performed by applying the same blending to the top, left, bottom, and right sub-block boundary pixels using motion information from neighboring sub-blocks. Enable this for sub-block-based codec tools. • Affine AMVP mode; • Affine Merge pattern and sub-block-based temporal motion vector prediction (SbTMVP). • Bilateral matching based on sub-blocks.

[0263] 2.1.16. Sample-based BDOF In sample-based BDOF, motion refinement is not based on block derivation (Vx, Vy), but is performed on a per-sample basis.

[0264] The encoding / decoding block is divided into 8x8 sub-blocks. For each sub-block, the SAD between two reference sub-blocks is checked using a relative threshold to determine whether BDOF should be applied. If it is decided to apply BDOF to the sub-block, a sliding 5x5 window is used for each sample in the sub-block, and the existing BDOF process is applied to derive Vx and Vy for each sliding window. The derived motion refinement (Vx, Vy) is applied to adjust the bidirectional prediction sample values ​​for the center sample of the window.

[0265] 2.1.17. Interpolation The 8-tap interpolation filter used in VVC has been replaced by a 12-tap filter. The interpolation filter is derived from the sinc function, its frequency response is cut off at the Nyquist frequency, and is clipped by a cosine window function. Table 4 shows the filter coefficients for all 16 phases. Figure 28 The frequency responses of the interpolation filter and the VVC interpolation filter were compared, and all frequency responses were at half-pixel phase.

[0266] Table 4. Filter coefficients of the 12-tap interpolation filter

[0267] 2.1.18. Multiple Hypothesis Prediction (MHP) In the multi-hypothesis inter-frame prediction mode (JVET-M0425), in addition to the regular bidirectional prediction signal, one or more additional motion-compensated prediction signals are transmitted via signal transmission. The resulting overall prediction signal is obtained by sample-by-sample weighted superposition. The bidirectional prediction signal is utilized... p bi and the first additional inter-frame prediction signal / hypothesis h 3. The obtained prediction signal p 3 is obtained as follows:

[0268] The weighting factor α is specified by the new syntax element add_hyp_weight_idx according to the following mapping:

[0269] Similar to the above, more than one additional prediction signal can be used. The resulting overall prediction signal is iteratively accumulated with each additional prediction signal.

[0270]

[0271] The resulting overall prediction signal is obtained as the final p n (i.e., has the largest index) n of p n Within this EE, a maximum of two additional prediction signals can be used (i.e., n Limited to 2).

[0272] The motion parameters for each additional prediction hypothesis can be explicitly transmitted via signaling by specifying the reference index, motion vector prediction value index, and motion vector difference, or implicitly transmitted via signaling by specifying the merge index. A separate multi-hypothesis merge flag distinguishes between these two signaling modes.

[0273] For inter-frame AMVP mode, MHP is applied only when unequal weights in BCW are selected in bidirectional prediction mode.

[0274] Combining MHP and BDOF is possible; however, BDOF is only applied to the bidirectional prediction signal portion of the predicted signal (i.e., the ordinary first two assumptions).

[0275] 2.1.19. Adaptive Reordering of Merge Candidates with Template Matching (ARMC-TM) Merge candidates are adaptively reordered using Template Matching (TM). The reordering method is applied to the regular Merge pattern, Template Matching (TM) Merge pattern, and Affine Merge pattern (excluding SbTMVP candidates). For the TM Merge pattern, Merge candidates are reordered before the refinement process.

[0276] After constructing the Merge candidate list, the Merge candidates are divided into several subgroups. For the regular Merge pattern and the TM Merge pattern, the subgroup size is set to 5. For the affine Merge pattern, the subgroup size is set to 3. Within each subgroup, the Merge candidates are reordered in ascending order based on template matching and their cost value. For simplicity, the Merge candidates in the last subgroup (not the first subgroup) are not reordered.

[0277] The template matching cost of the merge candidate is measured by the sum of absolute differences (SAD) between the samples of the current block's template and their corresponding reference samples. The template includes a set of reconstructed samples adjacent to the current block. The reference samples of the template are located using the motion information of the merge candidate.

[0278] When the merge candidate utilizes bidirectional prediction, the reference samples for the merge candidate's template are also generated through bidirectional prediction, such as... Figure 29 As shown.

[0279] For a sub-block-based merge candidate with a sub-block size equal to Wsub × Hsub, the upper template includes several sub-templates of size Wsub × 1, and the left template includes several sub-templates of size 1 × Hsub. For example... Figure 30 As shown, the motion information of the sub-blocks in the first row and first column of the current block is used to derive the reference sample points of each sub-template.

[0280] 2.1.20. Geometric Partitioning Pattern (GPM) with Merge Motion Vector Difference (MMVD) The GPM in VVC is extended by applying motion vector refinement on top of the existing unidirectional MV of the GPM. First, a flag is transmitted to the GPMCU to specify whether to use this mode. If this mode is used, each geometric segment of the GPM CU can further determine whether to transmit MVD via signal transmission. If MVD is transmitted via signal transmission for each geometric segment, the segment's motion is further refined using the transmitted MVD information after selecting a GPM Merge candidate. All other procedures remain the same as in the GPM.

[0281] Similar to MMVD, MVD transmits range and direction pairs via signals. In GPM with MMVD (GPM-MMVD), nine candidate ranges are involved. The MVD has eight pixel values ​​(1, 2, 3, 4, 6, 8, and 16 pixels) and eight candidate directions (four horizontal / vertical directions and four diagonal directions). Additionally, when `pic_fpel_mmvd_enabled_flag` equals 1, the MVD is shifted left by 2, as in MMVD.

[0282] 2.1.21. Geometric Segmentation Pattern (GPM) with Template Matching (TM) Template matching is applied to GPM. When GPM mode is enabled for CU, the CU level flag is signaled to indicate whether TM is applied to the two geometric segments. Motion information for each geometric segment is refined using TM. When TM is selected, a template is constructed using left, top, or left and top neighboring samples, depending on the segmentation angle, as shown in Table 5. The motion is then refined by minimizing the difference between the current template and the template in the reference image using the same search style of Merge mode with the half-pixel interpolation filter disabled.

[0283] Table 5 . For the templates of the first and second geometric segments, where A indicates using the upper sample point and L indicates using the left sample point. Sample points, and L+A indicates that both left and top samples are used.

[0284]

[0285] The GPM candidate list is constructed as follows: 1. The interleaved list 0 MV candidates and list 1 MV candidates are derived directly from the regular merge candidate list, where list 0 MV candidates have a higher priority than list 1 MV candidates. A deduplication method with an adaptive threshold based on the current CU size is applied to remove redundant MV candidates.

[0286] 2. The interleaved list 1 MV candidates and list 0 MV candidates are further derived directly from the regular merge candidate list, where list 1 MV candidates have a higher priority than list 0 MV candidates. The same deduplication method with an adaptive threshold is also applied to remove redundant MV candidates.

[0287] 3. Zero MV candidates are filled until the GPM candidate list is full.

[0288] GPM-MMVD and GPM-TM are exclusively enabled to a single GPM CU. This is accomplished first by signaling the GPM-MMVD syntax. When both GPM-MMVD control flags are equal to "false" (i.e., GPM-MMVD is disabled for both GPM segments), the GPM-TM flag is signaled to indicate whether template matching is applied to both GPM segments. Otherwise (if at least one GPM-MMVD flag is equal to "true"), the value of the GPM-TM flag is presumed to be false.

[0289] 2.1.22. GPM with inter-frame and intra-frame prediction (GPM inter-frame - intra-frame) By utilizing GPM inter-frame / intra-frame prediction, in addition to the Merge candidates for each non-rectangular partitioned region in the CU where GPM is applied, a predefined intra-frame prediction mode for the geometric partition line can also be selected. In the proposed method, a flag from the encoder is used to determine whether it is an intra-frame or inter-frame prediction mode for each GPM-separated region. When it is an inter-frame prediction mode, a unidirectional prediction signal is generated using the MV from the Merge candidate list. On the other hand, when it is an intra-frame prediction mode, a unidirectional prediction signal is generated based on the neighboring pixels specified by the index from the encoder for the intra-frame prediction mode. The possible variations of the intra-frame prediction mode are limited by the geometry. Finally, the two unidirectional prediction signals are mixed in the same manner as with ordinary GPM.

[0290] 2.1.23. Adaptive Decoder-Side Motion Vector Refinement (ADMVR) The adaptive decoder-side motion vector refinement method consists of two new Merge modes, which are introduced to refine the motion vectors only in one direction (L0 or L1) of the bidirectional predictions of the Merge candidates that satisfy the DMVR condition. A multi-pass DMVR process is applied to the selected Merge candidates to refine the motion vectors; however, in the first pass (i.e., PU level) of the DMVR, either MVD0 or MVD1 is set to zero.

[0291] Similar to the regular Merge pattern, the Merge candidates for the proposed Merge pattern are derived from spatially adjacent encoded / decoded blocks, TMVP, non-adjacent blocks, HMVP, and paired candidates. The difference is that only those satisfying the DMVR condition are added to the candidate list. Both proposed Merge patterns use the same list of Merge candidates and encode / decode the Merge index as in the regular Merge pattern.

[0292] 2.1.24. Two-sided matching AMVP-MERGE pattern (AMVP-MERGE) In the AMVP-Merge model, the bidirectional prediction consists of the AMVP prediction in one direction and the Merge prediction in the other direction.

[0293] The AMVP portion of the proposed pattern is signaled as a regular unidirectional AMVP, i.e., the reference index and MVD are signaled, and it has a derived MVP index if template matching (TM_AMVP) is used, or the MVP index is signaled when template matching is disabled. The Merge index is not signaled, and the Merge prediction is selected from a list of candidates with the minimum template or bilateral matching cost.

[0294] When the selected Merge and AMVP predictions satisfy the DMVR condition (i.e., there is at least one reference image from the past and one reference image from the future relative to the current image, and the distances from both reference images to the current image are the same), bilateral matching MV refinement is applied to the Merge MV candidate and AMVP MVP, which serve as the starting points. Otherwise, if template matching is enabled, template matching MV refinement is applied to either the Merge or AMVP prediction, which has a higher template matching cost.

[0295] The third pass of the 8x8 sub-PU BDOF refinement of the multi-pass DMVR is enabled for blocks encoded and decoded in AMVP-Merge mode.

[0296] 3. Problem In ECM-8.0, LIC can be used on blocks encoded and decoded by AMVP and Merge, and the LIC parameters are calculated on the fly based on the relationship between the current template and the reference template. However, LIC parameters can also be derived / inherited from previously encoded and decoded blocks, which can improve encoding and decoding efficiency.

[0297] 4. Detailed Solution The detailed solutions below should be considered as examples for explaining general concepts. These solutions should not be interpreted in a narrow sense. Furthermore, these solutions can be combined in any way.

[0298] The terms “video unit” or “code-decoder unit” or “block” can refer to code-decoder tree block (CTB), code-decoder tree unit (CTU), code-decoder block (CB), CU, PU, ​​TU, PB, TB.

[0299] In this disclosure, with respect to "block encoded or decoded in mode N", "mode N" can be a predictive mode (e.g., MODE_INTRA, MODE_INTER, MODE_PLT, MODE_IBC, etc.) or an encoding / decoding technique (e.g., AMVP, Merge, SMVD, BDOF, PROF, DMVR, AMVR, TM, affine, CIIP, GPM, GEO, TPM, MMVD, BCW, HMVP, SbTMVP, etc.).

[0300] In the following discussion, the LIC parameters can refer to two parameters derived from a linear model (such as scaling parameter "a" and offset parameter "b"), which are used to map the relationship between the current template and the reference template. Furthermore, the LIC parameters can be used to estimate the predicted values ​​of samples within the current video cell.

[0301] In the following discussion, AMVP mode can refer to a predictive mode in which an indication of motion / block vector difference is transmitted via signaling. For example, it can be a regular AMVP mode, an affine AMVP mode, an SMVD mode, an IBC AMVP mode, an MHP mode with the basic assumptions of AMVP encoding and decoding, and / or an AMVP-MERGE mode.

[0302] In the following discussion, MERGE mode can refer to a prediction mode in which no motion / block vector difference is transmitted through the signal. For example, it can be a regular MERGE mode, an affine MERGE mode, a TM MERGE mode, a GPM mode, a CIIP mode, an ADMVR mode, a sbTMVP mode, an IBC MERGE mode, an MHP with the basic assumption of MERGE encoding and decoding, etc.

[0303] It should be noted that the terms mentioned below are not limited to the specific terms defined in existing standards. Any changes to codec tools also apply.

[0304] 1. For example, the LIC parameters of a video block can be stored and used in the following encoding and decoding processes.

[0305] a. In one example, in addition to mode information (such as intra / inter / IBC mode, LIC flag, etc.) and motion information (such as motion vectors, reference indexes, reference lists, etc.), LIC parameters such as scaling and offset can also be stored in the cache.

[0306] b. In one example, the LIC parameter can be stored in association with motion information for each video block.

[0307] c. In one example, the LIC parameter can be stored in M×N cells (such as 4×4 cells).

[0308] d. In one example, the LIC parameter can be stored in a lookup table (e.g., a history-based HMVP table).

[0309] e. In one example, it can be stored in a local cache representing the data within the current block / CU / PU / TU / VPDU / CTU / CTU line / slice / slice group / strip / subpicture / picture.

[0310] f. In one example, it can be stored in a time-domain cache representing data for a time-domain reference unit.

[0311] 2. For example, a motion candidate may include LIC parameters, and the LIC parameters associated with a decoded motion candidate may be used (e.g., to directly copy) the current block (e.g., without needing to compute from the current template and the reference template).

[0312] a. In one example, which LIC parameters are used for the current block based on the decoded motion candidates.

[0313] b. In one example, motion candidates can be derived based on previously encoded / decoded blocks.

[0314] c. In one example, motion candidates can be derived based on neighboring blocks.

[0315] d. In one example, motion candidates can be derived based on non-adjacent blocks.

[0316] e. In one example, motion candidates can be derived based on temporal blocks in a reference image.

[0317] f. In one example, motion candidates can be derived based on the HMVP table.

[0318] g. In one example, motion candidates can be generated based on the encoding and decoding information of more than one video block.

[0319] i. In one example, it can follow predefined rules to combine the motion information of the first two candidates.

[0320] h. In one example, motion candidates can be candidates from the list of inter-frame / affine / TM / BM / GPM / CIIP / sbTMVP / MHP / IBCMERGE.

[0321] i. In one example, motion candidates can be candidates from the list of inter-frame / affine / amvp merge / MHP / IBC AMVP.

[0322] j. In one example, a motion candidate can be a candidate from the LIC motion candidate list.

[0323] k. Alternatively, specific modifications to the LIC parameters associated with the decoded motion candidate (e.g., scaling of “a” and / or additional offsets applied to “b”) can be applied to derive the final LIC parameters for the current block.

[0324] 3. For example, a candidate list for LIC parameter derivation can be generated for blocks encoded or decoded by LIC.

[0325] a. In one example, all candidates in the candidate list can be encoded and decoded using LIC.

[0326] b. In one example, the candidates in the candidate list can come from neighboring neighbors, non-neighboring neighbors, time-domain blocks, HMVP tables, etc.

[0327] c. In one example, a dummy candidate can be inserted into the LIC candidate list.

[0328] i. In one example, a virtual candidate can be generated based on at least two existing LIC candidates.

[0329] d. In one example, at least one candidate indicator LIC parameter is calculated on the fly (e.g., by adjusting the relationship between the current template and the reference template).

[0330] e. In one example, candidate indices can be transmitted via signaling in the bitstream to provide hints on how to generate LIC parameters for the current block.

[0331] f. In one example, the first set of LIC parameters can be compared with at least one set of LIC parameters in the LIC parameter list before the first set of LIC parameters is added to the LIC parameter list.

[0332] i. In one example, if the first set of LIC parameters is the same as or similar to a set of LIC parameters in the LIC parameter list, then the first set of LIC parameters may not be included in the LIC parameter list.

[0333] ii. Alternatively, the similarity between two sets of LIC parameters can be measured based on predefined rules.

[0334] 1) In one example, it can depend on the block dimensions (width and / or height).

[0335] 2) In one example, it can depend on the order (index) of a set of LIC parameters in the LIC parameter list.

[0336] 4. In one example, the LIC HMVP table can be maintained.

[0337] a. In one example, the LIC parameters of a previously encoded / decoded LIC block can be stored in the LIC HMVP table.

[0338] b. In one example, the LIC parameters of the previously encoded / decoded block can be stored in a regular HMVP table along with regular motion information (e.g., motion vectors, predicted directions, reference indices, etc.).

[0339] c. In one example, in an HMVP table, the LIC parameter can be represented as a pair of {scale, offset}. For example, an HMVP table can contain multiple elements such as {scale0, offset0}, {scale1, offset1}, {scale2, offset2}, ... i. Alternatively, the scaling factor and offset value can be represented independently. For example, two HMVP tables can be maintained, one containing elements such as {scale0, scale1, scale2, …} and the other containing elements such as {offset0, offset1, offset2, …}.

[0340] 1) In one example, the LIC parameter for the current block can be a combination of scaling factors and offset values ​​from two HMVP tables. For example, it could be {scaleX, offsetY}, where X != Y.

[0341] d. In one example, the length of the LIC HMVP table can be equal to N.

[0342] i. In one example, N is a constant (such as N = 5 / 6 / 7 / 8 / 9 / 10).

[0343] ii. Alternatively, N can be adapted to encoding / decoding information (such as CTU size, image / sequence resolution, etc.).

[0344] e. In one example, the contents of the LIC HMVP table can be reset at the image / sub-image / slice / slice group / CTU row / strip level.

[0345] f. In one example, the first set of LIC parameters can be compared with at least one set of LIC parameters in the LIC HMVP table before being placed into the LIC HMVP table.

[0346] i. In one example, if the first set of LIC parameters is the same as or similar to a set of LIC parameters in the LIC HMVP table, then the first set of LIC parameters may not be included in the LIC HMVP table.

[0347] ii. Alternatively, the similarity between two sets of LIC parameters can be measured based on predefined rules.

[0348] 1) In one example, it can depend on the block dimensions (width and / or height).

[0349] 2) In one example, it can depend on the order (index) of a set of LIC parameters in the LIC HMVP table.

[0350] 5. For example, whether to compute LIC parameters (e.g., compute LIC parameters on the fly based on the current template and the reference template) or to inherit LIC parameters (e.g., copy stored LIC parameters directly from motion candidates encoded and decoded by LIC) can be based on predefined rules.

[0351] a. In one example, it can be transmitted via signals through (multiple) syntax elements.

[0352] b. In one example, it can be transmitted via signals through mode flags / indexes (e.g., block-level LIC mode types).

[0353] i. In one example, the first flag can indicate whether LIC is used, and the second flag can indicate the LIC mode type (e.g., whether the LIC parameter is evaluated on the fly or directly inherited).

[0354] ii. Furthermore, in one example, the LIC parameters of the first LIC mode type can be deduced by on-the-fly computation, and the LIC parameters of the second LIC mode type can be deduced by inheritance.

[0355] c. In one example, it can be inherited from a motion candidate.

[0356] i. In one example, once the LIC mode type of a motion candidate is decoded, whether the LIC parameter is computed or inherited for the current block can be inherited depending on whether the LIC parameter of the motion candidate is computed or inherited.

[0357] 6. For example, the final prediction of a LIC-encoded block can be generated by fusing the LIC prediction with at least one prediction of a non-LIC-encoded block.

[0358] 7. For example, the final prediction of a LIC-encoded block can be generated by fusing more than one intermediate LIC prediction.

[0359] a. In one example, at least one intermediate prediction can be derived from the inherited LIC parameter.

[0360] b. In one example, the first prediction can be generated by the inherited LIC parameter, and the second prediction can be generated by the LIC parameter calculated on the fly.

[0361] c. In one example, the first prediction can be generated using the first set of inherited LIC parameters, and the second prediction can be generated using the second set of inherited LIC parameters.

[0362] 8. For example, the disclosed method can be applied to AMVP blocks encoded and decoded by LIC.

[0363] a. In one example, it could be a regular inter-frame AMVP block.

[0364] b. In one example, it could be an affine AMVP block.

[0365] c. In one example, it could be an AMVP-MERGE block.

[0366] d. In one example, it could be an MHP block with the basic assumptions of being encoded and decoded by AMVP.

[0367] e. In one example, it could be an IBC AMVP block.

[0368] 9. For example, the disclosed method can be applied to LIC-encoded MERGE blocks.

[0369] a. In one example, it could be a regular inter-frame merge block.

[0370] b. In one example, it could be an affine MERGE block.

[0371] c. In one example, it could be the sbTMVP / GPM / CIIP / TM / BM / ADMVR MERGE block.

[0372] d. In one example, it could be an MHP block with the basic assumption of being encoded and decoded via MERGE.

[0373] e. In one example, it could be an IBC MERGE block.

[0374] 10. For example, the disclosed method can be applied to unidirectional prediction blocks encoded and decoded by LIC.

[0375] 11. For example, the disclosed method can be applied to bidirectional prediction blocks encoded and decoded by LIC.

[0376] a. For example, the disclosed method can be applied to both directions of a bidirectional prediction block.

[0377] b. For example, the disclosed method can be applied to one direction of a bidirectional prediction block.

[0378] 12. The syntax elements disclosed above can be binarized into flags, fixed-length codes, EG(x) codes, unary codes, rounded unary codes, rounded binary codes, etc. These can be signed or unsigned.

[0379] 13. The grammatical elements disclosed above can be encoded and decoded using at least one context model. Alternatively, they can be encoded and decoded using a bypass method.

[0380] 14. The syntax elements disclosed above can be transmitted via signals in a conditional manner. For example, SE may be transmitted via signals only if the corresponding function applies.

[0381] 15. For example, the LIC parameters disclosed in the documentation may include only the LIC parameters for a specific component (such as the Y component).

[0382] 16. For example, LIC parameters disclosed in the documentation may include LIC parameters for multiple components.

[0383] a. For example, different components (such as Y and Cb / Cr) can have different LIC parameters.

[0384] 17. Whether and / or how the methods disclosed above can be applied to be transmitted via signaling at the sequence level / picture group level / picture level / strip level / piece group level, such as in the sequence header / picture header / SPS / VPS / DPS / DCI / PPS / APS / strip header / piece group header.

[0385] 18. Whether and / or how the methods disclosed above can be applied to transmit signals at PB / TB / CB / PU / TU / CU / VPDU / CTU / CTU lines / strips / films / sub-images / other types of areas containing more than one sample point or pixel.

[0386] 19. Whether and / or how the methods disclosed above are applied may depend on the encoded / decoded information, such as block size, color format, single-tree / dual-tree segmentation, color components, and stripe / picture type.

[0387] The methods presented in this document can be used in other encoding / decoding tools that require solving linear / nonlinear regression models.

[0388] As used herein, the term "video unit" or "video block" can be a sequence, picture, strip, slice, brick, sub-picture, codec tree unit (CTU) / codec tree block (CTB), CTU / CTB row, one or more codec units (CU) / codec blocks (CB), one or more CTU / CTB, one or more virtual pipeline data units (VPDU), or a sub-region within a picture / strip / slice / brick. The term "reference row" can refer to a row and / or column of reconstructed samples that are adjacent or non-adjacent to the current block, used to derive intra-prediction of the current video unit along a specific direction via an interpolation filter, and the specific direction is determined by an intra-prediction mode (e.g., conventional intra-prediction with an intra-prediction mode), or by deriving intra-prediction of the current video unit by weighting the reference samples of the reference row using a matrix or vector (e.g., MIP).

[0389] Figure 31 A flowchart of a method 3100 for video processing according to an embodiment of the present disclosure is shown. Method 3100 is implemented during the conversion between video units of a video and a bitstream of a video.

[0390] At box 3110, for the conversion between video units and bitstreams of video units, a set of local illumination compensation (LIC) parameters for the previously encoded and decoded video units is obtained. In some embodiments, a video unit includes at least one of the following: color components, prediction blocks (PB), transform blocks (TB), codec blocks (CB), prediction units (PU), transform units (TU), codec tree blocks (CTB), codec units (CU), codec tree units (CTU), CTU rows, CTU groups, stripes, slices, sub-pictures, blocks, sub-regions within blocks, or regions containing more than one sample point or pixel.

[0391] At box 3120, a set of LIC parameters are applied to the video unit.

[0392] At box 3130, a conversion is performed based on a set of LIC parameters. In some embodiments, the conversion may include encoding video units into a bitstream. Alternatively or additionally, the conversion may include decoding video units from the bitstream. In this way, the encoding / decoding efficiency and performance of intra-frame TMPs can be improved.

[0393] In some embodiments, in addition to mode information and motion information, a set of LIC parameters is also stored in a cache. In some embodiments, a set of LIC parameters is stored in association with motion information for each video unit. In some embodiments, a set of LIC parameters is stored in M×N units, where M and N are integers. In some embodiments, a set of LIC parameters is stored in a lookup table. In some embodiments, a set of LIC parameters is stored in a local cache representing data within one of the following: current block, codec unit (CU), prediction unit (PU), transform unit (TU), virtual pipeline data unit (VPDU), codec tree unit (CTU), CTU row, slice, slice group, stripe, subpicture, or picture. In some embodiments, a set of LIC parameters is stored in a temporal cache representing data for a temporal reference unit.

[0394] In some embodiments, a motion candidate includes a set of LIC parameters, and a set of LIC parameters associated with the decoded motion candidate is used for the current block. In some embodiments, which LIC parameters are used for the current block is based on the decoded motion candidate. In some embodiments, the motion candidate is derived based on previously encoded / decoded blocks.

[0395] In some embodiments, motion candidates are derived based on neighboring blocks. In some embodiments, motion candidates are derived based on non-neighboring blocks.

[0396] In some embodiments, motion candidates are derived based on temporal blocks in a reference image. In some embodiments, motion candidates are derived based on a history-based motion vector prediction (HMVP) table.

[0397] In some embodiments, motion candidates are generated based on the encoding and decoding information of multiple video units. In some embodiments, the motion information of the first two candidates is combined based on predefined rules.

[0398] In some embodiments, motion candidates are candidates from one of the following: inter-frame merge list, affine merge list, template matching (TM) merge list, bilateral matching (BM) merge list, geometric segmentation mode (GPM) merge list, inter-frame intra-frame joint prediction (CIIP) merge list, sub-block-based temporal motion vector prediction (SbTMVP) merge list, multiple hypothesis prediction (MHP) merge list, or intra-frame block copy (IBC) merge list. In some embodiments, motion candidates are candidates from one of the following: inter-frame advanced motion vector prediction (AMVP) list, affine AMVP list, AMVP merge list, MHPAMVP list, or IBC AMVP list.

[0399] In some embodiments, motion candidates are candidates in the LIC motion candidate list. In some embodiments, modifications to a set of LIC parameters associated with the decoded motion candidates are applied to derive the final LIC parameters for the current block.

[0400] In some embodiments, a candidate list for LIC parameter derivation is generated for a LIC-encoded block. In some embodiments, all candidates in the candidate list are LIC-encoded. In some embodiments, the candidates in the candidate list come from at least one of the following: neighboring neighbors, non-neighboring neighbors, time-domain blocks, or HMVP tables.

[0401] In some embodiments, dummy candidates are inserted into a candidate list for LIC parameters. For example, dummy candidates are generated based on at least two existing LIC candidates.

[0402] In some embodiments, at least one candidate indication LIC parameter is dynamically calculated. For example, the LIC parameter is calculated based on the relationship between the current template and a reference template.

[0403] In some embodiments, candidate indices are transmitted via signaling in a bitstream to indicate how LIC parameters for the current block should be generated. In some embodiments, the first set of LIC parameters is compared with at least one set of LIC parameters in the candidate list of LIC parameters before being added to the candidate list. For example, if the first set of LIC parameters is the same as or similar to a set of LIC parameters in the candidate list, the first set of LIC parameters is not added to the candidate list of LIC parameters.

[0404] In some embodiments, the similarity between two sets of LIC parameters is measured based on predefined rules. For example, the method of measuring similarity depends on the block dimension. In some embodiments, the block dimension includes at least one of the width or height of the block. In some embodiments, the method of measuring similarity depends on the order or index of a set of LIC parameters in a candidate list for LIC parameters.

[0405] In some embodiments, a LIC HMVP table is maintained. In some embodiments, the LIC parameters of previously encoded / decoded LIC blocks are stored in the LIC HMVP table. In some embodiments, the LIC parameters of previously encoded / decoded blocks are stored together with regular motion information in a regular HMVP table.

[0406] In some embodiments, the LIC parameter is represented as a pair of scaling factors and offset values ​​in the HMVP table. In some embodiments, the HMVP table contains multiple pairs of scaling factors and offset values. In some embodiments, the scaling factor and offset value of the LIC parameter are represented independently. For example, two HMVP tables are maintained, one of which includes the scaling factor, and the other of which includes the offset value.

[0407] In some embodiments, the LIC parameter for the current block is a combination of scaling factors and offset values ​​from two HMVP tables. For example, the LIC parameter is represented as {scaleX, offsetY}, where X is not equal to Y.

[0408] In some embodiments, the length of the LIC HMVP table is equal to N. For example, N is a constant value. In some embodiments, N is equal to one of the following: 5, 6, 7, 8, 9, or 10. In some embodiments, N is adapted to codec information. In some embodiments, the contents of the LIC HMVP table are reset at one of the following: picture level, sub-picture level, slice level, slice group level, CTU line level, or stripe level.

[0409] In some embodiments, the first set of LIC parameters is compared with at least one set of LIC parameters in the LIC HMVP table before being added to the LIC HMVP table. For example, if the first set of LIC parameters is the same as or similar to a set of LIC parameters in the LIC HMVP table, then the first set of LIC parameters is not added to the LIC HMVP table.

[0410] In some embodiments, the similarity between two sets of LIC parameters is measured based on predefined rules. In some embodiments, the similarity is measured based on the block dimension. In some other embodiments, the similarity is measured based on the order or index of a set of LIC parameters in the LIC HMVP table.

[0411] In some embodiments, whether a set of LIC parameters is evaluated or inherited is based on predefined rules. For example, whether a set of LIC parameters is evaluated or inherited is indicated by one or more syntax elements. In some embodiments, whether a set of LIC parameters is evaluated or inherited is indicated by a pattern flag or index.

[0412] In some embodiments, a first flag indicates whether a LIC is used, and a second flag indicates the LIC mode type. For example, a first LIC mode type indicates that a set of LIC parameters is deduced through on-the-fly computation, and a second LIC mode type indicates that a set of LIC parameters is deduced through inheritance.

[0413] In some embodiments, it is determined whether a set of LIC parameters is computed or inherited from a set of LIC parameters inherited from a motion candidate. For example, if the LIC mode type of a motion candidate is decoded, it is determined whether a set of LIC parameters for the current block is computed or inherited from a set of LIC parameters of the motion candidate.

[0414] In some embodiments, the final prediction of a LIC-encoded block is generated by combining a LIC prediction with at least one prediction of a non-LIC-encoded block. In some other embodiments, the final prediction of a LIC-encoded block is generated by combining multiple intermediate LIC predictions. For example, at least one intermediate prediction is derived from inherited LIC parameters. In some embodiments, a first prediction is generated using inherited LIC parameters, and a second prediction is generated using dynamically computed LIC parameters. In some other embodiments, a first prediction is generated using a first set of inherited LIC parameters, and a second prediction is generated using a second set of inherited LIC parameters.

[0415] In some embodiments, the video unit is an AMVP block encoded and decoded by a LIC. For example, the video unit is a regular inter-frame AMVP block. In some embodiments, the video unit is an affine AMVP block. In some other embodiments, the video unit is an AMVP-MERGE block.

[0416] In some embodiments, the video unit is an MHP block with the basic assumption of being encoded and decoded using AMVP. In some other embodiments, the video unit is an IBC AMVP block.

[0417] In some embodiments, the video unit is a LIC-encoded MERGE block. For example, the video unit is a regular inter-frame MERGE block. In some embodiments, the video unit is an affine MERGE block. In some other embodiments, the video unit is one of the following: an sbTMVP block, a GPM block, a CIIP block, a TM block, a BM block, or an ADMVR MERGE block.

[0418] In some embodiments, the video unit is an MHP block with the basic assumption of MERGE encoding / decoding. In some other embodiments, the video unit is an IBC MERGE block.

[0419] In some embodiments, the video unit is a unidirectional prediction block encoded with LIC. In some other embodiments, the video unit is a bidirectional prediction block encoded with LIC. For example, the method is applied to both directions of the bidirectional prediction block. Alternatively, the method is applied to one direction of the bidirectional prediction block.

[0420] In some embodiments, syntax elements are binarized into one of the following: a flag, a fixed-length code, an EG(x) code, a unary code, a rounded unary code, or a rounded binary code. In some embodiments, syntax elements are signed or unsigned.

[0421] In some embodiments, syntax elements are bypassed for encoding and decoding. In some embodiments, syntax elements are encoded and decoded using at least one context model.

[0422] In some embodiments, syntax elements are transmitted conditionally via signals. For example, a syntax element is transmitted via a signal if the corresponding function applies.

[0423] In some embodiments, a set of LIC parameters includes only LIC parameters for the target component. In some embodiments, the target component is the Y component.

[0424] In some embodiments, a set of LIC parameters includes LIC parameters for multiple components. For example, different components may have different LIC parameters.

[0425] In some embodiments, an indication of whether and / or how a set of LIC parameters is obtained is given at one of the following: sequence level, picture group level, picture level, stripe level, or slice group level. In some other embodiments, an indication of whether and / or how a set of LIC parameters is obtained is given at one of the following: sequence header, picture header, sequence parameter set (SPS), video parameter set (VPS), dependency parameter set (DPS), decoding capability information (DCI), picture parameter set (PPS), adaptive parameter set (APS), stripe header, or slice group header. In some embodiments, an indication of whether and / or how a set of LIC parameters is obtained is included in one of the following: prediction block (PB), transform block (TB), codec block (CB), prediction unit (PU), transform unit (TU), codec unit (CU), virtual pipeline data unit (VPDU), codec tree unit (CTU), CTU row, stripe, slice, subpicture, or region containing more than one sample or pixel.

[0426] In some embodiments, whether and / or how a set of LIC parameters is obtained is determined based on the encoded information of the video unit. The encoded information may include at least one of the following: block size, color format, single-tree segmentation and / or dual-tree segmentation, color components, stripe type, or picture type. In some embodiments, the method of obtaining a set of LIC parameters is used in other codec tools that require solving linear or nonlinear regression models.

[0427] According to another embodiment of this disclosure, a non-transitory computer-readable recording medium is provided. This non-transitory computer-readable recording medium stores a bitstream of video generated by a method performed by an apparatus for video processing. The method includes: obtaining a set of local illumination compensation (LIC) parameters for previously encoded / decoded video units stored therein; applying the set of LIC parameters to the video units of the video; and generating a bitstream based on the set of LIC parameters.

[0428] According to further embodiments of this disclosure, a method for storing a bitstream of video is provided. The method includes: obtaining a set of local illumination compensation (LIC) parameters for previously encoded / decoded video units; applying the set of LIC parameters to the video units of the video; generating a bitstream based on the set of LIC parameters; and storing the bitstream in a non-transitory computer-readable recording medium.

[0429] Implementations of this disclosure may be described in accordance with the following entries, and its features may be combined in any reasonable manner.

[0430] Item 1. A method for video processing, comprising: converting between video units of a video and a bitstream of the video units to obtain a set of local illumination compensation (LIC) parameters of a previously encoded / decoded video unit stored therein; applying the set of LIC parameters to the video unit; and performing the conversion based on the set of LIC parameters. In this manner, encoding / decoding performance and efficiency are improved.

[0431] Item 2. The method according to Item 1, wherein, in addition to pattern information and motion information, the set of LIC parameters is also stored in a cache.

[0432] Item 3. The method according to Item 1, wherein the set of LIC parameters is stored in association with motion information for each video unit.

[0433] Item 4. According to the method described in Item 1, wherein the set of LIC parameters is stored in M×N units, where M and N are integers.

[0434] Item 5. The method described in Item 1, wherein the set of LIC parameters is stored in a lookup table.

[0435] Item 6. According to the method described in Item 1, wherein the set of LIC parameters is stored in a local cache, the local cache representing data within one of the following: current block, codec unit (CU), prediction unit (PU), transform unit (TU), virtual pipeline data unit (VPDU), codec tree unit (CTU), CTU row, slice, slice group, stripe, sub-picture, or picture.

[0436] Item 7. The method according to Item 1, wherein the set of LIC parameters is stored in a time-domain cache, the time-domain cache representing data for a time-domain reference cell.

[0437] Item 8. The method according to Item 1, wherein the motion candidate includes the set of LIC parameters, and the set of LIC parameters associated with the decoded motion candidate is used for the current block.

[0438] Item 9. The method described in Item 8, wherein which LIC parameters are used for the current block is based on the decoded motion candidate.

[0439] Item 10. The method according to Item 8, wherein the motion candidate is derived based on previously encoded and decoded blocks.

[0440] Item 11. The method according to Item 8, wherein the motion candidate is derived based on neighboring blocks.

[0441] Item 12. The method according to Item 8, wherein the motion candidate is derived based on non-adjacent blocks.

[0442] Item 13. The method according to Item 8, wherein the motion candidate is derived based on temporal blocks in a reference image.

[0443] Item 14. The method according to Item 8, wherein the motion candidate is derived from a history-based motion vector prediction (HMVP) table.

[0444] Item 15. The method according to Item 8, wherein the motion candidate is generated based on the encoding and decoding information of a plurality of video units.

[0445] Item 16. The method described in Item 8, wherein the motion information of the first two candidates is combined based on predefined rules.

[0446] Item 17. The method according to Item 8, wherein the motion candidate is a candidate of one of the following: inter-frame merge list, affine merge list, template matching (TM) merge list, bilateral matching (BM) merge list, geometric segmentation mode (GPM) merge list, inter-frame and intra-frame joint prediction (CIIP) merge list, sub-block-based temporal motion vector prediction (SbTMVP) merge list, multiple hypothesis prediction (MHP) merge list, or intra-frame block copy (IBC) merge list.

[0447] Item 18. The method according to Item 8, wherein the motion candidate is a candidate of one of the following: an inter-frame advanced motion vector prediction (AMVP) list, an affine AMVP list, an AMVP merge list, an MHP AMVP list, or an IBC AMVP list.

[0448] Item 19. The method according to Item 8, wherein the motion candidate is a candidate in the LIC motion candidate list.

[0449] Item 20. The method according to Item 8, wherein modifications to the set of LIC parameters associated with the decoded motion candidate are applied to derive the final LIC parameters for the current block.

[0450] Item 21. The method according to Item 1, wherein the candidate list for LIC parameter derivation is generated for LIC encoded / decoded blocks.

[0451] Item 22. The method according to Item 21, wherein all candidates in the candidate list are LIC encoded or decoded.

[0452] Item 23. The method according to Item 21, wherein the candidates in the candidate list are from at least one of the following: neighboring neighbors, non-neighboring neighbors, time-domain blocks, or HMVP tables.

[0453] Item 24. The method according to Item 21, wherein a virtual candidate is inserted into the candidate list for the LIC parameter.

[0454] Item 25. The method according to Item 24, wherein the virtual candidate is generated based on at least two existing LIC candidates.

[0455] Item 26. The method according to Item 21, wherein at least one candidate indicator LIC parameter is dynamically calculated.

[0456] Item 27. The method according to Item 26, wherein the LIC parameter is calculated based on the relationship between the current template and the reference template.

[0457] Item 28. The method according to Item 21, wherein the candidate index is transmitted by signaling in the bitstream to indicate how the LIC parameter for the current block is generated.

[0458] Item 29. The method according to Item 21, wherein the first set of LIC parameters is compared with at least one set of LIC parameters in the candidate list for LIC parameters before the first set of LIC parameters is placed in the candidate list for LIC parameters.

[0459] Item 30. The method according to Item 29, wherein if the first set of LIC parameters is the same as or similar to a set of LIC parameters in the candidate list for LIC parameters, then the first set of LIC parameters is not placed in the candidate list for LIC parameters.

[0460] Item 31. The method described in Item 29, wherein the similarity of two sets of LIC parameters is measured based on predefined rules.

[0461] Item 32. The method according to Item 31, wherein the manner in which the similarity is measured depends on the block dimension.

[0462] Item 33. The method according to Item 32, wherein the block dimension includes at least one of the width or height of the block.

[0463] Item 34. The method according to Item 31, wherein the manner of measuring the similarity depends on the order or index of a set of LIC parameters in the candidate list for the LIC parameters.

[0464] Item 35. The method described in Item 1, wherein the LIC HMVP table is maintained.

[0465] Item 36. The method according to Item 35, wherein the LIC parameters of the previously encoded / decoded LIC block are stored in the LIC HMVP table.

[0466] Item 37. The method according to Item 35, wherein the LIC parameters of the previously encoded / decoded block are stored together with the regular motion information in a regular HMVP table.

[0467] Item 38. The method described in Item 35, wherein in the HMVP table, the LIC parameter is represented as a pair of scaling factors and offset values.

[0468] Item 39. The method according to Item 38, wherein the HMVP table contains multiple pairs of scaling factors and offset values.

[0469] Item 40. The method according to Item 35, wherein the scaling factor and offset value of the LIC parameter are represented independently.

[0470] Item 41. The method according to Item 40, wherein two HMVP tables are maintained, one of the two HMVP tables includes a scaling factor, and the other of the two HMVP tables includes an offset value.

[0471] Item 42. The method according to Item 40, wherein the LIC parameter for the current block is composed of a combination of scaling factors and offset values ​​from the two HMVP tables.

[0472] Item 43. The method according to Item 42, wherein the LIC parameter is represented as {scaleX,offsetY}, where X is not equal to Y.

[0473] Item 44. The method according to Item 35, wherein the length of the LIC HMVP table is equal to N.

[0474] Item 45. The method described according to Item 44, wherein N is a constant value.

[0475] Item 46. The method according to Item 45, wherein N equals one of the following: 5, 6, 7, 8, 9, or 10.

[0476] Item 47. The method according to Item 44, wherein N is adapted to the encoding / decoding information.

[0477] Item 48. The method described in Item 35, wherein the contents of the LIC HMVP table are reset at one of the following levels: image level, sub-image level, slice level, slice group level, CTU row level, or stripe level.

[0478] Item 49. The method according to Item 35, wherein the first set of LIC parameters is compared with at least one set of LIC parameters in the LIC HMVP table before the first set of LIC parameters is placed into the LIC HMVP table.

[0479] Item 50. The method according to Item 49, wherein if the first set of LIC parameters is the same as or similar to a set of LIC parameters in the LIC HMVP table, then the first set of LIC parameters is not placed in the LIC HMVP table.

[0480] Item 51. The method described in Item 49, wherein the similarity between two sets of LIC parameters is measured based on predefined rules.

[0481] Item 52. The method according to Item 51, wherein the manner in which the similarity is measured depends on the block dimension.

[0482] Item 53. The method according to Item 51, wherein the manner in which the similarity is measured depends on the order or index of a set of LIC parameters in the LIC HMVP table.

[0483] Item 54. The method described in Item 1, wherein the set of LIC parameters is calculated or inherited based on a predefined rule.

[0484] Item 55. The method according to Item 54, wherein whether to compute the set of LIC parameters or to inherit the set of LIC parameters is indicated by one or more syntax elements.

[0485] Item 56. The method according to Item 54, wherein whether the set of LIC parameters is calculated or inherited is indicated by a pattern flag or index.

[0486] Item 57. The method according to Item 54, wherein the first flag indicates whether a LIC is used, and the second flag indicates the LIC mode type.

[0487] Item 58. The method according to Item 57, wherein a first LIC mode type indicates that the set of LIC parameters is derived by on-the-fly computation, and a second LIC mode type indicates that the set of LIC parameters is derived by inheritance.

[0488] Item 59. The method according to Item 54, wherein the set of LIC parameters is calculated or inherited from the motion candidate.

[0489] Item 60. The method according to Item 59, wherein if the LIC mode type of the motion candidate is decoded, whether the computation or inheritance of the set of LIC parameters for the current block is based on whether the set of LIC parameters of the motion candidate is computed or inherited.

[0490] Item 61. The method according to Item 1, wherein the final prediction of a LIC-encoded block is generated by combining the LIC prediction with at least one prediction of a non-LIC-encoded block.

[0491] Item 62. The method according to Item 1, wherein the final prediction of the LIC-encoded block is generated by combining multiple intermediate LIC predictions.

[0492] Item 63. The method according to Item 62, wherein at least one intermediate prediction is derived from the inherited LIC parameter.

[0493] Item 64. The method according to Item 62, wherein the first prediction is generated by inherited LIC parameters and the second prediction is generated by dynamically calculated LIC parameters.

[0494] Item 65. The method according to Item 62, wherein the first prediction is generated by a first set of inherited LIC parameters, and the second prediction is generated by a second set of inherited LIC parameters.

[0495] Item 66. The method according to any one of items 1 to 65, wherein the video unit is an AMVP block encoded and decoded by LIC.

[0496] Item 67. The method according to Item 66, wherein the video unit is a regular inter-frame AMVP block.

[0497] Item 68. The method of claim 66, wherein the video unit is an affine AMVP block.

[0498] Item 69. The method according to Item 66, wherein the video unit is an AMVP-Merge block.

[0499] Item 70. The method according to Item 66, wherein the video unit is an MHP block having the basic assumption of being encoded and decoded by AMVP.

[0500] Item 71. The method according to Item 66, wherein the video unit is an IBC AMVP block.

[0501] Item 72. The method according to any one of items 1-65, wherein the video unit is a merge block encoded and decoded by LIC.

[0502] Item 73. The method according to Item 72, wherein the video unit is a regular inter-frame merge block.

[0503] Item 74. The method according to Item 72, wherein the video unit is an affine Merge block.

[0504] Item 75. The method according to Item 72, wherein the video unit is one of the following: sbTMVP block, GPM block, CIIP block, TM block, BM block or ADMVR Merge block.

[0505] Item 76. The method according to Item 72, wherein the video unit is an MHP block having the basic assumption of being encoded and decoded via Merge.

[0506] Item 77. The method according to Item 72, wherein the video unit is an IBC Merge block.

[0507] Item 78. The method according to any one of items 1 to 65, wherein the video unit is a unidirectional prediction block encoded and decoded by LIC.

[0508] Item 79. The method according to any one of items 1 to 65, wherein the video unit is a bidirectional prediction block encoded and decoded by LIC.

[0509] Item 80. The method according to Item 79, wherein the method is applied to both directions of the bidirectional prediction block.

[0510] Item 81. The method according to Item 79, wherein the method is applied to one direction of a bidirectional prediction block.

[0511] Item 82. The method according to any one of items 1 to 81, wherein the syntax element is binarized into one of the following: a flag, a fixed-length code, an EG(x) code, a unary code, a rounded unary code, or a rounded binary code.

[0512] Item 83. The method according to Item 82, wherein the syntax element is signed or unsigned.

[0513] Item 84. The method according to any one of items 1 to 81, wherein the syntax element is bypassed or wherein the syntax element is encoded or decoded using at least one context model.

[0514] Item 85. The method according to any one of items 1 to 81, wherein the syntax element is transmitted via signal in a conditional manner.

[0515] Item 86. The method according to Item 85, wherein the syntax element is transmitted via a signal if the corresponding function is applicable.

[0516] Item 87. The method according to any one of items 1-86, wherein the set of LIC parameters includes only LIC parameters for the target component.

[0517] Item 88. The method according to Item 87, wherein the target component is the Y component.

[0518] Item 89. The method according to any one of items 1 to 86, wherein the set of LIC parameters includes LIC parameters for multiple components.

[0519] Item 90. The method described in Item 89, wherein different components have different LIC parameters.

[0520] Item 91. The method according to any one of items 1 to 90, wherein an indication of whether and / or how the set of LIC parameters is obtained is indicated at one of the following: sequence level, picture group level, picture level, strip level, or slice group level.

[0521] Item 92. The method according to any one of items 1 to 90, wherein an indication of whether and / or how the set of LIC parameters is obtained is indicated in one of the following: sequence header, picture header, sequence parameter set (SPS), video parameter set (VPS), dependency parameter set (DPS), decoding capability information (DCI), picture parameter set (PPS), adaptive parameter set (APS), strip header or slice header.

[0522] Item 93. The method according to any one of items 1 to 90, wherein an indication of whether and / or how the set of LIC parameters is obtained is included in one of the following: prediction block (PB), transform block (TB), codec block (CB), prediction unit (PU), transform unit (TU), codec unit (CU), virtual pipeline data unit (VPDU), codec tree unit (CTU), CTU row, strip, slice, sub-picture, or region containing more than one sample point or pixel.

[0523] Item 94. The method according to any one of items 1 to 90 further comprises: determining, based on the encoded and decoded information of the video unit, whether and / or how the set of LIC parameters is obtained, the encoded and decoded information including at least one of the following: block size, color format, single-tree segmentation and / or dual-tree segmentation, color components, stripe type, or picture type.

[0524] Item 95. The method according to any one of items 1 to 90, wherein the manner in which the set of LIC parameters is obtained is used in other encoding / decoding tools that need to solve linear or nonlinear regression models.

[0525] Item 96. The method according to any one of items 1 to 95, wherein the conversion includes encoding the video unit into the bitstream.

[0526] Item 97. The method according to any one of items 1 to 95, wherein the conversion comprises decoding the video unit from the bitstream.

[0527] Item 98. An apparatus for video processing, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform the method according to any one of items 1 to 97.

[0528] Item 99. A non-transitory computer-readable storage medium storing instructions that cause a processor to perform the method according to any one of items 1 to 97.

[0529] Item 100. A non-transitory computer-readable recording medium storing a bitstream of video generated by a method performed by means of a video processing apparatus, wherein the method includes: obtaining a set of local illumination compensation (LIC) parameters for previously encoded and decoded video units stored therein; applying the set of LIC parameters to the video units of the video; and generating the bitstream based on the set of LIC parameters.

[0530] Item 101. A method for storing a bitstream of video, comprising: obtaining a set of local illumination compensation (LIC) parameters for previously encoded and decoded video units stored therein; applying the set of LIC parameters to the video units of the video; generating the bitstream based on the set of LIC parameters; and storing the bitstream in a non-transitory computer-readable recording medium.

[0531] Example device Figure 32A block diagram of a computing device 3200 in which various embodiments of the present disclosure may be implemented is shown. The computing device 3200 may be implemented as a source device 110 (or video encoder 114 or 200) or a destination device 120 (or video decoder 124 or 300), or may be included in a source device 110 (or video encoder 114 or 200) or a destination device 120 (or video decoder 124 or 300).

[0532] It should be understood that, Figure 8 , Figure 11 and Figure 32 The computing device 3200 shown is for illustrative purposes only and is not intended to imply any limitation on the functionality and scope of the embodiments of this disclosure.

[0533] like Figure 8 , Figure 11 and Figure 32 As shown, computing device 3200 includes general-purpose computing device 3200. Computing device 3200 may include at least one or more processors or processing units 3210, memory 3220, storage unit 3230, one or more communication units 3240, one or more input devices 3250, and one or more output devices 3260.

[0534] In some embodiments, the computing device 3200 can be implemented as any user terminal or server terminal with computing capabilities. The server terminal can be a server provided by a service provider, a large computing device, etc. The user terminal can be, for example, any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, stations, units, devices, multimedia computers, multimedia tablet computers, internet nodes, communicators, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio receivers, e-book devices, gaming devices, or any combination thereof, and includes accessories and peripherals of these devices, or any combination thereof. It is conceivable that the computing device 3200 can support any type of interface to the user (such as "wearable" circuitry devices, etc.).

[0535] Processing unit 3210 can be a physical processor or a virtual processor, and can perform various processes based on programs stored in memory 3220. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capabilities of computing device 3200. Processing unit 3210 may also be referred to as a central processing unit (CPU), microprocessor, controller, or microcontroller.

[0536] Computing device 3200 typically includes various computer storage media. Such media can be any media accessible by computing device 3200, including but not limited to volatile and non-volatile media, or removable and non-removable media. Memory 3220 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (such as read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), or flash memory) or any combination thereof. Storage cell 3230 can be any removable or non-removable media and may include machine-readable media, such as memory, flash drives, disks, or other media that can be used to store information and / or data and can be accessed within computing device 3200.

[0537] The computing device 3200 may also include additional removable / non-removable storage media, volatile / non-volatile storage media. Although in Figure 8 , Figure 11 and Figure 32 Not shown, but may provide disk drives for reading from and / or writing to removable non-volatile disks, and optical disc drives for reading from and / or writing to removable non-volatile optical discs. In this case, each drive may be connected to a bus (not shown) via one or more data media interfaces.

[0538] Communication unit 3240 communicates with another computing device via a communication medium. Furthermore, the functionality of the components in computing device 3200 can be implemented by a single computing cluster or by multiple computing machines communicating via communication connections. Therefore, computing device 3200 can operate in a networked environment using logical connections to one or more other servers, networked personal computers (PCs), or other general-purpose network nodes.

[0539] Input device 3250 can be one or more of various input devices, such as a mouse, keyboard, trackball, voice input device, etc. Output device 3260 can be one or more of various output devices, such as a monitor, speaker, printer, etc. With the aid of communication unit 3240, computing device 3200 can also communicate with one or more external devices (not shown), such as storage devices and display devices. Computing device 3200 can also communicate with one or more devices that enable a user to interact with computing device 3200, or any device that enables computing device 3200 to communicate with one or more other computing devices (e.g., network card, modem, etc.), if needed. Such communication can be performed via an input / output (I / O) interface (not shown).

[0540] In some embodiments, some or all components of computing device 3200 may not be integrated into a single device, but may be deployed in a cloud computing architecture. In a cloud computing architecture, components may be provided remotely and work together to achieve the functionality described herein. In some embodiments, cloud computing provides computing, software, data access, and storage services without requiring end users to know the physical location or configuration of the systems or hardware providing these services. In various embodiments, cloud computing provides services via a wide area network (WAN), such as the Internet, using suitable protocols. For example, a cloud computing provider offers applications via a WAN that can be accessed through a web browser or any other computing component. The software or components of the cloud computing architecture, along with the corresponding data, may be stored on servers at a remote location. Computing resources in a cloud computing environment may be consolidated or distributed at locations in remote data centers. Cloud computing infrastructure may provide services through shared data centers, although they may appear as a single access point for users. Therefore, cloud computing architectures can be used to provide the components and functionality described herein from service providers at remote locations. Alternatively, they may be provided from conventional servers, or directly installed or otherwise installed on client devices.

[0541] In embodiments of this disclosure, computing device 3200 can be used to implement video encoding / decoding. Memory 3220 may include one or more video codec modules 3225 having one or more program instructions. These modules can be accessed and executed by processing unit 3210 to perform the functions of the various embodiments described herein.

[0542] In an example embodiment of performing video encoding, input device 3250 may receive video data as input 3270 to be encoded. The video data may be processed, for example, by video codec module 3225 to generate an encoded bitstream. The encoded bitstream may be provided as output 3280 via output device 3260.

[0543] In an example embodiment of performing video decoding, input device 3250 may receive an encoded bitstream as input 3270. The encoded bitstream may be processed, for example, by a video codec module 3225 to generate decoded video data. The decoded video data may be provided as output 3280 via output device 3260.

[0544] While this disclosure has been specifically shown and described with reference to preferred embodiments, those skilled in the art will understand that various changes in form and detail may be made without departing from the spirit and scope of this application as defined by the appended claims. These changes are intended to be covered by the scope of this application. Therefore, the foregoing description of embodiments of this application is not intended to be limiting.

Claims

1. A method for video processing, comprising: for a conversion between a video unit of a video and a bitstream of the video unit, obtaining a set of local illumination compensation (LIC) parameters of a previously coded video unit stored; applying the set of LIC parameters to the video unit; and performing the conversion based on the set of LIC parameters.

2. The method of claim 1, wherein the set of LIC parameters is stored in a cache in addition to mode information and motion information.

3. The method of claim 1, wherein the set of LIC parameters is stored in association with motion information for each video unit.

4. The method of claim 1, wherein the set of LIC parameters is stored in MxN units, where M and N are integers.

5. The method of claim 1, wherein the set of LIC parameters is stored in a lookup table.

6. The method of claim 1, wherein the set of LIC parameters is stored in a local cache, the local cache representing data within a current block, a coding unit (CU), a prediction unit (PU), a transform unit (TU), a virtual pipeline data unit (VPDU), a coding tree unit (CTU), a CTU row, a tile, a tile group, a slice, a subpicture, or a picture.

7. The method of claim 1, wherein the set of LIC parameters is stored in a temporal cache, the temporal cache representing data for a temporal reference unit.

8. The method of claim 1, wherein a motion candidate includes the set of LIC parameters, and the set of LIC parameters associated with a decoded motion candidate is used for a current block.

9. The method of claim 8, wherein which LIC parameters are used for the current block is based on the decoded motion candidate.

10. The method of claim 8, wherein the motion candidate is derived based on a previously coded block.

11. The method of claim 8, wherein the motion candidate is derived based on a neighboring neighboring block.

12. The method of claim 8, wherein the motion candidate is derived based on a non- neighboring block.

13. The method of claim 8, wherein the motion candidate is derived based on a temporal block in a reference picture.

14. The method of claim 8, wherein the motion candidate is derived from a history-based motion vector prediction (HMVP) table.

15. The method of claim 8, wherein the motion candidate is generated from coding information of a plurality of video units.

16. The method of claim 8, wherein motion information of the first two candidates is combined based on a predefined rule.

17. The method of claim 8, wherein the motion candidate is a candidate in one of the following: an inter MERGE list, an affine MERGE list, a template matching (TM) MERGE list, a bilateral matching (BM) MERGE list, a geometric partition mode (GPM) MERGE list, a Inter Intra Combined Prediction (CIIP) MERGE list, Subblock-based Temporal Motion Vector Prediction (SbTMVP) MERGE list, Multi-hypothesis prediction (MHP) MERGE list, or Intra Block Copy (IBC) MERGE list.

18. The method of claim 8, wherein the motion candidate is a candidate in one of: Inter Advanced Motion Vector Prediction (AMVP) list, Affine AMVP list, AMVP Merge list, MHP AMVP list, or IBC AMVP list.

19. The method of claim 8, wherein the motion candidate is a candidate in a LIC motion candidate list.

20. The method of claim 8, wherein a modification to the set of LIC parameters associated with a decoded motion candidate is applied to derive final LIC parameters for the current block.

21. The method of claim 1, wherein a candidate list for LIC parameter derivation is generated for a LIC coded block.

22. The method of claim 21, wherein candidates in the candidate list are all LIC coded.

23. The method of claim 21, wherein candidates in the candidate list are from at least one of: a neighboring neighbor, a non-neighboring neighbor, a temporal block, or a HMVP table.

24. The method of claim 21, wherein a virtual candidate is inserted into the candidate list for LIC parameters.

25. The method of claim 24, wherein the virtual candidate is generated based on at least two of existing LIC candidates.

26. The method of claim 21, wherein at least one candidate indicates that LIC parameters are dynamically computed.

27. The method of claim 26, wherein the LIC parameters are computed according to a relationship between a current template and the reference template.

28. The method of claim 21, wherein a candidate index is signaled in the bitstream to indicate how the LIC parameters for a current block are generated.

29. The method of claim 21, wherein a first set of LIC parameters is compared with at least one set of LIC parameters in the candidate list for LIC parameters before the first set of LIC parameters is put into the candidate list for LIC parameters.

30. The method of claim 29, wherein if the first set of LIC parameters is identical or similar to a set of LIC parameters in the candidate list for LIC parameters, the first set of LIC parameters is not put into the candidate list for LIC parameters.

31. The method of claim 29, wherein a way of measuring similarity of two sets of LIC parameters is based on a predefined rule.

32. The method of claim 31, wherein the way of measuring the similarity depends on a block dimension.

33. The method of claim 32, wherein the block dimension includes at least one of a width or a height of a block.

34. The method of claim 31, wherein the way of measuring the similarity depends on an order or index of a set of LIC parameters in the candidate list for LIC parameters.

35. The method of claim 1, wherein an LIC HMVP table is maintained.

36. The method of claim 35, wherein LIC parameters of previously coded LIC blocks are stored in the LIC HMVP table.

37. The method of claim 35, wherein LIC parameters of previously coded blocks are stored in a regular HMVP table together with regular motion information.

38. The method of claim 35, wherein in a HMVP table, an LIC parameter is represented as a pair of a scaling factor and an offset value.

39. The method of claim 38, wherein a HMVP table contains multiple pairs of scaling factors and offset values.

40. The method of claim 35, wherein a scaling factor and an offset value of an LIC parameter are represented independently.

41. The method of claim 40, wherein two HMVP tables are maintained, one of the two HMVP tables includes scaling factors, and the other of the two HMVP tables includes offset values.

42. The method of claim 40, wherein the LIC parameter for the current block is composed of a combination of a scaling factor and an offset value from the two HMVP tables.

43. The method of claim 42, wherein the LIC parameter is represented as {scaleX, offsetY}, where X is not equal to Y.

44. The method of claim 35, wherein a length of the LIC HMVP table is equal to N.

45. The method of claim 44, wherein N is a constant value.

46. The method of claim 45, wherein N is equal to one of: 5, 6, 7, 8, 9, or 10.

47. The method of claim 44, wherein N is adaptive to coding information.

48. The method of claim 35, wherein content of the LIC HMVP table is reset at one of: picture level, sub-picture level, tile level, tile group level, CTU row level, or slice level.

49. The method of claim 35, wherein before a first set of LIC parameters is put into the LIC HMVP table, the first set of LIC parameters is compared with at least one set of LIC parameters in the LIC HMVP table.

50. The method of claim 49, wherein if the first set of LIC parameters is identical or similar to a set of LIC parameters in the LIC HMVP table, the first set of LIC parameters is not put into the LIC HMVP table.

51. The method of claim 49, wherein a way of measuring similarity of two sets of LIC parameters is based on a pre-defined rule.

52. The method of claim 51, wherein the way of measuring the similarity depends on block dimension.

53. The method of claim 51, wherein the manner of measuring the similarity depends on an order or index of a set of LIC parameters in the LIC HMVP table.

54. The method of claim 1, wherein whether the set of LIC parameters is computed or inherited is based on a pre-defined rule.

55. The method of claim 54, wherein whether the set of LIC parameters is computed or inherited is indicated by one or more syntax elements.

56. The method of claim 54, wherein whether the set of LIC parameters is computed or inherited is indicated by a mode flag or index.

57. The method of claim 54, wherein a first flag indicates whether LIC is used and a second flag indicates a LIC mode type.

58. The method of claim 57, wherein a first LIC mode type indicates that the set of LIC parameters is derived by instant computation and a second LIC mode type indicates that the set of LIC parameters is derived by inheritance.

59. The method of claim 54, wherein whether the set of LIC parameters is computed or inherited is inherited from a motion candidate.

60. The method of claim 59, wherein if a LIC mode type of the motion candidate is decoded, whether the set of LIC parameters for the current block is computed or inherited is inherited according to whether the set of LIC parameters of the motion candidate is computed or inherited.

61. The method of any of claims 1-60, wherein the video unit is an AMVP block that is LIC coded.

62. The method of claim 61, wherein the video unit is a regular inter AMVP block.

63. The method of claim 61, wherein the video unit is an affine AMVP block.

64. The method of claim 61, wherein the video unit is an AMVP-MERGE block.

65. The method of claim 61, wherein the video unit is an MHP block with a basic assumption coded by AMVP.

66. The method of claim 61, wherein the video unit is an IBC AMVP block.

67. The method of any of claims 1-60, wherein the video unit is a MERGE block that is LIC coded.

68. The method of claim 67, wherein the video unit is a regular inter MERGE block.

69. The method of claim 67, wherein the video unit is an affine MERGE block.

70. The method of claim 67, wherein the video unit is one of: a sbTMVP block, a GPM block, a CIIP block, a TM block, a BM block, or an ADMVR MERGE block.

71. The method of claim 67, wherein the video unit is an MHP block with a basic assumption coded by MERGE.

72. The method of claim 67, wherein the video unit is an IBC MERGE block.

73. The method of any of claims 1 to 72, wherein the video unit is a LIC coded uni- directional prediction block.

74. The method of any of claims 1 to 72, wherein the video unit is a LIC coded bi- directional prediction block.

75. The method of claim 74, wherein the method is applied to both directions of the bi-directional prediction block.

76. The method of claim 74, wherein the method is applied to one direction of the bi- directional prediction block.

77. The method of claim 1, wherein the final prediction of the LIC coded block is generated by combining the LIC prediction with at least one prediction of a non-LIC coded block.

78. The method of claim 1, wherein the final prediction of the LIC coded block is generated by combining a plurality of intermediate LIC predictions.

79. The method of claim 78, wherein at least one intermediate prediction is derived from inherited LIC parameters.

80. The method of claim 78, wherein a first prediction is generated by inherited LIC parameters and a second prediction is generated by dynamically calculated LIC parameters.

81. The method of claim 78, wherein a first prediction is generated by a first set of inherited LIC parameters and a second prediction is generated by a second set of inherited LIC parameters.

82. The method of any of claims 1 to 81, wherein the syntax element is binarized as one of the following: a flag, a fixed length code, an EG(x) code, a unary code, a truncated unary code, or a truncated binary code.

83. The method of claim 82, wherein the syntax element is signed or unsigned.

84. The method of any of claims 1 to 81, wherein the syntax element is bypass coded, or wherein the syntax element is coded with at least one context model.

85. The method of any of claims 1 to 81, wherein the syntax element is signaled in a conditional manner.

86. The method of claim 85, wherein the syntax element is signaled if a corresponding functionality is applicable.

87. The method of any of claims 1 to 86, wherein the set of LIC parameters includes LIC parameters for only a target component.

88. The method of claim 87, wherein the target component is the Y component.

89. The method of any of claims 1 to 86, wherein the set of LIC parameters includes LIC parameters for a plurality of components.

90. The method of claim 89, wherein different components have different LIC parameters.

91. The method of any of claims 1 to 90, wherein an indication of whether and / or how the set of LIC parameters is obtained is indicated at one of the following: a sequence level, a group of pictures level, a picture level, a slice level, or a tile group level.

92. The method according to any one of claims 1 to 90, wherein an indication of whether and / or how to obtain the set of LIC parameters is indicated in one of the following: Sequence header, Image header, Sequence Parameter Set (SPS) Video Parameter Set (VPS) Dependency Parameter Set (DPS) Decoding Capability Information (DCI) Image Parameter Set (PPS) Adaptive Parameter Set (APS) strip head, or The beginning of the film.

93. The method according to any one of claims 1 to 90, wherein an indication of whether and / or how the set of LIC parameters is obtained is included in one of the following: Predicted blocks (PB). Transform block (TB) Code block (CB) Prediction Unit (PU) Transformer Unit (TU) Codec Unit (CU) Virtual Pipeline Data Unit (VPDU). Code-decode tree unit (CTU) CTU line, strip, piece, Sub-images, or A region containing more than one sample point or pixel.

94. The method according to any one of claims 1 to 90, further comprising: Based on the encoded and decoded information of the video unit, determine whether and / or how to obtain the set of LIC parameters, wherein the encoded and decoded information includes at least one of the following: Block size, Color format, Single-tree partitioning and / or dual-tree partitioning, Color components, Strip type, or Image type.

95. The method according to any one of claims 1 to 90, wherein the manner in which the set of LIC parameters is obtained is used in other encoding / decoding tools that need to solve linear or nonlinear regression models.

96. The method according to any one of claims 1 to 95, wherein the conversion comprises encoding the video unit into the bitstream.

97. The method according to any one of claims 1 to 95, wherein the conversion comprises decoding the video unit from the bitstream.

98. An apparatus for video processing, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 97.

99. A non-transitory computer-readable storage medium storing instructions that cause a processor to perform the method according to any one of claims 1 to 97.

100. A non-transitory computer-readable recording medium storing a bitstream of video generated by a method performed by means of a video processing apparatus, wherein the method includes: Obtain a set of Local Illumination Compensation (LIC) parameters for the previously encoded and decoded video units stored in the database; Apply the set of LIC parameters to the video units of the video; as well as The bitstream is generated based on the set of LIC parameters.

101. A method for storing a bitstream of video, comprising: Obtain a set of Local Illumination Compensation (LIC) parameters for the previously encoded and decoded video units stored in the database; Apply the set of LIC parameters to the video units of the video; The bitstream is generated based on the set of LIC parameters; as well as The bitstream is stored in a non-transitory computer-readable recording medium.

Images