Local illumination compensation model merging mode

Abstract

Apparatuses and methods including techniques for encoding and decoding video data are disclosed. The disclosed techniques provide for encoding and decoding an inter predicted video block in a local illumination compensation (LIC) model merge mode, including constructing a list of LIC model candidates, selecting an LIC model from the list of LIC model candidates, and applying the selected LIC model to a reference block of the video block to generate an illumination compensated prediction block.

Description

Cross-reference to related applications

[0001] This application claims the benefit of European Patent Application No. 23306575.4, filed on 22 September 2023, which is incorporated herein by reference in its entirety. Background Technology

[0002] In predictive video coding and decoding, the currently encoded (or decoded) video block from a video frame can be predicted based on a reference video block from another (reference) video frame that is temporally preceding or following the current video frame. To compensate for illumination variations across video frames, the prediction of the current video block can be corrected using a Local Illumination Compensation (LIC) process. In this LIC process, an LIC model is applied to a reference video block to generate an illumination-compensated predicted block. As implemented in recent video codecs, the LIC model is estimated based on template regions in the immediate neighborhood of the current video block and its reference video block. However, if the correlation between corresponding samples from these template regions does not represent the correlation between corresponding samples from the video block and its reference block, the estimated LIC model has limited ability to compensate for illumination variations, and therefore applying it may impair coding and decoding efficiency. Summary of the Invention

[0003] The aspects described in this disclosure describe a method for encoding video data. The method includes encoding video blocks of video data into a bitstream. Encoding the video blocks includes constructing a candidate list of LIC models, selecting a LIC model from the candidate list, and applying the selected LIC model to a reference block of the video block to generate a prediction block for the video block. The aspects described in this disclosure also describe a method for decoding video data. The method includes decoding video blocks of video data from a bitstream. Decoding the video blocks includes constructing a candidate list of LIC models, selecting a LIC model from the candidate list, and applying the selected LIC model to a reference block of the video block to generate a prediction block for the video block.

[0004] The aspects described in this disclosure describe an apparatus for encoding video data. The apparatus includes at least one processor and a memory storing instructions. When executed by the at least one processor, the instructions cause the apparatus to encode video blocks of video data into a bitstream. Encoding the video blocks includes constructing a candidate list of LIC models, selecting a LIC model from the candidate list, and applying the selected LIC model to a reference block of the video block to generate a prediction block for the video block. The aspects described in this disclosure also describe an apparatus for decoding video data. The apparatus includes at least one processor and a memory storing instructions. When executed by the at least one processor, the instructions cause the apparatus to decode video blocks of video data from a bitstream. Decoding the video blocks includes constructing a candidate list of LIC models, selecting a LIC model from the candidate list, and applying the selected LIC model to a reference block of the video block to generate a prediction block for the video block.

[0005] A further aspect described in this disclosure describes a non-transitory computer-readable medium comprising instructions executable by at least one processor to perform a method for encoding video data. The method includes encoding video blocks of video data into a bitstream. Encoding the video blocks includes constructing a candidate list of LIC models, selecting a LIC model from the candidate list, and applying the selected LIC model to a reference block of the video block to generate a prediction block for the video block. An aspect described in this disclosure also describes a non-transitory computer-readable medium comprising instructions executable by at least one processor to perform a method for decoding video data. The method includes decoding video blocks of video data from a bitstream. Decoding the video blocks includes constructing a candidate list of LIC models, selecting a LIC model from the candidate list, and applying the selected LIC model to a reference block of the video block to generate a prediction block for the video block.

[0006] This synopsis is intended to introduce some concepts in a simplified form, which will be further described in the detailed embodiments below. This synopsis is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to features that address any or all of the shortcomings pointed out in any part of this disclosure. Attached Figure Description

[0007] Figure 1 This is a block diagram of an exemplary system, according to which aspects of this embodiment can be implemented.

[0008] Figure 2 This is a block diagram of an exemplary video encoder, according to which aspects of this embodiment can be implemented.

[0009] Figure 3This is a block diagram of an exemplary video decoder, according to which aspects of this embodiment can be implemented.

[0010] Figure 4 This is a schematic diagram illustrating the derivation of the LIC model when operating in bidirectional prediction mode, based on which aspects of this embodiment can be implemented.

[0011] Figure 5 This is a schematic diagram illustrating the derivation of the LIC model when operating in sub-block mode, based on which aspects of this embodiment can be implemented.

[0012] Figure 6 This is a schematic diagram showing the location of adjacent blocks in space, based on which aspects of this embodiment can be implemented.

[0013] Figure 7 This is a schematic diagram showing the location of non-adjacent blocks in space, based on which aspects of this embodiment can be implemented.

[0014] Figure 8 This is a schematic diagram showing the location of a time-adjacent block, based on which aspects of this embodiment can be implemented.

[0015] Figure 9 This is a schematic diagram showing the position of the time-adjacent block during the shift, based on which aspects of this embodiment can be implemented.

[0016] Figure 10 This is a schematic diagram showing the location of non-adjacent blocks in space, based on which aspects of this embodiment can be implemented.

[0017] Figure 11 This is a flowchart illustrating an application of LIC mode, according to which aspects of this embodiment can be implemented.

[0018] Figure 12 This is a flowchart illustrating the application of the LIC model merging mode, according to which aspects of this embodiment can be implemented.

[0019] Figure 13 This is a schematic diagram illustrating a detailed version of the LIC model, according to which aspects of this embodiment can be implemented.

[0020] Figure 14 This is a schematic diagram illustrating another refinement of the LIC model, according to which aspects of this embodiment can be implemented.

[0021] Figure 15 This is a flowchart of an exemplary method for encoding video data, according to which aspects of this embodiment can be implemented.

[0022] Figure 16This is a flowchart of an exemplary method for decoding video data, according to which aspects of this embodiment can be implemented. Detailed Implementation

[0023] This disclosure discloses apparatus and methods for encoding and decoding video data. A LIC model merging mode is introduced here, which allows the LIC model to be inherited from neighboring blocks of video blocks predicted in an inter-frame prediction mode (i.e., inter-frame predicted blocks). References are then made to this disclosure. Figures 1 to 3 Describe traditional systems and methods for predictive video coding and decoding, then refer to Figures 4 to 16 Describe aspects of this disclosure.

[0024] Figure 1 A block diagram of an exemplary system 100 is shown. System 100 may be embodied as a device and may be configured to perform one or more of the aspects described in this application. Examples of such devices include, but are not limited to, a variety of electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. Components of system 100 may be embodied individually or in combination in integrated circuits, multiple integrated circuits, and / or discrete devices. For example, in at least one embodiment, the processing 110 and encoder / decoder 130 components of system 100 are distributed across multiple integrated circuits and / or discrete devices. In various embodiments, system 100 may be communicatively coupled to other systems or other electronic devices, for example, via a communication bus or through dedicated input and / or output ports.

[0025] System 100 includes at least one processor 110, which can be configured to execute instructions loaded thereon to implement various aspects, such as those described in this application. Processor 110 may include embedded memory, input and output interfaces, and various other circuitry known in the art. System 100 includes at least one memory 120, such as a volatile memory device and / or a non-volatile memory device. System 100 includes a storage device 140, which may include non-volatile memory and / or volatile memory, such as EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash memory, disk drive, and / or optical disk drive. Storage device 140 may be, for example, an internal storage device, an attached storage device, and / or a network-accessible storage device.

[0026] System 100 includes an encoder / decoder module 130 configured to process data to provide encoded or decoded video data. The encoder / decoder module 130 may include its own processor and memory. The encoder / decoder module 130 may be implemented as a separate element of system 100, or it may be incorporated into processor 110 as a combination of hardware and / or software, as known to those skilled in the art. Furthermore, the encoder / decoder module 130 represents a module that can be implemented in a separate device to perform encoding and / or decoding functions.

[0027] Program code to be loaded into processor 110 or encoder / decoder 130 to execute the various aspects described in this application may be stored in storage device 140 and subsequently loaded into memory 120 for execution by processor 110. According to various embodiments, one or more of processor 110, memory 120, storage device 140, and encoder / decoder module 130 may store one or more items of various kinds during the execution of the processes described in this application. Such stored items may include, but are not limited to, input video, decoded video or portions thereof, bitstreams, matrices, variables, operational logic, and intermediate or final results from processing equations, formulas, and operations.

[0028] In several embodiments, the memory within processor 110 and / or encoder / decoder 130 is used to store instructions and provide working memory for processing functions required during encoding or decoding. However, in other embodiments, external memory (wherein, for example, processor 110 or encoder / decoder module 130) may be used for one or more of these functions. External memory may be memory 120 and / or storage device 140, which may include, for example, dynamically volatile memory and / or non-volatile flash memory. In several embodiments, external non-volatile flash memory is used to store the television's operating system. In at least one embodiment, fast external dynamically volatile memory, such as RAM, is used as working memory for video encoding and decoding operations.

[0029] Inputs to the components of system 100 may be provided by a variety of input devices as indicated in box 105. Such input devices include, but are not limited to, (i) an RF section that receives, for example, RF signals transmitted over the air by a broadcaster, (ii) a composite input terminal (COMP), (iii) a USB input terminal, and / or (iv) an HDMI input terminal.

[0030] In various embodiments, the input device of block 105 has relevant corresponding input processing elements known in the art. For example, the RF section may be associated with elements suitable for: (i) selecting a desired frequency (also known as selecting a signal, or band-limiting a signal to a band), (ii) down-converting the selected signal, (iii) band-limiting it again to a narrower band to select, for example, a signal band, which may be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. The RF section in various embodiments includes one or more elements performing these functions, such as frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF section may include a tuner that performs some of these functions, such as down-converting a received signal to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or baseband. In one set-top box embodiment, the RF section and its associated input processing elements receive RF signals transmitted via a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and re-filtering to a desired frequency band. Various embodiments rearrange the order of the above (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions. Adding elements may include inserting elements between existing elements, such as inserting amplifiers and analog-to-digital converters. In many embodiments, the RF section includes an antenna.

[0031] Furthermore, the USB and / or HDMI terminals may include corresponding interface processors for connecting system 100 to other electronic devices across USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented as needed, for example, within a separate input processing integrated circuit or within processor 110. Similarly, aspects of USB or HDMI interface processing may be implemented as needed, either within a separate interface integrated circuit or within processor 110. The demodulated, error-corrected, and demultiplexed streams are provided to various processing elements (e.g., including processor 110 and an encoder / decoder 130 operating in combination with memory and storage elements) to process the data streams as needed for presentation on the output device.

[0032] Various components of system 100 can be provided within an integrated housing. Within the integrated housing, various components can be interconnected and transmit data therebetween using suitable connection means 115 (e.g., internal buses known in the art, including I2C buses, wiring, and printed circuit boards).

[0033] System 100 includes a communication interface 150 that enables communication with other devices via a communication channel 190. The communication interface 150 may include, but is not limited to, a transceiver configured to send and receive data via the communication channel 190. The communication interface 150 may include, but is not limited to, a modem or network interface card (NIC). The communication channel 190 may be implemented, for example, within a wired and / or wireless medium.

[0034] In various embodiments, data can be streamed to system 100 using a Wi-Fi network such as IEEE 802.11. In these embodiments, the Wi-Fi signal is received via communication channel 190 and a communication interface 150 adapted for Wi-Fi communication. Communication channel 190 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other over-the-top services to communicate. In other embodiments, data can be streamed to system 100 using a set-top box that delivers data via an HDMI connection to input box 105, or data can be streamed to system 100 using an RF connection to input box 105.

[0035] System 100 can provide output signals to a variety of output devices, including display device 165, audio device (e.g., speaker) 175, and other peripheral devices 185. In various exemplary embodiments, the other peripheral devices 185 include one or more of a standalone DVR, optical disc player, stereo system, lighting system, and other devices that provide functionality based on the output of system 100. In various embodiments, signaling such as AV.link, CEC, or other communication protocols is used to transmit control signals between system 100 and display device 165, audio device 175, or other peripheral devices 185, enabling device-to-device control with or without user intervention. Output devices can be communicatively coupled to system 100 via dedicated connections through corresponding interfaces 160, 170, and 180. Alternatively, output devices can be connected to system 100 via communication interface 150 using communication channel 190. Display device 165 and audio device 175 can be integrated into a single unit within an electronic device such as a television set, along with other components of system 100. In various embodiments, the display interface 160 includes a display driver, such as a timing controller (T Con) chip.

[0036] Alternatively, display device 165 and audio device 175 may be separate from one or more other components, for example, if the RF portion of input 105 is part of a separate set-top box. In various embodiments where display device 165 and audio device 175 are external components, the output signal may be provided via a dedicated output connection, such as an HDMI port, USB port, or COMP output.

[0037] Figure 2 A functional block diagram of an exemplary video encoder 200 is shown. The video encoder 200 can be constructed by... Figure 1 The system 100 described herein employs [the following]. For example, the video encoder 200 may be an encoder that operates according to encoding standards such as Advanced Video Coding (AVC, H.264 / MPEG-4 | ISO / IEC 14496-10), High Efficiency Video Coding (HEVC, ITU-T H.265 | ISO / IEC 23008-2), or Multifunctional Video Coding (VVC, standard ITU-T H.266, ISO / IEC 23090-3, 2020).

[0038] Before encoding, the video data may be preprocessed by a pre-encoding processor (not shown). This preprocessing may include applying color model transformations to the color components of the input video frames (e.g., converting from RGB 4:4:4 to YCbCr 4:2:0) or mapping the color components of the input video frames to obtain a more robust signal distribution for compression (e.g., applying a histogram equalizer and / or denoising filter to the color components of one or more video frames). Preprocessing may also include associating metadata with the video data, which may be appended to the encoded video bitstream.

[0039] In encoder 200, video frames are encoded by encoder elements, as generally described below. The original video images (frames) to be encoded are divided into coding units (i.e., raw blocks) by image partitioner 202. Typically, a coding unit (CU) contains a luma block and a corresponding chroma block; therefore, generally, the operations applied to the CU are applied to the luma block and the corresponding chroma block. After partitioning 202, each CU can be encoded using either intra-frame prediction mode or inter-frame prediction mode. In intra-frame prediction mode, CU prediction is performed by intra-frame predictor 260. In intra-frame prediction mode, the content of a CU within a frame is predicted based on the content of one or more other CUs from the same frame using reconstructed versions of those other CUs (obtainable from the output of adder 255). In inter-frame prediction mode, motion estimation and motion compensation are performed by motion estimator 275 and motion compensator 270, respectively. In inter-frame prediction mode, the content of a CU within a frame is predicted based on the content of one or more other CUs from neighboring frames using reconstructed versions of those other CUs (obtainable from the reference image buffer 280). The encoder 205 determines which prediction result (one obtained through operations in intra-frame prediction mode 260, and one obtained through operations in inter-frame prediction modes 270, 275) to use to encode the CU, and indicates the selected prediction mode, for example, through a prediction mode flag. The selected prediction result can then be enhanced (e.g., filtered) by prediction enhancer 285 to output the corresponding prediction block. Once a prediction block has been generated for each CU, the corresponding residual block is computed, for example, by subtracting the predicted CU (i.e., the prediction block) from the CU (i.e., the original block).

[0040] Then, the corresponding residual block or its partition (i.e., transform block) of the CU is transformed into a coefficient block by transformer 220—that is, the residual samples of the transform block are transformed into the transform coefficients of the coefficient block. The resulting coefficient block is quantized by quantizer 230. Next, entropy encoder 245 is used to entropy encode the quantized coefficient block and the corresponding coding parameters (e.g., syntax elements including motion vectors and other control data). Thus, the entropy-encoded quantized coefficient block and the corresponding coding parameters associated with each video frame of the original video are packed into the bitstream of the encoded video data.

[0041] Simultaneously with the encoding of the original video blocks as described above, encoder 200 reconstructs the encoded original blocks to provide a reference for future predictions. Therefore, the quantized coefficient blocks (provided by quantizer 230) are inversely quantized by inverse quantizer 240 and then inversely transformed by inverse transformer 250 to reconstruct (decode) the residual blocks of the corresponding original blocks. The reconstructed residual blocks are added 255 times to the corresponding prediction blocks to obtain the corresponding reconstructed original blocks. Loop filters 265 can then be applied to the reconstructed images (formed from the reconstructed original blocks), such as performing deblocking filtering and / or Sample Adaptive Offset (SAO) filtering to reduce coding artifacts. The filtered reconstructed images can then be stored in reference image buffer 280 for future predictions in inter-frame prediction mode. Therefore, encoder 200 also performs decoding operations 240 and 250 to reconstruct the encoded images (frames). The reconstructed images can then be stored in reference image buffer 280 and used to facilitate motion estimation 275 and compensation 270, as described above.

[0042] Figure 3 A functional block diagram of an exemplary video decoder 300 is shown. The video decoder 300 can be composed of... Figure 1 The system 100 described herein employs the following approach. Generally, the operational aspects of the video decoder 300 are inverses of those of the video encoder 200. In the decoder 300, the bitstream of encoded video data generated by the video encoder 200 is first entropy-decoded by the entropy decoder 330 to decode quantized coefficient blocks and various coding parameters from the bitstream. The quantized coefficient blocks are dequantized by the inverse quantizer 340 and then inverse-transformed by the inverse transformer 350 to decode (reconstruct) the corresponding residual blocks. The reconstructed residual blocks are added 355 to the corresponding prediction blocks to obtain the corresponding reconstructed original blocks. Depending on the selected prediction mode, the predicted original blocks 370 can be obtained from the intra-frame predictor 360 or from the motion compensator 375, and can then be enhanced (e.g., filtered) by the prediction enhancer 390 to generate prediction blocks. A loop filter 365 can be applied to the reconstructed picture (formed from the reconstructed original blocks) to output the reconstructed (decoded) video frame. The filtered reconstructed picture is also stored in the reference picture buffer 380 to facilitate motion compensation 375.

[0043] A post-decoding processor (not shown) can further process the reconstructed video. For example, post-decoding processing may include inverse color model transformation (e.g., a conversion from YCbCr 4:2:0 to RGB 4:4:4) or inverse mapping to reverse the mapping process performed by the pre-encoding processor. The post-decoding processor may use metadata derived by the pre-encoding processor and / or signaled in the video bitstream.

[0044] The aspects disclosed herein are described with reference to video blocks (i.e., blocks) of video images (i.e., frames). However, the described aspects can similarly be applied to any region of a video frame to which the encoder 200 or decoder 300 may apply encoding tools. Generally, the aspects described herein can be applied to video data regions of any shape or size, formed by video partitions. A video region may be a coding tree unit (CTU), or a CU (to which inter-frame prediction or intra-frame prediction may be applied), or a partition thereof, each of which may include a luminance component. Y and chromaticity components Cr and Cb (This text also uses) C express).

[0045] LIC has been introduced into recent video codecs. LIC aims to compensate for local illumination differences across frames, which can lead to discrepancies between the values ​​of corresponding pixels in the block currently being encoded or decoded (i.e., the current block) and the reference block used for its prediction (i.e., the prediction block). Specifically, for the current block predicted in inter-frame prediction mode, LIC modifies the corresponding reference block into an illumination-compensated prediction block by applying the LIC model to the pixel values ​​of the reference block to compensate for illumination differences. Typically, the parameters of the LIC model are estimated based on neighboring reconstructed samples. That is, for the current block, the parameters of the LIC model are estimated based on reconstructed samples in the neighborhood of the current block (i.e., the current template) and corresponding reconstructed samples in the neighborhood of the reference block (i.e., the reference template). To obtain a LIC model that accurately models the illumination differences between the current block and its reference block, more than one LIC model should be considered. Doing so can improve compression efficiency (i.e., reduce the bitrate while maintaining the same quality, or equivalently, improve the quality while maintaining the same bitrate). The following describes how LIC is applied in recent video codecs (when relying on a LIC model).

[0046] In inter-frame prediction operation modes such as those implemented using Exploratory Coding Model (ECM), LIC is employed to compensate for potential temporal illumination variations across images (see, “CE4-3.1a and CE4-3.1b: Unidirectional localillumination compensation with affine prediction,” document JVET-O0066, 15th meeting, Gothenburg, Sweden, July 3-12, 2019). The LIC model is a linear model applied to samples from a reference block to predict the value of the corresponding sample in the current block. This linear model can be represented as follows: (1) in The position in the current block The predicted pixel value of the sample at that location; where It is the position in the reference block. The pixel value of the corresponding sample at that location, where These are motion vectors obtained through inter-frame prediction; where These are linear model parameters. This represents the scaling parameter. This represents the offset parameter. Derivation of the LIC model—that is, the linear model parameters. The estimation of the parameters can be performed using the minimum mean square error (LMSE) estimation method. In the LMSE method, the parameters are... The estimated values ​​are those parameters that minimize LMSE. LMSE can be represented as follows: (2) in This represents the number of template samples. The parameter that minimizes the LMSE error in equation (2) can be calculated as follows: , and (3) (4) in Indicates the number of template samples; where It is a location The pixel value of the template sample (i.e., the current template) associated with the current block; where It is a location The pixel value of the template sample (i.e., the reference template) associated with the reference block. Note that... via motion vector and They correspond in space.

[0047] Therefore, when the encoder (e.g., encoder 200) applies LIC to the current block, it signals the LIC flag to indicate to the decoder (e.g., decoder 300) that the predicted block (i.e., its reference block) of the current block has been modified to compensate for illumination changes. If LIC is not applicable to the current block, the LIC flag is not signaled. When the LIC flag is signaled, it can be added to the motion information of the current block and can be context-encoded using a single context. Note that the decoder can derive the LIC model independently in the same way as the encoder. This is because the parameters of the LIC model are based on the current template. T and reference template T rThe corresponding reconstructed samples are used for estimation, as discussed above for equations (1)-(4). Therefore, the encoder does not need to include the LIC model itself (e.g., parameters). and The decoder is notified by a signal.

[0048] LIC can also be applied to bidirectional prediction blocks (see “EE2-Test2.7: Improvements on local illumination compensation”, document JVET-AD0213, 30th meeting, Antalya, Turkey, April 21-28, 2023, hereinafter referred to as “JVET-AD0213”), as referenced below. Figure 4 As described.

[0049] Figure 4 This is a schematic diagram illustrating the derivation of the LIC model 400 when operating in bidirectional prediction mode. Figure 4 The example shows block 425 of the current bidirectional predictive coding (or decoding) of the current image 420, and a template region including reconstructed samples located along the left boundary of the block and reconstructed samples located along the upper boundary of the block. 450. Reference images L0 410 and L1 430 are also shown, including corresponding reference blocks 415, 435 for predicting the current block 425 in inter-frame prediction mode. 440 and 460 is the template region in the corresponding reference frames L0 410 and L1 430, corresponding to the template region of the current image. 450. In this bidirectional predictive coding of the current block 425, two LIC models are derived. The parameters of the first model are... based on T The samples in and their T The corresponding sample in 0 (by motion vector) The parameters of the second model are estimated using a 470-point pointer. based on T The samples in and their T The corresponding sample in 1 (by motion vector) The LMSE error in equation (2) is used to estimate the parameters of the first or second model (expressed as 480 for simplicity). The following can be calculated: , and (5) , (6) in Indicates the number of template samples; where It is a location The pixel value of the template sample associated with the current block; where It is a location The pixel values ​​of the template sample associated with the reference block (L0 410 or L1 430). Note that... via motion vector and They correspond in space.

[0050] The derivation of the LIC model (e.g., based on equations (5) and (6)) can be computationally demanding. To reduce computational complexity, one can... 450 440 and 460 subsampling is used to reduce the number of template samples used. For example, for block sizes greater than 8, a 2:1 sampling rate can be used to subsample the template samples, i.e., only using... Figure 4 Patterned samples in the data are used to estimate and It should be noted that the LIC model (independently or in combination) can be derived from the color components of the video block—such as the luminance component Y and the chrominance components Cr and Cb.

[0051] Once the two LIC models are derived, they can be applied to the corresponding motion vectors. 470 and 480 points to the reference block 415 and 435 (in reference images L0 410 and L1 430 respectively), as shown below: (7) (8) in and These are estimates of the scaling and offset parameters of the first LIC model. and These are estimates of the scaling and offset parameters of the second LIC model. Next, the two illumination-compensated prediction blocks are combined. and To obtain bidirectional prediction blocks As shown below: (9) in This represents the weighting parameter. When the application employs CU-based weighted (BCW) bidirectional prediction, this weight is indicated by the BCW index.

[0052] The parameters can be iteratively applied. and Make estimates to derive and For example, as described below with respect to equations (10)-(13). In the first iteration, the parameters are calculated according to equations (5) and (6). Then by subtracting Update the corresponding samples in the data. T The pixel values ​​of the template sample in the image. The derivation is done using the calculated parameters as shown below: , (10) (11) In the second iteration, the updated template is used according to equations (5) and (6). T' To calculate parameters Then by subtracting Update again using the corresponding samples in the database. T The pixel values ​​of the template sample in the image. It is derived using calculated parameters, as shown below: , (12) (13) Then, in the third iteration, the updated equations are used according to equations (5) and (6). T'' To refine the parameters .

[0053] As described in JVET-AD0213, according to the current LIC design, when using bidirectional prediction with AMVP mode, a flag is signaled to indicate the LIC mode; while when using bidirectional prediction with merging mode, this flag is inherited from the merging candidate used. It should be noted that LIC operation is disabled when certain tools (such as decoder-side motion vector refinement (DMVR) (including multipass DMVR, adaptive DMVR, and affine DMVR) and bidirectional optical flow (BDOF)) are applied.

[0054] Similarly, LIC can be applied to sub-blocks of the current block (e.g., when operating in a sub-block mode such as affine mode). In this case, the LIC model is derived from the stencil region associated with the corresponding sub-block, as per [the relevant context]. Figure 5 Further description.

[0055] Figure 5 This is a schematic diagram illustrating the derivation of LIC model 500 when operating in sub-block mode. For example... Figure 5As shown in the example, the current block 515 of the current image 510 is divided into sub-blocks, labeled A, B, C, D, E, F, and G. These sub-blocks are associated with corresponding reference sub-blocks of the reference image 520, labeled Aref, Bref, Cref, Dref, Eref, Fref, and Gref. The reference sub-blocks are pointed to by various motion vectors (indicated by arrows). The gray areas to the left and above the current block 515 represent the current template, and the gray areas to the left and above the reference sub-blocks represent the reference templates. In this case, for example, the template sample above sub-block A of the current block 515 corresponds to the template sample above sub-block Aref; the template sample to the left of sub-block A of the current block 515 corresponds to the template sample to the left of sub-block Aref; the template sample above sub-block B of the current block 515 corresponds to the template sample above sub-block Bref; the template sample to the left of sub-block E of the current block 515 corresponds to the template sample to the left of sub-block Eref; and so on. The reference template and the corresponding samples in the current template can be used to estimate the LIC model (e.g., according to equations (3) and (4)).

[0056] Furthermore, LIC can also be applied to the intra-block copy (IBC) mode (see, “EE2-3.6: IBC-CIIP, IBC-GPM, and IBC-LIC”, document JVET-AC0112, 29th meeting, teleconference, January 11-20, 2023). The combined application of IBC and LIC (i.e., in IBC-LIC mode) aims to compensate for illumination variations within the same image—that is, illumination variations between the CU being encoded and its IBC prediction block. In this case, the LIC model is derived in the same manner as described above for inter-frame prediction, except that the reference template is associated with the reference block pointed to by the block vector given by the IBC-LIC mode. IBC-LIC can be applied in conjunction with IBC AMVP mode and IBC merging mode. For IBC AMVP mode, the IBC-LIC flag is signaled to indicate the application of LIC. For IBC merging mode, the IBC-LIC flag is inferred from the merging candidate.

[0057] In recent video codecs, template-based techniques have been introduced to predict chrominance samples from corresponding luma samples—the so-called cross-component prediction (CCP). Several CCP models are employed in ECM, including the cross-component linear model (CCLM), multi-model CCLM (MMLM), convolutional cross-component model (CCCM), and gradient linear model (GLM). These CCP models leverage the correlations that may exist between corresponding luma and chrominance regions associated with the current block. To improve efficiency (e.g., by reducing signaling overhead), the cross-component merging (CCMerge) mode is introduced as a novel CCP mode (see “EE2-1.6: Non-local cross-component prediction and cross-component merge mode”, document JVET-AD0188, 30th meeting, Antalya, Turkey, April 21-28, 2023, hereinafter referred to as “JVET-AD0188”). The CCMerge mode can be used to inherit one of the CCP models used by the spatially neighboring block (JVET-AD0188) or the temporally neighboring block (see “EE2-2.7: Cross-component prediction merge mode with temporal candidates”, document JVET-AE0043, 31st meeting, Geneva, Switzerland, July 11-19, 2023, hereinafter referred to as “JVET-AE0043”), as further described below.

[0058] In the CCMerge mode, the CCP model used to predict chroma blocks can be 1) inherited from spatially adjacent or non-adjacent neighboring blocks of the block; 2) a history-based model; or 3) a default model. To this end, a candidate list is created for the current block, which can include CCP models such as CCLM, MMLM, CCCM, GLM, chroma fusion, and the CCMerge mode. To construct this list, spatially adjacent and non-adjacent neighboring blocks are considered first, and their respective CCP models are added to the list (if available). Next, if the list has not yet reached its maximum size, the CCP models listed in the history-based table are added to the list, followed by the default CCP model. To avoid redundant CCP models in the list, a pruning operation can be applied. After constructing the list, the CCP models in the list can be reordered based on the cost of applying these models to the template regions of the current block (e.g., using the sum of absolute differences (SAD) metric).

[0059] Figure 6This is a schematic diagram showing the location 600 of spatially adjacent neighboring blocks. Spatially adjacent blocks are those that are adjacent to... Figure 6 The blocks indicated by B0, B1, B2, A0, and A1 shown are spatially adjacent blocks. These spatially adjacent blocks are considered in the following order: B1, A1, B0, A0, then B2; and if each of these blocks uses the CCP model, it can be added to the list as a CPP model candidate.

[0060] Figure 7 This is a schematic diagram showing the location 700 of a non-adjacent neighboring block in space. Figure 7 The example shows 59 locations indicated by points relative to the current block layout, whose distances can be determined based on the width and height of the current block. Spatially non-adjacent blocks are those that overlap with these locations. Therefore, if the CCP model candidate list is not full after adding CCP models for spatially adjacent blocks, CCP models for spatially non-adjacent blocks can be added to the list (if available). The order in which spatially non-adjacent blocks are considered can be based on their distances from the current block and / or their frequency of occurrence.

[0061] As mentioned above, CCP model candidates can be history-based candidates, extracted from a history-based table. A history-based table (e.g., size 6) can be maintained to include recently used CCP models and can be reset periodically (e.g., at the beginning of each CTU row). If the CCP model candidate list is not full after adding CCP model candidates for spatially adjacent and non-adjacent blocks, CCP models from the history-based table can be added to the list.

[0062] Next, if the CCP model candidate list is still not full (after adding available CCP models for spatially adjacent and non-adjacent adjacent blocks, as well as historical CCP models), a default candidate can be added. Therefore, if the list does not include CCLM candidates, a default CCLM candidate can be added to the list with scaling parameters of 0, 1 / 8, -1 / 8, 2 / 8, -2 / 8, 3 / 8, -3 / 8, 4 / 8, -4 / 8, 5 / 8, -5 / 8, and 6 / 8. Otherwise, a default CCLM candidate can be added to the list with a scaling parameter of 0, and the scaling parameter is equal to the scaling parameter of the first CCLM candidate in the list plus 1 / 8, -1 / 8, 2 / 8, -2 / 8, 3 / 8, -3 / 8, 4 / 8, -4 / 8, 5 / 8, -5 / 8, and 6 / 8. The offset parameter can be derived as shown in equation (14) below.

[0063] Therefore, a chroma block (of the current block) can be predicted based on a CCP model inherited from neighboring blocks encoded in CCLM, CCCM, GLM, chroma fusion, or CCMerge modes (or a chroma block can be predicted based on multiple CCP models inherited from neighboring blocks encoded in MMLM or multi-model CCCM). Inheritance is performed as follows.

[0064] • Chroma blocks can inherit the following information from candidates in the CCP model candidate list: model parameters, GLM mode index (when the CCP model is GLM), and classification threshold (when the CPP is MMLM or multi-model CCCM).

[0065] • The offset parameters (if defined for a CCP model) are not inherited, but are derived as follows: , (14) in These are inherited scaling parameters. It is the average value of the neighboring reconstructed brightness samples. It is the average value of the neighboring reconstructed chromaticity samples.

[0066] Therefore, to indicate whether CCMerge mode is used to predict the chroma samples of the current block, a signaling flag can be used (e.g., after signaling in the cclm_mode_flag syntax element). If this flag is set to true (i.e., CCMerge mode is used), the index is further signaled to indicate which candidate in the list is used for the current block. However, if the maximum number of allowed candidates is modified to 1, there is no need to signal the index. Currently, the maximum number of allowed candidates is set to 12 by default. The signaled index can be shared by the Cb and Cr color components. Each bin (bit) of the index is context-encoded with a separate context. Note that CCMerge mode is not allowed for intra-frame prediction of chroma blocks when the CU (including the chroma block) is encoded by intra-fractional sub-segmentation (ISP) with a single tree, or when the size of the chroma block is less than or equal to 16.

[0067] As mentioned above, CCP models for temporally neighboring blocks can also be added to the CCP model candidate list (see JVET-AE0043). In this case, CCP models for both temporally neighboring blocks and shifted temporally neighboring blocks can be added to the list. CCP models for temporally neighboring blocks can be added after adding CCP models for spatially adjacent blocks, while CCP models for shifted temporally neighboring blocks can be added after adding CCP models for spatially non-adjacent blocks. Signaling efficiency can then be further improved by reordering the CCP model candidates in the input list based on the cost of applying them to predict chromaticity samples in the template region associated with the current block (e.g., using a SAD metric). Candidates can be stored on an 8×8 grid and made available for later predictions. For 16×16 prediction blocks, only candidate information on the top-left 8×8 grid is stored for access. Temporally neighboring blocks and shifted temporally neighboring blocks will be related to... Figure 8 and Figure 9 Further description.

[0068] Figure 8 This is a schematic diagram showing the position 800 of the time neighbor block. The time neighbor block is selected from the co-position relative to the current block 820 in the co-position picture 810. The co-position picture 810 is defined by an index of the co-position reference picture used to derive the co-position CU, and this index is explicitly indicated by a signal in the strip header (as in ECM-8.0). Figure 8 In the example, two sets of locations are highlighted: C0, indicated by a black border, and C1, indicated by a gray border. The locations of these temporally adjacent blocks are also numbered to indicate the order in which the corresponding CCP model candidate (if available) was added to the CCP model candidate list. This order is defined as: C01, C02, ... C0 10 If C0 i Unavailable (i.e., C0) i The location is outside the image / strip boundary, or outside the current CTU line) and C1 i If available, C1 will be used. i Replace C0 i , where 1 ≤ i ≤ 10. Otherwise (C0) i and C1 i (None of them are available), check the next location C0 i+1 .

[0069] Figure 9 This is a schematic diagram showing the positions 900 of the time-adjacent blocks during the shift. These positions are selected from the co-position relative to the current block 920 in the co-position image 910. However, in this case, the position of the co-position block 920 is shifted by the selected neighboring motion vector (as indicated by the arrow). Therefore, C0 i and C1 iThe position is also shifted by the same motion vector. The CCP models of the shifted temporally neighboring blocks are added to the CCP model candidate list in the same order as the temporally neighboring blocks. The motion vectors are selected from the motion vectors of the blocks adjacent to the current chroma block 930 (i.e., the neighboring blocks that overlap with positions A0, A1, B0, B1, and B2). L0 motion vector 940 and / or L1 motion vector 950 can be associated with each of these neighboring blocks. The order in which the motion vectors of the neighboring blocks are checked is as follows: L0B1, L1B1, L0A1, L1A1, L0B0, L1B0, L0A0, L1A0, L0B2, and L1B2. The first motion vector that uses the co-location image 910 as the reference image is selected. If no such motion vector is found, the CCP model of the shifted temporally neighboring block is not added to the list.

[0070] Merge mode is also applied when using IBC-LIC, namely IBC-LIC model merge mode. In IBC-LIC model merge mode, IBC-LIC model parameters are inherited from previously coded blocks (see “Non-EE2: IBC-LIC Model Mergemode”, document JVET-AE0073, 31st meeting, Geneva, Switzerland, July 11-19, 2023). More specifically, for the current block, the LIC model can be obtained as follows.

[0071] a) Construct a candidate list of LIC models. This list consists of LIC models used by spatially adjacent and non-adjacent neighboring blocks, history-based models, and a default model. The size of the candidate list is predetermined (e.g., twelve). Similar to the CCMerge pattern, it selects models from those located at adjacent and non-adjacent positions (e.g., the position of spatially non-adjacent neighboring block 1000 in the adjacent and non-adjacent blocks). Figure 10 The LIC models are collected from blocks (previously encoded in IBC-LIC mode and IBC-LIC model merging mode, as shown in the diagram). Additionally, a historical IBC-LIC model table (e.g., of size six) is maintained. LIC models from spatially neighboring blocks and the historical IBC-LIC model table are added to the IBC-LIC model merging candidate list. If the list is not full, a default model can then be added. To avoid redundant models, a pruning operation can be applied.

[0072] b) The offset parameters of the IBC-LIC model candidates can be adjusted. Adjustment value The calculation can be performed as follows: (15) (16) in and These are the pixel values ​​of the reconstructed samples from the templates of the current block and the reference block, respectively; That's the total number of samples. Then offset. quilt Adjustment, i.e. .

[0073] c) Select an IBC-LIC model from the candidate list, with its index signaled in the bitstream.

[0074] A signaling flag is used to indicate whether the IBC-LIC model merge mode is applied. If this flag is true (i.e., IBC-LIC model merge mode is applied), the index is further signaled to indicate which candidate in the list the current block uses. In IBC-AMVP mode, this flag is signaled when the flag indicating operation in IBC-LIC mode is true. In IBC-Merge mode, this flag is signaled only if the current block is not encoded in IBC-CIIP, IBC-GPM, TM-Merge, or Skip mode. If the IBC-LIC model merge flag is true, the normal inheritance of IBC-LIC flags is not applied, and the current block is treated as normal IBC-LIC by other blocks.

[0075] The aspects disclosed in this paper improve the coding efficiency of inter-frame coded blocks using LIC. In state-of-the-art LIC designs, when using LIC, only a single LIC model can be applied—that is, a model estimated based on template regions in the immediate neighborhood of the current block and its reference block. If this estimated model is poorly compensated for illumination differences, applying it to correct the prediction of the current block may introduce additional errors, which may degrade coding efficiency. In particular, when using merge mode, the LIC flag of the current block is inherited from the merge candidate. Regardless of whether it is optimal, the prediction of the current block is then corrected using the LIC model of the merge candidate. As mentioned above, in CCMerge mode, the inherited CCP model is applied to predict the chroma samples of the block, and a similar concept has been applied when using IBC-LIC mode. However, the concept of inheriting the LIC model from previously coded blocks does not apply to inter-frame predicted blocks.

[0076] According to the specifications, the LIC model merging mode is used for inter-frame prediction blocks. In LIC model merging mode, the LIC model is inherited from previously encoded blocks to compensate for illumination differences. For this purpose, the LIC model merging mode can be explicitly signaled as a sub-mode of the LIC mode, whether using AMVP mode or merging mode. When using LIC model merging mode, a list of LIC model candidates is constructed; then, a LIC model is selected from the list and (optionally) refined before applying it to compensate for illumination variations. Furthermore, the LIC model merging mode can be used in combination with sub-block modes and bidirectional prediction block modes. The following describes: the general operation of the disclosed LIC model merging mode; the construction of the LIC model candidate list; the adaptation of the LIC model merging mode to sub-block modes and bidirectional prediction block modes; and the refinement of the inherited LIC model parameters.

[0077] LIC model merging mode.

[0078] Figure 11 This is a flowchart illustrating application 1100 in LIC mode. Figure 11 In the example, the LIC model (e.g., a linear model) is derived and applied to the blocks encoding inter-frame predictions. Accordingly, in step 1110, the LIC flag is determined. lic_flag (Set by the encoder and signaled in the bitstream, decoded by the decoder from the bitstream) Whether the current block predicted in the inter-frame prediction mode is set to true. When LIC is enabled for the inter-frame prediction block, in step 1120, the current template (including reconstructed neighbor samples of the current block) and the reference template (including reconstructed neighbor samples of the reference block) are obtained. Based on these templates, in step 1130, the LIC model is derived—for example, the parameters of the linear model are derived (i.e., estimated) as described with respect to equations (1)-(4). Then, in step 1140, in order to compensate for the illumination change, the derived LIC model is applied to the samples in the reference block, thereby generating an illumination-compensated prediction block, which is used as the prediction for the current block.

[0079] like Figure 11 As shown, a single LIC model is applied based on the current template and the reference template. However, such a LIC model may be insufficient to correct for lighting variations experienced by the current block. This is because the LIC model is derived based on a limited number of corresponding samples from the current and reference templates. Furthermore, the correlation between corresponding samples from the reference and current templates may not represent the correlation between corresponding samples from the reference and current blocks.

[0080] To mitigate these drawbacks, this paper introduces a LIC model merging mode, where a LIC model can be inherited from a previously encoded block with the LIC flag enabled. In this mode, multiple LIC models can be collected from previously encoded blocks and treated as candidates for the LIC model of the current block.

[0081] The LIC model merging mode disclosed in this paper can be explicitly signaled as a sub-mode of the LIC mode, regardless of whether the AMVP mode or the merging mode is used. Signaling flags can be used for blocks predicted between frames. lic_model_merge_ flag Its conditions are based on LIC flags to indicate whether the LIC model merging mode is applied. If lic_model_merge_flag If set to true, the index is further signaled to indicate which candidate LIC model the current block should use. However, in this case, the index can be implicitly inferred (e.g., a predefined candidate could be used, such as the first available candidate in the LIC model candidate list)—therefore, the index does not need to be signaled. In AMVP mode, the flag is signaled when the LIC flag is true. lic_model_merge_flag In merge mode, the flag is signaled only if the LIC flag of the current block inferred from the merge candidate is true. lic_model_merge_flag .

[0082] Figure 12 This is a flowchart illustrating the application of LIC model merging mode 1200. Figure 12 In the example, in step 1210, the LIC model merge mode flag is determined. lic_model_merge_flag (Set by the encoder and can be signaled in the bitstream, and can be decoded from the bitstream by the decoder) Whether to set to true for the current block encoded in inter-frame prediction mode. lic_model_merge_flag Indicates whether to apply the LIC model merging mode to blocks predicted by inter-frames. If lic_model_merge_flag If set to false, then the regular LIC procedure (e.g., ...) will be executed in steps 1220 and 1230. Figure 11 (As shown). That is, in step 1220, the current template and the reference template are obtained. Based on these templates, in step 1230, the LIC model is derived—for example, the parameters of the linear model are derived (i.e., estimated) as described with respect to equations (1)-(4). .if lic_model_merge_ flagIf set to true, the LIC model merging mode is applied in steps 1240 and 1250. In step 1240, a candidate list of LIC models is constructed. Then, in step 1250, a LIC model is selected from the candidate list. The selected LIC model is indicated by an index (which is determined by the encoder and can be signaled in the bitstream and decoded from the bitstream by the decoder). Then, in step 1260, to compensate for illumination variations, the LIC model (derived in step 1230 or selected from the candidate list in step 1250) is applied to samples in the reference block, thereby generating an illumination-compensated prediction block, which is used as the prediction for the current block.

[0083] According to the approach, when applying the AMVP pattern, the LIC model merge mode can be signaled before signaling the LIC flag, and the LIC flag is only signaled if the LIC model merge mode is false. In this case, if lic_model_merge_flag If true, the proposed LIC model merge pattern will be applied to the current block, and there is no need to signal the LIC flag; if lic_model_merge_flag If the result is false, the LIC flag is signaled, and if the LIC flag is true, the LIC procedure is executed on the current block; if... lic_model_merge_flag If both the LIC and COM flags are false, the LIC procedure will not be executed on the current block. When a merge mode is applied, the flags are only signaled if the current block is not encoded in CIIP, GPM, TM-Merge, or Skip mode. lic_model_merge_flag Regardless of the LIC flag inferred from the merge candidates. In merge mode, if the LIC model merges the flag... lic_model_merge_flag If true, the LIC flag should not be inherited from the merge candidate in the normal way, which means that the current block will still perform the LIC process even if the LIC flag inherited from its merge candidate is false.

[0084] lic_model_merge_flag It can be implicitly deduced. For example, it can be assumed that it is always true or equal to the LIC flag. lic_model_merge_flag It can also be derived based on encoding parameters or other conditions, such as the current block size or the number of neighboring blocks (or reconstructed samples) with LIC enabled. For example, the LIC model merge mode can be used only in merge mode or only in AMVP mode. In another example, the LIC model merge mode can be disabled for blocks of a specific size, such as blocks with fewer (or more) N luminance samples (e.g., N=32).

[0085] On one hand, when the LIC model merging mode or the regular LIC mode is applied to the current block, the LIC flag is stored as true in the block's motion information and can be used for subsequent predictions. This current block will be treated as using the regular LIC mode by subsequent blocks, which will use the current block's LIC model as a candidate in their LIC model candidate list.

[0086] Construction of the LIC model candidate list.

[0087] The candidate list of LIC models can consist of the following: LIC models of spatially adjacent neighboring blocks; LIC models of spatially non-adjacent neighboring blocks; LIC models of temporally adjacent blocks; LIC models of shifted temporally adjacent blocks; history-based LIC models; the default LIC model; or a combination thereof. The order in which candidates are added to the list may differ. For example, a LIC model from a shifted temporally adjacent block may be included in the list after a history-based LIC model. In one aspect, a maximum number of candidates can be defined for each candidate type. For example, the maximum number of LIC models for spatially adjacent blocks can be set to 5, and the maximum number of LIC models for temporally adjacent blocks can be set to 2.

[0088] To avoid including redundant LIC models in the list, a pruning operation can be applied. For example, a LIC model candidate can be added to the list only if it is not already in the list. Once the list of LIC model candidates is built, the LIC models in the list can be reordered based on the cost of applying these models to the template regions of the current coding block (e.g., using the SAD metric).

[0089] The size of the merge list can be signaled in the bitstream. For example, the list size can be signaled in the Sequence Parameter Set (SPS), Viewpoint Parameter Set (VPS), Picture Parameter Set (PPS), Adaptive Parameter Set (APS), or Picture Header (PH) of the encoded video data. Alternatively, the size of the merge list can be determined implicitly. That is, the size of the merge list can be predetermined or determined based on encoding parameters or other conditions, such as block size, color components, prediction mode, QP value, or sequence class and configuration. In one aspect, the maximum allowed size of the merge list can be predetermined (e.g., 12).

[0090] For each block encoded using the LIC model merging mode, the index of the selected (and used) LIC model in the LIC model candidate list can be encoded (informed by signaling) in the bitstream by the encoder (e.g., using truncated unary binarization). Alternatively, the index of the selected LIC model can be implicitly determined by the decoder. When signaled, the first bin of the index can be context-coded, and the other bins can be bypass-coded; or each bin of the index can be encoded using a separate context-based adaptive binary arithmetic coding (CABAC).

[0091] The LIC model candidate list can include LIC models from spatially adjacent and / or non-adjacent neighbor blocks encoded in LIC patterns. For spatially adjacent neighbor blocks, the locations of multiple spatially adjacent blocks can be used (e.g., ...). Figure 6 (as shown), and their respective LIC models (if available) can be added to the list in the following order: B1, A1, B0, A0, and B2. For spatially non-adjacent neighbor blocks, various location styles can be used (e.g., such as...). Figure 7 and Figure 10 (As shown). The distance between non-adjacent neighboring blocks and the current coding block can be based on the width and height of the current coding block. The order in which spatially adjacent and non-adjacent neighboring candidate LIC models are added to the list can be based on predetermined criteria (e.g., distance and / or statistical measures). For example, the LIC model of the nearest neighboring block relative to the current block can be added first, and the LIC model of the farthest neighboring block can be added last; or, neighboring blocks can be divided into two groups, such that the LIC models of blocks in the first group are added to the list first, and then, if the list is not full, the LIC models of blocks in the second group are added.

[0092] The LIC model candidate list can include LIC models from temporal neighbor blocks and / or shifted temporal neighbor blocks encoded in LIC mode. These temporal neighbor blocks and shifted temporal neighbor blocks are respectively referenced from... Figure 8 and Figure 9 Describe it.

[0093] The history-based LIC model maintained in the history-based table can also be added to the LIC model candidate list after LIC models from spatially and temporally neighboring blocks (if available) are added. Redundancy checks can be applied to any history-based LIC model to be added to the list. The history-based table includes LIC models of blocks previously encoded in LIC mode. Multiple LIC models (e.g., 6) can be maintained in the history-based table during encoding / decoding. The history-based table can be reset (cleared) when a new CTU row is encountered. Whenever a block is encoded in LIC mode, the LIC model used can be added as the last entry in the table as a new history-based LIC model. When a new LIC model is inserted into the table, a first-in, first-out (FIFO) rule can be used, where a redundancy check is first applied to determine if the same LIC model already exists in the table. If found, the same LIC model is removed from the table, and all subsequent history-based LIC models are shifted forward in the table, and the new LIC model is inserted as the last entry in the table.

[0094] If the LIC model candidate list is not full—even after adding LIC models from spatially neighboring blocks, temporally neighboring blocks, and history-based tables—a default candidate can be added to the list until it is full (i.e., the maximum number of LIC model candidates has been reached). For example, the default scaling parameter of the default LIC model can be determined based on the scaling parameter of the first LIC model candidate plus 0, 1 / 8, -1 / 8, 2 / 8, -2 / 8, 3 / 8, -3 / 8, 4 / 8, -4 / 8, 5 / 8, -5 / 8, and / or 6 / 8. The offset parameter can be derived as follows: , (17) in This is the default scaling parameter. It is the average value of the samples in the current template. It is the average value of the samples in the reference template.

[0095] The LIC model merging pattern disclosed in this paper can also be applied when using the sub-block pattern. In the sub-block pattern, each sub-block has its own motion vector, so the LIC model can be derived based on the current template and reference template associated with the sub-block, such as the reference template. Figure 5 As described above. When the LIC model merging mode is used in conjunction with the sub-block mode, the LIC model candidate list can include LIC models from multiple reference blocks pointed to by motion vectors associated with each sub-block of the current block. For example... Figure 5The current block 515 shows 16 sub-blocks. The LIC models of reference blocks associated with these sub-blocks (if present) can be added to the merge list. For example, the LIC models used by the individual reference blocks Aref, Bref, Cref, ..., Gref can be added in this order. Therefore, the reference block pointed to by the motion vector of the corresponding sub-block can be examined: if the reference block (e.g., Aref) is encoded in LIC mode and the corresponding LIC model has not yet been added to the list, then the corresponding LIC model can be added to the list. On one hand, the added LIC model can be the LIC model of the reference block pointed to by the motion vector derived based on one or more motion vectors associated with the individual sub-blocks of the video block. On the other hand, the LIC model can be derived from the reference block pointed to by the motion vector representing the center of the current block.

[0096] The LIC model merging pattern disclosed herein can be applied when using bidirectional prediction to predict the current block (i.e., bidirectional prediction LIC model merging pattern). Therefore, LIC can be extended to bidirectional prediction blocks, where two LIC models are applied to generate two prediction blocks, and then the two prediction blocks are combined to generate the final bidirectional prediction block, as described with respect to equations (5)-(9).

[0097] To indicate operation in the bidirectional predictive LIC model merging mode, signal notification flags can be used. bic_ model_merge_flag (replace lic_model_merge_flag ).if bic_model_merge_flag Setting this to true allows you to select two LIC models from the LIC model candidate list, and the associated index can be signaled in the bitstream. As previously mentioned, the list can contain LIC models from spatially and temporally neighboring blocks, LIC models from history-based tables, or the default LIC model. For example, the list can include LIC model pairs, i.e., pairs of L0 LIC models and L1 LIC models. These LIC model pairs can be derived from spatially and temporally neighboring blocks encoded with bidirectional predictive LIC modes, and can be added to the list if they are not already there.

[0098] On one hand, the two LIC models can be selected from different LIC model pairs in the LIC model candidate list using two corresponding indices, which can be signaled in the bitstream. For example, a first LIC model (e.g., L0 LIC model) can be selected from a first LIC model pair in the list using a first index, and a second LIC model (e.g., L1 LIC model) can be selected from a second LIC model pair in the list using a second index.

[0099] In another aspect, the LIC model pair can be selected from two separate LIC model candidate lists, so the two corresponding indices can be signaled to the bitstream. For example, an L0 LIC model list and an L1 LIC model list can be generated and used to derive paired L0 LIC models and L1 LIC models, respectively. In another example, for temporally adjacent blocks, the LIC model pair can be: 1) the LIC model from the reference block pointed to by the corresponding MV_L0 in the reference picture list L0; and 2) the LIC model from the reference block pointed to by the corresponding MV_L1 in the reference picture list L1.

[0100] In another aspect, one LIC model (e.g., the L0 LIC model) is selected from a list, while another LIC model (e.g., the L1 LIC model) is derived from the current template and the reference template (e.g., ...). Figure 11 (As shown). In this case, two flags can be signaled for the bidirectional predictive LIC block. bic_model_merge_flag_L0 and bic_model_merge_flag_L1 This indicates whether to use the merge mode to derive the L0 LIC model and the L1 LIC model, respectively. For example, if... bic_model_ merge_flag_L0 If set to true, an L0 LIC model can be selected from the L0 LIC model candidate list based on the L0 LIC index, which can be signaled in the bitstream. Furthermore, if... bic_model_merge_flag_L1 Setting it to false allows the L1 LIC model to be derived from the current template and the reference template (e.g., Figure 11 (As shown).

[0101] Alternatively, a single LIC model can be selected from the list of LIC model candidates and then applied to two prediction blocks (e.g., in equations (7)-(8)). and For example, for a LIC model derived from a temporally neighboring block, a single LIC model can be derived by blending two LIC models inherited from two reference blocks in the reference image lists L0 and L1. The blending process can be a simple averaging or a weighted averaging (e.g., using weights indicated by the BCW index).

[0102] It should be noted that the aspects of the bidirectional prediction LIC model merging mode described above can be similarly applied to bidirectional prediction IBC blocks.

[0103] Refinement of inherited LIC model parameters.

[0104] The inherited LIC model can be refined to improve local illumination compensation performance. To do this, the parameters of the inherited LIC model can be adjusted—for example, refined scaling parameters and / or refined offset parameters can be calculated based on the template associated with the corresponding video block. For instance, the inherited scaling parameters can remain unchanged, while the inherited offset parameters can be refined based on the current template and a reference template (associated with the current block and its reference block, respectively), as follows: (18) (19) in and These are the pixel values ​​of the reconstructed samples from the current template and the reference template, respectively. N That's the total number of samples. Then offset. use Adjustment, i.e. .

[0105] When applying bidirectional prediction, the inherited L0 LIC model and L1 LIC model can be refined using the following steps: 1) Inherited L0 LIC model Applied to L0 template T For a sample of 0, we get: ; 2) From the template of the current block Subtract from the corresponding samples The sample was obtained ; 3) Template-based and T Minimum difference between 1s refines the inherited L1 LIC offset : , (20) ; (twenty one) 4) Apply the inherited L1 LIC model to the L1 template. The sample was obtained ;as well as 5) By from Subtract from the corresponding samples The samples and recalculated offsets To refine the inherited L0 LIC offset .

[0106] In one variant, only the L0 LIC model is inherited, while the L1 LIC model and (optionally) the L0 LIC model are iteratively refined. For example, inheriting the L0 LIC model... And derive the L1 LIC model using the following steps. : 1) Inherited L0 LIC model Applied to L0 template From the sample, we obtained: ; 2) From the template of the current block Subtract from the corresponding samples From the samples, we obtain updated templates. ; 3) Use the updated equations (5) and (6) and L1 template Derivation of the L1 LIC model .

[0107] In the case where the LIC model inherits from the time candidate, the scaling parameter Refinement can be achieved based on image sequence count (POC) distance, and offset. Refinement can be performed, for example, using the refinement scaling parameters according to equations (20)-(21). This refinement will be about Figure 13 Further description.

[0108] Figure 13 This is a schematic diagram showing a refinement of the LIC model at 1300. In Figure 13 In the example, the current block 1315 (belonging to the current image 1310) references a reference block A 1325 (belonging to the reference image 1320) that may contain a temporally neighboring block. The LIC model of this temporally neighboring block can be applied to the reference block B 1335 to generate a predicted block. In one aspect, when the current block 1315 inherits the LIC model from this temporally neighboring block, the scaling parameters of the inherited LIC model can be based on distance. 1340 and 1350 is further refined, among which It is the difference in Proof of Concept (POC) between reference image 1320 and the current image 1310. This is the difference in Proof of Concept (POC) between reference image 1330 and reference image 1320. The refinement of the scaling parameter can be normalized to constrain its range. For example, the scaling parameter... It can be refined into as follows: (twenty two) Figure 14 This is a schematic diagram illustrating another refined version 1400 of the LIC model. Figure 14 In the example, the scaling parameters of the inherited LIC model Use adjustment parameters (by...) (to indicate) to refine or adjust to As shown below: , and (23) , (twenty four) in This represents the average value of the reference sample. For example... Figure 14 As shown, this refinement revolves around Tilt or rotate the LIC model. Adjust parameters. These can be values ​​within a predefined range. On one hand, the encoder can use signals in the bitstream to notify adjustments to the parameters. .

[0109] Figure 15 This is a flowchart of an exemplary method 1500 for encoding video data. According to the aspects described herein, method 1500 can be applied to encode video blocks (belonging to video data) into a bitstream. Encoding 1510 of the video block can begin in step 1520 by constructing a candidate list of LIC models. In step 1530, a LIC model can be selected from the candidate list. Then, in step 1540, the selected LIC model can be applied to a reference block of the video block to thereby generate a prediction block for the video block. As described herein, the candidate list of LIC models can consist of LIC models including: LIC models of spatially adjacent neighboring blocks; LIC models of spatially non-adjacent neighboring blocks; LIC models of temporally adjacent blocks; LIC models of shifted temporally adjacent blocks; history-based LIC models; default LIC models; or combinations thereof. In one aspect, the constructed candidate list of LIC models can also include LIC models of corresponding reference blocks pointed to by motion vectors associated with sub-blocks of the video block. Alternatively, the candidate list of LIC models can include LIC models of reference blocks pointed to by motion vectors derived from one or more motion vectors associated with the respective sub-blocks of the video block.

[0110] Figure 16 This is a flowchart of an exemplary method 1600 for decoding video data. According to the aspects described herein, method 1600 can be applied to decode video blocks (belonging to video data) from a bitstream. Decoding of the video block 1610 can begin in step 1620 by constructing a candidate list of LIC models. In step 1630, a LIC model can be selected from the candidate list. Then, in step 1640, the selected LIC model can be applied to a reference block of the video block to thereby generate a prediction block for the video block. The selection of a LIC model from the candidate list can be based on an index decoded from the bitstream. Alternatively, the selection of a LIC model from the candidate list can be based on an implicitly derived index.

[0111] On one hand, a flag is decoded from the bitstream indicating whether the current video block should be decoded in a bidirectional prediction LIC model merging mode. If the flag is set to true, the construction (steps 1520 and 1620), selection (steps 1530 and 1630), and application (steps 1540 and 1640) performed by methods 1500 and 1600 may include: constructing a list of LIC model candidate pairs; selecting a LIC model pair from the list; and then applying the LIC model pair to the corresponding reference block of the video block to generate a bidirectional prediction block for the video block. The LIC model pair selected from the LIC model candidate list may be based on an index that may be decoded from the bitstream. On the other hand, a first LIC model is selected from the LIC model pairs in the list based on a first index decoded from the bitstream, and a second LIC model is selected from another LIC model pair in the list based on a second index that may be decoded from the bitstream.

[0112] According to a further aspect, the construction (steps 1520 and 1620), selection (steps 1530 and 1630), and application (steps 1540 and 1640) performed by methods 1500 and 1600 may include: constructing a first list and a second list of LIC model candidates; selecting a first LIC model from the first list and a second LIC model from the second list; and then applying the first LIC model and the second LIC model to the corresponding reference block of the video block to generate a bidirectional prediction block for the video block. Alternatively, the construction, selection, and application may include: 1) constructing a first list and a second list of LIC model candidates; 2) for each model (in the first LIC model and the second LIC model) and its respective list (in the constructed first list and the second list), decoding a flag from the bitstream indicating whether the model should be selected from the corresponding list, selecting the model from the corresponding list if the flag is set to true, otherwise deriving the model from the template region of the video block and its reference block; and then 3) applying the first LIC model and the second LIC model to the corresponding reference block of the video block to generate a bidirectional prediction block for the video block.

[0113] Several aspects and embodiments have been described in this disclosure. These aspects and embodiments, across different claim classes and types, provide at least the following outputs and results, including all combinations thereof: • Based on any aspect described herein, syntax elements that enable the decoder to decode the encoded video data will be encoded into the encoded video data.

[0114] • Includes a bitstream of one or more described syntax elements or variations thereof. The bitstream can be any dataset, whether or not it is transmitted, stored, or otherwise provided.

[0115] • Create, transmit, receive, and / or decode the bitstream.

[0116] • An electronic device (e.g., a television, set-top box, mobile phone, or tablet computer) tunes (e.g., using a tuner) a channel to receive the bitstream, or (e.g., using an antenna) receives the bitstream over the air. The electronic device decodes syntax elements from the bitstream and, optionally, displays (e.g., using a monitor, screen, or any other type of display) the resulting image.

[0117] This disclosure also supports and envisions a variety of other generalized and specific outputs, results, implementations, and claims.

[0118] This document describes various methods, and each method includes one or more steps or actions for implementing the described method. Unless a specific order of steps or actions is required for the correct operation of the method, the order and / or use of specific steps and / or actions can be modified or combined. Furthermore, terms such as "first," "second," etc., can be used in various embodiments to modify elements, components, steps, operations, etc., for example, "first decoding" and "second decoding." The use of such terms does not imply a requirement for the order of the modified operations unless specifically required. Therefore, in this example, the first decoding does not need to be performed before the second decoding, and can occur, for example, before, during, or within an overlapping period of the second decoding.

[0119] The various methods and other aspects described in this application can be used to modify the module, for example, Figure 2 and Figure 3 The modules of the video encoder 200 and video decoder 300 are shown. Furthermore, this aspect is not limited to a specific standard (e.g., VVC or HEVC) and can be applied, for example, to other standards and recommendations, as well as any extensions of such standards and recommendations. Unless otherwise stated or technically excluded, the aspects described in this application may be used alone or in combination.

[0120] Various numerical values ​​are used in this application. The specific numerical values ​​are exemplary, and the aspects described are not limited to these specific numerical values.

[0121] Various implementations involve decoding. As used herein, “decoding” can encompass all or part of the process performed, for example, on a received encoded sequence to produce a final output suitable for display. In various embodiments, these processes include one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. Whether the phrase “decoding process” is intended to specifically refer to a subset of operations or generally to a broader decoding process will be clear from the specific description and will be well understood by those skilled in the art.

[0122] Various implementations involve encoding. Similar to the discussion of "decoding" above, the term "encoding," as used in this application, can encompass all or part of the process performed on input video data to produce an encoded bitstream. Furthermore, the terms "reconstruction" and "decoding" are used interchangeably, as are "encoded" and "coded," and the terms "image," "picture," and "frame" are used interchangeably. Typically, but not necessarily, the term "reconstruction" is used at the encoder end, while the term "decoding" is used at the decoder end.

[0123] It should be noted that the grammatical elements used in this article are descriptive terms. Therefore, the use of other grammatical element names is not excluded.

[0124] This disclosure describes various types of information, such as syntax, that can be transmitted or stored. This information can be packaged or arranged in various ways, including those common in video standards, such as placing the information in SPS, PPS, NAL units, headers (e.g., NAL unit headers, or stripe headers), or SEI messages. Other methods are also available, including those common in system-level or application-level standards, such as signaling information to one or more of the following: a. SDP (Session Description Protocol), a format for describing multimedia communication sessions, used for session announcements and session invitations (e.g., as described in the RFC and used in conjunction with RTP (Real-Time Transport Protocol) transmission).

[0125] b. DASH MPD (Media Presentation Description) descriptors, such as those used in DASH and transmitted via HTTP. Descriptors are associated with representations or sets of representations to provide additional characteristics to the content representation.

[0126] c. RTP header extensions, for example, used during RTP streaming.

[0127] d. ISO basic media file format, for example, uses and employs boxes in OMAF, which are object-oriented building blocks (also called "atoms" in some specifications) defined by unique type identifiers and lengths.

[0128] e. HLS (HTTP Live Streaming) manifests, transmitted via HTTP. Manifests can be associated with, for example, versions or sets of versions of content to provide characteristics of that version or set of versions.

[0129] The implementations and aspects described herein can be implemented, for example, as methods or procedures, apparatus, software programs, data streams, or signals. Even if discussed only in the context of a single implementation form (e.g., discussed only as a method), the features under discussion can be implemented in other forms (e.g., apparatus or program). Apparatus can be implemented, for example, in appropriate hardware, software, and firmware. Methods can be implemented, for example, in apparatus, such as processors, which generally refer to processing devices, including computers, microprocessors, integrated circuits, or programmable logic devices. Processors also include communication devices, such as computers, mobile phones, portable / personal digital assistants (PDAs), and other devices that facilitate information communication between end users.

[0130] References to “aspect / one aspect”, “an embodiment / one embodiment”, or “an implementation / one implementation” and their variations mean that a particular feature, structure, characteristic, etc., described in connection with that aspect / an embodiment / implementation is included in at least one embodiment. Therefore, the phrases “in an aspect / one aspect”, “in an embodiment / one embodiment”, or “in an implementation / one implementation” and any other variations appearing throughout this application do not necessarily refer to the same embodiment.

[0131] Furthermore, this application may refer to "determining" various types of information. Determining information may include, for example, one or more of the following: estimation information, calculation information, prediction information, or information retrieved from memory.

[0132] Furthermore, this application may refer to "accessing" various types of information. Accessing information may include, for example, receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information, or one or more of these.

[0133] Furthermore, this application may refer to "receiving" various types of information. Like "accessing," "receiving" is intended to be a broad term. Receiving information may, for example, include accessing information or retrieving information (e.g., from memory). Moreover, "receiving" is generally involved in some way during operation, such as storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0134] It should be understood that the use of any of the following symbols " / ", "and / or", and "...at least one of", such as in "A / B", "A and / or B", and "at least one of A and B", is intended to cover selecting only the first listed option (A), or only the second listed option (B), or both options (A and B). As a further example, in the cases of "A, B, and / or C" and "at least one of A, B, and C", such wording is intended to cover selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A, B, and C). This will be apparent to those skilled in the art and related fields and can be extended to any number of the listed items.

[0135] Furthermore, as used herein, the term "signal" refers, among other things, to instruct the corresponding decoder to do something. For example, in some embodiments, the encoder signals the quantization parameters used for dequantization. Thus, in one embodiment, the same parameter is used on both the encoder and decoder sides. Therefore, for example, the encoder can transmit (explicitly signal) a specific parameter to the decoder so that the decoder can use the same specific parameter. Conversely, if the decoder already has that specific parameter as well as other parameters, it can signal without transmitting (implicitly signal) to simply let the decoder know and select that specific parameter. Bit savings are achieved in many embodiments by avoiding the transmission of any actual data. It should be understood that signaling can be implemented in various ways. For example, in many embodiments, one or more syntax elements, flags, etc., are used to signal information to the corresponding decoder. Although the foregoing refers to the verb form of the term "signal," the term "signal" can also be used as a noun herein.

[0136] It will be apparent to those skilled in the art that implementations can generate various signals that are formatted to carry information, such as information that can be stored or transmitted. This information may include, for example, instructions for performing a method, or data generated by one of the described implementations. For example, the signal may be formatted to carry a bitstream of the described embodiment. Such a signal may be formatted, for example, as an electromagnetic wave (e.g., using a portion of the radio frequency spectrum) or a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal can be transmitted via a variety of different wired or wireless links. The signal may be stored on a processor-readable medium.

Claims

1. A method for decoding video data, comprising: Decoding video blocks of the video data from the bitstream, the decoding including: Construct a candidate list of Local Illumination Compensation (LIC) models. Select a LIC model from the LIC model candidate list, and The selected LIC model is applied to the reference block of the video block to generate a prediction block for the video block.

2. The method according to claim 1, wherein The construction includes constructing a list of LIC model candidate pairs; The selection includes selecting a pair of LIC models from the list of candidate LIC models; and The application includes applying the LIC model to the corresponding reference block of the video block to generate a bidirectional prediction block for the video block.

3. The method according to claim 2, further comprising: The index is decoded from the bitstream, wherein the LIC model pair is selected from the LIC model candidate list based on the decoded index.

4. The method according to claim 2, further comprising: Decoding the first and second indices from the bitstream, wherein selecting the LIC model pair includes selecting the first LIC model and the second LIC model: The first LIC model is selected from the LIC model pairs in the LIC model candidate pair list based on the decoded first index, and The second LIC model is selected from another LIC model pair in the list of LIC model candidate pairs based on the decoded second index.

5. The method according to claim 1, wherein The construction includes constructing a first list and a second list of LIC model candidates; The selection includes selecting a first LIC model from the first list and selecting a second LIC model from the second list; and The application includes applying the first LIC model and the second LIC model to the corresponding reference blocks of the video block to generate bidirectional prediction blocks for the video block.

6. The method of claim 1, wherein... The construction includes constructing a first list and a second list of LIC model candidates; and The selection includes: For each model in the first LIC model and the second LIC model, and the corresponding list in the first and second lists constructed. Decode a flag from the bitstream indicating whether the model should be selected from the corresponding list, and if the flag indicates that the model should be selected from the corresponding list, then select the model from the corresponding list; otherwise, deduce the model. The application includes applying the first LIC model and the second LIC model to the corresponding reference blocks of the video block to generate bidirectional prediction blocks for the video block.

7. The method according to any one of claims 2 to 6, further comprising: Decode from the bitstream a flag indicating whether the video block should be decoded in a bidirectional predictive LIC model merging mode.

8. A method for encoding video data, comprising: Encoding video blocks of the video data into a bitstream, wherein the encoding of the video blocks includes: Construct a candidate list of LIC models. Select a LIC model from the LIC model candidate list, and The LIC model is applied to a reference block of the video block to generate a prediction block for the video block.

9. The method according to claim 8, further comprising: At least one parameter of the selected LIC model is refined based on the template region in the neighborhood of the video block and the reference block.

10. The method according to claim 8 or 9, further comprising: At least one parameter of the selected LIC model is refined based on a first distance between an image containing the video block and an image containing the reference block, and a second distance between an image containing the reference block and an image containing a reference block of the reference block.

11. The method according to any one of claims 8 to 10, further comprising: Refine at least one parameter of the selected LIC model based on parameter adjustment; as well as The adjustment parameters are encoded into the bitstream.

12. An apparatus for decoding video data, comprising: At least one processor; as well as A memory storing instructions that, when executed by the at least one processor, cause the device to: Decoding video blocks of the video data from the bitstream, the decoding including: Construct a candidate list of LIC models. Select a LIC model from the LIC model candidate list, and The selected LIC model is applied to the reference block of the video block to generate a prediction block for the video block.

13. The apparatus of claim 12, wherein the instructions further cause the apparatus to: Decode from the bitstream a flag indicating whether the video block should be decoded in LIC model merge mode.

14. The apparatus of claim 12, wherein a flag indicating whether the video block is to be decoded in LIC model merging mode is implicitly derived.

15. The apparatus according to any one of claims 12 to 14, wherein the instructions further cause the apparatus to: The index is decoded from the bitstream, wherein the selection of the LIC model from the LIC model candidate list is based on the decoded index.

16. The apparatus according to any one of claims 12 to 14, wherein the selection of the LIC model from the LIC model candidate list is based on an implicitly deduced index.

17. An apparatus for encoding video data, comprising: At least one processor; as well as A memory storing instructions that, when executed by the at least one processor, cause the device to: Encoding video blocks of the video data into a bitstream, wherein the encoding of the video blocks includes: Construct a candidate list of LIC models. Select a LIC model from the LIC model candidate list, and The LIC model is applied to a reference block of the video block to generate a prediction block for the video block.

18. The apparatus of claim 17, wherein the LIC model candidate list is composed of LIC models and includes: LIC model for spatially adjacent neighboring blocks; LIC model for spatially non-adjacent neighboring blocks; LIC model for temporally adjacent blocks; LIC model for shifted temporally adjacent blocks; history-based LIC model; default LIC model; or a combination thereof.

19. The apparatus of claim 17 or 18, wherein the LIC model candidate list is composed of LIC models and includes: The LIC model of the corresponding reference block pointed to by the motion vector associated with the sub-block of the video block.

20. The apparatus according to any one of claims 17 to 19, wherein the LIC model candidate list is composed of LIC models and includes: A LIC model of a reference block to which motion vectors, derived from one or more motion vectors associated with the various sub-blocks of the video block, point.

21. The apparatus of any one of claims 17 to 20, wherein the candidates in the LIC model candidate list are reordered based on the corresponding cost of applying the candidates to the template region of the video block.

22. The apparatus according to any one of claims 17 to 21, wherein the instructions further cause the apparatus to: Syntax elements indicating the size of the LIC model candidate list are encoded into the bitstream, and these syntax elements are encoded in the sequence parameter set (SPS), viewpoint parameter set (VPS), picture parameter set (PPS), adaptive parameter set (APS), or picture header (PH) of the encoded video data.

23. The apparatus according to any one of claims 17 to 21, wherein the size of the LIC model candidate list is implicitly determined.

24. A non-transitory computer-readable medium comprising instructions executable by at least one processor to perform a method for decoding video data, the method comprising: Decoding video blocks of the video data from the bitstream, the decoding including: Construct a candidate list of LIC models. Select a LIC model from the LIC model candidate list, and The selected LIC model is applied to the reference block of the video block to generate a prediction block for the video block.

25. A non-transitory computer-readable medium comprising instructions executable by at least one processor to perform a method for encoding video data, the method comprising: Encoding video blocks of the video data into a bitstream, wherein the encoding of the video blocks includes: Construct a candidate list of LIC models. Select a LIC model from the LIC model candidate list, and The LIC model is applied to a reference block of the video block to generate a prediction block for the video block.

Images