Video encoding / decoding method and device, and recording medium storing bitstream

Abstract

In an image decoding method and device according to the present disclosure, the method may comprise the steps of: deriving an illumination compensation parameter for a current block; and performing inter prediction on the basis of the illumination compensation parameter, wherein the deriving is performed on the basis of a predefined inter prediction mode.

Description

Video encoding / decoding method and device and recording medium storing bitstream

[0001] The present invention relates to a video signal processing method and device.

[0002] The market demand for high-resolution video is growing, necessitating technologies capable of efficiently compressing high-resolution images. To address this market need, the ISO / IEC's Moving Picture Expert Group (MPEG) and the ITU-T's Video Coding Expert Group (VCEG) jointly formed the Joint Collaborative Team on Video Coding (JCT-VC). They completed development of the HEVC (High Efficiency Video Coding) video compression standard in January 2013 and have been actively conducting research and development on next-generation compression standards.

[0003] Video compression largely consists of intra-prediction, inter-prediction, transform, quantization, entropy coding, and in-loop filtering. Among these, inter-prediction refers to a technology that generates a prediction block for the current block based on adjacent frames that have been encoded / decoded. The encoder encodes the inter-prediction mode used for inter-prediction, and the decoder decodes the encoded inter-prediction mode to perform inter-prediction.

[0004] The present disclosure seeks to provide a method and apparatus for performing inter prediction for a current block by deriving a luminance compensation parameter.

[0005] The present disclosure provides a method and device for deriving a second direction illuminance compensation parameter based on a first direction illuminance compensation parameter.

[0006] The present disclosure provides a method and apparatus for subsampling a template for deriving a luminance compensation parameter.

[0007] The image decoding method and device according to the present disclosure include the steps of deriving an illumination compensation parameter for a current block; and the step of performing inter prediction based on the illumination compensation parameter, wherein the derivation can be performed based on a predefined inter prediction mode.

[0008] In the video decoding method and device according to the present disclosure, the predefined inter prediction mode may include a merge mode or an AMVP mode.

[0009] In the image decoding method and device according to the present disclosure, the illumination compensation parameter can be derived based on at least one candidate included in the inter prediction candidate list.

[0010] In the image decoding method and device according to the present disclosure, the illumination compensation parameter may include at least one of an illumination compensation parameter in a first direction or an illumination compensation parameter in a second direction.

[0011] In the image decoding method and device according to the present disclosure, at least one of the first direction illumination compensation parameter or the second direction illumination compensation parameter can be derived based on a template of a current block and a template of a surrounding block.

[0012] In the image decoding method and device according to the present disclosure, the illumination compensation parameter in the second direction can be derived based on the illumination compensation parameter in the first direction.

[0013] In the image decoding method and device according to the present disclosure, the size of the template can be determined based on the size of the current block.

[0014] In the image decoding method and device according to the present disclosure, at least one of the first direction illumination compensation parameter or the second direction illumination compensation parameter can be derived based on subsampled samples within the template.

[0015] A video encoding method and device according to the present disclosure comprises the steps of deriving an illumination compensation parameter for a current block; and performing inter prediction based on the illumination compensation parameter, wherein the derivation can be performed based on a predefined inter prediction mode.

[0016] A computer-readable recording medium according to the present disclosure can store a bitstream encoded by the image encoding method.

[0017] According to the present disclosure, the complexity of a decoder / encoder can be reduced by deriving a light compensation parameter based on at least one candidate constituting an inter prediction candidate list.

[0018] According to the present disclosure, when performing prediction for both directions, the complexity of the encoder / decoder can be reduced by omitting the process of deriving the illumination compensation parameters for at least one direction.

[0019] According to the present disclosure, compression efficiency can be improved by subsampling a template for deriving a luminance compensation parameter.

[0020] FIG. 1 is a block diagram showing an image encoding device according to the present disclosure.

[0021] FIG. 2 is a block diagram showing an image decoding device according to the present disclosure.

[0022] FIG. 3 illustrates a method for performing inter prediction for a current block as an embodiment according to the present disclosure.

[0023] FIG. 4 illustrates an embodiment of subsampling a template of a current block and templates of surrounding blocks to derive a luminance compensation parameter according to the present disclosure.

[0024] The image decoding method and device according to the present disclosure include a step of deriving an illumination compensation parameter for a current block; and a step of performing inter prediction based on the illumination compensation parameter, wherein the derivation can be performed based on a predefined inter prediction mode.

[0025] In the video decoding method and device according to the present disclosure, the predefined inter prediction mode may include a merge mode or an AMVP mode.

[0026] In the image decoding method and device according to the present disclosure, the illumination compensation parameter can be derived based on at least one candidate included in an inter prediction candidate list.

[0027] In the image decoding method and device according to the present disclosure, the illumination compensation parameter may include at least one of an illumination compensation parameter in a first direction or an illumination compensation parameter in a second direction.

[0028] In the image decoding method and device according to the present disclosure, at least one of the first direction illumination compensation parameter or the second direction illumination compensation parameter can be derived based on a template of a current block and a template of a surrounding block.

[0029] In the image decoding method and device according to the present disclosure, the illumination compensation parameter in the second direction can be derived based on the illumination compensation parameter in the first direction.

[0030] In the image decoding method and device according to the present disclosure, the size of the template can be determined based on the size of the current block.

[0031] In the image decoding method and device according to the present disclosure, at least one of the first direction illumination compensation parameter or the second direction illumination compensation parameter can be derived based on subsampled samples within the template.

[0032] A video encoding method and device according to the present disclosure comprises the steps of: deriving an illumination compensation parameter for a current block; and performing inter prediction based on the illumination compensation parameter, wherein the derivation can be performed based on a predefined inter prediction mode.

[0033] A computer-readable recording medium according to the present disclosure can store a bitstream encoded by the image encoding method.

[0034] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings attached to this specification so that those skilled in the art can easily implement the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In addition, in the drawings, parts irrelevant to the description have been omitted to clearly explain the present invention, and similar parts have been designated with similar reference numerals throughout the specification.

[0035] Throughout this specification, when a part is said to be 'connected' to another part, this includes not only cases where they are directly connected, but also cases where they are electrically connected with another element in between.

[0036] Additionally, whenever a part throughout this specification is said to "include" a component, this does not mean that other components are excluded, but rather that other components may be included, unless specifically stated otherwise.

[0037] Additionally, while terms such as first, second, etc. may be used to describe various components, these components should not be limited by these terms. These terms are used solely to distinguish one component from another.

[0038] Additionally, in the embodiments of the devices and methods described herein, some components of the devices or some steps of the methods may be omitted. Furthermore, the order of some components of the devices or some steps of the methods may be changed. Furthermore, other components or other steps may be inserted into some components of the devices or some steps of the methods.

[0039] Additionally, some components or some steps of the first embodiment of the present invention may be added to the second embodiment of the present invention, or some components or some steps of the second embodiment may be replaced.

[0040] In addition, the components shown in the embodiments of the present invention are independently depicted to represent different characteristic functions, and this does not mean that each component is composed of separate hardware or a single software component. That is, each component is described by listing each component for convenience of explanation, and at least two components among each component may be combined to form a single component, or a single component may be divided into multiple components to perform a function. Such integrated and separate embodiments of each component are also included in the scope of the present invention as long as they do not deviate from the essence of the present invention.

[0041] In this specification, a block can be variously expressed as a unit, an area, a unit, a partition, etc., and a sample can be variously expressed as a pixel, a pel, a pixel, etc.

[0042] Hereinafter, embodiments of the present invention will be described in more detail with reference to the attached drawings. In describing the present invention, duplicate descriptions of identical components will be omitted.

[0043] FIG. 1 is a block diagram showing an image encoding device according to the present disclosure.

[0044] Referring to FIG. 1, a video encoding device (100) may include a picture segmentation unit (110), a prediction unit (120, 125), a transformation unit (130), a quantization unit (135), a reordering unit (160), an entropy encoding unit (165), an inverse quantization unit (140), an inverse transformation unit (145), a filter unit (150), and a memory (155).

[0045] The picture segmentation unit (110) can segment the input picture into at least one processing unit. At this time, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). Hereinafter, in the embodiments of the present disclosure, the coding unit may be used to mean a unit that performs encoding or a unit that performs decoding.

[0046] A prediction unit may be divided into at least one square or rectangular shape of the same size within a single coding unit, or may be divided such that one prediction unit among the divided prediction units within a single coding unit has a different shape and / or size from another prediction unit. When a prediction unit that performs intra prediction based on a coding unit is generated and is not the minimum coding unit, intra prediction can be performed without being divided into a plurality of NxN prediction units.

[0047] The prediction unit (120, 125) may include an inter prediction unit (120) that performs inter prediction or inter prediction, and an intra prediction unit (125) that performs intra prediction or intra prediction. It may determine whether to use inter prediction or intra prediction for a prediction unit, and determine specific information (e.g., intra prediction mode, motion vector, reference picture, etc.) according to each prediction method. A residual value (residual block) between the generated prediction block and the original block may be input to the transformation unit (130). In addition, prediction mode information, motion vector information, etc. used for prediction may be encoded together with the residual value by the entropy encoding unit (165) and transmitted to the decoder.

[0048] The inter prediction unit (120) may predict a prediction unit based on information of at least one picture among the previous or subsequent pictures of the current picture, and in some cases, may predict a prediction unit based on information of a portion of an encoded region within the current picture. The inter prediction unit (120) may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

[0049] The reference picture interpolation unit can receive reference picture information from the memory (155) and generate pixel information less than an integer pixel from the reference picture. In the case of luminance pixels, a DCT-based 8-tap interpolation filter with different filter coefficients can be used to generate pixel information less than an integer pixel in units of 1 / 4 pixels. In the case of a chrominance signal, a DCT-based 4-tap interpolation filter with different filter coefficients can be used to generate pixel information less than an integer pixel in units of 1 / 8 pixels.

[0050] The motion prediction unit can perform motion prediction based on a reference picture interpolated by the reference picture interpolation unit. Various methods such as FBMA (Full search-based Block Matching Algorithm), TSS (Three Step Search), and NTS (New Three-Step Search Algorithm) can be used to derive a motion vector. The motion vector can have a motion vector value in units of 1 / 2 or 1 / 4 pixels based on the interpolated pixel. The motion prediction unit can predict the current prediction unit by using different motion prediction methods. Various methods such as Skip Mode, Merge Mode, AMVP Mode, Affine Mode, and Affine Merge Mode can be used as motion prediction methods.

[0051] The intra prediction unit (125) can generate a prediction unit based on reference pixel information surrounding the current block, which is pixel information within the current picture. If the surrounding block of the current prediction unit is a block on which inter prediction has been performed and the reference pixel is a pixel on which inter prediction has been performed, the reference pixel included in the block on which inter prediction has been performed can be replaced and used with reference pixel information of the surrounding block on which intra prediction has been performed. That is, if the reference pixel is not available, the unavailable reference pixel information can be replaced and used with at least one reference pixel among the available reference pixels.

[0052] Additionally, a residual block containing residual value information, which is the difference between the prediction unit that performed the prediction based on the prediction unit generated in the prediction unit (120, 125) and the original block of the prediction unit, can be generated. The generated residual block can be input to the transformation unit (130).

[0053] In the transformation unit (130), the residual block including the residual value information of the prediction unit generated through the original block and the prediction unit (120, 125) can be transformed using a transformation method such as DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), or KLT. Whether to apply DCT, DST, or KLT to transform the residual block can be determined based on the intra prediction mode information of the prediction unit used to generate the residual block.

[0054] The quantization unit (135) can quantize values ​​converted to the frequency domain by the transformation unit (130). The quantization coefficients may vary depending on the block or the importance of the image. The values ​​produced by the quantization unit (135) can be provided to the dequantization unit (140) and the reordering unit (160).

[0055] The rearrangement unit (160) can perform rearrangement of coefficient values ​​for quantized residual values.

[0056] The rearrangement unit (160) can change a two-dimensional block-shaped coefficient into a one-dimensional vector form through a coefficient scanning method. For example, the rearrangement unit (160) can change the two-dimensional block-shaped coefficient into a one-dimensional vector form by scanning from the DC coefficient to the coefficient of the high-frequency region using a zig-zag scan method. Depending on the size of the transformation unit and the intra prediction mode, a vertical scan that scans the two-dimensional block-shaped coefficient in the column direction or a horizontal scan that scans the two-dimensional block-shaped coefficient in the row direction may be used instead of the zig-zag scan. That is, depending on the size of the transformation unit and the intra prediction mode, it is possible to determine which scan method among the zig-zag scan, the vertical scan, and the horizontal scan is to be used.

[0057] The entropy encoding unit (165) can perform entropy encoding based on the values ​​produced by the rearrangement unit (160). Entropy encoding can use various encoding methods such as, for example, Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC). In this regard, the entropy encoding unit (165) can encode residual value coefficient information of the encoding unit from the rearrangement unit (160) and the prediction units (120, 125). In addition, according to the present disclosure, it is possible to signal and transmit information indicating that motion information is derived and used on the decoder side and information on a technique used to derive motion information.

[0058] The inverse quantization unit (140) and the inverse transformation unit (145) inversely quantize the values ​​quantized in the quantization unit (135) and inversely transform the values ​​transformed in the transformation unit (130). The residual values ​​generated in the inverse quantization unit (140) and the inverse transformation unit (145) can be combined with the predicted prediction units predicted through the motion estimation unit, motion compensation unit, and intra prediction unit included in the prediction unit (120, 125) to generate a reconstructed block.

[0059] The filter unit (150) may include at least one of a deblocking filter, an offset correction unit, and an ALF (Adaptive Loop Filter). The deblocking filter may remove block distortion caused by boundaries between blocks in a restored picture. The offset correction unit may correct the offset from the original image on a pixel-by-pixel basis for the image on which deblocking has been performed. In order to perform offset correction for a specific picture, a method may be used in which the pixels included in the image are divided into a certain number of regions, the regions to be offset are determined, and the offset is applied to the regions, or the offset is applied by considering edge information of each pixel. The ALF (Adaptive Loop Filtering) may be performed based on a value obtained by comparing the filtered restored image with the original image. After dividing the pixels included in the image into a predetermined group, one filter to be applied to the group is determined, and filtering may be performed differentially for each group.

[0060] The memory (155) can store a restored block or picture produced through the filter unit (150), and the stored restored block or picture can be provided to the prediction unit (120, 125) when performing inter prediction.

[0061] FIG. 2 is a block diagram showing an image decoding device according to the present disclosure.

[0062] Referring to FIG. 2, the image decoding device (200) may include an entropy decoding unit (210), a rearrangement unit (215), an inverse quantization unit (220), an inverse transformation unit (225), a prediction unit (230, 235), a filter unit (240), and a memory (245).

[0063] When a video bitstream is input to a video encoding device, the input bitstream can be decoded in the opposite procedure to that of the video encoding device.

[0064] The entropy decoding unit (210) can perform entropy decoding in a procedure opposite to that of the entropy encoding unit of the video encoder. For example, various methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) can be applied in response to the method performed in the video encoder.

[0065] The entropy decoding unit (210) can decode information related to intra prediction and inter prediction performed in the encoder.

[0066] The reordering unit (215) can perform reordering based on the method by which the bitstream entropy-decoded by the entropy decoding unit (210) is reordered by the encoding unit. The coefficients expressed in the form of a one-dimensional vector can be reordered by restoring them back to coefficients in the form of a two-dimensional block.

[0067] The inverse quantization unit (220) can perform inverse quantization based on the quantization parameters provided by the encoder and the coefficient values ​​of the rearranged block.

[0068] The inverse transform unit (225) can perform inverse transform, i.e., inverse DCT, inverse DST, and inverse KLT, on the transforms performed by the transform unit, i.e., DCT, DST, and KLT, on the quantization result performed by the image encoder. The inverse transform can be performed based on the transmission unit determined by the image encoder. In the inverse transform unit (225) of the image decoder, a transform technique (e.g., DCT, DST, KLT) can be selectively performed according to a plurality of pieces of information, such as a prediction method, the size of the current block, and the prediction direction.

[0069] The prediction unit (230, 235) can generate a prediction block based on prediction block generation related information provided from the entropy decoding unit (210) and previously decoded block or picture information provided from the memory (245).

[0070] As described above, when performing intra prediction or intra prediction in the same manner as the operation in the image encoder, if the size of the prediction unit and the size of the transformation unit are the same, intra prediction for the prediction unit is performed based on the pixels on the left side of the prediction unit, the pixels on the upper left side, and the pixels on the upper side. However, when performing intra prediction, if the size of the prediction unit and the size of the transformation unit are different, intra prediction can be performed using reference pixels based on the transformation unit. In addition, intra prediction using NxN division only for the minimum coding unit can be used.

[0071] The prediction unit (230, 235) may include a prediction unit determination unit, an inter prediction unit, and an intra prediction unit. The prediction unit determination unit may receive various information such as prediction unit information input from the entropy decoding unit (210), prediction mode information of an intra prediction method, and motion prediction-related information of an inter prediction method, and may distinguish a prediction unit from a current encoding unit and determine whether the prediction unit performs inter prediction or intra prediction. On the other hand, if the encoder (100) does not transmit motion prediction-related information for the inter prediction, but instead transmits information indicating that motion information is to be derived and used on the decoder side and information on a technique used to derive motion information, the prediction unit determination unit determines whether the inter prediction unit (230) performs prediction based on the information transmitted from the encoder (100).

[0072] The inter prediction unit (230) can perform inter prediction on the current prediction unit based on information included in at least one picture among the previous picture or the subsequent picture of the current picture including the current prediction unit, using information required for inter prediction of the current prediction unit provided by the image encoder. In order to perform inter prediction, it can be determined based on the encoding unit whether the motion prediction method of the prediction unit included in the corresponding encoding unit is one of Skip Mode, Merge Mode, AMVP Mode, Affine Mode, and Affine Merge Mode.

[0073] The intra prediction unit (235) can generate a prediction block based on pixel information within the current picture. If the prediction unit is a prediction unit that has performed intra prediction, intra prediction can be performed based on intra prediction mode information of the prediction unit provided by the image encoder.

[0074] The intra prediction unit (235) may include an Adaptive Intra Smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter is a unit that performs filtering on the reference pixels of the current block and can determine whether to apply the filter based on the prediction mode of the current prediction unit and apply it. AIS filtering can be performed on the reference pixels of the current block using the prediction mode and AIS filter information of the prediction unit provided by the image encoder. If the prediction mode of the current block is a mode that does not perform AIS filtering, the AIS filter may not be applied.

[0075] The reference pixel interpolation unit can interpolate the reference pixel to generate a reference pixel of a pixel unit less than an integer value when the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on the pixel value interpolated from the reference pixel. When the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixel, the reference pixel may not be interpolated. The DC filter can generate a prediction block through filtering when the prediction mode of the current block is the DC mode.

[0076] The restored block or picture may be provided to a filter unit (240). The filter unit (240) may include a deblocking filter, an offset correction unit, and an ALF.

[0077] Information about whether a deblocking filter has been applied to a corresponding block or picture can be received from a video encoding device, and if a deblocking filter has been applied, information about whether a strong or weak filter has been applied. The deblocking filter of the video decoder can receive information related to the deblocking filter provided by the video encoder, and the video decoder can perform deblocking filtering on the corresponding block.

[0078] The offset correction unit can perform offset correction on the restored image based on the type of offset correction applied to the image during encoding and information on the offset value. ALF can be applied to the encoding unit based on information on whether ALF is applied and ALF coefficient information provided from the encoder. This ALF information can be provided by being included in a specific parameter set.

[0079] The memory (245) can store a restored picture or block so that it can be used as a reference picture or reference block, and can also provide the restored picture to an output unit.

[0080] Compensation is performed to reduce errors in prediction. Typically, compensation can refer to motion compensation or illumination compensation.

[0081] If illumination changes occur between adjacent frames of the current frame, even if motion estimation accurately locates the same object, residual values ​​can be high. Therefore, illumination compensation was developed to compensate for illumination changes across multiple frames.

[0082] Illumination compensation may include global illumination compensation or local illumination compensation.

[0083] One embodiment of the present disclosure, local illumination compensation, is a method for performing prediction on a current block based on a predetermined illumination compensation parameter. The illumination compensation parameter may be derived based on the illumination compensation parameter of a neighboring block of the current block. Alternatively, the illumination compensation parameter may be derived based on a linear equation (or linear model) derived from a template of the current block and templates of neighboring blocks. The illumination compensation parameter may be derived by the least square method. The illumination compensation parameter may include at least one of a weight or an offset.

[0084] The template of the current block may be a region adjacent to the current block, which may be a region that was previously decoded / encoded. For example, the template of the current block may include at least one of the upper region or the left region adjacent to the current block. The template of the surrounding block may be a region adjacent to the surrounding block, which may be a region that was previously decoded / encoded. For example, the template of the surrounding block may include at least one of the upper region or the left region adjacent to the surrounding block.

[0085] FIG. 3 illustrates a method for performing inter prediction based on illumination compensation as an embodiment according to the present disclosure.

[0086] Referring to Fig. 3, the illumination compensation parameters for the current block can be derived (S310).

[0087] The illumination compensation parameters can be derived based on the inter-prediction mode of the current block. Below, we will examine the cases where the inter-prediction mode of the current block is merge mode and AMVR mode, respectively.

[0088] Example 1

[0089] When merge mode is applied to the current block, a merge candidate list for the current block may be generated. The merge candidate list may include multiple merge candidates. The merge candidates may include at least one of a spatial merge candidate, a temporal merge candidate, or a history-based merge candidate.

[0090] Each merge candidate has motion information, which may include at least one of prediction direction information, a motion vector, or a reference picture index. At least one of the plurality of merge candidates may have an illumination compensation parameter. The illumination compensation parameter may refer to a local illumination compensation parameter.

[0091] Each merge candidate may have motion information in at least one direction, either the L0 direction or the L1 direction. The LO direction and the L1 direction may each point in either the forward or backward direction.

[0092] The luminance compensation parameters of a merge candidate can be derived based on the surrounding blocks. The encoder can also explicitly signal a flag to the decoder indicating whether luminance compensation should be performed on the surrounding blocks.

[0093] For example, a flag and / or illumination compensation parameter indicating whether to perform illumination compensation according to the merge candidate index value can be defined as in Table 1 below.

[0094] IndexL0Ref idx / (x,y)L1Ref idx / (x,y)LIC flagLIC parameterL0 / L100 / (68, -36)0 / (-68, 36)True(a0, b0) / (a1, b1)10 / (-72, -28)1 / (72, 28)True(a0, b0) / (a1, b1)21 / (27, -16)True(a0, b0)30 / (110, -31)1 / (74, -6)False-4-1 / (96, -37)True(a1, b1)50 / (-1, 0)0 / (4, 0)True(a0, b0) / (a1, b1)60 / (-72, -28)-False-… … … … … N0 / (-2, -32)0 / (2, 32)False-

[0095] Specifically, if illumination compensation is performed in a surrounding block, the illumination compensation parameters of the first merge candidate can be derived based on the illumination compensation parameters used in the surrounding block.

[0096] If illumination compensation is performed in one direction in the surrounding blocks, the illumination compensation parameters of the first merge candidate can be derived based on the illumination compensation parameters of the corresponding direction.

[0097] Alternatively, if illumination compensation has been performed on a neighboring block, the illumination compensation parameters used in the neighboring block can be modified to derive the illumination compensation parameters of the first merge candidate. For example, the illumination compensation parameters can be modified by calculating the illumination change amount of the current block and neighboring blocks. This calculation can be performed using SAD, Mean Reduced SAD (MRSAD), etc.

[0098] If the illumination compensation for both directions is performed in the surrounding blocks, the illumination compensation parameters of the first merge candidate can be derived based on the illumination compensation parameters for at least one of the first direction or the second direction. Here, the first direction can be either the L0 direction or the L1 direction, and the second direction can be the other direction.

[0099] Alternatively, if bidirectional illumination compensation is performed in the surrounding blocks, multiple illumination compensation parameters can be combined to derive the illumination compensation parameters of the first merge candidate. For example, a weighted sum operation can be performed on at least two illumination compensation parameters.

[0100] As another example, if illumination compensation is not performed on a surrounding block, a new illumination compensation parameter can be derived for the surrounding block, and the illumination compensation parameter of the first merge candidate can be derived based on the illumination compensation parameter. Here, the newly derived illumination compensation parameter for the surrounding block can be derived based on the illumination compensation parameter pre-defined in the encoder and decoder, or can be derived based on the signaled default parameter.

[0101] Alternatively, if the illumination compensation is not performed in the surrounding blocks, the illumination compensation parameters of the first merge candidate can be derived based on the illumination compensation parameters used in the adjacent blocks of the surrounding blocks.

[0102] Alternatively, if illumination compensation is not performed in the surrounding blocks, the illumination compensation parameters of the first merge candidate can be derived based on the illumination compensation parameters of the previously generated second merge candidate within the merge candidate list. For example, the illumination compensation parameters of the second merge candidate may be the illumination compensation parameters of the merge candidate that has motion information most similar to that of the first merge candidate.

[0103] Even though a merge candidate list is generated according to at least one of the above-described embodiments, if the illumination compensation parameter of the first merge candidate does not exist, the illumination compensation parameter of the first merge candidate can be derived based on the illumination compensation parameter of any merge candidate in the list. Here, the arbitrary merge candidate may be a merge candidate having a highest or lowest index in the merge candidate list. Or, it may be a merge candidate having a highest or lowest index in the reordered merge candidate list. The merge candidates in the merge candidate list can be reordered through ARMC-TM (Adaptive Reordering of Merge Candidate-Template Matching). Here, ARMC-TM may be a method of obtaining a template difference through template matching of a current block and surrounding blocks and reordering the merge candidates in the list based on the template difference.

[0104] The method for deriving the luminance compensation parameters of the merge candidates described above can be applied in the same / similar manner when constructing a candidate list based on a motion vector or a block vector.

[0105] A merge candidate list can be generated using the above-described method. The encoder can encode a merge index indicating one of a plurality of merge candidates in the merge candidate list and signal it to the decoder. The decoder can specify one of the plurality of merge candidates in the merge candidate list based on the signaled merge index. The illumination compensation parameter of the current block can be derived based on the illumination compensation parameter of the specified merge candidate.

[0106] Alternatively, the illumination compensation parameter of a merge candidate of an index other than the illumination compensation parameter corresponding to the merge index may be used.

[0107] Alternatively, in addition to the illumination compensation parameter corresponding to the merge index (hereinafter referred to as the first illumination compensation parameter), the illumination compensation parameter of the merge candidate of another index (hereinafter referred to as the second illumination compensation parameter) may be used.

[0108] For example, a new illuminance compensation parameter can be derived based on a combination of the first illuminance compensation parameter and the second illuminance compensation parameter. A new illuminance compensation parameter can be derived through a weighted sum of the first illuminance compensation parameter and the second illuminance compensation parameter.

[0109] Here, the weights for the weighted sum can be explicitly signaled or implicitly derived in the same manner in the encoder and decoder. Alternatively, the weights can be derived based on template differences (e.g., SAD, SATD) calculated via ARMC-TM.

[0110] Alternatively, the second light compensation parameter may be the light compensation parameter of the merge candidate having the highest or lowest index in the merge candidate list. Alternatively, the second light compensation parameter may be the light compensation parameter of the merge candidate having the highest or lowest index in the reordered merge candidate list.

[0111] Example 2

[0112] When AMVP mode is applied to the current block, an AMVP candidate list for the current block may be generated. The AMVP candidate list may include multiple AMVP candidates. The AMVP candidates may include at least one of a spatial candidate, a temporal candidate, or a history-based candidate.

[0113] Each AMVP candidate can have motion information including motion vectors.

[0114] Each AMVP candidate may have motion information in at least one direction, either the L0 direction or the L1 direction. The LO direction and the L1 direction may each point in either the forward or backward direction.

[0115] The illumination compensation parameters for the current block can be derived for either one direction or both directions.

[0116] When bidirectional prediction is performed on the current block, the illumination compensation parameters can be derived for the first direction and the second direction, respectively. Here, the first direction can be either the L0 direction or the L1 direction, and the second direction can be the other one. The illumination compensation parameters for the first direction can be derived based on a linear equation (or linear model) derived from a template of the current block and a template of a reference block (in particular, a reference block in the first direction). Here, the reference block can be specified based on a motion vector in the first direction with respect to the current block. The illumination compensation parameters for the second direction can be derived based on a linear equation (or linear model) derived from a template of the current block and a template of a reference block (in particular, a reference block in the second direction). Here, the reference block can be specified based on a motion vector in the second direction with respect to the current block.

[0117] Alternatively, when bidirectional prediction is performed for the current block, the illumination compensation parameter for the first direction can be derived through the above-described method, and the illumination compensation parameter for the second direction can be derived based on the derived illumination compensation parameter for the first direction.

[0118] For example, the illuminance compensation parameter for the second direction can be the same as the illuminance compensation parameter for the first direction. In other words, the illuminance compensation parameter for the second direction can be the same as the illuminance compensation parameter for the first direction.

[0119] For example, the illuminance compensation parameter for the second direction may have an opposite sign to the illuminance compensation parameter for the first direction. Both the weight and offset for the second direction may have an opposite sign to the first direction. Alternatively, either the weight or the offset for the second direction may have the same sign as the first direction, and the other may have a different sign than the first direction.

[0120] For example, the illumination compensation parameter for the second direction can be derived by adding a predetermined offset (L) to the illumination compensation parameter for the first direction. Here, L can be a real number with a positive or negative sign. The offset for the second direction can be derived by adding a predetermined offset to the offset for the first direction.

[0121] For example, the illumination compensation parameter for the second direction may be the illumination compensation parameter for the first direction multiplied by a predetermined scaling factor (K). Here, K may be a real number with a positive or negative sign. The weight for the second direction may be derived by multiplying the weight for the first direction by a predetermined scaling factor.

[0122] The above-described offset (L) and scaling factor (K) can be explicitly signaled from the encoder or implicitly derived from the decoder / decoder.

[0123] For example, the illumination compensation parameters for the second direction can be derived by interpolating and / or extrapolating a linear equation (or linear model) for deriving the illumination compensation parameters for the first direction.

[0124] For example, the illumination compensation parameter for the second direction can be derived through a dependent operation of the illumination compensation parameter for the first direction. The dependent operation can include a weighted sum operation based on weights and offsets. The weights for the weighted sum operation can be explicitly signaled by the encoder. Alternatively, they can be implicitly derived in the decoder / decoder from information about the weights.

[0125] For example, the illumination compensation parameter for the second direction can be derived by calculating the MRSAD for the first region for deriving the illumination compensation parameter for the first direction. For example, if the MRSAD of the first region for deriving the illumination compensation parameter for the first direction is greater than the MRSAD of the second region for deriving the illumination compensation parameter for the second direction, the weight can be modified to be closer to 1, and the offset can be modified to be closer to 0. The first region can include at least one of the template of the current block or the template of the reference block in the first direction. The second region can include at least one of the template of the current block or the template of the reference block in the second direction.

[0126] For example, the illumination compensation parameter for the first direction can be derived and set as an initial value, and the illumination compensation parameter for the second direction can be derived by modifying the initial value.

[0127] For example, based on samples of a first region for deriving illuminance compensation parameters for a first direction and a linear equation (or linear model) for deriving illuminance compensation parameters for a second direction, illuminance compensation parameters for a second direction can be derived. One or more samples belonging to the first region for deriving illuminance compensation parameters for the first direction can be added to the linear equation (or linear model) for deriving illuminance compensation parameters for the second direction, thereby deriving illuminance compensation parameters for the second direction.

[0128] When bidirectional prediction is performed on the current block, there may be multiple previously restored reference frames in the first direction and the second direction. When multiple illumination compensation parameters for the first direction are derived, illumination compensation parameters for the second direction can be derived based on the multiple illumination compensation parameters for the first direction.

[0129] For example, the illumination compensation parameter for the second direction may be the same as any one of the plurality of illumination compensation parameters for the first direction.

[0130] Alternatively, the illumination compensation parameter for the second direction can be derived by combining at least two of the plurality of illumination compensation parameters for the first direction.

[0131] With respect to the reference frame used to derive the illumination compensation parameters, the index of the reference frame for the second direction and the index of the reference frame for the first direction may be the same or different.

[0132] Referring to FIG. 3, inter prediction for the current block can be performed based on the illumination compensation parameters of the current block (S320).

[0133] Inter prediction can be performed on the current block based on a predetermined inter prediction mode. The predetermined inter prediction mode can be a merge mode, AMVP mode, Affine Merge mode, or Affine AMVP mode pre-defined in the decoder / decoder.

[0134] When a predetermined inter prediction mode is applied to the current block, inter prediction can be performed for the current block based on the illumination compensation parameters derived in S310.

[0135] Specifically, when the merge mode is applied to the current block, illumination compensation can be performed on the current block based on the illumination compensation parameters derived in S310.

[0136] Alternatively, when the merge mode is applied to the current block, the illumination compensation parameter derived from S310 may be set to an initial value, and the illumination compensation parameter set to the initial value may be modified. Illumination compensation may be performed for the current block based on the modified illumination compensation parameter.

[0137] Alternatively, the illumination compensation parameter of at least one merge candidate in the merge candidate list may be modified, and illumination compensation may be performed based on the modified illumination compensation parameter. For example, if the merge candidate has a bidirectional motion vector, the illumination compensation parameter of the merge candidate may be modified based on the difference in the Picture Order Count (POC) of each reference frame.

[0138] Meanwhile, in performing the illumination compensation, the index (hereinafter, the first index) used to derive motion information from the inter prediction candidate list and the index (hereinafter, the second index) used to derive the illumination compensation parameter may be different from each other. The first index and the second index may be independently encoded and signaled. Alternatively, either the first index or the second index may be derived based on the other. Alternatively, the second index may be a fixed value pre-defined in the encoder / decoder, which may indicate an inter prediction candidate at a pre-defined position within the inter prediction candidate list.

[0139] For example, the motion information of the current block can be derived based on the motion information of the Pth inter prediction candidate in the inter prediction candidate list, and the illumination compensation parameter of the current block can be derived based on the illumination compensation parameter of the Qth inter prediction candidate in the inter prediction candidate list.

[0140] Alternatively, the motion information of the current block may be derived based on the motion information of the Pth inter prediction candidate in the inter prediction candidate list, and the illumination compensation parameter of the current block may be derived based on the illumination compensation parameter of the Rth inter prediction candidate in the inter prediction candidate list. In this case, the illumination compensation parameter of the Rth inter prediction candidate may be derived based on a weighted sum operation of at least two or more of the illumination compensation parameters belonging to the inter prediction candidate list.

[0141] Meanwhile, according to the present disclosure, the template of the current block and / or the template of the surrounding blocks for deriving the illumination compensation parameter may have a predetermined size. For example, the size of the upper template of the current block and / or the surrounding blocks may be W×TA, and the size of the left template may be TL×H. Here, W and H may represent the width and height of the current block, respectively. In addition, TA and TL may represent the height of the upper template and the width of the left template, respectively.

[0142] For example, the size of the template may be determined as a fixed value in the encoder / decoder.

[0143] As another example, the size of the template can be derived as a variable value in the encoder / decoder.

[0144] For example, depending on the size of the current block, the TA and / or TL of the current block and / or surrounding blocks may be determined. At this time, TA may be set to be less than or equal to H or greater than or equal to H. TL may be set to be less than or equal to W or greater than or equal to W. Here, TA and TL may each be an integer greater than or equal to 1.

[0145] For example, depending on the ratio of the current block, the TA and / or TL of the current block and / or surrounding blocks may be determined.

[0146] Alternatively, the TA and / or TL of the current block and / or surrounding blocks can be determined randomly in the area where samples exist within the template.

[0147] Meanwhile, when determining the size of the template according to the aforementioned embodiment, if the upper template and the left template of the current block and / or surrounding blocks overlap, the overlapping samples can be removed. Alternatively, the overlapping samples can be used in duplicate.

[0148] Alternatively, the TA and / or TL of the current block and / or surrounding blocks may be determined based on the QP (Quantization Parameter) of the current frame and / or reference frame. For example, in a high QP section of a low-bitrate environment, the TA and / or TL of the current block and / or surrounding blocks may be determined as half of their respective initial values.

[0149] Alternatively, the TA and / or TL of the current block and / or surrounding blocks may be determined to be the same as the TA and / or TL of the template of the surrounding block on which the illumination compensation is performed.

[0150] As another example, when BCW is applied to surrounding blocks, the TA and / or TL of the current block and / or surrounding blocks may be determined based on the weights. For example, when the BCW indices of the surrounding blocks have unequal weights, the TA and / or TL of the current block and / or surrounding blocks may be determined to be twice the initial value.

[0151] FIG. 4 is a diagram illustrating an embodiment of subsampling a template of a current block and / or a template of a surrounding block to derive a luminance compensation parameter according to the present disclosure.

[0152] Fig. 4 (a) illustrates the entire template. As shown in Fig. 4 (a), the illumination compensation parameters can be derived using all samples within the template.

[0153] Alternatively, only some samples within the template can be subsampled, and the illumination compensation parameters can be derived based only on the subsampled samples. FIGS. 4 (b) to (f) illustrate embodiments of obtaining subsampled samples, excluding some of the samples within the template. As illustrated in FIGS. 4 (b) to (f), the illumination compensation parameters can be derived based on the subsampled samples within the template.

[0154] For example, as illustrated in FIG. 4 (b), some of the samples belonging to either an odd column or an odd row may be excluded from at least one of the top template or the left template.

[0155] For example, as illustrated in FIG. 4 (c), the subsampled samples may belong to one or more even columns and not to odd columns in at least one of the top template or the left template.

[0156] For example, as illustrated in FIG. 4 (d), the subsampled samples may belong to one or more even rows and not to odd rows in at least one of the top template or the left template.

[0157] For example, as illustrated in FIG. 4 (e), the subsampled samples may belong to one or more even rows of the upper template and may not belong to odd rows.

[0158] Additionally, as illustrated in FIG. 4 (e), the subsampled samples may belong to one or more even columns of the left template and may not belong to odd columns.

[0159] If the coordinates of the upper left sample of the current block are (0, 0), the even rows / columns and odd rows / columns described above can be defined as follows. An even row can mean a sample row to which samples whose y-axis coordinates are expressed as at least one of 0, -2, or -4 belong. An even column can mean a sample column to which samples whose x-axis coordinates are expressed as at least one of 0, -2, or -4 belong. An odd row can mean a sample row to which samples whose y-axis coordinates are expressed as at least one of -1, -3, or -5 belong. An odd column can mean a sample column to which samples whose y-axis coordinates are expressed as at least one of -1, -3, or -5 belong.

[0160] The subsampling factor can be explicitly signaled in the encoder or implicitly derived in the decoder / sub-decoder.

[0161] For example, the template may be subsampled using any of the following methods: 2D Nearest Neighbor, Bi-linear, or Bi-cubic. The subsampling may also be performed only if the size of the current block is greater than or equal to a threshold Z, or less than or equal to Z.

[0162] For example, the template may be subsampled for at least one of the horizontal, vertical, or diagonal units. The subsampling may be performed only if the size of the current block is greater than or equal to a threshold Z, or less than or equal to Z.

[0163] For example, a template may be subsampled only for samples that are not adjacent to the current block. The subsampled samples may be determined based on distance. The subsampling may also be performed only if the size of the current block is greater than or equal to a threshold Z, or less than or equal to Z.

[0164] For example, if the size of the current block is asymmetric, subsampling can be performed on the templates of the current block and surrounding blocks.

[0165] For example, whether to subsample and the subsampling method can be determined based on the QP of the current frame and the reference frame.

[0166] For example, if illumination compensation is applied to a surrounding block of the current block, the subsampled template area of ​​the surrounding block can be inherited and used in the current block.

[0167] For example, whether to subsample the template of the current block and the subsampling method can be determined based on the prediction mode derived from the surrounding blocks of the current block.

[0168] The various embodiments of the present disclosure are not intended to list all possible combinations but rather to illustrate representative aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combinations of two or more.

[0169] Additionally, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. In the case of hardware implementation, the embodiments may be implemented by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), general processors, controllers, microcontrollers, microprocessors, etc.

[0170] The scope of the present disclosure includes software or machine-executable instructions (e.g., operating systems, applications, firmware, programs, etc.) that cause operations according to the methods of various embodiments to be executed on a device or a computer, and a non-transitory computer-readable medium having such software or instructions stored thereon and executable on the device or computer.

Claims

1. A step of deriving the light compensation parameters for the current block; and Comprising a step of performing inter prediction based on the above illumination compensation parameters, An image decoding method, wherein the above-mentioned illumination compensation parameters are derived based on a predefined inter prediction mode.

2. In paragraph 1, A method for decoding an image, wherein the above-described inter prediction mode includes a merge mode or an AMVP mode.

3. In paragraph 1, An image decoding method, wherein the above-mentioned illuminance compensation parameter is derived based on at least one candidate included in an inter prediction candidate list.

4. In paragraph 1, An image decoding method, wherein the above illumination compensation parameter includes at least one of an illumination compensation parameter in a first direction or an illumination compensation parameter in a second direction.

5. In paragraph 4, An image decoding method, wherein at least one of the first direction illumination compensation parameter or the second direction illumination compensation parameter is derived based on a template of a current block and templates of surrounding blocks.

6. In paragraph 4, An image decoding method, wherein the second direction illumination compensation parameter is derived based on the first direction illumination compensation parameter.

7. In paragraph 5, A method for decoding an image, wherein the size of the above template is determined based on the size of the current block.

8. In paragraph 5, An image decoding method, wherein at least one of the first direction illumination compensation parameter or the second direction illumination compensation parameter is derived based on sub-sampled samples within the template.

9. A step of deriving the light compensation parameters for the current block; and Comprising a step of performing inter prediction based on the above illumination compensation parameters, A method for encoding an image, wherein the above derivation is performed based on a predefined inter prediction mode.

10. A computer-readable recording medium storing a bitstream generated based on the image encoding method according to Article 9.

Images