Image encoding / decoding method and apparatus, and recording medium having bitstream stored therein

Abstract

The present invention provides an image encoding method and an image decoding method. The image decoding method of the present invention may comprise the steps of: deriving inter prediction information of the current block; determining, for the current block, whether to perform weighted sum prediction on an inter prediction block and an intra prediction block; generating a prediction block of the current block on the basis of whether the weighted sum prediction is performed; and reconstructing the current block on the basis of the prediction block and a residual block of the current block.

Description

Video encoding / decoding method and device, and recording medium storing bitstream

[0001] The present invention relates to an encoder and a decoder, and more particularly, to a method and device for encoding and decoding for generating a prediction block of a current block.

[0002] The market demand for high-resolution video is growing, necessitating technologies capable of efficiently compressing high-resolution images. To address this market need, the ISO / IEC's Moving Picture Expert Group (MPEG) and the ITU-T's Video Coding Expert Group (VCEG) jointly formed the Joint Collaborative Team on Video Coding (JCT-VC), which completed development of the HEVC (High Efficiency Video Coding) video compression standard in January 2013 and has been actively conducting research and development on next-generation compression standards.

[0003] Technologies applied to video compression can broadly include intra-prediction, inter-prediction, transform, quantization, entropy coding, and / or in-loop filters. Meanwhile, as demand for high-resolution video increases, so does the demand for stereoscopic video content as a new video service. Accordingly, active discussions are underway on video compression technologies to effectively provide high-resolution and ultra-high-resolution stereoscopic video content.

[0004] The present disclosure provides a method and device for determining a reference frame for inter prediction of a current block.

[0005] The present disclosure provides a method and device for generating a prediction block of a current block by weighting inter and intra prediction signals.

[0006] The present disclosure provides a method and apparatus for determining a weighted sum mask that assigns differential weights based on spatial location in order to efficiently combine multiple prediction signals.

[0007] The present disclosure provides a method and apparatus for deriving motion model parameters for inter prediction.

[0008] The present disclosure provides a method and device for determining whether to perform weighted summation based on a signaled flag when the inter prediction mode of the current block is a warp prediction mode.

[0009] The present disclosure provides a method and device for determining an intra prediction mode of a current block when performing weighted sum using inter and intra prediction.

[0010] The present disclosure provides a method and apparatus for deriving a transform kernel of a residual block of a current block based on a current prediction block generated based on a weighted sum between inter and intra prediction signals.

[0011] The video decoding method and device according to the present disclosure can derive inter prediction information of a current block, determine whether to perform weighted sum prediction of an inter prediction block and an intra prediction block for the current block, generate a prediction block of the current block based on whether to perform the weighted sum prediction, and reconstruct the current block based on the prediction block and a residual block of the current block. Here, the inter prediction information can include at least one of reference frame information, a motion vector coding mode, or a motion mode of the current block, and the reference frame information can include prediction directionality information of the current block.

[0012] In the video decoding method and device according to the present disclosure, when the motion vector coding mode of the current block is Warped motion vector prediction (WARPMV), the translational parameters of the current block can be derived based on a warp parameter list, and the warp parameter list can be generated based on a reference block restored through warp prediction around the current block.

[0013] In the video decoding method and device according to the present disclosure, whether to perform weighted sum prediction may be determined based on a first flag indicating whether to perform weighted sum prediction, and whether to signal the first flag may be determined based on at least one of the width or height of the current block, the reference frame information, or the inter prediction information.

[0014] In the video decoding method and device according to the present disclosure, the signaling of the first flag may be omitted based on the fact that bidirectional prediction is performed for the current block according to the prediction direction information of the current block, and in this case, the value of the first flag may be set to 0.

[0015] In the video decoding method and device according to the present disclosure, the video decoding method, wherein the first flag is signaled based on the frame of the current block not being a TIP (Temporal Interpolated Prediction) frame according to the reference frame information.

[0016] In the video decoding method and device according to the present disclosure, the first flag may be signaled based on the motion vector coding mode of the current block being set to WARPMV and the motion mode of the current block being set to either the WARP_CASUAL mode or the WARP_DELTA mode.

[0017] In the video decoding method and device according to the present disclosure, the WARP_CASUAL mode may be a mode in which a non-translational parameter of the current block is derived based on a motion vector of a block reconstructed based on the same reference frame as the current block among pre-reconstructed reference blocks around the current block, according to a least squares regression method, and the WARP_DELTA mode may be a mode in which a non-translational parameter of the current block is derived based on a warp parameter list generated based on a reference block reconstructed through warp prediction around the current block.

[0018] In the video decoding method and device according to the present disclosure, when the motion vector coding mode of the current block is set to WARPMV, and the motion mode of the current block is set to either the WARP_CASUAL mode or the WARP_DELTA mode, and when the width and height of the current block are equal to sizes pre-defined for the encoder and decoder, the first flag may be signaled. Conversely, when the width and height of the current block are not equal to the pre-defined sizes, the signaling of the first flag may be omitted, and in this case, the value of the first flag may be set to 0.

[0019] In the video decoding method and device according to the present disclosure, when the motion vector coding mode of the current block is WARPMV and weighted sum prediction of the inter prediction block and the intra prediction block is performed, the intra prediction block can be generated based on the intra prediction mode, and the intra prediction mode can be any one of a DC mode, a vertical mode, a horizontal mode, or a smooth mode.

[0020] In the video decoding method and device according to the present disclosure, when weighted sum prediction of the inter prediction block and the intra prediction block is performed, the weighted sum prediction can be performed based on a weighted sum mask, and the weighted sum mask can be derived based on a weighted sum mask flag.

[0021] In the video decoding method and device according to the present disclosure, based on the value of the weighted sum mask flag being the first value, the weighted sum mask can be derived based on the intra prediction mode of the current block.

[0022] In the image decoding method and device according to the present disclosure, when the intra prediction mode of the current block is the DC mode, a weight mask can be derived in which the same proportion of weights is applied to the inter prediction block and the intra prediction block for all samples in the current block.

[0023] In the video decoding method and device according to the present disclosure, based on the value of the weighted sum mask flag being the second value, the weighted sum mask may be determined as one of N weighted sum masks that are identically defined in the encoder and decoder based on weighted sum mask information, and the weighted sum mask information may include a parameter for at least one of a direction, an angle, or a distance of a weighted sum boundary line. Here, N may be an integer greater than or equal to 1.

[0024] In the image decoding method and device according to the present disclosure, the residual block can be derived by applying a predetermined transformation kernel to transformation coefficients obtained from a bitstream, and the predetermined transformation kernel can be determined based on a HoG (Histogram of Gradient) generated based on reference samples around the current block.

[0025] The video encoding method and device according to the present disclosure may include the steps of deriving inter prediction information of a current block, determining whether to perform weighted sum prediction of an inter prediction block and an intra prediction block for the current block, generating a prediction block of the current block based on whether to perform the weighted sum prediction, and determining a transform kernel of the current block based on the prediction block of the current block. Here, the inter prediction information may include at least one of reference frame information, a motion vector coding mode, or a motion mode of the current block, and the reference frame information may include prediction directionality information of the current block.

[0026] A computer-readable storage medium according to the present disclosure can store a bitstream encoded by the image encoding method.

[0027] The bitstream transmission method and device according to the present disclosure can transmit a bitstream generated by the image encoding method.

[0028] According to the present disclosure, by signaling a flag for determining whether to perform bidirectional prediction during inter prediction based on at least one of the width or height of the current block, more efficient prediction is possible, thereby improving the compression performance of the encoder and decoder.

[0029] According to the present disclosure, by determining a weighted sum mask used for weighted sum of inter and intra prediction signals through a plurality of signaled parameters, more accurate prediction becomes possible, thereby improving the compression performance of an encoder and decoder.

[0030] According to the present disclosure, when generating an inter prediction block of a current block, more accurate prediction is possible by deriving translation parameters based on warp parameters of surrounding pre-restored reference blocks, thereby improving the compression performance of an encoder and decoder.

[0031] According to the present disclosure, by determining whether to signal a flag that determines whether to perform weighted summation of inter and intra prediction signals of a current block according to the motion vector coding mode of the current block, more efficient prediction becomes possible, thereby improving the compression performance of an encoder and decoder.

[0032] According to the present disclosure, by deriving an intra prediction mode of a current block based on a motion vector coding mode of the current block, more efficient prediction is possible, thereby improving the compression performance of an encoder and a decoder.

[0033] According to the present disclosure, a more adaptive transformation is possible by deriving a transformation kernel based on a Histogram of Gradient (HoG) generated based on surrounding reference samples for a residual block of a current block derived based on a prediction block of the current block generated by a weighted sum of inter and intra prediction signals, thereby improving the compression performance of an encoder and decoder.

[0034] FIG. 1 is a block diagram showing an image encoding device according to the present disclosure.

[0035] FIG. 2 is a block diagram showing an image decoding device according to the present disclosure.

[0036] FIG. 3 is a flowchart illustrating an image decoding method performed in an image decoding device according to the present disclosure.

[0037] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings attached to this specification so that those skilled in the art can easily implement the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In addition, in the drawings, parts irrelevant to the description have been omitted to clearly explain the present invention, and similar parts have been designated with similar reference numerals throughout the specification.

[0038] Throughout this specification, when a part is said to be 'connected' to another part, this includes not only cases where they are directly connected, but also cases where they are electrically connected with another element in between.

[0039] Additionally, whenever a part throughout this specification is said to "include" a component, this does not mean that other components are excluded, but rather that other components may be included, unless specifically stated otherwise.

[0040] Additionally, while terms such as "first," "second," etc. may be used to describe various components, these components should not be limited by these terms. These terms are used solely to distinguish one component from another.

[0041] Additionally, in the embodiments of the devices and methods described herein, some components of the devices or some steps of the methods may be omitted. Furthermore, the order of some components of the devices or some steps of the methods may be changed. Furthermore, other components or other steps may be inserted into some components of the devices or some steps of the methods.

[0042] The components shown in the embodiments of the present invention are depicted independently to represent different characteristic functions, and this does not mean that each component is composed of separate hardware or a single software component. That is, each component is described by listing each component for convenience of explanation, and at least two of the components may be combined to form a single component, or a single component may be divided into multiple components to perform a function. Such integrated and separate embodiments of each component are also included in the scope of the present invention, as long as they do not deviate from the essence of the present invention.

[0043] First, the terms used in this application are briefly explained as follows.

[0044] The decoding device (Video Decoding Apparatus) described below may be a device included in a server terminal such as a civilian security camera, a civilian security system, a military security camera, a military security system, a personal computer (PC), a laptop computer, a portable multimedia player (PMP), a wireless communication terminal, a smart phone, a TV application server, a service server, etc., and may refer to various devices including user terminals such as various devices, communication devices such as a communication modem for communicating with a wired or wireless communication network, a memory for storing various programs and data for decoding a video or predicting between or within a screen for decoding, and a microprocessor for executing a program for calculation and control.

[0045] In addition, an image encoded into a bitstream by an encoder can be transmitted to an image decoding device through wired or wireless communication networks such as the Internet, a local area network, a wireless LAN, a WiBro network, a mobile communication network, etc. in real time or non-real time, or through various communication interfaces such as a cable or a universal serial bus (USB), and can be decoded, restored into an image, and played back. Alternatively, the bitstream generated by the encoder can be stored in memory. The memory can include both volatile memory and non-volatile memory. In this specification, the memory can be expressed as a recording medium that stores a bitstream.

[0046] Typically, a video may be composed of a series of pictures, and each picture may be divided into coding units such as blocks. In addition, those skilled in the art to which this embodiment pertains will understand that the term "picture" described below may be replaced with other terms having equivalent meanings, such as "image" or "frame." In addition, those skilled in the art to which this embodiment pertains will understand that the term "coding unit" may be replaced with other terms having equivalent meanings, such as "unit block" or "block."

[0047] Hereinafter, embodiments of the present invention will be described in more detail with reference to the attached drawings. In describing the present invention, duplicate descriptions of identical components will be omitted.

[0048] Fig. 1 is a block diagram illustrating a video encoding device according to the present invention. Referring to Fig. 1, a conventional video encoding device (100) may include a picture segmentation unit (110), a prediction unit (120, 125), a transformation unit (130), a quantization unit (135), a rearrangement unit (160), an entropy encoding unit (165), an inverse quantization unit (140), an inverse transformation unit (145), a filter unit (150), and a memory (155).

[0049] The picture segmentation unit (110) can segment the input picture into at least one processing unit. At this time, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). In the following embodiments of the present invention, the coding unit may be used to mean a unit that performs encoding or a unit that performs decoding.

[0050] A prediction unit may be divided into at least one square or rectangular shape of the same size within a single coding unit, or may be divided such that one prediction unit among the divided prediction units within a single coding unit has a different shape and / or size from another prediction unit. When a prediction unit that performs intra prediction based on a coding unit is generated and is not the minimum coding unit, intra prediction can be performed without being divided into a plurality of NxN prediction units.

[0051] The prediction unit (120, 125) may include an inter prediction unit (120) that performs inter prediction or inter-screen prediction, and an intra prediction unit (125) that performs intra prediction or intra-screen prediction. It may determine whether to use inter prediction or intra prediction for a prediction unit, and determine specific information (e.g., intra prediction mode, motion vector, reference picture, etc.) according to each prediction method. A residual value (residual block) between the generated prediction block and the original block may be input to the transformation unit (130). In addition, prediction mode information, motion vector information, etc. used for prediction may be encoded together with the residual value by the entropy encoding unit (165) and transmitted to the decoder. However, when the motion information derivation technique on the decoder side according to the present invention is applied, the encoder does not generate the prediction mode information, motion vector information, etc., and therefore the corresponding information is not transmitted to the decoder. On the other hand, it is possible to signal and transmit information indicating that motion information is to be derived and used from the decoder side and information about the technique used to derive the motion information from the encoder.

[0052] The inter prediction unit (120) may predict a prediction unit based on information of at least one picture among the previous or subsequent pictures of the current picture, and in some cases, may predict a prediction unit based on information of a portion of an encoded region within the current picture. The inter prediction unit (120) may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

[0053] The reference picture interpolation unit can receive reference picture information from the memory (155) and generate pixel information less than an integer pixel from the reference picture. In the case of luminance pixels, a DCT-based 8-tap interpolation filter with different filter coefficients can be used to generate pixel information less than an integer pixel in units of 1 / 4 pixels. In the case of a chrominance signal, a DCT-based 4-tap interpolation filter with different filter coefficients can be used to generate pixel information less than an integer pixel in units of 1 / 8 pixels.

[0054] The motion prediction unit can perform motion prediction based on a reference picture interpolated by the reference picture interpolation unit. Various methods such as FBMA (Full search-based Block Matching Algorithm), TSS (Three Step Search), and NTS (New Three-Step Search Algorithm) can be used to derive a motion vector. The motion vector can have a motion vector value in units of 1 / 2 or 1 / 4 pixels based on the interpolated pixels. The motion prediction unit can predict the current prediction unit by using different motion prediction methods. Various methods such as the Skip method, the Merge method, the Advanced Motion Vector Prediction (AMVP) method, and the Intra Block Copy method can be used as the motion prediction method. In addition, when applying the motion information derivation technique on the decoder side according to the present invention, a template matching method and a bilateral matching method utilizing a motion trajectory can be applied as methods performed in the motion prediction unit.

[0055] The intra prediction unit (125) can generate a prediction unit based on reference pixel information surrounding the current block, which is pixel information within the current picture. If the surrounding block of the current prediction unit is a block on which inter prediction has been performed and the reference pixel is a pixel on which inter prediction has been performed, the reference pixel included in the block on which inter prediction has been performed can be replaced and used with reference pixel information of the surrounding block on which intra prediction has been performed. That is, if the reference pixel is not available, the unavailable reference pixel information can be replaced and used with at least one reference pixel among the available reference pixels.

[0056] Additionally, a residual block containing residual value information, which is the difference between the prediction unit that performed the prediction based on the prediction unit generated in the prediction unit (120, 125) and the original block of the prediction unit, can be generated. The generated residual block can be input to the transformation unit (130).

[0057] In the transformation unit (130), the residual block including the residual value information of the prediction unit generated through the original block and the prediction unit (120, 125) can be transformed using a transformation method such as DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), or KLT. Whether to apply DCT, DST, or KLT to transform the residual block can be determined based on the intra prediction mode information of the prediction unit used to generate the residual block.

[0058] The quantization unit (135) can quantize values ​​converted to the frequency domain by the transformation unit (130). The quantization coefficients can vary depending on the block or the importance of the image. The values ​​produced by the quantization unit (135) can be provided to the dequantization unit (140) and the reordering unit (160).

[0059] The rearrangement unit (160) can perform rearrangement of coefficient values ​​for quantized residual values.

[0060] The rearrangement unit (160) can change a two-dimensional block-shaped coefficient into a one-dimensional vector form through a coefficient scanning method. For example, the rearrangement unit (160) can change the two-dimensional block-shaped coefficient into a one-dimensional vector form by scanning from the DC coefficient to the coefficient of the high-frequency region using a zig-zag scan method. Depending on the size of the transformation unit and the intra prediction mode, a vertical scan that scans the two-dimensional block-shaped coefficient in the column direction or a horizontal scan that scans the two-dimensional block-shaped coefficient in the row direction may be used instead of the zig-zag scan. That is, depending on the size of the transformation unit and the intra prediction mode, it is possible to determine which scan method among the zig-zag scan, the vertical scan, and the horizontal scan is to be used.

[0061] The entropy encoding unit (165) can perform entropy encoding based on the values ​​produced by the rearrangement unit (160). Entropy encoding can use various encoding methods such as, for example, Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC). In this regard, the entropy encoding unit (165) can encode residual value coefficient information of an encoding unit from the rearrangement unit (160) and the prediction units (120, 125). In addition, according to the present invention, it is possible to signal and transmit information indicating that motion information is derived and used from the decoder side and information on a technique used to derive motion information.

[0062] The inverse quantization unit (140) and the inverse transformation unit (145) inversely quantize the values ​​quantized in the quantization unit (135) and inversely transform the values ​​transformed in the transformation unit (130). The residual values ​​generated in the inverse quantization unit (140) and the inverse transformation unit (145) can be combined with the predicted prediction units predicted through the motion estimation unit, motion compensation unit, and intra prediction unit included in the prediction unit (120, 125) to generate a reconstructed block.

[0063] The filter unit (150) may include at least one of a deblocking filter, an offset correction unit, and an ALF (Adaptive Loop Filter). The deblocking filter may remove block distortion caused by boundaries between blocks in a restored picture. The offset correction unit may correct the offset from the original image on a pixel-by-pixel basis for the image on which deblocking has been performed. In order to perform offset correction for a specific picture, a method may be used in which the pixels included in the image are divided into a certain number of regions, the regions to be offset are determined, and the offset is applied to the regions, or the offset is applied by considering edge information of each pixel. The ALF (Adaptive Loop Filtering) may be performed based on a value obtained by comparing the filtered restored image with the original image. After dividing the pixels included in the image into a predetermined group, one filter to be applied to the group is determined, and filtering may be performed differentially for each group.

[0064] The memory (155) can store a restored block or picture produced through the filter unit (150), and the stored restored block or picture can be provided to the prediction unit (120, 125) when performing inter prediction.

[0065] Figure 2 is a block diagram illustrating an image decoding device according to the present invention. Referring to Figure 2, an image decoder (200) may include an entropy decoding unit (210), a rearrangement unit (215), an inverse quantization unit (220), an inverse transformation unit (225), a prediction unit (230, 235), a filter unit (240), and a memory (245).

[0066] When a video bitstream is input to a video encoder, the input bitstream can be decoded in the opposite procedure to that of the video encoder.

[0067] The entropy decoding unit (210) can perform entropy decoding in a procedure opposite to that of the entropy encoding unit of the video encoder. For example, various methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) can be applied in response to the method performed in the video encoder.

[0068] The entropy decoding unit (210) can decode information related to intra prediction and inter prediction performed in the encoder.

[0069] The reordering unit (215) can perform reordering based on the method by which the bitstream entropy-decoded by the entropy decoding unit (210) is reordered by the encoding unit. The coefficients expressed in the form of a one-dimensional vector can be reordered by restoring them back to coefficients in the form of a two-dimensional block.

[0070] The inverse quantization unit (220) can perform inverse quantization based on the quantization parameters provided by the encoder and the coefficient values ​​of the rearranged block.

[0071] The inverse transform unit (225) can perform inverse transform, i.e., inverse DCT, inverse DST, and inverse KLT, on the transforms performed by the transform unit, i.e., DCT, DST, and KLT, on the quantization result performed by the image encoder. The inverse transform can be performed based on the transmission unit determined by the image encoder. In the inverse transform unit (225) of the image decoder, a transform technique (e.g., DCT, DST, KLT) can be selectively performed according to a plurality of pieces of information, such as a prediction method, the size of the current block, and the prediction direction.

[0072] The prediction unit (230, 235) can generate a prediction block based on prediction block generation related information provided from the entropy decoding unit (210) and previously decoded block or picture information provided from the memory (245).

[0073] As described above, when performing intra prediction or intra-screen prediction in the same manner as the operation in the video encoder, if the size of the prediction unit and the size of the transformation unit are the same, intra prediction for the prediction unit is performed based on the pixels on the left side of the prediction unit, the pixels on the upper left side, and the pixels on the upper side. However, when performing intra prediction, if the size of the prediction unit and the size of the transformation unit are different, intra prediction can be performed using reference pixels based on the transformation unit. In addition, intra prediction using NxN division only for the minimum coding unit can be used.

[0074] The prediction unit (230, 235) may include a prediction unit determination unit, an inter prediction unit, and an intra prediction unit. The prediction unit determination unit may receive various information such as prediction unit information input from the entropy decoding unit (210), prediction mode information of an intra prediction method, and motion prediction-related information of an inter prediction method, and may distinguish a prediction unit from a current encoding unit and determine whether the prediction unit performs inter prediction or intra prediction. On the other hand, if the encoder (100) does not transmit motion prediction-related information for the inter prediction, but instead transmits information indicating that motion information is to be derived and used on the decoder side and information on a technique used to derive motion information, the prediction unit determination unit determines whether the inter prediction unit (23) performs prediction based on the information transmitted from the encoder (100).

[0075] The inter prediction unit (230) can perform inter prediction on the current prediction unit based on information included in at least one picture among the previous picture or the next picture of the current picture including the current prediction unit, using information necessary for inter prediction of the current prediction unit provided by the video encoder. In order to perform inter prediction, it can be determined based on the coding unit whether the motion prediction method of the prediction unit included in the corresponding coding unit is one of Skip Mode, Merge Mode, AMVP Mode, and Intra Block Copy Mode. Alternatively, the inter prediction unit (230) can perform inter prediction by deriving motion information on its own from information provided by the video encoder that instructs the decoder to use the motion information and information on a technique used to derive the motion information.

[0076] The intra prediction unit (235) can generate a prediction block based on pixel information in the current picture. If the prediction unit is a prediction unit that has performed intra prediction, intra prediction can be performed based on intra prediction mode information of the prediction unit provided by the image encoder. The intra prediction unit (235) can include an AIS (Adaptive Intra Smoothing) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter is a part that performs filtering on the reference pixels of the current block, and can determine and apply whether to apply the filter according to the prediction mode of the current prediction unit. AIS filtering can be performed on the reference pixels of the current block using the prediction mode and AIS filter information of the prediction unit provided by the image encoder. If the prediction mode of the current block is a mode that does not perform AIS filtering, the AIS filter may not be applied.

[0077] The reference pixel interpolation unit can interpolate the reference pixel to generate a reference pixel of a pixel unit less than an integer value when the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on the pixel value interpolated from the reference pixel. When the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixel, the reference pixel may not be interpolated. The DC filter can generate a prediction block through filtering when the prediction mode of the current block is the DC mode.

[0078] The restored block or picture may be provided to a filter unit (240). The filter unit (240) may include a deblocking filter, an offset correction unit, and an ALF.

[0079] Information about whether a deblocking filter has been applied to a corresponding block or picture can be received from the video encoder, and if a deblocking filter has been applied, information about whether a strong or weak filter has been applied. The deblocking filter of the video decoder can receive information related to the deblocking filter provided by the video encoder, and the video decoder can perform deblocking filtering on the corresponding block.

[0080] The offset correction unit can perform offset correction on the restored image based on the type of offset correction applied to the image during encoding and information on the offset value. ALF can be applied to the encoding unit based on information on whether ALF is applied and ALF coefficient information provided from the encoder. This ALF information can be provided by being included in a specific parameter set.

[0081] The memory (245) can store a restored picture or block so that it can be used as a reference picture or reference block, and can also provide the restored picture to an output unit.

[0082] FIG. 3 is a flowchart illustrating an image decoding method performed in an image decoding device according to the present disclosure.

[0083] Referring to FIG. 3, inter prediction information of the current block can be derived (S300).

[0084] Inter prediction information of the current block may include at least one of reference frame information, motion vector coding mode, or motion mode.

[0085] The reference frame information may include at least one of information on the number of reference frames to be used by the current block (i.e., prediction direction information of the current block) and reference frame index information.

[0086] The prediction directionality information of the current block may include information regarding whether the current block is unidirectional or bidirectionally predicted. Whether the current block is unidirectionally or bidirectionally predicted may be determined based on the number of reference frames used by the current block, which may be one (unidirectional prediction) or two (bidirectional prediction). Furthermore, the number of reference frames may be determined by parsing the signaled bitstream.

[0087] For example, if the value of the signaled TIP (Temporal Interpolated Prediction) flag is 1, the L0 reference frame of the current block may be determined as a TIP frame, in which case unidirectional prediction may be performed on the current block. Alternatively, if the value of the TIP flag is 0 and segmentation-based inter prediction is performed on the current block, the L0 reference frame of the current block may be determined as a frame that is temporally closest to the current frame among frames preceding the current frame, and unidirectional prediction may be performed using the frame as a reference frame. The segmentation-based inter prediction may be a method of dividing the current block into multiple regions (segments) and then independently performing prediction on each segment.

[0088] As another example, at a level higher than the current block, it may be determined whether to perform 1) only unidirectional prediction, 2) only bidirectional prediction, or 3) select between unidirectional or bidirectional prediction for all blocks within that level, and the number of reference frames used by the current block may be determined based on the determination. In this case, if it is determined that unidirectional or bidirectional prediction is selectable at the level higher than the current block, the number of reference frames of the current block may be determined based on an additionally signaled flag.

[0089] As another example, a flag for prediction directionality information of the current block may be signaled based on at least one of the width or height of the current block. More specifically, if at least one of the width or height of the current block is greater than or equal to a predetermined size defined equally for the encoder and decoder, a flag for determining whether unidirectional or bidirectional prediction of the current block may be signaled. Conversely, if the width and height of the current block are less than the predetermined size, the signaling of the flag may be omitted, and the current block may be determined to perform unidirectional prediction.

[0090] When the number of reference frames used by the current block is determined, i.e., when it is determined whether to perform unidirectional or bidirectional prediction of the current block, a reference frame to be used for prediction among a reference frame list can be determined based on a signaled reference frame index. For example, the reference frame list of the current block can be constructed based on a value calculated between a reference frame in a decoded picture buffer (DPB) and a frame including the current block (hereinafter, referred to as the current frame). More specifically, a lower value can be placed at a higher position in the reference frame list, and a lower index number can be assigned. At this time, the value can be calculated based on a difference in a picture order count (POC) between the reference frame and the current frame, a difference in a frame-based quantization parameter (QP), etc.

[0091] A motion vector coding mode (MoVCM) is a method for representing motion information of the current block. It may be a mode that defines how motion vectors are determined and how information about the motion vectors is obtained. Depending on the mode, motion vectors may be explicitly signaled, derived, or selected from a list of candidates and used.

[0092] For example, the motion vector coding mode of the current block may be any one of a mode in which a global motion model determined at a higher level (e.g., frame, picture, etc.) than the current block is used for the current block (GLOBALMV), a mode in which a motion vector index and a motion vector difference (MVD) selected from a motion vector candidate list are signaled (NEWMV), a mode in which a motion vector index is signaled and an MVD is not signaled (NEARMV), a mode in which a motion vector index and an MVD with simplified precision are signaled (AMVDNEWMV), or a mode in which translational parameters for warp prediction of the current block are derived based on non-translational parameters of a reference block determined based on a signaled index from a warp parameter candidate list (Warped motion vector prediction, WARPMV).

[0093] Specifically, when the motion vector coding mode is GLOBALMV, the motion model of the current block can be determined based on the motion model derived from a higher level or its corresponding parameters. In addition, when the motion vector coding mode is NEWMV, the motion vector of the current block can be derived based on the signaled MVP (Motion Vector Predictor) index and MVD, and when the motion vector coding mode is NEARMV, the motion vector of the current block can be derived based on the signaled MVP index. In addition, when the motion vector coding mode is AMVDNEWMV, the motion vector of the current block can be derived based on the signaled MVP index and MVD index.

[0094] In addition, when the motion vector coding mode is WARPMV, the warp parameter, which is a motion model parameter for performing warp prediction of the current block, is composed of a translational parameter and a non-translational parameter, and each can be derived in a separate manner.

[0095] Among these, the translational parameters for warp prediction of the current block can be derived using a method that is identically defined for the encoder and decoder based on the derived warp parameters. For example, the translational parameters (e curr , f curr ) can be calculated as shown in the following mathematical formula 1.

[0096]

[0097] In mathematical expression 1, x center , y center can represent the x-axis coordinate and y-axis coordinate of the center position of the current block, respectively. Also, x' center , y' center can represent the x-axis coordinate and y-axis coordinate of the center position of the reference block within the reference frame, respectively.

[0098] The above warp parameters (a, b, c, d, e, and f) can be determined based on a signaled index among a warp parameter candidate list. Here, the warp parameter candidate list can be constructed based on warp parameters obtained from a reference block restored based on warp prediction around the current block. At this time, the warp parameter candidate list can be constructed based on a bank stored in a FIFO (First In First Out) format. Alternatively, if a motion vector of the current block exists, a warp parameter generated through regression based on the motion vector and the motion vector of a previously restored reference block around the current block can be included in the warp parameter candidate list. Alternatively, if a global warp parameter is defined at a higher level, the warp parameter candidate list can also be constructed based on the parameter, etc.

[0099] As another example, if the motion vector coding mode is WARPMV, the delta translational parameter (e) in the above mathematical expression 1 delta , f delta ), the translational parameter (e) for warp prediction of the current block is additionally considered. curr , f curr ) can be derived as in the following mathematical expression 2. At this time, the warp parameters (a, b, c, d, e, and f) may be warp parameters determined from among the warp parameter candidate list.

[0100]

[0101] In mathematical expression 2, e delta can represent the delta translational parameter for the x-axis, and f delta can represent the delta translational parameter for the y-axis.

[0102] Whether to consider the delta translational parameter may be determined based on a signaled flag. When the value of the flag is 1, the delta translational parameter may be signaled for each of the x-axis and the y-axis. Alternatively, when the value of the flag is 1, the direction information of the delta translational parameter may be signaled first, and then the delta translational parameter may be signaled for axes that are required according to the direction information, and signaling may be omitted for axes that are not required. That is, for each of the x-axis and the y-axis, information indicating whether the delta translational parameter is 0 (direction information of the delta translational parameter) may be signaled first, and then the delta translational parameter may be acquired for axes that are not 0. At this time, the delta translational parameter for each of the x-axis and the y-axis may be acquired based on the signaled index through a table that is identically defined in the encoder and decoder.

[0103] Meanwhile, when the motion vector coding mode is WARPMV, non-translational parameters for warp prediction of the current block can be derived based on the motion mode of the current block. The motion mode can be determined based on the motion vector coding mode of the current block, which may relate to the method of acquiring and applying motion information for prediction of the current block. Depending on the motion mode, the type and acquisition method of motion parameters used for prediction may vary.

[0104] For example, if the motion vector coding mode of the current block is WARPMV, the motion mode of the current block can be determined as any one of the WARP_CASUAL mode, the WARP_DELTA mode, or the WARP_INTRA mode, and the determination can be performed based on a signaled flag. Alternatively, if the number of reference blocks surrounding the current block that are reconstructed with reference to the same reference frame as the current block is less than a predetermined threshold, the signaling of the flag can be omitted and the motion mode of the current block can be determined as the WARP_DELTA mode.

[0105] First, when the motion mode of the current block is determined to be WARP_CASUAL mode, non-translational parameters for performing warp prediction of the current block can be derived based on pre-reconstructed reference blocks surrounding the current block using a method identically defined for the encoder and decoder. Here, the pre-reconstructed reference blocks may be N reference blocks reconstructed by referring to the same reference frame as the current block based on motion vectors using an inter prediction method. At this time, when the difference between the translational parameters of the current block and the motion vectors of the surrounding reference blocks is greater than or equal to a predetermined threshold, the reference block can be excluded from the process of deriving non-translational parameters of the current block.

[0106] For example, when the above N is 4, the non-translational parameters (a, b, c, and d) for performing warp prediction of the current block whose motion mode is WARP_CASUAL can be calculated through the least squares regression formula according to the following mathematical formulas 3 and 4, based on four reference blocks surrounding the current block.

[0107]

[0108]

[0109] In equations 3 and 4, (x k , y k ) may represent the center position coordinates of the k (0, 1, 2, 3, or 4)th block among the N reference blocks and the current block. Here, when k is 0, the block may mean the current block. In this case, the center position is the width of the kth block w k and the height is h k Assuming that (w k / 2 + 1, h k / 2 +1) or (w k / 2 - 1, h k / 2 - 1) can be either one of them.

[0110] Also, (x k ', y k ') may represent the motion vector (or translational parameter) of the k (0, 1, 2, 3, or 4)th block among the N reference blocks and the current block. Here, when k is 0, the block may mean the current block, and (x0', y0') is (e) derived according to mathematical expression 1 or mathematical expression 2. curr , f curr ) may be.

[0111] Meanwhile, if the motion mode of the current block is determined to be WARP_DELTA mode, the non-translational parameter for performing warp prediction of the current block can be derived by correcting one non-translational parameter determined based on a signaled index among the warp parameter candidate list by applying an additionally obtained delta parameter. The delta parameter can be obtained based on a signaled bitstream.

[0112] For example, if the motion mode of the current block is WARP_DELTA, a 4-parameter warp prediction including rotation and scale information can be performed, and the warp parameters can be in the form of the following mathematical expression 5.

[0113]

[0114] The above warp parameter is a delta parameter (a) determined based on a list of warp parameter candidates. delta and b delta ) can be applied to finally form a form as in the following mathematical expression 6.

[0115]

[0116] The above delta parameter (a delta and b delta ) can be determined through a signaled index based on a list that is identically defined for the encoder and decoder.

[0117] On the other hand, if the motion mode of the current block is determined to be WARP_INTRA mode, the non-translational parameters for performing warp prediction of the current block may be the warp parameters determined based on the signaled index among the warp parameter candidate list in the above-described mathematical expression 1 as they are. Alternatively, they may be calculated using the least squares regression formula according to the above-described mathematical expressions 3 and 4 based on the surrounding reference blocks.

[0118] The motion vector coding mode of the current block may be determined based on whether unidirectional or bidirectional prediction is performed on the current block. For example, if unidirectional prediction is performed on the current block, any one of GLOBALMV, NEWMV, NEARMV, AMVDNEWMV, or WARPMV may be determined as the motion vector coding mode of the current block based on the signaled bitstream.

[0119] As another example, when bidirectional prediction is performed on the current block, a motion vector coding mode for each reference frame (L0 and L1) can be derived based on a motion vector coding mode list that is identically defined for the encoder and decoder. More specifically, the motion vector coding mode list can be formed by combining all or part of the motion vector coding modes available for unidirectional prediction. Thereafter, one of the combinations is selected based on the signaled index, and thus a motion vector coding mode for each reference frame can be determined.

[0120] As another example, when bidirectional prediction is performed for the current block, the motion vector coding mode for each reference frame (L0, L1) can be determined as one of (NEARMV, NEARMV), (NEARMV, NEWMV), (NEWMV, NEARMV), (GLOBALMV, GLOVALMV), or (NEWMV, NEWMV).

[0121] As another example, when bidirectional prediction is performed on the current block, a motion vector coding mode may be used in which MVD information or an MVD index is signaled for one of the L0 reference frames or the L1 reference frame, and MVD information is derived from the other reference frame without separate signaling. In this case, the derived MVD information may be derived based on the MVD information or the MVD index of the signaled reference direction.

[0122] Referring to FIG. 3, it is possible to determine whether to perform weighted summation between inter and intra prediction signals of the current block (S310).

[0123] According to one embodiment of the present disclosure, whether weighted summation of inter and intra prediction signals (hereinafter, “inter / intra weighted summation”) is performed for a current block can be determined based on a first flag indicating whether inter / intra weighted summation is performed. In this case, if the value of the first flag is 1, it can indicate that inter / intra weighted summation is performed, and conversely, if the value of the first flag is 0, it can indicate that inter / intra weighted summation is not performed.

[0124] For example, the first flag may be signaled if the reference frame of the current block is not a TIP frame.

[0125] As another example, the first flag may be signaled if the current block is unidirectionally predicted by an inter prediction method. In other cases (e.g., bidirectional prediction), signaling may be omitted, and the value of the first flag may be set to 0.

[0126] Alternatively, the signaling of the first flag may be omitted, and the value of the first flag may be derived based on at least one of the width or height of the current block. In this case, the derived value may be 0.

[0127] As another example, the first flag may be additionally signaled when the motion vector coding mode of the current block is WARPMV and the motion mode is determined as WARP_CASUAL or WARP_DELTA. In this case, the first flag may be signaled when at least one of the width or the height of the current block is equal to a predetermined size defined equally in the encoder and the decoder, and in other cases, the signaling may be omitted and the corresponding value may be derived as 0. In addition, when the value of the first flag is 1, the signaling of the flag for determining whether to additionally consider the delta translational parameter to derive the translational parameter of the current block may be omitted and the corresponding flag's value may be derived as 0.

[0128] According to another embodiment of the present disclosure, whether inter / intra weighted summation is performed can be determined depending on whether the motion mode of the current block is WARP_INTRA mode.

[0129] For example, if the motion vector coding mode of the current block is WARPMV and the motion mode of the current block is determined to be WARP_INTRA mode based on the signaled index to determine the motion mode, inter / intra weighted summation can be performed.

[0130] Referring to FIG. 3, information for generating a prediction block of the current block can be derived based on whether weighted summation between inter and intra prediction signals is performed (S320).

[0131] When inter / intra weighted summation is performed on the current block, information for generating a prediction block of the current block may include at least one of information for generating an inter prediction block of the current block, information for deriving an intra prediction mode of the current block, or information for deriving a weighted summation mask used in weighted summation between the inter prediction block and the intra prediction block generated based on the intra prediction mode. Here, information for generating an inter prediction block of the current block may include information for deriving a motion model parameter for generating an inter prediction block.

[0132] The information for generating the inter prediction block may include a second flag indicating whether the inter prediction block is generated based on warp prediction. The second flag may be derived based on a value of the first flag. In this case, when the value of the second flag is 1, it may indicate that warp prediction is performed, and conversely, when the value of the second flag is 0, it may indicate that warp prediction is not performed. In addition, whether the second flag is signaled may be determined based on at least one of the width or height of the current block, or a motion vector coding mode of the current block.

[0133] For example, the second flag may be signaled when the motion vector coding mode of the current block is NEARMV, and may not be signaled otherwise. The second flag may be signaled when the motion vector coding mode of the current block is either NEARMV or AMVDNEWMV, and may not be signaled otherwise. If the signaling of the second flag is omitted, the value of the second flag may be derived as 0. In this case, if the value of the signaled second flag is 1, the translational parameter among the warp parameters for warp prediction of the current block may be set to be identical to the MV (motion vector) derived according to the motion vector coding mode.

[0134] In addition, when the value of the signaled second flag is 1, the non-translational parameter for warp prediction of the current block can be derived based on previously restored reference blocks surrounding the current block. For example, the non-translational parameter of a reference block restored based on warp prediction among the reference blocks can be set as the non-translational parameter of the current block. More specifically, the surrounding reference blocks can be searched according to the order defined in the encoder and decoder, and the non-translational parameter of the first reference block restored through warp prediction can be set as the non-translational parameter of the current block. Alternatively, when there are multiple reference blocks restored based on warp prediction, a warp parameter candidate list can be constructed, and the non-translational parameter of the current block can be derived based on the signaled index.

[0135] As another example, the non-translational parameters of the current block can be calculated using the least squares regression formula according to Equations 3 and 4 based on the MVs of the previously reconstructed reference blocks around the current block.

[0136] As another example, if the motion vector coding mode of the current block is WARPMV, the second flag may be signaled. More specifically, if the motion mode of the current block is a mode that derives non-translational parameters of the current block based on at least one of the warp parameters or motion vectors of a pre-reconstructed reference block located around the current block, the second flag may be signaled.

[0137] When weighted sum of inter and intra prediction signals is performed for the current block, information for generating a prediction block of the current block may include information for deriving an intra prediction mode of the current block.

[0138] The intra prediction mode of the current block may be determined according to criteria predefined for the encoder and decoder. For example, the intra prediction mode of the current block may be determined as either a DC mode or a smooth mode according to criteria equally defined for the encoder and decoder. Here, the smooth mode may be a mode in which the average of prediction values ​​generated based on the smooth vertical mode and the smooth horizontal mode is used as the prediction value of the current pixel, and the smooth vertical mode may be a mode in which the prediction value of the current pixel is generated through distance-based weighted interpolation using a reference sample located above and a reference sample located at the lower left with respect to the position of the current sample, and the smooth horizontal mode may be a mode in which the prediction value of the current pixel is generated through distance-based weighted interpolation using a reference sample located at the left and a reference sample located at the upper right with respect to the position of the current sample.

[0139] As another example, the intra prediction mode of the current block may be determined based on the signaled index as one of the modes defined identically for the encoder and decoder. In this case, the modes defined identically for the encoder and decoder may include at least one of a DC mode, a vertical mode, a horizontal mode, or a smooth mode.

[0140] Alternatively, the intra prediction mode of the current block may be determined based on the motion vector coding mode of the current block when inter / intra weighted sum prediction is performed on the current block. For example, if the motion vector coding mode of the current block is either WARPMV or NEARMV, any one of the intra prediction modes that are identically defined for the encoder and decoder based on the signaled index may be determined as the intra prediction mode of the current block. As another example, if the motion vector coding mode of the current block is either NEWMV or AMVDNEWMV, the signaling of the index may be omitted, and the intra prediction mode that is identically defined for the encoder and decoder may be set as the intra prediction mode of the current block.

[0141] Alternatively, the intra prediction mode of the current block may be derived based on surrounding reference blocks of the current block. For example, prediction may be performed on a template region of the current block based on the intra prediction mode of a reference block reconstructed through intra prediction among surrounding reference blocks, a template cost may be calculated for each prediction result, and then the intra prediction mode of the current block may be derived based on the template cost. More specifically, the intra prediction mode with the smallest template cost may be set as the intra prediction mode of the current block. Alternatively, a list of intra prediction mode candidates may be generated based on P intra prediction modes selected in descending order of template cost among the intra prediction modes of surrounding reference blocks, and the intra prediction mode of the current block may be determined based on a signaled index. The P may be a natural number greater than or equal to 1.

[0142] Prediction for the above template region can be performed using a reference line of the template region. At this time, the number of rows or columns of samples included in the reference line of the template region can be 1. In addition, the template region of the current block may be an L-shaped region formed by combining regions adjacent to the left and top of the current block as a region composed of pre-reconstructed samples located around the current block, and may be set to include at least one direction among the left and top, but not the other direction. The template cost can be calculated as at least one of the SAD (Sum of Absolute Difference) or SATD (Sum of Absolute Transformed Difference) between the pre-reconstructed samples in the template region and the prediction result.

[0143] When weighted summation of inter and intra prediction signals is performed for a current block, information for generating a prediction block of the current block may include information for deriving a weighted summation mask used in the weighted summation. The weighted summation mask includes a weight corresponding to each sample position of the current block, and may be used to determine the contribution of the inter prediction signal and the intra prediction signal for each sample. In addition, the weighted summation mask may include a boundary line that separates the contribution of the inter prediction signal and the intra prediction signal. The weighted summation boundary line may be a line that serves as a reference for distinguishing between samples in the current block in which the weight of the inter prediction signal is greater than the weight of the intra prediction signal, and vice versa. For example, with the weighted summation boundary line as a reference, a greater weight may be assigned to the inter prediction signal in one area of ​​the boundary line, and a greater weight may be assigned to the intra prediction signal in the opposite area.

[0144] Information for deriving a weighted sum mask may include a weighted sum mask flag. The weighted sum mask flag may indicate whether the weighted sum mask of the current block is determined based on the intra prediction mode of the current block.

[0145] For example, when the value of the weighted sum mask flag is the first value, a weighted sum mask may be derived based on the intra prediction mode of the current block. In this case, a weighted sum mask may be derived in which a higher weight is assigned to a sample predicted using the intra prediction mode as the distance from the reference block referred to when generating the intra prediction block of the current block is closer. Conversely, a lower weight may be assigned to a sample in the current block that is farther from the reference block. In this case, when the intra prediction mode of the current block is the DC mode, weights of the same proportion may be assigned to the inter prediction signal and the intra prediction signal for all samples in the current block.

[0146] As another example, when the value of the weighted sum mask flag is the second value, a weighted sum mask may be determined based on an additionally signaled bitstream. At this time, the bitstream may include a parameter for at least one of a direction, an angle, or a distance of a weighted sum boundary line. A weighted sum boundary line may be derived through the parameter obtained from the bitstream, and one of the weighted sum masks defined identically in the encoder and decoder based on the weighted sum boundary line may be determined as the weighted sum mask of the current block.

[0147] Here, the first value may be 1 and the second value may be 0, or the first value may be 0 and the second value may be 1.

[0148] The parameters for the direction, angle, or distance of the above weighted sum boundary line can be used to determine the weights assigned to samples within the current block. More specifically, the direction parameter can have a value between 0 and 3, and the angle parameter can have a value between 0 and 4. The slope angle of the weighted sum boundary line can be derived based on the combination of the values ​​of the direction parameter and the angle parameter.

[0149] For example, when the value of the direction parameter is 0 and the value of the angle parameter is 0, 1, 2, 3, or 4, the slope angle of the weighted sum boundary line may be 0 degrees, 14 degrees, 27 degrees, 45 degrees, or 63 degrees, respectively. As another example, when the value of the direction parameter is 1 and the value of the angle parameter is 0, 1, 2, 3, or 4, the slope angle of the weighted sum boundary line may be 90 degrees, 117 degrees, 135 degrees, 153 degrees, or 166 degrees, respectively. Here, when the slope angle is 0 degrees and when it is 180 degrees, the same weighted sum boundary line may be indicated. In this case, based on the value of the direction parameter, it may be determined which region's sample among the two regions separated by the weighted sum boundary line is given a higher weight.

[0150] Additionally, based on the slope angle of the weighted sum boundary line, the distance parameter may be determined to have a value between 0 and 2 or a value between 0 and 3. For example, when the slope angle is 0 degrees, 90 degrees, or 180 degrees or greater, the distance parameter may be determined to have a value between 0 and 2, and in other cases, the distance parameter may be determined to have a value between 0 and 3. Based on the value of the distance parameter, the distance between the weighted sum boundary line and a sample at a central position within the current block may be derived.

[0151] As described above, a weighted sum mask used in weighted summing of inter and intra prediction signals of the current block can be derived based on the parameters of the weighted sum boundary included in the signaled bitstream. Furthermore, the weighted sum mask can be calculated and derived in real time based on the signaled parameters.

[0152] Referring to FIG. 3, a prediction block of the current block can be generated (S330).

[0153] After the inter prediction block (signal) and the intra prediction block (signal) for the current block are each generated, the inter prediction block and the intra prediction block can be weighted based on the weighted sum mask derived by the above-described method, and the prediction block of the current block can be generated through the weighted sum.

[0154] Referring to FIG. 3, the current block can be restored based on the predicted block of the current block (S340).

[0155] The current block can be reconstructed based on the generated prediction block and the residual block of the current block. Here, the residual block can be generated by applying at least one of inverse quantization or inverse transformation to the transform coefficients obtained from the bitstream. The inverse transformation can be performed based on a predetermined transform kernel (or a set of transform kernels).

[0156] The above transformation kernel can be derived based on whether weighted sum is performed between inter and intra prediction signals of the current block.

[0157] For the current block, when an inter-prediction block and an intra-prediction block generated based on warp prediction are weighted and combined to generate a prediction block, a transform kernel for inverse transform of the transform coefficients of the current block can be derived according to an embodiment of the present disclosure. Here, the transform kernel can include at least one of a primary transform kernel or a secondary transform kernel.

[0158] According to one embodiment of the present disclosure, the transform kernel or set of transform kernels may be derived based on at least one of the width or height of the current block, or at least one of the intra prediction modes. In this case, one of the derived transform kernel sets may be selected based on a signaled flag or index, and determined as the transform kernel of the current block.

[0159] According to another embodiment of the present disclosure, the transform kernel or set of transform kernels may be derived based on at least one of the width or height of the current block, or at least one of the gradient values ​​of the current block. More specifically, an intra prediction mode may be derived based on the gradient value of the current block, and the transform kernel or set of transform kernels of the current block may be determined based on the derived intra prediction mode.

[0160] The gradient value of the current block can be derived by applying a predetermined filter to a surrounding reference area of ​​the current block. The gradient value can include at least one of a horizontal gradient value (Gh) or a vertical gradient value (Gv), and each gradient value can be calculated based on a filter applied in the horizontal direction of the reference area (hereinafter, referred to as a horizontal filter) or a filter applied in the vertical direction (hereinafter, referred to as a vertical filter). Table 1 below is an example of a combination of the horizontal filter and the vertical filter.

[0161]

[0162] Based on the horizontal gradient value (Gh) and vertical gradient value (Gv) calculated by the combination of the horizontal filter and the vertical filter, the angle (θ) and size information (I) can be calculated as in the following mathematical expression 7.

[0163]

[0164] The angle (θ) derived through mathematical expression 7 can be mapped to one of a plurality of directional intra prediction modes. By assigning the size information (I) calculated through mathematical expression 7 to the directional intra prediction mode corresponding to the angle (θ) as the amplitude value of the corresponding mode, a histogram reflecting the directionality of the intra prediction mode can be constructed. At this time, among the intra prediction modes in the histogram, the directional intra prediction mode with the largest amplitude value can be determined as the intra prediction mode for deriving the transformation kernel or a set of transformation kernels of the current block.

[0165] Alternatively, if the amplitude value of the directional intra prediction mode with the largest amplitude value in the histogram is less than or equal to a predetermined threshold, the intra prediction mode for deriving the transformation kernel or set of transformation kernels of the current block may be determined as the DC mode or the smooth mode.

[0166] Alternatively, if the amplitude value of the directional intra prediction mode with the largest amplitude value in the histogram is smaller than a predetermined threshold, the intra prediction mode with a smaller template cost calculated based on the template area of ​​the current block among the DC mode or the smooth mode may be determined as the intra prediction mode for deriving the transformation kernel or the set of transformation kernels.

[0167] After an intra prediction mode is derived based on the above histogram, a transformation kernel or a set of transformation kernels of the current block can be determined based on the derived intra prediction mode.

[0168] The encoding method and device according to the present disclosure can derive inter-prediction information for a current block and determine whether to perform weighted summation between inter- and intra-prediction signals for the current block. Furthermore, based on the above-described performance, information for generating a prediction block for the current block can be derived, and a prediction block for the current block can be generated based on the above-described information. The method for generating the prediction block is as described in FIG. 3.

[0169] The encoding method and device according to the present disclosure can derive a residual block based on the prediction block. In addition, at least one of transformation and quantization can be performed on the residual block, thereby deriving a transform coefficient of the current block. In addition, the transform coefficient can be encoded. The method for determining a transform kernel (or a set of transform kernels) for transforming the residual block is as mentioned in FIG. 3.

[0170] The various embodiments of the present disclosure are not intended to list all possible combinations but rather to illustrate representative aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combinations of two or more.

[0171] Additionally, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. In the case of hardware implementation, the embodiments may be implemented by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), general processors, controllers, microcontrollers, microprocessors, etc.

[0172] The scope of the present disclosure includes software or machine-executable instructions (e.g., operating systems, applications, firmware, programs, etc.) that cause operations according to the methods of various embodiments to be executed on a device or a computer, and a non-transitory computer-readable medium having such software or instructions stored thereon and executable on the device or computer.

[0173] The present disclosure can be industrially utilized in the fields of methods, devices, and recording media for video encoding / decoding.

Claims

1. A step of deriving inter prediction information of the current block; A step of determining whether to perform weighted sum prediction of an inter prediction block and an intra prediction block for the current block; A step of generating a prediction block of the current block based on whether the weighted sum prediction is performed; and A step of restoring the current block based on the prediction block and the residual block of the current block, The inter prediction information includes at least one of reference frame information of the current block, a motion vector coding mode, or a motion mode, A method for decoding an image, wherein the above reference frame information includes prediction direction information of the current block.

2. In paragraph 1, If the motion vector coding mode of the current block is Warped motion vector prediction (WARPMV), the translational parameters of the current block are derived based on the warp parameter list, A method for decoding an image, wherein the above warp parameter list is generated based on a reference block restored through warp prediction around the current block.

3. In paragraph 1, Whether to perform the above weighted sum prediction is determined based on a first flag indicating whether to perform the weighted sum, A video decoding method, wherein whether the first flag is signaled is determined based on at least one of the width or height of the current block, at least one of the reference frame information or the inter prediction information.

4. In paragraph 3, Based on the fact that bidirectional prediction is performed for the current block according to the prediction direction information of the current block, the signaling of the first flag is omitted, A method for decoding an image, wherein the value of the first flag is set to 0.

5. In paragraph 3, A video decoding method, wherein the first flag is signaled based on the frame of the current block not being a TIP (Temporal Interpolated Prediction) frame according to the above reference frame information.

6. In paragraph 3, A video decoding method, wherein the first flag is signaled based on the motion vector coding mode of the current block being set to WARPMV and the motion mode of the current block being set to either the WARP_CASUAL mode or the WARP_DELTA mode.

7. In paragraph 6, The above WARP_CASUAL mode is a mode that derives the non-translational parameters of the current block based on the motion vector of a block restored based on the same reference frame as the current block among the previously restored reference blocks around the current block, according to the least squares regression method. The above WARP_DELTA mode is a video decoding method in which the non-translational parameters of the current block are derived based on a list of warp parameters generated based on a reference block restored through warp prediction around the current block.

8. In paragraph 3, If the motion vector coding mode of the current block is set to WARPMV and the motion mode of the current block is set to either the WARP_CASUAL mode or the WARP_DELTA mode, If the width and height of the current block are equal to the sizes pre-defined for the encoder and decoder, the first flag is signaled, A method for decoding an image, wherein if the width and height of the current block are not equal to the pre-defined size, signaling of the first flag is omitted and the value of the first flag is set to 0.

9. In paragraph 1, If the motion vector coding mode of the current block is WARPMV and weighted sum prediction of the inter prediction block and intra prediction block is performed, The above intra prediction block is generated based on the intra prediction mode, A method for decoding an image, wherein the intra prediction mode is any one of DC mode, vertical mode, horizontal mode, or smooth mode.

10. In paragraph 1, When weighted sum prediction of the above inter prediction block and intra prediction block is performed, The above weighted sum prediction is performed based on the weighted sum mask, An image decoding method wherein the above weighted sum mask is derived based on a weighted sum mask flag.

11. In paragraph 10, A method for decoding an image, wherein the weighted sum mask is derived based on the intra prediction mode of the current block, based on the value of the weighted sum mask flag being the first value.

12. In paragraph 10, If the intra prediction mode of the current block above is DC mode, A method for decoding an image, wherein a weight mask is derived in which equal weights are given to inter-prediction blocks and intra-prediction blocks for all samples within the current block.

13. In paragraph 10, Based on the value of the above weighted sum mask flag being the second value, the weighted sum mask is determined as one of N weighted sum masks that are identically defined in the encoder and decoder based on the weighted sum mask information, The above weighted sum mask information includes a parameter for at least one of the direction, angle, or distance of the weighted sum boundary line, A method for decoding an image, wherein N is an integer greater than or equal to 1.

14. In paragraph 1, The above residual block is derived by applying a predetermined transformation kernel to the transformation coefficients obtained from the bitstream, An image decoding method, wherein the above-described transformation kernel is determined based on a HoG (Histogram of Gradient) generated based on reference samples around the current block.

15. A step of deriving inter prediction information of the current block; A step of determining whether to perform weighted sum prediction of an inter prediction block and an intra prediction block for the current block; A step of generating a prediction block of the current block based on whether the weighted sum prediction is performed; and A step of determining a transformation kernel of the current block based on a prediction block of the current block, The inter prediction information includes at least one of reference frame information of the current block, a motion vector coding mode, or a motion mode, A video encoding method, wherein the above reference frame information includes prediction direction information of the current block.

16. A computer-readable storage medium for storing a bitstream generated by the image encoding method according to Article 15.

17. A method for transmitting a bitstream generated by a video encoding method according to Article 15.

Images