Video coding method and apparatus, and recording medium having bitstream stored therein

Abstract

An image encoding / decoding method and apparatus according to the present disclosure may: configure a candidate list for the current block; derive motion information regarding the current block on the basis of the candidate list; and perform inter prediction on the current block on the basis of the motion information. Here, the candidate list may include a plurality of candidates, and the plurality of candidates may include at least one of a spatial candidate and a temporal candidate.

Description

Video coding method and device, and recording medium storing bitstream

[0001] The present invention relates to a video signal processing method and device.

[0002] The market demand for high-resolution video is growing, necessitating technologies capable of efficiently compressing high-resolution images. To address this market need, the ISO / IEC's Moving Picture Expert Group (MPEG) and the ITU-T's Video Coding Expert Group (VCEG) jointly formed the Joint Collaborative Team on Video Coding (JCT-VC). They completed development of the HEVC (High Efficiency Video Coding) video compression standard in January 2013 and have been actively conducting research and development on next-generation compression standards.

[0003] Video compression largely consists of intraprediction, interprediction, transform, quantization, entropy coding, and in-loop filtering. Among these, intraprediction refers to a technique that generates a prediction block for the current block using reconstructed pixels surrounding the current block. The encoder encodes the intraprediction mode used for intraprediction, and the decoder performs intraprediction by reconstructing the encoded intraprediction mode.

[0004] The present disclosure provides a method and device for constructing a candidate list for inter prediction.

[0005] The present disclosure provides a method and device for deriving a spatial candidate.

[0006] The present disclosure provides a method and device for deriving a temporal candidate.

[0007] The present disclosure provides a method and device for deriving pairwise candidates.

[0008] The video decoding method and device according to the present disclosure can construct a candidate list for a current block, derive motion information of the current block based on the candidate list, and perform inter prediction on the current block based on the motion information. Here, the candidate list can include a plurality of candidates, and the plurality of candidates can include at least one of a spatial candidate and a temporal candidate.

[0009] In the image decoding method and device according to the present disclosure, the spatial candidate may be derived based on a candidate block spatially adjacent to the current block. The candidate block may include at least one of a neighboring block adjacent to the current block or a block spatially adjacent to the neighboring block.

[0010] In the video decoding method and device according to the present disclosure, the video decoding method may include a step of constructing a spatial candidate list for the current block and a step of calculating costs for candidates included in the spatial candidate list. Here, the candidates included in the spatial candidate list may be derived based on motion information of candidate blocks spatially neighboring the current block.

[0011] In the image decoding method and device according to the present disclosure, the top K candidates in ascending order of the costs can be added as the spatial candidates of the candidate list.

[0012] In the video decoding method and device according to the present disclosure, the motion vector of the temporal candidate can be derived based on the motion vector of the call block corresponding to the current block and a predetermined scaling factor. The scaling factor can be derived based on the POC difference between the current picture to which the current block belongs and the reference picture.

[0013] In the image decoding method and device according to the present disclosure, the reference picture can be determined based on the reference index of the call block.

[0014] In the image decoding method and device according to the present disclosure, the reference picture can be determined based on the POC difference between the call picture to which the call block belongs and the reference picture of the call block.

[0015] In the video decoding method and device according to the present disclosure, the step of constructing the candidate list may include the step of calculating costs for candidates already added to the candidate list, the step of deriving one or more pairwise candidates through at least two combinations among the top N candidates in ascending order of the calculated costs, and the step of adding the derived one or more pairwise candidates to the candidate list.

[0016] The video encoding method and device according to the present disclosure can construct a candidate list for a current block, derive motion information of the current block based on the candidate list, and perform inter prediction on the current block based on the motion information. The candidate list may include a plurality of candidates, and the plurality of candidates may include at least one of a spatial candidate and a temporal candidate.

[0017] A bitstream transmission method and device according to the present disclosure may configure a candidate list for a current block, derive motion information of the current block based on the candidate list, perform inter prediction on the current block based on the motion information to generate a bitstream, and transmit the generated bitstream. The candidate list may include a plurality of candidates, and the plurality of candidates may include at least one of a spatial candidate and a temporal candidate.

[0018] A computer-readable recording medium according to the present disclosure can store a bitstream encoded by the image encoding method.

[0019] According to the present disclosure, by expanding the range of candidate blocks for inter prediction, more accurate prediction is possible, thereby improving the compression performance of the encoder / decoder.

[0020] According to the present disclosure, by variably deriving the reference index of a temporal candidate, more accurate prediction is possible, thereby improving the compression performance of the encoder / decoder.

[0021] According to the present disclosure, efficient prediction is possible by deriving pairwise candidates through various combinations, thereby improving the compression performance of the encoder / decoder.

[0022] FIG. 1 is a block diagram showing an image encoding device according to the present disclosure.

[0023] FIG. 2 is a block diagram showing an image decoding device according to the present disclosure.

[0024] Figure 3 illustrates an inter prediction method according to the present disclosure.

[0025] The video decoding method and device according to the present disclosure can construct a candidate list for a current block, derive motion information of the current block based on the candidate list, and perform inter prediction on the current block based on the motion information. Here, the candidate list can include a plurality of candidates, and the plurality of candidates can include at least one of a spatial candidate and a temporal candidate.

[0026] In the image decoding method and device according to the present disclosure, the spatial candidate may be derived based on a candidate block spatially adjacent to the current block. The candidate block may include at least one of a neighboring block adjacent to the current block or a block spatially adjacent to the neighboring block.

[0027] In the video decoding method and device according to the present disclosure, the video decoding method may include a step of constructing a spatial candidate list for the current block and a step of calculating costs for candidates included in the spatial candidate list. Here, the candidates included in the spatial candidate list may be derived based on motion information of candidate blocks spatially neighboring the current block.

[0028] In the image decoding method and device according to the present disclosure, the top K candidates in ascending order of the costs can be added as the spatial candidates of the candidate list.

[0029] In the video decoding method and device according to the present disclosure, the motion vector of the temporal candidate can be derived based on the motion vector of the call block corresponding to the current block and a predetermined scaling factor. The scaling factor can be derived based on the POC difference between the current picture to which the current block belongs and the reference picture.

[0030] In the image decoding method and device according to the present disclosure, the reference picture can be determined based on the reference index of the call block.

[0031] In the image decoding method and device according to the present disclosure, the reference picture can be determined based on the POC difference between the call picture to which the call block belongs and the reference picture of the call block.

[0032] In the video decoding method and device according to the present disclosure, the step of constructing the candidate list may include the step of calculating costs for candidates already added to the candidate list, the step of deriving one or more pairwise candidates through at least two combinations among the top N candidates in ascending order of the calculated costs, and the step of adding the derived one or more pairwise candidates to the candidate list.

[0033] The video encoding method and device according to the present disclosure can construct a candidate list for a current block, derive motion information of the current block based on the candidate list, and perform inter prediction on the current block based on the motion information. The candidate list may include a plurality of candidates, and the plurality of candidates may include at least one of a spatial candidate and a temporal candidate.

[0034] A bitstream transmission method and device according to the present disclosure may configure a candidate list for a current block, derive motion information of the current block based on the candidate list, perform inter prediction on the current block based on the motion information to generate a bitstream, and transmit the generated bitstream. The candidate list may include a plurality of candidates, and the plurality of candidates may include at least one of a spatial candidate and a temporal candidate.

[0035] A computer-readable recording medium according to the present disclosure can store a bitstream encoded by the image encoding method.

[0036] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings attached to this specification so that those skilled in the art can easily implement the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In addition, in the drawings, parts irrelevant to the description have been omitted to clearly explain the present invention, and similar parts have been designated with similar reference numerals throughout the specification.

[0037] Throughout this specification, when a part is said to be 'connected' to another part, this includes not only cases where they are directly connected, but also cases where they are electrically connected with another element in between.

[0038] Additionally, whenever a part throughout this specification is said to "include" a component, this does not mean that other components are excluded, but rather that other components may be included, unless specifically stated otherwise.

[0039] Additionally, while terms such as first, second, etc. may be used to describe various components, these components should not be limited by these terms. These terms are used solely to distinguish one component from another.

[0040] Additionally, in the embodiments of the devices and methods described herein, some components of the devices or some steps of the methods may be omitted. Furthermore, the order of some components of the devices or some steps of the methods may be changed. Furthermore, other components or other steps may be inserted into some components of the devices or some steps of the methods.

[0041] Additionally, some components or some steps of the first embodiment of the present invention may be added to the second embodiment of the present invention, or some components or some steps of the second embodiment may be replaced.

[0042] In addition, the components shown in the embodiments of the present invention are independently depicted to represent different characteristic functions, and this does not mean that each component is composed of separate hardware or a single software component. That is, each component is described by listing each component for convenience of explanation, and at least two components among each component may be combined to form a single component, or a single component may be divided into multiple components to perform a function. Such integrated and separate embodiments of each component are also included in the scope of the present invention as long as they do not deviate from the essence of the present invention.

[0043] In this specification, a block can be variously expressed as a unit, an area, a unit, a partition, etc., and a sample can be variously expressed as a pixel, a pel, a pixel, etc.

[0044] Hereinafter, embodiments of the present invention will be described in more detail with reference to the attached drawings. In describing the present invention, duplicate descriptions of identical components will be omitted.

[0045] FIG. 1 is a block diagram showing an image encoding device according to the present disclosure.

[0046] Referring to FIG. 1, a video encoding device (100) may include a picture segmentation unit (110), a prediction unit (120, 125), a transformation unit (130), a quantization unit (135), a reordering unit (160), an entropy encoding unit (165), an inverse quantization unit (140), an inverse transformation unit (145), a filter unit (150), and a memory (155).

[0047] The picture segmentation unit (110) can segment the input picture into at least one processing unit. At this time, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). Hereinafter, in the embodiments of the present disclosure, the coding unit may be used to mean a unit that performs encoding or a unit that performs decoding.

[0048] A prediction unit may be divided into at least one square or rectangular shape of the same size within a single coding unit, or may be divided such that one prediction unit among the divided prediction units within a single coding unit has a different shape and / or size from another prediction unit. When a prediction unit that performs intra prediction based on a coding unit is generated and is not the minimum coding unit, intra prediction can be performed without being divided into a plurality of NxN prediction units.

[0049] The prediction unit (120, 125) may include an inter prediction unit (120) that performs inter prediction or inter prediction, and an intra prediction unit (125) that performs intra prediction or intra prediction. It may determine whether to use inter prediction or intra prediction for a prediction unit, and determine specific information (e.g., intra prediction mode, motion vector, reference picture, etc.) according to each prediction method. A residual value (residual block) between the generated prediction block and the original block may be input to the transformation unit (130). In addition, prediction mode information, motion vector information, etc. used for prediction may be encoded together with the residual value by the entropy encoding unit (165) and transmitted to the decoder.

[0050] The inter prediction unit (120) may predict a prediction unit based on information of at least one picture among the previous or subsequent pictures of the current picture, and in some cases, may predict a prediction unit based on information of a portion of an encoded region within the current picture. The inter prediction unit (120) may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

[0051] The reference picture interpolation unit can receive reference picture information from the memory (155) and generate pixel information less than an integer pixel from the reference picture. In the case of luminance pixels, a DCT-based 8-tap interpolation filter with different filter coefficients can be used to generate pixel information less than an integer pixel in units of 1 / 4 pixels. In the case of a chrominance signal, a DCT-based 4-tap interpolation filter with different filter coefficients can be used to generate pixel information less than an integer pixel in units of 1 / 8 pixels.

[0052] The motion prediction unit can perform motion prediction based on a reference picture interpolated by the reference picture interpolation unit. Various methods such as FBMA (Full search-based Block Matching Algorithm), TSS (Three Step Search), and NTS (New Three-Step Search Algorithm) can be used to derive a motion vector. The motion vector can have a motion vector value in units of 1 / 2 or 1 / 4 pixels based on the interpolated pixel. The motion prediction unit can predict the current prediction unit by using different motion prediction methods. Various methods such as Skip Mode, Merge Mode, AMVP Mode, Intra Block Copy Mode, and Affine Mode can be used as motion prediction methods.

[0053] The intra prediction unit (125) can generate a prediction unit based on reference pixel information surrounding the current block, which is pixel information within the current picture. If the surrounding block of the current prediction unit is a block on which inter prediction has been performed and the reference pixel is a pixel on which inter prediction has been performed, the reference pixel included in the block on which inter prediction has been performed can be replaced and used with reference pixel information of the surrounding block on which intra prediction has been performed. That is, if the reference pixel is not available, the unavailable reference pixel information can be replaced and used with at least one reference pixel among the available reference pixels.

[0054] Additionally, a residual block containing residual value information, which is the difference between the prediction unit that performed the prediction based on the prediction unit generated in the prediction unit (120, 125) and the original block of the prediction unit, can be generated. The generated residual block can be input to the transformation unit (130).

[0055] In the transformation unit (130), the residual block including the residual value information of the prediction unit generated through the original block and the prediction unit (120, 125) can be transformed using a transformation method such as DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), or KLT. Whether to apply DCT, DST, or KLT to transform the residual block can be determined based on the intra prediction mode information of the prediction unit used to generate the residual block.

[0056] The quantization unit (135) can quantize values ​​converted to the frequency domain by the transformation unit (130). The quantization coefficients can vary depending on the block or the importance of the image. The values ​​produced by the quantization unit (135) can be provided to the dequantization unit (140) and the reordering unit (160).

[0057] The rearrangement unit (160) can perform rearrangement of coefficient values ​​for quantized residual values.

[0058] The rearrangement unit (160) can change a two-dimensional block-shaped coefficient into a one-dimensional vector form through a coefficient scanning method. For example, the rearrangement unit (160) can change the two-dimensional block-shaped coefficient into a one-dimensional vector form by scanning from the DC coefficient to the coefficient of the high-frequency region using a zig-zag scan method. Depending on the size of the transformation unit and the intra prediction mode, a vertical scan that scans the two-dimensional block-shaped coefficient in the column direction or a horizontal scan that scans the two-dimensional block-shaped coefficient in the row direction may be used instead of the zig-zag scan. That is, depending on the size of the transformation unit and the intra prediction mode, it is possible to determine which scan method among the zig-zag scan, the vertical scan, and the horizontal scan is to be used.

[0059] The entropy encoding unit (165) can perform entropy encoding based on the values ​​produced by the rearrangement unit (160). Entropy encoding can use various encoding methods such as, for example, Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC). In this regard, the entropy encoding unit (165) can encode residual value coefficient information of the encoding unit from the rearrangement unit (160) and the prediction units (120, 125). In addition, according to the present disclosure, it is possible to signal and transmit information indicating that motion information is derived and used on the decoder side and information on a technique used to derive motion information.

[0060] The inverse quantization unit (140) and the inverse transformation unit (145) inversely quantize the values ​​quantized in the quantization unit (135) and inversely transform the values ​​transformed in the transformation unit (130). The residual values ​​generated in the inverse quantization unit (140) and the inverse transformation unit (145) can be combined with the predicted prediction units predicted through the motion estimation unit, motion compensation unit, and intra prediction unit included in the prediction unit (120, 125) to generate a reconstructed block.

[0061] The filter unit (150) may include at least one of a deblocking filter, an offset correction unit, and an ALF (Adaptive Loop Filter). The deblocking filter may remove block distortion caused by boundaries between blocks in a restored picture. The offset correction unit may correct the offset from the original image on a pixel-by-pixel basis for the image on which deblocking has been performed. In order to perform offset correction for a specific picture, a method may be used in which the pixels included in the image are divided into a certain number of regions, the regions to be offset are determined, and the offset is applied to the regions, or the offset is applied by considering edge information of each pixel. The ALF (Adaptive Loop Filtering) may be performed based on a value obtained by comparing the filtered restored image with the original image. After dividing the pixels included in the image into a predetermined group, one filter to be applied to the group is determined, and filtering may be performed differentially for each group.

[0062] The memory (155) can store a restored block or picture produced through the filter unit (150), and the stored restored block or picture can be provided to the prediction unit (120, 125) when performing inter prediction.

[0063] FIG. 2 is a block diagram showing an image decoding device according to the present disclosure.

[0064] Referring to FIG. 2, the image decoding device (200) may include an entropy decoding unit (210), a rearrangement unit (215), an inverse quantization unit (220), an inverse transformation unit (225), a prediction unit (230, 235), a filter unit (240), and a memory (245).

[0065] When a video bitstream is input to a video encoding device, the input bitstream can be decoded in the opposite procedure to that of the video encoding device.

[0066] The entropy decoding unit (210) can perform entropy decoding in a procedure opposite to that of the entropy encoding unit of the video encoder. For example, various methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) can be applied in response to the method performed in the video encoder.

[0067] The entropy decoding unit (210) can decode information related to intra prediction and inter prediction performed in the encoder.

[0068] The reordering unit (215) can perform reordering based on the method by which the bitstream entropy-decoded by the entropy decoding unit (210) is reordered by the encoding unit. The coefficients expressed in the form of a one-dimensional vector can be reordered by restoring them back to coefficients in the form of a two-dimensional block.

[0069] The inverse quantization unit (220) can perform inverse quantization based on the quantization parameters provided by the encoder and the coefficient values ​​of the rearranged block.

[0070] The inverse transform unit (225) can perform inverse transform, i.e., inverse DCT, inverse DST, and inverse KLT, on the transforms performed by the transform unit, i.e., DCT, DST, and KLT, on the quantization result performed by the image encoder. The inverse transform can be performed based on the transmission unit determined by the image encoder. In the inverse transform unit (225) of the image decoder, a transform technique (e.g., DCT, DST, KLT) can be selectively performed according to a plurality of pieces of information, such as a prediction method, the size of the current block, and the prediction direction.

[0071] The prediction unit (230, 235) can generate a prediction block based on prediction block generation related information provided from the entropy decoding unit (210) and previously decoded block or picture information provided from the memory (245).

[0072] As described above, when performing intra prediction or intra prediction in the same manner as the operation in the image encoder, if the size of the prediction unit and the size of the transformation unit are the same, intra prediction for the prediction unit is performed based on the pixels on the left side of the prediction unit, the pixels on the upper left side, and the pixels on the upper side. However, when performing intra prediction, if the size of the prediction unit and the size of the transformation unit are different, intra prediction can be performed using reference pixels based on the transformation unit. In addition, intra prediction using NxN division only for the minimum coding unit can be used.

[0073] The prediction unit (230, 235) may include a prediction unit determination unit, an inter prediction unit, and an intra prediction unit. The prediction unit determination unit may receive various information such as prediction unit information input from the entropy decoding unit (210), prediction mode information of an intra prediction method, and motion prediction-related information of an inter prediction method, and may distinguish a prediction unit from a current encoding unit and determine whether the prediction unit performs inter prediction or intra prediction. On the other hand, if the encoder (100) does not transmit motion prediction-related information for the inter prediction, but instead transmits information indicating that motion information is to be derived and used on the decoder side and information on a technique used to derive motion information, the prediction unit determination unit determines whether the inter prediction unit (230) performs prediction based on the information transmitted from the encoder (100).

[0074] The inter prediction unit (230) can perform inter prediction on the current prediction unit based on information included in at least one picture among the previous picture or the subsequent picture of the current picture including the current prediction unit, using information required for inter prediction of the current prediction unit provided by the image encoder. In order to perform inter prediction, it can be determined based on the encoding unit whether the motion prediction method of the prediction unit included in the corresponding encoding unit is one of Skip Mode, Merge Mode, AMVP Mode, Intra Block Copy Mode, and Affine Mode.

[0075] The intra prediction unit (235) can generate a prediction block based on pixel information within the current picture. If the prediction unit is a prediction unit that has performed intra prediction, intra prediction can be performed based on intra prediction mode information of the prediction unit provided by the image encoder.

[0076] The intra prediction unit (235) may include an Adaptive Intra Smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter is a unit that performs filtering on the reference pixels of the current block and can determine whether to apply the filter based on the prediction mode of the current prediction unit and apply it. AIS filtering can be performed on the reference pixels of the current block using the prediction mode and AIS filter information of the prediction unit provided by the image encoder. If the prediction mode of the current block is a mode that does not perform AIS filtering, the AIS filter may not be applied.

[0077] The reference pixel interpolation unit can interpolate the reference pixel to generate a reference pixel of a pixel unit less than an integer value when the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on the pixel value interpolated from the reference pixel. When the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixel, the reference pixel may not be interpolated. The DC filter can generate a prediction block through filtering when the prediction mode of the current block is the DC mode.

[0078] The restored block or picture may be provided to a filter unit (240). The filter unit (240) may include a deblocking filter, an offset correction unit, and an ALF.

[0079] Information about whether a deblocking filter has been applied to a corresponding block or picture can be received from a video encoding device, and if a deblocking filter has been applied, information about whether a strong or weak filter has been applied. The deblocking filter of the video decoder can receive information related to the deblocking filter provided by the video encoder, and the video decoder can perform deblocking filtering on the corresponding block.

[0080] The offset correction unit can perform offset correction on the restored image based on the type of offset correction applied to the image during encoding and information on the offset value. ALF can be applied to the encoding unit based on information on whether ALF is applied and ALF coefficient information provided from the encoder. This ALF information can be provided by being included in a specific parameter set.

[0081] The memory (245) can store a restored picture or block so that it can be used as a reference picture or reference block, and can also provide the restored picture to an output unit.

[0082] Figure 3 illustrates an inter prediction method according to the present disclosure.

[0083] Referring to FIG. 3, a candidate list for the current block can be constructed (S300).

[0084] The candidate list for the current block may include multiple candidates. Here, a candidate may refer to a motion vector or a candidate with motion information. The motion information may include at least one of a motion vector, a reference index for specifying a reference picture, or prediction direction information indicating either the L0 or L1 direction.

[0085] The multiple candidates may include at least one of a spatial candidate, a temporal candidate, a history-based candidate, a pair-wise candidate, or a candidate with a zero motion vector. The spatial candidate may include at least one of an adjacent candidate derived from a neighboring block that is spatially adjacent to the current block, or a non-adjacent candidate derived from a neighboring block that is not spatially adjacent to the current block. In the embodiments described below, the spatial candidate may be understood as being replaced with an adjacent candidate.

[0086] A spatial candidate according to the present disclosure can be derived based on motion information of candidate blocks spatially neighboring the current block. A candidate block can be defined as a neighboring block available for deriving a spatial candidate of the current block. The candidate block can include neighboring blocks adjacent to the current block. Here, the neighboring blocks can include at least one of an upper neighboring block (B1), a left neighboring block (A1), an upper-right neighboring block (B0), a lower-left neighboring block (A0), or an upper-left neighboring block (B2). A spatial candidate can be derived based on motion information of at most N candidate block(s) among the aforementioned candidate blocks. Here, N can be an integer less than or equal to 4. However, the present invention is not limited thereto, and N can also be 5.

[0087] The above candidate blocks can be sequentially searched based on a pre-defined search order. For example, the pre-defined search order may be B1, A1, B0, A0, B2.

[0088] Depending on the search order, the availability of motion information for a given candidate block can be determined. If the motion information for a given candidate block is determined to be available, a spatial candidate can be derived based on the motion information for that candidate block. At this time, the availability of motion information for a candidate block can be determined by checking for redundancy with the motion information for a candidate block with a previous search order.

[0089] For example, A1 can check whether it has the same motion information as B1. B0 can check whether it has the same motion information as B1. A0 can check whether it has the same motion information as A1. B2 can check whether it has the same motion information as A1.

[0090] A candidate block according to the present disclosure may further include a block spatially adjacent to at least one of the aforementioned surrounding blocks. For example, the candidate block may include at least one of the upper block (B4) of B1, the upper block (B5) of B0, the upper right block (B6) of B0, the right block (B3) of B0, the left block (A3) of A1, the left block (A4) of A0, the lower left block (A5) of A0, or the lower block (A2) of A0.

[0091] A maximum of 13 candidate blocks can be used by increasing the number of candidate blocks by up to 8. Spatial candidates can be derived based on the motion information of up to M candidate blocks among the aforementioned candidate blocks. Here, M can be an integer greater than or equal to 1. M can be less than or equal to 13.

[0092] The above-described increased candidate blocks can also be sequentially searched based on a pre-defined search order. For example, assuming a total of 8 candidate blocks are increased, the increased candidate blocks can be searched in the search order A2, B3, A3, B4, A4, B5, A5, B6. Alternatively, the increased candidate blocks can be searched in the search order B3, A2, B4, A3, B5, A4, B6, A5. However, this is merely an example, and other combinable search orders may also be used.

[0093] In addition to the candidate list for the current block, a spatial candidate list may also be constructed for the current block. The spatial candidate list may include candidates derived based on the motion information of the aforementioned (augmented) candidate blocks.

[0094] A predetermined cost can be calculated for each candidate in the spatial candidate list. Here, the cost can be calculated based on the difference (e.g., SAD, SATD) between the template region of the current block and the template region of the reference block. The template region of the current block may be a neighboring region adjacent to the current block. The template region of the reference block may be a neighboring region adjacent to the reference block, and may be an region of the same size corresponding to the template region of the current block. The template region may be composed of one or more sample lines. The reference block may be specified based on the motion information of the candidate. That is, within the reference picture specified by the reference index of the candidate, the block specified by the motion vector of the candidate may be the reference block.

[0095] The candidates in the spatial candidate list can be reordered in ascending order of the above cost. Within the reordered spatial candidate list, the top K candidate(s) can be added as spatial candidate(s) to the candidate list of the current block. Here, K can be an integer greater than or equal to 1. The top K candidates can be candidates with candidate indices between 0 and (K-1) within the spatial candidate list.

[0096] A temporal candidate according to the present disclosure can be derived based on motion information (or motion vector) of a co-located block within a co-located picture. Here, the co-located block may be a block including a lower right sample position adjacent to the current block within the co-located picture (hereinafter referred to as a first co-located block) or a block including a central sample position of the current block (hereinafter referred to as a second co-located block).

[0097] For example, it may first be determined whether motion information of the first call block is available. If the motion information of the first call block is determined to be available, a temporal candidate may be derived based on the motion information of the first call block. Conversely, if the motion information of the first call block is determined to be unavailable, a temporal candidate may be derived based on the motion information of the second call block. If the first call block is encoded in intra mode, if the first call block is outside the boundary of a coding tree unit (CTU), or if the first call block is outside the boundary of a picture, the motion information of the first call block may be determined to be unavailable.

[0098] The number of call blocks available for deriving temporal candidates may be one or more.

[0099] The motion vector of a temporal candidate can be derived by applying a predetermined scaling factor to the motion vector of the called block. The scaling factor can be derived based on the difference in picture order count (POC) between the current picture and the reference picture, and the difference in POC between the called picture and the reference picture of the called block. For example, the scaling factor can be derived as shown in the following mathematical expression 1.

[0100]

[0101] In mathematical expression 1, POC curr_PU represents the POC of the current picture, and POC curr_ref can represent the POC of the reference picture referenced by the current picture. POC col_PU represents the POC of the call picture, and POC col_ref can represent the POC of the reference picture referenced by the call block.

[0102] POC curr_PU , POC curr_ref , POC col_PU , POC col_ref When are 5, 4, 6, and 0 respectively, ScaleFactor can be derived as 1 / 6 according to Equation 1. At this time, assuming that the motion vector of the call block is (6, 6), the motion vector of the temporal candidate can be derived as (1, 1).

[0103] The reference picture referenced by the current picture can be specified based on the reference index of the temporal candidate. The reference index among the motion information of the temporal candidate can be fixedly set to 0. In other words, the reference picture referenced by the current picture can be a reference picture with a reference index of 0 within the reference picture list for the current block.

[0104] Alternatively, the reference index of the temporal candidate may be set based on the reference index of the called block. If the reference index of the called block is greater than 0, the reference index of the temporal candidate may be greater than 0. That is, if the reference index of the called block is greater than 0, a reference picture with a reference index greater than 0 in the reference picture list for the current block may be used as a reference picture referenced by the current picture. The reference index of the temporal candidate may be set to the same value as the reference index of the called block.

[0105] For example, if the reference index of the call block is 1, the reference index of the temporal candidate can be set to 1, and the reference picture with the reference index of 1 in the reference picture list for the current block can be used as the reference picture referenced by the current picture. In this case, POC curr_PU , POC curr_ref , POC col_PU , POC col_ref If are 5, 2, 6, and 0 respectively, ScaleFactor can be derived as 1 / 2 according to Equation 1. Assuming that the motion vector of the call block is (6, 6), the motion vector of the temporal candidate can be derived as (3, 3).

[0106] Based on the POC difference between the call picture and the reference picture of the call block, the reference picture referenced by the current picture may be determined. For example, if the POC difference between the call picture and the reference picture of the call block is N, the reference picture referenced by the current picture may be determined as the reference picture having the reference index M within the reference picture list for the current block. Here, N and M may be integers greater than or equal to 1.

[0107] A pairwise candidate according to the present disclosure can be derived based on a combination of at least two of the pre-added candidates in the candidate list.

[0108] A predetermined cost can be calculated for each of the pre-added candidates. Here, the cost can be calculated based on the difference (e.g., SAD, SATD) between the template region of the current block and the template region of the reference block. The template region of the current block may be a neighboring region adjacent to the current block. The template region of the reference block may be a neighboring region adjacent to the reference block, and may be an region of the same size corresponding to the template region of the current block. The template region may be composed of one or more sample lines. The reference block may be specified based on the motion information of the candidate. That is, within the reference picture specified by the reference index of the candidate, the block specified by the motion vector of the candidate may be the reference block.

[0109] If a candidate has bidirectional motion information in the L0 and L1 directions, the cost for the candidate may be calculated based on at least one of the template region of the reference block in the L0 direction or the template region of the reference block in the L1 direction. For example, a first difference between the template region of the current block and the template region of the reference block in the L0 direction may be calculated, and a second difference between the template region of the current block and the template region of the reference block in the L1 direction may be calculated. The cost may be calculated based on an average of the calculated first and second differences. Alternatively, the cost may be calculated based only on the template region of the reference block in either the L0 direction or the L1 direction.

[0110] If a candidate has unidirectional motion information in the L0 or L1 direction, the cost for that candidate can be calculated based on the template area of ​​the reference block in that direction.

[0111] One or more pairwise candidates can be derived through at least two combinations among the top N candidates in ascending order of the generated costs.

[0112] For example, pairwise candidates can be derived based on combinations between the candidate with the lowest cost and T candidates with the next lowest cost, where T can be an integer greater than or equal to 1.

[0113] When T is 4, four pairwise candidates can be derived based on combinations between the candidate with the lowest cost and the four candidates with the next lowest costs. That is, a pairwise candidate can be derived based on the combination of the candidate with the lowest cost and the candidate with the second lowest cost. A pairwise candidate can be derived based on the combination of the candidate with the lowest cost and the candidate with the third lowest cost. A pairwise candidate can be derived based on the combination of the candidate with the lowest cost and the candidate with the fourth lowest cost. A pairwise candidate can be derived based on the combination of the candidate with the lowest cost and the candidate with the fifth lowest cost.

[0114] A pairwise candidate can be derived based on the combination of the candidate with the second lowest cost and the candidate with the third lowest cost.

[0115] Pairwise candidates can be derived based on the combination of the candidate with the lowest cost, the candidate with the second lowest cost, and the candidate with the third lowest cost.

[0116] Pairwise candidates can be derived based on the combination of the candidate with the smallest cost, the candidate with the second smallest cost, the candidate with the third smallest cost, and the candidate with the fourth smallest cost.

[0117] Alternatively, one or more pairwise candidates may be derived through combinations among multiple candidates, specified based on candidate indices in the candidate list, regardless of cost.

[0118] For example, pairwise candidates can be derived based on a combination of all or some candidates with even candidate indices. Pairwise candidates can also be derived based on a combination of all or some candidates with odd candidate indices. Here, some candidates can be defined as candidates whose candidate indices are less than or equal to a predetermined threshold. The threshold can be a predefined value that is identical for both the encoding device and the decoding device.

[0119] Alternatively, one or more pairwise candidates may be derived through combinations between multiple candidates, which are specified based on the candidate indices of the candidate list and the calculated costs.

[0120] For example, pairwise candidates can be derived based on the combination of the candidate with the smallest cost among candidates whose candidate indices are even and the candidate with the smallest cost among candidates whose candidate indices are odd.

[0121] A pairwise candidate can be derived based on the combination of the candidate with the smallest cost among candidates whose candidate indices are even and the candidate with the second smallest cost among candidates whose candidate indices are odd.

[0122] A pairwise candidate can be derived based on the combination of the candidate with the smallest cost among candidates with odd candidate indices and the candidate with the second smallest cost among candidates with even candidate indices.

[0123] A pairwise candidate can be derived based on the combination of the candidate with the second smallest cost among candidates whose candidate indices are odd and the candidate with the second smallest cost among candidates whose candidate indices are even.

[0124] A pairwise candidate can be derived based on a combination of at least two of the top K candidates in ascending order of cost among candidates with odd candidate indices or the top K candidates in ascending order of cost among candidates with even candidate indices. Here, K can be an integer of 1, 2, 3, or more. K can be a value predefined in the encoding device and the decoding device.

[0125] The motion vector of a pairwise candidate can be derived based on the average of the motion vectors of the combined candidates. In this case, the motion vector of a pairwise candidate can be derived through an average operation in each of the L0 and L1 directions. The reference index of a pairwise candidate can be derived based on the reference index of the candidate with the lowest cost among the combined candidates.

[0126] If one of the candidates to be combined (hereinafter referred to as the first candidate) has only an L1 motion vector and the other of the candidates to be combined (hereinafter referred to as the second candidate) has bidirectional motion vectors (i.e., L0 and L1 motion vectors), the L0 motion vector of the pairwise candidate can be derived from the L0 motion vector of the second candidate, and the L1 motion vector of the pairwise candidate can be derived as the average between the L1 motion vector of the first candidate and the L1 motion vector of the second candidate. At this time, the L0 reference index of the pairwise candidate can be derived from the L0 reference index of the second candidate. The L1 reference index of the pairwise candidate can be derived from the L1 reference index of the candidate having the smallest cost among the first and second candidates.

[0127] The average of the above motion vectors can be calculated based on a shift operation. For example, the average of the motion vectors can be calculated as shown in the following mathematical expression 2.

[0128]

[0129] In mathematical expression 2, x can be the sum of the motion vectors of the candidates, and N can be an integer greater than or equal to 1. N can be a variable value depending on the number of candidates to be combined.

[0130] Alternatively, as in Equation 2, the average of the motion vectors can be calculated based on a division operation.

[0131] For example, when pairwise candidates are derived based on a combination of three or more candidates in a candidate list, the average of the motion vectors of the three or more candidates can be calculated based on a division operation.

[0132] For example, when pairwise candidates are derived based on a combination of four or more candidates in a candidate list, the average of the motion vectors of the four or more candidates can be calculated by selectively using either a division operation or a shift operation.

[0133] For example, when pairwise candidates are derived based on combinations between an odd number of candidates in a candidate list, the average of the motion vectors of the odd number of candidates can be calculated based on at least one of a shift operation or a division operation.

[0134] When pairwise candidates are derived based on combinations between an even number of candidates in a candidate list, the average of the motion vectors of the even number of candidates can be calculated based on at least one of a shift operation or a division operation.

[0135] A pairwise candidate list can be constructed based on the pairwise candidates derived through the above-described method. A predetermined cost can be calculated for each pairwise candidate. Here, the cost can be calculated based on the difference (e.g., SAD, SATD) between the template region of the current block and the template region of the reference block. The template region of the current block may be a neighboring region adjacent to the current block. The template region of the reference block may be a neighboring region adjacent to the reference block, and may be an region of the same size corresponding to the template region of the current block. The template region may be composed of one or more sample lines. The reference block may be specified based on the motion information of the pairwise candidate. That is, within the reference picture specified by the reference index of the pairwise candidate, the block specified by the motion vector of the pairwise candidate may be the reference block.

[0136] Based on the costs of the pairwise candidates, all or some of the pairwise candidates may be added to the candidate list. The candidate list, with the pairwise candidate(s) added, may be reordered in ascending order of their costs.

[0137] Meanwhile, the pairwise candidate list may further include a pairwise candidate with the reference index of the candidate with the highest cost among the combined candidates. Here, the motion vector of the pairwise candidate is derived based on the average of the motion vectors of the combined candidates, as discussed above.

[0138] For example, pairwise candidates can be derived based on the combination of the candidate with the lowest cost and the candidate with the second-lowest cost. In this case, at least one of the pairwise candidates with the reference index of the candidate with the lowest cost or the pairwise candidate with the reference index of the candidate with the second-lowest cost can be derived.

[0139] A pairwise candidate can be derived as a combination of the candidate with the second-lowest cost and the candidate with the third-lowest cost. In this case, at least one of the pairwise candidates having the reference index of the candidate with the second-lowest cost or the pairwise candidate having the reference index of the candidate with the third-lowest cost can be derived.

[0140] A pairwise candidate can be derived based on a combination of a candidate with the lowest cost and a candidate with an even candidate index. In this case, at least one of the pairwise candidates with the reference index of the candidate with the lowest cost or the reference index of the candidate with an even candidate index can be derived.

[0141] A pairwise candidate can be derived based on a combination of the candidate with the lowest cost and the candidate with an odd candidate index. In this case, at least one of the pairwise candidates with the reference index of the candidate with the lowest cost or the pairwise candidate with the reference index of the candidate with an odd candidate index can be derived.

[0142] Among candidates with even candidate indices, pairwise candidates can be derived based on the combination of the candidate with the lowest cost and the candidate with the second-lowest cost. In this case, at least one of the pairwise candidates with the reference index of the candidate with the lowest cost or the pairwise candidate with the reference index of the candidate with the second-lowest cost can be derived.

[0143] Among candidates with odd candidate indices, pairwise candidates can be derived based on the combination of the candidate with the lowest cost and the candidate with the second-lowest cost. In this case, at least one of the pairwise candidates with the reference index of the candidate with the lowest cost or the pairwise candidate with the reference index of the candidate with the second-lowest cost can be derived.

[0144] A pairwise candidate can be derived based on a combination of the candidate with the smallest cost among candidates with even indices (hereinafter referred to as the first even candidate) and the candidate with the smallest cost among candidates with odd indices (hereinafter referred to as the first odd candidate). In this case, at least one of the pairwise candidate having the reference index of the first even candidate or the pairwise candidate having the reference index of the first odd candidate can be derived.

[0145] A pairwise candidate can be derived based on a combination of the candidate with the lowest cost among the candidates with odd indices (hereinafter referred to as the first odd candidate) and the candidate with the second lowest cost among the candidates with even indices (hereinafter referred to as the second even candidate). In this case, at least one of the pairwise candidate having the reference index of the first odd candidate or the pairwise candidate having the reference index of the second even candidate can be derived.

[0146] The aforementioned candidate list construction method may be applied when the current block is encoded in merge mode or when the current block is encoded in AMVP mode. The aforementioned candidate list construction method may also be applied when the current block is encoded in geometric partitioning mode.

[0147] Referring to Fig. 3, movement information of the current block can be derived based on the candidate list (S310).

[0148] For example, if the current block is encoded in merge mode, motion information of the current block can be derived based on motion information of any one of the multiple candidates included in the candidate list. In this case, the motion information may include at least one of a motion vector or a reference index.

[0149] Alternatively, if the current block is encoded in AMVP mode, the motion vector of the current block can be derived based on any one of the candidates in the candidate list.

[0150] Referring to FIG. 3, inter prediction of the current block can be performed based on the motion information of the current block (S320).

[0151] The aforementioned inter prediction method can be performed in the decoding process of a decoding device. Furthermore, the aforementioned inter prediction method can be performed in the same manner in the encoding process of an encoding device.

[0152] The various embodiments of the present disclosure are not intended to list all possible combinations but rather to illustrate representative aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combinations of two or more.

[0153] Additionally, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. In the case of hardware implementation, the embodiments may be implemented by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), general processors, controllers, microcontrollers, microprocessors, etc.

[0154] The scope of the present disclosure includes software or machine-executable instructions (e.g., operating systems, applications, firmware, programs, etc.) that cause operations according to the methods of various embodiments to be executed on a device or a computer, and a non-transitory computer-readable medium having such software or instructions stored thereon and executable on the device or computer.

Claims

1. Step of constructing a candidate list for the current block; A step of deriving movement information of the current block based on the candidate list; and A step of performing inter prediction on the current block based on the above motion information, The above candidate list includes multiple candidates, A method for decoding an image, wherein the plurality of candidates include at least one of a spatial candidate and a temporal candidate.

2. In paragraph 1, The above spatial candidate is derived based on a candidate block spatially neighboring the current block, A method for decoding an image, wherein the candidate block includes at least one of a neighboring block adjacent to the current block or a block spatially adjacent to the neighboring block.

3. In paragraph 1, The above video decryption method is, A step of constructing a spatial candidate list for the current block; wherein the spatial candidate list includes candidates derived based on motion information of candidate blocks spatially neighboring the current block, and An image decoding method, comprising a step of calculating costs for the candidates included in the spatial candidate list.

4. In paragraph 3, An image decoding method, wherein the top K candidates in ascending order of the above costs are added as the spatial candidates of the above candidate list.

5. In paragraph 1, The motion vector of the above temporal candidate is derived based on the motion vector of the call block corresponding to the current block and a predetermined scaling factor, A method for decoding an image, wherein the scaling factor is derived based on the POC difference between the current picture to which the current block belongs and the reference picture.

6. In paragraph 5, A method for decoding an image, wherein the above reference picture is determined based on the reference index of the call block.

7. In paragraph 5, A method for decoding an image, wherein the above reference picture is determined based on the POC difference between the call picture to which the call block belongs and the reference picture of the call block.

8. In paragraph 1, The steps for constructing the above candidate list are: A step of calculating costs for candidates added to the above candidate list; A step of deriving one or more pairwise candidates through at least two combinations among the top N candidates in ascending order of the above-mentioned calculated costs; and A method for decoding an image, comprising the step of adding one or more of the derived pairwise candidates to the candidate list.

9. Step of constructing a candidate list for the current block; A step of deriving movement information of the current block based on the candidate list; and A step of performing inter prediction on the current block based on the above motion information, The above candidate list includes multiple candidates, A method for encoding an image, wherein the plurality of candidates include at least one of a spatial candidate and a temporal candidate.

10. A computer-readable storage medium for storing a bitstream generated by the image encoding method according to Article 9.

11. Step of constructing a candidate list for the current block; A step of deriving movement information of the current block based on the above candidate list; A step of generating a bitstream by performing inter prediction on the current block based on the motion information; and Including a step of transmitting the generated bitstream, The above candidate list includes multiple candidates, A method for transmitting a bitstream, wherein the plurality of candidates include at least one of a spatial candidate and a temporal candidate.

Images