Image decoding methods, image encoding methods, and computer-readable invariant media.

Abstract

The invention relates to a method for decoding images, which may include the following steps: acquiring first reference patterns located at the top left of the current block; acquiring second reference patterns located at the right and bottom of the current block; and acquiring a prediction pattern for the current block on the basis of the first and second reference patterns, wherein the second reference patterns may be acquired on the basis of the first reference patterns and the provisional prediction patterns are generated by performing provisional internal prediction on the current block on the basis of provisional internal prediction mode.

Description

Video signal processing method and device

[0001] The present invention relates to a video signal processing method and device.

[0002] Recently, the demand for high-resolution, high-quality images, such as HD (High Definition) and UHD (Ultra High Definition) images, is increasing across various application fields. As image data becomes higher in resolution and quality, the relative amount of data increases compared to conventional image data. Therefore, transmitting image data using existing media such as wired and wireless broadband lines or storing it using existing storage media leads to increased transmission and storage costs. To address these issues arising from the increasing resolution and quality of image data, high-efficiency image compression technologies can be utilized.

[0003] There are various technologies for image compression, such as inter-picture prediction technology that predicts pixel values ​​included in the current picture from pictures before or after the current picture, intra-picture prediction technology that predicts pixel values ​​included in the current picture using pixel information in the current picture, and entropy encoding technology that assigns short codes to values ​​with high frequency of appearance and long codes to values ​​with low frequency of appearance. Using these image compression technologies, image data can be effectively compressed and transmitted or stored.

[0004] Meanwhile, as demand for high-resolution video grows, so does the demand for stereoscopic video content as a new video service. Discussions are underway on video compression technologies to effectively deliver high-resolution and ultra-high-resolution stereoscopic video content.

[0005] The purpose of the present invention is to provide a method and device capable of efficiently performing intra prediction on a block to be encoded / decoded when encoding / decoding a video signal.

[0006] The purpose of the present invention is to provide a method and device for performing intra prediction using right and bottom reference samples in encoding / decoding a video signal.

[0007] The purpose of the present invention is to provide a method and device for performing intra prediction that considers both forward and backward directions of an intra prediction mode when encoding / decoding a video signal.

[0008] The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention belongs from the description below.

[0009] A video signal decoding method and device according to the present invention can derive first reference samples located at the top and left of a current block, derive second reference samples located at the right and bottom of the current block, and obtain a prediction sample for the current block based on the first reference sample and the second reference sample. At this time, the second reference samples can be derived based on temporary prediction samples generated by performing temporary intra prediction for the current block based on the first reference samples and a temporary intra prediction mode.

[0010] A video signal encoding method and device according to the present invention can derive first reference samples located at the top and left of a current block, derive second reference samples located at the right and bottom of the current block, and obtain a prediction sample for the current block based on the first reference sample and the second reference sample. At this time, the second reference samples can be derived based on temporary prediction samples generated by performing temporary intra prediction for the current block based on the first reference samples and a temporary intra prediction mode.

[0011] In the video signal encoding / decoding method and device according to the present invention, the second reference sample can be derived by copying a temporary prediction sample located at the boundary of the current block.

[0012] In the video signal encoding / decoding method and device according to the present invention, the second reference sample can be derived based on a temporary prediction sample located at the boundary of the current block and a neighboring temporary prediction sample neighboring the temporary prediction sample.

[0013] In the video signal encoding / decoding method and device according to the present invention, the prediction sample is obtained based on a weighted sum operation of the first reference sample and the second reference sample, and the weight used in the weighted sum operation can be determined based on the position of the prediction sample, the position of the first reference sample, and the position of the second reference sample.

[0014] In the video signal encoding / decoding method and device according to the present invention, the first reference sample and the second reference sample may be positioned in a direction indicated by the intra prediction mode of the current block in the prediction target sample.

[0015] In the video signal encoding / decoding method and device according to the present invention, one of the first reference sample and the second reference sample may be positioned in a direction indicated by the intra prediction mode of the current block in the prediction target sample, and the other may be positioned in an opposite direction to the direction indicated by the intra prediction mode.

[0016] In the video signal encoding / decoding method and device according to the present invention, the temporary intra prediction mode may be a planar or DC mode.

[0017] The features briefly summarized above of the present invention are merely exemplary aspects of the detailed description of the present invention described below and do not limit the scope of the present invention.

[0018] According to the present invention, intra prediction can be efficiently performed on a block to be encoded / decoded.

[0019] According to the present invention, there is an advantage in that the efficiency of intra prediction can be increased by using right and bottom reference samples.

[0020] According to the present invention, there is an advantage in that the efficiency of intra prediction can be increased by considering both the forward and backward directions of the intra prediction mode.

[0021] The effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by a person having ordinary skill in the art to which the present invention belongs from the description below.

[0022] FIG. 1 is a block diagram showing an image encoding device according to an embodiment of the present invention.

[0023] FIG. 2 is a block diagram showing an image decoding device according to an embodiment of the present invention.

[0024] FIG. 3 is a diagram illustrating partition mode candidates that can be applied to a coding block when the coding block is encoded with inter-screen prediction.

[0025] FIG. 4 illustrates an example of hierarchically dividing coding blocks based on a tree structure, as an embodiment of the invention.

[0026] FIG. 5 is a diagram illustrating a partition form that allows binary tree-based division as an embodiment to which the present invention is applied.

[0027] Figure 6 shows a triple tree partitioning form.

[0028] Figure 7 is a diagram showing an example where only a specific type of binary tree-based split is allowed.

[0029] FIG. 8 is a diagram illustrating an example in which information related to the number of allowed binary tree splits is encoded / decoded as an embodiment to which the present invention is applied.

[0030] FIG. 9 illustrates an example of an intra prediction mode pre-defined in an image encoder / decoder to which the present invention is applied.

[0031] Figure 10 is a diagram showing intra prediction modes available under the extended intra mode.

[0032] FIG. 11 is a flowchart schematically illustrating an intra prediction method as an embodiment to which the present invention is applied.

[0033] Figure 12 is a drawing illustrating reference sample line candidates.

[0034] FIG. 13 illustrates an embodiment to which the present invention is applied, and illustrates a method for correcting a prediction sample of a current block based on differential information of surrounding samples.

[0035] Figures 14 and 15 are diagrams showing a one-dimensional reference sample group in which reference samples are rearranged in a row.

[0036] Figure 16 is a diagram showing an example of deriving a right reference sample or a bottom reference sample using multiple reference samples.

[0037] FIG. 17 and FIG. 18 are diagrams for explaining determining a right reference sample and a bottom reference sample for a non-square block according to one embodiment of the present invention.

[0038] Figure 19 is a drawing for explaining an example of deriving a second reference sample using a first reference sample.

[0039] Figure 20 is a diagram showing an example of deriving a right reference sample and a bottom reference sample using a temporary prediction sample obtained based on the planar mode.

[0040] Figure 21 is a diagram illustrating reference samples that constitute a one-dimensional reference sample group.

[0041] Figures 22 and 23 are drawings showing the positions of the first reference target sample and the second reference target sample.

[0042] Figure 24 is a diagram showing an example of obtaining a prediction sample based on a weighted sum operation of a first reference target sample and a second reference target sample.

[0043] Figure 25 is a diagram showing an area to which bidirectional intra prediction is applied.

[0044] Figure 26 identifies directional prediction modes in which bidirectional intra prediction is allowed.

[0045] Figure 27 is a flowchart illustrating a process for determining whether to apply a bidirectional intra prediction mode according to the present invention.

[0046] The present invention is susceptible to various modifications and embodiments. Specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and technical scope of the present invention. Throughout the description of each drawing, similar reference numerals have been used to designate similar components.

[0047] While terms such as "first" and "second" may be used to describe various components, these components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a "second component," and similarly, a second component may also be referred to as a "first component." The term "and / or" includes a combination of multiple related items described herein or any of multiple related items described herein.

[0048] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0049] The terminology used in this application is only used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprise" or "have" indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0050] Hereinafter, with reference to the attached drawings, preferred embodiments of the present invention will be described in more detail. Hereinafter, identical components in the drawings will be designated by the same reference numerals, and redundant descriptions of identical components will be omitted.

[0051]

[0052] FIG. 1 is a block diagram showing an image encoding device according to an embodiment of the present invention.

[0053] Referring to FIG. 1, a video encoding device (100) may include a picture segmentation unit (110), a prediction unit (120, 125), a transformation unit (130), a quantization unit (135), a reordering unit (160), an entropy encoding unit (165), an inverse quantization unit (140), an inverse transformation unit (145), a filter unit (150), and a memory (155).

[0054] Each component shown in Fig. 1 is independently depicted to indicate different characteristic functions in the video encoding device, and does not mean that each component is composed of separate hardware or a single software component. That is, each component is listed and included as a separate component for convenience of explanation, and at least two components among each component may be combined to form a single component, or one component may be divided into multiple components to perform a function, and such integrated and separate embodiments of each component are also included in the scope of the present invention as long as they do not deviate from the essence of the present invention.

[0055] Additionally, some components may not be essential components that perform essential functions of the present invention, but may be optional components merely used to enhance performance. The present invention may be implemented by including only components essential to implementing the essence of the present invention, excluding components used solely for performance enhancement. A structure that includes only essential components, excluding optional components used solely for performance enhancement, is also within the scope of the present invention.

[0056] The picture splitting unit (110) can split the input picture into at least one processing unit. At this time, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). The picture splitting unit (110) can split one picture into a combination of multiple coding units, prediction units, and transform units, and select one combination of coding units, prediction units, and transform units based on a predetermined criterion (e.g., a cost function) to encode the picture.

[0057] For example, a picture can be split into multiple coding units. A recursive tree structure, such as a quad tree structure, can be used to split a coding unit in a picture. A coding unit that is split into other coding units starting from an image or the largest coding unit as the root can be split into as many child nodes as the number of split coding units. A coding unit that cannot be split any further according to a certain restriction becomes a leaf node. That is, assuming that only a square split is possible for a coding unit, a coding unit can be split into at most four different coding units.

[0058] Hereinafter, in the embodiments of the present invention, the encoding unit may be used to mean a unit that performs encoding or may be used to mean a unit that performs decoding.

[0059] A prediction unit may be divided into at least one square or rectangular shape of the same size within a single coding unit, or may be divided such that one prediction unit among the divided prediction units within a single coding unit has a different shape and / or size from another prediction unit.

[0060] When generating a prediction unit that performs intra prediction based on a coding unit, if it is not the minimum coding unit, intra prediction can be performed without dividing it into multiple prediction units NxN.

[0061] The prediction unit (120, 125) may include an inter prediction unit (120) that performs inter prediction and an intra prediction unit (125) that performs intra prediction. It may be determined whether to use inter prediction or intra prediction for a prediction unit, and specific information (e.g., intra prediction mode, motion vector, reference picture, etc.) according to each prediction method may be determined. At this time, the processing unit where prediction is performed and the processing unit where the prediction method and specific contents are determined may be different. For example, the prediction method and prediction mode may be determined in the prediction unit, and the prediction may be performed in the transformation unit. The residual value (residual block) between the generated prediction block and the original block may be input to the transformation unit (130). In addition, the prediction mode information, motion vector information, etc. used for prediction may be encoded together with the residual value in the entropy encoding unit (165) and transmitted to the decoder. When using a specific encoding mode, it is also possible to encode the original block as is and transmit it to the decoding unit without generating a prediction block through the prediction unit (120, 125).

[0062] The inter prediction unit (120) may predict a prediction unit based on information of at least one picture among the previous or subsequent pictures of the current picture, and in some cases, may predict a prediction unit based on information of a portion of an encoded region within the current picture. The inter prediction unit (120) may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

[0063] The reference picture interpolation unit can receive reference picture information from the memory (155) and generate pixel information less than an integer pixel from the reference picture. In the case of luminance pixels, a DCT-based 8-tap interpolation filter with different filter coefficients can be used to generate pixel information less than an integer pixel in units of 1 / 4 pixels. In the case of a chrominance signal, a DCT-based 4-tap interpolation filter with different filter coefficients can be used to generate pixel information less than an integer pixel in units of 1 / 8 pixels.

[0064] The motion prediction unit can perform motion prediction based on a reference picture interpolated by the reference picture interpolation unit. Various methods can be used to derive a motion vector, such as FBMA (Full search-based Block Matching Algorithm), TSS (Three Step Search), and NTS (New Three-Step Search Algorithm). The motion vector can have a motion vector value in units of 1 / 2 or 1 / 4 pixels based on the interpolated pixel. The motion prediction unit can predict the current prediction unit by using different motion prediction methods. Various methods can be used as motion prediction methods, such as the Skip method, the Merge method, the AMVP (Advanced Motion Vector Prediction) method, and the Intra Block Copy method.

[0065] The intra prediction unit (125) can generate a prediction unit based on reference pixel information surrounding the current block, which is pixel information within the current picture. If the surrounding block of the current prediction unit is a block on which inter prediction has been performed and the reference pixel is a pixel on which inter prediction has been performed, the reference pixel included in the block on which inter prediction has been performed can be replaced and used with reference pixel information of the surrounding block on which intra prediction has been performed. That is, if the reference pixel is not available, the unavailable reference pixel information can be replaced and used with at least one reference pixel among the available reference pixels.

[0066] In intra prediction, the prediction mode can have a directional prediction mode that uses reference pixel information according to the prediction direction, and a non-directional mode that does not use directional information when performing prediction. The mode for predicting luminance information and the mode for predicting chrominance information can be different, and the intra prediction mode information used to predict luminance information or the predicted luminance signal information can be utilized to predict chrominance information.

[0067] When performing intra prediction, if the size of the prediction unit and the size of the transformation unit are the same, intra prediction for the prediction unit can be performed based on the pixels located to the left of the prediction unit, the pixels located at the upper left, and the pixels located at the upper. However, when performing intra prediction, if the sizes of the prediction unit and the transformation unit are different, intra prediction can be performed using reference pixels based on the transformation unit. In addition, intra prediction using NxN division can be used only for the minimum coding unit.

[0068] The intra prediction method can generate a prediction block after applying an Adaptive Intra Smoothing (AIS) filter to a reference pixel according to a prediction mode. The type of AIS filter applied to the reference pixel may be different. In order to perform the intra prediction method, the intra prediction mode of the current prediction unit can be predicted from the intra prediction modes of prediction units existing around the current prediction unit. When the prediction mode of the current prediction unit is predicted using the mode information predicted from the surrounding prediction units, if the intra prediction modes of the current prediction unit and the surrounding prediction units are the same, information indicating that the prediction modes of the current prediction unit and the surrounding prediction units are the same can be transmitted using predetermined flag information, and if the prediction modes of the current prediction unit and the surrounding prediction units are different, entropy encoding can be performed to encode the prediction mode information of the current block.

[0069] Additionally, a residual block containing residual value information, which is the difference between the prediction unit that performed the prediction based on the prediction unit generated in the prediction unit (120, 125) and the original block of the prediction unit, can be generated. The generated residual block can be input to the transformation unit (130).

[0070] In the transformation unit (130), the residual block including the residual value information of the prediction unit generated through the original block and the prediction unit (120, 125) can be transformed using a transformation method such as DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), or KLT. Whether to apply DCT, DST, or KLT to transform the residual block can be determined based on the intra prediction mode information of the prediction unit used to generate the residual block.

[0071] The quantization unit (135) can quantize values ​​converted to the frequency domain by the transformation unit (130). The quantization coefficients can vary depending on the block or the importance of the image. The values ​​produced by the quantization unit (135) can be provided to the dequantization unit (140) and the reordering unit (160).

[0072] The rearrangement unit (160) can perform rearrangement of coefficient values ​​for quantized residual values.

[0073] The rearrangement unit (160) can change a two-dimensional block-shaped coefficient into a one-dimensional vector form through a coefficient scanning method. For example, the rearrangement unit (160) can change the two-dimensional block-shaped coefficient into a one-dimensional vector form by scanning from the DC coefficient to the coefficient of the high-frequency region using a zig-zag scan method. Depending on the size of the transformation unit and the intra prediction mode, a vertical scan that scans the two-dimensional block-shaped coefficient in the column direction or a horizontal scan that scans the two-dimensional block-shaped coefficient in the row direction may be used instead of the zig-zag scan. That is, depending on the size of the transformation unit and the intra prediction mode, it is possible to determine which scan method among the zig-zag scan, the vertical scan, and the horizontal scan is to be used.

[0074] The entropy encoding unit (165) can perform entropy encoding based on the values ​​produced by the rearrangement unit (160). Entropy encoding can use various encoding methods such as, for example, Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

[0075] The entropy encoding unit (165) can encode various information such as residual value coefficient information of an encoding unit, block type information, prediction mode information, division unit information, prediction unit information, transmission unit information, motion vector information, reference frame information, block interpolation information, and filtering information from the rearrangement unit (160) and the prediction unit (120, 125).

[0076] The entropy encoding unit (165) can entropy encode the coefficient values ​​of the encoding unit input from the rearrangement unit (160).

[0077] The inverse quantization unit (140) and the inverse transformation unit (145) inversely quantize the values ​​quantized in the quantization unit (135) and inversely transform the values ​​transformed in the transformation unit (130). The residual values ​​generated in the inverse quantization unit (140) and the inverse transformation unit (145) can be combined with the predicted prediction units predicted through the motion estimation unit, motion compensation unit, and intra prediction unit included in the prediction unit (120, 125) to generate a reconstructed block.

[0078] The filter unit (150) may include at least one of a deblocking filter, an offset correction unit, and an ALF (Adaptive Loop Filter).

[0079] A deblocking filter can remove block distortion caused by boundaries between blocks in a reconstructed picture. To determine whether to perform deblocking, a deblocking filter can be applied to the current block based on the pixels contained in several columns or rows within the block. When applying a deblocking filter to a block, a strong filter or a weak filter can be applied depending on the required deblocking filtering strength. Furthermore, when applying a deblocking filter, horizontal and vertical filtering can be processed in parallel when performing vertical and horizontal filtering.

[0080] The offset correction unit can correct the offset from the original image on a pixel-by-pixel basis for an image that has undergone deblocking. To perform offset correction for a specific picture, the pixels contained in the image can be divided into a certain number of regions, the regions to be offset can be determined, and the offset can be applied to those regions. Alternatively, the offset can be applied by considering the edge information of each pixel.

[0081] Adaptive Loop Filtering (ALF) can be performed based on the comparison of the filtered restored image with the original image. After dividing the pixels included in the image into predetermined groups, a filter to be applied to each group can be determined, and filtering can be performed differentially for each group. Information regarding whether to apply ALF can be transmitted by luminance signal for each coding unit (CU), and the shape and filter coefficients of the ALF filter to be applied can vary depending on each block. Furthermore, an ALF filter of the same form (fixed form) can be applied regardless of the characteristics of the target block.

[0082] The memory (155) can store a restored block or picture produced through the filter unit (150), and the stored restored block or picture can be provided to the prediction unit (120, 125) when performing inter prediction.

[0083]

[0084] FIG. 2 is a block diagram showing an image decoding device according to an embodiment of the present invention.

[0085] Referring to FIG. 2, the image decoder (200) may include an entropy decoding unit (210), a reordering unit (215), an inverse quantization unit (220), an inverse transformation unit (225), a prediction unit (230, 235), a filter unit (240), and a memory (245).

[0086] When a video bitstream is input to a video encoder, the input bitstream can be decoded in the opposite procedure to that of the video encoder.

[0087] The entropy decoding unit (210) can perform entropy decoding in a procedure opposite to that of the entropy encoding unit of the video encoder. For example, various methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) can be applied in response to the method performed in the video encoder.

[0088] The entropy decoding unit (210) can decode information related to intra prediction and inter prediction performed in the encoder.

[0089] The reordering unit (215) can perform reordering based on the method in which the bitstream entropy-decoded by the entropy decoding unit (210) is reordered by the encoding unit. The coefficients expressed in the form of a one-dimensional vector can be reordered by restoring them back to coefficients in the form of a two-dimensional block. The reordering unit (215) can perform reordering by receiving information related to the coefficient scanning performed by the encoding unit and performing reverse scanning based on the scanning order performed by the corresponding encoding unit.

[0090] The inverse quantization unit (220) can perform inverse quantization based on the quantization parameters provided by the encoder and the coefficient values ​​of the rearranged block.

[0091] The inverse transform unit (225) can perform inverse transform, i.e., inverse DCT, inverse DST, and inverse KLT, on the transforms performed by the transform unit, i.e., DCT, DST, and KLT, on the quantization result performed by the image encoder. The inverse transform can be performed based on the transmission unit determined by the image encoder. In the inverse transform unit (225) of the image decoder, a transform technique (e.g., DCT, DST, KLT) can be selectively performed according to a plurality of pieces of information, such as a prediction method, the size of the current block, and the prediction direction.

[0092] The prediction unit (230, 235) can generate a prediction block based on the prediction block generation related information provided by the entropy decoding unit (210) and the previously decoded block or picture information provided by the memory (245).

[0093] As described above, when performing intra prediction in the same manner as the operation in the image encoder, if the size of the prediction unit and the size of the transformation unit are the same, intra prediction for the prediction unit is performed based on the pixels on the left side of the prediction unit, the pixels on the upper left side, and the pixels on the upper side. However, when performing intra prediction, if the size of the prediction unit and the size of the transformation unit are different, intra prediction can be performed using reference pixels based on the transformation unit. In addition, intra prediction using NxN division only for the minimum coding unit can be used.

[0094] The prediction unit (230, 235) may include a prediction unit determination unit, an inter prediction unit, and an intra prediction unit. The prediction unit determination unit may receive various information such as prediction unit information input from the entropy decoding unit (210), prediction mode information of an intra prediction method, and motion prediction-related information of an inter prediction method, and may distinguish a prediction unit from a current encoding unit and determine whether the prediction unit performs inter prediction or intra prediction. The inter prediction unit (230) may perform inter prediction on the current prediction unit based on information included in at least one of a previous picture or a subsequent picture of the current picture including the current prediction unit, using information necessary for inter prediction of the current prediction unit provided from the image encoder. Alternatively, inter prediction may be performed based on information of a pre-restored portion of the current picture including the current prediction unit.

[0095] In order to perform inter prediction, it is possible to determine whether the motion prediction method of the prediction unit included in the encoding unit is Skip Mode, Merge Mode, AMVP Mode, or Intra Block Copy Mode based on the encoding unit.

[0096] The intra prediction unit (235) can generate a prediction block based on pixel information in the current picture. If the prediction unit is a prediction unit that has performed intra prediction, intra prediction can be performed based on intra prediction mode information of the prediction unit provided by the image encoder. The intra prediction unit (235) can include an AIS (Adaptive Intra Smoothing) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter is a part that performs filtering on the reference pixels of the current block, and can determine and apply whether to apply the filter according to the prediction mode of the current prediction unit. AIS filtering can be performed on the reference pixels of the current block using the prediction mode and AIS filter information of the prediction unit provided by the image encoder. If the prediction mode of the current block is a mode that does not perform AIS filtering, the AIS filter may not be applied.

[0097] The reference pixel interpolation unit can interpolate the reference pixel to generate a reference pixel of a pixel unit less than an integer value when the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on the pixel value interpolated from the reference pixel. When the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixel, the reference pixel may not be interpolated. The DC filter can generate a prediction block through filtering when the prediction mode of the current block is the DC mode.

[0098] The restored block or picture may be provided to a filter unit (240). The filter unit (240) may include a deblocking filter, an offset correction unit, and an ALF.

[0099] Information about whether a deblocking filter has been applied to a corresponding block or picture can be received from the video encoder, and if a deblocking filter has been applied, information about whether a strong or weak filter has been applied. The deblocking filter of the video decoder can receive information related to the deblocking filter provided by the video encoder, and the video decoder can perform deblocking filtering on the corresponding block.

[0100] The offset correction unit can perform offset correction on the restored image based on the type of offset correction applied to the image during encoding and offset value information.

[0101] ALF can be applied to a coding unit based on information such as whether ALF is applied and ALF coefficient information provided from the encoder. This ALF information can be provided by being included in a specific parameter set.

[0102] The memory (245) can store a restored picture or block so that it can be used as a reference picture or reference block, and can also provide the restored picture to an output unit.

[0103] As described above, in the following embodiments of the present invention, for convenience of explanation, the term coding unit is used as an encoding unit, but it may also be a unit that performs not only encoding but also decoding.

[0104] In addition, the current block represents a block to be encoded / decoded, and may represent a coding tree block (or coding tree unit), an encoding block (or encoding unit), a transform block (or transform unit), or a prediction block (or prediction unit), depending on the encoding / decoding step. In this specification, a 'unit' represents a basic unit for performing a specific encoding / decoding process, and a 'block' may represent a sample array of a predetermined size. Unless otherwise specified, 'block' and 'unit' may be used with the same meaning. For example, in the embodiment described below, an encoding block (coding block) and an encoding unit (coding unit) may be understood to have the same meaning.

[0105]

[0106] A picture can be encoded / decoded by partitioning it into square or non-square basic blocks. At this time, the basic block can be called a coding tree unit. The coding tree unit can be defined as a coding unit of the largest size allowed in a sequence or slice. Information indicating whether the coding tree unit is square or non-square or information related to the size of the coding tree unit can be signaled through a sequence parameter set, a picture parameter set, or a slice header. The coding tree unit can be partitioned into partitions of smaller sizes. At this time, if a partition created by partitioning the coding tree unit is said to have depth 1, the partition created by partitioning the partition of depth 1 can be defined as having depth 2. In other words, a partition created by partitioning a partition of depth k within the coding tree unit can be defined as having depth k+1.

[0107] A partition of arbitrary size generated by splitting a coding tree unit can be defined as a coding unit. A coding unit can be split recursively, or split into basic units for performing prediction, quantization, transformation, or in-loop filtering. For example, a partition of arbitrary size generated by splitting a coding unit can be defined as a coding unit, or a transformation unit or prediction unit, which are basic units for performing prediction, quantization, transformation, or in-loop filtering.

[0108] Alternatively, a prediction block having the same size as or smaller than the coding block can be determined through prediction partitioning of the coding block. For prediction partitioning of the coding block, any one of partition mode (Part_mode) candidates indicating a partition shape of the coding block can be specified. Information for determining a partition index indicating any one of the partition mode candidates can be signaled through a bitstream. Alternatively, the partition index of the coding block can be determined based on at least one of the size, shape, or encoding mode of the coding block. The size or shape of the prediction block can be determined based on the partition mode specified by the partition index. The partition mode candidate can include an asymmetric partition shape (e.g., nLx2N, nRx2N, 2NxnU, 2NxnD). The number or type of asymmetric partition mode candidates that the coding block can use can be determined based on at least one of the size, shape, or encoding mode of the coding block.

[0109] FIG. 3 is a diagram illustrating partition mode candidates that can be applied to a coding block when the coding block is encoded with inter-screen prediction.

[0110] When a coding block is encoded with inter-screen prediction, any one of the eight partition mode candidates illustrated in FIG. 3 can be applied to the coding block.

[0111] On the other hand, if the coding block is encoded with intra-picture prediction, only square partitioning can be applied to the coding block. That is, if the coding block is encoded with intra-picture prediction, the partition mode PART_2Nx2N or PART_NxN can be applied to the coding block.

[0112] PART_NxN can be applied when the coding block has a minimum size. Here, the minimum size of the coding block may be predefined in the encoder and decoder. Alternatively, information regarding the minimum size of the coding block may be signaled via the bitstream. For example, the minimum size of the coding block may be signaled via the slice header. Accordingly, the minimum size of the coding block may be determined differently for each slice.

[0113] As another example, the partition mode candidates available to a coding block may be determined differently depending on at least one of the size or shape of the coding block. For example, the number or type of partition mode candidates available to a coding block may be determined differently depending on at least one of the size or shape of the coding block.

[0114] Alternatively, the type or number of asymmetric partition mode candidates that the coding block can use may be determined based on the size or shape of the coding block. The number or type of asymmetric partition mode candidates that the coding block can use may be determined differently depending on at least one of the size or shape of the coding block. For example, if the coding block has a non-square shape in which the width is greater than the height, at least one of PART_2NxN, PART_2NxnU, or PART_2NxnD may not be used as a partition mode candidate of the coding block. If the coding block has a non-square shape in which the height is greater than the width, at least one of PART_Nx2N, PART_nLx2N, or PART_nRx2N may not be used as a partition mode candidate of the coding block.

[0115] In general, the size of the prediction block can be from 64x64 to 4x4. However, if the coding block is encoded with inter-screen prediction, the prediction block may not have a size of 4x4 to reduce memory bandwidth when performing motion compensation.

[0116] It is also possible to recursively partition a coding block based on the partition mode. That is, a coding block can be partitioned based on the partition mode determined by the partition index, and each partition generated as a result of the coding block partitioning can be defined as a coding block.

[0117] Hereinafter, a method for dividing a coding unit will be described in more detail. In the embodiments described below, a coding unit may refer to a coding tree unit or a coding unit included in a coding tree unit. Furthermore, a "partition" generated by dividing a coding block may refer to a "coding block." The dividing method described below may also be applied to dividing a coding block into multiple prediction blocks or multiple transform blocks.

[0118] A coding unit may be divided by at least one line. At this time, the angle of the line dividing the coding unit may be a value within the range of 0 to 360 degrees. For example, the angle of a horizontal line may be 0 degrees, the angle of a vertical line may be 90 degrees, the angle of a diagonal line in the upper right direction may be 45 degrees, and the angle of a diagonal line in the upper left direction may be 135 degrees.

[0119] When a coding unit is divided into multiple lines, the multiple lines may all have the same angle. Alternatively, at least one of the multiple lines may have a different angle from the other lines. Alternatively, the multiple lines dividing the coding tree unit or the coding unit may have a predefined angle difference (e.g., 90 degrees).

[0120] Information about the lines that divide a coding unit may be determined by the partition mode. Alternatively, information about at least one of the number of lines, their direction, angle, or the position of the lines within a block may be encoded.

[0121] For convenience of explanation, in the embodiments described below, it is assumed that a coding unit is divided into a plurality of coding units using at least one of a vertical line or a horizontal line.

[0122] The number of vertical or horizontal lines partitioning a coding unit may be at least one. For example, a coding unit may be partitioned into two partitions using one vertical line or one horizontal line. Alternatively, a coding unit may be partitioned into three partitions using two vertical lines or two horizontal lines. Alternatively, a coding unit may be partitioned into four partitions, each of which has a width and height half that of the coding unit, using one vertical line and one horizontal line.

[0123] When a coding unit is divided into multiple partitions using at least one vertical line or at least one horizontal line, the partitions may have uniform sizes. Alternatively, one partition may have a different size from the other partitions, or each partition may have a different size. For example, when a coding unit is divided using two horizontal lines or two vertical lines, the coding unit may be divided into three partitions. In this case, the width or height ratio of the three partitions may be n:2n:n, 2n:n:n, or n:n:2n.

[0124] In the embodiments described below, the partitioning of a coding unit into four partitions is referred to as quad-tree-based partitioning. Furthermore, the partitioning of a coding unit into two partitions is referred to as binary-tree-based partitioning. Furthermore, the partitioning of a coding unit into three partitions is referred to as triple-tree-based partitioning.

[0125] In the drawings described below, one vertical line and / or one horizontal line will be illustrated as being used to divide a coding unit, but it will be understood that it is also within the scope of the present invention to divide a coding unit into more or fewer partitions than illustrated by using more vertical lines and / or more horizontal lines than illustrated.

[0126] FIG. 4 illustrates an example of hierarchically dividing coding blocks based on a tree structure, as an embodiment to which the present invention is applied.

[0127] The input video signal is decoded in units of a predetermined block, and the basic unit for decoding the input video signal in this way is called a coding block. The coding block can be a unit that performs intra / inter prediction, transformation, and quantization. In addition, a prediction mode (e.g., an intra-screen prediction mode or an inter-screen prediction mode) is determined in units of the coding block, and the prediction blocks included in the coding block can share the determined prediction mode. The coding block can be a square or non-square block having any size in the range of 8x8 to 64x64, and can be a square or non-square block having a size of 128x128, 256x256, or more.

[0128] Specifically, the coding block can be hierarchically partitioned based on at least one of a quad tree partitioning method, a binary tree partitioning method, or a triple tree partitioning method. The quad tree-based partitioning may refer to a method in which a 2Nx2N coding block is partitioned into four NxN coding blocks. The binary tree-based partitioning may refer to a method in which one coding block is partitioned into two coding blocks. The triple tree-based partitioning may refer to a method in which one coding block is partitioned into three coding blocks. Even if the binary tree- or triple tree-based partitioning is performed, square coding blocks may exist in lower depths.

[0129] Partitions generated by binary tree-based partitioning can be symmetrical or asymmetrical. Furthermore, coding blocks partitioned based on binary trees can be square or non-square blocks (e.g., rectangular).

[0130] Fig. 5 is a diagram showing a partition form of a coding block based on binary tree partitioning. The partition form of a coding block based on binary tree partitioning may include a symmetric type such as 2NxN (horizontal non-square coding unit) or Nx2N (vertical non-square coding unit), or an asymmetric type such as nLx2N, nRx2N, 2NxnU, or 2NxnD. Either the symmetric type or the asymmetric type may be allowed as a partition form of a coding block.

[0131] The triple tree partitioning may include at least one of partitioning a coding block into two vertical lines or partitioning a coding block into two horizontal lines. The triple tree partitioning may generate three non-square partitions.

[0132] Figure 6 shows a triple tree partitioning form.

[0133] The triple tree partitioning form may include a form that divides a coding block into two horizontal lines or a form that divides a coding block into two vertical lines. The width or height ratio of the partitions generated as a result of the partitioning of the coding block may be n:2n:n, 2n:n:n, or n:n:2n.

[0134] The location of the partition with the largest width or height among the three partitions may be predefined in the encoder and decoder. Alternatively, information indicating the partition with the largest width or height among the three partitions may be signaled through the bitstream.

[0135] Only square or non-square symmetric partitions may be allowed for a coding unit. In this case, partitioning a coding unit into square partitions, which are organized into blocks, may correspond to quad-tree CU partitioning, and partitioning a coding unit into symmetric non-square partitions may correspond to binary tree partitioning. Partitioning a coding tree unit into square partitions and symmetric non-square partitions may correspond to quad-tree and binary tree CU partitioning (QTBT).

[0136] Binary tree or triple tree-based partitioning can be performed on a coding block for which quad tree-based partitioning is no longer performed. A coding block generated as a result of binary tree or triple tree-based partitioning can be partitioned into smaller coding blocks. At this time, at least one of quad tree partitioning, triple tree partitioning, or binary tree partitioning can be set not to be applied to the coding block. Alternatively, binary tree partitioning in a predetermined direction or triple tree partitioning in a predetermined direction may not be allowed for the coding block. For example, quad tree partitioning and triple tree partitioning can be set not to be allowed for a coding block generated as a result of binary tree or triple tree-based partitioning. Only binary tree partitioning can be allowed for the coding block.

[0137] Alternatively, only the largest coding block among the three coding blocks generated as a result of the triple-tree-based partitioning may be partitioned into smaller coding blocks. Alternatively, binary-tree-based partitioning or triple-tree-based partitioning may be allowed only for the largest coding block among the three coding blocks generated as a result of the triple-tree-based partitioning.

[0138] The partitioning form of a lower depth partition may be determined dependently on the partitioning form of the upper depth partition. For example, if the upper partition and the lower partition are partitioned based on a binary tree, only partitions based on a binary tree of the same form as the binary tree partitioning form of the upper depth partition may be allowed in the lower depth partition. For example, if the binary tree partitioning form of the upper depth partition is of the 2NxN form, the binary tree partitioning form of the lower depth partition may also be set to the 2NxN form. Alternatively, if the binary tree partitioning form of the upper depth partition is of the Nx2N form, the partitioning form of the lower depth partition may also be set to the Nx2N form.

[0139] Alternatively, the partition with the largest size among the partitions generated as a result of the triple-tree-based partitioning may be set to not allow binary tree partitioning with the same splitting direction as that of the upper depth partition or triple-tree partitioning with the same splitting direction as that of the upper depth partition.

[0140] Alternatively, the partition shape of the lower depth partition may be determined by considering the partition shape of the upper depth partition and the partition shape of the neighboring lower depth partition. Specifically, if the upper depth partition is partitioned based on a binary tree, the partition shape of the lower depth partition may be determined so as not to produce the same result as if the upper depth partition were partitioned based on a quad tree. For example, if the partition shape of the upper depth partition is 2NxN and the partition shape of the neighboring lower depth partition is Nx2N, the partition shape of the current lower depth partition cannot be set to Nx2N. This is because if the partition shape of the current lower depth partition is Nx2N, the same result as if the upper depth partition were partitioned using an NxN quad tree would occur. If the partition shape of the upper depth partition is Nx2N and the partition shape of the neighboring lower depth partition is 2NxN, the partition shape of the current lower depth partition cannot be set to 2NxN. That is, if the binary tree partitioning form of the upper depth partition and the binary tree partitioning form of the neighboring lower depth partition are different, the binary tree partitioning form of the current lower depth partition can be set to be the same as the binary tree partitioning form of the upper depth partition.

[0141] Alternatively, the binary tree partitioning form of the lower depth partition can be set differently from the binary tree partitioning form of the upper depth partition.

[0142] Allowable binary tree partitioning shapes can be determined on a per-sequence, slice, or coding unit basis. For example, the allowable binary tree partitioning shapes for a coding tree unit can be restricted to 2NxN or Nx2N shapes. The allowable partitioning shapes may be predefined in the encoder or decoder. Alternatively, information regarding allowable or disallowed partitioning shapes can be encoded and signaled through the bitstream.

[0143] Figure 7 is a diagram showing an example where only a specific type of binary tree-based split is allowed.

[0144] Figure 7 (a) shows an example in which only binary tree-based division in the form of Nx2N is allowed, and Figure 7 (b) shows an example in which only binary tree-based division in the form of 2NxN is allowed.

[0145] To indicate various partitioning forms, information regarding quad-tree partitioning, information regarding binary tree partitioning, or information regarding triple-tree partitioning may be used. The information regarding quad-tree partitioning may include at least one of information indicating whether quad-tree-based partitioning is performed or information regarding the size / depth of a coding block for which quad-tree-based partitioning is allowed. The information regarding binary tree partitioning may include at least one of information indicating whether binary tree-based partitioning is performed, information indicating whether binary tree-based partitioning is vertical or horizontal, information regarding the size / depth of a coding block for which binary tree-based partitioning is allowed, or information regarding the size / depth of a coding block for which binary tree-based partitioning is not allowed. The information regarding triple-tree partitioning may include at least one of information indicating whether triple-tree-based partitioning is performed, information indicating whether triple-tree-based partitioning is vertical or horizontal, information regarding the size / depth of a coding block for which triple-tree-based partitioning is allowed, or information regarding the size / depth of a coding block for which triple-tree-based partitioning is not allowed. Information about the size of a coding block may indicate the minimum or maximum value of at least one of the width, height, product of width and height, or ratio of width and height of the coding block.

[0146] For example, if the width or height of a coding block is less than or equal to the minimum size allowed for binary tree splitting, or if the split depth of the coding block is greater than the maximum depth allowed for binary tree splitting, binary tree-based splitting may not be allowed for the coding block.

[0147] For example, if the width or height of a coding block is less than or equal to the minimum size allowed for triple-tree splitting, or if the split depth of the coding block is greater than the maximum depth allowed for triple-tree splitting, triple-tree-based splitting may not be allowed for the coding block.

[0148] Information regarding the conditions for allowing a binary tree or triple tree-based segmentation can be signaled via a bitstream. The information can be encoded in units of a sequence, picture, or fragment image. The fragment image can mean at least one of a slice, a tile group, a tile, a brick, a coding block, a prediction block, or a transform block.

[0149] For example, the syntax 'max_mtt_depth_idx_minus1', which indicates the maximum depth at which binary tree / triple tree splitting is allowed, can be encoded / decoded through the bitstream. In this case, max_mtt_depth_idx_minus1+1 can indicate the maximum depth at which binary tree / triple tree splitting is allowed.

[0150] For example, at least one of the number of allowed binary tree / triple tree splits, the maximum depth allowed binary tree / triple tree splits, or the number of depths allowed binary tree / triple tree splits can be signaled at the sequence or slice level. Accordingly, at least one of the number of allowed binary tree / triple tree splits, the maximum depth allowed binary tree / triple tree splits, or the number of depths allowed binary tree / triple tree splits can be different for the first slice and the second slice. For example, in the first slice, binary tree / triple tree splits can be allowed at only one depth, whereas in the second slice, binary tree / triple tree splits can be allowed at two depths.

[0151] Referring to the example illustrated in Fig. 8, in Fig. 8, binary tree splitting is illustrated as being performed on a coding unit with a depth of 2 and a coding unit with a depth of 3. Accordingly, at least one of information indicating the number of times binary tree splitting is performed within a coding tree unit (2 times), information indicating the maximum depth of a partition generated by binary tree splitting within a coding tree unit (depth 3), or information indicating the number of partition depths to which binary tree splitting is applied within a coding tree unit (2, depth 2 and depth 3) can be encoded / decoded through a bitstream.

[0152] Alternatively, the number of times binary tree / triple tree splitting is allowed, the depth at which binary tree / triple tree splitting is allowed, or the number of depths at which binary tree / triple tree splitting is allowed may be predefined in the encoder and decoder. Alternatively, the number of times binary tree / triple tree splitting is allowed, the depth at which binary tree / triple tree splitting is allowed, or the number of depths at which binary tree / triple tree splitting is allowed may be determined based on at least one of an index of a sequence or a slice or a size / shape of a coding unit. For example, in a first slice, binary tree / triple tree splitting is allowed at one depth, and in a second slice, binary tree / triple tree splitting is allowed at two depths.

[0153] As another example, at least one of the number of times binary tree splitting is allowed, the depth at which binary tree splitting is allowed, or the number of depths at which binary tree splitting is allowed may be set differently depending on the temporal identifier (TemporalID) of the slice or picture. Here, the temporal identifier (TemporalID) is for identifying each of a plurality of layers of an image having at least one or more scalability among view, spatial, temporal, or quality.

[0154] As illustrated in FIG. 4, a first coding block 300 having a split depth of k may be split into a plurality of second coding blocks based on a quad tree. For example, the second coding blocks 310 to 340 are square blocks having a size half the width and height of the first coding block, and the split depth of the second coding block may be increased to k+1.

[0155] A second coding block 310 having a split depth of k+1 may be split into a plurality of third coding blocks having a split depth of k+2. Splitting of the second coding block 310 may be performed selectively using either a quad tree or a binary tree depending on the splitting method. Here, the splitting method may be determined based on at least one of information indicating a quad tree-based splitting or information indicating a binary tree-based splitting.

[0156] When the second coding block 310 is split based on a quad tree, the second coding block 310 is split into four third coding blocks 310a each having a size equal to half the width and height of the second coding block, and the split depth of the third coding block 310a can be increased to k+2. On the other hand, when the second coding block 310 is split based on a binary tree, the second coding block 310 can be split into two third coding blocks. At this time, each of the two third coding blocks is a non-square block in which one of the width and height of the second coding block is half the size, and the split depth can be increased to k+2. The second coding block can be determined as a horizontal or vertical non-square block depending on the splitting direction, and the splitting direction can be determined based on information about whether the binary tree-based splitting is in the vertical or horizontal direction.

[0157] Meanwhile, the second coding block 310 may be determined as a terminal coding block that is no longer split based on a quad tree or binary tree, in which case the coding block may be used as a prediction block or a transformation block.

[0158] The third coding block 310a may be determined as a terminal coding block, similar to the division of the second coding block 310, or may be further divided based on a quad tree or binary tree.

[0159] Meanwhile, the third coding block 310b divided based on the binary tree may be further divided based on the binary tree into a vertical coding block (310b-2) or a horizontal coding block (310b-3), and the division depth of the corresponding coding block may be increased to k+3. Alternatively, the third coding block 310b may be determined as a terminal coding block (310b-1) that is no longer divided based on the binary tree, and in this case, the corresponding coding block (310b-1) may be used as a prediction block or a transform block. However, the above-described division process may be performed restrictively based on at least one of information about the size / depth of a coding block for which quad-tree-based division is allowed, information about the size / depth of a coding block for which binary tree-based division is allowed, or information about the size / depth of a coding block for which binary tree-based division is not allowed.

[0160] The number of possible sizes that a coding block can have may be limited to a predetermined number, or the size of a coding block within a predetermined unit may have a fixed value. For example, the size of a coding block within a sequence or the size of a coding block within a picture may be limited to any one of 256x256, 128x128, or 32x32. Information indicating the size of a coding block within a sequence or picture may be signaled through a sequence header or a picture header.

[0161] As a result of the partitioning based on quad trees and binary trees, the coding units can be square or rectangular of arbitrary size.

[0162] As illustrated in FIG. 4, a first coding block 300 having a split depth of k may be split into a plurality of second coding blocks based on a quad tree. For example, the second coding blocks 310 to 340 are square blocks having a size half the width and height of the first coding block, and the split depth of the second coding block may be increased to k+1.

[0163] A second coding block 310 having a split depth of k+1 may be split into a plurality of third coding blocks having a split depth of k+2. Splitting of the second coding block 310 may be performed selectively using either a quad tree or a binary tree depending on the splitting method. Here, the splitting method may be determined based on at least one of information indicating a quad tree-based splitting or information indicating a binary tree-based splitting.

[0164] When the second coding block 310 is split based on a quad tree, the second coding block 310 is split into four third coding blocks 310a each having a size equal to half the width and height of the second coding block, and the split depth of the third coding block 310a can be increased to k+2. On the other hand, when the second coding block 310 is split based on a binary tree, the second coding block 310 can be split into two third coding blocks. At this time, each of the two third coding blocks is a non-square block in which one of the width and height of the second coding block is half the size, and the split depth can be increased to k+2. The second coding block can be determined as a horizontal or vertical non-square block depending on the splitting direction, and the splitting direction can be determined based on information about whether the binary tree-based splitting is in the vertical or horizontal direction.

[0165] Meanwhile, the second coding block 310 may be determined as a terminal coding block that is no longer split based on a quad tree or binary tree, in which case the coding block may be used as a prediction block or a transformation block.

[0166] The third coding block 310a may be determined as a terminal coding block, similar to the division of the second coding block 310, or may be further divided based on a quad tree or binary tree.

[0167] Meanwhile, the third coding block 310b divided based on the binary tree may be further divided based on the binary tree into a vertical coding block (310b-2) or a horizontal coding block (310b-3), and the division depth of the corresponding coding block may be increased to k+3. Alternatively, the third coding block 310b may be determined as a terminal coding block (310b-1) that is no longer divided based on the binary tree, and in this case, the corresponding coding block (310b-1) may be used as a prediction block or a transform block. However, the above-described division process may be performed restrictively based on at least one of information about the size / depth of a coding block for which quad-tree-based division is allowed, information about the size / depth of a coding block for which binary tree-based division is allowed, or information about the size / depth of a coding block for which binary tree-based division is not allowed.

[0168] The number of possible sizes that a coding block can have may be limited to a predetermined number, or the size of a coding block within a predetermined unit may have a fixed value. For example, the size of a coding block within a sequence or the size of a coding block within a picture may be limited to any one of 256x256, 128x128, or 32x32. Information indicating the size of a coding block within a sequence or picture may be signaled through a sequence header or a picture header.

[0169] As a result of the partitioning based on quad trees and binary trees, the coding units can be square or rectangular of arbitrary size.

[0170] Transform skip may be set not to be used in coding units generated as a result of binary tree-based or triple tree-based partitioning. Alternatively, transform skip may be set to be applicable to non-square coding units only in at least one of the vertical direction or the horizontal direction. For example, when transform skip is applied in the horizontal direction, it means that only scaling is performed without transformation / inverse transformation in the horizontal direction, and transformation / inverse transformation using DCT or DST is performed in the vertical direction. When transform skip is applied in the vertical direction, it means that only scaling is performed without transformation / inverse transformation in the vertical direction, and transformation / inverse transformation using DCT or DST is performed in the horizontal direction.

[0171] Information indicating whether to skip the inverse transform for the horizontal direction or whether to skip the inverse transform for the vertical direction can be signaled via the bitstream. For example, information indicating whether to skip the inverse transform for the horizontal direction can be a 1-bit flag, 'hor_transform_skip_flag', and information indicating whether to skip the inverse transform for the vertical direction can be a 1-bit flag, 'ver_transform_skip_flag'.

[0172] The encoder can determine whether to encode 'hor_transform_skip_flag' or 'ver_transform_skip_flag' depending on the size and / or shape of the current block. For example, if the current block is in the form of Nx2N, hor_transform_skip_flag can be encoded and ver_transform_skip_flag can be omitted. If the current block is in the form of 2NxN, ver_transform_skip_flag can be encoded and hor_transform_skip_flag can be omitted.

[0173] Alternatively, based on the size and / or shape of the current block, it may be determined whether to skip transformation in the horizontal direction or in the vertical direction. For example, if the current block is in the form of Nx2N, a transformation skip may be applied in the horizontal direction and transformation / inverse transformation may be performed in the vertical direction. If the current block is in the form of 2NxN, a transformation skip may be applied in the vertical direction and transformation / inverse transformation may be performed in the horizontal direction. The transformation / inverse transformation may be performed based on at least one of DCT or DST.

[0174] As a result of partitioning based on a quad tree, binary tree, or triple tree, coding blocks that are no longer partitioned can be used as prediction blocks or transformation blocks. In other words, coding blocks, prediction blocks, or transformation blocks generated as a result of quad tree partitioning or binary tree partitioning can be used. For example, a prediction image can be generated in units of coding blocks, and a residual signal, which is the difference between the original image and the predicted image, can be transformed in units of coding blocks. In order to generate a prediction image in units of coding blocks, motion information can be determined based on the coding block, or an intra-prediction mode can be determined based on the coding block. Accordingly, the coding block can be encoded using at least one of skip mode, intra-picture prediction, or inter-picture prediction.

[0175] Alternatively, a plurality of coding blocks generated by dividing a coding block may be configured to share at least one of motion information, a merge candidate, a reference sample, a reference sample line, or an intra prediction mode. For example, when a coding block is triple-tree partitioned, the partitions generated by dividing the coding block may share at least one of motion information, a merge candidate, a reference sample, a reference sample line, or an intra prediction mode depending on the size or shape of the coding block. Alternatively, only some of the plurality of coding blocks may be configured to share the information, and the remaining coding blocks may not share the information.

[0176] As another example, it is possible to split a coding block and use a prediction block or transformation block that has a smaller size than the coding block.

[0177] Hereinafter, a method for performing intra prediction on a prediction block or transformation block generated by dividing a coding block or coding blocks will be described in detail.

[0178] FIG. 9 illustrates an example of an intra prediction mode pre-defined in an image encoder / decoder to which the present invention is applied.

[0179] A video encoder / decoder can perform intra prediction using any one of the predefined intra prediction modes. The predefined intra prediction modes for intra prediction can be composed of a non-directional prediction mode (e.g., planar mode, DC mode) and 33 directional prediction modes.

[0180] Alternatively, to improve the accuracy of intra prediction, a greater number of directional prediction modes than 33 can be defined. That is, M extended directional prediction modes can be defined by further subdividing the angles of the directional prediction modes (M>33). A directional prediction mode different from the existing 33 directional prediction modes can be derived based on at least one of the existing 33 directional prediction modes.

[0181] Accordingly, a greater number of intra prediction modes than the 35 intra prediction modes illustrated in FIG. 9 can be defined. A number of available intra prediction modes greater than the 35 illustrated in FIG. 9 can be referred to as extended intra modes.

[0182] Figure 10 is a diagram illustrating the intra prediction modes available under the extended intra mode. Under the extended intra mode, the available intra prediction modes can be composed of two non-directional prediction modes and 65 extended directional prediction modes.

[0183] The number of available intra prediction modes can be set equally for both the luminance and chrominance components. Alternatively, the number of available intra prediction modes can vary for each color component. For example, one of 67 intra prediction modes can be selected and used for the luminance component, while one of 35 intra prediction modes can be selected and used for the chrominance component.

[0184] Alternatively, the number of available intra prediction modes may vary depending on the chrominance format. For example, under the 4:2:0 format, 67 intra prediction modes may be set to be available for the luminance component, and 35 intra prediction modes may be set to be available for the chrominance component. Under the 4:4:4 format, 67 intra prediction modes may be set to be available for both the luminance component and the chrominance component.

[0185] Alternatively, the number of available intra prediction modes may vary depending on the size and / or shape of the block. Specifically, the number of available intra prediction modes may be determined as 35 or 67 depending on the size and / or shape of the PU, TU, or CU. For example, when the size of the CU, TU, or PU is less than 64x64, or when the CU, TU, or PU is an asymmetric partition, the number of available intra prediction modes may be 35. On the other hand, when the size of the CU, TU, or PU is 64x64 or greater, or when the CU, TU, or PU is a symmetric partition, the number of available intra prediction modes may be set to 67.

[0186] Alternatively, for Intra_2Nx2N partitions, the number of available directional intra prediction modes can be set to 65. On the other hand, for Intra_NxN partitions, the number of available directional intra prediction modes can be set to 33.

[0187] Whether the extended intra mode is applied can be determined based on at least one of the size, shape, or color component of the block. At this time, information indicating the size or shape of the block to which the extended intra mode is applied can be signaled through the bitstream. The information can be signaled at the sequence, picture, or slice level. Accordingly, the size of the block to which the extended intra mode is applied can be different for each sequence, picture, or slice. For example, in a first slice, the extended intra mode can be set to be applied to a block larger than 64x64 (e.g., CU, TU, or PU), and in a second slice, the extended intra mode can be set to be applied to a block larger than 32x32.

[0188] Information indicating the size of a block to which the extended intra mode is applied may be obtained by subtracting a predetermined value from the logarithm of the size of the reference block. For example, 'log2_extended_intra_mode_size_minus4', which is obtained by subtracting the integer 4 from the logarithm of the size of the reference block, may be encoded. For example, a value of log2_extended_intra_mode_size_minus4 of 0 may indicate that the extended intra mode is applied to a block having a size of 16x16 or more or a block having a size larger than 16x16, and a value of log2_extended_intra_mode_size_minus4 of 1 may indicate that the extended intra mode is applied to a block having a size of 32x32 or more or a block having a size larger than 32x32.

[0189] As described above, the number of available intra prediction modes can be determined based on at least one of the chrominance component, the chrominance format, the block size, or the shape. Intra prediction for the current block can be performed based on at least one of the available intra prediction modes.

[0190] In addition to the examples described, the intra prediction mode candidates (e.g., the number of MPMs) used to determine the intra prediction mode of the encoding / decoding target block may also be determined based on at least one of the chrominance component, the chrominance format, the size or shape of the block. In addition, it is also possible to use a greater number of intra prediction modes than those illustrated in FIG. 9. For example, it is also possible to further subdivide the directional prediction modes illustrated in FIG. 9 to use 129 directional prediction modes and 2 non-directional prediction modes. Whether to use a greater number of intra prediction modes than those illustrated in FIG. 9 may be determined by considering at least one of the chrominance component, the chrominance component, the size or shape of the block, as in the examples described above.

[0191] With reference to the drawings described below, a method for determining an intra prediction mode of a block to be encoded / decoded and a method for performing intra prediction using the determined intra prediction mode will be described.

[0192]

[0193] With reference to the drawings described below, a method for determining an intra prediction mode of a block to be encoded / decoded and a method for performing intra prediction using the determined intra prediction mode will be described.

[0194] FIG. 11 is a flowchart schematically illustrating an intra prediction method as an embodiment to which the present invention is applied.

[0195] First, a reference sample line index of the current block can be determined (S1110). The reference sample line index can be used to determine a reference sample line used to perform intra prediction of the current block. At least one reference sample line indicated by the reference sample line index among a plurality of reference sample lines can be used to perform intra prediction of the current block.

[0196] Figure 12 is a drawing illustrating reference sample line candidates.

[0197] The Nth reference sample line may include an upper reference sample whose y-coordinate is N less than that of the uppermost row of the current block and a left reference sample whose x-coordinate is N less than that of the leftmost column of the current block. Here, the Nth reference sample line represents a reference sample line whose index is N-1 in the example illustrated in FIG. 12. The Nth reference sample line may include upper reference samples from P(-N, -N) to P(2W+N-1, -N) and left reference samples from P(-N, -N) to P(-N, 2H+N-1). As an example, reference sample line 1 may include upper reference samples from P(-2, _2) to P(2W+1, -2) and left reference samples from P(-2, -2) to P(-2, 2H+1).

[0198] The number of reference sample lines that can be used as reference sample line candidates may be 1, 2, 3, 4, or more. For example, in the example illustrated in FIG. 12, reference sample line 0, reference sample line 1, and reference sample line 3 can be used as reference sample line candidates.

[0199] The number of reference sample lines that can be used as reference sample line candidates or the positions of the reference sample lines can be determined based on at least one of the size, shape, intra prediction mode, or position of the current block. For example, if the current block is adjacent to the boundary of a CTU or the boundary of a tile, the number of reference sample line candidates can be 1 (e.g., reference sample line 0). If the current block is not adjacent to the boundary of a CTU or the boundary of a tile, the number of reference sample line candidates can be 3 (e.g., reference sample line 0, reference sample line 1, reference sample line 3). For example, if the intra prediction mode of the current block belongs to the first range, reference sample line 0, reference sample line 1, and reference sample line 3 can be used as reference sample line candidates. If the intra prediction mode of the current block belongs to the second range, reference sample line 0, reference sample line 2, and reference sample line 2 can be used as reference sample line candidates.

[0200] Information specifying at least one of the reference sample line candidates may be signaled via the bitstream. If the number of available reference sample line candidates is 1, encoding of the index information may be omitted. If the index information is not encoded, reference sample line 0 adjacent to the current block may be considered selected.

[0201] Alternatively, at least one of the reference sample line candidates may be selected based on at least one of the size, shape, position, or intra prediction mode of the current block. For example, if at least one of the width, height, and size of the current block is smaller than a predefined value, reference sample line 0 may be selected. For example, if the current block touches the top boundary of a CTU or tile, reference sample line 0 may be selected.

[0202] Alternatively, a reference sample line may be selected based on whether the current block is divided into sub-blocks. For example, if the current block is divided into sub-blocks, reference sample line 0 may be selected.

[0203] Alternatively, if the current block is divided into multiple sub-blocks, the reference sample line can be determined for each sub-block. Alternatively, all sub-blocks can be defined to have the same reference sample line index.

[0204] If the current block is divided into multiple sub-blocks, intra prediction can be performed on a sub-block basis.

[0205] Multiple reference sample lines may be selected for the current block. Whether to perform intra prediction using the multiple reference sample lines may be adaptively determined based on the size, shape, or intra prediction mode of the current block. For example, if the intra prediction mode of the current block is a non-directional prediction mode or a predefined directional intra prediction mode, multiple reference sample lines may not be used. The predefined directional intra prediction mode may include at least one of a vertical intra prediction mode, a horizontal intra prediction mode, or a diagonal intra prediction mode.

[0206] The multiple reference sample lines may include a reference sample line selected by index information and a reference sample line having a predefined value added or subtracted from the index of the reference sample line. Here, the predefined value may be 1 or 2.

[0207] Alternatively, multiple index information can be signaled via the bistream, each pointing to a different reference sample line.

[0208] The prediction sample may be obtained based on at least one of a weighted sum operation, an average operation, a minimum operation, or a maximum operation of multiple reference samples. Here, the index of a reference sample line including at least one of the multiple reference samples may be different from the index of the reference sample lines including the remaining reference samples.

[0209] Next, the intra prediction mode of the current block can be determined (S1220).

[0210] In order to determine the intra prediction mode of the current block, a Most Probable Mode (MPM) candidate can be derived based on the intra prediction mode of a neighboring block adjacent to the current block. Here, the neighboring block can include at least one of blocks adjacent to the top, bottom, left, right, or corner of the current block. For example, an MPM candidate can be derived based on the intra prediction mode of the top neighboring block and the intra prediction mode of the left neighboring block. The top neighboring block can include an upper neighboring sample at a predefined position whose y-coordinate value is smaller than that of the top row of the current block. The predefined position can be (0, -1), (W / 2, -1), (W-1, -1), or (W, -1). (0, 0) represents the position of the upper left sample included in the current block, and W represents the width of the current block. The left neighboring block can include a left neighboring sample at a predefined position whose x-coordinate value is smaller than that of the leftmost row of the current block. The predefined positions can be (-1, 0), (-1, H / 2), (-1, H-1), or (-1, H). H represents the height of the current block. If the neighboring block is encoded with inter prediction, the intra prediction mode of the neighboring block or the collocated block of the current block can be used to include the MPM candidate.

[0211] The number of MPM (Most Probable Mode) candidates included in the candidate list may be 3, 4, 5, 6, or more. The maximum number of MPM candidates may be a fixed value preset in the video encoder / decoder. Alternatively, the maximum number of MPM candidates may be determined based on attributes of the current block. The attributes may include at least one of the position / size / shape of the current block, the number / type of intra prediction modes available to the current block, the color type (luma / chroma) of the current block, the chrominance format of the current block, and whether the current block is divided into multiple sub-blocks. Alternatively, information indicating the maximum number of MPM candidates may be signaled through the bitstream. The information indicating the maximum number may be signaled at least one of the sequence level, the picture level, the slice level, and the block level.

[0212] The intra prediction mode of a neighboring block, a directional intra prediction mode similar to the neighboring block, or a default mode may be set as MPM candidates. The directional intra prediction mode similar to the neighboring block may be derived by adding or subtracting a predefined value to the intra prediction mode of the neighboring block. The predefined value may be an integer of 1, 2, or a larger integer. The predefined value may be adaptively determined depending on the number of available intra prediction modes. For example, when the number of available intra prediction modes is 35, the predefined value may be set to 1, and when the number of available intra prediction modes is 67, the predefined value may be set to 2. Furthermore, when the number of available intra prediction modes is 131, the predefined value may be set to 4. If both the intra prediction mode of the first neighboring block and the intra prediction mode of the second neighboring block are directional prediction modes, a similar directional intra prediction mode can be derived based on the maximum value of the intra prediction mode of the first neighboring block and the intra prediction mode of the second neighboring block. The default mode can include at least one of a DC mode, a planar mode, a horizontal direction prediction mode, a vertical direction prediction mode, an upper-right diagonal mode, a lower-left diagonal mode, or an upper-left diagonal mode. If the number of MPM candidates included in the MPM candidate list is less than the maximum number, a default mode different from the MPM candidates already inserted in the MPM candidate list can be inserted into the MPM candidate list. The number, type, or priority of the default mode can differ depending on at least one of a reference sample line index of the current block, whether a sub-block unit intra prediction technique is applied to the current block, or a segmentation type of the current block.

[0213] The MPM candidate index may be determined according to a predefined order. For example, if the intra prediction mode of the left neighboring block and the intra prediction mode of the upper neighboring block are different, the intra prediction mode of the left neighboring block may have a lower index value than the intra prediction mode of the upper neighboring block.

[0214] Alternatively, the MPM candidate index may be determined based on the size / shape of the current block. For example, if the current block is a non-square shape with a height greater than its width, the intra prediction mode of the upper neighboring block may have a lower index value than the intra prediction mode of the left neighboring block. If the current block is a non-square shape with a width greater than its height, the intra prediction mode of the left neighboring block may have a lower index value than the intra prediction mode of the upper neighboring block.

[0215] Alternatively, only N predefined intra prediction modes can be used as MPM candidates. If the intra prediction mode of a surrounding block is different from the N predefined intra prediction modes, the intra prediction mode of the surrounding block can be converted into one of the N predefined intra prediction modes, and the converted intra prediction mode can be set as an MPM candidate. A predefined table can be used to convert the intra prediction mode, or a scaling operation based on a predetermined value can be used. Here, the predefined table can define a mapping relationship between intra prediction modes.

[0216] For example, if a neighboring block is encoded in a non-extended intra mode (i.e., there are 35 available intra prediction modes) and the current block is encoded in an extended intra mode (i.e., there are 67 available intra prediction modes), the intra prediction mode index of the neighboring block can be converted into an intra prediction mode index under the extended intra mode. For example, if the intra prediction mode of the left neighboring block is 10 (horizontal mode), it can be converted into index 18 corresponding to the horizontal mode under the extended intra mode.

[0217] Alternatively, if the surrounding blocks are encoded in the extended intra mode and the current block is encoded in the non-extended intra mode, the intra prediction mode index of the surrounding blocks can be converted to an intra prediction mode index under the non-extended intra mode. For example, if the intra prediction mode index of the upper surrounding block is 50 (vertical mode), it can be converted to an index 26 corresponding to the vertical mode under the non-extended intra mode.

[0218] If the index of the reference sample line selected through step S1110 is greater than or equal to a predefined value, the candidate list may be set not to include the DC mode and / or the planar mode. The predefined value may be an integer greater than or equal to 1.

[0219] If the current block is divided into multiple sub-blocks, the current candidate list may be configured not to include DC mode and / or planar mode. Additionally, the candidate list may include a default mode. Depending on the division pattern of the current block, the number or type of default modes may vary.

[0220] Information indicating whether an MPM candidate identical to the intra prediction mode of the current block is included in the candidate list can be signaled via the bitstream. For example, an MPM flag can be signaled via the bitstream. A value of the MPM flag of 0 indicates that an MPM candidate identical to the intra prediction mode of the current block is not included in the candidate list. A value of the MPM flag of 1 indicates that an MPM candidate identical to the intra prediction mode of the current block is included in the candidate list.

[0221] If the above MPM flag indicates that there is an MPM candidate having the same intra prediction mode as the current block, index information specifying one of the MPM candidates included in the candidate list may be signaled through the bitstream. The MPM candidate specified by the index information may be set as the intra prediction mode of the current block. If the encoding / signaling of the above MPM flag is omitted, it may be determined that an MPM candidate having the same intra prediction mode as the current block is included in the candidate list.

[0222] On the other hand, if the MPM flag indicates that there is no MPM candidate identical to the intra prediction mode of the current block, residual mode information may be signaled through the bitstream. The residual mode information is used to specify one of the residual intra prediction modes excluding the MPM candidates included in the candidate list. The intra prediction mode of the current block may be determined using the residual mode information. If the MPM flag indicates that there is no MPM candidate identical to the intra prediction mode of the current block, the MPM candidates may be rearranged in ascending order. Thereafter, the mode value indicated by the residual mode information may be sequentially compared with the rearranged MPM candidates to derive the intra prediction mode of the current block. For example, if the mode value indicated by the residual mode information is less than or equal to the rearranged MPM candidate, 1 may be added to the mode value. If there is no MPM candidate less than or equal to the updated mode value, the updated mode value may be determined as the intra prediction mode of the current block.

[0223] If the index of the reference sample line selected through step S1110 is greater than or equal to a predefined value, the encoding of the MPM flag may be omitted. Accordingly, if the index of the reference sample line is greater than or equal to a predefined value, the intra prediction mode of the current block may be set to an MPM candidate indicated by the index information.

[0224] As previously discussed, if the index of the reference sample line is greater than or equal to a predefined value, the candidate list can be configured to not include DC mode and / or planar mode. Accordingly, if the index of the reference sample line is greater than or equal to a predefined value, DC mode and / or planar mode may not be available for the current block.

[0225] Information indicating whether the intra prediction mode of the current block is a default mode may be signaled through the bitstream. The default mode may include at least one of a DC, a planar, a horizontal mode, a vertical mode, or a diagonal mode. As an example, a default mode flag indicating whether the intra prediction mode of the current block is the same as the default mode may be signaled through the bitstream. The default mode flag may include at least one of a flag indicating whether the intra prediction mode of the current block is a planar mode, a flag indicating whether the intra prediction mode of the current block is a DC mode, a flag indicating whether the intra prediction mode of the current block is a vertical mode, a flag indicating whether the intra prediction mode of the current block is a horizontal mode, or a flag indicating whether the intra prediction mode of the current block is a diagonal mode.

[0226] For example, the is_planar_not_flag flag can be signaled via the bitstream. A value of 0 for the flag indicates that the intra prediction mode of the current block is planar mode. A value of 1 for the flag indicates that the intra prediction mode of the current block is not planar mode.

[0227] The default mode flag may be signaled if the MPM flag indicates that an MPM candidate exists that has the same intra prediction mode as the current block. If the default mode flag indicates that the intra prediction mode of the current block is not the default mode, the MPM candidate specified by the index information may be set as the intra prediction mode of the current block.

[0228] If the current block is divided into multiple sub-blocks, the multiple sub-blocks can share the intra-prediction mode of the current block. Alternatively, the intra-prediction mode can be determined for each sub-block. For example, the above information and / or the residual mode can be encoded / decoded for each sub-block. Alternatively, information indicating whether the intra-prediction mode of the sub-block is the same as that of a previously encoded / decoded sub-block can be signaled through the bitstream. Alternatively, the intra-prediction mode of the current sub-block can be derived by adding / subtracting an offset to the intra-prediction mode of a previously encoded / decoded sub-block.

[0229] If the current block is divided into multiple sub-blocks, encoding of the MPM flag may be omitted. Accordingly, if the current block is divided into multiple sub-blocks, the intra prediction mode of the current block may be set to an MPM candidate indicated by the index information.

[0230] Multiple sub-blocks can share the intra prediction mode of the current block.

[0231] The intra prediction modes of each of the luminance component and the chrominance component can be determined independently. Alternatively, the intra prediction mode of the chrominance component can be determined dependent on the intra prediction mode of the luminance component.

[0232] Specifically, the intra prediction mode of the chrominance component can be determined based on the intra prediction mode of the luminance component, as shown in Table 1 below.

[0233] Intra_chroma_pred_mode[xCb][yCb]IntraPredModeY[xCb][yCb]026101X(0<=X<=34)0340000126342626262101034101031113414026101X

[0234] In Table 1, intra_chroma_pred_mode represents information signaled to specify the intra prediction mode of the chrominance component, and IntraPredModeY represents the intra prediction mode of the luminance component.

[0235]

[0236] Next, reference samples for the current block can be derived (S1130). For example, if the Nth reference sample line is selected through step S1110, upper reference samples from P(-N, -N) to P(2W+N-1, -N) and left reference samples from P(-N, -N) to P(-N, 2H+N-1) can be derived.

[0237] The reference sample can be derived from a restored sample whose encoding / decoding was completed before the current block. The restored sample can refer to a state before the in-loop filter is applied or a state after the in-loop filter is applied.

[0238] A predetermined intra filter may be applied to reference samples. Filtering the reference samples using the intra filter may be referred to as reference sample smoothing. The intra filter may include at least one of a first intra filter applied in a horizontal direction or a second intra filter applied in a vertical direction. Depending on the position of the reference sample, either the first intra filter or the second intra filter may be selectively applied. Alternatively, two intra filters may be applied to one reference sample in duplicate. The filter coefficient of at least one of the first intra filter or the second intra filter may be (1, 2, 1), but is not limited thereto.

[0239] The filtering may be adaptively performed based on at least one of the intra prediction mode of the current block or the size of the transform block for the current block. For example, if the intra prediction mode of the current block is DC mode, vertical mode, or horizontal mode, filtering may not be performed. If the size of the transform block is NxM, filtering may not be performed. Here, N and M may be the same or different values, and may be any of 4, 8, 16, or more. For example, if the size of the transform block is 4x4, filtering may not be performed. Alternatively, whether to perform filtering may be determined based on a comparison result between the difference between the intra prediction mode of the current block and the vertical mode (or horizontal mode) and a pre-defined threshold. For example, filtering may be performed only when the difference between the intra prediction mode of the current block and the vertical mode is greater than the threshold. The threshold may be defined for each size of the transform block, as shown in Table 2.

[0240] 8x8 transform16x16 transform32x32 transformThreshold710

[0241] The intra filter may be determined as one of a plurality of intra filter candidates predefined in the video encoder / decoder. For this purpose, a separate index specifying the intra filter of the current block among the plurality of intra filter candidates may be signaled. Alternatively, the intra filter may be determined based on at least one of the size / shape of the current block, the size / shape of the transform block, information about the filter strength, or the variation of surrounding samples.

[0242] Next, intra prediction can be performed using the intra prediction mode of the current block and the reference sample (S1140).

[0243] A prediction sample can be obtained using the intra prediction mode of the current block and a reference sample. When multiple reference sample lines are selected, a prediction sample can be obtained based on a weighted sum operation or an average operation of reference samples belonging to different reference sample lines. For example, a prediction sample can be derived based on a weighted sum operation of a first reference sample belonging to a first reference sample line and a second reference sample belonging to a second reference sample line. At this time, the weights applied to the first reference sample and the second reference sample can have the same value. Alternatively, the weights applied to each reference sample can be determined based on the distance between the prediction target sample and the reference sample. For example, a weight applied to a reference sample having a closer distance from the prediction target sample among the first reference sample and the second reference sample can have a larger value than a weight applied to the other reference samples.

[0244] However, intra prediction may result in a reduction in the image quality of the predicted image because it utilizes boundary samples from surrounding blocks. Therefore, a correction process may be performed on the prediction samples generated through the above-described prediction process, which will be examined in detail below with reference to Figure 13. However, the correction process described below is not limited to intra prediction samples, and can also be applied to inter prediction samples or restored samples.

[0245]

[0246] FIG. 13 illustrates an embodiment to which the present invention is applied, and illustrates a method for correcting a prediction sample of a current block based on differential information of surrounding samples.

[0247] A prediction sample of a current block can be corrected based on differential information of multiple surrounding samples for the current block. The correction may be performed on all prediction samples belonging to the current block, or may be performed only on prediction samples belonging to a predetermined partial region. The partial region may be one row / column or multiple rows / columns, which may be a region pre-configured for correction in a video encoder / decoder. For example, correction may be performed on one row / column located at the boundary of the current block, or on multiple rows / columns from the boundary of the current block. Alternatively, the partial region may be variably determined based on at least one of the size / shape or the intra prediction mode of the current block.

[0248] The surrounding samples may belong to at least one of the surrounding blocks located at the top, left, or upper-left corner of the current block. The number of surrounding samples used for correction may be two, three, four, or more. The positions of the surrounding samples may be determined variably depending on the positions of the prediction samples to be corrected within the current block. Alternatively, some of the surrounding samples may have fixed positions regardless of the positions of the prediction samples to be corrected, while the rest may have variable positions depending on the positions of the prediction samples to be corrected.

[0249] The difference information of surrounding samples may refer to a difference sample between surrounding samples, or may refer to a value obtained by scaling the difference sample by a predetermined constant value (e.g., 1, 2, 3, etc.). Here, the predetermined constant value may be determined by considering the position of the prediction sample to be corrected, the position of the column or row to which the prediction sample to be corrected belongs, the position of the prediction sample within the column or row, etc.

[0250] For example, if the intra prediction mode of the current block is vertical mode, the final prediction sample can be obtained as in the following mathematical expression 1 by using the difference sample between the surrounding sample p(-1,y) adjacent to the left boundary of the current block and the upper left surrounding sample p(-1,-1).

[0251]

[0252] For example, if the intra prediction mode of the current block is horizontal mode, the final prediction sample can be obtained as in the following mathematical expression 2 by using the difference sample between the surrounding sample p(x,-1) adjacent to the upper boundary of the current block and the upper left surrounding sample p(-1,-1).

[0253]

[0254] For example, if the intra prediction mode of the current block is vertical mode, the final prediction sample can be obtained by using the difference sample between the surrounding sample p(-1, y) adjacent to the left boundary of the current block and the surrounding sample p(-1, -1) at the upper left. At this time, the difference sample can be added to the prediction sample, or the difference sample can be scaled by a predetermined constant value and then added to the prediction sample. The predetermined constant value used for scaling can be determined differently according to the column and / or row. For example, the prediction sample can be corrected as in the following mathematical expressions 3 and 4.

[0255]

[0256]

[0257] For example, if the intra prediction mode of the current block is horizontal mode, the final prediction sample can be obtained by using the difference sample between the surrounding sample p(x,-1) adjacent to the upper boundary of the current block and the upper left surrounding sample p(-1,-1), as described above in the vertical mode. For example, the prediction sample can be corrected as in the following mathematical expressions 5 and 6.

[0258]

[0259]

[0260] If the intra prediction mode of the current block is a directional prediction mode, intra prediction of the current block can be performed based on the directionality of the directional prediction mode. As an example, Table 3 shows the intra direction parameters (intraPredAng) of Mode 2 to Mode 34, which are directional intra prediction modes illustrated in FIG. 9.

[0261] predModeIntra12345678910111213141516intraPredAng-32262117139520-2-5-9-13-17-21predMo deIntra18192021222324252627282930313233intraPredAng-32-26-21-17-13-9-5-2025913172126

[0262] Table 3 illustrates 33 directional intra prediction modes, but it is possible to define more or fewer directional intra prediction modes. The intra directional parameters for the current block can be determined based on a lookup table that defines the mapping relationship between directional intra prediction modes and intra directional parameters. Alternatively, the intra directional parameters for the current block can be determined based on information signaled via the bitstream.

[0263] Intra prediction of a current block can be performed using at least one of a left reference sample or an upper reference sample, depending on the directionality of the directional intra prediction mode. Here, the upper reference sample may refer to reference samples having a y-axis coordinate smaller than a prediction target sample (x, 0) included in the top row in the current block (e.g., from (-1, -1) to (2W-1, -1)), and the left reference sample may refer to reference samples having an x-axis coordinate smaller than a prediction target sample (0, y) included in the leftmost column in the current block (e.g., from (-1, -1) to (-1, 2H-1)).

[0264] Depending on the directionality of the intra prediction mode, the reference samples of the current block can be arranged one-dimensionally. Specifically, if both the top and left reference samples must be used for intra prediction of the current block, the reference samples for each target prediction sample can be selected assuming that they are arranged in a vertical or horizontal line.

[0265] For example, when the intra direction parameter is negative (e.g., for intra prediction modes corresponding to Mode 11 to Mode 25 in Table 3), the upper reference samples and the left reference samples can be rearranged along the horizontal or vertical direction to form a one-dimensional reference sample group (P_ref_1D).

[0266] Figures 14 and 15 are diagrams showing a one-dimensional reference sample group in which reference samples are rearranged in a row.

[0267] Whether the reference samples are rearranged vertically or horizontally can be determined based on the directionality of the intra prediction mode. For example, if the intra prediction mode is oriented to the left (e.g., the index of the intra prediction mode is between 11 and 18 in the example illustrated in FIG. 9), as in the example illustrated in FIG. 14, the upper reference samples of the current block can be rotated counterclockwise to generate a one-dimensional reference sample group in which the left reference samples and the upper reference samples are arranged vertically.

[0268] On the other hand, when the intra prediction mode is upward-facing (e.g., the index of the intra prediction mode is between 19 and 25 in the example illustrated in FIG. 9), as in the example illustrated in FIG. 15, the left reference samples of the current block can be rotated clockwise to generate a one-dimensional reference sample group in which the left reference samples and the top reference samples are arranged in the horizontal direction.

[0269] If the intra direction parameter of the current block is non-negative, intra prediction for the current block can be performed using only the left reference samples or the top reference samples. Accordingly, for intra prediction modes in which the intra direction parameter is non-negative, a one-dimensional reference sample group can be formed using only the left reference samples or the top reference samples.

[0270] Based on the intra-directional parameters, a reference sample decision index iIdx can be derived to specify at least one reference sample used to predict the prediction target sample. In addition, a weight-related parameter i used to determine a weight applied to each reference sample based on the intra-directional parameters fact can be derived. As an example, the following mathematical expression 7 shows an example of deriving the reference sample decision index and weight-related parameters.

[0271]

[0272]

[0273] As shown in mathematical expression 7, iIdx and i fact is variably determined according to the slope of the directional intra prediction mode. At this time, the reference sample specified by iIdx may correspond to an integer pel.

[0274] Based on the reference sample determination index, at least one reference sample can be specified for each target sample to be predicted. For example, based on the reference sample determination index, the location of a reference sample within a one-dimensional reference sample group for predicting the target sample within the current block can be specified. Based on the reference sample at the specified location, a predicted image (i.e., a predicted sample) for the target sample to be predicted can be generated.

[0275]

[0276] Depending on the intra prediction mode of the current block, a prediction image for a prediction target sample can be generated based on one or more reference samples.

[0277] For example, if a virtual angular line extending from a prediction target sample passes through an integer pel position (i.e., a reference sample at an integer position) within a one-dimensional reference sample group, a prediction image for the prediction target sample can be generated by copying the reference sample at the integer pel position or scaling the reference sample by considering the position between the reference sample at the integer pel position and the prediction target sample. The virtual angular line may mean a line extended in one direction / two directions along the angle of the intra prediction mode of the current block or the slope of the intra prediction mode. As an example, the following mathematical expression 8 illustrates an example of generating a prediction image P(x, y) for the prediction target sample at the position (x, y) by copying a reference sample P_ref_1D(x+iIdx+1) specified by the intra prediction mode of the current block.

[0278]

[0279] On the other hand, if a virtual angle line extending from a prediction target sample does not pass through an integer pel position, a prediction image for the prediction target sample can be obtained using multiple reference samples. The prediction image for the prediction target sample can be generated by linearly interpolating a reference sample adjacent to a position where the virtual angle line passes and at least one neighboring reference sample adjacent to the reference sample. Alternatively, the prediction image for the prediction target sample can be obtained by performing tap filter-based interpolation on the reference sample and the at least one neighboring reference sample. The number of taps of the interpolation filter can be a natural number greater than or equal to 2. Specifically, depending on the number of reference samples to be interpolated, the number of taps of the tap filter can be 2, 3, 4, 5, 6, or an integer greater than or equal to 2.

[0280] For example, when a virtual angle line extending from a prediction target sample passes between two integer pel positions, a prediction image for the prediction target sample can be generated using at least one of the reference samples on both sides of the position through which the virtual angle line passes or the reference samples of the two integer pel positions and at least one neighboring reference sample. Here, the neighboring reference sample can include at least one of the reference samples adjacent to the left / right or above / below the reference sample. As an example, the following mathematical expression 9 shows an example of generating a prediction sample P(x, y) for the prediction target sample by interpolating two or more reference samples.

[0281]

[0282] The coefficients of the interpolation filter are the weight-related parameters i fact can be determined based on. For example, the coefficients of the interpolation filter can be determined based on the distance between a fractional pel located on an angular line and an integer pel (i.e., the integer position of each reference sample).

[0283] The following mathematical expression 10 is an example of a case where the number of taps in the tap filter is 4.

[0284]

[0285] As in the example shown in the above mathematical expression 10, a prediction image for a prediction target sample can be obtained by interpolating a plurality of consecutive reference samples. At this time, if at least one of the N consecutive reference samples is not included in the one-dimensional reference sample group, the value of the reference sample can be replaced with a predefined value or the value of a neighboring reference sample. For example, if the sample at the position (x+iIdx-1) is not included in the one-dimensional reference sample group, the value of the reference sample at the position can be replaced with a predefined value or the value of an adjacent reference sample (e.g., P_ref_1D(x+iIdx)). Alternatively, if the sample at the position (x+iIdx+2) is not included in the one-dimensional reference sample group, the value of the reference sample at the position can be replaced with a predefined value, a pre-calculated value, or the value of an adjacent reference sample (e.g., P_ref(x+iIdx+1)). Here, the predefined value may be an integer including 0. The pre-calculated value may be a value determined by bit depth. Alternatively, the pre-calculated value may be calculated based on the average, minimum, or maximum values ​​of at least one reference sample.

[0286] The multi-tap filter may be linear. For example, a linear multi-tap filter may be applied that utilizes multiple reference samples that are continuous in a horizontal or vertical direction. Alternatively, the multi-tap filter may be polygonal, such as a square or cross-shaped shape. For example, a cross-shaped multi-tap filter may be applied that utilizes a reference sample and reference samples adjacent to the reference sample on all sides. The shape of the multi-tap filter may be variably determined based on the size, shape, or intra prediction mode of the current block.

[0287] As shown in mathematical expressions 8 to 10, the technique of generating a prediction sample by interpolating a reference sample using the directionality of intra prediction can be called an intra prediction sample interpolation technique.

[0288] When using an intra prediction sample interpolation technique, a large number of taps in a tap filter does not necessarily guarantee improved prediction accuracy. For example, if the current block is an asymmetric coding unit with a significantly larger height or width than the other, such as 2x16, or a small block, such as 4x4, using a tap filter with more than 4 taps may result in excessive smoothing of the predicted image. Therefore, the type of tap filter can be adaptively determined based on the size, shape, or intra prediction mode of the current block. Here, the type of tap filter can be defined by at least one of the number of taps, filter coefficients, filter strength (strong / weak), filtering direction, and filter shape. The number of filter taps or filter coefficients can be variably determined based on the filter strength. In addition, the application direction of the tap filter can be determined, such as horizontal interpolation, vertical interpolation, or both horizontal and vertical interpolation, depending on the type of tap filter. The application direction of the tap filter can also be variably set on a line-by-line (row or column) basis or a sample-by-sample basis within the current block.

[0289] Specifically, the type of tap filter to be used can be determined based on the width or height of the current block. For example, if at least one of the width or height of the current block is smaller than a predefined value, the intra prediction sample interpolation technique can be performed using a 2-tap filter instead of a 4-tap filter. On the other hand, if both the width and height of the current block are equal to or larger than the predefined value, the intra prediction sample interpolation technique can be performed using a 4-tap filter. Here, the predefined value can represent a value such as 4, 8, or 16.

[0290] Alternatively, the type of tap filter to use can be determined based on whether the width and height of the current block are the same value. For example, if the width and height of the current block are different values, a 2-tap filter can be used instead of a 4-tap filter to perform intra-prediction sample interpolation. Conversely, if the width and height of the current block are the same value, a 4-tap filter can be used to perform intra-prediction sample interpolation.

[0291] Alternatively, the type of tap filter to be used can be determined based on the ratio of the width to height of the current block. For example, if the ratio of the width (w) to the height (h) of the current block (i.e., w / h or h / w) is less than a predefined threshold, a 2-tap filter can be used instead of a 4-tap filter to perform intra prediction sample interpolation. On the other hand, if the ratio of the width to the height of the current block is greater than or equal to a predefined threshold, a 4-tap filter can be used to perform intra prediction sample interpolation.

[0292] Alternatively, the type of tap filter may be determined based on the intra prediction mode, shape, or size of the current block. For example, if the current block is a coding unit having a shape of 2x16 and the intra prediction mode of the current block is an intra prediction mode that falls within the horizontal range, the intra prediction sample interpolation technique may be performed using a tap filter having a tap number of n. On the other hand, if the current block is a coding unit having a shape of 2x16 and the intra prediction mode of the current block is an intra prediction mode that falls within the vertical range, the intra prediction sample interpolation technique may be performed using a tap filter having a tap number of m.

[0293] On the other hand, if the current block is a coding unit of the shape of 16x2 and the intra prediction mode of the current block is an intra prediction mode belonging to the horizontal direction range, the intra prediction sample interpolation technique can be performed using a tap filter with the number of taps n. On the other hand, if the current block is a coding unit of the shape of 16x2 and the intra prediction mode of the current block is an intra prediction mode belonging to the vertical direction range, the intra prediction sample interpolation technique can be performed using a tap filter with the number of taps m.

[0294] Here, the horizontal range may represent a predetermined range including horizontal intra prediction modes, and the vertical range may represent a predetermined range including vertical intra prediction modes. For example, based on 35 intra prediction modes, the horizontal range may represent intra prediction modes from mode 11 to mode 18, and the vertical range may represent intra prediction modes from mode 19 to mode 27.

[0295] Additionally, n and m are constants greater than 0, and n and m may have different values. Alternatively, n and m may be set to have the same value, but at least one of the filter coefficients or filter strengths of the n-tap filter and the m-tap filter may be set to be different.

[0296] When intra prediction is performed based on directional prediction mode or DC mode, there is a risk of image quality degradation at block boundaries. On the other hand, when intra prediction is performed based on planar mode, there is an advantage of relatively less image quality degradation at block boundaries compared to the above prediction modes.

[0297] In intra prediction based on the planar mode, a predicted image can be obtained by weighting a first predicted image in the horizontal direction and a second predicted image in the vertical direction.

[0298] Here, the first predicted image may be generated based on reference samples positioned horizontally with respect to the prediction target sample. For example, the first predicted image may be generated based on a weighted sum of reference samples positioned horizontally with respect to the prediction target sample. At this time, a weight applied to each reference sample may be determined based on at least one of a distance from the prediction target sample or a size of the current block. The reference samples positioned horizontally with respect to the prediction target sample may include a left reference sample having the same y-coordinate as the prediction target sample and a right reference sample having the same y-coordinate as the prediction target sample. The right reference sample may be derived from an upper reference sample of the current block. For example, the right reference sample may be derived by copying a value of an upper reference sample positioned on the same vertical line as the right reference sample. Alternatively, the right reference sample may be derived as a weighted sum or average value of multiple upper reference samples. Here, the upper reference sample positioned on the same vertical line as the right reference sample may include a reference sample adjacent to the upper right corner of the current block. The reference sample adjacent to the upper right corner may have the same x-coordinate as the right reference sample. Alternatively, the position of the upper reference sample used to derive the right reference sample may be variably determined depending on the shape, size, or position of the target sample of the current block.

[0299] The second predicted image may be generated based on reference samples positioned in the vertical direction of the prediction target sample. For example, the second predicted image may be generated based on a weighted sum of reference samples positioned in the vertical direction of the prediction target sample. At this time, the weight applied to each reference sample may be determined in consideration of the distance from the prediction target sample or the size of the current block. The reference samples positioned in the vertical direction of the prediction target sample may include an upper reference sample having the same x-coordinate as the prediction target sample and a lower reference sample having the same x-coordinate as the prediction target sample. The lower reference sample may be derived from a left reference sample of the current block. For example, the lower reference sample may be derived by copying the value of a left reference sample positioned on the same horizontal line as the lower reference sample. Alternatively, the lower reference sample may be derived as a weighted sum or average value of multiple left reference samples. Here, the left reference sample positioned on the same horizontal line as the lower reference sample may include a reference sample adjacent to the lower left corner of the current block. The reference sample adjacent to the lower left corner may have the same y-coordinate as the lower reference sample. Alternatively, the position of the upper reference sample used to derive the lower reference sample may be variably determined depending on the size, shape, or position of the target sample of the current block.

[0300] Alternatively, at least one of the right reference sample or the bottom reference sample can be derived using both the left reference sample and the top reference sample.

[0301] For example, the weighted sum or average of the top reference sample and the left reference sample of the current block can be determined as the value of at least one of the right reference sample or the bottom reference sample.

[0302] Alternatively, the lower right reference sample adjacent to the lower right corner of the current block may be derived using the lower left reference sample and the upper right reference sample, and then the right reference sample and the lower reference sample may be derived using the derived lower right reference sample. The lower right reference sample may be derived based on a weighted sum or average of the upper right reference sample and the left reference sample of the current block. At this time, the weights applied to the upper right reference sample and the left reference sample may have the same value, or may be determined based on the width / height of the current block.

[0303] Once the lower right reference sample is determined, the lower right reference sample and the upper right reference sample can be interpolated to derive the right reference sample, and the lower right reference sample and the lower left reference sample can be interpolated to derive the lower reference sample. At this time, the coefficients of the interpolation filter can be determined based on the size of the current block, the shape of the current block, the distance to the lower right reference sample, the distance to the upper right reference sample, or the distance to the lower left reference sample.

[0304] To derive the right reference sample or the left reference sample, a reference sample at a fixed position can be used, or a reference sample that is adaptively selected depending on the position of the prediction target sample can be used. For example, the right reference sample can be derived using the upper right reference sample regardless of the position of the prediction target sample, or a left reference sample (e.g., a reference sample having the same y-axis coordinate as the prediction target sample) or an upper reference sample (e.g., a reference sample having the same x-axis coordinate as the prediction target sample) selected depending on the position of the prediction target sample. Alternatively, the bottom reference sample can be derived using the lower left reference sample regardless of the position of the prediction target sample, or a left reference sample (e.g., a reference sample having the same y-axis coordinate as the prediction target sample) or an upper reference sample (e.g., a reference sample having the same x-axis coordinate as the prediction target sample) selected depending on the position of the prediction target sample.

[0305] Figure 16 is a diagram illustrating an example of deriving a right reference sample or a bottom reference sample using multiple reference samples. It is assumed that the current block is a block having a size of WxH.

[0306] Referring to (a) of Fig. 16, first, the lower right reference sample P(W, H) can be generated based on the weighted sum or average value of the upper right reference sample P(W, -1) and the lower left sample P(-1, H) of the current block. At this time, the weights applied to the upper right reference sample and the left reference sample can be set to be the same, or determined based on the width (W) and height (H) of the current block. For example, when the current block is non-square, the weight applied to the upper right reference sample can be determined as W / (W+H), and the weight applied to the lower left reference sample can be determined as H / (W+H).

[0307] And, based on the lower right reference sample P(W, H) and the upper right reference sample P(W, -1), the right reference sample P(W, y) for the target prediction sample (x, y) can be generated. For example, the right prediction sample P(W, y) can be calculated as a weighted sum or an average value of the lower right reference sample P(W, H) and the upper right reference sample P(W, -1). In addition, the lower reference sample P(x, H) for the target prediction sample (x, y) can be generated based on the lower right reference sample P(W, H) and the lower left reference sample P(-1, H). For example, the lower reference sample P(x, H) can be calculated as a weighted sum or an average value of the lower right reference sample P(W, H) and the left reference sample P(-1, H).

[0308] As shown in (b) of Fig. 16, when the right reference sample and the bottom reference sample are generated, the first prediction sample P for the prediction target sample is generated using the generated reference sample. h (x, y) and the second prediction sample P v (x, y) can be generated. At this time, the first prediction sample P h (x, y) is generated based on the weighted sum of the left reference sample P(-1, y) and the right reference sample P(W, y), and the second prediction sample P v (x, y) can be generated based on the weighted sum of the upper reference sample P(x, -1) and the lower reference sample P(x, H).

[0309] FIG. 17 and FIG. 18 are diagrams for explaining determining a right reference sample and a bottom reference sample for a non-square block according to one embodiment of the present invention.

[0310] As in the example illustrated in Fig. 17, if the current block is a non-square block of size (N / 2)xN, the right reference sample can be derived based on the upper right reference sample P(N / 2, -1), and the bottom reference sample can be derived based on the lower left reference sample P(-1, N).

[0311] Alternatively, the right reference sample or the bottom reference sample may be derived based on at least one of the weighted sum, average, minimum value, or maximum value of the upper right reference sample P(N / 2, -1) and the lower left reference sample P(-1, N). For example, the right reference sample may be derived based on the weighted sum or average of P(N / 2, -1) and P(-1, N), or the lower right reference sample P(N / 2, N) may be derived based on P(N / 2, -1) and P(-1, N), and then the right reference sample may be derived by interpolating the lower right reference sample and the upper right reference sample. Alternatively, the bottom reference sample may be derived based on the weighted sum or average of P(N / 2, -1) and P(-1, N), or the lower right reference sample P(N / 2, N) may be derived based on P(N / 2, -1) and P(-1, N), and then the bottom reference sample may be derived by interpolating the lower right reference sample and the lower left reference sample.

[0312] On the other hand, as in the example illustrated in Fig. 18, if the current block is a non-square block of size Nx(N / 2), the right reference sample can be derived based on the upper right reference sample P(N, -1), and the bottom reference sample can be derived based on the lower left reference sample P(-1, N / 2).

[0313] Alternatively, the right reference sample or the bottom reference sample may be derived based on at least one of the weighted sum, average, minimum value, or maximum value of the upper left reference sample P(N, -1) and the lower left reference sample P(-1, N / 2). For example, the right reference sample may be derived based on the weighted sum or average of P(N, -1) and P(-1, N / 2), or the lower right reference sample P(N, N / 2) may be derived based on P(N, -1) and P(-1, N / 2), and then the right reference sample may be derived by interpolating the lower right reference sample and the upper right reference sample. Alternatively, the bottom reference sample may be derived based on the weighted sum or average of P(N, -1) and P(-1, N / 2), or the lower right reference sample P(N, N / 2) may be derived based on P(N, -1) and P(-1, N / 2), and then the bottom reference sample may be derived by interpolating the lower right reference sample and the lower left reference sample.

[0314] In the examples described with reference to FIGS. 16 to 18, the lower reference sample is derived based on at least one of the lower left reference sample of the current block located on the same horizontal line as the lower reference sample or the upper right reference sample of the current block located on the same vertical line as the right reference sample, and the right reference sample is derived based on at least one of the upper right reference sample of the current block located on the same vertical line as the right reference sample or the lower left reference sample of the current block located on the same horizontal line as the lower reference sample. Unlike the described examples, the right reference sample may be derived based on at least one of the upper middle reference sample or the left middle reference sample, or the left reference sample may be derived. For example, the lower middle sample may be derived using the upper middle sample and the lower left reference sample, and then the lower samples may be generated through interpolation or extrapolation of the lower middle sample and the lower left sample. In addition, the right middle sample may be derived using the left middle sample and the upper right sample, and then the lower samples may be generated through interpolation or extrapolation of the right middle sample and the upper right sample.

[0315] The positions of the reference samples used to generate the first and second predicted images may be determined differently depending on the size or shape of the current block. For example, the positions of the upper or left reference samples used to derive the right or lower reference samples may be determined differently depending on the size or shape of the current block.

[0316] For example, if the current block is a square block of size NxN, the right reference sample may be derived based on the upper right reference sample P(N, -1), while the bottom reference sample may be derived based on the lower left reference sample P(-1, N). Alternatively, if the current block is a square block of size NxN, the right reference sample and the bottom reference sample may be derived based on at least one of the weighted sum, average, minimum value, or maximum value of the upper right reference sample P(N, -1) and the lower left reference sample P(-1, N).

[0317] On the other hand, if the current block is a non-square block of size Nx2 / N, the lower middle reference sample P(N / 2, -1) can be derived based on the upper middle reference sample P(N / 2, -1) and the lower left reference sample P(-1, N / 2), and the lower reference samples can be derived based on the derived lower middle reference sample. For example, the lower reference samples can be derived through interpolation or extrapolation of the lower middle reference sample and the lower left reference sample. Alternatively, if the current block is a non-square block of size N / 2xN, the right middle reference sample P(N / 2, -1) can be derived based on the upper right reference sample P(N / 2, -1) and the left middle reference sample P(-1, N / 2), and the right reference samples can be derived based on the derived right middle reference sample. For example, the right reference samples can be derived by interpolation or extrapolation of the right middle reference sample and the right upper reference sample.

[0318] The first predicted image can be calculated based on weighted predictions of reference samples that lie on the same horizontal line as the target sample. Furthermore, the second predicted image can be calculated based on weighted predictions of reference samples that lie on the same vertical line as the target sample.

[0319] In addition to the examples described above, the first predicted image or the second predicted image can also be generated using the average value, minimum value, or maximum value of the reference samples.

[0320] Depending on whether the target sample for prediction is included in a predetermined area within the current block, the size or shape of the current block, etc., the method for deriving the reference sample can be set differently, or the method for deriving the first predicted image or the second predicted image can be set differently. Specifically, depending on the location of the target sample for prediction, the number of reference samples used for using the right or bottom reference sample, or the location of the reference samples, or the weights or the number of reference samples used for deriving the first predicted image or the second predicted image can be set differently.

[0321] For example, the right reference sample used when generating the first prediction image of the prediction target samples included in the predetermined area can be derived using only the upper reference sample, and the right reference sample used when generating the first prediction image of the prediction target samples included outside the predetermined area can be derived based on the weighted sum or average of the upper reference sample and the left reference sample.

[0322] For example, as in the example illustrated in FIG. 17, if the current block is a non-square block whose height is longer than its width, the right reference sample of the prediction target sample at the (x, y) position included in the predetermined area within the current block can be derived from P(N / 2, -1). For example, the right reference sample of the prediction target sample included in the predetermined area can be generated by copying the value of the P(N / 2, -1) reference sample. On the other hand, the right reference sample of the prediction target sample at the (x', y') position included outside the predetermined area within the current block can be derived based on the weighted sum or average value of P(N / 2, -1) and P(-1, N). For example, the right reference sample of the prediction target sample included outside the predetermined area can be generated through interpolation of the lower right reference sample P(N / 2, N) and the upper right reference sample P(N / 2, -1), which are derived based on P(N / 2, -1) and P(-1, N).

[0323] Alternatively, as in the example illustrated in FIG. 18, if the current block is a non-square block whose width is longer than its height, the lower reference sample of the prediction target sample at the (x, y) position included in the predetermined area within the current block can be derived from P(-1, N / 2). For example, the lower reference sample of the prediction target sample at the (x, y) position included in the predetermined area can be generated by copying the value of the P(-1, N / 2) reference sample. On the other hand, the lower reference sample of the prediction target sample at the (x', y') position included outside the predetermined area within the current block can be derived based on the weighted sum or average value of P(N, -1) and P(-1, N / 2). For example, the lower target reference sample of the prediction target sample included outside the predetermined area can be generated through interpolation of the lower right reference sample P(N, N / 2) and the lower left reference sample P(-1, N / 2), which are derived based on P(N, -1) and P(-1, N / 2).

[0324] As another example, the prediction target samples included in the predetermined area generate the first prediction image or the second prediction image based on the weighted sum of the reference samples, and the prediction target samples outside the predetermined area generate the first prediction image or the second prediction image with the average value, minimum value, or maximum value of the reference samples, or generate the first prediction image or the second prediction image using only one of the reference samples at a predefined position. For example, as in the example illustrated in FIG. 17, when the current block is a non-square block in which the height is longer than the width, the prediction target sample at the position (x, y) included in the predetermined area within the current block can generate the first prediction image using only one of the right reference sample P(N / 2, y) derived from P(N / 2, -1) or the left reference sample at the position P(-1, y). On the other hand, the prediction target sample at the position (x', y') that is not included in the above-mentioned predetermined area can generate the first prediction image based on the weighted sum or average of the right reference sample P(N / 2, y') derived from P(N / 2, -1) and the reference samples at the position P(-1, y').

[0325] Alternatively, as in the example illustrated in FIG. 18, when the current block is a non-square block whose width is longer than its height, the prediction target sample at the position (x, y) included in a predetermined area within the current block can generate a second prediction image using only one of the lower reference sample P(x, N / 2) derived from P(-1, N / 2) or the upper reference sample at the position P(x, -1). On the other hand, the prediction target sample at the position (x', y') not included in the predetermined area can generate a second prediction image based on a weighted sum or average of the lower reference sample P(x', N / 2) derived from P(-1, N / 2) and the reference samples at the position P(-1, y').

[0326] In the above-described embodiment, the predetermined area may be at least one sample line adjacent to the boundary of the current block or a remaining area excluding these. Here, the boundary of the current block may include at least one of a left boundary, a right boundary, an upper boundary, or a lower boundary. In addition, the number or position of the boundaries used to define the predetermined area may be set differently depending on the shape of the current block. Alternatively, the predetermined area may be a block shape that borders one corner of the current block. In this case, the size and shape of the predetermined area may be determined based on at least one of the size or shape of the current block.

[0327] Under planar mode, the final predicted image can be derived based on the weighted sum, average, minimum or maximum value of the first predicted image and the second predicted image.

[0328] For example, the following mathematical expression 11 is the first predicted image P h and the second predicted image P v An example of generating a final predicted image P based on the weighted sum of .

[0329]

[0330] In the above mathematical expression 11, the prediction weight w may be different depending on the shape and size of the current block or the location of the target sample to be predicted.

[0331] For example, the prediction weight w can be derived by considering the width of the current block, the height of the current block, or the width-height ratio of the current block. If the current block is a non-square block whose width is greater than its height, w can be set so that more weight is given to the first predicted image. Conversely, if the current block is a non-square block whose height is greater than its width, w can be set so that more weight is given to the second predicted image.

[0332] For example, if the current block is square, the prediction weight w may have a value of 1 / 2. On the other hand, if the current block is a non-square block whose height is greater than its width (e.g., (N / 2)xN), the prediction weight w may be set to 1 / 4, and if the current block is a non-square block whose width is greater than its height (e.g., Nx(N / 2)), the prediction weight w may be set to 3 / 4.

[0333]

[0334] In addition to the planar mode, intra prediction can also be performed using reference samples other than the left reference sample and the top reference sample in the DC mode or the directional intra prediction mode. In the following embodiments, the left reference sample and / or the top reference sample will be referred to as first reference samples, and reference samples other than the left reference sample and the top reference sample will be referred to as second reference samples. For example, the second reference sample may include the right reference sample and / or the bottom reference sample of the current block. Here, the bottom reference sample may mean reference samples having a y-axis coordinate greater than a prediction target sample of the bottommost row in the current block, and the right reference sample may mean reference samples having an x-axis coordinate greater than a prediction target sample of the rightmost column in the current block.

[0335] Whether to perform intra prediction using the second reference sample may be determined based on at least one of the size, shape, intra prediction mode, or position of the prediction target sample of the current block. For example, whether to perform intra prediction using the second reference sample may be determined based on whether the intra prediction mode of the current block is vertical mode, horizontal mode, or diagonal mode. Alternatively, intra prediction may be performed using the second reference sample for prediction target samples included in a predetermined area within the current block, while intra prediction may be performed using the first reference sample for prediction target samples not included in the predetermined area within the current block.

[0336] Alternatively, information indicating whether a second reference sample is used may be signaled via the bitstream. The information may be a 1-bit flag, an index used to determine the intra prediction mode of the current block, or the like.

[0337] Alternatively, it may be determined whether to use the second reference sample based on whether a second reference sample was used in a surrounding block of the current block.

[0338] The second reference sample may be generated based on the first reference sample. For example, the second reference sample may be constructed by changing the order of the first reference samples, or the second reference sample may be derived using the first reference sample at a specific location.

[0339] Figure 19 is a drawing for explaining an example of deriving a second reference sample using a first reference sample.

[0340] First, the bottom right reference sample P(W, H) can be derived based on the top right reference sample r(W, -1) and the bottom left reference sample r(-1, H) of the current block. Specifically, the bottom right reference sample can be derived through a weighted sum or average operation of the top right reference sample and the bottom left reference sample. The following mathematical expression 12 illustrates an example of deriving the bottom right reference sample.

[0341]

[0342] As shown in the above mathematical expression 12, the lower right reference sample can be calculated based on the weighted sum between the upper right reference sample and the lower left reference sample. At this time, the weights applied to the upper right reference sample and the lower left reference sample can be determined according to the width and height of the current block. For example, if the current block is square, the same weights can be applied to the upper right reference sample and the lower left reference sample, whereas if the current block is non-square, different weights can be applied to the upper right reference sample and the lower left reference sample. However, the weight setting method shown in mathematical expression 12 is only an example of the present invention, and the present invention is not limited thereto. In addition to the example shown in mathematical expression 12, the weights may be determined based on at least one of the size and shape of the current block, the intra prediction mode, the availability of reference samples, the availability of neighboring blocks, whether neighboring blocks are encoded in the intra prediction mode, or the intra prediction mode of the neighboring blocks.

[0343] The right reference sample can be derived based on the upper right reference sample and the lower right reference sample. For example, the right reference sample can be obtained by interpolating the upper right reference sample and the lower right reference sample. Mathematical expression 13 below illustrates an example of deriving the right reference sample.

[0344]

[0345] As shown in the above mathematical expression 13, the right reference sample P r (W, y) (where y is an integer between 0 and the CU height (cu_height)) can be obtained by weighted prediction of the upper right reference sample r(W, -1) and the lower right reference sample P(W, H). At this time, the weights applied to the upper right reference sample and the lower right reference sample can be determined based on at least one of the width, the height of the current block, or the position of the right reference sample. For example, as in the example shown in Equation 13, a weight of (H-1-y) / H can be applied to the upper right reference sample, while a weight of (y+1) / H can be applied to the lower right reference sample. However, the weight setting method shown in Equation 13 is only an example of the present invention, and the present invention is not limited thereto. In addition to the example shown in Equation 13, the weights may be determined based on at least one of the size and shape of the current block, the intra prediction mode, the availability of reference samples, the availability of neighboring blocks, whether neighboring blocks are encoded in the intra prediction mode, or the intra prediction mode of the neighboring blocks.

[0346] The bottom reference sample can be derived based on the bottom left reference sample and the bottom right reference sample. For example, the bottom reference sample can be obtained by interpolating the bottom left reference sample and the bottom right reference sample. Mathematical expression 14 below illustrates an example of deriving the bottom reference sample.

[0347]

[0348] As shown in the above mathematical expression 14, the lower reference sample P b(x, H) (where x is an integer between 0 and the CU width (cu_width)) can be obtained by weighted prediction of the lower left reference sample r(-1, H) and the lower right reference sample P(W, H). At this time, the weights applied to the lower left reference sample and the lower right reference sample can be determined based on at least one of the width, the height, or the position of the lower reference sample of the current block. For example, as in the example shown in Equation 14, a weight of (W-1-x) / W can be applied to the lower left reference sample, while a weight of (x+1) / H can be applied to the lower right reference sample. However, the weight setting method shown in Equation 14 is merely an example of the present invention, and the present invention is not limited thereto. In addition to the example shown in Equation 14, the weights may be determined based on at least one of the size and shape of the current block, the intra prediction mode, the availability of reference samples, the availability of neighboring blocks, whether neighboring blocks are encoded in the intra prediction mode, or the intra prediction mode of the neighboring blocks.

[0349] If the current block is non-square, the right reference sample and the bottom reference sample may be derived based on the examples described above through FIGS. 17 and 18.

[0350] As in the above examples, second reference samples, such as the right reference sample and the bottom reference sample, can be derived using first reference samples at fixed positions, such as the upper right reference sample and the lower left reference sample. Unlike the above examples, the second reference sample can also be derived using first reference samples at positions different from the upper right reference sample and / or the lower left reference sample. For example, the right reference sample and the bottom reference sample can be derived using first reference samples, such as the upper middle reference sample of the current block or the left middle sample of the current block.

[0351] Alternatively, the first reference sample used to derive the second reference sample may be determined based on the intra prediction mode of the current block. For example, the right reference sample and / or the bottom reference sample may be derived based on the left reference sample and / or the top reference sample, which are specified by the intra prediction mode directionality of the current block.

[0352] Alternatively, the second reference sample may be determined using a plurality of left reference samples and / or a plurality of top reference samples. For example, at least one of the right reference sample, the bottom reference sample, or the bottom-right reference sample may be generated based on a weighted sum, average, maximum, or minimum value of the plurality of left reference samples, or may be generated based on a weighted sum, average, maximum, or minimum value of the plurality of top reference samples.

[0353] Alternatively, a second reference sample may be generated by copying the first reference sample. In this case, the first reference sample used to generate the second reference sample may have a fixed position, or may be adaptively determined based on the size, shape, intra prediction mode of the current block, or the position of the second reference sample.

[0354] In the above example, the number of bottom reference samples is W and the number of right reference samples is H, but a larger number of bottom reference samples and / or right reference samples can be derived. For example, the bottom reference samples can be derived to the same vertical line as the rightmost top reference sample r(2W-1, -1), or the right reference samples can be derived to the same horizontal line as the bottommost left reference sample r(-1, 2H-1).

[0355] At this time, the bottom reference sample having an x-coordinate greater than W can be generated by extrapolating the bottom left reference sample and the bottom right reference sample, or by interpolating the bottom right reference sample P(W, H) and the rightmost bottom reference sample P(2W-1, H). The rightmost bottom reference sample can be generated by copying the rightmost top reference sample r(2W-1, -1), or by performing a weighted sum operation between the rightmost top reference sample and the bottom left reference sample. The right reference sample having a y-coordinate greater than H can be generated by extrapolating the top right reference sample and the bottom right reference sample, or by interpolating the bottom right reference sample P(W, H) and the bottom right reference sample P(W, 2H-1). At this time, the bottom right reference sample can be generated by copying the bottom left reference sample r(-1, 2H-1), or by performing a weighted sum operation between the bottom left reference sample and the leftmost top reference sample.

[0356] Based on first reference samples and a predetermined intra prediction mode, intra prediction can be performed on a current block, and second reference samples can be derived based on the prediction samples generated by the intra prediction. Here, the second reference samples can be derived based on a prediction sample according to a result of performing the intra prediction or a restored sample based on the predicted sample. The restored value can be a value before the in-loop filter is applied or a value after the in-loop filter is applied.

[0357] The above prediction sample is only used to derive the second reference samples and may not be used as the actual prediction sample of the current block. That is, even if intra prediction is performed based on the predetermined intra prediction mode to derive the second reference samples, the actual prediction sample of the current block can be obtained by reapplying an intra prediction mode that is the same as or different from the predetermined intra prediction mode.

[0358] Alternatively, the final prediction sample of the current block may be obtained based on the prediction sample obtained based on the predetermined intra prediction mode and an additional prediction sample obtained based on a prediction mode different from the predetermined intra prediction mode. The final prediction sample may be obtained based on a weighted sum operation or an average operation between the prediction sample and the additional prediction sample.

[0359] In the embodiments described below, the prediction samples used to derive the second reference samples are referred to as temporary prediction samples.

[0360] A given intra prediction mode for obtaining a temporary prediction sample may include at least one of a planar mode, a DC mode, a horizontal mode, a vertical mode, or a diagonal mode.

[0361] Figure 20 is a diagram showing an example of deriving a right reference sample and a bottom reference sample using a temporary prediction sample obtained based on the planar mode.

[0362] Based on the planar mode, a temporary prediction sample for the current block can be obtained. Specifically, a first prediction sample and a second prediction sample for the prediction target sample can be used to generate a temporary prediction sample for the prediction target sample. The first prediction sample can be obtained using a reference sample located in the horizontal direction of the prediction target sample. For example, the first prediction sample can be generated based on a weighted sum operation of a left reference sample having the same y-coordinate as the prediction target sample and a reference sample adjacent to the upper right corner. The second prediction sample can be obtained using a reference sample located in the vertical direction of the prediction target sample. For example, the second prediction sample can be generated based on a weighted sum operation of an upper reference sample having the same x-coordinate as the prediction target sample and a reference sample adjacent to the lower left corner.

[0363] The second reference sample may be generated by copying a temporary prediction sample adjacent to the second reference sample. For example, the right reference sample may be generated by copying a temporary prediction sample located at the right boundary of the current block, and the bottom reference sample may be generated by copying a temporary prediction sample located at the bottom boundary of the current block.

[0364] Alternatively, the second reference sample may be derived by adding an offset to a temporary prediction sample adjacent to the second reference sample. For example, the right reference sample may be derived by adding an offset to a temporary prediction sample located at the right boundary of the current block, and the bottom reference sample may be derived by adding an offset to a temporary prediction sample located at the bottom boundary of the current block. The offset may be a difference value between the temporary prediction sample and a neighboring temporary prediction sample adjacent to the temporary prediction sample.

[0365] Alternatively, the second reference sample may be generated by interpolating multiple temporary prediction samples, or based on a weighted sum operation or an average operation of multiple temporary prediction samples. For example, the right reference sample may be generated by interpolating a temporary prediction sample located at the right boundary of the current block and a neighboring temporary prediction sample neighboring the temporary prediction sample, or based on a weighted sum operation or an average operation of the temporary prediction samples. The bottom reference sample may be generated by interpolating a temporary prediction sample located at the bottom boundary of the current block and a neighboring temporary prediction sample neighboring the temporary prediction sample, or based on a weighted sum operation or an average operation of the temporary prediction samples.

[0366] Alternatively, the second reference sample may be derived based on a weighted sum operation or an average operation of surrounding reference samples. For example, the right reference sample adjacent to the right boundary of the current block may be derived by copying a temporary reference sample located at the right boundary of the current block, and the bottom boundary sample adjacent to the bottom boundary of the current block may be derived by copying a temporary reference sample located at the bottom boundary of the current block. In this case, the reference sample adjacent to the bottom right corner of the current block (e.g., P(W, H)) may be derived based on a weighted sum operation or an average operation of the neighboring right reference sample (e.g., P(W, H-1)) and the neighboring bottom reference sample (e.g., P(W-1, H)).

[0367] A predetermined intra prediction mode for obtaining temporary prediction samples may be predefined in the encoder and decoder. Alternatively, information for determining the predetermined intra prediction mode may be signaled through the bitstream. Alternatively, the predetermined intra prediction mode may be determined based on the intra prediction mode used to obtain the actual prediction samples.

[0368] The first reference samples may be arranged one-dimensionally to create a first one-dimensional reference sample group, and the second reference samples may be arranged one-dimensionally to create a second one-dimensional reference sample group. The first one-dimensional reference sample group may be composed only of the first reference samples, and the second one-dimensional reference sample group may be composed only of the second reference samples. Alternatively, the first one-dimensional reference sample group may be configured to include not only the first reference samples but also at least one or more of the second reference samples, and the second one-dimensional reference sample group may be configured to include not only the second reference samples but also at least one or more of the first reference samples.

[0369] Figure 21 is a diagram illustrating reference samples that constitute a one-dimensional reference sample group.

[0370] As in the example illustrated in (a) of Fig. 21, the first one-dimensional reference sample group may be composed of left reference samples and top reference samples of the current block.

[0371] On the other hand, as in the example illustrated in (b) of FIG. 21, the second one-dimensional reference sample group may be configured to further include some left reference samples and some upper reference samples in addition to the right reference samples and the lower reference samples of the current block.

[0372] That is, among the left reference samples, the lower left reference sample r(-1, H) and the left reference samples having a y-axis coordinate greater than the lower left reference sample can be included in both the first one-dimensional reference sample group and the second one-dimensional reference sample group. In addition, among the upper reference samples, the upper right reference sample r(W, -1) and the upper reference samples having a x-axis coordinate greater than the upper right reference sample can be included in both the first one-dimensional reference sample group and the second one-dimensional reference sample group.

[0373] Alternatively, based on at least one of the size, shape, or intra prediction mode of the current block, some of the first reference samples may be included only in the first one-dimensional reference sample group, or some of the first reference samples may be included only in the second one-dimensional reference sample group. Not only the composition of the one-dimensional reference sample group, but also the arrangement order of the reference samples constituting the one-dimensional reference sample group may be variably determined based on at least one of the size, shape, or intra prediction mode of the current block.

[0374] For convenience of explanation, in the embodiments described below, a reference sample group including a left reference sample and an upper reference sample of the current block will be referred to as a first reference sample group (e.g., a first one-dimensional reference sample group), and a reference sample group including a right reference sample and a lower reference sample of the current block will be referred to as a second reference sample group (e.g., a second one-dimensional reference sample group). For example, the first reference sample group and the second reference sample group may be distinguished depending on whether the right reference sample and the lower reference sample are included. In addition, in order to perform intra prediction of a prediction target sample, a reference sample selected from the first reference sample group will be referred to as a first reference target sample, and a reference sample selected from the second reference sample group will be referred to as a second reference target sample.

[0375] Intra prediction of a current block may be performed using at least one of a first reference sample group or a second reference sample group. For example, a prediction value of a prediction target sample in the current block may be obtained based on at least one of a first reference target sample selected from the first reference sample group or a second reference target sample selected from the second reference sample group. In this case, the first reference target sample and / or the second reference target sample may be determined based on at least one of a shape, a size, or an intra prediction mode of the current block. For example, when an intra prediction mode of the current block is determined, a first reference target sample for the prediction target sample may be specified according to the determined intra prediction mode direction, and a second reference target sample for the prediction target sample may be specified according to the reverse direction of the determined intra prediction mode. Alternatively, the first reference target sample and the second reference target sample may be specified according to the determined intra prediction mode direction.

[0376] Figures 22 and 23 are drawings showing the positions of the first reference target sample and the second reference target sample.

[0377] Figure 22 illustrates an example in which a reference sample located in the direction indicated by the intra prediction mode of the current block is determined as a first reference target sample, and a reference sample located in the opposite direction to the direction is determined as a second reference target sample.

[0378] If the intra prediction mode of the current block is the upper right diagonal direction, a reference sample located in the upper right diagonal direction from the prediction target sample among the reference samples included in the first reference sample group can be determined as the first reference target sample. In addition, a reference sample located in the opposite direction (i.e., the lower left diagonal direction) from the reference samples included in the second reference sample group can be determined as the second reference target sample.

[0379] Figure 23 illustrates an example in which reference samples located in the direction indicated by the intra prediction sample of the current block are determined as the first reference target sample and the second reference target sample, respectively.

[0380] When the intra prediction mode of the current block points to the upper right direction, a reference sample located in the upper right direction of the prediction target sample among the reference samples included in the first reference sample group may be determined as the first reference target sample, and a reference sample located in the upper right direction of the prediction target sample among the reference samples included in the second reference sample group may be determined as the second reference target sample. In (a) of Fig. 23, the upper reference sample is selected as the first reference target sample, and the right reference sample is selected as the second reference target sample.

[0381] When the intra prediction mode of the current block points toward the lower left, a reference sample located toward the lower left of the prediction target sample among the reference samples included in the first reference sample group may be determined as the first reference target sample, and a reference sample located toward the lower left of the prediction target sample among the reference samples included in the second reference sample group may be determined as the second reference target sample. In (b) of Fig. 23, the left reference sample is selected as the first reference target sample, and the bottom reference sample is selected as the second reference target sample.

[0382] Alternatively, the position of the second reference target sample may be determined based on the position of the first reference target sample, or the position of the first reference target sample may be determined based on the position of the second reference target sample. For example, the second reference target sample having the same x-coordinate or the same y-coordinate as the first reference target sample may be selected, or the second reference target sample having a value obtained by adding an offset to the x-coordinate or y-coordinate of the first reference target sample may be selected. Here, the offset may have a fixed value, or may be adaptively determined according to the size, shape, or intra prediction mode of the current block.

[0383] Alternatively, the positions of the first reference target sample and / or the second reference target sample may be determined based on the positions of the prediction target sample. For example, the first reference target sample and / or the second reference target sample having the same x-coordinate or the same y-coordinate as the prediction target sample may be selected, or the first reference target sample and / or the second reference target sample having a value obtained by adding an offset to the x-coordinate or the y-coordinate of the prediction target sample may be selected. Here, the offset may have a fixed value, or may be adaptively determined according to the size, shape, or intra prediction mode of the current block.

[0384] The predicted value of the prediction target sample may be generated based on at least one of a first predicted image based on the first reference target sample or a second predicted image based on the second reference target sample. In this case, the first predicted image may be generated based on the aforementioned mathematical expressions 8 to 10.

[0385] The second predicted image can be generated by interpolating or copying a second reference target sample specified according to the slope of the intra prediction mode of the current block. As an example, Equation 15 is a diagram illustrating an example of deriving a second predicted image by copying a second reference target sample.

[0386]

[0387] In the above mathematical expression 15, P2(x, y) represents the second predicted image, and P_2nd_1D(x+iIdx+1+f) represents the second reference target sample.

[0388] When the slope of the intra prediction mode of the current block cannot be expressed by only one second reference target sample, a plurality of second reference target samples can be interpolated to generate a second prediction image. Specifically, when a virtual angle line according to the slope and / or angle of the intra prediction mode does not pass through an integer pel (i.e., a reference sample at an integer position), the second prediction image can be obtained by interpolating the second reference samples adjacent to the left, right, above, and below the angle line. As an example, mathematical expression 16 shows an example of obtaining a second prediction image by interpolating second reference samples.

[0389]

[0390] The coefficients of the interpolation filter are the weight-related parameters i factcan be determined based on. For example, the coefficients of the interpolation filter can be determined based on the distance between a fractional pel located on an angular line and an integer pel (i.e., the integer position of each reference sample).

[0391] In mathematical expression 16, an interpolation filter with a tap count of 2 is exemplified, but an interpolation filter with a tap count greater than 2 can also be used.

[0392] A final predicted image of a target sample can be obtained based on at least one of the first predicted image or the second predicted image. For example, the first predicted image can be determined as the final predicted image of the target sample, or the second predicted image can be determined as the final predicted image of the target sample. Alternatively, the final predicted image of the target sample can be determined based on a weighted sum or average operation of the first and second predicted images.

[0393] Figure 24 is a diagram showing an example of obtaining a prediction sample based on a weighted sum operation of a first reference target sample and a second reference target sample.

[0394] The predicted value of the prediction target sample can be obtained based on a weighted sum operation of the first prediction image and the second prediction image. In other words, the predicted value can be obtained based on a weighted sum operation of the first reference target sample and the second reference target sample.

[0395] Mathematical expression 17 shows an example of obtaining a final prediction sample based on weighted operations of the first prediction image and the second prediction image.

[0396]

[0397] In the above mathematical expression 17, P1(x, y) represents the first predicted image or the first reference target sample, and P2(x, y) represents the second predicted image or the second reference target sample. In addition, w(x, y) represents the weight assigned to the first predicted image.

[0398] The weight applied to each prediction target sample can be determined based on the location of the prediction target sample. As an example, the following mathematical expression 18 illustrates an example of determining the weight w.

[0399]

[0400] In the above mathematical expression 18, x represents the x-axis coordinate of the prediction target sample, and y represents the y-axis coordinate of the prediction target sample. N represents the L1 norm between the first reference target sample and the second reference target sample. For example, if the coordinate of the first reference target sample is (x0, y0) and the coordinate of the second reference target sample is (x1, y1), then N can be defined as |x0-x1|+|y0-y1|.

[0401] Alternatively, the weights assigned to the first prediction image and the second prediction image may be determined based on at least one of the position of the prediction target sample, the size and shape of the current block, or the intra prediction mode. As an example, Equation 19 illustrates an example in which the weights are determined based on the size of the current block and the position of the prediction target sample.

[0402]

[0403] In the above mathematical expression 19, W and H represent the width and height of the current block, respectively, and (x, y) represents the coordinates of the prediction target sample.

[0404] As in the example shown in the above mathematical expression 19, the weight given to the first prediction image can be set to be larger as the prediction target sample is closer to the upper left corner of the current block, and the weight given to the second prediction image can be set to be larger as the prediction target sample is closer to the lower right corner of the current block.

[0405] Alternatively, weights may be derived from neighboring blocks of the current block. Here, the neighboring blocks of the current block may include at least one of the upper neighboring block, the left neighboring block, or the neighboring block adjacent to a corner (e.g., the upper left neighboring block, the upper right neighboring block, or the lower left neighboring block) of the current block.

[0406] Alternatively, information for determining weights may be signaled via the bitstream. The information may indicate weight values ​​applied to the first or second predicted image, or may indicate weight differences between the current block and neighboring blocks.

[0407] Alternatively, in the first region of the current block, the first predicted image may be set as the final predicted image, and in the second region of the current block, the second predicted image may be set as the final prediction. The first region and the second region may be generated by dividing the current block into two based on a vertical line, a horizontal line, or a diagonal line. For example, the first region and the second region may be rectangular or triangular partitions generated by dividing the current block into two. The predicted value of a position adjacent to the boundary between the first region and the second region may be obtained by weighted prediction of the first predicted image and the second predicted image. At this time, the weights assigned to the first predicted image and the second predicted image may be determined based on at least one of the position of the prediction target sample, the size and shape of the current block, or the intra prediction mode.

[0408] As in the example described above, obtaining a final prediction image based on multiple prediction images can be called bi-intra prediction.

[0409] Bidirectional intra prediction can be applied only to some regions within the current block. The regions to which bidirectional intra prediction is applied may be predefined by the encoder and decoder. For example, bidirectional intra prediction can be applied to a block of a predetermined size (e.g., 4x4) adjacent to the lower right corner within the current block. Alternatively, the region to which bidirectional intra prediction is applied can be adaptively determined based on the size, shape, or intra prediction mode of the current block. Alternatively, information for determining the region to which bidirectional intra prediction is applied (e.g., information indicating the size or position of the region) may be signaled through the bitstream.

[0410] Figure 25 is a diagram showing an area to which bidirectional intra prediction is applied.

[0411] Within a region where bidirectional intra prediction is applied, the first and second predicted images can be weighted and predicted to obtain a final predicted sample. Conversely, within a region where bidirectional intra prediction is not applied, either the first or second predicted image can be determined as the final predicted sample.

[0412] The application area of ​​bidirectional intra prediction can be rectangular or triangular.

[0413] In the above example, it has been described that bidirectional intra prediction is performed using a first reference target sample selected from a first sample group and a second reference target sample selected from a second sample group. Unlike the described example, it is also possible to perform bidirectional intra prediction by selecting multiple reference samples from the first sample group, or to perform bidirectional intra prediction by selecting multiple reference samples from the second sample group. For example, when the intra prediction mode of the current block is the upper right diagonal direction or the lower left diagonal direction, bidirectional intra prediction can be performed by selecting the upper reference sample and the left reference sample from the first sample group. That is, the first reference image obtained based on the upper reference sample and the second reference image obtained based on the lower reference sample can be weighted-predicted to obtain the final prediction sample of the current block.

[0414] Alternatively, depending on the intra prediction mode, bidirectional intra prediction may be performed by selecting the right reference sample and the bottom reference sample from the second sample group.

[0415] Bidirectional intra prediction can be defined as an independent intra prediction mode. For example, by defining N directional prediction modes and N bidirectional intra prediction modes corresponding to the N directional prediction modes, a total of 2N+2 intra prediction modes can be defined. For example, by adding a bidirectional intra prediction mode to the intra prediction modes illustrated in FIG. 9, a total of 68 intra prediction modes (i.e., 2 non-directional intra prediction modes, 33 directional intra prediction modes, and 33 bidirectional intra prediction modes) can be defined. Of course, it is also possible to use more or fewer directional intra prediction modes or bidirectional intra prediction modes than 33.

[0416] Alternatively, after determining the intra prediction mode of the current block, it is also possible to determine whether to use the determined intra prediction mode as a bidirectional prediction mode. For example, once the intra prediction mode of the current block is determined, information on whether to use the determined intra prediction mode as a bidirectional intra prediction mode can be decoded. The information may be a 1-bit flag (e.g., bi_intra_flag), but is not limited thereto. A value of bi_intra_flag of 0 indicates that directional intra prediction is performed, and a value of bi_intra_flag of 1 indicates that bidirectional intra prediction is performed. That is, when the value of bi_intra_flag is 0, the first prediction image is determined as the final prediction sample of the current block, whereas when the value of bi_intra_flag is 1, a weighted prediction of the first prediction image and the second prediction image may be determined as the final prediction sample of the current block.

[0417] Alternatively, whether the current block uses the bidirectional intra prediction mode can be determined based on whether the neighboring blocks adjacent to the current block used the bidirectional intra prediction mode. For example, if the intra prediction mode of the current block is identical to a candidate derived based on the intra prediction mode of the neighboring blocks (i.e., an MPM candidate), whether the current block uses the bidirectional intra prediction mode can be determined in the same manner as whether the neighboring blocks used the bidirectional intra prediction mode.

[0418] Alternatively, it may be determined whether to perform bidirectional intra prediction based on the size and / or shape of the current block. For example, bidirectional intra prediction may be configured to be allowed only for blocks larger than 32x32. Accordingly, bidirectional intra prediction will not be applied when the current block size is smaller than 32x32, whereas bidirectional intra prediction can be applied when the current block size is larger than 32x32.

[0419] As another example, bidirectional intra prediction could be allowed only for square blocks, or bidirectional intra prediction could be allowed only for non-square blocks.

[0420] Alternatively, bidirectional intra prediction may be applied only to some directional intra prediction modes. As an example, FIG. 26 identifies directional prediction modes in which bidirectional intra prediction is allowed. As in the example illustrated in FIG. 26, bidirectional intra prediction may be set to be allowed only for some intra prediction modes between the horizontal direction and the vertical direction. In this case, when an intra prediction mode within the above range is selected, bidirectional intra prediction may be performed by default, and when an intra prediction mode within the above range is selected, whether to perform the bidirectional intra prediction mode may be determined based on at least one of information parsed through the bitstream, the size or shape of the current block.

[0421] The intra prediction modes that allow bidirectional intra prediction are not limited to the examples illustrated in FIG. 26. The intra prediction modes that allow bidirectional intra prediction may be predefined in the encoder and decoder, or may be adaptively determined based on the size and / or shape of the current block. Alternatively, information for determining the intra prediction modes that allow bidirectional intra prediction may be signaled via the bitstream.

[0422] Figure 27 is a flowchart illustrating a process for determining whether to apply a bidirectional intra prediction mode according to the present invention.

[0423] First, it can be determined whether the intra prediction mode of the current block is a non-directional mode (S2710). The non-directional mode may include at least one of a planar or DC mode.

[0424] Information indicating whether the intra prediction mode of the current block is a non-directional mode may be signaled through the bitstream. For example, at least one of a flag indicating whether the intra prediction mode of the current block is a non-directional mode, a flag indicating whether the intra prediction mode of the current block is a planar mode, or a flag indicating whether the intra prediction mode of the current block is a DC mode may be signaled through the bitstream.

[0425] Alternatively, the intra prediction mode of the current block can be determined based on the MPM candidate or residual mode information.

[0426] If the intra prediction mode of the current block is non-directional mode, bidirectional intra prediction is not applied to the current block (S2720).

[0427] If the intra prediction mode of the current block is not a non-directional mode, it is possible to determine whether bidirectional intra prediction is applied to the current block (S2730). Whether bidirectional intra prediction is applied to the current block can be determined based on information parsed from the bitstream, the shape, size, or intra prediction mode of the current block.

[0428] For example, it may be determined whether bidirectional intra prediction is applied to the current block based on a flag (e.g., bi_pred_flag) parsed from the bitstream. A value of bi_pred_flag of 0 indicates that bidirectional intra prediction is not applied to the current block, and a value of bi_pred_flag of 1 indicates that bidirectional intra prediction is applied to the current block. Alternatively, it may be determined whether bidirectional intra prediction is applied to the current block based on whether the intra prediction mode of the current block is a directional prediction mode to which bidirectional intra prediction is applied.

[0429] To perform unidirectional or bidirectional intra prediction, reference samples of the current block can be derived. The reference samples may include first reference samples adjacent to the left and top of the current block and second reference samples adjacent to the right and bottom. The second reference samples can only be derived when bidirectional intra prediction is applied to the current block.

[0430] If bidirectional intra prediction is not applied to the current block, unidirectional intra prediction may be performed based on the intra prediction mode of the current block (S2740). In unidirectional intra prediction, the prediction sample may be acquired based on at least one of the first reference samples.

[0431] When bidirectional intra prediction is applied to the current block, bidirectional intra prediction can be performed according to the intra prediction mode of the current block (S2750). In the bidirectional intra prediction, a prediction sample can be obtained based on a first prediction image obtained based on at least one of the first reference target samples and a second prediction image obtained based on at least one of the second reference samples. The first reference target sample and the second reference target sample can be determined based on the directionality of the intra prediction mode, or can be determined based on the size and shape of the current block or the position of another reference target sample. When the first prediction image and the second prediction image are obtained, the first prediction image and the second prediction image can be weighted-predicted to obtain the prediction sample.

[0432]

[0433] As another example, after dividing the current block into multiple regions, intra prediction can be performed on each of the multiple regions. For example, for a first region included in the current block, intra prediction can be performed using reference samples included in a first reference sample group. On the other hand, for a second region included in the current block, intra prediction can be performed using reference samples derived based on a predicted value or a restored value of the first region, reference samples included in the first reference sample group, or reference samples included in the second reference sample group.

[0434] At this time, the first intra prediction mode applied to the first region and the second intra prediction mode applied to the second region may be the same or different from each other. For example, either the first intra prediction mode or the second intra prediction mode may be predefined in the encoder and the decoder. For example, either the first intra prediction mode or the second intra prediction mode may be a planar mode, a DC mode, a horizontal mode, a vertical mode, or a diagonal mode. Alternatively, either the first intra prediction mode or the second intra prediction mode may be variably determined based on at least one of the size, shape, or division type of the current block. For example, when the width of the current block is greater than the height, either the first intra prediction mode or the second intra prediction mode may be determined as a vertical mode. On the other hand, when the height of the current block is greater than the width, either the first intra prediction mode or the second intra prediction mode may be determined as a horizontal mode. If the width and height of the current block are the same, either the first intra prediction mode or the second intra prediction mode can be determined as the planar mode or the DC mode.

[0435] The other of the first intra prediction mode and the second intra prediction mode can be determined using a derivation method based on the MPM candidate.

[0436] The first region and the second region may each include at least one sample included in the current block. For example, the first region may include at least one of n columns on the left (or right) of the current block or m rows on the top (or bottom) of the current block. n and m may be integers of 1, 2, or more. Alternatively, n and m may be determined based on at least one of the size, shape, or intra prediction mode of the current block. The second region may be a residual region excluding the first region or a portion of the residual region.

[0437]

[0438] Applying the embodiments described above, focusing on the decoding or encoding process, to the encoding or decoding process is within the scope of the present invention. Changing the embodiments described above, in a given order, to a different order is also within the scope of the present invention.

[0439]

[0440] Although the above-described embodiments have been described based on a series of steps or a flowchart, this does not limit the chronological order of the invention, and may be performed simultaneously or in a different order as needed. In addition, each of the components (e.g., units, modules, etc.) constituting the block diagram in the above-described embodiments may be implemented as a hardware device or software, or a plurality of components may be combined to be implemented as a single hardware device or software. The above-described embodiments may be implemented in the form of program commands that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, etc., singly or in combination. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program commands such as ROMs, RAMs, and flash memories. The above hardware device may be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa.

[0441] The present invention can be applied to an electronic device capable of encoding / decoding an image.

Claims

1. A step of deriving first reference samples located at the top and left of the current block; A step of deriving second reference samples located on the right and bottom of the current block; and A step of obtaining a prediction sample for the current block based on a first reference sample and a second reference sample, A method for decoding an image, characterized in that the second reference samples are derived based on temporary prediction samples generated by performing temporary intra prediction on the current block based on the first reference samples and a temporary intra prediction mode.

2. In paragraph 1, A method for decoding an image, characterized in that the second reference sample is derived by copying a temporary prediction sample located at the boundary of the current block.

3. In paragraph 1, A method for decoding an image, characterized in that the second reference sample is derived based on a temporary prediction sample located at the boundary of the current block and a neighboring temporary prediction sample adjacent to the temporary prediction sample.

4. In paragraph 1, The above prediction sample is obtained based on a weighted sum operation of the first reference sample and the second reference sample, An image decoding method, characterized in that the weights used in the weighted sum operation are determined based on the position of the prediction sample, the position of the first reference sample, and the position of the second reference sample.

5. In paragraph 1, A method for decoding an image, characterized in that the first reference sample and the second reference sample are positioned in a direction indicated by the intra prediction mode of the current block in the prediction target sample.

6. In paragraph 1, An image decoding method, characterized in that one of the first reference sample and the second reference sample is located in a direction indicated by the intra prediction mode of the current block in the prediction target sample, and the other is located in a direction opposite to the direction indicated by the intra prediction mode.

7. In paragraph 1, A method for decoding an image, characterized in that the temporary intra prediction mode is a planar or DC mode.

8. A step of deriving first reference samples located at the top and left of the current block; A step of deriving second reference samples located on the right and bottom of the current block; and A step of obtaining a prediction sample for the current block based on a first reference sample and a second reference sample, A video encoding method, characterized in that the second reference samples are derived based on temporary prediction samples generated by performing temporary intra prediction on the current block based on the first reference samples and a temporary intra prediction mode.

9. In paragraph 8, A video encoding method, characterized in that the second reference sample is derived by copying a temporary prediction sample located at the boundary of the current block.

10. In paragraph 8, A video encoding method, characterized in that the second reference sample is derived based on a temporary prediction sample located at the boundary of the current block and a neighboring temporary prediction sample adjacent to the temporary prediction sample.

11. In paragraph 8, The above prediction sample is obtained based on a weighted sum operation of the first reference sample and the second reference sample, A video encoding method, characterized in that the weights used in the weighted sum operation are determined based on the position of the prediction sample, the position of the first reference sample, and the position of the second reference sample.

12. In paragraph 8, A video encoding method, characterized in that the first reference sample and the second reference sample are positioned in a direction indicated by the intra prediction mode of the current block in the prediction target sample.

13. In paragraph 8, A video encoding method, characterized in that one of the first reference sample and the second reference sample is located in a direction indicated by the intra prediction mode of the current block in the prediction target sample, and the other is located in a direction opposite to the direction indicated by the intra prediction mode.

14. In paragraph 8, A method for decoding an image, characterized in that the temporary intra prediction mode is a planar or DC mode.

15. Including an intra prediction unit that derives first reference samples located at the top and left of the current block, derives second reference samples located at the right and bottom of the current block, and obtains a prediction sample for the current block based on the first reference sample and the second reference sample. An image decoding device, characterized in that the second reference samples are derived based on temporary prediction samples generated by performing temporary intra prediction on the current block based on the first reference samples and a temporary intra prediction mode.

Images