Video decoding methods, video encoding methods, and non-temporal computer-readable media.

Abstract

The invention relates to a video decoding method, a video encoding method, a video decoding device, and a video encoding device. The video decoding method under the invention may include the following steps: extracting reference sample lines for the current block, selecting at least two of the reference sample lines, and performing a prediction in the image for the current block based on the selected reference sample lines.

Description

Video signal processing method and device

[0001] The present invention relates to a video signal processing method and device.

[0002] Recently, the demand for high-resolution, high-quality images, such as HD (High Definition) and UHD (Ultra High Definition) images, is increasing across various application fields. As image data becomes higher in resolution and quality, the relative amount of data increases compared to conventional image data. Therefore, transmitting image data using existing media such as wired and wireless broadband lines or storing it using existing storage media leads to increased transmission and storage costs. To address these issues arising from the increasing resolution and quality of image data, high-efficiency image compression technologies can be utilized.

[0003] There are various technologies for image compression, such as inter-picture prediction technology that predicts pixel values ​​included in the current picture from pictures before or after the current picture, intra-picture prediction technology that predicts pixel values ​​included in the current picture using pixel information in the current picture, and entropy encoding technology that assigns short codes to values ​​with high frequency of appearance and long codes to values ​​with low frequency of appearance. Using these image compression technologies, image data can be effectively compressed and transmitted or stored.

[0004] Meanwhile, as demand for high-resolution video grows, so does the demand for stereoscopic video content as a new video service. Discussions are underway on video compression technologies to effectively deliver high-resolution and ultra-high-resolution stereoscopic video content.

[0005] The purpose of the present invention is to provide a method and device capable of efficiently performing intra prediction on a block to be encoded / decoded when encoding / decoding a video signal.

[0006] The present invention aims to provide a method and device capable of performing intra prediction on a block to be encoded / decoded by selecting at least one of a plurality of reference lines when encoding / decoding a video signal.

[0007] The purpose of the present invention is to provide a method and device for deriving a reference line based on a block of a predetermined shape or a predetermined size or larger when encoding / decoding a video signal.

[0008] The present invention aims to provide a method and device for applying an intra filter to at least one of a plurality of reference lines when encoding / decoding a video signal.

[0009] The present invention aims to provide a method and device for adaptively determining an intra prediction mode or the number of intra prediction modes according to a reference line used for intra prediction of a current block when encoding / decoding a video signal.

[0010] The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention belongs from the description below.

[0011] A video signal decoding method and device according to the present invention can derive a plurality of reference sample lines for a current block, select two or more of the plurality of reference sample lines, and perform intra prediction for the current block based on the two or more selected reference sample lines.

[0012] A video signal encoding method and device according to the present invention can derive a plurality of reference sample lines for a current block, select two or more of the plurality of reference sample lines, and perform intra prediction for the current block based on the two or more selected reference sample lines.

[0013] In the video signal encoding / decoding method and device according to the present invention, selecting two or more of the plurality of reference sample lines may correspond to selecting different reference sample lines for each predetermined area of ​​the current block.

[0014] In the video signal encoding / decoding method and device according to the present invention, different reference sample lines can be selected between a first area adjacent to one boundary of the current block and a second area excluding the first area.

[0015] In the video signal encoding / decoding method and device according to the present invention, the one-side boundary can be determined according to the intra prediction mode of the current block.

[0016] In the video signal encoding / decoding method and device according to the present invention, the prediction sample of the current block can be obtained based on a weighted sum operation of reference samples included in different reference sample lines.

[0017] In the video signal encoding / decoding method and device according to the present invention, the weight applied to each of the reference samples can be determined based on at least one of the intra prediction mode of the current block, the distance between the prediction sample and the reference sample, or the position of the reference sample line to which the reference sample belongs.

[0018] In the video signal encoding / decoding method and device according to the present invention, the plurality of reference sample lines can be derived based on a block of a predetermined shape or a predetermined size or larger, including the current block.

[0019] In the video signal encoding / decoding method and device according to the present invention, the current block can share the plurality of reference sample lines with a neighboring block included in the block having the predetermined shape or a predetermined size or larger.

[0020] The features briefly summarized above of the present invention are merely exemplary aspects of the detailed description of the present invention described below and do not limit the scope of the present invention.

[0021] According to the present invention, intra prediction can be efficiently performed on a block to be encoded / decoded.

[0022] According to the present invention, intra prediction can be performed on a block to be encoded / decoded by selecting at least one of a plurality of reference lines.

[0023] According to the present invention, a reference line can be derived based on a block having a predetermined shape or a predetermined size or larger.

[0024] According to the present invention, an intra filter can be applied to at least one of a plurality of reference lines.

[0025] According to the present invention, the intra prediction mode or the number of intra prediction modes can be adaptively determined according to the reference line used for intra prediction of the current block.

[0026] The effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by a person having ordinary skill in the art to which the present invention belongs from the description below.

[0027] FIG. 1 is a block diagram showing an image encoding device according to an embodiment of the present invention.

[0028] FIG. 2 is a block diagram showing an image decoding device according to an embodiment of the present invention.

[0029] FIG. 3 illustrates an example of hierarchically dividing coding blocks based on a tree structure, as an embodiment of the present invention.

[0030] FIG. 4 is a diagram showing a partition form that allows binary tree-based division as an embodiment to which the present invention is applied.

[0031] FIG. 5 is a diagram showing an example in which only a specific type of binary tree-based division is allowed, as an example to which the present invention is applied.

[0032] FIG. 6 is a diagram illustrating an example in which information related to the number of allowed binary tree splits is encoded / decoded as an embodiment to which the present invention is applied.

[0033] FIG. 7 is a diagram illustrating a partition mode that can be applied to a coding block as an embodiment to which the present invention is applied.

[0034] FIG. 8 illustrates an example of an intra prediction mode pre-defined in an image encoder / decoder to which the present invention is applied.

[0035] FIG. 9 illustrates an example of an extended intra prediction mode to which the present invention is applied.

[0036] FIG. 10 is a flowchart schematically illustrating an intra prediction method as an embodiment to which the present invention is applied.

[0037] FIG. 11 illustrates an embodiment to which the present invention is applied, and illustrates a method for correcting a prediction sample of a current block based on differential information of surrounding samples.

[0038] FIG. 12 and FIG. 13 illustrate an embodiment to which the present invention is applied, and illustrate a method for correcting a prediction sample based on a predetermined correction filter.

[0039] FIG. 14 is an example of an application of the present invention, showing the range of reference samples for intra prediction.

[0040] Figures 15 to 17 illustrate an example of reference sample filtering, as an embodiment to which the present invention is applied.

[0041] FIG. 18 is a drawing illustrating a plurality of reference sample lines as an embodiment to which the present invention is applied.

[0042] FIG. 19 is a drawing showing an example in which the use of an extended reference line is determined according to the shape of the current block, as an embodiment to which the present invention is applied.

[0043] FIG. 20 is a flowchart illustrating a method for performing intra prediction based on an extended reference line, as an embodiment to which the present invention is applied.

[0044] FIG. 21 is a drawing illustrating a plurality of reference lines for a non-square block as an example to which the present invention is applied.

[0045] FIG. 22 is a drawing for explaining an example of replacing an unavailable reference sample using an available reference sample located at the shortest distance from the unavailable reference sample, as an embodiment to which the present invention is applied.

[0046] FIG. 23 and FIG. 24 are drawings for explaining an example in which the position of an available reference sample is adaptively determined according to the distance between an unavailable reference sample and an available reference sample belonging to the same reference line as the unavailable reference sample, as an embodiment to which the present invention is applied.

[0047] FIG. 25 and FIG. 26 are drawings showing reference samples used to derive the average value of the reference line as an embodiment to which the present invention is applied.

[0048] Figure 27 is a diagram showing an example of obtaining a prediction sample using multiple reference lines.

[0049] Figure 28 is a drawing for explaining an example of determining a reference line based on the location information of a sample to be decrypted.

[0050] Figure 29 is a diagram showing an example of performing intra prediction using a reference sample derived based on a block of a predetermined shape.

[0051] Figure 30 is a diagram showing an example of performing intra prediction using a plurality of reference lines derived based on a block of a predetermined shape.

[0052] Figure 31 is a flowchart illustrating a process for obtaining a residual sample as an embodiment to which the present invention is applied.

[0053] The present invention is susceptible to various modifications and embodiments. Specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and technical scope of the present invention. Throughout the description of each drawing, similar reference numerals have been used to designate similar components.

[0054] While terms such as "first" and "second" may be used to describe various components, these components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a "second component," and similarly, a second component may also be referred to as a "first component." The term "and / or" includes a combination of multiple related items described herein or any of multiple related items described herein.

[0055] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0056] The terminology used in this application is only used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprise" or "have" indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0057] Hereinafter, with reference to the attached drawings, preferred embodiments of the present invention will be described in more detail. Hereinafter, identical components in the drawings will be designated by the same reference numerals, and redundant descriptions of identical components will be omitted.

[0058]

[0059] FIG. 1 is a block diagram showing an image encoding device according to an embodiment of the present invention.

[0060] Referring to FIG. 1, a video encoding device (100) may include a picture segmentation unit (110), a prediction unit (120, 125), a transformation unit (130), a quantization unit (135), a reordering unit (160), an entropy encoding unit (165), an inverse quantization unit (140), an inverse transformation unit (145), a filter unit (150), and a memory (155).

[0061] Each component shown in Fig. 1 is independently depicted to indicate different characteristic functions in the video encoding device, and does not mean that each component is composed of separate hardware or a single software component. That is, each component is listed and included as a separate component for convenience of explanation, and at least two components among each component may be combined to form a single component, or one component may be divided into multiple components to perform a function, and such integrated and separate embodiments of each component are also included in the scope of the present invention as long as they do not deviate from the essence of the present invention.

[0062] Additionally, some components may not be essential components that perform essential functions of the present invention, but may be optional components merely used to enhance performance. The present invention may be implemented by including only components essential to implementing the essence of the present invention, excluding components used solely for performance enhancement. A structure that includes only essential components, excluding optional components used solely for performance enhancement, is also within the scope of the present invention.

[0063] The picture splitting unit (110) can split the input picture into at least one processing unit. At this time, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). The picture splitting unit (110) can split one picture into a combination of multiple coding units, prediction units, and transform units, and select one combination of coding units, prediction units, and transform units based on a predetermined criterion (e.g., a cost function) to encode the picture.

[0064] For example, a picture can be split into multiple coding units. A recursive tree structure, such as a quad tree structure, can be used to split a coding unit in a picture. A coding unit that is split into other coding units starting from an image or the largest coding unit as the root can be split into as many child nodes as the number of split coding units. A coding unit that cannot be split any further according to a certain restriction becomes a leaf node. That is, assuming that only a square split is possible for a coding unit, a coding unit can be split into at most four different coding units.

[0065] Hereinafter, in the embodiments of the present invention, the encoding unit may be used to mean a unit that performs encoding or may be used to mean a unit that performs decoding.

[0066] A prediction unit may be divided into at least one square or rectangular shape of the same size within a single coding unit, or may be divided such that one prediction unit among the divided prediction units within a single coding unit has a different shape and / or size from another prediction unit.

[0067] When generating a prediction unit that performs intra prediction based on a coding unit, if it is not the minimum coding unit, intra prediction can be performed without dividing it into multiple prediction units NxN.

[0068] The prediction unit (120, 125) may include an inter prediction unit (120) that performs inter prediction and an intra prediction unit (125) that performs intra prediction. It may be determined whether to use inter prediction or intra prediction for a prediction unit, and specific information (e.g., intra prediction mode, motion vector, reference picture, etc.) according to each prediction method may be determined. At this time, the processing unit where prediction is performed and the processing unit where the prediction method and specific contents are determined may be different. For example, the prediction method and prediction mode may be determined in the prediction unit, and the prediction may be performed in the transformation unit. The residual value (residual block) between the generated prediction block and the original block may be input to the transformation unit (130). In addition, the prediction mode information, motion vector information, etc. used for prediction may be encoded together with the residual value in the entropy encoding unit (165) and transmitted to the decoder. When using a specific encoding mode, it is also possible to encode the original block as is and transmit it to the decoding unit without generating a prediction block through the prediction unit (120, 125).

[0069] The inter prediction unit (120) may predict a prediction unit based on information of at least one picture among the previous or subsequent pictures of the current picture, and in some cases, may predict a prediction unit based on information of a portion of an encoded region within the current picture. The inter prediction unit (120) may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

[0070] The reference picture interpolation unit can receive reference picture information from the memory (155) and generate pixel information less than an integer pixel from the reference picture. In the case of luminance pixels, a DCT-based 8-tap interpolation filter with different filter coefficients can be used to generate pixel information less than an integer pixel in units of 1 / 4 pixels. In the case of a chrominance signal, a DCT-based 4-tap interpolation filter with different filter coefficients can be used to generate pixel information less than an integer pixel in units of 1 / 8 pixels.

[0071] The motion prediction unit can perform motion prediction based on a reference picture interpolated by the reference picture interpolation unit. Various methods can be used to derive a motion vector, such as FBMA (Full search-based Block Matching Algorithm), TSS (Three Step Search), and NTS (New Three-Step Search Algorithm). The motion vector can have a motion vector value in units of 1 / 2 or 1 / 4 pixels based on the interpolated pixel. The motion prediction unit can predict the current prediction unit by using different motion prediction methods. Various methods can be used as motion prediction methods, such as the Skip method, the Merge method, the AMVP (Advanced Motion Vector Prediction) method, and the Intra Block Copy method.

[0072] The intra prediction unit (125) can generate a prediction unit based on reference pixel information surrounding the current block, which is pixel information within the current picture. If the surrounding block of the current prediction unit is a block on which inter prediction has been performed and the reference pixel is a pixel on which inter prediction has been performed, the reference pixel included in the block on which inter prediction has been performed can be replaced and used with reference pixel information of the surrounding block on which intra prediction has been performed. That is, if the reference pixel is not available, the unavailable reference pixel information can be replaced and used with at least one reference pixel among the available reference pixels.

[0073] In intra prediction, the prediction mode can have a directional prediction mode that uses reference pixel information according to the prediction direction, and a non-directional mode that does not use directional information when performing prediction. The mode for predicting luminance information and the mode for predicting chrominance information can be different, and the intra prediction mode information used to predict luminance information or the predicted luminance signal information can be utilized to predict chrominance information.

[0074] When performing intra prediction, if the size of the prediction unit and the size of the transformation unit are the same, intra prediction for the prediction unit can be performed based on the pixels located to the left of the prediction unit, the pixels located at the upper left, and the pixels located at the upper. However, when performing intra prediction, if the sizes of the prediction unit and the transformation unit are different, intra prediction can be performed using reference pixels based on the transformation unit. In addition, intra prediction using NxN division can be used only for the minimum coding unit.

[0075] The intra prediction method can generate a prediction block after applying an Adaptive Intra Smoothing (AIS) filter to a reference pixel according to a prediction mode. The type of AIS filter applied to the reference pixel may be different. In order to perform the intra prediction method, the intra prediction mode of the current prediction unit can be predicted from the intra prediction modes of prediction units existing around the current prediction unit. When the prediction mode of the current prediction unit is predicted using the mode information predicted from the surrounding prediction units, if the intra prediction modes of the current prediction unit and the surrounding prediction units are the same, information indicating that the prediction modes of the current prediction unit and the surrounding prediction units are the same can be transmitted using predetermined flag information, and if the prediction modes of the current prediction unit and the surrounding prediction units are different, entropy encoding can be performed to encode the prediction mode information of the current block.

[0076] Additionally, a residual block containing residual value information, which is the difference between the prediction unit that performed the prediction based on the prediction unit generated in the prediction unit (120, 125) and the original block of the prediction unit, can be generated. The generated residual block can be input to the transformation unit (130).

[0077] In the transformation unit (130), the residual block including the residual value information of the prediction unit generated through the original block and the prediction unit (120, 125) can be transformed using a transformation method such as DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), or KLT. Whether to apply DCT, DST, or KLT to transform the residual block can be determined based on the intra prediction mode information of the prediction unit used to generate the residual block.

[0078] The quantization unit (135) can quantize values ​​converted to the frequency domain by the transformation unit (130). The quantization coefficients can vary depending on the block or the importance of the image. The values ​​produced by the quantization unit (135) can be provided to the dequantization unit (140) and the reordering unit (160).

[0079] The rearrangement unit (160) can perform rearrangement of coefficient values ​​for quantized residual values.

[0080] The rearrangement unit (160) can change a two-dimensional block-shaped coefficient into a one-dimensional vector form through a coefficient scanning method. For example, the rearrangement unit (160) can change the two-dimensional block-shaped coefficient into a one-dimensional vector form by scanning from the DC coefficient to the coefficient of the high-frequency region using a zig-zag scan method. Depending on the size of the transformation unit and the intra prediction mode, a vertical scan that scans the two-dimensional block-shaped coefficient in the column direction or a horizontal scan that scans the two-dimensional block-shaped coefficient in the row direction may be used instead of the zig-zag scan. That is, depending on the size of the transformation unit and the intra prediction mode, it is possible to determine which scan method among the zig-zag scan, the vertical scan, and the horizontal scan is to be used.

[0081] The entropy encoding unit (165) can perform entropy encoding based on the values ​​produced by the rearrangement unit (160). Entropy encoding can use various encoding methods such as, for example, Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

[0082] The entropy encoding unit (165) can encode various information such as residual value coefficient information of an encoding unit, block type information, prediction mode information, division unit information, prediction unit information, transmission unit information, motion vector information, reference frame information, block interpolation information, and filtering information from the rearrangement unit (160) and the prediction unit (120, 125).

[0083] The entropy encoding unit (165) can entropy encode the coefficient values ​​of the encoding unit input from the rearrangement unit (160).

[0084] The inverse quantization unit (140) and the inverse transformation unit (145) inversely quantize the values ​​quantized in the quantization unit (135) and inversely transform the values ​​transformed in the transformation unit (130). The residual values ​​generated in the inverse quantization unit (140) and the inverse transformation unit (145) can be combined with the predicted prediction units predicted through the motion estimation unit, motion compensation unit, and intra prediction unit included in the prediction unit (120, 125) to generate a reconstructed block.

[0085] The filter unit (150) may include at least one of a deblocking filter, an offset correction unit, and an ALF (Adaptive Loop Filter).

[0086] A deblocking filter can remove block distortion caused by boundaries between blocks in a reconstructed picture. To determine whether to perform deblocking, a deblocking filter can be applied to the current block based on the pixels contained in several columns or rows within the block. When applying a deblocking filter to a block, a strong filter or a weak filter can be applied depending on the required deblocking filtering strength. Furthermore, when applying a deblocking filter, horizontal and vertical filtering can be processed in parallel when performing vertical and horizontal filtering.

[0087] The offset correction unit can correct the offset from the original image on a pixel-by-pixel basis for an image that has undergone deblocking. To perform offset correction for a specific picture, the pixels contained in the image can be divided into a certain number of regions, the regions to be offset can be determined, and the offset can be applied to those regions. Alternatively, the offset can be applied by considering the edge information of each pixel.

[0088] Adaptive Loop Filtering (ALF) can be performed based on the comparison of the filtered restored image with the original image. After dividing the pixels included in the image into predetermined groups, a filter to be applied to each group can be determined, and filtering can be performed differentially for each group. Information regarding whether to apply ALF can be transmitted by luminance signal for each coding unit (CU), and the shape and filter coefficients of the ALF filter to be applied can vary depending on each block. Furthermore, an ALF filter of the same form (fixed form) can be applied regardless of the characteristics of the target block.

[0089] The memory (155) can store a restored block or picture produced through the filter unit (150), and the stored restored block or picture can be provided to the prediction unit (120, 125) when performing inter prediction.

[0090]

[0091] FIG. 2 is a block diagram showing an image decoding device according to an embodiment of the present invention.

[0092] Referring to FIG. 2, the image decoder (200) may include an entropy decoding unit (210), a reordering unit (215), an inverse quantization unit (220), an inverse transformation unit (225), a prediction unit (230, 235), a filter unit (240), and a memory (245).

[0093] When a video bitstream is input to a video encoder, the input bitstream can be decoded in the opposite procedure to that of the video encoder.

[0094] The entropy decoding unit (210) can perform entropy decoding in a procedure opposite to that of the entropy encoding unit of the video encoder. For example, various methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) can be applied in response to the method performed in the video encoder.

[0095] The entropy decoding unit (210) can decode information related to intra prediction and inter prediction performed in the encoder.

[0096] The reordering unit (215) can perform reordering based on the method in which the bitstream entropy-decoded by the entropy decoding unit (210) is reordered by the encoding unit. The coefficients expressed in the form of a one-dimensional vector can be reordered by restoring them back to coefficients in the form of a two-dimensional block. The reordering unit (215) can perform reordering by receiving information related to the coefficient scanning performed by the encoding unit and performing reverse scanning based on the scanning order performed by the corresponding encoding unit.

[0097] The inverse quantization unit (220) can perform inverse quantization based on the quantization parameters provided by the encoder and the coefficient values ​​of the rearranged block.

[0098] The inverse transform unit (225) can perform inverse transform, i.e., inverse DCT, inverse DST, and inverse KLT, on the transforms performed by the transform unit, i.e., DCT, DST, and KLT, on the quantization result performed by the image encoder. The inverse transform can be performed based on the transmission unit determined by the image encoder. In the inverse transform unit (225) of the image decoder, a transform technique (e.g., DCT, DST, KLT) can be selectively performed according to a plurality of pieces of information, such as a prediction method, the size of the current block, and the prediction direction.

[0099] The prediction unit (230, 235) can generate a prediction block based on the prediction block generation related information provided by the entropy decoding unit (210) and the previously decoded block or picture information provided by the memory (245).

[0100] As described above, when performing intra prediction in the same manner as the operation in the image encoder, if the size of the prediction unit and the size of the transformation unit are the same, intra prediction for the prediction unit is performed based on the pixels on the left side of the prediction unit, the pixels on the upper left side, and the pixels on the upper side. However, when performing intra prediction, if the size of the prediction unit and the size of the transformation unit are different, intra prediction can be performed using reference pixels based on the transformation unit. In addition, intra prediction using NxN division only for the minimum coding unit can be used.

[0101] The prediction unit (230, 235) may include a prediction unit determination unit, an inter prediction unit, and an intra prediction unit. The prediction unit determination unit may receive various information such as prediction unit information input from the entropy decoding unit (210), prediction mode information of an intra prediction method, and motion prediction-related information of an inter prediction method, and may distinguish a prediction unit from a current encoding unit and determine whether the prediction unit performs inter prediction or intra prediction. The inter prediction unit (230) may perform inter prediction on the current prediction unit based on information included in at least one of a previous picture or a subsequent picture of the current picture including the current prediction unit, using information necessary for inter prediction of the current prediction unit provided from the image encoder. Alternatively, inter prediction may be performed based on information of a pre-restored portion of the current picture including the current prediction unit.

[0102] In order to perform inter prediction, it is possible to determine whether the motion prediction method of the prediction unit included in the encoding unit is Skip Mode, Merge Mode, AMVP Mode, or Intra Block Copy Mode based on the encoding unit.

[0103] The intra prediction unit (235) can generate a prediction block based on pixel information in the current picture. If the prediction unit is a prediction unit that has performed intra prediction, intra prediction can be performed based on intra prediction mode information of the prediction unit provided by the image encoder. The intra prediction unit (235) can include an AIS (Adaptive Intra Smoothing) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter is a part that performs filtering on the reference pixels of the current block, and can determine and apply whether to apply the filter according to the prediction mode of the current prediction unit. AIS filtering can be performed on the reference pixels of the current block using the prediction mode and AIS filter information of the prediction unit provided by the image encoder. If the prediction mode of the current block is a mode that does not perform AIS filtering, the AIS filter may not be applied.

[0104] The reference pixel interpolation unit can interpolate the reference pixel to generate a reference pixel of a pixel unit less than an integer value when the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on the pixel value interpolated from the reference pixel. When the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixel, the reference pixel may not be interpolated. The DC filter can generate a prediction block through filtering when the prediction mode of the current block is the DC mode.

[0105] The restored block or picture may be provided to a filter unit (240). The filter unit (240) may include a deblocking filter, an offset correction unit, and an ALF.

[0106] Information about whether a deblocking filter has been applied to a corresponding block or picture can be received from the video encoder, and if a deblocking filter has been applied, information about whether a strong or weak filter has been applied. The deblocking filter of the video decoder can receive information related to the deblocking filter provided by the video encoder, and the video decoder can perform deblocking filtering on the corresponding block.

[0107] The offset correction unit can perform offset correction on the restored image based on the type of offset correction applied to the image during encoding and offset value information.

[0108] ALF can be applied to a coding unit based on information such as whether ALF is applied and ALF coefficient information provided from the encoder. This ALF information can be provided by being included in a specific parameter set.

[0109] The memory (245) can store a restored picture or block so that it can be used as a reference picture or reference block, and can also provide the restored picture to an output unit.

[0110] As described above, in the following embodiments of the present invention, for convenience of explanation, the term coding unit is used as an encoding unit, but it may also be a unit that performs not only encoding but also decoding.

[0111] In addition, the current block represents a block to be encoded / decoded, and may represent a coding tree block (or coding tree unit), an encoding block (or encoding unit), a transformation block (or transformation unit), or a prediction block (or prediction unit), depending on the encoding / decoding step.

[0112]

[0113] A picture can be encoded / decoded by partitioning it into square or non-square basic blocks. At this time, the basic block can be called a coding tree unit. The coding tree unit can be defined as a coding unit with the largest size allowed in a sequence or slice. Information related to whether the coding tree unit is square or non-square or the size of the coding tree unit can be signaled through a sequence parameter set, a picture parameter set, or a slice header. The coding tree unit can be partitioned into partitions of smaller sizes. At this time, if a partition created by partitioning the coding tree unit is said to have depth 1, the partition created by partitioning the partition with depth 1 can be defined as having depth 2. In other words, a partition created by partitioning a partition with depth k within the coding tree unit can be defined as having depth k+1.

[0114] A partition of arbitrary size generated by splitting a coding tree unit can be defined as a coding unit. A coding unit can be split recursively, or split into basic units for performing prediction, quantization, transformation, or in-loop filtering. For example, a partition of arbitrary size generated by splitting a coding unit can be defined as a coding unit, or a transformation unit or prediction unit, which are basic units for performing prediction, quantization, transformation, or in-loop filtering.

[0115] Partitioning of a coding tree unit or coding unit may be performed based on at least one of a vertical line or a horizontal line. In addition, the number of vertical lines or horizontal lines partitioning the coding tree unit or coding unit may be at least one. For example, a coding tree unit or coding unit may be divided into two partitions using one vertical line or one horizontal line, or a coding tree unit or coding unit may be divided into three partitions using two vertical lines or two horizontal lines. Alternatively, a coding tree unit or coding unit may be divided into four partitions, each having a length and width of 1 / 2, using one vertical line and one horizontal line.

[0116] When a coding tree unit or a coding unit is divided into multiple partitions using at least one vertical line or at least one horizontal line, the partitions may have uniform sizes or different sizes. Alternatively, one partition may have a different size from the other partitions.

[0117] In the embodiments described below, it is assumed that the coding tree unit or coding unit is partitioned into a quad tree or binary tree structure. However, partitioning of the coding tree unit or coding unit using a larger number of vertical lines or a larger number of horizontal lines is also possible.

[0118] FIG. 3 illustrates an example of hierarchically dividing coding blocks based on a tree structure, as an embodiment of the present invention.

[0119] The input video signal is decoded in units of a predetermined block, and the basic unit for decoding the input video signal in this way is called a coding block. The coding block can be a unit that performs intra / inter prediction, transformation, and quantization. In addition, a prediction mode (e.g., an intra-screen prediction mode or an inter-screen prediction mode) is determined in units of the coding block, and the prediction blocks included in the coding block can share the determined prediction mode. The coding block can be a square or non-square block having any size in the range of 8x8 to 64x64, and can be a square or non-square block having a size of 128x128, 256x256, or more.

[0120] Specifically, the coding block can be hierarchically partitioned based on at least one of a quad tree and a binary tree. Here, the quad tree-based partitioning may mean a method in which a 2Nx2N coding block is partitioned into four NxN coding blocks, and the binary tree-based partitioning may mean a method in which one coding block is partitioned into two coding blocks. Even if the binary tree-based partitioning is performed, square coding blocks may exist in lower depths.

[0121] Binary tree-based partitioning can be performed symmetrically or asymmetrically. Coding blocks partitioned based on binary trees can be square blocks or non-square blocks such as rectangles. For example, partition shapes that allow binary tree-based partitioning can include at least one of symmetric 2NxN (horizontal non-square coding units) or Nx2N (vertical non-square coding units), and asymmetric nLx2N, nRx2N, 2NxnU, or 2NxnD, as in the example illustrated in FIG. 4.

[0122] Binary tree-based partitioning may restrictively allow only one of the symmetric or asymmetric partitioning types. In this case, configuring a coding tree unit with square blocks corresponds to quad-tree CU partitioning, while configuring a coding tree unit with symmetric non-square blocks corresponds to binary tree partitioning. Configuring a coding tree unit with square blocks and symmetric non-square blocks corresponds to quad and binary tree CU partitioning.

[0123] Binary tree-based partitioning can be performed on coding blocks for which quad tree-based partitioning is no longer performed. Quad tree-based partitioning may no longer be performed on coding blocks that have been partitioned based on binary trees.

[0124] In addition, the partitioning of a lower depth may be determined dependently on the partitioning form of a higher depth. For example, if binary tree-based partitioning is allowed in two or more depths, only binary tree-based partitioning of the same form as the binary tree partitioning form of the higher depth may be allowed in the lower depths. For example, if binary tree-based partitioning in the form of 2NxN is performed in the upper depth, binary tree-based partitioning in the form of 2NxN may also be performed in the lower depths. Alternatively, if binary tree-based partitioning in the form of Nx2N is performed in the upper depth, binary tree-based partitioning in the form of Nx2N may also be allowed in the lower depths.

[0125] Conversely, at lower depths, it is also possible to allow only binary tree-based splits that have a different form from the binary tree splits at higher depths.

[0126] For a sequence, slice, coding tree unit, or coding unit, only binary tree-based partitions of a specific form may be restricted to be used. For example, for a coding tree unit, only binary tree-based partitions of the form 2NxN or Nx2N may be restricted to be allowed. The allowed partition forms may be predefined in the encoder or decoder, or information regarding the allowed or disallowed partition forms may be encoded and signaled through the bitstream.

[0127] FIG. 5 is a diagram showing an example in which only a specific type of binary tree-based partitioning is allowed. FIG. 5 (a) shows an example in which only a binary tree-based partitioning of an Nx2N type is allowed, and FIG. 5 (b) shows an example in which only a binary tree-based partitioning of a 2NxN type is allowed. In order to implement the above quad-tree or binary tree-based adaptive partitioning, information indicating quad-tree-based partitioning, information regarding the size / depth of a coding block in which quad-tree-based partitioning is allowed, information indicating binary tree-based partitioning, information regarding the size / depth of a coding block in which binary tree-based partitioning is allowed, information regarding the size / depth of a coding block in which binary tree-based partitioning is not allowed, or information regarding whether binary tree-based partitioning is vertical or horizontal may be used.

[0128] Additionally, for a coding tree unit or a given coding unit, the number of times binary tree splitting is allowed, the depth at which binary tree splitting is allowed, or the number of depths at which binary tree splitting is allowed, etc. can be obtained. The above information can be encoded in units of coding tree units or coding units and transmitted to a decoder via a bitstream.

[0129] For example, the syntax 'max_binary_depth_idx_minus1', which indicates the maximum depth at which binary tree splitting is allowed, can be encoded / decoded through the bitstream. In this case, max_binary_depth_idx_minus1+1 can indicate the maximum depth at which binary tree splitting is allowed.

[0130] Referring to the example illustrated in FIG. 6, in FIG. 6, binary tree splitting is illustrated as being performed on a coding unit with a depth of 2 and a coding unit with a depth of 3. Accordingly, at least one of information indicating the number of times binary tree splitting is performed within the coding tree unit (2 times), information indicating the maximum depth (depth 3) at which binary tree splitting is allowed within the coding tree unit, or information indicating the number of depths (2, depth 2 and depth 3) at which binary tree splitting is allowed within the coding tree unit can be encoded / decoded through the bitstream.

[0131] As another example, at least one of the number of times binary tree splitting is allowed, the depth at which binary tree splitting is allowed, or the number of depths at which binary tree splitting is allowed may be obtained for each sequence or slice. As an example, the information may be encoded on a sequence, picture, or slice basis and transmitted via a bitstream. Accordingly, at least one of the number of binary tree splittings, the maximum depth at which binary tree splitting is allowed, or the number of depths at which binary tree splitting is allowed may be different for the first slice and the second slice. As an example, in the first slice, binary tree splitting may be allowed at only one depth, whereas in the second slice, binary tree splitting may be allowed at two depths.

[0132] As another example, at least one of the number of times binary tree splitting is allowed, the depth at which binary tree splitting is allowed, or the number of depths at which binary tree splitting is allowed may be set differently depending on the temporal identifier (TemporalID) of the slice or picture. Here, the temporal identifier (TemporalID) is for identifying each of a plurality of layers of an image having at least one or more scalability among view, spatial, temporal, or quality.

[0133] As illustrated in FIG. 3, a first coding block 300 having a split depth of k may be split into a plurality of second coding blocks based on a quad tree. For example, the second coding blocks 310 to 340 are square blocks having a size half the width and height of the first coding block, and the split depth of the second coding block may be increased to k+1.

[0134] A second coding block 310 having a split depth of k+1 may be split into a plurality of third coding blocks having a split depth of k+2. Splitting of the second coding block 310 may be performed selectively using either a quad tree or a binary tree depending on the splitting method. Here, the splitting method may be determined based on at least one of information indicating a quad tree-based splitting or information indicating a binary tree-based splitting.

[0135] When the second coding block 310 is split based on a quad tree, the second coding block 310 is split into four third coding blocks 310a each having a size equal to half the width and height of the second coding block, and the split depth of the third coding block 310a can be increased to k+2. On the other hand, when the second coding block 310 is split based on a binary tree, the second coding block 310 can be split into two third coding blocks. At this time, each of the two third coding blocks is a non-square block in which one of the width and height of the second coding block is half the size, and the split depth can be increased to k+2. The second coding block can be determined as a horizontal or vertical non-square block depending on the splitting direction, and the splitting direction can be determined based on information about whether the binary tree-based splitting is in the vertical or horizontal direction.

[0136] Meanwhile, the second coding block 310 may be determined as a terminal coding block that is no longer split based on a quad tree or binary tree, in which case the coding block may be used as a prediction block or a transformation block.

[0137] The third coding block 310a may be determined as a terminal coding block, similar to the division of the second coding block 310, or may be further divided based on a quad tree or binary tree.

[0138] Meanwhile, the third coding block 310b divided based on the binary tree may be further divided based on the binary tree into a vertical coding block (310b-2) or a horizontal coding block (310b-3), and the division depth of the corresponding coding block may be increased to k+3. Alternatively, the third coding block 310b may be determined as a terminal coding block (310b-1) that is no longer divided based on the binary tree, and in this case, the corresponding coding block (310b-1) may be used as a prediction block or a transform block. However, the above-described division process may be performed restrictively based on at least one of information about the size / depth of a coding block for which quad-tree-based division is allowed, information about the size / depth of a coding block for which binary tree-based division is allowed, or information about the size / depth of a coding block for which binary tree-based division is not allowed.

[0139] The size that a coding block can have may be limited to a certain number, or the size of a coding block within a certain unit may have a fixed value. For example, the size of a coding block within a sequence or a coding block within a picture may be limited to 256x256, 128x128, or 32x32. Information indicating the size of a coding block within a sequence or picture may be signaled through a sequence header or a picture header.

[0140] As a result of the partitioning based on quad trees and binary trees, the coding units can be square or rectangular in shape of arbitrary size.

[0141]

[0142] A coding block is encoded using at least one of skip mode, intra-picture prediction, inter-picture prediction, or skip method. Once a coding block is determined, a prediction block can be determined through prediction partitioning of the coding block. The prediction partitioning of the coding block can be performed by a partition mode (Part_mode) indicating the partitioning form of the coding block. The size or shape of the prediction block can be determined according to the partition mode of the coding block. For example, the size of the prediction block determined according to the partition mode can have a value equal to or smaller than the size of the coding block.

[0143] Figure 7 is a diagram illustrating a partition mode that can be applied to a coding block when the coding block is encoded with inter-screen prediction.

[0144] When a coding block is encoded with inter-screen prediction, any one of eight partition modes can be applied to the coding block, as in the example illustrated in FIG. 7.

[0145] If a coding block is encoded with intra-picture prediction, the coding block may have a partition mode PART_2Nx2N or PART_NxN applied.

[0146] PART_NxN can be applied when the coding block has a minimum size. Here, the minimum size of the coding block may be predefined in the encoder and decoder. Alternatively, information regarding the minimum size of the coding block may be signaled through the bitstream. For example, the minimum size of the coding block may be signaled through the slice header, and thus the minimum size of the coding block may be defined for each slice.

[0147] In general, the size of the prediction block can be from 64x64 to 4x4. However, if the coding block is encoded with inter-screen prediction, the prediction block may not have a size of 4x4 to reduce memory bandwidth when performing motion compensation.

[0148]

[0149] FIG. 8 illustrates an example of an intra prediction mode pre-defined in an image encoder / decoder to which the present invention is applied.

[0150] A video encoder / decoder can perform intra prediction using any one of the predefined intra prediction modes. The predefined intra prediction modes for intra prediction can be composed of a non-directional prediction mode (e.g., planar mode, DC mode) and 33 directional prediction modes.

[0151] Alternatively, a greater number of directional prediction modes than the 33 directional prediction modes can be used to improve the accuracy of intra prediction. That is, the angles of the directional prediction modes can be further refined to define M extended directional prediction modes (M>33), or a directional prediction mode with a predetermined angle can be derived and used by utilizing at least one of the 33 pre-defined directional prediction modes.

[0152] It is also possible to use more intra prediction modes than the 35 intra prediction modes illustrated in FIG. 8. For example, by further segmenting the angle of the directional prediction mode or by decoding a directional prediction mode having a predetermined angle using at least one of a predetermined number of predefined directional modes, it is possible to use more intra prediction modes than the 35 intra prediction modes. In this case, using more intra prediction modes than the 35 intra prediction modes can be referred to as an extended intra prediction mode.

[0153] Figure 9 illustrates an example of an extended intra prediction mode, which may be comprised of two non-directional prediction modes and 65 extended directional prediction modes. The extended intra prediction mode may be used equally for both luminance and chrominance components, or may use different numbers of intra prediction modes for each component. For example, 67 extended intra prediction modes may be used for the luminance component, and 35 intra prediction modes may be used for the chrominance component.

[0154] Alternatively, intra prediction can be performed using different numbers of intra prediction modes depending on the chrominance format. For example, in the 4:2:0 format, intra prediction can be performed using 67 intra prediction modes for the luminance component and 35 intra prediction modes for the chrominance component, and in the 4:4:4 format, intra prediction can be performed using 67 intra prediction modes for both the luminance component and the chrominance component.

[0155] Alternatively, intra prediction can be performed using different numbers of intra prediction modes depending on the size and / or shape of the block. That is, intra prediction can be performed using 35 intra prediction modes or 67 intra prediction modes depending on the size and / or shape of the PU or CU. For example, if the size of the CU or PU is smaller than 64x64 or is an asymmetric partition, intra prediction can be performed using 35 intra prediction modes, and if the size of the CU or PU is equal to or larger than 64x64, intra prediction can be performed using 67 intra prediction modes. Intra_2Nx2N may allow 65 directional intra prediction modes, and Intra_NxN may allow only 35 directional intra prediction modes.

[0156] The size of the block to which the extended intra prediction mode is applied may be set differently for each sequence, picture, or slice. For example, in the first slice, the extended intra prediction mode may be set to be applied to a block larger than 64x64 (e.g., CU or PU), and in the second slice, the extended intra prediction mode may be set to be applied to a block larger than 32x32. Information indicating the size of the block to which the extended intra prediction mode is applied may be signaled for each sequence, picture, or slice. For example, information indicating the size of the block to which the extended intra prediction mode is applied may be defined as 'log2_extended_intra_mode_size_minus4', which is obtained by taking the logarithm of the block size and then subtracting the integer 4. For example, a value of 0 for log2_extended_intra_mode_size_minus4 may indicate that the extended intra prediction mode can be applied to a block having a size of 16x16 or more or a block having a size greater than 16x16, and a value of 1 for log2_extended_intra_mode_size_minus4 may indicate that the extended intra prediction mode can be applied to a block having a size of 32x32 or more or a block having a size greater than 32x32.

[0157] As described above, the number of intra prediction modes can be determined by considering at least one of a chrominance component, a chrominance format, and a block size or shape. In addition to the described example, intra prediction mode candidates (e.g., the number of MPMs) used to determine an intra prediction mode of a block to be encoded / decoded may also be determined based on at least one of a chrominance component, a chrominance format, and a block size or shape. With reference to the drawings described below, a method for determining an intra prediction mode of a block to be encoded / decoded and a method for performing intra prediction using the determined intra prediction mode will be described.

[0158]

[0159] FIG. 10 is a flowchart schematically illustrating an intra prediction method as an embodiment to which the present invention is applied.

[0160] Referring to FIG. 10, the intra prediction mode of the current block can be determined (S1000).

[0161] Specifically, the intra prediction mode of the current block can be derived based on a candidate list and an index. Here, the candidate list includes multiple candidates, and the multiple candidates can be determined based on the intra prediction modes of neighboring blocks adjacent to the current block. The neighboring blocks can include at least one of blocks located above, below, to the left, to the right, or at a corner of the current block. The index can specify any one of the multiple candidates in the candidate list. The candidate specified by the index can be set as the intra prediction mode of the current block.

[0162] The intra prediction mode used by the surrounding block for intra prediction may be set as a candidate. Additionally, an intra prediction mode with a similar directionality to the intra prediction mode of the surrounding block may also be set as a candidate. Here, the intra prediction mode with a similar directionality may be determined by adding or subtracting a predetermined constant value from the intra prediction mode of the surrounding block. The predetermined constant value may be an integer of 1, 2, or a larger number.

[0163] The above candidate list may further include a default mode. The default mode may include at least one of a planar mode, a DC mode, a vertical mode, and a horizontal mode. The default mode may be adaptively added based on the maximum number of candidates that can be included in the candidate list of the current block.

[0164] The maximum number of candidates that can be included in the candidate list can be 3, 4, 5, 6, or more. The maximum number of candidates that can be included in the candidate list can be a fixed value preset in the video encoder / decoder, or can be variably determined based on the properties of the current block. The properties can mean the position / size / shape of the block, the number / type of intra prediction modes that the block can use, chrominance properties, chrominance format, etc. Alternatively, information indicating the maximum number of candidates that can be included in the candidate list can be separately signaled, and the maximum number of candidates that can be included in the candidate list can be variably determined using this. The information indicating the maximum number of candidates can be signaled at least one of the sequence level, the picture level, the slice level, and the block level.

[0165] When the extended intra prediction mode and the 35 predefined intra prediction modes are optionally used, the intra prediction modes of the surrounding blocks can be converted into indices corresponding to the extended intra prediction mode or into indices corresponding to the 35 intra prediction modes to derive candidates. A predefined table can be used to convert the indices, or a scaling operation based on a predetermined value can be used. Here, the predefined table can define a mapping relationship between different intra prediction mode groups (e.g., the extended intra prediction mode and the 35 intra prediction modes).

[0166] For example, if the left neighboring block uses 35 intra prediction modes and the intra prediction mode of the left neighboring block is 10 (horizontal mode), this can be converted to index 16 corresponding to the horizontal mode in the extended intra prediction mode.

[0167] Alternatively, if the upper surrounding block uses the extended intra prediction mode and the intra prediction mode index of the upper surrounding block is 50 (vertical mode), it can be converted to index 26 corresponding to the vertical mode in the 35 intra prediction modes.

[0168] Based on the intra prediction mode determination method described above, intra prediction modes may be derived independently for each of the luminance component and the chrominance component, or the chrominance component may be derived as a dependency on the intra prediction mode of the luminance component.

[0169] Specifically, the intra prediction mode of the chrominance component can be determined based on the intra prediction mode of the luminance component, as shown in Table 1 below.

[0170] Intra_chroma_pred_mode[xCb][yCb]IntraPredModeY[xCb][yCb]026101X(0<=X<=34)0340000126342626262101034101031113414026101X

[0171] In Table 1, intra_chroma_pred_mode represents information signaled to specify the intra prediction mode of the chrominance component, and IntraPredModeY represents the intra prediction mode of the luminance component.

[0172] Referring to FIG. 10, a reference sample for intra prediction of the current block can be derived (S1010).

[0173] Specifically, reference samples for intra prediction can be derived based on surrounding samples of the current block. The surrounding samples may refer to reconstructed samples of the above-described surrounding blocks, which may be reconstructed samples before the in-loop filter is applied or reconstructed samples after the in-loop filter is applied.

[0174] A surrounding sample restored before the current block may be used as a reference sample, or a surrounding sample filtered based on a predetermined intra filter may be used as a reference sample. Filtering a surrounding sample using an intra filter may be referred to as reference sample smoothing. The intra filter may include at least one of a first intra filter applied to a plurality of surrounding samples positioned on the same horizontal line or a second intra filter applied to a plurality of surrounding samples positioned on the same vertical line. Depending on the position of the surrounding sample, either the first intra filter or the second intra filter may be selectively applied, or two intra filters may be applied redundantly. At this time, the filter coefficient of at least one of the first intra filter or the second intra filter may be (1, 2, 1), but is not limited thereto.

[0175] The filtering may be adaptively performed based on at least one of the intra prediction mode of the current block or the size of the transform block for the current block. For example, if the intra prediction mode of the current block is DC mode, vertical mode, or horizontal mode, filtering may not be performed. If the size of the transform block is NxM, filtering may not be performed. Here, N and M may be the same or different values, and may be any of 4, 8, 16, or more. For example, if the size of the transform block is 4x4, filtering may not be performed. Alternatively, filtering may be selectively performed based on a comparison result between the difference between the intra prediction mode of the current block and the vertical mode (or horizontal mode) and a pre-defined threshold. For example, filtering may be performed only when the difference between the intra prediction mode of the current block and the vertical mode is greater than the threshold. The threshold may be defined for each size of the transform block, as shown in Table 2.

[0176] 8x8 transform16x16 transform32x32 transformThreshold710

[0177] The intra filter may be determined as one of a plurality of intra filter candidates predefined in the video encoder / decoder. For this purpose, a separate index specifying the intra filter of the current block among the plurality of intra filter candidates may be signaled. Alternatively, the intra filter may be determined based on at least one of the size / shape of the current block, the size / shape of the transform block, information about the filter strength, or the variation of surrounding samples.

[0178] Referring to FIG. 10, intra prediction can be performed using the intra prediction mode and reference sample of the current block (S1020).

[0179] That is, the prediction sample of the current block can be obtained using the intra prediction mode determined in S500 and the reference sample derived in S510. However, in the case of intra prediction, since the boundary samples of the surrounding blocks are used, the problem of deterioration in the image quality of the predicted image may occur. Therefore, a correction process for the prediction sample generated through the above-described prediction process may be further performed, which will be examined in detail below with reference to FIGS. 11 to 13. However, the correction process described below is not limited to being applied only to intra prediction samples, and can of course also be applied to inter prediction samples or restored samples.

[0180]

[0181] FIG. 11 illustrates an embodiment to which the present invention is applied, and illustrates a method for correcting a prediction sample of a current block based on differential information of surrounding samples.

[0182] A prediction sample of a current block can be corrected based on differential information of multiple surrounding samples for the current block. The correction may be performed on all prediction samples belonging to the current block, or may be performed only on prediction samples belonging to a predetermined partial region. The partial region may be one row / column or multiple rows / columns, which may be a region pre-configured for correction in a video encoder / decoder. For example, correction may be performed on one row / column located at the boundary of the current block, or on multiple rows / columns from the boundary of the current block. Alternatively, the partial region may be variably determined based on at least one of the size / shape or the intra prediction mode of the current block.

[0183] The surrounding samples may belong to at least one of the surrounding blocks located at the top, left, or upper-left corner of the current block. The number of surrounding samples used for correction may be two, three, four, or more. The positions of the surrounding samples may be determined variably depending on the positions of the prediction samples to be corrected within the current block. Alternatively, some of the surrounding samples may have fixed positions regardless of the positions of the prediction samples to be corrected, while the rest may have variable positions depending on the positions of the prediction samples to be corrected.

[0184] The difference information of surrounding samples may refer to a difference sample between surrounding samples, or may refer to a value obtained by scaling the difference sample by a predetermined constant value (e.g., 1, 2, 3, etc.). Here, the predetermined constant value may be determined by considering the position of the prediction sample to be corrected, the position of the column or row to which the prediction sample to be corrected belongs, the position of the prediction sample within the column or row, etc.

[0185] For example, if the intra prediction mode of the current block is vertical mode, the final prediction sample can be obtained as in the following mathematical expression 1 by using the difference sample between the surrounding sample p(-1,y) adjacent to the left boundary of the current block and the upper left surrounding sample p(-1,-1).

[0186]

[0187] For example, if the intra prediction mode of the current block is horizontal mode, the final prediction sample can be obtained as in the following mathematical expression 2 by using the difference sample between the surrounding sample p(x,-1) adjacent to the upper boundary of the current block and the upper left surrounding sample p(-1,-1).

[0188]

[0189] For example, if the intra prediction mode of the current block is vertical mode, the final prediction sample can be obtained by using the difference sample between the surrounding sample p(-1, y) adjacent to the left boundary of the current block and the surrounding sample p(-1, -1) at the upper left. At this time, the difference sample can be added to the prediction sample, or the difference sample can be scaled by a predetermined constant value and then added to the prediction sample. The predetermined constant value used for scaling can be determined differently according to the column and / or row. For example, the prediction sample can be corrected as in the following mathematical expressions 3 and 4.

[0190]

[0191]

[0192] For example, if the intra prediction mode of the current block is horizontal mode, the final prediction sample can be obtained by using the difference sample between the surrounding sample p(x,-1) adjacent to the upper boundary of the current block and the upper left surrounding sample p(-1,-1), as described above in the vertical mode. For example, the prediction sample can be corrected as in the following mathematical expressions 5 and 6.

[0193]

[0194]

[0195]

[0196] FIG. 12 and FIG. 13 illustrate an embodiment to which the present invention is applied, and illustrate a method for correcting a prediction sample based on a predetermined correction filter.

[0197] The prediction sample can be corrected based on surrounding samples of the prediction sample to be corrected and a predetermined correction filter. The surrounding samples can be identified by the angular line of the directional prediction mode of the current block, and can be one or more samples located on the same angular line as the prediction sample to be corrected. Furthermore, the surrounding samples can be prediction samples belonging to the current block, or restored samples belonging to a neighboring block restored prior to the current block.

[0198] The number of taps, strength, or at least one filter coefficient of the correction filter may be determined based on at least one of the position of the prediction sample to be corrected, whether the prediction sample to be corrected is located at the boundary of the current block, the intra prediction mode of the current block, the angle of the directional prediction mode, the prediction mode (inter or intra mode) of the surrounding block, or the size / shape of the current block.

[0199] Referring to Fig. 12, when the index is 2 or 34 in the directional prediction mode, the final prediction sample can be obtained by using at least one prediction / reconstruction sample located at the lower left of the prediction sample to be corrected and a predetermined correction filter, as shown in Fig. 12. Here, the prediction / reconstruction sample at the lower left may belong to a previous line of the line to which the prediction sample to be corrected belongs, and may belong to the same block as the current sample or may belong to a neighboring block adjacent to the current block.

[0200] Filtering of predicted samples can be performed only on lines located at block boundaries, or on multiple lines. A compensation filter with at least one different filter tap or filter coefficient may be used for each line. For example, a (1 / 2, 1 / 2) filter may be used for the first left line closest to the block boundary, a (12 / 16, 4 / 16) filter may be used for the second line, a (14 / 16, 2 / 16) filter may be used for the third line, and a (15 / 16, 1 / 16) filter may be used for the fourth line.

[0201] Alternatively, when the index is a value between 3 and 6 or between 30 and 33 in the directional prediction mode, filtering can be performed at the block boundary as shown in Fig. 13, and a 3-tap correction filter can be used to correct the prediction sample. Filtering can be performed using a 3-tap correction filter that inputs the lower left sample of the prediction sample to be corrected, the lower sample of the lower left sample, and the prediction sample to be corrected. The positions of the surrounding samples used in the correction filter can be determined differently based on the directional prediction mode. The filter coefficients of the correction filter can also be determined differently depending on the directional prediction mode.

[0202] Different correction filters can be applied depending on whether the surrounding blocks are encoded in inter or intra mode. If the surrounding blocks are encoded in intra mode, a filtering method that weights prediction samples more heavily can be used than if the surrounding blocks are encoded in inter mode. For example, if the intra prediction mode is 34, a (1 / 2, 1 / 2) filter can be used if the surrounding blocks are encoded in inter mode, and a (4 / 16, 12 / 16) filter can be used if the surrounding blocks are encoded in intra mode.

[0203] Depending on the size / shape of the current block (e.g., coding block, prediction block), the number of lines filtered within the current block may vary. For example, if the current block size is less than or equal to 32x32, filtering may be performed on only one line at the block boundary. Otherwise, filtering may be performed on multiple lines, including one line at the block boundary.

[0204] Figures 12 and 13 are explained based on the case of using the 35 intra prediction modes mentioned in Figure 7, but can be applied in the same / similar manner even when using the extended intra prediction mode.

[0205]

[0206] FIG. 14 is an example of an application of the present invention, showing the range of reference samples for intra prediction.

[0207] Intra prediction of a current block can be performed using reference samples derived from reconstructed samples included in surrounding blocks. Here, the reconstructed sample means a sample whose encoding / decoding has been completed before encoding / decoding of the current block. For example, intra prediction for the current block can be performed based on at least one of reference samples P(-1,-1), P(-1,y) (0<= y<= 2N-1), P(x,-1) (0<= x<= 2N-1) around the current block. At this time, filtering for the reference sample can be selectively performed based on at least one of an intra prediction mode of the current block (e.g., an index, directionality, angle, etc. of the intra prediction mode) or a size of a transform block for the current block.

[0208] Filtering of reference samples can be performed using predefined intra filters in the encoder and decoder. For example, the final reference sample to be used for intra prediction can be derived using an intra filter with filter coefficients of (1,2,1) or an intra filter with coefficients of (2,3,6,3,2).

[0209] Alternatively, at least one of a plurality of intra filter candidates may be selected to perform filtering on a reference sample. Here, the plurality of intra filter candidates may differ from each other in at least one of filter strength, filter coefficients, or number of taps (e.g., number of filter coefficients, filter length). The plurality of intra filter candidates may be defined at at least one of the sequence, picture, slice, and block levels. That is, the sequence, picture, slice, or block to which the current block belongs may use the same plurality of intra filter candidates.

[0210] Hereinafter, for convenience of explanation, it is assumed that the multiple intra filter candidates include a first intra filter and a second intra filter, and that the first intra filter is a (1,2,1) 3-tap filter and the second intra filter is a (2,3,6,3,2) 5-tap filter.

[0211] When filtering a reference sample by applying the above first intra filter, the filtered reference sample can be derived as in the following mathematical expression 7.

[0212]

[0213]

[0214]

[0215] When filtering a reference sample by applying the above second intra filter, the filtered reference sample can be derived as in the following mathematical expression 8.

[0216]

[0217]

[0218]

[0219] In the above mathematical expressions 7 and 8, x may be an integer between 0 and 2N-2, and y may be an integer between 0 and 2N-2.

[0220] Alternatively, one of multiple intra filter candidates may be selected based on the location of the reference sample, and filtering may be performed on the reference sample using the selected intra filter. For example, a first intra filter may be applied to a reference sample belonging to a first range, and a second intra filter may be applied to a reference sample belonging to a second range. Here, the first range and the second range may be distinguished based on whether they are adjacent to the boundary of the current block, whether they are located at the top or left of the current block, or whether they are adjacent to a corner of the current block. For example, as illustrated in FIG. 15, the first intra filter may be applied to reference samples P(-1,-1), P(-1,0), P(-1,1), ..., P(-1,N-1) and P(0,-1), P(1,-1), ..., P(N-1,-1) adjacent to the current block boundary to perform filtering as in Equation 7, and the second intra filter may be applied to other reference samples that are not adjacent to the current block boundary to perform filtering as in Equation 8. Based on the transformation type used in the current block, one of the multiple intra filter candidates may be selected and filtering may be performed on the reference sample using the selected intra filter. Here, the transformation type may mean (1) a transformation technique such as DCT, DST, or KLT, (2) a transformation mode indicator such as 2D transformation, 1D transformation, or no transformation, or (3) a number of transformations such as 1st transformation or 2nd transformation. Hereinafter, for convenience of explanation, it is assumed that the transformation type refers to a transformation technique such as DCT, DST, or KLT.

[0221] For example, if the current block is encoded using DCT, filtering may be performed using the first intra filter, and if the current block is encoded using DST, filtering may be performed using the second intra filter. Alternatively, if the current block is encoded using DCT or DST, filtering may be performed using the first intra filter, and if the current block is encoded using KLT, filtering may be performed using the second intra filter.

[0222] Filtering may also be performed using a filter selected based on the transform type of the current block and the positions of the reference samples described above. For example, if the current block is encoded using DCT, the reference samples P(-1,-1), P(-1,0), P(-1,1), ..., P(-1,N-1) and P(0,-1), P(1,-1), ..., P(N-1,-1) may be filtered using the first intra filter, and the other reference samples may be filtered using the second intra filter. If the current block is encoded using DST, the reference samples P(-1,-1), P(-1,0), P(-1,1), ..., P(-1,N-1) and P(0,-1), P(1,-1), ..., P(N-1,-1) may be filtered using the second intra filter, and the other reference samples may be filtered using the first intra filter.

[0223] One of a plurality of intra filter candidates can be selected based on whether the transformation type of a neighboring block including a reference sample is the same as the transformation type of the current block, and filtering can be performed using the selected intra filter candidate for the reference sample. For example, if the current block and the neighboring block use the same transformation type, filtering can be performed using a first intra filter, and if the current block and the neighboring block use different transformation types, filtering can be performed using a second intra filter.

[0224] Based on the transformation type of the neighboring block, one of the multiple intra filter candidates can be selected and filtering can be performed on the reference sample using the selected intra filter. That is, a predetermined filter can be selected by considering the transformation type of the block to which the reference sample belongs. For example, as illustrated in FIG. 16, if the blocks adjacent to the left / bottom left of the current block are blocks encoded using DCT, and the blocks adjacent to the upper / upper right are blocks encoded using DST, filtering can be performed by applying a first intra filter to the reference sample adjacent to the left / bottom left, and filtering can be performed by applying a second intra filter to the reference sample adjacent to the upper / upper right.

[0225] For a given region unit, available filters can be defined for that region. Here, the given region unit can be any one of a sequence, a picture, a slice, a block group (e.g., a coding tree unit row), a block (e.g., a coding tree unit), or a separate region that shares one or more filters can be defined. Reference samples can be filtered using a filter mapped to the region to which the current block belongs.

[0226] For example, as illustrated in FIG. 17, filtering of reference samples can be performed using different filters on a CTU basis. In this case, information indicating whether the corresponding sequence or picture uses the same filter, the type of filter used for each CTU, an index specifying the filter used by the corresponding CTU among available intra filter candidates, etc. can be signaled in the sequence parameter set (SPS) or picture parameter set (PPS).

[0227] The intra filter described above may also be applied on a coding unit basis. For example, filtering may be performed by applying a first intra filter or a second intra filter to reference samples surrounding a coding unit.

[0228]

[0229] Once the intra prediction mode of the current block is determined, intra prediction can be performed using reference samples adjacent to the current block. For example, the prediction sample of the current block can be generated by averaging the reference samples, or by copying the reference samples in a specific direction considering the directionality of the intra prediction mode. As in the example discussed above through FIG. 14, the reference sample of the current block can be derived using reference samples derived from reconstructed samples included in neighboring blocks around the current block (e.g., P(-1,-1), P(-1,y) (0<= y<= 2N-1), P(x,-1) (0<= x<= 2N-1)).

[0230] At this time, if it is determined that a sample included in a neighboring block adjacent to the current block is not available as a reference sample, the sample that cannot be used as a reference sample can be replaced with a sample that can be used as a reference sample. For example, if the location of a sample included in a neighboring block exists outside the picture, if a sample included in a neighboring block exists in a different slice from the current block, or if a sample included in a neighboring block is included in a block encoded with inter prediction, etc., the neighboring sample may be determined to be unavailable as a reference sample. At this time, whether a sample included in a block encoded with inter prediction is unavailable can be determined based on information indicating whether a sample included in a block encoded with inter prediction is to be used as a reference sample during intra prediction of the current block. Here, the information may be, but is not limited to, a 1-bit flag (e.g., 'constrained_intra_prediction_flag'). For example, if 'constrained_intra_prediction_flag' is 1, samples included in a block encoded with inter prediction may be determined as unavailable as reference samples. Hereinafter, samples that cannot be used as reference samples are referred to as unavailable reference samples, and samples that can be used as reference samples are referred to as available reference samples.

[0231] In the example illustrated in FIG. 14, if the sample located at the lower left (e.g., P(-1, 2N-1)) is determined to be unavailable, the sample located at the lower left can be replaced using the first available reference sample found by scanning the available samples in a predetermined order. Here, the scanning order can be sequentially performed starting from the sample adjacent to the lower left sample. For example, in the example illustrated in FIG. 14, if the sample P(-1, 2N-1) is unavailable, the scanning can be performed in the order of P(-1, 2N-2) to P(-1, -1), and then P(-1, 0) to P(2N-1, -1). P(-1, 2N-1) can be replaced with the first available reference sample found as a result of the scanning.

[0232] If any left peripheral sample, except the leftmost peripheral sample, is unavailable, the left peripheral sample can be replaced with a peripheral sample adjacent to the bottom of the left peripheral sample. For example, an unavailable reference sample P(-1, y) between P(-1, 2N-1) and P(-1, -1) can be replaced with the reference sample (P-1, y+1).

[0233] If the upper peripheral sample is unavailable, the upper peripheral sample can be replaced with the peripheral sample adjacent to the left of the upper peripheral sample. For example, an unavailable reference sample P(x,-1) between P(0,-1) and P(2N-1,-1) can be replaced with the reference sample P(x-1,-1).

[0234] A set of reference samples referred to in intra prediction of the current block may be referred to as a 'reference line' (or, 'intra reference line' or 'reference sample line'). Here, the reference line represents a set of reference samples consisting of one row and / or one column. For example, in the example illustrated in FIG. 14, the 'reference line' may represent a set of reference samples including P(-1,2N-1) to P(-1,-1) and P(0,-1) to P(2N-2,-1).

[0235] Intra prediction of the current block can be performed based on reference samples included in the reference line, depending on the intra prediction mode of the current block. For example, if the intra prediction mode of the current block is DC mode, a prediction signal can be generated using an average and / or weighted prediction of the reference samples included in the reference line. If the intra prediction mode of the current block is DC mode, the prediction sample of the current block can be obtained according to the following mathematical expression 9.

[0236]

[0237]

[0238]

[0239] In mathematical expression 9, dcVal can be generated based on the average value of the remaining reference samples excluding P(-1,-1) among the reference samples included in the reference line.

[0240] The planar mode exhibits effective prediction efficiency in a smooth area without a strong edge, and is effective in improving discontinuity of block boundaries or image quality degradation of block boundaries. When the intra prediction mode of the current block is the planar mode, the horizontal preliminary prediction sample of the current block may be obtained using a reference sample adjacent to the upper right corner of the current block and a reference sample having the same y-coordinate as the horizontal preliminary prediction sample, and the vertical preliminary prediction sample of the current block may be obtained using a reference sample adjacent to the lower left corner of the current block and a reference sample having the same x-coordinate as the vertical preliminary prediction sample. For example, the horizontal preliminary prediction sample and the vertical preliminary prediction sample of the current block may be obtained by the following mathematical expression 10.

[0241]

[0242]

[0243] The prediction sample of the current block can be generated by adding the horizontal preliminary prediction sample and the vertical preliminary prediction sample and then shifting them by a value determined based on the size of the current block. For example, the prediction sample of the current block can be obtained using the following mathematical expression (11).

[0244]

[0245] Intra prediction of the current block may be performed using at least one of multiple reference lines. The lengths of the multiple reference lines may be determined to be all or part of the same length, or may be set differently.

[0246] For example, assuming that the current block has a size of WxH, the kth reference line may include reference samples located in the same row as p(-k, -k) and p(-k, -k) (e.g., reference samples from p(-k+1,-k) to p(W+H+2(k-1),-k) or reference samples from p(-k+1, -k) to p(2W+2(k-1), -k)) and reference samples located in the same column as p(-k, -k) (e.g., reference samples from p(-k, -k+1) to p(-k,W+H+2(k-1)) or reference samples from p(-k, -k+1) to p(-k,2H+2(k-1))).

[0247] Fig. 18 is a diagram illustrating multiple reference sample lines. As in the example illustrated in Fig. 18, when the first reference line adjacent to the boundary of the current block is referred to as 'reference line 0', the kth reference line can be configured adjacent to the k-1th reference line.

[0248] Alternatively, unlike as shown in FIG. 18, it is also possible to configure all reference lines to have the same number of reference samples.

[0249] Intra prediction of the current block can be performed by selecting at least one of a plurality of reference lines and based on reference samples included in the selected reference line. In this way, performing intra prediction by selecting at least one of a plurality of reference line candidates can be referred to as an "extended reference intra prediction method using an extended reference sample" or an "extended intra prediction method." Furthermore, the plurality of reference lines can be referred to as "extended reference lines."

[0250] Whether to perform intra prediction based on an extended reference line can be determined based on information signaled through a bitstream. In this case, the information may be a 1-bit flag, but is not limited thereto. Information regarding whether to perform intra prediction based on an extended reference line can be signaled in units of coding tree units, encoding units, or prediction units, or can be signaled in units of sequences, pictures, or slices. That is, whether to perform intra prediction based on an extended reference line can be determined in units of sequences, pictures, slices, CTUs, CUs, or PUs.

[0251] Alternatively, whether to perform intra prediction based on extended reference lines may be determined based on at least one of the size, shape, depth, or intra prediction mode of the current block. That is, the number of reference line candidates available to the current block may be limited based on at least one of the size, shape, depth, or intra prediction mode of the current block.

[0252] For example, whether to perform intra prediction using an extended reference line may be determined depending on whether the current block is square or non-square. For example, if the current block is square, intra prediction of the current block may be performed based on an extended reference line, as in the example illustrated in (a) of FIG. 19. That is, if the current block is square, intra prediction may be performed by selecting at least one of a plurality of reference line candidates surrounding the current block. On the other hand, if the current block is non-square, intra prediction of the current block may be performed based on a single reference line candidate or a predefined reference line, instead of performing intra prediction based on an extended reference line, as in the example illustrated in (b) of FIG. 19. That is, in non-square blocks, the use of multiple reference line candidates may be restricted.

[0253] Contrary to the example illustrated in FIG. 19, if the current block is non-square, intra prediction may be performed based on the extended reference line, and if the current block is square, intra prediction may be performed without being based on the extended reference line.

[0254] When it is decided to perform intra prediction based on extended reference lines, the number of reference lines available to the current block can be determined. At this time, the number of reference lines may have a fixed value, or may be adaptively determined according to the size, shape, or intra prediction mode of the current block. For example, if the intra prediction mode of the current block is a non-directional mode, the number of reference lines available to the current block is determined to be 1, whereas if the intra prediction mode of the current block is a directional mode, the number of reference lines available to the current block may be determined to be plural.

[0255] As another example, the number of reference lines available to the current block may be determined by information signaled in units of a sequence, picture, slice, or decoding target unit. Here, the decoding target unit unit may mean a coding tree unit, an encoding unit, a transformation unit, or a prediction unit. As an example, a syntax element 'max_intra_line_idx_minus2' indicating the number of reference lines available in a sequence or slice may be signaled through a sequence header or a slice header. In this case, max_intra_line_idx_minus2 + 2 may be set as the number of available reference lines.

[0256] Below, we will describe in detail how to perform intra prediction based on the extended reference line.

[0257] FIG. 20 is a flowchart illustrating a method for performing intra prediction based on an extended reference line according to the present invention.

[0258] First, the decoder can generate multiple reference lines (S2010). The reference samples included in each reference line can be generated based on the restored samples included in blocks decrypted prior to the current block.

[0259] If the intra prediction mode of the current block is a directional mode, the decoder can generate reference lines by considering the directionality of the intra prediction mode. Considering the directionality of the intra prediction mode, the k-th reference line may contain a larger number of reference samples than the k-1-th reference line. That is, reference lines further from the current block may contain a larger number of reference samples than reference lines closer to the current block.

[0260] At this time, the number of reference samples additionally included in the k-th reference line compared to the k-1th reference line can be variably determined depending on the size, shape, or intra prediction mode of the current block.

[0261] For example, if the current block has a size of 4x4, the k-th reference line may include four more reference samples (specifically, two in the horizontal direction and two in the vertical direction) than the k-1-th reference line. On the other hand, if the current block has a size of 8x8, the k-th reference line may include eight more reference samples (specifically, four in the horizontal direction and four in the vertical direction) than the k-1-th reference line.

[0262] In Fig. 18, since the size of the current block is 4x4, the first reference sample is shown to contain a total of 9 reference samples, and the second reference sample is shown to contain a total of 13 reference samples (9+2x2).

[0263] If the current block is non-square, the number of reference samples included in the reference line can be determined according to the horizontal and vertical lengths of the current block.

[0264] As an example, Fig. 21 is a diagram illustrating multiple reference lines for a non-square block. Comparing Figs. 18 and 21, it is illustrated that as the width of the current block is reduced by half, the number of upper reference samples, excluding the upper left reference sample included in reference line 0, is reduced from 8 to 4.

[0265] That is, according to FIG. 18 and FIG. 21, assuming that the current block has a size of WxH, the kth reference line may include a total of 2{(W+H)+2(k-1)}+1 reference samples, including W+H+2(k-1) top reference samples (or 2W+2(k-1) top reference samples) (i.e., horizontal reference samples), W+H+2(k-1) left reference samples (or 2H+2(k-1) left reference samples) (i.e., vertical reference samples), and top left reference samples.

[0266] Reference samples included in a reference line can be derived based on reconstructed samples that were decoded / encoded before the current block. For example, reconstructed samples at the same position as the reference samples can be derived as reference samples for intra prediction of the current block. At this time, if the reconstructed samples include unavailable reference samples that cannot be used for intra prediction of the current block, the unavailable reference samples can be replaced with available reference samples in the vicinity. The available reference samples that replace the unavailable reference samples can be reconstructed samples included in the same reference line as the unavailable reference samples, or can be reconstructed samples included in a different reference line from the unavailable reference samples.

[0267] For example, when performing intra prediction based on an extended reference line, if the location of a sample within the reference line is outside the picture, or exists in a different slice from the current block, or if the reconstructed sample is included in a block encoded with inter prediction, the sample may be determined to be an unavailable sample that cannot be used as a reference sample of the current block. A reconstructed sample included in a block encoded with inter prediction may be determined to be an unavailable reference sample only when it is set not to use the reconstructed sample included in the block encoded with inter prediction as a reference sample (e.g., when constrained_intra_prediction_flag is 1).

[0268] Alternatively, if a block encoded with intra prediction is set to be decoded before a block encoded with inter prediction, the block encoded with inter prediction will not be reconstructed yet when the block encoded with intra prediction is decoded. Accordingly, if the decoding order of the block encoded with intra prediction precedes the block encoded with inter prediction, the samples included in the block encoded with inter prediction may be determined as unavailable reference samples.

[0269] The reference sample used to replace the unavailable reference sample may be determined by considering the location of the unavailable reference sample, the distance between the unavailable reference sample and the available reference sample, etc. For example, the unavailable reference sample may be replaced with the available reference sample located at the shortest distance from the unavailable reference sample. That is, the distance (first offset) between the available reference sample and the unavailable reference sample included in the same reference line as the unavailable reference sample, and the distance (second offset) between the available reference sample and the unavailable reference sample included in a different reference line from the reference line including the unavailable reference sample, may be compared to replace the unavailable reference sample with the available reference sample having the shorter distance from the unavailable reference sample. Alternatively, the unavailable reference sample may be replaced with an available reference sample located in a predetermined direction with respect to the unavailable reference sample. Here, the predetermined direction may be a direction predefined in the encoder / decoder (e.g., left, right, up, or down, etc.). The predefined direction may be set differently depending on the location of the unavailable reference sample.

[0270] In the example illustrated in FIG. 22, the distance between the unavailable reference sample included in reference line 0 and the available reference sample included in reference line 0 is illustrated as 4, and the distance between the unavailable reference sample included in reference line 0 and the available reference sample included in reference line 2 is illustrated as 2. Accordingly, the unavailable reference sample included in reference line 0 can be replaced by the available reference sample included in reference line 2.

[0271] If the first offset and the second offset are the same, the unavailable reference sample can be replaced using an available reference sample included in the same reference line as the unavailable reference sample.

[0272] If the distance between an available reference sample and an unavailable reference sample included in the same reference line as the unavailable reference sample (i.e., the first offset) is N or greater, the unavailable reference sample may be replaced using an available reference sample belonging to a different reference line from the unavailable reference sample. Alternatively, even if the first offset is N or greater, the unavailable reference sample may be replaced using an available reference sample belonging to a different reference line from the unavailable reference sample only if the second offset is smaller than the first offset. Here, N may represent an integer greater than or equal to 1.

[0273] If the first offset is not greater than N, the unavailable reference sample can be replaced using an available reference sample included in the same reference line as the unavailable reference sample.

[0274] Figures 23 and 24 illustrate examples of replacing unavailable reference samples with available reference samples when N is 2. As in the example illustrated in Figure 23, if the distance between the unavailable reference sample included in reference line 0 and the available reference sample included in reference line 0 is 2, the unavailable reference sample included in reference line 0 can be replaced using the available reference sample included in reference line 1.

[0275] On the other hand, as in the example illustrated in FIG. 24, if the distance between the unavailable reference sample included in reference line 0 and the available reference sample included in reference line 0 is 1, the unavailable reference sample included in reference line 0 can be replaced using the available reference sample included in reference line 0.

[0276] An unavailable reference sample may be replaced using an available reference sample included in the same reference line as the unavailable reference sample, or an available reference sample included in a reference line adjacent to the reference line including the unavailable reference sample. Here, the reference line adjacent to the reference line including the unavailable reference sample may mean a reference line having an index difference of 1 from the reference line including the unavailable reference sample. Alternatively, the unavailable reference sample may be replaced using an available reference sample included in a reference line having an index difference of 2 or more from the reference line including the unavailable reference sample.

[0277] Alternatively, the unavailable reference sample may be replaced by an available reference sample included in a reference line having an index value greater than that of the reference line containing the unavailable reference sample, or an available reference sample included in a reference line having an index value less than that of the reference line containing the unavailable reference sample. For example, when a reference line having an index value greater than that of the reference line containing the unavailable reference sample is used, the unavailable reference sample may be replaced by a reference sample located to the left or above the unavailable reference sample.

[0278] The search for available reference samples to replace unavailable reference samples may be performed in a predefined direction. For example, only reference samples located in the same reference line as the unavailable reference sample and located in any one of the directions above, below, left, or right of the unavailable sample may be used to replace the unavailable sample. Alternatively, only reference samples located in a different reference line from the unavailable reference sample and located in any one of the directions above, below, left, or right of the unavailable sample may be used to replace the unavailable sample.

[0279] According to one embodiment of the present invention, a reference sample, reference line, or reference line candidate may be derived based on a block of a predetermined size or shape. A detailed description thereof will be provided later with reference to FIGS. 29 and 30.

[0280] The decoder can decode index information that specifies at least one of a plurality of reference lines from the bitstream (S2020). For example, as in the example illustrated in FIG. 18, if four reference lines are selectable candidates, the index information can specify at least one of the four reference lines.

[0281] The reference line for performing intra prediction on the current block may be adaptively determined based on the size of the current block, the shape of the current block, the intra prediction mode of the current block, index information in neighboring blocks, or the difference between the intra prediction mode of the current block and a given intra prediction mode.

[0282] Additionally, the number of reference lines selected for in-screen prediction of the current block may have a fixed value, or may be adaptively determined depending on the size, shape, or intra-prediction mode of the current block.

[0283] When at least one of the plurality of reference lines is determined, the decoder can perform intra prediction on the current block using the determined reference line (S2030). At this time, the position and / or range of the reference sample used for intra prediction of the current block in the selected reference line can be derived based on at least one of the intra prediction mode type or intra prediction mode directionality of the current block.

[0284] For example, if the intra prediction mode of the current block is DC mode, the prediction sample of the current block can be generated based on the average value (dcVal) of all or some of the reference samples included in the determined reference line. Referring to FIGS. 25 and 26, the calculation of the average value of the reference samples included in the reference line will be described in detail.

[0285] Alternatively, if the intra prediction mode of the current block is a directional mode, the prediction sample of the current block may be generated based on a reference sample specified by the directional mode among the reference samples included in the determined reference line. At this time, if a line segment extended from the prediction sample in the direction indicated by the directional mode points between the reference samples, the prediction sample of the current block may be generated based on a weighted sum (weighted prediction) of a first reference sample and a second reference sample located on both sides of a point indicated by the line segment extended in the direction indicated by the directional mode.

[0286]

[0287] When the intra prediction mode of the current block is DC mode, the average (dcVal) of the reference samples belonging to the reference line needs to be calculated to perform prediction for the current block. At this time, the average value of the reference samples of the kth reference line can be calculated using only some of the reference samples belonging to the kth reference line. At this time, the number of reference samples used to derive the average may be the same for each reference line or may vary for each reference line.

[0288] Alternatively, the average of the reference samples of the kth reference line may be derived using all of the reference samples belonging to the kth reference line. Alternatively, depending on the size and shape of the current block or the position of the reference line, it may be determined whether to derive the average using only some of the reference samples of the kth reference line or all of the reference samples of the kth reference line.

[0289] Figure 25 is a diagram illustrating reference samples used to derive the average value of the reference line.

[0290] The example illustrated in Fig. 25 illustrates an example of deriving the reference sample mean of the kth reference line by using some of the reference samples belonging to the reference line. For example, in the example illustrated in Fig. 25, the reference sample mean of the first reference line (reference line 0 illustrated in Fig. 25) adjacent to the current block can be calculated by using the upper reference sample and the left reference sample, excluding the reference sample adjacent to the upper left corner of the current block. That is, when the size of the current block is NxN, a total of 4N reference samples, including 2N upper reference samples and 2N left reference samples, can be used to calculate the mean of the first reference line.

[0291] The number of reference samples used to calculate the reference sample average of the kth reference line may have the same value as the number of reference samples used to calculate the reference sample average of the first reference line. In this case, the position of the reference sample used to calculate the average of the kth reference line may correspond to the position of the reference sample used to calculate the reference sample average of the first reference line.

[0292] The reference sample of the kth reference line corresponding to the reference sample of the first reference line may have the same x-coordinate or the same y-coordinate as the reference sample of the first reference line. For example, the upper reference sample included in the kth reference line corresponding to the upper reference sample P(i,j) included in the first reference line may be P(i,j-k+1) having the same x-coordinate as P(i,j). For example, the left reference sample of the kth reference line corresponding to the left reference sample P(i,j) included in the first reference line may be P(i-k+1,j) having the same y-coordinate as P(i,j).

[0293] Figure 25 illustrates reference samples of the second to fourth reference lines corresponding to the upper reference sample and the left reference sample of the first reference line. The average value of the reference samples of each reference line can be calculated using the reference samples illustrated in Figure 25.

[0294] In Fig. 25, it is assumed that the current block is square, but even if the current block is non-square, the above embodiment can be applied as is. For example, if the current block is a non-square block having a size of WxH, the reference sample average value of each reference line can be calculated using a total of 2 (W+H), including 2W upper reference samples and 2H left reference samples. Accordingly, as in the example illustrated in Fig. 26, the number of reference samples used to calculate the average value of the kth reference line can have the same value as the number of reference samples used to calculate the average value of the first reference line. In addition, the position of the reference sample used to calculate the average value of the kth reference line can correspond to the position of the reference sample used to calculate the reference sample average value of the first reference line.

[0295] In FIGS. 25 and 26, it is described that the upper reference samples, which are twice the current block width, and the left reference samples, which are twice the current block height, are used to calculate the reference sample average value of the reference line. Without being limited to the described example, the reference sample average value of the reference line may be calculated using a smaller or larger number of reference samples than those illustrated in FIGS. 25 and 26. For example, the reference sample average value of the reference line may be calculated using the same number of upper reference samples as the current block width, and the same number of left reference samples as the current block height.

[0296] The reference sample mean of the reference line can be calculated by assigning different weights to each reference sample depending on the shape of the current block and the position of the reference sample. For example, if the current block is square, the reference sample mean can be calculated by assigning equal weights to the top reference sample and the left reference sample. On the other hand, if the current block is non-square, the reference sample mean can be calculated by assigning a higher weight to either the top reference sample or the left reference sample than the other. For example, if the height of the current block is greater than the width, the mean can be calculated by assigning a greater weight to the top reference sample than the left reference sample. On the other hand, if the width of the current block is greater than the height, the mean can be calculated by assigning a greater weight to the left reference sample than the top reference sample.

[0297] For example, if the size of the current block is N / 2xN, the average value dcVal of the kth reference line can be calculated by the following mathematical expression 12.

[0298]

[0299] For example, if the size of the current block is NxN / 2, the average value dcVal of the kth reference line can be calculated by the following mathematical expression 13.

[0300]

[0301] In the above mathematical expressions 12 and 13, k can be set to a value between 1 and max_intra_line_idx_minus2+2.

[0302] As in the example described above, the average value can be calculated using reference samples that correspond to the reference samples included in the first intra reference line among the reference samples of the k-th intra reference line. In this case, the average value can also be calculated based on a predetermined weight. Here, the weight can be derived based on the distance of the first and k-th intra reference lines from the current block.

[0303] In the embodiment described through FIG. 20, it is exemplified that multiple reference lines are generated, and then index information specifying one of the multiple reference lines is decoded. Instead of deriving multiple reference lines, it is also possible to decode index information specifying one of the multiple reference lines, and then derive only the reference line specified by the index information.

[0304] In the embodiment described with reference to FIG. 20, it has been described that intra prediction for the current block is performed using at least one reference line specified by index information among a plurality of reference lines. That is, intra prediction for the current block can be performed using one reference line or two or more reference lines. When performing intra prediction for the current block, whether to use two or more reference lines can be determined based on information signaled from the bitstream, the size of the current block, the shape of the current block, the intra prediction mode of the current block, whether the intra prediction mode of the current block is non-directional, or the difference between the intra prediction mode of the current block and a predetermined intra prediction mode.

[0305] Two or more reference lines may be specified by multiple index information signaled from the bitstream. For example, if two reference lines are configured to be used, one of the two reference lines may be specified by the first index information, and the other may be specified by the second index information.

[0306] Alternatively, two or more reference lines may be located spatially contiguously. In this case, index information for specifying one of the two or more reference lines may be signaled through the bitstream. When one of the two or more reference lines is selected based on the index information, the remaining reference lines may be automatically selected based on spatial proximity to the selected reference line. For example, when two reference lines are configured to be used, if the index information indicates 'reference line 0', intra prediction of the current block may be performed based on reference line 0 and reference line 1 adjacent to reference line 0.

[0307] When multiple reference lines are set to be used, intra prediction of the current block can be performed based on the average value, maximum value, minimum value, or weighted sum of reference samples included in the multiple reference lines.

[0308] For example, assuming that the intra prediction mode of the current block is a directional mode (i.e., angular mode), the prediction sample of the current block can be generated based on a first reference sample and a second reference sample included in different reference lines, respectively. Here, the first reference line including the first reference sample and the second reference line including the second reference sample may be positioned adjacent to each other, but the present invention is not limited thereto. In addition, the positions of the first reference sample and the second reference sample may be determined by the intra prediction mode of the current block. Alternatively, the first reference sample and the second reference sample may be positioned adjacent to each other. The prediction sample of the current block may be generated by considering a weighted sum of the first reference sample and the second reference sample, or may be generated based on an average value, a minimum value, or a maximum value of the first reference sample and the second reference sample.

[0309] Figure 27 is a diagram showing an example of obtaining a prediction sample using multiple reference lines.

[0310] The prediction sample of the current block can be derived based on sample values ​​generated using multiple reference lines. For example, the prediction sample of the current block can be generated based on at least one of a minimum value, a maximum value, an average value, or a weighted average value of reference samples belonging to different reference lines. In the illustrated example of Fig. 27, assuming that intra prediction of the current block is performed using reference line 0 and reference line 1, the prediction sample of the current block can be obtained based on a minimum value, a maximum value, an average value, or a weighted sum between reference samples belonging to reference line 0 and reference samples belonging to reference line 1.

[0311] At this time, the number of reference samples belonging to different reference lines used to predict the decryption target sample within the current block may be the same or different. For example, in the example illustrated in FIG. 27, a prediction value for the decryption target sample included in the current block can be obtained using N samples belonging to reference line 0 and M reference samples (M is a natural number that is the same as or different from N) belonging to reference line 1.

[0312] When predicting a target sample to be decoded based on a weighted sum of reference samples belonging to different reference lines, the weight applied to each reference sample may be determined based on at least one of an intra prediction mode of a current block (e.g., a mode value, directionality, or angle of the intra prediction mode), a distance between the target sample to be decoded and the reference samples, or a position of the reference line to which the reference samples belong. Alternatively, the weight applied to the reference samples belonging to different reference lines may be a fixed value predefined according to the intra prediction mode (e.g., a mode value, directionality, or angle of the intra prediction mode).

[0313] In addition, the positions of the reference samples used in the weighted sum operation can be determined according to the intra prediction mode of the current block (e.g., the mode value, directionality, or angle of the intra prediction mode, etc.). For example, if the intra prediction mode of the current block points to the upper right diagonal direction, the decoding target sample located at (0,0) in the current block can be derived based on the weighted sum operation of the reference sample at position (1,-1) belonging to reference line 0 and the reference sample at position (2,-2) belonging to reference line 1.

[0314] Alternatively, the prediction sample of the current block may be generated based on a first intra prediction using at least one of a plurality of reference lines and a second intra prediction using at least one of the plurality of reference lines. Here, the reference line used for the first intra prediction and the reference line used for the second intra prediction may be different. For example, if the first intra prediction is performed using some of the plurality of reference lines, the second intra prediction may be performed using the remaining reference lines. In addition, the intra prediction mode for the first intra prediction and the intra prediction mode for the second intra prediction may be the same or different.

[0315] The prediction sample of the current block may be generated based on a first prediction sample generated by performing the first intra prediction and a second prediction sample generated by performing the second intra prediction. For example, the prediction sample may be generated based on a minimum value, maximum value, average value, or weighted sum of the first prediction sample and the second prediction sample.

[0316]

[0317] By selecting multiple reference lines, intra prediction is performed on the current block. At this time, different reference lines can be selected for each predetermined unit within the current block. For example, reference lines can be determined on a sample-by-sample basis, a row / column basis, or a predetermined region basis within the block. Accordingly, even if two decoding target samples are included in the same block, they can be predicted based on different reference lines.

[0318] Specifically, the reference line can be determined based on the position information of the sample to be decoded. Here, the position information of the sample to be decoded can include the x-axis position of the sample to be decoded, the y-axis position of the sample to be decoded, the row to which the sample to be decoded belongs, the column to which the sample to be decoded belongs, whether the sample to be decoded is located at a block boundary (e.g., a left / right boundary or an upper / lower boundary of a block), whether the sample to be decoded is located at a corner of the current block, or whether the sample to be decoded belongs to a predetermined area within the current block.

[0319] Not only the location of the reference line, but also the number and length of the reference lines can be determined differently based on the location information of the sample to be decoded. For example, even if two samples to be decoded are included in the same block, each can be predicted based on a different number of reference lines.

[0320] Figure 28 is a drawing for explaining an example of determining a reference line based on the location information of a sample to be decrypted.

[0321] Different reference lines may be used between the decoding target samples adjacent to a predetermined boundary of the current block and the remaining decoding target samples excluding the decoding target samples. For example, in the example illustrated in FIG. 28, the decoding target samples included in the first row of the current block may be predicted based on the second reference line (reference line 1 in FIG. 28), while the decoding target samples included in the second and third rows may be predicted based on the first reference line (reference line 0 in FIG. 28).

[0322] At this time, the predetermined boundary may be determined based on the intra prediction mode of the current block. For example, if the intra prediction mode of the current block has a vertical direction or a direction similar to the vertical direction, the predetermined boundary may be the top boundary (or the left, right, or bottom boundary) of the current block. On the other hand, if the intra prediction mode of the current block has a horizontal direction or a direction similar to the horizontal direction, the predetermined boundary may be the left boundary (or the top, bottom, or right boundary) of the current block. Here, the intra prediction mode having a vertical direction or a direction similar to the horizontal direction may mean that the mode value difference with the intra prediction mode in the vertical direction or the horizontal direction is within a predetermined range.

[0323] In addition to the example shown in Fig. 28, different reference lines can be used in units of samples, rows, columns, or predetermined areas.

[0324] It is also possible to encode and transmit information for selecting a reference line for each unit within the current block. For example, information identifying one of multiple reference lines can be encoded for each sample, row, column, or area.

[0325]

[0326] As in the examples illustrated in FIGS. 27 and 28, intra prediction for a current block may be performed based on a plurality of reference lines. At this time, as in the example illustrated in FIG. 27, reference lines may be determined on a block-by-block basis, or as in the example illustrated in FIG. 28, reference lines may be determined in units smaller than a block. Whether to determine reference lines on a block-by-block basis or in units smaller than a block may be determined based on information signaled through a bitstream. Here, the information may be a 1-bit flag, but the present invention is not limited thereto. Alternatively, the unit for determining reference lines may be adaptively determined based on at least one of the size, shape, or intra prediction mode of the current block.

[0327]

[0328] According to one embodiment of the present invention, a reference line or reference sample may be derived based on a block of a predetermined shape or a block of a predetermined size or larger. Here, the predetermined shape may refer to a square block or a non-square block with a width / height ratio of a predetermined value or larger. If the current block is not of the predetermined shape or larger than the predetermined size, intra prediction for the current block may be performed using a reference line or reference sample derived based on a block of the predetermined shape or a block of the predetermined size or larger. Accordingly, a plurality of blocks (e.g., a plurality of coding blocks, a plurality of transform blocks, or a plurality of prediction blocks) included in a block of the predetermined shape or a block of the predetermined size or larger may share a reference line or reference sample derived based on a block of the predetermined shape or a block of the predetermined size or larger.

[0329] Figure 29 is a diagram illustrating an example of performing intra prediction using reference samples derived from a block of a predetermined shape. For convenience of explanation, it is assumed that a single reference line is derived from a square block.

[0330] A non-square block can be predicted using reference samples derived from a square block that includes the non-square block. For example, if a coded block is a non-square block whose width or height is half that of a square block, intra prediction of the coded block can be performed based on reference samples from the square block.

[0331] When a reference line or reference sample is derived based on a block of a predetermined shape or a predetermined size or larger, blocks included in the block of the predetermined shape or a predetermined size or larger can share the reference line or reference sample. For example, in FIG. 29, non-square coding block 0 and non-square coding block 1 can share a reference sample derived based on a square block.

[0332] According to one embodiment of the present invention, a plurality of reference lines may be derived based on a block of a predetermined shape or a block of a predetermined size or larger. Accordingly, a plurality of blocks included in a block of a predetermined shape or a block of a predetermined size or larger may share a plurality of reference line candidates derived based on the block of a predetermined shape or a block of a predetermined size or larger.

[0333] Figure 30 is a diagram showing an example of performing intra prediction using a plurality of reference lines derived based on a block of a predetermined shape.

[0334] A non-square block can be predicted using a reference sample included in at least one of a plurality of reference lines derived based on a square block including the non-square block. For example, if a coded block is a non-square block whose horizontal or vertical length is half that of a square block, intra prediction of the coded block can be performed based on at least one of a plurality of reference lines derived based on the square block.

[0335] When multiple reference lines are derived based on blocks of a predetermined shape or a predetermined size or larger, blocks included in the blocks of the predetermined shape or a predetermined size or larger can share multiple reference line candidates. For example, in FIG. 30, multiple reference lines derived based on square blocks can be set as reference line candidates for non-square coding block 0 and reference line candidates for non-square coding block 1.

[0336]

[0337] A plurality of blocks included in a block of a predetermined shape or a block larger than a predetermined size can individually determine reference lines. For example, in the example illustrated in FIG. 30, coding block 0 can select at least one of a plurality of reference lines based on index information encoded for coding block 0, and coding block 1 can select at least one of a plurality of reference lines based on index information encoded for coding block 1.

[0338] Alternatively, multiple blocks included in a block of a predetermined shape or a block larger than a predetermined size may be restricted to use the same reference line. For example, in the example illustrated in FIG. 30, if coding block 0 performed intra prediction based on a reference sample included in reference line 0, coding block 1 may also perform intra prediction based on a reference sample included in reference line 0.

[0339] Whether each block is allowed to individually select a reference line or whether each block is restricted from individually selecting a reference line can be determined according to the shape or size of each of the plurality of blocks included in a block having a predetermined shape or a predetermined size or larger. For example, when performing intra prediction on a non-square coding block having a shape of 2NxN or Nx2N, if 2N (i.e., width or height) is greater than a threshold k, individual selection of a reference line for the block can be permitted. On the other hand, if 2N is less than the threshold k, individual selection of a reference line for the block can be restricted.

[0340] According to one embodiment of the present invention, a block unit sharing the same reference sample, the same reference line, or the same reference line candidate may be defined. In this case, the block unit may have a size and / or shape preset in the encoder / decoder. Alternatively, information defining the block unit may be encoded in the encoder and signaled to the decoder. The information may indicate the width / height, split depth, or size of the block unit. A plurality of blocks included in the block unit may share the same reference sample, the same reference line, or the same reference line candidate. In this case, the reference sample or at least one reference line may be derived based on the block unit, or may be derived based on at least one of the plurality of blocks included in the block unit.

[0341]

[0342] Reference sample smoothing may be performed on reference samples included in a reference line. That is, reference samples included in a reference line may be filtered using at least one of a plurality of filters. Intra prediction of the current block may be performed based on reference samples before filtering or filtered reference samples.

[0343] Reference sample smoothing may be performed on all of the multiple reference lines, or on a subset of the multiple reference lines. For example, reference sample smoothing may be performed on a reference line among the multiple reference lines used for intra prediction of the current block, e.g., a reference line specified by index information.

[0344] Whether to perform reference sample smoothing may be adaptively determined based on at least one of the size of the current block, the type of intra prediction mode, the directionality of the intra prediction mode, the position of the reference line, the number of reference lines, or the variation between reference lines. The variation between reference lines may mean a difference value between reference samples included in different reference lines. For example, when the size of the prediction block for the current block is 4x4 or the intra prediction mode of the current block is DC mode, reference sample smoothing may be omitted.

[0345] In addition, the number of times to apply a filter or the type of a filter may be adaptively determined depending on at least one of the size of the current block, the type of the intra prediction mode, the directionality of the intra prediction mode, the position of the reference line, the number of reference lines, or the amount of change between the reference lines. For example, if the size of the prediction block for the current block is 32x32 or less and the intra prediction mode of the current block is not a horizontal direction, a direction similar to the horizontal direction, a vertical direction, or a direction similar to the vertical direction, reference filter smoothing may be performed using the first intra filter. Conversely, if the size of the prediction block for the current block is 32x32 or more and the change between reference samples is less than a predetermined value (i.e., the change in the values ​​of the reference samples is not large), reference filter smoothing may be performed using the second intra filter.

[0346] When filtering reference samples included in a reference line using the first intra filter, the filtered reference samples can be derived as in mathematical expression 14.

[0347]

[0348]

[0349]

[0350] When filtering reference samples included in a reference line using a second intra filter, the filtered reference samples can be derived as in mathematical expression 15.

[0351]

[0352]

[0353] In the above mathematical expressions 14 and 15, x can have a value from 0 to 2N-2(k-1), and y can have a value from 0 to 2N-2(k-1). Here, k represents the index of the reference line, and N can represent the size of the current block.

[0354] Alternatively, the type of intra-filter or the number of times the intra-filter is applied may be variably determined depending on the reference line. For example, different intra-filters may be applied depending on the index or position of the reference line, or different intra-filters may be applied depending on the group to which the reference line belongs. Here, different filters may mean filters that differ in at least one of the filter length, filter coefficient, or filter strength.

[0355] For example, a plurality of reference lines may be classified into at least one group. Each group may include at least one reference line. The grouping may be performed based on at least one of the indices of the reference lines, the number of reference lines, or the proximity between the reference lines. For example, a plurality of reference lines may be classified into at least one group based on factors such as whether the indices of the reference lines are even or whether the indices of the reference lines are greater than or equal to a predetermined value.

[0356] For example, if the current block uses at least one of the first reference line or the second reference line, intra-reference smoothing may be performed using the first intra filter. On the other hand, if the current block uses at least one of the third reference line or the fourth reference line, intra-reference smoothing may be performed using the second intra filter. Here, the first intra filter and the second intra filter may differ in at least one of a filter coefficient, a filter tap, or a filter strength. For example, the filter coefficient of the first intra filter may be (1,2,1), and the filter coefficient of the second intra filter may be (2,12,2).

[0357] A filter type to be applied to a reference line may be determined based on at least one of the size of a current block, the type of intra prediction mode, the directionality of the intra prediction mode, the position of a reference line, the number of reference lines, or the amount of variation between reference lines, and any one of the filters belonging to the determined filter type may be applied to the reference line based on at least one of the above-listed elements. Here, the filter types may be distinguished based on at least one of a filter length (number of taps), a filter coefficient, or a filter strength. For example, first type filters may have the same filter length, but may have a different filter length from second type filters.

[0358] For example, if the size of the prediction block for the current block is 32x32 or less, and the intra prediction mode of the current block is not a horizontal direction, a direction similar to the horizontal direction, a vertical direction, or a direction similar to the vertical direction, it may be determined to use the first intra filter type. In addition, if the current block uses at least one of the first reference line or the second reference line, reference sample smoothing may be performed using the first intra filter belonging to the first intra filter type. On the other hand, if the current block uses at least one of the third reference line or the fourth reference line, reference sample smoothing may be performed using the second intra filter belonging to the first intra filter type. In this case, the filter lengths of the first intra filter and the second intra filter may be the same, while the filter coefficients may be different. For example, the filter coefficients of the first intra filter may be (1,2,1), and the filter coefficients of the second intra filter may be (2,12,2).

[0359] Depending on whether extended reference lines are used, the number of available intra prediction modes can be determined variably. That is, depending on whether extended reference lines are used, different numbers of directional intra prediction modes can be used per sequence slice, coding tree unit, coding unit, or prediction unit.

[0360] When extended reference lines are used, intra prediction efficiency increases, so there is no problem in performing efficient intra prediction even if a smaller number of intra prediction modes is used than when extended reference lines are not used. Accordingly, depending on whether extended reference lines are used, it can be determined whether to use N or a smaller number of directional prediction modes. For example, when extended reference lines are used, the basic intra prediction mode (i.e., 33 directional intra prediction modes) can be used, and when extended reference lines are not used, the extended intra prediction mode (i.e., 65 directional intra prediction modes) can be set to be used.

[0361] Depending on the location of the reference line used for intra prediction, the type or number of intra prediction modes available to the current block can be limited.

[0362] For example, as the distance between the current block and the reference line increases, the discontinuity between the current block and the reference line increases. Accordingly, as the distance between the current block and the reference line increases, the efficiency of intra prediction using a non-directional prediction mode such as the DC mode or the planar mode decreases. Accordingly, when a reference line whose distance from the current block is greater than a predetermined threshold is used, at least one non-directional prediction mode among the DC mode or the planar mode may be set not to be used. For example, when the index of the reference line used for intra prediction of the current block is L or greater, at least one non-directional prediction mode among the DC mode or the planar mode may not be used. Here, L is an integer including 0, and may be 0, 1, 2, 3, etc.

[0363] Alternatively, the number of available intra prediction modes may be adjusted depending on the position of the intra reference line used for intra prediction of the current block. For example, if the index of the reference line used for intra prediction of the current block is greater than or equal to L, the basic intra prediction mode may be used, and if the index of the reference line is less than L, the extended intra prediction mode may be used. For example, if L is 2, the extended intra prediction mode (i.e., 67 intra prediction modes) may be used for the first reference line or the second reference line, and the basic intra prediction mode (i.e., 35 intra prediction modes) may be used for the third reference line or the fourth reference line.

[0364] Depending on the intra prediction mode of the current block or the number of intra prediction modes available to the current block, the reference lines that the current block can select may be restricted. For example, if the intra prediction mode of the current block is a non-directional prediction mode such as DC mode or planar mode, reference lines with indices greater than L may be set not to be used for intra prediction of the current block.

[0365] Although the above-described embodiment has been described focusing on the decryption process, the encoding process may be performed in the same order as described or in the reverse order.

[0366]

[0367] Figure 31 is a flowchart illustrating a process for obtaining a residual sample as an embodiment to which the present invention is applied.

[0368] First, the residual coefficients of the current block can be obtained (S3110). The decoder can obtain the residual coefficients through a coefficient scanning method. For example, the decoder can perform coefficient scanning using a zig-zag scan, vertical scan, or horizontal scan, thereby obtaining the residual coefficients in the form of a two-dimensional block.

[0369] Dequantization can be performed on the residual coefficients of the current block (S3120).

[0370] It may be determined whether to skip the inverse transform on the inverse quantized residual coefficients of the current block (S3130). Specifically, the decoder may determine whether to skip the inverse transform in at least one of the horizontal direction or the vertical direction of the current block. If it is determined to apply the inverse transform to at least one of the vertical or horizontal directions of the current block, the residual sample of the current block may be obtained by inversely transforming the inverse quantized residual coefficients of the current block. Here, the inverse transform may be performed using at least one of DCT, DST, or KLT.

[0371] If inverse transformations are skipped in both the horizontal and vertical directions of the current block, inverse transformations are not performed in the horizontal and vertical directions of the current block. In this case, the inverse quantized residual coefficients can be scaled to a preset value to obtain the residual samples of the current block.

[0372] Omitting the horizontal inverse transformation means not performing the horizontal inverse transformation, but performing the vertical inverse transformation. In this case, scaling may be performed in the horizontal direction.

[0373] Omitting the vertical inverse transformation means not performing the vertical inverse transformation, but performing the horizontal inverse transformation. In this case, scaling may be performed in the vertical direction.

[0374] Depending on the partitioning pattern of the current block, it may be determined whether the inverse transform skip technique can be used for the current block. For example, if the current block is generated through binary tree-based partitioning, the inverse transform skip technique can be restricted from being used for the current block. Accordingly, if the current block is generated through binary tree-based partitioning, the residual sample of the current block can be obtained by inversely transforming the current block. Furthermore, if the current block is generated through binary tree-based partitioning, the encoding / decoding of information indicating whether inverse transform is skipped (e.g., transform_skip_flag) can be omitted.

[0375] Alternatively, if the current block is generated through binary tree-based partitioning, the inverse transform skip technique may be restricted to be allowed only in at least one of the horizontal or vertical directions. Here, the direction in which the inverse transform skip technique is restricted may be determined based on information decoded from the bitstream, or may be adaptively determined based on at least one of the size of the current block, the shape of the current block, or the intra prediction mode of the current block.

[0376] For example, if the current block is a non-square block with a width greater than its height, the inverse transform skip technique can be allowed only in the vertical direction, and the use of the inverse transform skip technique can be restricted in the horizontal direction. That is, if the current block is 2NxN, the inverse transform is performed in the horizontal direction of the current block, and the inverse transform can be optionally performed in the vertical direction.

[0377] On the other hand, if the current block is a non-square block whose height is greater than its width, the inverse transform skip technique can be allowed only in the horizontal direction, and the use of the inverse transform skip technique can be restricted in the vertical direction. That is, if the current block is Nx2N, the inverse transform is performed in the vertical direction of the current block, and the inverse transform can be optionally performed in the horizontal direction.

[0378] Conversely to the above example, if the current block is a non-square block with a width greater than its height, the inverse skip technique may be allowed only for the horizontal direction, and if the current block is a non-square block with a height greater than its width, the inverse skip technique may be allowed only for the vertical direction.

[0379] Information indicating whether to skip inverse transform in the horizontal direction or whether to skip inverse transform in the vertical direction can be signaled through the bitstream. For example, information indicating whether to skip inverse transform in the horizontal direction can be a 1-bit flag, 'hor_transform_skip_flag', and information indicating whether to skip inverse transform in the vertical direction can be a 1-bit flag, 'ver_transform_skip_flag'. The encoder can encode at least one of 'hor_transform_skip_flag' or 'ver_transform_skip_flag' depending on the shape of the current block. In addition, the decoder can determine whether inverse transform in the horizontal direction or the vertical direction is skipped by using at least one of 'hor_transform_skip_flag' or 'ver_transform_skip_flag'.

[0380] Depending on the partitioning form of the current block, inverse transformation may be set to be skipped for one direction. For example, if the current block is generated through binary tree-based partitioning, inverse transformation in the horizontal direction or vertical direction may be skipped. That is, if the current block is generated through binary tree-based partitioning, it is possible to determine to skip inverse transformation for at least one of the horizontal direction or the vertical direction for the current block without encoding / decoding information indicating whether inverse transformation of the current block is skipped (e.g., transform_skip_flag, hor_transform_skip_flag, ver_transform_skip_flag).

[0381]

[0382] Although the above-described embodiments have been described based on a series of steps or a flowchart, this does not limit the chronological order of the invention, and may be performed simultaneously or in a different order as needed. In addition, each of the components (e.g., units, modules, etc.) constituting the block diagram in the above-described embodiments may be implemented as a hardware device or software, or a plurality of components may be combined to be implemented as a single hardware device or software. The above-described embodiments may be implemented in the form of program commands that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, etc., singly or in combination. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program commands such as ROMs, RAMs, and flash memories. The above hardware device may be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa.

[0383] The present invention can be applied to an electronic device capable of encoding / decoding an image.

Claims

1. A step of deriving multiple reference sample lines for the current block; a step of selecting two or more of the above multiple reference sample lines; and An image decoding method, comprising the step of performing intra prediction for the current block based on two or more of the selected reference sample lines.

2. In paragraph 1, The step of selecting two or more of the above multiple reference sample lines is: An image decoding method, characterized in that the step comprises selecting different reference sample lines for each predetermined area of ​​the current block.

3. In paragraph 2, An image decoding method, characterized in that different reference sample lines are selected between a first region adjacent to one side boundary of the current block and a second region excluding the first region.

4. In paragraph 3, An image decoding method, characterized in that the above one-sided boundary is determined according to the intra prediction mode of the current block.

5. In paragraph 1, An image decoding method, characterized in that the prediction sample of the current block is obtained based on a weighted sum operation of reference samples included in different reference sample lines.

6. In paragraph 5, A method for decoding an image, characterized in that the weight applied to each of the above reference samples is determined based on at least one of an intra prediction mode of the current block, a distance between the prediction sample and the reference sample, or a position of a reference sample line to which the reference sample belongs.

7. In paragraph 1, An image decoding method, characterized in that the plurality of reference sample lines are derived based on a block having a predetermined shape or a predetermined size or larger, including the current block.

8. In paragraph 7, An image decoding method, characterized in that the current block shares the plurality of reference sample lines with neighboring blocks included in the block having the predetermined shape or a predetermined size or larger.

9. A step of deriving multiple reference sample lines for the current block; a step of selecting two or more of the above multiple reference sample lines; and An image encoding method, comprising the step of performing intra prediction for the current block based on two or more of the selected reference sample lines.

10. In paragraph 9, The step of selecting two or more of the above multiple reference sample lines is: An image encoding method, characterized in that it is a step of selecting different reference sample lines for each predetermined area of ​​the current block.

11. In clause 10, A video encoding method, characterized in that different reference sample lines are selected between a first area adjacent to one side boundary of the current block and a second area excluding the first area.

12. In paragraph 11, An image encoding method, characterized in that the above one-sided boundary is determined according to the intra prediction mode of the current block.

13. In paragraph 9, A video encoding method, characterized in that the plurality of reference sample lines are derived based on a block having a predetermined shape or a predetermined size or larger, including the current block.

14. In paragraph 13, A video encoding method, characterized in that the current block shares the plurality of reference sample lines with neighboring blocks included in the block having the predetermined shape or a predetermined size or larger.

15. An image decoding device, comprising an intra prediction unit that derives a plurality of reference sample lines for a current block, selects two or more of the plurality of reference sample lines, and performs intra prediction for the current block based on the two or more selected reference sample lines.

Images