Image encoding / decoding method and recording medium for storing bitstream

Abstract

An image encoding / decoding method according to the present invention may comprise the steps of: deriving filter coefficients for applying an extrapolation filter; and acquiring a prediction sample for a current block by inputting a plurality of samples to the extrapolation filter. In this case, the filter coefficients may be derived by applying the extrapolation filter to a template area around the current block.

Description

Video encoding / decoding method and recording medium for storing bitstream

[0001] The present disclosure relates to a video signal processing method and device.

[0002] Recently, the demand for high-resolution, high-quality images, such as HD (High Definition) and UHD (Ultra High Definition) images, is increasing across various application fields. As image data becomes higher in resolution and quality, the relative amount of data increases compared to conventional image data. Therefore, transmitting image data using existing media such as wired and wireless broadband lines or storing it using existing storage media leads to increased transmission and storage costs. To address these issues arising from the increasing resolution and quality of image data, high-efficiency image compression technologies can be utilized.

[0003] There are various technologies for image compression, such as inter-picture prediction technology that predicts pixel values ​​included in the current picture from pictures before or after the current picture, intra-picture prediction technology that predicts pixel values ​​included in the current picture using pixel information in the current picture, and entropy encoding technology that assigns short codes to values ​​with high frequency of appearance and long codes to values ​​with low frequency of appearance. Using these image compression technologies, image data can be effectively compressed and transmitted or stored.

[0004] Meanwhile, as demand for high-resolution video grows, so does the demand for stereoscopic video content as a new video service. Discussions are underway on video compression technologies to effectively deliver high-resolution and ultra-high-resolution stereoscopic video content.

[0005] The present disclosure aims to provide a method and device for predicting a chroma block using a restored luma block in encoding / decoding a video signal.

[0006] The present disclosure aims to provide a method and device for predicting a chroma block based on the linearity of a luma component and a chroma component in encoding / decoding a video signal.

[0007] The present disclosure aims to provide a method and device for predicting a chroma block in units of sub blocks when encoding / decoding a video signal.

[0008] The present disclosure aims to provide a method and device for deriving filter parameters for chroma block prediction by utilizing similarity between sub-blocks in encoding / decoding a video signal.

[0009] The present disclosure aims to provide a method and device for applying a convolution-based filter coefficient derivation method to single component prediction in encoding / decoding a video signal.

[0010] The technical problems to be achieved in the present disclosure are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by a person having ordinary skill in the technical field to which the present invention pertains from the description below.

[0011] A video encoding / decoding method according to the present disclosure may include a step of deriving filter coefficients for applying an extrapolation filter; and a step of inputting a plurality of samples into the extrapolation filter to obtain a prediction sample for a current block. In this case, the filter coefficients may be derived by applying the extrapolation filter to a template region surrounding the current block.

[0012] In the image decoding / encoding method according to the present disclosure, depending on the position of the predicted sample, it can be determined whether a sample predicted within the current block is included among the plurality of samples input to the extrapolation filter.

[0013] In the video decoding / encoding method according to the present disclosure, when the predicted sample is located at the top or right boundary of the current block, the plurality of samples may be configured to include only reference samples outside the current block, and when the predicted sample is not located at the top or right boundary of the current block, the plurality of samples may be configured to include pre-predicted samples within the current block.

[0014] In the image decoding / encoding method according to the present disclosure, the extrapolation filter may be one of a plurality of extrapolation filter candidates.

[0015] In the image decoding / encoding method according to the present disclosure, one of the plurality of extrapolation filter candidates can be selected by an index decoded from a bitstream.

[0016] In the image decoding / encoding method according to the present disclosure, each of the plurality of extrapolation filter candidates may differ in at least one of a filter shape, a number of filter taps, or an output position of a filter value.

[0017] In the image decoding / encoding method according to the present disclosure, one of the plurality of extrapolation filter candidates can be selected based on the intra prediction mode of the current block.

[0018] In the image decoding / encoding method according to the present disclosure, the plurality of extrapolation filter candidates may include at least one of a first extrapolation filter candidate having an output position at the lower right corner of a rectangular area, a second extrapolation filter candidate having an output position at the right outer area of ​​the rectangular area, or a third extrapolation filter candidate having an output position at the lower outer area of ​​the rectangular area.

[0019] In the image decoding / encoding method according to the present disclosure, the extrapolation filter can be applied only to prediction samples located at the upper or left boundary of the current block.

[0020] In the image decoding / encoding method according to the present disclosure, a final prediction sample can be obtained at the upper or left boundary of the current block based on a first prediction sample obtained using an extrapolation filter and a second prediction sample obtained based on an intra prediction mode.

[0021] In the image decoding / encoding method according to the present disclosure, at a location not adjacent to the upper or left border of the current block, a second prediction sample obtained based on the intra prediction mode can be set as a final prediction sample.

[0022] In the image decoding / encoding method according to the present disclosure, the final prediction sample for the current block can be obtained based on an average or weighted sum operation between the prediction sample obtained by applying the extrapolation filter and an additional prediction sample obtained based on the intra prediction mode of the current block.

[0023] A computer-readable recording medium storing a bitstream encoded by an image encoding method according to the present disclosure may be provided.

[0024] According to the present disclosure, encoding / decoding efficiency can be improved by predicting a chroma block using a restored luma block.

[0025] According to the present disclosure, the accuracy of intra prediction can be improved by predicting a chroma block based on the linearity of a luma component and a chroma component.

[0026] According to the present disclosure, intra prediction accuracy can be improved by predicting chroma blocks in sub-block units.

[0027] According to the present disclosure, intra prediction can improve accuracy by deriving filter parameters for chroma block prediction by utilizing similarity between sub-blocks.

[0028] According to the present disclosure, prediction accuracy can be improved by applying a convolution-based filter coefficient derivation method to single component prediction.

[0029] The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects that are not mentioned will be clearly understood by a person having ordinary skill in the art to which the present disclosure pertains from the description below.

[0030] FIG. 1 is a block diagram illustrating an image encoding device according to an embodiment of the present disclosure.

[0031] FIG. 2 is a block diagram showing an image decoding device according to an embodiment of the present disclosure.

[0032] FIG. 3 illustrates an image encoding / decoding method performed by an image encoding / decoding device according to the present disclosure.

[0033] FIG. 4 and FIG. 5 illustrate examples of multiple intra prediction modes according to the present disclosure.

[0034] FIG. 6 illustrates an intra prediction method based on a planar mode according to the present disclosure.

[0035] FIG. 7 illustrates an intra prediction method based on DC mode according to the present disclosure.

[0036] FIG. 8 illustrates an intra prediction method based on a directional mode according to the present disclosure.

[0037] Figure 9 illustrates a method for deriving samples of fractional positions.

[0038] Figures 10 and 11 illustrate that the tangent value for the angle is scaled by a factor of 32 for each intra prediction mode.

[0039] Figure 12 is a diagram illustrating an intra prediction aspect when the directional mode is one of modes 34 to 49.

[0040] Figure 13 is a drawing for explaining an example of generating an upper reference sample by interpolating left reference samples.

[0041] Figure 14 shows an example in which intra prediction is performed using reference samples arranged in a 1D array.

[0042] Figure 15 is a flowchart illustrating a method for predicting a chroma block using a restored luma block.

[0043] Figures 16 to 18 illustrate examples of downsampling a luma block.

[0044] Figure 19 is a drawing for explaining an example related to the location where down sampling is applied.

[0045] Figure 20 illustrates templates for each chroma block and its corresponding luma block.

[0046] Figure 21 shows the filter form applied to reference samples of the luma component.

[0047] Figure 22 shows an example of obtaining a prediction sample based on an extrapolation filter.

[0048] Figure 23 illustrates various forms of extrapolation filters.

[0049] Figure 24 shows a restoration area around the current block for deriving filter coefficients of an extrapolation filter.

[0050] Figure 25 illustrates locations excluded from the input of the extrapolation filter.

[0051] The present invention is susceptible to various modifications and embodiments. Specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and technical scope of the present invention. Throughout the description of each drawing, similar reference numerals have been used to designate similar components.

[0052] While terms such as "first" and "second" may be used to describe various components, these components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a "second component," and similarly, a second component may also be referred to as a "first component." The term "and / or" includes a combination of multiple related items described herein or any of multiple related items described herein.

[0053] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0054] The terminology used in this application is only used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprise" or "have" indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0055] Hereinafter, with reference to the attached drawings, preferred embodiments of the present invention will be described in more detail. Hereinafter, identical components in the drawings will be designated by the same reference numerals, and redundant descriptions of identical components will be omitted.

[0056] FIG. 1 is a block diagram showing an image encoding device according to an embodiment of the present invention.

[0057] Referring to FIG. 1, a video encoding device (100) may include a picture segmentation unit (110), a prediction unit (120, 125), a transformation unit (130), a quantization unit (135), a reordering unit (160), an entropy encoding unit (165), an inverse quantization unit (140), an inverse transformation unit (145), a filter unit (150), and a memory (155).

[0058] Each component shown in Fig. 1 is independently depicted to indicate different characteristic functions in the video encoding device, and does not mean that each component is composed of separate hardware or a single software component. That is, each component is listed and included as a separate component for convenience of explanation, and at least two components among each component may be combined to form a single component, or one component may be divided into multiple components to perform a function, and such integrated and separate embodiments of each component are also included in the scope of the present invention as long as they do not deviate from the essence of the present invention.

[0059] Additionally, some components may not be essential components that perform essential functions of the present invention, but may be optional components merely used to enhance performance. The present invention may be implemented by including only components essential to implementing the essence of the present invention, excluding components used solely for performance enhancement. A structure that includes only essential components, excluding optional components used solely for performance enhancement, is also within the scope of the present invention.

[0060] The picture splitting unit (110) can split the input picture into at least one processing unit. At this time, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). The picture splitting unit (110) can split one picture into a combination of multiple coding units, prediction units, and transform units, and select one combination of coding units, prediction units, and transform units based on a predetermined criterion (e.g., a cost function) to encode the picture.

[0061] For example, a picture can be split into multiple coding units. A recursive tree structure, such as a quad tree structure, can be used to split a coding unit in a picture. A coding unit that is split into other coding units starting from an image or the largest coding unit as the root can be split into as many child nodes as the number of split coding units. A coding unit that cannot be split any further according to a certain restriction becomes a leaf node. That is, assuming that only a square split is possible for a coding unit, a coding unit can be split into at most four different coding units.

[0062] Hereinafter, in the embodiments of the present invention, the encoding unit may be used to mean a unit that performs encoding or may be used to mean a unit that performs decoding.

[0063] A prediction unit may be divided into at least one square or rectangular shape of the same size within a single coding unit, or may be divided such that one prediction unit among the divided prediction units within a single coding unit has a different shape and / or size from another prediction unit.

[0064] When generating a prediction unit that performs intra prediction based on a coding unit, if it is not the minimum coding unit, intra prediction can be performed without dividing it into multiple prediction units NxN.

[0065] The prediction unit (120, 125) may include an inter prediction unit (120) that performs inter prediction and an intra prediction unit (125) that performs intra prediction. It may be determined whether to use inter prediction or intra prediction for a prediction unit, and specific information (e.g., intra prediction mode, motion vector, reference picture, etc.) according to each prediction method may be determined. At this time, the processing unit where prediction is performed and the processing unit where the prediction method and specific contents are determined may be different. For example, the prediction method and prediction mode may be determined in the prediction unit, and the prediction may be performed in the transformation unit. The residual value (residual block) between the generated prediction block and the original block may be input to the transformation unit (130). In addition, the prediction mode information, motion vector information, etc. used for prediction may be encoded together with the residual value in the entropy encoding unit (165) and transmitted to the decoding device. When using a specific encoding mode, it is also possible to encode the original block as is and transmit it to the decoding unit without generating a prediction block through the prediction unit (120, 125).

[0066] The inter prediction unit (120) may predict a prediction unit based on information of at least one picture among the previous or subsequent pictures of the current picture, and in some cases, may predict a prediction unit based on information of a portion of an encoded region within the current picture. The inter prediction unit (120) may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

[0067] The reference picture interpolation unit can receive reference picture information from the memory (155) and generate pixel information less than an integer pixel from the reference picture. In the case of luminance pixels, a DCT-based 8-tap interpolation filter with different filter coefficients can be used to generate pixel information less than an integer pixel in units of 1 / 4 pixels. In the case of a chrominance signal, a DCT-based 4-tap interpolation filter with different filter coefficients can be used to generate pixel information less than an integer pixel in units of 1 / 8 pixels.

[0068] The motion prediction unit can perform motion prediction based on a reference picture interpolated by the reference picture interpolation unit. Various methods can be used to derive a motion vector, such as FBMA (Full search-based Block Matching Algorithm), TSS (Three Step Search), and NTS (New Three-Step Search Algorithm). The motion vector can have a motion vector value in units of 1 / 2 or 1 / 4 pixels based on the interpolated pixel. The motion prediction unit can predict the current prediction unit by using different motion prediction methods. Various methods can be used as motion prediction methods, such as the Skip method, the Merge method, the AMVP (Advanced Motion Vector Prediction) method, and the Intra Block Copy method.

[0069] The intra prediction unit (125) can generate a prediction unit based on reference pixel information surrounding the current block, which is pixel information within the current picture. If the surrounding block of the current prediction unit is a block on which inter prediction has been performed and the reference pixel is a pixel on which inter prediction has been performed, the reference pixel included in the block on which inter prediction has been performed can be replaced and used with reference pixel information of the surrounding block on which intra prediction has been performed. That is, if the reference pixel is not available, the unavailable reference pixel information can be replaced and used with at least one reference pixel among the available reference pixels.

[0070] In intra prediction, the prediction mode can have a directional prediction mode that uses reference pixel information according to the prediction direction, and a non-directional mode that does not use directional information when performing prediction. The mode for predicting luminance information and the mode for predicting chrominance information can be different, and the intra prediction mode information used to predict luminance information or the predicted luminance signal information can be utilized to predict chrominance information.

[0071] When performing intra prediction, if the size of the prediction unit and the size of the transformation unit are the same, intra prediction for the prediction unit can be performed based on the pixels located to the left of the prediction unit, the pixels located at the upper left, and the pixels located at the upper. However, when performing intra prediction, if the sizes of the prediction unit and the transformation unit are different, intra prediction can be performed using reference pixels based on the transformation unit. In addition, intra prediction using NxN division can be used only for the minimum coding unit.

[0072] The intra prediction method can generate a prediction block after applying an Adaptive Intra Smoothing (AIS) filter to a reference pixel according to a prediction mode. The type of AIS filter applied to the reference pixel may be different. In order to perform the intra prediction method, the intra prediction mode of the current prediction unit can be predicted from the intra prediction modes of prediction units existing around the current prediction unit. When the prediction mode of the current prediction unit is predicted using the mode information predicted from the surrounding prediction units, if the intra prediction modes of the current prediction unit and the surrounding prediction units are the same, information indicating that the prediction modes of the current prediction unit and the surrounding prediction units are the same can be transmitted using predetermined flag information, and if the prediction modes of the current prediction unit and the surrounding prediction units are different, entropy encoding can be performed to encode the prediction mode information of the current block.

[0073] Additionally, a residual block containing residual value information, which is the difference between the prediction unit that performed the prediction based on the prediction unit generated in the prediction unit (120, 125) and the original block of the prediction unit, can be generated. The generated residual block can be input to the transformation unit (130).

[0074] In the transformation unit (130), the residual block including the residual value information of the prediction unit generated through the original block and the prediction unit (120, 125) can be transformed using a transformation method such as DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), or KLT. Whether to apply DCT, DST, or KLT to transform the residual block can be determined based on the intra prediction mode information of the prediction unit used to generate the residual block.

[0075] The quantization unit (135) can quantize values ​​converted to the frequency domain by the transformation unit (130). The quantization coefficients can vary depending on the block or the importance of the image. The values ​​produced by the quantization unit (135) can be provided to the dequantization unit (140) and the reordering unit (160).

[0076] The rearrangement unit (160) can perform rearrangement of coefficient values ​​for quantized residual values.

[0077] The rearrangement unit (160) can change a two-dimensional block-shaped coefficient into a one-dimensional vector form through a coefficient scanning method. For example, the rearrangement unit (160) can change the two-dimensional block-shaped coefficient into a one-dimensional vector form by scanning from the DC coefficient to the coefficient of the high-frequency region using a zig-zag scan method. Depending on the size of the transformation unit and the intra prediction mode, a vertical scan that scans the two-dimensional block-shaped coefficient in the column direction or a horizontal scan that scans the two-dimensional block-shaped coefficient in the row direction may be used instead of the zig-zag scan. That is, depending on the size of the transformation unit and the intra prediction mode, it is possible to determine which scan method among the zig-zag scan, the vertical scan, and the horizontal scan is to be used.

[0078] The entropy encoding unit (165) can perform entropy encoding based on the values ​​produced by the rearrangement unit (160). Entropy encoding can use various encoding methods such as, for example, Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

[0079] The entropy encoding unit (165) can encode various information such as residual value coefficient information of an encoding unit, block type information, prediction mode information, division unit information, prediction unit information, transmission unit information, motion vector information, reference frame information, block interpolation information, and filtering information from the rearrangement unit (160) and the prediction unit (120, 125).

[0080] The entropy encoding unit (165) can entropy encode the coefficient values ​​of the encoding unit input from the rearrangement unit (160).

[0081] The inverse quantization unit (140) and the inverse transformation unit (145) inversely quantize the values ​​quantized in the quantization unit (135) and inversely transform the values ​​transformed in the transformation unit (130). The residual values ​​generated in the inverse quantization unit (140) and the inverse transformation unit (145) can be combined with the predicted prediction units predicted through the motion estimation unit, motion compensation unit, and intra prediction unit included in the prediction unit (120, 125) to generate a reconstructed block.

[0082] The filter unit (150) may include at least one of a deblocking filter, an offset correction unit, and an ALF (Adaptive Loop Filter).

[0083] A deblocking filter can remove block distortion caused by boundaries between blocks in a reconstructed picture. Whether to apply a deblocking filter to the current block can be determined based on the pixels contained in several columns or rows within the block. When applying a deblocking filter to a block, a strong filter or a weak filter can be applied depending on the required deblocking filtering strength. Furthermore, when applying a deblocking filter, horizontal and vertical filtering can be processed in parallel when performing vertical and horizontal filtering.

[0084] The offset correction unit can correct the offset from the original image on a pixel-by-pixel basis for an image that has undergone deblocking. To perform offset correction for a specific picture, the pixels contained in the image can be divided into a certain number of regions, the regions to be offset can be determined, and the offset can be applied to those regions. Alternatively, the offset can be applied by considering the edge information of each pixel.

[0085] Adaptive Loop Filtering (ALF) can be performed based on the comparison of the filtered restored image with the original image. After dividing the pixels included in the image into predetermined groups, a filter to be applied to each group can be determined, and filtering can be performed differentially for each group. Information regarding whether to apply ALF can be transmitted by luminance signal for each coding unit (CU), and the shape and filter coefficients of the ALF filter to be applied can vary depending on each block. Furthermore, an ALF filter of the same form (fixed form) can be applied regardless of the characteristics of the target block.

[0086] The memory (155) can store a restored block or picture produced through the filter unit (150), and the stored restored block or picture can be provided to the prediction unit (120, 125) when performing inter prediction.

[0087] FIG. 2 is a block diagram showing an image decoding device according to an embodiment of the present invention.

[0088] Referring to FIG. 2, the image decoding device (200) may include an entropy decoding unit (210), a rearrangement unit (215), an inverse quantization unit (220), an inverse transformation unit (225), a prediction unit (230, 235), a filter unit (240), and a memory (245).

[0089] When a video bitstream is input to a video encoding device, the input bitstream can be decoded in the opposite procedure to that of the video encoding device.

[0090] The entropy decoding unit (210) can perform entropy decoding in a procedure opposite to that of the entropy encoding unit of the video encoding device. For example, various methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) can be applied in response to the method performed in the video encoding device.

[0091] The entropy decoding unit (210) can decode information related to intra prediction and inter prediction performed in the encoding device.

[0092] The reordering unit (215) can perform reordering based on the method in which the bitstream entropy-decoded by the entropy decoding unit (210) is reordered by the encoding unit. The coefficients expressed in the form of a one-dimensional vector can be reordered by restoring them back to coefficients in the form of a two-dimensional block. The reordering unit (215) can perform reordering by receiving information related to the coefficient scanning performed by the encoding unit and performing reverse scanning based on the scanning order performed by the corresponding encoding unit.

[0093] The dequantization unit (220) can perform dequantization based on the quantization parameters provided from the encoding device and the coefficient values ​​of the rearranged block.

[0094] The inverse transform unit (225) can perform inverse transform, i.e., inverse DCT, inverse DST, and inverse KLT, on the transforms performed by the transform unit, i.e., DCT, DST, and KLT, on the quantization result performed by the image encoding device. The inverse transform can be performed based on the transmission unit determined by the image encoding device. In the inverse transform unit (225) of the image decoding device, a transform technique (e.g., DCT, DST, KLT) can be selectively performed according to a plurality of pieces of information, such as a prediction method, the size of the current block, and the prediction direction.

[0095] The prediction unit (230, 235) can generate a prediction block based on the prediction block generation related information provided by the entropy decoding unit (210) and the previously decoded block or picture information provided by the memory (245).

[0096] As described above, when performing intra prediction in the same manner as the operation in the video encoding device, if the size of the prediction unit and the size of the transformation unit are the same, intra prediction for the prediction unit is performed based on the pixels on the left side of the prediction unit, the pixels on the upper left side, and the pixels on the upper side. However, when performing intra prediction, if the size of the prediction unit and the size of the transformation unit are different, intra prediction can be performed using a reference pixel based on the transformation unit. In addition, intra prediction using NxN division only for the minimum coding unit can be used.

[0097] The prediction unit (230, 235) may include a prediction unit determination unit, an inter prediction unit, and an intra prediction unit. The prediction unit determination unit may receive various information such as prediction unit information input from the entropy decoding unit (210), prediction mode information of an intra prediction method, and motion prediction-related information of an inter prediction method, and may distinguish a prediction unit from a current encoding unit and determine whether the prediction unit performs inter prediction or intra prediction. The inter prediction unit (230) may perform inter prediction on the current prediction unit based on information included in at least one of a previous picture or a subsequent picture of the current picture including the current prediction unit, using information necessary for inter prediction of the current prediction unit provided from the video encoding device. Alternatively, inter prediction may be performed based on information on a pre-restored portion of the current picture including the current prediction unit.

[0098] In order to perform inter prediction, it is possible to determine whether the motion prediction method of the prediction unit included in the encoding unit is Skip Mode, Merge Mode, AMVP Mode, or Intra Block Copy Mode based on the encoding unit.

[0099] The intra prediction unit (235) can generate a prediction block based on pixel information in the current picture. If the prediction unit is a prediction unit that has performed intra prediction, intra prediction can be performed based on intra prediction mode information of the prediction unit provided by the image encoding device. The intra prediction unit (235) can include an AIS (Adaptive Intra Smoothing) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter is a part that performs filtering on the reference pixels of the current block, and can determine and apply whether to apply the filter according to the prediction mode of the current prediction unit. AIS filtering can be performed on the reference pixels of the current block using the prediction mode and AIS filter information of the prediction unit provided by the image encoding device. If the prediction mode of the current block is a mode that does not perform AIS filtering, the AIS filter may not be applied.

[0100] The reference pixel interpolation unit can interpolate the reference pixel to generate a reference pixel of a pixel unit less than an integer value when the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on the pixel value interpolated from the reference pixel. When the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixel, the reference pixel may not be interpolated. The DC filter can generate a prediction block through filtering when the prediction mode of the current block is the DC mode.

[0101] The restored block or picture may be provided to a filter unit (240). The filter unit (240) may include a deblocking filter, an offset correction unit, and an ALF.

[0102] Information regarding whether a deblocking filter has been applied to a corresponding block or picture may be received from a video encoding device, and if a deblocking filter has been applied, information regarding whether a strong or weak filter has been applied. The deblocking filter of the video decoding device may receive information related to the deblocking filter provided by the video encoding device, and the video decoding device may perform deblocking filtering on the corresponding block.

[0103] The offset correction unit can perform offset correction on the restored image based on the type of offset correction applied to the image during encoding and offset value information.

[0104] ALF can be applied to an encoding unit based on information such as whether ALF is applied and ALF coefficient information provided from an encoding device. This ALF information can be provided by being included in a specific parameter set.

[0105] The memory (245) can store a restored picture or block so that it can be used as a reference picture or reference block, and can also provide the restored picture to an output unit.

[0106] As described above, in the following embodiments of the present invention, for convenience of explanation, the term “coding unit” is used as an encoding unit, but it may also be a unit that performs not only encoding but also decoding.

[0107] In addition, the current block represents a block to be encoded / decoded, and may represent a coding tree block (or coding tree unit), an encoding block (or encoding unit), a transform block (or transform unit), or a prediction block (or prediction unit), depending on the encoding / decoding step. In this specification, a 'unit' represents a basic unit for performing a specific encoding / decoding process, and a 'block' may represent a pixel array of a predetermined size. Unless otherwise specified, 'block' and 'unit' may be used with the same meaning. For example, in the embodiment described below, an encoding block (coding block) and an encoding unit (coding unit) may be understood to have the same meaning.

[0108] FIG. 3 illustrates an image encoding / decoding method performed by an image encoding / decoding device according to the present disclosure.

[0109] Referring to FIG. 3, a reference line for intra prediction of the current block can be determined (S300).

[0110] The current block can use one or more of a plurality of pre-defined reference line candidates in the video encoding / decoding device as reference lines for intra prediction. Here, the plurality of pre-defined reference line candidates can include neighboring reference lines adjacent to the current block to be decoded and N non-neighboring reference lines that are 1 to N samples away from the boundary of the current block. N can be 1, 2, 3, or an integer greater than or equal to 1. For convenience of explanation, it is assumed hereafter that the plurality of reference line candidates available to the current block are composed of neighboring reference line candidates and three non-neighboring reference line candidates, but the present invention is not limited thereto. That is, it goes without saying that the plurality of reference line candidates available to the current block can include four or more non-neighboring reference line candidates.

[0111] An image encoding device can determine an optimal reference line candidate from among a plurality of reference line candidates and encode an index for specifying the optimal reference line candidate. An image decoding device can determine a reference line of a current block based on an index signaled through a bitstream. The index can specify any one of the plurality of reference line candidates. The reference line candidate specified by the index can be used as a reference line of the current block.

[0112] The number of indexes signaled to determine the reference line of the current block may be 1, 2, or more. For example, when the number of indexes signaled is 1, the current block can perform intra prediction using only a single reference line candidate specified by the signaled index among a plurality of reference line candidates. Alternatively, when the number of indexes signaled is 2 or more, the current block can perform intra prediction using a plurality of reference line candidates specified by a plurality of indexes among a plurality of reference line candidates.

[0113] Referring to FIG. 3, the intra prediction mode of the current block can be determined (S310).

[0114] The intra prediction mode of the current block can be determined from among multiple intra prediction modes pre-defined in the video encoding / decoding device. The multiple pre-defined intra prediction modes will be described with reference to FIGS. 4 and 5.

[0115] FIG. 4 illustrates an example of multiple intra prediction modes according to the present disclosure.

[0116] Referring to FIG. 4, the multiple intra prediction modes pre-defined in the video encoding / decoding device may be configured as a non-directional mode and a directional mode. The non-directional mode may include at least one of a planar mode or a DC mode. The directional mode may include directional modes 2 to 66.

[0117] The directional mode can be further expanded than that shown in Fig. 4. Fig. 5 shows an example in which the directional mode is expanded.

[0118] In Fig. 5, modes -1 to -14 and modes 67 to 80 are exemplified as being added. These directional modes may be referred to as wide-angle intra prediction modes. Whether to use the wide-angle intra prediction mode may be determined depending on the shape of the current block. For example, if the current block is a non-square block whose width is greater than its height, some directional modes (e.g., 2 to 15) may be converted to wide-angle intra prediction modes between 67 and 80. On the other hand, if the current block is a non-square block whose height is greater than its width, some directional modes (e.g., 53 to 66) may be converted to wide-angle intra prediction modes between -1 and -14.

[0119] The range of available wide-angle intra prediction modes can be adaptively determined based on the width-to-height ratio of the current block. Table 1 shows the range of available wide-angle intra prediction modes based on the width-to-height ratio of the current block.

[0120] Available Wide Angle Intra Prediction Mode Ranges W / H = 1667~80 W / H = 867~78 W / H = 467~76 W / H = 267~74 W / H = 1 None W / H = 1 / 2-1~-8 W / H = 1 / 4-1~-10 W / H = 1 / 8-1~-12 W / H = 1 / 16-1~-14

[0121] Among the above multiple intra prediction modes, K candidate modes (most probable modes, MPMs) can be selected. A candidate list including the selected candidate modes can be generated. An index indicating one of the candidate modes in the candidate list can be signaled. The intra prediction mode of the current block can be determined based on the candidate mode indicated by the index. For example, the candidate mode indicated by the index can be set as the intra prediction mode of the current block. Alternatively, the intra prediction mode of the current block can be determined based on a value of the candidate mode indicated by the index and a predetermined difference value. The difference value can be defined as a difference between a value of the intra prediction mode of the current block and a value of the candidate mode indicated by the index. The difference value can be signaled through a bitstream. Alternatively, the difference value may be a pre-defined value in the video encoding / decoding device. Alternatively, the intra prediction mode of the current block may be determined based on a flag indicating whether a mode identical to the intra prediction mode of the current block exists in the candidate list. For example, when the flag has a first value, the intra prediction mode of the current block may be determined from the candidate list. In this case, an index indicating any one of a plurality of candidate modes belonging to the candidate list may be signaled. The candidate mode indicated by the index may be set as the intra prediction mode of the current block. On the other hand, when the flag has a second value, any one of the remaining intra prediction modes may be set as the intra prediction mode of the current block. The remaining intra prediction mode may mean a mode excluding a candidate mode belonging to the candidate list among the plurality of pre-defined intra prediction modes. When the flag has a second value, an index indicating any one of the remaining intra prediction modes may be signaled.An intra prediction mode indicated by a signaled index may be set as the intra prediction mode of the current block. The intra prediction mode of a chroma block may be selected from intra prediction mode candidates of multiple chroma blocks. To this end, index information indicating one of the intra prediction mode candidates of the chroma block may be explicitly encoded and signaled through the bitstream. Table 2 illustrates examples of intra prediction mode candidates of chroma blocks.

[0122] Intra prediction mode candidates for indexed chroma blocks Luma mode: 0 Luma mode: 50 Luma mode: 18 Luma mode: 1 Other 0 6 6 0 0 0 1 5 0 6 6 5 0 5 0 5 0 2 1 8 1 8 6 6 1 8 1 8 3 1 1 6 6 1 4 DM

[0123] In the example of Table 2, DM (Direct Mode) means setting the intra prediction mode of the luma block co-located with the chroma block to the intra prediction mode of the chroma block. Meanwhile, the luma block co-located with the chroma block can be determined based on the position of the upper left sample or the position of the center sample of the chroma block.

[0124] For example, if the intra prediction mode (luma mode) of the luma block is 0 (planar mode) and the index points to 2, the intra prediction mode of the chroma block can be determined as horizontal mode (18). For example, if the intra prediction mode (luma mode) of the luma block is 1 (DC mode) and the index points to 0, the intra prediction mode of the chroma block can be determined as planar mode (0).

[0125] Consequently, the intra prediction mode of the chroma block may also be set to one of the intra prediction modes illustrated in FIG. 4 or FIG. 5. The intra prediction mode of the current block may also be used to determine a reference line of the current block, in which case step S310 may be performed before step S300.

[0126] Referring to FIG. 3, intra prediction can be performed for the current block based on the reference line and intra prediction mode of the current block (S320).

[0127] Hereinafter, with reference to FIGS. 6 through 8, we will examine in detail the intra prediction method for each intra prediction mode. However, for convenience of explanation, it is assumed that a single reference line is used for intra prediction of the current block. However, even when multiple reference lines are used, the intra prediction method described below can be applied in the same / similar manner.

[0128] FIG. 6 illustrates an intra prediction method based on a planar mode according to the present disclosure.

[0129] Referring to Fig. 6, T represents a reference sample located at the upper right corner of the current block, and L represents a reference sample located at the lower left corner of the current block. P1 can be generated through horizontal interpolation. For example, P1 can be generated by interpolating T with a reference sample located on the same horizontal line as P1. P2 can be generated through vertical interpolation. For example, P2 can be generated by interpolating L with a reference sample located on the same vertical line as P2. The current sample within the current block can be predicted through a weighted sum of P1 and P2, as in the following mathematical expression 1.

[0130]

[0131] In Equation 1, weights α and β can be determined by considering the width and height of the current block. Depending on the width and height of the current block, weights α and β may have the same value or different values. If the width and height of the current block are the same, weights α and β can be set to the same value, and the prediction sample of the current sample can be set to the average value of P1 and P2. If the width and height of the current block are not the same, weights α and β can have different values. For example, if the width is greater than the height, a smaller value can be set for the weight corresponding to the width of the current block, and a larger value can be set for the weight corresponding to the height of the current block. Conversely, if the width is greater than the height, a larger value can be set for the weight corresponding to the width of the current block, and a smaller value can be set for the weight corresponding to the height of the current block. Here, the weight corresponding to the width of the current block can mean β, and the weight corresponding to the height of the current block can mean α.

[0132] FIG. 7 illustrates an intra prediction method based on DC mode according to the present disclosure.

[0133] Referring to FIG. 7, the average value of neighboring samples adjacent to the current block can be calculated, and the calculated average value can be set as the predicted value of all samples in the current block. Here, the neighboring samples can include the upper reference sample and the left reference sample of the current block. However, depending on the shape of the current block, the average value can be calculated using only the upper reference sample or the left reference sample. For example, if the width of the current block is greater than the height, the average value can be calculated using only the upper reference sample of the current block. Alternatively, if the ratio of the width to the height of the current block is greater than or equal to a predetermined threshold, the average value can be calculated using only the upper reference sample of the current block. Alternatively, if the ratio of the width to the height of the current block is less than or equal to a predetermined threshold, the average value can be calculated using only the upper reference sample of the current block. On the other hand, if the width of the current block is less than the height, the average value can be calculated using only the left reference sample of the current block. Alternatively, if the ratio of the width to the height of the current block is less than or equal to a predetermined threshold, the average value can be calculated using only the left reference sample of the current block. Alternatively, if the ratio of the width and height of the current block is greater than or equal to a predetermined threshold, the average value can be calculated using only the left reference sample of the current block.

[0134] FIG. 8 illustrates an intra prediction method based on a directional mode according to the present disclosure.

[0135] If the intra prediction mode of the current block is a directional mode, projection can be performed to a reference line according to the angle of the directional mode. If a reference sample exists at the projected position, the reference sample can be set as a prediction sample of the current sample. If a reference sample does not exist at the projected position, a sample corresponding to the projected position can be generated using one or more neighboring samples neighboring the projected position. For example, a sample corresponding to the projected position can be generated by performing interpolation based on two or more neighboring samples neighboring in both directions with respect to the projected position. Alternatively, one neighboring sample neighboring the projected position can be set as the sample corresponding to the projected position. In this case, among the plurality of neighboring samples neighboring the projected position, the neighboring sample closest to the projected position can be used. The sample corresponding to the projected position can be set as a prediction sample of the current sample.

[0136] Referring to FIG. 8, for the current sample B, when projection is performed with a reference line according to the angle of the intra prediction mode at the corresponding position, a reference sample exists at the projected position (i.e., a reference sample at an integer position, R3). In this case, the reference sample at the projected position can be set as a prediction sample of the current sample B. For the current sample A, when projection is performed with a reference line according to the angle of the intra prediction mode at the corresponding position, a reference sample (i.e., a reference sample at an integer position) does not exist at the projected position. In this case, a sample (r) at a fractional position can be generated by performing interpolation based on neighboring samples (e.g., R2 and R3) adjacent to the projected position. The generated sample (r) at the fractional position can be set as a prediction sample of the current sample A.

[0137] Figure 9 illustrates a method for deriving samples of fractional positions.

[0138] In the example of Fig. 9, the variable h represents the vertical distance (i.e., vertical distance) between the position of the predicted sample A and the reference sample line, and the variable w represents the horizontal distance (i.e., horizontal distance) between the position of the predicted sample A and the fractional position sample. In addition, the variable θ represents a predefined angle according to the directionality of the intra prediction mode, and the variable x represents the fractional position.

[0139] The variable w can be derived as shown in the following mathematical expression 2.

[0140]

[0141] Afterwards, by removing the integer position from the variable w, the fractional position can be finally derived.

[0142] Fractional position samples can be generated by interpolating adjacent integer position reference samples. For example, integer position reference sample R2 and integer position reference sample R3 can be interpolated to generate fractional position reference samples at the x position.

[0143] To avoid floating-point operations when deriving fractional position samples, a scaling factor can be used. For example, if the scaling factor f is set to 32, the distance between neighboring integer reference samples can be set to 32 instead of 1, as in the example illustrated in (b) of Fig. 8.

[0144] Additionally, the tangent value for the angle θ determined by the directionality of the intra prediction mode can also be scaled up using the same scaling factor (e.g., 32).

[0145] Figures 10 and 11 illustrate that the tangent value for the angle is scaled by a factor of 32 for each intra prediction mode.

[0146] Figure 10 shows the scaled results of tangent values ​​for the non-wide angle intra prediction mode, and Figure 11 shows the scaled results of tangent values ​​for the wide angle intra prediction mode.

[0147] If the tangent value (tanθ) for the angle value of the intra prediction mode is positive, intra prediction can be performed using only one of the reference samples belonging to the upper line of the current block (i.e., upper reference samples) or the reference samples belonging to the left line of the current block (i.e., left reference samples). On the other hand, if the tangent value for the angle value of the intra prediction mode is negative, both the reference samples located at the upper side and the reference samples located at the left side are used.

[0148] At this time, to simplify the implementation, the reference samples may be arranged in a 1D array form by projecting the left reference samples upward or the top reference samples to the left, and intra prediction may be performed using the reference samples in the 1D array form.

[0149] Figure 12 is a diagram illustrating an intra prediction aspect when the directional mode is one of modes 34 to 49.

[0150] If the intra prediction mode of the current block is one of modes 34 to 49, intra prediction is performed using not only the upper reference samples of the current block but also the left reference samples. At this time, as in the example illustrated in Fig. 12, the reference sample located on the left side of the current block can be copied to the position of the upper line, or the reference samples located on the left can be interpolated to generate the reference sample of the upper line.

[0151] For example, in case of obtaining a reference sample for position A at the top of the current block, projection can be performed from position A of the top line to the left line of the current block, considering the directionality of the intra prediction mode of the current block. If the projected position is a, the value corresponding to position a can be copied, or a fractional position value corresponding to a can be generated and set as the value of position A. For example, if position a is an integer position, the value of position A can be generated by copying the integer position reference sample. On the other hand, if position a is a fractional position, the reference sample located above position a and the reference sample located below position a can be interpolated, and the interpolated value can be set as the value of position A. Meanwhile, the direction of projection from position A at the top of the current block to the left line of the current block can be parallel to and opposite to the direction of the intra prediction mode of the current block.

[0152] Figure 13 is a drawing for explaining an example of generating an upper reference sample by interpolating left reference samples.

[0153] In Fig. 13, the variable h represents the horizontal distance between position A of the upper line and position a of the left line. The variable w represents the vertical distance between position A of the upper line and position a of the left line. In addition, the variable θ represents a predefined angle according to the directionality of the intra prediction mode, and the variable x represents a fractional position.

[0154] The variable h can be derived as shown in the following mathematical expression 3.

[0155]

[0156] Afterwards, by removing the integer position from the variable h, the fractional position can be finally derived.

[0157] To avoid real-valued operations when deriving fractional position samples, a scaling factor can be used. For example, the tangent value for variable θ can be scaled using the scaling factor f1. Here, since the direction projected to the left line is parallel and opposite to the directional prediction model, the scaled tangent value illustrated in FIGS. 10 and 11 can also be used.

[0158] When the scaling factor f1 is applied, Equation 3 can be transformed into Equation 4 below.

[0159]

[0160] In this manner, a 1D reference sample array can be constructed using only the reference samples belonging to the upper line. As a result, intra prediction for the current block can be performed using only the upper reference samples formed in the 1D array.

[0161] Figure 14 shows an example in which intra prediction is performed using reference samples arranged in a 1D array.

[0162] As in the example illustrated in Fig. 14, by projecting the left reference samples to generate the upper reference samples, prediction samples of the current block can be obtained using only the reference samples belonging to the upper line.

[0163] Contrary to what is shown in FIGS. 12 and 14, a 1D reference sample array can also be constructed using only the reference samples belonging to the left line by projecting the upper reference sample onto the left line. Specifically, for modes 19 to 33 among the directional modes in which the tangent value (tanθ) for the angle of the directional mode is negative, the reference samples belonging to the upper line can be projected onto the left line to generate the left reference sample.

[0164] Meanwhile, for color pictures, encoding / decoding can be performed for each of the luminance component and chroma component. For example, encoding / decoding can be performed for the luminance component of the current block (i.e., luminance block), and then encoding / decoding can be performed for the chroma component of the current block (i.e., chroma block).

[0165] Meanwhile, the size of the chroma block corresponding to the luminance block may vary depending on the chrominance format. For example, in the 4:4:4 format, the size of the luminance block and the size of the chroma block may be the same. On the other hand, in the 4:2:0 format, the width and height of the chroma block may be half the size of the width and height of the corresponding luminance block, respectively.

[0166] Meanwhile, predictions can be performed on chroma blocks using the restored luma block. A prediction model utilizing different color components, such as the above, can be called a Cross-Component Linear Model (CCLM). When CCLM is applied, the process of deriving the intra-prediction mode of the chroma block based on the intra-prediction mode of the luma block can be omitted.

[0167] Figure 15 is a flowchart illustrating a method for predicting a chroma block using a restored luma block.

[0168] In the embodiments described below, the chroma component of the current block will be referred to as a chroma block. In addition, the luminance component block of the current component or the luma component block corresponding to the chroma block will be referred to as a luma block. Meanwhile, the current block represents the current encoding / decoding target block and may be at least one of a coding unit, a prediction unit, or a transformation unit for the luminance component or the chroma component.

[0169] Additionally, unless explicitly stated otherwise, embodiments in the present disclosure that apply to the current block may be applicable to at least one of a luma block or a chroma block. That is, the current block may mean at least one of a luma block or a chroma block. For example, embodiments relating to configuring a template for the current block may be applicable to at least one of configuring a template for a luma block or a template for a chroma block.

[0170] Referring to Figure 15, first, prediction parameters can be derived for prediction of chroma blocks (S1510). At this time, the prediction parameters can be derived in different ways depending on the video format of the picture. The video format indicates the chroma subsampling rate and can be determined as one of 4:4:4, 4:2:2, or 4:2:0.

[0171] If the video format is not 4:4:4, the luma block is downsampled to match the size of the chroma block.

[0172] Figures 16 to 18 illustrate examples of downsampling a luma block.

[0173] For convenience of explanation, the video format is assumed to be 4:2:0.

[0174] When the video format is 4:2:0, the size of the chroma block corresponding to the 4x4 luma block is 2x2, as in the example illustrated in Fig. 16. In this case, a down-sampling filter can be applied to the luma block to reduce the 4x4 luma block to 2x2. The following mathematical expression 5 shows how the down-sampling filter is applied.

[0175]

[0176] In mathematical expression 5, Downsampled_Luma refers to a sample value within a downsampled luma block, and Luma refers to the value of a luma sample before downsampling. For example, Luma[0][0] may indicate the position of the upper left sample within a luma block before downsampling. Since the size of the downsampled luma block is 2x2, the variables x and y, which indicate the coordinates of the sample, may each have values ​​ranging from 0 to 1.

[0177] When applying the down-sampling filter according to mathematical expression 5, the value of the down-sampled luma sample can be obtained by applying a cross-shaped down-sampling filter to the luma samples. For example, the value of the down-sampled luma sample at the (0, 0) position can be obtained by applying the down-sampling filter to the luma sample at the (0, 0) position, the upper luma sample at the (0, 0) position, the left luma sample at the (0, 0) position, the lower luma sample at the (0, 0) position, and the right luma sample at the (0, 0) position.

[0178] Downsampling filters of different shapes than those illustrated in Fig. 17 may also be applied. For example, a 1D filter, a rectangular or square filter may be applied to obtain downsampled luma samples. The 1D filter may have a size of 1x3 or 3x1, the rectangular filter may have a size of 2x3 or 3x2, and the square filter may have a size of 2x2 or 3x2.

[0179] The shape of the filter may be predefined in the encoder and decoder.

[0180] Alternatively, the shape of the filter may be adaptively determined based on at least one of the size / shape of the current block, the intra prediction mode applied to the luma block, whether the positions of the chroma samples match the positions of the luma samples, or the image format.

[0181] Alternatively, information indicating one of multiple filter candidates may be encoded and signaled.

[0182] Alternatively, depending on the downsampling location, the filter type may differ. For example, a 1D filter or a rectangular filter may be applied to luma samples located at the boundary of a luma block, while a cross-shaped filter may be applied to luma samples not located at the boundary of a luma block.

[0183] As illustrated in Fig. 17, a downsampling filter can be applied to positions where both the x-axis coordinate and the y-axis coordinate are even numbers.

[0184] The application position of the downsampling filter can also be set differently from that shown in Fig. 17.

[0185] Figure 18 shows various examples of application locations of the downsampling filter.

[0186] After defining multiple candidates for the downsampling application location, one of the multiple candidates can be selected. For example, after defining the examples in (a) to (d) of FIG. 18 as multiple candidates, index information indicating one of the multiple examples can be encoded and signaled.

[0187] Alternatively, one of the multiple candidates may be selected based on whether the location of the chroma sample matches the location of the luma sample.

[0188] A downsampling filter can also be applied to reference samples surrounding a luma block. Here, the reference samples may represent previously restored samples. Specifically, a downsampling filter can be applied to at least one of an upper reference region adjacent to the upper side of the luma block or a left reference region adjacent to the left side to obtain a downsampled luma reference sample.

[0189] A number of downsampled luma reference samples equal to the number of reference samples included in the reference area of ​​the chroma block can be obtained.

[0190] Meanwhile, the reference area of ​​a luma block can be called a luma reference area, and the reference area of ​​a chroma block can be called a chroma reference area. The reference area can also be called a template.

[0191] The inter-component prediction mode can be divided into an upper inter-component prediction mode, a left inter-component prediction mode, and an upper and left inter-component prediction mode, depending on the configuration of the reference region. When the upper inter-component prediction mode is selected, the reference region of each of the luma block and the chroma block consists of only the upper reconstruction region. When the left inter-component prediction mode is selected, the reference region of each of the luma block and the chroma block consists of only the left reconstruction region. When the upper and left inter-component prediction modes are selected, the reference region of each of the luma block and the chroma block can consist of an upper reconstruction region and a left reconstruction region.

[0192] Information indicating which of the top inter-component prediction mode, the left inter-component prediction mode, and the top and left inter-component prediction modes is applied to the current block can be explicitly encoded and signaled. For example, index information indicating the type of inter-component prediction mode can be encoded and signaled.

[0193] Alternatively, one of the top inter-component prediction mode, the left inter-component prediction mode, and the top and left inter-component prediction mode may be selected based on at least one of the size / shape of the current block, whether the current block touches a CTU or picture boundary, or an intra prediction mode applied to the luma block.

[0194] For convenience of explanation, in the embodiments described below, it is assumed that the reference areas of each of the luma block and the chroma block include an upper reference area and a left reference area.

[0195] The shape of the down-sampling filter applied to the reference area of ​​the luma block may be the same as the down-sampling filter applied to the luma block. Alternatively, the shape of the down-sampling filter applied to the reference area of ​​the luma block may be different from the down-sampling filter applied to the luma block. Alternatively, the shape of the down-sampling filter applied to the upper reference area of ​​the luma block may be different from the shape of the down-sampling filter applied to the left reference area of ​​the luma block.

[0196] Meanwhile, the location where downsampling is applied within the reference region may be predefined in the encoder and decoder.

[0197] In other words, in a decoder, you can also decide for yourself where in the reference region downsampling should be applied, in the same way as the encoder.

[0198] Figure 19 is a drawing for explaining an example related to the location where down sampling is applied.

[0199] When the video format is 4:2:0, a 1x1 chroma block corresponds to a 2x2 luma block. Accordingly, a downsampling filter can be applied to one of the four luma reference samples to derive a downsampled luma reference sample corresponding to the chroma reference sample.

[0200] When four luma reference samples corresponding to one chroma reference sample are represented as A to D, downsampling can be performed on each of positions A to D within the reference area, and then the cost for each position can be calculated. Here, the cost for a specific position can be derived based on the sum of the differences between the downsampled luma reference sample obtained by applying a downsampling filter centered on the position and the chroma reference sample corresponding to the position, or the sum of the absolute values ​​of the differences. In this way, the cost derived based on the sum of the absolute values ​​of the differences can be referred to as SAD (Sum of Difference).

[0201] Afterwards, the location with the lowest cost is determined as the optimal location, and the prediction parameter derivation process described below can be performed using the downsampled luma samples at the optimal location.

[0202] Alternatively, information indicating one of the multiple positions to which the downsampling filter can be applied can be encoded and signaled. For example, in the example illustrated in FIG. 19, an index indicating one of positions A to D can be encoded and signaled. To this end, the encoder can obtain prediction parameters for each of the multiple positions to which the downsampling filter can be applied, and can encode and signal an index indicating a position used to derive an optimal prediction parameter among the multiple prediction parameters. Here, the optimal prediction parameter can be derived by the cost of each of the prediction parameters or by Rate Distortion Optimization (RDO).

[0203] Meanwhile, determining the optimal downsampling position within the upper reference area may be independent of determining the optimal downsampling position within the left reference area. In this case, the optimal downsampling position within the upper reference area and the optimal downsampling position within the left reference area may differ.

[0204] Prediction parameters for a chroma block can be derived using downsampled luma reference samples and chroma block reference samples. The prediction parameters may include weights α and offsets β. The prediction parameters can be derived using the least squares method, for example.

[0205] Alternatively, the weight α offset β can be derived based on the linearity of the maximum and minimum values ​​of the downsampled luma reference samples and the maximum and minimum values ​​of the chroma reference samples.

[0206] At this time, prediction parameters can be derived using only chroma reference samples at predefined locations and corresponding downsampled luma reference samples. In this case, the process of deriving prediction parameters can be simplified, thereby reducing the complexity of the encoder and decoder. For example, prediction parameters can be derived using chroma reference samples at locations exemplified in the following mathematical expression 6.

[0207]

[0208] In the above example, W and H represent the width and height of the chroma block, respectively. According to the above example, prediction parameters can be derived using four chroma reference samples and four corresponding downsampled luma reference samples.

[0209] Prediction parameters can also be obtained using reference samples at different locations than in the example above. For example, the locations of the reference samples can be determined as in Equations 7 and 8 below.

[0210]

[0211]

[0212] After defining multiple candidates for the locations of reference samples, one of the multiple candidates can be selected. For example, each of the examples in Equations 6 to 8 listed above can be set as a location candidate, and then reference samples can be selected based on one of the multiple location candidates.

[0213] Information for selecting one of multiple location candidates may be encoded and signaled. For example, an index pointing to one of the multiple location candidates may be encoded and signaled.

[0214] Alternatively, one of the plurality of location candidates may be adaptively selected based on at least one of the size / shape of the current block, the color format, or whether the location of the chroma sample matches the location of the luma sample.

[0215] For example, if the current block is square, the prediction parameter can be derived using the location candidate of Equation 6. On the other hand, if the current block is non-square, the prediction parameter can be derived using the location candidate of Equation 7 or Equation 8. For example, if the current block is non-square in which the width is greater than the height, the location candidate of Equation 7 can be used, and if the current block is non-square in which the height is greater than the width, the location candidate of Equation 8 can be used.

[0216] Once the prediction parameters are derived, a prediction sample of a chroma block can be obtained based on the downsampled luma sample (S1520). For example, a prediction sample of a chroma block can be obtained according to the following mathematical expression 9.

[0217]

[0218] In mathematical expression 9, PredChroma represents a prediction sample of a chroma block, and Downsampled_Luma represents a downsampled luma sample at a position corresponding to the chroma prediction sample.

[0219] Meanwhile, if the video format is 4:4:4, the above-described downsampling process may be omitted. That is, if the video format is 4:4:4, the process of performing downsampling on the restored samples within the luma block and the process of performing downsampling on the reference samples of the luma block may be omitted.

[0220] As another example, regardless of the image format, the reference region of the luma block may not be subjected to a downsampling filter. That is, when deriving prediction parameters, instead of using the minimum and maximum values ​​of the downsampled luma reference samples, the minimum and maximum values ​​of the luma reference samples may be used.

[0221] Prediction samples for chroma prediction blocks can also be derived using filters (or convolutions) composed of more filter coefficients than those in Equation 9. Specifically, templates can be set around each of the luma block and chroma block, and prediction parameters can be derived based on the samples included in the templates. For convenience of explanation, the template for the chroma block will be referred to as the chroma template, and the template for the luma block will be referred to as the luma template.

[0222] Figure 20 illustrates templates for each chroma block and its corresponding luma block.

[0223] The previously restored region surrounding the chroma block can be set as a template for the chroma block. Furthermore, the previously restored region surrounding the luma block corresponding to the chroma block can be set as a template for the luma block. That is, each template may be composed of restored samples. Meanwhile, the samples within each template may be referred to as reference samples.

[0224] In the example illustrated in FIG. 20, the region including the upper four lines and the left four lines of each of the luma block and the chroma block is exemplified as being set as a template. Meanwhile, the template of the luma block and the template of the chroma block may have the same shape. Depending on the color format, the template of the chroma block may have the same size as the template of the luma block (e.g., 4:4:4 format), or the template of the chroma block may have a smaller size than the template of the luma block (e.g., 4:2:2 or 4:2:0 format).

[0225] Filter coefficients can be derived based on reference samples within a luma template and reference samples within a chroma template. Specifically, the filter coefficients can be derived so that the output values ​​when a filter is applied to reference samples within a luma template are close to the values ​​of the reference samples within the chroma template.

[0226] At this time, the filter applied to the reference samples in the luma template may be a convolution filter using a mask.

[0227] Figure 21 shows the filter form applied to reference samples of the luma component.

[0228] As in the example illustrated in Fig. 21, when a cross-shaped filter is applied, a value E obtained by filtering luma reference samples can be obtained according to the following mathematical expression 10.

[0229]

[0230] In Equation 10, [i, j] represents the coordinates (e.g., [x, y]) of the reference sample within the template. T represents the template. That is, '[i, j] ∈ T' means that the reference sample is included in the template (i.e., luma template or chroma template).

[0231] Y can represent a reference sample within a chroma template.

[0232] C represents the position of a reference sample to which a filter is applied within the luma template. Specifically, C may be a reference sample within the luma template, and may be located at a position corresponding to the reference sample Y within the chroma template. N, S, E, and W represent samples adjacent to the top, bottom, right, and left of the reference sample C within the luma template, respectively.

[0233] Variable B can be derived by bit depth. For example, variable B can be derived according to the following mathematical expression (11).

[0234]

[0235] In the above mathematical expression 11, D may represent the bit depth of the sample. For example, if the bit depth is 10 bits, the variable B may be derived as the median value 512. Alternatively, if the bit depth is 8 bits, the variable B may be derived as the median value 128.

[0236] In the above mathematical expression 10, filter coefficients (i.e., filter parameters) w0 to w5 that minimize the filtered value E can be derived. Meanwhile, regression analysis can be used to derive the filter coefficients.

[0237] Once the filter coefficients are derived, a filter can be applied to luma samples within a luma block to obtain a prediction sample within a chroma block. For example, assuming that the cross filter illustrated in FIG. 21 is applied, a prediction sample Y' within a chroma block can be derived according to the following mathematical expression 12.

[0238]

[0239] In mathematical expression 12, M denotes a luma block corresponding to a chroma block. That is, '[i, j]∈M' may mean that the sample at the position [i, j] belongs to the luma block. C' may represent a sample within the luma block that is at the same position as the prediction sample Y' of the chroma block. N', S', E', and W' represent samples adjacent to the top, bottom, right, and left of the sample C' within the luma block, respectively.

[0240] Meanwhile, the shape or type of the filter is not limited to the example illustrated in FIG. 21. For example, a horizontal 1D filter (e.g., a 1xN-shaped filter), a vertical 1D filter (e.g., an Nx1-shaped filter), or a square-shaped filter (e.g., an NxN-shaped filter) may be used to derive prediction samples within a chroma block using filter coefficients or derived filter coefficients. In the above example, N is a natural number greater than or equal to 1, and may be 3, 4, or 5.

[0241] The filter type can be adaptively determined based on at least one of the size or shape of the current block. For example, if the current block is a non-square shape with a width greater than its height, a horizontal 1D filter can be used to derive filter coefficients or prediction samples within the chroma block. Conversely, if the current block is a non-square shape with a height greater than its width, a vertical 1D filter can be used to derive filter coefficients or prediction samples within the chroma block.

[0242] Alternatively, information indicating one of multiple filter type candidates may be encoded and signaled.

[0243] Filter coefficients are derived using the previously restored regions around the luma and chroma blocks. Accordingly, filter coefficients can be derived in the decoder using the same method as in the encoder.

[0244] As another example, information about a filter coefficient set, including filter coefficients, may be encoded and signaled to a decoder. For example, an index indicating one of multiple filter coefficient set candidates may be encoded and signaled. Each filter coefficient set candidate may be composed of multiple filter coefficients.

[0245] Alternatively, the values ​​of the filter coefficients that make up the filter coefficient set may be explicitly encoded and signaled.

[0246] As in the example described above, filter coefficients can be derived and prediction blocks can be generated using convolution. The inter-component prediction mode based on convolution can also be referred to as the Cross-Component Convolution Model (CCCM).

[0247] In the above-described mathematical expressions 9 and 12, it is exemplified that a prediction block of a chroma block is obtained based on a type of parameter derived using reconstructed samples surrounding the luma block and chroma block. Here, the use of a type of parameter indicates that a single prediction parameter or a single filter is used across the entire region of the current block.

[0248] In this way, a method of predicting a chroma block using one type of prediction parameter or a single type filter can be called a single-model method.

[0249] Extending the above-described embodiments, it is also possible to obtain a prediction block of a chroma block from a luma block using multiple parameters or multiple filters (or multiple convolutions).

[0250] In this way, a prediction method of a chroma block using multiple parameters or multiple filters can be called a multi-mode method.

[0251] When a multi-model approach is applied, multiple parameters can be derived. Specifically, the reference samples of the luma block and the reference samples of the chroma block can be classified into multiple groups. Then, parameters can be derived from each of the multiple groups.

[0252] For example, the average Ym of reference samples of a luma block is derived. At this time, luma reference samples around a luma block that are smaller than the average Ym can be classified into a first group. Furthermore, chroma reference samples at the same position as the luma reference samples classified into the first group can also be classified into the first group. On the other hand, luma reference samples around a luma block that are equal to or larger than the average Ym can be classified into a second group. Furthermore, chroma reference samples at the same position as the luma reference samples classified into the second group can also be classified into the second group.

[0253] Thereafter, prediction parameters can be derived for each group. For example, a first parameter can be derived using luma reference samples and chroma reference samples belonging to group 1. For example, the first parameter may include a first weight α1 and a first offset β1 for linear prediction.

[0254] Additionally, a second time parameter can be derived based on the luma reference samples and chroma reference samples belonging to group 2. For example, the second parameter may include a second weight α2 and a second offset β2 for linear prediction.

[0255] Depending on the group to which the luma sample in the luma block belongs, one of multiple parameters may be selected. Here, the luma sample represents a sample in the luma block at the same location as the location to be predicted (i.e., the prediction target location) in the chroma block.

[0256] For example, if a luma sample in a luma block belongs to the first group (e.g., if the value of the luma sample is smaller than the average Ym), a prediction sample of a chroma block can be obtained using a first parameter derived based on reference samples of the first group. For example, if a luma sample belongs to the first group, a prediction sample of a chroma block can be obtained according to the following mathematical expression 13.

[0257]

[0258] On the other hand, if the luma sample in the luma block belongs to the second group (e.g., if the value of the luma sample is equal to or greater than the average Ym), the prediction sample of the chroma block can be obtained using the second parameter derived based on the reference samples of the second group. For example, if the luma sample belongs to the second group, the prediction sample of the chroma block can be obtained according to the following mathematical expression 14.

[0259]

[0260] In the above mathematical expressions 13 and 14, (x, y) may indicate a sample location to be predicted within a chroma block.

[0261] Meanwhile, in the above-described example, the first parameter and the second parameter may be filter coefficients for convolution-based prediction. For example, the first parameter derived based on reference samples belonging to the first group may include first filter coefficients (w0_1 to w5_1). In addition, the second parameter derived based on reference samples belonging to the second group may include second filter coefficients (w0_2 to w5_2).

[0262] If the luma sample in the luma block belongs to the first group, the prediction sample of the chroma block can be derived using the first filter coefficients according to Equation 12. On the other hand, if the luma sample in the luma block belongs to the second group, the prediction sample of the chroma block can be derived using the second filter coefficients according to Equation 12.

[0263] Alternatively, one of the first parameter and the second parameter may be a prediction parameter for performing inter-component prediction based on linearity (e.g., CCLM), and the other may be filter coefficients for performing inter-component prediction based on convolution (e.g., CCCM).

[0264] Meanwhile, when classifying the first and second groups, instead of the average value Ym of the luma reference samples, the average value of the luma samples within the luma block or the average value of the luma reference samples and the luma samples may be used.

[0265] Alternatively, the first group and the second group may be classified based on the sample value at the middle position, taking into account the distribution of the mean luma reference samples and / or luma samples.

[0266]

[0267] The convolution-based filter coefficient derivation method described above can also be used for single-component prediction, rather than inter-component prediction. That is, the convolution-based filter coefficient derivation method is applied to the surrounding reconstructed region of the current block to derive filter coefficients, and then a prediction value for the current block can be obtained based on the derived filter coefficients. The prediction value for the current block can be obtained based on extrapolation. Furthermore, prediction using the filter coefficients can be applied to at least one of the luma component and the chroma component.

[0268] In the embodiments described below, an extrapolation filter means that the output value of the filter is located outside a predetermined area. For example, an extrapolation filter indicates that the output location exists on the boundary of a rectangular area containing input samples input to the extrapolation filter, or that the output location exists outside the input samples input to the extrapolation filter.

[0269] Figure 22 shows an example of obtaining a prediction sample based on an extrapolation filter.

[0270] As in the example illustrated in Fig. 22, an extrapolation filter may be utilized to obtain a prediction sample for the current block. Specifically, at least one of a reconstructed sample outside the current block and a previously predicted sample within the current block may be input into the extrapolation filter to derive a prediction sample within the current block.

[0271] The extrapolation filter illustrated in Fig. 22 is exemplified as outputting one sample value in response to eight input samples. That is, the lower right corner of the 3x3 area represents the location of the target sample to be predicted, and the remaining area represents the locations of the samples input to the extrapolation filter.

[0272] Accordingly, as in the example shown in (a) of Fig. 22, when the sample to be predicted is located at the upper left corner of the current block, the predicted sample can be obtained by inputting only the restored samples around the current block into the extrapolation filter.

[0273] On the other hand, as in the example shown in (b) of Fig. 22, if the sample to be predicted is located in the first or second row from the upper boundary of the current block, or in the first or second column from the left boundary of the current block, the predicted sample can be obtained by inputting the reconstructed sample around the current block and the previously predicted sample within the current block into the extrapolation filter.

[0274] As in the example shown in (c) of Fig. 22, when a sample to be predicted is included in the residual area within the current block, a predicted sample can be obtained by inputting only the pre-predicted samples within the current block into the extrapolation filter.

[0275] Mathematical expression 15 shows an example of obtaining a prediction sample based on the extrapolation filter illustrated in Fig. 22.

[0276]

[0277] In Equation 15, P[x, y] represents the predicted sample at the location [x, y], i.e., the output of the extrapolation filter. I represents the input value of the filter. xpos represents the x-axis coordinate difference between the output value of the filter and the input value of the filter, and ypos represents the y-axis coordinate difference between the output value of the filter and the input value of the filter.

[0278] Referring to the example illustrated in Fig. 22, xpos and ypos can each fall within the range of 0 to 2. Meanwhile, when both xpos and ypos are 0, it indicates the location of the target sample to be predicted. Accordingly, samples where both xpos and ypos are 0 cannot be input to the extrapolation filter.

[0279] N represents the number of input samples input to the extrapolation filter, and C[n] represents the filter coefficients (or weights or parameters) for each input sample. The number of filter coefficients may be equal to the number of input samples.

[0280] The shape or size of the extrapolation filter is not limited to the illustrated example. For example, a prediction sample can be derived by applying an extrapolation filter to a 4x4 area. Here, applying an extrapolation filter to a 4x4 area means that three samples are input and one prediction sample is output.

[0281] Alternatively, an extrapolation filter can be applied to a rectangular region with different widths and heights. Alternatively, a one-dimensional extrapolation filter (e.g., a horizontal filter (Nx1) or a vertical filter (1xN)) can be used to obtain prediction samples.

[0282] Alternatively, the sample location to be predicted through the extrapolation filter can be set to a location different from that shown.

[0283] Figure 23 illustrates various forms of extrapolation filters.

[0284] In the example illustrated in (a) of Fig. 23, the target sample for prediction is depicted as being located at the lower right corner within a region of the form (w+1)x(h+1). The remaining region excluding the lower right corner represents the locations of the samples input to the extrapolation filter. That is, the extrapolation filter can have a number of taps equal to ((w+1)x(h+1)-1).

[0285] Meanwhile, in the example illustrated in (a) of Fig. 23, the values ​​of w and h may be the same or different. In this case, unlike the example illustrated in Fig. 22, the value (w+1)x(h+1) may be set to an exponent of 2 (e.g., 2, 4, 8, 16, or 32).

[0286] In the example shown in (b) of Fig. 23, samples within the (wxh) region are used as input values ​​of the extrapolation filter, and the target sample is exemplified as being located at the bottom of the (wxh) region.

[0287] Meanwhile, the target sample for prediction can be located at the center of the bottom of the (wxh) region. This indicates that w0 has the same value as w1.

[0288] Alternatively, the target prediction sample can be located at the bottom left of the (wxh) region, indicating that w0 is set to 0.

[0289] Alternatively, the target sample for prediction can be located at the lower left of the (wxh) region, indicating that the size of w1 has the same value as w.

[0290] Alternatively, the target prediction sample can be located at the bottom right of the (wxh) region, indicating that w1 is set to 0.

[0291] Alternatively, the target sample for prediction can be located at the lower right corner of the (wxh) region, indicating that w0 has the same value as w.

[0292] That is, each of w0 and w1 can fall within the range from -1 to w.

[0293] On the other hand, in the example shown in (c) of Fig. 23, it is illustrated that samples within the (wxh) region are used as input values ​​of the extrapolation filter, and the prediction target sample is located on the right side of the (wxh) region.

[0294] Meanwhile, the target sample for prediction can be located in the center of the right side of the (wxh) region. This indicates that h0 has the same value as h1.

[0295] Alternatively, the target sample for prediction can be located at the upper right of the (wxh) region, indicating that h0 is set to 0.

[0296] Alternatively, the target sample for prediction may be located at the upper right of the (wxh) region, indicating that h1 has the same value as h.

[0297] Alternatively, the target prediction sample can be located at the bottom right of the (wxh) region, indicating that h1 is set to 0.

[0298] Alternatively, the target sample for prediction can be located at the lower right of the (wxh) region, indicating that h0 has the same value as h.

[0299] That is, each of h0 and h1 can fall within the range from 0 to h.

[0300] In the examples shown in (b) and (c) of Fig. 23, the values ​​of w and h may be the same or different. In this case, the (wxh) value may be set to an exponent of 2 (e.g., 2, 4, 8, 16, or 32).

[0301] Depending on the shape or number of taps of the extrapolation filter, the maximum values ​​of xpos and / or ypos in Equation 15 may be set differently.

[0302] The shape and / or number of taps of the extrapolation filter may be predefined in the encoder and decoder.

[0303] Alternatively, one of a plurality of extrapolation filter candidates may be applied to the current block. At this time, the extrapolation filter candidates may differ in at least one of the following: the number of filter taps, the filter shape, the filter coefficients, or the filter output position. Meanwhile, an index identifying the extrapolation filter applied to the current block among the plurality of extrapolation filter candidates may be encoded and signaled.

[0304] Alternatively, depending on the sample location to be predicted, the shape of the extrapolation filter, the number of taps, or the output location of the filter may be set differently. For example, if the sample to be predicted is located at the upper left corner of the current block, the extrapolation filter of the shape shown in (a) of Fig. 23 may be used. On the other hand, if the sample to be predicted is located at the top of the current block, the extrapolation filter of the shape shown in (b) of Fig. 23 may be used, and if the sample to be predicted is located at the left side of the current block, the extrapolation filter of the shape shown in (c) of Fig. 23 may be used.

[0305] Filter coefficients for the extrapolation filter can be derived by referring to the restoration area around the current block, i.e., the template.

[0306] As another example, at least one of the size or shape of the filter may be adaptively determined based on at least one of the size or shape of the current block. For example, the value of a variable (i.e., at least one of w, h, w0, w1, h0, or h1) that determines the shape and size of the filter illustrated in FIG. 23 may be determined differently depending on at least one of the size or shape of the current block. For example, if the current block is a rectangular shape in which the width is longer than the height, a rectangular filter in which the width is longer than the height may be used. On the other hand, if the current block is a square shape in which the width and height are equal, a square filter in which the width and height are equal may be used.

[0307] Figure 24 shows a restoration area around the current block for deriving filter coefficients of an extrapolation filter.

[0308] As in the example illustrated in FIG. 24, the restoration area may be configured to include at least one of an upper restoration area located at the top of the current block and a left restoration area located at the left of the current block.

[0309] Depending on the extrapolation filter candidate, the restoration area around the current block can be adaptively determined. For example, as in the example illustrated in (a) of Fig. 23, when an extrapolation filter is used in which the lower right corner of a (wxh) sized area is the predicted target location, the restoration area can be configured as illustrated in (a) of Fig. 24.

[0310] On the other hand, if the predicted target location exists at the bottom of an area of ​​size (wxh), as in the example shown in (b) of Fig. 23, the restoration area can be configured to include only the areas adjacent to the top of the current block, or the areas adjacent to the top and left of the current block, as in (b), (c) or (d) of Fig. 24.

[0311] Alternatively, as in the example shown in (c) of Fig. 23, when the predicted target location exists on the right side of the area of ​​size (wxh), the restoration area may be configured to include only the areas adjacent to the left side of the current block, or the areas adjacent to the top and left side of the current block, as in (d), (e) or (f) of Fig. 24.

[0312] Alternatively, one of multiple restoration area candidates may be selected. In this case, an index indicating at least one of the multiple restoration area candidates may be encoded.

[0313] Alternatively, the configuration of the restoration area may differ based on whether the current block touches the boundary of a CTU.

[0314] For example, if the upper boundary of the current block touches the boundary of the CTU, the restoration area can be configured using only the left restoration area. Conversely, if the upper boundary of the current block does not touch the boundary of the CTU, the restoration area can be configured to include the upper restoration area and the left restoration area.

[0315] By applying an extrapolation filter to the restoration region and comparing the output value of the extrapolation filter with the value of the corresponding restoration sample at the output location of the extrapolation filter, the filter coefficients of the extrapolation filter can be derived. For example, assuming that the extrapolation filter illustrated in (a) of Fig. 23 is applied, the filter coefficients of the extrapolation filter can be derived by modifying Equation 10 as follows.

[0316]

[0317] In Equation 16, [i, j] represents the template, i.e., the coordinates (e.g., [x, y]) of the reference sample within the restoration region. T represents the template. That is, '[i, j] ∈ T' means that the reference sample is included in the template.

[0318] Y represents the value of the restored sample at the output location of the extrapolation filter.

[0319] R0 to R7 represent samples input to the extrapolation filter within the restoration region.

[0320] Variable B can be derived by bit depth. For example, variable B can be derived according to Equation 11.

[0321] In the above mathematical expression 16, filter coefficients (i.e., filter parameters) w0 to w8 that minimize the sum E of the differences between the restored value and the output value of the extrapolation filter can be derived. Meanwhile, regression analysis can be used to derive the filter coefficients.

[0322] Meanwhile, information indicating whether prediction is performed on the current block can be encoded and signaled using an extrapolation filter. The information can be a 1-bit flag.

[0323] The above information can be encoded and signaled when the encoding mode of the current block is intra mode.

[0324] Alternatively, after determining the intra prediction mode of the current block, information indicating whether an extrapolation filter is applied to the current block can be encoded and signaled.

[0325] Alternatively, information indicating whether an extrapolation filter is applied to the current block may be encoded and signaled only if the intra prediction mode of the current block is a predefined intra prediction mode. Here, the predefined intra prediction mode may include at least one of a DC mode, a Planar mode, or a predefined directional prediction mode.

[0326] When an extrapolation filter is applied to the current block, the current block can be predicted using the extrapolation filter without performing intra prediction based on the determined intra prediction mode. In other words, the extrapolation filter can replace the intra prediction mode.

[0327] Alternatively, when an extrapolation filter is applied to the current block, the final prediction block of the current block can be obtained based on a first prediction block generated by performing intra prediction based on an intra prediction mode and a second prediction block generated using an extrapolation filter. For example, the final prediction block of the current block can be obtained based on an average or weighted sum operation of the first prediction block and the second prediction block.

[0328] Alternatively, prediction based on an extrapolation filter may be performed only on a portion of the current block. For example, prediction based on an extrapolation filter may be performed only on locations bordering the top boundary and / or the left boundary of the current block. Accordingly, at locations bordering the top boundary and / or the left boundary of the current block, a final prediction sample may be derived based on an average or weighted sum operation of a first prediction sample obtained based on an intra prediction mode and a second prediction sample obtained based on an extrapolation filter. On the other hand, at locations that do not border the top boundary and / or the left boundary of the current block, a prediction sample obtained based on an intra prediction mode may be set as the final prediction sample.

[0329] Alternatively, the prediction sample derivation method may be set differently based on whether the intra prediction mode is a predefined intra prediction mode. For example, if the intra prediction mode is a predefined intra prediction mode, the final prediction sample may be obtained by averaging or weighting the first prediction sample obtained based on the intra prediction mode and the second prediction sample obtained using an extrapolation filter. On the other hand, if the intra prediction mode is not a predefined intra prediction mode, the prediction sample obtained using an extrapolation filter may be set as the final prediction sample without performing intra prediction based on the intra prediction mode.

[0330] If the extrapolation filter is not applied, a prediction sample of the current block can be obtained based on the determined intra prediction mode.

[0331] After determining the intra prediction mode, if it is determined whether to apply an extrapolation filter, the shape and / or number of taps of the extrapolation filter may be determined depending on the intra prediction mode. For example, depending on the intra prediction mode, one of the extrapolation filter candidates illustrated in (a), (b), and (c) of FIG. 23 may be selected. For example, if the intra prediction mode of the current block is an intra prediction mode in the upper left diagonal direction, the extrapolation filter illustrated in (a) of FIG. 23 may be used. On the other hand, if the intra prediction mode of the current block is an intra prediction mode in the vertical direction (i.e., number 50), the extrapolation filter illustrated in (b) of FIG. 23 may be used, and if the intra prediction mode of the current block is an intra prediction mode in the horizontal direction (i.e., number 18), the extrapolation filter illustrated in (c) of FIG. 23 may be used.

[0332] Alternatively, in the example illustrated in FIG. 23, the values ​​of w0 and w1 or h0 and h1 may be determined differently depending on the intra prediction mode. For example, when the vertical intra prediction mode is used, the filter illustrated in (b) of FIG. 23 is used, and in this case, the output sample may be located at the bottom center of the 3x3 area (i.e., the value of w0 is equal to the value of w1).

[0333] On the other hand, when the upper left diagonal intra prediction mode is used, the filter shown in (b) of Fig. 23 is used, and at this time, the output sample can be located at the lower right of the 3x3 area (i.e., the value of w0 is equal to w).

[0334] Alternatively, a prediction sample can be derived by adding an offset to the output of Equation 15. Equation 17 shows another example of deriving a prediction sample using an extrapolation filter.

[0335]

[0336] In the above mathematical expression 17, variable B represents a sample value according to bit depth. For example, variable B may represent an intermediate value of bit depth, as in the example of mathematical expression 11.

[0337] Alternatively, variable B can be set to the average of the input values ​​input to the extrapolation filter or the average of the samples within the restoration region.

[0338] Alternatively, variable B can be set to the average of reference samples adjacent to the current block.

[0339] Alternatively, information pointing to variable B can be encoded and signaled.

[0340] Comparing Equations 15 and 17, it can be seen that the number of taps of the extrapolation filter increases by 1 as variable B is added. That is, in Equation 15, N filter coefficients are used, but in Equation 17, (N+1) filter coefficients are used.

[0341] To keep the number of taps of the extrapolation filter as N, Equation 17 can be transformed into Equation 18 below.

[0342]

[0343] Mathematical expression 18 represents an example in which one of the input values ​​of the preset positions input to Mathematical expressions 15 and 17 is excluded. In this case, the position not input to the extrapolation filter may be located in the opposite direction to the output position of the extrapolation filter.

[0344] Figure 25 illustrates locations excluded from the input of the extrapolation filter.

[0345] As in the example shown in (a) of Fig. 24, when the output position of the extrapolation filter is located at the corner of a rectangular area, samples located in the diagonal direction from the output position of the extrapolation filter can be excluded from the input.

[0346] Alternatively, as in the example shown in (b) of Fig. 24, when the output location of the extrapolation filter is located at the right center of a rectangular area, samples located at the left center within the rectangular area can be excluded from the input.

[0347] Alternatively, as in the example shown in (c) of Fig. 24, if the output location of the extrapolation filter is located at the bottom center of a rectangular area, the sample located at the top center within the rectangular area can be excluded from the input.

[0348] The values ​​output through the extrapolation filter can be clipped and the clipped values ​​can be set as prediction samples. Equation 19 shows an example of obtaining prediction samples through clipping.

[0349]

[0350] In Equation 19, m represents the minimum clipping value, and M represents the maximum clipping value. In addition, v represents the input value of the clipping function. For example, the output value P[x, y] of Equation 15, Equation 17, or Equation 18 may be the input value of the clipping function. V represents the output value of the clipping function.

[0351] If the output value P[x, y] of the extrapolation filter exists between m and M, the output value of the extrapolation filter can be set as the prediction sample as is. On the other hand, if the output value P[x, y] of the extrapolation filter is smaller than m, the minimum value m can be set as the prediction sample. Alternatively, if the output value P[x, y] of the extrapolation filter is larger than M, the maximum value M can be set as the prediction sample.

[0352] The minimum value m and maximum value M of the clipping function can be derived from the restoration region. For example, the minimum value of samples within the restoration region can be set to m, and the maximum value of samples within the restoration region can be set to M.

[0353] Alternatively, the minimum value m and the maximum value M may be preset values. m may be set to the minimum value among the values ​​expressible by the bit depth, and M may be set to the maximum value among the values ​​expressible by the bit depth. For example, when the bit depth is 10 bits, m may be set to 0 and M may be set to 1023.

[0354] Alternatively, information indicating the minimum value m and the maximum value M may be encoded and signaled. The information may be encoded and signaled for each predetermined unit. Here, the predetermined unit may be at least one of a coding unit, a set of coding units (e.g., a CTU), or a set of coding tree units. Alternatively, the predetermined unit may be encoded and signaled for each unit in which parallel processing is performed, for example, for each slice or tile.

[0355] Alternatively, multiple minimum-maximum value pairs can be encoded and signaled on a picture or slice basis. Subsequently, for each predetermined unit, index information identifying one of the multiple minimum-maximum value pairs can be encoded and signaled.

[0356]

[0357] The names of the syntaxes used in the above-described embodiments are merely named for convenience of explanation.

[0358] Applying the embodiments described above, focusing on the decoding or encoding process, to the encoding or decoding process is within the scope of the present disclosure. Changing the embodiments described above, in a given order, to a different order is also within the scope of the present disclosure.

[0359] Although the above-described disclosure is described based on a series of steps or a flowchart, this does not limit the chronological order of the invention, and may be performed simultaneously or in a different order as needed. In addition, each component (e.g., unit, module, etc.) constituting the block diagram in the above-described disclosure may be implemented as a hardware device or software, or multiple components may be combined to be implemented as a single hardware device or software. For example, the hardware device may include at least one of a processor for performing calculations, a memory for storing data, a transmitter for transmitting data, and a receiver for receiving data.

[0360] The above-described disclosure may be implemented in the form of program commands that can be executed by various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, etc., either singly or in combination.

[0361] In addition, according to the present disclosure, a computer-readable recording medium can be provided that stores a bitstream generated by the above-described encoding method. The bitstream can be transmitted by an encoding device, and a decoding device can receive the bitstream and decode an image.

[0362] Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program instructions, such as ROMs, RAMs, and flash memories. The hardware devices may be configured to operate as one or more software modules to perform processing according to the present disclosure, and vice versa.

[0363] The present disclosure may be applied to a computing or electronic device capable of encoding / decoding a video signal.

Claims

1. A step of deriving filter coefficients for applying an extrapolation filter; and A step of inputting multiple samples into the extrapolation filter to obtain a prediction sample for the current block, An image decoding method, characterized in that the filter coefficients are derived by applying the extrapolation filter to a template area around the current block.

2. In paragraph 1, A video decoding method, characterized in that, depending on the location of the predicted sample, it is determined whether a sample predicted within the current block is included among the plurality of samples input to the extrapolation filter.

3. In paragraph 2, If the above prediction sample is located at the upper or right border of the current block, the plurality of samples are composed only of reference samples outside the current block, A method for decoding an image, characterized in that, if the predicted sample is not located at the upper or right boundary of the current block, the plurality of samples are configured to include a pre-predicted sample within the current block.

4. In paragraph 2, An image decoding method, characterized in that the extrapolation filter is one of a plurality of extrapolation filter candidates.

5. In paragraph 4, A video decoding method, characterized in that one of the plurality of extrapolation filter candidates is selected by an index decoded from a bitstream.

6. In paragraph 4, An image decoding method, wherein each of the plurality of extrapolation filter candidates is different in at least one of a filter shape, a number of filter taps, or an output position of a filter value.

7. In paragraph 4, An image decoding method, characterized in that one of the plurality of extrapolation filter candidates is selected based on the intra prediction mode of the current block.

8. In paragraph 4, An image decoding method, characterized in that the plurality of extrapolation filter candidates include at least one of a first extrapolation filter candidate having an output location in the lower right corner of a rectangular area, a second extrapolation filter candidate having an output location in the right outer area of ​​the rectangular area, or a third extrapolation filter candidate having an output location in the lower outer area of ​​the rectangular area.

9. In paragraph 1, An image decoding method, characterized in that the extrapolation filter is applied only to prediction samples located at the upper or left boundary of the current block.

10. In paragraph 9, An image decoding method, characterized in that a final prediction sample is obtained based on a first prediction sample obtained using an extrapolation filter and a second prediction sample obtained based on an intra prediction mode at the upper or left boundary of the current block.

11. In paragraph 10, An image decoding method, characterized in that, at a location not adjacent to the upper or left border of the current block, a second prediction sample obtained based on the intra prediction mode is set as a final prediction sample.

12. In paragraph 1, A method for decoding an image, characterized in that the final prediction sample for the current block is obtained based on an average or weighted sum operation between the prediction sample obtained by applying the extrapolation filter and an additional prediction sample obtained based on the intra prediction mode of the current block.

13. In paragraph 1, An image decoding method, characterized in that the extrapolation filter is available only when the encoding mode of the current block is intra prediction.

14. A step of deriving filter coefficients for applying an extrapolation filter; and A step of inputting multiple samples into the extrapolation filter to obtain a prediction sample for the current block, A video encoding method, characterized in that the filter coefficients are derived by applying the extrapolation filter to a template area around the current block.

15. A step of deriving filter coefficients for applying an extrapolation filter; and A step of inputting multiple samples into the extrapolation filter to obtain a prediction sample for the current block, A computer-readable recording medium storing a bitstream generated by an image encoding method, characterized in that the filter coefficients are derived by applying the extrapolation filter to a template area around the current block.

Images