Image encoding / decoding method and apparatus for transmitting compressed video data

Abstract

An image decoding method according to the present disclosure may comprise the steps of: acquiring a prediction block of the current block; acquiring a residual block of the current block by applying inverse transform to the current block; and reconstructing the current block by adding the prediction block and the residual block. Here, the inverse transform includes second inverse transform for second transform and first inverse transform for first transform, a second transform kernel set for the second inverse transform is selected on the basis of an intra prediction mode of the current block, and when an encoding mode of the current block is not intra prediction, the intra prediction mode of the current block can be derived by referring to the prediction block of the current block or an area previously reconstructed before the current block.

Description

Video encoding / decoding method and device for transmitting compressed video data

[0001] The present disclosure relates to a video signal processing method and device.

[0002] Recently, the demand for high-resolution, high-quality images, such as HD (High Definition) and UHD (Ultra High Definition) images, is increasing across various application fields. As image data becomes higher in resolution and quality, the relative amount of data increases compared to conventional image data. Therefore, transmitting image data using existing media such as wired and wireless broadband lines or storing it using existing storage media leads to increased transmission and storage costs. To address these issues arising from the increasing resolution and quality of image data, high-efficiency image compression technologies can be utilized.

[0003] There are various technologies for image compression, such as inter-picture prediction technology that predicts pixel values ​​included in the current picture from pictures before or after the current picture, intra-picture prediction technology that predicts pixel values ​​included in the current picture using pixel information in the current picture, and entropy encoding technology that assigns short codes to values ​​with high frequency of appearance and long codes to values ​​with low frequency of appearance. Using these image compression technologies, image data can be effectively compressed and transmitted or stored.

[0004] Meanwhile, as demand for high-resolution video grows, so does the demand for stereoscopic video content as a new video service. Discussions are underway on video compression technologies to effectively deliver high-resolution and ultra-high-resolution stereoscopic video content.

[0005] The present disclosure aims to provide a method for deriving an intra prediction mode of a block to which intra prediction is not applied and a device therefor.

[0006] The present disclosure aims to provide a method for determining a transform kernel set of a block to which intra prediction is not applied and a device therefor.

[0007] The present disclosure aims to provide a method and a device for performing a second transformation / second inverse transformation on a block to which a partial transformation has been applied.

[0008] The technical problems to be achieved in the present disclosure are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by a person having ordinary skill in the technical field to which the present disclosure belongs from the description below.

[0009] A video decoding method according to the present disclosure may include the steps of: obtaining a prediction block of a current block; applying an inverse transform to the current block to obtain a residual block of the current block; and reconstructing the current block by combining the prediction block and the residual block. In this case, the inverse transform includes a second inverse transform for a second transform and a first inverse transform for a first transform, and a second transform kernel set for the second inverse transform is selected based on an intra prediction mode of the current block, and when the encoding mode of the current block is not intra prediction, the intra prediction mode of the current block may be derived by referring to the prediction block of the current block or a previously reconstructed region prior to the current block.

[0010] In the image decoding method according to the present disclosure, the intra prediction mode can be derived from a histogram derived from a reference block of the current block.

[0011] In the image decoding method according to the present disclosure, when the encoding mode is inter prediction, the reference block can be indicated by a motion vector of the current block within a reference picture of the current block.

[0012] In the video decoding method according to the present disclosure, when the encoding mode is intra block copy, the reference block can be indicated by a block vector of the current block in the current picture.

[0013] In the image decoding method according to the present disclosure, the histogram may be in the form of an accumulated amplitude value of each intra prediction mode.

[0014] In the image decoding method according to the present disclosure, the amplitude value can be derived based on the horizontal slope and the vertical slope of the reference sample within the reference block.

[0015] In the image decoding method according to the present disclosure, when a partial transformation is applied to the current block, the histogram may be derived from a portion of the reference block, and the portion of the reference block may correspond to a transformation target portion of the current block.

[0016] In the video decoding method according to the present disclosure, the intra prediction mode may be one having the lowest cost among a plurality of intra prediction modes.

[0017] In the image decoding method according to the present disclosure, the cost of each of the plurality of intra prediction modes can be derived based on the difference between a temporary prediction block obtained by performing intra prediction on the current block and the prediction block.

[0018] In the image decoding method according to the present disclosure, the cost of each of the plurality of intra prediction modes can be derived based on the difference between a temporary prediction block obtained by performing intra prediction on a reference region around the current block and the reference region.

[0019] In the image decoding method according to the present disclosure, when a prediction mode combining intra prediction and inter prediction is applied to the current block, the prediction block may be obtained by weighting an intra prediction block and an inter prediction block, and the intra prediction mode may be used to obtain the intra prediction block of the current block.

[0020] In the image decoding method according to the present disclosure, when the encoding mode of the current block is intra prediction and the intra prediction mode is a planar mode, the second transform kernel set can be determined according to the type of the planar mode.

[0021] In the image decoding method according to the present disclosure, when the type of the planar mode is a vertical direction planar mode, the second transformation kernel set may be selected by considering that the intra prediction mode of the current block is a vertical direction mode, and when the type of the planar mode is a horizontal direction planar mode, the second transformation kernel set may be selected by considering that the intra prediction mode of the current block is a horizontal direction mode.

[0022] A video encoding method according to the present disclosure may include the steps of: obtaining a prediction block of a current block; obtaining a residual block of the current block based on the prediction block; and applying a transform to the residual block. In this case, the transform includes a first transform and a second transform, and a second transform kernel set for the second transform is selected based on an intra prediction mode of the current block, and when the encoding mode of the current block is not intra prediction, the intra prediction mode of the current block may be derived by referring to the prediction block of the current block or a previously reconstructed area prior to the current block.

[0023] According to the present disclosure, a computer-readable recording medium having recorded thereon a command for storing / transmitting a bitstream generated by an image encoding method can be provided.

[0024] According to the present disclosure, a computer-readable recording medium having recorded thereon a command for performing an image decoding method or an image encoding method can be provided.

[0025] The features briefly summarized above regarding the present disclosure are merely exemplary aspects of the detailed description of the present disclosure that follows and do not limit the scope of the present disclosure.

[0026] According to the present disclosure, encoding / decoding efficiency can be improved by providing a method for deriving an intra prediction mode of a block to which intra prediction is not applied.

[0027] According to the present disclosure, encoding / decoding efficiency can be improved by providing a method for determining a transform kernel set of a block to which intra prediction is not applied.

[0028] According to the present disclosure, encoding / decoding efficiency can be improved by providing a method for performing a second transformation / second inverse transformation on a block to which a partial transformation has been applied.

[0029] The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects that are not mentioned will be clearly understood by a person having ordinary skill in the art to which the present disclosure pertains from the description below.

[0030] FIG. 1 is a block diagram illustrating an image encoding device according to an embodiment of the present disclosure.

[0031] FIG. 2 is a block diagram showing an image decoding device according to an embodiment of the present disclosure.

[0032] FIG. 3 illustrates an image encoding / decoding method performed by an image encoding / decoding device according to the present disclosure.

[0033] FIG. 4 illustrates an example of multiple intra prediction modes according to the present disclosure.

[0034] Figure 5 shows an example of an extended directional mode.

[0035] FIG. 6 illustrates an intra prediction method based on a planar mode according to the present disclosure.

[0036] FIG. 7 illustrates an intra prediction method based on DC mode according to the present disclosure.

[0037] FIG. 8 illustrates an intra prediction method based on a directional mode according to the present disclosure.

[0038] Figure 9 illustrates a method for deriving samples of fractional positions.

[0039] Figures 10 and 11 illustrate that the tangent value for the angle is scaled by a factor of 32 for each intra prediction mode.

[0040] Figure 12 is a diagram illustrating an intra prediction aspect when the directional mode is one of modes 34 to 49.

[0041] Figure 13 is a drawing for explaining an example of generating an upper reference sample by interpolating left reference samples.

[0042] Figure 14 shows an example in which intra prediction is performed using reference samples arranged in a 1D array.

[0043] Figure 15 is a drawing for explaining an example of setting a reference area.

[0044] Figure 16 is a drawing showing an example of the configuration of a reference area.

[0045] Figure 17 illustrates filter coefficients for each of the Sobel mask and the Prewitt mask.

[0046] Figure 18 shows the locations where the vertical and horizontal slopes are obtained within the reference area.

[0047] Figure 19 shows an example of grouping directional modes into multiple intra prediction mode groups.

[0048] Figure 20 is a drawing illustrating a reference area around the current block.

[0049] Figure 21 shows an example of performing intra prediction for a reference region based on the planar mode.

[0050] Figure 22 shows an example of performing intra prediction for a reference region based on the DC mode.

[0051] Figure 23 shows an example of performing intra prediction for a reference region based on a directional mode.

[0052] Figure 24 is a diagram schematically illustrating the process of performing inter prediction in an encoder and decoder.

[0053] Figure 25 shows an example in which motion estimation is performed.

[0054] Figures 26 and 27 illustrate examples in which a prediction block of a current block is generated based on motion information generated through motion estimation.

[0055] Figure 28 shows the locations referenced to derive motion vector prediction values.

[0056] Figure 29 is a flowchart of a method for encoding a residual block in an encoder.

[0057] Figure 30 is a flowchart of a method for restoring a residual block in a decoder.

[0058] Figures 31 and 32 are drawings showing examples in which the second transformation is applied.

[0059] Figures 33 and 34 illustrate a second transformation based on a second transformation kernel of an asymmetric shape.

[0060] Figure 35 shows an example in which information on partial transformations is sequentially encoded / decoded.

[0061] Figure 36 shows an example of a case where a partial transformation is applied to the current block.

[0062] Figure 37 shows an example of deriving an intra prediction mode of a current block using a reference block of the current block.

[0063] Figure 38 illustrates an example of deriving an intra prediction mode of a current block by performing multiple intra predictions on the current block.

[0064] The present disclosure may be modified in various ways and encompasses numerous embodiments. Specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present disclosure to specific embodiments, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and technical scope of the present disclosure. Similar reference numerals have been used to designate similar components throughout the description of each drawing.

[0065] While terms such as "first" and "second" may be used to describe various components, these components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present disclosure, a first component could be referred to as a "second component," and similarly, a second component could also be referred to as a "first component." The term "and / or" includes a combination of multiple related items described herein or any of multiple related items described herein.

[0066] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0067] The terminology used in this application is only used to describe specific embodiments and is not intended to limit the present disclosure. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprise" or "have" indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0068] Hereinafter, preferred embodiments of the present disclosure will be described in more detail with reference to the attached drawings. Hereinafter, identical components in the drawings will be designated by the same reference numerals, and redundant descriptions of identical components will be omitted.

[0069] FIG. 1 is a block diagram illustrating an image encoding device according to an embodiment of the present disclosure.

[0070] Referring to FIG. 1, a video encoding device (100) may include a picture segmentation unit (110), a prediction unit (120, 125), a transformation unit (130), a quantization unit (135), a reordering unit (160), an entropy encoding unit (165), an inverse quantization unit (140), an inverse transformation unit (145), a filter unit (150), and a memory (155).

[0071] Each component shown in Fig. 1 is independently depicted to represent different characteristic functions in the video encoding device, and does not mean that each component is composed of separate hardware or a single software component. That is, each component is listed and included as a separate component for convenience of explanation, and at least two components among each component may be combined to form a single component, or one component may be divided into multiple components to perform a function, and such integrated and separate embodiments of each component are also included in the scope of the present disclosure as long as they do not deviate from the essence of the present disclosure.

[0072] Additionally, some components may not be essential components that perform the essential functions of the present disclosure, but may be optional components merely used to enhance performance. The present disclosure may be implemented by including only components essential to implementing the essence of the present disclosure, excluding components used solely for performance enhancement. A structure that includes only essential components, excluding optional components used solely for performance enhancement, is also within the scope of the present disclosure.

[0073] The picture splitting unit (110) can split the input picture into at least one processing unit. At this time, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). The picture splitting unit (110) can split one picture into a combination of multiple coding units, prediction units, and transform units, and select one combination of coding units, prediction units, and transform units based on a predetermined criterion (e.g., a cost function) to encode the picture.

[0074] For example, a picture can be split into multiple coding units. A recursive tree structure such as a quad tree, a ternary tree, or a binary tree can be used to split a coding unit in a picture. A coding unit that is split into other coding units starting from an image or the largest coding unit as the root can be split into as many child nodes as the number of split coding units. A coding unit that cannot be split any further according to a certain restriction becomes a leaf node. For example, assuming that a quad tree split is applied to a coding unit, a coding unit can be split into at most four different coding units.

[0075] Hereinafter, in the embodiments of the present disclosure, the encoding unit may be used to mean a unit that performs encoding or may be used to mean a unit that performs decoding.

[0076] A prediction unit may be divided into at least one square or rectangular shape of the same size within a single coding unit, or may be divided such that one prediction unit among the divided prediction units within a single coding unit has a different shape and / or size from another prediction unit.

[0077] When predicting within a screen, the transformation unit and the prediction unit can be set to be the same. In this case, the encoding unit can be divided into multiple transformation units, and then intra-screen prediction can be performed for each transformation unit. The encoding unit can be divided in the horizontal direction or the vertical direction. The number of transformation units generated by dividing the encoding unit can be 2 or 4, depending on the size of the encoding unit. Alternatively, when the size of the transformation unit is small, multiple transformation units can be set as a single prediction unit.

[0078] The prediction unit (120, 125) may include an inter-prediction unit (120) that performs inter-prediction and an intra-prediction unit (125) that performs intra-prediction. It may be determined whether to use inter-prediction or intra-prediction for an encoding unit, and specific information (e.g., reference sample line, intra-prediction mode, motion vector, reference picture, etc.) according to each prediction method may be determined. At this time, the processing unit where prediction is performed and the processing unit where the prediction method and specific contents are determined may be different. For example, the prediction method and prediction mode, etc. are determined in the encoding unit, and the prediction may be performed in the prediction unit or the transformation unit. The residual value (residual block) between the generated prediction block and the original block may be input to the transformation unit (130). In addition, the prediction mode information, motion vector information, etc. used for prediction may be encoded together with the residual value in the entropy encoding unit (165) and transmitted to the decoding device. When using a specific encoding mode, it is also possible to encode the original block as is and transmit it to the decoding unit without generating a prediction block through the prediction unit (120, 125).

[0079] The inter-screen prediction unit (120) may predict a prediction unit based on information of at least one picture among the previous or subsequent pictures of the current picture, and in some cases, may predict a prediction unit based on information of a portion of an encoded region within the current picture. The inter-screen prediction unit (120) may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

[0080] The reference picture interpolation unit can receive reference picture information from the memory (155) and generate pixel information less than an integer pixel from the reference picture. In the case of luminance pixels, a DCT-based 8-tap interpolation filter with different filter coefficients can be used to generate pixel information less than an integer pixel in units of 1 / 4 pixels. In the case of a chrominance signal, a DCT-based 4-tap interpolation filter with different filter coefficients can be used to generate pixel information less than an integer pixel in units of 1 / 8 pixels.

[0081] The motion prediction unit can perform motion prediction based on a reference picture interpolated by the reference picture interpolation unit. Various methods can be used to derive a motion vector, such as FBMA (Full search-based Block Matching Algorithm), TSS (Three Step Search), and NTS (New Three-Step Search Algorithm). The motion vector can have a motion vector value in units of 1 / 2 or 1 / 4 pixels based on the interpolated pixel. The motion prediction unit can predict the current prediction unit by using different motion prediction methods. Various methods can be used as motion prediction methods, such as the Skip method, the Merge method, the AMVP (Advanced Motion Vector Prediction) method, and the Intra Block Copy method.

[0082] The on-screen prediction unit (125) can generate a prediction block based on reference pixel information, which is pixel information within the current picture. The reference pixel information can be derived from one selected from among a plurality of reference pixel lines. The Nth reference pixel line among the plurality of reference pixel lines can include left pixels having an x-axis difference of N from the upper left pixel within the current block and upper pixels having a y-axis difference of N from the upper left pixel. The number of reference pixel lines that the current block can select can be 1, 2, 3, or 4.

[0083] If the neighboring blocks of the current prediction unit are blocks that have performed inter-screen prediction and the reference pixel is a pixel that has performed inter-screen prediction, the reference pixel included in the block that has performed inter-screen prediction can be replaced with the reference pixel information of the neighboring block that has performed intra-screen prediction. That is, if the reference pixel is unavailable, the unavailable reference pixel information can be replaced with information from at least one of the available reference pixels.

[0084] In intra-screen prediction, the prediction mode can have a directional prediction mode that uses reference pixel information according to the prediction direction, and a non-directional mode that does not use directional information when performing prediction. The mode for predicting luminance information and the mode for predicting chrominance information can be different, and the intra-screen prediction mode information used to predict luminance information or the predicted luminance signal information can be utilized to predict chrominance information.

[0085] When performing intra-screen prediction, if the size of the prediction unit and the size of the transformation unit are the same, intra-screen prediction for the prediction unit can be performed based on the pixels on the left side of the prediction unit, the pixels on the upper left side, and the pixels on the upper side.

[0086] The on-screen prediction method can generate prediction blocks by applying a smoothing filter to reference pixels according to the prediction mode. Depending on the selected reference pixel line, whether or not the smoothing filter is applied can be determined.

[0087] In order to perform an intra-screen prediction method, the intra-screen prediction mode of the current prediction unit can be predicted from the intra-screen prediction modes of prediction units existing around the current prediction unit. When the prediction mode of the current prediction unit is predicted using mode information predicted from the surrounding prediction units, if the intra-screen prediction modes of the current prediction unit and the surrounding prediction units are the same, information indicating that the prediction modes of the current prediction unit and the surrounding prediction units are the same can be transmitted using predetermined flag information, and if the prediction modes of the current prediction unit and the surrounding prediction units are different, entropy encoding can be performed to encode the prediction mode information of the current block.

[0088] Additionally, a residual block containing residual value information, which is the difference between the prediction unit that performed the prediction based on the prediction unit generated in the prediction unit (120, 125) and the original block of the prediction unit, can be generated. The generated residual block can be input to the transformation unit (130).

[0089] In the transformation unit (130), the residual block including the residual value information of the prediction unit generated through the original block and the prediction unit (120, 125) can be transformed using a transformation method such as DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), or KLT. Whether to apply DCT, DST, or KLT to transform the residual block can be determined based on at least one of the size of the transformation unit, the shape of the transformation unit, the prediction mode of the prediction unit, or the prediction mode information within the screen of the prediction unit. Meanwhile, the transformation can be performed by separating the horizontal direction and the vertical direction.

[0090] After performing transformations in the horizontal and vertical directions, a secondary transformation can be performed. The secondary transformation may be in a form in which the horizontal and vertical directions are not separated. The secondary transformation can be performed on the transformation coefficients obtained by the primary transformation to generate final transformation coefficients. Meanwhile, the number of final transformation coefficients output by the secondary transformation may be smaller than the number of transformation coefficients input for the secondary transformation. Specifically, the secondary transformation can be performed using a reduced transformation matrix having different numbers of columns and rows.

[0091] The quantization unit (135) can quantize values ​​converted to the frequency domain by the transformation unit (130). The quantization coefficients can vary depending on the block or the importance of the image. The values ​​produced by the quantization unit (135) can be provided to the dequantization unit (140) and the reordering unit (160).

[0092] The rearrangement unit (160) can perform rearrangement of coefficient values ​​for quantized residual values.

[0093] The reordering unit (160) can change a two-dimensional block-shaped coefficient into a one-dimensional vector form through a coefficient scanning method. For example, the reordering unit (160) can change the two-dimensional block-shaped coefficient into a one-dimensional vector form by scanning from the DC coefficient to the coefficient of the high-frequency region using a zig-zag scan method. Depending on the size of the conversion unit and the intra-screen prediction mode, a vertical scan that scans the two-dimensional block-shaped coefficient in the column direction, a horizontal scan that scans the two-dimensional block-shaped coefficient in the row direction, or a diagonal scan that scans the two-dimensional block-shaped coefficient in the diagonal direction may be used instead of the zig-zag scan. That is, depending on the size of the conversion unit and the intra-screen prediction mode, it is possible to determine which scan method among the zig-zag scan, the vertical scan, the horizontal scan, or the diagonal scan is to be used.

[0094] The entropy encoding unit (165) can perform entropy encoding based on the values ​​produced by the rearrangement unit (160). Entropy encoding can use various encoding methods such as, for example, Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

[0095] The entropy encoding unit (165) can encode various information such as residual value coefficient information of the encoding unit, block type information, prediction mode information, division unit information, prediction unit information, transmission unit information, motion vector information, reference frame information, block interpolation information, and filtering information from the rearrangement unit (160) and the prediction unit (120, 125).

[0096] The entropy encoding unit (165) can entropy encode the coefficient values ​​of the encoding unit input from the rearrangement unit (160).

[0097] The inverse quantization unit (140) and the inverse transformation unit (145) inversely quantize the values ​​quantized in the quantization unit (135) and inversely transform the values ​​transformed in the transformation unit (130). The residual values ​​generated in the inverse quantization unit (140) and the inverse transformation unit (145) can be combined with the predicted prediction units predicted through the motion estimation unit, motion compensation unit, and intra-screen prediction unit included in the prediction unit (120, 125) to generate a reconstructed block.

[0098] The filter unit (150) may include at least one of a deblocking filter, an offset correction unit, and an ALF (Adaptive Loop Filter).

[0099] A deblocking filter can remove block distortion caused by boundaries between blocks in a reconstructed picture. To determine whether to perform deblocking, a deblocking filter can be applied to the current block based on the pixels contained in several columns or rows within the block. When applying a deblocking filter to a block, a strong filter or a weak filter can be applied depending on the required deblocking filtering strength. Furthermore, when applying a deblocking filter, horizontal and vertical filtering can be processed in parallel when performing vertical and horizontal filtering.

[0100] The offset correction unit can correct the offset from the original image on a pixel-by-pixel basis for an image that has undergone deblocking. To perform offset correction for a specific picture, the pixels contained in the image can be divided into a certain number of regions, the regions to be offset can be determined, and the offset can be applied to those regions. Alternatively, the offset can be applied by considering the edge information of each pixel.

[0101] Adaptive Loop Filtering (ALF) can be performed based on the comparison of the filtered restored image with the original image. After dividing the pixels included in the image into predetermined groups, a filter to be applied to each group can be determined, and filtering can be performed differentially for each group. Information regarding whether to apply ALF can be transmitted by luminance signal for each coding unit (CU), and the shape and filter coefficients of the ALF filter to be applied can vary depending on each block. Furthermore, an ALF filter of the same shape (fixed shape) can be applied regardless of the characteristics of the target block.

[0102] The memory (155) can store a restoration block or picture produced through the filter unit (150), and the stored restoration block or picture can be provided to the prediction unit (120, 125) when performing inter-screen prediction.

[0103] FIG. 2 is a block diagram showing an image decoding device according to an embodiment of the present disclosure.

[0104] Referring to FIG. 2, the image decoding device (200) may include an entropy decoding unit (210), a rearrangement unit (215), an inverse quantization unit (220), an inverse transformation unit (225), a prediction unit (230, 235), a filter unit (240), and a memory (245).

[0105] When a video bitstream is input to a video encoding device, the input bitstream can be decoded in the opposite procedure to that of the video encoding device.

[0106] The entropy decoding unit (210) can perform entropy decoding in a procedure opposite to that of the entropy encoding unit of the video encoding device. For example, various methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) can be applied in response to the method performed in the video encoding device.

[0107] The entropy decoding unit (210) can decode information related to intra-screen prediction and inter-screen prediction performed in the encoding device.

[0108] The reordering unit (215) can perform reordering based on the method in which the bitstream entropy-decoded by the entropy decoding unit (210) is reordered by the encoding unit. The coefficients expressed in the form of a one-dimensional vector can be reordered by restoring them back to coefficients in the form of a two-dimensional block. The reordering unit (215) can perform reordering by receiving information related to the coefficient scanning performed by the encoding unit and performing reverse scanning based on the scanning order performed by the corresponding encoding unit.

[0109] The dequantization unit (220) can perform dequantization based on the quantization parameters provided from the encoding device and the coefficient values ​​of the rearranged block.

[0110] The inverse transform unit (225) can perform an inverse transform of the transform performed by the transform unit on the quantization result performed by the image encoding device. That is, at least one of an inverse transform of a secondary transform (secondary inverse transform) or an inverse transform for DCT, DST, and KLT (i.e., first inverse transform) can be performed. The inverse transform can be performed based on a transmission unit determined by the image encoding device. The inverse transform unit (225) of the image decoding device can determine a transform matrix for the second inverse transform or a transform technique (e.g., DCT, DST, KLT) for the first inverse transform according to a plurality of pieces of information such as a prediction method, the size and shape of the current block, the prediction mode, and the prediction direction within the screen. Alternatively, information for determining the transform matrix or the transform technique may be explicitly encoded and signaled.

[0111] The prediction unit (230, 235) can generate a prediction block based on the prediction block generation related information provided by the entropy decoding unit (210) and the previously decoded block or picture information provided by the memory (245).

[0112] As described above, when performing intra-screen prediction in the same manner as the operation in the video encoding device, if the size of the prediction unit and the size of the transformation unit are the same, intra-screen prediction for the prediction unit is performed based on the pixels on the left side of the prediction unit, the pixels on the upper left side, and the pixels on the upper side. However, when performing intra-screen prediction, if the size of the prediction unit and the size of the transformation unit are different, intra-screen prediction can be performed using reference pixels based on the transformation unit. In addition, intra-screen prediction using NxN division only for the minimum coding unit can be used.

[0113] The prediction unit (230, 235) may include a prediction unit determination unit, an inter-screen prediction unit, and an intra-screen prediction unit. The prediction unit determination unit may receive various information such as prediction unit information input from the entropy decoding unit (210), prediction mode information of an intra-screen prediction method, and motion prediction-related information of an inter-screen prediction method, and may distinguish a prediction unit from a current encoding unit and determine whether the prediction unit performs inter-screen prediction or intra-screen prediction. The inter-screen prediction unit (230) may perform inter-screen prediction on the current prediction unit based on information included in at least one of a previous picture or a subsequent picture of the current picture including the current prediction unit, using information necessary for inter-screen prediction of the current prediction unit provided from the video encoding device. Alternatively, inter-screen prediction may be performed based on information on a pre-restored portion of the current picture including the current prediction unit.

[0114] In order to perform inter-screen prediction, it is possible to determine whether the motion prediction method of the prediction unit included in the encoding unit is Skip Mode, Merge Mode, AMVP Mode, or Intra-screen Block Copy Mode based on the encoding unit.

[0115] The intra-screen prediction unit (235) can generate a prediction block based on pixel information within the current picture. If the prediction unit is a prediction unit that has performed intra-screen prediction, intra-screen prediction can be performed based on intra-screen prediction mode information of the prediction unit provided by the video encoding device. The intra-screen prediction unit (235) can include an AIS (Adaptive Intra Smoothing) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter is a part that performs filtering on the reference pixels of the current block, and can determine and apply whether to apply the filter according to the prediction mode of the current prediction unit. AIS filtering can be performed on the reference pixels of the current block using the prediction mode and AIS filter information of the prediction unit provided by the video encoding device. If the prediction mode of the current block is a mode that does not perform AIS filtering, the AIS filter may not be applied.

[0116] The reference pixel interpolation unit can generate a reference pixel of a pixel unit less than an integer value by interpolating the reference pixel when the prediction mode of the prediction unit is a prediction unit that performs intra-screen prediction based on the pixel value interpolated from the reference pixel. If the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixel, the reference pixel may not be interpolated. The DC filter can generate a prediction block through filtering when the prediction mode of the current block is the DC mode.

[0117] The restored block or picture may be provided to a filter unit (240). The filter unit (240) may include a deblocking filter, an offset correction unit, and an ALF.

[0118] Information regarding whether a deblocking filter has been applied to a corresponding block or picture may be received from a video encoding device, and if a deblocking filter has been applied, information regarding whether a strong or weak filter has been applied. The deblocking filter of the video decoding device may receive information related to the deblocking filter provided by the video encoding device, and the video decoding device may perform deblocking filtering on the corresponding block.

[0119] The offset correction unit can perform offset correction on the restored image based on the type of offset correction applied to the image during encoding and offset value information.

[0120] ALF can be applied to an encoding unit based on information such as whether ALF is applied and ALF coefficient information provided from an encoding device. This ALF information can be provided by being included in a specific parameter set.

[0121] The memory (245) can store a restored picture or block so that it can be used as a reference picture or reference block, and can also provide the restored picture to an output unit.

[0122] As described above, in the following embodiments of the present disclosure, for convenience of explanation, the term coding unit is used as an encoding unit, but it may also be a unit that performs not only encoding but also decoding.

[0123] In addition, the current block represents a block to be encoded / decoded, and may represent a coding tree block (or coding tree unit), an encoding block (or encoding unit), a transform block (or transform unit), a prediction block (or prediction unit), or a block to which an in-loop filter is applied, depending on the encoding / decoding step. In this specification, a 'unit' represents a basic unit for performing a specific encoding / decoding process, and a 'block' may represent a pixel array of a predetermined size. Unless otherwise distinguished, 'block' and 'unit' may be used with the same meaning. For example, in the embodiment described below, an encoding block (coding block) and an encoding unit (coding unit) may be understood to have the same meaning.

[0124] Additionally, the encoding parameters for the current block may be commonly applied to multiple color components for the current block. For example, if the encoding mode of the current block is determined, prediction for the Y component block, the Cb component block, and the Cr component block may be performed based on the encoding mode.

[0125] Alternatively, depending on the color component to be encoded / decoded, the current block may mean a Y component block, a Cb component block, or a Cr component block.

[0126] Furthermore, we will refer to the picture that contains the current block as the current picture.

[0127] In an encoder, the current picture can be divided into multiple reference blocks. Here, the reference blocks can be called Coding Tree Units (CTUs) or Coding Tree Blocks (CTBs).

[0128] The size of the reference block may be predefined in the encoder and decoder. Alternatively, information related to the size of the reference block may be encoded and signaled to the decoder. The information may be encoded / decoded via an upper header. For example, the information may be encoded / decoded via a sequence parameter set or a picture header.

[0129] The reference block may be further divided into multiple blocks (i.e., multiple coding blocks) based on a tree structure partitioning. Here, the tree structure partitioning may include at least one of a quad tree partitioning, a binary tree partitioning, or a ternary tree partitioning.

[0130] A prediction block for the current block can be obtained by performing a prediction block on the current block generated by dividing the reference block. Specifically, a prediction block for the current block can be obtained through inter-prediction or intra-prediction.

[0131] Inter-prediction may be used to remove redundant data between pictures, and intra-prediction may be used to remove redundant data within a picture. For example, a prediction block of the current block may be generated from a reference picture using motion information of the current block, or an intra-prediction mode of the current block may be determined, and then a prediction block of the current block may be generated from reference samples of the current block. Here, the motion information may include at least one of a motion vector, a reference picture index, and a prediction direction.

[0132] FIG. 3 illustrates an image encoding / decoding method performed by an image encoding / decoding device according to the present disclosure.

[0133] Referring to FIG. 3, a reference line for intra prediction of the current block can be determined (S300).

[0134] The current block can use one or more of a plurality of pre-defined reference line candidates in the video encoding / decoding device as reference lines for intra prediction. Here, the plurality of pre-defined reference line candidates can include neighboring reference lines adjacent to the current block to be decoded and N non-neighboring reference lines that are 1 to N samples away from the boundary of the current block. N can be 1, 2, 3, or an integer greater than or equal to 1. For convenience of explanation, it is assumed hereafter that the plurality of reference line candidates available to the current block are composed of neighboring reference line candidates and three non-neighboring reference line candidates, but the present invention is not limited thereto. That is, it goes without saying that the plurality of reference line candidates available to the current block can include four or more non-neighboring reference line candidates.

[0135] An image encoding device can determine an optimal reference line candidate from among a plurality of reference line candidates and encode an index for specifying the optimal reference line candidate. An image decoding device can determine a reference line of a current block based on an index signaled through a bitstream. The index can specify any one of the plurality of reference line candidates. The reference line candidate specified by the index can be used as a reference line of the current block.

[0136] The number of indexes signaled to determine the reference line of the current block may be 1, 2, or more. For example, when the number of indexes signaled is 1, the current block can perform intra prediction using only a single reference line candidate specified by the signaled index among a plurality of reference line candidates. Alternatively, when the number of indexes signaled is 2 or more, the current block can perform intra prediction using a plurality of reference line candidates specified by a plurality of indexes among a plurality of reference line candidates.

[0137] Referring to FIG. 3, the intra prediction mode of the current block can be determined (S310).

[0138] The intra prediction mode of the current block can be determined from among multiple intra prediction modes pre-defined in the video encoding / decoding device. The multiple pre-defined intra prediction modes will be described with reference to FIGS. 4 and 5.

[0139] FIG. 4 illustrates an example of multiple intra prediction modes according to the present disclosure.

[0140] Referring to FIG. 4, the multiple intra prediction modes pre-defined in the video encoding / decoding device may be configured as a non-directional mode and a directional mode. The non-directional mode may include at least one of a planar mode or a DC mode. The directional mode may include directional modes 2 to 66.

[0141] The directional mode can be further expanded than that shown in Fig. 4. Fig. 5 shows an example in which the directional mode is expanded.

[0142] In Fig. 5, modes -1 to -14 and modes 67 to 80 are exemplified as being added. These directional modes may be referred to as wide-angle intra prediction modes. Whether to use the wide-angle intra prediction mode may be determined depending on the shape of the current block. For example, if the current block is a non-square block whose width is greater than its height, some directional modes (e.g., 2 to 15) may be converted to wide-angle intra prediction modes between 67 and 80. On the other hand, if the current block is a non-square block whose height is greater than its width, some directional modes (e.g., 53 to 66) may be converted to wide-angle intra prediction modes between -1 and -14.

[0143] The range of available wide-angle intra prediction modes can be adaptively determined based on the width-to-height ratio of the current block. Table 1 shows the range of available wide-angle intra prediction modes based on the width-to-height ratio of the current block.

[0144] Available Wide Angle Intra Prediction Mode Ranges W / H = 1667~80 W / H = 867~78 W / H = 467~76 W / H = 267~74 W / H = 1 None W / H = 1 / 2-1~-8 W / H = 1 / 4-1~-10 W / H = 1 / 8-1~-12 W / H = 1 / 16-1~-14

[0145] Among the above multiple intra prediction modes, K candidate modes (most probable modes, MPMs) can be selected. A candidate list including the selected candidate modes can be generated. An index indicating one of the candidate modes in the candidate list can be signaled. The intra prediction mode of the current block can be determined based on the candidate mode indicated by the index. For example, the candidate mode indicated by the index can be set as the intra prediction mode of the current block. Alternatively, the intra prediction mode of the current block can be determined based on a value of the candidate mode indicated by the index and a predetermined difference value. The difference value can be defined as a difference between a value of the intra prediction mode of the current block and a value of the candidate mode indicated by the index. The difference value can be signaled through a bitstream. Alternatively, the difference value may be a pre-defined value in the video encoding / decoding device. Alternatively, the intra prediction mode of the current block may be determined based on a flag indicating whether a mode identical to the intra prediction mode of the current block exists in the candidate list. For example, when the flag has a first value, the intra prediction mode of the current block may be determined from the candidate list. In this case, an index indicating any one of a plurality of candidate modes belonging to the candidate list may be signaled. The candidate mode indicated by the index may be set as the intra prediction mode of the current block. On the other hand, when the flag has a second value, any one of the remaining intra prediction modes may be set as the intra prediction mode of the current block. The remaining intra prediction mode may mean a mode excluding a candidate mode belonging to the candidate list among the plurality of pre-defined intra prediction modes. When the flag has a second value, an index indicating any one of the remaining intra prediction modes may be signaled.The intra prediction mode indicated by the signaled index can be set as the intra prediction mode of the current block.

[0146] The intra prediction mode of a chroma block can be selected from among intra prediction mode candidates of multiple chroma blocks. To this end, index information indicating one of the intra prediction mode candidates of the chroma block can be explicitly encoded and signaled through the bitstream. Table 2 illustrates intra prediction mode candidates of the chroma block.

[0147] Intra prediction mode candidates for indexed chroma blocks Luma mode: 0 Luma mode: 50 Luma mode: 18 Luma mode: 1 Other 0 6 6 0 0 0 1 5 0 6 6 5 0 5 0 5 0 2 1 8 1 8 6 6 1 8 1 8 3 1 1 6 6 1 4 DM

[0148] In the example of Table 2, DM (Direct Mode) means setting the intra prediction mode of the luma block co-located with the chroma block to the intra prediction mode of the chroma block. Meanwhile, the luma block co-located with the chroma block can be determined based on the position of the upper left sample or the position of the center sample of the chroma block.

[0149] For example, if the intra prediction mode (luma mode) of the luma block is 0 (planar mode) and the index points to 2, the intra prediction mode of the chroma block can be determined as horizontal mode (18). For example, if the intra prediction mode (luma mode) of the luma block is 1 (DC mode) and the index points to 0, the intra prediction mode of the chroma block can be determined as planar mode (0).

[0150] Consequently, the intra prediction mode of the chroma block may also be set to one of the intra prediction modes illustrated in FIG. 4 or FIG. 5. The intra prediction mode of the current block may also be used to determine a reference line of the current block, in which case step S310 may be performed before step S300.

[0151] Meanwhile, in the present disclosure, a chroma block may represent at least one of a Cb component block or a Cr component block.

[0152] Referring to FIG. 3, intra prediction can be performed for the current block based on the reference line and intra prediction mode of the current block (S320).

[0153] Hereinafter, with reference to FIGS. 6 through 8, we will examine in detail the intra prediction method for each intra prediction mode. However, for convenience of explanation, it is assumed that a single reference line is used for intra prediction of the current block. However, even when multiple reference lines are used, the intra prediction method described below can be applied in the same / similar manner.

[0154] FIG. 6 illustrates an intra prediction method based on a planar mode according to the present disclosure.

[0155] Referring to Fig. 6, T represents a reference sample located at the upper right corner of the current block, and L represents a reference sample located at the lower left corner of the current block. P1 can be generated through horizontal interpolation. For example, P1 can be generated by interpolating T with a reference sample located on the same horizontal line as P1. P2 can be generated through vertical interpolation. For example, P2 can be generated by interpolating L with a reference sample located on the same vertical line as P2. The current sample within the current block can be predicted through a weighted sum of P1 and P2, as in the following mathematical expression 1.

[0156]

[0157] In Equation 1, weights α and β can be determined by considering the width and height of the current block. Depending on the width and height of the current block, weights α and β may have the same value or different values. If the width and height of the current block are the same, weights α and β can be set to the same value, and the prediction sample of the current sample can be set to the average value of P1 and P2. If the width and height of the current block are not the same, weights α and β can have different values. For example, if the width is greater than the height, a smaller value can be set for the weight corresponding to the width of the current block, and a larger value can be set for the weight corresponding to the height of the current block. Conversely, if the width is greater than the height, a larger value can be set for the weight corresponding to the width of the current block, and a smaller value can be set for the weight corresponding to the height of the current block. Here, the weight corresponding to the width of the current block can mean β, and the weight corresponding to the height of the current block can mean α.

[0158] FIG. 7 illustrates an intra prediction method based on DC mode according to the present disclosure.

[0159] Referring to FIG. 7, the average value of neighboring samples adjacent to the current block can be calculated, and the calculated average value can be set as the predicted value of all samples in the current block. Here, the neighboring samples can include the upper reference sample and the left reference sample of the current block. However, depending on the shape of the current block, the average value can be calculated using only the upper reference sample or the left reference sample. For example, if the width of the current block is greater than the height, the average value can be calculated using only the upper reference sample of the current block. Alternatively, if the ratio of the width to the height of the current block is greater than or equal to a predetermined threshold, the average value can be calculated using only the upper reference sample of the current block. Alternatively, if the ratio of the width to the height of the current block is less than or equal to a predetermined threshold, the average value can be calculated using only the upper reference sample of the current block. On the other hand, if the width of the current block is less than the height, the average value can be calculated using only the left reference sample of the current block. Alternatively, if the ratio of the width to the height of the current block is less than or equal to a predetermined threshold, the average value can be calculated using only the left reference sample of the current block. Alternatively, if the ratio of the width and height of the current block is greater than or equal to a predetermined threshold, the average value can be calculated using only the left reference sample of the current block.

[0160] FIG. 8 illustrates an intra prediction method based on a directional mode according to the present disclosure.

[0161] If the intra prediction mode of the current block is a directional mode, projection can be performed to a reference line according to the angle of the directional mode. If a reference sample exists at the projected position, the reference sample can be set as a prediction sample of the current sample. If a reference sample does not exist at the projected position, a sample corresponding to the projected position can be generated using one or more neighboring samples neighboring the projected position. For example, a sample corresponding to the projected position can be generated by performing interpolation based on two or more neighboring samples neighboring in both directions with respect to the projected position. Alternatively, one neighboring sample neighboring the projected position can be set as the sample corresponding to the projected position. In this case, among the plurality of neighboring samples neighboring the projected position, the neighboring sample closest to the projected position can be used. The sample corresponding to the projected position can be set as a prediction sample of the current sample.

[0162] Referring to FIG. 8, for the current sample B, when projection is performed with a reference line according to the angle of the intra prediction mode at the corresponding position, a reference sample exists at the projected position (i.e., a reference sample at an integer position, R3). In this case, the reference sample at the projected position can be set as a prediction sample of the current sample B. For the current sample A, when projection is performed with a reference line according to the angle of the intra prediction mode at the corresponding position, a reference sample (i.e., a reference sample at an integer position) does not exist at the projected position. In this case, a sample (r) at a fractional position can be generated by performing interpolation based on neighboring samples (e.g., R2 and R3) adjacent to the projected position. The generated sample (r) at the fractional position can be set as a prediction sample of the current sample A.

[0163] Figure 9 illustrates a method for deriving samples of fractional positions.

[0164] In the example of Fig. 9, the variable h represents the vertical distance (i.e., vertical distance) between the position of the predicted sample A and the reference sample line, and the variable w represents the horizontal distance (i.e., horizontal distance) between the position of the predicted sample A and the fractional position sample. In addition, the variable θ represents a predefined angle according to the directionality of the intra prediction mode, and the variable x represents the fractional position.

[0165] The variable w can be derived as shown in the following mathematical expression 2.

[0166]

[0167] Afterwards, by removing the integer position from the variable w, the fractional position can be finally derived.

[0168] Fractional position samples can be generated by interpolating adjacent integer position reference samples. For example, integer position reference sample R2 and integer position reference sample R3 can be interpolated to generate fractional position reference samples at the x position.

[0169] To avoid floating-point operations when deriving fractional position samples, a scaling factor can be used. For example, if the scaling factor f is set to 32, the distance between neighboring integer reference samples can be set to 32 instead of 1, as in the example illustrated in (b) of Fig. 8.

[0170] Additionally, the tangent value for the angle θ determined by the directionality of the intra prediction mode can also be scaled up using the same scaling factor (e.g., 32).

[0171] Figures 10 and 11 illustrate that the tangent value for the angle is scaled by a factor of 32 for each intra prediction mode.

[0172] Figure 10 shows the scaled results of tangent values ​​for the non-wide angle intra prediction mode, and Figure 11 shows the scaled results of tangent values ​​for the wide angle intra prediction mode.

[0173] If the tangent value (tanθ) for the angle value of the intra prediction mode is positive, intra prediction can be performed using only one of the reference samples belonging to the upper line of the current block (i.e., upper reference samples) or the reference samples belonging to the left line of the current block (i.e., left reference samples). On the other hand, if the tangent value for the angle value of the intra prediction mode is negative, both the reference samples located at the upper side and the reference samples located at the left side are used.

[0174] At this time, to simplify the implementation, the reference samples may be arranged in a 1D array form by projecting the left reference samples upward or the top reference samples to the left, and intra prediction may be performed using the reference samples in the 1D array form.

[0175] Figure 12 is a diagram illustrating an intra prediction aspect when the directional mode is one of modes 34 to 49.

[0176] If the intra prediction mode of the current block is one of modes 34 to 49, intra prediction is performed using not only the upper reference samples of the current block but also the left reference samples. At this time, as in the example illustrated in Fig. 12, the reference sample located on the left side of the current block can be copied to the position of the upper line, or the reference samples located on the left can be interpolated to generate the reference sample of the upper line.

[0177] For example, in case of obtaining a reference sample for position A at the top of the current block, projection can be performed from position A of the top line to the left line of the current block, considering the directionality of the intra prediction mode of the current block. If the projected position is a, the value corresponding to position a can be copied, or a fractional position value corresponding to a can be generated and set as the value of position A. For example, if position a is an integer position, the value of position A can be generated by copying the integer position reference sample. On the other hand, if position a is a fractional position, the reference sample located above position a and the reference sample located below position a can be interpolated, and the interpolated value can be set as the value of position A. Meanwhile, the direction of projection from position A at the top of the current block to the left line of the current block can be parallel to and opposite to the direction of the intra prediction mode of the current block.

[0178] Figure 13 is a drawing for explaining an example of generating an upper reference sample by interpolating left reference samples.

[0179] In Fig. 13, the variable h represents the horizontal distance between position A of the upper line and position a of the left line. The variable w represents the vertical distance between position A of the upper line and position a of the left line. In addition, the variable θ represents a predefined angle according to the directionality of the intra prediction mode, and the variable x represents a fractional position.

[0180] The variable h can be derived as shown in the following mathematical expression 3.

[0181]

[0182] Afterwards, by removing the integer position from the variable h, the fractional position can be finally derived.

[0183] To avoid real-valued operations when deriving fractional position samples, a scaling factor can be used. For example, the tangent value for variable θ can be scaled using the scaling factor f1. Here, since the direction projected to the left line is parallel and opposite to the directional prediction model, the scaled tangent value illustrated in FIGS. 10 and 11 can also be used.

[0184] When the scaling factor f1 is applied, Equation 3 can be transformed into Equation 4 below.

[0185]

[0186] In this manner, a 1D reference sample array can be constructed using only the reference samples belonging to the upper line. As a result, intra prediction for the current block can be performed using only the upper reference samples formed in the 1D array.

[0187] Figure 14 shows an example in which intra prediction is performed using reference samples arranged in a 1D array.

[0188] As in the example illustrated in Fig. 14, by projecting the left reference samples to generate the upper reference samples, prediction samples of the current block can be obtained using only the reference samples belonging to the upper line.

[0189] Contrary to what is shown in FIGS. 12 and 14, a 1D reference sample array can also be constructed using only the reference samples belonging to the left line by projecting the upper reference sample onto the left line. Specifically, for modes 19 to 33 among the directional modes in which the tangent value (tanθ) for the angle of the directional mode is negative, the reference samples belonging to the upper line can be projected onto the left line to generate the left reference sample.

[0190] The intra prediction mode of the current block can also be derived using reference samples surrounding the current block. Specifically, the horizontal and vertical gradients for the reference samples are calculated, and the intra prediction mode of the current block can be derived using the calculated gradients.

[0191] Figure 15 is a drawing for explaining an example of setting a reference area.

[0192] For convenience of explanation, we assume that the current block size is 4x4.

[0193] A reference region can be set to derive the intra prediction mode of the current block. For example, in Fig. 15, it is assumed that w0 columns adjacent to the left of the current block and h0 rows adjacent to the top of the current block are set as the reference region.

[0194] The number of columns (w0) and / or the number of rows (h0) constituting the reference region may be fixed in the encoder and decoder. Alternatively, the number of columns (w0) and / or the number of rows (h0) may be determined based on at least one of the size / shape of the current block, whether Intra Sub-Partitioning (ISP) is applied to the current block, or whether the current block is adjacent to a CTU boundary.

[0195] As another example, the size of the reference region may be determined depending on the type of filter applied to the reference region. Specifically, the horizontal and vertical lengths of the filter may be set to the number of columns w0 and the number of rows h0, respectively. For example, assuming that a 3x3 mask as illustrated in FIG. 17 described below is used, the number of columns w0 and the number of rows h0 may each be set to 3.

[0196] The reference area may extend beyond the right boundary and / or bottom boundary of the current block. For example, in the example illustrated in FIG. 15, the reference area is illustrated as extending from the right boundary of the current block by w1 and from the bottom boundary of the current block by h1.

[0197] The right extension distance w1 and / or the bottom extension distance h1 may be set to be equal to the width and / or height of the current block. For example, if the size of the current block is 4x4, the right extension distance w1 may be set to 4, equal to the width of the current block, and the bottom extension distance h1 may be set to 4, equal to the height of the current block.

[0198] As another example, a reference area may be set, as in the example illustrated in FIG. 16.

[0199] Specifically, as in the example illustrated in (a) of Fig. 16, the right extension distance w1 and / or the bottom extension distance h1 can be set to 0. Furthermore, as in the example illustrated in (b) of Fig. 16, the upper reference area can be formed only with reference samples whose x-axis coordinates are between 0 and (w-1), and the left reference area can be formed only with reference samples whose y-axis coordinates are between 0 and (h-1). Here, w represents the width of the current block, and h represents the height of the current block.

[0200] As another example, at least one of the reference line candidates or reference line candidates for intra prediction of the current block may be set as a reference region.

[0201] As another example, depending on whether the current block is adjacent to a CTU boundary, the reference area may be formed using only the upper reference area, or only the left reference area.

[0202] Filtering (i.e., convolution) can be performed using a mask within the reference region. In this case, the filter used may be at least one of a Sobel mask or a Prewitt mask that outputs a gradient value.

[0203] Figure 17 illustrates filter coefficients for each of the Sobel mask and the Prewitt mask.

[0204] A different type of filter than that illustrated in FIG. 17 may be applied to the reference region. For example, instead of a 3x3 square filter, a 1D filter of 1x3 or 3x1 size, a 2x3 or 2x3 rectangular filter, a cross-shaped filter, or a diamond-shaped filter may be applied to the reference region. Alternatively, a filter of a different size than that illustrated in FIG. 17 (e.g., 2x2, 4x4, or 5x5, etc.) may be applied to the reference region.

[0205] The type of filter applied to the reference region may be predefined in the encoder and decoder. Alternatively, multiple filter candidates may be predefined, and index information pointing to one of the multiple filter candidates may be encoded and explicitly signaled through the bitstream.

[0206] As another example, at least one of the plurality of filter candidates may be adaptively selected based on at least one of the following: the size / shape of the current block, whether ISP is applied to the current block, the size of the reference region, the intra prediction mode of a neighboring block, or whether the current block borders a CTU boundary. Here, the neighboring block may include at least one of the upper neighboring block or the left neighboring block of the current block.

[0207] The type of filter applied to the upper reference area may be different from the type of filter applied to the left reference area.

[0208] By applying a vertical mask to a specific reference sample within a reference region, a vertical gradient Dy for the reference sample can be obtained. Furthermore, by applying a horizontal mask to a specific reference sample within a reference region, a horizontal gradient Dx for the reference sample can be obtained.

[0209] Figure 18 shows the locations where the vertical and horizontal slopes are obtained within the reference area.

[0210] Assuming that a 3x3 sized mask is applied, as in the example illustrated in Fig. 17, a vertical gradient Dy and a horizontal gradient Dx can be obtained for each of the reference samples that are not adjacent to the boundary of the reference area. For example, if w0 and h0 are 3, and w1 and h1 are 4, as in the example illustrated in Fig. 17, 17 vertical gradients Dy and 17 horizontal gradients Dx can be obtained for each of the 17 reference samples.

[0211] When a filter of a different size or shape than that shown in Fig. 17 is applied, the vertical slope Dy and the horizontal slope Dx can be obtained for more / fewer reference samples than shown.

[0212] Based on the vertical gradient Dy and horizontal gradient Dx of each of the reference samples, an intra prediction mode can be determined for each of the reference samples.

[0213] We will explain how to determine the intra prediction mode of a reference sample using the vertical slope Dy and the horizontal slope Dx. Horizontal direction

[0214] For example, if either the vertical slope Dy or the horizontal slope Dx is 0, the directional mode of the corresponding reference sample can be determined as the horizontal mode (No. 18) or the vertical mode (No. 50). Specifically, if the horizontal slope Dx is 0 and the vertical slope Dy is not 0, the intra prediction mode of the corresponding reference sample can be determined as the vertical mode (No. 50). Conversely, if the vertical slope Dy is 0 and the horizontal slope Dx is not 0, the intra prediction mode of the corresponding reference sample can be determined as the horizontal mode (No. 18).

[0215] If both the vertical slope Dy and the horizontal slope Dx are not 0, one of the remaining directional modes, excluding the horizontal mode and the vertical mode, can be determined as the intra prediction mode of the reference sample.

[0216] Here, the intra prediction mode group to which the intra prediction mode of the reference sample belongs can be determined by comparing the absolute values ​​of the vertical slope Dy and the horizontal slope Dx. Here, the intra prediction mode group may be composed of multiple directional modes of similar directionality.

[0217] Figure 19 shows an example of grouping directional modes into multiple intra prediction mode groups.

[0218] In Fig. 19, the directional modes are exemplified as being classified into four intra prediction mode groups (a to d) based on the horizontal mode (no. 18), the diagonal mode (no. 34), and the vertical mode (no. 50).

[0219] In the illustrated example, groups a and b are symmetrical structures based on the horizontal mode (No. 18), and groups c and d are symmetrical structures based on the vertical mode (No. 50).

[0220] Additionally, the angles of the directional modes 36 to 66 are identical to the transposed angles of the modes 2 to 34.

[0221] If the absolute value of the horizontal slope Dx of the reference sample is greater than the absolute value of the vertical slope Dy, the directional mode of the reference sample may belong to group a or group b.

[0222] Conversely, if the absolute value of the vertical slope Dy of the reference sample is greater than the slope of the horizontal slope Dx, the directional mode of the reference sample may belong to group c or group d.

[0223] Table 3 shows the intra prediction mode group to which the intra prediction mode of the reference sample belongs, according to the magnitude of the horizontal gradient Dx and the vertical gradient Dy.

[0224] if (|Dx| > |Dy|)ElseDx >= 0Dy >= 0bDx >= 0Dy >= 0cDx < 0Dy >= 0aDx < 0Dy >= 0dDx >= 0Dy < 0aDx >= 0Dy < 0dDx < 0Dy < 0bDx < 0Dy < 0c

[0225] Using the horizontal slope Dx and the vertical slope Dy of the reference sample, the slope of the directional mode to be assigned to the reference sample can be derived. To this end, a variable R representing the ratio between the horizontal slope and the vertical slope can be derived, as in Equation 5 below.

[0226]

[0227] As illustrated in mathematical expression 5, the variable R can be derived by using the larger absolute value of the horizontal slope Dx and the vertical slope Dy as the denominator.

[0228] Thereafter, the directional mode of the reference sample can be determined by comparing the tangent value (tanθ) for each angle of the variable R and the directional modes. Specifically, the directional mode having the same tangent value as the variable R or the most similar tangent value can be assigned to the reference sample.

[0229] At this time, if the tangent value for each angle of the directional modes is stored in the encoder and decoder in a scaled state, as in the example illustrated in FIG. 10 or FIG. 11, the directional mode of the reference sample can be determined by scaling the variable R using the same scaling factor.

[0230] Next, the amplitude of each reference sample can be derived. The amplitude can be derived as the sum of the absolute value of the horizontal slope Dx and the absolute value of the vertical slope Dy, as shown in the following mathematical expression 6.

[0231]

[0232] Next, for each of the intra prediction modes, the amplitude values ​​of each of the reference samples to which the same intra prediction mode is assigned can be accumulated.

[0233]

[0234] In mathematical expression 7, intra_mode represents an intra prediction mode. For example, the amplitude accumulation value for a directional mode with a mode number of N can be derived by summing the amplitude values ​​of reference samples to which mode N is assigned within a reference region, and the amplitude accumulation value for a directional mode with a mode number of M can be derived by summing the amplitude values ​​of reference samples to which mode M is assigned within a reference region.

[0235] The buffer storing the amplitude accumulation value can be initialized on a block-by-block basis. For example, when specifying a reference area around the current block, the amplitude accumulation value for each intra prediction mode can be initialized to 0.

[0236] Through the above process, when a histogram recording the amplitude accumulation value for each intra prediction mode is derived, at least one intra prediction mode can be selected in descending order of the amplitude accumulation value in the histogram. The number of intra prediction modes to be selected can be M, and M can be a natural number greater than or equal to 1. The value of M can be predefined in the encoder and decoder. Alternatively, the value of M can be adaptively determined by considering at least one of the size / shape of the current block and whether ISP is applied to the current block. That is, M intra prediction modes can be selected in descending order of the amplitude accumulation value.

[0237] At least one intra prediction mode selected from the histogram may be set as the intra prediction mode of the current block, and a prediction block of the current block may be obtained based on the intra prediction mode of the current block. For example, if one intra prediction mode is selected from the histogram, a prediction block obtained based on the selected intra prediction mode may be used as the final prediction block of the current block.

[0238] When multiple intra prediction modes are selected from the histogram, intra prediction can be performed based on each of the multiple intra prediction modes. Accordingly, when multiple prediction blocks are generated, the final prediction block of the current block can be obtained through an average operation or a weighted sum operation of the multiple prediction blocks.

[0239] At this time, for the weighted sum operation, the weight applied to each prediction block can be determined based on the amplitude of the intra prediction mode. That is, the largest weight can be assigned to the prediction block derived based on the intra prediction mode with the largest amplitude among multiple intra prediction modes, and the smallest weight can be assigned to the prediction block derived based on the intra prediction mode with the smallest amplitude.

[0240] At this time, the weight assigned to each prediction block can be determined based on the ratio between amplitudes. Alternatively, the weight values ​​for each amplitude rank can be stored in advance, and then the weights mapped to the amplitude ranks of the corresponding intra prediction mode can be applied to the prediction block.

[0241] A prediction block of the current block can be obtained by further considering at least one default mode along with at least one intra prediction mode selected from the histogram. For example, intra prediction can be performed based on each of the intra prediction modes selected from the histogram and the default mode, thereby obtaining multiple prediction blocks for the current block. Thereafter, a final prediction block of the current block can be obtained by performing an average operation or a weighted sum operation on the multiple prediction blocks.

[0242] The number of default modes, N, can be an integer greater than or equal to 0 or 1. When M intra prediction modes are selected from the histogram, intra prediction can be performed based on each of the M intra prediction modes and the N default modes, so that (M+N) prediction blocks can be obtained. Thereafter, the final prediction block of the current block can be obtained through an average operation or a weighted sum operation of the (M+N) prediction blocks.

[0243] The number N of default modes may be predefined in the encoder and decoder. Alternatively, the number N of default modes may be adaptively determined based on at least one of the size / shape of the current block, whether ISP is applied to the current block, or whether at least one intra prediction mode selected from the histogram includes the default mode.

[0244] The default mode may include at least one of a planar mode, a DC mode, or a predefined directional mode.

[0245] The encoder and decoder can also be configured to use a predefined mode (e.g., planar mode) from among the modes listed above as the default mode.

[0246] Alternatively, the type of default mode may be adaptively determined based on the type of directional mode selected through the histogram. For example, if at least one directional mode selected through the histogram is a vertical mode or a horizontal mode, the planar mode or DC mode may be set as the default mode. Conversely, if no vertical and / or horizontal mode is selected through the histogram, the vertical mode or the horizontal mode may be set as the default mode.

[0247] Depending on the shape of the current block, the availability of a wide-angle intra prediction mode may be determined. For example, if the current block is a square shape with equal width and height, the directional modes selected from the histogram may be composed of non-wide-angle intra prediction modes. Conversely, if the current block is a non-square shape with different widths and heights, some of the directional modes selected from the histogram may be converted to a wide-angle intra prediction mode.

[0248] When performing intra prediction based on an intra prediction mode derived through a histogram, a predefined reference line may be utilized. Here, the predefined reference line may be an adjacent reference line (i.e., index 0) adjacent to the current block or a non-adjacent reference line (e.g., index 1).

[0249] Information indicating whether to apply the method of performing intra prediction by selecting an intra prediction mode through a histogram as described above may be encoded and signaled through the bitstream. The information may be a 1-bit flag.

[0250] Alternatively, whether to select an intra prediction mode may be determined via a histogram based on at least one of the size / shape of the current block, whether ISP is applied to the current block, whether the current block borders a CTU boundary, or whether a neighboring block is encoded with intra prediction.

[0251] For example, if at least one of the upper neighboring block or the left neighboring block of the current block is not encoded with intra prediction, a method of selecting an intra prediction mode through a histogram can be applied to the current block.

[0252] The method for selecting an intra prediction mode using a histogram can be applied to both the luma component and the chroma component. Alternatively, the above-described method can be applied only to the luma component. Alternatively, whether to select an intra prediction mode using a histogram can be determined independently for each of the luma and chroma components.

[0253] The intra prediction mode of the current block can also be derived by utilizing the surrounding area of ​​the current block. The surrounding area used as a reference for deriving the intra prediction mode of the current block can be referred to as a reference area.

[0254] Figure 20 is a drawing illustrating a reference area around the current block.

[0255] In Fig. 20, it is illustrated that the width w and height h of the current block are both 4.

[0256] As in the example illustrated in Fig. 20, a surrounding area adjacent to the current block can be set as a reference area. Specifically, a left reference area adjacent to the left of the current block and an upper reference area adjacent to the top of the current block can each be set.

[0257] The size of the left reference area may be expressed as w0, and the size of the upper reference area may be expressed as h0. For example, w0 represents the number of reference sample lines (i.e., reference sample columns) included in the left reference area, and h0 represents the number of reference sample lines (i.e., reference sample rows) included in the upper reference area. At this time, w0 and h0 may each be a natural number greater than or equal to 1. In addition, w0 and h0 may be predefined in the encoder and decoder.

[0258] For example, as in the example illustrated in FIG. 20, if the size of the current block is 4x4 or 2x2, the 4x4 or 2x2 area on the left of the current block can be set as the left reference area, and the 4x4 or 2x2 area on the top of the current block can be set as the top reference area.

[0259] Alternatively, at least one of the sizes w0 and / or h0 of the reference area may be adaptively determined based on at least one of the size of the current block, the shape of the current block, whether Intra Sub-partitioning (ISP) is applied to the current block, or whether the current block is adjacent to a CTU boundary. Here, the size of the current block represents at least one of the width, the height, or the product of the width and the height of the current block. As an example, at least one of the left reference area and the upper reference area may be determined to be equal to the size of the current block. Alternatively, the left reference area may be set to a square area in which one side length is equal to the height of the current block, and the upper reference area may be set to a square area in which one side length is equal to the width of the current block.

[0260] Alternatively, the size of the reference area can be determined by comparing the size of the current block with a threshold value. For example, if the size of the current block is greater than or equal to the threshold value, the size of at least one of the left reference area or the upper reference area can be set to 4x4. Conversely, if the size of the current block is less than the threshold value, the size of at least one of the left reference area or the upper reference area can be set to 2x2.

[0261] Intra prediction can be performed on the reference region using reference samples of the reference region. Here, the reference samples for the left reference region may belong to a column adjacent to the left of the left reference region, and the reference samples for the upper reference region may belong to a row adjacent to the top of the upper reference region.

[0262] In the example illustrated in Fig. 20, w1 and h1 are variables representing the range of reference samples used to perform intra prediction for the reference region. Specifically, w1 may represent the number of reference samples present in the upper right region of the upper reference region, and h1 may represent the number of reference samples present in the lower left region of the left reference region.

[0263] In the example illustrated in Fig. 20, both w1 and h1 are illustrated as being 4.

[0264] At this time, w1 and h1 may be predefined in the encoder and decoder. For example, each of w1 and h1 may be a natural number greater than or equal to 0 or 1.

[0265] Alternatively, at least one of w1 or h1 may be adaptively determined based on at least one of the size of the current block, the shape of the current block, whether Intra Sub-partitioning (ISP) is applied to the current block, or whether the current block is adjacent to a CTU boundary. Here, the size of the current block represents at least one of the width, height, or the product of the width and height of the current block.

[0266] For example, if the size (e.g., width or height) of the current block is greater than or equal to the threshold, at least one of w1 or h1 may be set to 8 or 16. Conversely, if the size (e.g., width or height) of the current block is less than the threshold, at least one of w1 or h1 may be set to 4.

[0267] Meanwhile, under the above conditions, w1 can be determined depending on the width w of the current block, and h1 can be determined depending on the height h of the current block.

[0268] Alternatively, if the current block is square, w1 and h1 may be equal to each other. On the other hand, if the current block is non-square, w1 and h1 may be different from each other.

[0269] Intra prediction can be performed on a reference region using reference samples for the reference region. Specifically, intra prediction can be performed on the reference region based on multiple intra prediction modes, and then the cost for each prediction result can be calculated.

[0270] Figure 21 shows an example of performing intra prediction for a reference region based on the planar mode.

[0271] Specifically, in FIG. 21, reference samples used to perform intra prediction based on the planar mode for the left reference area and reference samples used to perform intra prediction based on the planar mode for the upper reference area are illustrated.

[0272] As in the example illustrated in Figure 21, the reference samples may be included in a line adjacent to the left of the left reference area and a line adjacent to the top of the upper reference area.

[0273] Accordingly, the left reference samples for the left reference region may be adjacent to the left reference region, while the top reference samples for the left reference region may not be adjacent to the left reference region.

[0274] Additionally, the upper reference samples for the upper reference area may be adjacent to the upper reference area, while the left reference samples for the upper reference area may not be adjacent to the upper reference area.

[0275] Figure 22 shows an example of performing intra prediction for a reference region based on the DC mode.

[0276] When intra prediction based on the DC mode is performed, the prediction samples can be set to the average value of the reference samples. At this time, as in the example illustrated in Fig. 22, the average value for the upper reference area (i.e., DCval) can be calculated using only the reference samples adjacent to the upper reference area (i.e., the upper reference samples), and the average value for the left reference area can be calculated using only the reference samples adjacent to the left reference area (i.e., the left reference samples).

[0277] Figure 23 shows an example of performing intra prediction for a reference region based on a directional mode.

[0278] Meanwhile, depending on the directional mode, intra prediction for the reference region can be performed using only reference samples belonging to the upper row of the upper reference region, or intra prediction for the reference region can be performed using only reference samples belonging to the left column of the left reference region.

[0279] As an example, FIG. 23 shows an example in which intra prediction for a reference region is performed using only reference samples belonging to the upper row of the upper reference region.

[0280] For example, if the index of the directional mode is equal to or greater than the index of the upper left diagonal mode (i.e., 34), intra prediction for the reference regions (i.e., the upper reference region and the left reference region) can be performed using only the reference samples belonging to the upper row of the upper reference region.

[0281] Meanwhile, if the index of the directional mode is smaller than the index of the vertical mode (i.e., 50), reference samples belonging to the left column of the left reference region (i.e., left reference samples) can be projected to the upper row of the upper reference region according to the directionality of the directional mode, thereby deriving reference samples belonging to the upper row (i.e., upper reference samples). Meanwhile, if the position projected from the left reference sample is not an integer position, the upper reference sample can be obtained by interpolating the left reference samples.

[0282] Although not shown, if the index of the directional mode is smaller than the index of the upper left diagonal mode, intra prediction for the reference region (i.e., the upper reference region and the left reference region) can be performed using only the reference samples belonging to the left column of the left reference region.

[0283] Meanwhile, if the index of the directional mode is greater than the index of the horizontal mode (i.e., 18), the reference samples belonging to the upper row of the upper reference area (i.e., the upper reference samples) can be projected to the left column of the left reference area according to the directionality of the directional mode, thereby deriving the reference samples belonging to the left column (i.e., the left reference samples). Meanwhile, if the position projected from the left reference sample is not an integer position, the upper reference sample can be obtained by interpolating the left reference samples.

[0284] After performing multiple intra predictions on a reference region based on multiple intra prediction modes, the cost for each intra prediction mode can be calculated. Specifically, the cost for each intra prediction mode can be calculated based on the difference between the reconstructed samples within the reference region and the predicted samples within the reference region obtained through intra prediction.

[0285] Meanwhile, the cost function for calculating the cost may include at least one of SAD (Sum of Absolute Difference), SATD (Sum of Absolute Transformed Differences), SSD (Sum of Squared Difference), or MR-SAD (Mean-Removed Sum of Absolute Differences).

[0286] Once the cost for each intra prediction mode is calculated, the intra prediction mode with the lowest cost can be selected.

[0287] Alternatively, N intra prediction modes with low cost can be selected, where N is a natural number greater than or equal to 1, such as 2, 3, or 4.

[0288] Thereafter, based on N intra prediction modes, N intra predictions can be performed for the current block to obtain N predicted blocks. Thereafter, the N predicted blocks can be weighted and combined to obtain the final predicted block of the current block.

[0289] Meanwhile, the weights for weighted summation can be determined as a ratio of the costs of each intra prediction mode. That is, if the cost of an intra prediction mode is low, a high weight can be assigned to the predicted block derived from that intra prediction mode. Conversely, if the cost of an intra prediction mode is high, a low weight can be assigned to the predicted block derived from that intra prediction mode.

[0290] Meanwhile, intra prediction modes can be derived for each of the upper reference region and the left reference region. For example, based on the results of performing multiple intra predictions on the upper reference region, a first intra prediction mode with the lowest cost can be selected, and based on the results of performing multiple intra predictions on the left reference region, a second intra prediction mode with the lowest cost can be selected. Thereafter, based on the first and second intra prediction modes, intra prediction can be performed twice on the current block to obtain a first prediction block and a second prediction block. Thereafter, the first prediction block and the second prediction block can be weighted or averaged to obtain the current block.

[0291] Alternatively, at least one intra prediction mode selected in descending order of cost may be inserted into the MPM list of the current block. For example, a first intra prediction mode derived from the upper reference region and a second intra prediction mode derived from the left reference region may be inserted into the MPM list of the current block.

[0292] Afterwards, intra prediction for the current block can be performed by selecting at least one of the intra prediction mode candidates included in the MPM list.

[0293] Figure 24 is a diagram schematically illustrating the process of performing inter prediction in an encoder and decoder.

[0294] As in the example illustrated in FIG. 24, to perform inter prediction, motion information for the current block can be acquired (S2410). Here, the motion information can include at least one of a motion vector, a reference picture index, or a weight applied to the prediction block. For the current block, motion information for at least one of the L0 direction or the L1 direction can be acquired.

[0295] In the encoder, motion information of the current block can be derived through motion estimation, and the derived motion information can be encoded and signaled to the decoder. Meanwhile, the encoding / decoding of motion information can be based on a motion information merging mode, a motion vector prediction mode, a template-based motion estimation method, or a bilateral matching method, which will be described later.

[0296] In the decoder, motion information of the current block can be derived based on the information transmitted from the encoder.

[0297] Alternatively, the motion information of the current block can be derived from the decoder in the same manner as in the encoder. This method can be referred to as decoder-side motion estimation.

[0298] Once motion information for the current block is derived, a prediction block for the current block can be obtained based on the derived motion information (S2420). For example, a reference block spaced apart by a motion vector from the current block's position within the reference picture can be set as the prediction block for the current block.

[0299] Below, we will explain in more detail the process of calculating inter predictions.

[0300] The motion information of the current block can be generated through motion estimation.

[0301] Figure 25 shows an example in which motion estimation is performed.

[0302] In Fig. 25, it is assumed that the POC (Picture Order Count) of the current picture is T, and the POC of the reference picture is (T-1).

[0303] A search range for motion estimation can be set from the same location as the reference point of the current block within the reference picture. Here, the reference point may be the location of the upper left sample of the current block.

[0304] For example, in Fig. 25, it is illustrated that a rectangle of size (w0+w01) and (h0+h1) is set as a search range centered on the reference point. In the above example, w0, w1, h0, and h1 may have the same value. Alternatively, at least one of w0, w1, h0, and h1 may be set to have a different value from the other. Alternatively, the sizes of w0, w1, h0, and h1 may be determined so as not to exceed a Coding Tree Unit (CTU) boundary, a slice boundary, a tile boundary, or a picture boundary.

[0305] Within the search range, reference blocks of the same size as the current block can be set, and the cost of each reference block relative to the current block can be measured. The cost can be calculated using the similarity between the two blocks.

[0306] For example, the cost can be calculated based on the absolute sum of the differences between the original samples in the current block and the original samples (or reconstructed samples) in the reference block. A smaller absolute sum can reduce the cost.

[0307] Afterwards, the cost of each reference block is compared, and the reference block with the optimal cost can be set as the prediction block of the current block.

[0308] Additionally, the distance between the current block and the reference block can be set as a motion vector. Specifically, the x-coordinate difference and the y-coordinate difference between the current block and the reference block can be set as the motion vector.

[0309] Furthermore, the index of the picture containing the reference block identified through motion estimation is set as the reference picture index.

[0310] Additionally, the prediction direction can be set based on whether the reference picture belongs to the L0 reference picture list or the L1 reference picture list.

[0311] Additionally, motion estimation can be performed for each of the L0 direction and the L1 direction. If prediction is performed for both the L0 direction and the L1 direction, motion information in the L0 direction and motion information in the L1 direction can be generated, respectively.

[0312] Figures 26 and 27 illustrate examples in which a prediction block of a current block is generated based on motion information generated through motion estimation.

[0313] Figure 26 shows an example of generating a prediction block with unidirectional (i.e., L0 direction) prediction, and Figure 27 shows an example of generating a prediction block with bidirectional (i.e., L0 and L1 direction) prediction.

[0314] In the case of unidirectional prediction, a prediction block of the current block is generated using a single motion information. For example, the motion information may include an L0 motion vector, an L0 reference picture index, and prediction direction information indicating the L0 direction.

[0315] In the case of bidirectional prediction, a prediction block is generated using two pieces of motion information. For example, a reference block in the L0 direction, determined based on motion information about the L0 direction (L0 motion information), can be set as an L0 prediction block, and an L1 prediction block can be generated based on a reference block in the L1 direction, determined based on motion information about the L1 direction (L1 motion information). Thereafter, the L0 prediction block and the L1 prediction block can be weighted and combined to generate a prediction block of the current block.

[0316] In the examples illustrated in FIGS. 25 to 27, the L0 reference picture is illustrated as existing in the previous direction of the current picture (i.e., having a POC value smaller than that of the current picture), and the L1 reference picture is illustrated as existing in the subsequent direction of the current picture (i.e., having a POC value larger than that of the current picture).

[0317] However, unlike the illustrated example, the L0 reference picture may exist in the subsequent direction of the current picture, or the L1 reference picture may exist in the previous direction of the current picture. For example, both the L0 reference picture and the L1 reference picture may exist in the previous direction of the current picture, or both may exist in the subsequent direction of the current picture. Alternatively, bidirectional prediction may be performed using the L0 reference picture existing in the subsequent direction of the current picture and the L1 reference picture existing in the previous direction of the current picture.

[0318] Motion information for blocks for which inter prediction has been performed can be stored in memory. At this time, the motion information can be stored on a sample-by-sample basis. Specifically, the motion information for a block to which a specific sample belongs can be stored as motion information for that specific sample. The stored motion information can be used to derive motion information for neighboring blocks to be encoded / decoded in the future.

[0319] In the encoder, information encoding residual samples corresponding to the difference between the sample of the current block (i.e., the original sample) and the predicted sample, and motion information required to generate a predicted block can be signaled to the decoder. The decoder can decode information about the signaled difference value to derive a difference sample, and add a prediction sample within the predicted block generated using the motion information to the difference sample to generate a restored sample.

[0320] At this time, in order to effectively compress the motion information signaled to the decoder, one of a plurality of inter prediction modes may be selected. Here, the plurality of inter prediction modes may include a motion information merging mode and a motion vector prediction mode.

[0321] The motion vector prediction mode is a mode that signals by encoding the difference between a motion vector and a motion vector prediction value. Here, the motion vector prediction value can be derived based on motion information of neighboring blocks or neighboring samples adjacent to the current block.

[0322] Figure 28 shows the locations referenced to derive motion vector prediction values.

[0323] For convenience of explanation, the current block is assumed to have a size of 4x4.

[0324] In the illustrated example, 'LB' represents a sample contained in the leftmost column and bottommost row within the current block. 'RT' represents a sample contained in the rightmost column and topmost row within the current block. A0 to A4 represent samples neighboring to the left of the current block, and B0 to B5 represent samples neighboring to the top of the current block. For example, A1 represents a sample neighboring to the left of LB, and B1 represents a sample neighboring to the top of RT.

[0325] Col indicates the location of a sample neighboring the lower right of the current block within a co-located picture. A co-located picture is a picture different from the current picture, and information for specifying the co-located picture (e.g., a co-located picture index) can be explicitly encoded and signaled in the bitstream. Alternatively, a reference picture having a predefined reference picture index can be set as the co-located picture.

[0326] The motion vector prediction value of the current block can be derived from at least one motion vector prediction candidate included in a motion vector prediction list.

[0327] The number of motion vector prediction candidates that can be inserted into the motion vector prediction list (i.e., the size of the list) may be predefined in the encoder and decoder. For example, the maximum number of motion vector prediction candidates may be 2.

[0328] A motion vector stored at the location of a neighboring sample adjacent to the current block or a scaled motion vector derived by scaling the motion vector can be inserted into the motion vector prediction list as a motion vector prediction candidate. At this time, the motion vector prediction candidates can be derived by scanning the neighboring samples adjacent to the current block in a predefined order.

[0329] For example, it is possible to check whether a motion vector is stored at each location in the order of A0 to A4. Then, according to the above scanning order, the first available motion vector found can be inserted into the motion vector prediction list as a motion vector prediction candidate.

[0330] As another example, in the order of A0 to A4, it is checked whether a motion vector is stored at each position, and the motion vector of the position that is found first and has the same reference picture as the current block can be inserted into the motion vector prediction list as a motion vector prediction candidate. If there is no neighboring sample that has the same reference picture as the current block, a motion vector prediction candidate can be derived based on the first found available vector. Specifically, the first found available motion vector can be scaled, and then the scaled motion vector can be inserted into the motion vector prediction list as a motion vector prediction candidate. At this time, the scaling can be performed based on the output order difference between the current picture and the reference picture (i.e., the POC difference) and the output order difference between the current picture and the reference picture of the neighboring sample (i.e., the POC difference).

[0331] Furthermore, it is possible to check whether a motion vector is stored at each location in the order of B0 to B5. Then, according to the above scanning order, the first available motion vector found can be inserted into the motion vector prediction list as a motion vector prediction candidate.

[0332] As another example, in the order of B0 to B5, it is checked whether a motion vector is stored at each position, and the motion vector of the position that has the same reference picture as the current block that is found first can be inserted into the motion vector prediction list as a motion vector prediction candidate. If there is no neighboring sample that has the same reference picture as the current block, a motion vector prediction candidate can be derived based on the first found available vector. Specifically, the first found available motion vector can be scaled, and then the scaled motion vector can be inserted into the motion vector prediction list as a motion vector prediction candidate. At this time, the scaling can be performed based on the output order difference between the current picture and the reference picture (i.e., the POC difference) and the output order difference between the current picture and the reference picture of the neighboring sample (i.e., the POC difference).

[0333] As in the example described above, a motion vector prediction candidate can be derived from a sample adjacent to the left of the current block, and a motion vector prediction candidate can be derived from a sample adjacent to the top of the current block.

[0334] At this time, the motion vector prediction candidate derived from the left sample may be inserted into the motion vector prediction list before the motion vector prediction candidate derived from the upper sample. In this case, the index assigned to the motion vector prediction candidate derived from the left sample may have a smaller value than the motion vector prediction candidate derived from the upper sample.

[0335] Conversely, the motion vector prediction candidate derived from the top sample may be inserted into the motion vector prediction list before the motion vector prediction candidate derived from the left sample.

[0336] Among the motion vector prediction candidates included in the above motion vector prediction list, the motion vector prediction candidate with the highest encoding efficiency can be set as the motion vector predictor (MVP) of the current block. In addition, index information indicating the motion vector prediction candidate set as the motion vector predictor of the current block among the plurality of motion vector prediction candidates can be encoded and signaled to a decoder. When the number of motion vector prediction candidates is two, the index information can be a 1-bit flag (e.g., an MVP flag). In addition, a motion vector difference (MVD), which is the difference between the motion vector of the current block and the motion vector predictor, can be encoded and signaled to a decoder.

[0337] The decoder can construct a motion vector prediction list, similar to the encoder. Furthermore, it can decode index information from the bitstream and select one of multiple motion vector prediction candidates based on the decoded index information. The selected motion vector prediction candidate can be set as the motion vector prediction value of the current block.

[0338] Additionally, the motion vector differential can be decoded from the bitstream. Afterwards, the motion vector of the current block can be derived by combining the motion vector prediction value and the motion vector differential value.

[0339] When bidirectional prediction is applied to the current block, a motion vector prediction list can be generated for each of the L0 and L1 directions. That is, the motion vector prediction list can be composed of motion vectors in the same direction. Accordingly, the motion vector of the current block and the motion vector prediction candidates included in the motion vector prediction list have the same direction.

[0340] When the motion vector prediction mode is selected, reference picture index and prediction direction information can be explicitly encoded and signaled to the decoder. For example, when there are multiple reference pictures in the reference picture list and motion estimation is performed for each of the multiple reference pictures, a reference picture index for specifying a reference picture from which motion information of the current block is derived among the multiple reference pictures can be explicitly encoded and signaled to the decoder.

[0341] At this time, if the reference picture list contains only one reference picture, encoding / decoding of the reference picture index may be omitted.

[0342] The prediction direction information may be an index pointing to one of L0 unidirectional prediction, L1 unidirectional prediction, or bidirectional prediction. Alternatively, an L0 flag indicating whether prediction is performed in the L0 direction and an L1 flag indicating whether prediction is performed in the L1 direction may be encoded and signaled, respectively.

[0343] Motion Information Merge Mode is a mode in which the motion information of the current block is set to be identical to the motion information of neighboring blocks. In Motion Information Merge Mode, motion information can be encoded / decoded using a motion information merge list.

[0344] Motion information merging candidates can be derived based on motion information from neighboring blocks or neighboring samples adjacent to the current block. For example, after defining reference locations around the current block, it is possible to check whether motion information exists at the defined reference locations. If motion information exists at the defined reference locations, the motion information at those locations can be inserted into the motion information merging list as a motion information merging candidate.

[0345] In the example of Fig. 28, the predefined reference positions may include at least one of A0, A1, B0, B1, B5, and Col. Furthermore, motion information merging candidates may be derived in the order of A1, B1, B0, A0, B5, and Col.

[0346] Among the motion information merge candidates included in the motion information merge list, the motion information of the motion information merge candidate with the optimal cost can be set as the motion information of the current block. Furthermore, index information (e.g., a merge index) indicating the motion information merge candidate selected from among the multiple motion information merge candidates can be encoded and transmitted to the decoder.

[0347] In the decoder, a motion information merge list can be constructed in the same manner as in the encoder. Furthermore, motion information merge candidates can be selected based on the merge index decoded from the bitstream. The motion information of the selected motion information merge candidate can be set as the motion information of the current block.

[0348] Unlike the motion vector prediction list, the motion information merge list is composed of a single list regardless of the prediction direction. That is, the motion information merge candidates included in the motion information merge list may have only L0 motion information or only L1 motion information, or may have bidirectional motion information (i.e., L0 motion information and L1 motion information).

[0349] After performing a prediction for the current block, the residual block can be obtained by differentiating the original block and the predicted block.

[0350] Figure 29 is a flowchart of a method for encoding a residual block in an encoder, and Figure 30 is a flowchart of a method for restoring a residual block in a decoder.

[0351] In the encoder, residual coefficients can be obtained by performing at least one of transformation or quantization on the residual block (S2910, S2920). If quantization is omitted for the residual block, the residual coefficients may refer to transform coefficients obtained by transformation. Alternatively, if transformation and quantization are performed on the residual block, the residual coefficients may refer to quantized transform coefficients obtained by quantizing the transform coefficients. Alternatively, if transformation is omitted for the residual block, the residual coefficients may be obtained by quantizing residual samples.

[0352] Additionally, in the encoder, the residual coefficients can be encoded (specifically, entropy encoded) and the encoded data can be transmitted to the decoder (S2930).

[0353] The decoder decodes the encoded data to restore the residual coefficients (S3010). Then, by performing inverse quantization on the residual coefficients, transform coefficients (i.e., inverse quantized residual coefficients) can be derived, and by performing inverse transformation on the transform coefficients, a residual block can be derived (S3020, S3030).

[0354] Information indicating whether a transformation is applied to the current block may be encoded and signaled. For example, transform_skip_flag may be encoded and signaled. When transform_skip_flag is 1, it indicates that no transformation is applied to the current block. Here, the transformation may include not only the first transformation described below, but also the second transformation. When transform_skip_flag is 0, it indicates that a transformation is applied to the current block. When transform_skip_flag is 0, the first transformation is necessarily applied to the current block, while the second transformation may be optionally applied.

[0355] The transformation for the current block may be performed based on at least one of a plurality of transformation kernel candidates. For example, the transformation kernel applicable to the current block may be a transformation kernel of the Discrete Cosine Transform (DCT) family or a transformation kernel of the Discrete Sine Transform (DST) family.

[0356] Mathematical expressions 8 to 10 represent the basis functions of the transform kernels applicable to the current block. Mathematical expression 8 represents the basis function for DCT-2, Mathematical expression 9 represents the basis function for DCT-8, and Mathematical expression 10 represents the basis function for DST-7.

[0357]

[0358]

[0359]

[0360] When multiple transform kernel candidates exist, information indicating a transform kernel applied to the current block among the multiple transform kernel candidates can be encoded and signaled. Here, the transform kernel candidate can include at least one of DCT-2, DST-7, or DCT-8. Additionally, the information can be an index indicating one of the multiple transform kernel candidates.

[0361] Meanwhile, the horizontal and vertical transformations of the current block can be separated. At this time, a common transformation kernel can be applied to both the horizontal and vertical directions. That is, when one of multiple transformation kernel candidates is selected, the selected transformation kernel can be applied to both the horizontal and vertical transformations of the current block.

[0362] Alternatively, the transform kernels for the horizontal and vertical directions can be determined independently. In this case, the encoder can encode and signal information indicating the transform kernel for the horizontal direction and information indicating the transform kernel for the vertical direction, respectively.

[0363] Alternatively, multiple transform kernel combination candidates indicating combinations of horizontal transform kernels and vertical transform kernels may be defined, and then information indicating one of the multiple transform kernel combination candidates may be encoded and signaled. Table 4 illustrates multiple transform kernel combination candidates.

[0364] Index 01234 Horizontal Conversion DCT-2DST-7DCT-8DST-7DCT-8 Vertical Conversion DCT-2DST-7DST-7DCT-8DCT-8

[0365] In the decoder, a transformation kernel for the horizontal direction and a transformation kernel for the vertical direction can be determined based on an index indicating one of a plurality of transformation kernel combination candidates.

[0366] Meanwhile, information indicating whether the transformation kernels for horizontal and vertical transformations are determined integrally may be encoded / decoded. The information may be a 1-bit flag.

[0367] For example, if the information indicates that the transform kernels for horizontal and vertical transformations are determined integrally, an index indicating one of the multiple transform kernel candidates may be encoded / decoded. The transform kernel indicated by the index may be applied to both the horizontal and vertical transformations of the current block.

[0368] On the other hand, if the above information indicates that the transformation kernels for the horizontal and vertical transformations are not determined integrally, an index indicating one of a plurality of transformation kernel combination candidates may be encoded / decoded. The combination of the horizontal and vertical transformation kernels indicated by the index may be applied to the current block.

[0369] After performing the above-described transformation, an additional transformation may be performed on the current block. For the convenience of explanation, the transformation performed by the DCT or DST-based transformation kernel will be referred to as the first transformation, and the transformation additionally applied to the result of the first transformation will be referred to as the second transformation. In addition, the transformation coefficients generated as a result of the first transformation will be referred to as the first transformation coefficients, and the transformation coefficients generated as a result of the second transformation will be referred to as the second transformation coefficients.

[0370] In the encoder, a second transform can be performed on the first transform coefficients generated as a result of performing the first transform, thereby generating second transform coefficients.

[0371] When both the first transform and the second transform are performed in the encoder, the decoder can perform a second inverse transform (i.e., an inverse transform of the second transform) on the inverse quantized residual coefficients (i.e., the second transform coefficients) to generate the first transform coefficients. Then, by performing a first inverse transform (i.e., an inverse transform for the first transform) on the first transform coefficients, the residual samples can be obtained.

[0372] The second transform may be applied to at least some of the first transform coefficients. For example, depending on the size of the second transform kernel, the second transform may be applied to 16, 48, or 64 first transform coefficients. The shape of the region containing the first transform coefficients to which the second transform is applied may be square, non-square, or polygonal.

[0373] Mathematical expression 11 shows the application aspect of the second transformation.

[0374]

[0375] When the second transformation is performed, the first transformation coefficients can be aligned in one dimension. For example, in the above mathematical expression 11, A Nx1 represents the first transformation coefficients consisting of N rows and 1 column. Also, B Rx1 represents the second transformation coefficients consisting of R rows and 1 column. T RXN represents the second transformation kernel consisting of R rows and N columns.

[0376] Figures 31 and 32 are drawings showing examples in which the second transformation is applied.

[0377] Fig. 31 illustrates an example in which the second transform kernel is 64x64 in size. The first transform coefficients generated as a result of the first transform within an 8x8 block can be arranged in one dimension. At this time, the first transform coefficients can be scanned using a predetermined scan method to generate the one-dimensional array. The predetermined scan method can include at least one of a diagonal scan, a horizontal scan, a vertical scan, or a raster scan.

[0378] When an input matrix of size 64x1 is generated through the above rearrangement, the second transformation coefficient can be derived through matrix multiplication between the second transformation kernel of size 64x64 and the input matrix of size 64x1.

[0379] As a result of performing the second transform, 64 second transform coefficients are generated, and the second transform coefficients within an 8x8 block can be rearranged. After quantizing the 8x8 block in which the second transform coefficients are rearranged, the quantized transform block can be encoded.

[0380] Fig. 32 illustrates an example in which the second transform kernel is 48x48 in size. Among the first transform coefficients generated as a result of the first transform within an 8x8 block, 48 first transform coefficients can be rearranged into one dimension. At this time, the 48 first transform coefficients may be included in a polygonal area excluding a 4x4 sized sub-block at the lower right within the 8x8 block.

[0381] When the 48 first transformation coefficients are rearranged into one dimension to generate an input matrix of size 48x1, the second transformation coefficients can be derived through matrix multiplication between the second transformation kernel of size 48x48 and the input matrix of size 48x1.

[0382] As a result of performing the second transformation, 48 second transformation coefficients are generated, and the second transformation coefficients can be rearranged within an 8x8 block. For example, the 48 second transformation coefficients can be rearranged within a polygonal region excluding a 4x4 sized sub-block at the lower right within an 8x8 block.

[0383] In areas where the second transform coefficients are not placed, the first transform coefficients can be maintained as is. After applying quantization to a block including the second transform coefficients and the first transform coefficients, the quantized transform block can be encoded.

[0384] Alternatively, the transform coefficients in the area where the second transform coefficients are not placed may be set to 0. That is, the values ​​of the transform coefficients in the area where the second transform is not applied may be set to 0, and then quantization and encoding may be performed.

[0385] The size of the second transform kernel can be determined based on the size of the current block. For example, if at least one of the width or height of the current block is 4, the second transform can be applied to 16 first transform coefficients. Conversely, if the width and height of the current block are 8 or greater, the second transform can be applied to 48 or 64 first transform coefficients.

[0386] Alternatively, information indicating the size and type of the second transform kernel may be encoded and signaled. The information may be signaled at the block level. For example, information specifying at least one of the number of rows or the number of columns of the transform size may be encoded. Alternatively, a different index may be assigned to each combination of the number of rows and the number of columns, and then an index specifying one of the combinations may be encoded. Alternatively, a different index may be assigned to each of a plurality of second transform kernel candidates, and then an index specifying one of the second transform kernel candidates may be encoded. Here, each of the plurality of second transform kernel candidates may differ in at least one of the size or the coefficient.

[0387] Alternatively, based on the size of the current block, the size of the second transformation kernel may be determined, and then an index specifying one of a plurality of second transformation kernel candidates having the determined size may be encoded.

[0388] In the examples illustrated in Figures 31 and 32, a second transformation kernel is used, with the same number of rows and columns. To simplify the second transformation, it is also possible to set the number of rows and columns differently.

[0389] Figures 33 and 34 illustrate a second transformation based on a second transformation kernel of an asymmetric shape.

[0390] The number of rows R of the second transformation kernel can be set to a value smaller than the number of columns N. For example, the number of rows R can be set to 8, and the number of columns N can be set to 48.

[0391] As the number of rows of the second transformation kernel decreases, the number of second transformation coefficients output as a result of the second transformation also decreases. For example, when a matrix multiplication is performed between a second transformation kernel of size 8x48 and an input matrix of size 48x1, second transformation coefficients of size 8x1 are generated.

[0392] The eight second transform coefficients can be rearranged within an 8x8 block. At this time, in an area where the second transform coefficients are not allocated within the application area of ​​the second transform (i.e., an area including the first transform coefficients to which the second transform is applied), the values ​​of the transform coefficients can be set to 0. For example, if a polygonal area including 48 samples is the application area of ​​the second transform, the values ​​of the transform coefficients can be set to 0 in the remaining area excluding the area to which the eight second transform coefficients are allocated among the polygonal area.

[0393] In areas where the second transformation is not applied, the first transformation coefficients can be maintained as is.

[0394] Alternatively, encoding may be performed by converting at least some of the first transform coefficients in the area to which the second transform is not applied to 0. FIG. 34 illustrates an example in which at least some of the area to which the second transform is not applied is converted to 0.

[0395] As in the example illustrated in (a) of Fig. 34, the values ​​of the first transform coefficients corresponding to the high-frequency region within the region where the second transform is not performed can be converted to 0. For example, the values ​​of the first transform coefficients whose sum of the x-axis and y-axis coordinates is greater than or equal to a threshold value can be converted to 0.

[0396] Alternatively, depending on a specific form, the first transform coefficients that are converted to 0 can be selected. For example, as in the example illustrated in (b) of FIG. 34, the first transform coefficients included in the lower n rows within the area where the second transform is not performed can be converted to 0. Alternatively, as in the example illustrated in (c) of FIG. 34, the first transform coefficients included in the right n columns within the area where the second transform is not performed can be converted to 0.

[0397] Alternatively, as in the example shown in (d) of Fig. 34, all first transform coefficients within the area where the second transform is not performed may be transformed to 0.

[0398] The shape of the region containing the first transform coefficients that are transformed to 0 can be determined based on at least one of the size, shape, intra prediction mode, or transform kernel of the current block. Alternatively, an index specifying one of the plurality of candidate shapes that matches the region can be encoded and signaled.

[0399] Whether the second transform is allowed may be determined based on at least one of the encoding mode of the current block or the first transform kernel. Here, the encoding mode refers to intra prediction or inter prediction. For example, if the current block is encoded with intra prediction, the second transform may be allowed, whereas if the current block is encoded with inter prediction, the second transform may not be allowed.

[0400] Information indicating whether a second transformation has been applied may be encoded and signaled. The information may be a 1-bit flag. Depending on whether the flag is true or false, it may be determined whether the second transformation has been applied to the current block. Alternatively, the information may be index information. An index value of 0 indicates that the second transformation has not been applied to the current block. On the other hand, an index value greater than 0 indicates that the second transformation has been applied to the current block. If the index value is greater than 0, the second transformation kernel can be specified by the index.

[0401] Information indicating whether a second transformation has been performed on the current block can be individually encoded for each color component. For example, information indicating whether a second transformation has been performed can be encoded for each of the luma component (Y), the first chrominance component (Cb), and the second chrominance component (Cr).

[0402] Alternatively, information indicating whether a second transformation has been performed on the chrominance components may be integrated and encoded. For example, whether or not a second transformation has been applied to each of the chrominance components (Cb, Cr) may be jointly determined. That is, the first chrominance component (Cb) and the second chrominance component (Cr) may share information indicating whether or not a second transformation has been performed.

[0403] Alternatively, based on the tree structure, it can be determined whether the information is encoded for each color component. For example, if the luma component and the chroma component have the same tree structure, the three color components (i.e., Y, Cb, Cr) can share information indicating whether the second transformation is performed. On the other hand, if the luma component and the chroma component have different tree structures, information indicating whether the second transformation is performed can be signaled for each of the luma component and the chroma component.

[0404] A plurality of second transformation kernel candidates can be grouped into at least a plurality of groups. One of the groups can be identified based on at least one of the size, shape, or intra prediction modes of the current block. Once the group is identified, the index information can be used to identify at least one of the plurality of second transformation kernel candidates included in the identified group.

[0405] Table 5 shows an example in which one of multiple sets of second transform kernels is selected based on the intra prediction mode.

[0406] In-screen prediction mode 2nd transform kernel set In-screen prediction mode < 010 <= In-screen prediction mode <= 102 <= In-screen prediction mode <= 12113 <= In-screen prediction mode <= 23224 <= In-screen prediction mode <= 44345 <= In-screen prediction mode <= 55256 <= In-screen prediction mode <= 80181 <= In-screen prediction mode <= 830

[0407] In the example of Table 5, it is illustrated that one of the four second transformation kernel sets (i.e., the second transformation kernel sets from index 0 to index 3) is selected depending on the intra prediction mode. A lookup table defining the mapping relationship between the intra prediction modes of Table 5 and the second transformation kernel sets may be pre-stored in the encoder and decoder.

[0408] Once the second set of transformation kernels is determined, one of the transformation kernels included in the second set of transformation kernels can be selected via an index.

[0409] Meanwhile, in the example of Table 5, intra prediction modes 81 to 83 may represent Cross-Component Linear Model (CCLM) modes for chrominance components. When the CCLM mode is applied, prediction samples of the chrominance components may be derived by applying prediction parameters to luma component restoration samples. Here, the prediction parameters may include at least one of a weight or an offset.

[0410] Meanwhile, prediction parameters can be derived based on restored samples surrounding the luma and chroma blocks. Depending on the intra prediction mode, the range of restored samples used to derive prediction parameters may vary.

[0411] For example, mode 81 may derive prediction parameters based on top reconstructed samples of the luma block and chroma block, mode 82 may derive prediction parameters using top reconstructed samples and left reconstructed samples of the luma block and chroma block, and mode 83 may derive prediction parameters using left reconstructed samples of the luma block and chroma block.

[0412] Transformation / inverse transformation may also be performed only for some areas within the current block. Here, the transformation may include at least one of the first transformation and the second transformation.

[0413] For example, transformation may be performed only on a portion of a current block, and quantization and entropy encoding may be performed only on the transformation coefficients of the portion of the current block. Accordingly, residual coefficient information may be encoded and signaled only for a portion of a current block, and residual coefficient information may not be encoded / decoded for the remaining portion.

[0414] Accordingly, the decoder can obtain residual samples by performing inverse quantization and inverse transformation only on some regions within the current block. For the remaining regions within the current block, the values ​​of the residual coefficients (or residual samples) can all be set to 0.

[0415] As described above, performing transformation / inverse transformation on only a portion of the current block can be referred to as a partial transformation. For example, the current block can be divided into two regions, and transformation / inverse transformation can be performed on only one of the divided regions. In this case, the region where transformation / inverse transformation is performed can be referred to as the "transformation application region."

[0416] Information about partial transformations may be encoded and signaled. The information about partial transformations may include information indicating whether partial transformations have been applied to the current block. The information may be a 1-bit flag (e.g., sbt_flag).

[0417] When a partial transformation is applied to the current block, information about the partial transformation may further include at least one of information indicating a division ratio of the current block, information indicating a division direction of the current block, and information indicating a location of an area to which the transformation is applied within the current block.

[0418] The information indicating the split ratio of the current block indicates whether the split ratio of the current block is 1:3 or 1:1. The information indicating the split ratio of the current block may be a 1-bit flag (e.g., sbt_quad_flag). A split ratio of 1:3 indicates that the current block is split into an area 1 / 4 the size of the current block and an area 3 / 4 the size of the current block. A split ratio of 1:1 indicates that the current block is split into two areas 1 / 2 the size of two current blocks. The current block may be split into two areas at a ratio different from the above example. For example, the split ratio may be 7:1 or 15:1.

[0419] Information indicating the split direction of the current block indicates whether the split direction of the current block is horizontal or vertical. Information indicating the split direction of the current block may be a 1-bit flag (e.g., sbt_hor_flag).

[0420] Information indicating the position of the area to which transformation is applied within the current block indicates whether the area to which transformation is applied is the first area within the current block or the second area within the current block. Information indicating the position of the area to which transformation is applied within the current block may be a 1-bit flag (e.g., sbt_pos_flag). In this case, if the split ratio is not 1:1, the area to which transformation is applied indicated by sbt_pos_flag may be the larger area of ​​the two areas. Alternatively, if the split ratio is not 1:1, the area to which transformation is applied indicated by sbt_pos_flag may be the smaller area of ​​the two areas.

[0421] Figure 35 shows an example in which information on partial transformations is sequentially encoded / decoded.

[0422] As in the example illustrated in FIG. 35, if information indicating whether a partial transformation has been applied to the current block (e.g., sbt_flag) indicates that a partial transformation has been applied to the current block, information indicating a division ratio of the current block (e.g., sbt_quad_flag), information indicating a division direction of the current block (sbt_hor_flag), and information indicating a position of an area to which a transformation is applied within the current block (sbt_pos_flag) may be additionally encoded / decoded.

[0423] Figure 36 shows an example of a case where a partial transformation is applied to the current block.

[0424] In Fig. 36, area A represents an area where transformation / inverse transformation is performed. Depending on sbt_pos_flag, the first area or the second area within the current block can be set as the transformation application area.

[0425] For the residual region excluding region A, transformation / inverse transformation may not be performed, and accordingly, the values ​​of residual coefficients / residual samples within the residual region may be derived as 0.

[0426] w and h represent the width and height of the current block, respectively. w1 and h1 can be variables representing the sizes of the segmented areas.

[0427] For example, if the splitting direction is vertical, w1 can be set to w / 2 or w / 4, depending on the value of sbt_quad_flag.

[0428] If the split direction is horizontal, h1 can be set to h / 2 or h / 4, depending on the value of sbt_quad_flag.

[0429] The transformation kernel of the transformation application area may be predefined in the encoder and decoder. Specifically, a corresponding transformation kernel may be predefined for each size of the transformation target area or each position of the transformation target area.

[0430] For example, if the current block is vertically divided ((a) and (b) of FIG. 31), the vertical transform kernel of the transformation target region can be determined as DST-7. On the other hand, if the transformation target region is the first region in the current block, the horizontal transform kernel can be determined as DCT-8. On the other hand, if the transformation target region is the second region in the current block, the horizontal transform kernel can be determined as DST-7. Here, the first region represents the left region in the current block, and the second region represents the right region in the current block.

[0431] If the current block is divided horizontally ((c) and (d) of Fig. 31), the horizontal kernel of the target region for transformation can be determined as DST-7. On the other hand, if the target region for transformation is the first region in the current block, the vertical transform kernel can be determined as DCT-8. On the other hand, if the target region for transformation is the second region in the current block, the vertical transform kernel can be determined as DST-7. Here, the first region represents the upper region in the current block, and the second region represents the lower region in the current block.

[0432] As another example, the transformation kernel of the transformation application area can be adaptively determined depending on the size of the current block or the transformation application area.

[0433] For example, if at least one of the width or height of the current block or transform application area is greater than a threshold, both the horizontal transform kernel and the vertical transform kernel may be set to DCT-2.

[0434] Meanwhile, partial transformation can be applied to chrominance components as well as luma components. In this case, the transformation kernel of the transformation application area can be determined differently depending on the color component.

[0435] For example, for the luma component, the transform kernel of the transform application area can be determined according to the example shown in Fig. 34.

[0436] On the other hand, for the chrominance component, both the horizontal and vertical transformation kernels of the transformation target region can be determined by DCT-2 (or DCT-8).

[0437] Meanwhile, depending on the encoding mode of the current block, it can be determined whether partial transformation is allowed to be applied. Here, the encoding mode can indicate intra prediction or inter prediction.

[0438] For example, if the current block is encoded using inter prediction, applying a partial transform to the current block may not be permitted. Accordingly, if the encoding mode of the current block is inter prediction, encoding / decoding information about the partial transform may be omitted. Furthermore, the decoder may infer the value of information indicating whether a partial transform has been applied to the current block (i.e., sbt_flag) as False.

[0439] Alternatively, it may be permissible to apply partial transforms both when the encoding mode of the current block is intra prediction and when it is inter prediction.

[0440] Meanwhile, the second transformation / second inverse transformation can be performed only when the encoding mode of the current block is intra prediction. In this case, a second transformation kernel set for the second transformation / second inverse transformation can be determined based on the intra prediction mode of the current block.

[0441] As another example, even if the encoding mode of the current block is inter prediction, it may be permissible to apply the second transform / second inverse transform. However, if the encoding mode of the current block is inter prediction, since the intra prediction mode of the current block does not exist, the second transform kernel set cannot be selected based on the intra prediction mode. Accordingly, if the encoding mode of the current block is inter prediction, index information indicating one of the plurality of second transform kernel sets can be encoded and signaled.

[0442] As another example, if the encoding mode of the current block is inter prediction, the decoder can derive an intra prediction mode for the current block in the same manner as the encoder. The derived intra prediction mode may only be used to select a second transform kernel set and may not be used to predict the current block.

[0443] Figure 37 shows an example of deriving an intra prediction mode of a current block using a reference block of the current block.

[0444] By applying a horizontal direction filter and a vertical direction filter to a reference block indicated by the motion information of the current block, a horizontal direction slope Dx and a vertical direction slope Dy for samples within the reference block can be obtained.

[0445] For example, by applying the Sobel mask or the Prewitt mask illustrated in FIG. 17 to a reference block, the horizontal slope Dx and the vertical slope Dy for samples within the reference block can be obtained.

[0446] Meanwhile, when using a 3x3 mask, the horizontal gradient Dx and the vertical gradient Dy may not be obtained for samples adjacent to the boundary of the reference block. That is, when the size of the reference block is NxM, the horizontal gradient Dx and the vertical gradient Dy may be obtained for (N-2)x(M-2) samples.

[0447] Alternatively, the horizontal gradient Dx and the vertical gradient Dy can be obtained by using the restored samples around the reference block, up to the samples adjacent to the boundary of the reference block.

[0448] Based on the horizontal slope Dx and the vertical slope Dy of the samples in the reference block, the amplitude of each sample in the reference block can be calculated, and a histogram can be generated based on the amplitude of each sample in the reference block.

[0449] Afterwards, the intra prediction mode with the largest amplitude on the histogram can be set as the intra prediction mode of the current block.

[0450] Meanwhile, in the present embodiment, the reference block may represent a predicted block of the current block. That is, in the present embodiment, the reference block may be replaced with a predicted block of the current block obtained through inter prediction.

[0451] As another example, multiple intra predictions can be performed on the current block based on multiple intra prediction modes, and the results of the multiple intra predictions can be compared with the predicted block of the current block (i.e., the inter prediction results) to derive the intra prediction mode of the current block. Here, intra prediction for the current block can be performed based on reference samples surrounding the current block.

[0452] Figure 38 illustrates an example of deriving an intra prediction mode of a current block by performing multiple intra predictions on the current block.

[0453] Specifically, the cost for an intra-prediction mode can be calculated based on the difference between a prediction block obtained by applying intra-prediction to the current block (i.e., an intra-prediction block) and a prediction block of the current block induced by the motion information of the current block (i.e., an inter-prediction block). For example, the SAD between the intra-prediction block and the inter-prediction block can be set as the cost of the intra-prediction mode. In other words, an intra-prediction mode that generates an intra-prediction block that is most similar to the inter-prediction block can be set as the intra-prediction mode of the current block.

[0454] Alternatively, as in the examples illustrated in FIGS. 21 to 23, multiple intra predictions may be performed on a reference region surrounding the current block, and then the cost of each intra prediction mode may be calculated. Here, the cost of the intra prediction mode may be calculated based on the difference between the predicted sample obtained by performing intra prediction on the reference region and the restored sample within the reference region.

[0455] Afterwards, based on the intra prediction mode with the lowest cost, a second set of transformation kernels for the current block can be selected.

[0456] As another example, after deriving multiple intra prediction modes for the current block, it is also possible to determine whether a second transformation / second inverse transformation is applied to the current block based on the derived multiple intra prediction modes.

[0457] For example, if the cost is calculated for each intra prediction mode, two intra prediction modes with lower costs can be selected.

[0458] Thereafter, based on the costs of the two selected intra prediction modes, it may be determined whether to apply the second transformation / second inverse transformation to the current block. For example, the difference between the two costs may be compared with a first threshold to determine whether to apply the second transformation / second inverse transformation to the current block. Only when the difference is smaller than the first threshold may the second transformation / second inverse transformation be applied to the current block.

[0459] Alternatively, the ratio between the two costs can be compared with a second threshold to determine whether to apply the second transformation / second inverse transformation to the current block. For example, the second transformation / second inverse transformation can be applied to the current block only if the ratio is less than the second threshold.

[0460] Meanwhile, when a partial transformation is applied, the second transformation / second inverse transformation may not be performed on the current block. That is, for the transformation target area within the current block, only the first transformation / first inverse transformation may be performed, and the second transformation / second inverse transformation may not be performed.

[0461] Accordingly, if it is determined that a partial transformation is applied to the current block, encoding / decoding of information indicating whether to perform the second transformation / second inverse transformation may be omitted.

[0462] Alternatively, after applying a partial transform based on the first transform / first inverse transform to the current block, a second transform / second inverse transform may be performed on the intermediate transform coefficients induced by the first transform / first inverse transform.

[0463] For example, if the encoding mode of the current block is intra prediction, a first transform may be performed on the transform application region of the current block according to a partial transform, and then a second transform may be performed on the intermediate transform coefficients within the transform application region. In the decoder, a second inverse transform may be performed on the transform application region of the current block, and then a first inverse transform may be performed on the intermediate transform coefficients within the transform application region.

[0464] Meanwhile, even if the encoding mode of the current block is inter prediction, it may be permissible to apply a partial transformation. In this case, the intra prediction mode of the current block may be derived using only prediction samples within the transformation application area (i.e., the area corresponding to the transformation application area within the reference block).

[0465] For example, the horizontal gradient Dx and the vertical gradient Dy may be calculated only for prediction samples belonging to the transformation application area within the current block, and the horizontal gradient and the vertical gradient may not be calculated for prediction samples belonging to the remaining area.

[0466] Alternatively, multiple intra predictions can be performed only for the transformation-applied region within the current block. In this case, the cost for each intra prediction mode can be derived by comparing the intra prediction samples in the transformation-applied region with the inter prediction samples in the transformation-applied region.

[0467] Depending on certain conditions, it may be determined whether the second transformation / second inverse transformation is applied to the transformation application area. For example, it may be determined whether the second transformation / second inverse transformation is applied to the transformation application area based on at least one of whether the size of the transformation application area is greater than or equal to a threshold or whether the transformation application area is the first area within the current block.

[0468] Meanwhile, when the combined prediction mode is applied, the prediction block of the current block can be derived by weighting the prediction block obtained by intra prediction (i.e., the intra-prediction block) and the block obtained by inter prediction (i.e., the inter-prediction block). When the combined prediction mode is applied, a second transformation / second inverse transformation can be applied to the current block. In this case, the second transformation kernel set of the current block can be selected based on the intra-prediction mode used to derive the intra-prediction block.

[0469] Meanwhile, when a combined prediction mode is applied, intra prediction can be performed based on the default mode. Here, the default mode can be a planar mode or a DC mode.

[0470] As another example, when the combined prediction mode is applied, the intra prediction mode of the current block may be derived based on a reference block (i.e., an inter prediction block) indicated by the motion information of the current block or an inter prediction block of the current block.

[0471] For example, according to the example illustrated in FIG. 37, the intra prediction mode of the current block can be derived from the histogram.

[0472] Alternatively, according to the example illustrated in FIG. 38, the intra prediction mode of the current block can be derived by comparing the inter prediction block and the intra prediction block.

[0473] Meanwhile, when the combined prediction mode is applied, the prediction block of the current block is derived by weighting the intra-prediction block and the inter-prediction block. At this time, based on the weights for the weighted sum operation, it can be determined whether to perform the second transformation / second inverse transformation on the current block.

[0474] For example, a second transformation / second inverse transformation can be performed on the current block only if the weight assigned to the intra prediction block is greater than or equal to the weight assigned to the inter prediction block.

[0475] Additionally, instead of selecting a second set of transform kernels based on the intra prediction mode used to obtain the intra prediction block, a new intra prediction mode can be derived based on the prediction block of the current block (i.e., the prediction block derived by weighting the intra prediction block and the inter prediction block). Based on the newly derived intra prediction mode, a second set of transform kernels for the current block can be determined.

[0476] For example, according to the example illustrated in FIG. 37, a histogram can be constructed based on the prediction block of the current block, and then an intra prediction mode of the current block can be newly derived from the histogram.

[0477] Alternatively, according to the example illustrated in FIG. 38, the cost for each intra prediction mode may be calculated by comparing the final prediction block of the current block with each of the intra prediction blocks obtained through multiple intra predictions.

[0478] Meanwhile, in FIG. 6 and mathematical expression 1, it is exemplified that when the planar mode is applied, the prediction sample of the current block is derived through a weighted sum operation of the horizontal direction interpolation sample P1 and the vertical direction interpolation sample P2.

[0479] As another example, the horizontal interpolation sample P1 can be set as the prediction sample of the current block, or the vertical interpolation sample P2 can be set as the prediction sample of the current block. Setting the horizontal interpolation sample P1 as the prediction sample of the current block can be defined as the horizontal planar mode, and setting the vertical interpolation sample P2 as the prediction sample of the current block can be defined as the vertical planar mode. In addition, obtaining the prediction sample of the current block through a weighted sum operation between the horizontal interpolation sample P1 and the vertical interpolation sample P2 can be defined as the normal planar mode.

[0480] Information indicating the type of planar mode may be encoded and signaled. The information may be an index indicating one of the normal planar mode, the horizontal planar mode, or the vertical planar mode.

[0481] Alternatively, after encoding / decoding a flag indicating whether the general planar mode is applied, if the flag indicates that the general planar mode is not applied, a flag indicating the direction of the planar mode may be additionally encoded / decoded. The direction flag may indicate either a horizontal direction or a vertical direction.

[0482] Meanwhile, if the horizontal planar mode is applied to the current block, the second transformation kernel set of the current block can be selected assuming that the intra prediction mode is the horizontal mode (i.e., mode 18). In addition, if the vertical planar mode is applied to the current block, the second transformation kernel set of the current block can be selected assuming that the intra prediction mode is the vertical mode (i.e., mode 50).

[0483] Alternatively, conversely, if the horizontal planar mode is applied to the current block, the intra prediction mode may be assumed to be the vertical mode, and if the vertical planar mode is applied to the current block, the intra prediction mode may be assumed to be the horizontal mode.

[0484] Meanwhile, in the example illustrated in FIG. 7, when the intra prediction mode of the current block is the DC mode, the average value (DCval) is derived using at least one of the upper reference samples or the left reference samples of the current block, and the derived average value is set as the prediction sample of the current block. That is, the DC mode can be subdivided into a normal DC mode that derives the average value using both the upper reference samples and the left reference samples, an upper DC mode that derives the average value using only the upper reference samples, and a left DC mode that derives the average value using only the left reference samples.

[0485] Depending on the DC mode type, the second transform kernel set of the current block can be selected. For example, when the general DC mode is applied, the second transform kernel set with index 1 can be selected according to the example in Table 5. On the other hand, when the left DC mode is applied to the current block, the second transform kernel set of the current block can be selected assuming that the intra prediction mode is a horizontal direction mode (i.e., mode 18). In addition, when the top DC mode is applied to the current block, the second transform kernel set of the current block can be selected assuming that the intra prediction mode is a vertical direction mode (i.e., mode 50).

[0486] Alternatively, conversely, if the left DC mode is applied to the current block, the intra prediction mode may be assumed to be a vertical mode, and if the top DC mode is applied to the current block, the intra prediction mode may be assumed to be a horizontal mode.

[0487] In the example of Table 5, if the current block is a chroma component block and the intra prediction mode of the current block is a prediction mode that applies prediction parameters to the reconstructed luma block (i.e., CCLM mode), a preset one of the second transformation kernel sets is selected for the current block. That is, in Table 5, if the intra prediction mode of the current block is CCLM mode, the second transformation kernel set with an index of 0 is selected.

[0488] Unlike the example in Table 5, if the intra prediction mode of the chroma block is a prediction mode that applies prediction parameters to the restored luma block, a new intra prediction mode can be derived based on the prediction block of the chroma block, and then a second set of transform kernels of the chroma block can be selected based on the newly derived intra prediction mode.

[0489] That is, the prediction block of the chroma block can be derived by applying prediction parameters to the reconstructed luma block, while the second set of transform kernels of the chroma block can be derived based on the newly derived intra prediction mode.

[0490] Here, the new intra prediction mode may be derived through a histogram obtained by applying a predefined filter to a reference area around a chroma block (see FIGS. 15 to 18).

[0491] Alternatively, a new intra prediction mode may be derived by performing multiple intra predictions based on multiple intra prediction modes in a reference area around a chroma block, and then based on the cost of each of the multiple intra prediction modes (see FIGS. 20 to 23).

[0492] Alternatively, if the intra prediction mode of the chroma block is a prediction mode that applies prediction parameters to the reconstructed luma block, a second set of transform kernels of the chroma block may be selected based on the intra prediction mode of the luma block.

[0493] Meanwhile, in the above-described example, it has been exemplified that the intra prediction mode of the current block is derived, and then the second set of transform kernels for the current block is selected based on the derived intra prediction mode. Not limited to the illustrated example, a set of transform kernels or a set of transform kernels for the first transform / first inverse transform may be selected based on the intra prediction mode of the current block, or at least one of whether to split the current block or the splitting direction may be determined.

[0494]

[0495] Applying the embodiments described above, focusing on the decoding or encoding process, to the encoding or decoding process is within the scope of the present disclosure. Changing the embodiments described above, in a given order, to a different order is also within the scope of the present disclosure.

[0496] Although the above-described disclosure is described based on a series of steps or a flowchart, this does not limit the chronological order of the invention, and may be performed simultaneously or in a different order as needed. In addition, each component (e.g., unit, module, etc.) constituting the block diagram in the above-described disclosure may be implemented as a hardware device or software, or multiple components may be combined to be implemented as a single hardware device or software. For example, the hardware device may include at least one of a processor for performing calculations, a memory for storing data, a transmitter for transmitting data, and a receiver for receiving data.

[0497] The above-described disclosure may be implemented in the form of program commands that can be executed by various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, etc., either singly or in combination.

[0498] In addition, according to the present disclosure, a computer-readable recording medium can be provided that stores a bitstream generated by the above-described encoding method. The bitstream can be transmitted by an encoding device, and a decoding device can receive the bitstream and decode an image.

[0499] Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program instructions, such as ROMs, RAMs, and flash memories. The hardware devices may be configured to operate as one or more software modules to perform processing according to the present disclosure, and vice versa.

[0500] The present disclosure may be applied to a computing or electronic device capable of encoding / decoding a video signal.

Claims

1. A step of obtaining a prediction block of the current block; A step of obtaining a residual block of the current block by applying an inverse transformation to the current block; and A step of restoring the current block by combining the above prediction block and the above residual block, The above inverse transformation includes a second inverse transformation for the second transformation and a first inverse transformation for the first transformation, The second set of transform kernels for the second inverse transform is selected based on the intra prediction mode of the current block, An image decoding method, characterized in that, if the encoding mode of the current block is not intra prediction, the intra prediction mode of the current block is derived by referring to the predicted block of the current block or a previously restored area before the current block.

2. In paragraph 1, An image decoding method, characterized in that the intra prediction mode is derived from a histogram derived from a reference block of the current block.

3. In paragraph 2, An image decoding method, characterized in that when the encoding mode is inter prediction, the reference block is indicated by a motion vector of the current block within a reference picture of the current block.

4. In paragraph 2, A video decoding method, characterized in that when the encoding mode is an intra block copy, the reference block is indicated by a block vector of the current block in the current picture.

5. In paragraph 2, An image decoding method, characterized in that the above histogram is in the form of an accumulated amplitude value of each intra prediction mode.

6. In paragraph 5, An image decoding method, characterized in that the amplitude value is derived based on the horizontal slope and vertical slope of the reference sample within the reference block.

7. In paragraph 2, An image decoding method, wherein when a partial transformation is applied to the current block, the histogram is derived from a portion of an area within the reference block, and the portion of the area corresponds to a transformation target area within the current block.

8. In paragraph 1, A video decoding method, wherein the intra prediction mode is the one having the lowest cost among a plurality of intra prediction modes.

9. In paragraph 8, A method for decoding an image, characterized in that the cost of each of the plurality of intra prediction modes is derived based on the difference between a temporary prediction block obtained by performing intra prediction on the current block and the prediction block.

10. In paragraph 8, A method for decoding an image, characterized in that the cost of each of the plurality of intra prediction modes is derived based on the difference between a temporary prediction block obtained by performing intra prediction on a reference area around the current block and the reference area.

11. In paragraph 1, When a prediction mode combining intra prediction and inter prediction is applied to the current block, the prediction block is obtained by weighting the intra prediction block and the inter prediction block, A video decoding method, wherein the intra prediction mode is used to obtain the intra prediction block of the current block.

12. In paragraph 1, The encoding mode of the current block is intra prediction, An image decoding method, characterized in that, when the intra prediction mode is a planar mode, the second transformation kernel set is determined according to the type of the planar mode.

13. In paragraph 12, If the type of the above planar mode is a vertical direction planar mode, the second transformation kernel set is selected by considering that the intra prediction mode of the current block is a vertical direction mode, An image decoding method characterized in that, when the type of the planar mode is a horizontal direction planar mode, the second transform kernel set is selected by considering that the intra prediction mode of the current block is a horizontal direction mode.

14. Step of obtaining a prediction block of the current block; A step of obtaining a residual block of the current block based on the above prediction block; and Comprising a step of applying a transformation to the residual block, The above transformation includes a first transformation and a second transformation, The second transformation kernel set for the second transformation is selected based on the intra prediction mode of the current block, A video encoding method, characterized in that, if the encoding mode of the current block is not intra prediction, the intra prediction mode of the current block is derived by referring to the predicted block of the current block or a previously restored area before the current block.

15. A processor that generates compressed video data; and In a device including a transmitter for transmitting the compressed video data, The process of generating the above compressed video is as follows: A step of obtaining a prediction block of the current block; A step of obtaining a residual block of the current block based on the above prediction block; and Comprising a step of applying a transformation to the residual block, The above transformation includes a first transformation and a second transformation, The second transformation kernel set for the second transformation is selected based on the intra prediction mode of the current block, A device for transmitting compressed video data, characterized in that, if the encoding mode of the current block is not intra prediction, the intra prediction mode of the current block is derived by referring to the predicted block of the current block or a previously restored area before the current block.

Images