Image encoding / decoding method and recording medium for storing bitstream

Abstract

The image decoding method according to the present disclosure may comprise the steps of: determining whether a partial transform is applied to a current block; decoding residual coefficients belonging to a first region within the current block when the partial transform is applied to the current block; dequantizing the residual coefficients within the first region; and performing an inverse transform on the dequantized residual coefficients within the first region, thereby obtaining residual samples. Here, when the partial transform is applied to the current block, the division direction of the current block may be one of the horizontal direction, the vertical direction, the upper right diagonal direction, and the upper left diagonal direction.

Description

Video encoding / decoding method and recording medium for storing bitstream

[0001] The present disclosure relates to a video signal processing method and device.

[0002] Recently, the demand for high-resolution, high-quality images, such as HD (High Definition) and UHD (Ultra High Definition) images, is increasing across various application fields. As image data becomes higher in resolution and quality, the relative amount of data increases compared to conventional image data. Therefore, transmitting image data using existing media such as wired and wireless broadband lines or storing it using existing storage media leads to increased transmission and storage costs. To address these issues arising from the increasing resolution and quality of image data, high-efficiency image compression technologies can be utilized.

[0003] There are various technologies for image compression, such as inter-picture prediction technology that predicts pixel values ​​included in the current picture from pictures before or after the current picture, intra-picture prediction technology that predicts pixel values ​​included in the current picture using pixel information in the current picture, and entropy encoding technology that assigns short codes to values ​​with high frequency of appearance and long codes to values ​​with low frequency of appearance. Using these image compression technologies, image data can be effectively compressed and transmitted or stored.

[0004] Meanwhile, as demand for high-resolution video grows, so does the demand for stereoscopic video content as a new video service. Discussions are underway on video compression technologies to effectively deliver high-resolution and ultra-high-resolution stereoscopic video content.

[0005] The present disclosure aims to provide a method and a device for applying transformation / inverse transformation to only a portion of a current block.

[0006] The present disclosure aims to provide a method and a device for diagonally dividing a current block in applying a partial transformation to the current block.

[0007] The present disclosure aims to provide a method for rearranging residual samples / transformation coefficients included in a non-rectangular region and a device therefor.

[0008] The technical problems to be achieved in the present disclosure are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by a person having ordinary skill in the technical field to which the present disclosure belongs from the description below.

[0009] A video decoding method according to the present disclosure may include the steps of: determining whether a partial transform is applied to a current block; decoding residual coefficients belonging to a first region within the current block if the partial transform is applied to the current block; inversely quantizing the residual coefficients within the first region; and inversely transforming the inversely quantized residual coefficients within the first region to obtain residual samples. In this case, if the partial transform is applied to the current block, a division direction of the current block may be one of a horizontal direction, a vertical direction, an upper-right diagonal direction, and an upper-left diagonal direction.

[0010] In the image decoding method according to the present disclosure, the values ​​of residual coefficients belonging to the second area within the current block can be set to 0.

[0011] In the image decoding method according to the present disclosure, when the current block is divided in the horizontal direction or the vertical direction, the first region may have a square shape, and when the current block is divided in the upper right diagonal direction or the upper left diagonal direction, the first region may have a triangular shape.

[0012] In the image decoding method according to the present disclosure, the inverse quantized residual coefficients within the first region having a triangular shape can be rearranged according to a predetermined scan order.

[0013] In the image decoding method according to the present disclosure, the starting point of the scan may be an inverse quantized residual coefficient located at a right angle vertex within the first region.

[0014] In the image decoding method according to the present disclosure, the scan order can be selected from among a plurality of scan type candidates.

[0015] In the image decoding method according to the present disclosure, one of the plurality of scan type candidates can be adaptively selected based on at least one of the size, shape, or position of the first region.

[0016] In the image decoding method according to the present disclosure, the inverse quantized residual coefficients can be rearranged into a two-dimensional rectangular shape.

[0017] In the image decoding method according to the present disclosure, each of the width and height of the two-dimensional square shape may have a value greater than or equal to the minimum value.

[0018] A video encoding method according to the present disclosure may include the steps of applying a transformation to residual samples of a first area within a current block; quantizing transform coefficients obtained by the transformation; and encoding residual coefficients obtained by the quantization. In this case, when partial transformation information for the current block is encoded and the partial transformation is applied to the current block, the division direction of the current block may be one of a horizontal direction, a vertical direction, an upper-right diagonal direction, and an upper-left diagonal direction.

[0019] According to the present disclosure, a computer-readable recording medium for storing a bitstream generated by an image encoding method can be provided.

[0020] According to the present disclosure, a computer-readable recording medium having recorded thereon a command for performing an image decoding method or an image encoding method can be provided.

[0021] The features briefly summarized above regarding the present disclosure are merely exemplary aspects of the detailed description of the present disclosure that follows and do not limit the scope of the present disclosure.

[0022] According to the present disclosure, there is an effect of reducing the amount of data to be encoded / decoded by applying transformation / inverse transformation only to a portion of the current block.

[0023] According to the present disclosure, when applying a partial transformation to a current block, there is an effect of increasing encoding / decoding efficiency by diagonally dividing the current block into two.

[0024] According to the present disclosure, there is an effect of increasing the compression efficiency by transformation by rearranging residual samples / transform coefficients included in a non-rectangular shape area.

[0025] The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects that are not mentioned will be clearly understood by a person having ordinary skill in the art to which the present disclosure pertains from the description below.

[0026] FIG. 1 is a block diagram illustrating an image encoding device according to an embodiment of the present disclosure.

[0027] FIG. 2 is a block diagram showing an image decoding device according to an embodiment of the present disclosure.

[0028] Figure 3 is a flowchart of a method for encoding a residual block in an encoder.

[0029] Figure 4 is a flowchart of a method for restoring a residual block in a decoder.

[0030] Figures 5 and 6 are drawings showing examples in which the second transformation is applied.

[0031] Figures 7 and 8 illustrate a second transformation based on a second transformation kernel of an asymmetric shape.

[0032] Figure 9 illustrates an example in which whether to encode information indicating whether to apply the second transformation is determined based on the position of the last non-zero coefficient.

[0033] Figure 10 illustrates restricted area candidates for a 4x4 sized block.

[0034] Figure 11 is a drawing showing an example in which a second transformation kernel of a predefined size is applied.

[0035] Figure 12 is a drawing for explaining an example in which partial transformation is applied to the current block.

[0036] Figures 13 and 14 illustrate the division types of the current block.

[0037] Figure 15 shows an example in which the current block is divided diagonally.

[0038] Figure 16 shows an example of selecting residual samples for transformation within the current block.

[0039] Figure 17 shows an example of a case where the division shape of the current block that is divided diagonally is asymmetric.

[0040] Figure 18 shows an example of selecting residual samples for transformation within the current block.

[0041] Figure 19 shows an example in which selected residual samples are arranged in a two-dimensional form.

[0042] Figures 20 and 21 are flowcharts showing the operation of an encoder and a decoder when partial transformation is applied.

[0043] The present disclosure may be modified in various ways and encompasses numerous embodiments. Specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present disclosure to specific embodiments, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and technical scope of the present disclosure. Throughout the description of each drawing, similar reference numerals have been used to designate similar components.

[0044] While terms such as "first" and "second" may be used to describe various components, these components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present disclosure, a first component could be referred to as a "second component," and similarly, a second component could also be referred to as a "first component." The term "and / or" includes a combination of multiple related items described herein or any of multiple related items described herein.

[0045] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0046] The terminology used in this application is only used to describe specific embodiments and is not intended to limit the present disclosure. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprise" or "have" indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0047] Hereinafter, preferred embodiments of the present disclosure will be described in more detail with reference to the attached drawings. Hereinafter, identical components in the drawings will be designated by the same reference numerals, and redundant descriptions of identical components will be omitted.

[0048] FIG. 1 is a block diagram illustrating an image encoding device according to an embodiment of the present disclosure.

[0049] Referring to FIG. 1, a video encoding device (100) may include a picture segmentation unit (110), a prediction unit (120, 125), a transformation unit (130), a quantization unit (135), a reordering unit (160), an entropy encoding unit (165), an inverse quantization unit (140), an inverse transformation unit (145), a filter unit (150), and a memory (155).

[0050] Each component shown in Fig. 1 is independently depicted to represent different characteristic functions in the video encoding device, and does not mean that each component is composed of separate hardware or a single software component. That is, each component is listed and included as a separate component for convenience of explanation, and at least two components among each component may be combined to form a single component, or one component may be divided into multiple components to perform a function, and such integrated and separate embodiments of each component are also included in the scope of the present disclosure as long as they do not deviate from the essence of the present disclosure.

[0051] Additionally, some components may not be essential components that perform the essential functions of the present disclosure, but may be optional components merely used to enhance performance. The present disclosure may be implemented by including only components essential to implementing the essence of the present disclosure, excluding components used solely for performance enhancement. A structure that includes only essential components, excluding optional components used solely for performance enhancement, is also within the scope of the present disclosure.

[0052] The picture splitting unit (110) can split the input picture into at least one processing unit. At this time, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). The picture splitting unit (110) can split one picture into a combination of multiple coding units, prediction units, and transform units, and select one combination of coding units, prediction units, and transform units based on a predetermined criterion (e.g., a cost function) to encode the picture.

[0053] For example, a picture can be split into multiple coding units. A recursive tree structure such as a quad tree, a ternary tree, or a binary tree can be used to split a coding unit in a picture. A coding unit that is split into other coding units starting from an image or the largest coding unit as the root can be split into as many child nodes as the number of split coding units. A coding unit that cannot be split any further according to a certain restriction becomes a leaf node. For example, assuming that a quad tree split is applied to a coding unit, a coding unit can be split into at most four different coding units.

[0054] Hereinafter, in the embodiments of the present disclosure, the encoding unit may be used to mean a unit that performs encoding or may be used to mean a unit that performs decoding.

[0055] A prediction unit may be divided into at least one square or rectangular shape of the same size within a single coding unit, or may be divided such that one prediction unit among the divided prediction units within a single coding unit has a different shape and / or size from another prediction unit.

[0056] When predicting within a screen, the transformation unit and the prediction unit can be set to be the same. In this case, the encoding unit can be divided into multiple transformation units, and then intra-screen prediction can be performed for each transformation unit. The encoding unit can be divided in the horizontal direction or the vertical direction. The number of transformation units generated by dividing the encoding unit can be 2 or 4, depending on the size of the encoding unit. Alternatively, when the size of the transformation unit is small, multiple transformation units can be set as a single prediction unit.

[0057] The prediction unit (120, 125) may include an inter-prediction unit (120) that performs inter-prediction and an intra-prediction unit (125) that performs intra-prediction. It may be determined whether to use inter-prediction or intra-prediction for an encoding unit, and specific information (e.g., reference sample line, intra-prediction mode, motion vector, reference picture, etc.) according to each prediction method may be determined. At this time, the processing unit where prediction is performed and the processing unit where the prediction method and specific contents are determined may be different. For example, the prediction method and prediction mode, etc. are determined in the encoding unit, and the prediction may be performed in the prediction unit or the transformation unit. The residual value (residual block) between the generated prediction block and the original block may be input to the transformation unit (130). In addition, the prediction mode information, motion vector information, etc. used for prediction may be encoded together with the residual value in the entropy encoding unit (165) and transmitted to the decoding device. When using a specific encoding mode, it is also possible to encode the original block as is and transmit it to the decoding unit without generating a prediction block through the prediction unit (120, 125).

[0058] The inter-screen prediction unit (120) may predict a prediction unit based on information of at least one picture among the previous or subsequent pictures of the current picture, and in some cases, may predict a prediction unit based on information of a portion of an encoded region within the current picture. The inter-screen prediction unit (120) may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

[0059] The reference picture interpolation unit can receive reference picture information from the memory (155) and generate pixel information less than an integer pixel from the reference picture. In the case of luminance pixels, a DCT-based 8-tap interpolation filter with different filter coefficients can be used to generate pixel information less than an integer pixel in units of 1 / 4 pixels. In the case of a chrominance signal, a DCT-based 4-tap interpolation filter with different filter coefficients can be used to generate pixel information less than an integer pixel in units of 1 / 8 pixels.

[0060] The motion prediction unit can perform motion prediction based on a reference picture interpolated by the reference picture interpolation unit. Various methods can be used to derive a motion vector, such as FBMA (Full search-based Block Matching Algorithm), TSS (Three Step Search), and NTS (New Three-Step Search Algorithm). The motion vector can have a motion vector value of 1 / 2 or 1 / 4 pixel unit based on the interpolated pixel. The motion prediction unit can predict the current prediction unit by using different motion prediction methods. Various methods can be used as motion prediction methods, such as the Skip method, the Merge method, the AMVP (Advanced Motion Vector Prediction) method, and the Intra Block Copy method.

[0061] The on-screen prediction unit (125) can generate a prediction block based on reference pixel information, which is pixel information within the current picture. The reference pixel information can be derived from one selected from among a plurality of reference pixel lines. The Nth reference pixel line among the plurality of reference pixel lines can include left pixels having an x-axis difference of N from the upper left pixel within the current block and upper pixels having a y-axis difference of N from the upper left pixel. The number of reference pixel lines that the current block can select can be 1, 2, 3, or 4.

[0062] If the neighboring blocks of the current prediction unit are blocks that have performed inter-screen prediction and the reference pixel is a pixel that has performed inter-screen prediction, the reference pixel included in the block that has performed inter-screen prediction can be replaced with the reference pixel information of the neighboring block that has performed intra-screen prediction. That is, if the reference pixel is unavailable, the unavailable reference pixel information can be replaced with information from at least one of the available reference pixels.

[0063] In intra-screen prediction, the prediction mode can have a directional prediction mode that uses reference pixel information according to the prediction direction, and a non-directional mode that does not use directional information when performing prediction. The mode for predicting luminance information and the mode for predicting chrominance information can be different, and the intra-screen prediction mode information used to predict luminance information or the predicted luminance signal information can be utilized to predict chrominance information.

[0064] When performing intra-screen prediction, if the size of the prediction unit and the size of the transformation unit are the same, intra-screen prediction for the prediction unit can be performed based on the pixels on the left side of the prediction unit, the pixels on the upper left side, and the pixels on the upper side.

[0065] The on-screen prediction method can generate prediction blocks by applying a smoothing filter to reference pixels according to the prediction mode. Depending on the selected reference pixel line, whether or not the smoothing filter is applied can be determined.

[0066] In order to perform an intra-screen prediction method, the intra-screen prediction mode of the current prediction unit can be predicted from the intra-screen prediction modes of prediction units existing around the current prediction unit. When the prediction mode of the current prediction unit is predicted using mode information predicted from the surrounding prediction units, if the intra-screen prediction modes of the current prediction unit and the surrounding prediction units are the same, information indicating that the prediction modes of the current prediction unit and the surrounding prediction units are the same can be transmitted using predetermined flag information, and if the prediction modes of the current prediction unit and the surrounding prediction units are different, entropy encoding can be performed to encode the prediction mode information of the current block.

[0067] Additionally, a residual block containing residual value information, which is the difference between the prediction unit that performed the prediction based on the prediction unit generated in the prediction unit (120, 125) and the original block of the prediction unit, can be generated. The generated residual block can be input to the transformation unit (130).

[0068] In the transformation unit (130), the residual block including the residual value information of the prediction unit generated through the original block and the prediction unit (120, 125) can be transformed using a transformation method such as DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), or KLT. Whether to apply DCT, DST, or KLT to transform the residual block can be determined based on at least one of the size of the transformation unit, the shape of the transformation unit, the prediction mode of the prediction unit, or the prediction mode information within the screen of the prediction unit. Meanwhile, the transformation can be performed by separating the horizontal direction and the vertical direction.

[0069] After performing transformations in the horizontal and vertical directions, a secondary transformation can be performed. The secondary transformation may be in a form in which the horizontal and vertical directions are not separated. The secondary transformation can be performed on the transformation coefficients obtained by the primary transformation to generate final transformation coefficients. Meanwhile, the number of final transformation coefficients output by the secondary transformation may be smaller than the number of transformation coefficients input for the secondary transformation. Specifically, the secondary transformation can be performed using a reduced transformation matrix having different numbers of columns and rows.

[0070] The quantization unit (135) can quantize values ​​converted to the frequency domain by the transformation unit (130). The quantization coefficients may vary depending on the block or the importance of the image. The values ​​produced by the quantization unit (135) can be provided to the dequantization unit (140) and the reordering unit (160).

[0071] The rearrangement unit (160) can perform rearrangement of coefficient values ​​for quantized residual values.

[0072] The reordering unit (160) can change a two-dimensional block-shaped coefficient into a one-dimensional vector form through a coefficient scanning method. For example, the reordering unit (160) can change the two-dimensional block-shaped coefficient into a one-dimensional vector form by scanning from the DC coefficient to the coefficient of the high-frequency region using a zig-zag scan method. Depending on the size of the conversion unit and the intra-screen prediction mode, a vertical scan that scans the two-dimensional block-shaped coefficient in the column direction, a horizontal scan that scans the two-dimensional block-shaped coefficient in the row direction, or a diagonal scan that scans the two-dimensional block-shaped coefficient in the diagonal direction may be used instead of the zig-zag scan. That is, depending on the size of the conversion unit and the intra-screen prediction mode, it is possible to determine which scan method among the zig-zag scan, the vertical scan, the horizontal scan, or the diagonal scan is to be used.

[0073] The entropy encoding unit (165) can perform entropy encoding based on the values ​​produced by the rearrangement unit (160). Entropy encoding can use various encoding methods such as, for example, Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

[0074] The entropy encoding unit (165) can encode various information such as residual value coefficient information of the encoding unit, block type information, prediction mode information, division unit information, prediction unit information, transmission unit information, motion vector information, reference frame information, block interpolation information, and filtering information from the rearrangement unit (160) and the prediction unit (120, 125).

[0075] The entropy encoding unit (165) can entropy encode the coefficient values ​​of the encoding unit input from the rearrangement unit (160).

[0076] The inverse quantization unit (140) and the inverse transformation unit (145) inversely quantize the values ​​quantized in the quantization unit (135) and inversely transform the values ​​transformed in the transformation unit (130). The residual values ​​generated in the inverse quantization unit (140) and the inverse transformation unit (145) can be combined with the predicted prediction units predicted through the motion estimation unit, motion compensation unit, and intra-screen prediction unit included in the prediction unit (120, 125) to generate a reconstructed block.

[0077] The filter unit (150) may include at least one of a deblocking filter, an offset correction unit, and an ALF (Adaptive Loop Filter).

[0078] A deblocking filter can remove block distortion caused by boundaries between blocks in a reconstructed picture. To determine whether to perform deblocking, a deblocking filter can be applied to the current block based on the pixels contained in several columns or rows within the block. When applying a deblocking filter to a block, a strong filter or a weak filter can be applied depending on the required deblocking filtering strength. Furthermore, when applying a deblocking filter, horizontal and vertical filtering can be processed in parallel when performing vertical and horizontal filtering.

[0079] The offset correction unit can correct the offset from the original image on a pixel-by-pixel basis for an image that has undergone deblocking. To perform offset correction for a specific picture, the pixels contained in the image can be divided into a certain number of regions, the regions to be offset can be determined, and the offset can be applied to those regions. Alternatively, the offset can be applied by considering the edge information of each pixel.

[0080] Adaptive Loop Filtering (ALF) can be performed based on the comparison of the filtered restored image with the original image. After dividing the pixels included in the image into predetermined groups, a filter to be applied to each group can be determined, and filtering can be performed differentially for each group. Information regarding whether to apply ALF can be transmitted by luminance signal for each coding unit (CU), and the shape and filter coefficients of the ALF filter to be applied can vary depending on each block. Furthermore, an ALF filter of the same form (fixed form) can be applied regardless of the characteristics of the target block.

[0081] The memory (155) can store a restoration block or picture produced through the filter unit (150), and the stored restoration block or picture can be provided to the prediction unit (120, 125) when performing inter-screen prediction.

[0082] FIG. 2 is a block diagram showing an image decoding device according to an embodiment of the present disclosure.

[0083] Referring to FIG. 2, the image decoding device (200) may include an entropy decoding unit (210), a rearrangement unit (215), an inverse quantization unit (220), an inverse transformation unit (225), a prediction unit (230, 235), a filter unit (240), and a memory (245).

[0084] When a video bitstream is input to a video encoding device, the input bitstream can be decoded in the opposite procedure to that of the video encoding device.

[0085] The entropy decoding unit (210) can perform entropy decoding in a procedure opposite to that of the entropy encoding unit of the video encoding device. For example, various methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) can be applied in response to the method performed in the video encoding device.

[0086] The entropy decoding unit (210) can decode information related to intra-screen prediction and inter-screen prediction performed in the encoding device.

[0087] The reordering unit (215) can perform reordering based on the method in which the bitstream entropy-decoded by the entropy decoding unit (210) is reordered by the encoding unit. The coefficients expressed in the form of a one-dimensional vector can be reordered by restoring them back to coefficients in the form of a two-dimensional block. The reordering unit (215) can perform reordering by receiving information related to the coefficient scanning performed by the encoding unit and performing reverse scanning based on the scanning order performed by the corresponding encoding unit.

[0088] The dequantization unit (220) can perform dequantization based on the quantization parameters provided from the encoding device and the coefficient values ​​of the rearranged block.

[0089] The inverse transform unit (225) can perform an inverse transform of the transform performed by the transform unit on the quantization result performed by the image encoding device. That is, at least one of an inverse transform of a secondary transform (secondary inverse transform) or an inverse transform for DCT, DST, and KLT (i.e., first inverse transform) can be performed. The inverse transform can be performed based on a transmission unit determined by the image encoding device. The inverse transform unit (225) of the image decoding device can determine a transform matrix for the second inverse transform or a transform technique (e.g., DCT, DST, KLT) for the first inverse transform according to a plurality of pieces of information such as a prediction method, the size and shape of the current block, the prediction mode, and the prediction direction within the screen. Alternatively, information for determining the transform matrix or the transform technique may be explicitly encoded and signaled.

[0090] The prediction unit (230, 235) can generate a prediction block based on the prediction block generation related information provided by the entropy decoding unit (210) and the previously decoded block or picture information provided by the memory (245).

[0091] As described above, when performing intra-screen prediction in the same manner as the operation in the video encoding device, if the size of the prediction unit and the size of the transformation unit are the same, intra-screen prediction for the prediction unit is performed based on the pixels on the left side of the prediction unit, the pixels on the upper left side, and the pixels on the upper side. However, when performing intra-screen prediction, if the size of the prediction unit and the size of the transformation unit are different, intra-screen prediction can be performed using reference pixels based on the transformation unit. In addition, intra-screen prediction using NxN division only for the minimum coding unit can be used.

[0092] The prediction unit (230, 235) may include a prediction unit determination unit, an inter-screen prediction unit, and an intra-screen prediction unit. The prediction unit determination unit may receive various information such as prediction unit information input from the entropy decoding unit (210), prediction mode information of an intra-screen prediction method, and motion prediction-related information of an inter-screen prediction method, and may distinguish a prediction unit from a current encoding unit and determine whether the prediction unit performs inter-screen prediction or intra-screen prediction. The inter-screen prediction unit (230) may perform inter-screen prediction on the current prediction unit based on information included in at least one of a previous picture or a subsequent picture of the current picture including the current prediction unit, using information necessary for inter-screen prediction of the current prediction unit provided from the video encoding device. Alternatively, inter-screen prediction may be performed based on information on a pre-restored portion of the current picture including the current prediction unit.

[0093] In order to perform inter-screen prediction, it is possible to determine whether the motion prediction method of the prediction unit included in the encoding unit is Skip Mode, Merge Mode, AMVP Mode, or Intra-screen Block Copy Mode based on the encoding unit.

[0094] The intra-screen prediction unit (235) can generate a prediction block based on pixel information within the current picture. If the prediction unit is a prediction unit that has performed intra-screen prediction, intra-screen prediction can be performed based on intra-screen prediction mode information of the prediction unit provided by the video encoding device. The intra-screen prediction unit (235) can include an AIS (Adaptive Intra Smoothing) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter is a part that performs filtering on the reference pixels of the current block, and can determine and apply whether to apply the filter according to the prediction mode of the current prediction unit. AIS filtering can be performed on the reference pixels of the current block using the prediction mode and AIS filter information of the prediction unit provided by the video encoding device. If the prediction mode of the current block is a mode that does not perform AIS filtering, the AIS filter may not be applied.

[0095] The reference pixel interpolation unit can generate a reference pixel of a pixel unit less than an integer value by interpolating the reference pixel when the prediction mode of the prediction unit is a prediction unit that performs intra-screen prediction based on the pixel value interpolated from the reference pixel. If the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixel, the reference pixel may not be interpolated. The DC filter can generate a prediction block through filtering when the prediction mode of the current block is the DC mode.

[0096] The restored block or picture may be provided to a filter unit (240). The filter unit (240) may include a deblocking filter, an offset correction unit, and an ALF.

[0097] Information regarding whether a deblocking filter has been applied to a corresponding block or picture may be received from a video encoding device, and if a deblocking filter has been applied, information regarding whether a strong or weak filter has been applied. The deblocking filter of the video decoding device may receive information related to the deblocking filter provided by the video encoding device, and the video decoding device may perform deblocking filtering on the corresponding block.

[0098] The offset correction unit can perform offset correction on the restored image based on the type of offset correction applied to the image during encoding and offset value information.

[0099] ALF can be applied to an encoding unit based on information such as whether ALF is applied and ALF coefficient information provided from an encoding device. This ALF information can be provided by being included in a specific parameter set.

[0100] The memory (245) can store a restored picture or block so that it can be used as a reference picture or reference block, and can also provide the restored picture to an output unit.

[0101] As described above, in the following embodiments of the present disclosure, for convenience of explanation, the term coding unit is used as an encoding unit, but it may also be a unit that performs not only encoding but also decoding.

[0102] In addition, the current block represents a block to be encoded / decoded, and may represent a coding tree block (or coding tree unit), an encoding block (or encoding unit), a transform block (or transform unit), a prediction block (or prediction unit), or a block to which an in-loop filter is applied, depending on the encoding / decoding step. In this specification, a 'unit' represents a basic unit for performing a specific encoding / decoding process, and a 'block' may represent a pixel array of a predetermined size. Unless otherwise distinguished, 'block' and 'unit' may be used with the same meaning. For example, in the embodiment described below, an encoding block (coding block) and an encoding unit (coding unit) may be understood to have the same meaning.

[0103] Furthermore, we will refer to the picture that contains the current block as the current picture.

[0104] In an encoder, the current picture can be divided into multiple reference blocks. Here, the reference blocks can be called Coding Tree Units (CTUs) or Coding Tree Blocks (CTBs).

[0105] The size of the reference block may be predefined in the encoder and decoder. Alternatively, information related to the size of the reference block may be encoded and signaled to the decoder. The information may be encoded / decoded via an upper header. For example, the information may be encoded / decoded via a sequence parameter set or a picture header.

[0106] The reference block may be further divided into multiple blocks (i.e., multiple coding blocks) based on a tree structure partitioning. Here, the tree structure partitioning may include at least one of a quad tree partitioning, a binary tree partitioning, or a ternary tree partitioning.

[0107] A prediction block for the current block can be obtained by performing a prediction block on the current block generated by dividing the reference block. Specifically, a prediction block for the current block can be obtained through inter-prediction or intra-prediction.

[0108] Inter-prediction may be used to remove redundant data between pictures, and intra-prediction may be used to remove redundant data within a picture. For example, a prediction block of the current block may be generated from a reference picture using motion information of the current block, or an intra-prediction mode of the current block may be determined, and then a prediction block of the current block may be generated from reference samples of the current block. Here, the motion information may include at least one of a motion vector, a reference picture index, and a prediction direction.

[0109] After performing a prediction for the current block, the residual block can be obtained by differentiating the original block and the predicted block.

[0110] Figure 3 is a flowchart of a method for encoding a residual block in an encoder, and Figure 4 is a flowchart of a method for restoring a residual block in a decoder.

[0111] In the encoder, residual coefficients can be obtained by performing at least one of transformation or quantization on the residual block (S310, S320). If quantization is omitted for the residual block, the residual coefficients may refer to transform coefficients obtained by transformation. Alternatively, if quantization is performed on the residual block, the residual coefficients may refer to quantized transform coefficients obtained by quantizing the transform coefficients.

[0112] Additionally, in the encoder, the residual coefficients can be encoded (specifically, entropy encoded) and the encoded data can be transmitted to the decoder (S330).

[0113] The decoder decodes the encoded data to restore the residual coefficients (S410). Then, by performing inverse quantization on the residual coefficients, transform coefficients (i.e., inverse quantized residual coefficients) can be derived, and by performing inverse transformation on the transform coefficients, a residual block can be derived (S420, S430).

[0114] Information indicating whether a transformation is applied to the current block may be encoded and signaled. For example, transform_skip_flag may be encoded and signaled. When transform_skip_flag is 1, it indicates that no transformation is applied to the current block. Here, the transformation may include not only the first transformation described below, but also the second transformation. When transform_skip_flag is 0, it indicates that a transformation is applied to the current block. When transform_skip_flag is 0, the first transformation is necessarily applied to the current block, while the second transformation may be optionally applied.

[0115] The transformation for the current block may be performed based on at least one of a plurality of transformation kernel candidates. For example, the transformation kernel applicable to the current block may be a transformation kernel of the Discrete Cosine Transform (DCT) family or a transformation kernel of the Discrete Sine Transform (DST) family.

[0116] Mathematical expressions 1 to 3 represent the basis functions of the transform kernels applicable to the current block. Mathematical expression 1 represents the basis function for DCT-2, Mathematical expression 2 represents the basis function for DCT-8, and Mathematical expression 3 represents the basis function for DST-7.

[0117]

[0118]

[0119]

[0120] When multiple transform kernel candidates exist, information indicating a transform kernel applied to the current block among the multiple transform kernel candidates can be encoded and signaled. Here, the transform kernel candidate can include at least one of DCT-2, DST-7, or DCT-8. Additionally, the information can be an index indicating one of the multiple transform kernel candidates.

[0121] Meanwhile, the horizontal and vertical transformations of the current block can be separated. At this time, a common transformation kernel can be applied to both the horizontal and vertical directions. That is, when one of multiple transformation kernel candidates is selected, the selected transformation kernel can be applied to both the horizontal and vertical transformations of the current block.

[0122] Alternatively, the transform kernels for the horizontal and vertical directions can be determined independently. In this case, the encoder can encode and signal information indicating the transform kernel for the horizontal direction and information indicating the transform kernel for the vertical direction, respectively.

[0123] Alternatively, multiple transform kernel combination candidates indicating combinations of horizontal transform kernels and vertical transform kernels may be defined, and then information indicating one of the multiple transform kernel combination candidates may be encoded and signaled. Table 1 illustrates multiple transform kernel combination candidates.

[0124] Index 01234 Horizontal Conversion DCT-2DST-7DCT-8DST-7DCT-8 Vertical Conversion DCT-2DST-7DST-7DCT-8DCT-8

[0125] In the decoder, a transformation kernel for the horizontal direction and a transformation kernel for the vertical direction can be determined based on an index indicating one of a plurality of transformation kernel combination candidates.

[0126] Meanwhile, information indicating whether the transformation kernels for horizontal and vertical transformations are determined integrally may be encoded / decoded. The information may be a 1-bit flag.

[0127] For example, if the information indicates that the transform kernels for horizontal and vertical transformations are determined integrally, an index indicating one of the multiple transform kernel candidates may be encoded / decoded. The transform kernel indicated by the index may be applied to both the horizontal and vertical transformations of the current block.

[0128] On the other hand, if the above information indicates that the transformation kernels for the horizontal and vertical transformations are not determined integrally, an index indicating one of a plurality of transformation kernel combination candidates may be encoded / decoded. The combination of the horizontal and vertical transformation kernels indicated by the index may be applied to the current block.

[0129] After performing the above-described transformation, an additional transformation may be performed on the current block. For the convenience of explanation, the transformation performed by the DCT or DST-based transformation kernel will be referred to as the first transformation, and the transformation additionally applied to the result of the first transformation will be referred to as the second transformation. In addition, the transformation coefficients generated as a result of the first transformation will be referred to as the first transformation coefficients, and the transformation coefficients generated as a result of the second transformation will be referred to as the second transformation coefficients.

[0130] In the encoder, a second transform can be performed on the first transform coefficients generated as a result of performing the first transform, thereby generating second transform coefficients.

[0131] When both the first transform and the second transform are performed in the encoder, the decoder can perform a second inverse transform (i.e., an inverse transform of the second transform) on the inverse quantized residual coefficients (i.e., the second transform coefficients) to generate the first transform coefficients. Then, by performing a first inverse transform (i.e., an inverse transform for the first transform) on the first transform coefficients, the residual samples can be obtained.

[0132] The second transform may be applied to at least some of the first transform coefficients. For example, depending on the size of the second transform kernel, the second transform may be applied to 16, 48, or 64 first transform coefficients. The shape of the region containing the first transform coefficients to which the second transform is applied may be square, non-square, or polygonal.

[0133] Mathematical expression 4 shows the application aspect of the second transformation.

[0134]

[0135] When the second transformation is performed, the first transformation coefficients can be aligned in one dimension. For example, in the above mathematical expression 4, A Nx1 represents the first transformation coefficients consisting of N rows and 1 column. Also, B Rx1 represents the second transformation coefficients consisting of R rows and 1 column. T RXN represents the second transformation kernel consisting of R rows and N columns.

[0136] Figures 5 and 6 are drawings showing examples in which the second transformation is applied.

[0137] Fig. 5 illustrates an example in which the second transform kernel is 64x64 in size. The first transform coefficients generated as a result of the first transform within an 8x8 block can be arranged in one dimension. At this time, the first transform coefficients can be scanned using a predetermined scan method to generate the one-dimensional array. The predetermined scan method can include at least one of a diagonal scan, a horizontal scan, a vertical scan, or a raster scan.

[0138] When an input matrix of size 64x1 is generated through the above rearrangement, the second transformation coefficient can be derived through matrix multiplication between the second transformation kernel of size 64x64 and the input matrix of size 64x1.

[0139] As a result of performing the second transform, 64 second transform coefficients are generated, and the second transform coefficients within an 8x8 block can be rearranged. After quantizing the 8x8 block in which the second transform coefficients are rearranged, the quantized transform block can be encoded.

[0140] Fig. 6 illustrates an example in which the second transform kernel is 48x48 in size. Among the first transform coefficients generated as a result of the first transform within an 8x8 block, 48 first transform coefficients can be rearranged into one dimension. At this time, the 48 first transform coefficients may be included in a polygonal area excluding a 4x4 sized sub-block at the lower right within the 8x8 block.

[0141] When the 48 first transformation coefficients are rearranged into one dimension to generate an input matrix of size 48x1, the second transformation coefficients can be derived through matrix multiplication between the second transformation kernel of size 48x48 and the input matrix of size 48x1.

[0142] As a result of performing the second transformation, 48 second transformation coefficients are generated, and the second transformation coefficients can be rearranged within an 8x8 block. For example, the 48 second transformation coefficients can be rearranged within a polygonal region excluding a 4x4 sized sub-block at the lower right within an 8x8 block.

[0143] In areas where the second transform coefficients are not placed, the first transform coefficients can be maintained as is. After applying quantization to a block including the second transform coefficients and the first transform coefficients, the quantized transform block can be encoded.

[0144] Alternatively, the transform coefficients in the area where the second transform coefficients are not placed may be set to 0. That is, the values ​​of the transform coefficients in the area where the second transform is not applied may be set to 0, and then quantization and encoding may be performed.

[0145] The size of the second transform kernel can be determined based on the size of the current block. For example, if at least one of the width or height of the current block is 4, the second transform can be applied to 16 first transform coefficients. Conversely, if the width and height of the current block are 8 or greater, the second transform can be applied to 48 or 64 first transform coefficients.

[0146] Alternatively, information indicating the size and type of the second transform kernel may be encoded and signaled. The information may be signaled at the block level. For example, information specifying at least one of the number of rows or the number of columns of the transform size may be encoded. Alternatively, a different index may be assigned to each combination of the number of rows and the number of columns, and then an index specifying one of the combinations may be encoded. Alternatively, a different index may be assigned to each of a plurality of second transform kernel candidates, and then an index specifying one of the second transform kernel candidates may be encoded. Here, each of the plurality of second transform kernel candidates may differ in at least one of the size or the coefficient.

[0147] Alternatively, based on the size of the current block, the size of the second transformation kernel may be determined, and then an index specifying one of a plurality of second transformation kernel candidates having the determined size may be encoded.

[0148] In the examples illustrated in Figures 5 and 6, a second transformation kernel is used, with the same number of rows and columns. To simplify the second transformation, the number of rows and the number of columns can also be set differently.

[0149] Figures 7 and 8 illustrate a second transformation based on a second transformation kernel of an asymmetric shape.

[0150] The number of rows R of the second transformation kernel can be set to a value smaller than the number of columns N. For example, the number of rows R can be set to 8, and the number of columns N can be set to 48.

[0151] As the number of rows of the second transformation kernel decreases, the number of second transformation coefficients output as a result of the second transformation also decreases. For example, when a matrix multiplication is performed between a second transformation kernel of size 8x48 and an input matrix of size 48x1, second transformation coefficients of size 8x1 are generated.

[0152] The eight second transform coefficients can be rearranged within an 8x8 block. At this time, in an area where the second transform coefficients are not allocated within the application area of ​​the second transform (i.e., an area including the first transform coefficients to which the second transform is applied), the values ​​of the transform coefficients can be set to 0. For example, if a polygonal area including 48 samples is the application area of ​​the second transform, the values ​​of the transform coefficients can be set to 0 in the remaining area excluding the area to which the eight second transform coefficients are allocated among the polygonal area.

[0153] In areas where the second transformation is not applied, the first transformation coefficients can be maintained as is.

[0154] Alternatively, encoding may be performed by converting at least some of the first transform coefficients in the area to which the second transform is not applied to 0. FIG. 8 illustrates an example in which at least some of the area to which the second transform is not applied is converted to 0.

[0155] As in the example illustrated in (a) of Fig. 8, the values ​​of the first transform coefficients corresponding to the high-frequency region within the region where the second transform is not performed can be converted to 0. For example, the values ​​of the first transform coefficients whose sum of the x-axis and y-axis coordinates is greater than or equal to a threshold value can be converted to 0.

[0156] Alternatively, depending on a specific form, the first transform coefficients that are converted to 0 may be selected. For example, as in the example illustrated in (b) of FIG. 8, the first transform coefficients included in the lower n rows within the area where the second transform is not performed may be converted to 0. Alternatively, as in the example illustrated in (c) of FIG. 8, the first transform coefficients included in the right n columns within the area where the second transform is not performed may be converted to 0.

[0157] Alternatively, as in the example shown in (d) of Fig. 8, all first transform coefficients within the region where the second transform is not performed may be transformed to 0.

[0158] The shape of the region containing the first transform coefficients that are transformed to 0 can be determined based on at least one of the size, shape, intra prediction mode, or transform kernel of the current block. Alternatively, an index specifying one of the plurality of candidate shapes that matches the region can be encoded and signaled.

[0159] Whether the second transform is allowed may be determined based on at least one of the encoding mode of the current block or the first transform kernel. Here, the encoding mode refers to intra prediction or inter prediction. For example, if the current block is encoded with intra prediction, the second transform may be allowed, whereas if the current block is encoded with inter prediction, the second transform may not be allowed.

[0160] Information indicating whether a second transformation has been applied may be encoded and signaled. The information may be a 1-bit flag. Depending on whether the flag is true or false, it may be determined whether the second transformation has been applied to the current block. Alternatively, the information may be index information. An index value of 0 indicates that the second transformation has not been applied to the current block. On the other hand, an index value greater than 0 indicates that the second transformation has been applied to the current block. If the index value is greater than 0, the second transformation kernel can be specified by the index.

[0161] Information indicating whether a second transformation has been performed on the current block can be individually encoded for each color component. For example, information indicating whether a second transformation has been performed can be encoded for each of the luma component (Y), the first chrominance component (Cb), and the second chrominance component (Cr).

[0162] Alternatively, information indicating whether a second transformation has been performed on the chrominance components may be integrated and encoded. For example, whether or not a second transformation has been applied to each of the chrominance components (Cb, Cr) may be jointly determined. That is, the first chrominance component (Cb) and the second chrominance component (Cr) may share information indicating whether or not a second transformation has been performed.

[0163] Alternatively, based on the tree structure, it can be determined whether the information is encoded for each color component. For example, if the luma component and the chroma component have the same tree structure, the three color components (i.e., Y, Cb, Cr) can share information indicating whether the second transformation is performed. On the other hand, if the luma component and the chroma component have different tree structures, information indicating whether the second transformation is performed can be signaled for each of the luma component and the chroma component.

[0164] A plurality of second transformation kernel candidates can be grouped into at least a plurality of groups. One of the groups can be identified based on at least one of the size, shape, or intra prediction modes of the current block. Once the group is identified, the index information can be used to identify at least one of the plurality of second transformation kernel candidates included in the identified group.

[0165] Based on the position of the last non-zero coefficient within the current block, it may be determined whether to encode information indicating whether a second transformation is applied.

[0166] Figure 9 illustrates an example in which whether to encode information indicating whether to apply the second transformation is determined based on the position of the last non-zero coefficient.

[0167] For convenience of explanation, it is assumed that when the second transformation is performed, the values ​​of the transformation coefficients are set to 0 in the remaining areas other than the areas where the second transformation coefficients are rearranged.

[0168] As a result of performing the second transformation, a number of second transformation coefficients equal to the number R of rows of the second transformation kernel are generated. As in the example described above, the values ​​of the remaining transformation coefficients excluding the second transformation coefficients are all set to 0, so that there are no non-zero coefficients in the remaining area excluding the area where the R second transformation coefficients are rearranged. According to the above principle, the area where the R second transformation coefficients are rearranged can be set as a restricted area.

[0169] In (a) to (c) of Fig. 9, it is exemplified that the upper left 4x4 block within the 8x8 block is set as a restricted area.

[0170] The presence of a non-zero coefficient outside the restricted area indicates that the second transformation has not been applied to the current block. Accordingly, if the last non-zero coefficient is outside the restricted area, the encoding of information indicating whether the second transformation has been applied to the current block can be omitted. For example, as in the example illustrated in (a) of FIG. 9, if the last non-zero coefficient is outside the restricted area, the encoding of information indicating whether the second transformation has been applied can be omitted.

[0171] The decryptor may also decide not to decrypt the information if the last non-zero coefficient is outside the restricted area and not to apply the second inverse transform to the current block.

[0172] As in the example illustrated in (b) of Fig. 9, when the second transform is applied to the current block, non-zero transform coefficients may exist only within the restricted region. Alternatively, as in the example illustrated in (c) of Fig. 9, even when the second transform is not applied to the current block, there may be cases where non-zero transform coefficients exist only within the restricted region. Accordingly, if the last non-zero coefficient exists within the restricted region, information indicating whether the second transform was applied can be encoded.

[0173] Based on the above information, the decoder can decide whether to apply a second inverse transform to the current block.

[0174] Alternatively, if the last non-zero coefficient is within the restricted area, encoding of the above information may be omitted and the second transformation may be necessarily applied.

[0175] The size of the restriction region can be determined based on the size of the second transformation kernel. For example, if the second transformation kernel is a matrix of size RxN, a rectangular region whose width and height are each Log2R can be set as the restriction region.

[0176] Alternatively, the area to which the second transformation is applied can be set as a restricted area.

[0177] Alternatively, information indicating at least one of the size or shape of the restricted area may be encoded and signaled. The information may be signaled via a higher level, such as a sequence, picture header, or slice header.

[0178] Alternatively, at least one of the size or shape of the restricted area may be predefined for the encoder and decoder. For example, it may be predefined between the encoder and decoder to set the upper left 4x4 block within the current block as the restricted area.

[0179] Alternatively, at least one of the size or shape of the restriction region may be adaptively determined based on at least one of the size, shape, first transform kernel, or intra prediction mode of the current block.

[0180] Alternatively, multiple restriction area candidates can be defined and then an index specifying one of the multiple restriction area candidates can be encoded and signaled.

[0181] Figure 10 illustrates restricted area candidates for a 4x4 sized block.

[0182] If the current block is 4x4 in size, index information specifying one of the multiple restricted area candidates shown in FIG. 12 can be encoded.

[0183] At least one of the restriction region candidates illustrated in FIG. 10 may be applied not only to blocks of size 4x4, but also to blocks of size larger than 4x4. For example, at least one of the restriction region candidates illustrated in FIG. 10 may be applied to a block of which at least one of the width or the height is 4 and the other is larger than 4.

[0184] Alternatively, the size or number of restricted area candidates may be set differently depending on the size of the current block.

[0185] Instead of encoding an index that specifies one of the restriction area candidates, one of the restriction area candidates may be specified based on the size or shape of the current block.

[0186] In the above example, it was explained that the size of the second transformation kernel can be adaptively selected. As another example, a second transformation kernel of a predefined size can be applied to all blocks. For example, a second transformation kernel of size 16x48 can be used for all blocks. In this case, the second transformation can be applied to 48 first transformation coefficients.

[0187] Figure 11 is a drawing showing an example in which a second transformation kernel of a predefined size is applied.

[0188] When a second transform kernel of size 16x48 is used, the second transform may be applied to an area excluding a 4x4 sized sub-block at the lower right within an 8x8 sized block, as in the example illustrated in (a) of Fig. 11. For example, the second transform may be applied to the first transform coefficients included in the polygonal area illustrated in (a) of Fig. 11.

[0189] At this time, if at least one of the width or height of the current block is less than 8, the second transformation may not be applied to the current block.

[0190] Alternatively, if at least one of the width or height of the current block is less than 8, the area to which the second transformation is applied may be transformed into a rectangular shape such as 4x12 or 12x4, and then the second transformation may be performed.

[0191] Alternatively, if at least one of the width or height of the current block is less than 8, the upper left position of the current block and the upper left area of ​​the area to which the second transformation is applied may be matched, and the second transformation may be applied only to the overlapping area between the current block and the area to which the second transformation is applied.

[0192] Figure 11 (b) shows an example in which the second transformation is performed only in the overlapping area.

[0193] The first transformation coefficients included in the overlapping area can be arranged in one dimension and then set as the input matrix for the second transformation.

[0194] When the first and second transformations are applied to the current block, the decoder can derive residual samples by performing an inverse transformation for the second transformation (second inverse transformation) and then performing an inverse transformation for the first transformation (first inverse transformation) on the result of the second inverse transformation.

[0195] The second inverse transformation can be performed based on the transpose matrix of the second transformation kernel. For example, if the second transformation kernel has a size of 8x48, the second inverse transformation can be performed by a transformation kernel of a size of 48x8.

[0196] The second transform coefficients generated by the second transform can be set as the input matrix of the second inverse transform. For example, when the second transform is performed by a second transform kernel of size 8x48, an 8x1 input matrix composed of 8 coefficients can be used when performing the second inverse transform. Thereafter, the transform coefficients for which the second inverse transform is performed can be output by matrix multiplication between the transpose matrix of the second transform kernel and the input matrix. For example, 48 transform coefficients can be output by matrix multiplication between a transform kernel of size 48x8 and an input matrix of size 8x1.

[0197] After rearranging the transformation coefficients within the current block, the first inverse transformation can be applied to the rearranged block.

[0198] Meanwhile, transformation may be performed only on some areas within the current block. That is, the transformed coefficients may be output / exist only for some areas within the current block, and the values ​​of the transformed coefficients may be set to 0 for the remaining areas. Performing transformation only on some areas within the current block can be defined as a partial transformation.

[0199] Figure 12 is a drawing for explaining an example in which partial transformation is applied to the current block.

[0200] In Figure 12, W and H represent the width and height of the current block (specifically, the residual block of the current block), respectively.

[0201] When a partial transformation is applied, the current block can be divided into two regions. For example, in the example illustrated in FIG. 12, the current block is vertically divided to create a first region R0 of size (W0xH) and a second region R1 of size ((W-W0)xH). In this case, each region may be referred to as a sub-block, a transformation block, or a sub-partition.

[0202] Transformation / inverse transformation may be performed only on one of multiple regions belonging to the current block. Furthermore, at least one of quantization / inverse quantization or entropy addition / decoding on residual coefficients may be performed only on the region where transformation has been performed.

[0203] On the other hand, for the residual region, transformation / inverse transformation can be omitted. Furthermore, quantization / inverse quantization and entropy addition / decoding for residual coefficients can be omitted for the residual region. Residual coefficients in regions where transformation / inverse transformation is not performed can be set to 0.

[0204] That is, for regions where transformation / inverse transformation is performed within the current block, information about residual coefficients can be encoded and signaled to the decoder. On the other hand, for regions where transformation / inverse transformation is not performed within the current block, the values ​​of residual coefficients may be set to 0, and information about residual coefficients may not be encoded / decoded.

[0205] Meanwhile, information related to partial transformation may be encoded and signaled to the decoder. The information related to partial transformation may include at least one of: information indicating whether partial transformation is applied to the current block, information indicating the division direction of the current block, information indicating the sizes of the divided regions, or information indicating a region among multiple regions within the current block where transformation / inverse transformation is performed.

[0206] For example, information indicating whether a partial transformation is applied to the current block may be a 1-bit flag.

[0207] If the above flag indicates that a partial transformation is applied to the current block, at least one of information indicating the splitting direction, information indicating the size of the split regions, or information indicating the region where the transformation / inverse transformation is performed may be additionally encoded / decoded.

[0208] On the other hand, if the flag indicates that no partial transformation is applied to the current block, encoding / decoding of information indicating the splitting direction, information indicating the sizes of the split regions, and information indicating the region where transformation / inverse transformation is performed may be omitted.

[0209] The information indicating the splitting direction may be a flag indicating whether the splitting direction of the current block is vertical or horizontal. For example, a flag value of 0 (or 1) indicates that the current block is split vertically, and a flag value of 1 (or 0) indicates that the current block is split horizontally.

[0210] The information indicating the sizes of the divided regions may be a flag indicating whether the current block is divided in a symmetrical shape. For example, a value of 0 (or 1) of the flag indicates that the current block is divided into two regions, each half the size of the current block (i.e., a symmetrical shape). On the other hand, a value of 1 (or 0) of the flag indicates that the current block is divided into a region, each quarter the size of the current block, and a region, each three quarters the size of the current block (i.e., an asymmetrical shape). The number of residual samples / transformation coefficients on which transformation / inverse transformation is performed may be determined by the information indicating the sizes of the divided regions.

[0211] As another example, the information indicating the size of the divided regions may be an index indicating one of a plurality of size candidates. Here, the plurality of size candidates may include at least one of 1 / 2, 1 / 4, 1 / 8, 1 / 16, or 1 / 32. One of the two regions within the current block may have a size indicated by one of the plurality of size candidates compared to the current block. For example, if the 1 / N size candidate is selected, one of the two regions may have a size that is 1 / N of the current block. The other of the two regions may be set to the remaining size within the current block.

[0212] As another example, if it is determined that the current block is to be split into an asymmetrical shape, the size of each region may be determined based on at least one of the size and / or shape of the current block. For example, if the size of the current block is smaller than a threshold, the current block may be split into a region that is 1 / 4 the size of the current block and a region that is 3 / 4 the size of the current block, whereas if the size of the current block is equal to or greater than the threshold, the current block may be split into a region that is 1 / 8 the size of the current block and a region that is 7 / 8 the size of the current block.

[0213] The information indicating the area where transformation / inverse transformation is performed may be a flag indicating whether transformation / inverse transformation is performed on the first area among the two areas. For example, a value of the flag of 0 (or 1) indicates that transformation / inverse transformation is applied to the first area within the current block, and that transformation / inverse transformation is not applied to the second area. On the other hand, a value of the flag of 1 (or 0) indicates that transformation / inverse transformation is applied to the second area within the current block, and that transformation / inverse transformation is not applied to the first area.

[0214] Here, the first area represents the area placed on the left or top within the current block, and the second area represents the area placed on the right or bottom within the current block.

[0215] Figures 13 and 14 illustrate the division types of the current block.

[0216] Figure 13 illustrates the split types of the current block when the size candidate is 1 / 2, and Figure 14 illustrates the split types of the current block when the size candidate is 1 / 4.

[0217] Meanwhile, if the current block is split asymmetrically, additional information indicating the location of the smaller and larger regions within the current block may be required. Accordingly, if the current block is split asymmetrically, information indicating whether the left region or upper region is larger than the right region or lower region may be additionally encoded / decoded.

[0218] Alternatively, if the information for determining the size between regions is omitted and the current block is split into an asymmetrical shape, the transformation / inverse transformation may be applied to the smaller region (e.g., a region of 1 / 4 size) of the two regions, and the transformation / inverse transformation may not be applied to the larger region (e.g., a region of 3 / 4 size). In this case, both the size and location of each region may be determined based on the information indicating the region on which the transformation / inverse transformation is performed.

[0219] Conversely, if the current block is split into an asymmetrical shape, the transformation / inverse transformation may be applied to the larger size region (e.g., the 3 / 4 size region) of the two regions, and the transformation / inverse transformation may not be applied to the smaller size region (e.g., the 1 / 4 size region).

[0220] The decoder, after receiving information related to partial transformation from the encoder, can derive residual samples by performing at least one of entropy decoding, inverse quantization, or inverse transformation only on some regions within the current block. On the other hand, for the remaining regions within the current block, the entropy decoding, inverse quantization, and inverse transformation can be omitted, and the values ​​of the residual samples can be set to 0.

[0221] That is, residual samples belonging to some areas within the restored residual block are obtained by performing at least one of entropy decoding, inverse quantization, or inverse transformation, whereas residual samples belonging to the residual area can all be set to 0 without entropy decoding, inverse quantization, or inverse transformation.

[0222] Once the prediction block for the current block is obtained, the reconstructed residual block and the prediction block can be combined to derive the reconstructed block for the current block.

[0223] Meanwhile, in the above-described embodiments, the current block is exemplified as being divided into two rectangular regions. Beyond the illustrated example, the current block may also be divided diagonally.

[0224] Figure 15 shows an example in which the current block is divided diagonally.

[0225] As in the example illustrated in Fig. 15, the current block can be divided into two regions along the upper right diagonal direction or the upper left diagonal direction.

[0226] When the current block is split diagonally, information indicating the split direction of the current block can be encoded / decoded. For example, the range of values ​​that the information indicating the split direction can take can be expanded, such that the information indicating the split direction can be set to indicate one of the following directions: vertical, horizontal, upper-right diagonal, or upper-left diagonal.

[0227] Alternatively, information indicating whether the current block is divided in a diagonal direction, for example, a 1-bit flag, may be encoded. For example, if the flag is 0 (or 1), it indicates that the current block is divided in a vertical direction or a horizontal direction. In this case, one of the vertical direction and the horizontal direction can be selected based on information indicating the division direction of the current block. On the other hand, if the flag is 1 (or 0), it indicates that the current block is divided in a top-right diagonal direction or a top-left diagonal direction. In this case, one of the top-right diagonal direction and the top-left diagonal direction can be selected based on information indicating the division direction of the current block.

[0228] Meanwhile, the number of components constituting the transformation kernel for transformation / inverse transformation is expressed as a power of 2, and accordingly, the number of residual samples / transform coefficients on which transformation / inverse transformation is performed can also be expressed as a power of 2. When the current block is divided into triangular regions, a predetermined scan order can be applied to determine the residual samples / transform coefficients on which transformation / inverse transformation is performed.

[0229] Figure 16 shows an example of selecting residual samples for transformation within the current block.

[0230] In the example shown in Fig. 16, r0 to r63 represent residual samples within the current block.

[0231] Assuming that the current block is divided along the upper right diagonal direction and that the transformation is applied to the upper left area of ​​the current block, a zigzag scan can be applied to the residual samples at the upper left position within the area where the transformation is performed to select the residual samples to which the transformation / inverse transformation is applied. For example, as in the example shown, if the current block is 8x8, half of the residual samples within the current block can be determined as the target of the transformation.

[0232] Meanwhile, the scan start position may be a sample at the upper left position of the area to which the transformation is applied.

[0233] Alternatively, the scan start position may be a residual sample located at a right angle vertex within the area to which the transformation is applied. For example, in the example illustrated in (a) of Fig. 15, if it is determined that the transformation is to be performed on the first area (i.e., R0), the residual samples to be transformed can be determined through a scan starting from the residual sample at the upper left position within the area (r0 in Fig. 16) as the starting point. On the other hand, if it is determined that the transformation is to be performed on the second area (i.e., R1), the residual samples to be transformed can be determined through a scan starting from the residual sample at the lower right position within the area (r63 in Fig. 16) as the starting point.

[0234] For example, in the example illustrated in (b) of Fig. 15, if it is determined that the transformation is to be performed on the first region (i.e., R0), the residual samples to be transformed can be determined through a scan starting from the residual sample at the lower left position within the region (r56 in Fig. 16) as a starting point. On the other hand, if it is determined that the transformation is to be performed on the second region (i.e., R1), the residual samples to be transformed can be determined through a scan starting from the residual sample at the upper right position within the region (r7 in Fig. 16) as a starting point.

[0235] Meanwhile, even when the current block is diagonally divided, only 1 / 4, 1 / 8, 1 / 16, or 1 / 32 residual samples within the current block can be selected as transformation targets. Specifically, by dividing the current block into two regions based on a diagonal line that does not pass through both vertices of the current block, only 1 / 4, 1 / 8, 1 / 16, or 1 / 32 residual samples within the current block can be determined as transformation targets.

[0236] Figure 17 shows an example of a case where the division shape of the current block that is divided diagonally is asymmetric.

[0237] As in the example shown, two regions of asymmetrical shape can be created by dividing the current block based on a diagonal line that does not pass through the two diagonals of the current block.

[0238] To support the above division, the position information of the division line may be additionally encoded / decoded. For example, if the current block is divided in the upper right diagonal direction, one of the pieces of information indicating the position of the division line dividing the current block may be encoded. For example, the information may be an index indicating one of a plurality of candidate positions, and the plurality of candidate positions may include a symmetric division position ((a) of FIG. 16), a position shifted in the upper left direction from the symmetric division position ((a) of FIG. 17), and a position shifted in the lower right direction from the symmetric division position ((b) of FIG. 17).

[0239] Alternatively, after encoding / decoding information indicating whether the diagonal division is performed in a symmetrical manner, if the information indicates that the diagonal division is performed in an asymmetrical manner, information indicating the location of the division line may be additionally encoded / decoded. For example, the number of residual samples on which transformation is performed within the current block can be determined based on the information indicating the location of the division line.

[0240] Alternatively, the location of the dividing line that divides the current block can be adaptively determined based on at least one of the size or shape of the current block. For example, if the size of the current block is smaller than a threshold, the current block can be divided so that 1 / 4 of the samples within the current block are selected as conversion targets. Conversely, if the size of the current block is equal to or greater than the threshold, 1 / 8 of the samples within the current block can be selected as conversion targets.

[0241] Alternatively, for performing intra prediction or inter prediction, if the current block is divided into multiple partitions, the location of the division line that divides the current block can be determined based on the division type applied to the current block.

[0242] For example, during intra-prediction or inter-prediction, if the current block is split horizontally, the current block can be split horizontally to apply a partial transformation. Conversely, during intra-prediction or inter-prediction, if the current block is split vertically, the current block can be split vertically to apply a partial transformation. In other words, the splitting direction of the current block for applying a partial transformation can be the same as the splitting direction of the current block for prediction.

[0243] Alternatively, if a prediction based on geometric partitioning (e.g., inter prediction based on geometric transformation, intra prediction based on geometric transformation, or combined prediction based on geometric transformation (i.e., prediction combining intra prediction and inter prediction)) is applied to the current block, the current block can be divided into two regions in the same manner as geometric partitioning.

[0244] In the above case, the encoding / decoding of information indicating the splitting direction of the current block may be omitted.

[0245] Figure 18 shows an example of selecting residual samples for transformation within the current block.

[0246] In the example shown in Fig. 18, r0 to r63 represent residual samples within the current block.

[0247] As explained through Figure 16, the target of transformation application can be determined by applying a predefined scan method from residual samples located at orthogonal vertices within the area where transformation is performed.

[0248] For example, if the size of the current block is 8x8 and 1 / 4 residual samples within the current block are to be selected as transformation targets, 16 residual samples can be selected in a zigzag scan order from the residual sample r0 at the upper left position within the current block, as in the example shown in FIG. 18.

[0249] As in the examples illustrated in Figures 16 and 18, the residual samples to which the transformation is applied can be determined according to a predetermined scan order. Here, the predetermined scan order may be, but is not limited to, a zigzag scan. For example, at least one of a diagonal scan, an inverse diagonal scan, a horizontal scan, or a vertical scan may be used.

[0250] The scan order for determining residual samples may be predefined in the encoder and decoder.

[0251] Alternatively, information indicating one of multiple scan types may be encoded and signaled.

[0252] Alternatively, the scan order for selecting residual samples to which the transformation is applied may be adaptively determined based on the location, size, or shape of the region to which the transformation is applied.

[0253] Meanwhile, in the decoder, transform coefficients for performing the inverse transform can be selected in the same manner. For example, if the transform is performed according to the example illustrated in Fig. 16, the transform coefficients at positions from r0 to r28 selected according to the zigzag scan can be selected as the transform coefficients for performing the inverse transform.

[0254] All residual coefficients / transform coefficients within the area where the transformation is not applied can be encoded / decoded with a value of 0.

[0255] Once residual samples for which transformation is to be performed are selected, the selected residual samples can be arranged in one dimension and then the transformation can be performed. For example, in the example illustrated in Fig. 16, residual samples from r0 to r28 selected according to a zigzag scan can be arranged in one dimension and then the transformation can be performed.

[0256] As another example, the selected residual samples can be arranged in two dimensions and then transformed.

[0257] Figure 19 shows an example in which selected residual samples are arranged in a two-dimensional form.

[0258] For convenience of explanation, the number of residual samples selected for transformation is assumed to be 32.

[0259] When 32 residual samples are selected, the 32 residual samples can be arranged in an 8x4 or 4x8 format to perform the transformation.

[0260] Alternatively, it is possible to arrange the 32 residual samples into a 16x2 or 2x16 size and perform the transformation.

[0261] Information regarding the size of the array of residual samples can be encoded and signaled. The decoder can rearrange the transform coefficients according to the instructions of the information.

[0262] Alternatively, in the encoder, the array size of the residual samples may be predefined. In the decoder, the transform coefficients may be rearranged to have the same size as that defined in the encoder.

[0263] Alternatively, the array size of the residual samples may be adaptively determined based on at least one of the size, location, or shape of the region where the transformation is performed.

[0264] When arranging residual samples in two dimensions, the width and height can be set to be greater than or equal to a minimum value. For example, the width and height can be set to values ​​greater than or equal to 4.

[0265] The minimum value may be a value predefined in the encoder and decoder. Alternatively, information indicating the minimum value may be encoded and signaled. This information may be encoded and signaled via an upper header.

[0266] Meanwhile, the array form or array size of the residual samples may differ between the first and second transformations. For example, for the first transformation, the residual samples may be arranged in two dimensions, while for the second transformation, the first transformation coefficients may be arranged in one dimension.

[0267] Alternatively, the values ​​of all residual samples located in the area where the transformation is not performed can be converted to 0, and then the transformation can be performed using a transformation kernel that is tailored to the size of the current block. For example, if the size of the current block is 8x8, an 8x8-sized transformation kernel can be used, but the values ​​of the residual samples in the area where the transformation is not applied within the current block can be set to 0.

[0268] Figures 20 and 21 are flowcharts showing the operation of an encoder and a decoder when partial transformation is applied.

[0269] In the examples shown in FIGS. 20 and 21, it is assumed that among the two regions generated by dividing the current block, the first region is a region to which transformation / inverse transformation is applied, and the second region is a region to which transformation / inverse transformation is not applied.

[0270] Referring to FIG. 20, the encoder can divide the current block into a first region and a second region, and perform transformation only on residual samples belonging to the first region to generate transformation coefficients (S2010, S2020).

[0271] Transform coefficients can be quantized to generate residual coefficients, and information related to the generated residual coefficients and partial transformation can be encoded (S2030, S2040).

[0272] Meanwhile, the values ​​of the residual coefficients in the second region can all be set to 0, and thus, entropy encoding for the residual coefficients in the second region can be omitted.

[0273] Referring to FIG. 21, the decoder decodes information related to partial transformation, and if it is determined that partial transformation is applied to the current block, the current block can be divided into a first region and a second region (S2110, S2120).

[0274] Then, the residual coefficients belonging to the first region can be decoded, and the decoded residual coefficients can be dequantized (S2130).

[0275] Afterwards, by performing inverse transformation on the inverse quantized residual coefficients (i.e., transform coefficients), residual samples can be obtained (S2140).

[0276] Meanwhile, the values ​​of residual coefficients or residual samples in the second region can all be set to 0, and for the second region, entropy decoding of residual coefficients can be omitted.

[0277]

[0278] Applying the embodiments described above, focusing on the decoding or encoding process, to the encoding or decoding process is within the scope of the present disclosure. Changing the embodiments described above, in a given order, to a different order is also within the scope of the present disclosure.

[0279] Although the above-described disclosure is described based on a series of steps or a flowchart, this does not limit the chronological order of the invention, and may be performed simultaneously or in a different order as needed. In addition, each component (e.g., unit, module, etc.) constituting the block diagram in the above-described disclosure may be implemented as a hardware device or software, or multiple components may be combined to be implemented as a single hardware device or software. For example, the hardware device may include at least one of a processor for performing calculations, a memory for storing data, a transmitter for transmitting data, and a receiver for receiving data.

[0280] The above-described disclosure may be implemented in the form of program commands that can be executed by various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, etc., either singly or in combination.

[0281] In addition, according to the present disclosure, a computer-readable recording medium can be provided that stores a bitstream generated by the above-described encoding method. The bitstream can be transmitted by an encoding device, and a decoding device can receive the bitstream and decode an image.

[0282] Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program instructions, such as ROMs, RAMs, and flash memories. The hardware devices may be configured to operate as one or more software modules to perform processing according to the present disclosure, and vice versa.

[0283] The present disclosure may be applied to a computing or electronic device capable of encoding / decoding a video signal.

Claims

1. A step for determining whether partial transformation is applied to the current block; A step of decoding residual coefficients belonging to a first region within the current block when a partial transformation is applied to the current block; a step of inversely quantizing the residual coefficients within the first region; and A step of obtaining residual samples by inversely transforming the inversely quantized residual coefficients in the first region, An image decoding method, characterized in that when the partial transformation is applied to the current block, the division direction of the current block is one of a horizontal direction, a vertical direction, an upper-right diagonal direction, and an upper-left diagonal direction.

2. In paragraph 1, An image decoding method, characterized in that the values of residual coefficients belonging to the second area within the current block are set to 0.

3. In paragraph 1, When the current block is divided in the horizontal direction or the vertical direction, the first region is a square shape, An image decoding method, characterized in that when the current block is divided in the upper right diagonal direction or the upper left diagonal direction, the first region has a triangular shape.

4. In paragraph 3, An image decoding method, characterized in that the inverse quantized residual coefficients within the first region having a triangular shape are rearranged according to a predetermined scan order.

5. In paragraph 4, An image decoding method, characterized in that the starting point of the scan is a dequantized residual coefficient located at a right angle vertex within the first region.

6. In paragraph 4, An image decoding method, characterized in that the above scan order is selected from among a plurality of scan type candidates.

7. In paragraph 6, An image decoding method, characterized in that one of the plurality of scan type candidates is adaptively selected based on at least one of the size, shape or position of the first region.

8. In paragraph 4, An image decoding method, characterized in that the above-mentioned inverse quantized residual coefficients are rearranged into a two-dimensional rectangular shape.

9. In paragraph 8, An image decoding method, characterized in that each of the width and height of the above two-dimensional rectangular shape has a value greater than or equal to the minimum value.

10. In paragraph 8, An image decoding method, characterized in that the size of the two-dimensional square shape is determined based on at least one of the size, shape, or position of the first region.

11. A step of applying transformation to residual samples of the first region within the current block; A step of quantizing the transform coefficients obtained by the above transformation; and Including a step of encoding the residual coefficients obtained by the above quantization, Partial transformation information for the current block is encoded, An image encoding method, characterized in that when the partial transformation is applied to the current block, the division direction of the current block is one of a horizontal direction, a vertical direction, an upper-right diagonal direction, and an upper-left diagonal direction.

12. In paragraph 11, An image encoding method, characterized in that the values of residual coefficients belonging to the second area within the current block are set to 0.

13. In paragraph 11, When the current block is divided in the horizontal direction or the vertical direction, the first region is a square shape, An image encoding method, characterized in that when the current block is divided in the upper right diagonal direction or the upper left diagonal direction, the first region has a triangular shape.

14. In paragraph 13, An image decoding method, characterized in that the residual samples within the first region having a triangular shape are rearranged according to a predetermined scan order.

15. A step of applying transformation to residual samples of the first region within the current block; A step of quantizing the transform coefficients obtained by the above transformation; and Including a step of encoding the residual coefficients obtained by the above quantization, Partial transformation information for the current block is encoded, A computer-readable recording medium storing a bitstream generated by an image encoding method, characterized in that when the partial transformation is applied to the current block, the division direction of the current block is one of a horizontal direction, a vertical direction, an upper-right diagonal direction, and an upper-left diagonal direction.

Images