Method and apparatus for processing video signals

Abstract

A video decoding method according to the present disclosure may include the following steps: determining whether to skip inverse transformation in a current block; decoding residual coefficients of the current block; and selectively applying inverse transformation to the residual coefficients based on the determination. When decoding the residual coefficients, either a first syntax indicating whether the residual coefficients are greater than 0 or a second syntax indicating the absolute value of the residual coefficients may be selectively decoded.

Description

Technical Field

[0001] The present disclosure relates to methods and apparatuses for processing video signals. Background Art

[0002] Recently, there has been an increasing demand for high-resolution and high-quality images such as HD (High Definition) images and UHD (Ultra High Definition) images in various application fields. Since the amount of data relatively increases as image data becomes high-resolution and high-quality compared to existing image data, transmission costs and storage costs increase when transmitting image data through a medium such as existing wired and wireless broadband circuits or storing image data using existing storage media. These problems generated due to image data becoming high-resolution and high-quality can be solved by using efficient image compression techniques.

[0003] There are various techniques, such as an inter prediction technique that predicts pixel values included in a current picture based on a previous or subsequent picture of the current picture by using an image compression technique, an intra prediction technique that predicts pixel values included in the current picture by using pixel information in the current picture, an entropy coding technique that assigns short symbols to values with high occurrence frequencies and long symbols to values with low occurrence frequencies, etc., and image data can be effectively compressed, transmitted, or stored by using these image compression techniques.

[0004] On the other hand, as the demand for high-resolution images increases, the demand for stereoscopic image content as a new image service also increases. Video compression techniques for effectively providing high-resolution and ultra-high-resolution stereoscopic image content have been discussed. Summary of the Invention

[0005] Technical Objectives

[0006] An object of the present disclosure is to provide a method and apparatus for effectively encoding / decoding residual coefficients when encoding / decoding a video signal.

[0007] An object of the present disclosure is to provide a method and apparatus for additionally applying a second transform to the result of a first transform when encoding / decoding a video signal.

[0008] The technical effects of the present disclosure may not be limited to the above-mentioned technical effects, and other unmentioned technical effects can be clearly understood by those of ordinary skill in the technical field to which the present disclosure belongs from the following description.

[0009] Technical Solutions

[0010] A method for decoding a video signal according to the present disclosure may include: determining whether to skip inverse transformation for a current block; decoding residual coefficients of the current block; and selectively applying inverse transformation to the residual coefficients based on the determination. In this case, when decoding the residual coefficients, one of a first syntax indicating whether the residual coefficients are greater than 0 and a second syntax indicating the absolute values of the residual coefficients may be selectively decoded.

[0011] A method for encoding a video signal according to the present disclosure may include: determining whether to skip transformation for a current block; quantizing a result of applying transformation or a result of skipping transformation; and encoding residual coefficients output as a result of the quantization. In this case, when encoding the residual coefficients, one of a first syntax indicating whether the residual coefficients are greater than 0 and a second syntax indicating the absolute values of the residual coefficients may be selectively encoded.

[0012] In the method for decoding a video signal according to the present disclosure, it may be determined whether to decode the first syntax or the second syntax by comparing a threshold with the number of binary bits decoded by using context information.

[0013] In the method for decoding a video signal according to the present disclosure, when at least one of the following is decoded, the number of binary bits decoded by using context information may be increased: the first syntax, at least one gt_N_flag indicating whether the absolute value has a value greater than (2N - 1), or a parity flag indicating whether the absolute value is even.

[0014] In the method for decoding a video signal according to the present disclosure, when the first syntax is decoded and the first syntax indicates that the residual coefficients have non - zero values, a gt_1_flag indicating whether the absolute value of the residual coefficients has a value greater than 1 may be additionally decoded.

[0015] In the method for decoding a video signal according to the present disclosure, when the gt_1_flag indicates that the absolute value has a value greater than 1, a parity flag indicating whether the absolute value is even and a gt_2_flag indicating whether the absolute value is greater than 3 may be additionally decoded.

[0016] In the method for decoding a video signal according to the present disclosure, the threshold may be determined based on the size of the current block.

[0017] The features briefly summarized above for the present disclosure are merely illustrative aspects of the detailed description of the present disclosure to follow and do not limit the scope of the present disclosure.

[0018] Technical effects

[0019] According to the present disclosure, the encoding / decoding efficiency can be improved by differently setting encoding methods of residual coefficients according to the number of binary bits encoded by using context information.

[0020] According to the present disclosure, the encoding / decoding efficiency can be improved by additionally applying a second transform to the result of a first transform.

[0021] The effects obtainable from the present disclosure may not be limited to the above-mentioned effects, and those skilled in the art to which the present disclosure pertains can clearly understand other unmentioned effects from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a block diagram showing an image encoding apparatus according to an embodiment of the present disclosure.

[0023] Figure 2 is a block diagram showing an image decoding apparatus according to an embodiment of the present disclosure.

[0024] Figure 3 is a flowchart showing an intra prediction method according to an embodiment of the present disclosure.

[0025] Figure 4 shows types of intra prediction modes.

[0026] Figure 5 is a diagram for describing an example of obtaining a prediction sample in a planar mode.

[0027] Figure 6 represents examples of generating prediction samples in horizontal and vertical modes.

[0028] Figure 7 and Figure 8 is a diagram showing an example of applying a second transform.

[0029] Figure 9 and Figure 10 show a second transform based on a second transform kernel having an asymmetric shape.

[0030] Figure 11 represents an example of determining whether to encode information indicating whether to apply a second transform based on the position of a last non-zero coefficient.

[0031] Figure 12 shows candidates for a limited region for a block of size 4×4.

[0032] Figure 13 is a diagram showing an example of applying a second transform kernel of a predetermined size.

[0033] Figure 14Shows a scanning method.

[0034] Figure 15 Is a flowchart showing the process of encoding residual coefficients in an encoder.

[0035] Figure 16 Is a flowchart showing the process of encoding the size information of residual coefficients.

[0036] Figure 17 Is a flowchart showing the process of decoding residual coefficients in a decoder.

[0037] Figure 18 Is a diagram showing the process of decoding the size information of residual coefficients.

[0038] Figure 19 and Figure 20 Is a diagram showing an example of counting the number of binary bits using context information.

[0039] Figures 21 to 23 Shows examples where the priorities between grammars encoded using context information are different.

[0040] Figure 24 and Figure 25 Shows the surrounding reconstruction region referred to for determining context information.

[0041] Figure 26 Shows the number of context information that can be referred to when encoding the flag sig_flag.

[0042] Figure 27 Shows the number of context information that can be referred to when encoding gt_N_flag or par_flag. Detailed implementation

[0043] Since the present disclosure can be made in various changes and has several embodiments, specific embodiments will be illustrated in the drawings and described in detail. However, it is not intended to limit the present disclosure to specific embodiments, and it should be understood that the present disclosure includes all changes, equivalents, or alternatives included in the spirit and technical scope of the present disclosure. When describing each drawing, similar reference numerals are used for similar components.

[0044] Terms such as first, second, etc. can be used to describe various components, but the components should not be limited by the terms. These terms are only used to distinguish one component from other components. For example, without exceeding the scope of the rights of the present disclosure, the first component can be referred to as the second component, and similarly, the second component can also be referred to as the first component. The term "and / or" includes combinations of multiple relative input items or any item of multiple relative input items.

[0045] When a component is referred to as being "linked" or "connected" to another component, it should be understood that the component can be directly linked or connected to the other component, but other components may be present in between. On the other hand, when a component is referred to as being "directly linked" or "directly connected" to another component, it should be understood that no other components are present in between.

[0046] Since the terms used in this application are only used to describe specific embodiments, they are not intended to limit the present disclosure. Expressions in the singular include expressions in the plural unless they clearly have a different meaning in the context. In this application, it should be understood that terms such as "including" or "having" refer to the presence of the features, numbers, stages, movements, components, parts, or combinations thereof entered in the specification, but do not exclude the presence or the possibility of addition of one or more other features, numbers, stages, movements, components, parts, or combinations thereof.

[0047] Hereinafter, with reference to the accompanying drawings, desired embodiments of the present disclosure will be described in more detail. Hereinafter, the same reference numerals in the drawings are used for the same components, and repeated descriptions of the same components are omitted.

[0048] Figure 1 is a block diagram showing an image encoding apparatus according to an embodiment of the present disclosure.

[0049] Referring to Figure 1 , the image encoding apparatus 100 may include: a picture segmentation unit 110, a prediction unit 120 and a prediction unit 125, a transform unit 130, a quantization unit 135, a rearrangement unit 160, an entropy encoding unit 165, a dequantization unit 140, an inverse transform unit 145, a filter unit 150, and a memory 155.

[0050] Since Figure 1 each structural unit in is shown independently to show different characteristic functions in the image encoding apparatus, this does not mean that each structural unit is constituted by separate hardware or a software unit. That is, since each structural unit is listed as each structural unit for the convenience of description, at least two of the structural units in each structural unit may be combined to form one structural unit, or one structural unit may be divided into multiple structural units to perform functions, and even the integrated embodiments and separate embodiments of each structural unit are included in the scope of the rights of the present disclosure as long as they do not deviate from the essence of the present disclosure.

[0051] In addition, some components may be optional components only for improving performance and not necessary components for performing the basic functions in the present disclosure. The present disclosure can be implemented by excluding the components only for improving performance and including only the structural units necessary for implementing the essence of the present disclosure, and the structure that includes only the necessary components and excludes the optional components only for improving performance is also included in the scope of the rights of the present disclosure.

[0052] The picture segmentation unit 110 may segment an input picture into at least one processing unit. In this regard, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). In the picture segmentation unit 110, a picture may be segmented into a combination of multiple coding units, prediction units, and transform units, and the picture may be encoded by selecting a combination of a coding unit, a prediction unit, and a transform unit according to a predetermined criterion (e.g., a cost function).

[0053] For example, a picture may be segmented into multiple coding units. To segment the coding units in a picture, a recursive tree structure such as a quadtree structure may be used, and the coding unit that is segmented into other coding units by using one picture or the largest coding unit as a path may be segmented with as many child nodes as the number of the segmented coding units. The coding unit that is no longer segmented according to a specific limitation becomes a leaf node. In other words, when it is assumed that only square segmentation is possible for a coding unit, a coding unit may be segmented into up to four other coding units.

[0054] Hereinafter, in the embodiments of the present disclosure, the coding unit may be used as a unit for encoding or may be used as a unit for decoding.

[0055] The prediction unit may be segmented in at least one square or rectangular shape, etc. with the same size in a coding unit, or may be segmented such that any one of the prediction units segmented in a coding unit may have a different shape and / or size from another prediction unit.

[0056] When generating a prediction unit that performs intra prediction based on an encoded block, when the prediction unit is not the smallest coding unit, intra prediction may be performed without performing segmentation into multiple prediction units N×N.

[0057] The prediction units 120 and 125 may include an inter - prediction unit 120 that performs inter - frame prediction and an intra - prediction unit 125 that performs intra - frame prediction. It may be determined whether to perform inter - frame prediction or intra - frame prediction for the prediction unit, and the details according to each prediction method (e.g., intra - frame prediction mode, motion vector, reference picture, etc.) may be determined. In this regard, the processing unit that performs the prediction may be different from the processing unit that determines the prediction method and details. For example, the prediction method, prediction mode, etc. may be determined in the prediction unit, and the prediction may be performed in the transform unit. The residual value (residual block) between the generated prediction block and the original block may be input to the transform unit 130. In addition, the prediction mode information, motion vector information, etc. for prediction may be encoded using the residual value in the entropy encoding unit 165 and sent to the decoding device. When using a specific encoding mode, the original block may be encoded as it is without generating a prediction block through the prediction unit 120 or 125 and sent to the decoding unit.

[0058] The inter - prediction unit 120 may predict the prediction unit based on information of at least one of a previous picture or a subsequent picture of the current picture, or in some cases, may predict the prediction unit based on information of some encoded regions in the current picture. The inter - prediction unit 120 may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

[0059] The reference picture interpolation unit may receive reference picture information from the memory 155 and generate pixel information equal to or less than integer pixels in the reference picture. For luminance pixels, a DCT - based 8 - tap interpolation filter with different filter coefficients may be used to generate pixel information equal to or less than integer pixels in units of 1 / 4 pixels. For chrominance signals, a DCT - based 4 - tap interpolation filter with different filter coefficients may be used to generate pixel information equal to or less than integer pixels in units of 1 / 8 pixels.

[0060] The motion prediction unit may perform motion prediction based on the reference picture interpolated by the reference picture interpolation unit. As a method for calculating the motion vector, various methods such as FBMA (Full Search - based Block Matching Algorithm), TSS (Three - Step Search), NTS (New Three - Step Search Algorithm), etc. may be used. The motion vector may have a motion vector value in units of 1 / 2 or 1 / 4 pixels based on the interpolated pixels. The motion prediction unit may predict the current prediction unit by changing the motion prediction method. Various methods such as a skip method, a merge method, an Advanced Motion Vector Prediction (AMVP) method, an Intra - Block Copy method, etc. may be used as the motion prediction method.

[0061] The intra prediction unit 125 may generate a prediction unit based on reference pixel information around a current block, where the reference pixel information is pixel information in a current picture. When an adjacent block in a current prediction unit is a block that performs inter prediction and thus the reference pixel is a pixel that performs inter prediction, the reference pixel included in the block that performs inter prediction may be used by replacing it with reference pixel information of a surrounding block that has performed intra prediction. In other words, when the reference pixel is unavailable, the unavailable reference pixel information may be used by replacing it with at least one of the available reference pixels.

[0062] The prediction mode of intra prediction may have a directional prediction mode that uses reference pixel information according to a prediction direction when performing prediction and a non-directional mode that does not use direction information. The mode for predicting luminance information may be different from the mode for predicting chrominance information, and the intra prediction mode information for predicting luminance information or the predicted luminance signal information may be used to predict chrominance information.

[0063] When the size of a prediction unit is the same as the size of a transform unit during intra prediction, intra prediction of the prediction unit may be performed based on pixels at the left position, the upper left position, and the top position of the prediction unit. However, when the size of the prediction unit is different from the size of the transform unit during intra prediction, intra prediction may be performed by using reference pixels based on the transform unit. In addition, intra prediction using N×N partitioning may be used only for the smallest coding unit.

[0064] In the intra prediction method, a prediction block may be generated after applying an adaptive intra smoothing (AIS) filter to reference pixels according to a prediction mode. The type of the AIS filter applied to the reference pixels may be different. To perform the intra prediction method, the intra prediction mode in a current prediction unit may be predicted according to the intra prediction mode in prediction units around the current prediction unit. When predicting the prediction mode in the current prediction unit by using the mode information predicted according to surrounding prediction units, if the intra prediction mode in the current prediction unit is the same as the intra prediction mode in the surrounding prediction units, information indicating that the intra prediction mode in the current prediction unit is the same as the intra prediction mode in the surrounding prediction units may be sent by using predetermined flag information, and if the prediction mode in the current prediction unit is different from the intra prediction mode in the surrounding prediction units, the prediction mode information of the current block may be encoded by performing entropy coding.

[0065] In addition, a residual block including information about a residual value, which is a difference between a prediction unit that performs prediction based on the prediction units generated in the prediction unit 120 and the prediction unit 125 and an original block in the prediction unit, may be generated. The generated residual block may be input to the transform unit 130.

[0066] The transform unit 130 may transform the original block and the following residual block by using a transform method such as DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), KLT. The residual block includes residual value information in the prediction unit generated by the prediction unit 120 and the prediction unit 125. Whether to apply DCT, DST or KLT to transform the residual block may be determined based on the intra-prediction mode information in the prediction unit used to generate the residual block.

[0067] The quantization unit 135 may quantize the values transformed into the frequency domain in the transform unit 130. The quantization coefficients may vary according to the importance or blocks of the image. The values calculated in the quantization unit 135 may be provided to the dequantization unit 140 and the rearrangement unit 160.

[0068] The rearrangement unit 160 may perform rearrangement on the coefficient values of the quantized residual values.

[0069] The rearrangement unit 160 may change the coefficients in the shape of a two-dimensional block into the shape of a one-dimensional vector by a coefficient scanning method. For example, the rearrangement unit 160 may scan from the DC coefficient to the coefficients in the high-frequency domain by using the zigzag scanning method and change it into the shape of a one-dimensional vector. Depending on the size of the transform unit and the intra-prediction mode, instead of the zigzag scanning, a vertical scan that scans the coefficients in the shape of a two-dimensional block in the column direction or a horizontal scan that scans the coefficients in the shape of a two-dimensional block in the row direction may be used. In other words, which scanning method among the zigzag scanning, the vertical-oriented scanning, and the horizontal-oriented scanning will be used may be determined according to the size of the transform unit and the intra-prediction mode.

[0070] The entropy coding unit 165 may perform entropy coding based on the values calculated by the rearrangement unit 160. The entropy coding may use various coding methods, such as Exponential Golomb, CAVLC (Context Adaptive Variable Length Coding), and CABAC (Context Adaptive Binary Arithmetic Coding).

[0071] The entropy coding unit 165 may encode various information from the rearrangement unit 160 and the prediction unit 120 and the prediction unit 125, such as the residual value coefficient information and block type information in the coding unit, prediction mode information, segmentation unit information, prediction unit information, and transmission unit information, motion vector information, reference frame information, block interpolation information, filtering information, etc.

[0072] The entropy coding unit 165 may perform entropy coding on the coefficient values in the coding unit input from the rearrangement unit 160.

[0073] The dequantization unit 140 and the inverse transformation unit 145 perform dequantization on the values quantized in the quantization unit 135 and perform inverse transformation on the values transformed in the transformation unit 130. The residual values generated by the dequantization unit 140 and the inverse transformation unit 145 can be combined with the prediction units predicted by the motion prediction unit, the motion compensation unit, and the intra prediction unit included in the prediction unit 120 and the prediction unit 125 to generate a reconstructed block.

[0074] The filter unit 150 may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

[0075] The deblocking filter may remove block distortion generated by the boundaries between blocks in the reconstructed picture. To determine whether to perform deblocking, it may be determined whether to apply the deblocking filter to the current block based on the pixels included in several rows or columns included in the block. When applying the deblocking filter to the block, a strong filter or a weak filter may be applied according to the required deblocking filtering strength. In addition, when applying the deblocking filter, when performing horizontal filtering and vertical filtering, the horizontal directional filtering and the vertical directional filtering may be set to be processed in parallel.

[0076] The offset correction unit may correct the offset from the original image in pixel units for the image on which deblocking is performed. To perform offset correction on a specific picture, after dividing the pixels included in the image into a certain number of regions, the region on which the offset will be performed may be determined, and a method of applying the offset to the corresponding region or a method of applying the offset by considering the edge information of each pixel may be used.

[0077] Adaptive loop filtering (ALF) may be performed based on the value obtained by comparing the filtered reconstructed image with the original image. After dividing the pixels included in the image into predetermined groups, filtering may be performed on each group separately by determining one filter to be applied to the corresponding group. Information related to whether ALF will be applied may be transmitted for the luminance signal in coding units (CUs), and the shape and filter coefficients of the ALF filter to be applied may vary according to each block. In addition, an ALF filter having the same shape (fixed shape) may be applied regardless of the characteristics of the block to be applied.

[0078] The memory 155 may store the reconstructed blocks or pictures calculated by the filter unit 150, and when performing inter prediction, may provide the stored reconstructed blocks or pictures to the prediction unit 120 and the prediction unit 125.

[0079] Figure 2 is a block diagram showing an image decoding apparatus according to an embodiment of the present disclosure.

[0080] Refer to Figure 2, the image decoding device 200 may include: an entropy decoding unit 210, a rearrangement unit 215, a dequantization unit 220, an inverse transformation unit 225, a prediction unit 230 and a prediction unit 235, a filter unit 240, and a memory 245.

[0081] When an image bitstream is input from the image encoding device, the input bitstream may be decoded according to a process opposite to that of the image encoding device.

[0082] The entropy decoding unit 210 may perform entropy decoding according to a process opposite to that of performing entropy encoding in the entropy encoding unit of the image encoding device. For example, in response to the method performed in the image encoding device, various methods such as exponential Golomb, CAVLC (context-adaptive variable length coding), and CABAC (context-adaptive binary arithmetic coding) may be applied.

[0083] The entropy decoding unit 210 may decode information related to intra prediction and inter prediction performed in the encoding device.

[0084] The rearrangement unit 215 may perform rearrangement based on the method of rearranging the bitstream entropy decoded by the entropy decoding unit 210 in the encoding unit. The coefficients represented in a one-dimensional vector form may be rearranged by the coefficients reconstructed in a two-dimensional block form. The rearrangement unit 215 may receive information related to the coefficient scanning performed in the encoding unit, and perform rearrangement by a method in which scanning is performed in reverse based on the scanning order performed in the corresponding encoding unit.

[0085] The dequantization unit 220 may perform dequantization based on the quantization parameter provided from the encoding device and the coefficient values of the rearranged block.

[0086] The inverse transformation unit 225 may perform the transformation performed in the transformation unit, that is, the inverse transformation for DCT, DST, and KLT, that is, the inverse DCT, inverse DST, and inverse KLT for the quantization result performed in the image encoding device. The inverse transformation may be performed based on the transmission unit determined in the image encoding device. In the inverse transformation unit 225 of the image decoding device, the transformation technique (e.g., DCT, DST, KLT) may be selectively performed according to a plurality of information such as the prediction method, the size of the current block, and the prediction direction.

[0087] The prediction unit 230 and the prediction unit 235 may generate a prediction block based on the information related to the generation of the prediction block provided from the entropy decoding unit 210 and the pre-decoded block or picture information provided from the memory 245.

[0088] As described above, when performing intra prediction in the same manner as the operations in the image encoding device, when the size of the prediction unit is the same as the size of the transform unit, intra prediction of the prediction unit can be performed based on the pixels at the left position, the upper left position, and the top position of the prediction unit. However, when the size of the prediction unit is different from the size of the transform unit during intra prediction, intra prediction of the prediction unit can be performed by using the reference pixels based on the transform unit. In addition, intra prediction using N×N partitioning can be used only for the minimum coding unit.

[0089] The prediction unit 230 and the prediction unit 235 may include: a prediction unit determination unit, an inter prediction unit, and an intra prediction unit. The prediction unit determination unit may receive various information such as prediction unit information, prediction mode information of an intra prediction method, and motion prediction related information of an inter prediction method input from the entropy decoding unit 210, partition the prediction unit in the current coding unit, and determine whether the prediction unit performs inter prediction or intra prediction. The inter prediction unit 230 may perform inter prediction on the current prediction unit by using the information required for inter prediction in the current prediction unit provided from the image encoding device, based on the information included in at least one of the previous picture or the subsequent picture of the current picture including the current prediction unit. Alternatively, inter prediction may be performed based on the information about some regions pre-reconstructed in the current picture including the current prediction unit.

[0090] To perform inter prediction, it is possible to determine whether the motion prediction method included in the prediction unit included in the corresponding coding unit is a skip mode, a merge mode, an AMVP mode, or an intra block copy mode based on the coding unit.

[0091] The intra prediction unit 235 may generate a prediction block based on the pixel information in the current picture. When the prediction unit is a prediction unit that has performed intra prediction, intra prediction can be performed based on the intra prediction mode information in the prediction unit provided from the image encoding device. The intra prediction unit 235 may include an adaptive intra smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. As a part of filtering the reference pixels of the current block, the AIS filter may be applied by determining whether to apply the filter according to the prediction mode in the current prediction unit. By using the prediction mode and the AIS filter information in the prediction unit provided from the image encoding device, AIS filtering can be performed on the reference pixels of the current block. When the prediction mode of the current block is a mode that does not perform AIS filtering, the AIS filter may not be applied.

[0092] When the prediction mode in the prediction unit is a prediction unit that performs intra prediction based on the pixel values obtained by interpolating reference pixels, the reference pixel interpolation unit may interpolate the reference pixels to generate reference pixels in units of pixels equal to or less than integer values. When the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixels, the reference pixels may not be interpolated. When the prediction mode of the current block is the DC mode, the DC filter may generate a prediction block by filtering.

[0093] A reconstructed block or picture may be provided to the filter unit 240. The filter unit 240 may include a deblocking filter, an offset correction unit, and an ALF.

[0094] Information on whether to apply the deblocking filter to the corresponding block or picture and information on whether to apply a strong filter or a weak filter when the deblocking filter is applied may be provided from the image encoding device. Information related to the deblocking filter provided from the image encoding device may be provided in the deblocking filter of the image decoding device, and deblocking filtering of the corresponding block may be performed in the image decoding device.

[0095] The offset correction unit may perform offset correction on the reconstructed image based on the offset value information and the type of offset correction applied to the image during encoding.

[0096] The ALF may be applied to the coding unit based on the information on whether to apply the ALF provided from the encoding device, the ALF coefficient information, etc. Such ALF information may be provided by including it in a specific parameter set.

[0097] The memory 245 may store the reconstructed picture or block to be used as a reference picture or reference block, and provide the reconstructed picture to the output unit.

[0098] As described above, hereinafter, in the embodiments of the present disclosure, for ease of description, the coding unit is used as a term for the coding unit, but it may be a unit that performs both decoding and encoding.

[0099] In addition, since the current block represents the block to be encoded / decoded, depending on the encoding / decoding step, it may represent a coding tree block (or coding tree unit), a coding block (or coding unit), a transform block (or transform unit), or a prediction block (or prediction unit), etc. In this specification, "unit" may represent the basic unit for performing a specific encoding / decoding process, and "block" may represent an array of pixels of a predetermined size. Unless otherwise classified, "block" and "unit" may be used interchangeably. For example, in the embodiments described later, it may be understood that the coding block (coding block) and the coding unit (coding unit) are used interchangeably.

[0100] Images can be encoded / decoded in units of blocks. Encoding blocks can be recursively divided based on a tree structure. In an example, an encoding block can be divided by at least one of quadtree division, binary tree division, or ternary tree division.

[0101] In addition, an encoding block can be divided into a plurality of prediction blocks or a plurality of transform blocks.

[0102] Figure 3 is a flowchart showing an intra prediction method according to an embodiment of the present disclosure.

[0103] Referring to Figure 3 , an index S301 of a reference sample row of a current block can be determined. The index can specify one of a plurality of reference sample row candidates. The plurality of reference sample row candidates can include an adjacent reference sample row adjacent to the current block and at least one non-adjacent reference sample row not adjacent to the current block.

[0104] In an example, an adjacent reference sample row including an adjacent row whose y-axis coordinate is 1 less than the uppermost row of the current block and an adjacent column whose x-axis coordinate is 1 less than the leftmost column of the current block can be used as a reference sample row candidate.

[0105] A first non-adjacent reference sample row including a non-adjacent row whose y-axis coordinate is 2 less than the uppermost row of the current block and a non-adjacent column whose x-axis coordinate is 2 less than the leftmost column of the current block can be used as a reference sample row candidate.

[0106] A second non-adjacent reference sample row including a non-adjacent row whose y-axis coordinate is 3 less than the uppermost row of the current block and a non-adjacent column whose x-axis coordinate is 3 less than the leftmost column of the current block can be used as a reference sample row candidate.

[0107] The index can indicate one of an adjacent reference sample row, a first non-adjacent reference sample row, or a second non-adjacent reference sample row. In an example, when the index is 0, it means that the adjacent reference sample row is selected, when the index is 1, it means that the first non-adjacent reference sample row is selected, and when the index is 2, it means that the second non-adjacent reference sample row is selected.

[0108] An index specifying one of a plurality of reference sample row candidates can be signaled in a bitstream.

[0109] Alternatively, the index can be signaled for a luminance component block, and signaling of the index can be omitted for a chrominance component block. When signaling of the index is omitted, the index can be inferred as 0. In other words, for a chrominance component block, intra prediction can be performed by using an adjacent reference sample row.

[0110] Reconstructed samples included in the selected reference sample row can be obtained as reference samples.

[0111] Next, the intra prediction mode of the current block can be determined S302.

[0112] Figure 4 The types of intra prediction modes are shown. As Figure 4 In the example shown, the intra prediction modes include non - directional prediction modes (DC and planar) and directional prediction modes. Figure 4 65 directional prediction modes are shown.

[0113] A flag indicating whether the intra prediction mode of the current block is the same as the MPM (Most Probable Mode) can be signaled in the bitstream. In the example, when the value of the MPM flag is 1, it indicates that there is an MPM that is the same as the intra prediction mode of the current block. On the other hand, when the value of the MPM flag is 0, it indicates that there is no MPM that is the same as the intra prediction mode of the current block.

[0114] When the value of the MPM flag is 1, a flag indicating whether the intra prediction mode of the current block is the same as the default intra prediction mode can be signaled. The default intra prediction mode can be at least one of DC, planar, vertical directional prediction mode, or horizontal directional prediction mode. In the example, intra_not_planar_flag can be signaled, which is a flag indicating whether the intra prediction mode of the current block is a planar mode. When the value of the flag intra_not_planar_flag is 0, this indicates that the intra prediction mode of the current block is planar. On the other hand, when the value of the flag intra_not_planar_flag is 1, this indicates that the intra prediction mode of the current block is not planar. When the value of the flag intra_not_planar_flag is 1, the index specifying one of the MPM candidates can be signaled. The intra prediction mode of the current block can be set to be the same as the MPM indicated by the MPM index.

[0115] Based on the intra prediction mode and the reference samples belonging to the reference sample row, the predicted sample can be obtained S303.

[0116] When the intra prediction mode of the current block is a directional prediction mode, the predicted sample can be obtained by using the reference samples located on the line following the angle of the directional prediction mode.

[0117] When the intra prediction mode of the current block is a planar mode, the predicted sample can be obtained by using the reference samples in the vertical direction of the sample to be predicted and the reference samples in the horizontal direction of the sample to be predicted.

[0118] Figure 5 It is a diagram for describing an example of obtaining a predicted sample in the planar mode.

[0119] InFigure 5 where T represents a reference sample adjacent to the upper right corner of the current block, and L represents a reference sample adjacent to the lower left corner of the current block.

[0120] In the planar mode, for a sample to be predicted, a horizontally oriented prediction sample P1 and a vertically oriented prediction sample P2 can be obtained.

[0121] The horizontally oriented prediction sample P1 can be generated by performing linear interpolation on the upper right reference sample T and the reference sample H that are located on the same horizontal line as the sample to be predicted.

[0122] The vertically oriented prediction sample P2 can be generated by performing linear interpolation on the lower left reference sample L and the reference sample V that are located on the same vertical line as the sample to be predicted.

[0123] Subsequently, based on the weighted sum operation of the horizontally oriented prediction sample P1 and the vertically oriented prediction sample P2, a prediction sample can be obtained. Equation 1 shows an example of obtaining the prediction sample P through the weighted sum operation of the horizontally oriented prediction sample P1 and the vertically oriented prediction sample P2.

[0124] [Equation 1]

[0125] P = (α × P1 + β × P2) / (α + β)

[0126] In Equation 1, α represents the weight applied to the horizontally oriented prediction sample P1, and β represents the weight applied to the vertically oriented prediction sample P2.

[0127] The weights α and β can be determined based on the size or shape of the current block. Specifically, the weights α and β can be determined by considering at least one of the width or height of the current block. In an example, when the width and height of the current block are the same, the weights α and β can be set to the same value. When the weights α and β are the same, the prediction sample can be obtained as the average of the horizontally oriented prediction sample P1 and the vertically oriented prediction sample P2. On the other hand, when the width and height of the current block are different, the weights α and β can be set differently. In an example, when the width of the current block is greater than the height, the weight β can be set to a value greater than the weight α, and when the height of the current block is greater than the width, the weight α can be set to a value greater than the weight β. Alternatively, conversely, when the width of the current block is greater than the height, the weight α can be set to a value greater than the weight β, and when the height of the current block is greater than the width, the weight β can be set to a value greater than the weight α.

[0128] In another example, the weights α and β can be derived from one of multiple weight set candidates. In the example, when a weight candidate set (1, 1), (3, 1), and (1, 3) that predefines combinations of the weights α and β is available, the weights α and β can be selected to be the same as one of the weight candidate sets.

[0129] An index indicating one of multiple weight set candidates can be signaled in the bitstream. The index can be signaled at the block level. In the example, the index can be signaled in units of coded blocks or transform blocks.

[0130] Alternatively, the index can be signaled at the level of coding tree units, slices, pictures, or sequences. Blocks included in the index transmission unit can determine the weights α and β by referring to the index signaled at a higher level. In other words, for blocks included in the index transmission unit, the weights α and β can be set to be the same.

[0131] In Figure 5 the example, it is shown that the upper-right reference sample T is used to derive the horizontally-directed prediction sample P1, and the lower-left reference sample L is used to derive the vertically-directed prediction sample P2.

[0132] The horizontally-directed prediction sample P1 can be derived by using reference samples other than the upper-right reference sample, or the vertically-directed prediction sample P2 can be derived by using reference samples other than the lower-left reference sample. In the example, a reference sample set candidate for the first reference sample used to derive the horizontally-directed prediction sample P1 and the second reference sample used to derive the vertically-directed prediction sample P2 can be configured, and the horizontally-directed prediction sample P1 and the vertically-directed prediction sample P2 can be derived by using one candidate selected from multiple reference sample set candidates.

[0133] An index identifying one of multiple reference sample set candidates can be signaled in the bitstream. The index can be signaled in units of blocks, sub-blocks, or samples.

[0134] Alternatively, the reference sample set candidate can be selected based on the position of the sample to be predicted.

[0135] In the directional prediction mode, prediction samples can be generated by using reconstructed pixels around the current block.

[0136] Figure 6 Examples of generating prediction samples in the horizontal mode and the vertical mode are shown.

[0137] As Figure 6 shown in the example, in the horizontal mode, prediction samples can be generated by using the reconstructed samples in the horizontal direction of the sample to be predicted.

[0138] In the vertical mode, a predicted sample can be generated by using a reconstructed sample in the vertical direction of the sample to be predicted.

[0139] After performing intra prediction based on an intra prediction mode, a residual block can be obtained by subtracting the predicted block from the original block.

[0140] In this case, a prediction method using one of the 0th mode to the 66th mode can be used, or a limited prediction method can also be used. In the limited prediction method, only the intra prediction mode in the horizontal direction (the 18th) or the intra prediction mode in the vertical direction (the 50th) can be used. In this case, the intra prediction mode can be specified by 1-bit information. Alternatively, diagonal orientation prediction modes such as the lower left diagonal direction (the 2nd) and the upper right diagonal direction (the 66th) in addition to the vertical and horizontal directions can be added as available candidates. In this case, the intra prediction mode can be specified by 2-bit information. Alternatively, two of the three diagonal direction modes such as the lower left diagonal direction (the 2nd), the upper left diagonal direction (the 34th), and the upper right diagonal direction (the 66th) can be added as available candidates.

[0141] The number of available intra prediction modes can be encoded and signaled to the decoder. Alternatively, the number of intra prediction modes used in the encoder and decoder can be fixed. Alternatively, the number of available intra prediction modes can be determined based on the size or shape of the current block.

[0142] After performing prediction, a residual block can be obtained by subtracting the predicted block from the original block. When the residual block is obtained, residual coefficients can be obtained by performing at least one of transformation or quantization on the residual block.

[0143] Information indicating whether transformation is applied to the current block can be encoded and signaled. In an example, transform_skip_flag can be encoded and signaled. When transform_skip_flag is 1, this indicates that transformation is not applied to the current block. Here, the transformation can include a second transformation as well as a first transformation to be described later. When transform_skip_flag is 0, this indicates that transformation is applied to the current block. When transform_skip_flag is 0, the first transformation must be applied to the current block, and the second transformation can be optionally applied.

[0144] The transformation can be performed based on at least one of a DCT-based transformation kernel or a DST-based transformation kernel. Here, the DCT-based transformation kernel can include at least one of DCT-2 or DCT-8, and the DST-based transformation kernel can include DST-7. An additional transformation can be applied to the result of the transform residual samples. Hereinafter, for ease of description, the transformation performed by the DCT- or DST-based transformation kernel is referred to as the first transformation, and the transformation additionally applied to the result of the first transformation is referred to as the second transformation. In addition, the transform coefficients generated from the result of the first transformation are referred to as first transform coefficients, and the transform coefficients generated from the result of the second transformation are referred to as second transform coefficients.

[0145] The second transformation can be applied to at least a portion of the first transform coefficients. In an example, depending on the size of the second transformation kernel, the second transformation can be applied to 16, 48, or 64 first transform coefficients. The shape of the region including the first transform coefficients to which the second transformation is applied can be square, non-square, or polygonal.

[0146] Equation 2 represents the application aspect of the second transformation.

[0147] [Equation 2]

[0148] B R×1 = T R×N · A N×1

[0149] In Equation 2, B_R×l represents the second transform coefficients consisting of R rows and 1 column. T_R×N represents the second transformation kernel consisting of R rows and N columns. A_N×1 represents the first transform coefficients consisting of N rows and 1 column.

[0150] Figure 7 and Figure 8 are diagrams showing examples of applying the second transformation.

[0151] Figure 7 Shows an example where the second transformation kernel has a size of 64×64. The first transform coefficients generated from the result of the first transformation in an 8×8 block can be arranged in one dimension. In this case, a one-dimensional array can be generated by scanning the first transform coefficients in a predetermined scanning method. The predetermined scanning method can include at least one of diagonal-oriented scanning, horizontal-oriented scanning, vertical-oriented scanning, or raster scanning.

[0152] When generating a 64×1 size input matrix by rearrangement, the second transform coefficients can be obtained by a matrix product between a 64×64 size second transformation kernel and a 64×1 size input matrix.

[0153] As a result of performing the second transform, 64 second transform coefficients can be generated and the second transform coefficients in an 8×8 block can be rearranged. After quantizing the 8×8 block in which the second transform coefficients are rearranged, the quantized transform block can be coded.

[0154] Figure 8 An example where the second transform kernel has a size of 48×48. Among the first transform coefficients generated as a result of the first transform in an 8×8 block, 48 first transform coefficients can be rearranged in one dimension. In this case, the 48 first transform coefficients can be included in a polygon region in the 8×8 block excluding the lower right sub-block having a size of 4×4.

[0155] When generating a 48×1 size input matrix by rearranging 48 first transform coefficients in one dimension, the second transform coefficients can be obtained by a matrix product between a 48×48 size second transform kernel and the 48×1 size input matrix.

[0156] As a result of performing the second transform, 48 second transform coefficients can be generated and the second transform coefficients in an 8×8 block can be rearranged. In the example, the 48 second transform coefficients can be rearranged in a polygon region in the 8×8 block excluding the lower right sub-block having a size of 4×4.

[0157] In a region not used for arranging the second transform coefficients, the first transform coefficients can remain as they are. After applying quantization to a block including the second transform coefficients and the first transform coefficients, the quantized transform block can be coded.

[0158] Alternatively, the transform coefficients in a region not used for arranging the second transform coefficients can be set to 0. In other words, quantization and coding can be performed after setting the values of the transform coefficients in a region where the second transform is not applied to 0.

[0159] The size of the second transform kernel can be determined based on the size of the current block. In the example, when at least one of the width or height of the current block is 4, the second transform can be applied to 16 first transform coefficients. On the other hand, when the width and height of the current block are equal to or greater than 8, the second transform can be applied to 48 or 64 first transform coefficients.

[0160] Alternatively, information indicating the size and type of the second transform kernel can be encoded and signaled. The index can be signaled at the block level. In an example, information specifying at least one of the number of columns or rows of the transform size can be encoded. Alternatively, after different indices are assigned to each combination of the number of rows and columns, an index of one of the specified combinations can be encoded. Alternatively, after different indices are assigned to each of a plurality of second transform kernel candidates, an index of one of the specified second transform kernel candidates can be encoded. Here, for each of the plurality of second transform kernel candidates, at least one of the size or the coefficients can be different.

[0161] Alternatively, based on the size of the current block, after determining the size of the second transform kernel, an index specifying one of a plurality of second transform kernel candidates having the determined size can be encoded.

[0162] Figure 7 and Figure 8 The example shown in and shows the use of a second transform kernel with the same number of rows and columns. To simplify the second transform, the number of rows and columns can be set differently.

[0163] Figure 9 and Figure 10 shows a second transform based on a second transform kernel having an asymmetric shape.

[0164] The number of rows R of the second transform kernel can be set to a value less than the number of columns N. For example, the number of rows R can be set to 8, and the number of columns N can be set to 48.

[0165] When the number of rows of the second transform kernel decreases, the number of second transform coefficients output as a result of the second transform also decreases. For example, when performing a matrix multiplication between a second transform kernel of size 8×48 and an input matrix of size 48×1, second transform coefficients of size 8×1 are generated.

[0166] The 8 second transform coefficients can be rearranged in an 8×8 block. In this case, the values of the transform coefficients can be set to 0 in the region where the second transform coefficients are not assigned within the region where the second transform is applied (i.e., the region including the first transform coefficients to which the second transform is applied). For example, when a polygon region including 48 samples is the region where the second transform is applied, the values of the transform coefficients can be set to 0 in the remaining region of the polygon region excluding the region where the 8 second transform coefficients are assigned.

[0167] In the region where the second transform is not applied, the first transform coefficients can remain as they are.

[0168] Alternatively, at least a part of the first transform coefficients in the region where the second transform is not applied can be converted to 0 and encoded. In Figure 10An example is shown in which at least a part of a region where a second transform is not applied is converted to 0.

[0169] As Figure 10 shown in (a) of, within a region where the second transform is not performed, the value of a first transform coefficient corresponding to a high-frequency domain can be converted to 0. In the example, the value of a first transform coefficient whose sum of x-axis and y-axis coordinates is equal to or greater than a threshold can be converted to 0.

[0170] Alternatively, depending on the shape, the first transform coefficients to be converted to 0 can be selected. In the example, as Figure 10 shown in (b) of, the first transform coefficients included in the n bottom rows within a region where the second transform is not performed can be converted to 0. Alternatively, as Figure 10 shown in (c) of, the first transform coefficients included in the n right columns within a region where the second transform is not performed can be converted to 0.

[0171] Alternatively, as Figure 10 shown in (d) of, all the first transform coefficients within a region where the second transform is not performed can be converted to 0.

[0172] The shape of a region including the first transform coefficients converted to 0 can be determined based on at least one of the size or shape of the current block, the intra prediction mode, or the transform kernel. Alternatively, an index specifying one candidate shape corresponding to the region among a plurality of candidate shapes can be encoded and signaled.

[0173] Whether to allow the second transform can be determined based on at least one of the coding mode of the current block or the first transform kernel. Here, the coding mode indicates intra prediction or inter prediction. In the example, although the second transform is allowed when the current block is encoded by intra prediction, it may not be allowed when the current block is encoded by inter prediction.

[0174] Information indicating whether the second transform has been applied can be encoded and signaled. This information can be a 1-bit flag. Depending on whether the flag is true or false, it can be determined whether to apply the second transform to the current block. Alternatively, this information can be index information. When the value of the index is 0, this indicates that the second transform has not been applied to the current block. On the other hand, when the value of the index is greater than 0, this indicates that the second transform has been applied to the current block. When the value of the index is greater than 0, the second transform kernel can be specified by the index.

[0175] Information indicating whether to perform the second transform on the current block can be separately encoded for each color component. In the example, for each of the luminance component (Y), the first chrominance component (Cb), and the second chrominance component (Cr), information indicating whether the second transform has been performed can be encoded.

[0176] Alternatively, for the chrominance components, information indicating whether the second transform has been performed can be jointly encoded. In an example, for each chrominance component (Cb, Cr), it can be jointly determined whether to apply the second transform. In other words, the first chrominance component (Cb) and the second chrominance component (Cr) can share the information indicating whether the second transform has been performed.

[0177] Alternatively, it can be determined whether information is encoded by color component based on a tree structure. In an example, when the luma component and the chrominance components have the same tree structure, the three color components (i.e., Y, Cb, Cr) can share the information indicating whether the second transform is performed. On the other hand, when the luma component and the chrominance components have different tree structures, information indicating whether the second transform is performed can be signaled for each of the luma component and the chrominance components.

[0178] A plurality of second transform kernel candidates can be grouped into at least a plurality of groups. Based on at least one of the size or shape of the current block or the intra prediction mode therein, one of the plurality of groups can be specified. When a group is specified, at least one of the plurality of second transform kernel candidates included in the specified group can be specified by using index information.

[0179] It can be determined whether to encode information indicating whether to apply the second transform based on the position of the last non-zero coefficient in the current block.

[0180] Figure 11 An example showing that it is determined whether to encode information indicating whether to apply the second transform based on the position of the last non-zero coefficient.

[0181] For ease of description, it is assumed that when the second transform is performed, in the remaining region excluding the region where the second transform coefficients are rearranged, the values of the transform coefficients are set to 0.

[0182] As a result of performing the second transform, as many second transform coefficients as the number of rows R of the second transform kernel are generated. As in the above example, since all values of the remaining transform coefficients excluding the second transform coefficients are set to 0, there are no non-zero coefficients in the remaining region excluding the region where R second transform coefficients are rearranged. According to this principle, the region where R second transform coefficients are rearranged can be set as a limited region.

[0183] In Figure 11 (a) to (c) of, it is shown that the upper left 4×4 block in an 8×8 sized block is set as the limited region.

[0184] When non-zero coefficients exist outside the finite region, this indicates that the second transform is not applied to the current block. Therefore, when the last non-zero coefficient exists outside the finite region, the encoding of the information indicating whether the second transform is applied to the current block can be omitted. In the example, as Figure 11 shown in (a) of

[0185] When the last non-zero coefficient exists outside the finite region, the encoding of the information indicating whether the second transform is applied can be omitted. The decoder can determine that the second inverse transform is not applied to the current block without decoding the information.

[0186] As Figure 11 shown in (b) of Figure 11 When the second transform is applied to the current block, non-zero transform coefficients can exist only in the finite region. Alternatively, as

[0187] shown in (c) of

[0188] Even though the second transform is not applied to the current block, it is possible that non-zero transform coefficients exist only in the finite region. Therefore, when the last non-zero coefficient exists within the finite region, the information indicating whether the second transform is applied can be encoded. The decoder can determine whether to apply the second inverse transform to the current block based on this information.

[0189] Alternatively, when the last non-zero coefficient exists within the finite region, the encoding of the information can be omitted and it can be defaulted that the second transform is applied. The size of the finite region can be determined based on the size of the second transform kernel. In the example, when the second transform kernel is a matrix of size R×N, a rectangular region with width and height of Log2R can be set as the finite region.

[0190] Alternatively, the region where the second transform is applied can be set as the finite region.

[0191] Alternatively, the information indicating at least one of the size or shape of the finite region can be encoded and signaled. This information can be signaled at a higher level such as in a sequence, picture header, or slice header.

[0192] Alternatively, at least one of the size or shape of the finite region can be predefined in the encoder and decoder. In the example, between the encoder and the decoder, it can be pre-agreed that the upper left block of size 4×4 in the current block is set as the finite region.

[0193] Alternatively, at least one of the size or shape of the finite region can be adaptively determined based on at least one of the size or shape of the current block, the first transform kernel, or the intra prediction mode.

[0194] Alternatively, after defining a plurality of finite region candidates, an index specifying one of the plurality of finite region candidates may be encoded and signaled.

[0195] Figure 12 Finite region candidates for a 4×4 sized block are shown.

[0196] When the current block has a size of 4×4, index information specifying one of the plurality of finite region candidates shown may be encoded. Figure 12

[0197] At least one of the finite region candidates shown in Figure 12 may be applied to blocks having a size greater than 4×4 as well as 4×4 sized blocks. In an example, at least one of the finite region candidates shown in Figure 12 may also be applied to blocks having at least one of a width or height of 4 and the other greater than 4.

[0198] Alternatively, the size or number of finite region candidates may be set differently according to the size of the current block.

[0199] Instead of encoding an index specifying one of the finite region candidates, one of the finite region candidates may be specified based on the size or shape of the current block.

[0200] In the above example, it was described that the size of the second transform kernel can be adaptively selected. In other examples, a second transform kernel of a predefined size may be applied to all blocks. In an example, a 16×48 sized second transform kernel may be used for all blocks. In this case, the second transform may be applied to 48 first transform coefficients.

[0201] Figure 13 is a diagram showing an example where a second transform kernel of a predetermined size is applied.

[0202] When using a 16×48 sized second transform kernel, in the example shown in Figure 13 (a), the second transform may be applied to the region excluding the lower right sub-block of 4×4 size in an 8×8 sized block. In an example, the second transform may be applied to the first transform coefficients included in the polygonal region shown in Figure 13 (a).

[0203] In this case, when at least one of the width or height of the current block is less than 8, the second transform may not be applied to the current block.

[0204] Alternatively, when at least one of the width or height of the current block is less than 8, the second transform may be performed after transforming the region to which the second transform will be applied into a rectangular shape such as 4×12 or 12×4. ​

[0205] Alternatively, when at least one of the width or height of the current block is less than 8, after matching the upper left position of the current block with the upper left area of the region to which the second transform is applied, the second transform can be applied only to the overlapping area between the current block and the region to which the second transform is applied.

[0206] Figure 13 Subfigure (b) shows an example where the second transform is performed only on the overlapping area.

[0207] After arranging the first transform coefficients included in the overlapping area in a one-dimensional manner, the first transform coefficient can be set as the input matrix for the second transform.

[0208] When the first transform and the second transform are applied to the current block, the decoder can obtain the residual samples by performing the inverse transform of the first transform (first inverse transform) on the result of performing the inverse transform of the second transform (second inverse transform).

[0209] The second inverse transform can be performed based on the transposed matrix of the second transform kernel. In an example, when the second transform kernel has a size of 8×48, the second inverse transform can be performed by a transform kernel with a size of 48×8.

[0210] The second transform coefficients generated by the second transform can be set as the input matrix for the second inverse transform. In an example, when the second transform is performed by a second transform kernel with a size of 8×48, an input matrix of 8×1 consisting of 8 coefficients can be used when performing the second inverse transform. Subsequently, the transform coefficients after performing the second inverse transform can be output through the matrix product between the input matrix and the transposed matrix of the second transform kernel. In an example, 48 transform coefficients can be output through the matrix product between a transform kernel with a size of 48×8 and an input matrix with a size of 8×1.

[0211] After rearranging the transform coefficients in the current block, the first inverse transform can be applied to the rearranged block.

[0212] After quantizing the transform coefficients generated by the transform residual samples, the quantized transform coefficients can be encoded. Alternatively, quantization can be omitted and the transform coefficients can be encoded.

[0213] When no transform is applied to the current block, the residual samples can be quantized and the quantized residual samples can be encoded.

[0214] When the transform is skipped, the quantization information can be additionally encoded block by block. In an example, the quantization information for transform skip encoded by applying DPCM (Differential Pulse Code Modulation) to the quantization information sent through the sequence, picture header, or slice header can be additionally signaled.

[0215] After performing quantization, run-length encoding can be applied. In other words, the quantized coefficients generated from the quantization result can be encoded by the run-length method. Here, a run refers to the same data being consecutive, and the run length refers to the length of the consecutive data. In an example, when there is a string aaaaaabbccccccc, a is consecutive 6 times, b is consecutive 2 times, and c is consecutive 7 times, so it can be expressed as 6a2b7c or a6b2c7 and encoded accordingly.

[0216] The encoding method can be defined as a run-length encoding method.

[0217] For effective run-length encoding, an optimal scanning method can be determined.

[0218] Figure 14 A scanning method is shown.

[0219] According to Figure 14 the shown scanning method, the coefficients are scanned according to a specific directionality. In Figure 14 the shown scanning method, when the same values are arranged consecutively, it can be determined as the optimal scanning method.

[0220] When a transform skip is applied to the current block, the information specifying the scanning method for the current block can be encoded and signaled. This information can be an index specifying one of multiple scanning methods.

[0221] The number or type of available scanning method candidates can be set differently based on at least one of the size, shape, or intra prediction mode of the current block. In an example, when the intra prediction mode of the current block is in the horizontal direction or the vertical direction, only Figure 4 the 2 shown scanning methods can be set as candidates. On the other hand, when the intra prediction mode of the current block is in the diagonal direction (e.g., 2, 34, or 66), Figure 14 all 4 shown scanning methods can be set as candidates. Therefore, depending on the intra prediction mode of the current block, the bit length assigned to the index for specifying the scanning method can be different. In an example, when the intra prediction mode of the current block is in the horizontal direction or the vertical direction, the index can have a 1-bit length. On the other hand, when the intra prediction mode of the current block is in the diagonal direction, the index can have a 2-bit length.

[0222] Alternatively, the scanning method can be determined based on at least one of the size, shape, or intra prediction mode of the current block. In an example, when the intra prediction mode is in the horizontal direction, the horizontal directional scanning method or the vertical directional scanning method shown in Figure 14 (a) can be applied.

[0223] Instead of encoding the quantized coefficients by a run - length method, an encoding method applying additional prediction to the quantized coefficients can be applied. In an example, when generating a residual block by intra - prediction in a current block, quantization can be performed by skipping the transform on the residual block. When outputting the coefficients quantized by quantization, DPCM can be applied to the output values.

[0224] One of a plurality of direction candidates can be used for DPCM. In an example, horizontal - direction DPCM or vertical - direction DPCM can be applied to the quantized coefficients.

[0225] The encoder can encode and signal information for specifying the DPCM direction applied to the quantized residual coefficients. Alternatively, the prediction direction for generating a prediction block can be set as the DPCM direction.

[0226] The DPCM direction can be used when predicting an intra - prediction mode. In an example, when horizontal - direction DPCM is applied to a neighboring block when obtaining the MPM candidates for the current block, the MPM can be obtained by regarding the intra - prediction mode of the neighboring block as the horizontal direction. Alternatively, when vertical - direction DPCM is applied to a neighboring block when obtaining the MPM candidates for the current block, the MPM can be obtained by regarding the intra - prediction mode of the neighboring block as the vertical direction. Alternatively, when diagonal - direction DPCM is applied to a neighboring block, the MPM can be obtained by regarding the intra - prediction mode of the neighboring block as a diagonal direction (e.g., 2, 34, or 66) or a non - directional mode (e.g., planar or DC).

[0227] In this case, the MPM directly obtained from the DPCM direction can have the highest or lowest priority among the MPM candidates. Here, the highest priority can mean that the lowest index among the MPM candidates is assigned (i.e., set as the first MPM), and the lowest priority can mean that the highest index among the MPM candidates is assigned (i.e., set as the last MPM).

[0228] For ease of description, the data related to the residual signal encoded in the encoder is referred to as residual coefficients. For example, depending on whether transform or quantization is applied, the residual coefficients can mean at least one of the quantized transform coefficients, transform coefficients, or quantized residual samples.

[0229] A flag indicating whether there are non - zero residual coefficients in the current block can be encoded and signaled. When there are non - zero residual coefficients in the current block, the position of the last non - zero residual coefficient in the scan order can be encoded.

[0230] In addition, sub-block flags indicating whether there are non-zero residual coefficients in a sub-block can be encoded in units of sub-blocks in the current block. When there are non-zero residual coefficients in a sub-block, information about each residual coefficient can additionally be encoded in scan order.

[0231] In this case, for sub-blocks scanned before the sub-block including the last non-zero residual coefficient, encoding of the sub-block flag can be omitted. Since the sub-block does not include non-zero residual coefficients, the value of the sub-block flag can be inferred to be 0.

[0232] In addition, for the sub-block including the last non-zero residual coefficient, encoding of the sub-block flag can be omitted. Since the sub-block necessarily includes non-zero residual coefficients, the value of the sub-block flag can be inferred to be 1.

[0233] In another example, encoding of the position information of the last non-zero residual coefficient can be omitted. When encoding of the position information of the last non-zero residual coefficient is omitted, sub-block flags can be encoded for all sub-blocks in the current block.

[0234] In this case, when it is determined that there are no non-zero residual coefficients in the remaining sub-blocks excluding the sub-block with the last scan order, it can be understood that non-zero residual coefficients should be included in the last sub-block. Therefore, for the last sub-block, encoding of the sub-block flag can be omitted and the value can be inferred to be 1.

[0235] Information indicating whether the position information of the last non-zero coefficient is encoded can additionally be encoded. When encoding the position information of the last non-zero coefficient, the value of this information can be set to 1. In this case, sub-block flags can be encoded starting from the sub-block where the last non-zero coefficient exists. On the other hand, when the position information of the last non-zero coefficient is not encoded, the value of this information can be set to 0. In this case, sub-block flags can be encoded starting from the first scanned sub-block.

[0236] When there are non-zero residual coefficients in the current block, it can be assumed that for the first sub-block in the current block, non-zero residual coefficients are necessarily included. Therefore, for the first sub-block, encoding of the sub-block flag indicating whether there are non-zero residual coefficients can be omitted.

[0237] Information about each residual coefficient can include at least one of a flag indicating whether the residual coefficient has a non-zero value, information indicating the magnitude of the residual coefficient, and information indicating the sign of the residual coefficient.

[0238] The residual coefficients can be encoded in a predetermined scan order. In this case, the encoding order of the residual coefficients can be different based on whether the transform is skipped in the current block. In an example, when the transform is not skipped in the current block, the residual coefficients at the lower-right position in the sub-block can be encoded first, and the residual coefficients at the upper-left position can be encoded last. In other words, the scan order among the residual coefficients can be determined according to reverse diagonal scan, reverse horizontal scan, or reverse vertical scan. On the other hand, when the transform is skipped in the current block, the residual coefficients at the upper-left position in the sub-block can be encoded first, and the residual coefficients at the lower-right position can be encoded last. In other words, the scan order among the residual coefficients can be determined according to diagonal scan, horizontal scan, or vertical scan.

[0239] Alternatively, when the transform is skipped in the current block, the scan order among the residual coefficients can be determined according to reverse diagonal scan, reverse horizontal scan, or reverse vertical scan.

[0240] The scan order of the residual coefficients can be predefined in the encoder and decoder. Alternatively, the information indicating the scan order of the residual coefficients can be encoded and signaled. Alternatively, the scan order can be determined based on at least one of the following: the size or shape of the current block, the intra prediction mode, whether the transform is skipped, or whether a second transform is performed.

[0241] Figure 15 is a flowchart showing the process of encoding the residual coefficients in the encoder.

[0242] First, a flag significant_flag indicating whether the residual coefficient has a non-zero value can be encoded S1510. When the value of the residual coefficient is 0, the value of the flag sig_flag can be set to 0 and encoded. On the other hand, when the value of the residual coefficient is not 0, the value of the flag sig_flag can be set to 1 and encoded. When the value of the residual coefficient is not 0, the magnitude information of the residual coefficient can also be encoded S1520.

[0243] Figure 16 is a flowchart showing the process of encoding the magnitude information of the residual coefficient.

[0244] The absolute value of the residual coefficient can be encoded by using at least one or more gt_N_flag. In this case, N can be a natural number equal to or greater than 1. The flag gt_N_flag can indicate whether the absolute value of the residual coefficient has a value greater than 2(N - 1). The number of gt_N_flag used for encoding the absolute value of the residual coefficient can be determined based on whether the transform is skipped in the current block. In an example, when the transform is not skipped in the current block, 2 gt_N_flag can be used (N ranges from 1 to 2). On the other hand, when the transform is skipped in the current block, 3 or more gt_N_flag can be used (e.g., 3, 4, or 5). In this embodiment, it is assumed that 2 gt_N_flag are used.

[0245] The flag gt1_flag indicating whether the absolute value of the residual coefficient is greater than 1 can be encoded at S1610. When the absolute value of the residual coefficient is 1, the value of the flag gt1_flag can be encoded with 0. On the other hand, when the absolute value of the residual coefficient is greater than 1, the value of the flag gt1_flag can be encoded with 1.

[0246] When the absolute value of the residual coefficient is greater than 1, the flag par_flag indicating whether the absolute value of the residual coefficient is even or odd can be encoded at S1620. When the absolute value of the residual coefficient is even, the flag par_flag can be set to 0 and encoded. On the other hand, when the absolute value of the residual coefficient is odd, the flag par_flag can be set to 1 and encoded. Alternatively, conversely, when the absolute value of the residual coefficient is even, the flag par_flag can be set to 1, and when the absolute value of the residual coefficient is odd, the flag par_flag can be set to 0.

[0247] Next, the flag gt_2_flag indicating whether the absolute value of the residual coefficient is greater than 3 can be encoded at S1630. When the absolute value of the residual coefficient is equal to or less than 3, the value of the flag gt_2_flag can be set to 0. On the other hand, when the absolute value of the residual coefficient is greater than 0, the value of the flag gt_2_flag can be set to 1.

[0248] When the absolute value of the residual coefficient is greater than 3, the rem_level indicating the residual magnitude can be encoded at S1640. The syntax rem_level can be obtained by shifting the value obtained by subtracting 4 from the absolute value of the residual coefficient to the right by 1.

[0249] In addition to Figure 16In addition to the shown flags gt_1_flag and gt_2_flag, it is possible to additionally encode gt_N_flags such as gt_3_flag, gt_4_flag, or gt_5_flag. In this case, when the value of gt_(N - 1)_flag is 1, it is possible to additionally encode gt_N_flag.

[0250] gt_N_flag can indicate whether the absolute value of the residual coefficient has a value greater than (2N - 1). When gt_N_flag is used additionally, rem_level can be obtained by shifting the value obtained by subtracting 2N from the absolute value of the residual coefficient to the right by 1.

[0251] In the above example, it is shown that the absolute value of the residual coefficient is encoded using sig_flag, gt_1_flag, par_flag, gt_2_flag, and rem_level. In another example, the absolute value of the residual coefficient can be encoded as it is. In the example, it is possible to encode the syntax abs_level representing the absolute value of the residual coefficient. The method of selecting the method for encoding the absolute value of the residual coefficient will be described below.

[0252] After encoding the size information of the residual coefficient, it is possible to encode the sign_flag indicating the sign of the residual coefficient S1030. When the value of the sign_flag is 0, it indicates that the residual coefficient is positive. On the other hand, when the value of the sign_flag is 1, it indicates that the residual coefficient is negative.

[0253] Table 1 shows the values assigned to each syntax when the value of the residual coefficient is -21 and 2 gt_N_flags are used.

[0254] [Table 1]

[0255]

[0256]

[0257] In Table 1, coeff represents the value of the residual coefficient, and "formula" represents the formula for obtaining the value of each syntax.

[0258] Table 2 shows the values assigned to each syntax when the value of the residual coefficient is -21 and 5 gt_N_flags are used.

[0259] [Table 2]

[0260] Division Formula Value Residual coefficient (Coeff) coeff -21 sig_flag coeff != 0 1 gt_1_flag !!(|coeff| - 1) 1 par_flag (|coeff| - 2) & 1 1 gt_2_flag |coeff| >= 4 1 gt_3_flag |coeff| >= 6 1 gt_4_flag |coeff| >= 8 1 gt_5_flag |coeff| >= 10 1 rem_level (|coeff| - 10) >> 1 5 sign_flag 1

[0261] Figure 17It is a flowchart showing the process of decoding residual coefficients in a decoder.

[0262] When it is determined that non-zero residual coefficients are included in a sub-block, the residual coefficients can be reconstructed based on a predetermined scan order.

[0263] First, a flag sig_flag indicating whether the residual coefficient has a non-zero value can be decoded S1710. When the value of the flag sig_flag is 0, it indicates that the value of the residual coefficient is 0. On the other hand, when the value of the flag sig_flag is 1, it indicates that the value of the residual coefficient is not 0. When the value of the flag sig_flag is 1, the magnitude information of the residual coefficient can also be decoded S1720.

[0264] Figure 18 It is a diagram showing the process of decoding the magnitude information of the residual coefficient.

[0265] For ease of description, it is assumed that the residual coefficients are encoded using up to 2 gt_N_flag.

[0266] A flag gt1_flag indicating whether the absolute value of the residual coefficient is greater than 1 can be decoded S1810. When the value of the flag gt_1_flag is 0, it indicates that the absolute value of the residual coefficient is 1. On the other hand, when the value of the flag gt_1_flag is 1, it indicates that the absolute value of the residual coefficient is greater than 1.

[0267] When the value of the flag gt_1_flag is 1, a flag par_flag indicating whether the absolute value of the residual coefficient is even or odd can be decoded S1820. When the value of the flag par_flag is 0, it indicates that the absolute value of the residual coefficient is even, and when the value of the flag par_flag is 1, it indicates that the absolute value of the residual coefficient is odd.

[0268] Next, a flag gt_2_flag indicating whether the absolute value of the residual coefficient is greater than 3 can be decoded S1830. When the value of the flag gt_2_flag is 0, it indicates that the absolute value of the residual coefficient is less than 3. When the value of the flag gt_2_flag is 0, the absolute value of the residual coefficient can be determined to be 2 or 3 according to the value of the flag par_flag.

[0269] When the value of the flag gt_2_flag is 1, it indicates that the absolute value of the residual coefficient is greater than 3.

[0270] When the value of the flag gt_2_flag is 1, a rem_level indicating the residual magnitude can be decoded S1840. The absolute value of the residual coefficient can be obtained by adding 3 or 4 to the value obtained by shifting the value representing the syntax rem_level to the left by 1.

[0271] In addition to Figure 18 the flags gt_1_flag and gt_2_flag shown, the gt_N_flag such as gt_3_flag, gt_4_flag or gt_5_flag can be additionally decoded. In this case, when the value of gt_(N - 1)_flag is 1, the gt_N_flag can be additionally decoded.

[0272] The gt_N_flag can indicate whether the absolute value of the residual coefficient has a value greater than (2N - 1). When the gt_N_flag is additionally used, the rem_level can be set to the value obtained by shifting the value obtained by subtracting 2N from the absolute value of the residual coefficient to the right by 1.

[0273] In the above example, it is shown that the absolute value of the residual coefficient is decoded by using sig_flag, gt_1_flag, par_flag, gt_2_flag and rem_level. In another example, the absolute value of the residual coefficient can be decoded as it is. In the example, the syntax abs_level representing the absolute value of the residual coefficient can be decoded. The method of selecting the method for decoding the absolute value of the residual coefficient will be described below.

[0274] After decoding the magnitude information of the residual coefficient, the sign_flag representing the sign of the residual coefficient can be decoded S1230. When the value of the sign_flag is 0, it indicates that the residual coefficient is positive. On the other hand, when the value of the sign_flag is 1, it indicates that the residual coefficient is negative.

[0275] Table 3 shows an example of decoding a residual coefficient with a value of -21 by using 2 gt_N_flags.

[0276] [Table 3]

[0277]

[0278] In Table 3, the variable tmp_coeff represents the temporary reconstruction coefficient. When the value of gt_2_flag is 0, the temporary reconstruction coefficient tmp_coeff can be set to the absolute value of the residual coefficient. On the other hand, when the value of gt_2_flag is 1, the absolute value of the residual coefficient can be obtained by updating the temporary reconstruction coefficient tmp_coeff based on the syntax rem_level.

[0279] Table 4 shows an example of decoding a residual coefficient with a value of -21 by using 5 gt_N_flags.

[0280] [Table 4]

[0281]

[0282]

[0283] In Table 3, the variable tmp_coeff represents the temporary reconstruction coefficient. When gt_N_flag is 0, the temporary reconstruction coefficient tmp_coeff can be set to the absolute value of the residual coefficient. On the other hand, when gt_N_flag is 1, the temporary reconstruction coefficient can be updated (e.g., tmp_coeff += sig_N_flag << 1), and the next syntax can be parsed.

[0284] As described above, the residual coefficient can be encoded by at least one syntax. The residual coefficient can be changed into a plurality of binary bits during the binarization process of the (one or more) syntax, and the changed binary bits can be encoded by entropy coding.

[0285] Entropy coding can be divided into coding using context information and coding without using context information. Context represents the probability that the value of a binary bit is 0 or 1.

[0286] A threshold can be set to limit the number of binary bits encoded using context information. For the binary bits in the generated binary bits whose count value is less than the threshold, coding using context information is performed. When the count value is equal to or greater than the threshold, coding using context information can no longer be utilized.

[0287] The threshold can be determined based on the number of non-zero residual coefficients in the current block. In an example, the number of non-zero residual coefficients in the current block multiplied by the value of a real number or added to or subtracted by an offset value can be set as the threshold.

[0288] Alternatively, the threshold can be determined based on the number of pixels included in the current block. In an example, the number of pixels in the current block multiplied by the value of a real number or added to or subtracted by an offset value can be set as the threshold.

[0289] Alternatively, the information representing the threshold can be signaled in the bitstream. This information can be encoded by a higher-level header such as a sequence, picture header, or slice header.

[0290] Alternatively, the threshold can be determined based on at least one of the size or shape of the current block.

[0291] Alternatively, the threshold can be determined based on at least one of the following: whether transform skip is applied, the transform kernel applied to the current block, or the quantization parameter.

[0292] When counting the number of binary bits encoded using context information, the counter can be set to no operation when encoding the information indicating the position of the last non-zero residual coefficient. In other words, this information can be excluded from the count.

[0293] Alternatively, the counter can be set to no operation when encoding the flag indicating whether there is a non-zero residual coefficient in each sub-block in the current block. In other words, this flag can be excluded from the count.

[0294] According to an embodiment of the present disclosure, in order to limit the number of binary bits encoded using context information, when the number of binary bits encoded using context information is equal to or greater than a threshold, the absolute value of the residual coefficient can be encoded as it is, instead of encoding the residual coefficient sequentially using such as gt_N_flag. In an example, when the number of binary bits encoded using context information is less than the threshold, the absolute value of the residual coefficient can be encoded using at least one of sig_flag, sign_flag, gt_1_flag, par_flag, gt_2_flag, gt_3_flag, gt_4_flag, gt_5_flag, or rem_level shown in Tables 1 to 4. On the other hand, when the number of binary bits encoded using context information is equal to or greater than the threshold, the syntax abs_level representing the absolute value of the residual coefficient can be encoded.

[0295] Whenever the binary bits encoded using context information are decoded, the decoder can also operate the counter. When the value of the counter is less than the threshold, the absolute value of the residual coefficient can be reconstructed using at least one of sig_flag, sign_flag, gt_1_flag, par_flag, gt_2_flag, gt_3_flag, gt_4_flag, gt_5_flag, or rem_level. On the other hand, when the value of the counter is equal to or greater than the threshold, the absolute value of the residual coefficient can be reconstructed using the syntax abs_level.

[0296] Figure 19 It is a diagram showing an example of counting the number of binary bits using context information.

[0297] For ease of description, it is assumed that there are 16 residual coefficients in the sub-block, and it is assumed that each of the coefficients is C0 to C15. Here, C15 means the residual coefficient at the lower right position in the sub-block, and C0 means the residual coefficient at the upper left position in the sub-block.

[0298] In addition, the residual coefficients are generated by transformation, and thus, it is assumed that the scanning order is determined in the order of C15 to C0.

[0299] In addition, it is assumed that the maximum number of binary bits encoded by using context information is 36, and it is assumed that a flag indicating whether there is a non-zero residual coefficient in a sub-block and information indicating the position of the last non-zero residual coefficient are excluded from the count.

[0300] In Figure 19 one pass represents the syntax encoded by using context information. Two passes 2-1, two passes 2-2, and three passes other than one pass represent the syntax encoded without using context information.

[0301] A pass represents the encoding and decoding order. In the example, in the decoder, the syntax belonging to two passes 2-1 can be decoded after decoding all the syntax belonging to one pass. In addition, the syntax belonging to three passes can be decoded after decoding all the syntax belonging to two passes 2-1.

[0302] In the shown example, two passes 2-2 represent the alternate path of one pass, two passes 2-1, and three passes.

[0303] When the coefficient of the binary bits encoded by using context information is less than the threshold, the absolute value of the residual coefficient can be encoded by one pass and two passes 2-1. On the other hand, when the coefficient of the binary bits encoded by using context information is equal to or greater than the threshold, the absolute value of the residual coefficient can be encoded by two passes 2-2.

[0304] In the example, as in the example shown in Table 2, when the first residual coefficient C15 is -21, the flags sig_flag, gt_1_flag, par_flag, gt_2_flag, and rem_level can be encoded. When encoding the first residual coefficient C15, when a total of 4 binary bits are used to encode the syntax encoded by using context information (i.e., sig_flag, gt_1_flag, par_flag, gt_2_flag), the counter is incremented to 4.

[0305] Since the counter value is less than the threshold 36 after encoding the first residual coefficient C15, the syntax encoded by using context information can also be used for the second residual coefficient C14. When it is assumed that 4 syntaxes encoded by using context information are used for each of C15 to C7, after encoding the absolute value of the residual coefficient C7, the counter value is set to 36 equal to the threshold.

[0306] Therefore, when encoding the next residual coefficient C6, the absolute value of the residual coefficient C6 can be encoded as it is through the syntax abs_level without using the syntax encoded by using context information. In other words, for the residual coefficients C6 to C0, the absolute values of the residual coefficients can be encoded by using the syntax abs_level belonging to the 2-2 round instead of the four syntaxes (i.e., sig_flag, gt_1_flag, par_flag, gt_2_flag) belonging to the 1st round and the syntax rem_level belonging to the 2-1 round.

[0307] In this case, although the number of bits encoded by using context information is less than the threshold, when the difference between the number and the threshold is less than the maximum number of syntaxes encoded by using context information, abs_level can be set to be encoded without using context information. For example, when sig_flag, gt_1_flag, par_flag, and gt_2_flag are set to be encoded by using context information, the syntax can be encoded only when the difference between the number and the threshold is greater than 4. On the other hand, when the difference is less than 4, abs_level can be encoded.

[0308] In the shown example, it is shown that only the four syntaxes belonging to the 1st round are encoded by using context information. Different from the said example, at least one of rem_level (the syntax belonging to the 2-1 round) or sign_flag (the syntax belonging to the 3rd round) can be encoded by using context information. In the example, when rem_level is encoded by using context information, the counter can be increased to the same number as the number of bits assigned to the syntax rem_level.

[0309] In Figure 19 the shown example, the flag par_flag can be set to be not encoded by using context information. Figure 20 Indicates an example thereof.

[0310] In Figure 20 it, the 1st round represents the syntax encoded by using context information. The 2nd round, 3-1st round, 3-2nd round, and 4th round other than the 1st round represent the syntax encoded without using context information. When the coefficient of the bits encoded by using context information is less than the threshold, the absolute values of the residual coefficients can be encoded by the 1st round, 2nd round, and 3-1st round. On the other hand, when the coefficient of the bits encoded by using context information is equal to or greater than the threshold, the absolute values of the residual coefficients can be encoded by the 3-2nd round.

[0311] When the flag par_flag is set to be encoded without using context information, the counter can be set so that it does not increase for the number of binary bits (i.e., 1) assigned to the flag par_flag.

[0312] Therefore, for each residual coefficient, the counter is incremented only for the binary bits assigned to the three syntaxes sig_flag, gt_1_flag, and gt_2_flag.

[0313] When it is assumed that the syntaxes sig_flag, gt_1_flag, and gt_2_flag are encoded for each of the residual coefficients C15 to C4, after encoding the syntax for the residual coefficient C4, the counter is set to 36, which is equal to the threshold.

[0314] Therefore, when encoding the residual coefficient C3, the absolute value of the residual coefficient C3 can be encoded as it is by the syntax abs_level included in the 3 - 2 rounds. In other words, for the residual coefficients C3 to C0, the absolute values of the residual coefficients can be encoded by using the syntax abs_level belonging to the 3 - 2 rounds instead of the syntax belonging to the 1st round and the 2 - 1 rounds.

[0315] Priorities can be set among the syntaxes encoded by using context information. In this case, after counting the number of binary bits assigned to the syntax with a high priority, the number of binary bits assigned to the syntax with a low priority can be counted.

[0316] Figure 21 An example showing that the priorities among the syntaxes encoded by using context information are different.

[0317] The transform is skipped in the current block, and therefore, it is assumed that the scan order is determined in the order of C0 to C15.

[0318] In Figure 21 the example, the syntaxes belonging to the 1st round and the 2nd round can be encoded by using context information. In this case, when the syntax belonging to the 1st round has a higher priority than the syntax belonging to the 2nd round, when counting the number of binary bits encoded by using context information, the number of binary bits assigned to the syntax belonging to the 2nd round can be counted after counting the number of binary bits assigned to the syntax belonging to the 1st round.

[0319] In the example, when it is assumed that 16 residual coefficients are encoded and the threshold is 96, after encoding the syntax belonging to the 1st round for all 16 residual coefficients, the value of the counter is set to 64. Since the value of the counter is less than the threshold, the syntax belonging to the 2nd round can also be encoded by using context information.

[0320] In the example shown, after encoding the syntax belonging to 2 rounds for the residual coefficient C7, the value of the counter is set to 96. Therefore, the syntax belonging to 2 rounds can be encoded for the next residual coefficient C8 without using context information.

[0321] In Figure 21 the example shown, the flag par_flag can be set to be encoded without using context information. Figure 22 An example thereof is shown.

[0322] In Figure 22 it, rounds 1 and 3 represent the syntax encoded by using context information. Rounds 2 and 4 represent the syntax encoded without using context information.

[0323] The flag par_flag can be encoded without using context information. Therefore, when encoding the flag par_flag, the counter can be set not to increase. After encoding the syntax belonging to 3 rounds for the residual coefficient C11, the value of the counter is set to 96 which is equal to the threshold. Therefore, starting from the next residual coefficient C12, when encoding the syntax belonging to 3 rounds, context information can be not used.

[0324] In Figure 21 and Figure 22 it, the distribution of gt_N_flag is shown to be scattered in different rounds. In the example, in Figure 21 the example shown, it is shown that when gt_1_flag belongs to round 1, gt_2_flag belongs to round 2.

[0325] The distribution of the syntax can also be set to be different from Figure 21 and Figure 22 . In the example, all gt_N_flag can be assigned to a single round or gt_1_pass and gt_2_pass can be assigned to a single round.

[0326] Instead of setting the flag par_flag to a separate round, the par_flag can be set to the same round as gt_1_flag or gt_2_flag, but it can be set not to use context information when encoding the flag par_flag. Figure 23 An example thereof is shown. In Figure 23 the example shown, it is shown that the flag par_flag is assigned to the same round as gt_2_flag, gt_3_flag, gt_4_flag and gt_5_flag.

[0327] The flag par_flag can be assigned to a round lower than gt_N_flag. In the example, inFigure 22 In the example shown, the two rounds including par_flag can be changed to three rounds, and the existing three rounds can be changed to two rounds. In this case, the syntax that does not use context information can be encoded after first encoding the syntax using context information.

[0328] par_flag and rem_level can be combined into three rounds and encoded.

[0329] As in the above example, the absolute value of the residual coefficient can be encoded by using at least one of sig_flag, par_flag, gt_N_flag, or rem_level. In this case, the residual syntax other than rem_level can be encoded by referring to various context information according to the characteristics of the surrounding coefficients. In the example, the flag sig_flag indicating whether the residual coefficient is 0 can be encoded by referring to various context information according to the characteristics of the surrounding residual coefficients. In this case, the number of context information that can be referred to can be determined according to the position of the pixel.

[0330] Figure 24 and Figure 25 represents the surrounding reconstruction region that is referred to determine the context information.

[0331] Figure 24 is an example of the case of encoding the residual coefficients in the scanning order from the lower right residual coefficient to the upper left residual coefficient. In the example, Figure 24 can be applied to the case where the transform is not skipped in the current block.

[0332] Figure 25 is an example of the case of encoding the residual coefficients in the scanning order from the upper left residual coefficient to the lower right residual coefficient. In the example, Figure 25 can be applied to the case where the transform is skipped in the current block.

[0333] Refer to Figure 24 and Figure 25 In the example, up to 2 or up to 5 reconstructed coefficients can be referred to. In the example, when the position of the residual coefficient is (x, y), the region including the reconstructed coefficients with the absolute value of the sum of the x coordinate difference and the y coordinate difference of the residual coefficient equal to or less than 1 or the region including the reconstructed coefficients with the absolute value equal to or less than 2 can be set as the surrounding reconstruction region.

[0334] Alternatively, when there are reconstructed coefficients that have not been reconstructed in the scanning order or are outside the block boundary among the reconstructed coefficients in the reconstruction region with the residual coefficient, the unavailable reconstructed coefficients can be excluded from the reference.

[0335] Alternatively, for an unavailable reconstruction coefficient, context information can be selected by treating the information at the corresponding position as a default value. In the example, as shown in (a) of Figure 25 in the example shown in (a) of Figure 25 , when a reconstruction region is set and the current residual coefficient to be encoded is included in the leftmost column in the current block, the sig_flag value of the left reconstruction coefficient of the current residual coefficient can be inferred as 0 or 1.

[0336] Alternatively, when a reconstruction coefficient outside the block boundary is included in a sub-block different from the current residual coefficient but is included in the same coding block, the corresponding reconstruction coefficient can be set as available.

[0337] Context information can be determined by referring to more or fewer reconstruction coefficients than those in the example shown. In the example, Figure 24 not shown in Figure 24 , but context information can be determined using only the reconstruction coefficients at the right position and the lower position of the residual coefficient.

[0338] Alternatively, after assigning an index to each of multiple reconstruction region candidates, the index specifying one of them can be encoded and sent to the decoder. Alternatively, the reconstruction region can be adaptively determined according to the size or shape of the current block. Alternatively, the reconstruction region can be determined based on the quantization state information QState. In the example, when the variable QState is 0 or 1, a reconstruction region including up to 2 reconstruction coefficients can be used. Alternatively, when QState is 2 or 3, a reconstruction region including up to 5 reconstruction coefficients can be used.

[0339] Alternatively, instead of setting the surrounding reconstruction region, it can be set to one of N fixed context information. In the example, N can be 1. Alternatively, the value of N can be determined according to the position of the residual coefficient. In the example, when the sum of x and y is less than the threshold, N can be set to 1, and when the sum of x and y is equal to or greater than the threshold, N can be set to 2. The threshold can be sent to the encoder through the higher layer header. Alternatively, the threshold can be pre-agreed upon in the encoder and the decoder.

[0340] When encoding / decoding the sig_flag, the values of the sig_flag of the reconstruction coefficients included in the reconstruction region around the residual coefficient can be summed up.

[0341] Alternatively, the absolute value of the reconstruction coefficient or partial reconstruction coefficient included in the reconstruction region around the residual coefficient can be calculated. Here, the absolute value of the partial reconstruction coefficient can mean a temporary reconstruction coefficient based on the syntax included in 1 round such as (sig_flag + gt_1_flag + par_flag + (gt_2_flag << 1)).

[0342] One of multiple context information items can be specified by using the derived value.

[0343] Figure 26 Shows the number of context information items that can be referred to when encoding the flag sig_flag.

[0344] Figure 26 (a) of is an example of a luminance component, and Figure 26 (b) of is an example of a chrominance component.

[0345] The current block can be divided into multiple regions, and the type of context information that can be referred to can be set differently for each region. In the example, in each of the first region where the sum of the x coordinate and the y coordinate is less than 2, the second region where the sum of the x coordinate and the y coordinate is equal to or greater than 2 and less than 5, and the third region where the sum of the x coordinate and the y coordinate is equal to or greater than 5, the type of context information that can be referred to can be different.

[0346] In addition, the number of context information items that can be referred to in each region can be different. In the example, the number of context information items that can be referred to in each region can be fixed.

[0347] Alternatively, the number of context information items that can be referred to in each region can be different. In the example, in each of the first region where the sum of the x coordinate and the y coordinate is less than 2, the second region where the sum of the x coordinate and the y coordinate is equal to or greater than 2 and less than 5, and the third region where the sum of the x coordinate and the y coordinate is equal to or greater than 5, the number of context information items that can be referred to can be different.

[0348] In another example, the type of context information that can be referred to can be set differently according to quantization state information. The variable QState representing quantization state information can have values from 0 to 3. In the example, when the variable QState is 0 and 1, type 1 context information can be referred to, when QState is 2, type 2 context information can be referred to, and when QState is 3, type 3 context information can be referred to.

[0349] In Figure 26 In the example shown in (a) of, it is shown that a luminance block is divided into 3 regions, and the number of context information items that can be referred to in each region is 4. When assuming 3 types of context information are available according to quantization state information, a total of 36 (3×4×3) context information items in the luminance block can be set to be referable.

[0350] In Figure 26In the example shown in (b), the chrominance block is divided into two regions, and the number of context information that can be referred to in each region is 4. When assuming that three types of context information are available according to the quantization status information, a total of 24 (2 × 4 × 3) context information in the chrominance block can be set to be referable.

[0351] According to the size of the reconstruction region (i.e., the number of reconstruction coefficients included in the reconstruction region), the number of context information that can be referred to can be set differently.

[0352] Alternatively, the number of context information that can be referred to can be different according to whether transform skip is applied to the current block. In the example, 3 or 5 context information can be referred to when transform skip is applied to the current block, while 4 context information can be referred to when transform is applied to the current block.

[0353] When using the sum of the sig_flag values of the surrounding reconstruction coefficients, the obtained sum can be compared with a threshold. Here, the threshold can be set to be the same as the number of context information that can be referred to. According to the number of reconstruction coefficients included in the reconstruction region, the sum can be in the range of 0 to 2 or 0 to 5. When the sum is greater than the threshold, the sum can be converted to the threshold. In the example, when the sum is 5 and the threshold is 4, the sum can be changed to 4. Subsequently, one of the multiple context information can be specified based on the sum. In other words, the sum can be used as an index to specify one of the multiple context information.

[0354] In the case of using the sum of the absolute values of the surrounding reconstruction coefficients, after calculating the sum of the absolute values of the reconstruction coefficients in the surrounding reconstruction region, the obtained value can be divided by a predetermined value. For example, the predetermined value can be a natural number such as 2, 3, 4, or 5. Alternatively, the sum of the absolute values can be divided by the number of reconstruction coefficients in the surrounding reconstruction region.

[0355] The result value obtained by the division operation can be compared with a threshold. In this case, when the result value is greater than the threshold, the obtained value can be converted to the threshold. In the example, when the threshold is 3 and the result value is greater than 3, the result value can be converted to 3. Therefore, the result value can be set to a value from 0 to 3.

[0356] According to the result value, the context information that should be referred to when encoding / decoding the corresponding residual coefficient can be specified. In other words, the result value can be used as an index to specify one of the multiple context information. Therefore, the threshold can be determined based on the number of context information that can be referred to in the region including the residual coefficient.

[0357] The residual coefficients can be encoded / decoded by using simplified context information. In an example, when the variable QState is 0 or 1, the number of context information that can be referred to can be 4. On the other hand, when the variable QState is 2 or 3, the number of context information that can be referred to can be 2.

[0358] Alternatively, the number of references that can be set may be different for each region. In an example, when the variable QState is 2 or 3, the number of context information that can be referred to is set to 4 in the third region, while the number of context information that can be referred to is set to 2 in the first and second regions.

[0359] When the number of context information that can be referred to in encoding / decoding the residual coefficients is 2, the threshold can be set to 1. In this case, the result value has a value of 0 or 1, so one of the 2 context information can be specified by the result value.

[0360] Alternatively, when the number of context information that can be referred to is 2, the context information can be specified based on whether the value obtained by modulo operation with 2 and the sum of the absolute values of the reconstructed coefficients in the surrounding reconstructed region is 0.

[0361] Even when gt_N_flag or par_flag is encoded / decoded, the context information can be determined by referring to the surrounding reconstructed region.

[0362] Figure 27 Shows the number of context information that can be referred to when encoding gt_N_flag or par_flag.

[0363] Figure 27 (a) of shows a luminance block, and Figure 27 (b) of shows a chrominance block.

[0364] For a luminance block, the block can be divided into multiple regions. In Figure 27 the example shown in (a) of, it is shown that the luminance block is divided into: a first region where the luminance block includes the residual coefficient at the (0,0) position; a second region where the sum of the x-axis coordinate and the y-axis coordinate is equal to or greater than 1 and less than 3; a third region where the sum of the x-axis coordinate and the y-axis coordinate is equal to or greater than 3 and less than 10; and a fourth region other than the first region to the third region.

[0365] In this case, the last non-zero residual coefficient can be set as the fifth region. Since the last non-zero residual coefficient is encoded / decoded first in the scan order, for the last non-zero residual coefficient, only 1 context information can be set as the reference.

[0366] The chrominance block may have fewer partition regions than the number of luminance blocks. In the example, in Figure 27 (b) of Figure 27 , it is shown that the chrominance block is partitioned into a first region including the residual coefficient (0,0) and a second region other than the first region.

[0367] In this case, it may be set as a third region including the last non-zero residual coefficient.

[0368] In Figure 27 the example shown, it is shown that the number of context information that can be referred to in each region other than the region including the last non-zero residual coefficient is 5. In this case, the types of context information that can be referred to in each region may be different. Therefore, the total number of context information in the luminance block may be 21 (4×5 + 1), and the total number of context information in the chrominance block may be 11 (2×5 + 1).

[0369] As described above, the number and / or type of context information that can be referred to may be set differently according to the region or quantization status information.

[0370] When encoding / decoding the gt_N_flag or par_flag, at least one of the sum of the absolute values of the residual coefficients included in the surrounding reconstructed region and the sum of the sig_flag can be obtained. Subsequently, the result value obtained by subtracting the sum of the sig_flag from the sum of the absolute values can be compared with a threshold. In this case, when the result value is greater than the threshold, the result value can be converted into the threshold. In the example, when the threshold is 4, the result value greater than 4 can be converted into 4. Therefore, the result value is set to a value from 0 to 4. Based on the result value, one of the 5 context information can be specified. In other words, the result value can be used as an index for specifying one of the multiple context information.

[0371] The context information that should be referred to can be determined by a simplified method. In the example, when encoding the residual coefficients in each region, only one predefined context information can be referred to, and the process of obtaining the sum of the absolute values of the reconstructed coefficients in the surrounding reconstructed region can be omitted. The simplified method for determining the context information can be applied to both the luminance block and the chrominance block, or can be applied to only one of the luminance block and the chrominance block.

[0372] In the example, although the method of using one of the multiple context information specified based on the result value in the luminance block can be used, the method of using the predefined context information can be used in the chrominance block.

[0373] The method of dividing the region and the number of regions are not limited to the examples shown. In the examples, at least one of the division method or the number can be determined by considering at least one of the size or shape of the current block, whether a transform skip is applied, or the position of the last non-zero residual coefficient. Alternatively, information specifying at least one of the region division method or the number of regions in the current block can be signaled by encoding via a higher-level header.

[0374] Depending on the size of the reconstructed region (i.e., the number of reconstruction coefficients), the number of context information that can be referred to can be set differently.

[0375] In the examples, when determining context information by using 5 reconstruction coefficients, up to one of 5 context information can be specified in encoding gt_N_flag or par_flag. On the other hand, when determining context information by using 2 reconstruction coefficients, up to one of 3 context information can be specified. When the number of context information that can be referred to decreases, the threshold can decrease together.

[0376] In addition, the size of the reconstructed region can be set differently for each region, or the size of the reconstructed region can be set differently for each color component.

[0377] Alternatively, context information can be obtained by comparing the information of each reconstruction coefficient instead of summing the information of each reconstruction coefficient in the surrounding reconstructed region. In the examples, when encoding / decoding par_flag, assume that the reconstructed region is set as shown in (a) of Figure 27 In this case, for the left and upper reconstruction coefficients of the residual coefficient, context information to be referred to when encoding / decoding par_flag for the current residual coefficient can be determined by referring to the par_flag of each reconstruction coefficient.

[0378] In the examples, the case where all par_flags are not encoded for the left and upper reconstruction coefficients (i.e., the case where at least one of sig_flag or gt_1_flag is 0 for each of the left and upper reconstruction coefficients) or the case where the values of the par_flags of the two reconstruction coefficients are different can be defined as the first case. The case where all values of the par_flags of the two reconstruction coefficients are 1 can be defined as the second case, and the case where all values of the par_flags of the two reconstruction coefficients are 0 can be defined as the third case. When encoding the par_flag of the current residual coefficient, it can be set to refer to different context information for each case. In other words, after assigning an index (from 0 to 2) to each case, one context information can be specified based on the index.

[0379] The syntax used in the above-described embodiments is named only for convenience of description.

[0380] When the embodiments described based on decoding processing or encoding processing are applied to encoding processing or decoding processing, they are included within the scope of the present disclosure. When the embodiments described in a predetermined order are changed in a different order, they are also included within the scope of the present disclosure.

[0381] The above-described embodiments are described based on a series of stages or flowcharts, but they do not limit the time series order of the present disclosure, and if necessary, they may be executed simultaneously or in a different order. Additionally, each component (e.g., unit, module, etc.) configuring the block diagrams in the above-described embodiments can be implemented as a hardware device or software, and multiple components can be combined and implemented as one hardware device or software. The above-described embodiments can be recorded in a computer-readable recording medium implemented in the form of program instructions executable by various computer components. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. Hardware devices specifically configured to store and execute program instructions such as magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs, DVDs, magneto-optical media such as magneto-optical floppy disks, and hardware devices such as ROMs, RAMs, flash memories, etc. are included in the computer-readable recording medium. The hardware device can be configured to operate as one or more software modules to execute the processing according to the present disclosure, and the software device can be configured to operate as one or more hardware modules to execute the processing according to the present disclosure.

[0382] Industrial Applicability

[0383] The present disclosure can be applied to an electronic device capable of encoding / decoding an image.

Claims

1. A video decoding method, comprising: Determining whether to skip inverse transformation for a current block; When skipping the inverse transformation for the current block, obtaining residual coefficients of the current block based on a predefined scan order; Performing inverse quantization on the current block; And Obtaining residual samples of the current block based on the inverse quantized residual coefficients of the current block, wherein, based on the number of context-coded binary bits decoded so far and a threshold, determining whether to decode at least one context-coded syntax used to obtain the residual coefficients of the current block, wherein, when the number of context-coded binary bits decoded so far is less than the threshold, decoding the at least one context-coded syntax from a bitstream to obtain the residual coefficients of the current block, and wherein the at least one context-coded syntax includes at least one of the following: a valid flag indicating whether the residual coefficient is non-zero, a greater than 1 flag indicating whether the absolute value of the residual coefficient is greater than 1, a sign flag indicating whether the residual coefficient has a positive or negative value, and a parity flag indicating whether the absolute value of the residual coefficient is odd or even.

2. The method according to claim 1, wherein When the number of context-coded binary bits decoded so far is equal to or greater than the threshold, obtaining the residual coefficients of the current block by decoding the remaining level information representing the absolute value of the residual coefficient from the bitstream without decoding the at least one context-coded syntax.

3. The method according to claim 1, wherein, Only when the valid flag indicates that the residual coefficient is non-zero, decoding the greater than 1 flag from the bitstream.

4. The method according to claim 1, wherein, The threshold is determined based on the size of the current block.

5. A video encoding method, comprising: Determining whether to skip transformation for a current block; Quantizing the transform coefficients obtained by skipping transformation for the current block; And Encoding the residual coefficients generated by the quantization, wherein, when skipping the transformation for the current block, encoding the residual coefficients based on a predefined scan order, wherein, based on the number of context-coded binary bits encoded so far and a threshold, determining whether to encode at least one context-coded syntax for the residual coefficients of the current block, wherein, when the number of context-coded binary bits encoded so far is less than the threshold, encoding the at least one context-coded syntax into the bitstream for the residual coefficients of the current block, and wherein the at least one context-coded syntax includes at least one of the following: a valid flag indicating whether the residual coefficient is non-zero, a greater than 1 flag indicating whether the absolute value of the residual coefficient is greater than 1, a sign flag indicating whether the residual coefficient has a positive or negative value, and a parity flag indicating whether the absolute value of the residual coefficient is odd or even.

6. The method according to claim 5, wherein, When the number of context-coded binary bits encoded so far is equal to or greater than the threshold, encoding the remaining level information representing the absolute value of the residual coefficient for the residual coefficients of the current block instead of the at least one context-coded syntax.

7. The method according to claim 5, wherein The greater than 1 flag is coded into the bitstream only when the valid flag indicating that the residual coefficient is non-zero is coded.

8. The method according to claim 5, wherein, The threshold is determined based on the size of the current block.

9. An apparatus for transmitting compressed video data, comprising: a processor configured to obtain the compressed video data, and a transmission unit configured to transmit the compressed video data, wherein obtaining the compressed video data includes: determining whether to skip transformation for a current block; quantizing the transform coefficients obtained by skipping transformation for the current block; and coding the residual coefficients resulting from the quantization, wherein when the transformation is skipped for the current block, the residual coefficients are coded based on a predefined scan order, wherein it is determined whether to code at least one context-coded syntax for the residual coefficients of the current block based on the number of context-coded bits coded so far and a threshold, wherein when the number of context-coded bits coded so far is less than the threshold, the at least one context-coded syntax is coded into the bitstream for the residual coefficients of the current block, and wherein the at least one context-coded syntax includes at least one of the following: a valid flag indicating whether the residual coefficient is non-zero, a greater than 1 flag indicating whether the absolute value of the residual coefficient is greater than 1, a sign flag indicating whether the residual coefficient has a positive or negative value, and a parity flag indicating whether the absolute value of the residual coefficient is odd or even.

Images