Video signal encoding / decoding method and device therefor

Abstract

To provide a method for encoding / decoding residual coefficients when encoding / decoding a video signal, and an apparatus for the method.SOLUTION: An image decoding method comprises: generating a merge candidate list of a current block; specifying any one of a plurality of merge candidates included in the merge candidate list; deriving a first affine seed vector and a second affine seed vector of the current block on the basis of a first affine seed vector and a second affine seed vector of the specified merge candidate; by using the first affine seed vector and the second affine seed vector of the current block, deriving an affine vector for a subblock in the current block; and performing motion compensation prediction for the subblock on the basis of the affine vector.SELECTED DRAWING: Figure 12

Description

[Technical field]

[0001] The present invention relates to a video signal encoding / decoding method and an apparatus therefor. [Background technology]

[0002] With the trend of display panels becoming larger and larger, higher quality video services are required. The biggest problem with high-definition video services is the significant increase in data volume. To solve this problem, research is being actively conducted to improve video compression rates. As a representative example, in 2009, MPEG (Motion Picture Experts Group) and VCEG (Video Coding Experts Group) under ITU-T (International Telecommunication Union-Telecommunication) established JCT-VC (Joint Collaborative Team on Video Coding). JCT-VC proposed HEVC (High Efficiency Video Coding), a video compression standard with approximately twice the compression performance compared to H.264 / AVC, and it was approved on January 25, 2013. With the rapid development of high-definition video services, the performance of HEVC gradually revealed its limitations. Summary of the Invention [Problem to be solved by the invention]

[0003] The present invention aims to provide a method for encoding / decoding residual coefficients when encoding / decoding a video signal and an apparatus for the method.

[0004] The present invention aims to provide a method and an apparatus for encoding / decoding residual coefficients using a flag that compares the magnitude of the residual coefficients with a threshold when encoding / decoding a video signal.

[0005] The present invention aims to provide a method for encoding / decoding residual coefficients using a flag indicating whether the residual coefficients are even or odd when encoding / decoding a video signal, and an apparatus for the method.

[0006] The technical problems to be solved by the present invention are not limited to the above technical problems, and a person skilled in the art to which the present invention belongs may clearly understand other technical problems not mentioned in the present invention through the following description. [Means for solving the problem]

[0007] The video signal decoding method of the present invention includes: parsing a non-zero flag from a bitstream indicating whether a residual coefficient is non-zero; if the non-zero flag indicates that the residual coefficient is not non-zero, parsing absolute value information from the bitstream, where the absolute value information is used to determine an absolute value of the residual coefficient; and determining the absolute value of the residual coefficient based on the absolute value information, where the absolute value information may include a residual coefficient comparison flag indicating whether the residual coefficient exceeds a first value, and further parsing a parity flag from the bitstream only if the residual coefficient exceeds the first value.

[0008] The video signal encoding method of the present invention may include: encoding a non-zero flag indicating whether a residual coefficient is non-zero; and if the residual coefficient is not non-zero, encoding absolute value information, where the absolute value information is used to determine the absolute value of the residual coefficient. The absolute value information may include a residual coefficient comparison flag indicating whether the residual coefficient exceeds a first value, and further encoding a parity flag of the residual coefficient only if the residual coefficient exceeds the first value.

[0009] In the video signal decoding / encoding method of the present invention, the parity flag may indicate whether the value of the residual coefficient is even or odd.

[0010] In the video signal decoding / encoding method of the present invention, if the residual coefficient exceeds the first value, a first adjusted residual coefficient comparison flag can be further analyzed, and the first adjusted residual coefficient comparison flag indicates whether an adjusted residual coefficient derived by shifting the residual coefficient one bit to the right exceeds a second value.

[0011] In the video signal decoding / encoding method of the present invention, when the adjusted residual coefficient is equal to or less than a second value, it can be determined that the residual coefficient is 2N or 2N+1 according to a value of the parity flag.

[0012] In the video signal decoding / encoding method of the present invention, if the adjusted residual coefficient exceeds the second value, a second adjusted residual coefficient comparison flag can be further analyzed, and the second adjusted residual coefficient comparison flag indicates whether the adjusted residual coefficient exceeds a third value.

[0013] In the video signal decoding / encoding method of the present invention, if the adjusted residual coefficient exceeds the second value, the residual value information can be further analyzed, and the residual value information is a value obtained by subtracting the second value from the adjusted residual coefficient.

[0014] The above briefly summarized features of the present invention are merely exemplary embodiments of the detailed description of the invention that follows, and are not intended to limit the scope of the present invention. Effect of the Invention

[0015] According to the present invention, residual coefficients can be efficiently coded / decoded.

[0016] In accordance with the present invention, the residual coefficients can be efficiently coded / decoded by using a flag that compares the magnitude of the residual coefficient to a threshold.

[0017] According to the present invention, the residual coefficients can be efficiently coded / decoded by using a flag indicating whether the residual coefficient is even or odd.

[0018] The effects obtained by the present invention are not limited to the above effects, and a person skilled in the art to which the present invention belongs can clearly understand other effects not mentioned in the present invention through the following description. [Brief description of the drawings]

[0019] [Figure 1] FIG. 2 is a block diagram of a video encoder according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram of a video decoder according to an embodiment of the present invention. [Diagram 3] FIG. 2 is a diagram showing a basic coding tree unit according to an embodiment of the present invention. [Figure 4] FIG. 1 is a diagram showing various division forms of a coding block. [Diagram 5] FIG. 13 is a diagram illustrating an example of division of a coding tree unit. [Figure 6] FIG. 13 is a diagram showing an example in which a block smaller than a coding tree unit of a preset size appears at an image boundary. [Figure 7] FIG. 13 illustrates an example of performing quadtree division on a boundary block with an irregular boundary. [Figure 8] FIG. 13 illustrates an example of performing quadtree division on a block adjacent to an image boundary. [Figure 9] FIG. 13 is a diagram showing a division pattern of blocks adjacent to an image boundary. [Figure 10] FIG. 1 shows the coding pattern for blocks adjacent to an image border. [Figure 11] 2 is a flowchart of an inter prediction method according to an embodiment of the present invention. [Figure 12] 13 is a flowchart of a process for deriving motion information for a current block in merge mode. [Figure 13] FIG. 1 illustrates candidate blocks for deriving merging candidates. [Figure 14] FIG. 13 illustrates the location of a reference sample. [Figure 15]FIG. 1 illustrates candidate blocks for deriving merging candidates. [Figure 16] FIG. 13 is a diagram illustrating an example of changing the position of a reference sample. [Figure 17] FIG. 13 is a diagram illustrating an example of changing the position of a reference sample. [Figure 18] 13 is a diagram for explaining an example of updating the inter region motion information table. FIG. [Figure 19] 13 is a diagram illustrating an example of updating the inter region merge candidate table. FIG. [Figure 20] 13 is a diagram showing an example in which the indexes of stored inter region merge candidates are updated. FIG. [Figure 21] FIG. 13 is a diagram showing the position of a representative sub-block. [Figure 22] A figure showing an example of generating an inter region motion information table for each inter prediction mode. [Figure 23] 13 is a diagram showing an example of adding an inter region merging candidate included in the long-term motion information table to a merging candidate list. FIG. [Figure 24] FIG. 13 illustrates an example of performing redundancy checks on only a portion of merge candidates. [Diagram 25] FIG. 13 illustrates an example of omitting redundancy checks for a particular merge candidate. [Figure 26] FIG. 13 illustrates an example in which a candidate block included in the same parallel merge region as the current block is set to be unavailable for use as a merge candidate. [Figure 27] FIG. 13 is a diagram showing a temporary movement information table. [Figure 28] A figure showing an example of merging an inter-region motion information table and a temporary motion information table. [Figure 29] 2 is a flowchart of an intra prediction method according to an embodiment of the present invention. [Diagram 30] FIG. 2 is a diagram showing reference samples included in each reference sample line. [Diagram 31] FIG. 13 is a diagram illustrating intra-prediction modes. [Diagram 32]FIG. 13 is a diagram showing an example of a one-dimensional array in which reference samples are arranged in a line. [Diagram 33] FIG. 13 is a diagram showing an example of a one-dimensional array in which reference samples are arranged in a line. [Diagram 34] FIG. 13 illustrates the angle formed between directional intra prediction modes and a line parallel to the x-axis. [Diagram 35] FIG. 13 illustrates an example of obtaining prediction samples when the current block is non-square. [Diagram 36] FIG. 13 is a diagram illustrating a wide-angle intra prediction mode. [Figure 37] FIG. 1 shows the application of PDPC. [Figure 38] FIG. 2 illustrates sub-blocks in which a secondary transformation is performed. [Figure 39] FIG. 2 illustrates sub-blocks in which a secondary transformation is performed. [Diagram 40] FIG. 13 is a diagram for explaining an example of determining a transformation type of a current block. [Diagram 41] 1 is a flowchart of a method for coding residual coefficients. [Diagram 42] 11A to 11C are diagrams illustrating the arrangement order of residual coefficients in different scanning orders. [Diagram 43] 11A to 11C are diagrams illustrating the arrangement order of residual coefficients in different scanning orders. [Diagram 44] FIG. 13 is a diagram illustrating an example of encoding the position of the last non-zero coefficient. [Diagram 45] 13 is a flowchart of a process for encoding absolute values ​​of residual coefficients. [Figure 46] 13 is a flowchart of a process for encoding absolute values ​​of residual coefficients. [Figure 47] 13 is a flowchart of a process for encoding absolute values ​​of residual coefficients. [Figure 48] 1 is a flow chart of a process for determining block strength. [Figure 49] FIG. 2 illustrates predefined filter candidates. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0021] Video encoding and decoding is performed on a block-by-block basis. For example, encoding / decoding operations such as transform, quantization, prediction, in-loop filtering, or reconstruction may be performed on a coding block, a transform block, or a prediction block.

[0022] Hereinafter, the block to be coded / decoded is referred to as a “current block.” For example, according to the current coding / decoding process, the current block can refer to a coding block, a transformation block, or a prediction block.

[0023] Furthermore, the term "unit" as used herein refers to a basic unit for performing a specific encoding / decoding process, and "block" can be understood to refer to a sample array of a given size. Unless otherwise stated, "block" and "unit" can be used with the same meaning. For example, in the embodiments described below, a coding block and a coding unit can be understood to have the same meaning.

[0024] FIG. 1 is a block diagram of a video encoder according to an embodiment of the present invention.

[0025] Referring to FIG. 1, the video encoding device 100 may include an image division unit 110, a prediction unit 120, 125, a transformation unit 130, a quantization unit 135, a reordering unit 160, an entropy encoding unit 165, an inverse quantization unit 140, an inverse transform unit 145, a filter unit 150, and a memory 155.

[0026] Each component shown in Fig. 1 is illustrated independently to show different characteristic functions in the video encoding device, and does not mean that each component is composed of separate hardware or a single software component. In other words, for each component, at least two components can be combined into one component or one component can be divided into multiple components to perform the function for convenience of explanation, and an embodiment in which each component is integrated and an embodiment in which each component is separated are also included in the scope of protection of the present invention as long as they do not deviate from the essence of the present invention.

[0027] In addition, some of the components may not be components necessary for performing the essential functions of the present invention, but may be optional structural elements simply for improving performance. The present invention may include only components necessary for realizing the essence of the present invention other than components simply for improving performance, and a structure including only necessary structural elements other than components simply for improving performance is also included in the scope of protection of the present invention.

[0028] The image division unit 110 can divide the input image into at least one processing unit. In this case, the processing unit may be a prediction unit (PU: Prediction Unit), a transform unit (TU: Transform Unit), or a coding unit (CU: Coding Unit). The image division unit 110 can divide one image into a combination of multiple coding units, prediction units, and transform units, and select one combination of coding units, prediction units, and transform units based on a predetermined standard (such as a cost function) to code the image.

[0029] For example, an image can be divided into multiple coding units. To divide an image into coding units, a recursive tree structure such as a quad tree structure can be used, using one video or largest coding unit as a root, and the coding unit to be divided into other coding units can be divided with subnodes of the number of divided coding units. A coding unit that is not further divided according to a certain restriction becomes a leaf node. That is, assuming that only square division is possible for one coding unit, one coding unit can be divided into a maximum of four other coding units.

[0030] Hereinafter, in the embodiments of the present invention, a coding unit may refer to a unit that performs coding, or may refer to a unit that performs decoding.

[0031] The prediction units within a coding unit may be divided into at least one of squares and rectangles of the same size, or may be divided such that any one of the divided prediction units within a coding unit has a different shape and / or size than another prediction unit.

[0032] Performing intra prediction based on a coding unit If the prediction unit is not the smallest coding unit, intra prediction can be performed without splitting into multiple prediction units NxN.

[0033] The prediction units 120 and 125 may include an inter prediction unit 120 that performs inter prediction and an intra prediction unit 125 that performs intra prediction. It is possible to determine whether to perform inter prediction or intra prediction for a prediction unit, and to determine specific information (e.g., intra prediction mode, motion vector, reference image, etc.) based on each prediction method. In this case, the processing unit that performs the prediction may be different from the processing unit that determines the prediction method and specific content. For example, the prediction method and prediction mode, etc. may be determined by the prediction unit, and the prediction may be performed by the transformation unit. A residual value (residual block) between the generated prediction block and the original block may be input to the transformation unit 130. In addition, prediction mode information, motion vector information, etc. used for prediction may be coded by the entropy coding unit 165 together with the residual value and transmitted to the decoder. When a specific coding mode is used, the original block may be directly coded and transmitted to the decoder without generating a block predicted by the prediction units 120 and 125.

[0034] The inter prediction unit 120 may predict a prediction unit based on information of at least one image preceding or following the current image, and in some cases may also predict a prediction unit based on information of a portion of an encoded region within the current image. The inter prediction unit 120 may include a reference image interpolation unit, a motion prediction unit, and a motion compensation unit.

[0035] The reference image interpolation unit can receive reference image information from the memory 155 and generate pixel information of less than integer pixels from the reference image. For luminance pixels, a DCT-based 8-tap interpolation filter with different filter coefficients can be used to generate pixel information of less than integer pixels in 1 / 4 pixel units. For chroma signals, a DCT-based 4-tap interpolation filter with different filter coefficients can be used to generate pixel information of less than integer pixels in 1 / 8 pixel units.

[0036] The motion prediction unit can perform motion prediction based on the reference image interpolated by the reference image interpolation unit. Various methods for calculating a motion vector are possible, such as a full search-based block matching algorithm (FBMA), a three step search method (TSS), a new three step search algorithm (NTS), etc. The motion vector can have a motion vector value in units of 1 / 2 pixel or 1 / 4 pixel based on the interpolated pixel. The motion prediction unit can predict the current prediction unit by adopting different motion prediction methods. Various methods are available for motion prediction, such as a skip method, a merge method, an advanced motion vector prediction (AMVP) method, an intra block copy method, etc.

[0037] The intra prediction unit 125 can generate a prediction unit based on reference pixel information around the current block, which is pixel information in the current image. If the neighboring block of the current prediction unit is an inter-predicted block and the reference pixel is an inter-predicted pixel, the reference pixel included in the inter-predicted block can be used as the reference pixel information of the neighboring intra-predicted block. That is, if the reference pixel is not available, at least one reference pixel of the available reference pixels can be used to replace the unavailable reference pixel information.

[0038] In intra prediction, the prediction mode may have a directional prediction mode that uses reference pixel information according to the prediction direction, and a non-directional mode that does not use directional information when performing prediction. The mode for predicting luma information and the mode for predicting chroma information may be different, and the intra prediction mode information used to predict luma information or predicted luma signal information may be utilized to predict chroma information.

[0039] When performing intra prediction, if the size of the prediction unit is the same as the size of the transform unit, intra prediction can be performed on the prediction unit based on the pixel located to the left, the pixel located to the top left, and the pixel located on the top side. However, when performing intra prediction, if the size of the prediction unit is different from the size of the transform unit, intra prediction can be performed using reference pixels based on the transform unit. In addition, N×N division intra prediction can be used only for the smallest coding unit.

[0040] The intra prediction method may generate a prediction block after applying an Adaptive Intra Smoothing (AIS) filter to the reference pixels according to the prediction mode. The type of the AIS filter applied to the reference pixels may vary. To perform the intra prediction method, the intra prediction mode of the current prediction unit may be predicted from the intra prediction mode of the prediction unit existing around the current prediction unit. When predicting the prediction mode of the current prediction unit using the mode information predicted from the surrounding prediction units, if the intra prediction modes of the current prediction unit and the surrounding prediction units are the same, information indicating that the prediction modes of the current prediction unit and the surrounding prediction units are the same may be transmitted using predetermined flag information, and if the prediction modes of the current prediction unit and the surrounding prediction units are different, entropy coding may be performed to encode the prediction mode information of the current block.

[0041] In addition, a residual block including residual value information that is a difference between a prediction unit predicted based on the prediction unit generated by the prediction unit 120, 125 and an original block of the prediction unit may be generated. The generated residual block may be input to the conversion unit 130.

[0042] The transform unit 130 may use a transform method such as a discrete cosine transform (DCT), a discrete sine transform (DST), or a KL transform (KLT) to transform the original block and the residual block including residual coefficient information of the prediction unit generated by the prediction unit 120, 125. Whether to apply the DCT, the DST, or the KLT to transform the residual block may be determined according to intra prediction mode information of the prediction unit used to generate the residual block.

[0043] The quantization unit 135 may quantize the values ​​transformed into the frequency domain by the transform unit 130. The quantization coefficients may be transformed according to the importance of the block or video. The values ​​calculated by the quantization unit 135 may be provided to the inverse quantization unit 140 and the reordering unit 160.

[0044] Re-ordering unit 160 may perform reordering of coefficient values ​​on the quantized residual values.

[0045] The reordering unit 160 can change two-dimensional block shape coefficients into one-dimensional vector form by a coefficient scanning method. For example, the reordering unit 160 can use a zig-zag scan method to scan from DC coefficients to coefficients in a high frequency region and change them into one-dimensional vector form. Instead of zig-zag scanning, vertical scanning, which scans two-dimensional block shape coefficients in a column direction, and horizontal scanning, which scans two-dimensional block shape coefficients in a row direction, can be used according to the size of the transform unit and the intra prediction mode. That is, it is possible to determine which scanning method to use among zig-zag scanning, vertical scanning, and horizontal scanning, according to the size of the transform unit and the intra prediction mode.

[0046] The entropy coding unit 165 can perform entropy coding based on the value calculated by the rearrangement unit 160. For example, various coding methods such as Exponential Golomb coding, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) can be used for the entropy coding.

[0047] The entropy coding unit 165 can code various information such as residual value coefficient information and block type information of the coding unit from the rearrangement unit 160 and the prediction units 120 and 125, prediction mode information, division unit information, prediction unit information and transmission unit information, motion vector information, reference frame information, block interpolation information, and filtering information.

[0048] The entropy coding unit 165 may perform entropy coding on the coefficient values ​​of the coding unit input from the reordering unit 160 .

[0049] The inverse quantization unit 140 and the inverse transform unit 145 inverse quantize the multiple values ​​quantized by the quantization unit 135, and inverse transform the values ​​transformed by the transform unit 130. The residual values ​​generated by the inverse quantization unit 140 and the inverse transform unit 145 can be merged with prediction units predicted by the motion prediction unit, the motion compensation unit, and the intra prediction unit included in the prediction units 120 and 125 to generate a reconstructed block.

[0050] The filter unit 150 may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

[0051] A deblocking filter can remove block artifacts that occur at the boundaries between blocks in a restored image. To determine whether to perform deblocking, a deblocking filter can be applied to a current block based on pixels contained in several columns or rows contained in the block. When applying a deblocking filter to a block, a strong filter or a weak filter can be applied depending on the required deblocking filtering strength. In addition, when performing vertical filtering and horizontal filtering in the process of using a deblocking filter, horizontal filtering and vertical filtering can be processed in parallel.

[0052] The offset correction unit can correct the offset between the deblocked video and the original video on a pixel-by-pixel basis. In order to perform offset correction for a specific image, a method of dividing pixels included in the video into a certain number of regions, determining the region to perform offset, and applying the offset to the corresponding region, or a method of applying the offset by considering edge information of each pixel, can be used.

[0053] Adaptive Loop Filtering (ALF) can be performed based on a comparison between the filtered restored image and the original video. After dividing the pixels in the video into predetermined groups, it is possible to perform differential filtering for each group by determining one filter to be used for the corresponding group. Information regarding whether to apply ALF and a luminance signal can be transmitted by a coding unit (CU), and the shape and filter coefficients of the ALF filter applied depending on each block can be different. It is also possible to apply the same type (fixed type) of ALF filter regardless of the characteristics of the block to which it is applied.

[0054] The memory 155 can store the reconstructed blocks or images calculated by the filter unit 150, and can provide the stored reconstructed blocks or images to the prediction units 120 and 125 when performing inter prediction.

[0055] FIG. 2 is a block diagram of a video decoder according to an embodiment of the present invention.

[0056] Referring to FIG. 2, the video decoder 200 may include an entropy decoding unit 210, a reordering unit 215, an inverse quantization unit 220, an inverse transform unit 225, a prediction unit 230, a prediction unit 235, a filter unit 240, and a memory 245.

[0057] When a video bitstream is input to a video encoder, the input bitstream can be decoded in the reverse steps to the video encoder.

[0058] The entropy decoding unit 210 may perform entropy decoding in a reverse step to the step of performing entropy encoding in the entropy encoding unit of the video encoder. For example, various methods such as Exponential Golomb coding, Context-Adaptive Variable Length Coding (CAVLC), Context-Adaptive Binary Arithmetic Coding (CABAC), etc., may be applied in correspondence with the method performed in the video encoder.

[0059] The entropy decoding unit 210 may decode information regarding intra-prediction and inter-prediction performed by the encoder.

[0060] The re-ordering unit 215 may perform re-ordering based on a method for re-ordering the bit stream entropy decoded by the entropy decoding unit 210 in the encoding unit. A plurality of coefficients represented in the form of a one-dimensional vector may be restored to two-dimensional block-shaped coefficients and re-ordered. The re-ordering unit 215 may receive information regarding coefficient scanning performed in the encoding unit and perform re-ordering by an inverse scanning method based on the scanning order performed by the corresponding encoding unit.

[0061] The inverse quantization unit 220 may perform inverse quantization based on the quantization parameter provided by the encoder and the coefficient values ​​of the reordered block.

[0062] For the quantization result performed by the video encoder, the inverse transform unit 225 can perform inverse transform (i.e., inverse DCT, inverse DST, and inverse KLT) for the transform performed by the transform unit, i.e., DCT, DST, and KLT. The inverse transform can be performed based on the transmission unit determined by the video encoder. In the inverse transform unit 225 of the video decoder, the transform method (e.g., DCT, DST, KLT) can be selectively performed based on multiple information such as a prediction method, a size of a current block, and a prediction direction.

[0063] The prediction units 230, 235 may generate a prediction block based on information regarding the generation of the prediction block provided by the entropy decoding unit 210 and information regarding previously decoded blocks or images provided by the memory 245.

[0064] As described above, when performing intra prediction in a manner similar to the operation in a video encoder, if the size of the prediction unit and the size of the transform unit are the same, intra prediction is performed on the prediction unit based on the pixel located to the left, the pixel located to the upper left, and the pixel located above the prediction unit, and if the size of the prediction unit and the size of the transform unit differ when performing intra prediction, intra prediction can be performed using reference pixels based on the transform unit. Also, N×N division intra prediction can be used only for the minimum coding unit.

[0065] The prediction unit 230, 235 may include a prediction unit discrimination unit, an inter prediction unit, and an intra prediction unit. The prediction unit discrimination unit may receive various information, such as prediction unit information input from the entropy decoding unit 210, prediction mode information of the intra prediction method, and motion prediction related information of the inter prediction method, and may classify the prediction unit according to the current encoding unit to determine whether the prediction unit performs inter prediction or intra prediction. The inter prediction unit 230 may use information necessary for inter prediction of the current prediction unit provided by the video encoder to perform inter prediction on the current prediction unit based on information included in at least one image of the previous image or the subsequent image of the current image to which the current prediction unit belongs. Alternatively, inter prediction may be performed based on information of a part of the area restored in the current image to which the current prediction unit belongs.

[0066] To perform inter prediction, it is possible to determine, based on the coding unit, whether the motion prediction method of the prediction unit included in the corresponding coding unit is skip mode, merge mode, advanced motion vector prediction mode (AMVP mode), or intra block copy mode.

[0067] The intra prediction unit 235 may generate a prediction block based on pixel information in the current image. If the prediction unit is a prediction unit that has performed intra prediction, the intra prediction may be performed based on intra prediction mode information of the prediction unit provided by the video encoder. The intra prediction unit 235 may include an adaptive intra smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter is an element that performs filtering on the reference pixels of the current block, and may determine whether to apply the filter according to the prediction mode of the current prediction unit. The prediction mode and AIS filter information of the prediction unit provided by the video encoder may be used to perform AIS filtering on the reference pixels of the current block. If the prediction mode of the current block is a mode that does not perform AIS filtering, the AIS filter may not be applied.

[0068] If the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on pixel values ​​of the interpolated reference pixels, the reference pixel interpolation unit may generate integer or fractional pixel-based reference pixels by interpolating the reference pixels. If the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixels, the reference pixels may not be interpolated. If the prediction mode of the current block is a DC mode, the DC filter may generate a prediction block by filtering.

[0069] The reconstructed block or image may be provided to a filter unit 240. The filter unit 240 may include a deblocking filter, an offset correction unit, and an ALF.

[0070] From the video encoder, information regarding whether to apply a deblocking filter to a block or image, and whether to apply strong or weak filtering when applying the deblocking filter can be received. From a deblocking filter of the video decoder, related information of the deblocking filter provided by the video encoder can be received, and the video decoder can perform deblocking filtering on the corresponding block.

[0071] The offset compensation unit can perform offset compensation on the reconstructed video based on, for example, the type of offset compensation and offset amount information used on the video when it was encoded.

[0072] An ALF may be applied to a coding unit based on information provided by the encoder about whether to apply an ALF, ALF coefficient information, etc. Such ALF information may be provided by being included in a particular parameter set.

[0073] The memory 245 can store the reconstructed image or block, can use said image or block as a reference image or block, and can provide the reconstructed image to an output.

[0074] FIG. 3 is a diagram illustrating a basic coding tree unit according to an embodiment of the present invention.

[0075] A coding block of a maximum size can be defined as a coding tree block. An image can be divided into multiple coding tree units (CTUs). A coding tree unit is a coding unit of a maximum size, and can also be called a largest coding unit (LCU). Figure 3 shows an example in which an image is divided into multiple coding tree units.

[0076] The size of the coding tree unit can be defined at the picture level or at the sequence level, so information indicating the size of the coding tree unit can be signaled via the picture parameter set or the sequence parameter set.

[0077] For example, the coding tree unit size for all images in a sequence can be set to 128 x 128. Alternatively, the coding tree unit size can be determined at the image level to be either 128 x 128 or 256 x 256. For example, the coding tree unit size for a first image can be set to 128 x 128, and the coding tree unit size for a second image can be set to 256 x 256.

[0078] A coding block can be generated by dividing the coding tree unit. The coding block represents a basic unit for performing an encoding / decoding process. For example, prediction or transformation can be performed according to different coding blocks, or a predictive coding mode can be determined according to different coding blocks. Here, the predictive coding mode represents a method for generating a predicted image. For example, the predictive coding mode may include Intra Prediction, Inter Prediction, Current Picture Referencing (CPR, or Intra Block Copy (IBC)), or Combined Prediction. For a coding block, at least one predictive coding mode of Intra Prediction, Inter Prediction, Current Picture Referencing, or Combined Prediction can be used to generate a predictive block for the coding block.

[0079] Information indicating the predictive coding mode of the current block can be signaled via a bitstream. For example, the information can be a 1-bit flag indicating whether the predictive coding mode is an intra mode or an inter prediction mode. Only when it is determined that the predictive coding mode of the current block is an inter prediction mode, the current image reference or composite prediction can be used.

[0080] The current image reference is used to set the current image as a reference image and obtain a prediction block of the current block from an encoded / decoded area in the current image. Here, the current image means an image including the current block. Information indicating whether the current image reference is applied to the current block can be signaled via a bitstream. For example, the information can be a 1-bit flag. If the flag is true, the prediction coding mode of the current block can be determined as the current image reference, and if the flag is false, the prediction mode of the current block can be determined as inter prediction.

[0081] Alternatively, the predictive coding mode of the current block can be determined based on the reference image index. For example, if the reference image index refers to the current image, the predictive coding mode of the current block can be determined as current image reference. If the reference image index refers to other images other than the current image, the predictive coding mode of the current block can be determined as inter prediction. That is, the current image reference is a prediction method that uses information of the coded / decoded area in the current image, and the inter prediction is a prediction method that uses information of other coded / decoded images.

[0082] The composite prediction indicates a coding mode that combines two or more of intra prediction, inter prediction, and current image reference. For example, when the composite prediction is applied, a first prediction block can be generated based on either intra prediction, inter prediction, or current image reference, and a second prediction block can be generated based on the other one. When the first prediction block and the second prediction block are generated, a final prediction block can be generated through an average operation or an additive operation of the first prediction block and the second prediction block. Information indicating whether the composite prediction is applied can be signaled through a bitstream. The information can be a 1-bit flag.

[0083] FIG. 4 is a diagram showing various division forms of a coding block.

[0084] A coding block can be divided into multiple coding blocks based on a quadtree partition, a binary tree partition, or a ternary tree partition. A divided coding block can also be divided again into multiple coding blocks based on a quadtree partition, a binary tree partition, or a ternary tree partition.

[0085] Quadtree partitioning refers to a partitioning technique that splits the current block into four blocks. As a result of quadtree partitioning, the current block can be divided into four square partitions (see “SPLIT_QT” in Figure 4(a)).

[0086] Binary tree splitting refers to a splitting technique that splits the current block into two blocks. Splitting the current block into two blocks along the vertical direction (i.e., using a vertical line across the current block) can be called vertical binary tree splitting, and splitting the current block into two blocks along the horizontal direction (i.e., using a horizontal line across the current block) can be called horizontal binary tree splitting. Binary tree splitting allows the current block to be split into two non-square partitions. "SPLIT_BT_VER" in Figure 4(b) shows the vertical binary tree split result, and "SPLIT_BT_HOR" in Figure 4(c) shows the horizontal binary tree split result.

[0087] Ternary tree partitioning refers to a partitioning technique that partitions the current block into three blocks. Partitioning the current block into three blocks along the vertical direction (i.e., using two vertical lines across the current block) can be referred to as vertical ternary tree partitioning, and partitioning the current block into three blocks along the horizontal direction (i.e., using two horizontal lines across the current block) can be referred to as horizontal ternary tree partitioning. Ternary tree partitioning allows partitioning the current block into three non-square partitions. In this case, the width / height of the partition located in the middle of the current block can be twice the width / height of the other partitions. "SPLIT_TT_VER" in FIG. 4(d) shows the vertical ternary tree partitioning result, and "SPLIT_TT_HOR" in FIG. 4(e) shows the horizontal ternary tree partitioning result.

[0088] The number of times a coding tree unit is divided can be defined as the partitioning depth. The maximum partitioning depth of a coding tree unit can be determined at the sequence or image level. This allows the maximum partitioning depth of a coding tree unit to be different for different sequences or images.

[0089] Alternatively, the maximum division depth for each of the multiple division techniques can be determined separately. For example, the maximum division depth at which quadtree division is permitted can be different from the maximum division depth at which binary tree and / or ternary tree division is permitted.

[0090] The encoder may signal information via the bitstream indicating at least one of the partition shape or partition depth of the current block, and the decoder may determine the partition shape and partition depth of the coding tree unit based on the information parsed from the bitstream.

[0091] FIG. 5 is a diagram illustrating an example of division of a coding tree unit.

[0092] Partitioning a coding block using partitioning techniques such as quad-tree partitioning, binary tree partitioning, and / or ternary tree partitioning can be referred to as Multi Tree Partitioning.

[0093] The multiple coding blocks generated by applying the multi-tree partitioning to the coding block can be called sub-coding blocks. If the partitioning depth of the coding block is k, the partitioning depth of the multiple sub-coding blocks is set to k+1.

[0094] On the other hand, among multiple coding blocks with a division depth of k+1, a coding block with a division depth of k can be called a higher-order coding block.

[0095] A partition type of a current coding block can be determined based on at least one of a partition shape of a higher coding block or a partition type of an adjacent coding block, where the adjacent coding block is adjacent to the current coding block, and the adjacent coding block may include at least one of an upper adjacent block, a left adjacent block, or an adjacent block adjacent to the upper left corner of the current coding block, where the partition type may include at least one of whether to perform quadtree partitioning, whether to perform binary tree partitioning, a partition direction of the binary tree, whether to perform ternary tree partitioning, or a partition direction of the ternary tree.

[0096] To determine the partition shape of the coding block, information indicating whether the coding block is to be partitioned can be signaled via the bitstream. The information is a 1-bit flag "split_cu_flag", and if the current flag is true, the coding block can be partitioned by a multi-tree partitioning method.

[0097] If 'split_cu_flag' is true, information indicating whether the coding block is quadtree partitioned can be signaled via the bitstream. The information is a 1-bit flag 'split_qt_flag', and if the flag is true, the coding block can be split into four blocks.

[0098] 5 shows that the coding tree unit is subjected to quadtree division to generate four coding blocks with a division depth of 1. It also shows that quadtree division is applied again to the first coding block and the fourth coding block among the four coding blocks generated as a result of the quadtree division. As a result, four coding blocks with a division depth of 2 can be generated.

[0099] Moreover, by applying quadtree division again to a coding block with a division depth of two, a coding block with a division depth of three can be generated.

[0100] If quadtree partitioning is not applied to a coding block, it may be determined whether to perform binary tree partitioning or ternary tree partitioning on the coding block, taking into account at least one of the following: the size of the coding block, whether the coding block is located on an image boundary, the maximum partition depth, or the partition shape of an adjacent block. If it is determined to perform binary tree partitioning or ternary tree partitioning on the coding block, information indicating a partitioning direction may be signaled via a bitstream. The information may be a 1-bit flag "mtt_split_cu_vertical_flag". Based on the flag, it may be determined whether the partitioning direction is vertical or horizontal. Furthermore, information indicating whether binary tree partitioning or ternary tree partitioning is applied to the coding block may be signaled via a bitstream. The information may be a 1-bit flag "mtt_split_cu_binary_flag". Based on the flag, it may be determined whether to apply binary tree partitioning or ternary tree partitioning to the coding block.

[0101] For example, in the example shown in Figure 5, vertical binary tree partitioning is applied to a coding block with a partitioning depth of 1, and of the coding blocks generated as a result of the partitioning, vertical ternary tree partitioning is applied to the coding block on the left and vertical binary tree partitioning is applied to the coding block on the right.

[0102] When an image is divided into coding tree units, a block having a size smaller than a predefined size may exist in an area adjacent to the right or bottom boundary of the image. If the block is a coding tree unit, a coding tree unit having a size smaller than the predefined size may be generated at the right or bottom boundary of the image. In this case, the size of the coding tree unit may be determined based on information signaled via a sequence parameter set or a picture parameter set.

[0103] FIG. 6 is a diagram showing an example in which a block smaller than a coding tree unit of a preset size appears on an image boundary.

[0104] 6, when a 1292×1080 size image is divided into 128×128 size coding tree units, there will be blocks at the right and bottom boundaries of the image that are smaller than 128×128 in size. In the embodiment described below, the blocks of coding tree units smaller than the predefined size generated at the image boundaries will be called border blocks of irregular boundaries.

[0105] For boundary blocks of an irregular boundary, only predefined partitioning methods may be allowed, where the predefined partitioning methods may include at least one of quadtree partitioning, ternary tree partitioning, or binary tree partitioning.

[0106] For example, for border blocks of irregular boundaries, only quadtree partitioning may be allowed. In this case, quadtree partitioning may be repeated until the blocks of the image border are of the minimum quadtree partition size. Here, the minimum quadtree partition size may be predefined in the encoder and decoder. Alternatively, information indicating the minimum quadtree partition size may be signaled via the bitstream.

[0107] 7 is a diagram illustrating an example of performing quadtree division on a boundary block of an irregular boundary. For convenience of explanation, it is assumed that the size of the minimum quadtree division is 4×4.

[0108] The division into the boundary block of the irregular boundary can be performed based on a square block. The square block can be derived based on the larger of the width or height of the boundary block of the irregular boundary. For example, a power of 2 that is greater than and closest to the reference value can be regarded as the length of one side of the square block. For example, the 12×20 block shown in FIG. 7 can be regarded as belonging to a 32×32 block, and the division result of 32×32 can be applied to the 12×20 block.

[0109] When performing quadtree decomposition on a 12x20 block, it may be split into a 12x16 size block and a 12x4 size block. When performing quadtree decomposition on each of them again, the 12x16 block is split into two 8x8 blocks and two 4x8 blocks, and the 12x4 size block is split into an 8x4 size block and a 4x4 size block.

[0110] We perform quadtree decomposition again on the 4x8 blocks located on the image border, resulting in two blocks of size 4x4. Similarly, we perform quadtree decomposition again on the 8x4 blocks located on the image border, resulting in two blocks of size 4x4.

[0111] Alternatively, binary tree partitioning can be performed if at least one of the width or height of the block is equal to or smaller than the minimum quadtree partition size, where the minimum quadtree partition size can indicate the minimum quadtree partition width or the minimum quadtree partition height. For example, if the minimum quadtree partition size is 4×4, the minimum quadtree partition width and the minimum quadtree partition height can be 4.

[0112] In this case, if the width of the block is equal to or smaller than the minimum quadtree partition size, then a vertical binary tree partition can be performed, and if the height of the block is equal to or smaller than the minimum quadtree partition height, then a horizontal binary tree partition can be performed.

[0113] On the other hand, if the width or height of a block is larger than the minimum quadtree division size, quadtree division can be performed. For example, if the top right and bottom left positions of a block are outside the image and the width or height of the block is larger than the minimum quadtree division size, quadtree division can be applied to the corresponding block.

[0114] 8 is a diagram illustrating an example of performing quadtree division on a block that is simultaneously adjacent to the right and bottom boundaries of an image. For ease of explanation, it is assumed that the size of the minimum quadtree division is 4×4.

[0115] In the example shown in FIG. 8, when quadtree division is performed on a 32×32 block including a 12×20 block, four 16×16 blocks are generated. Quadtree division can be performed again on the two 16×16 blocks including these texture data. As a result, the x-axis coordinate and y-axis coordinate can escape the image boundary to generate an 8×8 block including 4×4 texture data. Since the width and height of the 8×8 block are larger than the minimum quadtree division size, quadtree division can be performed on the block.

[0116] When performing quadtree decomposition on a 12x20 block, it may be split into a 12x16 size block and a 12x4 size block. When performing quadtree decomposition on each of them again, the 12x16 block is split into two 8x8 blocks and two 4x8 blocks, and the 12x4 size block is split into an 8x4 size block and a 4x4 size block.

[0117] Since the width of a 4x8 block located on the image border is the same as the minimum quadtree division size, a binary tree division can be performed on the 4x8 block. Specifically, a vertical binary tree division can be performed based on a square block (i.e., 8x8) that contains the 4x8 block.

[0118] Also, since the width of an 8x4 block located on the image boundary is the same as the minimum quadtree division size, a binary tree division can be performed on the 8x4 block. Specifically, a horizontal binary tree division can be performed based on a square block (i.e., 8x8) that contains a 4x8 block.

[0119] The division results in blocks of 8×4 size, blocks of 4×4 size, and blocks of 4×8 size that are arranged adjacent to the image boundary.

[0120] Alternatively, binary tree partitioning can be performed if at least one of the width or height of the block is less than or equal to a threshold, and quad tree partitioning can be performed otherwise, where the threshold can be derived based on a minimum quad tree partition size. For example, if the minimum quad tree partition size is minQTsize, the threshold can be set to "minQTsize<<1". Alternatively, information for determining the threshold can be signaled via the bitstream.

[0121] 9 is a diagram showing the division pattern of blocks adjacent to an image boundary. For convenience of explanation, assume that the size of the minimum quadtree division is 4 × 4. The threshold can be set to 8.

[0122] First, perform quadtree decomposition on the 12x20 block. As a result, the block can be divided into a 12x16 size block and a 12x4 size block. Since both the width and height of the 12x16 size block exceed the threshold, quadtree decomposition can be applied to the block. This allows the block to be divided into two 8x8 blocks and two 4x8 blocks.

[0123] The width of the 12×4 block exceeds a threshold, so that the quadtree division can be applied to the 12×4 block, which can then be divided into an 8×4 block and a 4×4 block.

[0124] Then, binary tree decomposition can be applied to the 4x8 and 8x4 size blocks located on the image border since both the width and height of these blocks are smaller than or equal to the threshold.

[0125] Contrary to the above example, if at least one of the width or height of the block exceeds a threshold, a binary tree partitioning can be performed, otherwise a quad tree partitioning can be performed.

[0126] Alternatively, only quadtree partitioning or only binary tree partitioning can be applied to the border blocks of the irregular boundaries, for example, repeatedly performing quadtree partitioning until the blocks located on the image border have a minimum size, or repeatedly performing binary tree partitioning until the blocks located on the image border have a minimum size.

[0127] A boundary block of an atypical boundary is designated as a coding unit, and for the boundary block of an atypical boundary, a skip mode can be fixedly applied or all transform coefficients can be set to 0. In this way, a coded block flag (CBF), which indicates whether or not a boundary block of an atypical boundary has a transform coefficient other than 0, can be set to 0. A coding unit coded in skip mode or a coding unit with transform coefficients set to 0 can be called a boundary zero coding unit.

[0128] Alternatively, at least one of the width or height of the coding block generated by dividing the boundary block of the non-standard boundary may be compared with a threshold value to determine whether to set the coding unit as a boundary zero coding unit. For example, a coding block having at least one of the width or height of the coding block less than a threshold value may be coded in skip mode, or the transform coefficients of the coding block may be set to 0.

[0129] 10 shows the coding pattern for blocks adjacent to an image boundary. Assume the threshold is 8.

[0130] Among the coding blocks generated by dividing a boundary block of an irregular boundary, a coding block having at least one of a width and a height less than a threshold can be set as a boundary zero coding unit.

[0131] For example, in the example shown in Figure 10, a coding block of 4x16 size, a coding block of 8x4 size, and a coding block of 4x4 size can be set as boundary zero coding units, which allows the blocks to be coded in skip mode or the transform coefficients of the blocks to be set to zero.

[0132] For coding blocks whose width and height are greater than or equal to the threshold, it may be selectively decided whether to apply skip mode or set transform coefficients to 0. To this end, a flag indicating whether to apply skip mode or set transform coefficients to 0 for said coding block may be coded and signalled.

[0133] Alternatively, it may be permitted that only coding units generated by binary tree partitioning may be set as boundary-zero coding units. Alternatively, it may be permitted that only coding units generated by quad-tree partitioning may be set as boundary-zero coding units.

[0134] Inter prediction is a predictive coding mode that uses information of a previous image to predict a current block. For example, a block in a previous image that is at the same position as the current block (hereinafter referred to as a collocated block) can be set as a prediction block of the current block. Hereinafter, a prediction block generated based on a block that is at the same position as the current block is referred to as a collocated prediction block.

[0135] On the other hand, if an object that existed in a previous image moves to another position in the current image, the object's motion can be used to effectively predict the current block. For example, if the object's moving direction and size can be known by comparing the previous image with the current image, a predicted block (or predicted image) of the current block can be generated taking into account the object's motion information. Hereinafter, the predicted block generated using the motion information can be referred to as a motion predicted block.

[0136] A residual block can be generated by subtracting the predicted block from the current block, where in the presence of object motion, the energy of the residual block can be reduced by using the motion predicted block instead of the co-located predicted block, thereby improving the compression performance of the residual block.

[0137] As mentioned above, using motion information to generate a predictive block may be referred to as motion compensated prediction. In most inter predictions, a predictive block may be generated based on motion compensated prediction.

[0138] The motion information may include at least one of a motion vector, a reference image index, a prediction direction, or a bidirectional weight value index. The motion vector represents the moving direction and size of an object. The reference image index specifies a reference image of the current block among multiple reference images included in a reference image list. The prediction direction refers to either unidirectional L0 prediction, unidirectional L1 prediction, or bidirectional prediction (L0 prediction and L1 prediction). According to the prediction direction of the current block, at least one of the motion information in the L0 direction or the motion information in the L1 direction can be used. The bidirectional weight value index specifies a weight value applied to the L0 prediction block and a weight value applied to the L1 prediction block.

[0139] FIG. 11 is a flowchart of an inter prediction method according to an embodiment of the present invention.

[0140] Referring to FIG. 11, the inter prediction method includes determining an inter prediction mode of a current block (S1101), obtaining motion information of the current block according to the determined inter prediction mode (S1102), and performing motion compensation prediction of the current block based on the obtained motion information (S1103).

[0141] Here, the inter prediction mode represents various methods for determining the motion information of the current block, and may include an inter prediction mode using translation motion information and an inter prediction mode using affine motion information. For example, the inter prediction mode using translation motion information may include a merge mode and a high-level motion vector prediction mode, and the inter prediction mode using affine motion information may include an affine merge mode and an affine motion vector prediction mode. According to the inter prediction mode, the motion information of the current block may be determined based on information analyzed from a neighboring block or a bitstream adjacent to the current block.

[0142] The inter prediction method using translational motion information will be described in detail below.

[0143] The motion information of the current block may be derived from the motion information of another block of the current block. Here, the other block may be a block that has been coded / decoded by inter prediction before the current block. Setting the motion information of the current block to be the same as the motion information of another block may be defined as a merge mode. Also, setting the motion vector of another block to a predicted value of the motion vector of the current block may be defined as a motion vector prediction mode.

[0144] FIG. 12 is a flowchart of a process for deriving motion information for a current block in merge mode.

[0145] Merge candidates for a current block may be derived (S1201). The merging candidates for the current block may be derived from blocks that have been coded / decoded using inter prediction prior to the current block.

[0146] FIG. 13 is a diagram showing candidate blocks for deriving merging candidates.

[0147] The candidate block may include at least one of a neighboring block including samples adjacent to the current block, or a non-neighboring block including samples not adjacent to the current block. Hereinafter, the samples used to determine the candidate block are designated as reference samples. Also, the reference samples adjacent to the current block are referred to as neighboring reference samples, and the reference samples not adjacent to the current block are referred to as non-neighboring reference samples.

[0148] The adjacent reference sample may be included in the adjacent column of the leftmost column of the current block or in the adjacent row of the topmost row of the current block. For example, if the coordinates of the top left sample of the current block are (0, 0), at least one of the following may be used as a candidate block: a block with a reference sample at a position of (-1, H-1), a block including a reference sample at a position of (W-1, -1), a block including a reference sample at a position of (W, -1), a block including a reference sample at a position of (-1, H), or a block including a reference sample at a position of (-1, -1). Referring to the drawing, the adjacent blocks with indexes 0 to 4 may be used as candidate blocks.

[0149] The non-adjacent reference sample indicates a sample in which at least one of the x-axis distance or y-axis distance from the reference sample adjacent to the current block has a predefined value. For example, at least one of a block including a reference sample whose x-axis distance from the left reference sample is a predefined value, a block of non-adjacent samples whose y-axis distance from the top reference sample is a predefined value, or a block of non-adjacent samples whose x-axis distance and y-axis distance from the top left reference sample are predefined values ​​can be used as a candidate block. The predefined value can be an integer such as 4, 8, 12, 16, etc. Referring to the drawing, at least one of the blocks with indexes 5 to 26 can be used as a candidate block.

[0150] A sample that is not located on the same vertical, horizontal, or diagonal line as an adjacent reference sample can be set as a non-adjacent reference sample.

[0151] FIG. 14 is a diagram showing the positions of the reference samples.

[0152] As shown in the example of FIG. 14, the x coordinate of the top non-adjacent reference sample can be set to be different from the x coordinate of the top adjacent reference sample. For example, if the position of the top adjacent reference sample is (W-1, -1), the position of the top non-adjacent reference sample that is N away from the top adjacent reference sample along the y axis can be set to ((W / 2)-1, -1-N), and the position of the top non-adjacent reference sample that is 2N away from the top adjacent reference sample along the y axis can be set to (0, -1-2N). That is, the position of the non-adjacent reference sample can be determined based on the position of the adjacent reference sample and the distance between the adjacent reference sample.

[0153] Hereinafter, among the candidate blocks, a candidate block including adjacent reference samples will be referred to as an adjacent block, and a block including non-adjacent reference samples will be referred to as a non-adjacent block.

[0154] If the distance between the current block and the candidate block is greater than or equal to a threshold, the candidate block can be set as ineligible for use as a merging candidate. The threshold can be determined based on the size of the coding tree unit. For example, the threshold can be set to the height of the coding tree unit (ctu_height), or can be set to a value obtained by adding or subtracting an offset value from the height of the coding tree unit (e.g., ctu_height±N). The offset value N is a predefined value in the encoder and decoder, and can be set to 4, 8, 16, 32, or ctu_height.

[0155] If the difference between the y-axis coordinate of the current block and the y-axis coordinate of a sample contained in the candidate block exceeds a threshold, it can be determined that the candidate block cannot be used as a merging candidate.

[0156] Alternatively, a candidate block that does not belong to the same coding tree unit as the current block can be set as ineligible for use as a merging candidate, e.g., a candidate block that contains a reference sample can be set as ineligible for use as a merging candidate if the reference sample does not exceed the upper boundary of the coding tree unit that the current block belongs to.

[0157] If the upper boundary of the current block is adjacent to the upper boundary of the coding tree unit, multiple candidate blocks are set as not usable as merge candidates, which reduces the encoding / decoding efficiency of the current block. To solve the above problem, the candidate blocks can be set so that the number of candidate blocks located above the current block is greater than the number of candidate blocks to the left of the current block.

[0158] FIG. 15 is a diagram showing candidate blocks for deriving merging candidates.

[0159] 15, the top blocks belonging to the N block rows above the current block and the left blocks belonging to the M block rows to the left of the current block can be set as candidate blocks. In this case, by setting M larger than N, the number of left candidate blocks can be set larger than the number of top candidate blocks.

[0160] For example, the difference between the y-axis coordinate of a reference sample in the current block and the y-axis coordinate of a top block that can be used as a candidate block can be set to not exceed N times the height of the current block, and the difference between the x-axis coordinate of a reference sample in the current block and the x-axis coordinate of a left block that can be used as a candidate block can be set to not exceed M times the width of the current block.

[0161] For example, the example shown in FIG. 15 indicates that the blocks belonging to the two block columns above the current block and the blocks belonging to the five block columns to the left of the current block are set as candidate blocks.

[0162] As another example, if a candidate block does not belong to the same coding tree unit as the current block, a merging candidate can be derived instead of the candidate block using a block that belongs to the same coding tree unit as the current block or a block that contains reference samples adjacent to the coding tree unit boundary.

[0163] FIG. 16 is a diagram showing an example of changing the position of the reference sample.

[0164] If a reference sample is included in a different coding tree unit than the current block and the reference sample is not adjacent to the coding tree unit boundary, a reference sample adjacent to the coding tree unit boundary may be used instead of the reference sample to determine a candidate block.

[0165] For example, in the example shown in Figures 16(a) and 16(b), if the top boundary of the current block and the top boundary of the coding tree unit meet, the reference samples at the top of the current block will belong to a coding tree unit different from the current block. The samples adjacent to the top boundary of the coding tree unit can be used to replace the reference samples that belong to a coding tree unit different from the current block but are not adjacent to the top boundary of the coding tree unit.

[0166] For example, as shown in the example of Figure 16(a), the reference sample at position 6 is replaced with the sample at position 6' of the upper boundary of the coding tree unit, and as shown in the example of Figure 16(b), the reference sample at position 15 is replaced with the sample at position 15' of the upper boundary of the coding tree unit. In this case, the y coordinate of the replacement sample can be changed to an adjacent position of the coding tree unit, and the x coordinate of the replacement sample can be set to the same as the reference sample. For example, the sample at position 6' may have the same x coordinate as the sample at position 6, and the sample at position 15' may have the same x coordinate as the sample at position 15.

[0167] Alternatively, the x-coordinate of the replacement sample can be set to a value obtained by adding or subtracting an offset value from the x-coordinate of the reference sample. For example, if the x-coordinates of the adjacent reference sample at the top of the current block and the non-adjacent reference sample are the same, the x-coordinate of the replacement sample can be set to a value obtained by adding or subtracting an offset value from the x-coordinate of the reference sample. This is to prevent the replacement sample that replaces the non-adjacent reference sample from being in the same position as other non-adjacent reference samples or adjacent reference samples.

[0168] FIG. 17 is a diagram showing an example of changing the position of the reference sample.

[0169] When a sample located on a coding tree unit boundary is used to replace a reference sample that is contained in a coding tree unit different from the current block and is not adjacent to a coding tree unit boundary, the x coordinate of the replacement sample can be set to a value obtained by adding or subtracting an offset value from the x coordinate of the reference sample.

[0170] For instance, in the example shown in Figure 17, the reference sample at position 6 and the reference sample at position 15 can be replaced with the sample at position 6' and the sample at position 15', respectively, that have the same y coordinate as the row adjacent to the top boundary of the coding tree unit. In this case, the x coordinate of the sample at position 6' can be set to the x coordinate of the reference sample at position 6 minus W / 2, and the x coordinate of the sample at position 15' can be set to the x coordinate of the reference sample at position 15 minus W-1.

[0171] Unlike the examples shown in Figures 16 and 17, the y coordinate of the row located above the top row of the current block or the y coordinate of the upper boundary of the coding tree unit can also be set as the y coordinate of the replacement sample.

[0172] Although not shown, a sample to replace the reference sample may be determined based on the left boundary of the coding tree unit. For example, if the reference sample is not included in the same coding tree unit as the current block and is not adjacent to the left boundary of the coding tree unit, the reference sample may be replaced with a sample adjacent to the left boundary of the coding tree unit. In this case, the replacement sample may have the same y coordinate as the reference sample, or may have a y coordinate obtained by adding or subtracting an offset value from the y coordinate of the reference sample.

[0173] Then, the block containing the replacement samples can be set as a candidate block, and merging candidates for the current block can be derived based on the candidate block.

[0174] Merging candidates can also be derived from temporally adjacent blocks in a different image than the current block, for example from equally positioned blocks in an equally positioned image.

[0175] The motion information of the merging candidate may be set to be the same as the motion information of the candidate block, for example, at least one of the motion vector, the reference image index, the prediction direction, or the bidirectional weight value index of the candidate block may be set to the motion information of the merging candidate.

[0176] A merge candidate list including merge candidates may be generated (S1202). The merge candidates may be classified into adjacent merge candidates derived from adjacent blocks adjacent to the current block and non-adjacent merge candidates derived from non-adjacent blocks.

[0177] Indices for multiple merge candidates in the merge candidate list may be assigned according to a predetermined order. For example, indices assigned to adjacent merge candidates may have lower values ​​than indices assigned to non-adjacent merge candidates. Alternatively, an index may be assigned to each merge candidate based on the index of each block shown in FIG. 13 or FIG. 15.

[0178] If the merge candidate list includes multiple merge candidates, at least one of the multiple merge candidates may be selected (S1203). In this case, information indicating whether motion information of the current block is derived from an adjacent merge candidate may be signaled via the bitstream. The information may be a 1-bit flag. For example, a syntax element isAdjancentMergeFlag indicating whether motion information of the current block is derived from an adjacent merge candidate may be signaled via the bitstream. If the value of the syntax element isAdjancentMergeFlag is 1, the motion information of the current block may be derived based on the adjacent merge candidate. On the other hand, if the value of the syntax element isAdjancentMergeFlag is 0, the motion information of the current block may be derived based on a non-adjacent merge candidate.

[0179] Table 1 shows a syntax table including the syntax element isAdjancentMergeFlag. [Table 1] JPEG0007661557000002.jpg128166

[0180] Information for specifying one of multiple merging candidates can be signaled via the bitstream, for example, information indicating an index of one of the merging candidates included in the merging candidate list can be signaled via the bitstream.

[0181] The syntax element merge_idx for determining one of the adjacent merge candidates can be signaled if isAdjacentMergeflag is 1. The maximum value of the syntax element merge_idx can be set to the number of adjacent merge candidates minus 1.

[0182] If isAdjacentMergeflag is 0, the syntax element NA_merge_idx can be signaled to determine one of the non-adjacent merge candidates. The syntax element NA_merge_idx indicates a value obtained by subtracting the number of adjacent merge candidates from the index of the non-adjacent merge candidate. The decoder can select the non-adjacent merge candidate by adding the number of adjacent merge candidates to the index determined by NA_merge_idx.

[0183] If the number of merge candidates included in the merge candidate list is less than the maximum value, merge candidates included in the inter region motion information table may be added to the merge candidate list. The inter region motion information table may include merge candidates derived based on blocks encoded / decoded before the current block.

[0184] The inter-region motion information table includes merge candidates derived from blocks encoded / decoded based on inter prediction in a current image. For example, the motion information of the merge candidates included in the inter-region motion information table can be set to be the same as the motion information of the blocks encoded / decoded based on inter prediction. Here, the motion information may include at least one of a motion vector, a reference image index, a prediction direction, or a bidirectional weight value index.

[0185] For ease of explanation, the merge candidates included in the inter region motion information table are called inter region merge candidates.

[0186] In the encoder and decoder, the maximum number of merge candidates that can be included in the inter region motion information table can be predefined. For example, the maximum number of merge candidates that can be included in the inter region motion information table can be 1, 2, 3, 4, 5, 6, 7, 8, or more (e.g., 16, etc.).

[0187] Alternatively, information indicating the maximum number of merge candidates in the inter region motion information table can be signaled via the bitstream. The information can be signaled at the sequence level, the picture level or the slice level.

[0188] Alternatively, the maximum number of merge candidates of the inter-region motion information table can be determined according to the size of the image, the size of the slice, or the size of the coding tree unit.

[0189] The inter region motion information table can be initialized in units of an image, slice, tile, brick, coding tree unit, or coding tree unit line (row or column). For example, when a slice is initialized, the inter region motion information table is also initialized, and the inter region motion information table may not contain any merge candidates.

[0190] Alternatively, information indicating whether to initialize the inter region motion information table can be signaled via a bitstream. The information can be signaled at a slice level, a tile level, a brick level, or a block level. The configured inter region motion information table can be used until the information indicates that the inter region motion information table should be initialized.

[0191] Alternatively, information about a given inter region merging candidate can be signaled via a picture parameter set or slice header. When a slice is initialized, the inter region motion information table can include the initial inter region merging candidate. This allows the inter region merging candidate to be used even for the first coded / decoded block in the slice.

[0192] Blocks can be coded / decoded according to the coding / decoding order, and a plurality of blocks coded / decoded based on inter prediction according to the coding / decoding order can be sequentially set as inter region merging candidates.

[0193] FIG. 18 is a diagram for explaining an example of updating the inter region motion information table.

[0194] When performing inter prediction on a current block (S1801), an inter region merging candidate may be derived based on the current block (S1802). The motion information of the inter region merging candidate may be set to be the same as the motion information of the current block.

[0195] If the inter region motion information table is empty (S1803), an inter region merge candidate derived based on the current block may be added to the inter region motion information table (S1804).

[0196] If the inter region motion information table already includes the inter region merge candidate (S1803), a redundancy check can be performed on the motion information of the current block (or the inter region merge candidate derived based on the current block) (S1805). The redundancy check is used to determine whether the motion information of the inter region merge candidate stored in the inter region motion information table is the same as the motion information of the current block. The redundancy check can be performed with all the inter region merge candidates stored in the inter region motion information table as objects. Alternatively, the redundancy check can be performed with the inter region merge candidate whose index is equal to or greater than a threshold value or equal to or less than a threshold value among the inter region merge candidates stored in the inter region motion information table as objects.

[0197] If the inter region merging candidates do not include the inter region merging candidates with the same motion information as the motion information of the current block, the inter region merging candidates derived based on the current block may be added to the inter region motion information table (S1808). Whether the inter region merging candidates are the same may be determined based on whether the motion information (e.g., motion vectors and / or reference image indexes, etc.) of the inter region merging candidates are the same.

[0198] In this case, if the maximum number of inter region merge candidates are already stored in the inter region motion information table (S1806), the oldest inter region merge candidate can be deleted (S1807), and an inter region merge candidate derived based on the current block can be added to the inter region motion information table (S1808).

[0199] A number of inter region merge candidates may be identified according to their respective indices. When an inter region merge candidate derived from the current block is added to the inter region motion information table, the inter region merge candidate may be assigned the lowest index (e.g., 0) and the indexes of the stored inter region merge candidates may be incremented by 1. In this case, if the inter region motion information table already has a maximum number of inter region merge candidates stored, the inter region merge candidate with the highest index is removed.

[0200] Alternatively, when an inter region merge candidate derived from the current block is added to the inter region motion information table, the inter region merge candidate may be assigned the largest index. For example, if the number of inter region merge candidates stored in the inter region motion information table is less than a maximum value, the inter region merge candidate may be assigned an index with a value equal to the number of stored inter region merge candidates. Alternatively, if the number of inter region merge candidates stored in the inter region motion information table is equal to a maximum value, the inter region merge candidate may be assigned an index obtained by subtracting 1 from the maximum value. Also, the inter region merge candidate with the smallest index is removed, and the indexes of the remaining stored inter region merge candidates are decremented by 1.

[0201] FIG. 19 is a diagram illustrating an example of updating the inter region merge candidate table.

[0202] Assume that an inter region merge candidate derived from the current block is added to an inter region merge candidate table and assigned a maximum index to the inter region merge candidate. Assume further that the inter region merge candidate table already stores a maximum number of inter region merge candidates.

[0203] When adding the inter region merge candidate HmvpCand[n+1] derived from the current block to the inter region merge candidate table HmvpCandList, the inter region merge candidate HmvpCand[0] with the smallest index may be deleted from the stored inter region merge candidates, and the indexes of the remaining inter region merge candidates may be decremented by 1. Also, the index of the inter region merge candidate HmvpCand[n+1] derived from the current block may be set to the maximum value (which is n in the example shown in FIG. 19).

[0204] If an inter region merge candidate that is the same as the inter region merge candidate derived based on the current block is stored (S1805), the inter region merge candidate derived based on the current block may not be added to the inter region motion information table (S1809).

[0205] Alternatively, the inter region merging candidate derived based on the current block can be added to the inter region motion information table, while the stored inter region merging candidate that is the same as the inter region merging candidate can be deleted, which has the same effect as updating the index of the stored inter region merging candidate.

[0206] FIG. 20 is a diagram showing an example in which the indices of the stored inter region merge candidates are updated.

[0207] If the index of a stored inter region merge candidate that is the same as the inter region merge candidate mvCand derived based on the current block is hIdx, the stored inter region merge candidate may be deleted, and the indexes of the inter region merge candidates whose indexes exceed hIdx may be decremented by 1. For example, the example shown in Figure 20 indicates that HmvpCand[2] that is the same as mvCand is deleted from the inter region motion information table HvmpCandList, and the indexes from HmvpCand[3] to HmvpCand[n] are decremented by 1.

[0208] Furthermore, the inter region merge candidate mvCand derived based on the current block can be added to the end of the inter region motion information table.

[0209] Alternatively, the index of the stored inter region merge candidate that is the same as the inter region merge candidate derived based on the current block may be updated, e.g., the index of the stored inter region merge candidate may be changed to a minimum or maximum value.

[0210] The motion information of the blocks included in a predetermined region can be set to one that cannot be added to the inter region motion information table. For example, the inter region merge candidate derived based on the motion information of the blocks included in the parallel merge region can be not added to the inter region motion information table. Since the coding / decoding order of the blocks included in the parallel merge region is not specified, it is inappropriate to use any of the motion information of the blocks when performing inter prediction on other blocks. This makes it possible to not add the inter region merge candidate derived based on the blocks included in the parallel merge region to the inter region motion information table.

[0211] When motion compensation prediction is performed using subblock units, inter region merging candidates may be derived based on motion information of a representative subblock among a plurality of subblocks included in a current block. For example, when subblock merging candidates are used for a current block, inter region merging candidates may be derived based on motion information of a representative subblock among the subblocks.

[0212] The motion vector of a subblock may be derived in the following order. First, one of the merging candidates included in the merging candidate list of the current block may be selected, and an initial shift vector (shVector) may be derived based on the motion vector of the selected merging candidate. Then, the initial shift vector may be added to the position (xSb, ySb) of a reference sample (e.g., the upper left sample or the sample at the middle position) of each subblock in the coding block to derive a shift subblock whose reference sample position is (xColSb, yColSb). The following Equation 1 shows a formula for deriving the shift subblock.

number

[0213] Next, the motion vector of the equivalent position block corresponding to the center position of the sub-block including (xColSb, yColSb) is set as the motion vector of the sub-block including (xSb, ySb).

[0214] The representative sub-block may refer to the sub-block containing the top-left sample or the central sample containing the current block.

[0215] FIG. 21 is a diagram showing the position of the representative sub-block.

[0216] 21(a) shows an example in which a sub-block located at the top left of a current block is set as a representative sub-block, and FIG. 21(b) shows an example in which a sub-block located at the center of a current block is set as a representative sub-block. When motion compensation prediction is performed in units of sub-blocks, an inter region merging candidate for a current block can be derived based on a motion vector of a sub-block including a sample at the top left of the current block or a sub-block including a sample at the center of the current block.

[0217] It is also possible to determine whether or not to use the current block as an inter region merging candidate based on the inter prediction mode of the current block. For example, a block encoded / decoded based on an affine motion model can be set as one that cannot be used as an inter region merging candidate. In this way, even if the current block is encoded / decoded by inter prediction, if the inter prediction mode of the current block is an affine prediction mode, the inter region motion information table is not updated based on the current block.

[0218] Alternatively, the inter region merging candidate may be derived based on at least one subblock vector of the subblocks included in the block encoded / decoded based on the affine motion model. For example, the subblock located at the top left, the subblock located in the center, or the subblock located at the top right of the current block may be used to derive the inter region merging candidate. Alternatively, the average value of the subblock vectors of multiple subblocks may be set as the motion vector of the inter region merging candidate.

[0219] Alternatively, the inter region merging candidate may be derived based on an average value of affine seed vectors of blocks encoded / decoded based on an affine motion model, for example, an average value of at least one of the first affine seed vector, the second affine seed vector, or the third affine seed vector of the current block may be set as the motion vector of the inter region merging candidate.

[0220] Alternatively, an inter region motion information table may be configured for each inter prediction mode. For example, at least one of an inter region motion information table for a block coded / decoded by intra block copy, an inter region motion information table for a block coded / decoded based on a translational motion model, or an inter region motion information table for a block coded / decoded based on an affine motion model may be defined. According to the inter prediction mode of the current block, one of the multiple inter region motion information tables may be selected.

[0221] FIG. 22 is a diagram showing an example of generating an inter region motion information table for each inter prediction mode.

[0222] When a block is coded / decoded based on a non-affine motion model, an inter region merge candidate mvCand derived based on the block may be added to the inter region non-affine motion information table HmvpCandList, whereas when a block is coded / decoded based on an affine motion model, an inter region merge candidate mvAfCand derived based on the block may be added to the inter region affine motion information table HmvpAfCandList.

[0223] The affine seed vector of a block can be stored in an inter region merge candidate derived from a block that has been coded / decoded based on an affine motion model, such that the inter region merge candidate can be used as a merge candidate for deriving the affine seed vector of the current block.

[0224] In addition to the inter region motion information table described above, another inter region motion information table can also be defined. In addition to the inter region motion information table described above (hereinafter referred to as the first inter region motion information table), a long-term motion information table (hereinafter referred to as the second inter region motion information table) can also be defined. Here, the long-term motion information table includes long-term merge candidates.

[0225] When the first inter region motion information table and the second inter region motion information table are all empty, the inter region merge candidate can be added to the second inter region motion information table first. The inter region merge candidate can be added to the first inter region motion information table only after the number of inter region merge candidates available in the second inter region motion information table reaches the maximum number.

[0226] Alternatively, one inter region merging candidate can be added to both the second inter region motion information table and the first inter region motion information table.

[0227] In this case, the configured second inter region motion information table may not be updated any more. Alternatively, the second inter region motion information table may be updated if the decoding region is equal to or greater than a predetermined ratio of the slice. Alternatively, the second inter region motion information table may be updated every N coding tree unit lines.

[0228] Meanwhile, the first inter region motion information table may be updated every time a block encoded / decoded by inter prediction is generated, but the inter region merge candidates added to the second inter region motion information table may be set not to be used to update the first inter region motion information table.

[0229] Information for selecting either the first inter region motion information table or the second inter region motion information table may be signaled via the bitstream. If the number of merge candidates included in the merge candidate list is less than a maximum value, the merge candidates included in the inter region motion information table indicated by the information may be added to the merge candidate list.

[0230] Alternatively, the inter region motion information table can be selected based on the size, shape, inter prediction mode, whether to bi-predict or not, whether to refine the motion vector, or whether to triangulate or not of the current block.

[0231] Alternatively, if the number of merge candidates included in the merge candidate list does not exceed the maximum number of merges even if an inter region merge candidate included in the first inter region motion information table is added, an inter region merge candidate included in the second inter region motion information table can be added to the merge candidate list.

[0232] FIG. 23 is a diagram showing an example of adding an inter region merging candidate included in the long-term motion information table to a merging candidate list.

[0233] If the number of merge candidates included in the merge candidate list is less than the maximum number, the inter region merge candidates included in the first inter region motion information table HmvpCandList can be added to the merge candidate list. If the number of merge candidates included in the merge candidate list is less than the maximum number even after adding the inter region merge candidates included in the first inter region motion information table to the merge candidate list, the inter region merge candidates included in the long term motion information table HmvpLTCandList can be added to the merge candidate list.

[0234] Table 2 shows the process of adding the inter-region merge candidates included in the long-term motion information table to the merge candidate list. [Table 2]

[0235] An inter region merge candidate may be set to include additional information in addition to motion information. For example, the size, shape, or block division information of the inter region merge candidate may be additionally stored. When configuring a merge candidate list for a current block, only inter region merge candidates whose size, shape, or division information is the same or similar to that of the current block may be used, or inter region merge candidates whose size, shape, or division information is the same or similar to that of the current block may be preferentially added to the merge candidate list.

[0236] Alternatively, an inter-region motion information table can be generated for each block size, shape, or partition information. Among the multiple inter-region motion information tables, an inter-region motion information table corresponding to the shape, size, or partition information of the current block can be used to generate a merge candidate list for the current block.

[0237] Alternatively, an inter region motion information table can be generated for each resolution of the motion vector. For example, if the motion vector of the current block has a quarter pixel resolution, the inter region merge candidate derived from the current block can be added to the inter prediction region quarter pixel motion information list. If the motion vector of the current block has a one integer pixel resolution, the inter region merge candidate derived from the current block can be added to the inter prediction region integer pixel motion information list. If the motion vector of the current block has a four integer pixel resolution, the inter region merge candidate derived from the current block can be added to the inter prediction region four integer pixel motion information list. According to the motion vector resolution of the encoding / decoding object block, one of the multiple inter motion information lists can be selected.

[0238] When the merge offset vector coding method is applied to the current block, the inter region merge candidates derived based on the current block may be added to the inter prediction region merge offset motion information table HmvpHMVDCandList instead of being added to the inter region motion information table HmvpCandList. In this case, the inter region merge candidates may include motion vector offset information of the current block. HmvpHMVDCandList may be used to derive offsets of the block to which the merge offset vector coding method is applied.

[0239] If the number of merge candidates included in the merge candidate list of the current block is not the maximum value, the inter region merge candidates included in the inter region motion information table may be added to the merge candidate list. The above adding process is performed in ascending or descending order of index. For example, the inter region merge candidate with the largest index may be added to the merge candidate list first.

[0240] When an inter region merge candidate included in the inter region motion information table is added to the merge candidate list, a redundancy check may be performed between the inter region merge candidate and the multiple merge candidates stored in the merge candidate list.

[0241] For example, Table 3 shows the process of adding inter-region merge candidates to the merge candidate list. [Table 3]

[0242] It is also possible to perform the redundancy check only on a portion of the inter region merge candidates included in the inter region motion information table, for example, only on the inter region merge candidates whose indexes are greater than or equal to a threshold or less than or equal to a threshold.

[0243] Alternatively, the redundancy check may be performed only on a portion of the merge candidates stored in the merge candidate list, for example, on merge candidates whose index is greater than or equal to a threshold or less than or equal to a threshold or derived from a block at a particular location, where the particular location may include at least one of the left neighboring block, the top neighboring block, the top right neighboring block, or the bottom left neighboring block of the current block.

[0244] FIG. 24 is a diagram showing an example of performing a redundancy check on only a portion of the merging candidates.

[0245] When adding an inter-region merge candidate HmvpCand[j] to the merge candidate list, a redundancy check can be performed on the inter-region merge candidate with the two merge candidates with the highest indices mergeCandList[NumMerge-2] and mergeCandList[NumMerge-1], where NumMerge denotes the number of available spatial and temporal merge candidates.

[0246] When a merging candidate identical to the first inter region merging candidate is found, the redundancy check with the second inter region merging candidate may be omitted when performing a redundancy check with the second inter region merging candidate.

[0247] FIG. 25 is a diagram showing an example in which a redundancy check is omitted for a specific merging candidate.

[0248] When an inter-region merge candidate HmvpCand[i] with index i is added to a merge candidate list, a redundancy check can be performed between the inter-region merge candidate and the merge candidates stored in the merge candidate list. In this case, if a merge candidate mergeCandList[j] that is the same as the inter-region merge candidate HmvpCand[i] is found, a redundancy check can be performed between the inter-region merge candidate HmvpCand[i-1] with index i-1 and the merge candidate without adding the inter-region merge candidate HmvpCand[i] to the merge candidate list. In this case, the redundancy check between the inter-region merge candidate HmvpCand[i-1] and the merge candidate mergeCandList[j] can be omitted.

[0249] For example, in the example shown in Figure 25, it is determined that HmvpCand[i] and mergeCandList[2] are the same. As a result, HmvpCand[i] is not added to the merge candidate list, and a redundancy check can be performed on HmvpCand[i-1]. In this case, the redundancy check between HmvpCand[i-1] and mergeCandList[2] can be omitted.

[0250] If the number of merge candidates included in the current block merge candidate list is less than the maximum value, in addition to the inter-region merge candidates, the list may also include at least one of pairwise merge candidates or zero merge candidates. A pairwise merge candidate refers to a merge candidate whose motion vector is the average value of the motion vectors of two or more merge candidates, and a zero merge candidate refers to a merge candidate whose motion vector is 0.

[0251] The current block's merge candidate list can be populated with merge candidates in the following order:

[0252] Spatial merge candidates - Temporal merge candidates - Inter-region merge candidates - (Inter-prediction affine merge candidates) - Pairwise merge candidates - Zero merge candidates Spatial merge candidates refer to merge candidates derived from at least one of adjacent or non-adjacent blocks, while temporal merge candidates refer to merge candidates derived from a previous reference image. Inter-prediction affine merge candidates refer to inter-region merge candidates derived from blocks encoded / decoded with an affine motion model.

[0253] The high-level motion vector prediction mode can also use the inter-region motion information table. For example, when the number of motion vector prediction candidates included in the motion vector prediction candidate list of the current block is less than the maximum value, the inter-region merge candidate included in the inter-region motion information table can be set as the motion vector prediction candidate of the current block. Specifically, the motion vector of the inter-region merge candidate can be set as the motion vector prediction candidate.

[0254] If any of the motion vector prediction candidates included in the motion vector prediction candidate list of the current block is selected, the selected candidate can be set as the motion vector predictor of the current block. Then, after decoding the motion vector residual value of the current block, the motion vector of the current block can be obtained by adding the motion vector prediction value and the motion vector residual value.

[0255] The motion vector prediction candidate list for the current block can be constructed in the following order:

[0256] Spatial motion vector prediction candidate-Temporal motion vector prediction candidate-Inter decoding region merge candidate-(Inter decoding region affine merge candidate)-Zero motion vector prediction candidate A spatial motion vector prediction candidate refers to a motion vector prediction candidate derived from at least one of adjacent blocks or non-adjacent blocks, and a temporal motion vector prediction candidate refers to a motion vector prediction candidate derived from a previous reference image. An inter-prediction affine merge candidate refers to an inter-prediction motion vector prediction candidate derived from a block encoded / decoded with an affine motion model. A zero motion vector prediction candidate refers to a candidate whose motion vector value is 0.

[0257] A merge processing area having a size larger than the coding block can be specified. The coding blocks included in the merge processing area can be processed in parallel without sequential coding / decoding. Here, not sequential coding / decoding means that the coding / decoding order is not specified. This allows the coding / decoding process of the blocks included in the merge processing area to be processed independently. Alternatively, the blocks included in the merge processing area can share a merge candidate. Here, the merge candidate can be derived based on the merge processing area.

[0258] Depending on the above characteristics, the merge processing region may also be called a parallel processing region, a shared merge region (SMR), or a merge estimation region (MER).

[0259] Merge candidates for the current block may be derived based on the coding blocks, except that if the current block is included in a parallel merge region of a size larger than the current block, candidate blocks included in the same parallel merge region as the current block may be set as ineligible for use as merge candidates.

[0260] FIG. 26 illustrates an example in which a candidate block included in the same parallel merge region as the current block is set so as not to be used as a merge candidate.

[0261] In the example shown in FIG. 26(a), when encoding / decoding CU5, blocks including reference samples adjacent to CU5 can be set as candidate blocks. In this case, candidate blocks X3 and X4 included in the same parallel merge region as CU5 can be set as those that cannot be used as merge candidates for CU5. On the other hand, candidate blocks X0, X1, and X2 that are not included in the same parallel merge region as CU5 can be set as those that can be used as merge candidates.

[0262] In the example shown in FIG. 26(b), when encoding / decoding CU8, the blocks including the reference samples adjacent to CU8 can be set as candidate blocks. In this case, the candidate blocks X6, X7, and X8 included in the same parallel merge region as CU8 can be set as those that cannot be used as merge candidates. On the other hand, the candidate blocks X5 and X9 that are not included in the same parallel merge region as CU5 can be set as those that can be used as merge candidates.

[0263] The parallel merge region may be square or non-square. Information for determining the parallel merge region may be signaled via the bitstream. The information may include at least one of information indicating a shape of the parallel merge region and information indicating a size of the parallel merge region. If the parallel merge region is non-square, at least one of information indicating a size of the parallel merge region, information indicating a width and / or height of the parallel merge region, or information indicating a ratio of the width and height of the parallel merge region may be signaled via the bitstream.

[0264] The size of the parallel merge region may be determined based on at least one of information signaled via the bitstream, an image resolution, a slice size, or a tile size.

[0265] When motion compensated prediction is performed on blocks included in the parallel merge region, inter region merge candidates derived based on the motion information of the blocks on which motion compensated prediction is performed can be added to the inter prediction motion information table.

[0266] However, when an inter region merge candidate derived from a block included in the parallel merge region is added to the inter region motion information table, if the actual encoding / decoding of another block in the parallel merge region is delayed from the above block, the inter region merge candidate derived from the above block may be used. In other words, it is necessary to eliminate the dependency between blocks when encoding / decoding a block included in the parallel merge region, but motion prediction compensation may be performed using motion information of another block included in the parallel merge region. In order to solve this problem, even if encoding / decoding of a block included in the parallel merge region is completed, the motion information of the block that has been encoded / decoded may not be added to the inter region motion information table.

[0267] Alternatively, when performing motion compensation prediction on blocks included in a parallel merge region, inter region merge candidates derived from the blocks may be added to the inter region motion information table in a predefined order, where the predefined order may be determined based on a scanning order of coding blocks in the parallel merge region or coding tree unit. The scanning order may be at least one of a raster scan, a horizontal scan, a vertical scan, or a zigzag scan. Alternatively, the predefined order may be determined based on the motion information of each block or the number of blocks having the same motion information.

[0268] Alternatively, inter region merge candidates containing unidirectional motion information can be added to the inter prediction region merge list before inter region merge candidates containing bidirectional motion information, and conversely, inter region merge candidates containing bidirectional motion information can be added to the inter region merge candidate table before inter region merge candidates containing unidirectional motion information.

[0269] Alternatively, the inter region merge candidates can be added to the inter region motion information table according to the most or least frequently used order within the parallel merge region or coding tree unit.

[0270] If the current block is included in a parallel merge region and the number of merge candidates included in the merge candidate list of the current block is less than the maximum number, the inter region merge candidates included in the inter region motion information table may be added to the merge candidate list. In this case, the inter region merge candidates derived from blocks included in the same parallel merge region as the current block may be set not to be added to the merge candidate list of the current block.

[0271] Alternatively, if the current block is included in a parallel merge region, the inter region merge candidates included in the inter region motion information table can be set not to be used, that is, the inter region merge candidates included in the inter region motion information table can be not added to the merge candidate list even if the number of merge candidates included in the merge candidate list of the current block is less than the maximum number.

[0272] An inter region motion information table for a parallel merge region or a coding tree unit may be configured. The inter region motion information table serves to temporarily store motion information of blocks included in the parallel merge region. In order to distinguish between a general inter region motion information table and an inter region motion information table for a parallel merge region or a coding tree unit, the inter region motion information table for a parallel merge region or a coding tree unit is called a temporary motion information table. Furthermore, the inter region merge candidates stored in the temporary motion information table are called temporary merge candidates.

[0273] FIG. 27 is a diagram showing a temporary movement information table.

[0274] A temporary motion information table for a coding tree unit or a parallel merge region may be configured. If motion compensation prediction is performed for a current block included in a coding tree unit or a parallel merge region, motion information of the block may not be added to the inter region motion information table HmvpCandList. Instead, a temporary merge candidate derived from the block may be added to the temporary motion information table HmvpMERCandList. That is, a temporary merge candidate added to the temporary motion information table may not be added to the inter region motion information table. As a result, the inter region motion information table may not include an inter region merge candidate derived based on motion information of a block included in a coding tree unit or a parallel merge region including the current block.

[0275] The maximum number of merge candidates that the temporary motion information table can contain can be set to be the same as the maximum number of merge candidates that the inter-region motion information table can contain, or the maximum number of merge candidates that the temporary motion information table can contain can be determined according to the size of a coding tree unit or a parallel merge region.

[0276] It may be set so that a current block included in a coding tree unit or a parallel merge region does not use the temporary motion information table of the coding tree unit or the parallel merge region. That is, if the number of merge candidates included in the merge candidate list of the current block is less than a maximum value, the inter region merge candidates included in the inter region motion information table may be added to the merge candidate list, and the temporary merge candidates included in the temporary motion information table may not be added to the merge candidate list. This makes it possible to not use motion information of other blocks included in the same coding tree unit or parallel merge region as the current block for motion compensation prediction of the current block.

[0277] Once the encoding / decoding of all blocks contained in a coding tree unit or parallel merge region is completed, the inter region motion information table and the temporary motion information table can be merged.

[0278] FIG. 28 is a diagram showing an example of merging the inter-prediction motion information table and the temporary motion information table.

[0279] Once the encoding / decoding of all blocks included in a coding tree unit or parallel merge region is completed, the temporary merge candidates included in the temporary motion information table can be updated to an inter-region motion information table, as shown in FIG.

[0280] In this case, the temporary merge candidates included in the temporary motion information table may be added to the inter-frame motion information table according to the order in which they were inserted into the temporary motion information table (ie, ascending or descending order of index value).

[0281] As another example, the temporary merge candidates included in the temporary motion information table can be added to the inter region motion information table according to a predefined order.

[0282] Here, the predefined order may be determined based on a scanning order of the coding blocks in the parallel merge region or the coding tree unit, where the scanning order may be at least one of a raster scan, a horizontal scan, a vertical scan, or a zigzag scan. Alternatively, the predefined order may be determined based on the motion information of each block or the number of blocks having the same motion information.

[0283] Alternatively, a temporary merge candidate with unidirectional motion information may be added to the inter prediction merge list before a temporary merge candidate with bidirectional motion information, while a temporary merge candidate with bidirectional motion information may be added to the inter region merge candidate list before a temporary merge candidate with unidirectional motion information.

[0284] Alternatively, the temporary merge candidates can be added to the inter-region motion information table according to the most or least frequently used order within a parallel merge region or coding tree unit.

[0285] When a temporary merge candidate included in the temporary motion information table is added to the inter region motion information table, a redundancy check may be performed on the temporary merge candidate. For example, if an inter region merge candidate that is the same as the temporary merge candidate included in the temporary motion information table is stored in the inter region motion information table, the temporary merge candidate may not be added to the inter region motion information table. In this case, a redundancy check may be performed on some of the inter region merge candidates included in the inter region motion information table. For example, a redundancy check may be performed on the inter region merge candidates whose index is greater than or equal to a threshold value. For example, if the temporary merge candidate is identical to an inter region merge candidate whose index is greater than or equal to a predefined value, the temporary merge candidate may not be added to the inter region motion information table.

[0286] Intra prediction is to predict the current block using reconstructed samples that have been encoded / decoded around the current block. In this case, intra prediction of the current block can use reconstructed samples before the in-loop filter is applied.

[0287] The intra prediction technique includes intra prediction based on a matrix and general intra prediction considering the direction with surrounding reconstruction samples. Information indicating the intra prediction technique of the current block can be signaled through a bitstream. The information can be a 1-bit flag. Alternatively, the intra prediction technique of the current block can be determined based on at least one of the position, size, shape, or intra prediction technique of the neighboring block of the current block. For example, if the current block is located across an image boundary, the current block can be set so that the matrix-based intra prediction is not applied to the current block.

[0288] Matrix-based intra prediction is a method of obtaining a prediction block of a current block based on a matrix product between a matrix stored in the encoder and the decoder and reconstructed samples around the current block. Information for specifying one of a plurality of stored matrices can be signaled via a bitstream. The decoder can determine the matrix used for intra prediction of the current block based on the information and the size of the current block.

[0289] General intra prediction is a method of obtaining a prediction block for a current block based on a non-directional intra prediction mode or a directional intra prediction mode. Hereinafter, a process of performing intra prediction based on general intra prediction will be described in detail with reference to the drawings.

[0290] FIG. 29 is a flowchart of an intra prediction method according to an embodiment of the present invention.

[0291] A reference sample line of a current block can be determined (S2901). The reference sample line refers to a set of reference samples included in a line K-th away from the top and / or left side of the current block. The reference samples can be derived from reconstructed samples that have been completely encoded / decoded around the current block.

[0292] Index information for identifying a reference sample line of the current block among the multiple reference sample lines can be signaled via the bitstream. The multiple reference sample lines may include at least one of the 1st row / column, the 2nd row / column, the 3rd row / column, or the 4th row / column on the top and / or left side of the current block. Table 4 shows the indexes assigned to each of the reference sample lines. In Table 4, it is assumed that the 1st row / column, the 2nd row / column, and the 4th row / column are used as candidates for reference sample lines. [Table 4]

[0293] The reference sample line of the current block may be determined based on at least one of the position, size, shape, or predictive coding mode of the neighboring block of the current block. For example, if the current block borders a boundary of an image, a tile, a slice, or a coding tree unit, the first reference sample line may be determined as the reference sample line of the current block.

[0294] The reference sample line may include a top reference sample located at the top of the current block and a left reference sample located to the left of the current block. The top reference sample and the left reference sample may be derived from reconstructed samples around the current block. The reconstructed samples may be in a state before an in-loop filter is applied.

[0295] FIG. 30 is a diagram showing reference samples included in each reference sample line.

[0296] According to the intra prediction mode of the current block, at least one of the reference samples belonging to the reference sample line may be used to obtain a prediction sample.

[0297] Next, the intra-prediction mode of the current block can be determined (S2902). For the intra-prediction mode of the current block, at least one of the non-directional intra-prediction mode or the directional intra-prediction mode can be determined as the intra-prediction mode of the current block. The non-directional intra-prediction mode includes Planar (planer) and DC, and the directional intra-prediction mode includes 33 or 65 modes from the left-lower diagonal to the right-upper diagonal.

[0298] FIG. 31 is a diagram showing intra prediction modes.

[0299] FIG. 31(a) shows 35 intra prediction modes, and FIG. 31(b) shows 67 intra prediction modes.

[0300] More or less intra-prediction modes may be defined than those shown in FIG.

[0301] A Most Probable Mode (MPM) may be set based on the intra-prediction modes of adjacent blocks adjacent to the current block. Here, the adjacent blocks may include a left adjacent block adjacent to the left side of the current block and a top adjacent block adjacent to the top of the current block. If the coordinates of the top-left sample of the current block are (0, 0), the left adjacent block may include a sample at a (-1, 0), (-1, H-1) or (-1, (H-1) / 2) position, where H represents the height of the current block. The top adjacent block may include a sample at a (0, -1), (W-1, -1) or ((W-1) / 2, -1) position, where W represents the width of the current block.

[0302] When encoding adjacent blocks using general intra prediction, the MPM can be derived based on the intra prediction modes of the adjacent blocks. Specifically, the intra prediction mode of the left adjacent block can be set to a variable candIntraPredModeA, and the intra prediction mode of the top adjacent block can be set to a variable candIntraPredModeB.

[0303] In this case, if the neighboring block is unavailable (e.g., if the neighboring block has not yet been coded / decoded or the neighboring block is located away from the image border), if the neighboring block is coded with matrix-based intra prediction and if the neighboring block is coded with inter prediction or if the neighboring block is included in a different coding tree unit from the current block, a variable candIntraPredModeX (where X is A or B) derived based on the intra prediction mode of the neighboring block can be set as the default mode. Here, the default mode may include at least one of a Planar mode, a DC mode, a vertical mode, or a horizontal mode.

[0304] Alternatively, when encoding adjacent blocks by intra prediction based on a matrix, an intra prediction mode corresponding to an index value for specifying one of the matrices can be set to candIntraPredModeX. For this purpose, a lookup table indicating a mapping relationship between index values ​​for specifying a matrix and intra prediction modes can be stored in advance in the encoder and decoder.

[0305] The MPM may be derived based on the variables candIntraPredModeA and candIntraPredModeB. The number of MPMs included in the MPM list may be predefined in the encoder and decoder. For example, the number of MPMs may be 3, 4, 5, or 6. Alternatively, information indicating the number of MPMs may be signaled via the bitstream. Alternatively, the number of MPMs may be determined based on at least one of the predictive coding modes of neighboring blocks, the size or shape of the current block.

[0306] In the embodiment described below, the number of MPMs is assumed to be three, and the three MPMs are referred to as MPM[0], MPM[1], and MPM[2]. If the number of MPMs is more than three, the MPMs may include three MPMs as described in the embodiment described below.

[0307] If candIntraPredA and candIntraPredB are equal and candIntraPredA is planar mode or DC mode, MPM[0] and MPM[1] can be set to planar mode and DC mode, respectively. MPM[2] can be set to vertical intra prediction mode, horizontal intra prediction mode or diagonal intra prediction mode. Diagonal intra prediction mode can be left-bottom diagonal intra prediction mode, left-top intra prediction mode or right-top intra prediction mode.

[0308] If candIntraPredA and candIntraPredB are equal and candIntraPredA is a directional intra prediction mode, MPM[0] can be set to be the same as candIntraPredA. MPM[1] and MPM[2] can be set as intra prediction modes similar to candIntraPredA. An intra prediction mode similar to candIntraPredA can be an intra prediction mode where the index difference value of candIntraPredA is ±1 or ±2. A modulo operation (%) and an offset amount can be used to derive an intra prediction mode similar to candIntraPredA.

[0309] If candIntraPredA and candIntraPredB are different, MPM[0] may be set equal to candIntraPredA, and MPM[1] may be set equal to candIntraPredB. In this case, if both candIntraPredA and candIntraPredB are non-directional intra prediction modes, MPM[2] may be set to a vertical intra prediction mode, a horizontal intra prediction mode, or a diagonal intra prediction mode. Alternatively, if at least one of candIntraPredA and candIntraPredB is a directional intra prediction mode, MPM[2] may be set to Planar, DC, or an intra prediction mode derived by adding or subtracting an offset value to the one of candIntraPredA or candIntraPredB with the greater value. Here, the offset value may be 1 or 2.

[0310] An MPM list including multiple MPMs may be generated, and information indicating whether the same MPM as the intra prediction mode of the current block is included in the MPM list may be signaled via a bitstream. The information is a 1-bit flag and may be referred to as an MPM flag. If the MPM flag indicates that the same MPM as the current block is included in the MPM list, index information for identifying one of the MPMs may be signaled via a bitstream. The MPM designated by the index information may be set as the intra prediction mode of the current block. If the MPM flag indicates that the same MPM as the current block is not included in the MPM list, residual mode information indicating one of the remaining intra prediction modes except the MPM may be signaled via a bitstream. The residual mode information indicates an index value corresponding to the intra prediction mode of the current block when reassigning indexes to the remaining intra prediction modes except the MPM. The decoder may sort the MPMs in ascending order and compare the residual mode information with the MPM to determine the intra prediction mode of the current block. For example, if the residual mode information is less than or equal to the MPM, one may be added to the residual mode information to derive the intra prediction mode of the current block.

[0311] Instead of setting the default mode to the MPM, information indicating whether the intra prediction mode of the current block is the default mode may be signaled via the bitstream. The information is a 1-bit flag, and the flag may be referred to as a default mode flag. The default mode flag may be signaled only if the MPM flag indicates that the same MPM as the current block is included in the MPM list. As mentioned above, the default mode may include at least one of Planar, DC, vertical mode, or horizontal mode. For example, if Planar is set as the default mode, the default mode flag may indicate whether the intra prediction mode of the current block is Planar or not. If the default mode flag indicates that the intra prediction mode of the current block is not the default mode, any of the MPMs indicated by the index information may be set as the intra prediction mode of the current block.

[0312] When multiple intra prediction modes are set as a default mode, index information indicating one of the default modes may be signaled, and the intra prediction mode of the current block may be set as the default mode indicated by the index information.

[0313] If the index of the reference sample line of the current block is not 0, the default mode can be set to be unavailable, so that if the index of the reference sample line is not 0, the value of the default mode flag can be set to a predefined value (i.e., false) without signaling the default mode flag.

[0314] When the intra-prediction mode of the current block is determined, a prediction sample of the current block can be obtained based on the determined intra-prediction mode (S2903).

[0315] When the DC mode is selected, a prediction sample for the current block can be generated based on the average value of the reference samples. Specifically, the values ​​of the samples in the entire prediction block can be generated based on the average value of the reference samples. At least one of the top reference sample located at the top of the current block and the left reference sample located to the left of the current block can be used to derive the average value.

[0316] According to the shape of the current block, the number or range of reference samples used to derive the average value may be different. For example, if the current block is a non-square block with a width greater than its height, only the top reference samples may be used to calculate the average value. On the other hand, if the current block is a non-square block with a width less than its height, only the left reference samples may be used to calculate the average value. That is, if the width and height of the current block are different, only the reference samples adjacent to the one with the longer length may be used to calculate the average value. Alternatively, based on the ratio of the width and height of the current block, it may be determined whether to calculate the average value using only the top reference samples or only the left reference samples.

[0317] When the planar mode is selected, a prediction sample can be obtained using a horizontal prediction sample and a vertical prediction sample. Here, a horizontal prediction sample is obtained based on a left reference sample and a right reference sample located on the same horizontal line as the prediction sample, and a vertical prediction sample is obtained using a top reference sample and a bottom reference sample located on the same vertical line as the prediction sample. Here, a reference sample adjacent to the upper right corner of the current block can be copied to generate a right reference sample, and a reference sample adjacent to the lower left corner of the current block can be copied to generate a bottom reference sample. A horizontal prediction sample can be obtained based on an additive operation of the left reference sample and the right reference sample, and a vertical prediction sample can be obtained based on an additive operation of the top reference sample and the bottom reference sample. In this case, a weight value to be assigned to each reference sample can be determined based on the position of the prediction sample. A prediction sample can be obtained based on an average operation or an additive operation of the horizontal prediction sample and the vertical prediction sample. When an additive operation is performed, a weight value to be assigned to the horizontal prediction sample and the vertical prediction sample can be determined based on the position of the prediction sample.

[0318] When a directional prediction mode is selected, a parameter representing a prediction direction (or prediction angle) of the selected directional prediction mode can be determined. Table 5 below shows the intra prediction parameter intraPredAng for each intra prediction mode. [Table 5]

[0319] Table 5 shows the intra direction parameters of each intra prediction mode having an index of 2 to 34 when 35 intra prediction modes are defined. When more than 33 directional intra prediction modes are defined, Table 5 can be further subdivided to set the intra direction parameters of each directional intra prediction mode.

[0320] After aligning the top reference sample and the left reference sample of the current block, a prediction sample can be obtained based on the value of the intra direction parameter, in which case, if the value of the intra direction parameter is a negative number, the left reference sample and the top reference sample can be aligned.

[0321] 32 and 33 are diagrams showing examples of a one-dimensional array in which reference samples are arranged in a line.

[0322] Fig. 32 shows an example of a vertical one-dimensional array in which reference samples are arranged in the vertical direction, and Fig. 33 shows an example of a horizontal one-dimensional array in which reference samples are arranged in the horizontal direction. The examples of Fig. 32 and Fig. 33 will be described assuming that 35 intra prediction modes are defined.

[0323] If the intra prediction mode index is any of 11 to 18, a horizontal one-dimensional array in which the top reference sample is rotated counterclockwise is applied, and if the intra prediction mode index is any of 19 to 25, a vertical one-dimensional array in which the left reference sample is rotated clockwise is applied. When arranging the reference samples in a row, the angle of the intra prediction mode may be taken into consideration.

[0324] Based on the intra direction parameters, reference sample decision parameters may be determined. The reference sample decision parameters may include a reference sample index for specifying the reference sample and a weight value parameter for determining a weight value to be applied to the reference sample.

[0325] The reference sample index iIdx and the weight parameter ifact can be obtained via the following Equations 2 and 3, respectively.

number

number

[0326] In Equation 2 and Equation 3, Pang represents an intra direction parameter. The reference sample specified by the reference sample index iIdx corresponds to an integer pixel.

[0327] In order to derive a prediction sample, at least one or more reference samples may be specified. Specifically, the position of the reference sample used to derive the prediction sample may be specified in consideration of the slope of the prediction mode. For example, the reference sample used to derive the prediction sample may be specified using a reference sample index iIdx.

[0328] In this case, if the slope of the intra prediction mode is not represented by one reference sample, a prediction sample may be generated by interpolating multiple reference samples. For example, if the slope of the intra prediction mode is a value between the slope between the prediction sample and the first reference sample and the slope between the prediction sample and the second reference sample, the prediction sample may be obtained by interpolating the first reference sample and the second reference sample. That is, if an angular line according to the intra prediction angle does not pass through a reference sample located at an integer pixel, the prediction sample may be obtained by interpolating adjacent reference samples to the left and right or above and below the position where the angular line passes.

[0329] Equation 4 below shows an example of obtaining a predicted sample based on a reference sample.

number

[0330] In Equation 4, P represents a predicted sample, and Ref_1D represents one of the one-dimensionally arranged reference samples. In this case, the position of the reference sample can be determined based on the position (x, y) of the predicted sample and the reference sample index iIdx.

[0331] When the gradient of the intra prediction mode can be represented by one reference sample, the weight parameter ifact can be set to 0. Therefore, Equation 4 can be simplified as Equation 5 below.

number

[0332] Intra prediction may be performed on the current block based on multiple intra prediction modes, for example, an intra prediction mode may be derived for each prediction sample, and the prediction sample may be derived based on the intra prediction mode assigned to each prediction sample.

[0333] Alternatively, an intra prediction mode may be derived for each region, and intra prediction may be performed for each region based on the intra prediction mode assigned to each region, where the region may include at least one sample. At least one of the size or shape of the region may be adaptively determined based on at least one of the size, shape, or intra prediction mode of the current block. Alternatively, the encoder and decoder may predefine at least one of the size or shape of the region, regardless of the size or shape of the current block.

[0334] Alternatively, intra prediction may be performed based on each of a plurality of intra predictions, and a final prediction sample may be derived based on an average operation or additive operation of a plurality of prediction samples obtained through a plurality of intra predictions. For example, intra prediction may be performed based on a first intra prediction mode to obtain a first prediction sample, and intra prediction may be performed based on a second intra prediction mode to obtain a second prediction sample. Then, a final prediction sample may be obtained based on an average operation or additive operation between the first prediction sample and the second prediction sample. In this case, the weight values ​​assigned to each of the first prediction sample and the second prediction sample may be determined taking into account at least one of whether the first intra prediction mode is a non-directional / directional prediction mode, whether the second intra prediction mode is a non-directional / directional prediction mode, or the intra prediction mode of a neighboring block.

[0335] The multiple intra-prediction modes can be a combination of non-directional intra-prediction modes and directional prediction modes, a combination of directional prediction modes, or a combination of non-directional prediction modes.

[0336] FIG. 34 is a diagram illustrating the angle formed between the directional intra prediction modes and a line parallel to the x-axis.

[0337] As shown in Fig. 34, the directional prediction modes may exist between the lower left diagonal direction and the upper right diagonal direction. In terms of the angle formed by the x-axis and the directional prediction modes, the directional prediction modes may exist between 45 degrees (lower left diagonal direction) and -135 degrees (upper right diagonal direction).

[0338] If the current block is non-square in shape, a prediction sample is derived using reference samples located on a diagonal line according to the intra prediction angle based on the intra prediction mode of the current block, which are located farther away from the reference sample than the reference samples closer to the prediction sample.

[0339] FIG. 35 is a diagram illustrating an example of obtaining prediction samples when the current block is non-square.

[0340] For example, as shown in FIG. 35(a), assume that the current block is a non-square with the width greater than the height, and the intra prediction mode of the current block is a directional intra prediction mode with an angle between 0 and 45 degrees. In this case, when deriving a prediction sample A near the right column of the current block, it may occur that, among the reference samples located on the angle mode according to the angle, a left reference sample L far from the prediction sample is used instead of a top reference sample T close to the prediction sample.

[0341] As another example, assume that the current block is a non-square with height greater than width, as shown in FIG. 35(b), and the intra prediction mode of the current block is a directional intra prediction mode with an angle between -90 degrees and -135 degrees. In this case, when deriving a prediction sample A near the bottom of the current block, it may occur that, among the reference samples located on the angle mode according to the angle, a top reference sample T far from the prediction sample is used instead of a left reference sample L close to the prediction sample.

[0342] To solve the above problem, if the current block is non-square, the intra prediction mode of the current block may be replaced with a backward intra prediction mode. This allows a directional prediction mode having a larger or smaller angle than the directional prediction mode shown in FIG. 31 to be used for the non-square block. Such a directional intra prediction mode may be defined as a wide-angle intra prediction mode. The wide-angle intra prediction mode refers to a directional intra prediction mode that does not belong to the range from 45 degrees to -135 degrees.

[0343] FIG. 36 is a diagram illustrating the wide-angle intra prediction mode.

[0344] In the example shown in FIG. 36, intra prediction modes with indexes of -1 to -14 and intra prediction modes with indexes of 67 to 80 indicate wide-angle intra prediction modes.

[0345] In Figure 36, 14 wide-angle intra-prediction modes (-1 to -14) with angles greater than 45 degrees and 14 wide-angle intra-prediction modes (67 to 80) with angles less than -135 degrees are illustrated, however, a greater or lesser number of wide-angle intra-prediction modes can be defined.

[0346] When using a wide-angle intra prediction mode, the length of the top reference sample may be set to 2W+1, and the length of the left reference sample may be set to 2H+1.

[0347] When using a wide-angle intra prediction mode, reference sample T can be used to predict sample A shown in Figure 35(a), and reference sample L can be used to predict sample A shown in Figure 35(b).

[0348] A total of 67+N intra prediction modes can be used by adding the existing intra prediction modes and N wide-angle intra prediction modes. For example, Table 6 shows intra direction parameters of intra prediction modes when 20 wide-angle intra prediction modes are defined. [Table 6]

[0349] The intra direction parameters may be set differently based on at least one of the size, shape, or reference sample line of the current block. For example, the intra direction parameters of a particular intra prediction mode may be different when the current block is square and when the current block is non-square. For example, the intra direction parameter intraPredAngle of intra prediction mode 15 may have a larger value when the current block is square than when the current block is non-square.

[0350] Alternatively, the intra direction parameter intraPredAngle of the intra prediction mode 75 may have a larger value when the index of the reference sample line of the current block is 1 or greater than when the index of the reference sample line of the current block is 0.

[0351] If the current block is non-square and the intra-prediction mode of the current block obtained in step S2902 belongs to a transform range, the intra-prediction mode of the current block may be transformed to a wide-angle intra-prediction mode. The transform range may be determined based on at least one of the size, shape, or ratio of the current block. Here, the ratio may indicate a ratio between the width and height of the current block.

[0352] If the current block is a non-square with width greater than height, a transform range may be set from the upper right diagonal intra prediction mode index (e.g., 66) to (the upper right diagonal intra prediction mode index -N), where N may be determined based on the ratio of the current block. If the intra prediction mode of the current block belongs to the transform range, the intra prediction mode may be transformed to a wide-angle intra prediction mode. The transform may be performed by subtracting a predefined value from the intra prediction mode, where the predefined value may be the total number of intra prediction modes excluding the wide-angle intra prediction mode (e.g., 67).

[0353] Based on the above embodiment, the intra prediction modes between 66 and 53 can be converted into wide-angle intra prediction modes between −1 and −14, respectively.

[0354] If the current block is a non-square with height greater than width, a transform range may be set from the left-bottom diagonal intra-prediction mode index (e.g., 2) to (left-bottom diagonal intra-prediction mode index+M), where M may be determined based on the proportion of the current block. If the intra-prediction mode of the current block belongs to the transform range, the intra-prediction mode may be transformed to a wide-angle intra-prediction mode. The transform may be performed by adding a predefined value to the intra-prediction mode, where the predefined value may be the total number of directional intra-prediction modes excluding the wide-angle intra-prediction mode (e.g., 65).

[0355] Based on the above embodiment, the intra prediction modes between No. 2 and No. 15 can be converted into wide-angle intra prediction modes between No. 67 and No. 80 respectively.

[0356] Hereinafter, the intra prediction modes that belong to the transform range are referred to as wide-angle intra alternative prediction modes.

[0357] The transform range can be determined based on the ratio of the current block. For example, Tables 7 and 8 show the transform ranges when 35 intra prediction modes excluding the wide-angle intra prediction mode are defined and when 67 intra prediction modes are defined, respectively. [Table 7] [Table 8]

[0358] As shown in Tables 7 and 8, depending on the proportion of the current block, the number of wide-angle intra alternative prediction modes included in the transform range may be different.

[0359] Using a wide-angle intra-prediction mode in addition to an existing intra-prediction mode may reduce coding efficiency due to the increased resources required to code the wide-angle intra-prediction mode. Thus, coding efficiency can be improved by coding an alternative intra-prediction mode for the wide-angle intra-prediction mode instead of coding the wide-angle intra-prediction mode as is.

[0360] For example, when the current block is encoded using wide-angle intra prediction mode 67, the wide-angle alternative intra prediction mode 2 of 67 may be encoded as the intra prediction mode of the current block. Furthermore, when the current block is encoded using wide-angle intra prediction mode −1, the wide-angle alternative intra prediction mode 66 of −1 may be encoded as the intra prediction mode of the current block.

[0361] The decoder may decode the intra-prediction mode of the current block and determine whether the decoded intra-prediction mode is included in a transform range. If the decoded intra-prediction mode is a wide-angle alternative intra-prediction mode, the decoder may transform the intra-prediction mode to a wide-angle intra-prediction mode.

[0362] Alternatively, if the current block is coded in wide-angle intra-prediction mode, the wide-angle intra-prediction mode can also be coded directly.

[0363] Encoding of intra prediction modes may be realized based on the MPM list. Specifically, when encoding a neighboring block in a wide-angle intra prediction mode, the MPM may be set based on a wide-angle alternative intra prediction mode corresponding to the wide-angle intra prediction mode. For example, when encoding a neighboring block in a wide-angle intra prediction mode, a variable candIntraPredX (X is A or B) may be set as the wide-angle alternative intra prediction mode.

[0364] When a prediction block is generated as a result of performing intra prediction, the prediction samples can be updated based on the position of each prediction sample included in the prediction block. The above-mentioned updating method can be called a sample position-based intra weighted prediction method (or Position Dependent Prediction Combination (PDPC)).

[0365] It may be determined whether to use PDPC considering the intra prediction mode of the current block, the reference sample line of the current block, the size of the current block, or the color component. For example, PDPC may be used if the intra prediction mode of the current block is at least one of the Planar mode, the DC mode, the vertical mode, the horizontal mode, the mode with a smaller index value than the vertical direction, or the mode with a larger index value than the horizontal direction. Alternatively, PDPC may be used only if at least one of the width or height of the current block is greater than 4. Alternatively, PDPC may be used only if the index of the reference image line of the current block is 0. Alternatively, PDPC may be used only if the index of the reference image line of the current block is greater than or equal to a predefined value. Alternatively, PDPC may be used only for the luma component. Alternatively, it may be determined whether to use PDPC depending on whether two or more of the listed conditions are satisfied.

[0366] As another example, information indicating whether the PDPC applies can be signaled via the bitstream.

[0367] When a prediction sample is obtained through an intra prediction sample, a reference sample used to correct the prediction sample can be determined based on the position of the obtained prediction sample. For convenience of explanation, in the embodiment described below, the reference sample used to correct the prediction sample is called a PDPC reference sample. Furthermore, a prediction sample obtained through intra prediction is called a first prediction sample, and a prediction sample obtained by correcting the first prediction sample is called a second prediction sample.

[0368] FIG. 37 is a diagram showing the application of PDPC.

[0369] At least one PDPC reference sample may be used to correct the first predicted sample, which may include at least one of a reference sample adjacent to an upper left corner of the current block, a top reference sample located at the top of the current block, or a left reference sample located to the left of the current block.

[0370] At least one of the reference samples belonging to the reference sample line of the current block may be set as a PDPC reference sample. Alternatively, at least one of the reference samples belonging to the reference sample line of index 0 may be set as a PDPC reference sample, regardless of the reference sample line of the current block. For example, even if the first predicted sample is obtained using the reference sample included in the reference sample line of index 1 or index 2, the second predicted sample may be obtained using the reference sample included in the reference sample line of index 0.

[0371] The number or positions of PDPC reference samples used to correct the first predicted sample may be determined taking into account at least one of the intra prediction mode of the current block, the size of the current block, the shape of the current block, or the position of the first predicted sample.

[0372] For example, when the intra prediction mode of the current block is Planar mode or DC mode, the top reference sample and the left reference sample can be used to obtain the second prediction sample. In this case, the top reference sample can be a reference sample perpendicular to the first prediction sample (e.g., a reference sample with the same x coordinate), and the left reference sample can be a reference sample horizontal to the first prediction sample (e.g., a reference sample with the same y coordinate).

[0373] If the intra prediction mode of the current block is a horizontal intra prediction mode, the top reference sample may be used to obtain the second prediction sample, in which case the top reference sample may be a reference sample perpendicular to the first prediction sample.

[0374] If the intra prediction mode of the current block is a vertical intra prediction mode, the left reference sample may be used to obtain the second prediction sample, in which case the left reference sample may be a reference sample horizontal to the first prediction sample.

[0375] If the intra prediction mode of the current block is a left-bottom diagonal intra prediction mode or an right-top diagonal intra prediction mode, the second prediction sample can be obtained based on the top-left reference sample, the top reference sample, and the left reference sample. The top-left reference sample may be a reference sample adjacent to the top-left corner of the current block (e.g., a reference sample at a (-1, -1) position). The top reference sample may be a reference sample located in the upper right diagonal direction of the first prediction sample, and the left reference sample may be a reference sample located in the lower left diagonal direction of the first prediction sample.

[0376] In summary, if the position of the first prediction sample is (x, y), R(-1, -1) can be set as the top-left reference sample, R(x+y+1, -1) or R(x, -1) can be set as the top reference sample, and R(-1, x+y+1) or R(-1, y) can be set as the left reference sample.

[0377] As another example, the position of the left reference sample or the top reference sample may be determined taking into account at least one of the shape of the current block or whether a wide-angle intra mode is applied.

[0378] Specifically, when the intra prediction mode of the current block is a wide-angle intra prediction mode, reference samples that are separated from the reference samples diagonally opposite the first predicted sample by the offset value may be set as PDPC reference samples, for example, the top reference sample R(x+y+k+1,-1) and the left reference sample R(-1,x+y-k+1) may be set as PDPC reference samples.

[0379] In this case, the offset value k can be determined based on the wide-angle intra prediction mode. Equations 6 and 7 show examples of deriving the offset value based on the wide-angle intra prediction mode.

number

number

[0380] The second predicted sample may be determined based on an additive operation between the first predicted sample and the PDPC reference sample. For example, the second predicted sample may be obtained based on the following Equation 8.

number

[0381] In Equation 8, RL represents the left reference sample, RT represents the top reference sample, and RTL represents the top-left reference sample. pred(x, y) represents a predicted sample at the (x, y) position. wL represents a weight value assigned to the left reference sample, wT represents a weight value assigned to the top reference sample, and wTL represents a weight value assigned to the top-left reference sample. The weight value assigned to the first predicted sample can be derived by subtracting the weight value assigned to the reference sample from the maximum value. For convenience of explanation, the weight value assigned to the PDPC reference sample is referred to as the PDPC weight value.

[0382] The weight value assigned to each reference sample may be determined based on at least one of the intra prediction mode of the current block or the position of the first prediction sample.

[0383] For example, at least one of wL, wT, or wTL and at least one of the x-axis coordinate value or the y-axis coordinate value of the predicted sample may be in a proportional or inverse proportional relationship. Alternatively, at least one of wL, wT, or wTL and at least one of the width or height of the current block may be in a proportional or inverse proportional relationship.

[0384] If the intra prediction mode of the current block is the DC mode, the PDPC weight value may be determined as shown in Equation 9 below.

number

[0385] In Equation 9, x and y represent the position of the first predicted sample.

[0386] In Equation 9, the variable shift used in the bit shift operation can be determined based on the width or height of the current block. For example, the variable shift can be derived based on Equation 10 or 11 below.

number

number

[0387] Alternatively, the variable shift can be derived taking into account the intra direction parameters of the current block.

[0388] The number and type of parameters used to derive the variable shift may be determined differently depending on the intra prediction mode of the current block. For example, if the intra prediction mode of the current block is a planar mode, a DC mode, a vertical mode, or a horizontal mode, the variable shift may be derived using the width and height of the current block as shown in Equation 10 or Equation 11. If the intra prediction mode of the current block is an intra prediction mode with a larger index than the intra prediction mode in the vertical direction, the variable shift may be derived using the height and intra direction parameters of the current block. If the intra prediction mode of the current block is an intra prediction mode with a smaller index than the intra prediction mode in the horizontal direction, the variable shift may be derived using the width and intra direction parameters of the current block.

[0389] If the intra prediction mode of the current block is the planar mode, the value of wTL may be set to 0. wL and wT may be derived according to Equation 12 below.

number

[0390] If the intra prediction mode of the current block is a horizontal intra prediction mode, wT may be set to 0 and wTL and wL may be set to the same. On the other hand, if the intra prediction mode of the current block is a vertical intra prediction mode, wL may be set to 0 and wTL and wT may be set to the same.

[0391] If the intra prediction mode of the current block is an intra prediction mode in the upper right direction having an index value larger than the intra prediction mode in the vertical direction, the PDPC weight value can be derived as shown in Equation 13 below.

number

[0392] Meanwhile, if the intra prediction mode of the current block is an intra prediction mode in the left-bottom direction having an index value smaller than that of the horizontal intra prediction mode, the PDPC weight value can be derived as shown in Equation 14 below.

number

[0393] As in the above embodiment, the PDPC weight value can be determined based on the position x and y of the predicted sample.

[0394] As another example, the weight value assigned to each PDPC reference sample may be determined on a sub-block basis, and the prediction samples included in the sub-block may share the same PDPC weight value.

[0395] In the encoder and decoder, the size of the sub-block, which is the basic unit used to determine the weighting value, can be predefined. For example, a weighting value can be determined for each sub-block of size 2×2 or 4×4.

[0396] Alternatively, the size, shape, or number of sub-blocks may be determined according to the size or shape of the current block. For example, the coding block may be divided into four sub-blocks regardless of the size of the coding block. Alternatively, the coding block may be divided into four or sixteen sub-blocks based on the size of the coding block.

[0397] Alternatively, the size, shape, or number of sub-blocks can be determined based on the intra-prediction mode of the current block. For example, if the intra-prediction mode of the current block is a horizontal mode, N columns (or N rows) can be set as one sub-block, and conversely, if the intra-prediction mode of the current block is a vertical mode, N rows (or N columns) can be set as one sub-block.

[0398] Equations 15 to 17 show an example of determining the PDPC weight value of a sub-block of size 2×2. Equation 15 shows an example when the intra prediction mode of the current block is the DC mode.

number

[0399] In Equation 15, K may be a value determined based on the size of the sub-block or the intra-prediction mode.

[0400] Equation 16 shows an example where the intra prediction mode of the current block is an intra prediction mode in the upper right direction, which has an index value greater than the intra prediction mode in the vertical direction.

number

[0401] Equation 17 illustrates an example where the intra prediction mode of the current block is a bottom-left intra prediction mode having an index value smaller than the horizontal intra prediction mode.

number

[0402] In Equations 15 to 17, x and y indicate the position of the reference sample within the sub-block. The reference sample may be a sample located at the top left of the sub-block, a sample located at the center of the sub-block, or a sample located at the bottom right of the sub-block.

[0403] Equations 18 to 20 show an example of determining the PDPC weight value of a sub-block of size 4×4. Equation 18 shows an example when the intra prediction mode of the current block is the DC mode.

number

[0404] Equation 19 shows an example where the intra prediction mode of the current block is an upper-right intra prediction mode with an index value greater than the vertical intra prediction mode.

number

[0405] Equation 20 shows an example where the intra prediction mode of the current block is a bottom-left intra prediction mode having an index value smaller than the horizontal intra prediction mode.

number

[0406] In the above embodiment, the PDPC weight value is determined by taking into consideration the position of the first predicted sample or the predicted sample included in the sub-block. The PDPC weight value may be determined by further taking into consideration the shape of the current block.

[0407] For example, for DC mode, the derivation of the PDPC weight values ​​may differ depending on whether the current block is a non-square block with its width greater than its height or its height greater than its width.

[0408] Equation 21 shows an example of deriving PDPC weight values ​​when the current block is a non-square with its width greater than its height, and Equation 22 shows an example of deriving PDPC weight values ​​when the current block is a non-square with its height greater than its width.

number

number

[0409] If the current block is non-square, the wide-angle intra prediction mode may be used to predict the current block. Thus, even when the wide-angle intra prediction mode is applied, the PDPC may be applied to update the first prediction sample.

[0410] When wide-angle intra prediction is applied to the current block, the PDPC weighting value may be determined taking into account the shape of the coding block.

[0411] For example, if the current block is a non-square block with a width greater than its height, a top reference sample located to the upper right of the first predicted sample may be closer to the first predicted sample than a left reference sample located to the lower left of the first predicted sample based on the position of the first predicted sample. Thus, when correcting the first predicted sample, a weight value applied to the top reference sample may be set to be greater than a weight value applied to the left reference sample.

[0412] Meanwhile, when the current block is a non-square block with its height greater than its width, the left reference sample located at the lower left of the first predicted sample may be closer to the first predicted sample than the top reference sample located at the upper right of the first predicted sample based on the position of the first predicted sample. Thus, when correcting the first predicted sample, the weight applied to the left reference sample may be set to be greater than the weight applied to the top reference sample.

[0413] Equation 23 shows an example of deriving the PDPC weight value when the intra prediction mode of the current block is a wide-angle intra prediction mode with an index greater than 66.

number

[0414] Equation 24 shows an example of deriving the PDPC weight value when the intra prediction mode of the current block is a wide-angle intra prediction mode with an index less than 0.

number

[0415] The PDPC weight value may also be determined based on the ratio of the current block. The ratio of the current block indicates the ratio of the width to the height of the current block, and may be defined as Equation 25 below.

number

[0416] The method of deriving the PDPC weight values ​​may be variably determined based on the intra-prediction mode of the current block.

[0417] For example, Equation 26 and Equation 27 show an example of deriving a PDPC weight value when the intra prediction mode of the current block is a DC mode. Specifically, Equation 26 is an example of the current block being a non-square shape whose width is greater than its height, and Equation 27 is an example of the current block being a non-square shape whose height is greater than its width.

number

number

[0418] Equation 28 shows an example of deriving the PDPC weight value when the intra prediction mode of the current block is a wide-angle intra prediction mode with an index greater than 66.

number

[0419] Equation 29 shows an example of deriving the PDPC weighting value when the intra prediction mode of the current block is a wide-angle intra prediction mode with an index less than 0.

number

[0420] A residual image can be derived by subtracting a predicted image from an original image. In this case, when the residual image is in the frequency domain, removing high-frequency components from the frequency components does not significantly degrade the quality of the video. Thus, by reducing the value of the high-frequency components and setting the value of the high-frequency components to 0, it is possible to improve the compression efficiency without causing obvious visual distortion. Reflecting the above characteristics, the current block can be transformed to decompose the residual image into two-dimensional frequency components. The transformation can be performed using a transform method such as a Discrete Cosine Transform (DCT) or a Discrete Sine Transform (DST).

[0421] DCT uses a cosine transform to decompose (or transform) the residual image into two-dimensional frequency components, while DST uses a cosine transform to decompose (or transform) the residual image into two-dimensional frequency components. The frequency components resulting from the transformation of the residual image can be represented by a base image. For example, if a DCT transformation is performed on a block of size N×N, N2 base pattern components can be obtained. Through the transformation, the size of each base pattern component contained in the block of size N×N can be obtained. Based on the transformation technique used, the size of the base pattern components can be called DCT coefficients or DST coefficients.

[0422] The DCT transform technique is primarily used to transform images with a high distribution of non-zero low-frequency components. The DST transform technique is primarily used for images with a high distribution of high-frequency components.

[0423] Transformation techniques other than DCT or DST may also be used to transform the residual image.

[0424] Hereinafter, transforming the residual image into two-dimensional frequency components is referred to as two-dimensional image transform. Furthermore, the size of the basic pattern component obtained by the transform result is referred to as transform coefficient. For example, the transform coefficient may refer to a DCT coefficient or a DST coefficient. When the primary transform and the secondary transform described later are applied simultaneously, the transform coefficient may represent the size of the basic pattern component generated by the result of the secondary transform.

[0425] The transform technique can be determined on a block-by-block basis. The transform technique can be determined based on at least one of the predictive coding mode of the current block, the size of the current block, or the shape of the current block. For example, if the current block is coded in intra-prediction mode and the size of the current block is smaller than N×N, the transform technique DST can be used to perform the transform. On the other hand, if the above conditions are not met, the transform technique DCT can be used to perform the transform.

[0426] It is also possible to not perform a 2D image transform on some blocks in the residual image. Not performing a 2D image transform can be called a transform skip. When a transform skip is applied, quantization can be applied to the residual values ​​on which no transform has been performed.

[0427] A DCT or DST may be used to transform a current block and then retransform the transformed current block, in which case the transformation based on the DCT or DST may be defined as the primary transformation, and retransforming the block to which the primary transformation was applied may be defined as the secondary transformation.

[0428] The primary transform may be performed using any of a number of possible transform cores, for example, the primary transform may be performed using any of the DCT2, DCT8, or DCT7.

[0429] Different transform cores can also be used for the horizontal and vertical directions, and information indicating the combination of horizontal and vertical transform cores can also be signaled via the bitstream.

[0430] The execution units of the primary transform and the secondary transform may be different. For example, the primary transform may be performed on an 8×8 block, and the secondary transform may be performed on a 4×4 size sub-block of the transformed 8×8 block. In this case, the transform coefficients of the remaining area where the secondary transform is not performed may be set to 0.

[0431] Alternatively, a primary transform can be performed on the 4x4 block, and a secondary transform can be performed on an 8x8 sized region that contains the transformed 4x4 block.

[0432] Information indicating whether or not to perform a secondary transformation can be signaled via the bitstream.

[0433] Alternatively, the secondary transform may be performed based on whether the horizontal and vertical transform cores are the same. For example, the secondary transform may be performed only if the horizontal and vertical transform cores are the same. Alternatively, the secondary transform may be performed only if the horizontal and vertical transform cores are different.

[0434] Alternatively, a secondary transform may be possible only if the horizontal transform and the vertical transform use a predefined transform core, for example, a secondary transform may be possible if a DCT2 transform core is used for the horizontal transform and the vertical transform.

[0435] Alternatively, the determination of whether to perform the secondary transform can be based on the number of non-zero transform coefficients of the current block. For example, the secondary transform can be set not to be used if the non-zero transform coefficients of the current block are less than or equal to a threshold, and the secondary transform can be set to be used if the non-zero transform coefficients of the current block are greater than the threshold. The secondary transform can also be set to be used only if the current block is coded using intra prediction.

[0436] Based on the shape of the current block, the size and shape of the sub-blocks on which the secondary transformation is performed can be determined.

[0437] 38 and 39 show the sub-blocks in which the secondary transformation is performed.

[0438] If the current block is a square, the primary transform may be performed, and then the secondary transform may be performed on the upper left sub-block of the current block, which is N×N in size. For example, if the current block is an 8×8 coding block, the primary transform may be performed on the current block, and then the secondary transform may be performed on the upper left sub-block of the current block, which is 4×4 in size (see FIG. 38).

[0439] If the current block is non-square, with the width being more than four times larger than the height, then the primary transform may be performed, and then the secondary transform may be performed on the top-left sub-block of the current block, which is (kN)×(4kN). For example, if the current block is non-square, with a size of 16×4, then the secondary transform may be performed on the top-left sub-block of the current block, which is 2×8 (see FIG. 39(a)).

[0440] If the current block is non-square, with the height being more than four times greater than the width, then the primary transform can be performed, and then the secondary transform can be performed on the top-left sub-block of the current block, which is (4kN)×(kN). For example, if the current block is non-square, with a size of 16×4, then the secondary transform can be performed on the top-left sub-block of the current block, which is 2×8 (see FIG. 39(b)).

[0441] In the decoder, the inverse transform (second inverse transform) of the secondary transform is performed, and the inverse transform (first inverse transform) of the primary transform is performed on the result of the second inverse transform. As a result of the second inverse transform and the first inverse transform, the residual signal of the current block can be obtained.

[0442] Quantization is used to reduce the energy of a block, and the quantization process involves dividing the transform coefficients by a certain constant, which can be derived by a quantization parameter, which can be defined as a value between 1 and 63.

[0443] If the encoder performs transform and quantization, the decoder can obtain a residual block through inverse quantization and inverse transform, and the decoder can add the prediction block and the residual block to obtain a reconstructed block of the current block.

[0444] Information indicating the transform type of the current block may be signaled via the bitstream, which may be index information tu_mts_idx indicating one of a combination of horizontal and vertical transform types.

[0445] Based on the transform type candidates specified by the index information tu_mts_idx, the vertical transform core and the horizontal transform core can be determined. Tables 9 and 10 show combinations of transform types according to tu_mts_idx. [Table 9] [Table 10]

[0446] The transform type can be determined as either DCT2, DST7, DCT8, or transform skip. Alternatively, the transform core alone can be used to configure the candidate transform type combinations, excluding transform skip.

[0447] When using Table 9, if tu_mts_idx is 0, a transform skip may be applied horizontally and vertically. If tu_mts_idx is 1, a DCT2 may be applied horizontally and vertically. If tu_mts_idx is 3, a DCT8 may be applied horizontally and a DCT7 may be applied vertically.

[0448] When using Table 10, if tu_mts_idx is 0, DCT2 can be applied horizontally and vertically. If tu_mts_idx is 1, transform skip can be applied horizontally and vertically. If tu_mts_idx is 3, DCT8 can be applied horizontally and DCT7 can be applied vertically.

[0449] It may be determined whether to code the index information based on at least one of the size, shape, or number of non-zero coefficients of the current block. For example, if the number of non-zero coefficients is less than or equal to a threshold, a default transform type may be applied to the current block without signaling the index information. Here, the default transform type may be DST7. Alternatively, the default mode may be different depending on the size, shape, or intra prediction mode of the current block.

[0450] The threshold value may be determined based on the size or shape of the current block. For example, if the size of the current block is less than or equal to 32×32, the threshold value may be set to 2, and if the size of the current block is greater than 32×32 (e.g., the current block is a coding block of 32×64 or 64×32 size), the threshold value may be set to 4.

[0451] A plurality of lookup tables can be stored in advance in the encoder / decoder, and the plurality of lookup tables may differ in at least one of index values ​​assigned to the transform type combination candidates, the types of the transform type combination candidates, or the number of the transform type combination candidates.

[0452] The lookup table for the current block may be selected based on at least one of the size, shape, predictive coding mode, intra prediction mode, whether to apply a secondary transform, or whether to apply a transform skip to a neighboring block of the current block.

[0453] For example, if the size of the current block is less than or equal to 4x4 or if the current block is encoded using inter prediction, the lookup table of Table 9 can be used, and if the size of the current block is greater than 4x4 or if the current block is encoded using intra prediction, the lookup table of Table 10 can be used.

[0454] Alternatively, information indicating which of multiple look-up tables is used can be signaled via the bitstream, and the decoder can select the look-up table for the current block based on the information.

[0455] As another example, the index assigned to the candidate combination of transform types may be adaptively determined based on at least one of the size, shape, predictive coding mode, intra prediction mode, whether to apply a secondary transform, or whether to apply a transform skip to a neighboring block of the current block. For example, if the size of the current block is 4×4, the index value assigned to the transform skip may be smaller than the index value assigned to the transform skip if the size of the current block is greater than 4×4. Specifically, if the size of the current block is 4×4, the transform skip may be assigned an index of 0, and if the current block is greater than 4×4 and less than or equal to 16×16, the transform skip may be assigned an index greater than 0 (e.g., index 1). If the current block is greater than 16×16, the index of the transform skip may be assigned a maximum value (e.g., 5).

[0456] Alternatively, if the current block is coded with inter prediction, the transform skip may be assigned an index of 0. If the current block is coded with intra prediction, the transform skip may be assigned an index greater than 0 (e.g., index 1).

[0457] Alternatively, if the current block is a 4×4 sized block coded with inter prediction, the transform skip may be assigned an index of 0. On the other hand, if the current block is not coded with inter prediction or is larger than 4×4, the transform skip may be assigned an index greater than 0 (e.g., index 1).

[0458] It is also possible to define and use a combination candidate of a transform type different from the combination candidates of the transform types listed in Tables 9 and 10. For example, a transform skip may be applied to one of the horizontal transform or the vertical transform, and a combination candidate of a transform type in which a transform core such as DCT7, DCT8 or DST2 is applied may be used for the other. In this case, it may be determined whether to use a transform skip as a transform type candidate in the horizontal or vertical direction based on at least one of the size (e.g., width and / or height), shape, predictive coding mode, or intra prediction mode of the current block.

[0459] Alternatively, information indicating whether a particular transform type candidate can be used can be signaled through a bitstream. For example, a flag indicating whether horizontal and vertical transform skips can be used as a transform type candidate can be signaled. Based on the flag, it can be determined whether the particular transform type combination candidate is included in the multiple transform type combination candidates.

[0460] Alternatively, information indicating whether a particular transform type candidate is applied to the current block may be signaled via the bitstream. For example, a flag cu_mts_flag indicating whether DCT2 is applied in the horizontal and vertical directions may be signaled. If the value of cu_mts_flag is 1, DCT2 may be set as the transform core in the vertical and horizontal directions. If the value of cu_mts_flag is 0, DCT8 or DST7 may be set as the transform core in the vertical and horizontal directions. Alternatively, if the value of cu_mts_flag is 0, information tu_mts_idx specifying one of multiple transform type combination candidates may be signaled.

[0461] If the current block is a non-square block with width greater than height or height greater than width, the coding of cu_mts_flag may be omitted and the value of cu_mts_flag may be considered to be 0.

[0462] The number of available transform type combination candidates may be set differently depending on the size, shape, or intra-prediction mode of the current block. For example, if the current block is square, three or more transform type combination candidates may be used, and if the current block is non-square, two transform type combination candidates may be used. Alternatively, if the current block is square, only transform type combination candidates with different horizontal and vertical transform types may be used.

[0463] If the current block can use three or more transform type combination candidates, index information tu_mts_idx indicating one of the transform type combination candidates can be signaled. On the other hand, if the current block can use two transform type combination candidates, flag mts_flag indicating one of the transform type combination candidates can be signaled. Table 11 below shows the process of encoding information for specifying transform type combination candidates according to the shape of the current block. [Table 11]

[0464] Depending on the shape of the current block, the indices of the candidate transform type combinations may be rearranged (or reordered). For example, the indices assigned to the candidate transform type combinations when the current block is square may be different from the indices assigned to the candidate transform type combinations when the current block is non-square. For example, when the current block is square, the transform type combinations may be selected based on Table 12 below, and when the current block is non-square, the transform type combinations may be selected based on Table 13 below. [Table 12] [Table 13]

[0465] A transform type may be determined based on the number of horizontal non-zero coefficients or the number of vertical non-zero coefficients of the current block, where the number of horizontal non-zero coefficients indicates the number of non-zero coefficients contained in 1×N (where N is the width of the current block), and the number of vertical non-zero coefficients indicates the number of non-zero coefficients contained in N×1 (where N is the height of the current block). If the maximum value of the horizontal non-zero coefficients is less than or equal to a threshold, a primary transform type may be applied horizontally, and if the maximum value of the horizontal non-zero coefficients is greater than the threshold, a secondary transform type may be applied horizontally. If the maximum value of the vertical non-zero coefficients is less than or equal to a threshold, a primary transform type may be applied vertically, and if the maximum value of the vertical non-zero coefficients is greater than the threshold, a secondary transform type may be applied vertically.

[0466] FIG. 40 is a diagram for explaining an example of determining the transformation type of the current block.

[0467] For example, if the current block is encoded using intra prediction and the maximum value of the horizontal non-zero coefficients of the current block is less than or equal to 2 (see FIG. 40(a)), the horizontal transform type can be determined as DST7.

[0468] If the current block is encoded with intra prediction and the maximum value of the vertical non-zero coefficients of the current block is greater than 2 (see FIG. 40(b)), DCT2 or DCT8 can be determined as the vertical transform type.

[0469] Residual coefficients may be coded in a transform unit or a sub-transform unit, where the residual coefficients refer to transform coefficients generated by a transform, transform skip coefficients generated by a transform skip, or quantized coefficients generated by quantizing the transform coefficients or coefficients.

[0470] A transform unit can refer to a block on which a primary or secondary transform is performed. A sub-transform unit refers to a block smaller than a transform unit. For example, a sub-transform unit can be a block of size 4×4, 2×8 or 8×2.

[0471] At least one of the sizes or shapes of the sub-transformation units may be determined based on the size or shape of the current block. For example, if the current block is a non-square with width greater than height, the sub-transformation unit may also be set to a non-square with width greater than height (e.g., 8×2), and if the current block is a non-square with height greater than width, the sub-transformation unit may also be set to a non-square with height greater than width (e.g., 2×8). If the current block is a square, the sub-transformation unit may also be set to a square (e.g., 4×4).

[0472] If the current block includes multiple sub-transform units, the sub-transform units may be sequentially encoded / decoded. The residual coefficients may be encoded using entropy coding, such as arithmetic coding. Hereinafter, a method for encoding / decoding the residual coefficients will be described in detail with reference to the drawings.

[0473] FIG. 41 is a flow chart of a method for encoding residual coefficients.

[0474] In this embodiment, it is assumed that the current block includes one or more sub-transform units. It is further assumed that the size of the sub-transform units is 4 × 4. However, this embodiment can be applied as it is when the size of the sub-transform units is different or the shape of the sub-transform units is different.

[0475] It may be determined whether a non-zero coefficient exists in the current block (S4101). A non-zero coefficient refers to a residual coefficient whose absolute value is greater than 0. Information indicating whether a non-zero coefficient exists in the current block may be coded and signaled. For example, the information may be a 1-bit flag CBF (Coded Block Flag).

[0476] If a non-zero coefficient exists in the current block, it may be determined whether a non-zero coefficient exists in each sub-transform unit (S4102). Information indicating whether a non-zero coefficient exists in each sub-transform unit may be coded and signaled. For example, the information may be a 1-bit flag, a Coded SubBlock Flag (CSBF). The sub-transform units may be coded according to a selected scan order.

[0477] When the sub-transform unit has a non-zero coefficient, the residual coefficients in the sub-transform unit may be arranged one-dimensionally in order to encode the residual coefficients of the sub-transform unit (S4103). The residual coefficients may be arranged one-dimensionally according to a selected scanning order.

[0478] The scan order may include at least one of a diagonal scan, a horizontal scan, a vertical scan, or an inverse scan thereof.

[0479] 42 and 43 are diagrams showing the arrangement order of residual coefficients in different scanning orders.

[0480] 42(a) to 42(c) show diagonal scan, horizontal scan, and vertical scan, and FIGS. 43(a) to 43(c) show the inverse scans of these.

[0481] The residual coefficients may be arranged in one dimension according to a selected scanning order.

[0482] The scan order may be determined by considering at least one of the size, shape, intra prediction mode, transform core used for the primary transform, or whether to apply a secondary transform for the current block. For example, if the current block is a non-square block with a width greater than its height, the residual coefficients may be coded using an inverse horizontal scan. On the other hand, if the current block is a non-square block with a height greater than its width, the residual coefficients may be coded using an inverse vertical scan.

[0483] Alternatively, the rate distortion optimization (RDO) of each of multiple scanning orders can be calculated, and the scanning order with the lowest RDO can be determined as the scanning order for the current block. In this case, information indicating the scanning order of the current block can be coded and signaled.

[0484] The scan order candidates available for coding the transform skip coefficients may be different from the scan order candidates available for coding the transform coefficients. For example, the reverse direction of the scan order candidates available for coding the transform coefficients may be set as the scan order candidates available for coding the transform skip coefficients.

[0485] For example, if the transform coefficients are coded using either an inverse diagonal scan, an inverse horizontal scan, or an inverse vertical scan, the transform skip coefficients can be coded using either a diagonal scan, a horizontal scan, or a vertical scan.

[0486] Thereafter, the position of the non-zero coefficient located last in the scan order in the transform block can be coded (S4103). The x-axis and y-axis positions of the last non-zero coefficient can be coded, respectively.

[0487] FIG. 44 is a diagram showing an example of encoding the position of the last non-zero coefficient.

[0488] As shown in Figure 44, in the case of diagonal scanning, the residual coefficient located at the upper right corner of the transform block may be set as the last non-zero coefficient. The x-axis coordinate LastX of the residual coefficient is 3, and the y-axis coordinate LastY is 0. LastX may be separated into a prefix part last_sig_coeff_x_prefix and a suffix part last_sig_coeff_x_suffix and coded. LastY may also be separated into a prefix part last_sig_coeff_y_prefix and a suffix part last_sig_coeff_y_suffix and coded.

[0489] When the position of the last non-zero coefficient is determined, for each residual coefficient preceding the last non-zero coefficient in the scanning order, information indicating whether the residual coefficient is a non-zero coefficient or not may be coded (S4104). The information may be a 1-bit flag, sig_coeff_flag. If the residual coefficient is 0, the value of the non-zero coefficient flag sig_coeff_flag may be set to 0, and if the residual coefficient is 1, the value of the non-zero coefficient flag sig_coeff_flag may be set to 1. For the last non-zero coefficient, coding of the non-zero coefficient flag sig_coeff_flag may be omitted. If coding of the non-zero coefficient flag sig_coeff_flag is omitted, the residual coefficient is considered to be non-zero.

[0490] For transform skip coefficients, information about the location of the last non-zero coefficient may not be coded, in which case a non-zero coefficient flag sig_coeff_flag may be coded for all transform skip coefficients in the block.

[0491] Alternatively, information indicating whether information on the position of the last non-zero coefficient is encoded can be signaled. The information can be a 1-bit flag. The decoder can determine whether to decode information on the position of the last non-zero coefficient based on the value of the flag. If the position of the last non-zero coefficient is not decoded, the non-zero coefficient flag can be decoded for all transform skip coefficients. On the other hand, if the position of the last non-zero coefficient is decoded, information indicating whether the transform skip coefficient is a non-zero coefficient can be encoded for each transform skip coefficient that precedes the last non-zero coefficient in the scanning order.

[0492] The DC component of the transform coefficient or transform skip coefficient may not be set to zero. Here, the DC component may indicate the last sample in the scanning order or the top left sample of the block. For the DC component, the residual coefficient may omit the coding of the non-zero coefficient flag sig_coeff_flag. When the coding of the non-zero coefficient flag sig_coeff_flag is omitted, the residual coefficient is considered to be non-zero.

[0493] If the residual coefficient is not 0, the information indicating the absolute value of the residual coefficient and the information indicating the sign of the residual coefficient can be coded (S4105). The coding process of the information indicating the absolute value of the residual coefficient will be described in more detail with reference to FIG.

[0494] The encoding of the residual coefficients may be performed sequentially for all residual coefficients after the last non-zero coefficient and for each of the sub-transform units included in the current block (S4106, S4107).

[0495] FIG. 45 is a flow chart of a process for encoding the absolute values ​​of the residual coefficients.

[0496] If the absolute value of the residual coefficient exceeds 0, the residual coefficient or information indicating whether the residual coefficient is even or odd can be coded (S4501). Here, the residual coefficient can be defined as a value obtained by subtracting 1 from the absolute value of the residual coefficient. The information can be a 1-bit parity flag. For example, a value of 0 in the syntax element par_level_flag indicates that the residual coefficient or the residual coefficient is even, and a value of 1 in the syntax element par_level_flag indicates that the residual coefficient or the residual coefficient is odd. The value of the syntax element par_level_flag can be determined according to the following Equation 30.

number

[0497] In Equation 30, Tcoeff represents the residual coefficient, and abs() represents the absolute value function.

[0498] The residual coefficient or the residual coefficient may be divided by 2, or a bit shift operation may be applied to the residual coefficient or the residual coefficient to derive an adjusted residual coefficient (S4502). Specifically, the quotient of the residual coefficient divided by 2 may be set as the adjusted residual coefficient, or the value of the residual coefficient shifted by 1 to the right may be set as the adjusted residual coefficient.

[0499] For example, the adjusted residual coefficients can be derived based on the following Equation 31.

number

[0500] In Equation 31, ReRemLevel represents the adjusted residual coefficient, and RemLevel represents the residual coefficient.

[0501] Then, information indicating the size of the scaled residual coefficient may be encoded. The information indicating the size of the scaled residual coefficient may include information indicating whether the scaled residual coefficient is greater than N, where N may be an integer such as 1, 2, 3, 4, and so on.

[0502] For example, information indicating whether the value of the adjusted residual coefficient is greater than 1 can be coded (S4503). The information can be a 1-bit flag rem_abs_gt1_flag.

[0503] If the residual coefficient is 0 or 1, the value of rem_abs_gt1_flag can be set to 0. That is, if the residual coefficient is 1 or 2, the value of rem_abs_gt1_flag can be set to 0. In this case, if the residual coefficient is 0, par_level_flag can be set to 0, and if the residual coefficient is 1, par_level_flag can be set to 1.

[0504] If the adjusted residual coefficient is equal to or greater than 2 (S4504), the value of rem_abs_gt1_flag may be set to 1, and information indicating whether the value of the adjusted residual coefficient is greater than 2 may be encoded (S4505). The information may be a 1-bit flag rem_abs_gt2_flag.

[0505] If the residual coefficient is 1 or 2, the value of rem_abs_gt2_flag can be set to 0. That is, if the residual coefficient is 3 or 4, the value of rem_abs_gt2_flag can be set to 0. In this case, if the residual coefficient is 2, par_level_flag can be set to 0, and if the residual coefficient is 3, par_level_flag can be set to 1.

[0506] If the adjusted residual coefficient exceeds 2, residual value information obtained by subtracting 2 from the adjusted residual coefficient can be coded (S4506). That is, after subtracting 5 from the absolute value of the residual coefficient, the resultant value can be divided by 2 and coded as the residual value information.

[0507] Although not shown, the residual coefficient may be further coded using a flag indicating whether the adjusted residual coefficient exceeds 3 (e.g., rem_abs_gt3_flag) or a flag indicating whether the adjusted residual coefficient exceeds 4 (rem_abs_gt4_flag). In this case, the residual value may be set to a value obtained by subtracting the maximum value from the adjusted residual coefficient. The maximum value indicates the maximum N value of rem_abs_gtN_flag.

[0508] Instead of a flag that compares the value of the adjusted residual coefficient with a specified value, a flag that compares the absolute value of the adjusted coefficient or the residual coefficient with a specified value can also be used. For example, instead of rem_abs_gt1_flag, gr2_flag can be used, which indicates whether the absolute value of the residual coefficient exceeds 2, and instead of rem_abs_gt2_flag, gr4_flag can be used, which indicates whether the absolute value of the residual coefficient exceeds 4.

[0509] Table 14 shows a simplified process for encoding the residual coefficients using syntax elements. [Table 14]

[0510] As in the example above, the residual coefficients may be coded using par_level_flag and at least one rem_abs_gtN_flag (where N is an integer such as 1, 2, 3, 4, etc.).

[0511] rem_abs_gtN_flag indicates whether the residual coefficient exceeds 2N. For residual coefficients of 2N-1 or 2N, rem_abs_gt(N-1)_flag is set to true and rem_abs_gtN_flag is set to false. For residual coefficients of 2N-1, the par_level_flag value can be set to 0, and for residual coefficients of 2N, the par_level_flag value can be set to 1. In other words, residual coefficients of 2N or less can be coded using rem_abs_gtN_flag and par_level_flag.

[0512] For residual coefficients that are 2MAX+1 or more, residual value information can be coded by dividing the difference between the residual coefficient and 2MAX by 2. Here, MAX indicates the maximum value of N. For example, when using rem_abs_gt1_flag and rem_abs_gt2_flag, MAX can be 2.

[0513] The decoder may also decode the residual coefficients according to the order shown in Figure 45. Specifically, the decoder may determine the position of the last nonzero coefficient and decode sig_coeff_flag for each residual coefficient that precedes the last nonzero coefficient in the scan order.

[0514] If sig_coeff_flag is true, the par_level_flag of the residual coefficient may be decoded. Furthermore, the rem_abs_gt1_flag of the residual coefficient may be decoded. In this case, the rem_abs_gtN_flag may be further decoded based on the value of rem_abs_gt(N-1)_flag. For example, if the value of rem_abs_gt(N-1)_flag is 1, the rem_abs_gtN_flag may be decoded. For example, if the value of rem_abs_gt1_flag of the residual coefficient is 1, the rem_abs_gt2_flag of the residual coefficient may be further analyzed. If the value of rem_abs_gt(MAX)_flag is 1, the residual value information may be decoded.

[0515] When the value of rem_abs_gtN_flag of the residual coefficient is 0, the value of the residual coefficient can be determined to be 2N-1 or 2N based on the value of par_level_flag. Specifically, when par_level_flag is 0, the residual coefficient can be set to 2N-1, and when par_level_flag is 1, the residual coefficient can be set to 2N.

[0516] For example, when rem_abs_gt1_flag is 0, the absolute value of the residual coefficient can be set to 1 or 2 based on the par_level_flag value. Specifically, when the par_level_flag value is 0, the absolute value of the residual coefficient is 1, and when the par_level_flag value is 1, the absolute value of the residual coefficient is 2.

[0517] For example, when rem_abs_gt2_flag is 0, the absolute value of the residual coefficient can be set to 3 or 4 based on the par_level_flag value. Specifically, when the par_level_flag value is 0, the absolute value of the residual coefficient is 3, and when the par_level_flag value is 1, the absolute value of the residual coefficient is 4.

[0518] When decoding the residual value information, the residual coefficient can be set to 2(MAX+R)-1 or 2(MAX+R) based on the value of par_level_flag. Here, R indicates the value indicated by the residual value information. For example, when par_level_flag is 0, the residual coefficient can be set to 2(MAX+R)-1, and when par_level_flag is 1, the residual coefficient can be set to 2(MAX+R). For example, when MAX is 2, the residual coefficient can be derived based on the following Equation 32.

number

[0519] Based on the example shown in FIG. 45, when the residual coefficient is coded, the parity flags of all non-zero coefficients need to be coded. For example, even if the residual coefficient is 1, par_level_flag and rem_abs_gt1_flag need to be coded. The coding method as described above causes a problem of increasing the number of bits required to code the residual coefficient whose absolute value is 1. In order to avoid such a problem, the information indicating whether the residual coefficient exceeds 1 can be coded first, and then, if the residual coefficient exceeds 1, the parity flag can be coded.

[0520] FIG. 46 is a flowchart of a process for encoding the absolute values ​​of the residual coefficients.

[0521] For non-zero residual coefficients, information gr1_flag indicating whether the absolute value of the residual coefficient exceeds 1 can be coded (S4601). If the residual coefficient is 1, gr1_flag can be set to 0, and if the residual coefficient exceeds 1, gr1_flag can be set to 1.

[0522] If the absolute value of the residual coefficient exceeds 1, a parity flag indicating whether the residual coefficient or the residual coefficient is even or odd can be coded (S4602, S4603). Here, the residual coefficient can be set to a value obtained by subtracting 2 from the residual coefficient. For example, par_level_flag can be derived based on the following equation 33.

number

[0523] The residual coefficient or residual coefficient may be divided by 2 or the residual coefficient or residual coefficient may be shifted one bit to the right to derive an adjusted residual coefficient, and information indicating whether the adjusted residual coefficient exceeds 1 may be coded (S4604). For example, for a residual coefficient with gr1_flag set to 1, rem_abs_gt1_flag indicating whether the adjusted residual coefficient exceeds 1 may be coded.

[0524] If the residual coefficient is 0 or 1, the value of rem_abs_gt1_flag can be set to 0. That is, if the residual coefficient is 2 or 3, the value of rem_abs_gt1_flag can be set to 0. In this case, if the adjusted residual coefficient is 0, par_level_flag can be set to 0, and if the residual coefficient is 1, par_level_flag can be set to 1.

[0525] If the adjusted residual coefficient is greater than or equal to 2 (S4605), the value of rem_abs_gt1_flag is set to 1, and information indicating whether the adjusted residual coefficient exceeds 2 may be encoded (S4606). For example, for residual coefficients with rem_abs_gt1_flag set to 1, rem_abs_gt2_flag indicating whether the adjusted residual coefficient exceeds 2 may be encoded.

[0526] If the residual coefficient is 2 or 3, the value of rem_abs_gt2_flag can be set to 1. That is, if the residual coefficient is 4 or 5, the value of rem_abs_gt2_flag can be set to 0. In this case, if the residual coefficient is 2, par_level_flag can be set to 0, and if the residual coefficient is 3, par_level_flag can be set to 1.

[0527] If the adjusted residual coefficient exceeds 2 (S4607), the residual value information obtained by subtracting 2 from the adjusted residual coefficient can be coded (S4608). That is, after subtracting 6 from the absolute value of the residual coefficient, the resultant value can be divided by 2 and coded as the residual value information.

[0528] The residual coefficient may be further coded using a flag indicating whether the adjusted residual coefficient exceeds 3 (e.g., rem_abs_gt3_flag) or a flag indicating whether the adjusted residual coefficient exceeds 4 (rem_abs_gt4_flag). In this case, the residual value may be set to a value obtained by subtracting the maximum value from the adjusted residual coefficient. The maximum value indicates the maximum N value of rem_abs_gtN_flag.

[0529] Instead of a flag that compares the value of the adjusted residual coefficient with a specified value, a flag that compares the absolute value of the adjusted coefficient or the residual coefficient with a specified value can also be used. For example, instead of rem_abs_gt1_flag, gr3_flag can be used, which indicates whether the absolute value of the residual coefficient exceeds 3, and instead of rem_abs_gt2_flag, gr5_flag can be used, which indicates whether the absolute value of the residual coefficient exceeds 5.

[0530] Table 15 shows a simplified process for encoding the residual coefficients using syntax elements. [Table 15]

[0531] The decoder may also decode the residual coefficients according to the order shown in Figure 46. Specifically, the decoder may determine the position of the last nonzero coefficient and decode sig_coeff_flag for each residual coefficient that precedes the last nonzero coefficient in the scan order.

[0532] If sig_coeff_flag is true, the gr1_flag of the residual coefficient may be decoded. If gr1_flag is 0, the absolute value of the residual coefficient is determined to be 1, and if gr1_flag is 1, the par_level_flag of the residual coefficient may be decoded. Then, the rem_abs_gt1_flag of the residual coefficient may be decoded. In this case, the rem_abs_gtN_flag may be further decoded based on the value of rem_abs_gt(N-1)_flag. For example, if the value of rem_abs_gt(N-1)_flag is 1, the rem_abs_gtN_flag may be decoded. For example, if the value of rem_abs_gt1_flag of the residual coefficient is 1, the rem_abs_gt2_flag of the residual coefficient may be further analyzed. If the value of rem_abs_gt(MAX)_flag is 1, the residual value information can be decoded.

[0533] When the value of rem_abs_gtN_flag of the residual coefficient is 0, the value of the residual coefficient can be determined to be 2N-1 or 2N based on the value of par_level_flag. Specifically, when par_level_flag is 0, the residual coefficient can be set to 2N, and when par_level_flag is 1, the residual coefficient can be set to 2N+1.

[0534] For example, if rem_abs_gt1_flag is 0, the absolute value of the residual coefficient can be set to 2 or 3 based on the par_level_flag value. Specifically, if the par_level_flag value is 0, the absolute value of the residual coefficient is 2, and if the par_level_flag value is 1, the absolute value of the residual coefficient is 3.

[0535] For example, if rem_abs_gt2_flag is 0, the absolute value of the residual coefficient can be set to 4 or 5 based on the par_level_flag value. Specifically, if the par_level_flag value is 0, the absolute value of the residual coefficient is 4, and if the par_level_flag value is 1, the absolute value of the residual coefficient is 5.

[0536] When decoding the residual value information, the residual coefficient can be set to 2(MAX+R) or 2(MAX+R)+1 based on the value of par_level_flag, where R indicates the value of the residual value information. For example, if par_level_flag is 0, the residual coefficient can be set to 2(MAX+R), and if par_level_flag is 1, the residual coefficient can be set to 2(MAX+R)+1.

[0537] As another example, a parity flag may be encoded only if the value of a residual coefficient exceeds 2. For example, after encoding information indicating whether the value of a residual coefficient exceeds 1 and information indicating whether the value of a residual coefficient exceeds 2, if it is determined that the value of a residual coefficient exceeds 2, a parity flag may be encoded for the residual coefficient.

[0538] FIG. 47 is a flow chart of a process for encoding the absolute values ​​of the residual coefficients.

[0539] For non-zero residual coefficients, information gr1_flag indicating whether the absolute value of the residual coefficient exceeds 1 can be coded (S4701). If the residual coefficient is 1, gr1_flag can be set to 0, and if the residual coefficient exceeds 1, gr1_flag can be set to 1.

[0540] For residual coefficients whose absolute value exceeds 1, information gr2_flag indicating whether the absolute value of the residual coefficient exceeds 2 can be coded (S4702, S4703). If the residual coefficient is 2, gr2_flag can be set to 0, and if the residual coefficient exceeds 2, gr2_flag can be set to 1.

[0541] If the absolute value exceeds 2, a parity flag indicating whether the residual coefficient or the residual coefficient is even or odd can be coded (S4704, S4705). Here, the residual coefficient can be set to a value obtained by subtracting 3 from the residual coefficient. For example, par_level_flag can be derived based on the following Equation 34.

number

[0542] The residual coefficient or residual coefficients may be divided by 2 or shifted one bit to the right to derive an adjusted residual coefficient, and information indicating whether the adjusted residual coefficient exceeds 1 may be encoded. For example, for a residual coefficient with gr1_flag set to 1, rem_abs_gt1_flag indicating whether the adjusted residual coefficient exceeds 1 may be encoded (S4706).

[0543] If the residual coefficient is 0 or 1, the value of rem_abs_gt1_flag can be set to 0. That is, if the residual coefficient is 3 or 4, the value of rem_abs_gt1_flag can be set to 0. In this case, if the adjusted residual coefficient is 0, par_level_flag can be set to 0, and if the residual coefficient is 1, par_level_flag can be set to 1.

[0544] If the adjusted residual coefficient is 2 or greater (S4707), the value of rem_abs_gt1_flag is set to 1, and information indicating whether the adjusted residual coefficient exceeds 2 can be encoded (S4708). For example, for residual coefficients where rem_abs_gt1_flag is 1, rem_abs_gt2_flag indicating whether the adjusted residual coefficient exceeds 2 can be encoded.

[0545] If the residual coefficient is 2 or 3, the value of rem_abs_gt2_flag can be set to 1. That is, if the residual coefficient is 5 or 6, the value of rem_abs_gt2_flag can be set to 0. In this case, if the residual coefficient is 2, par_level_flag can be set to 0, and if the residual coefficient is 3, par_level_flag can be set to 1.

[0546] If the adjusted residual coefficient exceeds 2, the residual value obtained by subtracting 2 from the adjusted residual coefficient can be coded (S4709, S4710). In other words, 7 is subtracted from the absolute value of the residual coefficient, and the resultant value is divided by 2 and coded as the residual value.

[0547] The residual coefficient may be further coded using a flag indicating whether the adjusted residual coefficient exceeds 3 (e.g., rem_abs_gt3_flag) or a flag indicating whether the adjusted residual coefficient exceeds 4 (rem_abs_gt4_flag). In this case, the residual value may be set to a value obtained by subtracting the maximum value from the adjusted residual coefficient. The maximum value indicates the maximum N value of rem_abs_gtN_flag.

[0548] Instead of a flag that compares the value of the adjusted residual coefficient with a specified value, a flag that compares the absolute value of the adjusted coefficient or the residual coefficient with a specified value can also be used. For example, instead of rem_abs_gt1_flag, gr4_flag can be used, which indicates whether the absolute value of the residual coefficient exceeds 4, and instead of rem_abs_gt2_flag, gr6_flag can be used, which indicates whether the absolute value of the residual coefficient exceeds 6.

[0549] The decoder may also decode the residual coefficients in the same order as shown in Figure 47. Specifically, the decoder may determine the position of the last nonzero coefficient and decode sig_coeff_flag for each residual coefficient that precedes the last nonzero coefficient in the scan order.

[0550] If sig_coeff_flag is true, gr1_flag of the residual coefficient may be decoded. If gr1_flag is 0, the absolute value of the residual coefficient may be determined to be 1, and if gr1_flag is 1, gr2_flag may be decoded. If gr2_flag is 0, the absolute value of the residual coefficient may be determined to be 2, and if gr2_flag is 1, par_level_flag of the residual coefficient may be decoded. Then, rem_abs_gt1_flag of the residual coefficient may be decoded. In this case, rem_abs_gtN_flag may be further decoded based on the value of rem_abs_gt(N-1)_flag. For example, if the value of rem_abs_gt(N-1)_flag is 1, rem_abs_gtN_flag may be decoded. For example, if the value of rem_abs_gt1_flag of a residual coefficient is 1, the rem_abs_gt2_flag of the residual coefficient can be further analyzed. If the value of rem_abs_gt(MAX)_flag is 1, the residual value information can be decoded.

[0551] When the value of rem_abs_gtN_flag of the residual coefficient is 0, the value of the residual coefficient can be determined to be 2N+1 or 2(N+1) based on the value of par_level_flag. Specifically, when par_level_flag is 0, the residual coefficient can be set to 2N+1, and when par_level_flag is 1, the residual coefficient can be set to 2(N+1).

[0552] For example, if rem_abs_gt1_flag is 0, the absolute value of the residual coefficient can be set to 3 or 4 based on the par_level_flag value. Specifically, if the par_level_flag value is 0, the absolute value of the residual coefficient is 3, and if the par_level_flag value is 1, the absolute value of the residual coefficient is 4.

[0553] For example, when rem_abs_gt2_flag is 0, the absolute value of the residual coefficient can be set to 5 or 6 based on the par_level_flag value. Specifically, when the par_level_flag value is 0, the absolute value of the residual coefficient is 5, and when the par_level_flag value is 1, the absolute value of the residual coefficient is 6.

[0554] When decoding the residual value information, the residual coefficient can be set to 2(MAX+R) or 2(MAX+R)+1 based on the value of par_level_flag, where R indicates the value of the residual value information. For example, if par_level_flag is 0, the residual coefficient can be set to 2(MAX+R), and if par_level_flag is 1, the residual coefficient can be set to 2(MAX+R)+1.

[0555] At least one of the number or type of comparison flags for comparing the adjusted residual coefficients with the specified value can be determined based on at least one of the size, shape, whether or not to perform transform skipping, transform core, the number of non-zero coefficients, or the position of the last non-zero coefficient of the current block. For example, when encoding transform coefficients, only rem_abs_gt1_flag can be used. On the other hand, when encoding transform skip coefficients, rem_abs_gt1_flag and rem_abs_gt2_flag can be used.

[0556] Alternatively, within a sub-transform unit of size 4x4, the number of rem_abs_gt1_flag can be set to a maximum of 8, and the number of rem_abs_gt2_flag can be set to a maximum of 1. Alternatively, if the number of non-zero coefficient flags is (16-N), the number of rem_abs_gt1_flag can be set to a maximum of 8+(N / 2), and the number of rem_abs_gt2_flag can be set to a maximum of 1+(N-(N / 2)).

[0557] When a reconstructed block of a current block is obtained, information loss generated in the quantization and encoding process can be reduced through in-loop filtering. The in-loop filter may include at least one of a deblocking filter, a sample adaptive offset filter (SAO) or an adaptive loop filter (ALF). Hereinafter, the reconstructed block before the in-loop filter is applied is referred to as a first reconstructed block, and the reconstructed block after the in-loop filter is applied is referred to as a second reconstructed block.

[0558] At least one of a deblocking filter, SAO, or ALF may be applied to the first reconstructed block to obtain a second reconstructed block, in which case SAO or ALF may be applied after the deblocking filter is applied.

[0559] A deblocking filter is used to reduce image quality degradation (blocking artifacts) that occur at block boundaries when quantization is performed block by block. To apply the deblocking filter, a blocking strength (BS) between a first reconstruction block and an adjacent reconstruction block can be determined.

[0560] FIG. 48 is a flow chart of a process for determining block strength.

[0561] In the example shown in Figure 48, P represents the first reconstructed block, and Q represents the adjacent reconstructed block, where the adjacent reconstructed block may be the block adjacent to the left or top of the current block.

[0562] The example shown in Figure 48 illustrates that the block strength is determined taking into account the predictive coding modes of P and Q, whether non-zero transform coefficients are included, whether inter prediction is performed using the same reference picture, or whether the motion vector difference value is greater than or equal to a threshold.

[0563] Based on the block strength, it can be determined whether to apply a deblocking filter. For example, if the block strength is 0, no filtering may be performed.

[0564] SAO is used to reduce ringing artifacts that occur when quantization is performed in the frequency domain. SAO can be performed by adding or subtracting an offset that is determined by considering the pattern of the first restored image. Methods for determining the offset include edge offset (EO) or band offset. EO indicates a method for determining an offset for a current sample based on the pattern of surrounding pixels. BO indicates a method for applying a common offset to a group of pixels with similar luminance values ​​in a region. Specifically, pixel luminance can be divided into 32 equal intervals, and pixels with similar luminance values ​​can be set as one group. For example, among the 32 bands, four adjacent bands can be set as one group, and the same offset value can be applied to samples belonging to the four bands.

[0565] ALF is a method of applying a filter of a predefined size / shape to a first restored image or a restored image to which a deblocking filter has been applied to generate a second restored image. The following Equation 35 shows an application example of ALF.

number

[0566] One of the predefined filter candidates can be selected on an image, coding tree unit, coding block, prediction block or transform block basis. Either the size or shape of each filter candidate can be different.

[0567] FIG. 49 is a diagram showing predefined filter candidates.

[0568] In the example shown in FIG. 49, at least one of diamond sizes 5×5, 7×7, and 9×9 may be selected.

[0569] For the chroma component, only diamonds of size 5x5 can be used.

[0570] It is within the scope of the invention to apply the embodiments described with a focus on a decoding or encoding process to an encoding or decoding process. It is also within the scope of the invention to modify the embodiments described in a given order to perform steps in an order different from that described.

[0571] Although the above-mentioned embodiment has been described based on a series of steps or a flow chart, this does not limit the chronological order of the present invention, and the steps may be performed simultaneously or in other orders as necessary. In addition, in the above-mentioned embodiment, each component (e.g., unit, module, etc.) constituting the block diagram can be implemented by a hardware device or software, and multiple components can be combined and implemented as a single hardware device or software. The above-mentioned embodiment can be implemented in the form of program instructions that can be executed by various computer components, and the program instructions can be stored in a computer-readable storage medium. The computer-readable storage medium can include program instructions, data files, data structures, etc., alone or in combination. Examples of computer-readable storage media can include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical storage media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions, such as ROMs, RAMs, flash memories, etc. The hardware devices can be configured to operate as one or more software modules for performing the processes according to the present invention, and vice versa. [Industrial Applicability]

[0572] The present invention can be applied to electronic devices that encode / decode video.

Claims

1. 1. A video decoding method, comprising: parsing a non-zero flag from the bitstream indicating whether a residual coefficient is non-zero; for the position of the last non-zero coefficient, if the non-zero flag is omitted, then consider the residual coefficient to be non-zero; if the non-zero flag indicates that the residual coefficient is non-zero, parsing absolute value information from the bitstream, the absolute value information being used to determine an absolute value of the residual coefficient; determining absolute values ​​of the residual coefficients based on the absolute value information; the absolute value information includes a residual coefficient comparison flag indicating whether the residual coefficient exceeds a first value; Further parsing a parity flag from the bitstream only if the residual coefficient exceeds the first value, the parity flag indicating whether the value of the residual coefficient is even or odd.

2. If the residual coefficient exceeds the first value, further analyze a first adjusted residual coefficient comparison flag, the first adjusted residual coefficient comparison flag indicating whether an adjusted residual coefficient derived by shifting the residual coefficient one bit right exceeds a second value; The video decoding method of claim 1.

3. if the adjusted residual coefficient is less than or equal to a second value, determining that the residual coefficient is 2N or 2N+1, where N is the second value, according to a value of the parity flag; The video decoding method of claim 2.

4. If the adjusted residual coefficient exceeds the second value, further analyze the residual value information; the residual value information is a value obtained by subtracting the second value from the adjusted residual coefficient; The video decoding method of claim 2.

5. 1. A video encoding method, comprising: encoding a non-zero flag indicating whether the residual coefficient is non-zero; For the position of the last non-zero coefficient, if the coding of the non-zero flag is omitted, the residual coefficient is regarded as non-zero; if the residual coefficient is non-zero, encoding absolute value information, the absolute value information being used to determine an absolute value of the residual coefficient; the absolute value information includes a residual coefficient comparison flag indicating whether the residual coefficient exceeds a first value; 4. The video encoding method of claim 3, further comprising: encoding a parity flag of the residual coefficient only if the residual coefficient exceeds the first value, the parity flag indicating whether the value of the residual coefficient is even or odd.

6. If the residual coefficient exceeds the first value, further encode a first adjusted residual coefficient comparison flag, the first adjusted residual coefficient comparison flag indicating whether an adjusted residual coefficient derived by shifting the residual coefficient one bit to the right exceeds a second value.

6. The video encoding method of claim 5.

7. if the adjusted residual coefficient is less than or equal to the second value, determining that the residual coefficient is 2N or 2N+1 according to a value of the parity flag; 7. The video encoding method of claim 6.

8. if the adjusted residual coefficient exceeds the second value, further encoding residual value information; the residual value information is a value obtained by subtracting the second value from the adjusted residual coefficient; 7. The video encoding method of claim 6.

9. 1. A video decoding device, comprising: means for parsing a non-zero flag from the bitstream, the non-zero flag indicating whether a residual coefficient is non-zero; for a last non-zero coefficient position, if the non-zero flag is omitted, means for considering the residual coefficient as non-zero; means for parsing absolute value information from the bitstream if the non-zero flag indicates that the residual coefficient is non-zero, the absolute value information being used to determine an absolute value of the residual coefficient; and means for determining absolute values ​​of the residual coefficients based on the absolute value information; the absolute value information includes a residual coefficient comparison flag indicating whether the residual coefficient exceeds a first value; The video decoding device further analyzes a parity flag from the bitstream only if the residual coefficient exceeds the first value, the parity flag indicating whether a value of the residual coefficient is even or odd.

10. 1. A video encoding device, comprising: means for encoding a non-zero flag indicating whether a residual coefficient is non-zero; means for regarding the residual coefficient as non-zero if coding of a non-zero flag is omitted for the position of the last non-zero coefficient; means for encoding absolute value information if the residual coefficient is non-zero, the absolute value information being used to determine an absolute value of the residual coefficient; the absolute value information includes a residual coefficient comparison flag indicating whether the residual coefficient exceeds a first value; the video encoding device further encoding a parity flag of the residual coefficient only if the residual coefficient exceeds the first value, the parity flag indicating whether the value of the residual coefficient is even or odd.

11. 1. A video decoding device, comprising: A memory storing instructions and a processor, the processor executing the instructions to parsing a non-zero flag from the bitstream indicating whether a residual coefficient is non-zero; for the position of the last non-zero coefficient, if the non-zero flag is omitted, then consider the residual coefficient to be non-zero; if the non-zero flag indicates that the residual coefficient is non-zero, parsing absolute value information from the bitstream, the absolute value information being used to determine an absolute value of the residual coefficient; determining absolute values ​​of the residual coefficients based on the absolute value information; the absolute value information includes a residual coefficient comparison flag indicating whether the residual coefficient exceeds a first value; The video decoding device further analyzes a parity flag from the bitstream only if the residual coefficient exceeds the first value, the parity flag indicating whether a value of the residual coefficient is even or odd.

12. 1. A video encoding device, comprising: A memory storing instructions and a processor, the processor executing the instructions to encoding a non-zero flag indicating whether the residual coefficient is non-zero; For the position of the last non-zero coefficient, if the coding of the non-zero flag is omitted, the residual coefficient is regarded as non-zero; if the residual coefficient is non-zero, encoding absolute value information, the absolute value information being used to determine an absolute value of the residual coefficient; the absolute value information includes a residual coefficient comparison flag indicating whether the residual coefficient exceeds a first value; the video encoding device further encoding a parity flag of the residual coefficient only if the residual coefficient exceeds the first value, the parity flag indicating whether the value of the residual coefficient is even or odd.

13. 1. A computer-readable storage medium, comprising: A computer-readable storage medium storing a program for causing a computer to execute the video decoding method according to any one of claims 1 to 4.

14. 1. A computer-readable storage medium, comprising: A computer-readable storage medium storing a program for causing a computer to execute the video encoding method according to any one of claims 5 to 8.

Images