Video signal encoding / decoding method and device therefor

Abstract

To provide a video image decoding method for improving a coding / decoding efficiency, and provide a video image coding method.SOLUTION: A video decoding method includes steps of: dividing the current picture into a plurality of tiles; decoding division information indicating a slice type indicating whether or not a square shape slice is adopted; and decoding first width information and first height information of the slice in the case that the division information indicates that the square shape slice is adopted. In the case where the first width information indicates one tile column, and the first height information indicates one tile raw, a parsing is made for number information on the basis of a height of a tile in which the slice is contained.SELECTED DRAWING: Figure 33

Description

[Technical Field]

[0001] The present invention relates to a video signal encoding / decoding method and an apparatus therefor. [Background technology]

[0002] As display panels continue to grow larger, video services with higher image quality are increasingly required. The biggest problem with high-definition video services is the significant increase in data volume. To address this issue, active research is being conducted to improve video compression rates. As a representative example, in 2009, the Motion Picture Experts Group (MPEG) and the Video Coding Experts Group (VCEG) under the International Telecommunication Union-Telecommunication (ITU-T) formed the Joint Collaborative Team on Video Coding (JCT-VC). JCT-VC proposed High Efficiency Video Coding (HEVC), a video compression standard with approximately twice the compression performance of H.264 / AVC, which was approved as a standard on January 25, 2013. With the rapid development of high-definition video services, HEVC's performance is gradually reaching its limits. Summary of the Invention [Problem to be solved by the invention]

[0003] The present invention aims to provide a method for dividing a picture into a plurality of tiles or slices when encoding / decoding a video signal, and an apparatus for performing the method.

[0004] SUMMARY OF THE INVENTION An object of the present invention is to provide a method for dividing a picture into tiles and a device for performing the method, in which slices are separated based on tile indexes.

[0005] An object of the present invention is to provide a method for dividing a picture into a plurality of slices based on difference information from the previous slice, and an apparatus for carrying out the method.

[0006] The technical problems that the present invention aims to solve are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention pertains from the following description. [Means for solving the problem]

[0007] A video signal decoding method according to the present invention includes: dividing a current picture into a plurality of tiles; decoding partition information indicating a slice type, the partition information indicating whether a rectangular slice is applied; and decoding first width information and first height information of the slice if the partition information indicates that a rectangular slice is applied. Here, if the first width information indicates one tile column and the first height information indicates one tile row, whether or not to parse the number information may be determined based on the height of the tile including the slice.

[0008] A video signal encoding method according to the present invention may include dividing a current picture into a plurality of tiles, determining whether to apply rectangular slices, and encoding first width information and first height information of the slices if the rectangular slices are applied. Here, if the first width information indicates one tile column and the first height information indicates one tile row, it may be determined whether to encode the number information based on the height of the tile including the slice.

[0009] In the video signal decoding method according to the present invention, when a tile is configured in one coding tree unit row, parsing of the number information may be omitted.

[0010] In the video signal decoding method according to the present invention, when a tile is composed of multiple coding tree unit rows, number information is parsed, and the number information can indicate the number of slices for which slice height information needs to be explicitly signaled.

[0011] In the video signal decoding method according to the present invention, the second height information may be parsed for a first slice having an index smaller than the number indicated by the number information among at least one slice included in a tile.

[0012] In the video signal decoding method according to the present invention, the second height information may not be parsed for the second slice in which the number indicated by the number information is equal to or greater than the index.

[0013] In the video signal decoding method according to the present invention, the height of the second slice may be set to the smaller of the height of the third slice from which the second height information was last parsed or the remaining height in the tile.

[0014] The above briefly summarized features of the present invention are illustrative aspects of the detailed description of the invention that follows and are not intended to limit the scope of the invention. [Effects of the Invention]

[0015] According to the present invention, the coding / decoding efficiency can be improved by dividing a picture into multiple tiles or slices.

[0016] According to the present invention, by dividing slices based on tile indexes, it is possible to improve encoding / decoding efficiency.

[0017] According to the present invention, by dividing a slice based on difference information from the previous slice, it is possible to improve encoding / decoding efficiency.

[0018] The effects obtained by the present invention are not limited to those mentioned above, and other effects not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention pertains from the following description. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a block diagram of a video encoder according to an embodiment of the present invention;

[0020] [Figure 2] 1 is a block diagram of a video decoder according to one embodiment of the present invention;

[0021] [Figure 3] FIG. 2 illustrates a basic coding tree unit according to an embodiment of the present invention.

[0022] [Figure 4] 10A and 10B are diagrams illustrating various division shapes of coding blocks.

[0023] [Figure 5] FIG. 10 is a diagram illustrating an example of division of a coding tree unit.

[0024] [Figure 6] 1 is a flowchart of an inter-prediction method according to an embodiment of the present invention.

[0025] [Figure 7] FIG. 10 is a diagram illustrating non-linear movement of an object.

[0026] [Figure 8] 1 is a flowchart of an affine motion-based inter-prediction method according to an embodiment of the present invention.

[0027] [Figure 9] 10A and 10B are diagrams illustrating affine seed vectors for different affine motion models.

[0028] [Figure 10] FIG. 10 is a diagram illustrating affine vectors of sub-blocks under a four-parameter motion model.

[0029] [Figure 11] 10 is a flowchart of a process of deriving motion information of a current block in a merge mode.

[0030] [Figure 12] FIG. 1 illustrates candidate blocks used to derive merge candidates.

[0031] [Figure 13] FIG. 10 is a diagram illustrating an update aspect of the motion information table.

[0032] [Figure 14] FIG. 10 is a diagram showing an update state of the motion information table.

[0033] [Figure 15] FIG. 10 is a diagram illustrating an example in which the index of pre-stored motion information candidates is updated.

[0034] [Figure 16] FIG. 10 illustrates an example in which overlap checking is performed on only some of the merge candidates.

[0035] [Figure 17] FIG. 10 illustrates an example in which overlap checking with a specific merge candidate is omitted.

[0036] [Figure 18] FIG. 10 is a diagram illustrating an example in which a candidate block included in the same merge processing area as the current block is set as unavailable as a merge candidate.

[0037] [Figure 19]10A and 10B are diagrams illustrating an example of deriving merge candidates for a current block when the current block is included in a merge processing area;

[0038] [Figure 20] FIG. 10 is a diagram showing a temporary motion information table.

[0039] [Figure 21] 10A and 10B are diagrams illustrating an example of merging a motion information table and a temporary motion information table.

[0040] [Figure 22] 1 is a flowchart of an intra prediction method according to an embodiment of the present invention.

[0041] [Figure 23] FIG. 10 is a diagram illustrating intra-prediction modes.

[0042] [Figure 24] FIG. 10 is a diagram illustrating an example of a one-dimensional array in which reference samples are arranged in a row. [Figure 25] FIG. 10 is a diagram illustrating an example of a one-dimensional array in which reference samples are arranged in a row.

[0043] [Figure 26] 10 is a diagram illustrating angles formed by directional intra prediction modes with a line parallel to the x-axis.

[0044] [Figure 27] FIG. 10 illustrates how predicted samples are obtained when the current block is non-square.

[0045] [Figure 28] FIG. 10 is a diagram illustrating a wide-angle intra prediction mode.

[0046] [Figure 29] FIG. 10 is a diagram illustrating an example in which whether or not a transform skip is possible is determined for each sub-block.

[0047] [Figure 30] FIG. 10 is a diagram illustrating an example in which sub-blocks use the same transform type.

[0048] [Figure 31] 10 is a flowchart illustrating a process for determining block strength.

[0049] [Figure 32] Shows predefined filter candidates.

[0050] [Figure 33] FIG. 2 illustrates a picture partitioning method according to an embodiment of the present invention.

[0051] [Figure 34] 1 shows an example in which a picture is divided into multiple tiles.

[0052] [Figure 35] FIG. 10 is a diagram for explaining the generation aspect of bricks.

[0053] [Figure 36] FIG. 10 is a diagram illustrating an example in which slices are defined based on the raster order. [Figure 37] FIG. 10 is a diagram illustrating an example in which slices are defined based on the raster order.

[0054] [Figure 38] FIG. 10 shows an example in which only rectangular slices are allowed. DETAILED DESCRIPTION OF THE INVENTION

[0055] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0056] Video encoding and decoding is performed on a block-by-block basis. For example, encoding / decoding processes such as transform, quantization, prediction, in-loop filtering, or reconstruction may be performed on a coding block, a transform block, or a prediction block.

[0057] Hereinafter, a block to be coded / decoded is referred to as a “current block.” For example, the current block may be a coding block, a transform block, or a prediction block depending on the current coding / decoding process step.

[0058] As used herein, the term "unit" refers to a basic unit for performing a particular encoding / decoding process, and "block" may be understood to refer to a sample array of a predetermined size. Unless otherwise specified, "block" and "unit" may be used interchangeably. For example, in the embodiments described below, a coding block and a coding unit may be understood to have the same meaning.

[0059] FIG. 1 is a block diagram of a video encoder according to one embodiment of the present invention.

[0060] Referring to FIG. 1, the video encoding device 100 may include a picture division unit 110, prediction units 120, 125, a transformation unit 130, a quantization unit 135, a reordering unit 160, an entropy encoding unit 165, an inverse quantization unit 140, an inverse transform unit 145, a filter unit 150, and a memory 155.

[0061] 1 are illustrated independently to illustrate different characteristic functions of the video encoding device, and do not mean that each component is a separate hardware or software component. That is, each component is included side by side as a separate component for convenience of explanation, and at least two of the components may be combined into one component, or one component may be divided into multiple components to perform its function. Such integrated and separated embodiments of each component are within the scope of the present invention as long as they do not deviate from the essence of the present invention.

[0062] Furthermore, some components may not be essential components that perform essential functions in the present invention, but may be optional components simply for improving performance. The present invention may be realized by including only components that are essential for realizing the essence of the present invention other than components used simply for improving performance, and a structure including only essential components other than optional components used simply for improving performance is also included in the scope of the present invention.

[0063] The picture division unit 110 can divide an input picture into at least one processing unit. Here, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). The picture division unit 110 can divide one picture into a plurality of combinations of coding units, prediction units, and transform units, select one combination of coding units, prediction units, and transform units based on a predetermined criterion (e.g., a cost function), and code the picture.

[0064] For example, one picture may be divided into multiple coding units. To divide coding units in a picture, a recursive tree structure such as a quad tree structure may be used. A coding unit that is divided into other coding units with one image or a largest coding unit as the root may be divided into child nodes equal to the number of divided coding units. A coding unit that is not further divided due to a predetermined restriction becomes a leaf node. In other words, assuming that only square division is possible for one coding unit, one coding unit may be divided into up to four different coding units.

[0065] Hereinafter, in the embodiments of the present invention, the coding unit can be used to mean a unit for performing coding, or a unit for performing decoding.

[0066] The prediction units may be divided into at least one shape, such as a square or rectangle, of the same size within one coding unit, or may be divided into prediction units such that one of the divided prediction units within one coding unit has a different shape and / or size from the other prediction units.

[0067] When generating a prediction unit for performing intra prediction based on a coding unit, if the coding unit is not the minimum coding unit, intra prediction can be performed without dividing the coding unit into a plurality of N×N prediction units.

[0068] The prediction units 120 and 125 may include an inter prediction unit 120 for performing inter prediction and an intra prediction unit 125 for performing intra prediction. It may determine whether to use inter prediction or intra prediction for a prediction unit, and determine specific information (e.g., intra prediction mode, motion vector, reference picture, etc.) according to each prediction method. Here, the processing unit in which prediction is performed may differ from the processing unit in which the prediction method and its specific contents are determined. For example, the prediction method and prediction mode may be determined for each prediction unit, and the prediction may be performed for each transform unit. Residual values (residual blocks) between the generated prediction block and the original block may be input to the transform unit 130. In addition, prediction mode information, motion vector information, etc. used for prediction may be coded by the entropy coding unit 165 along with the residual values and transmitted to the decoder. When a specific coding mode is used, it is also possible to directly code the original block and transmit it to the decoder without generating a prediction block via the prediction units 120 and 125.

[0069] The inter prediction unit 120 may predict a prediction unit based on information of at least one of a picture preceding or following the current picture, and in some cases, may predict a prediction unit based on information of a partial region in the current picture for which encoding has been completed. The inter prediction unit 120 may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

[0070] The reference picture interpolation unit receives reference picture information from the memory 155 and can generate sub-integer pixel information from the reference picture. In the case of luminance pixels, a DCT-based 8-tap interpolation filter having different filter coefficients may be used to generate sub-integer pixel information in 1 / 4 pixel units. In the case of chrominance signals, a DCT-based 4-tap interpolation filter having different filter coefficients may be used to generate sub-integer pixel information in 1 / 8 pixel units.

[0071] The motion prediction unit may perform motion prediction based on the reference picture interpolated by the reference picture interpolator. Various methods, such as a full search-based block matching algorithm (FBMA), a three-step search algorithm (TSS), or a new three-step search algorithm (NTS), may be used to calculate a motion vector. The motion vector may have a motion vector value in half or quarter pixel units based on the interpolated pixel. The motion prediction unit may predict the current prediction unit using different motion prediction methods. Various methods, such as a skip method, a merge method, an advanced motion vector prediction method (AMVP), or an intra block copy method, may be used as the motion prediction method.

[0072] The intra prediction unit 125 may generate a prediction unit based on reference pixel information surrounding a current block, which is pixel information within a current picture. Since a neighboring block of the current prediction unit is an inter-predicted block, if the reference pixel is an inter-predicted pixel, the reference pixel included in the inter-predicted block may be used instead of reference pixel information of a neighboring intra-predicted block. That is, if the reference pixel is unavailable, the unavailable reference pixel information may be used instead of at least one of the available reference pixels.

[0073] Intra prediction may include a directional prediction mode that uses reference pixel information according to a prediction direction and a non-directional mode that does not use directional information when performing prediction. A mode for predicting luma information and a mode for predicting chroma information may be different, and intra prediction mode information used for predicting luma information or predicted luma signal information may be used to predict chroma information.

[0074] When performing intra prediction, if the size of the prediction unit and the size of the transform unit are the same, intra prediction for the prediction unit can be performed based on the pixel located to the left, the pixel located at the top left, and the pixel located at the top. However, when performing intra prediction, if the size of the prediction unit and the size of the transform unit are different, intra prediction can be performed using reference pixels based on the transform unit. Also, intra prediction using NxN division can be used only for the minimum coding unit.

[0075] The intra prediction method may generate a predicted block after applying an adaptive intra smoothing (AIS) filter to reference pixels according to a prediction mode. The type of AIS filter applied to the reference pixels may vary. To perform the intra prediction method, the intra prediction mode of a current prediction unit may be predicted from the intra prediction mode of a prediction unit existing in the vicinity of the current prediction unit. When predicting the prediction mode of the current prediction unit using mode information predicted from the surrounding prediction units, if the intra prediction modes of the current prediction unit and the surrounding prediction units are the same, information indicating that the prediction modes of the current prediction unit and the surrounding prediction units are the same may be transmitted using predetermined flag information. If the prediction modes of the current prediction unit and the surrounding prediction units are different, entropy coding may be performed to encode the prediction mode information of the current block.

[0076] In addition, a residual block may be generated that includes residual value information, which is a difference value between a prediction unit predicted based on the prediction unit generated by the prediction units 120 and 125 and an original block of the prediction unit. The generated residual block may be input to the conversion unit 130.

[0077] The transform unit 130 may transform a residual block including residual value information of the original block and the prediction unit generated by the prediction units 120 and 125 using a transform method such as a Discrete Cosine Transform (DCT) or a Discrete Sine Transform (DST). Here, the DCT transform core may include at least one of DCT2 or DCT8, and the DST transform core may include DST7. Whether to apply DCT or DST to transform the residual block may be determined based on intra-prediction mode information of the prediction unit used to generate the residual block. Transformation of the residual block may also be skipped. A flag indicating whether to skip transformation of the residual block may be coded. Transform skipping may be permitted for residual blocks whose size is equal to or smaller than a threshold, luma components, or chroma components in a 4:4:4 format.

[0078] The quantization unit 135 quantizes the values transformed into the frequency domain by the transformation unit 130. The quantization coefficient may vary depending on the block or the importance of the image. The values calculated by the quantization unit 135 may be provided to the inverse quantization unit 140 and the realignment unit 160.

[0079] The reordering unit 160 may reorder coefficient values for the quantized residual values.

[0080] The reordering unit 160 may convert two-dimensional block shape coefficients into one-dimensional vector shapes using a coefficient scanning method. For example, the reordering unit 160 may use a zig-zag scan method to scan from DC coefficients to coefficients in the high-frequency region and convert the coefficients into one-dimensional vector shapes. Instead of zig-zag scan, vertical scan, in which two-dimensional block shape coefficients are scanned in the column direction, or horizontal scan, in which two-dimensional block shape coefficients are scanned in the row direction, may be used depending on the size of the transform unit and the intra prediction mode. That is, it may be determined whether to use a zig-zag scan, vertical scan, or horizontal scan depending on the size of the transform unit and the intra prediction mode.

[0081] The entropy coding unit 165 may perform entropy coding based on the value calculated by the reordering unit 160. The entropy coding may use various coding methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

[0082] The entropy coding unit 165 can encode various information such as residual value coefficient information and block type information of the coding unit from the realignment unit 160 and the prediction units 120 and 125, prediction mode information, division unit information, prediction unit information and transmission unit information, motion vector information, reference frame information, block interpolation information, and filtering information.

[0083] The entropy coding unit 165 can entropy code the coefficient values of the coding unit input from the reordering unit 160 .

[0084] The inverse quantization unit 140 and the inverse transform unit 145 inversely quantize the values quantized by the quantization unit 135 and inversely transform the values transformed by the transform unit 130. The residual values generated by the inverse quantization unit 140 and the inverse transform unit 145 can be combined with prediction units predicted by the motion estimation unit, motion compensation unit, and intra prediction unit included in the prediction units 120 and 125 to generate a reconstructed block.

[0085] The filter unit 150 may include at least one of a deblocking filter, an offset correction unit, and an ALF (Adaptive Loop Filter).

[0086] A deblocking filter can remove block artifacts caused by boundaries between blocks in a reconstructed picture. To determine whether to perform deblocking, it can determine whether to apply a deblocking filter to a current block based on pixels included in several columns or rows included in the block. When applying a deblocking filter to a block, a strong filter or a weak filter can be applied depending on the required deblocking filtering strength. In addition, when performing vertical filtering and horizontal filtering when applying a deblocking filter, horizontal filtering and vertical filtering can be processed in parallel.

[0087] The offset correction unit may correct the offset between the deblocked image and the original image on a pixel-by-pixel basis. To perform offset correction for a specific picture, the offset correction unit may divide pixels included in the image into a predetermined number of regions, determine regions to be offset, and apply the offset to the regions, or may apply the offset by considering edge information of each pixel.

[0088] Adaptive Loop Filtering (ALF) may be performed based on the value obtained by comparing the filtered restored image with the original image. After dividing the pixels in the image into predetermined groups, a filter to be applied to each group is determined, and differential filtering can be performed for each group. Information regarding whether to apply ALF may be transmitted for each coding unit (CU) of the luminance signal, and the shape and filter coefficients of the ALF filter applied to each block may differ. Alternatively, an ALF filter of the same shape (fixed shape) may be applied regardless of the characteristics of the target block.

[0089] The memory 155 may store the reconstructed blocks or pictures calculated through the filter unit 150, and the stored reconstructed blocks or pictures may be provided to the prediction units 120 and 125 when performing inter-prediction.

[0090] FIG. 2 is a block diagram of a video decoder according to one embodiment of the present invention.

[0091] Referring to FIG. 2, the video decoder 200 may include an entropy decoding unit 210, a reordering unit 215, an inverse quantization unit 220, an inverse transform unit 225, prediction units 230 and 235, a filter unit 240, and a memory 245.

[0092] When a video bitstream is input from a video encoder, the input bitstream may be decoded in the reverse procedure to that of the video encoder.

[0093] The entropy decoding unit 210 may perform entropy decoding in a reverse procedure to the entropy encoding performed by the entropy encoding unit of the video encoder. For example, various methods such as Exponential Golomb, CAVLC (Context-Adaptive Variable Length Coding), and CABAC (Context-Adaptive Binary Arithmetic Coding) may be applied in accordance with the method used in the video encoder.

[0094] The entropy decoding unit 210 can decode information about the intra-prediction and inter-prediction performed in the encoder.

[0095] The rearrangement unit 215 may rearrange the bitstream entropy decoded by the entropy decoding unit 210 based on the rearrangement method used by the encoder. Coefficients expressed in one-dimensional vector form may be restored to two-dimensional block-shaped coefficients and rearranged. The rearrangement unit 215 may receive information about coefficient scanning performed by the encoder and perform rearrangement by scanning in reverse based on the scanning order performed by the encoder.

[0096] The inverse quantization unit 220 may perform inverse quantization based on the quantization parameter provided by the encoder and the coefficient values of the reordered blocks.

[0097] The inverse transform unit 225 may perform an inverse transform, i.e., an inverse DCT or inverse DST, on the transform performed by the transform unit, i.e., the DCT or DST, on the quantization result performed by the video encoder. Here, the DCT transform core may include at least one of DCT2 or DCT8, and the DST transform core may include DST7. Alternatively, if a transform is skipped in the video encoder, the inverse transform unit 225 may not perform an inverse transform. The inverse transform may be performed based on a transmission unit determined by the video encoder. The inverse transform unit 225 of the video decoder may selectively perform a transform technique (e.g., DCT or DST) depending on multiple pieces of information, such as a prediction method, a size of a current block, and a prediction direction.

[0098] The prediction units 230, 235 can generate prediction blocks based on prediction block generation related information provided from the entropy decoding unit 210 and previously decoded block or picture information provided from the memory 245.

[0099] As described above, when performing intra prediction, similar to the operation in a video encoder, if the size of the prediction unit and the size of the transform unit are the same, intra prediction for the prediction unit is performed based on the pixel located to the left, the pixel located at the top left, and the pixel located at the top. However, if the size of the prediction unit and the size of the transform unit are different when performing intra prediction, intra prediction can be performed using reference pixels based on the transform unit. In addition, intra prediction using NxN division can be used only for the minimum coding unit.

[0100] The prediction units 230 and 235 may include a prediction unit determination unit, an inter prediction unit, and an intra prediction unit. The prediction unit determination unit receives various information, such as prediction unit information input from the entropy decoding unit 210, prediction mode information for the intra prediction method, and motion prediction-related information for the inter prediction method, to classify a prediction unit from a current coding unit and determine whether the prediction unit performs inter prediction or intra prediction. The inter prediction unit 230 may perform inter prediction on the current prediction unit based on information included in at least one picture preceding or following the current picture including the current prediction unit, using information necessary for inter prediction of the current prediction unit provided from the video encoder. Alternatively, the inter prediction may be performed based on information of a partial region previously restored within the current picture including the current prediction unit.

[0101] To perform inter-prediction, it is possible to determine, based on a coding unit, whether the motion prediction method of the prediction unit included in the coding unit is skip mode, merge mode, motion vector prediction mode (AMVP mode), or intra block copy mode.

[0102] The intra prediction unit 235 can generate a prediction block based on pixel information in the current picture. If the prediction unit is a prediction unit for which intra prediction has been performed, the intra prediction can be performed based on intra prediction mode information of the prediction unit provided from the video encoder. The intra prediction unit 235 may include an AIS (Adaptive Intra Smoothing) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter, which filters the reference pixels of the current block, can determine whether to apply a filter depending on the prediction mode of the current prediction unit and apply the filter. AIS filtering can be performed on the reference pixels of the current block using the prediction mode of the prediction unit and AIS filter information provided from the video encoder. If the prediction mode of the current block is a mode that does not perform AIS filtering, the AIS filter may not be applied.

[0103] The reference pixel interpolation unit may interpolate reference pixels to generate reference pixels in pixel units of integer values or less when the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on pixel values obtained by interpolating reference pixels. When the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating reference pixels, the reference pixels may not be interpolated. When the prediction mode of the current block is a DC mode, the DC filter may generate a prediction block by filtering.

[0104] The reconstructed block or picture may be provided to a filter unit 240. The filter unit 240 may include a deblocking filter, an offset correction unit, and an ALF.

[0105] The video encoder may provide information regarding whether a deblocking filter is applied to the corresponding block or picture, and, if a deblocking filter is applied, information regarding whether a strong filter or a weak filter is applied. The deblocking filter of the video decoder may receive deblocking filter-related information provided by the video encoder and perform deblocking filtering on the corresponding block.

[0106] The offset correction unit can perform offset correction on the restored video based on the type of offset correction applied to the video during encoding and information on the offset value.

[0107] The ALF may be applied to a coding unit based on ALF applicability information, ALF coefficient information, etc. provided by the encoder. Such ALF information may be provided by being included in a specific parameter set.

[0108] The memory 245 can store the reconstructed pictures or blocks to make them available as reference pictures or blocks, and can provide the reconstructed pictures to an output.

[0109]

[0110] FIG. 3 is a diagram illustrating a basic coding tree unit according to one embodiment of the present invention.

[0111] A coding block of the largest size can be defined as a coding tree block. One picture is divided into a plurality of coding tree units (CTUs). A coding tree unit is a coding unit of the largest size and can also be called an LCU (Largest Coding Unit). Figure 3 shows an example in which one picture is divided into a plurality of coding tree units.

[0112] The size of a coding tree unit may be defined at the picture level or at the sequence level, and for this purpose, information indicating the size of the coding tree unit may be signaled via a picture parameter set or a sequence parameter set.

[0113] For example, the coding tree unit size for all pictures in a sequence may be set to 128 x 128. Alternatively, at the picture level, the coding tree unit size may be determined to be either 128 x 128 or 256 x 256. For example, the coding tree unit size for a first picture may be set to 128 x 128, and the coding tree unit size for a second picture may be set to 256 x 256.

[0114] A coding tree unit may be divided to generate coding blocks. A coding block represents a basic unit for encoding / decoding processing. For example, prediction or transformation may be performed for each coding block, or a predictive coding mode may be determined for each coding block. Here, the predictive coding mode represents a method for generating a predicted image. For example, the predictive coding mode may include intra prediction (intra prediction), inter prediction (inter prediction), current picture referencing (CPR or intra block copy (IBC)), or combined prediction. A predictive block for a coding block may be generated using at least one predictive coding mode from intra prediction, inter prediction, current picture referencing, or combined prediction for the coding block.

[0115] Information indicating the predictive coding mode of the current block may be signaled via a bitstream. For example, the information may be a one-bit flag indicating whether the predictive coding mode is intra-mode or inter-mode. Only when the predictive coding mode of the current block is determined to be inter-mode, can current picture reference or hybrid prediction be used.

[0116] The current picture reference is used to set the current picture as a reference picture and obtain a prediction block for the current block from an area in the current picture that has already been coded / decoded. Here, the current picture refers to a picture including the current block. Information indicating whether the current picture reference is applied to the current block may be signaled via a bitstream. For example, the information may be a 1-bit flag. If the flag is true, the prediction coding mode of the current block may be determined as current picture reference, and if the flag is false, the prediction mode of the current block may be determined as inter-prediction.

[0117] Alternatively, the predictive coding mode of the current block may be determined based on the reference picture index. For example, if the reference picture index indicates the current picture, the predictive coding mode of the current block may be determined as current picture reference. If the reference picture index indicates a picture other than the current picture, the predictive coding mode of the current block may be determined as inter prediction. That is, current picture reference is a prediction method using information on an area in the current picture that has been completely coded / decoded, and inter prediction is a prediction method using information on another picture that has been completely coded / decoded.

[0118] Hybrid prediction refers to a coding mode that combines two or more of intra prediction, inter prediction, and current picture reference. For example, when hybrid prediction is applied, a first predicted block may be generated based on one of intra prediction, inter prediction, or current picture reference, and a second predicted block may be generated based on the other. Once the first predicted block and the second predicted block are generated, a final predicted block may be generated by averaging or weighted summing the first predicted block and the second predicted block. Information indicating whether hybrid prediction is applied may be signaled via a bitstream. The information may be a 1-bit flag.

[0119] FIG. 4 is a diagram showing various division shapes of a coding block.

[0120] A coding block may be divided into multiple coding blocks based on a quad-tree, binary-tree, or triple-tree partitioning. The divided coding block may also be divided into multiple coding blocks based on a quad-tree, binary-tree, or triple-tree partitioning.

[0121] Quadtree partitioning refers to a partitioning technique that divides the current block into four blocks. As a result of quadtree partitioning, the current block may be divided into four square partitions (see "SPLIT_QT" in (a) of Figure 4).

[0122] Binary tree splitting refers to a splitting technique that splits the current block into two blocks. Splitting the current block into two blocks along the vertical direction (i.e., using a vertical line that crosses the current block) can be referred to as vertical binary tree splitting, and splitting the current block into two blocks along the horizontal direction (i.e., using a horizontal line that crosses the current block) can be referred to as horizontal binary tree splitting. As a result of binary tree splitting, the current block may be split into two non-square partitions. Figure 4(b) "SPLIT_BT_VER" shows the result of vertical binary tree splitting, and Figure 4(c) "SPLIT_BT_HOR" shows the result of horizontal binary tree splitting.

[0123] Triple tree partitioning refers to a partitioning technique that divides the current block into three blocks. Dividing the current block into three blocks along the vertical direction (i.e., using two vertical lines crossing the current block) can be referred to as vertical triple tree partitioning, and dividing the current block into three blocks along the horizontal direction (i.e., using two horizontal lines crossing the current block) can be referred to as horizontal triple tree partitioning. As a result of triple tree partitioning, the current block may be divided into three non-square partitions. Here, the width / height of the partition located in the center of the current block may be twice the width / height of the other partitions. Figure 4(d) "SPLIT_TT_VER" shows the result of vertical triple tree partitioning, and Figure 4(e) "SPLIT_TT_HOR" shows the result of horizontal triple tree partitioning.

[0124] The number of times a coding tree unit is divided can be defined as the partitioning depth. The maximum partitioning depth of a coding tree unit can be determined at the sequence or picture level. This allows the maximum partitioning depth of a coding tree unit to differ for each sequence or picture.

[0125] Alternatively, the maximum split depth for each splitting technique can be determined separately. As an example, the maximum split depth allowed for quad-tree splitting can be different from the maximum split depth allowed for binary tree splitting and / or triple tree splitting.

[0126] The encoder may signal information indicating at least one of the partition shape and partition depth of the current block via a bitstream, and the decoder may determine the partition shape and partition depth of the coding tree unit based on information parsed from the bitstream.

[0127] FIG. 5 is a diagram illustrating an example of division of a coding tree unit.

[0128] Partitioning a coding block using partitioning techniques such as quad-tree partitioning, binary-tree partitioning, and / or triple-tree partitioning may be referred to as multi-tree partitioning.

[0129] A coding block generated by applying multi-tree partitioning to a coding block may be referred to as a sub-coding block. If the partition depth of a coding block is k, the partition depth of the sub-coding block is set to k+1.

[0130] Conversely, a coding block with a division depth of k can be referred to as a superior coding block relative to a coding block with a division depth of k+1.

[0131] The partition type of the current coding block may be determined based on at least one of the partition shape of an upper coding block or the partition type of a neighboring coding block. Here, the neighboring coding block is a block adjacent to the current coding block and may include at least one of the top neighboring block, the left neighboring block, or the neighboring block adjacent to the top left corner of the current coding block. Here, the partition type may include at least one of whether quad tree partitioning is possible, whether binary tree partitioning is possible, the binary tree partitioning direction, whether triple tree partitioning is possible, or the triple tree partitioning direction.

[0132] To determine the partition shape of a coding block, information indicating whether the coding block is split may be signaled via the bitstream. The information is a 1-bit flag "split_cu_flag", and if the flag is true, it indicates that the coding block is split using the merge-tree partitioning technique.

[0133] If split_cu_flag is true, information indicating whether the coding block is quadtree partitioned may be signaled via the bitstream. The information is a 1-bit flag split_qt_flag, and if the flag is true, the coding block may be split into four blocks.

[0134] 5, a coding tree unit is quad-tree partitioned to generate four coding blocks with a partition depth of 1. Also, quad-tree partitioning is applied to the first and fourth coding blocks among the four coding blocks generated as a result of the quad-tree partitioning. As a result, four coding blocks with a partition depth of 2 may be generated.

[0135] Furthermore, by applying quadtree partitioning to a coding block with a partition depth of 2, a coding block with a partition depth of 3 can be generated.

[0136] If quad-tree partitioning is not applied to a coding block, it may be determined whether to perform binary tree partitioning or triple tree partitioning on the coding block, taking into account at least one of the size of the coding block, whether the coding block is located on a picture boundary, the maximum partition depth, or the partition shape of neighboring blocks. If it is determined that binary tree partitioning or triple tree partitioning is to be applied to the coding block, information indicating the partitioning direction may be signaled via the bitstream. The information may be a 1-bit flag mtt_split_cu_vertical_flag. Based on the flag, it may be determined whether the partitioning direction is vertical or horizontal. Furthermore, information indicating whether binary tree partitioning or triple tree partitioning is to be applied to the coding block may be signaled via the bitstream. The information may be a 1-bit flag mtt_split_cu_binary_flag. Based on the flag, it may be determined whether binary tree partitioning or triple tree partitioning is to be applied to the coding block.

[0137] As an example, in the example shown in Figure 5, vertical binary tree partitioning is applied to a coding block with a partitioning depth of 1, and of the coding blocks generated as a result of the partitioning, vertical triple tree partitioning is applied to the left coding block, and vertical binary tree partitioning is applied to the right coding block.

[0138]

[0139] Inter-prediction is a predictive coding mode that predicts a current block using information of a previous picture. For example, a block at the same position as the current block in the previous picture (hereinafter, referred to as a collocated block) may be set as a prediction block for the current block. Hereinafter, a prediction block generated based on a block at the same position as the current block is referred to as a collocated prediction block.

[0140] Meanwhile, if an object that existed in the previous picture moves to a different position in the current picture, the current block can be effectively predicted using the object's motion. For example, if the object's movement direction and size can be known by comparing the previous picture with the current picture, a predicted block (or predicted image) of the current block can be generated taking into account the object's motion information. Hereinafter, the predicted block generated using the motion information can be referred to as a motion predicted block.

[0141] A residual block can be generated by subtracting the predicted block from the current block. Here, if object motion is present, the energy of the residual block can be reduced by using a motion prediction block instead of a co-located prediction block, thereby improving the compression performance of the residual block.

[0142] As described above, generating a prediction block using motion information can be referred to as motion compensated prediction. In most inter-prediction cases, a prediction block can be generated based on motion compensated prediction.

[0143] The motion information may include at least one of a motion vector, a reference picture index, a prediction direction, or a bidirectional weight index. The motion vector indicates the movement direction and size of an object. The reference picture index identifies the reference picture of the current block from among the reference pictures included in the reference picture list. The prediction direction indicates one of unidirectional L0 prediction, unidirectional L1 prediction, or bidirectional prediction (L0 prediction and L1 prediction). Depending on the prediction direction of the current block, at least one of L0 direction motion information or L1 direction motion information may be used. The bidirectional weight index identifies the weight applied to the L0 prediction block and the weight applied to the L1 prediction block.

[0144] FIG. 6 is a flowchart of an inter-prediction method according to an embodiment of the present invention.

[0145] Referring to FIG. 6, the inter-prediction method includes a step of determining an inter-prediction mode of a current block (S601), a step of obtaining motion information of the current block based on the determined inter-prediction mode (S602), and a step of performing motion compensation prediction for the current block based on the obtained motion information (S603).

[0146] Here, the inter prediction mode indicates various techniques for determining motion information of the current block, and may include an inter prediction mode using translation motion information and an inter prediction mode using affine motion information. For example, the inter prediction mode using translation motion information may include a merge mode and a motion vector prediction mode, and the inter prediction mode using affine motion information may include an affine merge mode and an affine motion vector prediction mode. The motion information of the current block may be determined based on information parsed from neighboring blocks adjacent to the current block or a bitstream, depending on the inter prediction mode.

[0147] The inter-prediction method using affine motion information will now be described in detail.

[0148] FIG. 7 is a diagram illustrating the nonlinear movement of an object.

[0149] Non-linear motion of an object may occur within an image. For example, as shown in FIG. 7, non-linear motion of an object may occur due to camera zoom-in, zoom-out, rotation, or affine transformation. When non-linear motion of an object occurs, the object's motion cannot be effectively represented by a translational motion vector. Therefore, in areas where non-linear motion of an object occurs, affine motion may be used instead of translational motion to improve coding efficiency.

[0150] FIG. 8 is a flowchart of an affine motion based inter-prediction method according to an embodiment of the present invention.

[0151] Whether an affine motion-based inter-prediction technique is applied to the current block may be determined based on information parsed from the bitstream. Specifically, whether an affine motion-based inter-prediction technique is applied to the current block may be determined based on at least one of a flag indicating whether an affine merge mode is applied to the current block or a flag indicating whether an affine motion vector prediction mode is applied to the current block.

[0152] When an affine motion-based inter-prediction technique is applied to a current block, an affine motion model of the current block can be determined (S801). The affine motion model can be determined to be at least one of a six-parameter affine motion model and a four-parameter affine motion model. The six-parameter affine motion model represents affine motion using six parameters, and the four-parameter affine motion model represents affine motion using four parameters.

[0153] Equation 1 expresses affine motion using six parameters: Affine motion indicates translational movement relative to a given region as determined by an affine seed vector.

[0154]

number

[0155] When affine motion is expressed using six parameters, complex motion can be expressed, but the number of bits required to encode each parameter increases, which may result in reduced coding efficiency. Therefore, affine motion can also be expressed using four parameters. Equation 2 expresses affine motion using four parameters.

[0156]

number

[0157] Information for determining an affine motion model for a current block may be coded and signaled via a bitstream. As an example, the information may be a 1-bit flag "affine_type_flag." A flag value of 0 may indicate that a 4-parameter affine motion model is applied, and a flag value of 1 may indicate that a 6-parameter affine motion model is applied. The flag may be coded in units of a slice, tile, or block (e.g., a coding block or coding tree unit). If the flag is signaled at the slice level, the affine motion model determined at the slice level may be applied to all of the blocks belonging to the slice.

[0158] Alternatively, the affine motion model of the current block may be determined based on the affine inter-prediction mode of the current block. For example, when an affine merge mode is applied, the affine motion model of the current block may be determined to be a four-parameter motion model. On the other hand, when an affine motion vector prediction mode is applied, information for determining the affine motion model of the current block may be encoded and signaled via a bitstream. For example, when an affine motion vector prediction mode is applied to the current block, the affine motion model of the current block may be determined based on a 1-bit flag "affine_type_flag."

[0159] Next, an affine seed vector for the current block can be derived (S802). If a four-parameter affine motion model is selected, motion vectors at two control points of the current block can be derived. On the other hand, if a six-parameter affine motion model is selected, motion vectors at three control points of the current block can be derived. The motion vectors at the control points can be referred to as affine seed vectors. The control points can include at least one of the top left corner, top right corner, or bottom left corner of the current block.

[0160] FIG. 9 is a diagram illustrating affine seed vectors for different affine motion models.

[0161] In a four-parameter affine motion model, affine seed vectors for two of the upper left corner, the upper right corner, or the lower left corner can be derived. For example, as shown in (a) of FIG. 9, when a four-parameter affine motion model is selected, an affine vector can be derived using an affine seed vector sv0 for the upper left corner of the current block (e.g., the upper left sample (x1, y1)) and an affine seed vector sv1 for the upper right corner of the current block (e.g., the upper right sample (x1, y1)). It is also possible to use an affine seed vector for the lower left corner instead of the affine seed vector for the upper left corner, or to use an affine seed vector for the lower left corner instead of the affine seed vector for the upper right corner.

[0162] In a six-parameter affine motion model, affine seed vectors for the upper left corner, the upper right corner, and the lower left corner can be derived. As an example, as shown in (b) of Figure 9, when a six-parameter affine motion model is selected, affine vectors can be derived using an affine seed vector sv0 for the upper left corner of the current block (e.g., the upper left sample (x1, y1)), an affine seed vector sv1 for the upper right corner of the current block (e.g., the upper right sample (x1, y1)), and an affine seed vector sv2 for the upper left corner of the current block (e.g., the upper left sample (x2, y2)).

[0163] In the embodiments described below, under the four-parameter affine motion model, the affine seed vectors of the top-left and top-right control points are referred to as the first affine seed vector and the second affine seed vector, respectively. In the embodiments described below that use the first affine seed vector and the second affine seed vector, the affine seed vector of the bottom-left control point (third affine seed vector) or the affine seed vector of the bottom-right control point (fourth affine seed vector) can be used instead of at least one of the first affine seed vector and the second affine seed vector.

[0164] Furthermore, under the six-parameter affine motion model, the affine seed vectors of the top-left control point, the top-right control point, and the bottom-left control point are referred to as the first affine seed vector, the second affine seed vector, and the third affine seed vector, respectively. In an embodiment using the first affine seed vector, the second affine seed vector, and the third affine seed vector, which will be described later, the affine seed vector of the bottom-right control point (fourth affine seed vector) can be used instead of at least one of the first affine seed vector, the second affine seed vector, and the third affine seed vector.

[0165] An affine vector can be derived for each sub-block using the affine seed vector (S803). Here, the affine vector refers to a translational motion vector derived based on the affine seed vector. The affine vector of a sub-block can be referred to as an affine sub-block motion vector or a sub-block motion vector.

[0166] FIG. 10 is a diagram illustrating affine vectors of sub-blocks under a four-parameter motion model.

[0167] The affine vector of the sub-block may be derived based on the position of the control point, the position of the sub-block, and the affine seed vector. As an example, Equation 3 shows an example of deriving an affine sub-block motion vector.

[0168]

number

[0169] In Equation 3, (x, y) indicates the position of a sub-block. Here, the position of the sub-block indicates the position of a reference sample included in the sub-block. The reference sample may be a sample located at the upper left corner of the sub-block or a sample whose x-axis or y-axis coordinate is the center position. (x0, y0) indicates the position of the first control point, and (sv0x, sv0y) indicates the first affine seed vector. Also, (x1, y1) indicates the position of the second control point, and (sv1x, sv1y) indicates the second affine seed vector.

[0170] If the first and second control points correspond to the top-left and top-right corners of the current block, respectively, x1-x0 may be set to a value equal to the width of the current block.

[0171] Next, motion compensation prediction can be performed for each sub-block using the affine vector of each sub-block (S804). As a result of the motion compensation prediction, a prediction block for each sub-block may be generated. The prediction block of the sub-block may be set as the prediction block of the current block.

[0172] The affine seed vector of the current block may be derived based on the affine seed vectors of neighboring blocks adjacent to the current block. If the inter prediction mode of the current block is the affine merge mode, the affine seed vector of a merge candidate included in the merge candidate list may be determined as the affine seed vector of the current block. In addition, if the inter prediction mode of the current block is the affine merge mode, motion information including at least one of a reference picture index, a specific direction prediction flag, or a bidirectional weighting value of the current block may also be set similarly to the merge candidate.

[0173]

[0174] Next, the inter-prediction method using translational motion information will be described in detail.

[0175] The motion information of the current block may be derived from the motion information of the current block and another block. Here, the other block may be a block that has been coded / decoded using inter-prediction before the current block. Setting the motion information of the current block to be the same as the motion information of another block may be defined as a merge mode. Also, setting the motion vector of another block as a predicted value of the motion vector of the current block may be defined as a motion vector prediction mode.

[0176] FIG. 11 is a flowchart of a process for deriving motion information of a current block in a merge mode.

[0177] Merge candidates for the current block may be derived (S1101). The merge candidates for the current block may be derived from blocks that have been coded / decoded using inter prediction prior to the current block.

[0178] FIG. 12 is a diagram illustrating candidate blocks used to derive merge candidates.

[0179] The candidate block may include at least one of a neighboring block including samples adjacent to the current block and a non-neighboring block including samples not adjacent to the current block. Hereinafter, samples used to determine the candidate block are defined as reference samples. Also, reference samples adjacent to the current block are referred to as neighboring reference samples, and reference samples not adjacent to the current block are referred to as non-neighboring reference samples.

[0180] The neighboring reference sample may be included in a neighboring column of the leftmost column of the current block or a neighboring row of the topmost row of the current block. For example, when the coordinates of the top left sample of the current block are (0, 0), at least one of a block including a reference sample at a position (-1, H-1), a block including a reference sample at a position (W-1, -1), a block including a reference sample at a position (W, -1), a block including a reference sample at a position (-1, H), or a block including a reference sample at a position (-1, -1) may be used as a candidate block. Referring to the drawing, neighboring blocks with indexes 0 to 4 may be used as candidate blocks.

[0181] A non-neighboring reference sample refers to a sample in which at least one of the x-axis distance or y-axis distance from a reference sample adjacent to the current block has a predetermined value. For example, at least one of a block including a reference sample whose x-axis distance from the left reference sample is a predetermined value, a block including a non-neighboring sample whose y-axis distance from the top reference sample is a predetermined value, or a block including a non-neighboring sample whose x-axis distance and y-axis distance from the top left reference sample are predetermined values may be used as a candidate block. The predetermined value may be a natural number such as 4, 8, 12, or 16. Referring to the drawing, at least one of blocks with indexes 5 to 26 may be used as a candidate block.

[0182] A sample that is not located on the same vertical, horizontal, or diagonal line as a neighboring reference sample can also be set as a non-neighboring reference sample.

[0183] The motion information of the merge candidate may be set similarly to the motion information of the candidate block. For example, at least one of the motion vector, reference picture index, prediction direction, and bidirectional weight index of the candidate block may be set as the motion information of the merge candidate.

[0184] A merge candidate list containing the merge candidates can be generated (S1102).

[0185] The indices of the merge candidates in the merge candidate list may be assigned according to a predetermined order. For example, the indices may be assigned to the merge candidates derived from the left neighboring block, the merge candidates derived from the top neighboring block, the merge candidates derived from the top right neighboring block, the merge candidates derived from the bottom left neighboring block, the merge candidates derived from the top left neighboring block, and the merge candidates derived from the temporal neighboring blocks in this order.

[0186] If the merge candidate list includes multiple merge candidates, at least one of the multiple merge candidates can be selected (S1103). Specifically, information for identifying one of the multiple merge candidates may be signaled via the bitstream. As an example, information "merge_idx" indicating the index of one of the merge candidates included in the merge candidate list may be signaled via the bitstream.

[0187]

[0188] If the number of merge candidates included in the merge candidate list is less than a threshold, motion information candidates included in the motion information table may be added to the merge candidate list as merge candidates. Here, the threshold may be the maximum number of merge candidates that the merge candidate list may include or a value obtained by subtracting an offset from the maximum number of merge candidates. The offset may be a natural number, such as 1 or 2.

[0189] The motion information table includes motion information candidates derived from blocks encoded / decoded based on inter-prediction in the current picture. For example, the motion information of the motion information candidates included in the motion information table may be set to the same as the motion information of blocks encoded / decoded based on inter-prediction. Here, the motion information may include at least one of a motion vector, a reference picture index, a prediction direction, or a bidirectional weight index.

[0190] The motion information candidates included in the motion information table may also be referred to as inter-region merge candidates or prediction region merge candidates.

[0191] The maximum number of motion information candidates that the motion information table may include may be predefined in the encoder and decoder. For example, the maximum number of motion information candidates that the motion information table may include may be 1, 2, 3, 4, 5, 6, 7, 8, or more (e.g., 16).

[0192] Alternatively, information indicating the maximum number of motion information candidates that the motion information table may include may be signaled via a bitstream. The information may be signaled at the sequence, picture, or slice level. The information may indicate the maximum number of motion information candidates that the motion information table may include. Alternatively, the information may indicate the difference between the maximum number of motion information candidates that the motion information table may include and the maximum number of merge candidates that the merge candidate list may include.

[0193] Alternatively, the maximum number of motion information candidates that the motion information table may contain may be determined depending on the size of the picture, the size of the slice, or the size of the coding tree unit.

[0194] The motion information table may be initialized in units of pictures, slices, tiles, bricks, coding tree units, or coding tree unit lines (rows or columns). For example, when a slice is initialized, the motion information table may also be initialized, and the motion information table may not include any motion information candidates.

[0195] Alternatively, information indicating whether to initialize the motion information table may be signaled via the bitstream. The information may be signaled at the slice, tile, brick, or block level. A pre-configured motion information table may be used until the information indicates that the motion information table should be initialized.

[0196] Alternatively, information about initial motion information candidates may be signaled via a picture parameter set or a slice header. Even when a slice is initialized, the motion information table may include the initial motion information candidates. This allows the initial motion information candidates to be used even for the first block to be coded / decoded in the slice.

[0197] Alternatively, a motion information candidate included in the motion information table of the previous coding tree unit may be set as the initial motion information candidate. For example, a motion information candidate with the smallest index or the largest index among the motion information candidates included in the motion information table of the previous coding tree unit may be set as the initial motion information candidate.

[0198] The blocks are coded / decoded according to the coding / decoding order, and the blocks coded / decoded based on inter prediction can be set as motion information candidates in order according to the coding / decoding order.

[0199] FIG. 13 is a diagram for explaining how the motion information table is updated.

[0200] When inter-prediction is performed on a current block (S1301), motion information candidates can be derived based on the current block (S1302). The motion information of the motion information candidates may be set to the same as the motion information of the current block.

[0201] If the motion information table is empty (S1303), motion information candidates derived based on the current block can be added to the motion information table (S1304).

[0202] If the motion information table already includes motion information candidates (S1303), a redundancy check may be performed on the motion information of the current block (or motion information candidates derived based on the motion information) (S1305). The redundancy check is performed to determine whether the motion information of the motion information candidates pre-stored in the motion information table is the same as the motion information of the current block. The redundancy check may be performed on all motion information candidates pre-stored in the motion information table. Alternatively, the redundancy check may be performed on motion information candidates pre-stored in the motion information table whose indexes are greater than or less than a threshold value. Alternatively, the redundancy check may be performed on a predefined number of motion information candidates. For example, two motion information candidates with small indexes or two motion information candidates with large indexes may be determined to be the targets of the redundancy check.

[0203] If no motion information candidate having the same motion information as the motion information of the current block is included, a motion information candidate derived based on the current block may be added to the motion information table (S1308). Whether the motion information candidates are the same may be determined based on whether the motion information (e.g., motion vectors and / or reference picture indexes, etc.) of the motion information candidates are the same.

[0204] If the maximum number of motion information candidates is already stored in the motion information table (S1306), the oldest motion information candidate is deleted (S1307), and a motion information candidate derived based on the current block can be added to the motion information table (S1308). Here, the oldest motion information candidate can be the motion information candidate with the largest index or the motion information candidate with the smallest index.

[0205] Each motion information candidate may be identified by an index. When a motion information candidate derived from the current block is added to the motion information table, the lowest index (e.g., 0) may be assigned to the motion information candidate, and the index of the pre-stored motion information candidate may be incremented by 1. Here, if the maximum number of motion information candidates is already stored in the motion information table, the motion information candidate with the highest index is removed.

[0206] Alternatively, when a motion information candidate derived from the current block is added to the motion information table, the motion information candidate may be assigned the largest index. For example, if the number of motion information candidates pre-stored in the motion information table is less than the maximum value, the motion information candidate may be assigned an index with the same value as the number of pre-stored motion information candidates. Alternatively, if the number of motion information candidates pre-stored in the motion information table is the same as the maximum value, the motion information candidate may be assigned an index obtained by subtracting 1 from the maximum value. Furthermore, the motion information candidate with the smallest index is removed, and the indexes of the remaining pre-stored motion information candidates are decremented by 1.

[0207] FIG. 14 is a diagram showing how the motion information table is updated.

[0208] It is assumed that the motion information candidate derived from the current block is added to the motion information table, the largest index is assigned to the motion information candidate, and the maximum number of motion information candidates is already stored in the motion information table.

[0209] When adding the motion information candidate HmvpCand[n+1] derived from the current block to the motion information table HmvpCandList, the motion information candidate HmvpCand[0] with the smallest index among the pre-stored motion information candidates can be deleted, and the indexes of the remaining motion information candidates can be decremented by 1. In addition, the index of the motion information candidate HmvpCand[n+1] derived from the current block can be set to the maximum value (n in the example shown in FIG. 14).

[0210] If the same motion information candidate as the motion information candidate derived based on the current block is pre-stored (S1305), the motion information candidate derived based on the current block may not be added to the motion information table (S1309).

[0211] Alternatively, the motion information candidate derived based on the current block may be added to the motion information table, and the previously stored motion information candidate that is the same as the motion information candidate may be removed, which has the same effect as updating the index of the previously stored motion information candidate.

[0212] FIG. 15 is a diagram showing an example in which the indexes of pre-stored motion information candidates are updated.

[0213] If the index of a pre-stored motion information candidate identical to the motion information candidate mvCand derived based on the current block is hIdx, the pre-stored motion information candidate is deleted, and the index of a motion information candidate having an index greater than hIdx is decremented by 1. For example, in the example shown in FIG. 15, HmvpCand[2] identical to mvCand is deleted from the motion information table HvmpCandList, and the indices from HmvpCand[3] to HmvpCand[n] are decremented by 1.

[0214] Also, the motion information candidate mvCand derived based on the current block can be added to the end of the motion information table.

[0215] Alternatively, the index assigned to the pre-stored motion information candidate that is the same as the motion information candidate derived based on the current block may be updated, for example, the index of the pre-stored motion information candidate may be changed to the minimum or maximum value.

[0216] The motion information of blocks included in a predetermined region may be set not to be added to the motion information table. For example, motion information candidates derived based on the motion information of blocks included in the merge processing region may not be added to the motion information table. Because the encoding / decoding order is not defined for blocks included in the merge processing region, it is inappropriate to use any one of the motion information for inter-prediction of other blocks. Therefore, motion information candidates derived based on blocks included in the merge processing region may not be added to the motion information table.

[0217] Alternatively, motion information of blocks smaller than a predetermined size may be set not to be added to the motion information table. For example, motion information of coding blocks whose width or height is smaller than 4 or 8, or motion information candidates derived based on motion information of coding blocks of 4x4 size may not be added to the motion information table.

[0218] Whether to use the current block as a motion information candidate may also be determined based on the inter prediction mode of the current block. For example, a block encoded / decoded based on an affine motion model may be set as unavailable as a motion information candidate. As a result, even if the current block is encoded / decoded using inter prediction, if the inter prediction mode of the current block is an affine prediction mode, the motion information table may not be updated based on the current block.

[0219] A motion information candidate may be configured to include additional information in addition to motion information. For example, at least one of block size, shape, or block partition information may be further stored for the motion information candidate. When constructing a merge candidate list for the current block, only motion information candidates whose size, shape, or partition information is the same or similar to that of the current block may be used, or motion information candidates whose size, shape, or partition information is the same or similar to that of the current block may be added to the merge candidate list first.

[0220] If the number of merge candidates included in the merge candidate list of the current block is less than a threshold, motion information candidates included in the motion information table may be added to the merge candidate list as merge candidates. The addition process is performed according to the order in which the motion information candidate indices are sorted in ascending or descending order. For example, the motion information candidate with the largest index may be added to the merge candidate list of the current block.

[0221] When adding a motion information candidate included in the motion information table to the merge candidate list, a check for overlap between the motion information candidate and merge candidates previously stored in the merge candidate list may be performed. As a result of the check for overlap, a motion information candidate having the same motion information as a previously stored merge candidate may not be added to the merge candidate list.

[0222] The redundancy check may be performed only on some of the motion information candidates included in the motion information table. For example, the redundancy check may be performed only on motion information candidates whose indexes are greater than or equal to a threshold value or less than or equal to a threshold value. Alternatively, the redundancy check may be performed only on the N motion information candidates whose indexes are the largest or smallest.

[0223] Alternatively, the overlapping check may be performed only on some of the merge candidates pre-stored in the merge candidate list. For example, the overlapping check may be performed only on merge candidates whose indexes are greater than or less than a threshold or on merge candidates derived from a block at a specific position. Here, the specific position may include at least one of the left neighboring block, the top neighboring block, the top right neighboring block, or the bottom left neighboring block of the current block.

[0224] FIG. 16 illustrates an example in which overlap checking is performed on only some of the merge candidates.

[0225] When adding a motion information candidate HmvpCand[j] to the merge candidate list, the motion information candidate can be checked for overlap with the two merge candidates with the highest indices, mergeCandList[NumMerge-2] and mergeCandList[NumMerge-1], where NumMerge can indicate the number of available spatial and temporal merge candidates.

[0226] Unlike the illustrated example, when adding a motion information candidate HmvpCand[j] to the merge candidate list, the motion information candidate can be checked for overlap with up to two merge candidates with the smallest indices. For example, it can be checked whether mergeCandList[0] and mergeCandList[1] are identical to HmvpCand[j].

[0227] Alternatively, the overlapping check may be performed only on merge candidates derived from a specific position. For example, the overlapping check may be performed on at least one of merge candidates derived from neighboring blocks located to the left of the current block or merge candidates derived from neighboring blocks located above the current block. If there is no merge candidate derived from a specific position in the merge candidate list, the motion information candidate may be added to the merge candidate list without performing the overlapping check.

[0228] When adding a motion information candidate HmvpCand[j] to the merge candidate list, the motion information candidate can be checked for overlap with the two merge candidates with the highest indices, mergeCandList[NumMerge-2] and mergeCandList[NumMerge-1], where NumMerge can indicate the number of available spatial and temporal merge candidates.

[0229] The overlap check with the merge candidate may be performed only on some of the motion information candidates. For example, the overlap check may be performed only on the N motion information candidates with the largest indexes or the N motion information candidates with the smallest indexes among the motion information candidates included in the motion information table. For example, the overlap check may be performed only on motion information candidates with indexes whose difference from the number of motion information candidates included in the motion information table is less than or equal to a threshold. If the threshold is 2, the overlap check may be performed only on the three motion information candidates with the largest index values among the motion information candidates included in the motion information table. The overlap check may be omitted for motion information candidates other than the three motion information candidates. When the overlap check is omitted, the motion information candidate may be added to the merge candidate list regardless of whether it has the same motion information as the merge candidate.

[0230] Conversely, it may be set so that the duplication check is performed only on motion information candidates having indexes whose difference from the number of motion information candidates included in the motion information table is equal to or greater than a threshold value.

[0231] The number of motion information candidates for which the redundancy check is performed may be predefined in the encoder and decoder. For example, the threshold may be an integer such as 0, 1, or 2.

[0232] Alternatively, the threshold value may be determined based on at least one of the number of merge candidates included in the merge candidate list or the number of motion information candidates included in the motion information table.

[0233] If a merge candidate identical to the first motion information candidate is found, the overlapping check with the merge candidate identical to the first motion information candidate can be omitted when checking for overlapping with the second motion information candidate.

[0234] FIG. 17 is a diagram showing an example in which the overlap check with a specific merge candidate is omitted.

[0235] When adding a motion information candidate HmvpCand[i] with index i to the merge candidate list, a check is performed for overlap between the motion information candidate and merge candidates previously stored in the merge candidate list. If a merge candidate mergeCandList[j] identical to the motion information candidate HmvpCand[i] is found, the motion information candidate HmvpCand[i] is not added to the merge candidate list, and an overlap check is performed between the motion information candidate HmvpCand[i-1] with index i-1 and the merge candidate. The check for overlap between the motion information candidate HmvpCand[i-1] and the merge candidate mergeCandList[j] can be omitted.

[0236] For example, in the example shown in Figure 17, HmvpCand[i] and mergeCandList[2] are determined to be the same. As a result, HmvpCand[i] is not added to the merge candidate list, and a duplication check may be performed on HmvpCand[i-1]. Here, the duplication check between HmvpCand[i-1] and mergeCandList[2] may be omitted.

[0237] If the number of merge candidates included in the merge candidate list for the current block is less than a threshold, the merge candidate list may further include at least one of pairwise merge candidates or zero merge candidates in addition to motion information candidates. A pairwise merge candidate is a merge candidate whose motion vector is the average value of the motion vectors of two or more merge candidates, and a zero merge candidate is a merge candidate whose motion vector is 0.

[0238] The current block's merge candidate list may be populated with merge candidates in the following order:

[0239] Spatial merge candidate - Temporal merge candidate - Motion information candidate - (Affine motion information candidate) - Pairwise merge candidate - Zero merge candidate

[0240] The spatial merge candidate refers to a merge candidate derived from at least one of a neighboring block or a non-neighboring block, the temporal merge candidate refers to a merge candidate derived from a previous reference picture, and the affine motion information candidate refers to a motion information candidate derived from a block coded / decoded using an affine motion model.

[0241]

[0242] The motion information table may also be used in the motion vector prediction mode. For example, if the number of motion vector prediction candidates included in the motion vector prediction candidate list of the current block is less than a threshold, the motion information candidates included in the motion information table may be set as motion vector prediction candidates for the current block. Specifically, the motion vectors of the motion information candidates may be set as motion vector prediction candidates.

[0243] When one of the motion vector prediction candidates included in the motion vector prediction candidate list for the current block is selected, the selected candidate can be set as the motion vector predictor for the current block. Next, after decoding the motion vector residual value of the current block, the motion vector predictor and the motion vector residual value can be added together to obtain the motion vector of the current block.

[0244] The motion vector prediction candidate list for the current block may be constructed according to the following order:

[0245] Spatial motion vector prediction candidate - Temporal motion vector prediction candidate - Motion information candidate - (Affine motion information candidate) - Zero motion vector prediction candidate

[0246] The spatial motion vector prediction candidate refers to a motion vector prediction candidate derived from at least one of a neighboring block or a non-neighboring block, and the temporal motion vector prediction candidate refers to a motion vector prediction candidate derived from a previous reference picture. The affine motion information candidate refers to a motion information candidate derived from a block encoded / decoded using an affine motion model. The zero motion vector prediction candidate refers to a candidate whose motion vector value is 0.

[0247]

[0248] A merge processing area larger than a coding block may be defined. The coding blocks included in the merge processing area may be processed in parallel rather than sequentially encoded / decoded. Here, not sequentially encoded / decoded means that the encoding / decoding order is not defined. As a result, the encoding / decoding processes of the blocks included in the merge processing area may be processed independently. Alternatively, the blocks included in the merge processing area may share merge candidates. Here, the merge candidates may be derived based on the merge processing area.

[0249] Due to the above characteristics, the merge processing region can also be referred to as a parallel processing region, a shared merge region (SMR), or a merge estimation region (MER).

[0250] Merge candidates for the current block may be derived based on coding blocks. However, if the current block is included in a merge processing area that is larger than the current block, candidate blocks included in the same merge processing area as the current block may be set as unavailable as merge candidates.

[0251] FIG. 18 is a diagram showing an example in which a candidate block included in the same merge processing area as the current block is set as an unusable merge candidate.

[0252] In the example shown on the left side of Figure 18, when encoding / decoding CU5, blocks including reference samples adjacent to CU5 may be set as candidate blocks. Here, candidate blocks X3 and X4 included in the same merge processing area as CU5 may be set as unavailable as merge candidates for CU5. On the other hand, candidate blocks X0, X1, and X2 not included in the same merge processing area as CU5 may be set as available as merge candidates.

[0253] In the example shown on the right side of Figure 18, when encoding / decoding CU8, blocks including reference samples adjacent to CU8 may be set as candidate blocks. Here, candidate blocks X6, X7, and X8 included in the same merge processing area as CU8 may be set as unavailable merge candidates. On the other hand, candidate blocks X5 and X9 not included in the same merge area as CU8 may be set as available merge candidates.

[0254] Alternatively, when the current block is included in the merge processing region, neighboring blocks adjacent to the current block and neighboring blocks adjacent to the merge processing region may be set as candidate blocks.

[0255] FIG. 19 is a diagram illustrating an example of deriving merge candidates for a current block when the current block is included in a merge processing area.

[0256] As shown in the example of (a) of Figure 19, neighboring blocks adjacent to a current block may be set as candidate blocks for deriving merge candidates for the current block. Here, candidate blocks included in the same merge processing area as the current block may be set as unavailable as merge candidates. For example, when deriving merge candidates for a coding block CU3, an upper neighboring block y3 and an upper right neighboring block y4 included in the same merge processing area as the coding block CU3 may be set as unavailable as merge candidates for the coding block CU3.

[0257] Neighboring blocks adjacent to the current block may be scanned in a predefined order to derive merge candidates. For example, the predefined order may be y1, y3, y4, y0, and y2.

[0258] If the number of merging candidates that can be derived from neighboring blocks neighboring the current block is less than the maximum number of merging candidates or a value obtained by subtracting an offset from the maximum number of merging candidates, merging candidates for the current block may be derived using neighboring blocks neighboring the merge processing region, as shown in (b) of Figure 19. As an example, neighboring blocks neighboring the merge processing region including the coding block CU3 may be set as candidate blocks for the coding block CU3. Here, the neighboring blocks neighboring the merge processing region may include at least one of the left neighboring block x1, the top neighboring block x3, the bottom left neighboring block x0, the top right neighboring block x4, or the top left neighboring block x2.

[0259] Neighboring blocks adjacent to the merging processing region can be scanned in a predefined order to derive merging candidates, for example, x1, x3, x4, x0, and x2.

[0260] In summary, merge candidates for coding block CU3 included in the merge processing region may be derived by scanning the candidate blocks according to the following scan order:

[0261] (y1,y3,y4,y0,y2,x1,x3,x4,x0,x2)

[0262] However, the scanning order of the candidate blocks illustrated above is merely an example of the present invention, and the candidate blocks may be scanned in an order different from the above example. Alternatively, the scanning order may be adaptively determined based on at least one of the size and shape of the current block or the merge processing region.

[0263] The merge processing region may be square or non-square. Information for determining the merge processing region may be signaled via the bitstream. The information may include at least one of information indicating the shape of the merge processing region or information indicating the size of the merge processing region. If the merge processing region is non-square, at least one of information indicating the size of the merge processing region, information indicating the width and / or height of the merge processing region, or information indicating the ratio of the width and height of the merge processing region may be signaled via the bitstream.

[0264] The size of the merging region may be determined based on at least one of information signaled via the bitstream, picture resolution, slice size, or tile size.

[0265] When motion compensation prediction is performed on a block included in the merge processing area, motion information candidates derived based on the motion information of the block on which motion compensation prediction is performed can be added to the motion information table.

[0266] However, when adding motion information candidates derived from a block included in the merge processing area to the motion information table, it is possible that the motion information candidates derived from the block may be used when encoding / decoding other blocks in the merge processing area that are actually encoded / decoded later than the block. That is, when encoding / decoding a block included in the merge processing area, although it is necessary to eliminate dependency between blocks, motion prediction compensation may be performed using motion information of other blocks included in the merge processing area. To solve this problem, even if encoding / decoding of a block included in the merge processing area is completed, the motion information of the block whose encoding / decoding has been completed may not be added to the motion information table.

[0267] Alternatively, the motion information table may be updated using only blocks at predefined positions within the merge processing region. The predefined positions may include at least one of a block located at the upper left corner, a block located at the upper right corner, a block located at the lower left corner, a block located at the lower right corner, a block located in the center, a block adjacent to the right boundary, or a block adjacent to the bottom boundary within the merge processing region. For example, only the motion information of a block adjacent to the lower right corner within the merge processing region may be updated to the motion information table, and the motion information of other blocks may not be updated to the motion information table.

[0268] Alternatively, after all blocks included in the merge processing region have been decoded, motion information candidates derived from the blocks may be added to the motion information table. That is, the motion information table may not be updated while the blocks included in the merge processing region are being encoded / decoded.

[0269] For example, when motion compensation prediction is performed on a block included in a merge processing region, motion information candidates derived from the block may be added to the motion information table in a predefined order. Here, the predefined order may be determined according to a scan order of coding blocks in the merge processing region or a coding tree unit. The scan order may be at least one of raster scan, horizontal scan, vertical scan, and zigzag scan. Alternatively, the predefined order may be determined based on the motion information of each block or the number of blocks having the same motion information.

[0270] Alternatively, motion information candidates containing unidirectional motion information may be added to the motion information table before motion information candidates containing bidirectional motion information, and conversely, motion information candidates containing bidirectional motion information may be added to the motion information table before motion information candidates containing unidirectional motion information.

[0271] Alternatively, the motion information candidates can be added to the motion information table in order of most frequently used or least frequently used within the merge processing region or coding tree unit.

[0272] If the current block is included in a merge processing area and the number of merge candidates included in the merge candidate list of the current block is less than the maximum number, motion information candidates included in the motion information table may be added to the merge candidate list. Here, motion information candidates derived from blocks included in the same merge processing area as the current block may be set not to be added to the merge candidate list of the current block.

[0273] Alternatively, if the current block is included in the merge processing area, the motion information candidates included in the motion information table may be set not to be used. That is, even if the number of merge candidates included in the merge candidate list of the current block is less than the maximum number, the motion information candidates included in the motion information table may not be added to the merge candidate list.

[0274] As another example, a motion information table for a merge processing region or a coding tree unit may be configured. This motion information table serves to temporarily store motion information of blocks included in the merge processing region. To distinguish between a general motion information table and a motion information table for a merge processing region or a coding tree unit, the motion information table for a merge processing region or a coding tree unit is referred to as a temporary motion information table. In addition, motion information candidates stored in the temporary motion information table are referred to as temporary motion information candidates.

[0275] FIG. 20 is a diagram showing a temporary motion information table.

[0276] A temporary motion information table may be configured for a coding tree unit or a merge processing region. When motion compensation prediction is performed on a current block included in a coding tree unit or a merge processing region, the motion information of the block may not be added to the motion information table HmvpCandList. Instead, temporary motion information candidates derived from the block may be added to the temporary motion information table HmvpMERCandList. That is, temporary motion information candidates added to the temporary motion information table may not be added to the motion information table. As a result, the motion information table may not include motion information candidates derived based on the motion information of blocks included in the coding tree unit or merge processing region that includes the current block.

[0277] Alternatively, only motion information of some of the blocks included in the merge processing area may be added to the temporary motion information table. For example, only blocks at predefined positions within the merge processing area may be used to update the motion information table. The predefined positions may include at least one of a block located at the upper left corner, a block located at the upper right corner, a block located at the lower left corner, a block located at the lower right corner, a block located in the center, a block adjacent to the right boundary, or a block adjacent to the bottom boundary within the merge processing area. For example, only motion information of a block adjacent to the lower right corner of the merge processing area may be added to the temporary motion information table, and motion information of other blocks may not be added to the temporary motion information table.

[0278] The maximum number of temporary motion information candidates that the temporary motion information table may include may be set to be the same as the maximum number of motion information candidates that the motion information table may include, or may be determined based on the size of a coding tree unit or a merge processing area, or may be set to be smaller than the maximum number of motion information candidates that the motion information table may include.

[0279] A current block included in a coding tree unit or a merge processing area may be set not to use a temporary motion information table for the coding tree unit or the merge processing area. That is, if the number of merge candidates included in the merge candidate list of the current block is less than a threshold, motion information candidates included in the motion information table may be added to the merge candidate list, and temporary motion information candidates included in the temporary motion information table may not be added to the merge candidate list. As a result, motion information of other blocks included in the same coding tree unit or the same merge processing area as the current block may not be used for motion compensation prediction of the current block.

[0280] When the encoding / decoding of all blocks included in the coding tree unit or the merge processing area is completed, the motion information table and the temporary motion information table can be merged.

[0281] FIG. 21 is a diagram showing an example of merging a motion information table and a temporary motion information table.

[0282] When the encoding / decoding of all blocks included in the coding tree unit or merge processing area is completed, the temporary motion information candidates included in the temporary motion information table can be updated to the motion information table, as shown in the example of FIG. 21.

[0283] Here, the temporary motion information candidates included in the temporary motion information table may be added to the motion information table in the order in which they were inserted into the temporary motion information table (ie, in ascending or descending order of index values).

[0284] As another example, temporary motion information candidates included in the temporary motion information table may be added to the motion information table according to a predefined order. Here, the predefined order may be determined according to a scan order of coding blocks in a merge processing region or a coding tree unit. The scan order may be at least one of raster scan, horizontal scan, vertical scan, and zigzag scan. Alternatively, the predefined order may be determined based on the motion information of each block or the number of blocks having the same motion information.

[0285] Alternatively, temporary motion information candidates including unidirectional motion information may be added to the motion information table before temporary motion information candidates including bidirectional motion information, or conversely, temporary motion information candidates including bidirectional motion information may be added to the motion information table before temporary motion information candidates including unidirectional motion information.

[0286] Alternatively, the temporary motion information candidates may be added to the motion information table in order of high or low frequency of use within the merge processing region or coding tree unit.

[0287] When adding a temporary motion information candidate included in the temporary motion information table to the motion information table, a duplication check may be performed on the temporary motion information candidate. For example, if a motion information candidate identical to a temporary motion information candidate included in the temporary motion information table is pre-stored in the motion information table, the temporary motion information candidate may not be added to the motion information table. Here, the duplication check may be performed on some of the motion information candidates included in the motion information table. For example, the duplication check may be performed on motion information candidates whose indexes are equal to or greater than a threshold value or less than a threshold value. For example, if the temporary motion information candidate is identical to a motion information candidate whose index is equal to or greater than a predefined value, the temporary motion information candidate may not be added to the motion information table.

[0288] It is possible to restrict motion information candidates derived from blocks included in the same coding tree unit or the same merge processing region as the current block from being used as merge candidates for the current block. To this end, block address information can be further stored for the motion information candidates. The block address information may include at least one of a block position, a block address, a block index, a position of a merge processing region including the block, an address of a merge processing region including the block, an index of a merge processing region including the block, a position of a coding tree region including the block, an address of a coding tree region including the block, or an index of a coding tree region including the block.

[0289]

[0290] Intra prediction predicts a current block using reconstructed samples that have been coded / decoded around the current block. Here, reconstructed samples before an in-loop filter is applied may be used for intra prediction of the current block.

[0291] Intra prediction techniques include matrix-based intra prediction and general intra prediction that takes into account the directionality of surrounding reconstructed samples. Information indicating the intra prediction technique for the current block may be signaled via a bitstream. The information may be a 1-bit flag. Alternatively, the intra prediction technique for the current block may be determined based on at least one of the position, size, and shape of the current block or the intra prediction techniques of neighboring blocks. For example, if the current block is located across a picture boundary, it may be set so that matrix-based intra prediction is not applied to the current block.

[0292] Matrix-based intra prediction is a method of obtaining a prediction block of a current block based on matrix multiplication of a matrix pre-stored in an encoder and a decoder and reconstructed samples surrounding the current block. Information for identifying one of a plurality of pre-stored matrices may be signaled via a bitstream. The decoder can determine a matrix for intra prediction of the current block based on the information and the size of the current block.

[0293] General intra prediction is a method of obtaining a predicted block for a current block based on a non-directional intra prediction mode or a directional intra prediction mode. Hereinafter, a process of performing intra prediction based on general intra prediction will be described in more detail with reference to the accompanying drawings.

[0294] FIG. 22 is a flowchart of an intra prediction method according to one embodiment of the present invention.

[0295] A reference sample line of the current block may be determined (S2201). The reference sample line refers to a set of reference samples included in the line k-th away from the top and / or left end of the current block. The reference samples may be derived from reconstructed samples for which encoding / decoding of the current block has been completed.

[0296] Index information for identifying a reference sample line of a current block from among a plurality of reference sample lines may be signaled via a bitstream. For example, index information intra_luma_ref_idx for identifying a reference sample line of the current block may be signaled via a bitstream. The index information may be signaled on a coding block basis.

[0297] The plurality of reference sample lines may include at least one of the top and / or leftmost first, second, and third lines of the current block. Of the plurality of reference sample lines, a reference sample line consisting of a row adjacent to the top of the current block and a column adjacent to the left of the current block may be referred to as an adjacent reference sample line, and other reference sample lines may be referred to as non-adjacent reference sample lines.

[0298] Table 1 shows the index assigned to each candidate reference sample line.

[0299] [Table 1]

[0300] The reference sample line of the current block may be determined based on at least one of the position, size, and shape of the current block, or the predictive coding mode of a neighboring block. For example, if the current block borders a boundary of a picture, a tile, a slice, or a coding tree unit, the neighboring reference sample line may be determined as the reference sample line of the current block.

[0301] The reference sample line may include a top reference sample located at the top of the current block and a left reference sample located to the left of the current block. The top reference sample and the left reference sample may be derived from reconstructed samples around the current block. The reconstructed samples may be in a state before an in-loop filter is applied.

[0302] Next, an intra-prediction mode for the current block may be determined (S2202). The intra-prediction mode for the current block may be at least one of a non-directional intra-prediction mode and a directional intra-prediction mode. The non-directional intra-prediction modes include Planar and DC, and the directional intra-prediction modes include 33 or 65 modes from the bottom left diagonal to the top right diagonal.

[0303] FIG. 23 is a diagram showing intra prediction modes.

[0304] FIG. 23(a) shows 35 intra prediction modes, and FIG. 23(b) shows 67 intra prediction modes.

[0305] A greater or lesser number of intra prediction modes than those shown in FIG. 23 may be defined.

[0306] A Most Probable Mode (MPM) may be set based on the intra-prediction modes of neighboring blocks adjacent to the current block, where the neighboring blocks may include a left neighboring block adjacent to the left side of the current block and an upper neighboring block adjacent to the upper end of the current block.

[0307] The number of MPMs included in the MPM list may be preset in the encoder and decoder. For example, the number of MPMs may be 3, 4, 5, or 6. Alternatively, information indicating the number of MPMs may be signaled via a bitstream. Alternatively, the number of MPMs may be determined based on at least one of the predictive coding mode of a neighboring block, the size, shape, or reference sample line index of the current block. For example, when a neighboring reference sample line is determined as the reference sample line of the current block, N MPMs may be used, while when a non-neighboring reference sample line is determined as the reference sample line of the current block, M MPMs may be used. M is a natural number smaller than N. For example, N may be 6, and M may be 5, 4, or 3. Therefore, if the index of the reference sample line of the current block is 0 and the MPM flag is true, the intra prediction mode of the current block is determined to be one of the six candidate intra prediction modes, whereas if the index of the reference sample line of the current block is greater than 0 and the MPM flag is true, the intra prediction mode of the current block may be determined to be one of the five candidate intra prediction modes.

[0308] Alternatively, a fixed number of MPM candidates (eg, 6 or 5) may be used regardless of the index of the reference sample line of the current block.

[0309] An MPM list including multiple MPMs may be generated, and information indicating whether the MPM list includes the same MPM as the intra prediction mode of the current block may be signaled via a bitstream. The information is a 1-bit flag and may be referred to as an MPM flag. If the MPM flag indicates that the MPM list includes the same MPM as the current block, index information identifying one of the MPMs may be signaled via the bitstream. For example, index information mpm_idx identifying one of the multiple MPMs may be signaled via the bitstream. The MPM identified by the index information may be set as the intra prediction mode of the current block. If the MPM flag indicates that the MPM list does not include the same MPM as the current block, remaining mode information indicating one of the remaining intra prediction modes other than the MPM may be signaled via the bitstream. The remaining mode information indicates an index value corresponding to the intra prediction mode of the current block when indexes are reallocated to the remaining intra prediction modes other than the MPM. The decoder may sort the MPMs in ascending order and compare the remaining mode information with the MPM to determine the intra prediction mode of the current block. For example, if the residual mode information is equal to or smaller than the MPM, 1 may be added to the residual mode information to derive the intra prediction mode of the current block.

[0310] When deriving the intra prediction mode of the current block, comparison of some of the MPMs with the remaining mode information may be omitted. For example, among the MPMs, an MPM that is a non-directional intra prediction mode may be excluded from comparison. When a non-directional intra prediction mode is set to an MPM, it is clear that the remaining mode information indicates a directional intra prediction mode. Therefore, the intra prediction mode of the current block can be derived by comparing the remaining MPMs other than the non-directional intra prediction mode with the remaining mode information. Instead of excluding the non-directional intra prediction mode from comparison, the number of non-directional intra prediction modes may be added to the remaining mode information, and the resulting value may be compared with the remaining MPM.

[0311] Instead of setting the default mode to an MPM, information indicating whether the intra prediction mode of the current block is the default mode may be signaled via a bitstream. The information is a 1-bit flag, and the flag may be referred to as a default mode flag. The default mode flag may be signaled only if the MPM flag indicates that the same MPM as that of the current block is included in the MPM list. As described above, the default mode may include at least one of planner, DC, vertical mode, or horizontal mode. For example, if planner is set as the default mode, the default mode flag may indicate whether the intra prediction mode of the current block is planner. If the default mode flag indicates that the intra prediction mode of the current block is not the default mode, one of the MPMs indicated by the index information may be set as the intra prediction mode of the current block.

[0312] When a default mode flag is used, the intra prediction mode that is the same as the default mode may be set so as not to be set in the MPM. For example, when the default mode flag indicates whether the intra prediction mode of the current block is planner, the intra prediction mode of the current block may be guided using five MPMs other than the MPM corresponding to planner.

[0313] When multiple intra prediction modes are set as a default mode, index information indicating one of the default modes may be further signaled, and the intra prediction mode of the current block may be set to the default mode indicated by the index information.

[0314] It may be set so that the default mode cannot be used if the index of the reference sample line of the current block is not 0. As an example, if a non-adjacent reference sample line is determined to be the reference sample line of the current block, it may be set so that a non-directional intra prediction mode such as DC mode or planar mode is not used. Thus, if the index of the reference sample line is not 0, the default mode flag may not be signaled, and the value of the default mode flag may be set to a predefined value (i.e., false).

[0315] Once the intra prediction mode of the current block is determined, a prediction sample for the current block can be obtained based on the determined intra prediction mode (S2203).

[0316] When the DC mode is selected, a predicted sample for the current block is generated based on the average value of the reference samples. Specifically, the values of all samples in the predicted block may be generated based on the average value of the reference samples. The average value may be derived using at least one of a top reference sample located at the top of the current block and a left reference sample located to the left of the current block.

[0317] The number or range of reference samples used to derive the average value may vary depending on the shape of the current block. For example, if the current block is a non-square block whose width is greater than its height, the average value may be calculated using only the top reference sample. On the other hand, if the current block is a non-square block whose width is smaller than its height, the average value may be calculated using only the left reference sample. That is, if the width and height of the current block are different, the average value may be calculated using only the adjacent reference sample whose length is longer. Alternatively, it may be determined whether to calculate the average value using only the top reference sample or only the left reference sample based on the ratio of the width and height of the current block.

[0318] When the planar mode is selected, a prediction sample can be obtained using a horizontal prediction sample and a vertical prediction sample. Here, the horizontal prediction sample is obtained based on a left reference sample and a right reference sample located on the same horizontal line as the prediction sample, and the vertical prediction sample is obtained based on a top reference sample and a bottom reference sample located on the same vertical line as the prediction sample. Here, the right reference sample may be generated by copying a reference sample adjacent to the top right corner of the current block, and the bottom reference sample may be generated by copying a reference sample adjacent to the bottom left corner of the current block. The horizontal prediction sample may be obtained based on a weighted sum operation of the left reference sample and the right reference sample, and the vertical prediction sample may be obtained based on a weighted sum operation of the top reference sample and the bottom reference sample. Here, the weight assigned to each reference sample may be determined depending on the position of the prediction sample. The prediction sample may be obtained based on an average operation or a weighted sum operation of the horizontal prediction sample and the vertical prediction sample. When a weighted sum operation is performed, the weight assigned to the horizontal prediction sample and the vertical prediction sample may be determined based on the position of the prediction sample.

[0319] When a directional prediction mode is selected, a parameter indicating the prediction direction (or prediction angle) of the selected directional prediction mode can be determined. Table 2 below shows the intra direction parameter intraPredAng for each intra prediction mode.

[0320] [Table 2]

[0321] Table 2 shows the intra direction parameters for each intra prediction mode having an index of 2 to 34 when 35 intra prediction modes are defined. If more than 33 directional intra prediction modes are defined, Table 2 can be further subdivided to set the intra direction parameters for each directional intra prediction mode.

[0322] After aligning the top reference sample and the left reference sample of the current block, a predicted sample can be obtained based on the value of the intra direction parameter, where if the value of the intra direction parameter is negative, the left reference sample and the top reference sample can be aligned.

[0323] 24 and 25 are diagrams showing examples of a one-dimensional array in which reference samples are arranged in a line.

[0324] 24 shows an example of a one-dimensional vertical array in which reference samples are arranged vertically, and FIG. 25 shows an example of a one-dimensional horizontal array in which reference samples are arranged horizontally. The embodiments of FIGS. 24 and 25 will be described assuming that 35 intra prediction modes are defined.

[0325] If the intra prediction mode index is one of 11 to 18, a horizontal one-dimensional array in which the top reference sample is rotated counterclockwise is applied, and if the intra prediction mode index is one of 19 to 25, a vertical one-dimensional array in which the left reference sample is rotated clockwise is applied. When arranging the reference samples in a row, the angle of the intra prediction mode may be taken into consideration.

[0326] Based on the intra direction parameters, reference sample decision parameters can be determined, which may include a reference sample index for identifying the reference sample and a weight parameter for determining a weight to be applied to the reference sample.

[0327] The reference sample index iIdx and the weight parameter ifact may be obtained by the following Equations 4 and 5, respectively.

[0328]

number

[0329]

number

[0330] In Equation 4 and Equation 5, Pang represents an intra-direction parameter. The reference sample identified by the reference sample index iIdx corresponds to an integer pel.

[0331] At least one reference sample may be identified to derive a prediction sample. Specifically, the position of the reference sample used to derive the prediction sample may be identified by taking into account the gradient of the prediction mode. For example, the reference sample index iIdx may be used to identify the reference sample used to derive the prediction sample.

[0332] Here, if the gradient of the intra prediction mode cannot be expressed by one reference sample, a prediction sample can be generated by interpolating multiple reference samples. For example, if the gradient of the intra prediction mode is a value between the gradient between the prediction sample and a first reference sample and the gradient between the prediction sample and a second reference sample, the prediction sample can be obtained by interpolating the first and second reference samples. That is, if an angular line along the intra prediction angle does not pass through a reference sample located at an integer pel, the prediction sample can be obtained by interpolating reference samples located adjacent to the left, right, top, or bottom of the position where the angular line passes.

[0333] Equation 6 below shows an example of obtaining a predicted sample based on a reference sample.

[0334]

number

[0335] In Equation 6, P represents a predicted sample, and Ref_1D represents one of the one-dimensionally arranged reference samples, where the position of the reference sample may be determined by the position (x, y) of the predicted sample and the reference sample index iIdx.

[0336] If the gradient of the intra prediction mode can be expressed by one reference sample, the weight parameter ifact is set to 0. As a result, Equation 6 may be simplified as Equation 7 below.

[0337]

number

[0338] Intra prediction for the current block may be performed based on a plurality of intra prediction modes. For example, an intra prediction mode may be induced for each prediction sample, and the prediction sample may be induced based on the intra prediction mode assigned to each prediction sample.

[0339] Alternatively, an intra prediction mode may be derived for each region, and intra prediction for each region may be performed based on the intra prediction mode assigned to the region. Here, a region may include at least one sample. At least one of the size or shape of a region may be adaptively determined based on at least one of the size, shape, and intra prediction mode of a current block. Alternatively, at least one of the size or shape of a region may be predefined in an encoder and a decoder, independently of the size or shape of a current block.

[0340] FIG. 26 is a diagram illustrating angles formed by directional intra prediction modes with a line parallel to the x-axis.

[0341] As shown in the example of Fig. 26, the directional prediction modes can exist between the diagonal direction at the bottom left and the diagonal direction at the top right. In terms of the angle formed by the x-axis and the directional prediction modes, the directional prediction modes can exist between 45 degrees (the diagonal direction at the bottom left) and -135 degrees (the diagonal direction at the top right).

[0342] If the current block is non-square, depending on the intra prediction mode of the current block, it may occur that a prediction sample is derived using a reference sample that is farther away from the prediction sample, instead of a reference sample that is closer to the prediction sample among reference samples located on an angular line along the intra prediction angle.

[0343] FIG. 27 shows how predicted samples are obtained when the current block is non-square.

[0344] For example, assume that the current block is a non-square block with a width greater than its height, as shown in (a) of Figure 27, and that the intra prediction mode of the current block is a directional intra prediction mode with an angle between 0 and 45 degrees. In this case, when deriving a prediction sample A near the right column of the current block, it may occur that a left reference sample L far from the prediction sample is used instead of an upper reference sample T close to the prediction sample among reference samples located in an angular mode along the angle.

[0345] As another example, assume that the current block is a non-square block with its height greater than its width, as shown in (b) of Figure 27, and that the intra prediction mode of the current block is a directional intra prediction mode between -90 degrees and -135 degrees. In this case, when deriving a prediction sample A near the bottom row of the current block, it may happen that the top reference sample T, which is far from the prediction sample, is used instead of the left reference sample L, which is close to the prediction sample, among the reference samples located in the angular mode along the angle.

[0346] To solve the above problem, if the current block is non-square, the intra prediction mode of the current block can be replaced with a backward intra prediction mode. Therefore, for non-square blocks, a directional prediction mode having a larger or smaller angle than the directional prediction modes shown in FIG. 23 can be used. Such a directional intra prediction mode can be defined as a wide-angle intra prediction mode. The wide-angle intra prediction mode refers to a directional intra prediction mode that does not fall within the range of 45 degrees to -135 degrees.

[0347] FIG. 28 is a diagram illustrating the wide-angle intra prediction mode.

[0348] In the example shown in FIG. 28, intra prediction modes with indices from −1 to −14 and intra prediction modes with indices between 67 and 80 indicate wide-angle intra prediction modes.

[0349] Figure 28 illustrates 14 wide-angle intra-prediction modes with angles greater than 45 degrees (-1 to -14) and 14 wide-angle intra-prediction modes with angles less than -135 degrees (67 to 80), although a greater or lesser number of wide-angle intra-prediction modes may be defined.

[0350] When a wide-angle intra prediction mode is used, the length of the top reference sample may be set to 2W+1, and the length of the left reference sample may be set to 2H+1.

[0351] When using the wide-angle intra prediction mode, sample A shown in (a) of Figure 28 may be predicted using reference sample T, and sample A shown in (b) of Figure 28 may be predicted using reference sample L.

[0352] A total of 67+N intra prediction modes can be used by adding the existing intra prediction modes and the N wide-angle intra prediction modes. As an example, Table 3 shows intra direction parameters of the intra prediction modes when 20 wide-angle intra prediction modes are defined.

[0353] [Table 3]

[0354] If the current block is non-square and the intra prediction mode of the current block obtained in step S2502 belongs to a transform range, the intra prediction mode of the current block may be transformed to a wide-angle intra prediction mode. The transform range may be determined based on at least one of the size, shape, and ratio of the current block. Here, the ratio may indicate the ratio between the width and height of the current block.

[0355] If the current block is non-square with its width greater than its height, the transform range may be set from the intra prediction mode index (e.g., 66) in the upper right diagonal direction to (the intra prediction mode index in the upper right diagonal direction - N). Here, N may be determined based on the ratio of the current block. If the intra prediction mode of the current block belongs to the transform range, the intra prediction mode may be converted to a wide-angle intra prediction mode. The transform may be performed by subtracting a predefined value from the intra prediction mode, and the predefined value may be the total number of intra prediction modes other than the wide-angle intra prediction mode (e.g., 67).

[0356] Depending on the embodiment, the intra prediction modes between No. 66 and No. 53 may be converted to wide-angle intra prediction modes between No. -1 and No. -14, respectively.

[0357] If the current block is non-square with its height greater than its width, the transform range may be set from the intra-prediction mode index of the bottom-left diagonal (e.g., 2) to (the intra-prediction mode index of the bottom-left diagonal + M), where M may be determined based on the proportion of the current block. If the intra-prediction mode of the current block belongs to the transform range, the intra-prediction mode may be converted to a wide-angle intra-prediction mode. The transform may be performed by adding a predefined value to the intra-prediction mode, and the predefined value may be the total number of directional intra-prediction modes other than the wide-angle intra-prediction mode (e.g., 65).

[0358] Depending on the embodiment, each of the intra-prediction modes between number 2 and number 15 may be converted to a wide-angle intra-prediction mode between number 67 and number 80.

[0359] Hereinafter, the intra prediction modes that belong to the transform range are referred to as wide-angle intra alternative prediction modes.

[0360] The transform range may be determined based on the ratio of the current block. As an example, Tables 4 and 5 show the transform ranges when 35 intra prediction modes other than the wide-angle intra prediction mode are defined and when 67 intra prediction modes are defined, respectively.

[0361] [Table 4]

[0362] [Table 5]

[0363] As shown in the examples of Tables 4 and 5, the number of wide-angle intra alternative prediction modes included in the transform range may vary depending on the ratio of the current block.

[0364] The ratio of the current block can be further subdivided to set the conversion range as shown in Table 6 below.

[0365] [Table 6]

[0366] When a non-adjacent reference sample line is determined as the reference sample line of the current block, or when a multi-line intra prediction coding method that selects one of multiple reference sample lines is used, the wide-angle intra prediction mode may be set not to be used. That is, even if the current block is non-square and the intra prediction mode of the current block belongs to the conversion range, the intra prediction mode of the current block may not be converted to the wide-angle intra prediction mode.

[0367] Alternatively, when the intra prediction mode of the current block is determined to be a wide-angle intra prediction mode, the non-adjacent reference sample line may be set to one that cannot be used as the reference sample line of the current block, or a multi-line intra prediction encoding method that selects one of multiple reference sample lines may be set not to be used. When the multi-line intra prediction encoding method is not used, the adjacent reference sample line may be determined to be the reference sample line of the current block.

[0368] When a wide-angle intra prediction mode is not used, refW and refH may be set to the sum of nTbW and nTbH. Thus, except for the top-left reference sample, non-adjacent reference samples that are a distance i from the current block may include (nTbW + nTbH + offsetX[i]) top reference samples and (nTbW + nTbH + offsetY[i]) left reference samples. That is, non-adjacent reference samples that are a distance i from the current block may include (2nTbW + 2nTbH + offsetX[i] + offsetY[i] + 1) reference samples. For example, if the value of whRatio is greater than 1, the value of offsetX may be set greater than the value of offsetY. For example, the value of offsetX may be set to 1, and the value of offsetY may be set to 0. On the other hand, if the value of whRatio is less than 1, the value of offsetY may be set greater than the value of offsetX. As an example, the value of offsetX may be set to 0 and the value of offsetY may be set to 1.

[0369] When a wide-angle intra prediction mode is used in addition to an existing intra prediction mode, the resources required to encode the wide-angle intra prediction mode increase, which may result in reduced coding efficiency. Therefore, instead of directly encoding the wide-angle intra prediction mode, an alternative intra prediction mode for the wide-angle intra prediction mode may be encoded to improve coding efficiency.

[0370] For example, if the current block is encoded using wide-angle intra prediction mode No. 67, then the intra prediction mode of the current block may be No. 2, which is a wide-angle alternative intra prediction mode of No. 67. Also, if the current block is encoded in wide-angle intra prediction mode No. −1, then the intra prediction mode of the current block may be No. 66, which is a wide-angle alternative intra prediction mode of No. −1.

[0371] The decoder may decode the intra-prediction mode of the current block and determine whether the decoded intra-prediction mode is within the transform range. If the decoded intra-prediction mode is a wide-angle alternative intra-prediction mode, the intra-prediction mode may be transformed to the wide-angle intra-prediction mode.

[0372] Alternatively, if the current block is coded in the wide-angle intra-prediction mode, the wide-angle intra-prediction mode may be coded as is.

[0373] The encoding of the intra prediction mode may be performed based on the above-mentioned MPM list. Specifically, if a neighboring block is encoded in a wide-angle intra prediction mode, the MPM can be set based on a wide-angle alternative intra prediction mode corresponding to the wide-angle intra prediction mode.

[0374]

[0375] A residual image can be derived by subtracting a predicted image from an original image. When converting the residual image to the frequency domain, removing high-frequency components from the frequency components does not significantly reduce the subjective image quality of the image. Therefore, reducing the values of the high-frequency components or setting them to zero reduces visual distortion and increases compression efficiency. Reflecting these characteristics, the current block can be transformed to decompose the residual image into two-dimensional frequency components. The transformation can be performed using a transform technique such as DCT (Discrete Cosine Transform) or DST (Discrete Sine Transform).

[0376] DCT decomposes (or transforms) a residual image into two-dimensional frequency components using a cosine transform, while DST decomposes (or transforms) a residual image into two-dimensional frequency components using a sine transform. As a result of transforming the residual image, the frequency components may be represented by basis images. For example, when performing DCT transform on an N×N block, N2 basic pattern components may be obtained. The size of each basic pattern component included in the N×N block may be obtained through the transform. Depending on the transform technique used, the size of the basic pattern component may be referred to as a DCT coefficient or a DST coefficient.

[0377] The DCT transform technique is mainly used to transform images with a large distribution of non-zero low-frequency components, while the DST transform technique is mainly used for images with a large distribution of high-frequency components.

[0378] Transformation techniques other than DCT or DST can also be used to transform the residual video.

[0379] Hereinafter, converting a residual image into two-dimensional frequency components is referred to as two-dimensional image transform. The size of a basic pattern component obtained as a result of the transform is referred to as a transform coefficient. For example, the transform coefficient may refer to a DCT coefficient or a DST coefficient. When both a first transform and a second transform (described later) are applied, the transform coefficient may refer to the size of a basic pattern component generated as a result of the second transform. Furthermore, a residual sample to which a transform skip is applied is also referred to as a transform coefficient.

[0380] The transform technique may be determined on a block-by-block basis. The transform technique may be determined based on at least one of the predictive coding mode of the current block, the size of the current block, or the shape of the current block. For example, if the current block is coded in intra prediction mode and the size of the current block is smaller than NxN, the transform may be performed using the DST transform technique. On the other hand, if the condition is not met, the transform may be performed using the DCT transform technique.

[0381] 2D image transform may not be performed on some blocks of the residual image. Not performing 2D image transform may be called a transform skip. A transform skip indicates that the first transform and the second transform are not applied to the current block. When a transform skip is applied, quantization may be applied to residual values that are not transformed.

[0382] Whether to allow a transform skip for the current block may be determined based on at least one of the size or shape of the current block. For example, the transform skip may be applied only if the size of the current block is smaller than a threshold. The threshold may relate to at least one of the width, height, or number of samples of the current block, and may be defined as 32x32, for example. Alternatively, the transform skip may be allowed only for square blocks. For example, the transform skip may be allowed for square blocks of 32x32, 16x16, 8x8, or 4x4 size. Alternatively, the transform skip may be allowed only if a sub-partition intra-coding method is not used.

[0383] Alternatively, if a sub-partition intra-coding method is applied to the current block, it may be determined whether or not a transform skip is applied to each sub-partition.

[0384] FIG. 29 is a diagram showing an example in which whether or not transform skipping is possible is determined for each sub-block.

[0385] Transform skip may be applied to only some of the sub-blocks. For example, as shown in FIG. 29, the transform skip may be applied to the sub-blocks located at the top of the current block, and the transform skip may not be allowed for the sub-blocks located at the bottom.

[0386] The transform type of a sub-block for which transform skipping is not permitted may be determined based on information signaled from the bitstream. For example, the transform type may be determined based on tu_mts_idx, which will be described later.

[0387] Alternatively, the transform type of a sub-block may be determined based on the size of the sub-block. For example, the horizontal transform type may be determined based on whether the width of the sub-block is greater than or equal to a threshold and / or less than or equal to a threshold, and the vertical transform type may be determined based on whether the height of the sub-block is greater than or equal to a threshold and / or less than or equal to a threshold.

[0388] After transforming a current block using a DCT or DST, the transformed current block can be transformed again. Here, a transformation based on a DCT or DST can be defined as a first transformation, and a second transformation can be defined as a transformation of a block to which the first transformation has been applied.

[0389] The first transform may be performed using any one of a number of candidate transform cores, for example, a DCT2, a DCT8, or a DST7.

[0390] Different transform cores may be used for the horizontal and vertical directions, and information indicating the combination of horizontal and vertical transform cores may be signaled via the bitstream.

[0391] The first transform and the second transform may be performed in different units. For example, the first transform may be performed on an 8x8 block, and the second transform may be performed on a 4x4 sub-block of the transformed 8x8 block. Alternatively, the second transform may be performed on transform coefficients belonging to three 4x4 sub-blocks. The three sub-blocks may include a sub-block located at the top left corner of the current block, a sub-block adjacent to the right of the sub-block, and a sub-block adjacent to the bottom of the sub-block. Alternatively, the second transform may be performed on an 8x8 block.

[0392] The transform coefficients of the remaining areas where the second transform is not performed may be set to zero.

[0393] Alternatively, a first transform can be performed on a 4x4 block, and a second transform can be performed on an 8x8 sized region containing the transformed 4x4 block.

[0394] Information indicating whether to perform the second transform may be signaled via the bitstream. For example, a flag indicating whether to perform the second transform or index information specifying whether to perform the second transform and a transform kernel to be used for the second transform may be signaled. For example, index information of 0 indicates that the second transform is not performed on the current block. On the other hand, if the index information is greater than 0, the transform kernel for performing the second transform may be determined by the index information.

[0395] Alternatively, whether to perform the second transform may be determined based on whether the horizontal transform core and the vertical transform core are the same. As an example, the second transform may be performed only if the horizontal transform core and the vertical transform core are the same. Alternatively, the second transform may be performed only if the horizontal transform core and the vertical transform core are different.

[0396] Alternatively, the second transform may be allowed only when a predefined transform core is used for the horizontal and vertical transforms. For example, the second transform may be allowed only when a DCT2 transform core is used for the horizontal and vertical transforms. Alternatively, if a sub-partition intra coding method is applied to the current block, the second transform may be allowed only when a DCT2 transform core is used for the horizontal and vertical transforms.

[0397] Alternatively, it may be determined whether to perform the second transform based on the number of non-zero transform coefficients of the current block. For example, if the number of non-zero transform coefficients of the current block is equal to or smaller than a threshold, the second transform may be set not to be used, and if the number of non-zero transform coefficients of the current block is larger than the threshold, the second transform may be set to be used. The second transform may also be set to be used only if the current block is coded using intra prediction.

[0398] Alternatively, it may be determined whether to perform the second transform based on the position of the last non-zero transform coefficient of the current block. For example, if at least one of the x-axis coordinate or the y-axis coordinate of the last non-zero transform coefficient of the current block is greater than a threshold, or if at least one of the x-axis coordinate or the y-axis coordinate of the sub-block to which the last non-zero transform coefficient of the current block belongs is greater than a threshold, the second transform may not be performed. Here, the threshold may be predefined in the encoder and the decoder. Alternatively, the threshold may be determined based on the size or shape of the current block.

[0399] Alternatively, if only DC component transform coefficients exist in the current block, the second transform may be set not to be performed, where the DC component indicates the transform coefficient at the top left corner of the current block.

[0400] Alternatively, if matrix-based intra prediction is applied to the current block, the second transform may be set not to be performed.

[0401]

[0402] Information indicating the transform type of the current block may be signaled via the bitstream, and may be index information tu_mts_idx indicating one of a combination of a transform type for the horizontal direction and a transform type for the vertical direction.

[0403] Based on the transform type candidates identified by the index information tu_mts_idx, a transform core for the vertical direction and a transform core for the horizontal direction may be determined. Table 7 shows the transform type combinations by tu_mts_idx.

[0404] [Table 7]

[0405] The transform type may be determined to be one of DCT2, DST7 or DCT8, or a transform skip may be inserted into the transform type candidate.

[0406] When Table 7 is used, if tu_mts_idx is 0, a DCT2 may be applied horizontally and vertically. If tu_mts_idx is 2, a DCT8 may be applied horizontally and a DCT7 may be applied vertically.

[0407] When a sub-partition intra-coding method is applied, the transform cores of the sub-blocks may be determined independently. For example, information for identifying a transform type combination candidate for each sub-block may be coded and signaled. This allows the transform cores between the sub-blocks to be different.

[0408] Alternatively, sub-blocks may use the same transform type. In this case, tu_mts_idx, which identifies a candidate transform type combination, may be signaled only for the first sub-block. Alternatively, tu_mts_idx may be signaled at the coding block level, and the transform type of a sub-block may be determined by referring to tu_mts_idx signaled at the coding block level. Alternatively, the transform type may be determined based on at least one of the size, shape, or intra prediction mode of one of the sub-blocks, and the determined transform type may be set to be used for all sub-blocks.

[0409] FIG. 30 is a diagram showing an example in which sub-blocks use the same transform type.

[0410] When a coding block is divided horizontally, the transform types of the sub-block (Sub-CU0) located at the top and the sub-block (Sub-CU1) located at the bottom of the coding block can be set in the same way. For example, as shown in (a) of Figure 30, once the horizontal and vertical transform types are determined based on the tu_mts_idx signaled for the top sub-block, the determined transform type can also be applied to the bottom sub-block.

[0411] When a coding block is divided vertically, the transform types of the sub-block (Sub-CU0) located on the left side of the coding block and the sub-block (Sub-CU1) located on the right side of the coding block can be set in the same way. For example, as shown in (b) of Figure 30, once the horizontal and vertical transform types are determined based on the tu_mts_idx signaled for the left sub-block, the determined transform type can also be applied to the right sub-block.

[0412]

[0413] Whether to encode index information may be determined based on at least one of the size, shape, and number of non-zero coefficients of the current block, whether a secondary transform is performed, and whether a sub-partition intra-coding method is applied. For example, if a sub-partition intra-coding method is applied to the current block or if the number of non-zero coefficients is equal to or smaller than a threshold, signaling of index information may be omitted. If signaling of index information is omitted, a default transform type may be applied to the current block.

[0414] The default transform type may include at least one of DCT2 or DST7. If there are multiple default transform types, one of the multiple default transform types may be selected taking into consideration at least one of the size, shape, intra prediction mode, whether or not a secondary transform is performed, or whether or not a sub-partition intra coding method is applied to the current block. For example, one of the multiple transform types may be determined as a horizontal transform type based on whether or not the width of the current block falls within a preset range, and one of the multiple transform types may be determined as a vertical transform type based on whether or not the height of the current block falls within a preset range. Alternatively, the default mode may be determined differently depending on the size, shape, intra prediction mode, or whether or not a secondary transform is performed on the current block.

[0415] Alternatively, if only DC component transform coefficients exist in the current block, the horizontal transform type and the vertical transform type may be set to the default transform type. For example, if only DC component transform coefficients exist in the current block, the horizontal transform type and the vertical transform type may be set to DCT2.

[0416] The threshold may be determined based on the size or shape of the current block. For example, if the size of the current block is smaller than or equal to 32×32, the threshold may be set to 2, and if the current block is larger than 32×32 (e.g., if the current block is a coding block of 32×64 or 64×32 size), the threshold may be set to 4.

[0417] A plurality of lookup tables may be pre-stored in the encoder / decoder. The plurality of lookup tables may differ in at least one of index values assigned to transform type combination candidates, types of transform type combination candidates, or the number of transform type combination candidates.

[0418] The lookup table for the current block can be selected based on at least one of the size, shape, predictive coding mode, intra prediction mode, whether a secondary transform is applied to the current block, or whether a transform skip is applied to the neighboring block.

[0419] As an example, if the size of the current block is 4x4 or less or if the current block is coded using inter-prediction, the first lookup table can be used, and if the size of the current block is greater than 4x4 or if the current block is coded using intra-prediction, the second lookup table can be used.

[0420] Alternatively, information indicating one of a plurality of lookup tables may be signaled via a bitstream, and the decoder can select the lookup table for the current block based on the information.

[0421] As another example, an index assigned to a candidate transform type combination may be adaptively determined based on at least one of the size, shape, predictive coding mode, intra prediction mode, whether a secondary transform is applied to the current block, or whether a transform skip is applied to a neighboring block. For example, if the size of the current block is 4×4, the index assigned to the transform skip may have a smaller value than the index assigned to the transform skip if the size of the current block is larger than 4×4. Specifically, if the size of the current block is 4×4, the transform skip may be assigned an index of 0, and if the current block is larger than 4×4 but smaller than or equal to 16×16, the transform skip may be assigned an index greater than 0 (e.g., index 1). If the current block is larger than 16×16, the transform skip index may be assigned the maximum value (e.g., 5).

[0422] Alternatively, if the current block is coded using inter prediction, the transform skip may be assigned an index of 0. If the current block is coded using intra prediction, the transform skip may be assigned an index greater than 0 (e.g., index 1).

[0423] Alternatively, if the current block is a 4x4 block coded using inter prediction, the transform skip may be assigned an index of 0. On the other hand, if the current block is not coded using inter prediction or is larger than 4x4, the transform skip may be assigned an index of a value larger than 0 (e.g., index 1).

[0424] Transform type combination candidates different from the transform type combination candidates listed in Table 7 may also be defined and used. As an example, a transform type combination candidate may be used in which a transform skip is applied to one of the horizontal transform or the vertical transform, and a transform core such as DCT2, DCT8, or DST7 is applied to the other. Here, it may be determined whether to use a transform skip as a transform type candidate for the horizontal or vertical direction based on at least one of the size (e.g., width and / or height), shape, predictive coding mode, or intra prediction mode of the current block.

[0425]

[0426] Information indicating whether index information for determining the transform type of the current block is explicitly signaled may be signaled via a bitstream. For example, at the sequence level, information sps_explicit_intra_mts_flag indicating whether explicit transform type determination is allowed for a block coded using intra prediction and / or information sps_explicit_inter_mts_flag indicating whether explicit transform type determination is allowed for a block coded using inter prediction may be signaled.

[0427] If explicit transform type determination is allowed, the transform type of the current block may be determined based on index information tu_mts_idx signaled from the bitstream. On the other hand, if explicit transform type determination is not allowed, the transform type may be determined based on at least one of the size and shape of the current block, whether sub-block-based transforms are allowed, the position of a sub-block including a non-zero transform coefficient, whether a second transform is performed, or whether a sub-partition intra-coding method is applied. For example, the horizontal transform type of the current block may be determined based on the width of the current block, and the vertical transform type of the current block may be determined based on the height of the current block. For example, if the width of the current block is less than 4 or greater than 16, the horizontal transform type may be determined to be DCT2. Otherwise, the horizontal transform type may be determined to be DST7. If the height of the current block is less than 4 or greater than 16, the vertical transform type may be determined to be DCT2. Otherwise, the vertical transform type may be determined to be DST7. Here, the thresholds to be compared with the width and height to determine the horizontal transformation type and the vertical transformation type may be determined based on at least one of the size, shape, or intra prediction mode of the current block.

[0428] Alternatively, if the current block is a square with the same height and width, the horizontal and vertical transformation types can be set to be the same, while if the current block is a non-square with different heights and widths, the horizontal and vertical transformation types can be set to be different. For example, if the width of the current block is greater than the height, the horizontal transformation type can be determined to be DST7 and the vertical transformation type can be determined to be DCT2. If the height of the current block is greater than the width, the vertical transformation type can be determined to be DST7 and the horizontal transformation type can be determined to be DCT2.

[0429] The number and / or types of transform type candidates or the number and / or types of transform type combination candidates may differ depending on whether explicit transform type determination is allowed. For example, if explicit transform type determination is allowed, DCT2, DST7, and DCT8 may be used as transform type candidates. As a result, the horizontal transform type and the vertical transform type may be set to DCT2, DST8, or DCT8, respectively. If explicit transform type determination is not allowed, only DCT2 and DST7 may be used as transform type candidates. As a result, the horizontal transform type and the vertical transform type may be determined to be DCT2 or DST7, respectively.

[0430]

[0431] The decoder may perform an inverse transform of the second transform (second inverse transform) and then perform an inverse transform of the first transform (first inverse transform) on the result of the second inverse transform. A residual signal for the current block may be obtained as a result of performing the second inverse transform and the first inverse transform.

[0432] After the encoder performs the transform and quantization, the decoder can obtain the residual block by inverse quantization and inverse transform, and the decoder can add the prediction block and the residual block to obtain the reconstructed block for the current block.

[0433]

[0434] Once a reconstructed block for the current block is obtained, in-loop filtering can be used to reduce information loss that occurs during quantization and encoding. The in-loop filter may include at least one of a deblocking filter, a sample adaptive offset filter (SAO), or an adaptive loop filter (ALF). Hereinafter, the reconstructed block before the in-loop filter is applied is referred to as a first reconstructed block, and the reconstructed block after the in-loop filter is applied is referred to as a second reconstructed block.

[0435] A second reconstructed block can be obtained by applying at least one of a deblocking filter, SAO, or ALF to the first reconstructed block, where SAO or ALF may be applied after the deblocking filter is applied.

[0436] The deblocking filter is used to mitigate the image quality degradation (blocking artifacts) at block boundaries that occur due to block-based quantization. To apply the deblocking filter, the blocking strength (BS) between the first restored block and neighboring restored blocks can be determined.

[0437] FIG. 31 is a flowchart showing the process of determining block strength.

[0438] In the example shown in Figure 31, P indicates a first reconstruction block and Q indicates a neighboring reconstruction block, where the neighboring reconstruction block may be adjacent to the left or top edge of the current block.

[0439] In the example shown in Figure 31, the block strength is determined taking into account the predictive coding modes of P and Q, whether or not they contain non-zero transform coefficients, whether or not they are inter-predicted using the same reference picture, and whether or not the difference value of the motion vector is greater than or equal to a threshold.

[0440] Whether to apply a deblocking filter may be determined based on the block strength. For example, if the block strength is 0, no filtering may be performed.

[0441] SAO is used to mitigate ringing artifacts caused by quantization in the frequency domain. SAO can be performed by adding or subtracting an offset determined based on the pattern of the first restored image. Methods for determining the offset include edge offset (EO) and band offset. EO refers to a method of determining the offset of a current sample based on the pattern of surrounding pixels. BO refers to a method of applying a common offset to a group of pixels with similar brightness values within a region. Specifically, pixel brightness can be divided into 32 equal intervals, and pixels with similar brightness values can be grouped together. For example, four adjacent bands out of 32 bands can be grouped together, and the same offset value can be applied to samples belonging to the four bands.

[0442] ALF is a method of generating a second restored image by applying a filter of a predefined size / shape to a first restored image or a restored image to which a deblocking filter has been applied. Equation 8 below shows an application example of ALF.

[0443]

number

[0444] One of predefined filter candidates may be selected for each picture, coding tree unit, coding block, prediction block, or transform block, and each filter candidate may differ in either size or shape.

[0445] FIG. 32 shows predefined filter candidates.

[0446] As an example shown in FIG. 32, at least one of diamond shapes of 5×5, 7×7 or 9×9 size can be selected.

[0447] For the chroma components, only a 5x5 diamond shape may be used.

[0448]

[0449] For real-time or low-latency encoding of high-resolution video such as panoramic video, 360-degree video, or 4K / 8K UHD (Ultra High Definition) video, a method of dividing a picture into multiple regions and encoding / decoding the multiple regions in parallel can be considered. Specifically, a picture can be divided into tiles or slices (or tile groups) depending on the processing purpose.

[0450] A tile represents the basic unit of parallel encoding / decoding. Each tile may be processed in parallel. A tile may have a rectangular shape. Alternatively, tiles with non-rectangular shapes may be allowed.

[0451] Information indicating whether non-rectangular tiles are allowed or present may be signaled via the bitstream.

[0452] When encoding / decoding a tile, data from other tiles may be set not to be used. This can eliminate encoding / decoding dependencies between tiles and support parallel processing of tiles. Specifically, a CABAC (Context Adaptive Binary Arithmetic Coding) context probability table may be initialized on a tile-by-tile basis, and an in-loop filter may be set not to be applied at tile boundaries. In addition, data in other tiles may not be used as candidates for motion vector guidance. For example, data in other tiles may be set not to be used as merge candidates, motion vector prediction candidates (AMVP candidates), or motion information candidates. In addition, data in other tiles may be set not to be used for symbol context calculation.

[0453]

[0454] Video encoding / decoding information may be signaled via a slice header. The information signaled via the slice header may be commonly applied to coding tree units or tiles included in the slice. A slice may also be referred to as a tile group.

[0455] FIG. 33 is a diagram illustrating a picture division method according to one embodiment of the present invention.

[0456] First, it may be determined whether the current picture is to be divided into multiple processing units (S3310). Here, the processing units may include at least one of tiles or slices. As an example, a syntax element no_pic_partition_flag indicating whether the current picture is to be divided into multiple tiles or slices may be signaled via the bitstream. A value of 0 for the syntax element no_pic_partition_flag indicates that the current picture is to be divided into at least one tile or at least one slice. On the other hand, a value of 1 for the syntax element no_pic_partition_flag indicates that the current picture is not to be divided into multiple tiles or multiple slices.

[0457] If it is determined that the current picture is not to be divided into multiple processing units, the division process of the current picture may be terminated. In this case, the current picture may be considered to be composed of one tile and one slice (or tile group).

[0458] Alternatively, information indicating whether multiple tiles exist in a picture may be signaled via a bitstream, which may include at least one of a one-bit flag indicating whether multiple tiles exist in a picture or information specifying the number of tiles in a picture.

[0459] If it is determined that the current picture is to be divided into multiple processing units, tile division information may be signaled via the bitstream, and the picture may be divided into at least one tile based on the signaled tile division information (S3320).

[0460] If the current picture is divided into multiple tiles, the tiles can be merged or one tile can be divided to determine the slices (S3330).

[0461] The tile division method and slice determination method according to the present invention will be described in detail below.

[0462] FIG. 34 shows an example in which a picture is divided into multiple tiles.

[0463] A tile may include at least one coding tree unit. The boundaries of the tile may be set to coincide with the boundaries of the coding tree unit. That is, a division shape in which one coding tree unit is divided into multiple pieces may not be allowed.

[0464] When a picture is divided into a plurality of tiles, the height or width of adjacent tiles can be set to have the same value.

[0465] For example, the heights of tiles belonging to the same tile row and / or the widths of tiles belonging to the same tile column may be set to be the same, as in the example shown in Figure 34. Tiles belonging to the same tile row may also be referred to as a horizontal tile set, and tiles belonging to the same tile column may also be referred to as a vertical tile set.

[0466] Alternatively, information indicating whether the width and / or height of the tile to be encoded / decoded is set to be the same as the width and / or height of the previous tile may be signaled.

[0467] Information indicating the partition shape of a picture may be signaled via the bitstream: the information may be coded and signaled via a picture parameter set, a sequence parameter set, or a slice header.

[0468] Information for determining the size of a tile may be coded and signaled. For example, a syntax element tile_width_minus1[i] indicating the width of the i-th tile column and a syntax element tile_height_minus1[i] indicating the height of the i-th tile row may be coded into the bitstream.

[0469] If the sizes of consecutive tile columns are the same, width information may be signaled for only one of the consecutive tile columns, and encoding / decoding of width information for the remaining tile columns may be omitted.

[0470] As an example, if the widths of the i-th tile column to the last tile column in a picture are the same, the syntax tile_width_minus1[i] can be signaled only for the i-th tile column, and the remaining tile columns can be set to the same width as the i-th tile column.

[0471] Information indicating whether the widths of multiple consecutive tile columns are the same can be coded and signaled. For example, the syntax element rem_tile_col_equal_flag[i] indicates whether the widths of the tile columns from index i to index N-1 are the same. Here, N may indicate the number of tile columns. That is, the tile column with index N-1 may indicate the last tile column in the picture.

[0472] When the syntax rem_tile_col_equal_flag[i] is 1, the syntax tile_width_minus1[i] indicating the width is signaled for the tile column with index i, while the encoding / decoding of the syntax indicating the width of the tile column can be omitted for tile columns with index greater than i.

[0473] If the encoding / decoding of the syntax indicating the width of a tile column is omitted, the width of the next tile column can be set to the minimum value of the width of the i-th tile column and the width of the remaining area. As an example, if the width of the i-th tile column is set to the variable PrevTileColWidth and the width of the remaining area in the picture is set to the variable RemainingColWidth, the size of tile columns with indices greater than i may be set to the smaller value of the variable PrevTileColWidth and the variable RemainingColWidth.

[0474] As another example, the syntax rem_tile_col_equal_flag[i] can be set to indicate whether the widths from the tile column with index 0 to the tile column with index i are the same.

[0475] If consecutive tile rows have the same size, height information may be signaled for only one of the consecutive tile rows, and encoding / decoding of height information for the remaining tile rows may be omitted.

[0476] As an example, if the heights of the ith to last tile rows in a picture are the same, the syntax tile_height_minus1[i] can be signaled only for the ith tile row, and the remaining tile rows can be set to the same height as the ith tile row.

[0477] Information indicating whether the heights of consecutive tile rows are equal can be coded and signaled. For example, the syntax element rem_tile_row_equal_flag[i] indicates whether the heights of the tile rows from the tile row with index i to the tile row with index N-1 are equal. Here, N may indicate the number of tile rows. That is, the tile row with index N-1 may indicate the last tile row in the picture.

[0478] When the syntax rem_tile_row_equal_flag[i] is 1, the syntax tile_height_minus1[i] indicating the height is signaled for the tile row with index i, while the encoding / decoding of the syntax indicating the height of the tile row can be omitted for tile rows with index greater than i.

[0479] If the encoding / decoding of the syntax indicating the height of a tile row is omitted, the minimum value of the height of the i-th tile row and the height of the remaining area can be set as the height of the next tile row. As an example, if the height of the i-th tile row is set to the variable PrevTileRowHeight and the height of the remaining area in the picture is set to the variable RemainingRowHeight, the size of tile rows with indexes greater than i may be set to the smaller value of the variable PrevTileRowHeight and the variable RemainingRowHeight.

[0480] As another example, the syntax rem_tile_row_equal_flag[i] can be set to indicate whether the heights of the tile row with index 0 to the tile row with index i are the same.

[0481] Table 8 illustrates a syntax structure including the syntax rem_tile_col_equal_flag[i] and the syntax rem_tile_row_equal_flag[i].

[0482] [Table 8]

[0483] Information for specifying the number of tile columns in the current picture whose widths are explicitly signaled may also be signaled via a bitstream. As an example, a syntax element num_exp_tile_columns_minus1 for determining the number of tile columns whose widths are signaled may be signaled via a bitstream. The syntax element num_exp_tile_columns_minus1 may be a value obtained by subtracting 1 from the number of tile columns whose widths are signaled.

[0484] The number of syntax elements for specifying the width of tile columns determined based on the syntax element num_exp_tile_columns_minus1 may be coded and signaled. As an example, the syntax element tile_width_minus1[i] indicating the width of the i-th tile column may be signaled via a bitstream. The syntax element tile_width_minus1[i] may be obtained by subtracting 1 from the number of coding tree unit columns included in the tile column.

[0485] If the tile column index i is smaller than the number of tile columns whose width is explicitly signaled, the width of the tile column may be determined based on the syntax tile_width_minus1[i] signaled via the bitstream.

[0486] On the other hand, if the index j of a tile column is equal to or greater than the number of tile columns whose widths are explicitly signaled, the width of the tile column may be determined based on the syntax tile_width_minus1[l] that was last signaled. Here, l indicates the index of the tile column whose width was last signaled and may be an integer smaller than j. For example, if the value obtained by subtracting the width of the previous tile column from the width of the current picture is equal to or greater than the value obtained by adding 1 to the syntax tile_width_minus1[l], the width of tile column j may be set to the value obtained by adding 1 to the syntax tile_width_minus1[l]. On the other hand, if the value obtained by subtracting the width of the previous tile column from the width of the current picture is smaller than the value obtained by adding 1 to the syntax tile_width_minus1[l], the difference value obtained by subtracting the width of the previous tile column from the width of the current picture may be set to the width of tile column j.

[0487] Table 9 shows the process for determining the width of a tile column.

[0488] [Table 9]

[0489] In Table 9, the variable PicWidthInCtbsY indicates the number of coding tree unit sequences included in the current picture. As an example, the variable PicWidthInCtbsY may be derived as shown in Equation 9 below.

[0490]

number

[0491] The remainingWidthInCtbsY variable represents the PicWidthInCtbsY variable minus the accumulated width of the tile column. For example, for a tile column with index i, remainingWidthInCtbsY may be derived from PicWidthInCtbsY by subtracting the sum of the widths of tile columns 0 through i-1.

[0492] As shown in the example in Table 9, the variable uniformTileColWidth may be derived by adding 1 to the width of the last explicitly signaled tile column, tile_column_width_minus1 [num_exp_tile_columns_minus1]. Here, if the variable remainingWidthInCtbY indicating the number of remaining coding tree unit columns in the current picture is equal to or greater than the variable uniformTileColWidth, the remaining area may be divided into the size of the variable uniformTileColWidth.

[0493] On the other hand, if the variable remainingWidthInCtbY indicating the number of remaining coding tree unit columns in the current picture is smaller than the variable uniformTileColWidth, the remaining area may be set as the last tile column.

[0494] On the other hand, if the tile column index j is greater than or equal to the number of tile columns whose widths are explicitly signaled, then the width of that tile column may be set to the smaller of the variables uniformTileColWidth and remainingWidthInCtbY.

[0495] That is, the widths of the remaining tile columns other than the tile column whose width is explicitly signaled may have a value smaller than or equal to the width of the last tile column among the tile columns whose width is explicitly signaled.

[0496] Alternatively, information for specifying the number of tile rows for which the heights of tile rows in the current picture are signaled may be signaled via a bitstream. As an example, a syntax element num_exp_tile_rows_minus1 for determining the number of tile rows for which the heights of tile rows are signaled may be signaled via a bitstream. The syntax element num_exp_tile_rows_minus1 may be a value obtained by subtracting 1 from the number of tile rows for which the heights of tile rows are signaled.

[0497] The number of syntax elements for specifying the height of the tile rows determined based on the syntax element num_exp_tile_rows_minus1 may be coded and signaled. As an example, the syntax element tile_height_minus1[i] indicating the height of the i-th tile row may be signaled via the bitstream. The syntax element tile_height_minus1[i] may be the number of coding tree unit rows included in the tile row minus 1.

[0498] If the tile row index i is less than the number of tile rows whose height is explicitly signaled, the height of that tile row may be determined based on the syntax tile_height_minus1[i] signaled via the bitstream.

[0499] On the other hand, if the tile row index j is equal to or greater than the number of tile rows whose heights are explicitly signaled, the height of the tile row may be determined based on the last signaled syntax tile_height_minus1[l], where l indicates the index of the tile row whose height was last signaled and may be an integer smaller than j.

[0500] As an example, if the value obtained by subtracting the height of the previous tile row from the height of the current picture is equal to or greater than the value obtained by adding 1 to the syntax tile_height_minus1[l], the height of tile row j may be set to the value obtained by adding 1 to the syntax tile_height_minus1[l]. On the other hand, if the value obtained by subtracting the height of the previous tile row from the height of the current picture is less than the value obtained by adding 1 to the syntax tile_height_minus1[l], the height of tile row j may be set to the difference value obtained by subtracting the height of the previous tile row from the height of the current picture.

[0501] Table 10 shows the process for determining the height of a tile row.

[0502] [Table 10]

[0503] In Table 10, the variable PicHeightInCtbsY indicates the number of coding tree unit rows included in the current picture. As an example, the variable PicWidthInCtbsY may be derived as shown in Equation 10 below.

[0504]

number

[0505] The remainingHeightInCtbsY variable represents the PicHeightInCtbsY variable minus the cumulative tile row height. For example, for tile row index i, remainingHeightInCtbsY may be derived from PicHeightInCtbsY by subtracting the sum of the heights of tile rows 0 through i-1.

[0506] As shown in the example in Table 10, the variable uniformTileRowHeight may be derived by adding 1 to the height of the last explicitly signaled tile row, tile_row_height_minus1 [num_exp_tile_rows_minus1]. Here, if the variable remainingHeightInCtbY indicating the number of remaining coding tree unit rows in the current picture is equal to or greater than the variable uniformTileRowHeight, the remaining area may be divided into the size of the variable uniformTileRowHeight.

[0507] On the other hand, if the variable remainingHeightInCtbY indicating the number of remaining coding tree unit rows in the current picture is smaller than the variable uniformTileRowHeight, the remaining area may be set as the last tile row.

[0508] On the other hand, if the tile row index j is greater than or equal to the number of tile rows whose heights are explicitly signaled, then the height of that tile row may be set to the smaller of the variables uniformTileRowHeight and remainingHeightInCtbY.

[0509] That is, the heights of the remaining tile rows other than the tile row whose height is explicitly signaled may be smaller than or equal to the width of the last tile row among the tile rows whose height is explicitly signaled.

[0510] Table 11 illustrates an example syntax table that includes syntax indicating the number of tile columns whose width is explicitly signaled and syntax indicating the number of tile rows whose height is explicitly signaled.

[0511] [Table 11]

[0512] On the other hand, the syntax element loop_filter_across_tiles_enabled_flag indicates whether or not application of an in-loop filter is permitted at tile boundaries within a picture that references the picture parameter set. Here, the in-loop filter may include at least one of a deblocking filter, an ALF, or an SAO. A value of 1 for the flag loop_filter_across_tiles_enabled_flag indicates that application of an in-loop filter across tile boundaries within a picture that references the picture parameter set is permitted. On the other hand, a value of 0 for the flag loop_filter_across_tiles_enabled_flag indicates that application of an in-loop filter is not permitted at tile boundaries within a picture that references the picture parameter set.

[0513] The syntax element loop_filter_across_slices_enabled_flag indicates whether application of an in-loop filter is permitted at slice boundaries in a picture that references the picture parameter set. Here, the in-loop filter may include at least one of a deblocking filter, an ALF, or an SAO. A value of 1 for the flag loop_filter_across_slices_enabled_flag indicates that application of an in-loop filter across slice boundaries in a picture that references the picture parameter set is permitted. On the other hand, a value of 0 for the flag loop_filter_across_slices_enabled_flag indicates that application of an in-loop filter is not permitted at slice boundaries in a picture that references the picture parameter set.

[0514]

[0515] Tiles can be split recursively, for example, a single tile can be split into multiple tiles.

[0516] Each of the multiple tiles generated by dividing a tile may be referred to as a subtile or a brick. A brick may be a unit of parallel processing. For example, bricks may be encoded / decoded independently of each other. When encoding / decoding a block included in a brick, data of other blocks may be set not to be used. For example, samples included in other bricks may be set to be unavailable as reference samples for intra prediction. Alternatively, data in other bricks may be set not to be used as merge candidates, motion vector prediction candidates (AMVP candidates), or motion information candidates. Alternatively, data in other bricks may not be used for symbol context calculation.

[0517] FIG. 35 is a diagram for explaining the generation aspect of bricks.

[0518] Tiles can be split horizontally to generate bricks. For example, in the example shown in Figure 35, tiles belonging to the last tile column in the current picture are shown to be split into two bricks.

[0519] If raster scanning is applied to tiles, raster scanning may also be applied between bricks. For example, after all bricks included in a particular tile are scanned, the next tile may be scanned. In other words, a brick may have the same status as a tile.

[0520] The boundaries of a brick may coincide with the boundaries of a coding tree unit, i.e., at least one coding tree unit row in a tile may be defined as a brick.

[0521] Tiles and bricks may be identified by a tile index. A tile index may be assigned to each tile and brick according to the raster scan order. If a tile is divided into multiple bricks, a tile index may be assigned to each of the multiple bricks.

[0522] In the embodiments described below, the term "tile" may include not only tiles but also tiles generated by dividing tiles (i.e., subtiles or bricks).

[0523]

[0524] At least one tile may be defined as one processing unit. For example, a plurality of tiles may be defined as one slice. A slice may also be called a tile group.

[0525] Alternatively, one tile may be divided into a plurality of processing units. For example, one tile may be divided into a plurality of slices. Here, one slice may include at least one coding tree unit sequence. When one tile is divided into a plurality of slices, information indicating the height of each slice may be signaled via a bitstream.

[0526] Video encoding / decoding information may be signaled via a slice header, and the information signaled via the slice header may apply commonly to tiles and / or blocks belonging to the slice.

[0527] Information indicating the slice type can be signaled via a bitstream. The information indicates how a slice in the current picture is defined. As an example, a syntax element rect_slice_flag indicating the slice type may be signaled via a bitstream.

[0528] The syntax rec_slice_flag indicates whether slices are defined based on the raster scan order of tiles or whether they are defined as rectangular slices. For example, when rec_slice_flag is 0, slices are defined based on the raster scan order of tiles. On the other hand, when rec_slice_flag is 1, slices are defined as rectangular slices.

[0529] The above two methods for determining slices will now be described in detail.

[0530] The raster scan-based definition method identifies at least one tile according to a raster scan order, and then defines the identified at least one tile as a slice. In the raster scan-based definition method, one or more consecutive tiles may be defined as a slice. Here, the consecutive tiles may be determined according to the raster scan order. When the raster scan slice is applied, a non-rectangular slice may also be generated.

[0531] 36 and 37 are diagrams showing examples in which slices are defined based on the raster order.

[0532] As an example, in the example shown in FIG. 36, if the first slice "slice0" includes three tiles, the first slice "slice0" may be defined as including Tile0 to Tile2 in raster scan order. If the second slice "slice1" includes six tiles, the second slice "slice1" may be defined as including Tile3 to Tile8 in raster scan order. The last slice "slice2" may include the remaining tiles "Tile9 to Tile11" in raster scan order.

[0533] When slices are defined based on the raster scan order, information indicating the number of tiles included in each slice may be signaled, and for the last slice, signaling of information indicating the number of tiles included in the slice may be omitted.

[0534] When a slice includes multiple tiles, the tiles may have different widths or heights. For example, the height of Tile3 in the second slice, slice1, is shown as being different from the remaining tiles.

[0535] The quadrilateral slice definition method is a division method that allows only rectangular slices. When the quadrilateral slice definition method is applied, tiles located at the four corners of a slice belong to the same row or the same column.

[0536] FIG. 38 shows an example in which only rectangular slices are allowed.

[0537] 38, the fourth slice, slice3, includes Tile5, Tile6, Tile9, and Tile10. When a slice includes multiple tiles, as in the example shown, a rectangle with the upper left tile and lower right tile constituting the slice as its two vertices can be defined as one slice.

[0538] The boundaries of a slice can coincide with picture boundaries and / or tile boundaries. As an example, the left or top boundary of a slice may be set to a picture boundary, or the left or top boundary of a slice may be set to a tile boundary.

[0539] Alternatively, when a method for defining rectangular slices is applied, one tile can be divided into a plurality of rectangular slices.

[0540] When a rectangular slice definition method is applied (e.g., when rect_slice_flag is 1), information indicating whether a picture consists of a single slice may be signaled. As an example, a syntax element one_slice_in_pic_flag indicating whether the number of slices in a picture is one may be signaled via a bitstream. When the flag one_slice_in_pic_flag is 1, it indicates that the picture consists of one slice. On the other hand, when the flag one_slice_in_pic_flag is 0, it indicates that the picture consists of at least two slices. When the value of the flag one_slice_in_pic_flag is 0, slice configuration information may be further signaled. As an example, Table 12 illustrates a syntax table including the syntax element one_slice_in_pic_flag.

[0541] [Table 12]

[0542] As illustrated, when one_slice_in_pic_flag is 1, encoding of information about the slice configuration (described later, for example, syntax top_left_brick_idx[i] indicating the index of the tile included in the slice and / or syntax bottom_right_bric_idx_delta[i] indicating the index difference between the tile included in the first slice and the tile included in the second slice) may be omitted. That is, one_slice_in_pic_flag may be used to determine whether information about the slice configuration is encoded.

[0543] A picture may also indicate sub-pictures. A picture may be divided into at least one sub-picture. Sub-picture division-related information may be signaled at a sequence level. As an example, sub-picture division-related information signaled via a sequence parameter set may be commonly applied to all pictures that refer to the sequence parameter set.

[0544] The sub-picture division related information may include information indicating whether a picture is divided into multiple sub-pictures, information indicating the number of sub-pictures, information indicating the size of the sub-pictures, or information indicating whether the sub-pictures function as independent pictures.

[0545] For each subpicture, a syntax element one_slice_in_pic_flag may be signaled to indicate whether or not the subpicture is composed of a single slice.

[0546] Alternatively, at the sequence or picture level, a syntax element "one_slice_in_pic_flag" may be signaled to indicate whether each subpicture is composed of a single slice. For example, a flag "one_slice_in_pic_flag" of 1 indicates that all subpictures in the current picture are composed of a single slice. In this case, coding of slice configuration information may be omitted, and slices may be defined based on subpicture partition information.

[0547] On the other hand, if the flag one_slice_in_pic_flag is 0, it indicates that at least one sub-picture in the current picture is made up of multiple slices. In this case, slice configuration information may be coded separately from the sub-picture partition information.

[0548] When a picture is determined to be composed of multiple slices, a syntax indicating the number of slices in the picture may be coded and signaled. As an example, a syntax num_slices_in_pic_minus2 indicating the number of slices in the picture may be signaled via the bitstream. The syntax num_slices_in_pic_minus2 may indicate a value obtained by subtracting 2 from the number of slices in the picture. This allows the decoder to determine the total number of slices by adding 2 to the value indicated by the syntax num_slices_in_pic_minus2.

[0549] Instead of the syntax num_slices_in_pic_minus2, it is also possible to code the syntax num_slices_in_pic_minus1, which indicates a value obtained by subtracting 1 from the number of slices in a picture.

[0550] When a rectangular slice definition method is applied, information for identifying tiles included in each slice may be signaled to determine the tiles included in each slice. The information may be used to identify at least one of the first tile or the last tile of the slice. The order between tiles may be determined according to a predetermined scan order. For example, in the raster scan order, the first tile refers to the tile located at the upper left corner of the slice, and the last tile refers to the tile located at the lower right corner of the slice.

[0551] Information for identifying at least one of an index of a tile located at the upper left corner of a slice or an index of a tile located at the lower right corner of a slice may be signaled via a bitstream. For example, at least one of a syntax "top_left_tile_idx" for identifying an index of a tile located at the upper left corner of a slice or a syntax "bottom_right_tile_idx" for identifying an index of a tile located at the lower right corner of a slice may be signaled via a bitstream. For the last slice, coding of at least one of the syntax for identifying an index of a tile located at the upper left corner of a slice or the syntax for identifying an index of a tile located at the lower right corner of a slice may be omitted. The remaining area in the current picture other than the area occupied by the previous slice may be set as the last slice.

[0552] As an example, in the example shown in Figure 38, for slice0, slice1, and slice2, the index of the tile located at the upper left corner of each slice, top_left_tile_idx, and the index of the tile located at the lower right corner of each slice, bottom_right_tile_idx, may be signaled. On the other hand, for slice3, which is the last slice in the picture, coding of top_left_tile_idx and bottom_right_tile_idx may be omitted. The index of the upper left tile of slice3, which is the last slice in the picture, may be set to the index of the tile located at the upper left corner in the remaining area other than slice0, slice1, and slice2 in the picture, and the index of the lower right tile of slice3 may be set to the index of the tile located at the lower right corner in the remaining area (or the tile located at the lower right corner of the picture).

[0553] Alternatively, differential information for identifying the index of a tile included in a slice may be coded and signaled. For example, information indicating a differential value between the index of a tile located at the upper left corner of a slice and the index of a tile located at the lower right corner of the slice may be signaled via a bitstream. For example, for the i-th slice, a syntax element "top_left_tile_idx[i]" for identifying the index of the tile located at the upper left corner of the slice and a syntax element "bottom_right_tile_idx_delta[i]" indicating the difference between the index of the tile located at the upper left corner of the slice and the index of the tile located at the lower right corner of the slice may be signaled. The index of the tile located at the lower right corner of the i-th slice may be derived by adding together "top_left_tile_idx[i]" and "bottom_right_tile_idx_delta[i]." For the last slice, coding of at least one of the syntax elements indicating the index of the tile located at the upper left corner of the slice or the syntax elements indicating the difference between the index of the tile located at the upper left corner of the slice and the index of the tile located at the lower right corner of the slice may be omitted. The remaining area other than the area occupied by the previous slice in the current picture may be set as the last slice.

[0554] Alternatively, at least one of horizontal tile index difference information and vertical tile index difference information may be coded and signaled. The horizontal tile index difference information may indicate a difference between an index of a first tile and an index of a rightmost tile included in the same tile row as the first tile. The vertical tile index difference information may indicate a difference between an index of a first tile and an index of a bottommost tile included in the same tile column as the first tile.

[0555] The type of difference information to be coded may be determined depending on the position of the slice. For example, for a slice adjacent to the right or bottom boundary of a picture, horizontal tile index difference information or vertical index difference information may be coded and signaled. By coding horizontal tile index difference information or vertical index difference information instead of information indicating the index difference between the top-left tile and the top-right tile, the amount of bits can be reduced.

[0556] Alternatively, when a picture is divided into at least one slice row, information indicating an index difference between the top left tile and the bottom right tile may be coded for a slice adjacent to the left boundary of the picture. Slices belonging to the same row may be set to have the same height.

[0557] It is also possible to encode and signal difference information between the index of a tile included in the first slice and the index of a tile included in the second slice, where the first slice and the second slice are determined based on the scanning order, and the tiles occupied by each slice may be different.

[0558] For example, if the tiles constituting the first slice are different from the tiles constituting the second slice, the index of the second slice may be the index i of the first slice plus 1.

[0559] Alternatively, when a first tile is divided to include multiple slices, differential information between a first slice included in the first tile and a second slice including a second tile that is the next tile to the first tile in the scanning order or a second slice included in the second tile may be encoded. Here, differential information may be encoded only for the first slice or the last slice among the multiple slices included in the first tile.

[0560] The tile used to derive the difference information between the first slice and the second slice may be located at the top left, top right, bottom right, bottom left, or center of the slice.

[0561] Table 13 illustrates an example syntax table that includes differential information.

[0562] [Table 13]

[0563] As an example, a syntax element top_left_brick_idx_delta[i] may be signaled via the bitstream, indicating the index difference between the top left tile of the i-th slice and the top left tile of the previous slice (i.e., the i-1-th slice).

[0564] Based on the differential information, the position of the top-left tile in the i-th slice may be determined. Specifically, the index of the top-left tile of the i-th slice can be derived by summing the index of the top-left tile of the (i-1)-th slice and the syntax top_left_brick_idx_delta[i]. Equation 11 shows an example of deriving the index of the top-left tile of the i-th slice.

[0565]

number

[0566] In Equation 11, TopLeftBrickIdx[i-1] indicates the index of the top left tile of the i-1th slice.

[0567] For the first slice (i.e., the slice with index i=0), coding of the syntax element top_left_brick_idx_delta[i] indicating the difference information from the previous slice can be omitted. For the first slice, coding of the syntax element bottom_right_brick_idx_delta[i] indicating the index difference between the top-left tile and the top-right tile of the slice can be signaled.

[0568] Table 14 is an example showing the process of using the difference information to identify the slice to which each tile belongs.

[0569] [Table 14]

[0570] NumBricksInSlice[i] indicates the number of tiles contained in slice i. TopLeftBrickIdx[i] indicates the index of the top-left tile in slice i. botRightBkIdx indicates the index of the top-right tile. BrickColBd[j] indicates the index of the tile column to which tile j belongs. BrickRowBd[j] indicates the index of the tile row to which tile j belongs. BricksToSliceMap[j]=i indicates that tile j is to be added to slice i.

[0571] Instead of the syntax "top_left_brick_idx_delta[i]," the syntax "bottom_right_brick_idx_delta[i]" indicating the index difference between the bottom right tile of the i-th slice and the bottom right tile of the previous slice (i.e., the (i-1)-th slice) may be signaled via the bitstream. In this case, the index of the bottom right tile of the i-th slice may be derived by adding the syntax "bottom_right_brick_idx_delta[i]" to the index of the bottom right tile of the previous slice.

[0572] The bottom_right_brick_idx_delta[0] for the first slice may be set equal to the index of the bottom-right tile of the first slice, i.e., the index of the bottom-right tile of the first slice may be set equal to the syntax bottom_right_brick_idx_delta[0].

[0573] Information indicating whether the index of a tile included in the i-th slice in a picture is greater than the index of a tile included in the (i-1)-th slice may be coded and signaled via the bitstream. As an example, a syntax element bottom_right_increasing_order_flag may be signaled via the bitstream. When the flag bottom_right_increasing_order_flag is 1, it indicates that the index of a tile included in the i-th slice in a picture is always greater than the index of a tile included in the (i-1)-th slice. On the other hand, when the flag bottom_right_increasing_order_flag is 0, it indicates that the index of a tile included in at least one slice in a picture may be less than the index of a tile included in the previous slice.

[0574] A slice can be defined using information specifying the index of the top-left tile in the slice and at least one of the width and height of the slice. For example, at least one of the syntax slice_width_in_tiles_minus1[i] indicating the width of the i-th slice or the syntax slice_height_in_tiles_minus1[i] indicating the height of the i-th slice may be signaled via the bitstream.

[0575] The syntax slice_width_in_tiles_minus1[i] indicates the number of tile columns included in the i-th slice minus 1. The syntax slice_height_in_tiles_minus[i] indicates the number of tile rows included in the i-th slice minus 1.

[0576] The i-th slice may consist of a number of tile columns determined based on the syntax slice_width_in_tiles_minus1[i] and a number of tile rows determined based on the syntax slice_height_in_tiles_minus1[i], where the top-left tile of the i-th tile column may have an index value determined based on top_left_brick_idx_delta[i].

[0577] Alternatively, the syntax element slice_height_in_tiles_minus1[i] indicating the slice height may be signaled only for slices adjacent to the left boundary of the picture, and coding of the syntax element slice_height_in_tiles_minus1[i] may be omitted for other slices. The height of a slice for which coding of the syntax element slice_height_in_tiles_minus[i] is omitted may be set to the same height as the slice that is adjacent to the left boundary of the current picture among the slices included in the same row.

[0578]

[0579] The index of the top-left tile of the first slice, TopLeftBrickIdx[0], may be set to 0. As a result, in the second slice (i.e., the slice with index i=1), the syntax top_left_brick_idx_delta[i] effectively has the same value as the index of the top-left tile. As a result, for the second slice, the syntax top_left_brick_idx[i] can be signaled instead of the syntax top_left_brick_idx_delta[i], where top_left_brick_idx[i] indicates the index of the top-left tile of the i-th slice.

[0580] As another example, the syntax element top_left_brick_idx_delta[i] may be set to the difference between the first tile of the i-th slice and the first tile of the (i+1)-th slice. That is, the index of the top left tile of the (i+1)-th slice may be derived by adding the index of the top left tile of the i-th slice and the syntax element top_left_brick_idx_delta[i] signaled for the i-th slice. When the syntax element top_left_brick_idx_delta[i] indicates difference information between the i-th slice and the (i+1)-th slice, coding of the syntax element top_left_brick_idx_delta[i] for the last slice may be omitted.

[0581] A syntax table such as Table 15 below can also be used to determine the slice.

[0582] [Table 15-1] [Table 15-2]

[0583] A flag rect_slice_flag indicating the slice type is coded, and if the flag rect_slice_flag is 1, a syntax element num_slices_in_pic_minus1 indicating the number of slices in a picture can be coded.

[0584] If it is determined that a picture includes multiple slices, the syntax slice_width_in_tiles_minus1[i] indicating the width of each slice and the syntax slice_height_in_tiles_minus1[i] indicating the height of each slice can be coded and signaled.

[0585] It is also possible to signal the syntax tile_idx_delta[i], which indicates the difference between the index of the tile included in the i-th slice and the index of the tile included in the (i+1)-th slice. For the last tile, coding of the syntax tile_idx_delta[i] may be omitted.

[0586] It is also possible to encode a syntax derived by subtracting an offset from the tile index difference between two slices. The offset can be a natural number such as 1 or 2. As an example, the syntax tile_idx_delta_minus1[i] may be signaled via the bitstream. The difference between the index of the tile included in the i-th slice and the index of the tile included in the (i+1)-th slice may be derived by adding 1 to the syntax tile_idx_delta_minus1[i].

[0587] The tile index difference information may be coded / decoded when the current picture includes multiple slices. That is, when the current picture is composed of a single slice, the signaling of the syntax tile_idx_delta[i] indicating the tile index difference may be omitted. For example, when the flag one_slice_in_pic_flag indicating whether the current picture is composed of a single slice is 1, the signaling of the syntax tile_idx_delta[i] may be omitted.

[0588] Alternatively, the encoder can determine whether to encode the syntax element tile_idx_delta[i], which indicates the difference between tile indexes, and, based on the determination, encode a flag tile_idx_delta_present_flag, which indicates whether the syntax element tile_idx_delta[i] is encoded. A value of 0 for the flag tile_idx_delta_present_flag indicates that the syntax element tile_idx_delta[i] has not been encoded / decoded. A value of 1 for the flag tile_idx_delta_present_flag may indicate that the syntax element tile_idx_delta[i] is encoded / decoded for at least one slice.

[0589] A single tile can also be divided into multiple slices. For example, the tile can be divided horizontally to generate multiple slices.

[0590] If it is determined that a slice does not contain multiple tiles, it may be determined whether to divide the tile into multiple slices. For example, if the syntax slice_width_in_tiles_minus1[i] and the syntax slice_height_in_tiles_minus1[i] are both 0, this indicates that the slice consists of only one tile or that one tile is divided into multiple slices.

[0591] If the syntax slice_width_in_tiles_minus1[i] and the syntax slice_height_in_tiles_minus1[i] are both 0, information indicating whether the tile is divided into multiple slices may be signaled.

[0592] As an example, a syntax element num_slices_in_tile_minus1[i] indicating the number of slices included in a tile may be signaled. The syntax element num_slices_in_tile_minus1[i] indicates a value obtained by subtracting 1 from the number of slices included in the i-th tile.

[0593] When a tile including the i-th slice is composed of multiple slices, syntax indicating the height of each slice in the tile may be coded. As an example, syntax "slice_height_in_ctu_minus1[i][j]" may be signaled to indicate the height of the j-th slice (i.e., the i+j-th slice) in the tile including the i-th slice. The syntax "slice_height_in_ctu_minus1[i][j]" may indicate the number of coding tree unit rows included in the j-th slice in the tile including the i-th slice. Table 16 illustrates an example of a syntax structure including syntax "slice_height_in_ctu_minus1[i][j]" indicating the height of a slice.

[0594] [Table 16-1] [Table 16-2]

[0595] Here, whether or not to encode / decode the syntax indicating the number of slices in a tile may be determined based on the tile height. As an example, if a tile including an i-th slice is configured with a single coding tree unit row, encoding / decoding of the syntax num_slices_in_tile_minus1 indicating the number of slices in the tile may be omitted. When encoding / decoding of the syntax num_slices_in_tile_minus1 is omitted, its value may be regarded as 0. That is, the i-th slice and the tile including the i-th slice may be set to have the same size and shape. In Table 16, the variable TileIdx[i] indicates the index of the tile including the i-th slice, and the variable NumTileColumn indicates the number of tile columns. TileIdx[i] / NumTileColumn returns the row including the tile including the i-th slice. The variable RowHeight[TileIdx[i] / NumTileColumn] indicates the height of the tile or tile row including the i-th slice.

[0596] Here, since tiles belonging to the same tile row have the same height, it can also be understood that whether or not the syntax indicating the number of slices in a tile can be coded / decoded is determined based on the height of the tile row.

[0597] Whether or not the syntax indicating the height of a slice can be coded / decoded can be determined based on the height of the tile or tile row. For example, if the tile including the i-th slice is configured with a single coding tree unit row, or if the tile including the i-th slice is configured with two coding tree unit rows, coding / decoding of the syntax slice_height_in_ctu_minus1[i][j] indicating the height of the slice can be omitted. The height of the syntax slice_height_in_ctu_minus1[i][j] may be adaptively determined depending on the number of slices in the tile. For example, if the number of slices in the tile is one, the height of the slice may be set to the same as the height of the tile. Alternatively, if the number of slices in the tile is two and the height of the tile is two, the height of each slice may be set to 1. In Table 16, the variable TileIdx[i] indicates the index of the tile including the i-th slice, and the variable NumTileColumn indicates the number of tile columns. TileIdx[i] / NumTileColumn returns the row that contains the tile that contains the i-th slice. The variable RowHeight[TileIdx[i] / NumTileColumn] indicates the height of the tile or tile row that contains the i-th slice.

[0598] Instead of the syntax element num_slices_in_tile_minus1 indicating the number of slices described above, a syntax element num_exp_slices_in_tile[i] indicating information indicating the number of slice heights that needs to be explicitly signaled can also be coded. The syntax element num_exp_slices_in_tile[i] may have a value equal to or smaller than the number of slices included in the tile.

[0599] The syntax num_exp_slices_in_tile[i] being 0 indicates that the tile is not divided into multiple slices. The syntax num_exp_slices_in_tile[i] being greater than 0 indicates that the tile may be divided into multiple slices.

[0600] Information indicating the height of slices may be signaled as many times as the number of syntax elements num_exp_slices_in_tile[i]. As an example, when the syntax element num_exp_slices_in_tile is greater than 1, the syntax element exp_slice_height_in_ctu_minus1[j] indicating the height of the j-th slice in the tile may be signaled.

[0601] As in the above-described embodiment, whether or not the number information exp_slice_height_in_ctu_minus1[j] can be coded / decoded may be determined based on the height of the tile or tile row. As an example, if the tile including the i-th slice is configured with a single coding tree unit row, coding / decoding of the syntax exp_slice_height_in_ctu_minus1[j] may be omitted. If coding / decoding of the syntax exp_slice_height_in_ctu_minus1[j] is omitted, its value may be regarded as 0.

[0602] If slice index j is smaller than the number of slices whose heights are explicitly signaled, the width of slice j may be determined based on the syntax exp_slice_height_in_ctu_minus1[j] signaled via the bitstream. On the other hand, if slice index k is equal to or greater than the number of slices whose heights are explicitly signaled, the height of slice k may be determined based on the last signaled syntax exp_slice_height_in_ctu_minus1[l], where l indicates the index of the slice whose height was last signaled and may be an integer smaller than k.

[0603] As an example, if the value obtained by subtracting the height of the previous slice from the height of the tile is equal to or greater than the value obtained by adding 1 to the syntax exp_slice_height_in_ctu_minus1[l], the height of slice k may be set to the value obtained by adding 1 to the syntax exp_slice_height_in_ctu_minus1[l]. On the other hand, if the value obtained by subtracting the height of the previous slice from the height of the tile is less than the value obtained by adding 1 to the syntax exp_slice_height_in_ctu_minus1[l], the height of slice k can be set to the difference value obtained by subtracting the height of the previous slice from the height of the tile.

[0604] That is, the heights of the remaining slices other than the slice whose height is explicitly signaled may have a value smaller than or equal to the height of the last slice among the slices whose height is explicitly signaled.

[0605] In the above example, it is illustrated that whether or not to encode the syntax element one_slice_in_pic_flag, which indicates whether or not a picture is composed of a single slice, is determined only when a rectangular slice definition method is applied. Conversely to the example, it is also possible to first encode the syntax element one_slice_in_pic_flag, and then determine whether or not to encode the syntax element rect_slice_flag according to the value of the flag one_slice_in_pic_flag. For example, if the value of one_slice_in_pic_flag is 1, encoding of rect_slice_flag can be omitted.

[0606]

[0607] If the number of tile columns in the current picture is 1, coding of the syntax element slice_width_in_tiles_minus1 may be omitted. Also, if the number of tile rows in the current picture is 1, coding of the syntax element slice_height_in_tiles_minus1 may be omitted.

[0608] Alternatively, it is possible to omit encoding at least one of the information indicating the slice width and the information indicating the slice height.

[0609] As an example, the coding of the syntax element slice_width_in_tiles_minus1 indicating the slice width may be omitted, and the distance from a tile at a predetermined position in an adjacent slice may be set as the slice width. Specifically, the syntax element top_left_brick_idx_delta[i] may identify the top-left tile of the i+1-th slice adjacent to the right of the i-th slice. The width of the i-th slice may be derived as the difference between the x-coordinate of the top-left tile in the i-th slice (e.g., the x-coordinate of the top-left sample) and the x-coordinate of the top-left tile in the i+1-th slice (e.g., the x-coordinate of the top-left sample).

[0610] Alternatively, coding of the syntax element slice_height_in_tiles_minus1 indicating the slice height can be omitted, and the distance from a tile at a predetermined position in an adjacent slice can be set as the slice width. Specifically, the syntax element top_left_brick_idx_delta[j-1] may specify the top-left tile of the j-th slice located at the bottom edge of the i-th slice. The height of the i-th slice may be derived as the difference between the y-coordinate of the top-left tile in the i-th slice (e.g., the y-coordinate of the top-left sample) and the y-coordinate of the top-left tile in the j-th slice (e.g., the y-coordinate of the top-left sample).

[0611] Alternatively, information indicating whether differential value information is used when defining a rectangular slice may be signaled via a bitstream. As an example, tile_idx_delta_present_flag, which indicates whether differential value information is used, may be signaled via a bitstream. A value of 1 for the syntax tile_idx_delta_present_flag indicates that syntax indicating differential values of tile indices is coded and signaled. As an example, when the value of the syntax tile_idx_delta_present_flag is 1, the i-th slice may be defined by syntax slice_width_in_tiles_minus1[i] and syntax slice_height_in_tiles_minus1[i] indicating the size of the slice, and differential value information for determining the position of the top-left tile or the top-right tile in the slice (e.g., top_left_brick_idx_delta[i-1] or bottom_right_brick_idx_delta[i]).

[0612] When the value of the syntax tile_idx_delta_present_flag is 0, the i-th slice may be defined by the syntax slice_width_in_tiles_minus1[i] and the syntax slice_height_in_tiles_minus1[i], which indicate the size of the slice. When the value of the syntax tile_idx_delta_prsent_flag is 0, a slice located to the right of a slice that touches the left boundary of the current picture may be set to have the same height as the slice that touches the left boundary. This makes it possible to signal information indicating the slice height only for slices that touch the left boundary of the current picture, and to omit signaling information indicating the slice height for slices that do not touch the left boundary of the current picture.

[0613] The syntax element tile_idx_delta_present_flag, which indicates whether information indicating a difference between tile indexes is coded, may be coded / decoded if the number of slices included in the current picture is equal to or greater than a threshold. Here, the threshold may be set to an integer such as 0, 1, or 2. For example, the syntax element tile_idx_delta_present_flag may be coded / decoded only if the number of slices included in the current picture is equal to or greater than one or equal to two. If the number of slices included in the current picture is less than the threshold, coding / decoding of tile_idx_delta_present_flag may be omitted and its value may be set to 0. Slices may also be defined based on coding tree unit indices instead of tile indices.

[0614] In the above example, it is assumed that slices are defined by ordering tiles according to raster scan order. Alternatively, slices can be defined by ordering tiles according to vertical, horizontal, or diagonal scan order.

[0615] If a tile consists of multiple slices, slice size information and / or tile index difference information may be signaled only for the first slice, where the slice size information may include at least one of the syntax slice_width_in_tiles_minus1[i] indicating the slice width and the syntax slice_height_in_tiles_minus1[i] indicating the slice height.

[0616] Table 17 provides an example of this.

[0617] [Table 17-1] [Table 17-2]

[0618] A variable isFirstSliceInTile[i] may be set to indicate whether the slice is the first slice in the tile. A value of 1 for the variable isFirstSliceInTile[i] indicates that the slice is the first slice in the tile. Conversely, a value of 0 for the variable isFirstSliceInTile[i] indicates that the slice is not the first slice in the tile.

[0619] The variable isFirstSliceInTile[i] may be initialized to 1. This means that if the ith slice is not part of a tile but is composed of at least one more tile, the value of the variable isFirstSliceInTile[i] for the ith tile may be set to 1.

[0620] When the value of the variable isFirstSliceInTile[i] is 1, the syntax elements slice_width_in_tiles_minus1[i] and slice_height_in_tiles_minus1[i] indicating the size of the slice may be coded / decoded.

[0621] Here, if the syntax slice_width_in_tiles_minus1[i] and the syntax slice_height_in_tiles_minus1[i] are both 0, number information indicating the number of slices in the tile may be signaled. If a tile includes multiple slices, height information for at least one of the multiple slices may be encoded / decoded. Here, after decoding the height information of the second slice, the value of the variable isFirstTileInTile[i++] may be converted to 0.

[0622] As a result, coding of the syntax element slice_width_in_tiles_minus1[i] indicating the slice width and the syntax element slice_height_in_tiles_minus1[i] indicating the slice height may be omitted from the second slice onwards.

[0623] The syntax tile_idx_delta[i] indicating the tile index delta can be coded / decoded only for the first tile. As an example, the syntax tile_idx_delta[i] may be signaled when the syntax slice_width_in_tiles_minus1[i] is not 0, when the syntax slice_height_in_tiles_minus1[i] is not 0, or when the variable isFirstSliceInTile[i] is 1.

[0624] If a tile is divided into multiple slices (i.e., the syntax slice_width_in_tiles_minus1[i] and syntax slice_height_in_tiles_minus1[i] are both 0), the value of the variable isFirstSliceInTile is 1 only for the first slice, and the value of the variable isFirstSliceInTile is 0 from the second slice onwards. This means that the syntax tile_idx_delta[i] may be coded only for the first slice. If the last tile in a picture consists of multiple slices, tile_idx_delta[i] may not be signaled.

[0625]

[0626] It is within the scope of the present invention to apply an embodiment described primarily in terms of a decoding process or an encoding process to an encoding process or a decoding process. It is also within the scope of the present invention to change an embodiment described in a given order into an order different from that described.

[0627] The above-described embodiments are described based on a series of steps or flowcharts, but this does not limit the chronological order of the invention, and steps may be performed simultaneously or in a different order as necessary. Furthermore, in the above-described embodiments, each of the components (e.g., units, modules, etc.) constituting the block diagrams may be implemented as a hardware device or software, or multiple components may be combined to be implemented as a single hardware device or software. The above-described embodiments may be implemented in the form of program instructions that may be executed by various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., alone or in combination. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specially configured to store and execute program instructions, such as ROMs, RAMs, and flash memories. The hardware devices may be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa. [Industrial Applicability]

[0628] The present invention may be applied to electronic devices that encode / decode video.

Claims

1. 1. A method of encoding video, the method comprising: obtaining a current picture of the video; Dividing the current picture into a plurality of tiles; and encoding a flag indicating whether the number of slices in the current picture is one into a bitstream, wherein the value of the flag is: a first value indicating that the number of slices in the current picture is one; a second value indicating that the number of slices in the current picture is greater than one; Determining whether to encode slice configuration information into the bitstream, wherein determining whether to encode slice configuration information into the bitstream includes: refraining from encoding the slice configuration information into the bitstream in response to determining that the number of slices in the current picture is one; and encoding the slice configuration information into the bitstream in response to determining that the number of slices in the current picture is greater than one.

2. 10. The method of claim 1, determining that the number of slices in the current picture is greater than one; In response to determining that the number of slices in the current picture is greater than one, encoding the slice configuration information into the bitstream.

3. The method of claim 1 , wherein the slice configuration information includes a syntax indicating an index of a tile included in one of the slices.

4. 2. The method of claim 1, wherein the slice configuration information includes syntax indicating a difference between a first index of a first tile and a second index of a second tile, the first tile and the second tile being included in one of the slices.

5. It is a system one or more processors; and a memory storing instructions that, when executed by the one or more processors, cause the one or more processors to: obtaining a current picture of the video; Dividing the current picture into a plurality of tiles; and encoding a flag indicating whether the number of slices in the current picture is one into a bitstream, wherein the value of the flag is: a first value indicating that the number of slices in the current picture is one; a second value indicating that the number of slices in the current picture is greater than one; Determining whether to encode slice configuration information into the bitstream, wherein determining whether to encode slice configuration information into the bitstream includes: refraining from encoding the slice configuration information into the bitstream in response to determining that the number of slices in the current picture is one; and encoding the slice configuration information into the bitstream in response to determining that the number of slices in the current picture is greater than one.

6. 6. The system of claim 5, wherein the operation comprises: determining that the number of slices in the current picture is greater than one; In response to determining that the number of slices in the current picture is greater than one, encoding the slice configuration information into the bitstream.

7. The system of claim 5 , wherein the slice configuration information includes syntax indicating an index of a tile included in one of the slices.

8. 6. The system of claim 5, wherein the slice configuration information includes syntax indicating a difference between a first index of a first tile and a second index of a second tile, the first tile and the second tile being included in one of the slices.

9. 1. A computer program comprising instructions that, when executed by one or more processors, cause the one or more processors to: obtaining a current picture of the video; Dividing the current picture into a plurality of tiles; and encoding a flag indicating whether the number of slices in the current picture is one into a bitstream, wherein the value of the flag is: a first value indicating that the number of slices in the current picture is one; a second value indicating that the number of slices in the current picture is greater than one; Determining whether to encode slice configuration information into the bitstream, wherein determining whether to encode slice configuration information into the bitstream includes: refraining from encoding the slice configuration information into the bitstream in response to determining that the number of slices in the current picture is one; and encoding the slice configuration information into the bitstream in response to determining that the number of slices in the current picture is greater than one.

10. 10. The computer program of claim 9, wherein the operation comprises: determining that the number of slices in the current picture is greater than one; In response to determining that the number of slices in the current picture is greater than one, encoding the slice configuration information into the bitstream.

11. 10. The computer program of claim 9, wherein the slice configuration information includes syntax indicating an index of a tile included in one of the slices.

12. 10. The computer program of claim 9, wherein the slice configuration information includes syntax indicating a difference between a first index of a first tile and a second index of a second tile, the first tile and the second tile being included in one of the slices.

Images