Method for encoding / decoding image and device for transmitting compressed video data

By dividing video blocks into sub-blocks for intra prediction and weighting prediction blocks, along with adaptive residual coefficient grouping, the method improves encoding/decoding efficiency and accuracy for high-resolution video, addressing the challenges of high data volume and cost in existing technologies.

WO2026235376A1PCT designated stage Publication Date: 2026-11-12KT CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/KR2026/005925
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2025-06-10
Filing Date
2026-05-08
Publication Date
2026-11-12

AI Technical Summary

Technical Problem

The increasing demand for high-resolution and high-quality video data leads to higher data volumes, increasing transmission and storage costs, and existing video compression technologies struggle to efficiently handle stereoscopic video content.

Method used

A method is introduced that divides video blocks into sub-blocks for intra prediction, weights prediction blocks from intra and inter prediction, and adaptively configures residual coefficient groups for improved encoding/decoding efficiency.

Benefits of technology

This approach enhances prediction accuracy and encoding/decoding efficiency by optimizing video block processing, particularly for stereoscopic content, reducing data volume and costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure KR2026005925_12112026_PF_FP_ABST
    Figure KR2026005925_12112026_PF_FP_ABST
Patent Text Reader

Abstract

An image decoding method according to the present disclosure may comprise the steps of: generating a prediction block of a current block; acquiring residual coefficients of the current block; acquiring a residual block of the current block from the residual coefficients; and reconstructing the current block on the basis of the prediction block and the residual block. In this case, the prediction block is acquired by performing intra prediction on the current block in units of sub-blocks, and an intra prediction mode of each of the sub-blocks in the current block may be independently determined.
Need to check novelty before this filing date? Find Prior Art

Description

Video encoding / decoding method and device for transmitting compressed video data

[0001] The present disclosure relates to a video signal processing method and apparatus.

[0002] Recently, the demand for high-resolution, high-quality video, such as HD (High Definition) and UHD (Ultra High Definition) video, has been increasing across various application fields. As video data becomes higher in resolution and quality, the relative volume of data increases compared to conventional video data; consequently, transmission and storage costs increase when video data is transmitted using existing wired or wireless broadband lines or stored using conventional storage media. To address these issues arising from the increase in video data resolution and quality, high-efficiency video compression technologies can be utilized.

[0003] Various video compression technologies exist, such as inter-frame prediction technology that predicts pixel values ​​in the current picture from previous or subsequent pictures, intra-frame prediction technology that predicts pixel values ​​in the current picture using pixel information within the current picture, and entropy coding technology that assigns short codes to values ​​with high frequency and long codes to values ​​with low frequency; by utilizing these video compression technologies, video data can be effectively compressed for transmission or storage.

[0004] Meanwhile, along with the increasing demand for high-resolution video, the demand for stereoscopic video content as a new video service is also rising. Discussions are underway regarding video compression technologies to effectively provide high-resolution and ultra-high-resolution stereoscopic video content.

[0005] The present disclosure aims to provide a method for dividing a current block into a plurality of sub-blocks and then performing intra prediction on a sub-block basis.

[0006] The present disclosure aims to provide a method for obtaining a final prediction block by weighting a prediction block obtained by intra-prediction and a prediction block obtained by inter-prediction.

[0007] The present disclosure aims to provide a method for adaptively configuring residual coefficient groups in encoding / decoding residual coefficients in units of residual coefficient groups.

[0008] The present disclosure aims to provide an improved method for predicting the signs of residual coefficients.

[0009] The technical problems to be solved in this disclosure are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art to which this disclosure belongs from the description below.

[0010] A video decoding method according to the present disclosure may include: generating a prediction block of a current block; obtaining residual coefficients of the current block; obtaining a residual block of the current block from the residual coefficients; and restoring the current block based on the prediction block and the residual block. In this case, the prediction block is obtained by performing intra prediction on the current block in sub-block units, and the intra prediction mode of each sub-block within the current block may be determined independently.

[0011] In the image decoding method according to the present disclosure, the intra prediction mode of each of the sub-blocks can be obtained based on the histogram of each of the sub-blocks.

[0012] In the image decoding method according to the present disclosure, if the current sub-block is not the first sub-block among the sub-blocks, a histogram of the current sub-block may be generated based on the restored samples included in the previous sub-block of the current sub-block.

[0013] In the image decoding method according to the present disclosure, for each of the sub-blocks, information indicating whether to set the intra prediction mode of the sub-block to be the same as the current block may be decoded.

[0014] In the image decoding method according to the present disclosure, the prediction block may be obtained by weighting the sum of a first prediction block obtained by intra prediction and a second prediction block obtained by performing inter prediction on the current block.

[0015] In the image decoding method according to the present disclosure, the weights applied to the first prediction block and the second prediction block may be set differently on a sub-block basis.

[0016] In the image decoding method according to the present disclosure, the weights of each sub-block may be determined based on one of a plurality of previously defined weight lookup tables.

[0017] In the image decoding method according to the present disclosure, one of the plurality of weighted lookup tables may be selected by referring to the encoding mode of a neighbor block adjacent to the current block.

[0018] In the image decoding method according to the present disclosure, one of the plurality of weighted lookup tables may be selected based on the intra prediction mode derivation method of each of the sub-blocks.

[0019] In the image decoding method according to the present disclosure, the residual coefficients may be decoded in units of residual coefficient groups comprising a predetermined number of residual coefficients.

[0020] In the image decoding method according to the present disclosure, the residual coefficients are decoded in units of residual coefficient groups, and the first residual coefficient group may include residual coefficients existing up to the position of a residual coefficient with an absolute value of 0.

[0021] In the image decoding method according to the present disclosure, the second residual coefficient group may include residual coefficients existing from a residual coefficient with an absolute value of 0 up to a position of a residual coefficient with an absolute value not of 0.

[0022] In the image decoding method according to the present disclosure, for each of the first residual coefficient group and the second residual coefficient group, information on the number of residual coefficients can be decoded.

[0023] A video encoding method according to the present disclosure may include: a step of generating a prediction block of a current block; a step of obtaining a residual block of the current block based on the prediction block; a step of obtaining residual coefficients from the residual block; and a step of encoding the residual coefficients. In this case, the prediction block is obtained by performing intra prediction on the current block in sub-block units, and the intra prediction mode of each sub-block within the current block may be determined independently.

[0024] According to the present disclosure, a computer-readable recording medium may be provided that records instructions for storing / transmitting a bitstream generated by an image encoding method.

[0025] According to the present disclosure, a computer-readable recording medium may be provided that records instructions for performing an image decoding method or an image encoding method.

[0026] The features briefly summarized above regarding the present disclosure are merely exemplary aspects of the detailed description of the present disclosure that follows and do not limit the scope of the present disclosure.

[0027] According to the present disclosure, the prediction accuracy can be improved by dividing the current block into a plurality of sub-blocks and then performing intra prediction on a sub-block basis.

[0028] According to the present disclosure, prediction accuracy can be improved by obtaining a final prediction block through a weighted sum of a prediction block obtained by intra prediction and a prediction block obtained by inter prediction.

[0029] According to the present disclosure, encoding / decoding efficiency can be improved by providing a method for adaptively configuring residual coefficient groups when encoding / decoding residual coefficients in units of residual coefficient groups.

[0030] According to the present disclosure, encoding / decoding efficiency can be improved by providing an improved method for predicting the signs of residual coefficients.

[0031] The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present disclosure pertains from the description below.

[0032] FIG. 1 is a block diagram showing an image encoding device according to one embodiment of the present disclosure.

[0033] FIG. 2 is a block diagram showing an image decoding device according to an embodiment of the present disclosure.

[0034] FIG. 3 illustrates an image encoding / decoding method performed by an image encoding / decoding device according to the present disclosure.

[0035] FIG. 4 illustrates an example of a plurality of intra-prediction modes according to the present disclosure.

[0036] Figure 5 shows an example where the directional mode is extended.

[0037] FIG. 6 illustrates a planner mode-based intra-prediction method according to the present disclosure.

[0038] FIG. 7 illustrates a DC mode-based intra prediction method according to the present disclosure.

[0039] FIG. 8 illustrates an intra-prediction method based on a directional mode according to the present disclosure.

[0040] Figure 9 illustrates a method for deriving samples of fractional positions.

[0041] Figures 10 and 11 illustrate tangent values ​​for angles scaled by 32 times for each intra prediction mode.

[0042] FIG. 12 is a diagram illustrating an intra-prediction pattern when the directional mode is one of modes 34 to 49.

[0043] Figure 13 is a diagram illustrating an example of generating an upper reference sample by interpolating left reference samples.

[0044] Figure 14 shows an example in which intra prediction is performed using reference samples arranged in a 1D array.

[0045] Figure 15 is a diagram illustrating an example of setting a reference area.

[0046] Figure 16 is a diagram showing an example of the configuration of a reference area.

[0047] Figure 17 illustrates the filter coefficients for the Sobel mask and the Prewit mask, respectively.

[0048] Figure 18 shows the locations where the vertical and horizontal inclinations are obtained within the reference area.

[0049] Figure 19 shows an example of grouping directional modes into multiple intra-prediction mode groups.

[0050] FIG. 20 is a drawing illustrating a reference area around the current block.

[0051] Figures 21 and 22 illustrate an example of performing intra prediction on a reference area based on planner mode.

[0052] Figures 23 and 24 illustrate an example of performing intra prediction on a reference region based on DC mode.

[0053] Figures 25 and 26 illustrate an example of performing intra prediction on a reference region based on a directional mode.

[0054] FIG. 27 is a diagram illustrating an example in which the number of wide-angle intra prediction modes available to the current block is determined by referring to neighboring blocks adjacent to the current block.

[0055] FIG. 28 is a diagram showing the intra prediction modes available to the current block when both the first neighbor intra prediction mode and the second neighbor intra prediction mode are greater than the threshold value.

[0056] FIG. 29 is a drawing illustrating multiple reference sample lines.

[0057] Figure 30 shows an example in which intra prediction is performed by selecting one of the reference sample lines.

[0058] FIGS. 31 to 33 illustrate an example of implicitly restoring a reference sample line index on the decoder side.

[0059] Figure 34 shows an example where intra-prediction is performed on a sub-block basis.

[0060] Figure 35 illustrates the range of restored samples used to derive histograms by sub-block.

[0061] Figure 36 shows an example in which the current block is divided in the horizontal and vertical directions.

[0062] Figure 37 shows an example of calculating the amplitude accumulation value for each of the non-adjacent reference sample lines.

[0063] Figure 38 is a diagram illustrating the process of performing inter-prediction in the encoder and decoder.

[0064] Figure 39 shows an example where motion estimation is performed.

[0065] FIGS. 40 and FIGS. 41 illustrate an example in which a predicted block of the current block is generated based on motion information generated through motion estimation.

[0066] Figure 42 shows the location referenced to derive the motion vector prediction value.

[0067] Figure 43 is a diagram illustrating a template-based motion estimation method.

[0068] Figure 44 shows examples of template configurations.

[0069] Figure 45 is a diagram illustrating a motion estimation method based on a two-way matching method.

[0070] Figure 46 is a diagram illustrating a motion estimation method based on a unidirectional matching method.

[0071] Figures 47 and 48 illustrate examples in which prediction blocks are generated according to the precision of the motion vectors.

[0072] FIG. 49 shows an example in which motion compensation based on a translational model and a zooming model is performed for the current block.

[0073] FIG. 50 shows an example in which motion compensation based on a translational model and a rotational model is performed for the current block.

[0074] Figures 51 and 52 show an example of generating a prediction block for the current block using control point motion vectors.

[0075] Figure 53 shows an example of generating a prediction block for the current block using three control point motion vectors.

[0076] Figure 54 shows an example in which motion vectors are derived in sub-block units.

[0077] Figures 55 and 56 show examples in which motion vectors are induced in units of sub-blocks within the current block when SbTMVP is applied.

[0078] Figures 57 and 58 are diagrams illustrating examples in which a prediction block is derived according to the precision of the motion vector.

[0079] Figures 59 and 60 are diagrams illustrating the process of encoding and decoding motion vector difference values ​​when the AMVR method is applied, respectively.

[0080] FIG. 61 shows an example in which a first weight w1 and a second weight w2 are determined based on the encoding modes of neighboring blocks adjacent to the current block.

[0081] Figure 62 illustrates an example where weights are adaptively determined for each region.

[0082] Figure 63 illustrates the restoration region referenced to induce the intra-prediction mode of each region.

[0083] Figure 64 is a flowchart illustrating the process of encoding residual coefficients in an encoder.

[0084] Figure 65 is a flowchart illustrating the encoding process of magnitude information of residual coefficients.

[0085] Figure 66 is a flowchart illustrating the process of decoding residual coefficients in a decoder.

[0086] Figure 67 is a diagram showing the decoding process of the magnitude information of the residual coefficient.

[0087] FIG. 68 is a diagram illustrating the scan order of sub-blocks and the scan order of residual coefficients within sub-blocks.

[0088] Figure 69 shows the order in which the syntax elements of the residual coefficients are encoded / decoded.

[0089] Figure 70 shows the order in which the syntax elements of the residual coefficients are encoded / decoded.

[0090] Figure 71 shows the values ​​of the syntaxes for expressing the absolute value of the residual coefficient.

[0091] FIG. 72 is a flowchart of the encoding / decoding process of syntaxes representing the absolute value of a residual coefficient according to one embodiment of the present disclosure.

[0092] Figure 73 shows the reference residual coefficients of the current residual coefficients.

[0093] FIGS. 74 and 75 are intended to illustrate an example in which the current block is divided into multiple coefficient groups.

[0094] Figure 76 shows an example where residual coefficients with an absolute value of 0 or residual coefficients with an absolute value greater than 0 are set as a single coefficient group.

[0095] FIGS. 77 and 78 are flowcharts of a method for predicting the sign of a residual coefficient according to one embodiment of the present disclosure.

[0096] FIG. 79 shows an example in which different sign combinations are applied to residual coefficients to generate multiple restoration blocks.

[0097] Figure 80 shows an example of how the cost of a code combination is calculated.

[0098] Figure 81 illustrates an example in which a restored sample is selected by the directionality of the intra prediction mode.

[0099] Figure 82 is a diagram illustrating an example in which the cost of a symbol combination is calculated based on template matching.

[0100] FIGS. 83 and FIGS. 84 illustrate examples in which an intra prediction mode is induced for each of a plurality of reference sample groups.

[0101] FIG. 85 shows an example in which a horizontal gradient and a vertical gradient are output for reference samples belonging to a reference sample line adjacent to the current block.

[0102] FIG. 86 shows an example in which a horizontal gradient and a vertical gradient are calculated for reference samples belonging to a selected reference sample line among a plurality of reference sample lines.

[0103] Figure 87 shows an example in which restoration samples used to calculate the cost of code combinations are specified based on the intra-prediction mode of each group.

[0104] FIG. 88 is a diagram illustrating samples used to calculate the horizontal and vertical costs of a sign combination when i is 2.

[0105] FIG. 89 is intended to illustrate an example of calculating the cost of a code combination based on the intra-prediction mode of a neighboring block adjacent to the current block.

[0106] The present disclosure is susceptible to various modifications and may have various embodiments; specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present disclosure to specific embodiments, and it should be understood that it includes all modifications, equivalents, and substitutions that fall within the spirit and scope of the present disclosure. Similar reference numerals have been used for similar components in the description of each drawing.

[0107] Terms such as "first," "second," etc., may be used to describe various components, but said components should not be limited by said terms. Such terms are used solely for the purpose of distinguishing one component from another. For example, without departing from the scope of the present disclosure, the first component may be named the second component, and similarly, the second component may be named the first component. The term "and / or" includes a combination of a plurality of related described items or any of a plurality of related described items.

[0108] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. On the other hand, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

[0109] The terms used in this application are used merely to describe specific embodiments and are not intended to limit the disclosure. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as “comprising” or “having” are intended to specify the presence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0110] Hereinafter, preferred embodiments of the present disclosure will be described in more detail with reference to the attached drawings. Hereinafter, the same reference numerals are used for identical components in the drawings, and redundant descriptions of identical components are omitted.

[0111] FIG. 1 is a block diagram showing an image encoding device according to one embodiment of the present disclosure.

[0112] Referring to FIG. 1, the image encoding device (100) may include a picture splitting unit (110), a prediction unit (120, 125), a conversion unit (130), a quantization unit (135), a reordering unit (160), an entropy encoding unit (165), an inverse quantization unit (140), an inverse conversion unit (145), a filter unit (150), and a memory (155).

[0113] Each component shown in FIG. 1 is depicted independently to represent different characteristic functions of the image encoding device and does not imply that each component consists of separate hardware or a single software unit. That is, each component is listed and included as a separate component for convenience of explanation, but at least two of the components may be combined to form a single component, or a single component may be divided into multiple components to perform functions, and such integrated and separated embodiments of each component are included within the scope of the present disclosure as long as they do not deviate from the essence of the present disclosure.

[0114] Additionally, some components may not be essential components performing an essential function in the present disclosure, but may be optional components merely for enhancing performance. The present disclosure may be implemented by including only the components essential to embody the essence of the present disclosure, excluding components used merely for enhancing performance, and a structure including only the essential components, excluding optional components used merely for enhancing performance, is also included within the scope of the rights of the present disclosure.

[0115] The picture segmentation unit (110) can divide an input picture into at least one processing unit. At this time, the processing unit may be a Prediction Unit (PU), a Transform Unit (TU), or a Coding Unit (CU). The picture segmentation unit (110) can divide a picture into a combination of multiple coding units, prediction units, and transformation units, and can encode the picture by selecting one combination of coding units, prediction units, and transformation units based on a predetermined criterion (e.g., a cost function).

[0116] For example, a single picture can be divided into multiple coding units. To divide coding units within a picture, recursive tree structures such as a Quad Tree, Ternary Tree, or Binary Tree can be used. A coding unit divided into other coding units, with a single image or the largest coding unit as the root, can have as many child nodes as the number of divided coding units. A coding unit that is no longer divided according to certain limits becomes a leaf node. For example, assuming Quad Tree division is applied to a single coding unit, a single coding unit can be divided into up to four different coding units.

[0117] In the embodiments of the present disclosure below, the encoding unit may be used to mean a unit that performs encoding, or a unit that performs decoding.

[0118] A prediction unit may be divided into at least one shape, such as a square or rectangle, of the same size within a single encoding unit, or one of the prediction units divided within a single encoding unit may be divided such that any one prediction unit has a different shape and / or size from another prediction unit.

[0119] When performing intra-frame prediction, the transformation unit and the prediction unit may be set to be the same. In this case, the encoding unit may be divided into multiple transformation units, and intra-frame prediction may be performed for each transformation unit. The encoding unit may be divided in a horizontal or vertical direction. The number of transformation units generated by dividing the encoding unit may be two or four, depending on the size of the encoding unit. Alternatively, if the size of the transformation unit is small, multiple transformation units may be set as a single prediction unit.

[0120] The prediction unit (120, 125) may include an inter-frame prediction unit (120) that performs inter-frame prediction and an intra-frame prediction unit (125) that performs intra-frame prediction. It may determine whether to use inter-frame prediction or perform intra-frame prediction for a encoding unit, and determine specific information (e.g., reference sample line, intra-frame prediction mode, motion vector, reference picture, etc.) according to each prediction method. At this time, the processing unit in which the prediction is performed and the processing unit in which the prediction method and specific details are determined may be different. For example, the prediction method and prediction mode, etc., may be determined by the encoding unit, and the prediction may be performed by the prediction unit or the conversion unit. The residual value (residual block) between the generated prediction block and the original block may be input to the conversion unit (130). In addition, the prediction mode information, motion vector information, etc. used for prediction may be encoded together with the residual value in the entropy encoding unit (165) and transmitted to the decoding device. When using a specific encoding mode, it is also possible to encode the original block as is and transmit it to the decoding unit without generating a prediction block through the prediction unit (120, 125).

[0121] The inter-frame prediction unit (120) may predict a prediction unit based on information of at least one picture among the previous picture or the subsequent picture of the current picture, and in some cases, may predict a prediction unit based on information of a partially encoded area within the current picture. The inter-frame prediction unit (120) may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

[0122] In the reference picture interpolation unit, reference picture information is received from memory (155), and pixel information of integer pixels or less can be generated from the reference picture. In the case of luminance pixels, a DCT-based 8-tap interpolation filter with different filter coefficients can be used to generate pixel information of integer pixels or less in 1 / 4 pixel units. In the case of chrominance signals, a DCT-based 4-tap interpolation filter with different filter coefficients can be used to generate pixel information of integer pixels or less in 1 / 8 pixel units.

[0123] The motion prediction unit can perform motion prediction based on a reference picture interpolated by the reference picture interpolation unit. Various methods, such as FBMA (Full search-based Block Matching Algorithm), TSS (Three Step Search), and NTS (New Three-Step Search Algorithm), can be used to calculate motion vectors. Based on the interpolated pixels, the motion vector can have motion vector values ​​in units of 1 / 2 or 1 / 4 pixels. The motion prediction unit can predict the current prediction unit by using different motion prediction methods. Various motion prediction methods, such as the Skip method, Merge method, AMVP (Advanced Motion Vector Prediction) method, and Intra Block Copy method, can be used.

[0124] The in-screen prediction unit (125) can generate a prediction block based on reference pixel information, which is pixel information within the current picture. Reference pixel information can be derived from one selected from a plurality of reference pixel lines. The Nth reference pixel line among the plurality of reference pixel lines may include left pixels with an x-axis difference of N with the top-left pixel in the current block and top pixels with a y-axis difference of N with said top-left pixel. The number of reference pixel lines that the current block can select may be 1, 2, 3, or 4.

[0125] If a neighboring block of the current prediction unit is a block that has undergone inter-frame prediction, and the reference pixel is a pixel that has undergone inter-frame prediction, the reference pixel included in the block that has undergone inter-frame prediction can be replaced with the reference pixel information of a neighboring block that has undergone intra-frame prediction. That is, if the reference pixel is not available, the information of the unavailable reference pixel can be replaced with the information of at least one of the available reference pixels.

[0126] In intra-frame prediction, the prediction mode may include a directional prediction mode that uses reference pixel information according to the prediction direction, and a non-directional mode that does not use directional information when performing prediction. The mode for predicting luminance information and the mode for predicting chrominance information may be different, and the intra-frame prediction mode information used to predict luminance information or the predicted luminance signal information may be utilized to predict chrominance information.

[0127] When performing intra-frame prediction, if the size of the prediction unit and the size of the transformation unit are the same, intra-frame prediction for the prediction unit can be performed based on the pixels to the left of the prediction unit, the pixels at the top left, and the pixels at the top.

[0128] The in-frame prediction method can generate a prediction block after applying a smoothing filter to a reference pixel according to the prediction mode. Depending on the selected reference pixel line, it may be determined whether to apply the smoothing filter.

[0129] To perform an intra-frame prediction method, the intra-frame prediction mode of the current prediction unit can be predicted from the intra-frame prediction mode of the prediction unit existing in the vicinity of the current prediction unit. When predicting the prediction mode of the current prediction unit using the mode information predicted from the surrounding prediction unit, if the intra-frame prediction mode of the current prediction unit and the surrounding prediction unit are the same, information indicating that the prediction modes of the current prediction unit and the surrounding prediction unit are the same can be transmitted using predetermined flag information; if the prediction modes of the current prediction unit and the surrounding prediction unit are different, entropy coding can be performed to encode the prediction mode information of the current block.

[0130] Additionally, a residual block can be generated that includes residual value information, which is the difference between the prediction unit that performed the prediction based on the prediction unit generated in the prediction unit (120, 125) and the original block of the prediction unit. The generated residual block can be input to the conversion unit (130).

[0131] In the transformation unit (130), the residual block containing residual value information of the prediction unit generated through the original block and the prediction unit (120, 125) can be transformed using a transformation method such as DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), or KLT. Whether to apply DCT, DST, or KLT to transform the residual block can be determined based on at least one of the size of the transformation unit, the shape of the transformation unit, the prediction mode of the prediction unit, or the in-frame prediction mode information of the prediction unit. Meanwhile, the transformation can be performed by separating the horizontal direction and the vertical direction.

[0132] After performing transformations for the horizontal and vertical directions, a second transformation can be performed. The second transformation may be in a form where the horizontal and vertical directions are not separated. Final transformation coefficients can be generated by performing a second transformation on the transformation coefficients obtained by the first transformation. Meanwhile, the number of final transformation coefficients output by the second transformation may be smaller than the number of transformation coefficients input for the second transformation. Specifically, the second transformation can be performed using a reduced transformation matrix with different numbers of columns and rows.

[0133] The quantization unit (135) can quantize the values ​​converted into the frequency domain in the conversion unit (130). The quantization coefficient may vary depending on the block or the importance of the image. The values ​​produced by the quantization unit (135) may be provided to the inverse quantization unit (140) and the reordering unit (160).

[0134] The reordering unit (160) can perform reordering of coefficient values ​​for quantized residual values.

[0135] The reordering unit (160) can convert two-dimensional block-shaped coefficients into one-dimensional vector forms through a coefficient scanning method. For example, the reordering unit (160) can convert the coefficients from DC to high-frequency ranges into one-dimensional vector forms by scanning using a Zig-Zag Scan method. Depending on the size of the conversion unit and the in-frame prediction mode, instead of Zig-Zag Scan, a vertical scan that scans two-dimensional block-shaped coefficients in the column direction, a horizontal scan that scans two-dimensional block-shaped coefficients in the row direction, or a diagonal scan that scans two-dimensional block-shaped coefficients in the diagonal direction may be used. That is, depending on the size of the conversion unit and the in-frame prediction mode, it can be determined whether to use a Zig-Zag Scan, a vertical scan, a horizontal scan, or a diagonal scan.

[0136] The entropy encoding unit (165) can perform entropy encoding based on the values ​​calculated by the reordering unit (160). Entropy encoding can use various encoding methods, such as, for example, Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

[0137] The entropy encoding unit (165) can encode various information from the reordering unit (160) and the prediction unit (120, 125), such as residual value coefficient information of the encoding unit, block type information, prediction mode information, division unit information, prediction unit information and transmission unit information, motion vector information, reference frame information, block interpolation information, and filtering information.

[0138] In the entropy encoding unit (165), the coefficient value of the encoding unit input from the reordering unit (160) can be entropy encoded.

[0139] In the inverse quantization unit (140) and inverse transformation unit (145), the values ​​quantized in the quantization unit (135) are inversely quantized, and the values ​​transformed in the transformation unit (130) are inversely transformed. The residual value generated in the inverse quantization unit (140) and inverse transformation unit (145) can be combined with the predicted unit predicted through the motion estimation unit, motion compensation unit, and in-frame prediction unit included in the prediction unit (120, 125) to generate a reconstructed block.

[0140] The filter section (150) may include at least one of a deblocking filter, an offset correction section, and an ALF (Adaptive Loop Filter).

[0141] The deblocking filter can remove block distortion caused by boundaries between blocks in the restored picture. To determine whether to perform deblocking, the decision to apply the deblocking filter to the current block can be made based on the pixels contained in a certain number of columns or rows within the block. When applying the deblocking filter to a block, a Strong Filter or a Weak Filter can be applied depending on the required deblocking filtering strength. Additionally, when applying the deblocking filter, horizontal and vertical filtering can be processed in parallel.

[0142] The offset correction unit can correct the offset from the original image on a pixel-by-pixel basis for the image that has undergone deblocking. To perform offset correction for a specific picture, a method can be used in which pixels included in the image are divided into a certain number of regions, the region to be offset is determined, and the offset is applied to that region, or a method can be used in which the offset is applied by considering the edge information of each pixel.

[0143] Adaptive Loop Filtering (ALF) can be performed based on a comparison between the filtered restored image and the original image. After dividing the pixels included in the image into predetermined groups, a single filter to be applied to each group can be determined, allowing for differential filtering for each group. Information regarding whether to apply ALF can be transmitted per coding unit (CU), and the shape and filter coefficients of the ALF filter to be applied may vary depending on each block. Additionally, an ALF filter of the same form (fixed form) may be applied regardless of the characteristics of the block to be applied.

[0144] The memory (155) can store a restoration block or picture calculated through the filter unit (150), and the stored restoration block or picture can be provided to the prediction unit (120, 125) when performing inter-frame prediction.

[0145] FIG. 2 is a block diagram showing an image decoding device according to an embodiment of the present disclosure.

[0146] Referring to FIG. 2, the image decoding device (200) may include an entropy decoding unit (210), a reordering unit (215), an inverse quantization unit (220), an inverse transformation unit (225), a prediction unit (230, 235), a filter unit (240), and a memory (245).

[0147] When a video bitstream is input to a video encoding device, the input bitstream can be decoded by the reverse procedure of the video encoding device.

[0148] The entropy decoding unit (210) can perform entropy decoding in the opposite procedure to that which the entropy encoding unit of the image encoding device performed entropy encoding. For example, various methods such as Exponential Golomb, CAVLC (Context-Adaptive Variable Length Coding), and CABAC (Context-Adaptive Binary Arithmetic Coding) may be applied in correspondence with the method performed by the image encoding device.

[0149] The entropy decoding unit (210) can decode information related to intra-frame prediction and inter-frame prediction performed by the encoding device.

[0150] The reordering unit (215) can perform reordering based on the method of reordering the entropy-decoded bitstream in the encoding unit in the entropy decoding unit (210). It can reorder by restoring the coefficients expressed in the form of a one-dimensional vector back into coefficients in the form of a two-dimensional block. The reordering unit (215) can perform reordering by receiving information related to the coefficient scanning performed in the encoding unit and scanning in reverse based on the scanning order performed in the encoding unit.

[0151] The inverse quantization unit (220) can perform inverse quantization based on the coefficient values ​​of the rearranged block and the quantization parameters provided by the encoding device.

[0152] The inverse transform unit (225) can perform an inverse transform of the transform performed by the transform unit on the quantization result performed by the image encoding device. That is, it can perform at least one of an inverse transform of the second transform (second inverse transform) or an inverse transform for DCT, DST, and KLT (i.e., first inverse transform). The inverse transform can be performed based on a transmission unit determined by the image encoding device. The inverse transform unit (225) of the image decoder can determine a transform matrix for the second inverse transform or a transform technique for the first inverse transform (e.g., DCT, DST, KLT) according to a plurality of information such as a prediction method, the size and shape of the current block, a prediction mode, and an intra-frame prediction direction. Alternatively, information for determining the transform matrix or transform technique may be explicitly encoded and signaled.

[0153] The prediction unit (230, 235) can generate a prediction block based on the prediction block generation information provided by the entropy decoding unit (210) and the previously decoded block or picture information provided by the memory (245).

[0154] As described above, when performing intra-frame prediction identical to the operation in the video encoding device, if the size of the prediction unit and the size of the transform unit are the same, intra-frame prediction for the prediction unit is performed based on the pixels to the left of the prediction unit, the pixels to the top left, and the pixels to the top; however, if the size of the prediction unit and the size of the transform unit are different when performing intra-frame prediction, intra-frame prediction can be performed using reference pixels based on the transform unit. Additionally, intra-frame prediction using NxN partitioning only for the minimum encoding unit may also be used.

[0155] The prediction unit (230, 235) may include a prediction unit determination unit, an inter-frame prediction unit, and an intra-frame prediction unit. The prediction unit determination unit receives various information, such as prediction unit information input from the entropy decoding unit (210), prediction mode information of the intra-frame prediction method, and motion prediction related information of the inter-frame prediction method, distinguishes the prediction unit in the current encoding unit, and determines whether the prediction unit performs inter-frame prediction or intra-frame prediction. The inter-frame prediction unit (230) may perform inter-frame prediction for the current prediction unit based on information included in at least one picture among the previous picture or subsequent picture of the current picture containing the current prediction unit, using information necessary for inter-frame prediction of the current prediction unit provided by the video encoding device. Alternatively, it may perform inter-frame prediction based on information of a partially restored area within the current picture containing the current prediction unit.

[0156] To perform inter-frame prediction, based on the encoding unit, it is possible to determine whether the motion prediction method of the prediction unit included in the corresponding encoding unit is Skip Mode, Merge Mode, AMVP Mode, or Intra-frame Block Copy Mode.

[0157] The intra-frame prediction unit (235) can generate a prediction block based on pixel information within the current picture. If the prediction unit is a prediction unit that has performed intra-frame prediction, it can perform intra-frame prediction based on the intra-frame prediction mode information of the prediction unit provided by the video encoding device. The intra-frame prediction unit (235) may include an Adaptive Intra Smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter is a part that performs filtering on the reference pixel of the current block, and can determine whether to apply the filter based on the prediction mode of the current prediction unit. AIS filtering can be performed on the reference pixel of the current block using the prediction mode of the prediction unit and the AIS filter information provided by the video encoding device. If the prediction mode of the current block is a mode that does not perform AIS filtering, the AIS filter may not be applied.

[0158] The reference pixel interpolation unit can generate a reference pixel of an integer value or less by interpolating the reference pixel when the prediction mode of the prediction unit is a prediction unit that performs intra-frame prediction based on the pixel value interpolated from the reference pixel. If the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixel, the reference pixel may not be interpolated. The DC filter can generate a prediction block through filtering when the prediction mode of the current block is DC mode.

[0159] The restored block or picture may be provided to a filter unit (240). The filter unit (240) may include a deblocking filter, an offset correction unit, and an ALF.

[0160] Information regarding whether a deblocking filter has been applied to the corresponding block or picture can be received from the video encoding device, and if a deblocking filter has been applied, information regarding whether a strong filter or a weak filter has been applied. The deblocking filter of the video decoder receives information related to the deblocking filter provided by the video encoding device, and the video decoder can perform deblocking filtering on the corresponding block.

[0161] The offset correction unit can perform offset correction on the restored image based on the type of offset correction and offset value information applied to the image during encoding.

[0162] ALF can be applied to the encoding unit based on information on whether to apply ALF, ALF coefficient information, etc., provided by the encoding device. This ALF information can be provided included in a specific parameter set.

[0163] The memory (245) can store the restored picture or block so that it can be used as a reference picture or reference block, and can also provide the restored picture to the output unit.

[0164] As described above, in the embodiments of the present disclosure below, the term "Coding Unit" is used as "encoding unit" for convenience of explanation, but it may be a unit that performs not only encoding but also decoding.

[0165] Additionally, the current block represents a block to be encoded / decoded, and depending on the encoding / decoding stage, it may represent a coding tree block (or coding tree unit), an encoding block (or encoding unit), a conversion block (or conversion unit), a prediction block (or prediction unit), or a block to which an in-loop filter is applied. In this specification, 'unit' represents a basic unit for performing a specific encoding / decoding process, and 'block' may represent a pixel array of a predetermined size. Unless otherwise distinguished, 'block' and 'unit' may be used with the same meaning. For example, in the embodiments described below, the encoding block (coding block) and the encoding unit (coding unit) may be understood as having the same meaning.

[0166] In addition, encoding parameters for the current block may be commonly applied to multiple color components of the current block. For example, if the encoding mode of the current block is determined, predictions for the Y component block, Cb component block, and Cr component block can be performed based on the corresponding encoding mode.

[0167] Alternatively, depending on the color component to be encoded / decoded, the current block may refer to a Y component block, a Cb component block, or a Cr component block.

[0168] Furthermore, the picture containing the current block will be referred to as the current picture.

[0169] In the encoder, the current picture can be divided into multiple reference blocks. Here, the reference block may be referred to as a CTU (Coding Tree Unit) or CTB (Coding Tree Block).

[0170] The size of the reference block may be predefined in the encoder and decoder. Alternatively, information related to the size of the reference block may be encoded and signaled to the decoder. This information may be encoded / decoded through an upper header. For example, this information may be encoded / decoded through a sequence parameter set or a picture header.

[0171] The reference block may be further divided into multiple blocks (i.e., multiple coding blocks) based on tree structure partitioning. Here, the tree structure partitioning may include at least one of quad tree partitioning, binary tree partitioning, or ternary tree partitioning.

[0172] A prediction block for the current block can be obtained by performing a prediction block on the current block generated by dividing the reference block. Specifically, a prediction block for the current block can be obtained through inter-prediction or intra-prediction.

[0173] Inter-prediction may be intended to remove duplicate data between pictures, and intra-prediction may be intended to remove duplicate data within a picture. For example, a prediction block of the current block may be generated from a reference picture using motion information of the current block, or a prediction block of the current block may be generated from reference samples of the current block after determining the intra-prediction mode of the current block. Here, the motion information may include at least one of a motion vector, a reference picture index, and a prediction direction.

[0174] FIG. 3 illustrates an image encoding / decoding method performed by an image encoding / decoding device according to the present disclosure.

[0175] Referring to FIG. 3, a reference line for intra prediction of the current block can be determined (S300).

[0176] The current block may use one or more of the multiple reference line candidates predefined in the video encoding / decoding device as reference lines for intra-prediction. Here, the multiple reference line candidates predefined may include neighbor reference lines adjacent to the current block to be decoded and N non-neighbor reference lines located 1 to N samples away from the boundary of the current block. N may be 1, 2, 3, or more integers. For convenience of explanation, it is assumed that the multiple reference line candidates available to the current block consist of a neighbor reference line candidate and three non-neighbor reference line candidates, but are not limited thereto. That is, it is obvious that the multiple reference line candidates available to the current block may include four or more non-neighbor reference line candidates.

[0177] A video encoding device can determine an optimal reference line candidate among a plurality of reference line candidates and encode an index to specify it. A video decoding device can determine the reference line of the current block based on the index signaled through a bitstream. The index can specify any one of the plurality of reference line candidates. The reference line candidate specified by the index can be used as the reference line of the current block.

[0178] The number of signaled indices to determine the reference line of the current block may be one, two, or more. For example, if the number of signaled indices is one, the current block may perform intra prediction using only a single reference line candidate specified by the signaled index among multiple reference line candidates. Or, if the number of signaled indices is two or more, the current block may perform intra prediction using multiple reference line candidates specified by multiple indices among multiple reference line candidates.

[0179] Referring to FIG. 3, the intra prediction mode of the current block can be determined (S310).

[0180] The intra prediction mode of the current block can be determined from among a plurality of predefined intra prediction modes in the video encoding / decoding device. The plurality of predefined intra prediction modes will be examined with reference to FIGS. 4 and FIGS. 5.

[0181] FIG. 4 illustrates an example of a plurality of intra-prediction modes according to the present disclosure.

[0182] Referring to FIG. 4, a plurality of pre-defined intra-prediction modes in an image encoding / decoding device may be composed of non-directional modes and directional modes. The non-directional mode may include at least one of a planar mode or a DC mode. The directional mode may include directional modes 2 through 66.

[0183] The directional mode may be further extended than shown in FIG. 4. FIG. 5 shows an example of an extended directional mode.

[0184] In FIG. 5, modes -1 through -14 and modes 67 through 80 are shown as being added. These directional modes may be referred to as wide-angle intra-predicted modes. Whether to use wide-angle intra-predicted modes may be determined based on the shape of the current block. For example, if the current block is a non-square block where the width is greater than the height, some directional modes (e.g., 2 through 15) may be switched to wide-angle intra-predicted modes between 67 and 80. On the other hand, if the current block is a non-square block where the height is greater than the width, some directional modes (e.g., 53 through 66) may be switched to wide-angle intra-predicted modes between -1 and -14.

[0185] The range of available wide-angle intra prediction modes can be adaptively determined based on the width-to-height ratio of the current block. Table 1 shows the range of available wide-angle intra prediction modes based on the width-to-height ratio of the current block.

[0186] Width / Height Available Wide Angle Intra Predicted Mode Range W / H = 16 67~80 W / H = 8 67~78 W / H = 4 67~76 W / H = 2 67~74 W / H = 1 None W / H = 1 / 2 -1~-8 W / H = 1 / 4 -1~-10 W / H = 1 / 8 -1~-12 W / H = 1 / 16 -1~-14

[0187] Among the plurality of intra prediction modes mentioned above, K candidate modes (most probable mode, MPM) can be selected. A candidate list including the selected candidate modes can be generated. An index indicating any one of the candidate modes in the candidate list can be signaled. The intra prediction mode of the current block can be determined based on the candidate mode indicated by the index. For example, the candidate mode indicated by the index can be set as the intra prediction mode of the current block. Alternatively, the intra prediction mode of the current block may be determined based on the value of the candidate mode indicated by the index and a predetermined difference value. The difference value may be defined as the difference between the value of the intra prediction mode of the current block and the value of the candidate mode indicated by the index. The difference value may be signaled via a bitstream. Alternatively, the difference value may be a value pre-defined in the video encoding / decoding device. Alternatively, the intra prediction mode of the current block may be determined based on a flag indicating whether a mode identical to the intra prediction mode of the current block exists in the candidate list. For example, if the flag is a first value, the intra prediction mode of the current block may be determined from the candidate list. In this case, an index indicating any one of the multiple candidate modes belonging to the candidate list may be signaled. The candidate mode indicated by the index may be set as the intra prediction mode of the current block. On the other hand, if the flag is a second value, any one of the remaining intra prediction modes may be set as the intra prediction mode of the current block. The remaining intra prediction mode may refer to a mode among the pre-defined multiple intra prediction modes excluding the candidate mode belonging to the candidate list. If the flag is a second value, an index indicating any one of the remaining intra prediction modes may be signaled.The intra prediction mode indicated by the signaled index can be set to the intra prediction mode of the current block.

[0188] The intra prediction mode of a chroma block can be selected from among multiple intra prediction mode candidates of the chroma block. To this end, index information indicating one of the intra prediction mode candidates of the chroma block can be explicitly encoded and signaled through a bitstream. Table 2 is an example of intra prediction mode candidates of the chroma block.

[0189] Intra-prediction mode candidates for index chroma blocks: Luma Mode: 0 Luma Mode: 50 Luma Mode: 18 Luma Mode: 1 Others 0 6 6 0 0 0 1 5 0 6 6 5 0 5 5 0 2 1 8 1 8 6 6 1 8 1 8 3 1 1 1 6 6 1 4 DM

[0190] In the example of Table 2, DM (Direct Mode) means setting the intra prediction mode of the luminance block located at the same position as the chroma block to the intra prediction mode of the chroma block. Meanwhile, the luminance block located at the same position as the chroma block can be determined based on the position of the top-left sample or the position of the center sample of the chroma block.

[0191] For example, if the intra prediction mode (luminance mode) of the luminance block is 0 (planar mode) and the index points to 2, the intra prediction mode of the chroma block can be determined as horizontal mode (18). For example, if the intra prediction mode (luminance mode) of the luminance block is 1 (DC mode) and the index points to 0, the intra prediction mode of the chroma block can be determined as planner mode (0).

[0192] Consequently, the intra prediction mode of the chroma block may also be set to one of the intra prediction modes shown in FIG. 4 or FIG. 5. The intra prediction mode of the current block may also be used to determine the reference line of the current block, in which case step S310 may be performed before step S300.

[0193] Meanwhile, in the present disclosure, the chroma block may represent at least one of a Cb component block or a Cr component block.

[0194] Referring to FIG. 3, an intra prediction can be performed on the current block based on the reference line of the current block and the intra prediction mode (S320).

[0195] Hereinafter, with reference to FIGS. 6 to 8, we will examine in detail the intra prediction method for each intra prediction mode. However, for the sake of convenience of explanation, it is assumed that a single reference line is used for the intra prediction of the current block, but the intra prediction method described below can be applied in the same or similar way even when multiple reference lines are used.

[0196] FIG. 6 illustrates a planner mode-based intra-prediction method according to the present disclosure.

[0197] Referring to FIG. 6, T represents a reference sample located at the upper-right corner of the current block, and L represents a reference sample located at the lower-left corner of the current block. P1 can be generated through horizontal interpolation. For example, P1 can be generated by interpolating T with a reference sample located on the same horizontal line as P1. P2 can be generated through vertical interpolation. For example, P2 can be generated by interpolating L with a reference sample located on the same vertical line as P2. The current sample within the current block can be predicted through the weighted sum of P1 and P2 as shown in the following Equation 1.

[0198]

[0199] In Equation 1, weights α and β can be determined by considering the width and height of the current block. Depending on the width and height of the current block, weights α and β may have the same value or different values. If the width and height of the current block are the same, weights α and β can be set equally, and the predicted sample of the current sample can be set to the average value of P1 and P2. If the width and height of the current block are not the same, weights α and β may have different values. For example, if the width is greater than the height, a smaller value can be set for the weight corresponding to the width of the current block and a larger value can be set for the weight corresponding to the height of the current block. Conversely, if the width is greater than the height, a larger value can be set for the weight corresponding to the width of the current block and a smaller value can be set for the weight corresponding to the height of the current block. Here, the weight corresponding to the width of the current block may be β, and the weight corresponding to the height of the current block may be α.

[0200] FIG. 7 illustrates a DC mode-based intra prediction method according to the present disclosure.

[0201] Referring to FIG. 7, the average value of surrounding samples adjacent to the current block can be calculated, and the calculated average value can be set as the predicted value for all samples within the current block. Here, the surrounding samples may include the top reference sample and the left reference sample of the current block. However, depending on the shape of the current block, the average value may be calculated using only the top reference sample or only the left reference sample. For example, if the width of the current block is greater than the height, the average value may be calculated using only the top reference sample of the current block. Alternatively, if the ratio of the width to the height of the current block is greater than or equal to a predetermined threshold value, the average value may be calculated using only the top reference sample of the current block. Alternatively, if the ratio of the width to the height of the current block is less than or equal to a predetermined threshold value, the average value may be calculated using only the top reference sample of the current block. On the other hand, if the width of the current block is smaller than the height, the average value may be calculated using only the left reference sample of the current block. Alternatively, if the ratio of the width to the height of the current block is less than or equal to a predetermined threshold value, the average value may be calculated using only the left reference sample of the current block. Alternatively, if the ratio of the width to the height of the current block is greater than or equal to a predetermined threshold value, the average value can be calculated using only the left reference sample of the current block.

[0202] FIG. 8 illustrates an intra-prediction method based on a directional mode according to the present disclosure.

[0203] If the intra prediction mode of the current block is a directional mode, projection can be performed on a reference line according to the angle of the directional mode. If a reference sample exists at the projected location, that reference sample can be set as the prediction sample of the current sample. If no reference sample exists at the projected location, a sample corresponding to the projected location can be generated using one or more neighboring samples adjacent to the projected location. For example, a sample corresponding to the projected location can be generated by performing interpolation based on two or more neighboring samples adjacent in both directions relative to the projected location. Alternatively, a single neighboring sample adjacent to the projected location can be set as the sample corresponding to the projected location. In this case, among multiple neighboring samples adjacent to the projected location, the neighboring sample closest to the projected location may be used. The sample corresponding to the projected location can be set as the prediction sample of the current sample.

[0204] Referring to FIG. 8, for the current sample B, if projection is performed to a reference line according to the angle of the intra-prediction mode at that location, a reference sample exists at the projected location (i.e., a reference sample at an integer location, R3). In this case, the reference sample at the projected location can be set as the prediction sample for the current sample B. For the current sample A, if projection is performed to a reference line according to the angle of the intra-prediction mode at that location, a reference sample (i.e., a reference sample at an integer location) does not exist at the projected location. In this case, a sample (r) at a fractional location can be generated by performing interpolation based on neighboring samples (e.g., R2 and R3) adjacent to the projected location. The generated sample (r) at a fractional location can be set as the prediction sample for the current sample A.

[0205] Figure 9 illustrates a method for deriving samples of fractional positions.

[0206] In the example of Fig. 9, the variable h represents the vertical distance (i.e., vertical distance) from the position of predicted sample A to the reference sample line, and the variable w represents the horizontal distance (i.e., horizontal distance) from the position of predicted sample A to the fractional position sample. Additionally, the variable θ represents a predefined angle according to the directionality of the intra-prediction mode, and the variable x represents the fractional position.

[0207] The variable w can be derived as shown in the following mathematical equation 2.

[0208]

[0209] Subsequently, by removing the integer position from the variable w, the fractional position can finally be derived.

[0210] Fractional position samples can be generated by interpolating adjacent integer position reference samples. For example, fractional position reference samples at position x can be generated by interpolating integer position reference samples R2 and integer position reference samples R3.

[0211] In deriving fractional position samples, a scaling factor can be used to avoid real number operations. For example, if the scaling factor f is set to 32, the distance between neighboring integer reference samples can be set to 32 instead of 1, as in the example shown in FIG. 8 (b).

[0212] In addition, the tangent value for the angle θ determined by the directionality of the intra prediction mode can also be scaled up using the same scaling factor (e.g., 32).

[0213] Figures 10 and 11 illustrate tangent values ​​for angles scaled by 32 times for each intra prediction mode.

[0214] Figure 10 shows the scaled result of the tangent value for the non-wide angle intra prediction mode, and Figure 11 shows the scaled result of the tangent value for the wide angle intra prediction mode.

[0215] If the tangent value (tanθ) for the angle value in the intra prediction mode is positive, intra prediction can be performed using only one of the reference samples belonging to the top line of the current block (i.e., top reference samples) or the reference samples belonging to the left line of the current block (i.e., left reference samples). On the other hand, if the tangent value for the angle value in the intra prediction mode is negative, both the reference samples located at the top and the reference samples located at the left are utilized.

[0216] At this time, to simplify the implementation, the left reference samples may be projected upward or the top reference samples may be projected to the left to arrange the reference samples into a 1D array, and intra prediction may be performed using the reference samples in the 1D array.

[0217] FIG. 12 is a diagram illustrating an intra-prediction pattern when the directional mode is one of modes 34 to 49.

[0218] When the intra prediction mode of the current block is one of modes 34 to 49, intra prediction is performed using not only the upper reference samples of the current block but also the left reference samples. At this time, as in the example shown in FIG. 12, the reference samples located on the left side of the current block can be copied to the position of the upper line, or the reference samples located on the left side can be interpolated to generate the reference samples of the upper line.

[0219] For example, if one wishes to obtain a reference sample for position A at the top of the current block, projection can be performed from position A on the top line to the left line of the current block, taking into account the directionality of the intra prediction mode of the current block. If the projected position is denoted as 'a', the value corresponding to position 'a' can be copied, or a fractional position value corresponding to 'a' can be generated and set as the value of position A. For example, if position 'a' is an integer position, the value of position A can be generated by copying the integer position reference sample. On the other hand, if position 'a' is a fractional position, the reference sample located above position 'a' and the reference sample located below position 'a' can be interpolated, and the interpolated value can be set as the value of position A. Meanwhile, the direction of projection from position A at the top of the current block to the left line of the current block may be parallel to the direction of the intra prediction mode of the current block, while being opposite.

[0220] Figure 13 is a diagram illustrating an example of generating an upper reference sample by interpolating left reference samples.

[0221] In Fig. 13, the variable h represents the horizontal distance between position A on the top line and position a on the left line. The variable w represents the vertical distance between position A on the top line and position a on the left line. Additionally, the variable θ represents a predefined angle according to the directionality of the intra prediction mode, and the variable x represents a fractional position.

[0222] The variable h can be derived as shown in the following mathematical equation 3.

[0223]

[0224] Subsequently, by removing the integer position from the variable h, the fractional position can finally be derived.

[0225] In deriving fractional position samples, a scaling factor can be used to avoid real-valued operations. For example, the tangent value for the variable θ can be scaled using a scaling factor f1. Here, since the direction projected to the left line is parallel and opposite to the directional prediction model, the scaled tangent value shown in FIGS. 10 and FIGS. 11 may also be used.

[0226] When a scaling factor f1 is applied, Equation 3 can be modified and used as shown in Equation 4 below.

[0227]

[0228] In the above manner, a 1D reference sample array can be constructed using only the reference samples belonging to the top line. As a result, an intra prediction for the current block can be performed using only the top reference samples constructed as a 1D array.

[0229] Figure 14 shows an example in which intra prediction is performed using reference samples arranged in a 1D array.

[0230] As shown in the example illustrated in Fig. 14, by projecting the left reference samples to generate the top reference samples, the prediction samples of the current block can be obtained using only the reference samples belonging to the top line.

[0231] Contrary to what is shown in FIGS. 12 and 14, a 1D reference sample array may be constructed using only the reference samples belonging to the left line by projecting the top reference sample onto the left line. Specifically, for directional modes 19 through 33 among the directional modes where the tangent value (tanθ) for the angle of the directional mode is negative, the reference samples belonging to the top line may be projected onto the left line to generate the left reference sample.

[0232] The intra prediction mode of the current block can also be derived using reference samples surrounding the current block. Specifically, the gradients for the horizontal and vertical directions of the reference samples are calculated, and the calculated gradients are used to derive the intra prediction mode of the current block.

[0233] Figure 15 is a diagram illustrating an example of setting a reference area.

[0234] For the sake of convenience of explanation, the current block size is assumed to be 4x4.

[0235] A reference area can be set to induce an intra-prediction mode of the current block. For example, in FIG. 15, it is assumed that w0 columns adjacent to the left of the current block and h0 rows adjacent to the top of the current block are set as the reference area.

[0236] The number of columns (w0) and / or rows (h0) constituting the reference region may be fixed in the encoder and decoder. Alternatively, the number of columns (w0) and / or rows (h0) may be determined based on at least one of the size / shape of the current block, whether Intra Sub-Partitioning (ISP) is applied to the current block, or whether the current block is adjacent to a CTU boundary.

[0237] As another example, the size of the reference area may be determined according to the type of filter applied to the reference area. Specifically, the width and height of the filter can be set to the number of columns w0 and the number of rows h0, respectively. For example, assuming that a 3x3 mask as shown in FIG. 17, which will be described later, is used, the number of columns w0 and the number of rows h0 can each be set to 3.

[0238] The reference area may extend beyond the right boundary and / or bottom boundary of the current block. For example, in the example illustrated in FIG. 15, the reference area is shown as extending w1 from the right boundary of the current block and h1 from the bottom boundary of the current block.

[0239] The right extension distance w1 and / or bottom extension distance h1 can be set to be equal to the width and / or height of the current block. For example, if the size of the current block is 4x4, the right extension distance w1 can be set to 4, equal to the width of the current block, and the bottom extension distance h1 can be set to 4, equal to the height of the current block.

[0240] As another example, a reference area can also be set, as in the example shown in FIG. 16.

[0241] Specifically, as in the example illustrated in FIG. 16 (a), the right extension distance w1 and / or the bottom extension distance h1 can be set to 0. Furthermore, as in the example illustrated in FIG. 16 (b), the upper reference area can be formed using only reference samples with x-axis coordinates between 0 and (w-1), and the left reference area can be formed using only reference samples with y-axis coordinates between 0 and (h-1). Here, w represents the width of the current block, and h represents the height of the current block.

[0242] As another example, reference line candidates for intra prediction of the current block or at least one of the reference line candidates may be set as a reference region.

[0243] As another example, depending on whether the current block is adjacent to the CTU boundary, the reference area may be configured using only the top reference area or only the left reference area.

[0244] Filtering (i.e., convolution) using a mask within a reference region can be performed. In this case, the filter used may be at least one of a Sobel mask or a Prewitt mask that outputs a gradient value.

[0245] Figure 17 illustrates the filter coefficients for the Sobel mask and the Prewit mask, respectively.

[0246] Filters of a different type than those shown in FIG. 17 may also be applied to the reference area. For example, instead of a 3x3 square filter, a 1D filter of 1x3 or 3x1, a rectangular filter of 2x3 or 2x3, a cross-shaped filter, or a diamond-shaped filter may be applied to the reference area. Alternatively, filters of a different size than those shown in FIG. 17 (e.g., 2x2, 4x4, or 5x5, etc.) may also be applied to the reference area.

[0247] The type of filter applied to the reference area may be predefined in the encoder and decoder. Alternatively, multiple filter candidates may be predefined, and index information pointing to one of the multiple filter candidates may be encoded and explicitly signaled through the bitstream.

[0248] As another example, at least one of a plurality of filter candidates may be adaptively selected based on at least one of the size / shape of the current block, whether an ISP is applied to the current block, the size of the reference region, the intra-prediction mode of neighboring blocks, or whether the current block touches a CTU boundary. Here, the neighboring blocks may include at least one of the top neighboring block or the left neighboring block of the current block.

[0249] The type of filter applied to the top reference area and the type of filter applied to the left reference area may be different.

[0250] By applying a vertical direction mask to a specific reference sample within a reference region, the vertical direction slope Dy for the reference sample can be obtained. Additionally, by applying a horizontal direction mask to a specific reference sample within a reference region, the horizontal direction slope Dx for the reference sample can be obtained.

[0251] Figure 18 shows the locations where the vertical and horizontal inclinations are obtained within the reference area.

[0252] Assuming that a 3x3 mask is applied as in the example shown in FIG. 18, a vertical slope Dy and a horizontal slope Dx can be obtained for each of the reference samples that are not adjacent to the boundary of the reference region. For example, when w0 and h0 are 3 and w1 and h1 are 4, as in the example shown in FIG. 18, 17 vertical slopes Dy and 17 horizontal slopes Dx can be obtained for each of the 17 reference samples.

[0253] If a filter of a different size or shape than that shown in Fig. 18 is applied, the vertical slope Dy and the horizontal slope Dx can be obtained for more / fewer reference samples than shown.

[0254] Based on the vertical slope Dy and horizontal slope Dx of each reference sample, an intra-prediction mode can be determined for each reference sample.

[0255] We will explain how to determine the intra-prediction mode of a reference sample using the vertical slope Dy and the horizontal slope Dx.

[0256] For example, if either the vertical slope Dy or the horizontal slope Dx is 0, the directional mode of the reference sample can be determined as the horizontal mode (18) or the vertical mode (50). Specifically, if the horizontal slope Dx is 0 and the vertical slope Dy is not 0, the intra-prediction mode of the reference sample can be determined as the vertical mode (50). Conversely, if the vertical slope Dy is 0 and the horizontal slope Dx is not 0, the intra-prediction mode of the reference sample can be determined as the horizontal mode (18).

[0257] If the vertical slope Dy and the horizontal slope Dx are both not zero, one of the remaining directional modes, excluding the horizontal mode and the vertical mode, can be determined as the intra-prediction mode of the reference sample.

[0258] Here, the intra prediction mode group to which the intra prediction mode of the reference sample belongs can be determined by comparing the absolute values ​​of the vertical slope Dy and the horizontal slope Dx. Here, the intra prediction mode group may consist of multiple directional modes of similar directionality.

[0259] Figure 19 shows an example of grouping directional modes into multiple intra-prediction mode groups.

[0260] In FIG. 19, directional modes are exemplified as being classified into four intra-predicted mode groups (a to d) based on the horizontal direction mode (18), diagonal direction mode (34), and vertical direction mode (50).

[0261] In the illustrated example, groups a and b are symmetrical with respect to the horizontal direction mode (18), and groups c and d are symmetrical with respect to the vertical direction mode (50).

[0262] In addition, the angles of directional modes 36 through 66 are the same as the angles of modes 2 through 34 transposed.

[0263] If the absolute value of the horizontal slope Dx of a reference sample is greater than the absolute value of the vertical slope Dy, the directional mode of the reference sample may belong to group a or group b.

[0264] Conversely, if the absolute value of the vertical slope Dy of a reference sample is greater than the slope of the horizontal slope Dx, the directional mode of the reference sample may belong to group c or group d.

[0265] Table 3 shows the intra prediction mode groups to which the reference sample's intra prediction mode belongs, depending on the magnitudes of the horizontal slope Dx and the vertical slope Dy.

[0266] if (|Dx| > |Dy|)ElseDx >= 0Dy >= 0bDx >= 0Dy >= 0cDx < 0Dy >= 0aDx < 0Dy >= 0dDx >= 0Dy < 0aDx >= 0Dy < 0dDx < 0Dy < 0bDx < 0Dy < 0c

[0267] Using the horizontal slope Dx and vertical slope Dy of the reference sample, the slope of the directional mode to be assigned to the reference sample can be derived. To this end, a variable R representing the ratio between the horizontal slope and the vertical slope can be derived as shown in Equation 5 below.

[0268]

[0269] As exemplified in mathematical formula 5, the variable R can be derived by using the larger absolute value between the horizontal slope Dx and the vertical slope Dy as the denominator.

[0270] Subsequently, the directional mode of the reference sample can be determined by comparing the variable R with the tangent value (tanθ) for the angle of each directional mode. Specifically, a directional mode having the same tangent value as the variable R or the most similar tangent value can be assigned to the reference sample.

[0271] At this time, if the tangent values ​​for each angle of the directional modes are stored in the encoder and decoder in a scaled state as in the example illustrated in FIG. 10 or FIG. 11, the directional mode of the reference sample can be determined by scaling the variable R using the same scaling factor.

[0272] Next, the amplitude of each of the reference samples can be derived. The amplitude can be derived as the sum of the absolute value of the horizontal slope Dx and the absolute value of the vertical slope Dy, as shown in Equation 6 below.

[0273]

[0274] Next, for each of the intra prediction modes, the amplitude value of each of the reference samples assigned to the same intra prediction mode can be accumulated.

[0275]

[0276] In Equation 7, intra_mode represents an intra-predicted mode. For example, the amplitude accumulation value for a directional mode with mode number N is derived by summing the amplitude values ​​of reference samples assigned to mode N within a reference region, and the amplitude accumulation value for a directional mode with mode number M can be derived by summing the amplitude values ​​of reference samples assigned to mode M within a reference region.

[0277] The buffer storing the amplitude accumulation value can be initialized in blocks. For example, when specifying a reference area around the current block, the amplitude accumulation value for each intra prediction mode can be initialized to 0.

[0278] Through the above process, when a histogram recording the amplitude accumulation values ​​for each intra prediction mode is derived, at least one intra prediction mode can be selected in order of increasing amplitude accumulation values ​​within the histogram. The number of selected intra prediction modes may be M, and M may be a natural number greater than or equal to 1. The value of M may be predefined in the encoder and decoder. Alternatively, the value of M may be adaptively determined by considering at least one of the size / shape of the current block and whether an ISP is applied to the current block. That is, M intra prediction modes may be selected in descending order of amplitude accumulation values.

[0279] At least one intra prediction mode selected from the histogram can be set as the intra prediction mode of the current block, and a prediction block of the current block can be obtained based on the intra prediction mode of the current block. For example, if one intra prediction mode is selected from the histogram, a prediction block obtained based on the selected intra prediction mode can be used as the final prediction block of the current block.

[0280] When multiple intra prediction modes are selected from a histogram, intra prediction can be performed based on each of the multiple intra prediction modes. Accordingly, when multiple prediction blocks are generated, the final prediction block of the current block can be obtained through an average operation or a weighted sum operation of the multiple prediction blocks.

[0281] At this time, for the weighted sum operation, the weights applied to each prediction block can be determined based on the amplitude of the intra prediction mode. That is, among the multiple intra prediction modes, the largest weight can be assigned to the prediction block derived based on the intra prediction mode with the largest amplitude, and the smallest weight can be assigned to the prediction block derived based on the intra prediction mode with the smallest amplitude.

[0282] At this time, the weight assigned to each prediction block can be determined based on the ratio between amplitudes. Alternatively, the values ​​of the weights for each amplitude rank can be stored, and then the weights mapped to the amplitude ranks of the corresponding intra-prediction mode can be applied to the prediction blocks.

[0283] A prediction block for the current block can be obtained by considering at least one default mode along with at least one intra prediction mode selected from the histogram. For example, multiple prediction blocks for the current block can be obtained by performing intra prediction based on each of the intra prediction mode and the default mode selected from the histogram. Subsequently, a final prediction block for the current block can be obtained through an average operation or a weighted sum operation of the multiple prediction blocks.

[0284] The number of default modes N can be an integer greater than or equal to 0 or 1. When M intra prediction modes are selected from the histogram, intra prediction can be performed based on each of the M intra prediction modes and N default modes to obtain (M+N) prediction blocks. Subsequently, the final prediction block of the current block can be obtained through an average operation or a weighted sum operation of the (M+N) prediction blocks.

[0285] The number of default modes N may be predefined in the encoder and decoder. Alternatively, the number of default modes N may be adaptively determined based on at least one of the size / shape of the current block, whether an ISP is applied to the current block, or whether at least one intra-prediction mode selected from the histogram includes a default mode.

[0286] The default mode may include at least one of a planar mode, a DC mode, or a predefined directional mode.

[0287] The encoder and decoder may also be configured to use a predefined mode (e.g., planner mode) among the modes listed above as the default mode.

[0288] Alternatively, the type of default mode may be adaptively determined based on the type of directional mode selected via the histogram. For example, if at least one directional mode selected via the histogram is a vertical mode or a horizontal mode, the planar mode or DC mode may be set as the default mode. On the other hand, if a vertical and / or horizontal mode is not selected via the histogram, the vertical mode or horizontal mode may be set as the default mode.

[0289] Instead of setting the region adjacent to the current block as the reference region, you can also set the reference block indicated by the current block's block vector as the reference region.

[0290] Depending on the shape of the current block, the availability of wide-angle intra prediction modes may be determined. For example, if the current block is a square shape with equal width and height, the directional modes selected from the histogram may consist of non-wide-angle intra prediction modes. On the other hand, if the current block is a non-square shape with different widths and heights, some of the directional modes selected from the histogram may be converted into wide-angle intra prediction modes.

[0291] When performing intra prediction based on an intra prediction mode derived through a histogram, a predefined reference line may be used. Here, the predefined reference line may be an adjacent reference line (i.e., index 0) or a non-adjacent reference line (e.g., index 1) adjacent to the current block.

[0292] Information indicating whether to apply a method of performing intra prediction by selecting an intra prediction mode through the histogram described above can be encoded and signaled through a bitstream. The information may be a 1-bit flag.

[0293] Alternatively, whether to select an intra prediction mode through a histogram can be determined based on at least one of the size / shape of the current block, whether an ISP is applied to the current block, whether the current block touches a CTU boundary, or whether neighboring blocks are encoded with intra prediction.

[0294] For example, if at least one of the top neighbor block or left neighbor block of the current block is not encoded in intra prediction, a method for selecting an intra prediction mode through a histogram can be applied to the current block.

[0295] The method of selecting an intra-prediction mode via a histogram can be applied to both the luminance component and the chroma component. Alternatively, the method described above can be applied only to the luminance component. Or, for each of the luminance component and the chroma component, it may be determined independently whether to select an intra-prediction mode via a histogram.

[0296] The intra prediction mode of the current block can also be derived by utilizing the region surrounding the current block. The surrounding region referenced to derive the intra prediction mode of the current block can be referred to as the reference region.

[0297] FIG. 20 is a drawing illustrating a reference area around the current block.

[0298] In FIG. 20, the width w and height h of the current block are both 4.

[0299] As shown in the example illustrated in FIG. 20, a surrounding area adjacent to the current block can be set as a reference area. Specifically, a left reference area adjacent to the left of the current block and a top reference area adjacent to the top of the current block can be set, respectively.

[0300] The size of the left reference area can be represented by w0, and the size of the top reference area can be represented by h0. For example, w0 represents the number of reference sample lines (i.e., reference sample columns) included in the left reference area, and h0 represents the number of reference sample lines (i.e., reference sample rows) included in the top reference area. In this case, w0 and h0 can each be a natural number greater than or equal to 1. Additionally, w0 and h0 may be predefined in the encoder and decoder.

[0301] For example, as shown in the example illustrated in FIG. 20, if the size of the current block is 4x4 or 2x2, the 4x4 or 2x2 area to the left of the current block can be set as the left reference area, and the 4x4 or 2x2 area to the top of the current block can be set as the top reference area.

[0302] Alternatively, at least one of the size w0 and / or h0 of the reference area may be adaptively determined based on at least one of the size of the current block, the shape of the current block, whether Intra Sub-partitioning (ISP) is applied to the current block, or whether the current block is adjacent to a CTU boundary. Here, the size of the current block represents at least one of the width, height, or product of the width and height of the current block. For example, at least one of the left reference area and the top reference area may be determined to be equal to the size of the current block. Alternatively, the left reference area may be set as a square area with a side length equal to the height of the current block, and the top reference area may be set as a square area with a side length equal to the width of the current block.

[0303] Alternatively, the size of the reference area can be determined by comparing the size of the current block with a threshold value. For example, if the size of the current block is greater than or equal to the threshold value, the size of at least one of the left reference area or the top reference area can be set to 4x4. On the other hand, if the size of the current block is less than the threshold value, the size of at least one of the left reference area or the top reference area can be set to 2x2.

[0304] Alternatively, the size h0 of the top reference area can be determined based on the result of comparing the width of the current block with a threshold value. For example, if the width w of the current block is smaller than the threshold value, the size h0 of the top reference area can be set to 1. On the other hand, if the width w of the current block is larger than the threshold value, the size h0 of the top reference area can be set to 2.

[0305] Similarly, the size w0 of the left reference area can be determined based on the result of comparing the vertical length of the current block with the threshold value.

[0306] Alternatively, conversely to the above, the size w0 of the left reference area may be determined based on the result of comparing the width of the current block with the threshold value, and the size h0 of the top reference area may be determined based on the result of comparing the height of the current block with the threshold value.

[0307] Intra prediction can be performed on a reference region using reference samples from the reference region. Here, reference samples for the left reference region may belong to a column adjacent to the left of the left reference region, and reference samples for the top reference region may belong to a row adjacent to the top of the top reference region.

[0308] In the example illustrated in FIG. 20, w1 and h1 are variables representing the range of reference samples used to perform intra-prediction on a reference region. Specifically, w1 may represent the number of reference samples in the upper-right region of the upper reference region, and h1 may represent the number of reference samples in the lower-left region of the left reference region.

[0309] In the example illustrated in Fig. 20, w1 and h1 are both 4.

[0310] At this time, w1 and h1 may be predefined in the encoder and decoder. For example, w1 and h1 may each be a natural number greater than or equal to 0 or 1.

[0311] Alternatively, at least one of w1 or h1 may be adaptively determined based on at least one of the size of the current block, the shape of the current block, whether Intra Sub-partitioning (ISP) is applied to the current block, or whether the current block is adjacent to a CTU boundary. Here, the size of the current block represents at least one of the width, height, or the product of the width and height of the current block.

[0312] For example, if the current block size (e.g., width or height) is greater than or equal to a threshold value, at least one of w1 or h1 may be set to 8 or 16. On the other hand, if the current block size (e.g., width or height) is less than a threshold value, at least one of w1 or h1 may be set to 4.

[0313] Meanwhile, under the above conditions, w1 can be determined dependently on the width w of the current block, and h1 can be determined dependently on the height h of the current block.

[0314] Alternatively, if the current block is square, w1 and h1 may be identical. On the other hand, if the current block is non-square, w1 and h1 may be different.

[0315] Intra-prediction can be performed on a reference region using reference samples for the reference region. Specifically, after performing intra-prediction on the reference region based on multiple intra-prediction modes, the cost for each prediction result can be calculated.

[0316] Figures 21 and 22 illustrate an example of performing intra prediction on a reference area based on planner mode.

[0317] Specifically, in FIG. 21, reference samples used to perform intra prediction based on planer mode for the left reference area and reference samples used to perform intra prediction based on planer mode for the top reference area are shown.

[0318] As in the example illustrated in FIG. 21, reference samples may be included in the line adjacent to the left of the left reference area and the line adjacent to the top of the top reference area.

[0319] Accordingly, left reference samples for the left reference area are adjacent to the left reference area, whereas top reference samples for the left reference area may not be adjacent to the left reference area.

[0320] Additionally, the top reference samples for the top reference area are adjacent to the top reference area, whereas the left reference samples for the top reference area may not be adjacent to the top reference area.

[0321] Alternatively, as in the example illustrated in FIG. 22, the reference samples for the upper reference area may consist of upper reference samples adjacent to the upper reference area and left reference samples adjacent to the upper reference area, and the reference samples for the left reference area may consist of upper reference samples adjacent to the upper reference area and left reference samples adjacent to the upper reference area.

[0322] Figures 23 and 24 illustrate an example of performing intra prediction on a reference region based on DC mode.

[0323] When intra prediction based on DC mode is performed, the prediction samples can be set as the average value of the reference samples. In this case, as shown in the example illustrated in FIG. 23, the average value for the upper reference area (i.e., DCval) can be calculated using only the upper reference samples adjacent to the upper reference area, and the average value for the left reference area can be calculated using only the left reference samples adjacent to the left reference area.

[0324] Alternatively, as in the example illustrated in FIG. 24, the average value for the upper reference area can be derived using the left reference samples adjacent to the upper reference area together with the upper reference samples adjacent to the upper reference area, and the average value for the left upper reference area can be derived using the upper reference samples adjacent to the left reference area together with the left reference samples adjacent to the left reference area.

[0325] Figures 25 and 26 illustrate an example of performing intra prediction on a reference region based on a directional mode.

[0326] Meanwhile, depending on the directional mode, intra prediction for the reference region can be performed using only the reference samples belonging to the top row of the top reference region, or intra prediction for the reference region can be performed using only the reference samples belonging to the left column of the left reference region.

[0327] For example, FIG. 25 shows an example in which an intra prediction for a reference region is performed using only the reference samples belonging to the top row of the upper reference region.

[0328] For example, if the index of the directional mode is equal to or greater than the index of the top-left diagonal directional mode (i.e., 34), an intra prediction for the reference regions (i.e., the top reference region and the left reference region) can be performed using only the reference samples belonging to the top row of the top reference region. Accordingly, for the top reference region, reference samples adjacent to the top reference region are used, but for the left reference region, reference samples not adjacent to the left reference region may be used.

[0329] Alternatively, as in the example illustrated in FIG. 26, for the upper reference region, intra prediction may be performed using upper reference samples adjacent to the upper reference region, and for the left reference region, intra prediction may be performed using upper reference samples adjacent to the left reference region.

[0330] Meanwhile, if the index of the directional mode is smaller than the index of the vertical mode (i.e., 50), the reference samples belonging to the left column of the left reference area (i.e., left reference samples) can be projected to the top row of the top reference area according to the direction of the directional mode to derive the reference samples belonging to the top row (i.e., top reference samples). Meanwhile, if the position projected from the left reference samples is not an integer position, the left reference samples can be interpolated to obtain the top reference samples.

[0331] Although not explicitly stated, if the index of the directional mode is smaller than the index of the top-left diagonal directional mode, an intra prediction for the reference region (i.e., the top reference region and the left reference region) can be performed using only the reference samples belonging to the left column of the left reference region.

[0332] Meanwhile, if the index of the directional mode is greater than the index of the horizontal directional mode (i.e., 18), the reference samples belonging to the top row of the top reference area (i.e., top reference samples) can be projected to the left column of the left reference area according to the direction of the directional mode to derive the reference samples belonging to the left column (i.e., left reference samples). Meanwhile, if the position projected from the left reference sample is not an integer position, the left reference samples can be interpolated to obtain the top reference sample.

[0333] After performing multiple intra predictions on a reference region based on multiple intra prediction modes, the cost for each intra prediction mode can be calculated. Specifically, the cost for an intra prediction mode can be calculated based on the difference between the reconstructed samples within the reference region and the predicted samples within the reference region obtained through intra prediction.

[0334] Meanwhile, the cost function for calculating the cost may include at least one of SAD (Sum of Absolute Difference), SATD (Sum of Absolute Transformed Differences), SSD (Sum of Squared Difference), or MR-SAD (Mean-Removed Sum of Absolute Differences).

[0335] Once the cost for each intra prediction mode is calculated, the intra prediction mode with the lowest cost can be selected.

[0336] Alternatively, N intra-prediction modes with low costs can be selected. Here, N is a natural number greater than or equal to 1, such as 2, 3, or 4.

[0337] Subsequently, based on N intra prediction modes, N intra predictions are performed on the current block to obtain N prediction blocks. Subsequently, the final prediction block of the current block can be obtained by weighting the N prediction blocks.

[0338] Meanwhile, the weights for the weighted sum can be determined by the ratio of the costs of each intra-prediction mode. That is, if the cost of an intra-prediction mode is low, a high weight may be assigned to the prediction block derived from that intra-prediction mode. Conversely, if the cost of an intra-prediction mode is high, a low weight may be assigned to the prediction block derived from that intra-prediction mode.

[0339] Meanwhile, an intra prediction mode can be induced for each of the upper reference area and the left reference area. For example, based on the results of performing multiple intra predictions on the upper reference area, a first intra prediction mode with the lowest cost can be selected, and based on the results of performing multiple intra predictions on the left reference area, a second intra prediction mode with the lowest cost can be selected. Subsequently, based on the first intra prediction mode and the second intra prediction mode, two intra predictions can be performed on the current block to obtain the first prediction block and the second prediction block. Subsequently, the current block can be obtained by weighting the first prediction block and the second prediction block or averaging them.

[0340] Alternatively, at least one intra prediction mode selected in order of lowest cost may be inserted into the MPM list of the current block. For example, a first intra prediction mode derived from the top reference region and a second intra prediction mode derived from the left reference region may be inserted into the MPM list of the current block.

[0341] Subsequently, at least one of the intra prediction mode candidates included in the MPM list can be selected to perform an intra prediction for the current block.

[0342] Alternatively, for each of the left reference area and the top reference area, the cost for each intra prediction mode can be calculated. Subsequently, the area with the smaller cost among the left reference area and the top reference area can be selected, and the intra prediction mode having the smallest cost in the selected area can be set as the intra prediction mode of the current block. In this case, the cost of each reference area may be derived by summing the costs of the intra prediction modes for that reference area.

[0343] Meanwhile, the number and / or types of intra prediction modes applied to the left reference area and the intra prediction modes applied to the top reference area may be the same or different.

[0344] Alternatively, N intra prediction modes can be selected from the top reference area in order of decreasing cost, and N intra prediction modes can be selected from the left reference area in order of decreasing cost.

[0345] Subsequently, based on 2N intra prediction modes, the cost of 2N intra prediction modes can be calculated again by applying them to the upper reference area and the left reference area. That is, if the initial cost of an intra prediction mode was obtained by applying intra prediction to only one of the left reference area and the upper reference area, the cost of the intra prediction mode in this round can be obtained by applying intra prediction to the left reference area and the upper reference area.

[0346] Afterwards, the intra prediction mode with the smallest cost among 2N intra prediction modes, or M intra prediction modes selected in order of smallest cost, can be used for the intra prediction of the current block.

[0347] Meanwhile, in FIGS. 21 to 26, intra prediction for a reference region is exemplified as being performed using a single reference sample line adjacent to the reference region. Unlike the illustrated example, intra prediction for a reference region may also be performed using a reference sample line that is not adjacent to the reference region.

[0348] Specifically, by performing an intra-prediction on a reference region based on each of multiple reference sample lines, the cost can be calculated for each reference sample line. Accordingly, the cost can be calculated for a set combining the intra-prediction mode and the reference sample lines.

[0349] Subsequently, by selecting the combination of the intra prediction mode and reference sample line with the smallest cost, intra prediction for the current block can be performed.

[0350] To encode / decode the intra prediction mode of the current block, a list of intra prediction mode candidates for the current block can be constructed. The intra prediction mode candidate list may be an MPM list or an intra merge list.

[0351] At least one of the following can be inserted into the intra prediction mode candidate list: an intra prediction mode of a neighbor block adjacent to the current block, at least one intra prediction mode selected from a histogram, at least one intra prediction mode with a low cost of performing intra prediction in a reference area, or a predefined intra prediction mode.

[0352] Information indicating whether a candidate identical to the intra prediction mode of the current block is included in the intra prediction mode candidate list may be encoded and signaled. The information may be a 1-bit flag, and the flag may be referred to as the mode prediction flag.

[0353] If the intra prediction mode candidate list includes a candidate identical to the intra prediction mode of the current block, index information indicating the candidate identical to the intra prediction mode of the current block among the candidates included in the intra prediction mode candidate list can be encoded and signaled. The index information may be referred to as the mode prediction index.

[0354] Meanwhile, the candidate with the smallest index in the intra prediction mode candidate list (i.e., the candidate with an index of 0) may be a predefined intra prediction mode. A predefined intra prediction mode may be a planner mode or a DC mode.

[0355] The size N of the intra-prediction mode candidate list may be predefined in the encoder and decoder. Here, the size of the intra-prediction mode candidate list may represent the maximum number of candidates that the intra-prediction mode candidate list can include.

[0356] Alternatively, information indicating the size of the intra-prediction mode candidate list can be encoded and signaled through the upper header.

[0357] For the sake of convenience of explanation, the size N of the intra prediction mode candidate list is assumed to be 6 below.

[0358] If no candidate identical to the intra prediction mode of the current block is included in the intra prediction mode candidate list, the indices of the remaining intra prediction modes (i.e., MN intra prediction modes) can be reassigned, excluding the N candidates included in the intra prediction mode candidate list from among the M intra prediction modes. Here, M may represent the total number of intra prediction modes. For example, following the example of FIG. 4, the total number of intra prediction modes M may be 67. Alternatively, the total number of intra prediction modes M may be determined by including wide-angle intra prediction modes. For example, following the example of FIG. 5, the total number of intra prediction modes M may be 95.

[0359] Information indicating an index reassigned to the same intra prediction mode as the current block among the remaining intra prediction modes can be encoded and signaled. This information may be referred to as the remaining mode index.

[0360] That is, when the mode prediction flag is 1, a mode prediction index indicating one of the candidates included in the intra prediction mode candidate list can be encoded and signaled. On the other hand, when the mode prediction flag is 0, a residual mode index indicating one of the residual intra prediction modes can be encoded and signaled.

[0361] Before encoding / decoding the mode prediction index, information indicating whether the intra prediction mode of the current block is the same as the default mode may be encoded / decoded. The information may be a 1-bit flag, and the flag may be referred to as the default mode flag. The default mode flag may be encoded / decoded when the mode prediction flag is 1.

[0362] The default mode may be predefined in the encoder and decoder. For example, the candidate with the smallest index in the intra-prediction mode candidate list or a predefined intra-prediction mode may be set as the default mode. For example, the default mode may be planner mode.

[0363] If the intra prediction mode of the current block is not the default mode, the intra prediction mode of the current block can be induced through the mode prediction index. Meanwhile, if a candidate with index 0 is set to the default mode, the indices of the remaining candidates, excluding the candidate with index 0, can be reallocated. The mode prediction index can indicate the reallocated index of the candidate identical to the intra prediction mode of the current block.

[0364] Only a predefined number of M intra prediction modes may be available for the current block. For example, only Q of the M intra prediction modes may be available for the current block. In this case, the residual mode index may indicate the reallocated index of an intra prediction mode that is identical to the intra prediction mode of the current block among the (QN) intra prediction modes. For convenience of explanation, it is assumed that M is 95 and Q is 67.

[0365] If the current block is a square block, it can be determined that intra prediction modes from 0 to 66 are available for the current block.

[0366] On the other hand, if the current block is a non-square block, the number of directional modes facing the longer side of the current block's width and height is increased, and the number of directional modes facing the shorter side is decreased to determine 67 intra prediction modes available to the current block.

[0367] For example, if the current block is a non-square block where the width is greater than the height, q wide-angle intra prediction modes can be set to be available for the current block, starting with mode 67. Meanwhile, an equal number of opposite-direction intra prediction modes as the number of available wide-angle intra prediction modes can be set to be unavailable.

[0368] For example, when q is 4, wide-angle intra prediction modes 67 through 70 can be set to be available for the current block. On the other hand, directional intra prediction modes 2 through 5, which are opposite to wide-angle intra prediction modes 67 through 70, can be set to be unavailable for the current block. That is, non-directional intra prediction modes 0 through 1 and directional intra prediction modes 6 through 70 can be set to be available for the current block.

[0369] On the other hand, if the current block is a non-square block where the height is greater than the width, q wide-angle intra prediction modes, starting with mode -1, can be set to be available for the current block. Meanwhile, an equal number of opposite-direction intra prediction modes as the number of available wide-angle intra prediction modes can be set to be unavailable.

[0370] For example, when q is 4, wide-angle intra prediction modes -1 to -4 may be set to be available for the current block. On the other hand, directional intra prediction modes 63 to 66, which are opposite to the wide-angle intra prediction modes -1 to -4, may be set to be unavailable for the current block. That is, non-directional intra prediction modes 0 to 1 and directional intra prediction modes -1 to -4 and 2 to 62 may be set to be available for the current block.

[0371] The number of available wide-angle intra prediction modes (i.e., q) can be adaptively determined based on the ratio of the width and height of the current block. For example, the number of available intra prediction modes can be determined according to the following Table 4.

[0372] Width / Height q16 or 1 / 16148 or 1 / 8124 or 1 / 4102 or 1 / 2810

[0373] Meanwhile, for the convenience of encoding / decoding, even if wide-angle intra prediction modes are available, the index of the candidates inserted into the intra prediction mode candidate list may be limited to a range of 0 to 66.

[0374] Specifically, the index of the wide-angle intra prediction mode can be set to the index of the opposite direction's unavailable intra prediction mode. For example, if wide-angle intra prediction modes -1 to -4 are available for the current block, directional intra prediction modes 63 to 66 are unavailable. In this case, the index of the wide-angle intra prediction modes -1 to -4 can be updated by adding Q (i.e., 67), the total number of available intra prediction modes. Accordingly, the index of the wide-angle intra prediction modes -1 to -4 can be updated to 66 to 63.

[0375] If the index of the derived intra-prediction mode corresponds to an unavailable directional intra-prediction mode, the decoder can derive a wide-angle intra-prediction mode by differing the total number of available intra-prediction modes Q to the index of the intra-prediction mode. For example, if the index of the derived intra-prediction mode is 65, the decoder can derive a -2 wide-angle intra-prediction mode by differing the total number of available intra-prediction modes Q to the index.

[0376] Alternatively, if wide-angle intra prediction modes 67 through 70 are available for the current block, directional intra prediction modes 2 through 5 are not available. In this case, the indices of wide-angle intra prediction modes 67 through 70 can be updated by differing by the total number of available directional intra prediction modes (Q-2) (i.e., 65). Accordingly, the indices of wide-angle intra prediction modes 67 through 70 can be updated to 2 through 5.

[0377] If the index of the derived intra prediction mode corresponds to an unavailable directional intra prediction mode, the decoder can derive a wide-angle intra prediction mode by adding the total number of available directional intra prediction modes (Q-2) to the index of the intra prediction mode. For example, if the index of the derived intra prediction mode is 2, the decoder can derive a wide-angle intra prediction mode 67 by adding the total number of available directional intra prediction modes (Q-2) to the index.

[0378] Depending on the intra prediction mode of neighboring blocks, the number of wide-angle intra prediction modes available to the current block may also be determined.

[0379] FIG. 27 is a diagram illustrating an example in which the number of wide-angle intra prediction modes available to the current block is determined by referring to neighboring blocks adjacent to the current block.

[0380] In the example illustrated in FIG. 27, A represents the location of the bottom-left sample of the current block, and B represents the location of the top-right sample of the current block. L1 represents the location of the bottom-left adjacent sample located outside the current block and adjacent to the left of location A, and U1 represents the location of the top-right adjacent sample located outside the current block and adjacent to location B.

[0381] Searching can be performed from the upper-right adjacent sample location U1 of the current block to the upper-left adjacent sample location UL. At this time, the first discovered intra prediction mode can be set as the first neighbor intra prediction mode.

[0382] In addition, a search can be performed from the lower-left adjacent sample location L1 to the upper-left adjacent sample location UL. At this time, the first discovered intra prediction mode can be set to the second neighbor intra prediction mode.

[0383] Alternatively, a first neighbor intra prediction mode may be derived from a histogram obtained from the upper reference region of the current block, and a second neighbor intra prediction mode may be derived from a histogram obtained from the left reference region of the current block. That is, the first neighbor intra prediction mode may be the intra prediction mode with the largest amplitude value within the histogram obtained from the upper reference region, and the second neighbor intra prediction mode may be the intra prediction mode with the largest amplitude value within the histogram obtained from the left reference region.

[0384] Based on the first neighbor intra prediction mode and the second neighbor intra prediction mode, the number of wide-angle intra prediction modes available to the current block can be determined. For example, if the first neighbor intra prediction mode and the second neighbor intra prediction mode are greater than a threshold value, q1 wide-angle intra prediction modes among wide-angle intra prediction modes from 67 to 80 may be set as available for the current block. In addition, q1 directional intra prediction modes among directional intra prediction modes from mode 2 to 18 may be set as unavailable for the current block.

[0385] On the other hand, if one of the first neighbor intra prediction mode and the second neighbor intra prediction mode is greater than the threshold value and the other is less than the threshold value, wide angle intra prediction modes may not be available for the current block. Alternatively, if one of the first neighbor intra prediction mode and the second neighbor intra prediction mode is greater than the threshold value and the other is less than the threshold value, the number of available wide angle intra prediction modes may be determined based on the ratio between the width and height of the current block, according to the example in Table 4.

[0386] Alternatively, if the first neighbor intra prediction mode and the second neighbor intra prediction mode are less than the threshold value, q1 wide-angle intra prediction modes among -1 to -14 wide-angle intra prediction modes may be set to be available for the current block. Along with this, q1 directional intra prediction modes among modes 53 to 66 directional intra prediction modes may be set to be unavailable for the current block.

[0387] The threshold value may be predefined in the encoder and decoder. For example, the threshold value may be 34.

[0388] Alternatively, a threshold value for determining the number of available wide-angle intra prediction modes among the wide-angle intra prediction modes in the upper-right direction (i.e., wide-angle intra prediction modes from 67 to 80) and a threshold value for determining the number of available wide-angle intra prediction modes among the wide-angle intra prediction modes in the lower-left direction (i.e., wide-angle intra prediction modes from -1 to -14) may be set differently.

[0389] Meanwhile, the greater the difference between the first neighbor intra prediction mode and the second neighbor intra prediction mode, the greater the number of wide-angle intra prediction modes available to the current block. For example, the number of wide-angle intra prediction modes q1 available to the current block can be determined based on the difference between the first neighbor intra prediction mode and the second neighbor intra prediction mode, as shown in the example in Table 5 below.

[0390] | 1st Peripheral Mode - 2nd Peripheral Mode | q132 or more 1416 or more less than 32 128 or more less than 16 108 or less 8

[0391] Contrary to the example in Table 5, the smaller the difference between the first neighbor intra prediction mode and the second neighbor intra prediction mode, the greater the number of wide-angle intra prediction modes available to the current block. For example, the number of wide-angle intra prediction modes q1 available to the current block can be determined based on the difference between the first neighbor intra prediction mode and the second neighbor intra prediction mode, as shown in the example in Table 6 below.

[0392] | 1st Peripheral Mode - 2nd Peripheral Mode | q132 or more 816 or more less than 32 108 or more less than 16 128 or less 14

[0393] The number of wide-angle intra-prediction modes available to the current block may be determined by considering the range to which the first neighbor intra-prediction mode and the second neighbor intra-prediction mode belong. Table 7 shows an example in which the number of wide-angle intra-prediction modes q1 available to the current block is determined when both the first neighbor intra-prediction mode and the second neighbor intra-prediction mode are greater than the threshold value, and Table 8 shows an example in which the number of wide-angle intra-prediction modes q1 available to the current block is determined when both the first neighbor intra-prediction mode and the second neighbor intra-prediction mode are smaller than the threshold value.

[0394] First Neighbor Intra Prediction Mode Second Neighbor Intra Prediction Mode q134~5034~50834~5051 or more 1051 or more 34~501251 or more 51 or more 14

[0395] First neighbor intra-prediction mode Second neighbor intra-prediction mode q118~3418~34818~3417 or less 1017 or less 18~341217 or less 17 or less 14

[0396] Conversely to Table 7 or Table 8, the number of wide-angle intra prediction modes q1 available to the current block may also be determined. For example, the number of wide-angle intra prediction modes q1 available to the current block may also be determined according to the following Table 9 or Table 10.

[0397] First Neighbor Intra Prediction Mode Second Neighbor Intra Prediction Mode q134~5034~501434~5051 or more1251 or more34~501051 or more51 or more8

[0398] First Neighbor Intra Prediction Mode Second Neighbor Intra Prediction Mode q118~3418~341418~3417 or less 1217 or less 18~341017 or less 17 or less 8

[0399] Unlike in Tables 7 through 10, a mapping relationship between the intervals to which neighboring intra prediction modes belong and the number of available intra prediction modes may be established. Alternatively, the number of wide-angle intra prediction modes available to the current block may be determined by using a table that defines more or fewer mapping relationships than in Tables 7 through 10.

[0400] Alternatively, the boundaries of the mapping interval may be set differently depending on the threshold value.

[0401] In the example described above, it was assumed that the total number of intra prediction modes available to the current block remains fixed (i.e., Q) by setting the directional intra prediction modes to be unavailable by the number of available wide-angle intra prediction modes.

[0402] As another example, the total number of intra prediction modes available to the current block may be reduced to decrease the amount of bits encoded / decoded. Specifically, based on a first neighbor intra prediction mode and a second neighbor intra prediction mode, one group of directional intra prediction modes of a first group or a second group may be set to be available, and the other may be set to be unavailable. Here, the directional intra prediction modes of the first group may include left-direction directional intra prediction modes where the index is less than or equal to a reference value, and the directional intra prediction modes of the second group may include up-direction directional intra prediction modes where the index is greater than or equal to the reference value.

[0403] The reference value may be predefined in the encoder and decoder. For example, the reference value may be the index of the upper-left diagonal intra-prediction mode (i.e., 34). Alternatively, the reference value for determining the left-direction intra-prediction modes and the reference value for determining the upper-direction intra-prediction modes may be different.

[0404] For the sake of convenience of explanation, the reference value below is assumed to be 34.

[0405] If both the first neighbor intra prediction mode and the second neighbor intra prediction mode are greater than the threshold value, the directional intra prediction modes of the first group (i.e., left-direction directional intra prediction modes) are set to be unavailable for the current block, whereas the directional intra prediction modes of the second group (i.e., up-direction directional intra prediction modes) can be set to be available for the current block.

[0406] Instead of setting all directional intra-prediction modes of the second group as unavailable, only the remaining directional intra-prediction modes of the second group, excluding the intra-prediction mode indicating an integer position reference sample, may be set as unavailable. For example, even if both the first neighbor intra-prediction mode and the second neighbor intra-prediction mode are greater than the threshold value, at least one of the left-direction directional intra-prediction modes—specifically the lower-left diagonal intra-prediction mode (i.e., Intra-prediction Mode 2), the horizontal intra-prediction mode (i.e., Intra-prediction Mode 18), or the upper-left diagonal intra-prediction mode (i.e., Intra-prediction Mode 34)—may be determined to be available.

[0407] On the other hand, if both the first neighbor intra prediction mode and the second neighbor intra prediction mode are smaller than the threshold value, the directional intra prediction modes of the second group (i.e., the upward directional intra prediction modes) are set to be unavailable for the current block, whereas the directional intra prediction modes of the first group (i.e., the left directional intra prediction modes) can be set to be available for the current block.

[0408] Instead of setting all directional intra-prediction modes of the first group as unavailable, only the remaining directional intra-prediction modes of the first group, excluding the intra-prediction mode indicating an integer position reference sample, may be set as unavailable. For example, even if both the first neighbor intra-prediction mode and the second neighbor intra-prediction mode are smaller than the threshold value, at least one of the upper right diagonal intra-prediction mode (i.e., intra-prediction mode 66), the vertical intra-prediction mode (i.e., intra-prediction mode 50), or the upper left diagonal intra-prediction mode (i.e., intra-prediction mode 34) among the upper-direction directional intra-prediction modes may be determined to be available.

[0409] FIG. 28 is a diagram showing the intra prediction modes available to the current block when both the first neighbor intra prediction mode and the second neighbor intra prediction mode are greater than the threshold value.

[0410] When both the first neighbor intra prediction mode and the second neighbor intra prediction mode are greater than the threshold value, as in the example shown in FIG. 28, the non-directional intra prediction modes (i.e., planar mode and DC mode) and the upward directional intra prediction modes (i.e., intra prediction modes 34 through 80) are set to be available for the current block, whereas the left directional intra prediction modes (i.e., intra prediction modes -14 through 33) may be set to be unavailable for the current block.

[0411] Meanwhile, among the left-direction intra-prediction modes, the intra-prediction modes indicating integer position reference samples (e.g., the lower-left diagonal intra-prediction mode and the horizontal intra-prediction mode) can be determined to be available.

[0412] The availability of wide-angle intra prediction modes among the left-direction directional intra prediction modes or the top-direction directional intra prediction modes may be re-evaluated. For example, as in the example of Table 4, the number of available wide-angle intra prediction modes may be determined based on the ratio between the width and height of the current block, or the number of available wide-angle intra prediction modes may be determined based on the first neighbor intra prediction mode and the second neighbor intra prediction mode according to at least one of Tables 5 to 10.

[0413] Depending on the number of available wide-angle intra prediction modes, the number of unavailable directional intra prediction modes may also be adjusted.

[0414] For example, if the number of available upward wide-angle intra prediction modes q1 is 14, the remaining modes among the left-direction directional intra prediction modes may be determined to be unavailable, excluding three left-direction directional intra prediction modes (i.e., intra prediction modes 2, 18, and 34).

[0415] In contrast, when the number of available upward wide-angle intra prediction modes q1 is 12, the remaining modes, excluding the five left-direction directional intra prediction modes (i.e., intra prediction modes 2, 10, 18, 26, and 34), may be determined to be unavailable.

[0416] In contrast, when the number of available upward wide-angle intra prediction modes q1 is 10, the remaining modes, excluding the 9 left-direction directional intra prediction modes (i.e., intra prediction modes 2, 6, 10, 14, 18, 22, 26, 30, and 34), may be determined to be unavailable.

[0417] That is, as the number of available upward wide-angle intra prediction modes decreases, more left-direction directional intra prediction modes can be set to be available.

[0418] Likewise, depending on the number of available left-direction wide-angle intra-prediction modes, the number of unavailable upward-direction directional intra-prediction modes can be adjusted.

[0419] For example, if the number of available leftward wide-angle intra prediction modes q1 is 14, the remaining modes among the upwardward directional intra prediction modes may be determined to be unavailable, excluding three upward directional intra prediction modes (i.e., intra prediction modes 34, 50, and 66).

[0420] In contrast, when the number of available left-direction wide-angle intra-prediction modes q1 is 12, the remaining modes, excluding the 5 upward-direction directional intra-prediction modes (i.e., intra-prediction modes 34, 42, 50, 58, and 66), may be determined to be unavailable.

[0421] In contrast, when the number of available left-direction wide-angle intra-prediction modes q1 is 10, the remaining modes, excluding the 9 upward-direction directional intra-prediction modes (i.e., intra-prediction modes 34, 38, 42, 46, 50, 54, 58, 62, and 66), may be determined to be unavailable.

[0422] That is, as the number of available left-direction wide-angle intra-prediction modes decreases, more upward-direction directional intra-prediction modes can be set to be available.

[0423] After constructing a list of intra prediction mode candidates for the current block, the number of intra prediction modes available to the current block can be determined based on the candidates included in the list. For example, candidates to be inserted into the intra prediction mode candidate list can be derived from a fixed number (i.e., Q) of intra prediction modes, while the number of intra prediction modes available to the current block can be reduced to R (where R is a natural number less than or equal to Q). Here, R may be a value pre-set in the encoder and decoder. Alternatively, the value of R may be adaptively determined according to the size of the current block.

[0424] For example, the number of intra prediction modes available in the current block (i.e., R) can be determined based on the required modes among the 66 intra prediction modes. The required modes may be predefined in the encoder and decoder. For example, at least one of a non-directional intra prediction mode (i.e., at least one of DC mode or planar mode) or a predefined directional intra prediction mode (i.e., at least one of the lower-left diagonal mode (2), horizontal mode (18), upper-left diagonal mode (34), vertical mode (50), or upper-right diagonal mode (66)) may be set as the required mode.

[0425] The number of intra prediction modes available to the current block can be increased by adding and / or subtracting an offset to at least one of the mandatory modes. For example, an intra prediction mode corresponding to the value obtained by adding or subtracting an offset to the index of a mandatory mode can be determined to be available to the current block.

[0426] The offset can be set to +1, -1, +2, -2, +3, -3, or an integer whose absolute value is greater than this.

[0427] The number of intra prediction modes available to the current block can be increased by changing the absolute value of the offset in a predefined order until the number of intra prediction modes available to the current block reaches R. For example, the number of intra prediction modes available to the current block can be increased by adding / subtracting the offset for each of the horizontal mode (18) and the vertical mode (50). For example, in the first iteration, the (18+N), (18-N), (50+N), and (50-N) intra prediction modes can be set to be available to the current block.

[0428] Even if the number of intra prediction modes available to the current block has been increased, if the number of intra prediction modes available to the current block has not reached R, the absolute value of the offset can be changed to increase the number of intra prediction modes available to the current block. For example, in the second iteration, (18+M), (18-M), (50+M), and (50-M) intra prediction modes may be additionally set to be available to the current block.

[0429] The number of intra prediction mode candidates available to the current block can be increased by repeatedly adding or subtracting offsets for all candidates included in the intra prediction mode candidate list until the number of intra prediction modes available to the current block reaches R.

[0430] Here, R may be predefined in the encoder and decoder. Alternatively, information indicating R may be signaled through the upper header.

[0431] Meanwhile, the number of intra prediction modes available to the current block can be reduced (e.g., from Q to R) only when a predefined condition is satisfied. The predefined condition may be related to at least one of the size, shape, color component, or color format of the current block.

[0432] For example, if the number of samples included in the current block is within a predefined range, the number of intra prediction modes available to the current block can be reduced. Here, the predefined range may be at least one of 512 or less, 1024 or less, 512 or more, 1024 or more, or 512 or more and 1024 or less.

[0433] One of multiple reference sample lines can be selected, and the reference samples belonging to the selected reference sample line can be used to perform intra prediction of the current block.

[0434] FIG. 29 is a drawing illustrating multiple reference sample lines.

[0435] In FIG. 29, four reference sample lines are exemplified as being defined as selectable candidates. More or fewer reference sample lines than shown in FIG. 29 may be defined as selectable candidates.

[0436] Each of the reference sample lines may be assigned a unique index. For example, the first to fourth reference sample lines shown in FIG. 29 may be assigned indices from 0 to 3.

[0437] One of multiple reference sample line candidates can be selected, and based on the reference samples belonging to the selected reference sample line, an intra prediction of the current block can be performed.

[0438] Figure 30 shows an example in which intra prediction is performed by selecting one of the reference sample lines.

[0439] In the example illustrated in FIG. 30, the third reference sample line (i.e., the reference sample line with index 2) among the four reference sample lines is selected. When the intra prediction mode of the current block is horizontal, the left reference samples belonging to the third reference sample line can be copied horizontally to derive the prediction sample of the current block.

[0440] In the encoder, information indicating the index of a selected reference sample line can be encoded and signaled. This information may be referred to as the reference sample line index.

[0441] Depending on the selected reference sample line, the number of available intra prediction modes may be adaptively determined. For example, if a reference sample line adjacent to the current block (i.e., a first reference sample line (i.e., a reference sample line with index 0)) is selected, Q intra prediction modes may be available for the current block. That is, the intra prediction of the current block may be performed using at least one of the Q intra prediction modes. Here, Q may be 67.

[0442] On the other hand, if a reference sample line that is not adjacent to the current block (i.e., one of the second to fourth reference sample lines) is selected, R intra prediction modes may be available for the current block. Accordingly, intra prediction of the current block may be performed using at least one of the R intra prediction modes. Here, R may be a natural number smaller than Q.

[0443] Alternatively, if a reference sample line that is not adjacent to the current block is selected, only the candidates included in the intra prediction mode candidate list may be available for the current block. That is, if a reference sample line that is not adjacent to the current block is selected, the intra prediction mode of the current block must be derived from the intra prediction mode candidate list.

[0444] Accordingly, when a reference sample line that is not adjacent to the current block is selected, the encoding / decoding of the mode prediction flag can be omitted and the value of the mode prediction flag can be assumed to be 1 to induce the intra prediction mode of the current block. Since the value of the mode prediction flag is assumed to be 1, a mode prediction index indicating a candidate identical to the intra prediction mode of the current block can be encoded / decoded.

[0445] In summary, when the reference sample line index is 0, the mode prediction flag may be explicitly encoded / decoded. Depending on the value of the mode prediction flag, at least one of the mode prediction index, the default mode flag, or the residual mode index may be additionally encoded / decoded.

[0446] On the other hand, if the reference sample line index is not 0, the encoding / decoding of the mode prediction flag is omitted and its value may be considered 1. As the value of the mode prediction flag is considered 1, the mode prediction index may be encoded / decoded. Accordingly, the intra prediction mode of the current block may be set to be the same as the candidate indicated by the mode prediction index in the intra prediction mode candidate list.

[0447] Without explicitly encoding / decoding the reference sample line index, the decoder may implicitly derive the reference sample line index. Specifically, when a reference sample line that is not adjacent to the current block (e.g., one of the reference sample lines with indices 1 to 3) is selected, the decoder may implicitly derive the index of the reference sample line.

[0448] To this end, information indicating whether the reference sample line of the current block is index 0 may be encoded and signaled. That is, instead of a reference sample line index indicating the index of a selected reference sample line among the reference sample lines, a 1-bit flag indicating whether the reference sample line of the current block is an adjacent reference sample line may be encoded and signaled. The said flag may be referred to as the reference sample line flag.

[0449] When an adjacent reference sample line is not selected, the reference sample line of the current block can be determined by adding a process of calculating amplitude values ​​per reference sample line to the method of implicitly deriving the intra prediction mode of the current block.

[0450] FIGS. 31 to 33 illustrate an example of implicitly restoring a reference sample line index on the decoder side.

[0451] The number of reference sample lines N may be predefined in the encoder and decoder. Alternatively, information indicating the number of reference sample lines N may be encoded and signaled through the upper header.

[0452] For the sake of convenience of explanation, it is assumed that the number of reference sample lines is four or more (e.g., four or five).

[0453] If the reference sample line of the current block is not an adjacent reference sample line, one of the three non-adjacent reference sample lines must be selected. Accordingly, an amplitude value for each intra-prediction mode can be calculated for each of the non-adjacent reference sample lines.

[0454] Specifically, by applying a filter using reference samples belonging to the reference sample line to be calculated for amplitude as the center position, the horizontal and vertical slopes for the reference samples belonging to the reference sample line are obtained, and thereby, the amplitude of the reference samples can be derived.

[0455] FIG. 31 illustrates the locations of reference samples belonging to a second reference sample line where amplitude values ​​are obtained, FIG. 32 illustrates the locations of reference samples belonging to a third reference sample line where amplitude values ​​are obtained, and FIG. 33 illustrates the locations of reference samples belonging to a fourth reference sample line where amplitude values ​​are obtained.

[0456] For each of the reference sample lines, an amplitude value for each intra-prediction mode can be accumulated to generate a histogram. For example, according to the following mathematical formula 8, a histogram with accumulated amplitude values ​​can be generated for each of the reference sample lines.

[0457]

[0458] In mathematical formula 8, ref_idx represents the index of the reference sample line. That is, histogram[0] represents the histogram for the second reference sample line, histogram[1] represents the histogram for the third reference sample line, and histogram[2] represents the histogram for the fourth reference sample line.

[0459] From the histogram of each reference sample line, the sum of amplitude values ​​for each intra-prediction mode can be calculated. For example, if there are 67 intra-prediction modes, the sum of amplitude values ​​for each reference sample line can be derived according to the following mathematical formula 9.

[0460]

[0461] In Equation 9, histogram[ ref_idx - 1 ][n] represents the amplitude value of the intra prediction mode at index n on the histogram of the reference sample line at index ref_idx. That is, the sum of the amplitude values ​​of the reference sample line can be derived by summing the amplitude values ​​of the intra prediction modes on the corresponding histogram.

[0462] The reference sample line of the current block can be determined by comparing the sum of the amplitude values ​​of each of the reference sample lines. Specifically, the reference sample line with the largest sum of amplitude values ​​can be determined as the reference sample line of the current block.

[0463] If the sum of the amplitude values ​​of the reference sample lines is the same, the reference sample line with the lower index can be selected.

[0464] Alternatively, if the sum of the amplitude values ​​of the reference sample lines is the same, the reference sample line with the larger maximum amplitude value among the amplitude values ​​by intra-prediction mode within the histogram can be selected.

[0465] From the histogram of a determined reference sample line, an intra prediction mode of the current block can be derived. Specifically, at least one intra prediction mode can be selected in order of increasing amplitude value within the histogram of the determined reference sample line.

[0466] Instead of explicitly encoding / decoding the reference sample line flag, it is possible to determine whether to select an adjacent reference sample line by comparing the maximum, minimum, or average value of the sum of amplitude values ​​of non-adjacent reference sample lines with a threshold value.

[0467] For example, if the maximum value among the sum of amplitude values ​​of non-adjacent reference sample lines is smaller than the threshold value, an adjacent reference sample line (i.e., a reference sample line with an index of 0) can be selected to perform intra prediction. On the other hand, if the maximum value among the sum of amplitude values ​​of non-adjacent reference sample lines is equal to or smaller than the threshold value, an adjacent reference sample line with the largest sum of amplitude values ​​(i.e., one of the reference sample lines with indices of 1 to 3) can be selected to perform intra prediction.

[0468] Alternatively, histograms can be derived for adjacent reference sample lines as well as non-adjacent reference sample lines, and the reference sample line with the largest sum of amplitude values ​​can be selected.

[0469] At this time, the filter for deriving the horizontal and vertical slopes of reference samples belonging to an adjacent reference sample line and the filter for deriving the horizontal and vertical slopes of reference samples belonging to a non-adjacent reference sample line may be different. For example, the horizontal and vertical slopes of reference samples belonging to an adjacent reference sample line may be derived using a 1D filter (e.g., a filter of size (1xN) or (Nx1)).

[0470] Alternatively, reference sample lines may be reordered based on the sum of amplitude values, and the index of a selected reference sample line among the reordered reference sample lines may be explicitly encoded and signaled. That is, the reference sample line index may indicate the index of one of the reordered reference sample lines.

[0471] Reordering may involve reassigning the indices of reference sample lines in descending order of the sum of amplitude values. Accordingly, the larger the sum of amplitude values, the smaller the index reassigned to the reference sample line.

[0472] Meanwhile, reordering may be performed only on non-adjacent reference sample lines. In this case, the index assigned to the adjacent reference sample line (i.e., index 0) before and after reordering may remain the same.

[0473] Alternatively, for all reference sample lines, reordering can be performed.

[0474] As another example, a list can be generated using a predetermined number of combinations of intra-prediction modes and reference sample lines. For instance, M combinations of intra-prediction modes and reference sample lines can be selected in order of decreasing amplitude value, and the M combinations can be inserted into the list.

[0475] Subsequently, one of the candidates included in the list can be selected, and an intra prediction of the current block can be performed using a combination of the intra prediction mode and reference sample line indicated by the selected candidate. To this end, an index indicating one of the candidates included in the list can be encoded and signaled.

[0476] Depending on the value of the reference sample line flag, the encoding or decoding of the index may be determined. For example, if the reference sample line flag indicates that an adjacent reference sample line is used, the index may not be encoded or decoded. In this case, intra prediction of the current block may be performed using the adjacent reference sample line.

[0477] On the other hand, if the reference sample line flag indicates that an adjacent reference sample line has not been used, the index may be encoded / decoded. In this case, the intra prediction of the current block may be performed using the combination of the intra prediction mode and reference sample line indicated by the index.

[0478] The number of candidates M that the list can include may be predefined in the encoder and decoder. Alternatively, information indicating the number of candidates M that the list can include may be encoded and signaled through the upper header.

[0479] M being 1 indicates that there is only one candidate in the list. In this case, the encoding / decoding of the index indicating the index of the selected candidate in the list may be omitted.

[0480] Intra-prediction can also be performed at the sub-block level.

[0481] Figure 34 shows an example where intra-prediction is performed on a sub-block basis.

[0482] In the example illustrated in FIG. 34, it was assumed that the current block is divided into four sub-blocks (A to D), and that each of the sub-blocks has the same size. Specifically, FIG. 34 (a) shows an example in which the current block is divided horizontally to create four sub-blocks, and FIG. 34 (b) shows an example in which the current block is divided vertically to create four sub-blocks.

[0483] If the current block is not divided, the prediction unit may be the current block. On the other hand, if the current block is divided into multiple sub-blocks, the prediction unit may be a sub-block.

[0484] Sub-blocks can be encoded / decoded sequentially. In this case, the encoding / decoding order between sub-blocks may follow the raster scan order. For example, intra-prediction and encoding / decoding for the sub-blocks may proceed in the order from sub-block A to sub-block D.

[0485] That is, after the encoding / decoding of the current sub-block is completed, the encoding / decoding of the next sub-block can be performed. Accordingly, during the encoding / decoding of the current sub-block, the restored samples of the previous sub-block can be used as reference samples.

[0486] For example, in the example illustrated in FIG. 34 (a) and (b), when performing intra prediction on sub-block B, the restored samples present in sub-block A can be used as reference samples.

[0487] That is, intra prediction for the remaining subblocks, excluding the first subblock (i.e., subblock A), can be performed using the reconstructed samples of the previous subblock.

[0488] The number of sub-blocks created by splitting the current block can be adaptively determined based on the size of the current block. For example, if the number of samples contained in the current block (i.e., the product of the width and height) is equal to or less than the threshold value T0, the number of sub-blocks may be 2. On the other hand, if the number of samples contained in the current block is greater than the threshold value T0 but equal to or less than the threshold value T1, the number of sub-blocks may be 4. If the number of samples contained in the current block is greater than the threshold value T1, the number of sub-blocks may be 8.

[0489] The threshold values ​​T0 and T1 can each be 32 or 1024.

[0490] Unlike the example described, the number of sub-blocks may be odd (e.g., 3, 5, or 7).

[0491] Partition information of the current block can be encoded and signaled. The partition information may include at least one of information indicating whether the current block is divided into sub-blocks and information indicating the direction of partition of the current block.

[0492] Information indicating whether the current block is divided into sub-blocks may be a 1-bit flag. Only when the flag indicates that the current block is divided into sub-blocks, information indicating the direction of division of the current block may be encoded and signaled.

[0493] Information indicating the division direction of the current block may also be a 1-bit flag. The flag may indicate whether the division direction of the current block is horizontal or vertical.

[0494] Alternatively, the partition information of the current block may be an index indicating whether the current block is partitioned and the direction of partitioning the current block. For example, an index of 0 indicates that the current block is not partitioned into sub-blocks. On the other hand, an index greater than 0 indicates that the current block is partitioned into sub-blocks. In this case, an index of 1 indicates that the direction of partitioning the current block is horizontal (or vertical), and an index of 2 indicates that the direction of partitioning the current block is vertical (or horizontal).

[0495] The size of the sub-block must be within the acceptable range of the transformation block size. For example, the width of the sub-block must be within the range of the minimum and maximum values ​​of the transformation block width, and the height of the sub-block must be within the range of the minimum and maximum values ​​of the transformation block height.

[0496] The intra prediction modes of sub-blocks can be set identically. That is, once an intra prediction mode is determined for the current block, the sub-blocks can share the intra prediction modes of the current block. Accordingly, information related to the intra prediction mode can be encoded and signaled at the block level (i.e., at the coding block level).

[0497] As another example, an intra-prediction mode can be derived individually for each sub-block. Specifically, after generating a histogram for the sub-block, the intra-prediction mode of the sub-block can be derived from the histogram.

[0498] The histogram of the current sub-block can be derived based on the restored samples of the previous sub-block that were encoded / decoded prior to the current sub-block. If the current sub-block is the first sub-block, the histogram of the current sub-block can be derived based on the restored samples belonging to the reference region adjacent to the current block.

[0499] Figure 35 illustrates the range of restored samples used to derive histograms by sub-block.

[0500] The histogram of sub-block A is generated based on the reconstructed samples included in reference region A', and the histogram of sub-block B can be generated based on the reconstructed samples included in reference region B'. The histogram of sub-block C is generated based on the reconstructed samples included in reference region C', and the histogram of sub-block D can be generated based on the reconstructed samples included in reference region D'.

[0501] By applying a filter to the reconstructed samples, the horizontal and vertical gradients of each of the reconstructed samples can be derived. Based on the horizontal and vertical gradients of the reconstructed samples, an intra-prediction mode can be determined, and the amplitude values ​​derived based on the horizontal and vertical gradients can be accumulated in a buffer assigned to the corresponding intra-prediction mode on the histogram.

[0502] The intra prediction mode with the largest amplitude value on the histogram can be set as the intra prediction mode of the sub-block.

[0503] In this case, the intra prediction mode for each sub-block can be determined without encoding / decoding information for determining the intra prediction mode.

[0504] As another example, information indicating whether to set the intra-prediction mode of a sub-block to be the same as the intra-prediction mode derived for the current block may be encoded and signaled. Based on the information, it may be determined whether to set the intra-prediction mode of a sub-block to be the same as the intra-prediction mode of the current block or to derive the intra-prediction mode of a sub-block from the histogram of the sub-block. The information may be a 1-bit flag and may be encoded and signaled for each of the sub-blocks.

[0505] Here, the intra-prediction mode of the current block may be derived based on an MPM list or histogram.

[0506] Alternatively, based on the result of comparing the largest amplitude value on the histogram of the sub-block with the threshold value, it can be determined whether to set the intra prediction mode of the sub-block to be the same as the intra prediction mode of the current block.

[0507] For example, if the largest amplitude value on the histogram is smaller than the threshold value, the intra prediction mode of the sub-block can be set to be the same as the intra prediction mode of the current block.

[0508] On the other hand, if the largest amplitude value on the histogram is equal to or greater than the threshold value, the intra prediction mode having the largest amplitude value can be set as the intra prediction mode of the sub-block.

[0509] Alternatively, for each of the remaining sub-blocks excluding the first sub-block, information indicating whether the intra prediction mode of the current sub-block is the same as the intra prediction mode of the previous sub-block can be encoded and signaled.

[0510] If the intra prediction mode of the current sub-block is different from the intra prediction mode of the previous sub-block, the intra prediction mode of the current sub-block can be derived from the histogram of the current sub-block.

[0511] In the example described above, the intra prediction mode with the largest amplitude value on the histogram is set as the intra prediction mode of the current block or sub-block.

[0512] As another example, after selecting at least one intra prediction mode from the histogram, the intra prediction mode of the current block or sub-block may be determined based on an intra prediction mode candidate list that includes the selected intra prediction mode. For example, N intra prediction modes with large amplitude values ​​may be selected from the histogram, and the selected N intra prediction modes may be added to an intra prediction mode candidate list. For example, when N is 2, the intra prediction mode with the largest amplitude value and the intra prediction mode with the second largest amplitude value on the histogram may be selected.

[0513] Subsequently, one of the intra prediction mode candidates included in the intra prediction mode candidate list can be set as the intra prediction mode of the current block or sub-block. To this end, for the current block or sub-block, index information indicating one of the intra prediction mode candidates can be encoded / decoded.

[0514] The above intra prediction mode candidate list may be an MPM list.

[0515] As another example, after generating multiple histograms, at least one intra-prediction mode may be derived from each of the multiple histograms. For example, a first histogram may be generated based on reconstructed samples belonging to the left reference region of the current block or sub-block, and a second histogram may be generated based on reconstructed samples belonging to the top reference region of the current block or sub-block.

[0516] For example, the first histogram can be generated by the following mathematical formula 10, and the second histogram can be generated by the following mathematical formula 11.

[0517]

[0518]

[0519] The intra prediction mode with the largest amplitude value from the first histogram can be selected, and the intra prediction mode with the largest amplitude value from the second histogram can be selected. Subsequently, one of the two selected intra prediction modes can be set as the intra prediction mode of the current block or sub-block. To this end, information indicating one of the two intra prediction modes can be encoded and signaled.

[0520] Alternatively, both the intra prediction mode selected from the first histogram and the intra prediction mode selected from the second histogram may be used to perform intra prediction of the current block or sub-block. For example, an intra prediction may be performed on the current block or sub-block based on the intra prediction mode selected from the first histogram to generate a first prediction block, and an intra prediction may be performed on the current block or sub-block based on the intra prediction mode selected from the second histogram to generate a second prediction block. Subsequently, the first prediction block and the second prediction block may be averaged or weighted to obtain the final prediction block of the current block or sub-block.

[0521] Depending on the shape of the current block or sub-block, the range of reconstructed samples used to derive the histogram can be adaptively set. For example, if the width of the current block is greater than its height, the histogram of the current block can be derived using only the reconstructed samples belonging to the top reference area of ​​the current block. Conversely, if the height of the current block is greater than its width, the histogram of the current block can be derived using only the reconstructed samples belonging to the left reference area of ​​the current block.

[0522] In the example described above, the current block is illustrated as being divided in a horizontal or vertical direction. As another example, the current block can be configured to be divided in a vertical direction as well as a horizontal direction.

[0523] Figure 36 shows an example in which the current block is divided in the horizontal and vertical directions.

[0524] FIG. 36 (a) shows an example where the current block is divided into 4x4 sub-blocks, and FIG. 36 (b) shows an example where the current block is divided into 8x8 sub-blocks.

[0525] Information indicating the size of a sub-block may be encoded and signaled. The information may be an index indicating one of the sub-block size candidates. For example, an index value of 0 indicates that the sub-block has a size of 4x4, and an index value of 1 indicates that the sub-block may have a size of 8x8.

[0526] As shown in the example illustrated in FIG. 36, even when the current block is divided in the vertical direction as well as the horizontal direction to generate sub-blocks, the sub-blocks can be sequentially encoded / decoded. Accordingly, the current sub-block can perform intra-prediction using the reconstructed samples within the sub-blocks surrounding the current sub-block that have completed encoding / decoding.

[0527] For example, intra prediction of sub-block a1 can be performed using the upper reconstructed samples of sub-block a1 and the reconstructed samples belonging to sub-block a0 as reference samples.

[0528] Intra-prediction for a sub-block can be performed based on a reference sample line adjacent to the sub-block. That is, when encoding / decoding is performed on a sub-block basis, a reference sample line index indicating one of multiple reference sample lines can be set to a value indicating an adjacent reference sample line (i.e., 0).

[0529] As another example, intra-prediction for a sub-block can be performed using a reference sample line that is not adjacent to the sub-block. To this end, the reference sample line index can be encoded and signaled on a block-by-block or sub-block-by-block basis.

[0530] Encoding / decoding the reference sample line index on a block-by-block basis means that the reference sample line indices of the sub-blocks are set identically. Conversely, encoding / decoding the reference sample line index on a sub-block basis means that the reference sample line index of each sub-block is determined independently.

[0531] Alternatively, information indicating whether the intra prediction of sub-blocks is performed based on adjacent reference sample lines may be encoded and signaled. If the information indicates that the intra prediction of sub-blocks is performed based on adjacent reference sample lines, the intra prediction of each sub-block may be performed using adjacent reference sample lines. That is, the reference sample line index of each sub-block may be set to 0.

[0532] On the other hand, if the above information indicates that the intra prediction of the sub-blocks does not use adjacent reference sample lines, the intra prediction of each sub-block may be performed using a pre-set reference sample line. That is, the reference sample line index of each sub-block may be set to point to a pre-set reference sample line.

[0533] The index of the pre-set reference sample line can be 1.

[0534] Alternatively, if it is determined that the intra prediction of the current block or sub-block is performed based on non-adjacent reference sample lines, for each non-adjacent reference sample line, the cumulative amplitude value per reference sample line may be calculated, and then it may be decided to use the reference sample line with the larger cumulative amplitude value for the intra prediction. In this case, the cumulative amplitude value per reference sample line may be determined without distinguishing the intra prediction mode.

[0535] Figure 37 shows an example of calculating the amplitude accumulation value for each of the non-adjacent reference sample lines.

[0536] As shown in the example illustrated in FIG. 37, a horizontal gradient Dx and a vertical gradient Dy are calculated for a reconstructed sample belonging to the second reference sample line, and then an amplitude value can be calculated based on the horizontal gradient Dx and the vertical gradient Dy. Subsequently, the accumulated amplitude values ​​of the reconstructed samples belonging to the second reference sample line can be set as the accumulated amplitude value of the second reference sample line.

[0537] Likewise, for the reconstructed samples belonging to the third reference sample line, the horizontal gradient Dx and the vertical gradient Dy are calculated, and then the amplitude value can be calculated based on the horizontal gradient Dx and the vertical gradient Dy. Subsequently, the accumulated amplitude value of the reconstructed samples belonging to the third reference sample line can be set as the accumulated amplitude value of the third reference sample line.

[0538] The one with the larger amplitude accumulation value between the second and third reference sample lines can be set as the reference sample line of the current block or sub-block.

[0539] Alternatively, for each of the second reference sample line and the third reference sample line, a histogram is generated in which amplitude values ​​by intra-prediction mode are accumulated, and then the largest amplitude value on the histogram is compared to select one of the second reference sample line and the third reference sample line.

[0540] The intra-prediction mode can also be derived from the histogram. In this case, the encoding / decoding of information to determine the intra-prediction mode can be omitted.

[0541] For example, whether the intra prediction of a subblock is performed based on an adjacent reference sample line can be encoded and signaled. If the information indicates that the adjacent reference sample line is used, information for determining the intra prediction mode of the subblock can be encoded and signaled.

[0542] On the other hand, if the above information indicates that adjacent reference sample lines are not used, a histogram can be generated for each of the non-adjacent reference sample lines. Subsequently, the intra prediction mode and non-adjacent reference sample line having the largest amplitude value are selected to perform intra prediction of the sub-block.

[0543] Alternatively, after selecting one of the non-adjacent reference sample lines, the intra prediction mode of the sub-block may be derived using the reconstructed samples belonging to the selected non-adjacent reference sample line.

[0544] Figure 38 is a diagram illustrating the process of performing inter-prediction in the encoder and decoder.

[0545] As shown in the example illustrated in FIG. 38, motion information for the current block can be obtained to perform inter-prediction (S3810). Here, the motion information may include at least one of a motion vector, a reference picture index, or a weight applied to the prediction block. For the current block, motion information for at least one of the L0 direction or the L1 direction may be obtained.

[0546] In the encoder, motion information of the current block can be derived through motion estimation, and the derived motion information can be encoded and signaled to the decoder. Meanwhile, the encoding / decoding of motion information may be based on a motion information merging mode, a motion vector prediction mode, a template-based motion estimation method, or a two-way matching method, which will be described later.

[0547] In the decoder, movement information of the current block can be derived based on the information transmitted from the encoder.

[0548] Alternatively, motion information of the current block can be derived in the decoder in the same way as in the encoder. This method can be referred to as decoder-side motion estimation.

[0549] When motion information of the current block is derived, a prediction block for the current block can be obtained based on the derived motion information (S3820). For example, a reference block spaced apart by a motion vector from the position of the current block in the reference picture can be set as the prediction block of the current block.

[0550] Below, the process of performing inter-prediction will be explained in more detail.

[0551] The motion information of the current block can be generated through motion estimation.

[0552] Figure 39 shows an example where motion estimation is performed.

[0553] In Fig. 39, it was assumed that the Picture Order Count (POC) of the current picture is T, and the POC of the reference picture is (T-1).

[0554] A search range for motion estimation can be set from the same location as the reference point of the current block within the reference picture. Here, the reference point may be the location of the top-left sample of the current block.

[0555] For example, in FIG. 39, a rectangle of sizes (w0+w1) and (h0+h1) centered on a reference point is exemplified as being set as a search range. In the above example, w0, w1, h0, and h1 may have mutually identical values. Alternatively, at least one of w0, w1, h0, and h1 may be set to have a different value. Or, the sizes of w0, w1, h0, and h1 may be determined so as not to exceed the Coding Tree Unit (CTU) boundary, slice boundary, tile boundary, or picture boundary.

[0556] Within the search range, reference blocks of the same size as the current block can be set, and the cost of each reference block relative to the current block can be measured. The cost can be calculated using the similarity between the two blocks.

[0557] For example, the cost can be calculated based on the sum of the absolute differences between the original samples in the current block and the original samples (or restored samples) in the reference block. The smaller the sum of the absolute values, the lower the cost can be.

[0558] Afterward, the cost of each of the reference blocks is compared, and the reference block with the optimal cost can be set as the prediction block of the current block.

[0559] In addition, the distance between the current block and the reference block can be set as a motion vector. Specifically, the x-coordinate difference and the y-coordinate difference between the current block and the reference block can be set as a motion vector.

[0560] Furthermore, the index of the picture containing the reference block identified through motion estimation is set as the reference picture index.

[0561] In addition, the prediction direction can be set based on whether the reference picture belongs to the L0 reference picture list or the L1 reference picture list.

[0562] Additionally, motion estimation can be performed for the L0 direction and the L1 direction, respectively. If prediction is performed for both the L0 direction and the L1 direction, motion information for the L0 direction and motion information for the L1 direction can be generated, respectively.

[0563] FIGS. 40 and FIGS. 41 illustrate an example in which a predicted block of the current block is generated based on motion information generated through motion estimation.

[0564] Figure 40 shows an example of generating a prediction block with unidirectional (i.e., L0 direction) prediction, and Figure 41 shows an example of generating a prediction block with bidirectional (i.e., L0 and L1 directions) prediction.

[0565] In the case of unidirectional prediction, a prediction block of the current block is generated using a single motion information. For example, the motion information may include an L0 motion vector, an L0 reference picture index, and prediction direction information covering the L0 direction.

[0566] In the case of bidirectional prediction, a prediction block is generated using two sets of motion information. For example, a reference block for the L0 direction, specified based on motion information for the L0 direction (L0 motion information), can be set as the L0 prediction block, and a reference block for the L1 direction, specified based on motion information for the L1 direction (L1 motion information), can be generated as the L1 prediction block. Subsequently, the prediction block of the current block can be generated by performing a weighted sum of the L0 prediction block and the L1 prediction block.

[0567] In the examples illustrated in FIGS. 39 to 41, the L0 reference picture is shown as existing in the direction before the current picture (i.e., having a smaller POC value than the current picture), and the L1 reference picture is shown as existing in the direction after the current picture (i.e., having a larger POC value than the current picture).

[0568] However, unlike the illustrated example, the L0 reference picture may exist in the direction after the current picture, or the L1 reference picture may exist in the direction before the current picture. For example, both the L0 reference picture and the L1 reference picture may exist in the direction before the current picture, or both may exist in the direction after the current picture. Alternatively, bidirectional prediction may be performed using the L0 reference picture existing in the direction after the current picture and the L1 reference picture existing in the direction before the current picture.

[0569] The motion information of the block for which inter-prediction has been performed can be stored in memory. In this case, the motion information can be stored on a sample basis. Specifically, the motion information of the block to which a specific sample belongs can be stored as the motion information of that specific sample. The stored motion information can be used to derive the motion information of neighboring blocks to be encoded / decoded in the future.

[0570] In the encoder, information encoding residual samples corresponding to the difference value between the sample of the current block (i.e., the original sample) and the prediction sample, and motion information necessary to generate the prediction block, can be signaled to the decoder. In the decoder, information regarding the signaled difference value is decoded to derive a difference sample, and a prediction sample within the prediction block generated using the motion information is added to the difference sample to generate a reconstructed sample.

[0571] At this time, one of a plurality of inter-prediction modes may be selected to effectively compress motion information signaled to the decoder. Here, the plurality of inter-prediction modes may include a motion information merging mode and a motion vector prediction mode.

[0572] The motion vector prediction mode is a mode that signals by encoding the difference value between the motion vector and the motion vector prediction value. Here, the motion vector prediction value can be derived based on motion information of surrounding blocks or surrounding samples adjacent to the current block.

[0573] Figure 42 shows the location referenced to derive the motion vector prediction value.

[0574] For the sake of convenience of explanation, the current block is assumed to have a size of 4x4.

[0575] In the illustrated example, 'LB' represents a sample contained in the leftmost column and bottom row within the current block. 'RT' represents a sample contained in the rightmost column and top row within the current block. A0 through A4 represent samples adjacent to the left of the current block, and B0 through B5 represent samples adjacent to the top of the current block. For example, A1 represents a sample adjacent to the left of LB, and B1 represents a sample adjacent to the top of RT.

[0576] Col indicates the location of a sample adjacent to the bottom-right of the current block within the co-located picture. The co-located picture is a picture distinct from the current picture, and information to identify the co-located picture (e.g., co-located picture index) can be explicitly encoded and signaled in the bitstream. Alternatively, a reference picture having a predefined reference picture index can be set as the co-located picture.

[0577] The motion vector prediction value of the current block can be derived from at least one motion vector prediction candidate included in the Motion Vector Prediction List.

[0578] The number of motion vector prediction candidates that can be inserted into the motion vector prediction list (i.e., the size of the list) may be predefined in the encoder and decoder. For example, the maximum number of motion vector prediction candidates may be 2.

[0579] A motion vector stored at the location of a neighbor sample adjacent to the current block, or a scaled motion vector derived by scaling the said motion vector, can be inserted into the motion vector prediction list as a motion vector prediction candidate. At this time, the motion vector prediction candidate can be derived by scanning the neighbor samples adjacent to the current block according to a predefined order.

[0580] For example, it is possible to check whether a motion vector is stored at each location in the order from A0 to A4. Then, according to the above scan order, the first available motion vector found can be inserted into the motion vector prediction list as a motion vector prediction candidate.

[0581] As another example, checking whether a motion vector is stored at each location in the order from A0 to A4 allows the motion vector at the location with the same reference picture as the current block, found first, to be inserted into the motion vector prediction list as a motion vector prediction candidate. If no neighbor sample with the same reference picture as the current block exists, a motion vector prediction candidate can be derived based on the first available vector found. Specifically, the first available motion vector found can be scaled, and the scaled motion vector can be inserted into the motion vector prediction list as a motion vector prediction candidate. In this case, scaling can be performed based on the difference in output order between the current picture and the reference picture (i.e., POC difference) and the difference in output order between the current picture and the neighbor sample's reference picture (i.e., POC difference).

[0582] Furthermore, it is possible to check whether a motion vector is stored at each location in the order from B0 to B5. Then, according to the above scan order, the first available motion vector found can be inserted into the motion vector prediction list as a motion vector prediction candidate.

[0583] As another example, checking whether a motion vector is stored at each location in the order from B0 to B5 allows the motion vector at the location with the same reference picture as the current block, found first, to be inserted into the motion vector prediction list as a motion vector prediction candidate. If no neighbor sample with the same reference picture as the current block exists, a motion vector prediction candidate can be derived based on the first available vector found. Specifically, the first available motion vector found can be scaled, and the scaled motion vector can be inserted into the motion vector prediction list as a motion vector prediction candidate. In this case, scaling can be performed based on the difference in output order between the current picture and the reference picture (i.e., POC difference) and the difference in output order between the current picture and the neighbor sample's reference picture (i.e., POC difference).

[0584] Alternatively, the scaling process may be skipped during the above steps. In other words, the scaled motion vector may not be inserted into the motion vector prediction list.

[0585] As in the example described above, motion vector prediction candidates can be derived from samples adjacent to the left of the current block, and motion vector prediction candidates can be derived from samples adjacent to the top of the current block.

[0586] In this case, a motion vector prediction candidate derived from the left sample may be inserted into the motion vector prediction list before a motion vector prediction candidate derived from the top sample. In this case, the index assigned to the motion vector prediction candidate derived from the left sample may have a smaller value than that of the motion vector prediction candidate derived from the top sample.

[0587] Conversely, motion vector prediction candidates derived from the top sample may be inserted into the motion vector prediction list before motion vector prediction candidates derived from the left sample.

[0588] Among the motion vector prediction candidates included in the above motion vector prediction list, the motion vector prediction candidate with the highest encoding efficiency can be set as the motion vector prediction value (Motion Vector Predictor, MVP) of the current block. Additionally, index information pointing to the motion vector prediction candidate set as the motion vector prediction value of the current block among multiple motion vector prediction candidates can be encoded and signaled to the decoder. If the number of motion vector prediction candidates is two, the index information may be a 1-bit flag (e.g., an MVP flag). Furthermore, the motion vector difference value (Motion Vector Difference, MVD), which is the difference between the motion vector of the current block and the motion vector prediction value, can be encoded and signaled to the decoder.

[0589] The decoder can construct a motion vector prediction list in the same way as the encoder. Additionally, it can decode index information from the bitstream and select one of multiple motion vector prediction candidates based on the decoded index information. The selected motion vector prediction candidate can be set as the motion vector prediction value of the current block.

[0590] In addition, the motion vector difference value can be decoded from the bitstream. Subsequently, the motion vector prediction value and the motion vector difference value are combined to derive the motion vector of the current block.

[0591] When bidirectional prediction is applied to the current block, motion vector prediction lists can be generated for both the L0 and L1 directions. That is, the motion vector prediction lists can consist of motion vectors of the same direction. Accordingly, the motion vector of the current block and the motion vector prediction candidates included in the motion vector prediction lists have the same direction.

[0592] When the motion vector prediction mode is selected, the reference picture index and prediction direction information can be explicitly encoded and signaled to the decoder. For example, if multiple reference pictures exist on a reference picture list and motion estimation is performed for each of the multiple reference pictures, a reference picture index for identifying the reference picture from which the motion information of the current block was derived among the multiple reference pictures can be explicitly encoded and signaled to the decoder.

[0593] In this case, if the reference picture list contains only one reference picture, the encoding / decoding of the reference picture index may be omitted.

[0594] The prediction direction information may be an index indicating one of L0 unidirectional prediction, L1 unidirectional prediction, or bidirectional prediction. Alternatively, an L0 flag indicating whether a prediction for the L0 direction is performed and an L1 flag indicating whether a prediction for the L1 direction is performed may be encoded and signaled, respectively.

[0595] The motion information merging mode is a mode that sets the motion information of the current block to be identical to the motion information of neighboring blocks. In the motion information merging mode, motion information can be encoded or decoded using a motion information merging list.

[0596] Motion information merging candidates can be derived based on motion information from neighboring blocks or neighbor samples adjacent to the current block. For example, after defining reference locations around the current block, it is possible to check whether motion information exists at the defined reference locations. If motion information exists at the defined reference locations, the motion information at those locations can be inserted into the motion information merging list as a motion information merging candidate.

[0597] In the example of FIG. 42, the previously defined reference positions may include at least one of A0, A1, B0, B1, B5, and Col. Furthermore, motion information merging candidates can be derived in the order of A1, B1, B0, A0, B5, and Col.

[0598] The motion information of the motion information merge candidate with the optimal cost among the motion information merge candidates included in the motion information merge list can be set as the motion information of the current block. Furthermore, index information (e.g., merge index) pointing to the selected motion information merge candidate among multiple motion information merge candidates can be encoded and transmitted to a decoder.

[0599] In the decoder, a motion information merge list can be configured in the same way as in the encoder. Then, motion information merge candidates can be selected based on the merge index decoded from the bitstream. The motion information of the selected motion information merge candidate can be set as the motion information of the current block.

[0600] Unlike the motion vector prediction list, the motion information merging list consists of a single list regardless of the prediction direction. That is, the motion information merging candidates included in the motion information merging list may have only L0 motion information or L1 motion information, or they may have bidirectional motion information (i.e., L0 motion information and L1 motion information).

[0601] Movement information of the current block can also be derived using a restoration sample area around the current block. Here, the restoration sample area used to derive the movement information of the current block may be referred to as a template.

[0602] Figure 43 is a diagram illustrating a template-based motion estimation method.

[0603] In FIG. 39, it was explained that the predicted block of the current block is determined based on the cost between the current block and the reference block within the search range. According to the present embodiment, unlike FIG. 39, motion estimation for the current block can be performed based on the cost between a template adjacent to the current block (hereinafter referred to as the current template) and a reference template having the same size and shape as the current template.

[0604] For example, the cost can be calculated based on the sum of the absolute differences between the restored samples in the current template and the restored samples in the reference block. The smaller the sum of the absolute values, the lower the cost can be.

[0605] When a reference template with the optimal cost and the current template within the search range is determined, a reference block adjacent to the reference template can be set as the predicted block of the current block.

[0606] Additionally, movement information of the current block can be set based on the distance between the current block and the reference block, the index of the picture to which the reference block belongs, and whether the reference picture is included in the L0 or L1 reference picture list.

[0607] Since the template is defined by the previously restored area surrounding the current block, the decoder can perform motion estimation itself in the same manner as the encoder. Accordingly, when deriving motion information using a template, there is no need to encode and signal the motion information, except for information indicating whether the template is being used.

[0608] The current template may include at least one of an area adjacent to the top of the current block or an area adjacent to the left. In this case, the area adjacent to the top may include at least one row, and the area adjacent to the left may include at least one column.

[0609] Figure 44 shows examples of template configurations.

[0610] The current template can be configured following one of the examples shown in Fig. 44.

[0611] Alternatively, unlike the example shown in FIG. 44, the template may be configured using only the area adjacent to the left of the current block, or only the area adjacent to the top of the current block.

[0612] The size and / or shape of the current template may be predefined in the encoder and decoder.

[0613] Alternatively, multiple template candidates of different sizes and / or shapes can be defined, and index information specifying one of the multiple template candidates can be encoded and signaled to a decoder.

[0614] Alternatively, one of a plurality of template candidates may be adaptively selected based on at least one of the size, shape, or location of the current block. For example, if the current block touches the top boundary of the CTU, the current template may be configured using only the area adjacent to the left of the current block.

[0615] Motion estimation based on a template can be performed for each of the reference pictures stored in the reference picture list. Alternatively, motion estimation can be performed for only some of the reference pictures. For example, motion estimation can be performed only for the reference picture with a reference picture index of 0, or only for reference pictures with a reference picture index smaller than a threshold value, or for reference pictures with a POC difference with the current picture smaller than a threshold value.

[0616] Alternatively, after explicitly encoding and signaling the reference picture index, motion estimation can be performed only on the reference picture pointed to by the reference picture index.

[0617] Alternatively, motion estimation can be performed on a reference picture of a neighbor block corresponding to the current template. For example, if the template consists of a left neighbor area and a top neighbor area, at least one reference picture can be selected using at least one of the reference picture index of the left neighbor block or the reference picture index of the top neighbor block. Subsequently, motion estimation can be performed on the selected at least one reference picture.

[0618] Information indicating whether template-based motion estimation has been applied can be encoded and signaled to a decoder. The information may be a 1-bit flag. For example, if the flag is true (1), it indicates that template-based motion estimation is applied to the L0 and L1 directions of the current block. On the other hand, if the flag is false (0), it indicates that template-based motion estimation is not applied. In this case, motion information of the current block can be derived based on a motion information merging mode or a motion vector prediction mode.

[0619] Conversely to the above, if it is determined that the motion information merging mode and the motion vector prediction mode are not applied to the current block, then a template-based motion estimation may be applied. For example, if a first flag indicating whether the motion information merging mode is applied and a second flag indicating whether the motion vector prediction mode is applied are both 0, then a template-based motion estimation may be performed.

[0620] For each of the L0 and L1 directions, information indicating whether template-based motion estimation has been applied can be signaled. That is, whether template-based motion estimation is applied to the L0 direction and whether it is applied to the L1 direction can be determined independently of each other. Accordingly, while template-based motion estimation is applied to either the L0 or L1 direction, another mode (e.g., motion information merging mode or motion vector prediction mode) may be applied to the other.

[0621] If template-based motion estimation is applied to both the L0 and L1 directions, the prediction block of the current block can be generated based on the weighted sum operation of the L0 prediction block and the L1 prediction block. Alternatively, even if template-based motion estimation is applied to one of the L0 and L1 directions, but another mode is applied to the other, the prediction block of the current block can be generated based on the weighted sum operation of the L0 prediction block and the L1 prediction block.

[0622] Alternatively, a template-based motion estimation method may be inserted as a motion information merging candidate in the motion information merging mode or as a motion vector prediction candidate in the motion vector prediction mode. In this case, whether to apply the template-based motion estimation method may be determined based on whether the selected motion information merging candidate or the selected motion vector prediction candidate points to the template-based motion estimation method.

[0623] Based on the two-way matching method, movement information of the current block can also be generated.

[0624] Figure 45 is a diagram illustrating a motion estimation method based on a two-way matching method.

[0625] The two-way matching method can be performed only when the temporal order of the current picture (i.e., POC) exists between the temporal order of the L0 reference picture and the temporal order of the L1 reference picture.

[0626] When a two-way matching method is applied, a search range can be set for each of the L0 reference picture and the L1 reference picture. In this case, an L0 reference picture index for identifying the L0 reference picture and an L1 reference picture index for identifying the L1 reference picture can be encoded and signaled, respectively.

[0627] As another example, only the L0 reference picture index is encoded and signaled, and an L1 reference picture can be selected based on the distance between the current picture and the L0 reference picture (hereinafter referred to as the L0 POC difference). For example, among the L1 reference pictures included in the L1 reference picture list, an L1 reference picture can be selected in which the absolute value of the distance from the current picture (hereinafter referred to as the L1 POC difference) is equal to the absolute value of the distance between the current picture and the L0 reference picture. If there is no L1 reference picture having an L1 POC difference identical to the L0 POC difference, the L1 reference picture among the L1 reference pictures in which the L1 POC difference is most similar to the L0 POC difference can be selected.

[0628] At this time, among the L1 reference pictures, only L1 reference pictures that have a different temporal direction from the L0 reference picture can be used for two-way matching. For example, if the POC of the L0 reference picture is smaller than that of the current picture, one of the L1 reference pictures with a POC larger than that of the current picture can be selected.

[0629] Conversely to the above, only the L1 reference picture index is encoded and signaled, and the L0 reference picture is selected based on the distance between the current picture and the L1 reference picture.

[0630] Alternatively, a two-way matching method may be performed using the L0 reference picture closest to the current picture among the L0 reference pictures and the L1 reference picture closest to the current picture among the L1 reference pictures.

[0631] Alternatively, a two-way matching method may be performed using an L0 reference picture (e.g., index 0) assigned to a previously defined index in the L0 reference picture list and an L1 reference picture (e.g., index 0) assigned to a previously defined index in the L1 reference picture list.

[0632] Alternatively, LX (X is 0 or 1) reference picture may be selected based on an explicitly signaled reference picture index, and L|X-1| reference picture may be selected as the reference picture closest to the current picture among L|X-1| reference pictures, or as a reference picture having a predefined index within the L|X-1| reference picture list.

[0633] As another example, L0 and / or L1 reference pictures can be selected based on movement information of neighbor blocks of the current block. For example, L0 and / or L1 reference pictures to be used for bidirectional matching can be selected using the reference picture index of the left or top neighbor block of the current block.

[0634] The search range can be set within a predetermined range from the collocated blocks within the reference picture.

[0635] As another example, the search range can be set based on initial movement information. The initial movement information can be derived from the neighbor blocks of the current block. For example, the movement information of the current block's left neighbor block or top neighbor block can be set as the current block's initial movement information.

[0636] When the two-way matching method is applied, the L0 motion vector and the L1 motion vector are set in opposite directions. This indicates that the sign of the L0 motion vector and the L1 motion vector have opposite signs. Additionally, the magnitude of the LX motion vector can be proportional to the distance between the current picture and the LX reference picture (i.e., the POC difference).

[0637] Subsequently, motion estimation can be performed using the cost between a reference block (hereinafter referred to as the L0 reference block) within the search range of the L0 reference picture and a reference block (hereinafter referred to as the L1 reference block) within the search range of the L1 reference picture.

[0638] If an L0 reference block is selected with a vector (x, y) with respect to the current block, an L1 reference block can be selected at a location spaced (-Dx, -Dy) away from the current block. Here, D can be determined by the ratio of the distance between the current picture and the L0 reference picture to the distance between the L1 reference picture and the current picture.

[0639] For example, in the example illustrated in FIG. 45, the absolute value of the distance between the current picture (T) and the L0 reference picture (T-1) and the absolute value of the distance between the current picture (T) and the L1 reference picture (T+1) are mutually identical. Accordingly, in the illustrated example, the L0 motion vector (x0, y0) and the L1 motion vector (x1, y1) have the same magnitude but opposite distances. If the L1 reference picture with POC (T+2) is used, the L1 motion vector (x1, y1) will be set to (-2*x0, -2*y0).

[0640] When the L0 reference block and L1 reference block having the optimal cost are selected, the L0 reference block and L1 reference block can be set as the L0 prediction block and L1 prediction block of the current block, respectively. Subsequently, the final prediction block of the current block can be generated through a weighted sum operation of the L0 reference block and L1 reference block.

[0641] When a two-way matching method is applied, the decoder can perform motion estimation in the same way as the encoder. Accordingly, information indicating whether a two-way motion matching method is applied is explicitly encoded / decoded, while the encoding / decoding of motion information, such as motion vectors, can be omitted. As previously explained, at least one of the L0 reference picture index or the L1 reference picture index may be explicitly encoded / decoded.

[0642] As another example, information indicating whether a two-way matching method has been applied may be explicitly encoded / decoded; if the two-way matching method has been applied, the L0 motion vector or the L1 motion vector may be explicitly encoded and signaled. If the L0 motion vector is signaled, the L1 motion vector can be derived based on the POC difference between the current picture and the L0 reference picture and the POC difference between the current picture and the L1 reference picture. If the L1 motion vector is signaled, the L0 motion vector can be derived based on the POC difference between the current picture and the L0 reference picture and the POC difference between the current picture and the L1 reference picture. In this case, the encoder may explicitly encode the smaller of the L0 motion vector and the L1 motion vector.

[0643] Information indicating whether a two-way matching method is applied may be a 1-bit flag. For example, if the flag is true (e.g., 1), it may indicate that a two-way matching method is applied to the current block. If the flag is false (e.g., 0), it may indicate that a two-way matching method is not applied to the current block. In this case, a motion information merging mode or a motion vector prediction mode may be applied to the current block.

[0644] Conversely to the above, a two-way matching method may be applied only when it is determined that the motion information merging mode and the motion vector prediction mode are not applied to the current block. For example, if both the first flag indicating whether the motion information merging mode is applied and the second flag indicating whether the motion vector prediction mode is applied are 0, the two-way matching method may be applied.

[0645] Alternatively, a two-way matching method may be inserted as a motion information merging candidate in the motion information merging mode or as a motion vector prediction candidate in the motion vector prediction mode. In this case, whether to apply the two-way matching method may be determined based on whether the selected motion information merging candidate or the selected motion vector prediction candidate points to the two-way matching method.

[0646] In the two-way matching method, it was exemplified that the temporal order of the current picture must exist between the temporal order of the L0 reference picture and the temporal order of the L1 reference picture. A one-way matching method, to which the constraints of the above two-way matching method do not apply, may be applied to generate a predicted block of the current block. Specifically, in the one-way matching method, two reference pictures with a temporal order (i.e., POC) smaller than the current block or two reference pictures with a temporal order larger than the current block may be used. In this case, both of the two reference pictures may be derived from the L0 reference picture list or the L1 reference picture list. Alternatively, one of the two reference pictures may be derived from the L0 reference picture list and the other from the L1 reference picture list.

[0647] Figure 46 is a diagram illustrating a motion estimation method based on a unidirectional matching method.

[0648] A unidirectional matching method can be performed based on two reference pictures (i.e., Forward reference pictures) that have a smaller POC than the current picture, or two reference pictures (i.e., Backward reference pictures) that have a larger POC than the current picture. In FIG. 46, motion estimation based on a unidirectional matching method is exemplified by a first reference picture (T-1) and a second reference picture (T-2) that have a smaller POC than the current picture (T).

[0649] At this time, a first reference picture index for identifying the first reference picture and a second reference picture index for identifying the second reference picture can each be encoded and signaled. At this time, among the two reference pictures used in the unidirectional matching method, the reference picture with a smaller POC difference with the current picture can be set as the first reference picture. Accordingly, when the first reference picture is selected, only reference pictures among the reference pictures included in the reference picture list that have a POC difference with the current picture greater than that of the first reference picture can be set as the second reference picture. The second reference picture index can be set to point to the index of one of the reordered reference pictures after reordering the reference pictures that have the same temporal direction as the first reference picture and have a POC difference with the current picture greater than that of the first reference picture.

[0650] Conversely to the above, the reference picture with the larger POC difference with the current picture among the two reference pictures may be set as the first reference picture. In this case, the index of the second reference picture may be set to point to the index of one of the reordered reference pictures after reordering the reference pictures that have the same temporal direction as the first reference picture and have a smaller POC difference with the current picture than the first reference picture.

[0651] Alternatively, a unidirectional matching method may be performed using a reference picture assigned to a predefined index within the reference picture list and a reference picture having the same temporal direction. For example, a reference picture with an index of 0 within the reference picture list may be set as the first reference picture, and among the reference pictures with the same temporal direction as the first reference picture within the reference picture list, the reference picture with the smallest index may be selected as the second reference picture.

[0652] Both the first reference picture and the second reference picture can be selected from the L0 reference picture list or the L1 reference picture list. In FIG. 46, two L0 reference pictures are shown being used in a unidirectional matching method. Alternatively, the first reference picture may be selected from the L0 reference picture list and the second reference picture may be selected from the L1 reference picture list.

[0653] Information indicating whether the first reference picture and / or the second reference picture belongs to the L0 reference picture list or the L1 reference picture list may be additionally encoded / decoded.

[0654] Alternatively, unidirectional matching can be performed using one of the L0 reference picture list and the L1 reference picture list set as the default. Alternatively, two reference pictures can be selected from the L0 reference picture list and the L1 reference picture list that has a larger number of reference pictures.

[0655] Afterwards, a search range can be set within the first reference picture and the second reference picture.

[0656] The search range can be set within a predetermined range from the collocated blocks within the reference picture.

[0657] As another example, the search range can be set based on initial movement information. The initial movement information can be derived from the neighbor blocks of the current block. For example, the movement information of the current block's left neighbor block or top neighbor block can be set as the current block's initial movement information.

[0658] Subsequently, motion estimation can be performed using the cost between the first reference block within the search range of the first reference picture and the second reference block within the search range of the second reference picture.

[0659] At this time, under the unidirectional matching method, the magnitude of the motion vector should be set to increase in proportion to the distance between the current picture and the reference picture. Specifically, if a first reference block is selected with a vector (x, y) with respect to the current picture, the second reference block should be separated from the current block by (Dx, Dy). Here, D can be determined by the ratio of the distance between the current picture and the first reference picture to the distance between the current picture and the second reference picture.

[0660] For example, in the example of FIG. 46, the distance between the current picture and the first reference picture (i.e., POC difference) is 1, and the distance between the current picture and the second reference picture (i.e., POC difference) is 2. Accordingly, if the first motion vector for the first reference block in the first reference picture is (x0, y0), the second motion vector (x1, y1) for the second reference block in the second reference picture can be set to (2x0, 2y0).

[0661] When a first reference block and a second reference block having optimal costs are selected, the first reference block and the second reference block can be set as the first prediction block and the second prediction block of the current block, respectively. Subsequently, the final prediction block of the current block can be generated through a weighted sum operation of the first prediction block and the second prediction block.

[0662] When a unidirectional matching method is applied, the decoder can perform motion estimation in the same way as the encoder. Accordingly, information indicating whether a unidirectional motion matching method is applied is explicitly encoded / decoded, while the encoding / decoding of motion information, such as motion vectors, can be omitted. As previously explained, at least one of the first reference picture index or the second reference picture index may be explicitly encoded / decoded.

[0663] As another example, information indicating whether a unidirectional matching method has been applied may be explicitly encoded / decoded, and if a unidirectional matching method has been applied, a first motion vector or a second motion vector may be explicitly encoded and signaled. If the first motion vector is signaled, the second motion vector may be derived based on the POC difference between the current picture and the first reference picture and the POC difference between the current picture and the second reference picture. If the second motion vector is signaled, the first motion vector may be derived based on the POC difference between the current picture and the first reference picture and the POC difference between the current picture and the second reference picture. In this case, the encoder may explicitly encode the one with the smaller magnitude between the first motion vector and the second motion vector.

[0664] Information indicating whether a unidirectional matching method is applied may be a 1-bit flag. For example, if the flag is true (e.g., 1), it may indicate that a unidirectional matching method is applied to the current block. If the flag is false (e.g., 0), it may indicate that a unidirectional matching method is not applied to the current block. In this case, a motion information merging mode or a motion vector prediction mode may be applied to the current block.

[0665] Conversely to the above, a unidirectional matching method may be applied only when it is determined that the motion information merging mode and the motion vector prediction mode are not applied to the current block. For example, if both the first flag indicating whether the motion information merging mode is applied and the second flag indicating whether the motion vector prediction mode is applied are 0, a unidirectional matching method may be applied.

[0666] Alternatively, a unidirectional matching method may be inserted as a motion information merging candidate in the motion information merging mode or as a motion vector prediction candidate in the motion vector prediction mode. In this case, whether to apply the unidirectional matching method may be determined based on whether the selected motion information merging candidate or the selected motion vector prediction candidate points to the unidirectional matching method.

[0667] By adjusting the precision of the motion vector, the movement of an object between frames can also be detected. Specifically, the position of each pixel within a picture is specified as an integer. On the other hand, the movement of an object between frames may not be represented by an integer position.

[0668] Considering this, motion vectors can be explored in fractional pixel units by performing interpolation on the reference picture.

[0669] Figures 47 and 48 illustrate examples of prediction blocks being generated based on the precision of the motion vectors.

[0670] FIG. 47 shows the position of the current block in the current picture, and FIG. 48 illustrates an example in which a predicted block is acquired according to a motion vector.

[0671] Specifically, FIG. 48 (a) shows an example where the motion vector precision is in integer pixel units, and FIG. 48 (b) and (c) show examples where the motion vector precision is in 1 / 2 pixel units and 1 / 4 pixel units, respectively.

[0672] Motion vector precision can also be set in units smaller than those described. For example, motion vector precision can be set in units of 1 / 8 pixel, 1 / 16 pixel, or 1 / 32 pixel.

[0673] When the motion vector of the current block is expressed in integer units, a reference block composed of integer position samples can be set as the prediction block of the current block, as in the example illustrated in FIG. 48 (a).

[0674] On the other hand, when the motion vector of the current block is expressed in fractional units, a reference block composed of fractional position samples can be set as the prediction block of the current block, as in the examples illustrated in FIG. 48 (b) and (c). In this case, the fractional position samples within the reference block can be generated by interpolating integer position samples. The interpolation filter can have a size of 4 taps or 8 taps.

[0675] As another example, to reduce complexity, fractional position samples can be generated through linear interpolation using only integer position samples adjacent to the fractional position.

[0676] Information indicating the motion vector precision of the current block can be encoded and signaled. For example, after assigning different indices to each of multiple motion vector precision candidates, the index of the motion vector precision candidate corresponding to the motion vector precision of the current block can be encoded and signaled.

[0677] At this time, the number and / or types of available motion vector candidates may be determined based on at least one of the size of the current block, the shape of the current block, the reference picture, or the motion compensation model. Here, the motion compensation model may include at least one of a translation model, a zooming model, or a rotation model. A motion compensation model in which at least one of a zooming model or a rotation model is combined with a translation model may be referred to as an affine model.

[0678] An index indicating one of the motion vector candidates available for the current block can be encoded. Depending on the number of motion vector candidates available for the current block, the maximum number of bits required to encode the index can be determined.

[0679] By adjusting the precision of the motion vector, the motion vector can be explored more precisely, and accordingly, the prediction accuracy for the current block can be improved.

[0680] Meanwhile, motion vectors expressed as fractional positions can be scaled up to integers and encoded.

[0681] Compensation for the movement of an object may be performed based on at least one of a translation model to compensate for linear movement of the object (e.g., movement in the horizontal and / or vertical directions), a zooming model to compensate for changes in the size of the object, and a rotation model to compensate for rotational movement of the object. Here, zooming may refer to enlargement or reduction in size.

[0682] FIG. 49 shows an example in which motion compensation based on a translational model and a zooming model is performed for the current block.

[0683] For the convenience of explanation, the current block is assumed to have a size of 4x4, as shown in FIG. 47.

[0684] In FIG. 49, the variable α represents the scaling parameter. The size of the reference block can be derived by multiplying the size of the current block by the variable α.

[0685] A scaling parameter α less than 1 indicates that the reference block is smaller than the current block, and a scaling parameter α greater than 1 indicates that the reference block is larger than the current block.

[0686] Figures 49 (a) and (b) show examples where the scaling parameter α is less than 1, and Figure 49 (c) shows an example where the scaling parameter α is greater than 1.

[0687] Based on the motion vector of the current block, the top-left position of the reference block can be determined. Specifically, the top-left position of the reference block can be set to a position offset by the motion vector from the position corresponding to the top-left sample of the current block within the reference picture. Subsequently, a reference block can be set such that its width and height are each α times the width and height of the current block, respectively, according to a scaling parameter. Fractional position samples within the reference block can be generated by interpolating integer position samples.

[0688] The reference block derived by the motion vector and scaling parameter can be set as the prediction block of the current block.

[0689] Meanwhile, information regarding the size adjustment parameter α can be encoded and signaled. Specifically, a different index is assigned to each of the multiple size adjustment parameter candidates, and an index specifying the size adjustment parameter candidate applied to the current block can be encoded and signaled.

[0690] Alternatively, the size adjustment parameter of the current block may be derived based on the size adjustment parameter of a neighbor block. For example, the size adjustment parameter of a neighbor block at a predefined location can be set as the size adjustment parameter of the current block.

[0691] Alternatively, when multiple neighbor blocks are searched sequentially, the size adjustment parameter of the first available neighbor block found can be set as the size adjustment parameter of the current block.

[0692] Alternatively, a size control parameter of a neighboring block can be set as a size control parameter candidate. In this case, a list of size control parameter candidates containing multiple size control parameter candidates can be generated by sequentially searching multiple neighboring blocks. One of the multiple size control parameter candidates included in the list of multiple size control parameter candidates can be set as the size control parameter of the current block. In this case, an index indicating a candidate among the multiple size control parameter candidates that is identical to the size control parameter of the current block can be encoded and signaled.

[0693] Meanwhile, the neighbor blocks used to derive the size adjustment parameters of the current block may include at least one of the top neighbor block, left neighbor block, top-left neighbor block, top-right neighbor block, or bottom-left neighbor block.

[0694] FIG. 50 shows an example in which motion compensation based on a translational model and a rotational model is performed for the current block.

[0695] For the convenience of explanation, the current block is assumed to have a size of 4x4, as shown in FIG. 46.

[0696] First, as in the example illustrated in FIG. 50 (a), the position of a temporary block within a reference picture can be determined based on the motion vector of the current block. Specifically, a block position can be determined by taking a position spaced apart by the motion vector from a position corresponding to the top-left sample of the current block within the reference picture as the top-left sample.

[0697] Afterwards, the temporary block can be rotated as in the example shown in FIG. 50 (b). The block at the rotated position is set as a reference block, and the reference block can be set as a prediction block of the current block.

[0698] Meanwhile, a rotation matrix may be used when rotating a temporary block specified by a motion vector. That is, the predicted sample for the current block can be set to a sample at a position obtained by applying a rotation matrix to the sample position within the temporary block.

[0699] Mathematical equation 12 represents the rotation matrix.

[0700]

[0701] In the above mathematical equation 12, (pos_x, pos_y) represents the position of a sample within a temporary block. That is, (pos_x, pos_y) can be derived by adding a motion vector to the position of the target sample to be predicted within the current block.

[0702] (pos_x', pos_y') represents the position rotated from the position of the sample within the temporary block, and θ represents the rotation angle.

[0703] The sample value at position (pos_x', pos_y') within the reference picture can be set as the value of the predicted sample for the position of the sample to be predicted. If position (pos_x', pos_y') is a fractional position, the sample at that position can be generated by interpolating integer position samples.

[0704] Meanwhile, information representing the rotation angle θ can be encoded and signaled. For example, after assigning different indices to each of a plurality of rotation angle candidates, the index of the rotation angle candidate corresponding to the rotation angle of the current block can be encoded and signaled.

[0705] Alternatively, the rotation angle of the current block can be derived based on the rotation angle of a neighbor block. For example, the rotation angle of a neighbor block at a predefined position can be set as the rotation angle of the current block.

[0706] Alternatively, when multiple neighbor blocks are searched sequentially, the rotation angle of the first available neighbor block found can be set as the rotation angle of the current block.

[0707] Alternatively, the rotation angle of a neighboring block can be set as a rotation angle candidate. In this case, a rotation angle candidate list containing multiple rotation angle candidates can be generated by sequentially searching multiple neighboring blocks. One of the multiple rotation angle candidates included in the list of multiple rotation angle candidates can be set as the rotation angle of the current block. In this case, an index indicating the candidate among the multiple rotation angle candidates that is identical to the rotation angle of the current block can be encoded and signaled.

[0708] Meanwhile, the neighbor block used to induce the rotation angle of the current block may include at least one of the top neighbor block, left neighbor block, top-left neighbor block, top-right neighbor block, or bottom-left neighbor block.

[0709] Although not explicitly stated, motion compensation for the current block can also be performed by simultaneously applying translational, zooming, and rotational models.

[0710] Meanwhile, the motion vector precision for the current block or the number and / or types of motion vector precision candidates available for the current block may be determined differently depending on the motion compensation model.

[0711] For example, the number and / or types of motion vector precision candidates available for the current block may differ between the case where only a translation model is applied and the case where at least one of a zooming model or a rotation model is applied.

[0712] As a specific example, when a translation model is applied to the current block, candidates of at least 1 / 4 pixel unit may be available for the current block. On the other hand, when at least one of a zooming model or a rotation model is additionally applied along with the translation model to the current block, candidates of at least 1 / 16 pixel unit may be available for the current block.

[0713] Alternatively, if a translation model is applied to the current block, the motion vector precision of the current block may be set to 1 / 4 pixel units. On the other hand, if at least one of a zooming model or a rotation model is additionally applied to the current block along with the translation model, the motion vector precision of the current block may be set to 1 / 16 pixel units.

[0714] Meanwhile, available motion vector precision or available motion vector precision candidates for each motion compensation model may be stored in the encoder and decoder. Alternatively, information representing available motion vector precision or available motion vector precision candidates for each motion compensation model may be encoded and signaled through an upper header.

[0715] Motion compensation for an affine model, to which a zooming model and / or a rotation model are added to a translation model, can be performed using the motion vector of a control point. Here, the control point may correspond to a corner of the current block. For example, to perform motion compensation based on an affine model, at least one of the motion vector of the top-left corner, the motion vector of the top-right corner, or the motion vector of the bottom-left corner may be used.

[0716] Hereinafter, the motion vector of a control point will be referred to as the control point motion vector.

[0717] Figures 51 and 52 show an example of generating a prediction block for the current block using control point motion vectors.

[0718] For the convenience of explanation, the current block is assumed to have a size of 4x4, as shown in FIG. 46.

[0719] In FIG. 51 (a) and (b), a prediction block for the current block is exemplified by the motion vector of the first control point corresponding to the top-left corner of the current block (first control point motion vector, A) and the motion vector of the second control point corresponding to the top-right corner of the current block (second control point motion vector, B).

[0720] Beyond the illustrated examples, it is also possible to derive the predicted block of the current block by additionally utilizing the motion vector of the bottom-left corner or by using the motion vector of the bottom-left corner instead of the top-right corner.

[0721] Figure 53 shows an example of generating a prediction block for the current block using three control point motion vectors.

[0722] In FIG. 53 (a) and (b), a prediction block for the current block is exemplified by the motion vector of the first control point corresponding to the upper-left corner of the current block (first control point motion vector, A), the motion vector of the second control point corresponding to the upper-right corner of the current block (second control point motion vector, B), and the motion vector of the third control point corresponding to the lower-left corner of the current block (third control point motion vector, C).

[0723] As shown in the examples illustrated in FIGS. 51 to 53, translation, zooming, and rotational motion compensation for the current block can be performed using two or three control point motion vectors.

[0724] Information indicating the number of control point motion vectors can be encoded and signaled. The information can be signaled in blocks. For example, the information can indicate whether two control point motion vectors or three control point motion vectors are used in the current block.

[0725] Alternatively, the number of control point motion vectors can be adaptively determined based on at least one of the size or shape of the current block.

[0726] Alternatively, if the control point motion vectors of the current block are derived from neighboring blocks, the number of control point motion vectors for the current block can be set to be equal to the number of control point motion vectors of neighboring blocks.

[0727] Using control point motion vectors, sample-specific motion vectors within the current block can be derived. Equation 13 represents a formula for deriving a motion vector for each sample using two control point motion vectors.

[0728]

[0729] In the above mathematical formula 13, (mv x , mv y ) represents the motion vector at the (x, y) position within the current block. (mv Ax , mv Ay ) represents the first control point motion vector (A), and (mv Bx , mv By ) represents the second control point motion vector (B). W represents the width of the current block.

[0730] When three control point motion vectors are used, a motion vector per sample can be derived by the following mathematical formula 14.

[0731]

[0732] In the above mathematical formula 14, (mv Cx , mv Cy ) represents the third control point motion vector (C).

[0733] When motion vectors are derived for each sample, motion compensation can be performed for each sample, as in the example shown in FIG. 52. Specifically, a reference sample indicated by the motion vector of the sample to be predicted can be set as a prediction sample for the sample to be predicted.

[0734] Meanwhile, if the motion vector of the sample to be predicted is expressed in fractional units, integer position samples can be interpolated to generate fractional position samples, and the generated fractional position samples can be set as prediction samples for the sample to be predicted.

[0735] At this time, the precision of the motion vector for each sample may differ. For example, the motion vector for the first prediction target sample may be derived in units of 1 / 2 pixels, while the motion vector for the second prediction target sample may be derived in units of 1 / 4 pixels.

[0736] In this case, fractional position samples can be generated according to the motion vector precision for each of the prediction target samples. Alternatively, the motion vector of the prediction target sample can be adjusted according to the reference motion vector precision, and then prediction samples for the prediction target sample can be derived based on the adjusted motion vector. For example, if the reference motion vector precision is 1 / 2, the motion vector for the second prediction target sample can be adjusted in 1 / 4 pixel increments.

[0737] The reference motion vector precision can be determined in block units. Alternatively, the precision of the control point motion vectors can be set to the reference motion vector precision. Alternatively, the reference motion vector precision may be predefined in the encoder and decoder.

[0738] As another example, to reduce complexity, motion vectors can be derived at the sub-block level.

[0739] Figure 54 shows an example in which a motion vector is derived in sub-block units.

[0740] The size and / or shape of the sub-block may be predefined in the encoder and decoder. For example, the sub-block may be a square block of size 2x2 or 4x4.

[0741] Alternatively, the size and / or shape of the sub-block may be adaptively determined based on the size and / or shape of the current block. For example, if the current block is square, the sub-block may also be square. Conversely, if the current block is non-square, the sub-block may also be non-square.

[0742] Alternatively, information regarding at least one of the partitioning method or partitioning form of the current block may be explicitly encoded and signaled. For example, information regarding at least one of the size of a sub-block, the shape of a sub-block, the location of a partition line dividing the current block, or the number of partition lines may be explicitly encoded and signaled. The information may be encoded and signaled on a block-by-block basis, or it may be encoded and signaled through an upper header.

[0743] In Fig. 54, it was assumed that the sub-block is a square block of size 2x2.

[0744] The motion vector of a sub-block can be derived using the coordinates of a predefined location within the sub-block. Here, the predefined location may be one of the location of the top-left sample, the top-right sample, the bottom-left sample, the bottom-right sample, or the center location within the sub-block.

[0745] By substituting the coordinates of a predefined position within the sub-block into (x, y) of Equation 13, the motion vector of the sub-block can be derived.

[0746] As in the example described above, motion vectors can be derived in sub-block units based on an affine motion model.

[0747] Meanwhile, motion vectors can also be derived in sub-block units using collocated pictures. As described above, deriving motion vectors in sub-block units using collocated pictures can be referred to as SbTMVP (Sub-block Temporal Motion Vector Prediction).

[0748] A collocated picture may be one of the reference pictures included in the reference picture list. For example, a picture with index 0 in the reference picture list may be selected as the collocated picture.

[0749] Alternatively, information indicating the index of a reference picture set as a collocated picture within the reference picture list may be explicitly encoded and signaled.

[0750] Figures 55 and 56 show examples in which motion vectors are induced in sub-block units within the current block when SbTMVP is applied.

[0751] The size and / or shape of the sub-block may be predefined in the encoder and decoder.

[0752] Alternatively, the size and / or shape of the sub-block may be adaptively determined according to the size and / or shape of the current block. For example, if at least one of the width or height of the current block is greater than a threshold value, the size of the sub-block may be set to 8x8. Otherwise, the size of the sub-block may be set to 4x4.

[0753] Alternatively, information indicating the size and / or shape of the sub-block may be explicitly encoded and signaled.

[0754] In the example illustrated in FIG. 55, it is assumed that the current block size is 16x16 and the sub-block size is 4x4.

[0755] When SbTMVP is applied, the initial motion vector of the current block can be derived. The initial motion vector can be derived based on at least one of a motion vector prediction list or a motion information merge list. For example, an index indicating one of the motion vector prediction candidates included in the motion vector prediction list can be encoded and signaled. The initial motion vector can be derived by adding a motion vector difference value to the motion vector prediction candidate indicated by the index. Meanwhile, the motion vector difference value can also be explicitly encoded and signaled.

[0756] Alternatively, the encoding of the index may be omitted, and a motion vector prediction candidate with a predefined index within the motion vector prediction list may be set as the prediction value for the initial motion vector. Here, the motion vector prediction candidate with a predefined index may be a motion vector prediction candidate with an index of 0 or a motion vector prediction candidate with the largest index.

[0757] Alternatively, an index indicating one of the motion information merge candidates included in the motion information merge list may be encoded and signaled. The initial motion vector may be set to be identical to the motion vector of the motion information merge candidate indicated by the index.

[0758] Alternatively, the encoding of the index can be omitted, and an initial motion vector can be derived based on a motion information merging candidate having a predefined index within the motion information merging list. Here, the motion information merging candidate having a predefined index may be a motion information merging candidate with an index of 0 or a motion information merging candidate with the largest index.

[0759] Alternatively, an initial motion vector can be derived using the motion vector of a neighbor block at a predefined position. Here, the neighbor block at the predefined position may be a left neighbor block or an top neighbor block.

[0760] The motion vector of a neighbor block at a predefined position can be set as the predicted value of the initial motion vector, and the initial motion vector can be derived by adding a difference value to the predicted value.

[0761] Alternatively, the motion vector of a neighbor block at a predefined position can be set as the initial motion vector.

[0762] Alternatively, the initial motion vector can be derived using a template-based motion estimation method (i.e., a template matching method) or two-way matching.

[0763] The precision of the initial motion vector may be predefined in the encoder and decoder. For example, the precision of the initial motion vector may be fixed in integer pixel units.

[0764] Alternatively, information indicating the precision of the initial motion vector may be explicitly encoded and signaled. The information may be an index indicating one of a plurality of motion vector precision candidates.

[0765] When deriving an initial motion vector using motion vector prediction candidates, motion vector prediction candidates can be derived based on the motion vector precision of the initial motion vector. That is, after adjusting the motion vector prediction candidates to match the motion vector precision of the initial motion vector, the adjusted initial motion vector prediction candidates can be inserted into the motion vector prediction list.

[0766] When deriving initial motion vectors using motion information merging candidates, motion information merging candidates can be derived based on the motion vector precision of the initial motion vectors. That is, after adjusting the motion information merging candidates according to the motion vector precision of the initial motion vectors, the adjusted initial motion information merging candidates can be inserted into the motion information merging list.

[0767] Meanwhile, among the motion information merging candidates included in the motion information merging list, only those candidates whose reference picture is identical to the collocated picture of the current block can be used to derive the initial motion vector. That is, if the reference picture of a motion information merging candidate is different from the collocated picture of the current block, the initial motion vector may not be derived from that motion information merging candidate.

[0768] If there are multiple candidates among the motion information merging candidates for which the reference picture is identical to the collocated picture of the current block, an index indicating one of the multiple candidates can be encoded and signaled. Alternatively, if there are multiple candidates among the motion information merging candidates for which the reference picture is identical to the collocated picture of the current block, an initial motion vector can be derived from the candidate with the smallest index or the candidate with the largest index among the multiple candidates.

[0769] If a motion information merging candidate has both motion information in the L0 direction and motion information in the L1 direction, one of the motion information in the L0 direction and the motion information in the L1 direction is selected according to a preset priority, and an initial motion vector can be derived from the selected motion information.

[0770] The priority can be determined based on at least one of the magnitude of the motion vector of the motion merge candidate, the index of the reference picture of the motion merge candidate, or whether the reference picture of the motion merge candidate is the same as the collocated picture.

[0771] Alternatively, it may be set to always derive an initial motion vector based on motion information in the L0 direction.

[0772] When initial motion vectors are derived based on a template matching method, motion estimation can be performed according to the precision of the initial motion vectors. For example, if the precision of the initial motion vectors is in the integer pixel unit, motion estimation based on template matching can also be performed only at integer locations.

[0773] Similarly, when an initial motion vector is derived based on two-way matching, motion estimation can be performed according to the precision of the initial motion vector.

[0774] Meanwhile, as a result of the two-way matching, a motion vector for the L0 direction (L0 motion vector) and a motion vector for the L1 direction (L1 motion vector) are derived. In this case, according to a pre-set priority, one of the L0 motion vector and the L1 motion vector can be set as the initial motion vector.

[0775] Alternatively, it may be set to always derive an initial motion vector based on motion information in the L0 direction.

[0776] Alternatively, information indicating which of the L0 motion vector and L1 motion vector is set as the initial motion vector may be encoded and signaled.

[0777] Once an initial motion vector is derived, the position of a collocated block within a collocated block can be determined using the initial motion vector. For example, a block located at a position offset by the initial motion vector from a position corresponding to the current block within a reference picture can be set as a collocated block. In this case, the position of the collocated block can be determined based on a predefined position within the current block. Here, the predefined position may be the top-left position, top-right position, bottom-left position, bottom-right position, or center position.

[0778] Depending on the division method of the current block, the collocated block can be divided into multiple collocated sub-blocks. Additionally, the motion vector of each collocated sub-block within the collocated block can be set as the motion vector of each sub-block within the current block.

[0779] As another example, the positions of collocated sub-blocks corresponding to each of the sub-blocks within the current block in the collocated picture can be determined using initial motion vectors. In this case, the positions of the collocated sub-blocks can be derived based on predefined positions within the sub-blocks. Here, the predefined positions may be the top-left, top-right, bottom-left, bottom-right, or center positions.

[0780] Subsequently, the motion vector of the collocated sub-block corresponding to the sub-block can be set as the motion vector of the sub-block. Specifically, the motion vector stored at a position corresponding to a predefined position within the sub-block within the collocated sub-block can be set as the motion vector of the sub-block.

[0781] Meanwhile, if the motion information of the collocated sub-block is unavailable, a predefined motion vector can be set as the motion vector of the sub-block. Here, the predefined motion vector may be a zero vector (i.e., (0, 0)) or an initial motion vector.

[0782] Alternatively, if the motion information of the collocated sub-block corresponding to the sub-block is unavailable, the motion vector of the sub-block may be derived from another location within the collocated sub-block.

[0783] Specifically, when a position corresponding to a predefined position within a collocated sub-block is encoded by intra-prediction, there is no motion vector at that position. For example, if a predefined position is assumed to be a central position (e.g., c10 in FIG. 56), and no motion vector is stored at the central position, the motion vector of the sub-block cannot be derived.

[0784] In this case, the motion vector of the sub-block can be derived based on the motion vector stored at a location different from the center position. Specifically, the motion vector of the sub-block can be derived from the motion vector stored at a location adjacent to the center position (e.g., top adjacent position c6, left adjacent position c9, or top-left adjacent position c5).

[0785] Alternatively, if the center location is unavailable, samples within the collocated sub-block may be searched according to the scan order, and the first available motion vector found may be set as the motion vector of the sub-block. Here, the scan order may be a horizontal scan, a vertical scan, a diagonal scan, or a raster scan.

[0786] Alternatively, if the motion information of the collocated sub-block is unavailable, the motion vector of the sub-block can be set as the motion vector of the collocated block. For example, the motion vector stored at a position corresponding to a previously defined position within the current block within the collocated block can be set as the motion vector of the sub-block.

[0787] As in the example described above, motion vectors can be derived in sub-block units using an affine motion model or SbTMVP. When motion vectors are derived in sub-block units, motion compensation can be performed for each sub-block based on the motion vector of each sub-block.

[0788] By performing motion compensation for each of the sub-blocks, a prediction block for the current block can be obtained. That is, the prediction block may be composed of prediction samples for each of the sub-blocks.

[0789] When detecting movement between frames, the precision of the motion vector can be adjusted. Specifically, the position of each sample within a picture is defined as an integer position. However, the position reflecting the movement can be a real number rather than an integer position.

[0790] Considering this, motion vectors can be explored more precisely through reference picture interpolation.

[0791] Figures 57 and 58 are diagrams illustrating examples in which a prediction block is derived according to the precision of the motion vector.

[0792] FIG. 57 shows the position of the current block in the current picture, and FIG. 58 shows the position of the reference block according to the motion vector precision.

[0793] As shown in the examples illustrated in FIGS. 57 and 58, the motion vector of the current block can be defined as the distance from the sample corresponding to the top-left position of the current block in the reference picture to the sample corresponding to the top-left position of the reference block in the reference picture.

[0794] FIG. 58 (a) illustrates the case where the motion vector precision of the current block is an integer Pel, FIG. 58 (b) illustrates the case where the motion vector precision of the current block is 1 / 2 Pel. Also, FIG. 58 (c) illustrates the case where the motion vector precision of the current block is 1 / 4 Pel.

[0795] In Fig. 58, the vector precision is expressed up to 1 / 4, but the motion vector can be expressed with even greater precision, such as 1 / 8, 1 / 16, or 1 / 32.

[0796] Meanwhile, information for indicating the motion vector precision of the current block may be encoded and signaled. For example, the information may be an index identifying one of the motion vector precision candidates. Specifically, a different index may be assigned to each of the motion vector precision candidates, and the information may indicate the index of the motion vector precision candidate applied to the current block.

[0797] By adjusting the precision of the motion vectors used for cross-frame prediction, more precise motion vector detection may be possible. If the reference block indicated by the motion vector exists at a real-valued location, the samples at the real-valued location can be generated using samples at integer locations and an interpolation filter. Additionally, motion vectors represented by real numbers can be scaled up to integers for encoding / decoding.

[0798] Thus, the motion vector (MV), motion vector predicted value (MVP), and motion vector difference value (MVD) can be encoded / decoded into integer values ​​through integerization. Specifically, the motion vector, motion vector predicted value, and / or motion vector difference value can be integerized based on the motion vector precision.

[0799] For example, if the motion vector precision is 1 / N, the motion vector difference value MVD can be converted to an integer by multiplying it by N. For example, if the motion vector difference value MVD is (4 / 16, 8 / 16), the motion vector difference value MVD can be converted to an integer by multiplying it by 16. That is, the converted motion vector difference value MVD can be expressed as (4, 8).

[0800] Based on motion vector precision, the actual MVD can be derived from the integerized MVD. For example, if the motion vector precision is 1 / N, the actual MVD can be derived by dividing the integerized MVD by N. For example, if the integerized MVD is (4, 8) and the motion vector precision is 1 / 8, the actual MVD can be (4 / 8, 8 / 8). Or, if the integerized MVD is (4, 8) and the motion vector precision is 1 / 4, the actual MVD can be (4 / 4, 8 / 4).

[0801] Depending on the motion vector precision, the range of representation of the integerized MVD may differ. For example, assume that the motion vector difference value MVD is (4 / 16, 8 / 16) (i.e., (1 / 4, 2 / 4)). When the motion vector precision is 1 / 16, the integerized MVD is derived as (4, 8). On the other hand, when the motion vector precision is 1 / 4, the integerized MVD is derived as (1, 2).

[0802] Comparing the two cases above, if the motion vector precision is adjusted from 1 / 16 to 1 / 4, the value of the integerized MVD can be reduced from (4, 8) to (1, 2).

[0803] Consequently, depending on the motion vector precision, the number of bits required to encode / decode the integerized motion vector difference value MVD may vary. Accordingly, a motion vector precision that minimizes the number of bins can be selected when encoding / decoding the motion vector difference value MVD. Then, based on the selected motion vector precision, the motion vector difference value MVD can be converted to an integer, and the integerized motion vector difference value MVD can be encoded / decoded. In addition, information regarding the motion vector precision can be additionally encoded / decoded.

[0804] In the decoder, the actual MVD can be restored from the decoded MVD based on motion vector precision. Then, the motion vector MV can be derived by combining the restored MVD and the motion vector prediction value MVP.

[0805] As described above, adjusting the value of the motion vector difference value MVD, which is encoded / decoded based on motion vector precision, is called the AMVR (Adaptive Motion Vector Resolution) method.

[0806] Figures 59 and 60 are diagrams illustrating the process of encoding and decoding motion vector difference values ​​when the AMVR method is applied, respectively.

[0807] For the sake of convenience of explanation, it is assumed that the motion vector and the motion vector difference value are expressed in units of 1 / 16 before integerization is performed, and 1 / 16 is referred to as the original motion vector precision.

[0808] The motion vector difference value MVD can be derived by differencing the motion vector prediction value MVP from the motion vector MV (S5910).

[0809] The motion vector difference value MVD may consist of a horizontal component (i.e., the x-axis component) and a vertical component (i.e., the y-axis component).

[0810] When the motion vector difference value is 0, that is, when both the horizontal and vertical components are 0, the value of the motion vector difference value MVD to be encoded becomes 0 regardless of the motion vector precision. Therefore, when the motion vector difference value MVD is 0, the encoding of AMVR-related information can be omitted (S5920).

[0811] On the other hand, if the motion vector difference value is not zero, that is, if at least one of the horizontal component and the vertical component is not zero, the motion vector precision can be determined (S5930). Meanwhile, the motion vector precision can be encoded as AMVR-related information.

[0812] Information related to AMVR may include at least one of a flag (e.g., amvr_flag) indicating whether the AMVR method is applied to the current block and an index (e.g., amvr_prec_idx) indicating one of a plurality of motion precision candidates if the AMVR method is applied.

[0813] If the AMVR method is not applied to the current block, the motion vector precision can be set to a default value. In this case, amvr_flag can be encoded as a value of 0. Meanwhile, the default value can be 1, 1 / 2, 1 / 4, 1 / 8, or 1 / 16.

[0814] When the AMVR method is applied to the current block, an index indicating one of multiple motion vector precision candidates, i.e., amvr_prec_idx, may be additionally decoded. In this case, amvr_flag is encoded with a value of 1, and amvr_prec_idx may be encoded with a value from 0 to (n-1). Here, n represents the number of motion vector precision candidates. For example, multiple motion vector precision candidates may include at least one of 4, 2, 1, 1 / 2, 1 / 4, 1 / 8, or 1 / 16. Meanwhile, the default value may not be set to the multiple motion vector precision candidates indicated by the index. That is, if the motion vector precision of the current block is the default value, it is encoded and signaled as 0, which is the value of amvr_flag, and the encoding of amvr_prec_idx may be omitted.

[0815] In the encoder, the optimal motion vector precision can be determined by performing Rate Distortion Optimization (RDO) for each combination of amvr_flag and amvr_prec_idx. That is, by performing RDO for the following cases, the combination with the optimal cost can be selected.

[0816] 1) When amvr_flag is 0

[0817] 2) When amvr_flag is 1 and amvr_prec_idx is 0

[0818] 3) When amvr_flag is 1 and amvr_prec_idx is 1

[0819] 4) When amvr_flag is 1 and amvr_prec_idx is 2

[0820] Depending on the motion vector precision of the current block, a variable for scaling the motion vector difference value, i.e., a scaling parameter, can be set. For example, Table 11 shows the values ​​of the variable amvrshift according to the motion vector precision.

[0821] amvr_flagamvr_prec_idxamvrshift0 (1 / 4)-210 (1 / 2)311 (1-pel)412 (4-pel)6

[0822] If the finest motion vector precision applicable to the current block is 1 / 16, the motion vector precision can be expressed as in the following mathematical formula 15.

[0823]

[0824] As shown in Table 11, when the value of amvr_flag is 0, the variable amvrshift is set to 2. This indicates that the motion vector precision is 1 / 4 according to Equation 15.

[0825] When the value of amvr_flag is 1, the variable amvrshift can be determined according to the value of amvr_prec_idx. For example, when amvr_prec_idx is 1, the variable amvrshift is set to 4. This indicates that the motion vector precision is 1 according to Equation 15.

[0826] In the encoder, the motion vector difference value MVD can be scaled down and encoded using the variable amvrshift, which is based on the motion vector precision. As an example, Equation 16 shows an example of a scale-down operation being performed on the motion vector difference value MVD.

[0827]

[0828] In the above mathematical equation 16, MVD_x represents the horizontal component of the motion vector difference value, and MVD_y represents the vertical component of the motion vector difference value. MVD'_x and MVD'_y represent the results of performing a scale-down operation.

[0829] The encoder can encode motion vector difference values ​​and AMVR information with changed precision (S5940).

[0830] In the decoder, the motion vector difference value MVD can be decoded (S6010).

[0831] If the motion vector difference value is 0, the decoding of AMVR-related information is omitted, and the motion vector MV of the current block can be set to be the same as the motion vector prediction value (S6020).

[0832] On the other hand, if the motion vector difference value is not zero, that is, if at least one of the horizontal component and the vertical component is not zero, information related to AMVR can be additionally decoded (S6030).

[0833] Based on AMVR information, a variable amvrshift for scaling motion vector difference values ​​can be derived. For example, as shown in the example in Table 11, a variable amvrshfit can be derived based on amvr_flag and / or amvr_prec_idx.

[0834] Afterwards, the decoded MVD can be scaled up using the variable amvrshift to obtain the motion vector difference value MVD restored to the original precision (S6040). Equation 17 shows an example of applying a scale-up operation to the decoded MVD.

[0835]

[0836] In Equation 17, MVD' represents the decoded motion vector difference value. MVD represents the motion vector difference value restored to its original precision, i.e., 1 / 16, through a scale-up operation.

[0837] Afterwards, the motion vector MV can be obtained by combining the motion vector difference value MVD restored to the original precision and the motion vector prediction value MVP.

[0838] As in the example above, when a motion vector prediction mode is applied, the decoder can derive the motion vector MV by combining the motion vector prediction value MVP and the motion vector difference value MVD.

[0839] A prediction block for the current block can be generated by weighting the first prediction block and the second prediction block. Here, the first prediction block may be generated by performing an intra prediction on the current block. For example, the first prediction block may be obtained by performing an intra prediction on the current block based on a predefined intra prediction mode. The predefined intra prediction mode may be a planner mode or a DC mode.

[0840] The second prediction block may be generated by performing inter-prediction on the current block. For example, one of the merge candidates included in the motion information merge list may be selected, and the second prediction block may be obtained based on the motion information of the selected merge candidate.

[0841] Mathematical formula 18 shows an example in which the prediction block of the current block is generated through the weighted sum of the first prediction block and the second prediction block.

[0842]

[0843] In the above mathematical formula 18, (x, y) represents the coordinates of a sample within the current block of size WxH. For example, P(x, y) may represent a predicted sample at position (x, y) within the current block. P1(x, y) may represent a first predicted sample at position (x, y) within the first predicted block, and P2(x, y) may represent a second predicted sample at position (x, y) within the second predicted block.

[0844] The weights applied to the first prediction block are w1 / (w1+w2), and the weights applied to the second prediction block are w2 / (w1+w2). Since the sum of the weights is 1, there is no need to perform normalization.

[0845] To replace the division operation with a shift operation, Equation 18 can be modified as shown in Equation 19 below.

[0846]

[0847] In mathematical formula 19, the variable shift representing the shifting parameter can have a value of (Log2(w1+w2)). The variable offset for rounding processing can have a value of (shift>>1).

[0848] Since it cannot be guaranteed that the sum of the first weight w1 applied to the first prediction block and the second weight w2 applied to the second prediction block is 1, normalization of the first weight w1 and the second weight w2 can be performed using the variable shift. Meanwhile, the sum of the first weight w1 and the second weight w2 can be a natural number expressed as a power of 2, for example, 8 or 16.

[0849] A first weight w1 and a second weight w2 can be determined by referring to the encoding mode of a neighboring block adjacent to the current block.

[0850] FIG. 61 shows an example in which a first weight w1 and a second weight w2 are determined based on the encoding modes of neighboring blocks adjacent to the current block.

[0851] In the example illustrated in FIG. 61, the top neighbor block T is a block containing a sample located above sample a located at the top right of the current block, and the left neighbor block L is a block containing a sample located to the left of sample b located at the bottom left of the current block.

[0852] When both the upper neighbor block T and the left neighbor block L are encoded / decoded by intra prediction, the first weight can be determined as 3 and the second weight can be determined as 1.

[0853] If intra prediction is used only in one of the top neighbor block T and the left neighbor block L, the first weight and the second weight can each be determined to be 2.

[0854] If both the upper neighbor block T and the left neighbor block L are not encoded / decoded by intra prediction (e.g., if both the upper neighbor block T and the left neighbor block L are encoded / decoded by inter prediction), the first weight may be determined as 1 and the second weight may be determined as 3.

[0855] If the upper neighbor block T or the left neighbor block L is unavailable, the corresponding block is considered not to have been encoded / decoded by intra-prediction, and the first weight and the second weight can be determined.

[0856] In the example described above, the first prediction block is exemplified as being acquired based on planner mode or DC mode. As another example, the first prediction block may be acquired based on the intra prediction mode of a neighbor block of the current block. For example, the first prediction block may be acquired by using either the intra prediction mode of the upper neighbor block T or the intra prediction mode of the left neighbor block L.

[0857] Alternatively, if both the upper neighbor block T and the left neighbor block L are encoded by intra prediction, a first prediction block can be obtained by averaging or weighting the prediction block obtained by performing intra prediction based on the intra prediction mode of the upper neighbor block T and the prediction block obtained by performing intra prediction based on the intra prediction mode of the left neighbor block L.

[0858] Alternatively, if one of the intra prediction mode of the upper neighbor block T and the intra prediction mode of the left neighbor block L is a non-directional mode, the first prediction block can be obtained by performing intra prediction based only on the non-directional mode.

[0859] Alternatively, if one of the intra prediction mode of the upper neighbor block T and the intra prediction mode of the left neighbor block L is a directional mode, the first prediction block can be obtained by performing intra prediction based only on the directional mode.

[0860] Alternatively, a first prediction block may be generated based on an intra prediction mode derived from the histogram of the current block. Specifically, a horizontal gradient and a vertical gradient for a reconstructed sample belonging to a reference region adjacent to the current block may be obtained, and an intra prediction mode of the reconstructed sample may be determined based on the horizontal gradient and the vertical gradient. Subsequently, the amplitude values ​​obtained based on the horizontal gradient and the vertical gradient may be accumulated in a buffer assigned to the corresponding intra prediction mode on the histogram.

[0861] By selecting the intra prediction mode with the largest amplitude value on the histogram and performing intra prediction based on the selected intra prediction mode, a first prediction block can be obtained.

[0862] Alternatively, an intra prediction mode used to acquire the first prediction block can be determined based on the template cost per intra prediction mode. The template cost of the intra prediction mode can be obtained by summing the differences between the prediction samples obtained by performing intra prediction on the current template and the reconstructed samples within the current template. After determining a reference sample line for the current template, intra prediction can be performed on the current template.

[0863] After correcting the motion information merge candidates included in the motion information merge list of the current block, a second prediction block can be obtained based on the corrected motion information merge candidates.

[0864] Specifically, the point indicated by the motion information of the motion information merging candidate can be set as the starting point, and a search range can be set centered on the starting point. Subsequently, the reference template with the smallest cost relative to the current template within the search range can be searched, and the motion vector of the motion information merging candidate can be corrected to point to the searched reference template. That is, the distance between the starting point and the searched reference template is set as the offset vector, and the offset vector is added to the motion vector to obtain a corrected motion vector.

[0865] Motion information correction can be performed for each motion merge candidate included in the motion vector merge list. Alternatively, motion information correction can be performed for only some of the motion merge candidates included in the motion information merge list. For example, motion information correction can be performed only for N motion information merge candidates with small indices. N is a natural number greater than or equal to 1, e.g., 2.

[0866] If motion information merge candidates are corrected, only the corrected motion information merge candidates may be selectable. That is, an index indicating one of the motion information merge candidates can indicate one of the corrected motion information merge candidates.

[0867] The motion information merge candidates included in the motion information merge list can be reordered according to their template costs. That is, the indices of the motion information merge candidates can be reassigned in order of lowest template cost. Meanwhile, the number of motion information merge candidates subject to reordering may be predefined. For example, reordering can be performed only on M motion information merge candidates within the motion information merge list.

[0868] Subsequently, a second prediction block can be obtained based on the top M motion merge candidates within the motion information merge list. For example, a second prediction block can be obtained based on one of the top M motion merge candidates. In this case, an index indicating one of the motion information merge candidates can indicate one of the top M motion information merge candidates.

[0869] When performing a weighted sum of the first prediction block and the second prediction block, the first weight and the second weight can be set differently for each region. Specifically, the current block can be divided into multiple regions in the vertical or horizontal direction, and then the weights applied to the first prediction block and the second prediction block can be set differently for each region.

[0870] Figure 62 illustrates an example where weights are adaptively determined for each region.

[0871] When the intra prediction mode used to acquire the first prediction block has a left direction, the current block can be divided vertically to form multiple regions, as in the example shown in FIG. 62 (a). Here, the intra prediction mode having a left direction may represent a directional intra prediction mode with an index smaller than 34.

[0872] When the intra prediction mode used to acquire the first prediction block has an upward direction, the current block can be divided vertically to form multiple regions, as shown in the example illustrated in FIG. 62 (b). Here, the intra prediction mode having an upward direction may represent a small directional intra prediction mode with an index equal to or greater than 34.

[0873] Alternatively, conversely to the above, if the intra prediction mode has an upward direction, the current block can be divided horizontally, and if the intra prediction mode has a left direction, the current block can be divided vertically.

[0874] Alternatively, information indicating the split type of the current block can be encoded and signaled. The said information may indicate the splitting direction of the current block.

[0875] Setting weights by region may be applied only when the intra prediction mode used in the first prediction block is a directional prediction mode. When the intra prediction mode used in the first prediction block is a non-directional prediction mode, the first weight assigned to the first prediction block and the second weight assigned to the second prediction block may have uniform values ​​across the entire region of the current block.

[0876] The first weight and the second weight may differ depending on the area. For example, the first weight and the second weight of each area may be set as shown in Table 12 below.

[0877] Area Index 1st Weight w1 2nd Weight w2062153235326

[0878] In the example of Table 12, the sum of the first and second weights for each region is 8. The first and second weights for each region may be set differently from the example of Table 12.

[0879] Table 13 shows an example where the sum of the first and second weights for each region is 16.

[0880] Area Index 1st Weight w1 2nd Weight w20124110626103412

[0881] Different weight lookup tables may be used depending on the encoding mode of neighbor blocks adjacent to the current block. For example, if only one of the left neighbor block L and the top neighbor block T is encoded / decoded by intra prediction, the first weight and second weight of each region may be determined according to the example in Table 13.

[0882] On the other hand, when both the left neighbor block L and the top neighbor block T are encoded / decoded by intra prediction, the first weight and the second weight of each region can be determined according to the example in Table 14.

[0883] Area Index 1st Weight w1 2nd Weight w2014211242883610

[0884] If both the left neighbor block L and the top neighbor block T are not encoded / decoded by intra prediction, the first weight and the second weight of each region can be determined according to the example in Table 15.

[0885] Area Index 1st Weight w1 2nd Weight w2010618824123214

[0886] Alternatively, information indicating one of multiple weighted lookup tables may be encoded and signaled.

[0887] Alternatively, a weighted lookup table can be adaptively selected depending on the size of the current block.

[0888] Alternatively, a weighted lookup table may be adaptively selected based on the intra-prediction mode of the current block. For example, a weighted lookup table may be adaptively selected based on at least one of whether the intra-prediction mode of the current block is a directional mode or a non-directional mode.

[0889] Alternatively, a weighted lookup table may be adaptively selected according to the intra prediction mode derivation method for each region. The intra prediction mode derivation method for each region may include at least one of a method that uses the intra prediction mode of the current block as is or a method that derives the intra prediction mode based on a histogram. The intra prediction mode derivation method for each region is described in detail through the following embodiments.

[0890] When an intra-prediction mode of a region is derived from a histogram, the largest amplitude value within the histogram can be compared with a threshold value to adaptively select a weighted lookup table.

[0891] In the examples of Tables 12 to 15, it is also possible to set the first weight or the second weight to 0. For example, in the example of Table 14, for the region where the index is 0, the first weight can be set to 16 and the second weight can be set to 0.

[0892] Alternatively, depending on the size of the current block, it may be determined whether to set the first weight or the second weight within a specific area to 0. For example, if the size of the current block exists outside a pre-set range, the first weight or the second weight may be set to 0 for the area where the index is 0.

[0893] When an intra-prediction mode of a region is derived from a histogram, the largest amplitude value within the histogram can be compared with a threshold value to determine whether to set a first weight or a second weight to 0. For example, if the largest amplitude value within the histogram of a region is not greater than the threshold value, the first weight for that region can be set to 0.

[0894] For each of the first prediction block and the second prediction block, a histogram is generated, and the maximum amplitude values ​​in each prediction block are compared to derive the first weight and the second weight. Here, the first prediction block may be derived by performing intra-prediction based on a non-directional prediction mode.

[0895] The histograms of the first prediction block and the second prediction block, respectively, can be generated through a process of calculating the horizontal gradient and the vertical gradient for the prediction samples included in the prediction block.

[0896] If the maximum amplitude value in the histogram of the first prediction block is greater than the maximum amplitude value in the histogram of the second prediction block, the first weight can be set to a value greater than the second weight (i.e., w1 > w2). On the other hand, if the maximum amplitude value in the histogram of the first prediction block is smaller than the maximum amplitude value in the histogram of the second prediction block, the first weight can be set to a value smaller than the second weight (i.e., w1 < w2).

[0897] A weighted sum operation can be performed based on a first weight w1 and a second weight w2 assigned to the region to which the prediction sample belongs. For example, if the location (x, y) of the prediction sample belongs to a region with an index of 2, the first weight assigned to the first prediction sample P1(x, y) can be set to 3, and the second weight assigned to the second prediction sample P2(x, y) can be set to 5 to obtain the prediction sample P(x, y).

[0898] Different intra-prediction modes may be used for each region. That is, the first prediction block may be obtained by applying different intra-prediction modes for each region.

[0899] To induce a region-specific intra-prediction mode, a second prediction block may be used. Specifically, a region corresponding to the current region within the second prediction block may be set as a reference region, and a histogram may be generated based on samples included in the reference region. Here, the samples included in the reference region may represent prediction samples obtained through inter-prediction, i.e., reconstructed samples within the reference picture.

[0900] For each region, a reference region can be set, and a histogram can be generated for each region using samples within the reference region. Specifically, for samples belonging to the reference region, a horizontal gradient and a vertical gradient can be calculated, and an intra prediction mode can be determined based on the horizontal gradient and the vertical gradient. Subsequently, the amplitude values ​​derived based on the horizontal gradient and the vertical gradient can be accumulated in a buffer assigned to the corresponding intra prediction mode on the histogram. The intra prediction mode with the largest amplitude within the histogram can be set as the intra prediction mode for the current region.

[0901] Meanwhile, the division direction of the current block can be determined based on the intra prediction mode induced for the current block. For example, if the intra prediction mode of the current block has a left direction, the current block can be divided horizontally. By dividing the second prediction block horizontally as well, reference areas for each region within the current block can be established. On the other hand, if the intra prediction mode of the current block has an upward direction, the current block can be divided vertically. By dividing the second prediction block vertically as well, reference areas for each region within the current block can be established.

[0902] As another example, the intra prediction mode of each region can be derived based on the restoration region adjacent to each region. In this case, the restoration region referenced to derive the intra prediction mode of each region can be determined based on the intra prediction mode determined for the current block.

[0903] Figure 63 illustrates the restoration region referenced to induce the intra prediction mode of each region.

[0904] If the intra prediction mode for the current block is a directional intra prediction mode having an upward direction, the intra prediction mode for each region can be derived by referring to the upper restoration region of the current block, as in the example shown in FIG. 63 (a). Additionally, if the intra prediction mode for the current block is a directional intra prediction mode having a left direction, the intra prediction mode for each region can be derived by referring to the left restoration region of the current block, as in the example shown in FIG. 63 (b).

[0905] Alternatively, conversely to the above, if the intra prediction mode for the current block is a directional intra prediction mode having a left direction, the intra prediction mode for each region can be derived by referring to the upper restoration region of the current block, as in the example illustrated in FIG. 63 (a). In addition, if the intra prediction mode for the current block is a directional intra prediction mode having a left direction, the intra prediction mode for each region can be derived by referring to the left restoration region of the current block, as in the example illustrated in FIG. 63 (b).

[0906] If the division direction of the current block is determined by information that is explicitly encoded / decoded, the restoration area referenced to induce the intra prediction mode of each area can be determined according to the division direction indicated by the information. For example, if the information indicates a vertical division, the intra prediction mode of each area can be induced by referencing the upper restoration area, as in the example shown in FIG. 63 (a). On the other hand, if the information indicates a horizontal division, the intra prediction mode of each area can be induced by referencing the left restoration area, as in the example shown in FIG. 63 (b).

[0907] Based on the intra prediction mode derived for the current block and the intra prediction mode derived for each region, a first prediction block may also be derived.

[0908] For example, a first prediction block can be derived by averaging or weighting the prediction block obtained by performing intra prediction on the current block based on the intra prediction mode of the current block, and the prediction block obtained by performing intra prediction on each region based on the intra prediction mode of each region.

[0909] Alternatively, information indicating whether to acquire the first prediction block using the intra prediction mode of the current block or to acquire the first prediction block using the intra prediction mode derived per region may be encoded and signaled.

[0910] Alternatively, for each region, it may be determined whether to induce an intra prediction mode based on the histogram of the current region, based on the cost calculated when performing intra prediction on the current region based on the intra prediction mode of the current block. In this case, the cost may be derived as the sum of the differences between the prediction samples obtained by performing intra prediction on the current region and the samples included in the region corresponding to the current region within the second prediction block.

[0911] For example, if the induced cost is greater than the threshold value, the intra prediction mode derived from the histogram of that region can be set as the intra prediction mode of that region. On the other hand, if the induced cost is not greater than the threshold value, the intra prediction mode of the current block can be set as the intra prediction mode of that region.

[0912] Alternatively, information indicating a method for deriving the intra-prediction mode of each region may be encoded and signaled. The information may indicate whether the intra-prediction mode of a region is set to be identical to the intra-prediction mode of the current block, or whether the intra-prediction mode of a region is derived from the histogram of that region. The information may be encoded and signaled for the current block. Alternatively, the information may be encoded and signaled on a region-by-region basis.

[0913] Alternatively, for a pre-configured area within the current block, the intra prediction mode of the current block may be used fixedly. The pre-configured area may include at least one of the first area (i.e., the area with index 0) or the last area (i.e., the area with the largest index).

[0914] Regions can be encoded / decoded sequentially. In this case, intra prediction for the current region can be performed by using the reconstructed samples included in the region encoded / decoded prior to the current region as reference samples.

[0915] The encoder generates a residual block by differentiating the prediction block from the original block. Subsequently, it can be determined whether to perform a transformation on the residual values ​​present in the residual block, i.e., the residual samples. Depending on the above determination, residual coefficients can be generated by performing quantization on the residual samples for which no transformation was performed or on the transformation coefficients generated as a result of the transformation. Here, the residual coefficient may refer to at least one of the quantized transformation coefficients or quantized residual samples, depending on whether the transformation was applied. The residual coefficient may also be referred to as the residual signal. Subsequently, encoding can be performed on each of the residual coefficients.

[0916] In the decoder, residual samples can be recovered through the inverse process of the encoder. Specifically, after obtaining residual coefficients, inverse quantization can be performed on the obtained residual coefficients. Subsequently, residual samples can be obtained by performing an inverse transform on the inverse quantized residual coefficients or by omitting the inverse transform. Encoding of the residual coefficients can be performed based on at least one syntax element.

[0917] First, a flag indicating whether there is a non-zero residual coefficient in the current block can be encoded and signaled. If there is a non-zero residual coefficient in the current block, the position of the last non-zero residual coefficient in the scan order can be encoded.

[0918] In addition, a sub-block flag indicating whether there is a non-zero residual coefficient within a sub-block can be encoded at the sub-block level within the current block. If there is a non-zero residual coefficient within a sub-block, information for each residual coefficient can be additionally encoded according to the scan order.

[0919] At this time, for sub-blocks with a scan sequence earlier than the sub-block containing the last non-zero residual coefficient, the encoding of the sub-block flag may be omitted. Since the sub-block does not contain a non-zero residual coefficient, the value of the sub-block flag may be considered to be 0.

[0920] In addition, for sub-blocks containing the last non-zero residual coefficient, the encoding of the sub-block flag may be omitted. Since the sub-block must contain a non-zero residual coefficient, the value of the sub-block flag may be considered as 1.

[0921] As another example, the encoding of the location information of the last non-zero residual coefficient can be omitted. If the encoding of the location information of the last non-zero residual coefficient is omitted, the sub-block flag can be encoded for all sub-blocks within the current block.

[0922] In this case, if the scan sequence is determined to be such that the remaining sub-blocks, excluding the last sub-block, do not contain non-zero residual coefficients, it can be understood that the last sub-block must contain non-zero residual coefficients. Accordingly, for the last sub-block, the encoding of the sub-block flag can be omitted and its value can be considered to be 1.

[0923] For the first sub-block within the current block, that is, the sub-block at the top-left position of the current block, encoding / decoding of the sub-block flag can be omitted by assuming that there is always a non-zero residual coefficient. Accordingly, the value of the sub-block flag for the first sub-block within the current block can be derived to 1.

[0924] Information indicating whether the location information of the last non-zero count is encoded may be additionally encoded. If the location information of the last non-zero count is encoded, the value of the information may be set to 1. In this case, the sub-block flag may be encoded starting from the sub-block containing the last non-zero count. On the other hand, if the location information of the last non-zero count is not encoded, the value of the information may be set to 0. In this case, the sub-block flag may be encoded starting from the sub-block with the first scan order.

[0925] If a non-zero residual coefficient exists in the current block, it can be assumed that the first sub-block within the current block necessarily contains a non-zero residual coefficient. Accordingly, the encoding of the sub-block flag indicating whether a non-zero residual coefficient exists in the first sub-block may be omitted.

[0926] Information for encoding residual coefficients may include at least one of a flag indicating whether the residual coefficient has a non-zero value, information indicating the magnitude of the residual coefficient, and information indicating the sign of the residual coefficient.

[0927] Residual coefficients can be encoded according to a predetermined scan order. In this case, the encoding order of the residual coefficients may differ based on whether the transformation has been skipped in the current block. For example, if the transformation has not been skipped in the current block, the residual coefficient located at the bottom right of the sub-block may be encoded first, and the residual coefficient located at the top left may be encoded last. That is, the scan order among the residual coefficients may be determined along a reverse diagonal scan, a reverse horizontal scan, or a reverse vertical scan. Conversely, if the transformation has been skipped in the current block, the residual coefficient located at the top left of the sub-block may be encoded first, and the residual coefficient located at the bottom right may be encoded last. That is, the scan order among the residual coefficients may be determined along a diagonal scan, a horizontal scan, or a vertical scan.

[0928] Alternatively, even if the transformation is skipped in the current block, the scan order between residual coefficients can be determined along an inverse diagonal scan, an inverse horizontal scan, or an inverse vertical scan.

[0929] The scan order of residual coefficients may be predefined in the encoder and decoder. Alternatively, information indicating the scan order of residual coefficients may be encoded and signaled. Alternatively, the scan order may be determined based on at least one of the size, shape, intra-prediction mode, whether a transformation has been skipped, or whether a secondary transformation has been performed on the current block. For example, if a transformation has been skipped on the current block, residual coefficients may be encoded / decoded according to the diagonal (or anti-diagonal) scan order, and if a transformation has not been skipped on the current block, residual coefficients may be encoded / decoded according to the anti-diagonal (or diagonal) scan order.

[0930] Figure 64 is a flowchart illustrating the process of encoding residual coefficients in an encoder.

[0931] First, a flag, sig_coeff_flag, indicating whether the residual coefficient has a non-zero value can be encoded (S6410). If the value of the residual coefficient is 0, the value of the flag sig_coeff_flag can be encoded by setting it to 0. On the other hand, if the value of the residual coefficient is not 0, the value of the flag sig_coeff_flag can be encoded by setting it to 1. If the value of the residual coefficient is not 0, the magnitude information of the residual coefficient can be further encoded (S6420).

[0932] Figure 65 is a flowchart illustrating the encoding process of magnitude information of residual coefficients.

[0933] The absolute value of the residual coefficient can be encoded using at least one abs_level_gtx_flag[N], where N is an integer greater than or equal to 0. The flag abs_level_gtx_flag[N] may indicate whether the absolute value of the residual coefficient is greater than (2N+1). The number of abs_level_gtx_flag[N] used to encode the absolute value of the residual coefficient can be determined based on whether the transformation is skipped in the current block. For example, if the transformation is not skipped in the current block, two abs_level_gtx_flag[N] (N is from 0 to 1) may be used. On the other hand, if the transformation is skipped in the current block, three or more abs_level_gtx_flag[N] (e.g., three, four, or five) may be used. In this embodiment, it is assumed that two abs_level_gtx_flag[N] are used.

[0934] A flag abs_level_gtx_flag[0] indicating whether the absolute value of the residual coefficient is greater than 1 can be encoded (S6510). If the absolute value of the residual coefficient is 1, the value of the flag abs_level_gtx_flag[0] can be encoded by setting it to 0. On the other hand, if the absolute value of the residual coefficient is greater than 1, the value of the flag abs_level_gtx_flag[0] can be encoded by setting it to 1.

[0935] If the absolute value of the residual coefficient is greater than 1, a flag par_level_flag indicating whether the absolute value of the residual coefficient is even or odd can be encoded (S6520). If the absolute value of the residual coefficient is even, the flag par_level_flag can be encoded by setting it to 0. On the other hand, if the absolute value of the residual coefficient is odd, the flag par_level_flag can be encoded by setting it to 1. Alternatively, conversely, if the absolute value of the residual coefficient is even, the flag par_level_flag can be set to 1, and if the absolute value of the residual coefficient is odd, the flag par_level_flag can be set to 0.

[0936] Next, a flag abs_level_gtx_flag[1] indicating whether the absolute value of the residual coefficient is greater than 3 can be encoded (S6530). If the absolute value of the residual coefficient is 3 or less, the value of the flag abs_level_gtx_flag[1] can be set to 0. On the other hand, if the absolute value of the residual coefficient is greater than 3, the value of the flag abs_level_gtx_flag[1] can be set to 1.

[0937] If the absolute value of the residual coefficient is greater than 3, an abs_remainder representing the residual magnitude can be encoded (S6540). The syntax abs_remainder can be derived by differing the absolute value of the residual coefficient by 4 and shifting the value to the right by 1.

[0938] In addition to the abs_level_gtx_flag[0] and abs_level_gtx_flag[1] flags shown in FIG. 65, abs_level_gtx_flag[N], such as abs_level_gtx_flag[2], abs_level_gtx_flag[3], or abs_level_gtx_flag[4], may be additionally encoded. In this case, the abs_level_gtx_flag[N] flag may be additionally encoded / decoded only when the value of abs_level_gtx_flag[N-1] is 1.

[0939] abs_level_gtx_flag[N] can indicate whether the absolute value of the residual coefficient is greater than (2N+1). Finally, if the abs_level_gtx_flag[N] being encoded / decoded is abs_level_gtx_flag[L], abs_remainder can be derived by differing the absolute value of the residual coefficient by 2(L+1) and shifting the result to the right by 1.

[0940] In the example described above, the absolute value of the residual coefficient is encoded using sig_coeff_flag, abs_level_gtx_flag[0], par_level_flag, abs_level_gtx_flag[1], and abs_reminder. As another example, the absolute value of the residual coefficient may be encoded as is. For example, the syntax dec_abs_level representing the absolute value of the residual coefficient may be encoded.

[0941] After encoding the magnitude information of the residual coefficient, a flag coeff_sign_flag indicating the sign of the residual coefficient can be encoded (S6430). A value of 0 for the flag coeff_sign_flag indicates that the residual coefficient is positive. On the other hand, a value of 1 for the flag coeff_sign_flag indicates that the residual coefficient is negative.

[0942] Figure 66 is a flowchart illustrating the process of decoding residual coefficients in a decoder.

[0943] If it is determined that non-zero residual coefficients are included within a sub-block, the residual coefficients can be restored based on a predetermined scan order.

[0944] First, a flag, sig_coeff_flag, indicating whether the residual coefficient has a non-zero value can be decoded (S6610). A value of 0 for the flag sig_coeff_flag indicates that the residual coefficient is zero. On the other hand, a value of 1 for the flag sig_coeff_flag indicates that the residual coefficient is not zero. If the value of the flag sig_coeff_flag is 1, the magnitude information of the residual coefficient can be further decoded (S6620).

[0945] Figure 67 is a diagram showing the decoding process of the magnitude information of the residual coefficient.

[0946] For the sake of convenience of explanation, it is assumed that the residual coefficients are encoded using up to 2 abs_level_gtx_flag[N].

[0947] A flag abs_level_gtx_flag[0] indicating whether the absolute value of the residual coefficient is greater than 1 can be decoded (S6710). A value of 0 for flag abs_level_gtx_flag[0] indicates that the absolute value of the residual coefficient is 1. On the other hand, a value of 1 for flag abs_level_gtx_flag[0] indicates that the absolute value of the residual coefficient is greater than 1.

[0948] When the value of flag abs_level_gtx_flag[0] is 1, flag par_level_flag can be decoded to in...

Claims

1. A step of generating a predicted block of the current block; A step of obtaining the residual coefficients of the current block above; A step of obtaining a residual block of the current block from the above residual coefficients; and Based on the prediction block and the residual block, the step of restoring the current block is included, The above prediction block is obtained by performing intra prediction on the above current block in sub-block units, and An image decoding method characterized in that the intra prediction mode of each sub-block within the current block is determined independently.

2. In Paragraph 1, An image decoding method characterized in that the intra prediction mode of each of the above sub-blocks is obtained based on the histogram of each of the above sub-blocks.

3. In Paragraph 2, A video decoding method characterized by generating a histogram of the current sub-block based on restoration samples included in the previous sub-block of the current sub-block when the current sub-block is not the first sub-block among the said sub-blocks.

4. In Paragraph 2, An image decoding method characterized by decoding information that indicates whether to set the intra prediction mode of the sub-block to be the same as the current block for each of the above sub-blocks.

5. In Paragraph 1, An image decoding method characterized in that the above prediction block is obtained by weighting the sum of a first prediction block obtained by intra prediction and a second prediction block obtained by performing inter prediction on the current block.

6. In Paragraph 5, A video decoding method characterized in that the weights applied to the first prediction block and the second prediction block are set differently on a sub-block basis.

7. In Paragraph 6, An image decoding method characterized in that the weights of each sub-block are determined based on one of a plurality of predefined weight lookup tables.

8. In Paragraph 7, An image decoding method characterized by selecting one of the plurality of weighted lookup tables by referring to the encoding mode of a neighbor block adjacent to the current block.

9. In Paragraph 7, An image decoding method characterized by selecting one of the plurality of weighted lookup tables based on an intra prediction mode derivation method for each of the sub-blocks.

10. In Paragraph 1, An image decoding method characterized in that the above residual coefficients are decoded in units of residual coefficient groups containing a predetermined number of residual coefficients.

11. In Paragraph 1, The above residual coefficients are decoded in units of residual coefficient groups, and An image decoding method characterized in that the first residual coefficient group includes residual coefficients existing up to the position of a residual coefficient with an absolute value of 0.

12. In Paragraph 11, An image decoding method characterized in that the second residual coefficient group includes residual coefficients existing from the position of the residual coefficient with an absolute value of 0 up to the position of the residual coefficient with an absolute value not of 0.

13. In Paragraph 12, An image decoding method characterized by decoding the number information of residual coefficients for each of the first residual coefficient group and the second residual coefficient group.

14. Step of generating a predicted block of the current block; A step of obtaining a residual block of the current block based on the prediction block above; A step of obtaining residual coefficients from the above residual block; and The step of encoding the above residual coefficients is included, The above prediction block is obtained by performing intra prediction on the above current block in sub-block units, and A video encoding method characterized in that the intra prediction mode of each sub-block within the current block is determined independently.

15. A processor for acquiring compressed video data; and It includes a transmission unit that transmits the above-mentioned compressed video data, The above compressed video data is, Step of generating a predicted block of the current block; A step of obtaining a residual block of the current block based on the prediction block above; A step of obtaining residual coefficients from the above residual block; and The step of encoding the above residual coefficients is included, The above prediction block is obtained by performing intra prediction on the above current block in sub-block units, and A device for transmitting compressed video data, characterized in that each of the sub-blocks within the current block is determined independently.