Video signal encoding / decoding method and apparatus therefor

Abstract

This invention provides a method and apparatus for efficiently encoding / decoding residual coefficients when encoding / decoding video signals. [Solution] The method includes analyzing a non-zero flag from the bitstream that indicates whether the residual coefficient is non-zero, and if the non-zero flag indicates that the residual coefficient is not non-zero, analyzing absolute value information from the bitstream. The absolute value information is used to determine the absolute value of the residual coefficient. The method also includes determining the absolute value of the residual coefficient based on the absolute value information, where the absolute value information includes a residual coefficient comparison flag that indicates whether the residual coefficient exceeds a first value, and if the residual coefficient exceeds a first value, further analyzing a parity flag from the bitstream.

Description

Technical Field

[0001] The present invention relates to a video signal encoding / decoding method and a device therefor.

Background Art

[0002] With the trend of increasingly larger display panels, increasingly high-quality video services are being demanded. The biggest problem with high-resolution video services is the significant increase in data volume. To solve this problem, research to improve the video compression ratio has been actively carried out. As a typical example, in 2009, MPEG (Motion Picture Experts Group) and VCEG (Video Coding Experts Group) under the umbrella of ITU-T (International Telecommunication Union-Telecommunication) established JCT-VC (Joint Collaborative Team on Video Coding). JCT-VC proposed HEVC (High Efficiency Video Coding), a video compression standard having a compression performance approximately twice that of H.264 / AVC, which was approved on January 25, 2013. With the rapid development of high-resolution video services, the performance of HEVC has gradually shown its limitations.

Summary of the Invention

Problems to be Solved by the Invention

[0003] An object of the present invention is to provide a method for encoding / decoding residual coefficients and a device therefor when encoding / decoding a video signal.

[0004] An object of the present invention is to provide a method for encoding / decoding residual coefficients and a device therefor using a flag that compares the magnitude of a residual coefficient with a threshold when encoding / decoding a video signal.

[0005] The present invention aims to provide a method for encoding / decoding residual coefficients using a flag indicating whether the residual coefficients are even or odd when encoding / decoding a video signal, and apparatus for such a method.

[0006] The technical problems that this invention aims to solve are not limited to those described above, and those skilled in the art to which this invention belongs can clearly understand other technical problems not mentioned in this invention through the following description. [Means for solving the problem]

[0007] The video signal decoding method of the present invention includes: analyzing a non-zero flag from a bitstream indicating whether or not the residual coefficient is non-zero; if the non-zero flag indicates that the residual coefficient is not non-zero, analyzing absolute value information from the bitstream, wherein the absolute value information is used to determine the absolute value of the residual coefficient; and determining the absolute value of the residual coefficient based on the absolute value information. Here, the absolute value information may include a residual coefficient comparison flag indicating whether or not the residual coefficient exceeds a first value, and only if the residual coefficient exceeds a first value, a parity flag is further analyzed from the bitstream.

[0008] The video signal encoding method of the present invention may include encoding a non-zero flag indicating whether or not the residual coefficient is non-zero, and, if the residual coefficient is not non-zero, encoding absolute value information, the absolute value information is used to determine the absolute value of the residual coefficient. The absolute value information may include a residual coefficient comparison flag indicating whether or not the residual coefficient exceeds a first value, and only if the residual coefficient exceeds a first value, a parity flag of the residual coefficient is further encoded.

[0009] In the video signal decoding / encoding method of the present invention, the parity flag may indicate whether the value of the residual coefficient is even or odd.

[0010] In the video signal decoding / encoding method of the present invention, if the residual coefficient exceeds the first value, the first adjusted residual coefficient comparison flag can be further analyzed, and the first adjusted residual coefficient comparison flag indicates whether the adjusted residual coefficient derived by shifting the residual coefficient one bit to the right exceeds the second value.

[0011] In the video signal decoding / encoding method of the present invention, if the adjustment residual coefficient is less than or equal to the second value, the residual coefficient can be determined to be 2N or 2N+1 according to the value of the parity flag.

[0012] In the video signal decoding / encoding method of the present invention, if the adjustment residual coefficient exceeds the second value, the second adjustment residual coefficient comparison flag can be further analyzed, and the second adjustment residual coefficient comparison flag indicates whether or not the adjustment residual coefficient exceeds the third value.

[0013] In the video signal decoding / encoding method of the present invention, if the adjustment residual coefficient exceeds the second value, the residual value information can be further analyzed, and the residual value information is the value obtained by subtracting the second value from the adjustment residual coefficient.

[0014] The above-mentioned brief summary of the features of the present invention is merely an exemplary embodiment of the invention as described in the detailed description below, and does not limit the scope of the invention. [Effects of the Invention]

[0015] According to the present invention, residual coefficients can be efficiently encoded / decoded.

[0016] According to the present invention, residual coefficients can be efficiently encoded / decoded by using a flag that compares the magnitude of the residual coefficient with a threshold.

[0017] According to the present invention, residual coefficients can be efficiently encoded / decoded by using a flag that indicates whether the residual coefficient is even or odd.

[0018] The effects obtained by the present invention are not limited to the above effects, and those skilled in the art to which the present invention pertains can clearly understand other effects not mentioned in the present invention through the following description.

Brief Description of Drawings

[0019] [Figure 1] It is a block diagram of a video encoder according to an embodiment of the present invention. [Figure 2] It is a block diagram of a video decoder according to an embodiment of the present invention. [Figure 3] It is a diagram showing a basic encoding tree unit according to an embodiment of the present invention. [Figure 4] It is a diagram showing various division forms of an encoding block. [Figure 5] It is a diagram showing an example of division of an encoding tree unit. [Figure 6] It is a diagram showing an example where a block smaller than an encoding tree unit of a preset size appears at an image boundary. [Figure 7] It is a diagram showing an example of performing quadtree division on a boundary block of an irregular boundary. [Figure 8] It is a diagram showing an example of performing quadtree division on a block adjacent to an image boundary. [Figure 9] It is a diagram showing a division pattern of a block adjacent to an image boundary. [Figure 10] It is a diagram showing an encoding pattern of a block adjacent to an image boundary. [Figure 11] It is a flowchart of an inter prediction method according to an embodiment of the present invention. [Figure 12] It is a flowchart of a process for deriving motion information of a current block in merge mode. [Figure 13] It is a diagram showing candidate blocks for deriving merge candidates. [Figure 14] It is a diagram showing the position of a reference sample. [Figure 15]This diagram shows candidate blocks for deriving merge candidates. [Figure 16] This figure shows an example of changing the position of a reference sample. [Figure 17] This figure shows an example of changing the position of a reference sample. [Figure 18] This diagram illustrates an example of updating the inter-domain motion information table. [Figure 19] This figure shows an example of updating the inter-region merge candidate table. [Figure 20] This figure shows an example of how the index of stored inter-region merge candidates is updated. [Figure 21] This diagram shows the location of the representative subblocks. [Figure 22] This figure shows an example of generating an inter-domain motion information table for each inter-prediction mode. [Figure 23] This figure shows an example of adding inter-region merge candidates included in the long-term movement information table to the merge candidate list. [Figure 24] This figure shows an example of performing redundancy checks on only a portion of the merge candidates. [Figure 25] This figure shows an example of omitting redundancy checks for specific merge candidates. [Figure 26] This diagram shows an example where candidate blocks located in the same parallel merge region as the current block are configured to be unavailable as merge candidates. [Figure 27] This is a diagram showing a temporary movement information table. [Figure 28] This figure shows an example of merging the inter-domain motion information table and the temporary motion information table. [Figure 29] This is a flowchart of an intra-prediction method according to an embodiment of the present invention. [Figure 30] This figure shows the reference samples included in each reference sample line. [Figure 31] This figure shows the intra-prediction mode. [Figure 32]This figure shows an example of a one-dimensional array in which reference samples are arranged in a single column. [Figure 33] This figure shows an example of a one-dimensional array in which reference samples are arranged in a single column. [Figure 34] This figure shows the angle formed between the directional intra-prediction mode and a straight line parallel to the x-axis. [Figure 35] This figure shows an example of obtaining a predictive sample when the current block is not square. [Figure 36] This figure shows the wide-angle intra-predictive mode. [Figure 37] This is a diagram showing the application of PDPC. [Figure 38] This diagram shows the subblocks where the quadratic transformation is performed. [Figure 39] This diagram shows the subblocks where the quadratic transformation is performed. [Figure 40] This diagram illustrates an example of determining the transformation type of the current block. [Figure 41] This is a flowchart showing a method for encoding residual coefficients. [Figure 42] This figure shows the arrangement order of residual coefficients under different scanning orders. [Figure 43] This figure shows the arrangement order of residual coefficients under different scanning orders. [Figure 44] This figure shows an example of encoding the position of the last non-zero coefficient. [Figure 45] This is a flowchart of the process for encoding the absolute value of the residual coefficient. [Figure 46] This is a flowchart of the process for encoding the absolute value of the residual coefficient. [Figure 47] This is a flowchart of the process for encoding the absolute value of the residual coefficient. [Figure 48] This is a flowchart of the process for determining block strength. [Figure 49] This is a diagram showing predefined filter candidates. [Modes for carrying out the invention]

[0020] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0021] Video encoding and decoding are performed in blocks. For example, encoding / decoding processes such as transformation, quantization, prediction, in-loop filtering, or decompression can be performed on an encoding block, a transform block, or a prediction block.

[0022] Hereafter, the block being encoded / decoded will be referred to as the "current block." For example, depending on the current encoding / decoding process, the current block may represent an encoded block, a transformed block, or a predicted block.

[0023] Furthermore, as used herein, the term "unit" can be understood to refer to a basic unit for performing a particular encoding / decoding process, and "block" can be understood to refer to a sample array of a given size. Unless otherwise specified, "block" and "unit" can be used interchangeably. For example, in the embodiments described later, encoding blocks and encoding units can be understood to be mutually equivalent.

[0024] Figure 1 is a block diagram of a video encoder according to an embodiment of the present invention.

[0025] Referring to Figure 1, the video encoding device 100 may include an image splitting unit 110, prediction units 120, 125, a conversion unit 130, a quantization unit 135, a rearrangement unit 160, an entropy encoding unit 165, an inverse quantization unit 140, an inverse conversion unit 145, a filter unit 150, and a memory 155.

[0026] Each component shown in Figure 1 is illustrated independently to illustrate a different characteristic function in a video encoding device, and does not mean that each component consists of separate hardware or a single software component. In other words, for each component, for the sake of explanation, at least two components may be combined into one component, or one component may be divided into multiple components to perform its function, and embodiments that integrate these components and embodiments that separate them are also covered by the present invention, as long as they do not deviate from the essence of the invention.

[0027] Furthermore, some of the components may not be necessary to perform the essential functions of the present invention, but may be optional structural elements that merely improve performance. The present invention may include only the components necessary to realize the essence of the invention, other than components that merely improve performance, and structures that include only the necessary structural elements other than components that merely improve performance are also within the scope of protection of the present invention.

[0028] The image splitting unit 110 can split an input image into at least one processing unit. In this case, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). The image splitting unit 110 can split a single image into a combination of coding units, prediction units, and transform units, and then select one combination of coding units, prediction units, and transform units based on a predetermined standard (such as a cost function) to encode the image.

[0029] For example, a single image can be divided into multiple coding units. To divide an image into coding units, a recursive tree structure such as a quad tree structure can be used, with one video or the largest coding unit used as the root. Each coding unit that is divided into other coding units can have subnotes equal to the number of coding units it is divided into. Coding units that cannot be divided further according to certain constraints become leaf nodes. In other words, assuming that only square division is possible for a single coding unit, one coding unit can be divided into up to four other coding units.

[0030] In the embodiments of the present invention described below, the encoding unit may mean a unit that performs encoding, or a unit that performs decoding.

[0031] A prediction unit within a coding unit can be divided into at least one of the same-sized squares and rectangles, or within a coding unit, one prediction unit can be divided such that any of the divided prediction units has a different shape and / or size from another prediction unit.

[0032] If the prediction unit that performs intraprediction based on the coding unit is not the smallest coding unit, intraprediction can be performed without dividing it into multiple prediction units N×N.

[0033] The prediction units 120 and 125 may include an inter-prediction unit 120 that performs inter-prediction and an intra-prediction unit 125 that performs intra-prediction. For the prediction unit, it is possible to decide whether to perform inter-prediction or intra-prediction, and to determine specific information (e.g., intra-prediction mode, motion vector, reference image, etc.) based on each prediction method. In this case, the processing unit that performs the prediction may be different from the processing unit that determines the prediction method and specific content. For example, the prediction unit may determine the prediction method and prediction mode, and the conversion unit may perform the prediction. The residual value (residual block) between the generated prediction block and the original block can be input to the conversion unit 130. In addition, prediction mode information, motion vector information, etc. used for prediction can be encoded together with the residual value in the entropy encoding unit 165 and transmitted to the decoder. When using a specific encoding mode, the original block can also be directly encoded and transmitted to the decoder without generating predicted blocks by the prediction units 120 and 125.

[0034] The interpretation unit 120 can predict prediction units based on information from at least one image that precedes or follows the current image, and in some cases, it can also predict prediction units based on information from a portion of an encoded region within the current image. The interpretation unit 120 may include a reference image interpolation unit, a motion prediction unit, and a motion compensation unit.

[0035] The reference image interpolation unit receives reference image information from memory 155 and can generate pixel information of integer pixels or smaller from the reference image. For luminance pixels, a DCT-based 8-tap interpolation filter with different filter coefficients can be used to generate pixel information of integer pixels or smaller in 1 / 4 pixel units. For saturation signals, a DCT-based 4-tap interpolation filter with different filter coefficients can be used to generate pixel information of integer pixels or smaller in 1 / 8 pixel units.

[0036] The motion prediction unit can perform motion prediction based on the reference image interpolated by the reference image interpolation unit. Various methods are possible for calculating the motion vector, including the Full search-based Block Matching Algorithm (FBMA), the Three-Step Search (TSS), and the New Three-Step Search Algorithm (NTS). The motion vector can have motion vector values ​​in units of 1 / 2 pixel or 1 / 4 pixel based on the interpolated pixels. The motion prediction unit can predict the current prediction unit by employing different motion prediction methods. Various motion prediction methods are available, including the Skip method, Merge method, Advanced Motion Vector Prediction (AMVP), and Intra Block Copy method.

[0037] The intra-prediction unit 125 can generate prediction units based on reference pixel information around the current block, which is pixel information within the current image. If the adjacent block of the current prediction unit is a block where inter-prediction has been performed, and the reference pixel is a pixel where inter-prediction has been performed, the reference pixel included in the inter-predicted block can be used as the reference pixel information for the surrounding intra-predicted blocks. In other words, if a reference pixel is unavailable, at least one of the available reference pixels can be used to replace the unavailable reference pixel information.

[0038] In intra-prediction, the prediction mode may include a directional prediction mode that uses reference pixel information depending on the prediction direction, and a non-directional mode that does not use directional information when performing prediction. The mode for predicting luminance information and the mode for predicting saturation information may be different, and to predict saturation information, the intra-prediction mode information used to predict luminance information or the predicted luminance signal information can be utilized.

[0039] When performing intraprediction, if the size of the prediction unit is the same as the size of the transformation unit, intraprediction can be performed on the prediction unit based on the pixels to the left, the upper left, and the upper side of the prediction unit. However, when performing intraprediction, if the size of the prediction unit is different from the size of the transformation unit, intraprediction can be performed using reference pixels based on the transformation unit. In addition, N×N partitioning intraprediction can be used only for the minimum coding unit.

[0040] The intra-prediction method can generate a prediction block after applying an Adaptive Intra Smoothing (AIS) filter to a reference pixel, depending on the prediction mode. The type of AIS filter applied to the reference pixel may vary. To execute the intra-prediction method, the intra-prediction mode of the current prediction unit can be predicted from the intra-prediction modes of prediction units surrounding the current prediction unit. When predicting the prediction mode of the current prediction unit using mode information predicted from surrounding prediction units, if the intra-prediction modes of the current prediction unit and the surrounding prediction units are the same, information indicating that the prediction modes of the current prediction unit and the surrounding prediction units are the same can be transmitted using predetermined flag information. If the prediction modes of the current prediction unit and the surrounding prediction units are different, entropy coding can be performed to encode the prediction mode information of the current block.

[0041] Furthermore, residual blocks containing residual value information, which is the difference between the prediction unit and the original block of the prediction unit, can be generated based on the prediction units generated by the prediction units 120 and 125. The generated residual blocks can be input to the conversion unit 130.

[0042] The transformation unit 130 can transform the original block and the residual block containing residual coefficient information of the prediction units generated by the prediction units 120 and 125, using transformation methods such as discrete cosine transform (DCT), discrete sine transform (DST), or KL transform (KLT). Whether to apply DCT, DST, or KLT to transform the residual block can be determined according to the intra-prediction mode information of the prediction unit used to generate the residual block.

[0043] The quantization unit 135 can quantize the values ​​converted to the frequency domain by the conversion unit 130. The quantization coefficients can be converted according to the importance of blocks or videos. The values ​​calculated by the quantization unit 135 can be provided to the inverse quantization unit 140 and the rearrangement unit 160.

[0044] The rearrangement unit 160 can perform a rearrangement of coefficient values ​​for the quantized residual values.

[0045] The rearrangement unit 160 can convert two-dimensional block shape coefficients into one-dimensional vector shapes using a coefficient scanning method. For example, the rearrangement unit 160 can use a zig-zag scan method to scan from DC coefficients to high-frequency coefficients and convert them into one-dimensional vector shapes. Depending on the size of the conversion unit and the intra-prediction mode, vertical scanning, which scans the two-dimensional block shape coefficients in the column direction, and horizontal scanning, which scans the two-dimensional block shape coefficients in the row direction, can also be used instead of zig-zag scanning. In other words, depending on the size of the conversion unit and the intra-prediction mode, it is possible to determine which scanning method to use from zig-zag scanning, vertical scanning, and horizontal scanning.

[0046] The entropy coding unit 165 can perform entropy coding based on the value calculated by the rearrangement unit 160. For example, various coding methods can be used for entropy coding, such as Exponential Golomb coding, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

[0047] The entropy coding unit 165 can encode various information from the reordering unit 160 and the prediction units 120 and 125, such as residual coefficient information and block type information of the coding units, prediction mode information, divided unit information, prediction unit information and transmission unit information, motion vector information, reference frame information, block interpolation information, and filtering information.

[0048] The entropy coding unit 165 can perform entropy coding on the coefficient values ​​of the coding units input from the rearrangement unit 160.

[0049] The inverse quantization unit 140 and the inverse transformation unit 145 inverse quantize multiple values ​​quantized by the quantization unit 135 and inversely transform the values ​​transformed by the transformation unit 130. The residual values ​​generated by the inverse quantization unit 140 and the inverse transformation unit 145 can be merged with the prediction units predicted by the motion prediction unit, motion compensation unit, and intra prediction unit included in the prediction units 120 and 125 to generate a reconstructed block.

[0050] The filter section 150 may include at least one of the following: a deblocking filter, an offset correction section, and an adaptive loop filter (ALF).

[0051] A deblocking filter can remove block distortion that occurs at the boundaries between blocks in the restored image. To determine whether or not to deblock, it is possible to decide whether or not to apply a deblocking filter to the current block based on the pixels contained in some columns or rows within the block. When applying a deblocking filter to a block, a strong filter or a weak filter can be applied depending on the required deblocking filtering strength. In addition, if vertical filtering and horizontal filtering are performed in the process of using a deblocking filter, horizontal filtering and vertical filtering can be processed in parallel.

[0052] The offset correction unit can correct the offset from the original video on a pixel-by-pixel basis for the deblocked video. To perform offset correction on a specific image, one can divide the pixels contained in the video into a certain number of regions, determine the region to be offset, and apply the offset to the corresponding region, or apply the offset while considering the edge information of each pixel.

[0053] Adaptive Loop Filtering (ALF) can be performed based on a comparison of the filtered restored image with the original video. By dividing the pixels in the video into predetermined groups and then determining a single filter to be used for each group, differential filtering can be performed on each group. Information regarding whether or not to apply ALF, along with luminance signals, can be transmitted by a Coding Unit (CU), and the shape and filter coefficients of the ALF filter applied may differ depending on the block. Alternatively, the same type of ALF filter (fixed type) can be applied regardless of the characteristics of the block being processed.

[0054] The memory 155 can store the restored blocks or images calculated by the filter unit 150, and can provide the stored restored blocks or images to the prediction units 120 and 125 when performing interpretation.

[0055] Figure 2 is a block diagram of a video decoder according to an embodiment of the present invention.

[0056] Referring to Figure 2, the video decoder 200 may include an entropy decoding unit 210, a rearrangement unit 215, an inverse quantization unit 220, an inverse transform unit 225, a prediction unit 230, a prediction unit 235, a filter unit 240, and a memory 245.

[0057] When a video bitstream is input to a video encoder, the input bitstream can be decoded in the reverse steps of the video encoder.

[0058] The entropy decoding unit 210 can perform entropy decoding in the reverse steps of the steps performed by the entropy coding unit of the video encoder. For example, various methods such as Exponential Golomb coding, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) can be applied, corresponding to the methods performed by the video encoder.

[0059] The entropy decoding unit 210 can decode information regarding intra-predictions and inter-predictions performed by the encoder.

[0060] The re-sorting unit 215 can perform sorting based on the method by which the encoding unit sorts the bitstream that has been entropy-decoded by the entropy decoding unit 210. Multiple coefficients represented in the form of one-dimensional vectors can be restored to two-dimensional block-shaped coefficients and sorted again. The re-sorting unit 215 receives information about the coefficient scan performed by the encoding unit and can perform sorting by a reverse scan method based on the scan order performed by the corresponding encoding unit.

[0061] The inverse quantization unit 220 can perform inverse quantization based on the quantization parameters provided by the encoder and the coefficient values ​​of the rearranged blocks.

[0062] With respect to the quantization results performed by the video encoder, the inverse transformer 225 can perform inverse transforms (i.e., inverse DCT, inverse DST, and inverse KLT) with respect to the transforms performed by the transformer, i.e., DCT, DST, and KLT. The inverse transforms can be performed based on the transmission units determined by the video encoder. In the inverse transformer 225 of the video decoder, the transform method (e.g., DCT, DST, KLT) can be selectively performed based on multiple pieces of information, such as the prediction method, the current block size, and the prediction direction.

[0063] The prediction units 230 and 235 can generate prediction blocks based on information about the generation of prediction blocks provided by the entropy decoding unit 210 and information about previously decoded blocks or images provided by the memory 245.

[0064] As mentioned above, when performing intraprediction in a manner similar to that of a video encoder, if the size of the prediction unit and the size of the transform unit are the same, intraprediction can be performed on the prediction unit based on the pixels to the left, the upper left, and the upper side of the prediction unit. If the size of the prediction unit and the size of the transform unit are different when performing intraprediction, intraprediction can be performed using reference pixels based on the transform unit. In addition, N×N partitioned intraprediction can be used only for the smallest coding unit.

[0065] The prediction units 230 and 235 may include a prediction unit discrimination unit, an inter-prediction unit, and an intra-prediction unit. The prediction unit discrimination unit receives various information such as prediction unit information input from the entropy decoding unit 210, prediction mode information of the intra-prediction method, and motion prediction-related information of the inter-prediction method, and can determine whether a prediction unit performs inter-prediction or intra-prediction by classifying the prediction unit according to the current coding unit. The inter-prediction unit 230 can perform inter-prediction for the current prediction unit based on information contained in at least one image that precedes or follows the current image to which the current prediction unit belongs, using the information necessary for inter-prediction of the current prediction unit provided by the video encoder. Alternatively, it can perform inter-prediction based on information from a portion of the region recovered within the current image to which the current prediction unit belongs.

[0066] To perform interpretation, it is possible to determine, based on the encoded unit, which mode of motion prediction method the prediction unit contained within the corresponding encoded unit is using: Skip Mode, Merge Mode, Advanced Motion Vector Prediction Mode (AMVP Mode), or Intrablock Copy Mode.

[0067] The intra-prediction unit 235 can generate prediction blocks based on pixel information in the current image. If the prediction unit is the same prediction unit that performed intra-prediction, it can perform intra-prediction based on the intra-prediction mode information of the prediction unit provided by the video encoder. The intra-prediction unit 235 may include an Adaptive Intra Smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter is an element that performs filtering on the reference pixels of the current block and can determine whether or not to apply the filter according to the prediction mode of the current prediction unit. Using the prediction mode of the prediction unit and the AIS filter information provided by the video encoder, AIS filtering can be performed on the reference pixels of the current block. If the prediction mode of the current block is a mode that does not perform AIS filtering, the AIS filter may not be applied.

[0068] If the prediction mode of the prediction unit is a prediction unit that performs intra-prediction based on the pixel values ​​of interpolated reference pixels, the reference pixel interpolation unit can generate reference pixels of integer or fractional values ​​by interpolating the reference pixels. If the prediction mode of the current prediction unit is a prediction mode that generates prediction blocks without interpolating reference pixels, interpolation of reference pixels is not required. If the prediction mode of the current block is DC mode, the DC filter can generate prediction blocks by filtering.

[0069] The restored block or image can be provided to the filter unit 240. The filter unit 240 may include a deblocking filter, an offset correction unit, and an ALF.

[0070] The video decoder can receive information from the video encoder regarding whether or not to apply a deblocking filter to a block or image, and whether to apply strong or weak filtering when applying the deblocking filter. The video decoder can receive relevant information about the deblocking filter provided by the video encoder from the deblocking filter of the video decoder, and the video decoder can perform deblocking filtering on the corresponding block.

[0071] The offset correction unit can perform offset correction on the restored video based on information such as the type and amount of offset correction used for the video during encoding.

[0072] Based on information provided by the encoder, such as whether or not to apply ALF and ALF coefficient information, the ALF can be applied to the encoding unit. Such ALF information can be provided by being included in a specific set of parameters.

[0073] Memory 245 stores the restored image or block, allows the image or block to be used as a reference image or reference block, and can provide the restored image to the output unit.

[0074] Figure 3 shows a basic coding tree unit according to an embodiment of the present invention.

[0075] The largest coding block can be defined as a coding tree block. A single image can be divided into multiple coding tree units (CTUs). A coding tree unit is the largest coding unit and can also be called a largeest coding unit (LCU). Figure 3 shows an example where a single image is divided into multiple coding tree units.

[0076] The size of an encoded tree unit can be defined at the image level or the sequence level. Therefore, information indicating the size of an encoded tree unit can be signaled via an image parameter set or a sequence parameter set.

[0077] For example, the size of the coding tree unit for all images in a sequence can be set to 128x128. Alternatively, the coding tree unit size can be determined at the image level, either 128x128 or 256x256. For instance, the coding tree unit size for the first image can be set to 128x128, and for the second image, it can be set to 256x256.

[0078] Coding blocks can be generated by dividing coding tree units. Coding blocks represent basic units for performing coding / decoding processes. For example, prediction or transformation can be performed according to different coding blocks, or a predictive coding mode can be determined according to different coding blocks. Here, the predictive coding mode represents the method for generating the predicted image. For example, predictive coding modes may include intra-prediction, inter-prediction, current picture referencing (CPR), or intra-block copy (IBC), or combined prediction. For coding blocks, a predictive block can be generated for the coding block using at least one predictive coding mode from intra-prediction, inter-prediction, current picture referencing, or combined prediction.

[0079] Information indicating the current block's predictive coding mode can be signaled via a bitstream. For example, this information may be a one-bit flag indicating whether the predictive coding mode is intra-mode or inter-predictive mode. The current image reference or composite prediction can only be used if it is determined that the current block's predictive coding mode is inter-predictive mode.

[0080] The current image reference is used to set the current image as the reference image and to obtain the predicted block for the current block from the encoded / decoded region within the current image. Here, the current image means the image containing the current block. Information indicating whether or not to apply the current image reference to the current block can be signaled via a bitstream. For example, such information may be a 1-bit flag. If the flag is true, the predicted encoding mode for the current block can be determined as the current image reference; if the flag is false, the prediction mode for the current block can be determined as inter-prediction.

[0081] Alternatively, the predictive coding mode for the current block can be determined based on a reference image index. For example, if the reference image index points to the current image, the predictive coding mode for the current block can be determined as the current image reference. If the reference image index points to an image other than the current image, the predictive coding mode for the current block can be determined as the inter-prediction. In other words, the current image reference is a prediction method that uses information from the coded / decoded region within the current image, while the inter-prediction is a prediction method that uses information from other coded / decoded images.

[0082] A composite prediction refers to an encoding mode that combines two or more of the following: intra-prediction, inter-prediction, and current image reference. For example, when applying a composite prediction, a first prediction block can be generated based on either intra-prediction, inter-prediction, or current image reference, and a second prediction block can be generated based on the other one. When a first and second prediction block are generated, a final prediction block can be generated through averaging or additive averaging of the first and second prediction blocks. Information indicating whether or not to apply a composite prediction can be signaled via a bitstream. This information may be a 1-bit flag.

[0083] Figure 4 shows various partitioning configurations for encoded blocks.

[0084] A coded block can be divided into multiple coded blocks based on quadtree, binary, or ternary tree partitioning. A coded block that has been divided can also be further divided into multiple coded blocks based on quadtree, binary, or ternary tree partitioning.

[0085] Quadrarrow partitioning refers to a partitioning technique that divides the current block into four blocks. As a result of quadrarrow partitioning, the current block can be divided into four square partitions (see "SPLIT_QT" in Figure 4(a)).

[0086] Binary tree partitioning refers to a partitioning technique that divides the current block into two blocks. Dividing the current block into two blocks along a vertical direction (i.e., using vertical lines that cross the current block) can be called vertical binary tree partitioning, and dividing the current block into two blocks along a horizontal direction (i.e., using horizontal lines that cross the current block) can be called horizontal binary tree partitioning. Binary tree partitioning allows the current block to be divided into two non-square partitions. "SPLIT_BT_VER" in Figure 4(b) shows the result of vertical binary tree partitioning, and "SPLIT_BT_HOR" in Figure 4(c) shows the result of horizontal binary tree partitioning.

[0087] A ternary tree partitioning is a partitioning technique that divides the current block into three blocks. Dividing the current block into three blocks along the vertical direction (i.e., using two vertical lines that cross the current block) can be called a vertical ternary tree partitioning, and dividing the current block into three blocks along the horizontal direction (i.e., using two horizontal lines that cross the current block) can be called a horizontal ternary tree partitioning. A ternary tree partitioning can divide the current block into three non-square partitions. In this case, the width / height of the partition located in the center of the current block can be twice the width / height of the other partitions. "SPLIT_TT_VER" in Figure 4(d) shows the result of a vertical ternary tree partitioning, and "SPLIT_TT_HOR" in Figure 4(e) shows the result of a horizontal ternary tree partitioning.

[0088] The number of divisions in an encoded tree unit can be defined as the partitioning depth. The maximum partitioning depth of an encoded tree unit can be determined at the sequence or image level. This means that the maximum partitioning depth of an encoded tree unit can differ depending on the sequence or image.

[0089] Alternatively, the maximum partition depth for each of the multiple partitioning methods can be determined individually. For example, the maximum partition depth allowed for quadtree partitioning may be different from the maximum partition depth allowed for binary and / or ternary partitioning.

[0090] The encoder can signal information via the bitstream indicating at least one of the partition shape or partition depth of the current block. The decoder can determine the partition shape and partition depth of the coded tree unit based on the information analyzed from the bitstream.

[0091] Figure 5 shows an example of a coding tree unit partitioning.

[0092] The process of dividing an encoded block using partitioning techniques such as quadtree partitioning, binary tree partitioning, and / or ternary tree partitioning can be called multi-tree partitioning.

[0093] Multiple coded blocks generated by applying a multi-tree partition to a coded block can be called subcoded blocks. If the partitioning depth of a coded block is k, the partitioning depth of the multiple subcoded blocks is set to k+1.

[0094] On the other hand, for multiple coding blocks with a division depth of k+1, a coding block with a division depth of k can be called a higher-level coding block.

[0095] The partition type of the current coded block can be determined based on at least one of the partition shape of the upper coded block or the partition type of the adjacent coded blocks. Here, adjacent coded blocks are adjacent to the current coded block and may include at least one of the upper adjacent blocks, left adjacent blocks, or adjacent blocks adjacent to the upper left corner of the current coded block. Here, the partition type may include at least one of whether or not to partition into a quadtree, whether or not to partition into a binary tree, the direction of the binary tree partition, whether or not to partition into a ternary tree, or the direction of the ternary tree partition.

[0096] To determine the partition shape of the encoded block, information indicating whether or not the encoded block should be partitioned can be signaled via a bitstream. This information is a 1-bit flag "split_cu_flag", and if the current flag is true, the encoded block can be partitioned using a multi-tree partitioning method.

[0097] If "split_cu_flag" is true, information indicating whether or not the encoded block is quadtree split can be signaled via the bitstream. If the information is a 1-bit flag "split_qt_flag" and the flag is true, the encoded block can be split into four blocks.

[0098] For example, the example shown in Figure 5 demonstrates that a coding tree unit generates four coding blocks with a partition depth of 1 through quadtree partitioning. It also shows that the first and fourth coding blocks among the four coding blocks generated as a result of the quadtree partitioning are subjected to quadtree partitioning again. As a result, four coding blocks with a partition depth of 2 can be generated.

[0099] Furthermore, by applying a quadtree partition again to an encoded block with a partition depth of 2, an encoded block with a partition depth of 3 can be generated.

[0100] If a quadtree split is not applied to an encoded block, it is possible to determine whether to perform a binary tree split or a ternary tree split on the encoded block by considering at least one of the following: the size of the encoded block, whether the encoded block is located at an image boundary, the maximum split depth, or the split shape of adjacent blocks. If it is determined that a binary tree split or a ternary tree split should be performed on the encoded block, information indicating the split direction can be signaled via the bitstream. This information may be a 1-bit flag "mtt_split_cu_vertical_flag". Based on this flag, it is possible to determine whether the split direction is vertical or horizontal. Furthermore, information indicating whether a binary tree split or a ternary tree split is applied to the encoded block can be signaled via the bitstream. This information may be a 1-bit flag "mtt_split_cu_binary_flag". Based on this flag, it is possible to determine whether to apply a binary tree split or a ternary tree split to the encoded block.

[0101] For example, in the example shown in Figure 5, a vertical binary tree partition is applied to an encoded block with a partitioning depth of 1. As a result of the partitioning, a vertical ternary tree partition is applied to the left-hand encoded block, and a vertical binary tree partition is applied to the right-hand encoded block.

[0102] When an image is divided into encoded tree units, blocks smaller than a predefined size may exist in the region adjacent to the right or lower boundary of the image. If such blocks are encoded tree units, then encoded tree units smaller than the predefined size can be generated at the right or lower boundary of the image. In this case, the size of the encoded tree unit can be determined based on information signaled via a sequence parameter set or an image parameter set.

[0103] Figure 6 shows an example where blocks smaller than the preset size of the encoded tree unit appear at the image boundary.

[0104] As shown in the example in Figure 6, when a 1292×1080 image is divided into 128×128 encoded tree units, blocks smaller than 128×128 will exist at the right and bottom boundaries of the image. In the embodiments described later, blocks of encoded tree units smaller than the predefined size generated at the image boundaries are called unstructured boundary blocks.

[0105] For boundary blocks of undefined boundaries, only predefined partitioning methods are permitted. Here, a predefined partitioning method may include at least one of quadtree partitioning, ternary partitioning, or binary partitioning.

[0106] For example, for boundary blocks of unstructured boundaries, only quadtree splitting may be permitted. In this case, quadtree splitting can be repeated until the image boundary block reaches the minimum quadtree splitting size. Here, the minimum quadtree splitting size can be predefined in the encoder and decoder. Alternatively, information indicating the minimum quadtree splitting size can be signaled via the bitstream.

[0107] Figure 7 shows an example of performing a quadtree partition on a boundary block with an unstructured boundary. For ease of explanation, we assume that the minimum quadtree partition size is 4x4.

[0108] The division of a non-standard boundary into boundary blocks can be performed based on square blocks. These square blocks can be derived based on the greater of the width or height of the boundary blocks of the non-standard boundary. For example, a power series of 2 that is greater than and closest to the reference value can be considered the side length of a square block. For instance, the 12×20 block shown in Figure 7 can be considered to belong to a 32×32 block, and the division results of the 32×32 can be applied to the 12×20 block.

[0109] When a quadtree partition is performed on a 12x20 block, the block can be divided into a 12x16 block and a 12x4 block. When a quadtree partition is performed again on each of these, the 12x16 block is divided into two 8x8 blocks and two 4x8 blocks, and the 12x4 block is divided into an 8x4 block and a 4x4 block.

[0110] Performing another quadtree split on the 4x8 block located at the image boundary will generate two 4x4 blocks. Similarly, performing another quadtree split on the 8x4 block located at the image boundary will generate two 4x4 blocks.

[0111] Alternatively, a binary tree partition can be performed if at least one of the block's width or height is equal to or less than the minimum quadtree partition size. Here, the minimum quadtree partition size can refer to either the minimum quadtree partition width or the minimum quadtree partition height. For example, if the minimum quadtree partition size is 4x4, then the minimum quadtree partition width and minimum quadtree partition height could be 4.

[0112] In this case, if the width of the block is the same as or smaller than the minimum quadtree partition size, a vertical binary partition can be performed, and if the height of the block is the same as or smaller than the minimum quadtree partition height, a horizontal binary partition can be performed.

[0113] On the other hand, if the width or height of a block is greater than the minimum quadtree partition size, a quadtree partition can be performed. For example, if the top-right and bottom-left positions of a block are outside the image, and the width or height of the block is greater than the minimum quadtree partition size, a quadtree partition can be applied to the corresponding block.

[0114] Figure 8 shows an example of performing a quadtree partition on blocks that are simultaneously adjacent to the right and bottom boundaries of an image. For ease of explanation, we assume that the minimum size of the quadtree partition is 4x4.

[0115] In the example shown in Figure 8, when a quadtree split is performed on a 32x32 block containing a 12x20 block, four 16x16 blocks are generated. A quadtree split can then be performed again on two of these 16x16 blocks containing texture data. As a result, the x-axis and y-axis coordinates extend beyond the image boundary, generating an 8x8 block containing 4x4 texture data. Since the width and height of this 8x8 block are larger than the minimum quadtree split size, a quadtree split can be performed on this block.

[0116] When a quadtree partition is performed on a 12x20 block, the block can be divided into a 12x16 block and a 12x4 block. When a quadtree partition is performed again on each of these, the 12x16 block is divided into two 8x8 blocks and two 4x8 blocks, and the 12x4 block is divided into an 8x4 block and a 4x4 block.

[0117] Since the width of a 4x8 block located at the image boundary is the same as the minimum quadtree partition size, a binary tree partition can be performed on a 4x8 block. Specifically, a vertical binary tree partition can be performed based on a square block (i.e., 8x8) containing a 4x8 block.

[0118] Furthermore, since the width of an 8x4 block located at the image boundary is the same as the minimum quadtree partition size, a binary tree partition can be performed on an 8x4 block. Specifically, a horizontal binary tree partition can be performed based on a square block containing a 4x8 block (i.e., 8x8).

[0119] The resulting division allows for the placement of 8x4, 4x4, and 4x8 blocks adjacent to the image boundary.

[0120] Alternatively, a binary tree split can be performed if at least one of the block's width or height is less than or equal to a threshold, and a quadtree split otherwise. Here, the threshold can be derived based on the minimum quadtree split size. For example, if the minimum quadtree split size is minQTsize, the threshold can be set to "minQTsize<<1". Alternatively, information to determine the threshold can be signaled via a bitstream.

[0121] Figure 9 shows the partitioning pattern of blocks adjacent to the image boundary. For ease of explanation, we assume that the minimum quadtree partition size is 4x4. The threshold can be set to 8.

[0122] First, a quadtree partition is performed on the 12x20 block. As a result, the block can be divided into a 12x16 block and a 12x4 block. Since both the width and height of the 12x16 block exceed the threshold, a quadtree partition can be applied to the block. This divides the block into two 8x8 blocks and two 4x8 blocks.

[0123] The width of a 12x4 block exceeds the threshold. This allows a quadtree partition to be applied to the 12x4 block. As a result, the block can be divided into an 8x4 block and a 4x4 block.

[0124] Subsequently, since both the width and height of the 4x8 and 8x4 blocks located at the image boundary are less than or equal to the threshold, a binary tree partition can be applied to these blocks.

[0125] Conversely to the example above, if at least one of the block's width or height exceeds a threshold, a binary tree split can be performed; otherwise, a quadtree split can be performed.

[0126] Alternatively, for boundary blocks of unstructured boundaries, only quadtree partitioning or only binary tree partitioning can be applied. For example, quadtree partitioning can be repeatedly performed until the blocks located at the image boundary have the minimum size, or binary tree partitioning can be repeatedly performed until the blocks located at the image boundary have the minimum size.

[0127] By specifying a boundary block of an unstructured boundary as a coded unit, the skip mode can be fixedly applied to the boundary block of an unstructured boundary, or all conversion coefficients can be set to 0. This allows the Coded Block Flag (CBF), which indicates whether or not there are non-zero conversion coefficients in the boundary block of an unstructured boundary, to be set to 0. Coded units coded in skip mode or coded units with conversion coefficients set to 0 can be called boundary zero coded units.

[0128] Alternatively, the width or height of at least one of the encoding blocks generated by dividing a boundary block of an unstructured boundary can be compared to a threshold to determine whether or not to set the encoding unit as a boundary zero encoding unit. For example, encoding blocks whose width or height is less than the threshold can be encoded in skip mode, or the conversion coefficient of the encoding block can be set to 0.

[0129] Figure 10 shows the coding pattern of blocks adjacent to the image boundary. Assume the threshold is 8.

[0130] Among the coded blocks generated by dividing a boundary block of an unstructured boundary, a coded block whose width or height is less than a threshold can be set as a boundary zero coded unit.

[0131] For example, in the example shown in Figure 10, a 4x16 size coding block, an 8x4 size coding block, and a 4x4 size coding block can be set as boundary zero coding units. This allows the blocks to be coded in skip mode or the conversion coefficients of the blocks to be set to 0.

[0132] For coding blocks whose width and height are greater than or equal to a threshold, it is possible to selectively decide whether to apply a skip mode or set the conversion coefficient to 0. For this reason, a flag indicating whether or not to apply a skip mode to the coding block, or a flag indicating whether or not to set the conversion coefficient to 0, can be encoded and signaled.

[0133] Alternatively, it is possible to allow only coding units generated by binary tree partitioning to be set as boundary zero coding units. Alternatively, it is possible to allow only coding units generated by quadtree partitioning to be set as boundary zero coding units.

[0134] Collocated prediction is a predictive coding mode that uses information from the previous image to predict the current block. For example, a block at the same position as the current block in the previous image (hereinafter referred to as a collocated block) can be set as the predicted block for the current block. Hereafter, a predicted block generated based on a block at the same position as the current block will be called a collocated prediction block.

[0135] On the other hand, if an object that was present in the previous image moves to a different location in the current image, the object's movement can be used to effectively predict the current block. For example, if the direction and size of the object's movement can be determined by comparing the previous and current images, a predicted block (or predicted image) of the current block can be generated by considering the object's movement information. Hereafter, a predicted block generated using motion information can be referred to as a motion-predicted block.

[0136] A residual block can be generated by subtracting the predicted block from the current block. In this case, if object movement exists, the energy of the residual block can be reduced by using a motion prediction block instead of an equivalent position prediction block, thereby improving the compression performance of the residual block.

[0137] As mentioned earlier, generating prediction blocks using motion information can be called motion-compensated prediction. Most interpretations can generate prediction blocks based on motion-compensated prediction.

[0138] Motion information may include at least one of the following: motion vector, reference image index, prediction direction, or bidirectional weight index. The motion vector represents the direction and size of the object's movement. The reference image index specifies the reference image of the current block from among multiple reference images included in the reference image list. The prediction direction refers to either unidirectional L0 prediction, unidirectional L1 prediction, or bidirectional prediction (L0 and L1 prediction). Depending on the prediction direction of the current block, at least one of the motion information in the L0 direction or the motion information in the L1 direction can be used. The bidirectional weight index specifies the weight values ​​applied to L0 prediction blocks and the weight values ​​applied to L1 prediction blocks.

[0139] Figure 11 is a flowchart of the interpretation prediction method according to an embodiment of the present invention.

[0140] Referring to Figure 11, the inter-prediction method includes determining the inter-prediction mode of the current block (S1101), acquiring motion information of the current block according to the determined inter-prediction mode (S1102), and performing motion compensation prediction of the current block based on the acquired motion information (S1103).

[0141] Here, the interpretation mode represents various methods for determining the motion information of the current block and may include interpretation modes that use translational motion information and interpretation modes that use affine motion information. For example, the interpretation mode that uses translational motion information may include merge mode and high-level motion vector prediction mode, and the interpretation mode that uses affine motion information may include affine merge mode and affine motion vector prediction mode. According to the interpretation mode, the motion information of the current block can be determined based on information analyzed from adjacent blocks or bitstreams adjacent to the current block.

[0142] The following provides a detailed explanation of the interpretation prediction method using translational motion information.

[0143] The motion information of the current block can be derived from the motion information of other blocks. Here, the other blocks may be blocks that were encoded / decoded by interpretation before the current block. Setting the motion information of the current block to be the same as the motion information of other blocks can be defined as merge mode. Alternatively, setting the motion vector of other blocks to the predicted value of the motion vector of the current block can be defined as motion vector prediction mode.

[0144] Figure 12 is a flowchart of the process for deriving the movement information of the current block in merge mode.

[0145] Candidates for merging the current block can be derived (S1201). Candidates for merging the current block can be derived from blocks that have been coded / decoded by interpretation prior to the current block.

[0146] Figure 13 shows the candidate blocks used to derive merge candidates.

[0147] A candidate block may contain at least one of the following: an adjacent block containing samples adjacent to the current block, or a non-adjacent block containing samples not adjacent to the current block. Hereafter, the samples used to determine a candidate block will be designated as reference samples. Reference samples adjacent to the current block will be called adjacent reference samples, and reference samples not adjacent to the current block will be called non-adjacent reference samples.

[0148] Adjacent reference samples can be located in the adjacent column of the leftmost column of the current block, or in the adjacent row of the top row of the current block. For example, if the coordinates of the top-left sample of the current block are (0,0), then at least one of the following can be used as candidate blocks: a block containing a reference sample at (-1,H-1), a block containing a reference sample at (W-1,-1), a block containing a reference sample at (W,-1), a block containing a reference sample at (-1,H), or a block containing a reference sample at (-1,-1). Referring to the drawing, adjacent blocks with indices 0 through 4 can be used as candidate blocks.

[0149] A non-adjacent reference sample indicates a sample whose x-axis distance or y-axis distance from an adjacent reference sample to the current block is a predefined value. For example, at least one of the following can be used as a candidate block: a block containing a reference sample whose x-axis distance from the left reference sample is a predefined value; a block of a non-adjacent sample whose y-axis distance from the top reference sample is a predefined value; or a block of a non-adjacent sample whose x-axis and y-axis distances from the top-left reference sample are predefined values. The predefined value can be an integer such as 4, 8, 12, or 16. Referring to the drawing, at least one of the blocks with indices 5 through 26 can be used as a candidate block.

[0150] Samples that are not located on the same vertical, horizontal, or diagonal line as adjacent reference samples can be set as non-adjacent reference samples.

[0151] Figure 14 shows the location of the reference sample.

[0152] As shown in the example in Figure 14, the x-coordinate of an upper non-adjacent reference sample can be set to be different from the x-coordinate of an upper adjacent reference sample. For example, if the position of an upper adjacent reference sample is (W-1, -1), the position of an upper non-adjacent reference sample located N away from the upper adjacent reference sample along the y-axis can be set to ((W / 2)-1, -1-N), and the position of an upper non-adjacent reference sample located 2N away from the upper adjacent reference sample along the y-axis can be set to (0, -1-2N). In other words, the position of a non-adjacent reference sample can be determined based on the position of an adjacent reference sample and the distance between it and the adjacent reference sample.

[0153] Hereafter, candidate blocks that contain adjacent reference samples will be referred to as adjacent blocks, and blocks that contain non-adjacent reference samples will be referred to as non-adjacent blocks.

[0154] If the distance between the current block and a candidate block is greater than or equal to a threshold, the candidate block can be set to be unavailable as a merge candidate. The threshold can be determined based on the size of the coding tree unit. For example, the threshold can be set to the height of the coding tree unit (ctu_height), or to a value obtained by adding or subtracting an offset value from the height of the coding tree unit (e.g., ctu_height ± N). The offset value N is a predefined value in the encoder and decoder and can be set to 4, 8, 16, 32, or ctu_height.

[0155] If the difference between the y-axis coordinate of the current block and the y-axis coordinate of the sample included in the candidate block exceeds a threshold, it can be determined that the candidate block cannot be used as a merge candidate.

[0156] Alternatively, candidate blocks that do not belong to the same coding tree unit as the current block can be set as ineligible for merge. For example, if a reference sample does not extend beyond the upper boundary of the coding tree unit to which the current block belongs, a candidate block containing the reference sample can be set as ineligible for merge.

[0157] If the top boundary of the current block is adjacent to the top boundary of a coding tree unit, multiple candidate blocks are set to be ineligible for merging, which reduces the coding / decoding efficiency of the current block. To solve this problem, candidate blocks can be configured so that the number of candidate blocks located above the current block is greater than the number of candidate blocks located to the left of the current block.

[0158] Figure 15 shows the candidate blocks used to derive merge candidates.

[0159] As shown in the example in Figure 15, the upper blocks belonging to the N block columns above the current block and the left blocks belonging to the M block columns to the left of the current block can be set as candidate blocks. In this case, by setting M to be greater than N, the number of left candidate blocks can be set to be greater than the number of upper candidate blocks.

[0160] For example, the difference between the y-axis coordinate of a reference sample within the current block and the y-axis coordinate of the upper block that can be used as a candidate block can be set so as not to exceed N times the height of the current block. Similarly, the difference between the x-axis coordinate of a reference sample within the current block and the x-axis coordinate of the left-hand block that can be used as a candidate block can be set so as not to exceed M times the width of the current block.

[0161] For example, the example shown in Figure 15 indicates that the blocks belonging to the top two block columns of the current block and the blocks belonging to the left five block columns of the current block are set as candidate blocks.

[0162] As another example, if a candidate block does not belong to the same coding tree unit as the current block, a merge candidate can be derived using, instead of the candidate block, a block that belongs to the same coding tree unit as the current block, or a block containing a reference sample adjacent to the boundary of the coding tree unit.

[0163] Figure 16 shows an example of changing the position of a reference sample.

[0164] If a reference sample is located in a different coding tree unit than the current block, and the reference sample is not adjacent to the coding tree unit boundary, a candidate block can be determined using a reference sample adjacent to the coding tree unit boundary instead of the reference sample.

[0165] For example, in the examples shown in Figures 16(a) and 16(b), if the top boundary of the current block touches the top boundary of the coding tree unit, the reference samples at the top of the current block will belong to a different coding tree unit than the current block. A sample adjacent to the top boundary of the coding tree unit can be used to replace a reference sample belonging to a different coding tree unit than the current block that is not adjacent to the top boundary of the coding tree unit.

[0166] For example, as shown in Figure 16(a), the reference sample at position 6 is replaced with the sample at position 6' on the upper boundary of the coding tree unit, and as shown in Figure 16(b), the reference sample at position 15 is replaced with the sample at position 15' on the upper boundary of the coding tree unit. In this case, the y-coordinate of the replacement sample can be changed to an adjacent position in the coding tree unit, and the x-coordinate of the replacement sample can be set to be the same as that of the reference sample. For example, the sample at position 6' may have the same x-coordinate as the sample at position 6, and the sample at position 15' may have the same x-coordinate as the sample at position 15.

[0167] Alternatively, the x-coordinate of the replacement sample can be set to a value obtained by adding or subtracting an offset value from the x-coordinate of the reference sample. For example, if the x-coordinate of the adjacent reference sample at the top of the current block is the same as that of the non-adjacent reference sample, the x-coordinate of the replacement sample can be set to a value obtained by adding or subtracting an offset value from the x-coordinate of the reference sample. This is to prevent the replacement sample that replaces the non-adjacent reference sample from being in the same position as another non-adjacent or adjacent reference sample.

[0168] Figure 17 shows an example of changing the position of a reference sample.

[0169] When replacing a reference sample located in a different coding tree unit than the current block and not adjacent to the coding tree unit boundary using a sample located at the boundary of a coding tree unit, the x-coordinate of the replacement sample can be set to a value obtained by adding or subtracting an offset value from the x-coordinate of the reference sample.

[0170] For example, in the example shown in Figure 17, the reference sample at position 6 and the reference sample at position 15 can be replaced with the sample at position 6' and the sample at position 15', respectively, which have the same y-coordinate as the row adjacent to the upper boundary of the coding tree unit. In this case, the x-coordinate of the sample at position 6' can be set to the x-coordinate of the reference sample at position 6 minus W / 2, and the x-coordinate of the sample at position 15' can be set to the x-coordinate of the reference sample at position 15 minus W-1.

[0171] Unlike the examples shown in Figures 16 and 17, the y-coordinate of the replacement sample can also be set to the y-coordinate of the row located above the top row of the current block, or the y-coordinate of the top boundary of the coding tree unit.

[0172] Although not shown in the diagram, the sample to replace the reference sample can be determined based on the left boundary of the coding tree unit. For example, if the reference sample is not contained within the same coding tree unit as the current block and is not adjacent to the left boundary of the coding tree unit, the reference sample can be replaced with a sample adjacent to the left boundary of the coding tree unit. In this case, the replacement sample may have the same y-coordinate as the reference sample, or it may have a y-coordinate obtained by adding or subtracting an offset value from the y-coordinate of the reference sample.

[0173] Subsequently, a block containing the replacement sample can be set as a candidate block, and a merge candidate for the current block can be derived based on the candidate block.

[0174] It is also possible to derive merge candidates from temporally adjacent blocks contained in a different image from the current block. For example, merge candidates can be derived from equivalent-positioned blocks contained in equivalent-positioned images.

[0175] The motion information for merge candidates can be set in the same way as the motion information for candidate blocks. For example, at least one of the following can be set as the motion information for merge candidates: the motion vector of the candidate block, the reference image index, the predicted direction, or the bidirectional weight value index.

[0176] A list of merge candidates, including merge candidates, can be generated (S1202). These merge candidates can be classified into adjacent merge candidates derived from adjacent blocks adjacent to the current block, and non-adjacent merge candidates derived from non-adjacent blocks.

[0177] Indices can be assigned to multiple merge candidates within the merge candidate list according to a predetermined order. For example, the index assigned to an adjacent merge candidate may be smaller than the index assigned to a non-adjacent merge candidate. Alternatively, an index can be assigned to each merge candidate based on the index of each block shown in Figure 13 or Figure 15.

[0178] If the merge candidate list contains multiple merge candidates, at least one of the multiple merge candidates can be selected (S1203). In this case, information indicating whether the motion information of the current block is derived from an adjacent merge candidate can be signaled via the bitstream. This information may be a 1-bit flag. For example, a syntax element isAdjancentMergeFlag indicating whether the motion information of the current block is derived from an adjacent merge candidate can be signaled via the bitstream. If the value of the syntax element isAdjancentMergeFlag is 1, the motion information of the current block can be derived based on an adjacent merge candidate. On the other hand, if the value of the syntax element isAdjancentMergeFlag is 0, the motion information of the current block can be derived based on a non-adjacent merge candidate.

[0179] Table 1 shows the syntax table including the syntactic element isAdjancentMergeFlag. [Table 1(1)] [Table 1(2)]

[0180] Information to specify one of several merge candidates can be signaled via a bitstream. For example, information indicating the index of one of the merge candidates included in a merge candidate list can be signaled via a bitstream.

[0181] If isAdjacentMergeflag is 1, the syntactic element merge_idx can be signaled to determine which of the adjacent merge candidates is being considered. The maximum value of the syntactic element merge_idx can be set to the number of adjacent merge candidates minus 1.

[0182] If isAdjacentMergeflag is 0, the syntactic element NA_merge_idx can be signaled to determine one of the non-adjacent merge candidates. The syntactic element NA_merge_idx represents the value obtained by subtracting the number of adjacent merge candidates from the index of the non-adjacent merge candidate. The decoder can select a non-adjacent merge candidate by adding the number of adjacent merge candidates to the index determined by NA_merge_idx.

[0183] If the number of merge candidates in the merge candidate list is less than the maximum value, merge candidates included in the inter-domain motion information table can be added to the merge candidate list. The inter-domain motion information table may include merge candidates derived based on blocks encoded / decoded before the current block.

[0184] The inter-region motion information table contains merge candidates derived from blocks encoded / decoded based on inter-predictions in the current image. For example, the motion information of merge candidates included in the inter-region motion information table can be set to be the same as the motion information of blocks encoded / decoded based on inter-predictions. Here, the motion information may include at least one of the following: motion vectors, reference image indices, prediction direction, or bidirectional weight value indices.

[0185] For the sake of explanation, merge candidates included in the inter-domain motion information table are referred to as inter-domain merge candidates.

[0186] The encoder and decoder can predefine the maximum number of merge candidates that can be included in the inter-domain motion information table. For example, the maximum number of merge candidates that can be included in the inter-domain motion information table may be 1, 2, 3, 4, 5, 6, 7, 8 or more (e.g., 16).

[0187] Alternatively, information indicating the maximum number of merge candidates for the inter-region motion information table can be signaled via the bitstream. This information can be signaled at the sequence level, image level, or slice level.

[0188] Alternatively, the maximum number of merge candidates for the inter-region motion information table can be determined according to the image size, slice size, or encoding tree unit size.

[0189] The inter-region motion information table can be initialized at the level of an image, slice, tile, brick, coded tree unit, or coded tree unit line (row or column). For example, if a slice is initialized, the inter-region motion information table is also initialized, and the inter-region motion information table may contain no merge candidates.

[0190] Alternatively, information indicating whether or not to initialize the inter-region motion information table can be signaled via a bitstream. This information can be signaled at the slice level, tile level, brick level, or block level. The configured inter-region motion information table can be used until the information indicates that the inter-region motion information table should be initialized.

[0191] Alternatively, information about a given inter-region merge candidate can be signaled via an image parameter set or slice header. Even after the slice has been initialized, the inter-region motion information table may include an initial inter-region merge candidate. This allows the inter-region merge candidate to be used for the first block to be encoded / decoded within the slice.

[0192] Blocks can be encoded / decoded according to the encoding / decoding order, and multiple encoded / decoded blocks can be sequentially set as inter-region merge candidates based on inter-prediction according to the encoding / decoding order.

[0193] Figure 18 is a diagram illustrating an example of updating the inter-domain motion information table.

[0194] When performing inter-region prediction on the current block (S1801), inter-region merge candidates can be derived based on the current block (S1802). The movement information for inter-region merge candidates can be set in the same way as the movement information for the current block.

[0195] If the inter-region movement information table is empty (S1803), inter-region merge candidates derived based on the current block can be added to the inter-region movement information table (S1804).

[0196] If the inter-area movement information table already contains the inter-area merge candidate (S1803), a redundancy check can be performed on the movement information of the current block (or the inter-area merge candidate derived based on the current block) (S1805). The redundancy check is used to determine whether the movement information of the inter-area merge candidate stored in the inter-area movement information table is the same as the movement information of the current block. The redundancy check can be performed on all inter-area merge candidates stored in the inter-area movement information table as objects. Alternatively, the redundancy check can be performed on inter-area merge candidates stored in the inter-area movement information table whose index is above or below a threshold as objects.

[0197] If the current block does not contain any inter-region merge candidates with the same motion information, the inter-region merge candidates derived based on the current block can be added to the inter-region motion information table (S1808). Whether or not the inter-region merge candidates are the same can be determined based on whether or not their motion information (e.g., motion vectors and / or reference image indices) is the same.

[0198] In this case, if the inter-area movement information table already stores the maximum number of inter-area merge candidates (S1806), the oldest inter-area merge candidate can be deleted (S1807), and an inter-area merge candidate derived based on the current block can be added to the inter-area movement information table (S1808).

[0199] Multiple inter-region merge candidates can be identified according to each index. When adding inter-region merge candidates derived from the current block to the inter-region movement information table, the lowest index (such as 0) can be assigned to the inter-region merge candidate, and the index of the stored inter-region merge candidates can be incremented by 1. In this case, if the inter-region movement information table already stores the maximum number of inter-region merge candidates, the inter-region merge candidate with the highest index is removed.

[0200] Alternatively, when adding an inter-region merge candidate derived from the current block to the inter-region movement information table, the largest possible index can be assigned to the inter-region merge candidate. For example, if the number of inter-region merge candidates stored in the inter-region movement information table is less than the maximum value, an index equal to the number of stored inter-region merge candidates can be assigned to the inter-region merge candidate. Alternatively, if the number of inter-region merge candidates stored in the inter-region movement information table is equal to the maximum value, an index obtained by subtracting 1 from the maximum value can be assigned to the inter-region merge candidate. Furthermore, the inter-region merge candidate with the smallest index is removed, and the indices of the remaining stored inter-region merge candidates are decreased by 1 each.

[0201] Figure 19 shows an example of updating the inter-region merge candidate table.

[0202] Assume that the inter-region merge candidates derived from the current block are added to the inter-region merge candidate table, and that the largest possible index is assigned to each inter-region merge candidate. Furthermore, assume that the inter-region merge candidate table already contains the maximum number of inter-region merge candidates.

[0203] When adding the inter-region merge candidate HmvpCand[n+1] derived from the current block to the inter-region merge candidate table HmvpCandList, the inter-region merge candidate HmvpCand[0] with the smallest index can be removed from the stored inter-region merge candidates, and the indices of multiple remaining inter-region merge candidates can be decreased by 1 each time. Additionally, the index of the inter-region merge candidate HmvpCand[n+1] derived from the current block can be set to its maximum value (n in the example shown in Figure 19).

[0204] If an inter-area merge candidate that is the same as the inter-area merge candidate derived based on the current block is stored (S1805), the inter-area merge candidate derived based on the current block may not be added to the inter-area movement information table (S1809).

[0205] Alternatively, it is possible to add inter-region merge candidates derived based on the current block to the inter-region movement information table, while simultaneously deleting stored inter-region merge candidates that are identical to the aforementioned inter-region merge candidates. In this case, the same effect as updating the index of the stored inter-region merge candidates occurs.

[0206] Figure 20 shows an example of how the index of stored inter-region merge candidates is updated.

[0207] If the index of a stored inter-region merge candidate that is the same as the inter-region merge candidate mvCand derived based on the current block is hIdx, then the stored inter-region merge candidate can be deleted, and the indices of inter-region merge candidates whose index exceeds hIdx can be decreased by 1. For example, in the example shown in Figure 20, HmvpCand[2], which is the same as mvCand, is deleted from the inter-region movement information table HvmpCandList, and the indices from HmvpCand[3] to HmvpCand[n] are decreased by 1.

[0208] Furthermore, the inter-region merge candidate mvCand, derived based on the current block, can be added to the end of the inter-region motion information table.

[0209] Alternatively, you can update the index of a stored inter-region merge candidate that is the same as the inter-region merge candidate derived based on the current block. For example, you can change the index of a stored inter-region merge candidate to the minimum or maximum value.

[0210] The motion information of blocks included in a predetermined region can be set so that it cannot be added to the inter-region motion information table. For example, inter-region merge candidates derived based on the motion information of blocks included in a parallel merge region can not be added to the inter-region motion information table. Because the encoding / decoding order of blocks included in a parallel merge region is not specified, it is inappropriate to use the motion information of any of the said blocks when performing inter prediction on other blocks. This prevents inter-region merge candidates derived based on blocks included in a parallel merge region from being added to the inter-region motion information table.

[0211] When performing motion compensation prediction using subblock units, inter-region merge candidates can be derived based on the motion information of a representative subblock among the multiple subblocks contained in the current block. For example, when subblock merge candidates are used for the current block, inter-region merge candidates can be derived based on the motion information of a representative subblock among the subblocks.

[0212] The motion vector of a subblock can be derived in the following order. First, one of the merge candidates included in the current block's merge candidate list can be selected, and an initial shift vector (shVector) can be derived based on the motion vector of the selected merge candidate. Furthermore, by adding the initial shift vector to the position (xSb, ySb) of the reference sample (e.g., the top-left sample or the sample in the middle) of each subblock in the encoded block, a shift subblock with the reference sample position (xColSb, yColSb) can be derived. Equation 1 below shows the formula for deriving the shift subblock.

number

[0213] Next, the motion vector of the equivalent position block corresponding to the center position of the subblock containing (xColSb, yColSb) is set to the motion vector of the subblock containing (xSb, ySb).

[0214] A typical subblock can refer to a subblock containing the top-left sample that includes the current block, or a subblock that includes the central sample.

[0215] Figure 21 shows the location of representative subblocks.

[0216] Figure 21(a) shows an example where a subblock located in the upper left of the current block is set as the representative subblock, and Figure 21(b) shows an example where a subblock located in the center of the current block is set as the representative subblock. When motion compensation prediction is performed on a subblock basis, inter-region merge candidates for the current block can be derived based on the motion vectors of the subblock containing the upper left sample of the current block or the subblock containing the center sample of the current block.

[0217] Based on the current block's interprediction mode, it is also possible to determine whether or not to use the current block as an inter-region merge candidate. For example, blocks encoded / decoded based on an affine motion model can be set to not be used as inter-region merge candidates. This means that even if the current block is encoded / decoded with interprediction, if the current block's interprediction mode is affine prediction mode, the inter-region motion information table will not be updated based on the current block.

[0218] Alternatively, inter-region merge candidates can be derived based on at least one subblock vector from among the subblocks contained in a block encoded / decoded based on an affine motion model. For example, inter-region merge candidates can be derived using a subblock located in the upper left, center, or upper right of the current block. Alternatively, the motion vector of an inter-region merge candidate can be set to the average of the subblock vectors of multiple subblocks.

[0219] Alternatively, inter-region merge candidates can be derived based on the average value of the affine seed vectors of the encoded / decoded blocks based on the affine motion model. For example, the average value of at least one of the first, second, or third affine seed vectors of the current block can be set as the motion vector of the inter-region merge candidate.

[0220] Alternatively, an inter-domain motion information table can be configured for each inter-prediction mode. For example, at least one of the following can be defined: an inter-domain motion information table for blocks encoded / decoded by intra-block copying, an inter-domain motion information table for blocks encoded / decoded based on a translational motion model, or an inter-domain motion information table for blocks encoded / decoded based on an affine motion model. Depending on the inter-prediction mode of the current block, one of several inter-domain motion information tables can be selected.

[0221] Figure 22 shows an example of generating an inter-domain motion information table for each inter-prediction mode.

[0222] When encoding / decoding a block based on a non-affine motion model, the inter-region merge candidate mvCand derived based on the block can be added to the inter-region non-affine motion information table HmvpCandList. On the other hand, when encoding / decoding a block based on an affine motion model, the inter-region merge candidate mvAfCand derived based on the block can be added to the inter-region affine motion information table HmvpAfCandList.

[0223] The affine seed vector of a block can be stored in an inter-region merge candidate derived from the block, which has been encoded / decoded based on an affine motion model. This allows the inter-region merge candidate to be used as a merge candidate for deriving the affine seed vector of the current block.

[0224] In addition to the inter-domain motion information table described above, another inter-domain motion information table can also be defined. In addition to the inter-domain motion information table described above (hereinafter referred to as the first inter-domain motion information table), a long-term motion information table (hereinafter referred to as the second inter-domain motion information table) can also be defined. Here, the long-term motion information table includes long-term merge candidates.

[0225] If both the first and second inter-domain motion information tables are empty, an inter-domain merge candidate can first be added to the second inter-domain motion information table. Only after the number of available inter-domain merge candidates in the second inter-domain motion information table has reached its maximum can an inter-domain merge candidate be added to the first inter-domain motion information table.

[0226] Alternatively, a single inter-domain merge candidate can be added to both the second inter-domain motion information table and the first inter-domain motion information table.

[0227] In this case, the configured second inter-domain motion information table may no longer be updated. Alternatively, the second inter-domain motion information table may be updated if the decoding region is greater than or equal to a predetermined ratio of the slice. Alternatively, the second inter-domain motion information table may be updated for every N coded tree unit lines.

[0228] On the other hand, the first inter-domain motion information table can be updated each time an encoded / decoded block is generated by inter-prediction. However, it is also possible to configure the system so that inter-domain merge candidates added to the second inter-domain motion information table are not used to update the first inter-domain motion information table.

[0229] Information for selecting either the first inter-domain motion information table or the second inter-domain motion information table can be signaled via a bitstream. If the number of merge candidates in the merge candidate list is less than the maximum value, the merge candidates included in the inter-domain motion information table indicated by the information can be added to the merge candidate list.

[0230] Alternatively, an inter-region motion information table can be selected based on the current block size, shape, inter-prediction mode, whether to perform bidirectional prediction, whether to refine the motion vector, or whether to perform triangulation.

[0231] Alternatively, if adding inter-domain merge candidates included in the first inter-domain motion information table does not exceed the maximum number of merge candidates in the merge candidate list, then inter-domain merge candidates included in the second inter-domain motion information table can be added to the merge candidate list.

[0232] Figure 23 shows an example of adding inter-region merge candidates included in the long-term movement information table to the merge candidate list.

[0233] If the number of merge candidates in the merge candidate list is less than the maximum number, the inter-area merge candidates included in the first inter-area movement information table HmvpCandList can be added to the merge candidate list. If, even after adding the inter-area merge candidates included in the first inter-area movement information table to the merge candidate list, the number of merge candidates in the merge candidate list is still less than the maximum number, the inter-area merge candidates included in the long-term movement information table HmvpLTCandList can be added to the merge candidate list.

[0234] Table 2 shows the process of adding inter-region merge candidates included in the long-term movement information table to the merge candidate list. [Table 2]

[0235] Inter-region merge candidates can be configured to include additional information in addition to motion information. For example, the size, shape, or division information of the block can be added to the inter-region merge candidate. When configuring the merge candidate list for the current block, only inter-region merge candidates with the same or similar size, shape, or division information as the current block can be used, or inter-region merge candidates with the same or similar size, shape, or division information as the current block can be preferentially added to the merge candidate list.

[0236] Alternatively, an inter-region motion information table can be generated for each block size, shape, or division information. A list of merge candidates for the current block can be generated using the inter-region motion information table corresponding to the current block's shape, size, or division information from among multiple inter-region motion information tables.

[0237] Alternatively, an inter-region motion information table can be generated for each motion vector resolution. For example, if the motion vector of the current block has a resolution of 1 / 4 pixel, the inter-region merge candidate derived from the current block can be added to the inter-prediction region 1 / 4 pixel motion information list. If the motion vector of the current block has a resolution of 1 integer pixel, the inter-region merge candidate derived from the current block can be added to the inter-prediction region integer pixel motion information list. If the motion vector of the current block has a resolution of 4 integer pixels, the inter-region merge candidate derived from the current block can be added to the inter-prediction region 4 integer pixel motion information list. Depending on the motion vector resolution of the encoded / decoded object block, one of several inter-motion information lists can be selected.

[0238] When applying the merge offset vector coding method to the current block, instead of adding the inter-region merge candidates derived based on the current block to the inter-region motion information table HmvpCandList, they can be added to the inter-predicted region merge offset motion information table HmvpHMVDCandList. In this case, the inter-region merge candidates may include motion vector offset information for the current block. HmvpHMVDCandList can be used to derive the offsets of the block to which the merge offset vector coding method is applied.

[0239] If the current block's merge candidate list does not contain the maximum number of merge candidates, inter-area merge candidates from the inter-area movement information table can be added to the merge candidate list. This addition process is performed in ascending or descending order of the index. For example, the inter-area merge candidate with the highest index can be added to the merge candidate list.

[0240] When adding inter-region merge candidates included in the inter-region movement information table to the merge candidate list, a redundancy check can be performed between the inter-region merge candidate and the multiple merge candidates stored in the merge candidate list.

[0241] For example, Table 3 shows the process of adding inter-region merge candidates to the merge candidate list. [Table 3]

[0242] Redundancy checks can also be performed only on a subset of inter-region merge candidates included in the inter-region movement information table. For example, redundancy checks can be performed only on inter-region merge candidates whose index is above or below a threshold.

[0243] Alternatively, redundancy checks can be performed only on a subset of the merge candidates stored in the merge candidate list. For example, redundancy checks can be performed only on merge candidates whose index is above or below a threshold, or on merge candidates derived from a block at a specific location. Here, the specific location may include at least one of the left-side adjacent block, top-side adjacent block, top-right adjacent block, or bottom-left adjacent block of the current block.

[0244] Figure 24 shows an example of performing redundancy checks on only a portion of the merge candidates.

[0245] When adding the inter-region merge candidate HmvpCand[j] to the merge candidate list, a redundancy check can be performed on the inter-region merge candidate against the two merge candidates with the highest index, mergeCandList[NumMerge-2] and mergeCandList[NumMerge-1], where NumMerge indicates the number of available spatial and temporal merge candidates.

[0246] If the same merge candidate as the first inter-area merge candidate is found, when performing a redundancy check on the second inter-area merge candidate, the redundancy check with the same merge candidate as the first inter-area merge candidate can be omitted.

[0247] Figure 25 shows an example of omitting redundancy checks for specific merge candidates.

[0248] When adding the inter-area merge candidate HmvpCand[i], whose index is i, to the merge candidate list, a redundancy check can be performed between the inter-area merge candidate and the merge candidates stored in the merge candidate list. In this case, if the same merge candidate mergeCandList[j] as the inter-area merge candidate HmvpCand[i] is found, the inter-area merge candidate HmvpCand[i-1], whose index is i-1, can be added to the merge candidate list, and a redundancy check can be performed between the inter-area merge candidate HmvpCand[i-1] and the merge candidate mergeCandList[j]. In this case, the redundancy check between the inter-area merge candidate HmvpCand[i-1] and the merge candidate mergeCandList[j] can be omitted.

[0249] For example, in the example shown in Figure 25, HmvpCand[i] and mergeCandList[2] were determined to be the same. As a result, HmvpCand[i] is not added to the merge candidate list, and a redundancy check can be performed on HmvpCand[i-1]. In this case, the redundancy check between HmvpCand[i-1] and mergeCandList[2] can be omitted.

[0250] If the current block merge candidate list contains fewer merge candidates than the maximum value, it may also include at least one of the following: an inter-region merge candidate, a pairwise merge candidate, or a zero merge candidate. A pairwise merge candidate is a merge candidate whose motion vector is the average of the motion vectors of two or more merge candidates, while a zero merge candidate is a merge candidate whose motion vector is 0.

[0251] The current block's merge candidate list can be expanded by adding merge candidates in the following order:

[0252] Spatial merge candidates - Temporal merge candidates - Interdomain merge candidates - (Interpretation affine merge candidates) - Pairwise merge candidates - Zero merge candidates A spatial merge candidate refers to a merge candidate derived from at least one of adjacent blocks or non - adjacent blocks, and a temporal merge candidate refers to a merge candidate derived from a previous reference picture. An inter - prediction affine merge candidate refers to an inter - region merge candidate derived from a block encoded / decoded with an affine motion model.

[0253] The inter - region motion information table can also be used in the high - level motion vector prediction mode. For example, when the number of motion vector prediction candidates included in the motion vector prediction candidate list of the current block is less than the maximum value, the inter - region merge candidates included in the inter - region motion information table can be set as the motion vector prediction candidates of the current block. Specifically, the motion vector of the inter - region merge candidate can be set as the motion vector prediction candidate.

[0254] When any one of the motion vector prediction candidates included in the motion vector prediction candidate list of the current block is selected, the selected candidate can be set as the motion vector predictor of the current block. Then, after decoding the motion vector residual value of the current block, the motion vector of the current block can be obtained by adding the motion vector prediction value and the motion vector residual value.

[0255] The motion vector prediction candidate list of the current block can be configured in the following order.

[0256] Spatial motion vector prediction candidate - Temporal motion vector prediction candidate - Inter - decoded region merge candidate - (Inter - decoded region affine merge candidate) - Zero motion vector prediction candidate A spatial motion vector prediction candidate refers to a motion vector prediction candidate derived from at least one adjacent or non-adjacent block, while a temporal motion vector prediction candidate refers to a motion vector prediction candidate derived from the previous reference image. An interpredicted affine merge candidate refers to an interpredicted motion vector prediction candidate derived from blocks encoded / decoded by the affine motion model. A zero motion vector prediction candidate refers to a candidate whose motion vector value is 0.

[0257] A merge processing area larger than the encoded block can be specified. Encoded blocks contained within the merge processing area can be processed in parallel without sequential encoding / decoding. Here, not sequential encoding / decoding means that the encoding / decoding order is not specified. This allows the encoding / decoding process of the blocks contained within the merge processing area to be processed independently. Alternatively, blocks contained within the merge processing area can share merge candidates. Here, the merge candidates can be derived based on the merge processing area.

[0258] Depending on the characteristics described above, the merge processing region can also be called a parallel processing region, a shared merge region (SMR), or a merge estimation region (MER).

[0259] Candidates for merging the current block can be derived based on the encoded block. However, if the current block is contained within a parallel merge region larger than the current block, candidate blocks contained within the same parallel merge region as the current block can be set to be ineligible as merge candidates.

[0260] Figure 26 shows an example where candidate blocks located in the same parallel merge region as the current block are configured to be unavailable as merge candidates.

[0261] In the example shown in Figure 26(a), when encoding / decoding CU5, blocks containing reference samples adjacent to CU5 can be set as candidate blocks. In this case, candidate blocks X3 and X4, which are included in the same parallel merge region as CU5, can be set as unusable as merge candidates for CU5. On the other hand, candidate blocks X0, X1, and X2, which are not included in the same parallel merge region as CU5, can be set as usable as merge candidates.

[0262] In the example shown in Figure 26(b), when encoding / decoding CU8, blocks containing reference samples adjacent to CU8 can be set as candidate blocks. In this case, candidate blocks X6, X7, and X8, which are included in the same parallel merge region as CU8, can be set as ineligible for merge. On the other hand, candidate blocks X5 and X9, which are not included in the same parallel merge region as CU5, can be set as usable for merge.

[0263] The parallel merge region may be square or non-square. Information for determining the parallel merge region can be signaled via a bitstream. This information may include at least one of the following: information indicating the shape of the parallel merge region and information indicating the size of the parallel merge region. If the parallel merge region is non-square, at least one of the following can be signaled via a bitstream: information indicating the size of the parallel merge region, information indicating the width and / or height of the parallel merge region, or information indicating the ratio of the width to the height of the parallel merge region.

[0264] The size of the parallel merge region can be determined based on at least one of the following: information signaled via the bitstream, image resolution, slice size, or tile size.

[0265] When performing motion compensation prediction on blocks included in a parallel merge region, inter-region merge candidates derived based on the motion information of the blocks for which motion compensation prediction was performed can be added to the inter-predicted motion information table.

[0266] However, when adding inter-region merge candidates derived from blocks included in the parallel merge region to the inter-region motion information table, if the actual encoding / decoding of other blocks in the parallel merge region is delayed compared to the aforementioned block, the inter-region merge candidates derived from the aforementioned block may be used. In other words, it is necessary to eliminate inter-block dependencies when encoding / decoding blocks included in the parallel merge region, but motion prediction compensation may be performed using motion information from other blocks included in the parallel merge region. To solve this problem, even after encoding / decoding of a block included in the parallel merge region is completed, the motion information of the encoded / decoded block may not be added to the inter-region motion information table.

[0267] Alternatively, when performing motion compensation prediction on blocks included in a parallel merge region, inter-region merge candidates derived from the blocks can be added to the inter-region motion information table in a predefined order. Here, the predefined order can be determined based on the scan order of the encoded blocks within the parallel merge region or encoded tree unit. The scan order can be at least one of raster scan, horizontal scan, vertical scan, or zigzag scan. Alternatively, the predefined order can be determined based on the motion information of each block or the number of blocks having the same motion information.

[0268] Alternatively, inter-region merge candidates containing unidirectional motion information can be added to the inter-prediction-region merge list before inter-region merge candidates containing bidirectional motion information. Conversely, inter-region merge candidates containing bidirectional motion information can be added to the inter-region merge candidate table before inter-region merge candidates containing unidirectional motion information.

[0269] Alternatively, inter-region merge candidates can be added to the inter-region movement information table according to their frequency of use within the parallel merge region or coding tree unit, either in descending order of frequency or infrequency of frequency.

[0270] If the current block is located in a parallel merge region and the number of merge candidates in the current block's merge candidate list is less than the maximum number, then inter-region merge candidates included in the inter-region movement information table can be added to the merge candidate list. In this case, it can be configured not to add inter-region merge candidates derived from blocks located in the same parallel merge region as the current block to the current block's merge candidate list.

[0271] Alternatively, if the current block is located in a parallel merge region, it is possible to configure the system not to use inter-region merge candidates included in the inter-region movement information table. In other words, even if the number of merge candidates included in the current block's merge candidate list is less than the maximum number, inter-region merge candidates included in the inter-region movement information table are not added to the merge candidate list.

[0272] An inter-region motion information table can be configured for a parallel merge region or a coded tree unit. This inter-region motion information table serves to temporarily store motion information of blocks included in the parallel merge region. To distinguish it from a general inter-region motion information table, the inter-region motion information table for a parallel merge region or coded tree unit is called a temporary motion information table. Furthermore, inter-region merge candidates stored in the temporary motion information table are called temporary merge candidates.

[0273] Figure 27 shows a temporary movement information table.

[0274] A temporary motion information table for a symbolized tree unit or a parallel merge area can be configured. When motion compensation prediction of the current block included in the symbolized tree unit or the parallel merge area is performed, the motion information of the block cannot be added to the inter-region motion information table HmvpCandList. Instead, a temporary merge candidate derived from the block can be added to the temporary motion information table HmvpMERCandList. That is, the temporary merge candidate added to the temporary motion information table can be not added to the inter-region motion information table. Thereby, the inter-region motion information table may not include an inter-region merge candidate derived based on the motion information of the blocks included in the symbolized tree unit or the parallel merge area including the current block.

[0275] The maximum number of merge candidates that the temporary motion information table can include can be set to be the same as the maximum number of merge candidates that the inter-region motion information table can include. Alternatively, the maximum number of merge candidates that the temporary motion information table can include can be determined according to the size of the symbolized tree unit or the parallel merge area.

[0276] The current block included in the symbolized tree unit or the parallel merge area can be set not to use the temporary motion information table of the symbolized tree unit or the parallel merge area. That is, when the number of merge candidates included in the merge candidate list of the current block is less than the maximum value, the inter-region merge candidates included in the inter-region motion information table can be added to the merge candidate list, and the temporary merge candidates included in the temporary motion information table can be not added to the merge candidate list. Thereby, the motion information of other blocks included in the same symbolized tree unit or parallel merge area as the current block can be not used for the motion compensation prediction of the current block.

[0277] Once encoding / decoding is complete for all blocks contained within the encoded tree unit or parallel merge region, the inter-region motion information table and the temporary motion information table can be merged.

[0278] Figure 28 shows an example of merging the inter-predicted motion information table and the temporary motion information table.

[0279] Once encoding / decoding is complete for all blocks contained within the encoded tree unit or parallel merge region, the temporary merge candidates contained in the temporary motion information table can be updated in the inter-region motion information table, as shown in Figure 28.

[0280] In this case, temporary merge candidates included in the temporary movement information table can be added to the inter-frame movement information table in the order in which they were inserted into the temporary movement information table (i.e., in ascending or descending order of index value).

[0281] As another example, temporary merge candidates included in the temporary movement information table can be added to the inter-region movement information table according to a predefined order.

[0282] Here, the predefined order can be determined based on the scanning order of the encoded blocks within a parallel merge region or encoded tree unit. The scanning order may be at least one of raster scanning, horizontal scanning, vertical scanning, or zigzag scanning. Alternatively, the predefined order may be determined based on the motion information of each block or the number of blocks that have the same motion information.

[0283] Alternatively, temporary merge candidates containing unidirectional motion information can be added to the inter-prediction merge list before temporary merge candidates containing bidirectional motion information. On the other hand, temporary merge candidates containing bidirectional motion information can be added to the inter-region merge candidate list before temporary merge candidates containing unidirectional motion information.

[0284] Alternatively, temporary merge candidates can be added to the inter-region movement information table according to their frequency of use within the parallel merge region or coding tree unit, either in descending order of frequency or infrequency of use.

[0285] When adding temporary merge candidates from the temporary movement information table to the inter-area movement information table, a redundancy check can be performed on the temporary merge candidates. For example, if the same inter-area merge candidate as a temporary merge candidate in the temporary movement information table is stored in the inter-area movement information table, the temporary merge candidate may not be added to the inter-area movement information table. In this case, a redundancy check can be performed on some of the inter-area merge candidates included in the inter-area movement information table. For example, a redundancy check can be performed on inter-area merge candidates whose index is greater than or equal to a threshold. For example, if a temporary merge candidate is identical to an inter-area merge candidate whose index is greater than or equal to a predefined value, the temporary merge candidate may not be added to the inter-area movement information table.

[0286] Intra-prediction is the process of predicting the current block using the recovered samples that have been coded / decoded around the current block. In this case, the intra-prediction of the current block can use the recovered samples before the in-loop filter was applied.

[0287] Intra-prediction techniques include matrix-based intra-prediction and general intra-prediction that considers orientation with respect to peripherally restored samples. Information indicating the intra-prediction technique for the current block can be signaled via a bitstream. This information may be a 1-bit flag. Alternatively, the intra-prediction technique for the current block can be determined based on the current block's position, size, shape, or at least one of the intra-prediction techniques of adjacent blocks. For example, if the current block spans an image boundary, the current block can be set so that matrix-based intra-prediction is not applied to it.

[0288] Matrix-based intra-prediction is a method for obtaining a predicted block for the current block based on the matrix product of matrices stored in the encoder and decoder and the recovered samples around the current block. Information to specify one of several stored matrices can be signaled via a bitstream. The decoder can determine the matrix to be used for the intra-prediction of the current block based on this information and the size of the current block.

[0289] A typical intra-prediction is a method of obtaining predicted blocks for the current block based on either a non-directional intra-prediction mode or a directional intra-prediction mode. The process of performing intra-prediction based on a typical intra-prediction is described in detail below with reference to the diagrams.

[0290] Figure 29 is a flowchart of an intra-prediction method according to an embodiment of the present invention.

[0291] The reference sample line of the current block can be determined (S2901). The reference sample line refers to the set of reference samples contained in the line that is K-th away from the top and / or left of the current block. The reference sample can be derived from the reconstructed samples that have been encoded / decoded around the current block.

[0292] Index information for identifying the reference sample line of the current block within multiple reference sample lines can be signaled via the bitstream. Multiple reference sample lines may include at least one of the 1st, 2nd, 3rd, or 4th row / column at the top and / or left of the current block. Table 4 shows the index assigned to each of these reference sample lines. Table 4 assumes that the 1st, 2nd, and 4th rows / columns are used as candidate reference sample lines. [Table 4]

[0293] The reference sample line for the current block can be determined based on the current block's position, size, shape, or at least one of the predictive coding modes of adjacent blocks. For example, if the current block touches the boundary of an image, tile, slice, or coding tree unit, the first reference sample line can be determined as the reference sample line for the current block.

[0294] The reference sample line may include an upper reference sample located above the current block and a left reference sample located to the left of the current block. The upper and left reference samples can be derived from the restored samples around the current block. The restored samples may be in the state before the in-loop filter is applied.

[0295] Figure 30 shows the reference samples included in each reference sample line.

[0296] According to the current block's intra-prediction mode, a prediction sample can be obtained using at least one of the reference samples belonging to the reference sample line.

[0297] Next, the intra-prediction mode for the current block can be determined (S2902). In the case of the current block's intra-prediction mode, at least one of either a non-directional intra-prediction mode or a directional intra-prediction mode can be determined as the intra-prediction mode for the current block. The non-directional intra-prediction mode includes Planar (planar) and DC, while the directional intra-prediction mode includes 33 or 65 modes from the lower left diagonal direction to the upper right diagonal direction.

[0298] Figure 31 shows the intra-prediction mode.

[0299] Figure 31(a) shows 35 intra-prediction modes, and Figure 31(b) shows 67 intra-prediction modes.

[0300] More or fewer intra-prediction modes can be defined than those shown in Figure 31.

[0301] The Most Probable Mode (MPM) can be set based on the intra-prediction mode of the adjacent blocks adjacent to the current block. Here, adjacent blocks may include the left adjacent block adjacent to the left of the current block, and the top adjacent block adjacent to the top of the current block. If the coordinates of the top-left sample of the current block are (0, 0), the left adjacent block may include the sample at position (-1, 0), (-1, H-1), or (-1, (H-1) / 2), where H represents the height of the current block. The top adjacent block may include the sample at position (0, -1), (W-1, -1), or ((W-1) / 2, -1), where W represents the width of the current block.

[0302] When encoding adjacent blocks using general intraprediction, the MPM can be derived based on the intraprediction mode of the adjacent blocks. Specifically, the intraprediction mode of the left adjacent block can be set to the variable candIntraPredModeA, and the intraprediction mode of the upper adjacent block can be set to the variable candIntraPredModeB.

[0303] In this case, if the adjacent block is unavailable (for example, if the adjacent block has not yet been encoded / decoded, or if the adjacent block is located far from the image boundary), if the adjacent block is encoded with matrix-based intra-prediction and the adjacent block is encoded with inter-prediction, or if the adjacent block is in a different coding tree unit than the current block, the variable candIntraPredModeX (where X is A or B), derived based on the intra-prediction mode of the adjacent block, can be set as the default mode. Here, the default mode may include at least one of Planar mode, DC mode, vertical mode, or horizontal mode.

[0304] Alternatively, when encoding adjacent blocks using matrix-based intra-prediction, the intra-prediction mode corresponding to the index value for specifying one of the matrices can be set to candIntraPredModeX. To this end, a lookup table showing the mapping relationship between the index value for specifying the matrix and the intra-prediction mode can be stored in advance in the encoder and decoder.

[0305] The MPM can be derived based on the variables candIntraPredModeA and candIntraPredModeB. The number of MPMs included in the MPM list can be predefined in the encoder and decoder. For example, the number of MPMs could be 3, 4, 5, or 6. Alternatively, information indicating the number of MPMs can be signaled via the bitstream. Alternatively, the number of MPMs can be determined based on at least one of the following: the predictive coding mode of the adjacent block, the size of the current block, or its shape.

[0306] In the embodiments described later, we assume that the number of MPMs is 3, and these three MPMs are referred to as MPM[0], MPM[1], and MPM[2]. If the number of MPMs is greater than 3, the MPMs may include the three MPMs described in the embodiments described later.

[0307] If candIntraPredA and candIntraPredB are equal and candIntraPredA is in Planar mode or DC mode, then MPM[0] and MPM[1] can be set to Planar mode and DC mode, respectively. MPM[2] can be set to vertical intra-prediction mode, horizontal intra-prediction mode, or diagonal intra-prediction mode. Diagonal intra-prediction mode can be a lower-left diagonal intra-prediction mode, an upper-left intra-prediction mode, or an upper-right intra-prediction mode.

[0308] If candIntraPredA and candIntraPredB are equal and candIntraPredA is in directional intra-prediction mode, then MPM[0] can be set in the same way as candIntraPredA. MPM[1] and MPM[2] can be set as intra-prediction modes similar to candIntraPredA. An intra-prediction mode similar to candIntraPredA may be an intra-prediction mode where the index difference of candIntraPredA is ±1 or ±2. An intra-prediction mode similar to candIntraPredA can be derived using modulo operation (%) and an offset amount.

[0309] If candIntraPredA and candIntraPredB are different, MPM[0] can be set to be the same as candIntraPredA and MPM[1] can be set to be the same as candIntraPredB. In this case, if both candIntraPredA and candIntraPredB are in non-directional intra-prediction mode, MPM[2] can be set to vertical intra-prediction mode, horizontal intra-prediction mode, or diagonal intra-prediction mode. Alternatively, if at least one of candIntraPredA and candIntraPredB is in directional intra-prediction mode, MPM[2] can be set to Planar, DC, or an intra-prediction mode derived by adding or subtracting an offset value to the larger of candIntraPredA or candIntraPredB, where the offset value can be 1 or 2.

[0310] An MPM list containing multiple MPMs can be generated, and information indicating whether the same MPM as the current block's intra-prediction mode is included in the MPM list can be signaled via a bitstream. This information is a 1-bit flag and can be called the MPM flag. If the MPM flag indicates that the same MPM as the current block is included in the MPM list, index information for identifying one of the MPMs can be signaled via a bitstream. The MPM specified by the index information can be set as the current block's intra-prediction mode. If the MPM flag indicates that the same MPM as the current block is not included in the MPM list, residual mode information indicating one of the remaining intra-prediction modes excluding the MPM can be signaled via a bitstream. The residual mode information indicates the index value corresponding to the current block's intra-prediction mode when reassigning indices to the remaining intra-prediction modes excluding the MPM. The decoder can sort the MPMs in ascending order and compare the residual mode information with the MPMs to determine the current block's intra-prediction mode. For example, if the residual mode information is less than or equal to the MPM, the intra-predictive mode of the current block can be derived by adding 1 to the residual mode information.

[0311] Instead of setting the default mode to an MPM, information indicating whether the current block's intra-prediction mode is the default mode can be signaled via a bitstream. This information is a 1-bit flag, which may be called the default mode flag. The default mode flag can only be signaled if the MPM flag indicates that the same MPM as the current block is included in the MPM list. As previously mentioned, the default mode may include at least one of Planar, DC, vertical mode, or horizontal mode. For example, if Planar is set as the default mode, the default mode flag may indicate whether the current block's intra-prediction mode is Planar. If the default mode flag indicates that the current block's intra-prediction mode is not the default mode, one of the MPMs indicated by the index information can be set as the current block's intra-prediction mode.

[0312] When multiple intra-prediction modes are set as the default mode, an index information indicating one of the default modes can be signaled. The intra-prediction mode of the current block can be set as the default mode indicated by the index information.

[0313] If the index of the reference sample line in the current block is not available, the default mode can be disabled. This allows the default mode flag to be set to a predefined value (i.e., false) without signaling the default mode flag if the index of the reference sample line is not 0.

[0314] Once the intra-prediction mode for the current block is determined, prediction samples for the current block can be obtained based on the determined intra-prediction mode (S2903).

[0315] When DC mode is selected, predictive samples for the current block can be generated based on the mean value of the reference samples. Specifically, the values ​​of all samples within the predictive block can be generated based on the mean value of the reference samples. The mean value can be derived using at least one of the upper reference samples located above the current block and the left reference samples located to the left of the current block.

[0316] The number or range of reference samples used to derive the average may differ depending on the shape of the current block. For example, if the current block is a non-square block where the width is greater than the height, the average can be calculated using only the top reference samples. On the other hand, if the current block is a non-square block where the width is less than the height, the average can be calculated using only the left reference samples. In other words, if the width and height of the current block are different, the average can be calculated using only the reference samples adjacent to the longer side. Alternatively, the ratio of the width to height of the current block can be used to decide whether to calculate the average using only the top reference samples or only the left reference samples.

[0317] When Planar mode is selected, prediction samples can be obtained using horizontal and vertical prediction samples. Here, horizontal prediction samples are obtained based on left and right reference samples located on the same horizontal line as the prediction sample, and vertical prediction samples are obtained using top and bottom reference samples located on the same vertical line as the prediction sample. Here, a right reference sample can be generated by copying a reference sample adjacent to the upper right corner of the current block, and a bottom reference sample can be generated by copying a reference sample adjacent to the lower left corner of the current block. Horizontal prediction samples can be obtained based on an additive superposition of left and right reference samples, and vertical prediction samples can be obtained based on an additive superposition of top and bottom reference samples. In this case, the weight values ​​assigned to each reference sample can be determined based on the position of the prediction sample. Prediction samples can be obtained based on an averaging or additive superposition of horizontal and vertical prediction samples. When performing an additive superposition, the weight values ​​assigned to the horizontal and vertical prediction samples can be determined based on the position of the prediction sample.

[0318] When a direction prediction mode is selected, the parameters representing the predicted direction (or predicted angle) for the selected direction prediction mode can be determined. Table 5 below shows the intra-prediction parameters intraPredAng for each intra-prediction mode. [Table 5]

[0319] Table 5 shows the intra-direction parameters for each intra-prediction mode with an index ranging from 2 to 34, when 35 intra-prediction modes are defined. If more than 33 directional intra-prediction modes are defined, Table 5 can be further subdivided to set the intra-direction parameters for each directional intra-prediction mode.

[0320] After arranging the top and left reference samples of the current block in a single column, a predicted sample can be obtained based on the value of the intra-direction parameter. In this case, if the value of the intra-direction parameter is a negative number, the left and top reference samples can be arranged in a single column.

[0321] Figures 32 and 33 show examples of one-dimensional arrays in which reference samples are arranged in a single column.

[0322] Figure 32 shows an example of a vertical one-dimensional array in which reference samples are arranged vertically, and Figure 33 shows an example of a horizontal one-dimensional array in which reference samples are arranged horizontally. The embodiments of Figures 32 and 33 are described assuming that 35 intra-prediction modes are defined.

[0323] If the intra-prediction mode index is between 11 and 18, a horizontal one-dimensional array can be applied with the upper reference sample rotated counterclockwise. If the frame-in-prediction mode index is between 19 and 25, a vertical one-dimensional array can be applied with the left reference sample rotated clockwise. When arranging the reference samples in a single column, the angle of the intra-prediction mode can be taken into consideration.

[0324] Based on the intra-direction parameter, the reference sample determination parameter can be determined. The reference sample determination parameter may include a reference sample index for specifying the reference sample and a weight value parameter for determining the weight value applied to the reference sample.

[0325] The reference sample index iIdx and the weight parameter ifact can be obtained via Equations 2 and 3 below, respectively.

number

number

[0326] In equations 2 and 3, Pang represents the intra-direction parameter. The reference sample specified by the reference sample index iIdx corresponds to an integer pixel (Integer pel).

[0327] To derive prediction samples, at least one reference sample can be specified. Specifically, the location of the reference sample used to derive prediction samples can be specified, taking into account the slope of the prediction mode. For example, the reference sample index iIdx can be used to specify the reference sample used to derive prediction samples.

[0328] In this case, if the slope of the intra-prediction mode is not represented by a single reference sample, multiple reference samples can be interpolated to generate a prediction sample. For example, if the slope of the intra-prediction mode is a value between the slope between the prediction sample and the first reference sample and the slope between the prediction sample and the second reference sample, the first and second reference samples can be interpolated to obtain a prediction sample. In other words, if an angular line following the intra-prediction angle does not pass through a reference sample located at an integer pixel, the reference samples adjacent to the left, right, or above and below the position through which the angular line passes can be interpolated to obtain a prediction sample.

[0329] Equation 4 below shows an example of obtaining predicted samples based on reference samples.

number

[0330] In Equation 4, P represents a predicted sample, and Ref_1D represents one of the reference samples arranged in a one-dimensional array. In this case, the position of the reference sample can be determined based on the position (x, y) of the predicted sample and the reference sample index iIdx.

[0331] If the slope of the intra-prediction mode can be represented by a single reference sample, the weight value parameter ifact can be set to 0. Therefore, Equation 4 can be simplified to Equation 5 below.

number

[0332] Based on multiple intra-prediction modes, intra-prediction can be performed on the current block. For example, an intra-prediction mode can be derived for each prediction sample, and prediction samples can be derived based on the intra-prediction mode assigned to each prediction sample.

[0333] Alternatively, an intra-prediction mode can be derived for each region, and intra-prediction can be performed for each region based on the intra-prediction mode assigned to each region. Here, the region may contain at least one sample. The size or shape of the region can be adaptively determined based on at least one of the current block size, shape, or intra-prediction mode. Alternatively, the encoder and decoder can predefine the size or shape of the region regardless of the current block size or shape.

[0334] Alternatively, intraprediction can be performed based on each of multiple intrapredictions, and the final predicted sample can be derived based on the averaging or additive superposition of multiple predicted samples obtained through multiple intrapredictions. For example, an intraprediction can be performed based on a first intraprediction mode to obtain a first predicted sample, and an intraprediction can be performed based on a second intraprediction mode to obtain a second predicted sample. Then, the final predicted sample can be obtained based on the averaging or additive superposition of the first and second predicted samples. In this case, the weight values ​​assigned to the first and second predicted samples can be determined by considering whether the first intraprediction mode is a non-directional / directional prediction mode, whether the second intraprediction mode is a non-directional / directional prediction mode, or at least one of the intraprediction modes of the adjacent block.

[0335] Multiple intra-prediction modes may be combinations of non-directional intra-prediction modes and directional prediction modes, combinations of directional prediction modes, or combinations of non-directional prediction modes.

[0336] Figure 34 shows the angle formed between the directional intra-prediction mode and a straight line parallel to the x-axis.

[0337] As shown in Figure 34, the direction prediction mode can exist between the lower left diagonal and the upper right diagonal. In terms of the angle formed by the x-axis and the direction prediction mode, the direction prediction mode can exist between 45 degrees (lower left diagonal) and -135 degrees (upper right diagonal).

[0338] If the current block is not square in shape, based on the intra-prediction mode of the current block, the predicted sample is derived using a reference sample located further away from the predicted sample, rather than a reference sample close to the predicted sample, among the reference samples located on the corners that follow the intra-prediction angle.

[0339] Figure 35 shows an example of obtaining a prediction sample when the current block is not square.

[0340] For example, suppose the current block is a non-square shape, as shown in Figure 35(a), and the current block's intra-prediction mode is a directional intra-prediction mode with an angle between 0 and 45 degrees. In this case, when deriving a predicted sample A near the right-hand column of the current block, it is possible that, among the reference samples located on the angular mode following the angle, a left-hand reference sample L, which is far from the predicted sample, may be used instead of an upper reference sample T, which is close to the predicted sample.

[0341] As another example, suppose the current block is a non-square shape, with height greater than width, as shown in Figure 35(b), and the current block's intra-prediction mode is a directional intra-prediction mode with angles between -90 and -135 degrees. In this case, when deriving a predicted sample A near the bottom of the current block, it may occur that, among the reference samples located on the angular mode following the angle, an upper reference sample T, which is far from the predicted sample, is used instead of a left reference sample L, which is close to the predicted sample.

[0342] To solve the above problem, if the current block is not square, the intra-prediction mode for the current block can be replaced with the intra-prediction mode in the reverse direction. This allows for the use of directional prediction modes with angles larger or smaller than the directional prediction modes shown in Figure 31 for non-square blocks. Such directional intra-prediction modes can be defined as wide-angle intra-prediction modes. Wide-angle intra-prediction modes represent directional intra-prediction modes that do not fall within the range of 45 degrees to -135 degrees.

[0343] Figure 36 shows the wide-angle intra-prediction mode.

[0344] In the example shown in Figure 36, the intra-prediction mode with an index of -1 to -14 and the intra-prediction mode with an index of 67 to 80 represent the wide-angle intra-prediction mode.

[0345] Figure 36 illustrates 14 wide-angle intra-prediction modes with angles greater than 45 degrees (-1 to -14) and 14 wide-angle intra-prediction modes with angles less than -135 degrees (67 to 80), but it is possible to define a larger or smaller number of wide-angle intra-prediction modes.

[0346] When using wide-angle intra-prediction mode, the length of the upper reference sample can be set to 2W+1 and the length of the left reference sample can be set to 2H+1.

[0347] When using the wide-angle intra-prediction mode, a reference sample T can be used to predict sample A shown in Figure 35(a), and a reference sample L can be used to predict sample A shown in Figure 35(b).

[0348] By adding the existing intra-prediction modes to N wide-angle intra-prediction modes, a total of 67+N intra-prediction modes can be used. For example, Table 6 shows the intra-directional parameters of the intra-prediction modes when 20 wide-angle intra-prediction modes are defined. [Table 6]

[0349] The intra-direction parameter can be set differently based on at least one of the current block size, shape, or reference sample line. For example, the intra-direction parameter for a particular intra-prediction mode may differ depending on whether the current block is square or not. For instance, the intra-direction parameter intraPredAngle for intra-prediction mode 15 may have a larger value when the current block is square than when the current block is not square.

[0350] Alternatively, the intra-direction parameter intraPredAngle of intra-prediction mode 75 can have a larger value when the index of the reference sample line of the current block is 1 or greater than when the index of the reference sample line of the current block is 0.

[0351] If the current block is non-square and the intra-prediction mode of the current block obtained in step S2902 falls within the conversion range, the intra-prediction mode of the current block can be converted to a wide-angle intra-prediction mode. The conversion range can be determined based on at least one of the size, shape, or ratio of the current block, where the ratio may represent the ratio between the width and height of the current block.

[0352] If the current block is a non-square with a width greater than its height, the conversion range can be set from the intra-prediction mode index in the upper right diagonal direction (e.g., 66) to (the intra-prediction mode index in the upper right diagonal direction - N), where N can be determined based on the proportions of the current block. If the intra-prediction mode of the current block falls within the conversion range, the intra-prediction mode can be converted to a wide-angle intra-prediction mode. This conversion can be performed by subtracting a predefined value from the intra-prediction mode, which may be the total number of intra-prediction modes excluding the wide-angle intra-prediction mode (e.g., 67).

[0353] Based on the above embodiment, the intra-prediction modes between 66 and 53 can be converted to wide-angle intra-prediction modes between -1 and -14, respectively.

[0354] If the current block is a non-square with height greater than width, the conversion range can be set from the lower-left diagonal intra-prediction mode index (e.g., 2) to (lower-left diagonal intra-prediction mode index + M), where M can be determined based on the proportions of the current block. If the current block's intra-prediction mode falls within the conversion range, the intra-prediction mode can be converted to a wide-angle intra-prediction mode. This conversion can be performed by adding a predefined value to the intra-prediction mode, which may be the total number of directional intra-prediction modes excluding the wide-angle intra-prediction mode (e.g., 65).

[0355] Based on the above embodiment, the intra-prediction modes between numbers 2 and 15 can be converted to wide-angle intra-prediction modes between numbers 67 and 80, respectively.

[0356] Hereafter, intra-prediction modes that fall within the conversion range will be referred to as wide-angle intra-alternative prediction modes.

[0357] The conversion range can be determined based on the ratio of the current blocks. For example, Tables 7 and 8 show the conversion ranges when 35 intra-prediction modes are defined, excluding the wide-angle intra-prediction mode, and when 67 intra-prediction modes are defined, respectively. [Table 7] [Table 8]

[0358] As shown in Tables 7 and 8, the number of wide-angle intra-alternative prediction modes included in the conversion range can vary depending on the current block ratio.

[0359] Using a wide-angle intra-prediction mode in addition to existing intra-prediction modes may decrease encoding efficiency because it increases the resources required to encode the wide-angle intra-prediction mode. Therefore, encoding efficiency can be improved by encoding an alternative intra-prediction mode related to the wide-angle intra-prediction mode instead of encoding the wide-angle intra-prediction mode directly.

[0360] For example, when encoding the current block using wide-angle intra-prediction mode 67, the current block's intra-prediction mode can be encoded with its alternative wide-angle intra-prediction mode 2. Furthermore, when encoding the current block using wide-angle intra-prediction mode -1, the current block's intra-prediction mode can be encoded with its alternative wide-angle intra-prediction mode 66.

[0361] The decoder can decode the intra-prediction mode of the current block and determine whether the decoded intra-prediction mode falls within the conversion range. If the decoded intra-prediction mode is a wide-angle alternative intra-prediction mode, the intra-prediction mode can be converted to a wide-angle intra-prediction mode.

[0362] Alternatively, when encoding the current block in wide-angle intra-predictive mode, you can also encode wide-angle intra-predictive mode directly.

[0363] The encoding of the intra-prediction mode can be implemented based on the MPM list. Specifically, when encoding adjacent blocks in a wide-angle intra-prediction mode, the MPM can be set based on the wide-angle alternative intra-prediction mode corresponding to the wide-angle intra-prediction mode. For example, when encoding adjacent blocks in a wide-angle intra-prediction mode, the variable candIntraPredX (where X is A or B) can be set as the wide-angle alternative intra-prediction mode.

[0364] When prediction blocks are generated as a result of performing intraprediction, the prediction samples can be updated based on the position of each prediction sample contained in the prediction block. This update method can be called an intra-weighted prediction method based on sample position (or a position-dependent prediction combination (PDPC)).

[0365] The decision to use PDPC can be made by considering the current block's intra-prediction mode, the current block's reference sample line, the current block's size, or the color component. For example, PDPC can be used if the current block's intra-prediction mode is at least one of Planar mode, DC mode, vertical mode, horizontal mode, a mode with an index value smaller than vertical, or a mode with an index value larger than horizontal. Alternatively, PDPC can be used only if at least one of the current block's width or height is greater than 4. Alternatively, PDPC can be used only if the index of the current block's reference image line is 0. Alternatively, PDPC can be used only if the index of the current block's reference image line is greater than or equal to a predefined value. Alternatively, PDPC can be used only for the luminance component. Alternatively, the decision to use PDPC can be made depending on whether two or more of the above-listed conditions are met.

[0366] As another example, information indicating whether or not PDPC applies can be signaled via a bitstream.

[0367] When acquiring prediction samples via intra-prediction samples, a reference sample used to correct the acquired prediction sample can be determined based on the position of the acquired prediction sample. For convenience of explanation, in the embodiments described later, the reference sample used to correct the prediction sample will be referred to as the PDPC reference sample. Furthermore, the prediction sample acquired via intra-prediction will be referred to as the first prediction sample, and the prediction sample acquired by correcting the first prediction sample will be referred to as the second prediction sample.

[0368] Figure 37 shows the application of PDPC.

[0369] The first predicted sample can be corrected using at least one PDPC reference sample. The PDPC reference sample may include at least one of the following: a reference sample adjacent to the upper left corner of the current block, an upper reference sample located at the top of the current block, or a left reference sample located to the left of the current block.

[0370] At least one of the reference samples belonging to the current block's reference sample line can be set as the PDPC reference sample. Alternatively, at least one of the reference samples belonging to the reference sample line at index 0 can be set as the PDPC reference sample, regardless of the current block's reference sample line. For example, even if the first prediction sample is obtained using a reference sample in the reference sample line at index 1 or index 2, the second prediction sample can be obtained using a reference sample in the reference sample line at index 0.

[0371] The number or position of PDPC reference samples used to correct the first predicted sample can be determined by considering at least one of the current block's intra-prediction mode, current block size, current block shape, or the position of the first predicted sample.

[0372] For example, if the current block's intra-prediction mode is Planar mode or DC mode, a second prediction sample can be obtained using the top reference sample and the left reference sample. In this case, the top reference sample may be a reference sample perpendicular to the first prediction sample (e.g., a reference sample with the same x-coordinate), and the left reference sample may be a reference sample horizontal to the first prediction sample (e.g., a reference sample with the same y-coordinate).

[0373] If the current block's intra-prediction mode is horizontal intra-prediction mode, a second prediction sample can be obtained using the upper reference sample. In this case, the upper reference sample may be a reference sample perpendicular to the first prediction sample.

[0374] If the current block's intra-prediction mode is vertical intra-prediction mode, a second prediction sample can be obtained using the left-side reference sample. In this case, the left-side reference sample may be a reference sample horizontal to the first prediction sample.

[0375] If the current block's intra-prediction mode is either the lower-left diagonal intra-prediction mode or the upper-right diagonal intra-prediction mode, a second prediction sample can be obtained based on the upper-left reference sample, the top reference sample, and the left reference sample. The upper-left reference sample may be a reference sample adjacent to the upper-left corner of the current block (e.g., a reference sample at position (-1, -1)). The top reference sample may be a reference sample located diagonally to the upper right of the first prediction sample, and the left reference sample may be a reference sample located diagonally to the lower left of the first prediction sample.

[0376] In summary, if the position of the first predicted sample is (x, y), then R(-1, -1) can be set as the upper-left reference sample, and R(x+y+1, -1) or R(x, -1) can be set as the upper reference sample. Additionally, R(-1, x+y+1) or R(-1, y) can be set as the left-side reference sample.

[0377] As another example, the position of the left reference sample or the top reference sample can be determined by considering at least one of the following: the shape of the current block, or whether or not wide-angle intra mode is applied.

[0378] Specifically, if the current block's intra-prediction mode is wide-angle intra-prediction mode, a reference sample located diagonally opposite the first prediction sample, separated by an offset value, can be set as the PDPC reference sample. For example, the upper reference sample R(x+y+k+1, -1) and the left reference sample R(-1, x+y-k+1) can be set as the PDPC reference samples.

[0379] In this case, the offset value k can be determined based on the wide-angle intra-prediction mode. Equations 6 and 7 show an example of deriving the offset value based on the wide-angle intra-prediction mode.

number

number

[0380] A second predicted sample can be determined based on an additive superposition operation between the first predicted sample and the PDPC reference sample. For example, the second predicted sample can be obtained based on the following formula 8.

number

[0381] In Equation 8, RL represents the left reference sample, RT represents the top reference sample, and RTL represents the top-left reference sample. pred(x, y) represents the predicted sample at position (x, y). wL represents the weight assigned to the left reference sample, wT represents the weight assigned to the top reference sample, and wTL represents the weight assigned to the top-left reference sample. The weight assigned to the first predicted sample can be derived by subtracting the weight assigned to the reference sample from the maximum value. For convenience of explanation, the weight assigned to the PDPC reference sample is called the PDPC weight.

[0382] The weight value assigned to each reference sample can be determined based on at least one of the current block's intra-prediction mode or the position of the first predicted sample.

[0383] For example, at least one of wL, wT, or wTL may be proportional to or inversely proportional to at least one of the x-axis or y-axis coordinate values ​​of the predicted sample. Alternatively, at least one of wL, wT, or wTL may be proportional to or inversely proportional to at least one of the width or height of the current block.

[0384] If the current block's intra-prediction mode is DC mode, the PDPC weight values ​​can be determined as shown in Equation 9 below.

number

[0385] In Equation 9, x and y represent the positions of the first predicted sample.

[0386] In Equation 9, the variable `shift` used for bit shift operations can be determined based on the width or height of the current block. For example, the variable `shift` can be derived based on Equation 10 or Equation 11 below.

number

number

[0387] Alternatively, the variable `shift` can be derived by considering the intra-direction parameter of the current block.

[0388] The number and types of parameters used to derive the variable `shift` can be determined differently depending on the intra-prediction mode of the current block. For example, if the current block's intra-prediction mode is Planar mode, DC mode, vertical mode, or horizontal mode, the variable `shift` can be derived using the width and height of the current block, as shown in Equation 10 or Equation 11. If the current block's intra-prediction mode is an intra-prediction mode with a larger index than the vertical intra-prediction mode, the variable `shift` can be derived using the height and intra-direction parameter of the current block. If the current block's intra-prediction mode is an intra-prediction mode with a smaller index than the horizontal intra-prediction mode, the variable `shift` can be derived using the width and intra-direction parameter of the current block.

[0389] If the current block's intra-prediction mode is Planar mode, the value of wTL can be set to 0. wL and wT can be derived based on equation 12 below.

number

[0390] If the current block's intra-prediction mode is horizontal intra-prediction mode, you can set wT to 0 and set wTL and wL to the same value. On the other hand, if the current block's intra-prediction mode is vertical intra-prediction mode, you can set wL to 0 and set wTL and wT to the same value.

[0391] If the current block's intra-prediction mode is an intra-prediction mode directed upwards and to the right, with a larger index value than the vertical intra-prediction mode, the PDPC weight values ​​can be derived as shown in equation 13 below.

number

[0392] On the other hand, if the current block's intra-prediction mode is an intra-prediction mode directed downward to the left, with a smaller index value than the horizontal intra-prediction mode, the PDPC weight value can be derived as shown in equation 14 below.

number

[0393] As in the example described above, the PDPC weight values ​​can be determined based on the positions x and y of the predicted sample.

[0394] As another example, the weight values ​​assigned to each PDPC reference sample can also be determined on a subblock basis. Predicted samples within a subblock can share the same PDPC weight values.

[0395] The encoder and decoder can predefine the size of the subblocks, which are the basic units used to determine the weight values. For example, the weight values ​​can be determined for each subblock of a 2x2 or 4x4 size.

[0396] Alternatively, the size, shape, or number of subblocks can be determined based on the size or shape of the current block. For example, a coded block can be divided into four subblocks, regardless of its size. Or, based on its size, a coded block can be divided into four or sixteen subblocks.

[0397] Alternatively, the size, shape, or number of subblocks can be determined based on the current block's intra-prediction mode. For example, if the current block's intra-prediction mode is horizontal, N columns (or N rows) can be set as one subblock; conversely, if the current block's intra-prediction mode is vertical, N rows (or N columns) can be set as one subblock.

[0398] Equations 15-17 show examples of determining PDPC weight values ​​for a 2x2 subblock. Equation 15 shows an example where the intra-prediction mode of the current block is DC mode.

number

[0399] In Equation 15, K may be a value determined based on the size of the subblock or the intra-prediction mode.

[0400] Equation 16 shows an example where the current block's intra-prediction mode is an upward-right intra-prediction mode with a larger index value than the vertical intra-prediction mode.

number

[0401] Equation 17 shows an example where the current block's intra-prediction mode is a downward-left intra-prediction mode with a smaller index value than the horizontal intra-prediction mode.

number

[0402] In equations 15-17, x and y indicate the position of the reference sample within the subblock. The reference sample may be located in the upper left, center, or lower right of the subblock.

[0403] Equations 18-20 show examples of determining PDPC weight values ​​for a 4x4 subblock. Equation 18 shows an example where the intra-prediction mode of the current block is DC mode.

number

[0404] Equation 19 shows an example where the current block's intra-prediction mode is an upward-right intra-prediction mode with a larger index value than the vertical intra-prediction mode.

number

[0405] Equation 20 shows an example where the current block's intra-prediction mode is a downward-left intra-prediction mode with a smaller index value than the horizontal intra-prediction mode.

number

[0406] In the embodiments described above, the determination of PDPC weight values ​​was explained by considering the position of the first predicted sample or the predicted samples contained in the subblock. The PDPC weight values ​​can also be determined by further considering the shape of the current block.

[0407] For example, in DC mode, the method for deriving the PDPC weight values ​​can differ depending on whether the current block is a non-square shape where the width is greater than the height, or a non-square shape where the height is greater than the width.

[0408] Equation 21 shows an example of deriving PDPC weight values ​​when the current block is a non-square shape where the width is greater than the height, and Equation 22 shows an example of deriving PDPC weight values ​​when the current block is a non-square shape where the height is greater than the width.

number

number

[0409] If the current block is not square, the wide-angle intra-prediction mode can be used to predict the current block. In this way, even when the wide-angle intra-prediction mode is applied, PDPC can be applied to update the first prediction sample.

[0410] When wide-angle intra-prediction is applied to the current block, the PDPC weight values ​​can be determined by considering the shape of the encoded block.

[0411] For example, if the current block is a non-square with a width greater than its height, based on the position of the first predicted sample, the top reference sample located to the upper right of the first predicted sample may be closer to the first predicted sample than the left reference sample located to the lower left of the first predicted sample. This allows the weight applied to the top reference sample to be set to a larger value than the weight applied to the left reference sample when correcting the first predicted sample.

[0412] On the other hand, if the current block is a non-square with height greater than width, based on the position of the first predicted sample, the left reference sample located to the lower left of the first predicted sample may be closer to the first predicted sample than the upper reference sample located to the upper right of the first predicted sample. This allows the weight applied to the left reference sample to be set to a larger value than the weight applied to the upper reference sample when correcting the first predicted sample.

[0413] Equation 23 shows an example of deriving PDPC weight values ​​when the current block's intra-prediction mode is a wide-angle intra-prediction mode with an index greater than 66.

number

[0414] Equation 24 shows an example of deriving PDPC weight values ​​when the current block's intra-prediction mode is a wide-angle intra-prediction mode with an index less than 0.

number

[0415] The PDPC weight values ​​can also be determined based on the current block ratio. The current block ratio represents the ratio of the width to the height of the current block and can be defined as shown in Equation 25 below.

number

[0416] The method for deriving PDPC weight values ​​can be variably determined based on the current block's intra-prediction mode.

[0417] For example, equations 26 and 27 show examples of deriving PDPC weight values ​​when the intra-prediction mode of the current block is DC mode. Specifically, equation 26 is an example where the current block is a non-square shape with width greater than height, and equation 27 is an example where the current block is a non-square shape with height greater than width.

number

number

[0418] Equation 28 shows an example of deriving PDPC weight values ​​when the current block's intra-prediction mode is a wide-angle intra-prediction mode with an index greater than 66.

number

[0419] Equation 29 shows an example of deriving PDPC weight values ​​when the current block's intra-prediction mode is a wide-angle intra-prediction mode with an index less than 0.

number

[0420] The residual image can be derived by subtracting the predicted image from the original image. In this case, if the residual image is in the frequency domain, removing the high-frequency components will not significantly degrade the video quality. This means that by reducing the value of the high-frequency components or setting them to 0, it is possible to improve compression efficiency without causing obvious visual distortion. Reflecting the above characteristics, the current block can be transformed to decompose the residual image into two-dimensional frequency components. This transformation can be performed using transformation methods such as the Discrete Cosine Transform (DCT) or the Discrete Sine Transform (DST).

[0421] DCT uses the cosine transform to decompose (or transform) a residual image into two-dimensional frequency components, while DST uses the sine transform to decompose (or transform) a residual image into two-dimensional frequency components. As a result of the transformation of the residual image, the frequency components can be represented by fundamental images. For example, when performing a DCT transformation on an N×N block, N2 fundamental pattern components can be obtained. The transformation allows us to obtain the size of each fundamental pattern component contained in the N×N block. Based on the transformation technique used, the size of the fundamental pattern component can be called the DCT coefficient or DST coefficient.

[0422] The DCT (Digital Transformation) technique is primarily used to transform images with a high distribution of low-frequency components other than zero. The DST (Digital Transformation) technique is primarily used to transform images with a high distribution of high-frequency components.

[0423] Residual images can also be transformed using transformation techniques other than DCT or DST.

[0424] Hereinafter, the process of converting a residual image into two-dimensional frequency components will be referred to as two-dimensional image conversion. Furthermore, the size of the fundamental pattern components obtained as a result of the conversion will be called the conversion coefficient. For example, the conversion coefficient may refer to the DCT coefficient or the DST coefficient. When the linear and quadratic conversions described later are applied simultaneously, the conversion coefficient can represent the size of the fundamental pattern components generated as a result of the quadratic conversion.

[0425] The transformation technique can be determined on a block-by-block basis. The transformation technique can be determined based on at least one of the following: the predictive coding mode of the current block, the size of the current block, or the shape of the current block. For example, if the current block is coded in intra-predictive mode and the size of the current block is less than N×N, the transformation can be performed using the DST transformation technique. On the other hand, if the above conditions are not met, the transformation can be performed using the DCT transformation technique.

[0426] In residual images, it is possible to skip 2D image transformations for some blocks. This skipping of 2D image transformations is called a Transform Skip. When a Transform Skip is applied, quantization can be applied to residual values ​​that have not been transformed.

[0427] Using DCT or DST, the current block can be transformed, and then the transformed current block can be re-transformed. In this case, the transformation based on DCT or DST can be defined as a primary transformation, and the re-transformation of the block to which the primary transformation has been applied can be defined as a secondary transformation.

[0428] A primary transformation can be performed using one of several candidate transformation cores. For example, a primary transformation can be performed using DCT2, DCT8, or DCT7.

[0429] Different conversion cores can be used for the horizontal and vertical directions. Information indicating the combination of horizontal and vertical conversion cores can also be signaled via a bitstream.

[0430] The units for performing primary and secondary transformations may differ. For example, a primary transformation can be performed on an 8x8 block, and then a secondary transformation can be performed on a 4x4 subblock within the transformed 8x8 block. In this case, the transformation coefficient for the remaining area where the secondary transformation is not performed can also be set to 0.

[0431] Alternatively, a linear transformation can be performed on a 4x4 block, and then a quadratic transformation can be performed on an 8x8 area containing the transformed 4x4 block.

[0432] Information indicating whether or not to perform a quadratic transformation can be signaled via the bitstream.

[0433] Alternatively, the decision to perform a quadratic transformation can be based on whether the horizontal transformation core and the vertical transformation core are the same. For example, a quadratic transformation can only be performed if the horizontal transformation core and the vertical transformation core are identical. Or, a quadratic transformation can only be performed if the horizontal transformation core and the vertical transformation core are different.

[0434] Alternatively, a quadratic transformation may be possible only if the horizontal and vertical transformations use predefined transformation cores. For example, a quadratic transformation may be possible if the DCT2 transformation core is used for both the horizontal and vertical transformations.

[0435] Alternatively, you can decide whether or not to perform a quadratic transformation based on the number of non-zero transformation coefficients in the current block. For example, you can set it so that a quadratic transformation is not used if the number of non-zero transformation coefficients in the current block is less than or equal to a threshold, and to use a quadratic transformation if the number of non-zero transformation coefficients in the current block exceeds the threshold. You can also set it so that a quadratic transformation is used only if the current block is encoded with intra-prediction.

[0436] Based on the current block shape, the size and shape of the sub-blocks that will perform the quadratic transformation can be determined.

[0437] Figures 38 and 39 show the subblocks in which the quadratic transformation is performed.

[0438] If the current block is a square, a linear transformation can be performed, and then a quadratic transformation can be performed on the N×N subblock in the upper left corner of the current block. For example, if the current block is an 8×8 encoded block, a linear transformation can be performed on the current block, and then a quadratic transformation can be performed on the 4×4 subblock in the upper left corner of the current block (see Figure 38).

[0439] If the current block is a non-square with a width greater than four times its height, a linear transformation can be performed, followed by a quadratic transformation on the top-left subblock of the current block with a size of (kN) × (4kN). For example, if the current block is a 16 × 4 non-square, a linear transformation can be performed on the current block, followed by a quadratic transformation on the top-left subblock of the current block with a size of 2 × 8 (see Figure 39(a)).

[0440] If the current block is a non-square whose height is more than four times its width, a linear transformation can be performed, followed by a quadratic transformation on the top-left subblock of the current block with a size of (4kN) × (kN). For example, if the current block is a 16 × 4 non-square, a linear transformation can be performed on the current block, followed by a quadratic transformation on the top-left subblock of the current block with a size of 2 × 8 (see Figure 39(b)).

[0441] The decoder can perform the inverse of the quadratic transformation (second inverse transformation), and then perform the inverse of the linear transformation (first inverse transformation) on the result of that transformation. The results of the second and first inverse transformations, along with the residual signal of the current block, can be obtained.

[0442] Quantization is used to reduce the energy of a block, and the quantization process involves dividing the transformation coefficient by a specific constant. This constant can be derived from a quantization parameter, which can be defined as a value between 1 and 63.

[0443] When the encoder performs the transformation and quantization, the decoder can obtain the residual block via inverse quantization and inverse transformation. The decoder can then add the predicted block and the residual block to obtain the restored block of the current block.

[0444] Information indicating the transformation type of the current block can be signaled via the bitstream. This information may be an index, tu_mts_idx, indicating one of the combinations of horizontal and vertical transformation types.

[0445] Based on the candidate conversion types specified by the index information tu_mts_idx, the vertical and horizontal conversion cores can be determined. Tables 9 and 10 show the combinations of conversion types based on tu_mts_idx. [Table 9] [Table 10]

[0446] The conversion type can be determined as either DCT2, DST7, DCT8, or conversion skip. Alternatively, candidate combinations of conversion types can be constructed using only conversion cores, excluding conversion skip.

[0447] When using Table 9, if tu_mts_idx is 0, conversion skipping can be applied to both the horizontal and vertical directions. If tu_mts_idx is 1, DCT2 can be applied to both the horizontal and vertical directions. If tu_mts_idx is 3, DCT8 can be applied to the horizontal direction and DCT7 to the vertical direction.

[0448] When using Table 10, if tu_mts_idx is 0, DCT2 can be applied horizontally and vertically. If tu_mts_idx is 1, a conversion skip can be applied horizontally and vertically. If tu_mts_idx is 3, DCT8 can be applied horizontally and DCT7 can be applied vertically.

[0449] The decision of whether or not to encode index information can be made based on at least one of the current block's size, shape, or the number of non-zero coefficients. For example, if the number of non-zero coefficients is less than or equal to a threshold, the default transformation type can be applied to the current block without signaling the index information. Here, the default transformation type could be DST7. Alternatively, the default mode may differ depending on the current block's size, shape, or intra-prediction mode.

[0450] The threshold can be determined based on the size or shape of the current block. For example, if the size of the current block is less than or equal to 32x32, the threshold can be set to 2, and if the size of the current block is greater than 32x32 (for example, if the current block is a 32x64 or 64x32 encoded block), the threshold can be set to 4.

[0451] Multiple lookup tables can be stored in the encoder / decoder beforehand. At least one of the following may differ among the multiple lookup tables: the index value assigned to the candidate combination of transformation types, the type of candidate combination of transformation types, or the number of candidate combinations of transformation types.

[0452] The lookup table for the current block can be selected based on at least one of the following: the size, shape, predictive coding mode, intra-predictive mode, whether to apply a quadratic transformation, or whether to apply a transformation skip to an adjacent block.

[0453] For example, if the current block size is less than or equal to 4x4, or if the current block is encoded using interpretation, the lookup table in Table 9 can be used. If the current block size is greater than 4x4, or if the current block is encoded using intrablock prediction, the lookup table in Table 10 can be used.

[0454] Alternatively, information indicating one of several lookup tables can be signaled via the bitstream. Based on this information, the decoder can select the lookup table for the current block.

[0455] As another example, the index assigned to a candidate combination of transformation types can be adaptively determined based on at least one of the following: the size, shape, predictive coding mode, intra-predictive mode, whether to apply a quadratic transformation, or whether to apply a transformation skip to an adjacent block. For example, if the current block size is 4x4, the index value assigned to a transformation skip may be smaller than the index value assigned to a transformation skip if the current block size is greater than 4x4. Specifically, if the current block size is 4x4, an index of 0 can be assigned to the transformation skip, and if the current block is greater than 4x4 but less than or equal to 16x16, an index greater than 0 (e.g., index 1) can be assigned to the transformation skip. If the current block is greater than 16x16, the maximum value (e.g., 5) can be assigned to the transformation skip index.

[0456] Alternatively, if the current block is encoded with interpretation, you can assign index 0 to the transformation skip. If the current block is encoded with intrapretation, you can assign an index greater than 0 (e.g., index 1) to the transformation skip.

[0457] Alternatively, if the current block is a 4x4 size block encoded with interprediction, index 0 can be assigned to the transformation skip. On the other hand, if the current block is not encoded with interprediction, or if the current block is larger than 4x4, an index greater than 0 (e.g., index 1) can be assigned to the transformation skip.

[0458] It is also possible to define and use conversion type combination candidates different from those enumerated in Tables 9 and 10. For example, a conversion skip can be applied to either a horizontal or vertical conversion, and a conversion type combination candidate to which a conversion core such as DCT7, DCT8, or DST2 is applied can be used for the other. In this case, it can be determined whether to use a conversion skip as a horizontal or vertical conversion type candidate based on at least one of the current block size (e.g., width and / or height), shape, predictive coding mode, or intra-predictive mode.

[0459] Alternatively, information indicating whether a particular conversion type candidate is available can be signaled via the bitstream. For example, flags indicating whether horizontal and vertical conversion skips are available as conversion type candidates can be signaled. Based on these flags, it can be determined whether a particular conversion type candidate is included among a group of conversion type combination candidates.

[0460] Alternatively, information indicating whether a particular conversion type candidate is applicable to the current block can be signaled via the bitstream. For example, a flag cu_mts_flag indicating whether DCT2 is applied horizontally and vertically can be signaled. If the value of cu_mts_flag is 1, DCT2 can be set as the vertical and horizontal conversion core. If the value of cu_mts_flag is 0, DCT8 or DST7 can be set as the vertical and horizontal conversion core. Alternatively, if the value of cu_mts_flag is 0, information tu_mts_idx specifying one of several conversion type combination candidates can be signaled.

[0461] If the current block is a non-square with a width greater than its height, or a non-square with a height greater than its width, the encoding of cu_mts_flag can be omitted, and the value of cu_mts_flag can be considered as 0.

[0462] The number of available transformation type combinations can be set differently depending on the current block size, shape, or intra-prediction mode. For example, if the current block is square, three or more transformation type combinations can be used, and if the current block is not square, two transformation type combinations can be used. Alternatively, if the current block is square, only transformation type combinations where the horizontal transformation type and the vertical transformation type are different can be used.

[0463] If there are three or more possible combinations of transformation types that the current block can use, an index information tu_mts_idx indicating one of the possible combinations of transformation types can be signaled. On the other hand, if there are two possible combinations of transformation types that the current block can use, a flag mts_flag indicating one of the possible combinations of transformation types can be signaled. Table 11 below shows the process for encoding information to specify the possible combinations of transformation types based on the shape of the current block. [Table 11]

[0464] Depending on the shape of the current block, the index of the candidate combinations of transformation types can be rearranged (or rearranged). For example, the index assigned to the candidate combinations of transformation types may differ depending on whether the current block is square or not. For instance, if the current block is square, a combination of transformation types can be selected based on Table 12, and if the current block is not square, a combination of transformation types can be selected based on Table 13. [Table 12] [Table 13]

[0465] The transformation type can be determined based on the number of horizontal or vertical non-zero coefficients of the current block. Here, the number of horizontal non-zero coefficients is the number of non-zero coefficients contained in 1 × N (where N is the width of the current block), and the number of vertical non-zero coefficients is the number of non-zero coefficients contained in N × 1 (where N is the height of the current block). If the maximum number of horizontal non-zero coefficients is less than or equal to the threshold, a linear transformation type can be applied horizontally; if the maximum number of horizontal non-zero coefficients exceeds the threshold, a quadratic transformation type can be applied horizontally. If the maximum number of vertical non-zero coefficients is less than or equal to the threshold, a linear transformation type can be applied vertically; if the maximum number of vertical non-zero coefficients exceeds the threshold, a quadratic transformation type can be applied vertically.

[0466] Figure 40 is a diagram illustrating an example of determining the transformation type of the current block.

[0467] For example, if the current block is encoded in intraprediction and the maximum value of the horizontal non-zero coefficients of the current block is less than or equal to 2 (see Figure 40(a)), then DST7 can be determined as the horizontal transformation type.

[0468] If the current block is encoded using intraprediction and the maximum value of the vertical non-zero coefficients of the current block exceeds 2 (see Figure 40(b)), then DCT2 or DCT8 can be determined as the vertical transformation type.

[0469] The residual coefficients can be encoded in a transformation unit or a sub-transformation unit. Here, the residual coefficients refer to the transformation coefficients generated by the transformation, the transformation skip coefficients generated by the transformation skip, or the quantized coefficients generated by quantizing the said transformation coefficients or coefficients.

[0470] A transformation unit can represent a block on which a primary or secondary transformation has been performed. A subtransformation unit represents a block smaller than the transformation unit. For example, a subtransformation unit may be a 4x4, 2x8, or 8x2 sized block.

[0471] Based on the size or shape of the current block, at least one of the sizes or shapes of the sub-conversion unit can be determined. For example, if the current block is a non-square with a width greater than its height, the sub-conversion unit can also be set to a non-square with a width greater than its height (e.g., 8x2). If the current block is a non-square with a height greater than its width, the sub-conversion unit can also be set to a non-square with a height greater than its width (e.g., 2x8). If the current block is a square, the sub-conversion unit can also be set to a square (e.g., 4x4).

[0472] If the current block contains multiple subtransform units, the subtransform units can be encoded / decoded sequentially. The residual coefficients can be encoded using entropy coding, such as arithmetic coding. The method for encoding / decoding the residual coefficients is described in detail below with reference to the diagram.

[0473] Figure 41 is a flowchart of the method for encoding the residual coefficients.

[0474] In this embodiment, we assume that the current block contains one or more subconversion units. Furthermore, we assume that the subconversion units are 4x4 in size. However, this embodiment can still be applied even if the size or shape of the subconversion units is different.

[0475] It is possible to determine whether or not a non-zero coefficient exists in the current block (S4101). A non-zero coefficient is a residual coefficient whose absolute value is greater than 0. Information indicating whether or not a non-zero coefficient exists in the current block can be encoded and signaled. For example, this information may be a 1-bit flag CBF (Coded Block Flag).

[0476] If a non-zero coefficient exists in the current block, it can be determined whether or not a non-zero coefficient exists in each sub-conversion unit (S4102). Information indicating whether or not a non-zero coefficient exists in each sub-conversion unit can be encoded and signaled. For example, this information may be a 1-bit flag, a coded sub-block flag (CSBF). The sub-conversion units can be encoded according to the selected scan order.

[0477] If a subtransformation unit has non-zero coefficients, the residual coefficients within the subtransformation unit can be arranged one-dimensionally in order to encode the residual coefficients of the subtransformation unit (S4103). The residual coefficients can be arranged one-dimensionally according to the selected scanning order.

[0478] The scanning sequence may include at least one of diagonal scanning, horizontal scanning, vertical scanning, or inverse scanning of these.

[0479] Figures 42 and 43 show the arrangement order of residual coefficients in different scanning sequences.

[0480] Figures 42(a) to 42(c) show diagonal scan, horizontal scan, and vertical scan, while Figures 43(a) to 43(c) show the reverse scans of these.

[0481] The residual coefficients can be arranged one-dimensionally according to the selected scanning order.

[0482] The scan order can be determined by considering at least one of the following: the size, shape, intra-prediction mode, and whether to apply a transformation core used for the linear transformation or a quadratic transformation. For example, if the current block is a non-square with a width greater than its height, the residual coefficients can be encoded using inverse horizontal scanning. Conversely, if the current block is a non-square with a height greater than its width, the residual coefficients can be encoded using inverse vertical scanning.

[0483] Alternatively, the rate distortion optimization (RDO) for each of multiple scan sequences can be calculated, and the scan sequence with the lowest RDO can be determined as the current scan sequence for the block. In this case, information indicating the current scan sequence for the block can be encoded and signaled.

[0484] The scan order candidates that can be used when encoding conversion skip coefficients may differ from those that can be used when encoding conversion coefficients. For example, the reverse direction of the scan order candidates that can be used when encoding conversion coefficients can be set as the scan order candidates that can be used when encoding conversion skip coefficients.

[0485] For example, if the conversion coefficients are encoded using either inverse diagonal scanning, inverse horizontal scanning, or inverse vertical scanning, the conversion skip coefficients can be encoded using either diagonal scanning, horizontal scanning, or vertical scanning.

[0486] Subsequently, the position of the last non-zero coefficient in the scan order within the transformation block can be encoded (S4103). The x-axis and y-axis positions of the last non-zero coefficient can then be encoded, respectively.

[0487] Figure 44 shows an example of encoding the position of the last non-zero coefficient.

[0488] As shown in Figure 44, in the case of diagonal scanning, the residual coefficient located in the upper right corner of the transformation block can be set as the last non-zero coefficient. The x-axis coordinate LastX of the residual coefficient is 3, and the y-axis coordinate LastY is 0. LastX can be encoded by separating it into a prefix part last_sig_coeff_x_prefix and a suffix part last_sig_coeff_x_suffix. LastY can also be encoded by separating it into a prefix part last_sig_coeff_y_prefix and a suffix part last_sig_coeff_y_suffix.

[0489] Once the position of the last non-zero coefficient is determined, information indicating whether or not a residual coefficient is non-zero can be encoded for each residual coefficient that precedes the last non-zero coefficient in the scan order (S4104). This information may be a 1-bit flag, sig_coeff_flag. If the residual coefficient is 0, the value of the non-zero coefficient flag sig_coeff_flag can be set to 0, and if the residual coefficient is 1, the value of the non-zero coefficient flag sig_coeff_flag can be set to 1. For the last non-zero coefficient, encoding of the non-zero coefficient flag sig_coeff_flag can be omitted. If encoding of the non-zero coefficient flag sig_coeff_flag is omitted, the residual coefficient is considered not to be 0.

[0490] For transformation skip coefficients, information regarding the position of the last non-zero coefficient can be omitted. In this case, the non-zero coefficient flag sig_coeff_flag can be encoded for all transformation skip coefficients within the block.

[0491] Alternatively, information indicating whether or not the information about the position of the last non-zero coefficient is encoded can be encoded and signaled. This information may be a 1-bit flag. Based on the value of the flag, the decoder can decide whether or not to decode the information about the position of the last non-zero coefficient. If the position of the last non-zero coefficient is not decoded, the non-zero coefficient flag can be decoded for all transformation skip coefficients. On the other hand, if the position of the last non-zero coefficient is decoded, information indicating whether or not the transformation skip coefficient is a non-zero coefficient can be encoded for each transformation skip coefficient that precedes the last non-zero coefficient in the scan order.

[0492] The DC component of the conversion coefficient or conversion skip coefficient may not be set to zero. Here, the DC component may indicate the last sample in the scan order or the sample located in the upper left of the block. For the DC component, the encoding of the non-zero coefficient flag sig_coeff_flag may be omitted for the residual coefficient. If the encoding of the non-zero coefficient flag sig_coeff_flag is omitted, the residual coefficient is considered to be non-zero.

[0493] If the residual coefficient is not zero, information indicating the absolute value of the residual coefficient and information indicating the sign of the residual coefficient can be encoded (S4105). The encoding process for information indicating the absolute value of the residual coefficient will be explained in more detail with reference to Figure 45.

[0494] For all residual coefficients after the last non-zero coefficient and for each of the subtransform units contained in the current block, the encoding of the residual coefficients can be performed sequentially (S4106, S4107).

[0495] Figure 45 is a flowchart of the process for encoding the absolute value of the residual coefficient.

[0496] If the absolute value of the residual coefficient is greater than 0, information indicating whether the residual coefficient is even or odd can be encoded (S4501). Here, the residual coefficient can be defined as the value obtained by subtracting 1 from the absolute value of the residual coefficient. The information may be a 1-bit parity flag. For example, a value of 0 for the syntax element par_level_flag indicates that the residual coefficient or residual coefficient is even, and a value of 1 for the syntax element par_level_flag indicates that the residual coefficient or residual coefficient is odd. The value of the syntax element par_level_flag can be determined based on the following equation 30.

number

[0497] In Equation 30, Tcoeff represents the residual coefficient, and abs() represents the absolute value function.

[0498] The adjusted residual coefficient can be derived by dividing the residual coefficient or residual coefficient by 2, or by applying a bit shift operation to the residual coefficient or residual coefficient (S4502). Specifically, the quotient obtained by dividing the residual coefficient by 2 can be set as the adjusted residual coefficient, or the value obtained by shifting the residual coefficient one unit to the right can be set as the adjusted residual coefficient.

[0499] For example, the adjusted residual coefficient can be derived based on the following equation 31.

number

[0500] In Equation 31, ReRemLevel represents the adjusted residual coefficient, and RemLevel represents the residual coefficient.

[0501] Subsequently, information indicating the size of the adjusted residual coefficient can be encoded. This information may include whether the adjusted residual coefficient is greater than or equal to N, where N can be an integer such as 1, 2, 3, or 4.

[0502] For example, information indicating whether the value of the adjustment residual coefficient is greater than 1 can be encoded (S4503). This information may be a 1-bit flag rem_abs_gt1_flag.

[0503] If the residual coefficient is 0 or 1, the value of rem_abs_gt1_flag can be set to 0. In other words, if the residual coefficient is 1 or 2, the value of rem_abs_gt1_flag can be set to 0. In this case, if the residual coefficient is 0, par_level_flag can be set to 0, and if the residual coefficient is 1, par_level_flag can be set to 1.

[0504] If the adjustment residual coefficient is 2 or greater (S4504), the value of rem_abs_gt1_flag can be set to 1, and information indicating whether the value of the adjustment residual coefficient is greater than 2 can be encoded (S4505). This information may be a 1-bit flag rem_abs_gt2_flag.

[0505] If the residual coefficient is 1 or 2, the value of rem_abs_gt2_flag can be set to 0. In other words, if the residual coefficient is 3 or 4, the value of rem_abs_gt2_flag can be set to 0. In this case, if the residual coefficient is 2, par_level_flag can be set to 0, and if the residual coefficient is 3, par_level_flag can be set to 1.

[0506] If the adjusted residual coefficient exceeds 2, the residual value information obtained by subtracting 2 from the adjusted residual coefficient can be encoded (S4506). In other words, the residual value information can be encoded by subtracting 5 from the absolute value of the residual coefficient and then dividing the resulting value by 2.

[0507] Although not shown in the diagram, the residual coefficients can be encoded using additional flags, such as a flag indicating whether the adjusted residual coefficient is greater than 3 (e.g., rem_abs_gt3_flag) or a flag indicating whether the adjusted residual coefficient is greater than 4 (rem_abs_gt4_flag). In this case, the residual value can be set to the adjusted residual coefficient minus the maximum value. The maximum value is the largest N value among the rem_abs_gtN_flag.

[0508] Instead of using a flag to compare the value of the adjusted residual coefficient to a specified value, you can also use a flag to compare the absolute value of the adjustment coefficient or residual coefficient to a specified value. For example, instead of rem_abs_gt1_flag, you can use gr2_flag, which indicates whether the absolute value of the residual coefficient is greater than 2, and instead of rem_abs_gt2_flag, you can use gr4_flag, which indicates whether the absolute value of the residual coefficient is greater than 4.

[0509] Table 14 provides a simplified diagram of the process for encoding residual coefficients using syntactic elements. [Table 14]

[0510] As in the example above, the residual coefficients can be coded using par_level_flag and at least one rem_abs_gtN_flag (where N is an integer such as 1, 2, 3, 4, etc.).

[0511] The `rem_abs_gtN_flag` indicates whether the residual coefficient exceeds 2N. For residual coefficients of 2N-1 or 2N, set `rem_abs_gt(N-1)_flag` to true and `rem_abs_gtN_flag` to false. Additionally, for residual coefficients of 2N-1, the `par_level_flag` value can be set to 0, and for residual coefficients of 2N, the `par_level_flag` value can be set to 1. In other words, residual coefficients of 2N or less can be encoded using `rem_abs_gtN_flag` and `par_level_flag`.

[0512] For residual coefficients where the residual coefficient is greater than or equal to 2MAX+1, the residual value information obtained by dividing the difference from 2MAX by 2 can be encoded. Here, MAX represents the maximum value of N. For example, when using rem_abs_gt1_flag and rem_abs_gt2_flag, MAX can be 2.

[0513] The decoder can also decode the residual coefficients in the order shown in Figure 45. Specifically, the decoder can determine the position of the last non-zero coefficient and decode sig_coeff_flag for each residual coefficient whose scan order precedes the last non-zero coefficient.

[0514] If sig_coeff_flag is true, the par_level_flag of the residual coefficient can be decoded. Furthermore, the rem_abs_gt1_flag of the residual coefficient can be decoded. In this case, based on the value of rem_abs_gt(N-1)_flag, rem_abs_gtN_flag can be further decoded. For example, if the value of rem_abs_gt(N-1)_flag is 1, rem_abs_gtN_flag can be decoded. For example, if the value of the residual coefficient rem_abs_gt1_flag is 1, the rem_abs_gt2_flag of the residual coefficient can be further analyzed. If the value of rem_abs_gt(MAX)_flag is 1, the residual value information can be decoded.

[0515] If the value of the residual coefficient rem_abs_gtN_flag is 0, the value of the residual coefficient can be determined to be 2N-1 or 2N based on the value of par_level_flag. Specifically, if par_level_flag is 0, the residual coefficient can be set to 2N-1, and if par_level_flag is 1, the residual coefficient can be set to 2N.

[0516] For example, if rem_abs_gt1_flag is 0, the absolute value of the residual coefficient can be set to 1 or 2 based on the par_level_flag value. Specifically, if the par_level_flag value is 0, the absolute value of the residual coefficient will be 1, and if the par_level_flag value is 1, the absolute value of the residual coefficient will be 2.

[0517] For example, if rem_abs_gt2_flag is 0, the absolute value of the residual coefficient can be set to 3 or 4 based on the par_level_flag value. Specifically, if the par_level_flag value is 0, the absolute value of the residual coefficient will be 3, and if the par_level_flag value is 1, the absolute value of the residual coefficient will be 4.

[0518] When decoding residual value information, the residual coefficient can be set to 2(MAX+R)-1 or 2(MAX+R) based on the value of par_level_flag, where R represents the value indicated in the residual value information. For example, if par_level_flag is 0, the residual coefficient can be set to 2(MAX+R)-1, and if par_level_flag is 1, the residual coefficient can be set to 2(MAX+R). For example, if MAX is 2, the residual coefficient can be derived based on equation 32 below.

number

[0519] Based on the example shown in Figure 45, when encoding residual coefficients, it is necessary to encode the parity flags for all non-zero coefficients. For example, even when the residual coefficient is 1, it is necessary to encode par_level_flag and rem_abs_gt1_flag. The above encoding method causes the problem of increasing the number of bits required to encode residual coefficients with an absolute value of 1. To avoid this problem, information indicating whether the residual coefficient is greater than 1 can be encoded first, and then the parity flag can be encoded if the residual coefficient is greater than 1.

[0520] Figure 46 is a flowchart of the process for encoding the absolute value of the residual coefficient.

[0521] For non-zero residual coefficients, information gr1_flag indicating whether the absolute value of the residual coefficient is greater than 1 can be encoded (S4601). If the residual coefficient is 1, gr1_flag can be set to 0, and if the residual coefficient is greater than 1, gr1_flag can be set to 1.

[0522] If the absolute value of the residual coefficient exceeds 1, a parity flag indicating whether the residual coefficient or residual coefficient is even or odd can be encoded (S4602, S4603). Here, the residual coefficient can be set to the value obtained by subtracting 2 from the residual coefficient. For example, par_level_flag can be derived based on the following equation 33.

number

[0523] The residual coefficient or residual coefficient can be divided by 2, or the residual coefficient or residual coefficient can be shifted to the right by 1 bit to derive the adjusted residual coefficient, and information indicating whether the adjusted residual coefficient is greater than 1 can be encoded (S4604). For example, for a residual coefficient where gr1_flag is 1, rem_abs_gt1_flag, which indicates whether the adjusted residual coefficient is greater than 1, can be encoded.

[0524] If the residual coefficient is 0 or 1, the value of rem_abs_gt1_flag can be set to 0. In other words, if the residual coefficient is 2 or 3, the value of rem_abs_gt1_flag can be set to 0. In this case, if the adjusted residual coefficient is 0, par_level_flag can be set to 0, and if the residual coefficient is 1, par_level_flag can be set to 1.

[0525] If the adjusted residual coefficient is 2 or greater (S4605), the value of rem_abs_gt1_flag can be set to 1, and information indicating whether or not the adjusted residual coefficient is greater than 2 can be encoded (S4606). For example, for residual coefficients where rem_abs_gt1_flag is 1, rem_abs_gt2_flag, which indicates whether or not the adjusted residual coefficient is greater than 2, can be encoded.

[0526] If the residual coefficient is 2 or 3, the value of rem_abs_gt2_flag can be set to 1. In other words, if the residual coefficient is 4 or 5, the value of rem_abs_gt2_flag can be set to 0. In this case, if the residual coefficient is 2, par_level_flag can be set to 0, and if the residual coefficient is 3, par_level_flag can be set to 1.

[0527] If the adjusted residual coefficient exceeds 2 (S4607), the residual value information obtained by subtracting 2 from the adjusted residual coefficient can be encoded (S4608). In other words, the residual value information can be encoded by subtracting 6 from the absolute value of the residual coefficient and then dividing the resulting value by 2.

[0528] The residual coefficients can be further encoded using flags such as a flag indicating whether the adjusted residual coefficient is greater than 3 (e.g., rem_abs_gt3_flag) or a flag indicating whether the adjusted residual coefficient is greater than 4 (rem_abs_gt4_flag). In this case, the residual value can be set as the adjusted residual coefficient minus the maximum value. The maximum value is the largest N value among the rem_abs_gtN_flag.

[0529] Instead of using a flag that compares the value of the adjusted residual coefficient to a specified value, you can also use a flag that compares the absolute value of the adjustment coefficient or residual coefficient to a specified value. For example, instead of rem_abs_gt1_flag, you can use gr3_flag, which indicates whether the absolute value of the residual coefficient is greater than 3, and instead of rem_abs_gt2_flag, you can use gr5_flag, which indicates whether the absolute value of the residual coefficient is greater than 5.

[0530] Table 15 provides a simplified diagram of the process for encoding residual coefficients using syntactic elements. [Table 15]

[0531] The decoder can also decode the residual coefficients in the order shown in Figure 46. Specifically, the decoder can determine the position of the last non-zero coefficient and decode sig_coeff_flag for each residual coefficient whose scan order precedes the last non-zero coefficient.

[0532] If sig_coeff_flag is true, the gr1_flag of the residual coefficient can be decoded. If gr1_flag is 0, the absolute value of the residual coefficient is determined to be 1, and if gr1_flag is 1, the par_level_flag of the residual coefficient can be decoded. Then, the rem_abs_gt1_flag of the residual coefficient can be decoded. In this case, based on the value of rem_abs_gt(N-1)_flag, rem_abs_gtN_flag can be further decoded. For example, if the value of rem_abs_gt(N-1)_flag is 1, rem_abs_gtN_flag can be decoded. For example, if the value of the residual coefficient rem_abs_gt1_flag is 1, the rem_abs_gt2_flag of the residual coefficient can be further analyzed. If the value of rem_abs_gt(MAX)_flag is 1, the residual value information can be decoded.

[0533] If the value of the residual coefficient rem_abs_gtN_flag is 0, the value of the residual coefficient can be determined to be 2N-1 or 2N based on the value of par_level_flag. Specifically, if par_level_flag is 0, the residual coefficient can be set to 2N, and if par_level_flag is 1, the residual coefficient can be set to 2N+1.

[0534] For example, if rem_abs_gt1_flag is 0, the absolute value of the residual coefficient can be set to 2 or 3 based on the par_level_flag value. Specifically, if the par_level_flag value is 0, the absolute value of the residual coefficient will be 2, and if the par_level_flag value is 1, the absolute value of the residual coefficient will be 3.

[0535] For example, if rem_abs_gt2_flag is 0, the absolute value of the residual coefficient can be set to 4 or 5 based on the par_level_flag value. Specifically, if the par_level_flag value is 0, the absolute value of the residual coefficient will be 4, and if the par_level_flag value is 1, the absolute value of the residual coefficient will be 5.

[0536] When decoding residual value information, the residual coefficient can be set to 2(MAX+R) or 2(MAX+R)+1 based on the value of par_level_flag, where R represents the value of the residual value information. For example, if par_level_flag is 0, the residual coefficient can be set to 2(MAX+R), and if par_level_flag is 1, the residual coefficient can be set to 2(MAX+R)+1.

[0537] As another example, the parity flag can be encoded only if the value of the residual coefficient is greater than 2. For instance, after encoding information indicating whether the value of the residual coefficient is greater than 1 and information indicating whether the value of the residual coefficient is greater than 2, if it is determined that the value of the residual coefficient is greater than 2, the parity flag can be encoded for the residual coefficient.

[0538] Figure 47 is a flowchart of the process for encoding the absolute value of the residual coefficient.

[0539] For non-zero residual coefficients, information gr1_flag indicating whether the absolute value of the residual coefficient is greater than 1 can be encoded (S4701). If the residual coefficient is 1, gr1_flag can be set to 0, and if the residual coefficient is greater than 1, gr1_flag can be set to 1.

[0540] For residual coefficients whose absolute value exceeds 1, information gr2_flag indicating whether the absolute value of the residual coefficient exceeds 2 can be encoded (S4702, S4703). If the residual coefficient is 2, gr2_flag can be set to 0, and if the residual coefficient exceeds 2, gr2_flag can be set to 1.

[0541] If the absolute value exceeds 2, a parity flag indicating whether the residual coefficient or residual coefficient is even or odd can be encoded (S4704, S4705). Here, the residual coefficient can be set to the value obtained by subtracting 3 from the residual coefficient. For example, par_level_flag can be derived based on the following equation 34.

number

[0542] The residual coefficient or residual coefficient can be divided by 2, or the residual coefficient or residual coefficient can be shifted to the right by 1 bit to derive the adjusted residual coefficient, and information indicating whether the adjusted residual coefficient is greater than 1 can be encoded. For example, for residual coefficients where gr1_flag is 1, rem_abs_gt1_flag, which indicates whether the adjusted residual coefficient is greater than 1, can be encoded (S4706).

[0543] If the residual coefficient is 0 or 1, the value of rem_abs_gt1_flag can be set to 0. In other words, if the residual coefficient is 3 or 4, the value of rem_abs_gt1_flag can be set to 0. In this case, if the adjusted residual coefficient is 0, par_level_flag can be set to 0, and if the residual coefficient is 1, par_level_flag can be set to 1.

[0544] If the adjusted residual coefficient is 2 or greater (S4707), the value of rem_abs_gt1_flag can be set to 1, and information indicating whether or not the adjusted residual coefficient is greater than 2 can be encoded (S4708). For example, for residual coefficients where rem_abs_gt1_flag is 1, rem_abs_gt2_flag, which indicates whether or not the adjusted residual coefficient is greater than 2, can be encoded.

[0545] If the residual coefficient is 2 or 3, the value of rem_abs_gt2_flag can be set to 1. In other words, if the residual coefficient is 5 or 6, the value of rem_abs_gt2_flag can be set to 0. In this case, if the residual coefficient is 2, par_level_flag can be set to 0, and if the residual coefficient is 3, par_level_flag can be set to 1.

[0546] If the adjusted residual coefficient exceeds 2, the residual value obtained by subtracting 2 from the adjusted residual coefficient can be encoded (S4709, S4710). In other words, the residual value can be encoded by subtracting 7 from the absolute value of the residual coefficient and then dividing the resulting value by 2.

[0547] The residual coefficients can be further encoded using flags such as a flag indicating whether the adjusted residual coefficient is greater than 3 (e.g., rem_abs_gt3_flag) or a flag indicating whether the adjusted residual coefficient is greater than 4 (rem_abs_gt4_flag). In this case, the residual value can be set as the adjusted residual coefficient minus the maximum value. The maximum value is the largest N value among the rem_abs_gtN_flag.

[0548] Instead of using a flag that compares the value of the adjusted residual coefficient to a specified value, you can also use a flag that compares the absolute value of the adjustment coefficient or residual coefficient to a specified value. For example, instead of rem_abs_gt1_flag, you can use gr4_flag, which indicates whether the absolute value of the residual coefficient is greater than 4, and instead of rem_abs_gt2_flag, you can use gr6_flag, which indicates whether the absolute value of the residual coefficient is greater than 6.

[0549] The decoder can also decode the residual coefficients in the same order as shown in Figure 47. Specifically, the decoder can determine the position of the last non-zero coefficient and decode sig_coeff_flag for each residual coefficient whose scan order precedes the last non-zero coefficient.

[0550] If sig_coeff_flag is true, the gr1_flag of the residual coefficient can be decoded. If gr1_flag is 0, the absolute value of the residual coefficient is determined to be 1, and if gr1_flag is 1, the gr2_flag can be decoded. If gr2_flag is 0, the absolute value of the residual coefficient is determined to be 2, and if gr2_flag is 1, the par_level_flag of the residual coefficient can be decoded. Subsequently, the rem_abs_gt1_flag of the residual coefficient can be decoded. In this case, based on the value of rem_abs_gt(N-1)_flag, rem_abs_gtN_flag can be further decoded. For example, if the value of rem_abs_gt(N-1)_flag is 1, rem_abs_gtN_flag can be decoded. For example, if the value of the residual coefficient rem_abs_gt1_flag is 1, the residual coefficient rem_abs_gt2_flag can be further analyzed. If the value of rem_abs_gt(MAX)_flag is 1, the residual value information can be decoded.

[0551] If the value of the residual coefficient rem_abs_gtN_flag is 0, the value of the residual coefficient can be determined to be 2N+1 or 2(N+1) based on the value of par_level_flag. Specifically, if par_level_flag is 0, the residual coefficient can be set to 2N+1, and if par_level_flag is 1, the residual coefficient can be set to 2(N+1).

[0552] For example, if rem_abs_gt1_flag is 0, the absolute value of the residual coefficient can be set to 3 or 4 based on the par_level_flag value. Specifically, if the par_level_flag value is 0, the absolute value of the residual coefficient will be 3, and if the par_level_flag value is 1, the absolute value of the residual coefficient will be 4.

[0553] For example, if rem_abs_gt2_flag is 0, the absolute value of the residual coefficient can be set to 5 or 6 based on the par_level_flag value. Specifically, if the par_level_flag value is 0, the absolute value of the residual coefficient will be 5, and if the par_level_flag value is 1, the absolute value of the residual coefficient will be 6.

[0554] When decoding residual value information, the residual coefficient can be set to 2(MAX+R) or 2(MAX+R)+1 based on the value of par_level_flag, where R represents the value of the residual value information. For example, if par_level_flag is 0, the residual coefficient can be set to 2(MAX+R), and if par_level_flag is 1, the residual coefficient can be set to 2(MAX+R)+1.

[0555] Based on at least one of the following: the size, shape, whether to skip conversions, the number of conversion cores, the number of non-zero coefficients, or the position of the last non-zero coefficient, it is possible to determine at least one of the number or types of comparison flags used to compare the adjusted residual coefficients with a specified value. For example, when encoding conversion coefficients, only rem_abs_gt1_flag can be used. On the other hand, when encoding conversion skip coefficients, both rem_abs_gt1_flag and rem_abs_gt2_flag can be used.

[0556] Alternatively, within a 4x4 sub-conversion unit, the number of rem_abs_gt1_flags can be set to a maximum of 8, and the number of rem_abs_gt2_flags can be set to a maximum of 1. Or, if the number of non-zero coefficient flags is (16-N), the number of rem_abs_gt1_flags can be set to a maximum of 8+(N / 2), and the number of rem_abs_gt2_flags can be set to a maximum of 1+(N-(N / 2)).

[0557] When a restored block of the current block is obtained, the loss of information generated during the quantization and encoding processes can be reduced through in-loop filtering. The in-loop filter may include at least one of the following: a deblocking filter, a sample adaptive offset filter (SAO), or an adaptive loop filter (ALF). Hereinafter, the restored block before the application of the in-loop filter will be referred to as the first restored block, and the restored block after the application of the in-loop filter will be referred to as the second restored block.

[0558] The second restored block can be obtained by applying at least one of the following to the first restored block: a deblocking filter, SAO, or ALF. In this case, the SAO or ALF can be applied after the deblocking filter has been applied.

[0559] Deblocking filters are used to mitigate image degradation (blocking artifacts) that occur at block boundaries when quantization is performed in blocks. To apply a deblocking filter, the blocking strength (BS) between the first reconstructed block and adjacent reconstructed blocks can be determined.

[0560] Figure 48 is a flowchart of the process for determining block strength.

[0561] In the example shown in Figure 48, P represents the first restored block, and Q represents an adjacent restored block. Here, an adjacent restored block may be a block adjacent to the left or above the current block.

[0562] The example shown in Figure 48 demonstrates how block intensity is determined by considering the predictive coding modes of P and Q, whether non-zero conversion coefficients are included, whether interpretation is performed using the same reference image, and whether the difference in motion vectors is greater than or equal to a threshold.

[0563] Based on the block strength, it is possible to decide whether or not to apply a deblocking filter. For example, if the block strength is 0, filtering may not be performed.

[0564] SAO is used to mitigate ringing artifacts that occur when quantization is performed in the frequency domain. SAO can be performed by adding or subtracting an offset determined by considering the pattern of the first reconstructed image. Methods for determining the offset include edge offset (EO) or band offset. EO shows a method of determining the offset of the current sample based on the pattern of surrounding pixels. BO shows a method of applying a common offset to groups of pixels with similar luminance values ​​within a region. Specifically, pixel luminance can be divided into 32 equal intervals, and pixels with similar luminance values ​​can be set as one group. For example, out of 32 bands, four adjacent bands can be set as one group, and the same offset value can be applied to samples belonging to those four bands.

[0565] ALF is a method for generating a second reconstructed image by applying a predefined size / shape filter to a first reconstructed image or a reconstructed image to which a deblocking filter has been applied. Equation 35 below shows an example of ALF application.

number

[0566] One of the predefined filter candidates can be selected at the image, coding tree unit, coding block, prediction block, or transformation block level. Each filter candidate can differ in either size or shape.

[0567] Figure 49 shows a diagram of predefined filter candidates.

[0568] In the example shown in Figure 49, at least one of the following rhombuses can be selected: 5x5, 7x7, and 9x9 in size.

[0569] For the saturation component, only a 5x5 rhombus shape can be used.

[0570] Applying embodiments described with a focus on a decoding or encoding process to an encoding or decoding process is within the scope of the present invention. Modifications of embodiments described in a predetermined order, but in a different order than described, are also within the scope of the present invention.

[0571] The embodiments described above have been explained based on a series of steps or flowcharts, but this does not limit the chronological order of the present invention, and the steps can be performed simultaneously or in other orders as necessary. Also, in the embodiments described above, each component constituting the block diagram (e.g., unit, module, etc.) can be implemented by hardware devices or software, and multiple components can be combined to be implemented as a single hardware device or software. The embodiments described above can be implemented in the form of program instructions that can be executed by various computer components, and the program instructions can be stored in a computer-readable storage medium. The computer-readable storage medium can include program instructions, data files, data structures, etc., individually or in combination. Examples of computer-readable storage media can include magnetic media such as hard disks, floppy disks and magnetic tapes, optical storage media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. The hardware devices can be configured to operate as one or more software modules for performing the processing according to the present invention, and vice versa. [Industrial applicability]

[0572] This invention can be applied to electronic devices that encode / decode video.

Claims

1. A method for receiving a bitstream, Receiving a bitstream and The steps include decoding the bitstream by performing the following steps, the steps being: The steps include analyzing the bitstream for a non-zero flag indicating whether the residual coefficient is non-zero, If the non-zero flag is omitted for the position of the last non-zero coefficient, the residual coefficient is considered non-zero. If the non-zero flag indicates that the residual coefficient is non-zero, the step is to analyze absolute value information from the bitstream, wherein the absolute value information is used to determine the absolute value of the residual coefficient. The step of determining the absolute value of the residual coefficient based on the absolute value information is included, The absolute value information includes a residual coefficient comparison flag indicating whether the residual coefficient exceeds a first value. A method comprising further analyzing a parity flag from the bitstream only if the residual coefficient exceeds the first value, wherein the parity flag indicates whether the value of the residual coefficient is even or odd.

2. If the residual coefficient exceeds the first value, the first adjusted residual coefficient comparison flag is further analyzed, and the first adjusted residual coefficient comparison flag indicates whether the adjusted residual coefficient derived by shifting the residual coefficient one bit to the right exceeds the second value. The method according to claim 1.

3. If the adjusted residual coefficient is less than or equal to the second value, the residual coefficient is determined to be 2N or 2N+1 (where N is the second value) according to the value of the parity flag. The method according to claim 2.

4. If the adjusted residual coefficient exceeds the second value, the residual value information is further analyzed. The residual value information is the value obtained by subtracting the second value from the adjusted residual coefficient. The method according to claim 2.

5. A method for transmitting a bitstream, The following steps are performed to generate a bitstream, wherein the steps are: A step of encoding a non-zero flag that indicates whether the residual coefficient is non-zero, If encoding of the non-zero flag is omitted for the position of the last non-zero coefficient, the residual coefficient is treated as non-zero. The method includes, if the residual coefficient is non-zero, a step of encoding absolute value information, wherein the absolute value information is used to determine the absolute value of the residual coefficient. This includes transmitting the generated bitstream, The absolute value information includes a residual coefficient comparison flag indicating whether the residual coefficient exceeds a first value. A method comprising further encoding a parity flag of the residual coefficient only if the residual coefficient exceeds the first value, wherein the parity flag indicates whether the value of the residual coefficient is even or odd.

6. If the residual coefficient exceeds the first value, a first adjusted residual coefficient comparison flag is further encoded, and the first adjusted residual coefficient comparison flag indicates whether the adjusted residual coefficient derived by shifting the residual coefficient one bit to the right exceeds the second value. The method according to claim 5.

7. If the adjusted residual coefficient is less than or equal to the second value, the residual coefficient is determined to be 2N or 2N+1 according to the value of the parity flag. The method according to claim 6.

8. If the adjusted residual coefficient exceeds the second value, the residual value information is further encoded. The residual value information is the value obtained by subtracting the second value from the adjusted residual coefficient. The method according to claim 6.

9. A computer-readable storage medium storing computer programs and bitstreams, To generate the bitstream, the computer program causes the processor to perform the following steps, the steps being: A step of encoding a non-zero flag that indicates whether the residual coefficient is non-zero, If encoding of the non-zero flag is omitted for the position of the last non-zero coefficient, the residual coefficient is treated as non-zero. If the residual coefficient is non-zero, the step of encoding absolute value information, wherein the absolute value information is used to determine the absolute value of the residual coefficient, The absolute value information includes a residual coefficient comparison flag indicating whether the residual coefficient exceeds a first value. The computer-readable storage medium further encodes a parity flag of the residual coefficient only if the residual coefficient exceeds the first value, wherein the parity flag indicates whether the value of the residual coefficient is even or odd.

10. If the residual coefficient exceeds the first value, a first adjusted residual coefficient comparison flag is further encoded, and the first adjusted residual coefficient comparison flag indicates whether the adjusted residual coefficient derived by shifting the residual coefficient one bit to the right exceeds the second value. The computer-readable storage medium according to claim 9.

11. If the adjusted residual coefficient is less than or equal to the second value, the residual coefficient is determined to be 2N or 2N+1 according to the value of the parity flag. The computer-readable storage medium according to claim 10.

12. If the adjusted residual coefficient exceeds the second value, the residual value information is further encoded. The residual value information is the value obtained by subtracting the second value from the adjusted residual coefficient. The computer-readable storage medium according to claim 10.

Images