Device and method for encoding and decoding image by using intra prediction

Abstract

An image decoding method according to an embodiment may comprise the steps of: determining whether an intra blending mode is applied to the current block; when the intra blending mode is applied to the current block, deriving a first prediction mode of the current block on the basis of a region decoded prior to the current block; generating a first prediction block of the current block by performing intra prediction on the current block by using the first prediction mode; selecting a second prediction mode of the current block from among a plurality of candidate prediction modes on the basis of a template cost; generating a second prediction block of the current block by performing prediction corresponding to the second prediction mode on the current block by using the second prediction mode; and generating a final prediction block of the current block by combining the first prediction block and the second prediction block.

Description

Device and method for encoding and decoding images using intra-prediction

[0001] The present disclosure relates to the field of image encoding and decoding, and specifically, to an apparatus and method for encoding and decoding an image using intra-prediction.

[0002] In video encoding and decoding, the video is divided into blocks, and each block can be predictively encoded and predictedly decoded through inter prediction or intra prediction.

[0003] Inter-prediction is a technique that compresses images by eliminating temporal redundancy between images. In inter-prediction, blocks of the current image can be predicted using a reference image. The reference block most similar to the current block can be searched within a predetermined search range in the reference image. The current block is predicted based on the reference block, and a residual block can be generated by subtracting the predicted block resulting from the prediction result from the current block.

[0004] In standards such as H.264 AVC (Advanced Video Coding) and HEVC (High Efficiency Video Coding), the motion vectors of previously encoded blocks adjacent to the current block or blocks included in a previously encoded video can be used as the motion vector predictor for the current block to predict the motion vector of the current block. The motion vector difference, which is the difference between the motion vector of the current block and the motion vector predictor, can be signaled to the decoder side through a predetermined method.

[0005] Intra prediction is a technique that compresses images by eliminating spatial redundancy within the image. In intra prediction, depending on the intra prediction mode, a prediction block can be generated based on the surrounding pixels of the current block. Additionally, a residual block can be generated by subtracting the prediction block from the current block.

[0006] Residual blocks generated through inter-prediction or intra-prediction can be passed to a decoder after undergoing transformation and quantization. The decoder can inversely quantize and inversely transform the residual blocks, and reconstruct the current block by combining the prediction block of the current block with the residual blocks. In certain cases, the decoder can filter the reconstructed current block to remove artifacts within it.

[0007] A method for decoding an image according to one embodiment may include a step of determining whether an intra-blending mode is applied to the current block.

[0008] A method for decoding an image according to one embodiment may include the step of deriving a first prediction mode of the current block based on a region decoded prior to the current block when an intra-blending mode is applied to the current block.

[0009] A decoding method for an image according to one embodiment may include the step of generating a first prediction block of a current block by performing intra prediction on a current block using a first prediction mode.

[0010] A decoding method for an image according to one embodiment may include the step of selecting a second prediction mode of the current block among a plurality of candidate prediction modes based on a template cost.

[0011] In one embodiment, a plurality of candidate prediction modes may include at least one of an intra-mode based prediction mode, a block vector based prediction mode, or a motion vector based prediction mode.

[0012] A decoding method for an image according to one embodiment may include the step of generating a second prediction block of a current block by performing a prediction corresponding to the second prediction mode for the current block using a second prediction mode.

[0013] A decoding method for an image according to one embodiment may include the step of generating a final prediction block of the current block by combining a first prediction block and a second prediction block.

[0014] A method for decoding an image according to one embodiment may include a step of restoring the current block using the final predicted block of the current block.

[0015] An image decoding device according to one embodiment may include at least one memory storing at least one instruction; and at least one processor operating according to at least one instruction.

[0016] In one embodiment, at least one processor can determine whether an intra-blending mode is applied to the current block.

[0017] In one embodiment, at least one processor can derive a first prediction mode of the current block based on a region decoded prior to the current block when an intra-blending mode is applied to the current block.

[0018] In one embodiment, at least one processor can generate a first prediction block of the current block by performing intra prediction on the current block using a first prediction mode.

[0019] In one embodiment, at least one processor can select a second prediction mode of the current block from a plurality of candidate prediction modes based on a template cost.

[0020] In one embodiment, a plurality of candidate prediction modes may include at least one of an intra-mode based prediction mode, a block vector based prediction mode, or a motion vector based prediction mode.

[0021] In one embodiment, at least one processor can generate a second prediction block of the current block by performing a prediction corresponding to the second prediction mode for the current block using the second prediction mode.

[0022] In one embodiment, at least one processor can generate a final prediction block of the current block by combining a first prediction block and a second prediction block.

[0023] A method for encoding an image according to one embodiment may include a step of determining whether to apply an intra-blending mode to the current block.

[0024] A method for encoding an image according to one embodiment may include a step of inducing a first prediction mode of the current block based on a region decoded prior to the current block.

[0025] An image encoding method according to one embodiment may include the step of generating a first prediction block of a current block by performing intra prediction on a current block using a first prediction mode.

[0026] An image encoding method according to one embodiment may include the step of selecting a second prediction mode of the current block among a plurality of candidate prediction modes based on a template cost.

[0027] In one embodiment, a plurality of candidate prediction modes may include at least one of an intra-mode based prediction mode, a block vector based prediction mode, or a motion vector based prediction mode.

[0028] An image encoding method according to one embodiment may include the step of generating a second prediction block of a current block by performing a prediction corresponding to the second prediction mode for the current block using a second prediction mode.

[0029] An image encoding method according to one embodiment may include the step of generating a final prediction block of a current block by combining a first prediction block and a second prediction block.

[0030] An image encoding method according to one embodiment may include the step of encoding the current block using the final predicted block of the current block.

[0031] An image encoding device according to one embodiment may include at least one memory for storing at least one instruction; and at least one processor for operating according to at least one instruction.

[0032] In one embodiment, at least one processor can determine whether to apply an intra-blending mode to the current block.

[0033] In one embodiment, at least one processor can derive a first prediction mode of the current block based on a region decoded prior to the current block.

[0034] In one embodiment, at least one processor can generate a first prediction block of the current block by performing intra prediction on the current block using a first prediction mode.

[0035] In one embodiment, at least one processor can select a second prediction mode of the current block from a plurality of candidate prediction modes based on a template cost.

[0036] In one embodiment, a plurality of candidate prediction modes may include at least one of an intra-mode based prediction mode, a block vector based prediction mode, or a motion vector based prediction mode.

[0037] In one embodiment, at least one processor can generate a second prediction block of the current block by performing a prediction corresponding to the second prediction mode for the current block using the second prediction mode.

[0038] In one embodiment, at least one processor can generate a final prediction block of the current block by combining a first prediction block and a second prediction block.

[0039] In one embodiment, at least one processor can encode the current block using the final prediction block of the current block.

[0040] In a computer-readable recording medium that records a bitstream according to one embodiment, the bitstream may include the encoding result of the current block.

[0041] In one embodiment, the encoding result of the current block can be generated by determining whether to apply an intra-blending mode to the current block.

[0042] In one embodiment, the encoding result of the current block can be generated by deriving a first prediction mode of the current block based on a region decoded prior to the current block.

[0043] In one embodiment, the encoding result of the current block can be generated by generating a first prediction block of the current block by performing intra prediction on the current block using a first prediction mode.

[0044] In one embodiment, the encoding result of the current block can be generated by selecting a second prediction mode of the current block from a plurality of candidate prediction modes based on a template cost.

[0045] In one embodiment, a plurality of candidate prediction modes may include at least one of an intra-mode based prediction mode, a block vector based prediction mode, or a motion vector based prediction mode.

[0046] In one embodiment, the encoding result of the current block can be generated by generating a second prediction block of the current block by performing a prediction corresponding to the second prediction mode on the current block using the second prediction mode.

[0047] In one embodiment, the encoding result of the current block can be generated by combining the first prediction block and the second prediction block to generate the final prediction block of the current block.

[0048] In one embodiment, the encoding result of the current block can be generated by encoding the current block using the final predicted block of the current block.

[0049] FIG. 1 is a block diagram of an image decoding device according to one embodiment.

[0050] FIG. 2 is a block diagram of an image encoding device according to one embodiment.

[0051] FIG. 3 illustrates a process of determining at least one encoding unit by dividing the current encoding unit according to one embodiment.

[0052] FIG. 4 illustrates a process of determining at least one encoding unit by dividing a encoding unit that is in the shape of a non-square according to one embodiment.

[0053] FIG. 5 illustrates a process of dividing a encoding unit based on at least one of block shape information and division shape mode information according to one embodiment.

[0054] FIG. 6 illustrates a method for determining a predetermined encoding unit among an odd number of encoding units according to one embodiment.

[0055] FIG. 7 illustrates the order in which a plurality of encoding units are processed when a current encoding unit is divided to determine a plurality of encoding units according to one embodiment.

[0056] FIG. 8 illustrates a process for determining that, according to one embodiment, when the encoding unit cannot be processed in a predetermined order, the current encoding unit is divided into an odd number of encoding units.

[0057] FIG. 9 illustrates a process of determining at least one encoding unit by dividing a first encoding unit according to one embodiment.

[0058] FIG. 10 illustrates that, according to one embodiment, the shape that can be divided is limited when a second encoding unit of a non-square shape determined by dividing a first encoding unit satisfies a predetermined condition.

[0059] FIG. 11 illustrates a process of dividing square-shaped encoding units when, according to one embodiment, the divided shape mode information cannot represent division into four square-shaped encoding units.

[0060] FIG. 12 illustrates that, according to one embodiment, the processing order between a plurality of encoding units may vary depending on the division process of the encoding unit.

[0061] FIG. 13 illustrates a process in which, according to one embodiment, a encoding unit is recursively divided to determine a plurality of encoding units, and the depth of the encoding unit is determined as the shape and size of the encoding unit change.

[0062] FIG. 14 illustrates a depth and part index (hereinafter PID) for distinguishing between coding units that can be determined according to the shape and size of the coding units according to one embodiment.

[0063] FIG. 15 illustrates that a plurality of encoding units are determined according to a plurality of predetermined data units included in a picture according to one embodiment.

[0064] FIG. 16 illustrates the encoding units that can be determined for each picture when the combination of forms in which the encoding units can be divided according to one embodiment is different for each picture.

[0065] FIG. 17 illustrates various forms of encoding units that can be determined based on partitioned form mode information expressed as binary code according to one embodiment.

[0066] FIG. 18 illustrates another form of encoding unit that can be determined based on partitioned form mode information represented by binary code according to one embodiment.

[0067] FIG. 19 is a block diagram of an image encoding and decoding system that performs loop filtering according to one embodiment.

[0068] FIG. 20 is a block diagram illustrating the configuration of an image decoding device (2000) according to one embodiment.

[0069] FIG. 21 is a diagram illustrating intra-prediction modes according to one embodiment.

[0070] FIG. 22 is a diagram illustrating a method for inducing an intra-prediction mode based on the amount of change in surrounding sample values ​​according to one embodiment.

[0071] FIG. 23 is a diagram illustrating an intra-prediction mode-based image decoding method according to one embodiment.

[0072] FIG. 24 is a drawing illustrating a template area of ​​a current block according to one embodiment.

[0073] FIG. 25 is a diagram illustrating an intra-prediction mode-based prediction method for a template area according to one embodiment.

[0074] FIG. 26 is a diagram illustrating a block vector-based prediction method for a template region according to one embodiment.

[0075] FIG. 27 is a diagram illustrating an intra-prediction mode-based image decoding method according to one embodiment.

[0076] FIG. 28 is a diagram illustrating a motion vector-based prediction method for a template area according to one embodiment.

[0077] FIG. 29 is a diagram illustrating a motion vector-based prediction method for a template area according to one embodiment.

[0078] FIG. 30 is a diagram illustrating an intra-prediction mode-based image decoding method according to one embodiment.

[0079] Figure 31 is a flowchart illustrating the prediction mode determination process.

[0080] FIG. 32 is a flowchart illustrating a prediction mode determination process according to one embodiment.

[0081] FIG. 33 is a flowchart illustrating a prediction mode determination process according to one embodiment.

[0082] FIG. 34 is a flowchart of an image decoding method according to one embodiment.

[0083] FIG. 35 is a block diagram showing the configuration of an image encoding device according to one embodiment of the present disclosure.

[0084] FIG. 36 is a flowchart illustrating an image encoding method according to one embodiment of the present disclosure.

[0085] The present disclosure is capable of various modifications and may have various embodiments, and embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the embodiments of the present disclosure, and the present disclosure may include all modifications, equivalents, and substitutions that fall within the spirit and technical scope of the various embodiments.

[0086] In describing the embodiments, detailed descriptions of related prior art may be omitted if it is determined that such detailed descriptions would unnecessarily obscure the essence of the present disclosure. Additionally, numbers used in the description of the embodiments (e.g., first, second, etc.) may serve as identification symbols to distinguish one component from another.

[0087] In the present disclosure, the expression “at least one of a, b, or c” may refer to “a”, “b”, “c”, “a and b”, “a and c”, “b and c”, “a, b, and c all”, or variations thereof.

[0088] In the present disclosure, when one component is described as being "connected" or "connected" to another component, the one component may be directly connected to or directly connected to another component, but unless specifically stated otherwise, it may be connected or connected through another component in between.

[0089] In this disclosure, components expressed as ‘~part (unit)’, ‘module’, etc. may consist of two or more components combined into a single component, or a single component may be divided into two or more more subdivided components. Additionally, each component described below may additionally perform some or all of the functions of other components in addition to the primary function it is responsible for, and some of the primary functions of each component may be exclusively performed by other components.

[0090] In the present disclosure, 'image' may refer to a picture, a still image, a frame, a video composed of a plurality of consecutive still images, or a video.

[0091] In the present disclosure, 'sample' may refer to data assigned to a sampling location of an image that is subject to processing. For example, a pixel within a frame in a spatial domain may correspond to a sample. A unit comprising a plurality of samples may be defined as a block.

[0092] Hereinafter, with reference to FIGS. 1 to 19, an image encoding method and apparatus based on a tree structure encoding unit and a conversion unit according to one embodiment, an image decoding method and apparatus are disclosed.

[0093] FIG. 1 illustrates a block diagram of an image decoding device (100) according to one embodiment.

[0094] The video decoding device (100) may include a bitstream acquisition unit (110) and a decoding unit (120). The bitstream acquisition unit (110) and the decoding unit (120) may include at least one processor. Additionally, the bitstream acquisition unit (110) and the decoding unit (120) may include a memory that stores instructions to be executed by at least one processor.

[0095] The bitstream acquisition unit (110) can receive a bitstream. The bitstream contains information in which an image is encoded by an image encoding device (200) described later. Additionally, the bitstream can be transmitted from the image encoding device (200). The image encoding device (200) and the image decoding device (100) can be connected via wired or wireless connection, and the bitstream acquisition unit (110) can receive the bitstream via wired or wireless connection. The bitstream acquisition unit (110) can receive the bitstream from a storage medium such as an optical medium or a hard disk. The decoding unit (120) can restore the image based on information obtained from the received bitstream. The decoding unit (120) can obtain syntax elements for restoring the image from the bitstream. The decoding unit (120) can restore the image based on the syntax elements.

[0096] To explain in detail the operation of the video decoding device (100), the bitstream acquisition unit (110) can receive a bitstream.

[0097] The image decoder (100) can perform an operation of obtaining an empty string corresponding to a partitioning mode of the encoding unit from a bitstream. The image decoder (100) can also perform an operation of determining a partitioning rule for the encoding unit. Additionally, the image decoder (100) can perform an operation of partitioning the encoding unit into a plurality of encoding units based on at least one of the empty string corresponding to the partitioning mode and the partitioning rule. To determine the partitioning rule, the image decoder (100) can determine an allowable first range of the size of the encoding unit according to the ratio of the width and height of the encoding unit. To determine the partitioning rule, the image decoder (100) can determine an allowable second range of the size of the encoding unit according to the partitioning mode of the encoding unit.

[0098] In the following, the division of a encoding unit according to one embodiment of the present disclosure will be described in detail.

[0099] First, a picture may be divided into one or more slices or one or more tiles. A slice or a tile may be a sequence of one or more Coding Tree Units (CTUs). Depending on the embodiment, a slice may include one or more tiles, and a slice may include one or more Coding Tree Units. A slice containing one or more tiles may be determined within the picture.

[0100] In contrast to the Max Coding Unit (CTU), there is the Max Coding Tree Block (CTB). A Max Coding Tree Block (CTB) refers to an NxN block containing NxN samples (where N is an integer). Each color component can be divided into one or more Max Coding Tree Blocks.

[0101] When a picture has three sample arrays (sample arrays for Y, Cr, and Cb components), the maximum encoding unit (CTU) is a unit comprising a maximum encoding block for luminance samples and two corresponding maximum encoding blocks for chroma samples, and syntax structures used to encode the luminance samples and chroma samples. When a picture is a monochrome picture, the maximum encoding unit is a unit comprising a maximum encoding block for monochrome samples and syntax structures used to encode the monochrome samples. When a picture is encoded in color planes separated by color components, the maximum encoding unit is a unit comprising the picture and syntax structures used to encode the samples of the picture.

[0102] A single maximum coding block (CTB) can be divided into MxN coding blocks containing MxN samples (M and N are integers).

[0103] When a picture has sample arrays for Y, Cr, and Cb components, a coding unit (CU) is a unit comprising a coding block for luminance samples and two coding blocks for corresponding chroma samples, and syntax structures used to encode the luminance samples and chroma samples. When a picture is a monochrome picture, a coding unit is a unit comprising a coding block for monochrome samples and syntax structures used to encode the monochrome samples. When a picture is a picture encoded in color planes separated by color components, a coding unit is a unit comprising the picture and syntax structures used to encode the samples of the picture.

[0104] As explained above, the maximum encoding block and the maximum encoding unit are distinct concepts, and the encoding block and the encoding unit are distinct concepts. That is, the (maximum) encoding unit refers to a data structure that includes the (maximum) encoding block containing the corresponding sample and the corresponding syntax structure. However, since a person skilled in the art can understand that the (maximum) encoding unit or the (maximum) encoding block refers to a block of a predetermined size containing a predetermined number of samples, the maximum encoding block and the maximum encoding unit, or the encoding block and the encoding unit, are referred to without distinction in the following specification unless there are special circumstances.

[0105] The image can be divided into Coding Tree Units (CTUs). The size of the CTU can be determined based on information obtained from the bitstream. The shape of the CTU can be a square of equal size, but it is not limited to this.

[0106] For example, information about the maximum size of a luma-encoded block can be obtained from a bitstream. For example, the maximum size of a luma-encoded block indicated by the information about the maximum size of a luma-encoded block may be one of 4x4, 8x8, 16x16, 32x32, 64x64, 128x128, or 256x256.

[0107] For example, information regarding the maximum size of a two-divisionable luminance coding block and the difference in luminance block size can be obtained from a bitstream. The information regarding the difference in luminance block size may represent the size difference between the maximum luminance coding unit and the maximum two-divisionable luminance coding block. Therefore, by combining the information regarding the maximum size of the two-divisionable luminance coding block obtained from the bitstream with the information regarding the difference in luminance block size, the size of the maximum luminance coding unit can be determined. Using the size of the maximum luminance coding unit, the size of the maximum chroma coding unit can also be determined. For example, if the Y:Cb:Cr ratio according to the color format is 4:2:0, the size of the chroma block may be half the size of the luminance block, and similarly, the size of the maximum chroma coding unit may be half the size of the maximum luminance coding unit.

[0108] According to one embodiment, information regarding the maximum size of a binary splittable luminous encoding block is obtained from a bitstream, so the maximum size of the binary splittable luminous encoding block can be determined variably. Alternatively, the maximum size of a ternary splittable luminous encoding block can be fixed. For example, the maximum size of a ternary splittable luminous encoding block in picture I may be 32x32, and the maximum size of a ternary splittable luminous encoding block in picture P or picture B may be 64x64.

[0109] Additionally, the maximum encoding unit can be hierarchically divided into encoding units based on splitting mode information obtained from the bitstream. As splitting mode information, at least one of information indicating whether it is a quad split, information indicating whether it is a multi-split, splitting direction information, and splitting type information can be obtained from the bitstream.

[0110] For example, information indicating whether quad splitting is performed can indicate whether the current encoding unit will be quad split or not.

[0111] If the current encoding unit is not quad-splitting, information indicating multi-splitting can indicate whether the current encoding unit will no longer be split (NO_SPLIT) or whether it will be binary / ternary split.

[0112] If the current encoding unit is binary or binary split, the splitting direction information indicates that the current encoding unit is split in either the horizontal or vertical direction.

[0113] If the current encoding unit is split horizontally or vertically, the split type information indicates that the current encoding unit is split into binary or binary splits.

[0114] The splitting mode of the current encoding unit can be determined based on the splitting direction information and the splitting type information. The splitting mode when the current encoding unit is binary split in the horizontal direction can be determined as binary horizontal splitting (SPLIT_BT_HOR), when it is territorial split in the horizontal direction as territorial horizontal splitting (SPLIT_TT_HOR), when it is binary split in the vertical direction as binary vertical splitting (SPLIT_BT_VER), and when it is territorial split in the vertical direction as territorial vertical splitting (SPLIT_TT_VER).

[0115] The image decoding device (100) can obtain splitting mode information from a bitstream from a single empty string. The form of the bitstream received by the image decoding device (100) may include a fixed-length binary code, a unary code, a truncated unary code, a predetermined binary code, etc. The empty string represents information as a sequence of binary numbers. The empty string may consist of at least one bit. The image decoding device (100) can obtain splitting mode information corresponding to the empty string based on a splitting rule. Based on the single empty string, the image decoding device (100) can determine whether to quad split the encoding unit, whether not to split it, or the splitting direction and splitting type.

[0116] A coding unit may be smaller than or equal to a maximum coding unit. For example, since the maximum coding unit is a coding unit having the maximum size, it is also a coding unit. If the segmentation mode information for the maximum coding unit indicates that it is not segmented, the coding unit determined from the maximum coding unit has the same size as the maximum coding unit. If the segmentation mode information for the maximum coding unit indicates that it is segmented, the maximum coding unit may be segmented into coding units. Additionally, if the segmentation mode information for a coding unit indicates segmentation, the coding units may be segmented into coding units of smaller size. However, the segmentation of the image is not limited to this, and the maximum coding unit and the coding unit may not be distinguished. The segmentation of coding units is explained in more detail in FIGS. 3 through 16.

[0117] Additionally, one or more prediction blocks for prediction may be determined from the coding unit. The prediction blocks may be equal to or smaller than the coding unit. Additionally, one or more transformation blocks for transformation may be determined from the coding unit. The transformation blocks may be equal to or smaller than the coding unit.

[0118] The shape and size of the transformation block and the prediction block may not be related to each other.

[0119] In another embodiment, prediction can be performed using the encoding unit as a prediction block. Additionally, conversion can be performed using the encoding unit as a conversion block.

[0120] The division of the encoding unit is described in more detail in FIGS. 3 through 16. The current block and surrounding block of the present disclosure may represent one of the maximum encoding unit, the encoding unit, the prediction block, and the transformation block. Additionally, the current block or the current encoding unit is a block currently undergoing decoding or encoding, or a block currently undergoing division. The surrounding block may be a block restored prior to the current block. The surrounding block may be spatially or temporally adjacent to the current block. The surrounding block may be located on one of the lower-left, left, upper-left, upper, upper-right, right, or lower-right sides of the current block.

[0121] FIG. 3 illustrates a process in which an image decoding device (100) divides a current encoding unit to determine at least one encoding unit according to one embodiment.

[0122] The block shape may include 4Nx4N, 4Nx2N, 2Nx4N, 4NxN, Nx4N, 32NxN, Nx32N, 16NxN, Nx16N, 8NxN, or Nx8N. Here, N may be a positive integer. Block shape information is information indicating at least one of the shape, orientation, width, and height ratio or size of the encoding unit.

[0123] The shape of the encoding unit may include square and non-square. When the width and height of the encoding unit are the same (i.e., when the block shape of the encoding unit is 4Nx4N), the image decoder (100) may determine the block shape information of the encoding unit as square. The image decoder (100) may determine the shape of the encoding unit as non-square.

[0124] When the width and height of the encoding unit are different (i.e., when the block shape of the encoding unit is 4Nx2N, 2Nx4N, 4NxN, Nx4N, 32NxN, Nx32N, 16NxN, Nx16N, 8NxN, or Nx8N), the image decoder (100) can determine the block shape information of the encoding unit as non-square. When the shape of the encoding unit is non-square, the image decoder (100) can determine the ratio of the width and height among the block shape information of the encoding unit as at least one of 1:2, 2:1, 1:4, 4:1, 1:8, 8:1, 1:16, 16:1, 1:32, or 32:1. Additionally, based on the width and height of the encoding unit, the image decoding device (100) can determine whether the encoding unit is in a horizontal direction or a vertical direction. Additionally, based on at least one of the width, height, or width of the encoding unit, the image decoding device (100) can determine the size of the encoding unit.

[0125] According to one embodiment, the image decoding device (100) can determine the form of the encoding unit using block form information and can determine how the encoding unit is divided using division form mode information. That is, depending on what block form the block form information used by the image decoding device (100) represents, the method of dividing the encoding unit represented by the division form mode information can be determined.

[0126] The video decoder (100) can obtain split-form mode information from the bitstream. However, it is not limited thereto, and the video decoder (100) and the video encoding device (200) can determine pre-agreed split-form mode information based on block form information. The video decoder (100) can determine pre-agreed split-form mode information for a maximum encoding unit or a minimum encoding unit. For example, the video decoder (100) can determine the split-form mode information for the maximum encoding unit as quad split. Additionally, the video decoder (100) can determine the split-form mode information for the minimum encoding unit as "not split." Specifically, the video decoder (100) can determine the size of the maximum encoding unit to be 256x256. The video decoder (100) can determine the pre-agreed split-form mode information as quad split. Quad split is a split-form mode that divides both the width and height of the encoding unit into two equal parts. The image decoder (100) can obtain a 128x128 size encoding unit from a 256x256 size encoding unit based on the division mode information. Additionally, the image decoder (100) can determine the size of the minimum encoding unit to be 4x4. The image decoder (100) can obtain division mode information indicating "not divided" for the minimum encoding unit.

[0127] According to one embodiment, the image decoding device (100) may use block shape information indicating that the current encoding unit is in the shape of a square. For example, the image decoding device (100) may determine whether to not divide the square encoding unit, to divide it vertically, to divide it horizontally, or to divide it into four encoding units, etc., according to the division shape mode information. Referring to FIG. 3, when the block shape information of the current encoding unit (300) indicates a square shape, the decoding unit (120) may not divide the encoding unit (310a) having the same size as the current encoding unit (300) according to the division shape mode information indicating that it is not divided, or may determine the divided encoding units (310b, 310c, 310d, 310e, 310f, etc.) based on the division shape mode information indicating a predetermined division method.

[0128] Referring to FIG. 3, the image decoding device (100) can determine two encoding units (310b) that divide the current encoding unit (300) in the vertical direction based on splitting form mode information indicating that it is divided in the vertical direction according to one embodiment. The image decoding device (100) can determine two encoding units (310c) that divide the current encoding unit (300) in the horizontal direction based on splitting form mode information indicating that it is divided in the horizontal direction. The image decoding device (100) can determine four encoding units (310d) that divide the current encoding unit (300) in the vertical direction and the horizontal direction based on splitting form mode information indicating that it is divided in the vertical direction and the horizontal direction. The image decoding device (100) can determine three encoding units (310e) that divide the current encoding unit (300) in the vertical direction based on splitting form mode information indicating that it is divided ternary in the vertical direction according to one embodiment. The image decoding device (100) can determine three encoding units (310f) that divide the current encoding unit (300) horizontally based on division form mode information indicating horizontal division. However, the division form in which the square encoding unit can be divided should not be interpreted as being limited to the form described above, and may include various forms that the division form mode information can represent. The specific division forms in which the square encoding unit is divided will be described in detail below through various embodiments.

[0129] FIG. 4 illustrates a process in which, according to one embodiment, an image decoding device (100) divides a non-square-shaped encoding unit to determine at least one encoding unit.

[0130] According to one embodiment, the image decoding device (100) may use block shape information indicating that the current encoding unit is in a non-square shape. The image decoding device (100) may determine whether to not divide the current encoding unit of the non-square shape or to divide it in a predetermined way according to the division shape mode information. Referring to FIG. 4, when the block shape information of the current encoding unit (400 or 450) indicates a non-square shape, the image decoding device (100) may determine an encoding unit (410 or 460) having the same size as the current encoding unit (400 or 450) based on division shape mode information indicating that it is not divided, or determine divided encoding units (420a, 420b, 430a, 430b, 430c, 470a, 470b, 480a, 480b, 480c) based on division shape mode information indicating a predetermined division method. A predetermined division method in which a non-square encoding unit is divided will be specifically described below through various embodiments.

[0131] According to one embodiment, the image decoding device (100) can determine the form in which the encoding unit is divided using the division form mode information, and in this case, the division form mode information may indicate the number of at least one encoding unit generated by dividing the encoding unit. Referring to FIG. 4, when the division form mode information indicates that the current encoding unit (400 or 450) is divided into two encoding units, the image decoding device (100) can determine two encoding units (420a, 420b, or 470a, 470b) included in the current encoding unit by dividing the current encoding unit (400 or 450) based on the division form mode information.

[0132] According to one embodiment, when an image decoding device (100) divides a current encoding unit (400 or 450) in a non-square shape based on division shape mode information, the image decoding device (100) may divide the current encoding unit by considering the position of the long side of the current encoding unit (400 or 450) in a non-square shape. For example, the image decoding device (100) may determine a plurality of encoding units by dividing the current encoding unit (400 or 450) in a direction that divides the long side of the current encoding unit (400 or 450) by considering the shape of the current encoding unit (400 or 450).

[0133] According to one embodiment, when the segmentation mode information indicates that the encoding unit is divided into an odd number of blocks (terminal segmentation), the image decoder (100) can determine an odd number of encoding units included in the current encoding unit (400 or 450). For example, when the segmentation mode information indicates that the current encoding unit (400 or 450) is divided into three encoding units, the image decoder (100) can divide the current encoding unit (400 or 450) into three encoding units (430a, 430b, 430c, 480a, 480b, 480c).

[0134] According to one embodiment, the ratio of the width to the height of the current encoding unit (400 or 450) may be 4:1 or 1:4. When the ratio of the width to the height is 4:1, the block shape information may be in the horizontal direction because the width is longer than the height. When the ratio of the width to the height is 1:4, the block shape information may be in the vertical direction because the width is shorter than the height. The image decoder (100) may determine to divide the current encoding unit into an odd number of blocks based on the division shape mode information. Additionally, the image decoder (100) may determine the division direction of the current encoding unit (400 or 450) based on the block shape information of the current encoding unit (400 or 450). For example, if the current encoding unit (400) is in a vertical direction, the image decoding device (100) can determine the encoding units (430a, 430b, 430c) by dividing the current encoding unit (400) in a horizontal direction. Also, if the current encoding unit (450) is in a horizontal direction, the image decoding device (100) can determine the encoding units (480a, 480b, 480c) by dividing the current encoding unit (450) in a vertical direction.

[0135] According to one embodiment, the image decoding device (100) may determine an odd number of encoding units included in the current encoding unit (400 or 450), and the sizes of the determined encoding units may not all be the same. For example, among the determined odd number of encoding units (430a, 430b, 430c, 480a, 480b, 480c), the size of a certain encoding unit (430b or 480b) may have a different size from the other encoding units (430a, 430c, 480a, 480c). That is, the current encoding unit (400 or 450) can be divided and determined as a encoding unit, and the encoding unit can have multiple types of sizes, and in some cases, an odd number of encoding units (430a, 430b, 430c, 480a, 480b, 480c) may each have different sizes.

[0136] According to one embodiment, if the segmentation mode information indicates that the encoding unit is divided into an odd number of blocks, the image decoding device (100) can determine the odd number of encoding units included in the current encoding unit (400 or 450), and furthermore, the image decoding device (100) can impose a predetermined limit on at least one encoding unit among the odd number of encoding units generated by the segmentation. Referring to FIG. 4, the image decoding device (100) can perform the decoding process for the central encoding unit (430b, 480b) among the three encoding units (430a, 430b, 430c, 480a, 480b, 480c) generated by the segmentation of the current encoding unit (400 or 450) differently from the other encoding units (430a, 430c, 480a, 480c). For example, the video decoding device (100) may restrict the centrally located encoding unit (430b, 480b) from being further divided unlike other encoding units (430a, 430c, 480a, 480c), or restrict it to being divided only a predetermined number of times.

[0137] FIG. 5 illustrates a process in which an image decoding device (100) divides a encoding unit based on at least one of block form information and division form mode information according to one embodiment.

[0138] According to one embodiment, the image decoding device (100) may determine whether to divide a square-shaped first encoding unit (500) into encoding units or not to divide it based on at least one of block shape information and division shape mode information. According to one embodiment, if the division shape mode information indicates that the first encoding unit (500) is divided in a horizontal direction, the image decoding device (100) may divide the first encoding unit (500) in a horizontal direction to determine a second encoding unit (510). The first encoding unit, the second encoding unit, and the third encoding unit used according to one embodiment are terms used to understand the relationship before and after division between the encoding units. For example, if the first encoding unit is divided, the second encoding unit may be determined, and if the second encoding unit is divided, the third encoding unit may be determined. In the following, the relationship between the first encoding unit, the second encoding unit, and the third encoding unit used may be understood as following the features described above.

[0139] According to one embodiment, the image decoding device (100) may determine whether to divide the determined second encoding unit (510) into encoding units or not to divide it based on the division shape mode information. Referring to FIG. 5, the image decoding device (100) may divide the determined non-square second encoding unit (510) into at least one third encoding unit (520a, 520b, 520c, 520d, etc.) by dividing the first encoding unit (500) based on the division shape mode information, or may not divide the second encoding unit (510). The image decoding device (100) can obtain split-form mode information, and the image decoding device (100) can obtain a plurality of second encoding units (e.g., 510) of various forms by dividing the first encoding unit (500) based on the obtained split-form mode information, and the second encoding unit (510) can be divided according to the method in which the first encoding unit (500) was divided based on the split-form mode information. According to one embodiment, when the first encoding unit (500) is divided into the second encoding unit (510) based on the split-form mode information for the first encoding unit (500), the second encoding unit (510) can also be divided into third encoding units (e.g., 520a, 520b, 520c, 520d, etc.) based on the split-form mode information for the second encoding unit (510). That is, the encoding unit can be recursively partitioned based on partitioning mode information associated with each encoding unit. Thus, a square encoding unit can be determined from a non-square encoding unit, and a non-square encoding unit can be determined by recursively partitioning this square encoding unit.

[0140] Referring to FIG. 5, among the odd number of third encoding units (520b, 520c, 520d) determined by dividing a second encoding unit (510) of a non-square shape, a predetermined encoding unit (e.g., a central encoding unit or a square encoding unit) may be recursively divided. According to one embodiment, a third encoding unit (520b) of a non-square shape, which is one of the odd number of third encoding units (520b, 520c, 520d), may be divided horizontally into a plurality of fourth encoding units. A fourth encoding unit (530b or 530d) of a non-square shape, which is one of the plurality of fourth encoding units (530a, 530b, 530c, 530d), may again be divided into a plurality of encoding units. For example, a non-square fourth encoding unit (530b or 530d) may be further divided into an odd number of encoding units. Methods that can be used for the recursive division of encoding units will be described later through various embodiments.

[0141] According to one embodiment, the image decoding device (100) may divide each of the third encoding units (520a, 520b, 520c, 520d, etc.) into encoding units based on the division shape mode information. Additionally, the image decoding device (100) may decide not to divide the second encoding unit (510) based on the division shape mode information. According to one embodiment, the image decoding device (100) may divide the non-square second encoding unit (510) into an odd number of third encoding units (520b, 520c, 520d). The image decoding device (100) may place a certain limit on a certain third encoding unit among the odd number of third encoding units (520b, 520c, 520d). For example, the video decoding device (100) may limit the encoding unit (520c) located in the middle of the odd number of third encoding units (520b, 520c, 520d) so that it is not further divided or is limited to being divided a set number of times.

[0142] Referring to FIG. 5, the image decoding device (100) may limit the middle encoding unit (520c) among the odd number of third encoding units (520b, 520c, 520d) included in the second encoding unit (510) of a non-square shape to no longer be divided, to be divided into a predetermined division form (e.g., divided into only 4 encoding units or divided into a form corresponding to the divided form of the second encoding unit (510)), or to be divided only a predetermined number of times (e.g., divided only n times, n > 0). However, the above limitation on the middle encoding unit (520c) is merely a simple example and should not be interpreted as being limited to the above-described examples, but should be interpreted as including various limitations that allow the middle encoding unit (520c) to be decoded differently from the other encoding units (520b, 520d).

[0143] According to one embodiment, the image decoding device (100) can obtain splitting form mode information used to split the current encoding unit at a predetermined location within the current encoding unit.

[0144] FIG. 6 illustrates a method for an image decoding device (100) to determine a predetermined encoding unit among an odd number of encoding units according to one embodiment.

[0145] Referring to FIG. 6, the segmentation mode information of the current encoding unit (600, 650) can be obtained from a sample at a specific location among a plurality of samples included in the current encoding unit (600, 650) (e.g., a sample located in the center (640, 690)). However, the specific location within the current encoding unit (600) where at least one of such segmentation mode information can be obtained should not be interpreted as being limited to the center location shown in FIG. 6, and should be interpreted as including various locations within the current encoding unit (600) (e.g., top, bottom, left, right, top-left, bottom-left, top-right, or bottom-right, etc.). The image decoding device (100) can obtain the segmentation mode information obtained from the specific location and decide whether to divide the current encoding unit into encoding units of various shapes and sizes or not to divide it.

[0146] According to one embodiment, the image decoding device (100) may select one of the encoding units when the current encoding unit is divided into a predetermined number of encoding units. There may be various methods for selecting one of the multiple encoding units, and such methods will be described later through various embodiments below.

[0147] According to one embodiment, the image decoding device (100) can divide the current encoding unit into a plurality of encoding units and determine the encoding unit at a predetermined position.

[0148] According to one embodiment, the image decoding device (100) may use information indicating the location of each of the odd number of encoding units to determine the encoding unit located in the middle among the odd number of encoding units. Referring to FIG. 6, the image decoding device (100) may divide the current encoding unit (600) or the current encoding unit (650) to determine the odd number of encoding units (620a, 620b, 620c) or the odd number of encoding units (660a, 660b, 660c). The image decoding device (100) may determine the middle encoding unit (620b) or the middle encoding unit (660b) by using information regarding the location of the odd number of encoding units (620a, 620b, 620c) or the odd number of encoding units (660a, 660b, 660c). For example, the image decoding device (100) can determine the centrally located encoding unit (620b) by determining the positions of the encoding units (620a, 620b, 620c) based on information indicating the positions of a predetermined sample included in the encoding units (620a, 620b, 620c). Specifically, the image decoding device (100) can determine the centrally located encoding unit (620b) by determining the positions of the encoding units (620a, 620b, 620c) based on information indicating the positions of the upper-left samples (630a, 630b, 630c) of the encoding units (620a, 620b, 620c).

[0149] According to one embodiment, information indicating the location of the upper-left sample (630a, 630b, 630c) included in each of the encoding units (620a, 620b, 620c) may include information regarding the location or coordinates within the picture of the encoding units (620a, 620b, 620c). According to one embodiment, information indicating the location of the upper-left sample (630a, 630b, 630c) included in each of the encoding units (620a, 620b, 620c) may include information indicating the width or height of the encoding units (620a, 620b, 620c) included in the current encoding unit (600), and such width or height may correspond to information indicating the difference between coordinates within the picture of the encoding units (620a, 620b, 620c). That is, the image decoding device (100) can determine the centrally located encoding unit (620b) by directly using information about the position or coordinates of the encoding units (620a, 620b, 620c) within the picture, or by using information about the width or height of the encoding unit corresponding to the difference value between the coordinates.

[0150] According to one embodiment, information indicating the location of the upper left sample (630a) of the upper encoding unit (620a) may be represented by the (xa, ya) coordinates, information indicating the location of the upper left sample (530b) of the middle encoding unit (620b) may be represented by the (xb, yb) coordinates, and information indicating the location of the upper left sample (630c) of the lower encoding unit (620c) may be represented by the (xc, yc) coordinates. The image decoding device (100) can determine the middle encoding unit (620b) using the coordinates of the upper left samples (630a, 630b, 630c) included in each of the encoding units (620a, 620b, 620c). For example, when the coordinates of the upper-left samples (630a, 630b, 630c) are sorted in ascending or descending order, the encoding unit (620b) containing the coordinates (xb, yb) of the sample (630b) located in the middle can be determined as the encoding unit located in the middle among the encoding units (620a, 620b, 620c) determined by dividing the current encoding unit (600). However, the coordinates indicating the position of the upper-left samples (630a, 630b, 630c) may represent absolute positions within the picture, and furthermore, based on the position of the upper-left sample (630a) of the upper-left of the upper-left of the middle encoding unit (620b), the (dxb, dyb) coordinates, which represent the relative position of the upper-left sample (630b) of the middle encoding unit (620b), and the (dxc, dyc) coordinates, which represent the relative position of the upper-left sample (630c) of the lower encoding unit (620c) may also be used. In addition, the method of determining the encoding unit of a predetermined position by using the coordinates of the corresponding sample as information indicating the position of the sample included in the encoding unit should not be interpreted as being limited to the method described above, but should be interpreted as various arithmetic methods that can utilize the coordinates of the sample.

[0151] According to one embodiment, the image decoding device (100) can divide the current encoding unit (600) into a plurality of encoding units (620a, 620b, 620c) and select an encoding unit among the encoding units (620a, 620b, 620c) according to a predetermined criterion. For example, the image decoding device (100) can select an encoding unit (620b) of a different size among the encoding units (620a, 620b, 620c).

[0152] According to one embodiment, the image decoding device (100) can determine the width or height of each of the encoding units (620a, 620b, 620c) using the (xa, ya) coordinates, which are information indicating the location of the upper left sample (630a) of the upper encoding unit (620a), the (xb, yb) coordinates, which are information indicating the location of the upper left sample (630b) of the middle encoding unit (620b), and the (xc, yc) coordinates, which are information indicating the location of the upper left sample (630c) of the lower encoding unit (620c). The image decoding device (100) can determine the size of each of the encoding units (620a, 620b, 620c) using the (xa, ya), (xb, yb), and (xc, yc) coordinates, which are information indicating the location of the encoding units (620a, 620b, 620c). According to one embodiment, the image decoding device (100) may determine the width of the upper encoding unit (620a) as the width of the current encoding unit (600). The image decoding device (100) may determine the height of the upper encoding unit (620a) as yb-ya. According to one embodiment, the image decoding device (100) may determine the width of the middle encoding unit (620b) as the width of the current encoding unit (600). The image decoding device (100) may determine the height of the middle encoding unit (620b) as yc-yb. According to one embodiment, the image decoding device (100) may determine the width or height of the lower encoding unit using the width or height of the current encoding unit and the width and height of the upper encoding unit (620a) and the middle encoding unit (620b). The video decoding device (100) can determine a encoding unit having a different size from other encoding units based on the width and height of the determined encoding units (620a, 620b, 620c).Referring to FIG. 6, the image decoding device (100) can determine a middle encoding unit (620b) having a size different from that of the upper encoding unit (620a) and the lower encoding unit (620c) as the encoding unit of a predetermined position. However, since the process of the image decoding device (100) described above determining the encoding unit having a size different from other encoding units is merely one embodiment of determining the encoding unit of a predetermined position using the size of the encoding unit determined based on sample coordinates, various processes of determining the encoding unit of a predetermined position by comparing the size of the encoding unit determined according to the predetermined sample coordinates may be used.

[0153] The image decoding device (100) can determine the width or height of each of the encoding units (660a, 660b, 660c) using the (xd, yd) coordinates, which are information indicating the location of the upper-left sample (670a) of the left encoding unit (660a), the (xe, ye) coordinates, which are information indicating the location of the upper-left sample (670b) of the middle encoding unit (660b), and the (xf, yf) coordinates, which are information indicating the location of the upper-left sample (670c) of the right encoding unit (660c). The image decoding device (100) can determine the size of each of the encoding units (660a, 660b, 660c) using the (xd, yd), (xe, ye), and (xf, yf) coordinates, which are information indicating the location of the encoding units (660a, 660b, 660c).

[0154] According to one embodiment, the image decoding device (100) may determine the width of the left encoding unit (660a) as xe-xd. The image decoding device (100) may determine the height of the left encoding unit (660a) as the height of the current encoding unit (650). According to one embodiment, the image decoding device (100) may determine the width of the middle encoding unit (660b) as xf-xe. The image decoding device (100) may determine the height of the middle encoding unit (660b) as the height of the current encoding unit (600). According to one embodiment, the image decoding device (100) may determine the width or height of the right encoding unit (660c) using the width or height of the current encoding unit (650) and the width and height of the left encoding unit (660a) and the middle encoding unit (660b). The image decoding device (100) can determine a encoding unit having a different size from other encoding units based on the width and height of the determined encoding units (660a, 660b, 660c). Referring to FIG. 6, the image decoding device (100) can determine a middle encoding unit (660b) having a different size from the left encoding unit (660a) and the right encoding unit (660c) as the encoding unit of a predetermined position. However, since the process of the image decoding device (100) described above determining a encoding unit having a different size from other encoding units is merely one embodiment of determining the encoding unit of a predetermined position using the size of the encoding unit determined based on sample coordinates, various processes of determining the encoding unit of a predetermined position by comparing the size of the encoding unit determined according to the predetermined sample coordinates may be used.

[0155] However, the sample location considered to determine the location of the encoding unit should not be interpreted as being limited to the upper left corner described above, and can be interpreted as allowing the use of information regarding the location of any sample included in the encoding unit.

[0156] According to one embodiment, the image decoding device (100) may select an encoding unit at a predetermined position among an odd number of encoding units determined by dividing the current encoding unit, taking into account the shape of the current encoding unit. For example, if the current encoding unit is a non-square shape where the width is longer than the height, the image decoding device (100) may determine an encoding unit at a predetermined position according to the horizontal direction. That is, the image decoding device (100) may determine one of the encoding units at different positions in the horizontal direction and place a restriction on that encoding unit. If the current encoding unit is a non-square shape where the height is longer than the width, the image decoding device (100) may determine an encoding unit at a predetermined position according to the vertical direction. That is, the image decoding device (100) may determine one of the encoding units at different positions in the vertical direction and place a restriction on that encoding unit.

[0157] According to one embodiment, the image decoding device (100) may use information indicating the location of each of the even number of encoding units to determine the encoding unit at a predetermined location among the even number of encoding units. The image decoding device (100) may determine the even number of encoding units by dividing (binary division) the current encoding unit and may determine the encoding unit at a predetermined location using information regarding the locations of the even number of encoding units. Since the specific process for this may correspond to the process of determining the encoding unit at a predetermined location (e.g., the middle location) among the odd number of encoding units described above in FIG. 6, it is omitted.

[0158] According to one embodiment, when a current encoding unit in a non-square shape is divided into a plurality of encoding units, certain information regarding the encoding unit at a certain position may be used during the division process to determine the encoding unit at a certain position among the plurality of encoding units. For example, the image decoding device (100) may use at least one of block shape information and division shape mode information stored in a sample included in the middle encoding unit during the division process to determine the encoding unit located in the middle among the encoding units into which the current encoding unit is divided into a plurality of encoding units.

[0159] Referring to FIG. 6, the image decoding device (100) can divide the current encoding unit (600) into a plurality of encoding units (620a, 620b, 620c) based on the division form mode information, and can determine the encoding unit (620b) located in the middle among the plurality of encoding units (620a, 620b, 620c). Furthermore, the image decoding device (100) can determine the encoding unit (620b) located in the middle by considering the location where the division form mode information is obtained. That is, the segmentation mode information of the current encoding unit (600) can be obtained from a sample (640) located in the middle of the current encoding unit (600), and based on the segmentation mode information, if the current encoding unit (600) is divided into a plurality of encoding units (620a, 620b, 620c), the encoding unit (620b) containing the sample (640) can be determined as the encoding unit located in the middle. However, the information used to determine the encoding unit located in the middle should not be interpreted as being limited to segmentation mode information, and various types of information may be used in the process of determining the encoding unit located in the middle.

[0160] According to one embodiment, a predetermined information for identifying a coding unit at a predetermined location may be obtained from a predetermined sample included in the coding unit to be determined. Referring to FIG. 6, an image decoding device (100) may use a segmentation mode information obtained from a sample at a predetermined location within the current coding unit (600) (for example, a sample at the center of the current coding unit (600)) to determine a coding unit at a predetermined location (for example, a coding unit located in the center of the multiple divided coding units) among a plurality of coding units (620a, 620b, 620c) determined by dividing the current coding unit (600). That is, the image decoding device (100) can determine a sample at the predetermined position by considering the block shape of the current encoding unit (600), and the image decoding device (100) can determine a encoding unit (620b) containing a sample from which certain information (e.g., division shape mode information) can be obtained among a plurality of encoding units (620a, 620b, 620c) from which the current encoding unit (600) is divided and determined, and can impose a certain limit. Referring to FIG. 6, according to one embodiment, the image decoding device (100) can determine a sample (640) located in the middle of the current encoding unit (600) as a sample from which certain information can be obtained, and the image decoding device (100) can impose a certain limit on the encoding unit (620b) containing such a sample (640) during the decoding process. However, the location of the sample from which the specified information can be obtained should not be interpreted as being limited to the location described above, but can be interpreted as samples at any location included in the encoding unit (620b) to be determined for the purpose of imposing a limitation.

[0161] According to one embodiment, the location of a sample from which a predetermined information can be obtained may be determined according to the shape of the current encoding unit (600). According to one embodiment, block shape information may determine whether the shape of the current encoding unit is square or non-square, and may determine the location of a sample from which a predetermined information can be obtained according to the shape. For example, the image decoding device (100) may determine a sample located on a boundary that divides at least one of the width and height of the current encoding unit in half using at least one of the information regarding the width and height of the current encoding unit as a sample from which a predetermined information can be obtained. As another example, if the block shape information related to the current encoding unit indicates that it is a non-square shape, the image decoding device (100) may determine one of the samples adjacent to the boundary that divides the long side of the current encoding unit in half as a sample from which a predetermined information can be obtained.

[0162] According to one embodiment, when the image decoding device (100) divides the current encoding unit into a plurality of encoding units, it may use division form mode information to determine the encoding unit at a predetermined position among the plurality of encoding units. According to one embodiment, the image decoding device (100) may obtain division form mode information from a sample at a predetermined position included in the encoding unit, and the image decoding device (100) may divide the plurality of encoding units generated by dividing the current encoding unit using the division form mode information obtained from a sample at a predetermined position included in each of the plurality of encoding units. That is, the encoding unit may be recursively divided using the division form mode information obtained from a sample at a predetermined position included in each of the encoding units. Since the recursive division process of the encoding unit has been described in detail through FIG. 5, a detailed explanation will be omitted.

[0163] According to one embodiment, the image decoding device (100) can determine at least one encoding unit by dividing the current encoding unit, and can determine the order in which the at least one encoding unit is decoded according to a predetermined block (e.g., the current encoding unit).

[0164] FIG. 7 illustrates the order in which a plurality of encoding units are processed when an image decoding device (100) determines a plurality of encoding units by dividing a current encoding unit according to one embodiment.

[0165] According to one embodiment, the image decoding device (100) may determine a second encoding unit (710a, 710b) by dividing a first encoding unit (700) in a vertical direction according to the splitting form mode information, determine a second encoding unit (730a, 730b) by dividing the first encoding unit (700) in a horizontal direction, or determine a second encoding unit (750a, 750b, 750c, 750d) by dividing the first encoding unit (700) in both a vertical and a horizontal direction.

[0166] Referring to FIG. 7, the image decoding device (100) can determine the order of processing the determined second encoding units (710a, 710b) in the horizontal direction (710c) by dividing the first encoding unit (700) in the vertical direction. The image decoding device (100) can determine the processing order of the determined second encoding units (730a, 730b) in the vertical direction (730c) by dividing the first encoding unit (700) in the horizontal direction. The image decoding device (100) can determine the second encoding unit (750a, 750b, 750c, 750d) determined by dividing the first encoding unit (700) into vertical and horizontal directions, according to a predetermined order in which encoding units located in one row are processed and then encoding units located in the next row are processed (e.g., raster scan order or z scan order (750e), etc.).

[0167] According to one embodiment, the image decoding device (100) can recursively divide the encoding units. Referring to FIG. 7, the image decoding device (100) can divide the first encoding unit (700) to determine a plurality of encoding units (710a, 710b, 730a, 730b, 750a, 750b, 750c, 750d), and can recursively divide each of the determined plurality of encoding units (710a, 710b, 730a, 730b, 750a, 750b, 750c, 750d). A method of dividing multiple encoding units (710a, 710b, 730a, 730b, 750a, 750b, 750c, 750d) may be a method corresponding to a method of dividing the first encoding unit (700). Accordingly, the multiple encoding units (710a, 710b, 730a, 730b, 750a, 750b, 750c, 750d) may each be independently divided into multiple encoding units. Referring to FIG. 7, the image decoding device (100) may determine the second encoding units (710a, 710b) by dividing the first encoding unit (700) in a vertical direction, and furthermore, may determine whether to independently divide or not divide each of the second encoding units (710a, 710b).

[0168] According to one embodiment, the image decoding device (100) may divide the second encoding unit (710a) on the left side horizontally into third encoding units (720a, 720b), and may not divide the second encoding unit (710b) on the right side.

[0169] According to one embodiment, the processing order of the encoding units may be determined based on the process of dividing the encoding units. In other words, the processing order of the divided encoding units may be determined based on the processing order of the encoding units immediately before they are divided. The image decoding device (100) may determine the processing order of the third encoding units (720a, 720b), which are determined by dividing the second encoding unit (710a) on the left, independently of the second encoding unit (710b) on the right. Since the third encoding units (720a, 720b) ​​are determined by dividing the second encoding unit (710a) on the left in a horizontal direction, the third encoding units (720a, 720b) ​​may be processed in a vertical direction (720c). In addition, since the processing order of the second encoding unit (710a) on the left and the second encoding unit (710b) on the right corresponds to the horizontal direction (710c), the third encoding unit (720a, 720b) ​​included in the second encoding unit (710a) on the left can be processed in the vertical direction (720c) before the right encoding unit (710b) is processed. The above description is intended to explain the process in which the processing order of the encoding units is determined according to the encoding unit before division, and therefore should not be interpreted as being limited to the above-described embodiment, but should be interpreted as being used in various ways in which encoding units determined by division in various forms can be processed independently according to a predetermined order.

[0170] FIG. 8 illustrates a process in which, according to one embodiment, an image decoding device (100) determines that the current encoding unit is divided into an odd number of encoding units when the encoding unit cannot be processed in a predetermined order.

[0171] According to one embodiment, the image decoding device (100) may determine that the current encoding unit is divided into an odd number of encoding units based on acquired segmentation mode information. Referring to FIG. 8, a square-shaped first encoding unit (800) may be divided into non-square-shaped second encoding units (810a, 810b), and the second encoding units (810a, 810b) may each be independently divided into third encoding units (820a, 820b, 820c, 820d, 820e). According to one embodiment, the image decoding device (100) can determine a plurality of third encoding units (820a, 820b) by dividing the left encoding unit (810a) among the second encoding units in a horizontal direction, and the right encoding unit (810b) can be divided into an odd number of third encoding units (820c, 820d, 820e).

[0172] According to one embodiment, the image decoding device (100) can determine whether there are an odd number of divided encoding units by determining whether the third encoding units (820a, 820b, 820c, 820d, 820e) can be processed in a predetermined order. Referring to FIG. 8, the image decoding device (100) can determine the third encoding units (820a, 820b, 820c, 820d, 820e) by recursively dividing the first encoding unit (800). The video decoding device (100) can determine whether the first encoding unit (800), the second encoding unit (810a, 810b), or the third encoding unit (820a, 820b, 820c, 820d, 820e) are divided into an odd number of encoding units based on at least one of block form information and division form mode information. For example, the encoding unit located on the right among the second encoding units (810a, 810b) may be divided into an odd number of third encoding units (820c, 820d, 820e). The order in which a plurality of encoding units included in the first encoding unit (800) are processed may be a predetermined order (e.g., z-scan order (830)), and the image decoding device (100) may determine whether the right second encoding unit (810b) is divided into an odd number of determined third encoding units (820c, 820d, 820e) can be processed according to the predetermined order.

[0173] According to one embodiment, the image decoding device (100) can determine whether the third encoding unit (820a, 820b, 820c, 820d, 820e) included in the first encoding unit (800) satisfies a condition that the third encoding unit (820a, 820b, 820c, 820d, 820e) can be processed in a predetermined order, and the condition relates to whether at least one of the width and height of the second encoding unit (810a, 810b) is divided in half according to the boundary of the third encoding unit (820a, 820b, 820c, 820d, 820e). For example, the third encoding unit (820a, 820b) determined by dividing the height of the left second encoding unit (810a) in a non-square shape in half can satisfy the condition. Since the boundaries of the third encoding units (820c, 820d, 820e), which are determined by dividing the right second encoding unit (810b) into three encoding units, do not divide the width or height of the right second encoding unit (810b) in half, the third encoding units (820c, 820d, 820e) may be determined not to satisfy the condition. In the case of such non-satisfaction of the condition, the image decoding device (100) determines that there is a disconnection in the scan order, and based on the result of the determination, the right second encoding unit (810b) may be determined to be divided into an odd number of encoding units. According to one embodiment, when the image decoding device (100) is divided into an odd number of encoding units, it may place a certain restriction on the encoding unit at a certain position among the divided encoding units, and since the details of such restriction or the certain position, etc., have been described in detail through various embodiments, a detailed explanation will be omitted.

[0174] FIG. 9 illustrates a process in which an image decoding device (100) divides a first encoding unit (900) to determine at least one encoding unit according to one embodiment.

[0175] According to one embodiment, the image decoding device (100) may divide the first encoding unit (900) based on the segmentation form mode information obtained through the bitstream acquisition unit (110). The first encoding unit (900) in a square shape may be divided into four square-shaped encoding units or into a plurality of non-square-shaped encoding units. For example, referring to FIG. 9, the first encoding unit (900) is square and the segmentation form mode information indicates that it is divided into non-square encoding units, so the image decoding device (100) may divide the first encoding unit (900) into a plurality of non-square encoding units. Specifically, when the splitting mode information indicates that the first encoding unit (900) is divided in a horizontal or vertical direction to determine an odd number of encoding units, the image decoding device (100) can divide the square-shaped first encoding unit (900) into an odd number of encoding units, such as a second encoding unit (910a, 910b, 910c) determined by dividing in a vertical direction or a second encoding unit (920a, 920b, 920c) determined by dividing in a horizontal direction.

[0176] According to one embodiment, the image decoding device (100) can determine whether the second encoding unit (910a, 910b, 910c, 920a, 920b, 920c) included in the first encoding unit (900) satisfies a condition that the second encoding unit (910a, 910b, 910c, 920a, 920b, 920c) can be processed in a predetermined order, and the condition relates to whether at least one of the width and height of the first encoding unit (900) is divided in half according to the boundary of the second encoding unit (910a, 910b, 910c, 920a, 920b, 920c). Referring to FIG. 9, the boundaries of the second encoding units (910a, 910b, 910c), which are determined by dividing the square-shaped first encoding unit (900) in the vertical direction, do not divide the width of the first encoding unit (900) in half, so the first encoding unit (900) may be determined not to satisfy the condition of being processed in a predetermined order. Additionally, the boundaries of the second encoding units (920a, 920b, 920c), which are determined by dividing the square-shaped first encoding unit (900) in the horizontal direction, do not divide the height of the first encoding unit (900) in half, so the first encoding unit (900) may be determined not to satisfy the condition of being processed in a predetermined order. The image decoding device (100) determines that if these conditions are not satisfied, there is a disconnection in the scan order, and based on the result of the determination, the first encoding unit (900) may be divided into an odd number of encoding units. According to one embodiment, when the image decoding device (100) is divided into an odd number of encoding units, it may place a certain restriction on the encoding unit at a certain position among the divided encoding units. Since the details of such restriction or the certain position have been described in detail through various embodiments, a detailed explanation will be omitted.

[0177] According to one embodiment, the image decoding device (100) can divide the first encoding unit to determine various forms of encoding units.

[0178] Referring to FIG. 9, the image decoding device (100) can divide a square-shaped first encoding unit (900) and a non-square-shaped first encoding unit (930 or 950) into various types of encoding units.

[0179] FIG. 10 illustrates that, according to one embodiment, when a video decoding device (100) divides a first encoding unit (1000) and a second encoding unit of a non-square shape determined by the division satisfies a predetermined condition, the shape in which the second encoding unit can be divided is limited.

[0180] According to one embodiment, the image decoding device (100) may decide to divide a square-shaped first encoding unit (1000) into non-square-shaped second encoding units (1010a, 1010b, 1020a, 1020b) based on division shape mode information obtained through a bitstream acquisition unit (110). The second encoding units (1010a, 1010b, 1020a, 1020b) may be divided independently. Accordingly, the image decoding device (100) may decide to divide into a plurality of encoding units or not divide based on division shape mode information related to each of the second encoding units (1010a, 1010b, 1020a, 1020b). According to one embodiment, the image decoding device (100) may determine a third encoding unit (1012a, 1012b) by dividing the left second encoding unit (1010a), which is a non-square shape determined by dividing the first encoding unit (1000) in the vertical direction, in the horizontal direction. However, when the image decoding device (100) divides the left second encoding unit (1010a) in the horizontal direction, the right second encoding unit (1010b) may be restricted so that it cannot be divided in the same horizontal direction as the left second encoding unit (1010a). If the right second encoding unit (1010b) is divided in the same direction to determine the third encoding unit (1014a, 1014b), the left second encoding unit (1010a) and the right second encoding unit (1010b) may be divided independently in the horizontal direction to determine the third encoding unit (1012a, 1012b, 1014a, 1014b). However, this is the same result as the image decoding device (100) dividing the first encoding unit (1000) into four square-shaped second encoding units (1030a, 1030b, 1030c, 1030d) based on the division shape mode information, and this may be inefficient in terms of image decoding.

[0181] According to one embodiment, the image decoding device (100) may determine a third encoding unit (1022a, 1022b, 1024a, 1024b) by dividing a first encoding unit (1000) in a horizontal direction and a second encoding unit (1020a or 1020b) in a non-square shape in a vertical direction. However, if the image decoding device (100) divides one of the second encoding units (e.g., the upper second encoding unit (1020a)) in a vertical direction, it may restrict the other second encoding unit (e.g., the lower encoding unit (1020b)) from being divided in the same vertical direction as the upper second encoding unit (1020a) in accordance with the above-described reason.

[0182] FIG. 11 illustrates the process of a video decoder (100) dividing square-shaped encoding units when, according to one embodiment, the divided shape mode information cannot be divided into four square-shaped encoding units.

[0183] According to one embodiment, the image decoding device (100) can determine the second encoding unit (1110a, 1110b, 1120a, 1120b, etc.) by dividing the first encoding unit (1100) based on the division shape mode information. The division shape mode information may include information on various shapes in which the encoding unit can be divided, but the information on various shapes may not include information for dividing into four square-shaped encoding units. According to this division shape mode information, the image decoding device (100) cannot divide the square-shaped first encoding unit (1100) into four square-shaped second encoding units (1130a, 1130b, 1130c, 1130d). Based on the segmented form mode information, the image decoding device (100) can determine a non-square second encoding unit (1110a, 1110b, 1120a, 1120b, etc.).

[0184] According to one embodiment, the image decoding device (100) can independently divide each of the second encoding units (1110a, 1110b, 1120a, 1120b, etc.) in a non-square shape. Each of the second encoding units (1110a, 1110b, 1120a, 1120b, etc.) can be divided in a predetermined order through a recursive method, and this may be a division method corresponding to the method in which the first encoding unit (1100) is divided based on division shape mode information.

[0185] For example, the image decoding device (100) can determine a square-shaped third encoding unit (1112a, 1112b) by dividing the left second encoding unit (1110a) in a horizontal direction, and can determine a square-shaped third encoding unit (1114a, 1114b) by dividing the right second encoding unit (1110b) in a horizontal direction. Furthermore, the image decoding device (100) can determine a square-shaped third encoding unit (1116a, 1116b, 1116c, 1116d) by dividing both the left second encoding unit (1110a) and the right second encoding unit (1110b) in a horizontal direction. In this case, the encoding unit can be determined in the same form as the first encoding unit (1100) being divided into four square-shaped second encoding units (1130a, 1130b, 1130c, 1130d).

[0186] As another example, the video decoding device (100) may determine a square-shaped third encoding unit (1122a, 1122b) by dividing the upper second encoding unit (1120a) in a vertical direction, and determine a square-shaped third encoding unit (1124a, 1124b) by dividing the lower second encoding unit (1120b) in a vertical direction. Furthermore, the video decoding device (100) may determine a square-shaped third encoding unit (1126a, 1126b, 1126a, 1126b) by dividing both the upper second encoding unit (1120a) and the lower second encoding unit (1120b) in a vertical direction. In this case, the encoding unit can be determined in the same form as the first encoding unit (1100) being divided into four square-shaped second encoding units (1130a, 1130b, 1130c, 1130d).

[0187] FIG. 12 illustrates that, according to one embodiment, the processing order between a plurality of encoding units may vary depending on the division process of the encoding unit.

[0188] According to one embodiment, the image decoding device (100) may divide a first encoding unit (1200) based on division shape mode information. When the block shape is square and the division shape mode information indicates that the first encoding unit (1200) is divided in at least one of the horizontal direction and the vertical direction, the image decoding device (100) may divide the first encoding unit (1200) to determine a second encoding unit (e.g., 1210a, 1210b, 1220a, 1220b, etc.). Referring to FIG. 12, the second encoding units (1210a, 1210b, 1220a, 1220b) in a non-square shape determined by dividing the first encoding unit (1200) only in the horizontal direction or the vertical direction may be divided independently based on the division shape mode information for each. For example, the video decoding device (100) can determine the third encoding unit (1216a, 1216b, 1216c, 1216d) by dividing the second encoding unit (1210a, 1210b) generated by dividing the first encoding unit (1200) in the vertical direction, and can determine the third encoding unit (1226a, 1226b, 1226c, 1226d) by dividing the second encoding unit (1220a, 1220b) generated by dividing the first encoding unit (1200) in the horizontal direction, respectively, in the vertical direction. Since the process of dividing these second encoding units (1210a, 1210b, 1220a, 1220b) has been described in detail in relation to FIG. 11, a detailed explanation will be omitted.

[0189] According to one embodiment, the image decoding device (100) can process encoding units in a predetermined order. Since the characteristics of processing encoding units in a predetermined order have been described in detail in relation to FIG. 7, a detailed explanation will be omitted. Referring to FIG. 12, the image decoding device (100) can divide a square-shaped first encoding unit (1200) to determine four square-shaped third encoding units (1216a, 1216b, 1216c, 1216d, 1226a, 1226b, 1226c, 1226d). According to one embodiment, the image decoding device (100) can determine the processing order of the third encoding unit (1216a, 1216b, 1216c, 1216d, 1226a, 1226b, 1226c, 1226d) according to the form in which the first encoding unit (1200) is divided.

[0190] According to one embodiment, the image decoding device (100) can determine the third encoding unit (1216a, 1216b, 1216c, 1216d) by dividing the second encoding unit (1210a, 1210b) generated by dividing in the vertical direction into the horizontal direction, and the image decoding device (100) can process the third encoding unit (1216a, 1216b, 1216c, 1216d) according to the order (1217) of first processing the third encoding unit (1216a, 1216c) included in the left second encoding unit (1210a) in the vertical direction, and then processing the third encoding unit (1216b, 1216d) included in the right second encoding unit (1210b) in the vertical direction.

[0191] According to one embodiment, the image decoding device (100) can determine the third encoding unit (1226a, 1226b, 1226c, 1226d) by dividing the second encoding unit (1220a, 1220b) generated by dividing in the horizontal direction into the vertical direction, and the image decoding device (100) can process the third encoding unit (1226a, 1226b, 1226c, 1226d) according to the order (1227) of first processing the third encoding unit (1226a, 1226b) included in the upper second encoding unit (1220a) in the horizontal direction, and then processing the third encoding unit (1226c, 1226d) included in the lower second encoding unit (1220b) in the horizontal direction.

[0192] Referring to FIG. 12, the second encoding units (1210a, 1210b, 1220a, 1220b) are each divided to determine the square-shaped third encoding units (1216a, 1216b, 1216c, 1216d, 1226a, 1226b, 1226c, 1226d). The second encoding unit (1210a, 1210b) determined by dividing in the vertical direction and the second encoding unit (1220a, 1220b) determined by dividing in the horizontal direction are divided into different forms, but according to the third encoding unit (1216a, 1216b, 1216c, 1216d, 1226a, 1226b, 1226c, 1226d) determined thereafter, the result is that the first encoding unit (1200) is divided into encoding units of the same form. Accordingly, the image decoding device (100) recursively divides the encoding unit through different processes based on the division form mode information, so that even if the encoding units of the same form are determined as a result, the multiple encoding units determined in the same form can be processed in different orders.

[0193] FIG. 13 illustrates a process in which, according to one embodiment, a encoding unit is recursively divided to determine a plurality of encoding units, and the depth of the encoding unit is determined as the shape and size of the encoding unit change.

[0194] According to one embodiment, the image decoding device (100) may determine the depth of the encoding unit according to a predetermined standard. For example, the predetermined standard may be the length of the long side of the encoding unit. The image decoding device (100) may determine that if the length of the long side of the current encoding unit is divided by 2n (n>0) times the length of the long side of the encoding unit before division, the depth of the current encoding unit is increased by n compared to the depth of the encoding unit before division. In the following, the encoding unit with increased depth is expressed as a lower depth encoding unit.

[0195] Referring to FIG. 13, according to one embodiment, based on block shape information indicating that it is a square shape (for example, the block shape information may indicate '0: SQUARE'), an image decoding device (100) can determine a second encoding unit (1302), a third encoding unit (1304), etc. of a lower depth by dividing a first encoding unit (1300) that is square in shape. If the size of the first encoding unit (1300) that is square in shape is 2Nx2N, the second encoding unit (1302), which is determined by dividing the width and height of the first encoding unit (1300) by half, may have a size of NxN. Furthermore, the third encoding unit (1304), which is determined by dividing the width and height of the second encoding unit (1302) by half, may have a size of N / 2xN / 2. In this case, the width and height of the third encoding unit (1304) correspond to 1 / 4 times the width and height of the first encoding unit (1300). When the depth of the first encoding unit (1300) is D, the depth of the second encoding unit (1302), which is 1 / 2 times the width and height of the first encoding unit (1300), may be D+1, and the depth of the third encoding unit (1304), which is 1 / 4 times the width and height of the first encoding unit (1300), may be D+2.

[0196] According to one embodiment, based on block shape information representing a non-square shape (for example, the block shape information may represent '1: NS_VER' indicating that the height is longer than the width, or '2: NS_HOR' indicating that the width is longer than the height), the image decoding device (100) may divide a first encoding unit (1310 or 1320) that is a non-square shape to determine a second encoding unit (1312 or 1322), a third encoding unit (1314 or 1324), etc. of a lower depth.

[0197] The image decoding device (100) can determine a second encoding unit (e.g., 1302, 1312, 1322, etc.) by dividing at least one of the width and height of a first encoding unit (1310) of size Nx2N. That is, the image decoding device (100) can determine a second encoding unit (1302) of size NxN or a second encoding unit (1322) of size NxN / 2 by dividing the first encoding unit (1310) in a horizontal direction, and can also determine a second encoding unit (1312) of size N / 2xN by dividing it in a horizontal direction and a vertical direction.

[0198] According to one embodiment, the image decoding device (100) may determine a second encoding unit (e.g., 1302, 1312, 1322, etc.) by dividing at least one of the width and height of a first encoding unit (1320) of size 2NxN. That is, the image decoding device (100) may determine a second encoding unit (1302) of size NxN or a second encoding unit (1312) of size N / 2xN by dividing the first encoding unit (1320) in the vertical direction, and may determine a second encoding unit (1322) of size NxN / 2 by dividing it in the horizontal and vertical directions.

[0199] According to one embodiment, the image decoding device (100) may determine a third encoding unit (e.g., 1304, 1314, 1324, etc.) by dividing at least one of the width and height of a second encoding unit (1302) of size NxN. That is, the image decoding device (100) may determine a third encoding unit (1304) of size N / 2xN / 2 by dividing the second encoding unit (1302) in a vertical direction and a horizontal direction, or determine a third encoding unit (1314) of size N / 4xN / 2, or determine a third encoding unit (1324) of size N / 2xN / 4.

[0200] According to one embodiment, the image decoding device (100) may determine a third encoding unit (e.g., 1304, 1314, 1324, etc.) by dividing at least one of the width and height of a second encoding unit (1312) of size N / 2xN. That is, the image decoding device (100) may determine a third encoding unit (1304) of size N / 2xN / 2 or a third encoding unit (1324) of size N / 2xN / 4 by dividing the second encoding unit (1312) in a horizontal direction, or determine a third encoding unit (1314) of size N / 4xN / 2 by dividing it in a vertical direction and a horizontal direction.

[0201] According to one embodiment, the image decoding device (100) may determine a third encoding unit (e.g., 1304, 1314, 1324, etc.) by dividing at least one of the width and height of a second encoding unit (1322) of size NxN / 2. That is, the image decoding device (100) may determine a third encoding unit (1304) of size N / 2xN / 2 or a third encoding unit (1314) of size N / 4xN / 2 by dividing the second encoding unit (1322) in the vertical direction, or determine a third encoding unit (1324) of size N / 2xN / 4 by dividing it in the vertical and horizontal directions.

[0202] According to one embodiment, the image decoding device (100) may divide square-shaped encoding units (e.g., 1300, 1302, 1304) in a horizontal or vertical direction. For example, a first encoding unit (1300) of size 2Nx2N may be divided in a vertical direction to determine a first encoding unit (1310) of size Nx2N, or divided in a horizontal direction to determine a first encoding unit (1320) of size 2NxN. According to one embodiment, when the depth is determined based on the length of the longest side of the encoding unit, the depth of the encoding unit determined by dividing the first encoding unit (1300) of size 2Nx2N in a horizontal or vertical direction may be the same as the depth of the first encoding unit (1300).

[0203] According to one embodiment, the width and height of the third encoding unit (1314 or 1324) may correspond to 1 / 4 times the width and height of the first encoding unit (1310 or 1320). If the depth of the first encoding unit (1310 or 1320) is D, the depth of the second encoding unit (1312 or 1322), which is 1 / 2 times the width and height of the first encoding unit (1310 or 1320), may be D+1, and the depth of the third encoding unit (1314 or 1324), which is 1 / 4 times the width and height of the first encoding unit (1310 or 1320), may be D+2.

[0204] FIG. 14 illustrates a depth and part index (hereinafter PID) for distinguishing between coding units that can be determined according to the shape and size of the coding units according to one embodiment.

[0205] According to one embodiment, the image decoding device (100) can determine various shapes of second encoding units by dividing a square-shaped first encoding unit (1400). Referring to FIG. 14, the image decoding device (100) can determine second encoding units (1402a, 1402b, 1404a, 1404b, 1406a, 1406b, 1406c, 1406d) by dividing the first encoding unit (1400) in at least one of a vertical direction and a horizontal direction according to the division shape mode information. That is, the image decoding device (100) can determine the second encoding unit (1402a, 1402b, 1404a, 1404b, 1406a, 1406b, 1406c, 1406d) based on the segmented form mode information for the first encoding unit (1400).

[0206] According to one embodiment, the depth of the second encoding unit (1402a, 1402b, 1404a, 1404b, 1406a, 1406b, 1406c, 1406d), which is determined according to the segmented shape mode information for the first encoding unit (1400) in a square shape, can be determined based on the length of the longer side. For example, since the length of one side of the first encoding unit (1400) in a square shape and the length of the longer side of the second encoding unit (1402a, 1402b, 1404a, 1404b) in a non-square shape are the same, the depth of the first encoding unit (1400) and the second encoding unit (1402a, 1402b, 1404a, 1404b) in a non-square shape can be considered to be the same as D. In contrast, when the video decoding device (100) divides the first encoding unit (1400) into four square-shaped second encoding units (1406a, 1406b, 1406c, 1406d) based on the division shape mode information, since the length of one side of the square-shaped second encoding units (1406a, 1406b, 1406c, 1406d) is half the length of one side of the first encoding unit (1400), the depth of the second encoding units (1406a, 1406b, 1406c, 1406d) may be a depth of D+1, which is one depth lower than the depth D of the first encoding unit (1400).

[0207] According to one embodiment, the image decoding device (100) may divide a first encoding unit (1410), in which the height is longer than the width, into a plurality of second encoding units (1412a, 1412b, 1414a, 1414b, 1414c) by dividing it in a horizontal direction according to the division shape mode information. According to one embodiment, the image decoding device (100) may divide a first encoding unit (1420), in which the width is longer than the height, into a plurality of second encoding units (1422a, 1422b, 1424a, 1424b, 1424c) by dividing it in a vertical direction according to the division shape mode information.

[0208] According to one embodiment, a second encoding unit (1412a, 1412b, 1414a, 1414b, 1414c, 1422a, 1422b, 1424a, 1424b, 1424c) determined according to the segmented shape mode information for a first encoding unit (1410 or 1420) of a non-square shape may have its depth determined based on the length of the longer side. For example, since the length of one side of the square-shaped second encoding unit (1412a, 1412b) is half the length of one side of the non-square-shaped first encoding unit (1410) in which the height is longer than the width, the depth of the square-shaped second encoding unit (1412a, 1412b) is D+1, which is one depth lower than the depth D of the non-square-shaped first encoding unit (1410).

[0209] Furthermore, the image decoding device (100) may divide a first encoding unit (1410) in a non-square shape into an odd number of second encoding units (1414a, 1414b, 1414c) based on the division shape mode information. The odd number of second encoding units (1414a, 1414b, 1414c) may include a second encoding unit (1414a, 1414c) in a non-square shape and a second encoding unit (1414b) in a square shape. In this case, since the length of the longer side of the non-square second encoding unit (1414a, 1414c) and the length of one side of the square second encoding unit (1414b) are half the length of one side of the first encoding unit (1410), the depth of the second encoding unit (1414a, 1414b, 1414c) may be a depth of D+1, which is one depth lower than the depth D of the first encoding unit (1410). The image decoding device (100) may determine the depth of the encoding units associated with the non-square first encoding unit (1420), in a manner corresponding to the above method of determining the depth of the encoding units associated with the first encoding unit (1410).

[0210] According to one embodiment, when determining an index (PID) for distinguishing divided encoding units, the video decoding device (100) may determine the index based on the size ratio between the encoding units when the odd number of divided encoding units are not of the same size. Referring to FIG. 14, the encoding unit (1414b) located in the middle among the odd number of divided encoding units (1414a, 1414b, 1414c) may have the same width as the other encoding units (1414a, 1414c) but may have twice the height of the other encoding units (1414a, 1414c). That is, in this case, the encoding unit (1414b) located in the middle may include two of the other encoding units (1414a, 1414c). Accordingly, if the index (PID) of the encoding unit (1414b) located in the middle according to the scan order is 1, the encoding unit (1414c) located in the next order may have an index of 3, which is an increase of 2. That is, there may be a discontinuity in the index values. According to one embodiment, the image decoding device (100) may determine whether the encoding units divided into an odd number are not of the same size based on whether there is a discontinuity in the index for distinguishing between these divided encoding units.

[0211] According to one embodiment, the image decoding device (100) may determine whether a plurality of encoding units determined by dividing from the current encoding unit are divided into a specific division form based on the value of an index for distinguishing the plurality of encoding units. Referring to FIG. 14, the image decoding device (100) may divide a first encoding unit (1410) in the shape of a rectangle whose height is greater than its width to determine an even number of encoding units (1412a, 1412b) or an odd number of encoding units (1414a, 1414b, 1414c). The image decoding device (100) may use an index (PID) representing each encoding unit to distinguish each of the plurality of encoding units. According to one embodiment, the PID may be obtained from a sample at a predetermined position of each encoding unit (e.g., the upper left sample).

[0212] According to one embodiment, the image decoding device (100) can determine a coding unit at a predetermined position among the coding units determined by division using an index for distinguishing the coding units. According to one embodiment, if the division shape mode information for a first coding unit (1410) in the form of a rectangle whose height is longer than its width indicates that it is divided into three coding units, the image decoding device (100) can divide the first coding unit (1410) into three coding units (1414a, 1414b, 1414c). The image decoding device (100) can assign an index to each of the three coding units (1414a, 1414b, 1414c). The image decoding device (100) can compare the indices for each coding unit to determine the middle coding unit among the odd number of divided coding units. The image decoding device (100) may determine a encoding unit (1414b) having an index corresponding to the middle value among the indices based on the indices of the encoding units, as the encoding unit at the middle position among the encoding units determined by dividing the first encoding unit (1410). According to one embodiment, when determining an index for distinguishing the divided encoding units, the image decoding device (100) may determine the index based on the size ratio between the encoding units if the encoding units are not of the same size. Referring to FIG. 14, the encoding unit (1414b) generated by dividing the first encoding unit (1410) may have the same width as the other encoding units (1414a, 1414c) but may have a height twice that of the other encoding units (1414a, 1414c). In this case, if the index (PID) of the encoding unit (1414b) located in the middle is 1, the encoding unit (1414c) located in the next order may have an index of 3, which is increased by 2.In cases where the index increases uniformly but the rate of increase changes, such as in this case, the image decoding device (100) may determine that the current encoding unit is divided into a plurality of encoding units, including encoding units having different sizes from other encoding units. In one embodiment, if the division type mode information indicates that the current encoding unit is divided into an odd number of encoding units, the image decoding device (100) may divide the current encoding unit in such a way that the encoding unit at a predetermined position among the odd number of encoding units (e.g., the middle encoding unit) has a different size from other encoding units. In this case, the image decoding device (100) may determine the middle encoding unit having a different size by using an index (PID) for the encoding unit. However, the aforementioned index, the size or position of the encoding unit at a predetermined position to be determined, is specific for the purpose of explaining one embodiment and should not be interpreted as being limited thereto, and should be interpreted as allowing various indices, positions, and sizes of encoding units to be used.

[0213] According to one embodiment, the image decoding device (100) may use a predetermined data unit in which recursive division of the encoding unit begins.

[0214] FIG. 15 illustrates that a plurality of encoding units are determined according to a plurality of predetermined data units included in a picture according to one embodiment.

[0215] According to one embodiment, a predetermined data unit may be defined as a data unit in which the encoding unit begins to recursively divide using the division form mode information. That is, it may correspond to the highest depth encoding unit used in the process of determining multiple encoding units that divide the current picture. For convenience of explanation, such a predetermined data unit will be referred to as a reference data unit below.

[0216] According to one embodiment, the reference data unit may have a predetermined size and shape. According to one embodiment, the reference data unit may include MxN samples. Here, M and N may be the same and may be integers expressed as powers of 2. That is, the reference data unit may have a square or non-square shape and may subsequently be divided into an integer number of encoding units.

[0217] According to one embodiment, the image decoding device (100) can divide the current picture into a plurality of reference data units. According to one embodiment, the image decoding device (100) can divide the current picture into a plurality of reference data units using division form mode information for each reference data unit. This division process of reference data units may correspond to a division process using a quad-tree structure.

[0218] According to one embodiment, the image decoding device (100) can predetermine the minimum size that a reference data unit included in the current picture may have. Accordingly, the image decoding device (100) can determine reference data units of various sizes having a size greater than or equal to the minimum size, and can determine at least one encoding unit using segmentation form mode information based on the determined reference data unit.

[0219] Referring to FIG. 15, the image decoding device (100) may use a square-shaped reference encoding unit (1500) or a non-square-shaped reference encoding unit (1502). According to one embodiment, the shape and size of the reference encoding unit may be determined according to various data units (e.g., sequence, picture, slice, slice segment, tile, tile group, maximum encoding unit, etc.) that may include at least one reference encoding unit.

[0220] According to one embodiment, the bitstream acquisition unit (110) of the image decoding device (100) can acquire at least one of information regarding the shape of the reference encoding unit and information regarding the size of the reference encoding unit from the bitstream for each of the various data units. The process of determining at least one encoding unit included in the square-shaped reference encoding unit (1500) has been described in detail through the process of dividing the current encoding unit (300) of FIG. 3, and the process of determining at least one encoding unit included in the non-square-shaped reference encoding unit (1502) has been described in detail through the process of dividing the current encoding unit (400 or 450) of FIG. 4, so a detailed explanation is omitted.

[0221] According to one embodiment, the image decoding device (100) may use an index to identify the size and shape of a reference encoding unit in order to determine the size and shape of a reference encoding unit according to a portion of data units that are predetermined based on a predetermined condition. That is, the bitstream acquisition unit (110) may acquire only an index for identifying the size and shape of a reference encoding unit for each slice, slice segment, tile, tile group, maximum encoding unit, etc., among the various data units (e.g., sequence, picture, slice, slice segment, tile, tile group, maximum encoding unit, etc.) from the bitstream, as a data unit that satisfies a predetermined condition (e.g., a data unit having a size smaller than or equal to a slice). By using the index, the image decoding device (100) can determine the size and shape of a reference data unit for each data unit that satisfies the predetermined condition. When information regarding the form of the reference encoding unit and information regarding the size of the reference encoding unit are obtained from the bitstream for each data unit of a relatively small size and used, the utilization efficiency of the bitstream may be poor; therefore, instead of directly obtaining information regarding the form of the reference encoding unit and information regarding the size of the reference encoding unit, only the index may be obtained and used. In this case, at least one of the size and form of the reference encoding unit corresponding to the index representing the size and form of the reference encoding unit may be predetermined. That is, the image decoding device (100) can determine at least one of the size and form of the reference encoding unit included in the data unit that serves as the basis for obtaining the index by selecting at least one of the predetermined size and form of the reference encoding unit according to the index.

[0222] According to one embodiment, the image decoding device (100) may utilize at least one reference encoding unit included in one maximum encoding unit. That is, the maximum encoding unit that divides the image may include at least one reference encoding unit, and the encoding unit may be determined through a recursive division process of each reference encoding unit. According to one embodiment, at least one of the width and height of the maximum encoding unit may correspond to an integer multiple of at least one of the width and height of the reference encoding unit. According to one embodiment, the size of the reference encoding unit may be the size obtained by dividing the maximum encoding unit n times according to a quad tree structure. That is, the image decoding device (100) may determine the reference encoding unit by dividing the maximum encoding unit n times according to a quad tree structure, and according to various embodiments, the reference encoding unit may be divided based on at least one of block form information and division form mode information.

[0223] According to one embodiment, the image decoding device (100) may obtain and use block form information indicating the form of the current encoding unit or division form mode information indicating a method of dividing the current encoding unit from a bitstream. The division form mode information may be included in bitstreams associated with various data units. For example, the image decoding device (100) may use division form mode information included in a sequence parameter set, a picture parameter set, a video parameter set, a slice header, a slice segment header, a tile header, and a tile group header. Furthermore, the image decoding device (100) may obtain and use syntax elements corresponding to the block form information or division form mode information from the bitstream for each maximum encoding unit and reference encoding unit.

[0224] A method for determining a division rule according to one embodiment of the present disclosure will be described in detail below.

[0225] The image decoding device (100) can determine the segmentation rule of the image. The segmentation rule may be predetermined between the image decoding device (100) and the image encoding device (200). The image decoding device (100) can determine the segmentation rule of the image based on information obtained from a bitstream. The image decoding device (100) can determine the segmentation rule based on information obtained from at least one of a sequence parameter set, a picture parameter set, a video parameter set, a slice header, a slice segment header, a tile header, and a tile group header. The image decoding device (100) can determine the segmentation rule differently depending on the frame, slice, tile, temporal layer, maximum encoding unit, or encoding unit.

[0226] The image decoding device (100) can determine a partitioning rule based on the block shape of the encoding unit. The block shape may include the size, shape, ratio of width and height, and orientation of the encoding unit. The image encoding device (200) and the image decoding device (100) may predetermine to determine a partitioning rule based on the block shape of the encoding unit. However, they are not limited thereto. The image decoding device (100) can determine a partitioning rule based on information obtained from a bitstream received from the image encoding device (200).

[0227] The shape of the encoding unit may include square and non-square. When the width and height of the encoding unit are the same, the image decoder (100) may determine the shape of the encoding unit as square. Additionally, when the width and height of the encoding unit are not the same, the image decoder (100) may determine the shape of the encoding unit as non-square.

[0228] The size of the encoding unit may include various sizes such as 4x4, 8x4, 4x8, 8x8, 16x4, 16x8, ..., 256x256. The size of the encoding unit may be classified according to the length of the long side, the length of the short side, or the width of the encoding unit. The image decoding device (100) may apply the same partitioning rule to encoding units classified into the same group. For example, the image decoding device (100) may classify encoding units having the same long side length into the same size. Additionally, the image decoding device (100) may apply the same partitioning rule to encoding units having the same long side length.

[0229] The ratio of width to height of a encoding unit may include 1:2, 2:1, 1:4, 4:1, 1:8, 8:1, 1:16, 16:1, 32:1, or 1:32, etc. Additionally, the direction of the encoding unit may include a horizontal direction and a vertical direction. The horizontal direction may indicate a case where the width of the encoding unit is longer than the height. The vertical direction may indicate a case where the width of the encoding unit is shorter than the height.

[0230] The image decoder (100) can adaptively determine a splitting rule based on the size of the encoding unit. The image decoder (100) can determine different acceptable splitting mode types based on the size of the encoding unit. For example, the image decoder (100) can determine whether splitting is allowed based on the size of the encoding unit. The image decoder (100) can determine the splitting direction according to the size of the encoding unit. The image decoder (100) can determine an acceptable splitting type according to the size of the encoding unit.

[0231] Determining the division rule based on the size of the encoding unit may be a division rule predetermined between the image encoding device (200) and the image decoding device (100). Additionally, the image decoding device (100) may determine the division rule based on information obtained from the bitstream.

[0232] The image decoder (100) can adaptively determine a partitioning rule based on the position of the encoding unit. The image decoder (100) can adaptively determine a partitioning rule based on the position occupied by the encoding unit in the image.

[0233] Additionally, the video decoding device (100) can determine a splitting rule so that encoding units generated by different splitting paths do not have the same block shape. However, this is not limited thereto, and encoding units generated by different splitting paths may have the same block shape. Encoding units generated by different splitting paths may have different decoding processing orders. Since the decoding processing order has been explained together with FIG. 12, a detailed explanation is omitted.

[0234] FIG. 16 illustrates the encoding units that can be determined for each picture when the combination of forms in which the encoding units can be divided according to one embodiment is different for each picture.

[0235] Referring to FIG. 16, the image decoding device (100) may determine different combinations of division forms in which the encoding unit can be divided for each picture. For example, the image decoding device (100) may decode an image using a picture (1600) that can be divided into four encoding units, a picture (1610) that can be divided into two or four encoding units, and a picture (1620) that can be divided into two, three, or four encoding units among at least one picture included in the image. To divide the picture (1600) into multiple encoding units, the image decoding device (100) may use only division form information indicating that it is divided into four square encoding units. To divide the picture (1610), the image decoding device (100) may use only division form information indicating that it is divided into two or four encoding units. The image decoding device (100) may use only division form information indicating that the picture (1620) is divided into two, three, or four encoding units in order to divide the picture. Since the combination of division forms described above is merely an example for explaining the operation of the image decoding device (100), the combination of division forms described above should not be interpreted as being limited to the above example, but should be interpreted as allowing various combinations of division forms to be used for each predetermined data unit.

[0236] According to one embodiment, a bitstream acquisition unit (110) of an image decoding device (100) may acquire a bitstream including an index representing a combination of division form information for each predetermined data unit (e.g., sequence, picture, slice, slice segment, tile, or tile group, etc.). For example, the bitstream acquisition unit (110) may acquire an index representing a combination of division form information from a sequence parameter set, a picture parameter set, a slice header, a tile header, or a tile group header. The image decoding device (100) of the image decoding device (100) may determine a combination of division forms in which the encoding unit can be divided for each predetermined data unit using the acquired index, and accordingly, different combinations of division forms may be used for each predetermined data unit.

[0237] FIG. 17 illustrates various forms of encoding units that can be determined based on partitioned form mode information that can be expressed as binary code according to one embodiment.

[0238] According to one embodiment, the image decoding device (100) can divide the encoding unit into various forms using block form information and division form mode information obtained through the bitstream acquisition unit (110). The forms of the encoding unit that can be divided may correspond to various forms including the forms described through the embodiments above.

[0239] Referring to FIG. 17, the image decoding device (100) can divide a square-shaped encoding unit into at least one of a horizontal direction and a vertical direction based on the divided shape mode information, and can divide a non-square-shaped encoding unit into a horizontal direction or a vertical direction.

[0240] According to one embodiment, if the image decoding device (100) can divide a square-shaped encoding unit into four square encoding units by dividing it in the horizontal and vertical directions, there may be four types of division forms that can be represented by the division form mode information for the square encoding unit. According to one embodiment, the division form mode information may be expressed as a 2-digit binary code, and a binary code may be assigned to each division form. For example, if the encoding unit is not divided, the division form mode information may be expressed as (00)b; if the encoding unit is divided in the horizontal and vertical directions, the division form mode information may be expressed as (01)b; if the encoding unit is divided in the horizontal direction, the division form mode information may be expressed as (10)b; and if the encoding unit is divided in the vertical direction, the division form mode information may be expressed as (11)b.

[0241] According to one embodiment, when the image decoding device (100) divides a non-square-shaped encoding unit in a horizontal or vertical direction, the type of division form that can be represented by the division form mode information may be determined by how many encoding units are divided. Referring to FIG. 17, according to one embodiment, the image decoding device (100) may divide a non-square-shaped encoding unit into up to three. The image decoding device (100) may divide the encoding unit into two encoding units, in which case the division form mode information may be expressed as (10)b. The image decoding device (100) may divide the encoding unit into three encoding units, in which case the division form mode information may be expressed as (11)b. The image decoding device (100) may decide not to divide the encoding unit, in which case the division form mode information may be expressed as (0)b. That is, the video decoding device (100) can use variable length coding (VLC) rather than fixed length coding (FLC) to use binary code representing segmented form mode information.

[0242] Referring to FIG. 17 according to one embodiment, the binary code of the division type mode information indicating that the encoding unit is not divided can be represented as (0)b. If the binary code of the division type mode information indicating that the encoding unit is not divided is set to (00)b, then all 2 bits of the binary code of the division type mode information must be used even though there is no division type mode information set to (01)b. However, as illustrated in FIG. 17, if three division types for a non-square encoding unit are used, the image decoding device (100) can determine that the encoding unit is not divided even if a 1-bit binary code (0)b is used as the division type mode information, thus allowing for efficient use of the bitstream. However, the division type of the non-square encoding unit indicated by the division type mode information should not be interpreted as being limited only to the three types illustrated in FIG. 17, but should be interpreted as various types including the embodiments described above.

[0243] FIG. 18 illustrates another form of encoding unit that can be determined based on partitioned form mode information that can be represented as binary code according to one embodiment.

[0244] Referring to FIG. 18, the image decoding device (100) can divide a square-shaped encoding unit in a horizontal or vertical direction based on the division shape mode information, and can divide a non-square-shaped encoding unit in a horizontal or vertical direction. That is, the division shape mode information can indicate that a square-shaped encoding unit is divided in one direction. In this case, the binary code of the division shape mode information indicating that the square-shaped encoding unit is not divided can be represented as (0)b. If the binary code of the division shape mode information indicating that the encoding unit is not divided is set to (00)b, then all 2 bits of the binary code of the division shape mode information must be used even though there is no division shape mode information set to (01)b. However, as illustrated in FIG. 18, if three division forms for a square-shaped encoding unit are used, the image decoding device (100) can determine that the encoding unit is not divided even if a 1-bit binary code (0)b is used as division form mode information, thus allowing the bitstream to be used efficiently. However, the division form of the square-shaped encoding unit indicated by the division form mode information should not be interpreted as being limited only to the three forms illustrated in FIG. 18, but should be interpreted as various forms including the embodiments described above.

[0245] According to one embodiment, block form information or partition form mode information can be represented using binary code, and such information can be immediately generated as a bitstream. Additionally, block form information or partition form mode information that can be represented by binary code may not be immediately generated as a bitstream but may be used as binary code input in CABAC (context adaptive binary arithmetic coding).

[0246] According to one embodiment, the image decoding device (100) describes a process of obtaining syntax for block form information or partition form mode information through CABAC. A bitstream containing a binary code for the syntax can be obtained through a bitstream acquisition unit (110). The image decoding device (100) can detect a syntax element representing block form information or partition form mode information by debinding a bin string included in the obtained bitstream. According to one embodiment, the image decoding device (100) obtains a set of binary bin strings corresponding to the syntax element to be decoded, and can decode each bin using probability information, and the image decoding device (100) can repeat this process until the bin string composed of these decoded bins becomes equal to one of the previously obtained bin strings. The image decoding device (100) can determine the syntax element by performing debinding of the bin string.

[0247] According to one embodiment, the image decoding device (100) can determine the syntax for a bin string by performing a decoding process of adaptive binary arithmetic coding, and the image decoding device (100) can update a probability model for the bins obtained through a bitstream acquisition unit (110). Referring to FIG. 17, the bitstream acquisition unit (110) of the image decoding device (100) can obtain a bitstream representing a binary code representing partitioned mode information according to one embodiment. Using the obtained binary code having a size of 1 bit or 2 bits, the image decoding device (100) can determine the syntax for the partitioned mode information. To determine the syntax for the partitioned mode information, the image decoding device (100) can update the probability for each bit of the 2-bit binary code. That is, the image decoding device (100) can update the probability of having a value of 0 or 1 when decoding the next bin, depending on whether the value of the first bin of the 2-bit binary code is 0 or 1.

[0248] According to one embodiment, the image decoding device (100) can update the probability for the bins used in the process of decoding the bins of the empty string for the syntax during the process of determining the syntax, and the image decoding device (100) can determine that the probability is not updated for certain bits of the empty string and has the same probability.

[0249] Referring to FIG. 17, in the process of determining syntax using an empty string representing segmentation mode information for a non-square type encoding unit, the image decoder (100) can determine syntax for the segmentation mode information using one bin having a value of 0 when the non-square type encoding unit is not segmented. That is, when block type information indicates that the current encoding unit is a non-square type, the first bin of the empty string for the segmentation mode information may be 0 when the non-square type encoding unit is not segmented, and 1 when it is segmented into two or three encoding units. Accordingly, the probability that the first bin of the empty string for the segmentation mode information for a non-square type encoding unit is 0 may be 1 / 3, and the probability that it is 1 may be 2 / 3. As described above, since the image decoding device (100) can represent only a 1-bit empty string having a value of 0 as the segmentation mode information indicating that a non-square type encoding unit is not segmented, the image decoding device (100) can determine the syntax for the segmentation mode information by determining whether the second bin is 0 or 1 only when the first bin of the segmentation mode information is 1. According to one embodiment, the image decoding device (100) can decode the bin by considering that when the first bin of the segmentation mode information is 1, the probability that the second bin is 0 or 1 is equal to the probability.

[0250] According to one embodiment, the image decoding device (100) may use various probabilities for each bin in the process of determining the bins of the bin string for the segmented form mode information. According to one embodiment, the image decoding device (100) may determine the probability of the bins for the segmented form mode information differently depending on the direction of the non-square block. According to one embodiment, the image decoding device (100) may determine the probability of the bins for the segmented form mode information differently depending on the width or the length of the long side of the current encoding unit. According to one embodiment, the image decoding device (100) may determine the probability of the bins for the segmented form mode information differently depending on at least one of the shape of the current encoding unit and the length of the long side.

[0251] According to one embodiment, the image decoding device (100) may determine that the probability of bins for segmented form mode information is the same for encoding units of a predetermined size or larger. For example, based on the length of the long side of the encoding unit, the probability of bins for segmented form mode information is determined to be the same for encoding units of a size of 64 samples or larger.

[0252] According to one embodiment, the image decoding device (100) may determine the initial probability for the bins constituting the empty string of the segmented form mode information based on the slice type (e.g., I slice, P slice, or B slice).

[0253] Figure 19 is a block diagram of an image encoding and decoding system that performs loop filtering.

[0254] The encoding unit (1910) of the video encoding and decoding system (1900) transmits an encoded bitstream of the video, and the decoding unit (1950) receives the bitstream and decodes it to output a restored video. Here, the encoding unit (1910) may have a configuration similar to the video encoding device (200) described later, and the decoding unit (1950) may have a configuration similar to the video decoding device (100).

[0255] In the encoding unit (1910), the prediction encoding unit (1915) outputs prediction data through inter-prediction and intra-prediction, and the transformation and quantization unit (1920) outputs quantized transformation coefficients of residual data between the prediction data and the current input image. The entropy encoding unit (1925) encodes and transforms the quantized transformation coefficients and outputs them as a bitstream. The quantized transformation coefficients are restored to spatial domain data through the inverse quantization and inverse transformation unit (1930), and the restored spatial domain data is output as a restored image through the deblocking filtering unit (1935) and the loop filtering unit (1940). The restored image can be used as a reference image for the next input image through the prediction encoding unit (1915).

[0256] The encoded image data among the bitstreams received by the decoding unit (1950) is restored into spatial domain residual data through the entropy decoding unit (1955) and the inverse quantization and inverse transform unit (1960). The predicted data and residual data output from the prediction decoding unit (1975) are combined to form spatial domain image data, and the deblocking filtering unit (1965) and the loop filtering unit (1970) can perform filtering on the spatial domain image data to output a restored image for the current original image. The restored image can be used as a reference image for the next original image by the prediction decoding unit (1975).

[0257] The loop filtering unit (1940) of the encoding unit (1910) performs loop filtering using filter information input according to user input or system settings. The filter information used by the loop filtering unit (1940) is output to the entropy encoding unit (1925) and transmitted to the decoding unit (1950) together with the encoded image data. The loop filtering unit (1970) of the decoding unit (1950) can perform loop filtering based on the filter information input from the decoding unit (1950).

[0258] The various embodiments described above explain the operation related to the image decoding method performed by the image decoding device (100). Below, the operation of the image encoding device (200) that performs an image encoding method corresponding to the reverse process of the image decoding method will be explained through various embodiments.

[0259] FIG. 2 illustrates a block diagram of an image encoding device (200) capable of encoding an image based on at least one of block shape information and segmented shape mode information according to one embodiment.

[0260] The video encoding device (200) may include an encoding unit (220) and a bitstream generation unit (210). The encoding unit (220) may receive an input video and encode the input video. The encoding unit (220) may encode the input video to obtain at least one syntax element. The syntax element may include at least one of a skip flag, a prediction mode, a motion vector difference, a motion vector prediction method (or index), a transform quantized coefficient, a coded block pattern, a coded block flag, an intra prediction mode, a direct flag, a merge flag, a delta QP, a reference index, a prediction direction, and a transform index. The encoding unit (220) may determine a context model based on block shape information including at least one of the shape, direction, width, and height ratio or size of the encoding unit.

[0261] The bitstream generation unit (210) can generate a bitstream based on the encoded input image. For example, the bitstream generation unit (210) can generate a bitstream by entropy encoding syntax elements based on a context model. Additionally, the image encoding device (200) can transmit the bitstream to the image decoding device (100).

[0262] According to one embodiment, the encoding unit (220) of the image encoding device (200) can determine the shape of the encoding unit. For example, the encoding unit may be square or non-square in shape, and information indicating such shape may be included in block shape information.

[0263] According to one embodiment, the encoding unit (220) can determine how the encoding unit will be divided. The encoding unit (220) can determine the form of at least one encoding unit included in the encoding unit, and the bitstream generation unit (210) can generate a bitstream including division form mode information that includes information about the form of such encoding unit.

[0264] According to one embodiment, the encoding unit (220) may determine whether the encoding unit is divided or not. If the encoding unit (220) determines that only one encoding unit is included in the encoding unit or that the encoding unit is not divided, the bitstream generation unit (210) may generate a bitstream including division type mode information indicating that the encoding unit is not divided. Additionally, the encoding unit (220) may divide the encoding unit into a plurality of encoding units, and the bitstream generation unit (210) may generate a bitstream including division type mode information indicating that the encoding unit is divided into a plurality of encoding units.

[0265] According to one embodiment, information indicating how many encoding units to divide or in which direction to divide may be included in the division type mode information. For example, the division type mode information may indicate dividing in at least one of the vertical direction and the horizontal direction, or not dividing.

[0266] The video encoding device (200) determines information regarding the segmentation mode based on the segmentation mode of the encoding unit. The video encoding device (200) determines a context model based on at least one of the ratio or size of the shape, direction, width, and height of the encoding unit. Then, the video encoding device (200) generates information regarding the segmentation mode for segmenting the encoding unit based on the context model as a bitstream.

[0267] The video encoding device (200) may obtain an array for matching at least one of the ratio or size of the shape, direction, width, and height of an encoding unit with an index for the context model in order to determine the context model. The video encoding device (200) may obtain an index for the context model based on at least one of the ratio or size of the shape, direction, width, and height of an encoding unit in the array. The video encoding device (200) may determine the context model based on the index for the context model.

[0268] The video encoding device (200) may determine a context model based further on block shape information including at least one of the ratio or size of the shape, direction, width, and height of a surrounding encoding unit adjacent to the encoding unit, in order to determine the context model. Additionally, the surrounding encoding unit may include at least one of the encoding units located on the lower left, left, upper left, upper, upper right, right, or lower right side of the encoding unit.

[0269] Additionally, the video encoding device (200) can compare the width of an upper peripheral encoding unit with the width of an encoding unit in order to determine a context model. Additionally, the video encoding device (200) can compare the height of left and right peripheral encoding units with the height of an encoding unit. Additionally, the video encoding device (200) can determine a context model based on the comparison results.

[0270] The operation of the video encoding device (200) includes content similar to the operation of the video decoding device (100) described in FIGS. 3 to 19, so a detailed description is omitted.

[0271] FIG. 20 is a block diagram showing the configuration of an image decoding device according to one embodiment.

[0272] Referring to FIG. 20, the image decoding device (2000) may include a processor (2010) and a memory (2020).

[0273] In one embodiment of the present disclosure, the processor (2010) may include processing circuits and / or multiple processors. For example, the processor (2010) may include various processing circuits including at least one processor, and at least one of the at least one processor may be configured to perform the various functions described in the present disclosure individually and / or collectively in a distributed manner.

[0274] In one embodiment of the present disclosure, the memory (2020) may include one or more storage media that store at least one instruction. The processor (2010) may control the image decoding device (2000) by executing the instruction stored in the memory (2020). For example, the processor (2010) may control the image decoding device (2000) to perform an operation by executing the instruction stored in the memory (2020) individually or collectively. In one embodiment of the present disclosure, the operation performed by the image decoding device (2000) may be an operation performed by the processor (2010) of the image decoding device (2000).

[0275] In one embodiment of the present disclosure, the image decoding device (2000) may correspond to the image decoding device (100) shown in FIG. 1 and / or the decoding unit (1950) shown in FIG. 19.

[0276] The image decoder (2000) can obtain a bitstream generated as a result of encoding an image. The bitstream may include the encoding result for the current block. In one embodiment of the present disclosure, the image decoder (2000) can receive the bitstream from the image encoding device via a network.

[0277] In one embodiment of the present disclosure, an image decoding device (2000) can obtain a bitstream from a data storage medium comprising at least one of a magnetic medium such as a hard disk, a floppy disk and a magnetic tape, an optical recording medium such as a CD-ROM and a DVD, or a magneto-optical medium such as a floptical disk.

[0278] The image decoding device (2000) can obtain syntax elements for decoding an image from a bitstream. Values ​​corresponding to the syntax elements may be included in the bitstream according to the hierarchical structure of the image. In one embodiment of the present disclosure, the image decoding device (2000) can obtain syntax elements by entropy decoding bins included in the bitstream.

[0279] In one embodiment of the present disclosure, the bitstream may include information regarding the prediction mode of the current block in the current image. The current block may include at least one of a maximum encoding unit, an encoding unit, a transformation unit, or a prediction unit divided from the current image to be decoded.

[0280] In one embodiment, the image can be partitioned into tiles, slices, subpictures, CTU (Coding Tree Unit), CTB (Coding Tree Block), CU (Coding Unit), CB (Coding Block), PU (Prediction Unit), PB (Prediction Block), TU (Transform Unit), TB (Transform Block), etc., for various purposes. Subsequently, detailed processes in encoding and decoding can be performed in block units.

[0281] In one embodiment, prediction can be classified into intra prediction, inter prediction (or motion compensation prediction), and combined prediction utilizing both intra prediction and inter prediction (Combined Intra / Inter Prediction or Multi-hypothesis Prediction). Intra prediction refers to the process of generating a predicted image using restored samples (or pixels, images) within the current picture. Inter prediction refers to the process of generating a predicted image using motion information from a reference picture. Combined prediction refers to the process of generating a predicted image by combining the two types of prediction—intra prediction and inter prediction—in various forms.

[0282] After performing a prediction of the block, the image encoding device can transmit the residual (or difference) of the original image and the predicted image to the image decoding device (2000). A conversion and quantization process can be performed on the residual. The quantized residual data can be output as a bitstream through an entropy coding process and transmitted to the image decoding device (2000).

[0283] The image decoding device (2000) can obtain a decoded residual by performing inverse quantization and inverse transform processes on the quantized residual data, and obtain a restored image (or decoded image) by adding it to the predicted image. The restored image can be stored in a decoded pictures buffer (DPB) for a subsequent prediction process. The image decoding device (2000) can apply an in-loop filter to the decoded image before storing it in the DPB.

[0284] In one embodiment, the image decoding device (2000) may apply a deblocking filter to reduce blocking artifacts, a sample adaptive offset (SAO) to remove various noises in the image or to improve subjective image quality, a bilateral filter (BIF), an adaptive loop filter (ALF), a neural network based loop filter (NN), etc.

[0285] In one embodiment, in-loop filtering may be performed on a block, tile, slice, or picture basis. The image decoder (2000) may optionally perform post-processing at the output stage of the picture and then perform output. The image decoder (2000) may additionally perform a luminance mapping and / or chroma residual scaling process. Luminance mapping and chroma residual scaling may be applied as tone mapping techniques to improve coding efficiency.

[0286] In one embodiment, the intra prediction mode may be selected from a plurality of intra prediction modes. The plurality of intra prediction modes may include a non-directional intra prediction mode and a directional intra prediction mode. An intra prediction mode according to one embodiment of the present disclosure is described with reference to FIG. 21.

[0287] In one embodiment, the block copy mode may include an intra block copy mode (IBC). In one embodiment, the block copy mode may include an intra block copy mode. In one embodiment, the intra block copy mode may be a sub-mode of the intra mode, but is not limited thereto, and may represent a mode distinct from the intra mode.

[0288] In one embodiment, when the prediction mode of the current block is a template matching prediction mode, the image decoder (2000) can restore the current block using a reference block. The image decoder (2000) can obtain information regarding whether to use the template matching prediction mode. The image decoder (2000) can determine whether to use the template matching prediction mode based on the obtained information. The reference block may be determined based on at least one of the region included in the current image or the region included in the previously decoded image.

[0289] In one embodiment, the intermode can perform prediction using motion information. For example, the motion information may include a prediction direction, a reference picture index, and a motion vector. Within the reference picture, a reference block may be identified by the motion vector.

[0290] The video decoding device (2000) can restore the current block by performing a prediction according to the prediction mode for the current block according to the prediction mode of the current block.

[0291] In one embodiment, the image decoder (2000) can obtain information regarding the prediction mode of the current block from the bitstream. For example, the image decoder (2000) can obtain index information indicating the prediction mode of the current block from the bitstream.

[0292] In a prediction mode (e.g., intra mode) utilizing reference samples included in the current image, a prediction block of the current block can be generated based on the surrounding samples of the current block according to the prediction mode, under the assumption that there is continuity between the surrounding samples of the current block and the samples within the current block. An image decoding device (2000) according to one embodiment of the present disclosure may utilize spatial reference samples included in the current image, as well as the surrounding samples of the current block included in the current image, for intra prediction. When using samples restored before the current block, the size of residual data can be reduced by predicting the samples of the current block using not only samples immediately adjacent to the current block but also samples far from the current block.

[0293] In a prediction mode (e.g., inter mode) that utilizes reference samples included in a reference image rather than the current image, a prediction block of the current block can be generated based on a reference block (or reference sample) of the reference image according to the prediction mode, under the assumption that there is continuity between the current image and the reference image. An image decoding device (2000) according to one embodiment of the present disclosure can improve compression efficiency by increasing the efficiency of intra prediction.

[0294] The image decoding device (2000) can improve prediction accuracy by considering together the reference block (or reference sample) included in the current image and the reference block (or reference sample) included in an image other than the current image. The image decoding device (2000) according to one embodiment of the present disclosure can improve prediction accuracy by considering both the current image and an image other than the current image.

[0295] The image decoding device (2000) can perform deblocking filtering. The deblocking filter can improve image quality by smoothing the edges between blocks.

[0296] The image decoding device (2000) can perform filtering on samples of the current block where deblocking filtering has been performed using a Sample Adaptive Offset (SAO) filter and / or a Bilateral Filter (BIF). The SAO filter and BIF can improve image quality by reducing the error between the restored image and the original image. The SAO filter and BIF can perform filtering on a sample-by-sample basis.

[0297] The image decoding device (2000) can perform filtering using an Adaptive Loop Filter (ALF). The ALF can improve image quality by reducing the error between the restored image and the original image. The ALF can perform filtering in block units.

[0298] FIG. 21 is a diagram illustrating intra-prediction modes according to one embodiment.

[0299] In one embodiment of the present disclosure, the image decoder (2000) may determine an intra prediction mode of the current block (e.g., encoding block) to perform intra prediction. The image decoder (2000) may select an intra prediction mode of the current block from among a plurality of intra prediction modes.

[0300] Referring to FIG. 21, in one embodiment of the present disclosure, a plurality of intra prediction modes may include non-directional planar mode 0 (or, Intra_Planar mode), non-directional DC mode 1 (or, Intra_DC mode), and directional modes 2 through 66 (or, Intra_directional modes) (e.g., Intra_Angular2 .. Intra_Angular66).

[0301] In one embodiment of the present disclosure, the planner mode may mean a mode for determining a prediction sample based on a weighted average value according to the distance of a left reference sample, an upper reference sample, a lower-left sample of the current block, and an upper-right sample. In one embodiment of the present disclosure, the DC mode may mean a mode for determining the average value of the reference samples as the prediction sample.

[0302] In one embodiment of the present disclosure, a plurality of intra prediction modes may include wide-angle modes (or wide-angle directional modes, wide-angle intra directional modes). Referring to FIG. 21, a plurality of intra prediction modes may include wide-angle modes -14 through -1 and 67 through 80, indicated by dashed lines. The wide-angle modes may be used for intra prediction of a non-square current block.

[0303] For example, a wide-angle mode may be used to identify a reference sample of a non-square current block. The image decoder (2000) may determine an intra-prediction mode based on the width and height of the current block.

[0304] In one embodiment of the present disclosure, in intra-directional modes, the locations of reference samples for generating predicted samples of samples within the current block can be identified by considering the direction indicated by the intra-directional modes. For example, in mode 34, reference samples located at a 45-degree upper-left direction relative to the samples within the current block can be identified.

[0305] Recently, among intra prediction techniques, methods that generate two or more intra prediction images and combine them to generate various prediction images are being discussed. For example, Decoder-side Intra Mode Derivation (DIMD) or Occurrence-Based Intra Coding (OBIC) techniques fall into this category.

[0306] In the present disclosure, a method of combining a plurality of prediction blocks to generate a final prediction block is referred to as an intra-blending mode. As an example, the intra-blending mode may include DIMD or OBIC. However, the content of the present disclosure is not limited thereto. For example, the intra-blending mode may include Template-based Intra Mode Derivation (TIMD). TIMD represents a method of deriving an intra-prediction mode based on the difference between a predicted value and a reconstructed value for a template.

[0307] In the present disclosure, the intra-blending mode is not limited to its name and may be referred to as a blending mode, blending method, weighted sum mode, weighted sum method, combination mode, combination method, fusion mode, fusion method, intra-blending method, intra-weighted sum mode, intra-weighted sum method, intra-combined mode, intra-combined method, intra-fusion mode, intra-fusion method, etc.

[0308] According to one embodiment, when an intra-blending mode is applied, one or more intra-prediction modes may be derived based on a sample (or encoded / decoded sample) restored prior to the current block or previously used mode information. One intra-prediction mode may be derived, or two or more intra-prediction modes may be derived. When two or more intra-prediction modes are derived, block-unit or sample-unit weights may be determined for the prediction blocks generated by each prediction mode, and a weighted sum may be performed based on the determined weights.

[0309] In addition, in one embodiment, when an intra-blending mode is applied, a prediction block may be generated by a predetermined prediction mode in addition to the prediction block generated by the induced intra-prediction mode. A prediction block generated according to the predetermined prediction mode may be weighted to the intra-block generated by the induced intra-prediction mode. For each prediction block, a block-unit or sample-unit weight may be determined, and a weighted sum may be performed based on the determined weight. In one embodiment of the present disclosure, regarding the predetermined prediction mode, various prediction modes (or candidate prediction modes) may be defined, and among these various prediction modes, the prediction mode applied to the current block may be specified based on template matching. This will be described in detail later with reference to FIGS. 23 through 29.

[0310] In one embodiment, the image decoding device (2000) can generate a plurality of prediction blocks and generate a final predicted image (or prediction block) by performing a weighted sum on a block-by-block or sample-by-sample basis based on determined weights. In the present disclosure, the weighted sum may be expressed using terms such as blending, combining, or fusion.

[0311] Below, DIMD will be described in detail with reference to FIG. 22 as an example of an intra-blending mode.

[0312] FIG. 22 is a diagram illustrating a method for inducing an intra-prediction mode based on the amount of change in surrounding sample values ​​according to one embodiment.

[0313] Referring to FIG. 22, DIMD represents a method for deriving an intra-prediction mode based on the amount of change in sample values ​​within a restored region (or decoded region) around the current block (2200). In the present disclosure, the intra-prediction mode derived based on DIMD may be referred to as the DIMD mode. Additionally, in the present disclosure, the amount of change may be referred to as the gradient.

[0314] Referring to FIG. 22, the image decoder (2000) can induce an intra prediction mode (i.e., DIMD mode) of the current block (2200) based on the amount of change (i.e., gradient) between at least two samples belonging to the template (2210) of the current block (2200) within the restored area around the current block (2200).

[0315] In one embodiment, the image decoder (2000) can calculate a gradient using at least two samples belonging to the template (2210) of the current block (2200). The image decoder (2000) can generate gradient information by accumulating the calculated gradients. As an example, the intra prediction mode of the current block (2200) can be selected from the intra prediction modes described above in FIG. 21.

[0316] DIMD according to the present disclosure is a technique for directly inducing an intra-prediction mode in an image decoding device (2000) based on gradients. When the DIMD mode is applied to the current block, the image decoding device (2000) can induce a directional mode from the template (2210) of the current block (2200). The directional mode can be induced using gradient information constructed by collecting the gradients of the template (2210) of the current block (2200).

[0317] In one embodiment, the gradient may represent a variable calculated using at least two samples belonging to the template (2210) of the current block (2200). For example, the gradient may include at least one of a gradient in the horizontal direction or a gradient in the vertical direction. In the present disclosure, the gradient may collectively refer to the calculated gradient and gradient information obtained by accumulating the calculated gradient.

[0318] Gradient information may include information about an intra-prediction mode mapped to the calculated gradient and / or information about the amplitude (or intensity) of said intra-prediction mode. In the present disclosure, gradient information may be referred to as gradient intensity, gradient magnitude, gradient amplitude, histogram of gradient (HoG), histogram, histogram intensity, histogram magnitude, histogram amplitude, etc.

[0319] In one embodiment, the gradient between at least two samples belonging to the template (2210) of the current block (2200) can be obtained using a predefined filter. The gradient information of the current block (2210) can be generated based on the obtained gradient.

[0320] The template (2210) of the current block (2200) may include the left, upper-left, and upper regions of the current block (2200). As an example, the template (2210) may be defined as an L-shaped region of three pixel lines adjacent to the current block (2200), as shown in FIG. 22. A gradient may be calculated using pixels belonging to the template (2210) among the surrounding regions of the current block (2200).

[0321] In one embodiment, a gradient may be calculated through filtering of pixels belonging to a template (2210). In one embodiment, filtering may be performed on pixels within a 3x3 pixel area (2220) belonging to a template (2210). In the present disclosure, the pixel area (2220) to which filtering is applied may be referred to as a window, an operator, or an edge operator.

[0322] In one embodiment, a 3x3 Sobel filter may be applied to the window (2220). In this disclosure, the case where the window (2220) is 3x3 is described primarily, but this is merely an example and is not limited thereto. For example, the window (2220) may be defined with a size of 2x2, 4x4, 5x5, etc. The number of pixel lines that can be used as a template (2210) may be defined in various ways. For example, when the window (2220) is 2x2, the template (2210) may be defined as an L-shaped area of ​​2 pixel lines adjacent to the current block (2200). Depending on the size of the current block (2200), it may be determined as one of the predefined sizes of the window (2220). For example, the predefined sizes may include 2x2 and 3x3.

[0323] In one embodiment, a window (2220) may be determined centered on a pixel belonging to the central pixel line within a template (2210) of three pixel lines, and a gradient may be calculated by applying filtering to the window (2220).

[0324] The gradient may include at least one of a horizontal gradient or a vertical gradient. An intra-prediction mode may be determined based on the calculated gradient. An angle (or prediction direction) may be calculated based on the gradient, and the calculated angle may be mapped to the intra-prediction mode having the closest or most similar angle. In other words, an intra-prediction mode may be calculated from the gradient. Gradient information may be constructed using the intra-prediction mode calculated from the gradient.

[0325] In one embodiment, the image decoding device (2000) may use a Sobel filter to obtain a gradient. The Sobel filter may include at least one of a horizontal Sobel filter or a vertical Sobel filter. The Sobel filter may be applied to a 3x3 window (2220) centered on a pixel belonging to the central pixel line of the template (2210).

[0326] The angle (or prediction direction) calculated through filtering the window (2220) can be mapped (or transformed) to one of the predefined intra prediction modes. The angle derived from the gradient can be the texture angle or prediction direction of the window (2220). As an example, an intra prediction mode as described above in FIG. 21 can be derived from the gradient. That is, the angle calculated through filtering the window (2220) can be mapped to one of 65 directional intra prediction modes.

[0327] In one embodiment, gradient information may be generated (or updated) based on the accumulated intra-prediction mode and the intensity of the corresponding intra-prediction mode. That is, gradient information may be generated by accumulating the intra-prediction mode calculated from the gradient and the amplitude (or intensity) corresponding to the corresponding intra-prediction mode. Gradient information may be generated by summing the amplitudes for each intra-prediction mode collected from the template (2210) based on the intra-prediction mode.

[0328] In one embodiment, the gradient information may include at least one intra prediction mode and an amplitude corresponding to the at least one intra prediction mode.

[0329] In one embodiment, intra prediction modes are induced at all pixel locations (pixel locations belonging to the central pixel line) within the template (2210), and by summing the values ​​of the intensity of the induced intra prediction modes, the amplitudes for each intra prediction mode of the current block (2200) can be obtained.

[0330] In one embodiment, the image decoding device (2000) can induce a DIMD mode based on generated gradient information. As an example, the image decoding device (2000) can induce a plurality of DIMD modes based on gradient information. The maximum number of induced intra prediction modes can be predefined. For example, the maximum number can be defined as 2, 3, 4, 5, 6, 7, etc. For example, among the intra prediction modes included in the generated gradient information, a predetermined number of intra prediction modes can be induced as DIMD modes of the current block in order of largest amplitude. Also, as an example, the maximum number of induced intra prediction modes can be determined according to the size of the current block (2200).

[0331] The image decoding device (2000) can generate a prediction block of the current block by performing intra prediction on the current block using DIMD mode. A reference sample of the current block can be used for intra prediction.

[0332] The image decoding device (2000) can generate a predictor using a DIMD mode. In the present disclosure, the predictor generated by the image decoding device (2000) using a DIMD mode may be referred to as a prediction sample, a prediction block, a provisional prediction sample, a provisional prediction block, an initial prediction sample, or an initial prediction block.

[0333] The image decoding device (2000) can generate a final prediction block by weighting the predictors generated by the DIMD mode. For example, the image decoding device (2000) can generate a final prediction block by weighting the predictors generated by the DIMD mode and the predictors generated by a predetermined prediction mode. As described above, regarding the predetermined prediction mode, various prediction modes (or candidate prediction modes) may be defined, and among the various candidate prediction modes, a prediction mode applied to the current block may be specified based on template matching. In this regard, details will be described later with reference to FIGS. 23 to 29.

[0334] The image decoding device (2000) can restore the current block using a prediction block. As an example, the image decoding device (2000) can determine the prediction block as the restored current block. Alternatively, as an example, the image decoding device (2000) can generate the restored current block by combining the prediction block with residual data obtained from the bitstream.

[0335] Below, OBIC will be described in detail as another example of an intra-blending mode.

[0336] According to one embodiment of the present disclosure, when OBIC is applied, the image decoding device (2000) can induce an intra prediction mode of the current block based on the frequency of occurrence of an intra prediction mode of a region restored (or decoded region) prior to the current block. In the present disclosure, the intra prediction mode induced based on OBIC may be referred to as an OBIC mode.

[0337] In one embodiment, the image decoding device (2000) can induce the intra prediction mode of the current block by accumulating the frequency of occurrence of the intra prediction mode in a sample unit or block unit within a predetermined area within the area restored prior to the current block.

[0338] Alternatively, the image decoding device (2000) may induce an intra prediction mode of the current block by accumulating the frequency of occurrence of a sample-unit or block-unit intra prediction mode from a block at a predetermined location within a region restored prior to the current block. The block at the predetermined location may be referred to as a spatial neighbor block of the current block. A spatial neighbor block of the current block may include a spatial neighbor block spatially adjacent to the current block and / or a non-adjacent spatial neighbor block not spatially adjacent to the current block.

[0339] In one embodiment, the image decoding device (2000) can identify the intra prediction modes of adjacent and non-adjacent spatial neighbor blocks and collect the intra prediction modes of the blocks to generate an occurrence histogram. That is, instead of the gradient histogram (HoG) used in DIMD, in OBIC, an occurrence histogram (HoO, Histogram of Occurrences) composed of the intra prediction mode and the number of occurrences of the mode in a sample unit (or block unit) can be used to induce the intra prediction mode.

[0340] In one embodiment, the occurrence histogram may include at least one intra prediction mode and the number of occurrences corresponding to the at least one intra prediction mode. In the present disclosure, the occurrence histogram may be referred to as occurrence count information, occurrence information, intra mode occurrence information, intra mode occurrence histogram, etc.

[0341] In one embodiment, the image decoding device (2000) can induce an OBIC mode based on an occurrence histogram. The image decoding device (2000) can induce a predetermined number of intra prediction modes with the highest occurrence frequency into an OBIC mode. As an example, the image decoding device (2000) can induce a plurality of OBIC modes based on an occurrence histogram. The maximum number of intra prediction modes to be induced can be predefined. For example, the maximum number can be defined as 2, 3, 4, 5, 6, 7, etc. For example, among the intra prediction modes included in the occurrence histogram, a predetermined number of intra prediction modes can be induced into the OBIC mode of the current block in order of the highest occurrence frequency. Also, as an example, the maximum number of intra prediction modes to be induced can be determined according to the size of the current block.

[0342] The image decoder (2000) can generate a prediction block of the current block by performing intra prediction on the current block using OBIC mode. A reference sample of the current block can be used for intra prediction.

[0343] The image decoding device (2000) can generate a predictor using an OBIC mode. In the present disclosure, the predictor generated by the image decoding device (2000) using an OBIC mode may be referred to as a prediction sample, a prediction block, a provisional prediction sample, a provisional prediction block, an initial prediction sample, or an initial prediction block.

[0344] The image decoding device (2000) can generate a final prediction block by weighting the predictors generated by the OBIC mode. For example, the image decoding device (2000) can generate a final prediction block by weighting the predictors generated by the OBIC mode and the predictors generated by a predetermined prediction mode. As described above, regarding the predetermined prediction mode, various prediction modes (or candidate prediction modes) may be defined, and among the various candidate prediction modes, a prediction mode applied to the current block may be specified based on template matching. In this regard, details will be described later with reference to FIGS. 23 to 29.

[0345] The image decoding device (2000) can restore the current block using a prediction block. As an example, the image decoding device (2000) can determine the prediction block as the restored current block. Alternatively, as an example, the image decoding device (2000) can generate the restored current block by combining the prediction block with residual data obtained from the bitstream.

[0346] Hereinafter, the present disclosure proposes a method for improving the performance of intra-blending prediction techniques such as DIMD or OBIC described above.

[0347] FIG. 23 is a diagram illustrating an intra-prediction mode-based image decoding method according to one embodiment.

[0348] Referring to FIG. 23, the image decoding device (2000) can determine whether an intra-blending mode is applied to the current block (S2310). In other words, the image decoding device (2000) can determine whether to use an intra-blending method. As an example, flag information indicating whether to apply an intra-blending mode can be signaled through a bitstream.

[0349] An intra-blending mode represents a method for combining a plurality of prediction blocks to generate a final prediction block, and in one embodiment of the present disclosure, the intra-blending mode may be the DIMD described above in FIG. 22. Alternatively, the intra-blending mode may be the OBIC described above in FIG. 22. Alternatively, the intra-blending mode may include the DIMD and OBIC described above in FIG. 22.

[0350] When an intra-blending mode is not applied, the image decoder (2000) can perform a single intra-mode prediction (S2320, S2330). In the present disclosure, a single intra-mode prediction is a mode that does not perform blending and may represent a mode that performs prediction using a single intra-prediction mode.

[0351] Alternatively, in one embodiment, when an intra-blending mode is not applied, the image decoder (2000) may perform intra prediction using various intra modes in addition to the intra-blending mode according to one embodiment of the present disclosure. As an example, when an intra-blending mode is not applied, the image decoder (2000) may perform prediction using a mode that performs blending in a manner different from the intra-blending mode according to one embodiment of the present disclosure. In this case, the single intra prediction mode described above may always be included (or considered).

[0352] When an intra-blending mode is applied, the image decoding device (2000) can induce an intra-prediction mode using data that was decoded prior to the current block (e.g., restored sample information, decoded sample information, or prediction mode information, etc.) (S2320, S2340).

[0353] In other words, when an intra-blending mode is applied, the image decoder (2000) can first derive an intra-prediction mode of the current block based on a region decoded prior to the current block. As an example, a predetermined number of DIMD modes and / or OBIC modes may be derived. The image decoder (2000) can generate a prediction block (or prediction sample, predictor) by performing intra-prediction using the derived intra-prediction mode. Here, the embodiment described above in FIG. 22 may be applied, and redundant descriptions related thereto are omitted.

[0354] The image decoding device (2000) can select a prediction mode based on a template cost (or template matching cost) (S2350). As described above, a weighted sum can be performed using a prediction block generated by a predetermined prediction mode in an intra-blending mode. At this time, the predetermined prediction mode can be selected by the image decoding device (2000) based on template matching among several prediction modes.

[0355] In one embodiment, a plurality of candidate prediction modes may be defined. In the present disclosure, the plurality of candidate prediction modes may be referred to as a candidate prediction mode group. The image decoder (2000) calculates a template cost for each of the plurality of candidate prediction modes, and based on the calculated template cost, a candidate prediction mode applied to the current block among the plurality of candidate prediction modes may be determined as a predetermined prediction mode. The image decoder (2000) may select a candidate prediction mode that minimizes the template cost among the plurality of candidate prediction modes.

[0356] In one embodiment, the candidate prediction mode may include a non-directional prediction mode. The non-directional prediction mode may represent a mode that performs intra prediction according to a non-directional intra prediction mode. The non-directional intra prediction mode may include a Planar mode and a DC mode, as previously described in FIG. 21.

[0357] In one embodiment, the candidate prediction mode may include a planner prediction mode. The planner prediction mode may represent a mode that performs intra prediction according to the planner mode. If the candidate prediction mode includes only the planner prediction mode, the step S2350 of performing template matching may be omitted, and the image decoder (2000) may combine the prediction block generated according to the planner mode with the prediction block generated according to the intra prediction mode derived in step S2340 to generate a final prediction block.

[0358] In one embodiment, a candidate prediction mode (or group of candidate prediction modes) may include a planar prediction mode and a block vector (BV) prediction mode. A BV prediction mode represents a mode that determines a reference block specified by a BV as a prediction block while using a restored region of the current picture as a reference region. A reference block specified by a BV may be represented as a block at a location corresponding to the current block within the current picture. In the present disclosure, a BV prediction mode may be referred to as a BV mode, BV prediction, or BV-based prediction mode. As an example, a planar prediction mode may be replaced by a DC prediction mode or a directional prediction mode. A directional prediction mode may represent a mode that performs intra prediction according to a directional intra prediction mode.

[0359] In one embodiment, the BV can be acquired in various modes. For example, the BV can be acquired through an IBC mode or an IntraTMP (Intra Template Matching based Prediction) mode. In the present disclosure, the BV prediction mode is defined as a candidate prediction mode as a mode for performing prediction using the acquired BV, but the IBC mode or IntraTMP mode, which is a mode for acquiring the BV, may also be defined as a candidate prediction mode. In this case as well, the image decoding device (2000) can generate a prediction block using the BV acquired by the IBC mode or IntraTMP mode, and combine the generated prediction block with the prediction block generated in step S2340.

[0360] According to one embodiment of the present disclosure, a candidate prediction mode may include a planner prediction mode, a BV prediction mode, and a motion vector (MV) prediction mode. The MV prediction mode represents a mode that determines a reference block specified by the MV within a reference picture as a prediction block. The reference block specified by the MV may be represented as a block at a position corresponding to the current block within the reference picture. In the present disclosure, the MV prediction mode may be referred to as an MV mode, MV prediction, MV-based prediction mode, inter mode, or inter prediction mode. This is described in detail later in FIGS. 27 through 29. Alternatively, as an example, the BV prediction mode may be replaced with the MV prediction mode.

[0361] In one embodiment, the candidate prediction mode may include a mode that performs prediction according to a general intra prediction mode (e.g., DC, directional intra mode) in addition to the planner prediction mode. In this case, the general intra prediction mode may be the intra prediction mode described above in FIG. 21. The DC mode or a specific directional intra mode may be fixed as the candidate prediction mode.

[0362] In one embodiment, the candidate prediction mode may include a general intra prediction mode (e.g., planar, DC, directional intra mode) and a BV prediction mode.

[0363] In one embodiment, the candidate prediction mode may include a plurality of BV prediction modes. In this case, the image decoder (2000) may select a BV prediction mode that minimizes the template cost among the plurality of BV prediction modes. Alternatively, one BV prediction mode may include two BV candidates. In this case, the image decoder (2000) may select either the first BV or the second BV to use as a candidate prediction mode. Alternatively, the templates corresponding to the two BVs may be combined and considered as a single prediction.

[0364] The image decoder (2000) can calculate weight values ​​for each of the prediction modes (e.g., planner prediction mode or BV prediction mode) selected based on the intra modes derived in step S2340 and the template cost (S2360), and perform blending based on the calculated weights to generate a final prediction block (prediction image) (S2370).

[0365] In one embodiment, the weight value applied to the prediction mode selected based on the template cost may be predefined. Alternatively, the weight value applied to the prediction mode selected based on the template cost may be determined based on information signaled from the image encoding device. Alternatively, the weight value applied to the prediction mode selected based on the template cost may be determined based on predefined encoding information.

[0366] Below, a method for calculating template costs will be explained with reference to FIGS. 24 to 26.

[0367] FIG. 24 is a drawing illustrating a template area of ​​a current block according to one embodiment.

[0368] Referring to FIG. 24, the template area of ​​the current block (or current encoding unit) (2400) may be defined as a portion of the area (2410, 2420) adjacent to the left and upper sides of the current block. In one embodiment, when the size of the current block is W×H, the left template area (2410) may have a size of N×H, and the upper template area (2420) may have a size of W×M. Here, the values ​​of M and N may vary depending on various contexts. As an example, the values ​​of M and N may be determined based on the width or height of the current block.

[0369] The image decoder (2000) can generate prediction samples for each candidate prediction mode based on the template regions (2410, 2420) of the current block (2400) and calculate a matching cost with the restoration samples of the corresponding template regions (2410, 2420). The image decoder (2000) can determine the candidate prediction mode having the best cost as the final prediction mode.

[0370] FIG. 25 is a diagram illustrating an intra-prediction mode-based prediction method for a template area according to one embodiment.

[0371] According to one embodiment of the present disclosure, the template cost for an intra-prediction mode-based candidate prediction mode can be calculated based on the difference between the predicted value and the restored value for the template of the current block (2500).

[0372] Referring to FIG. 25, prediction for a template region can be performed using general intra prediction modes, including a planner mode. Specifically, the area surrounding the template region is set as a reference sample, and a prediction sample for the template region can be obtained from the reference sample according to the intra prediction mode.

[0373] In one embodiment, the reference sample may be defined in various forms as illustrated in FIG. 25. As illustrated in FIG. 25A, a single reference sample area (or reference sample line) (2510) used for both the left template and the upper template may be configured. Alternatively, as illustrated in FIG. 25B and FIG. 25C, reference sample areas (2520, 2530) adjacent to the left template and the upper template, respectively, may be configured individually.

[0374] The image decoding device (2000) can calculate the distortion between the predicted value and the restored value of the template region and use this as a cost. At this time, at least one of SAD (sum of absolute difference), SSD (sum of squared difference), SATD (sum of Absolute Transformed Difference), SSE (sum of squared error), or MR-SAD (mean removed SAD) may be used as a cost function for calculating the cost. The candidate prediction mode (or a predetermined number of candidate prediction modes) with the smallest cost value calculated using the cost function may be determined as the prediction mode for weighted prediction (i.e., blending).

[0375] In one embodiment, the image decoder (2000) can determine the error between the predicted value and the restored value of a template using at least one cost function among SAD, SATD, SSE, and MR-SAD. When the cost function is SAD, the image decoder (2000) can determine the error based on the sum of the absolute values ​​of the differences between the predicted sample and the restored sample of the template of the current block (2500). When the cost function is SSD, the image decoder (2000) can determine the error based on the sum of the squares of the differences between the predicted sample and the restored sample of the template of the current block (2500). The cost function may include a function representing the number of identical samples.

[0376] FIG. 26 is a diagram illustrating a block vector-based prediction method for a template region according to one embodiment.

[0377] According to one embodiment of the present disclosure, the template cost for a block vector-based candidate prediction mode can be calculated based on the difference between the predicted value and the restored value for the template of the current block (2600).

[0378] Referring to FIG. 26, within the restoration area of ​​the current picture, the reference block (2620) (or corresponding block, which may be referred to as a candidate block) of the current block (2600) can be identified by the BV (2610). The restoration samples of the left and upper template areas of the reference block (2620) corresponding to the left and upper template areas of the current block (2600) can be determined (or acquired) as prediction samples for the template of the current block (2600) in a block vector-based prediction mode.

[0379] In other words, the image decoding device (2000) can set the area corresponding to the template area of ​​the current block (2600) specified through the BV (2610) (i.e., the template area of ​​the reference block (2620)) as the predicted value for the template area of ​​the current block (2600).

[0380] The image decoding device (2000) can calculate the distortion between the predicted value and the restored value of the template region and use this as a cost. At this time, at least one of SAD (sum of absolute difference), SSD (sum of squared difference), SATD (sum of Absolute Transformed Difference), SSE (sum of squared error), or MR-SAD (mean removed SAD) may be used as a cost function for calculating the cost. The candidate prediction mode (or a predetermined number of candidate prediction modes) with the smallest cost value calculated using the cost function may be determined as the prediction mode for weighted prediction (i.e., blending).

[0381] In one embodiment, the image decoder (2000) can determine the error between templates using at least one cost function among SAD, SATD, SSE, and MR-SAD. When the cost function is SAD, the image decoder (2000) can determine the error based on the sum of the absolute values ​​of the differences between each sample of the template in the current block (2600) and each sample of the template in the reference block (2620). When the cost function is SSD, the image decoder (2000) can determine the error based on the sum of the squares of the differences between each sample of the template in the current block (2600) and each sample of the template in the reference block (2620). The cost function may include a function representing the number of identical samples.

[0382] In one embodiment, BV (2610) may have fractional sample precision. Alternatively, BV (2610) may be rounded to integer sample precision to account for complexity. As an example, if BV (2610) has fractional precision, a bi-linear interpolation method may be applied to reduce complexity.

[0383] In one embodiment, the BV (2610) may be a BV (2610) obtained from a block encoded / decoded before the current block (2600), or a newly determined BV (2610) for the current block (2600). The image decoding device (2000) may calculate a prediction cost for each using a plurality of BVs, and then compare the matching costs between the BVs to select an optimal prediction mode.

[0384] FIG. 27 is a diagram illustrating an intra-prediction mode-based image decoding method according to one embodiment.

[0385] According to one embodiment of the present disclosure, the candidate prediction mode group considered through the template cost in an intra-blending mode may be determined differently depending on the slice type (or whether it is an intra-picture). In other words, the image decoder (2000) determines a candidate prediction mode according to the slice type of the slice to which the current block belongs, and among the determined candidate prediction modes, determines a prediction mode applied to the current block using the template cost.

[0386] In FIG. 27, for convenience of explanation, steps S2310, S2320, and S2330 described in FIG. 23 are omitted, and the description focuses on the case where an intra-blending mode is applied to the current block, but is not limited thereto. The embodiment described in FIG. 23, including steps S2310, S2320, and S2330 described in FIG. 23, may be applied in the same way. Here, redundant descriptions are omitted.

[0387] Referring to FIG. 27, the image decoding device (2000) can induce an intra-prediction mode using data that was decoded prior to the current block (e.g., restored sample information, decoded sample information, or prediction mode information, etc.) (S2710).

[0388] When an intra-blending mode is applied, the image decoder (2000) can derive an intra-prediction mode of the current block based on a region decoded prior to the current block. As an example, a predetermined number of DIMD modes and / or OBIC modes may be derived. The image decoder (2000) can generate a prediction block (or prediction sample, predictor) by performing intra-prediction using the derived intra-prediction mode. Here, the embodiments described in FIGS. 22 and 23 above may be applied in the same way, and redundant descriptions related thereto are omitted.

[0389] Prior to step S2710, the image decoder (2000) can determine whether an intra-blending mode is applied to the current block, and if an intra-blending mode is not applied, the image decoder (2000) can perform a single intra-mode prediction.

[0390] The video decoding device (2000) can determine whether the current slice to which the current block belongs is an intra-slice (I-slice, Intra-slice) (S2720). An intra-blending mode according to one embodiment may operate differently in an intra-slice (or intra-picture) and an inter-slice (or non-intra-picture). An inter-slice may include a P-slice and / or a B-slice.

[0391] If the current slice is an intra-slice, the image decoder (2000) can select a prediction mode based on a template cost within a group of candidate prediction modes for the intra-slice (S2730). The image decoder (2000) can select which prediction to use between the planar prediction (or DC prediction, other directional mode prediction) and BV prediction described in FIG. 22 above and the prediction cost (or matching cost) of the template area.

[0392] In one embodiment, a candidate prediction mode group for an intra-slice may include an intra-mode based prediction mode and a BV prediction mode. An intra-mode based prediction mode represents a mode that performs intra-prediction according to a planner mode, a DC mode, or a directional intra-prediction mode. For example, a candidate prediction mode group for an intra-slice may include a non-directional prediction mode and a BV prediction mode. A candidate prediction mode group for an intra-slice may include a planner prediction mode and a BV prediction mode. As an example, the planner prediction mode may be replaced with a DC prediction mode or a directional prediction mode.

[0393] In one embodiment, the BV can be acquired in various modes. For example, the BV can be acquired through an IBC mode or an IntraTMP mode. In the present disclosure, the BV prediction mode is defined as a candidate prediction mode as a mode for performing prediction using the acquired BV, but the IBC mode or IntraTMP mode, which is a mode for acquiring the BV, may also be defined as a candidate prediction mode. In this case as well, the image decoding device (2000) can generate a prediction block using the BV acquired by the IBC mode or IntraTMP mode, and combine the generated prediction block with the prediction block generated in step S2710.

[0394] In one embodiment, the candidate prediction mode may include a plurality of BV prediction modes. In this case, the image decoder (2000) may select a BV prediction mode that minimizes the template cost among the plurality of BV prediction modes. Alternatively, one BV prediction mode may include two BV candidates. In this case, the image decoder (2000) may select either the first BV or the second BV to use as a candidate prediction mode. Alternatively, the templates corresponding to the two BVs may be combined and considered as a single prediction.

[0395] In one embodiment, when there are multiple BVs, the image decoding device (2000) may perform a template prediction-based cost calculation for a predetermined number of BV candidates. Alternatively, information explicitly indicating the BV applied to the current block may be signaled. When the BV is specified through signaling, there is an advantage in that the complexity of the template cost calculation can be reduced.

[0396] The image decoding device (2000) can select a prediction mode based on a template cost (or template matching cost) within the aforementioned group of candidate prediction modes. The image decoding device (2000) can select a candidate prediction mode that minimizes the template cost among a plurality of candidate prediction modes belonging to the group of candidate prediction modes.

[0397] If the current slice is not an intra-slice, the image decoder (2000) can select a prediction mode based on a template cost within a group of candidate prediction modes for non-intra-slices (S2740). If the current slice is not an intra-slice, the group of candidate prediction modes may additionally include an MV prediction mode.

[0398] In other words, an MV prediction mode may be included in a candidate prediction mode group only if it satisfies a predefined condition. In this case, the predefined condition may include the case where the current slice to which the current block belongs is not an intra-slice.

[0399] In one embodiment, a candidate prediction mode group for a non-intra slice may include an intra mode-based prediction mode, a BV prediction mode, and an MV prediction mode. An intra mode-based prediction mode represents a mode that performs intra prediction according to a planar mode, a DC mode, or a directional intra prediction mode. That is, a candidate prediction mode group for a non-intra slice may include an intra mode-based prediction mode and an MV prediction mode.

[0400] For example, a group of candidate prediction modes for a non-intra slice may include a non-directional prediction mode, a BV prediction mode, and an MV prediction mode. A group of candidate prediction modes for a non-intra slice may include a planar prediction mode, a BV prediction mode, and an MV prediction mode. As an example, the planar prediction mode may be replaced with a DC prediction mode or a directional prediction mode. Or, as an example, the BV prediction mode may be replaced with an MV prediction mode.

[0401] In one embodiment, the MV candidate of the MV prediction mode can be determined in various ways. For example, the MV can be determined according to the merge mode generally used in inter-prediction. In this case, index information regarding which candidate to select among a plurality of merge candidates may be additionally signaled. Alternatively, the image decoder (2000) can calculate a template matching-based prediction cost for a predetermined number of candidates among a plurality of merge candidates. The image decoder (2000) can select a candidate having the optimal cost among a plurality of merge candidates.

[0402] In one embodiment, the maximum number of MV candidates in the MV prediction mode may be set to be equal to or smaller than that of the general merge mode. The maximum number of MV candidates in the MV prediction mode may be signaled through a sequence parameter set, a picture parameter set, a picture header, a slice header, etc. Alternatively, the maximum number of MV candidates in the MV prediction mode may be predefined. As another example, the first candidate among the merge candidates (or merge candidate list) in the merge mode may be used as the MV in the MV prediction mode without separate signaling.

[0403] Additionally, in one embodiment, the image decoding device (2000) may set the motion vector of a neighboring block spatially adjacent to the current block as the MV of the MV prediction mode. The MV of the MV prediction mode may be set as the MV of a neighboring block at a predefined location.

[0404] For example, the image decoding device (2000) may use the MV of the left neighbor block of the current block as the MV of the MV prediction mode. If there is no MV at that location, the MV may be set to (0, 0). As another example, the image decoding device (2000) may calculate the respective template cost for two MVs existing in the left neighbor block and the upper neighbor block of the current block, and select the MV with the optimal cost among them as the MV of the MV prediction mode. Alternatively, the image decoding device (2000) may always fix the MV of the MV prediction mode to (0, 0).

[0405] In one embodiment, the image decoding device (2000) may obtain only MV information from the surrounding blocks of the fixed location described above. Alternatively, the image decoding device (2000) may inherit and use information such as whether illumination compensation is performed, and weight information for each direction in the case of bidirectional prediction, from the surrounding blocks of the fixed location described above.

[0406] The image decoder (2000) can calculate weight values ​​for each of the prediction modes (e.g., planer prediction mode, BV prediction mode, or MV prediction mode) selected based on the intra modes derived in step S2710 and the template cost (S2750), and can generate a final prediction image by performing blending based on the calculated weights (S2760).

[0407] In one embodiment, the weight value applied to the prediction mode selected based on the template cost may be predefined. Alternatively, the weight value applied to the prediction mode selected based on the template cost may be determined based on information signaled from the image encoding device. Alternatively, the weight value applied to the prediction mode selected based on the template cost may be determined based on predefined encoding information. Alternatively, the combination of prediction blocks may be performed through averaging.

[0408] FIG. 28 is a diagram illustrating a motion vector-based prediction method for a template area according to one embodiment.

[0409] According to one embodiment of the present disclosure, the template cost for a motion vector-based candidate prediction mode can be calculated based on the difference between the predicted value and the restored value for the template of the current block (2800). FIG. 28 assumes that there is only one motion vector (MV (2810)) used in the motion vector-based candidate prediction mode.

[0410] Referring to FIG. 28, the reference block (2820) (or corresponding block, which may be referred to as a candidate block) of the current block (2800) can be identified by the MV (2810) within the reference picture. The restoration samples of the left and upper template areas of the reference block (2820) corresponding to the left and upper template areas of the current block (2800) can be determined (or acquired) as prediction samples for the template of the current block (2800) in a motion vector-based prediction mode.

[0411] In other words, the video decoding device (2000) can set the area corresponding to the template area of ​​the current block (2800) specified through the MV (2810) (i.e., the template area of ​​the reference block (2820)) as the predicted value for the template area of ​​the current block (2800).

[0412] The image decoding device (2000) calculates the distortion between the predicted value and the restored value of the template region and can use this as a cost. At this time, at least one of SAD, SSD, SATD, SSE, or MR-SAD may be used as a cost function for calculating the cost. The candidate prediction mode (or a predetermined number of candidate prediction modes) with the smallest cost value calculated using the cost function may be determined as the prediction mode for weighted prediction (i.e., blending).

[0413] In one embodiment, the image decoder (2000) can determine the error between templates using at least one cost function among SAD, SATD, SSE, and MR-SAD. When the cost function is SAD, the image decoder (2000) can determine the error based on the sum of the absolute values ​​of the differences between each sample of the template in the current block (2800) and each sample of the template in the reference block (2820). When the cost function is SSD, the image decoder (2000) can determine the error based on the sum of the squares of the differences between each sample of the template in the current block (2800) and each sample of the template in the reference block (2820). The cost function may include a function representing the number of identical samples.

[0414] In one embodiment, MV (2810) may have fractional sample precision. Alternatively, MV (2810) may be rounded to integer sample precision for complexity considerations. As an example, if MV (2810) has fractional precision, a bi-linear motion compensated interpolation method may be applied to reduce complexity.

[0415] FIG. 29 is a diagram illustrating a motion vector-based prediction method for a template area according to one embodiment.

[0416] According to one embodiment of the present disclosure, the template cost for a motion vector-based candidate prediction mode can be calculated based on the difference between the predicted value and the restored value for the template of the current block (2900). FIG. 29 assumes that there are two motion vectors (MV0 (2910), MV1 (2930)) used in the motion vector-based candidate prediction mode.

[0417] Referring to FIG. 29, a first reference block (2920) of the current block (2900) can be identified by MV0 (2910) within reference picture 0, and a second reference block (2940) of the current block (2900) can be identified by MV1 (2930) within reference picture 1. The restoration samples of the left and upper template areas of the first reference block (2920) and the restoration samples of the left and upper template areas of the second reference block (2940), corresponding to the left and upper template areas of the current block (2900), can be used as prediction samples for the template of the current block (2900) in a motion vector-based prediction mode.

[0418] In one embodiment, the image decoding device (2000) can determine the predicted value of the template area in various ways. The image decoding device (2000) can determine the predicted value of the template area by combining the left and upper template areas of the first reference block (2920) and the left and upper template areas of the second reference block (2940), respectively. For example, the image decoding device (2000) can determine the predicted value by averaging the template areas of the first reference block (2920) and the second reference block (2940).

[0419] Alternatively, the image decoding device (2000) may select one of MV0 (2910) or MV1 (2930). Based on the selected MV, the predicted value of the template area may be determined according to the embodiment described in FIG. 28. Alternatively, the image decoding device (2000) may calculate the template cost for each of MV0 (2910) and MV1 (2930) and determine the average value of the two costs as the predicted value of the template area.

[0420] The image decoding device (2000) calculates the distortion between the predicted value and the restored value of the template region and can use this as a cost. At this time, at least one of SAD, SSD, SATD, SSE, or MR-SAD may be used as a cost function for calculating the cost. The candidate prediction mode (or a predetermined number of candidate prediction modes) with the smallest cost value calculated using the cost function may be determined as the prediction mode for weighted prediction (i.e., blending).

[0421] In one embodiment, MV0 (2910) and MV1 (2930) may have fractional sample precision. Alternatively, MV0 (2910) and MV1 (2930) may be rounded to integer sample precision to account for complexity. As an example, if MV0 (2910) and MV1 (2930) have fractional precision, a bi-linear motion compensated interpolation method may be applied to reduce complexity.

[0422] In one embodiment, whether to use the MV prediction mode as a candidate prediction mode in the intra-blending mode can be determined based on the size of the current block. Since there are more memory accesses for small blocks compared to relatively large blocks, the MV prediction mode may not be considered as a candidate prediction mode for small blocks in order to reduce complexity.

[0423] For example, if the size of the current block is smaller than a predefined size, the MV prediction mode may be excluded from the candidate prediction mode group. Or, if the area of ​​the current block (i.e., the product of the width and height of the current block) is smaller than a predetermined threshold K, the candidate prediction mode group may not include the MV prediction mode. As an example, the value of K may be set to 16, 32, or 64.

[0424] Alternatively, in one embodiment, whether to use an MV prediction mode as a candidate prediction mode in an intra-blending mode may be determined at the sequence, picture, or slice level. Information indicating whether to use an MV prediction mode as a candidate prediction mode in an intra-blending mode may be signaled through a sequence parameter set, a picture parameter set, a picture header, a slice header, etc.

[0425] Additionally, in one embodiment, whether to use the BV prediction mode as a candidate prediction mode in the intra-blending mode can be determined based on the size of the current block. Calculating the template cost for multiple BVs in a relatively small block can increase complexity. To reduce complexity, the BV prediction mode may not be considered as a candidate prediction mode for small blocks.

[0426] For example, if the size of the current block is smaller than a predefined size, the BV prediction mode may be excluded from the candidate prediction mode group. Or, if the area of ​​the current block (i.e., the product of the width and height of the current block) is smaller than a predetermined threshold S, the candidate prediction mode group may not include the BV prediction mode. As an example, the value of S may be set to 16, 32, or 64.

[0427] Alternatively, in one embodiment, whether to use a BV prediction mode as a candidate prediction mode in an intra-blending mode may be determined at the sequence, picture, or slice level. Information indicating whether to use a BV prediction mode as a candidate prediction mode in an intra-blending mode may be signaled through a sequence parameter set, a picture parameter set, a picture header, a slice header, etc.

[0428] FIG. 30 is a diagram illustrating an intra-prediction mode-based image decoding method according to one embodiment.

[0429] According to one embodiment of the present disclosure, a BV prediction mode may be allowed only when a predefined BV acquisition mode is applied to (or available for) the current block. That is, a BV prediction mode may be included in a candidate prediction mode group only when a predefined condition is satisfied. In this case, the predefined condition may include when a predefined BV acquisition mode is applied to (or available for) the current block. As an example, the predefined BV acquisition mode may include an IBC mode and / or an IntraTMP mode.

[0430] In one embodiment, the image decoding device (2000) can perform a template cost-based prediction mode selection process only when an IBC mode or an IntraTMP mode is applied.

[0431] In FIG. 30, for convenience of explanation, steps S2310, S2320, and S2330 described in FIG. 23 are omitted, and the description focuses on the case where an intra-blending mode is applied to the current block, but is not limited thereto. The embodiment described in FIG. 23, including steps S2310, S2320, and S2330 described in FIG. 23, may be applied in the same way. Here, redundant descriptions are omitted.

[0432] Referring to FIG. 30, the image decoding device (2000) can induce an intra-prediction mode using data that was decoded prior to the current block (e.g., restored sample information, decoded sample information, or prediction mode information, etc.) (S3010).

[0433] When an intra-blending mode is applied, the image decoder (2000) can derive an intra-prediction mode of the current block based on a region decoded prior to the current block. As an example, a predetermined number of DIMD modes and / or OBIC modes may be derived. The image decoder (2000) can generate a prediction block (or prediction sample, predictor) by performing intra-prediction using the derived intra-prediction mode. Here, the embodiments described in FIGS. 22 and 23 above may be applied in the same way, and redundant descriptions related thereto are omitted.

[0434] Prior to step S3010, the image decoder (2000) can determine whether an intra-blending mode is applied to the current block, and if an intra-blending mode is not applied, the image decoder (2000) can perform a single intra-mode prediction.

[0435] The video decoding device (2000) can check whether the IntraTMP flag value is 1 or the IBC flag value is 1 (S3020). The IntraTMP flag can indicate whether the IntraTMP mode is applied to the current block. The IBC flag can indicate whether the IBC mode is applied to the current block.

[0436] When the IntraTMP flag value is 1 or the IBC flag value is 1, the image decoder (2000) can select a prediction mode based on the template cost (or template matching cost) (S3030). That is, when the IBC mode or IntraTMP mode is applied to the current block, the image decoder (2000) can select a prediction mode based on the template cost.

[0437] As described above, a weighted sum can be performed using a prediction block generated by a predetermined prediction mode in an intra-blending mode. At this time, the predetermined prediction mode may be selected by the image decoding device (2000) based on template matching among several candidate prediction modes. If a BV prediction mode is available, the embodiment described in FIG. 23 or FIG. 27 above may be applied in the same way. Related redundant descriptions are omitted.

[0438] In one embodiment, if the IntraTMP flag value is not 1 and the IBC flag value is not 1, the image decoder (2000) may omit the template cost-based prediction mode selection process. That is, if neither the IBC mode nor the IntraTMP mode is applied to the current block, the image decoder (2000) may not perform the template cost-based prediction mode selection process. As an example, the image decoder (2000) may generate a prediction block based on a fixed intra prediction mode and combine it with the prediction block generated in step S3010. The fixed intra prediction mode may be a non-directional mode. The fixed intra prediction mode may be a planner mode or a DC mode.

[0439] Meanwhile, along with this embodiment which checks the condition regarding whether a predefined BV acquisition mode is applied to the current block, the candidate prediction mode determination process according to the slice type in FIG. 27 above may also be applied.

[0440] For example, if it is an intra-slice and the predefined BV acquisition mode is not applied to the current block, blending can be performed using a block predicted based on a fixed intra-prediction mode. If it is an inter-slice and the predefined BV acquisition mode is not applied to the current block, an MV prediction mode can be used as a candidate prediction mode along with an intra-mode-based prediction mode.

[0441] That is, in the case where it is a non-intra slice and neither the IBC mode nor the IntraTMP mode is applied to the current block, the image decoder (2000) can select the prediction mode of the current block that minimizes the template cost among the intra mode-based prediction mode and the MV prediction mode. In the case where it is a non-intra slice and neither the IBC mode nor the IntraTMP mode is applied to the current block, the BV prediction mode may be excluded from the group of candidate prediction modes for the non-intra slice.

[0442] Figure 31 is a flowchart illustrating the prediction mode determination process.

[0443] FIG. 31 illustrates a general prediction mode determination process in an intra slice. When the mode of the current block is an intra mode, it may be configured to determine whether to use an intra blending mode through signaling. The intra blending mode may include the DIMD and / or OBIC described earlier in FIG. 22.

[0444] Referring to FIG. 31, the image decoder (2000) can check whether a skip mode is applied to the current block (S3101). The image decoder (2000) can obtain skip flag information from the bitstream when an IBC is used. If the skip flag is 1, it can be determined to be an IBC skip mode (S3102).

[0445] When the skip flag is 0, the video decoder (2000) can perform an additional prediction mode determination process to determine whether the current block is in intra mode or another mode (S3103). FIG. 31 assumes a case where it is divided into intra mode and IBC mode.

[0446] The video decoder (2000) can check whether the current block is in intra mode (S3104). If the current block is in intra mode, the video decoder (2000) can determine whether to use (or apply) an intra blending mode (S3105).

[0447] The video decoder (2000) can perform a prediction according to the intra blending mode when the intra blending mode is applied (S3106, S3107), and otherwise acquire an intra prediction mode and perform a prediction according to the acquired intra prediction mode (S3106, S3108).

[0448] If the current block is not in intra mode, it may be determined to be in IBC mode, and the image decoder (2000) may acquire an IBC merge flag (S3109). The image decoder (2000) may selectively apply an IBC merge mode or an IBC mode (indicating a mode that directly signals the BV) according to the acquired IBC merge flag (S3110, S3111, S3112).

[0449] However, according to the embodiment of FIG. 31, when the BV prediction mode is used in the intra-blending mode, the operation of the IBC mode is performed within the intra-blending mode. That is, even though BV is obtained through the operation of the IBC mode as the BV prediction mode is used in the intra-blending mode, the structure determines whether to apply the intra-blending mode in the intra-blending mode rather than the IBC mode according to the branching of step S3104, so inefficiency may result in the determination of the prediction mode, and structural changes may be unavoidable when other prediction modes are added or deleted.

[0450] In order to improve these problems, the present disclosure proposes a method to modify the intra blending mode so that it operates in both the intra mode and the IBC mode.

[0451] FIG. 32 is a flowchart illustrating a prediction mode determination process according to one embodiment.

[0452] According to one embodiment of the present disclosure, signaling for an intra-blending mode may be performed in IBC mode, just as signaling for an intra-blending mode in intra mode may be performed. In the intra-blending mode in intra mode, BV may not be used. It may be configured to use BV only in the intra-blending mode in IBC mode.

[0453] Referring to FIG. 32, the image decoder (2000) can check whether the current block is in intra mode (S3201). If the current block is in intra mode, the image decoder (2000) can determine whether to use (or apply) an intra blending mode (S3202).

[0454] The video decoder (2000) can perform a prediction according to the intra blending mode when the intra blending mode is applied (S3203, S3204), and otherwise acquire an intra prediction mode and perform a prediction according to the acquired intra prediction mode (S3203, S3205). At this time, the BV prediction mode may not be used in the intra blending mode branched into the intra mode. The intra blending mode may be performed based on a general intra prediction mode. The general intra prediction mode may be the intra prediction mode described in FIG. 21 above.

[0455] In one embodiment, if the candidate prediction mode includes only one fixed intra prediction mode, the step of performing template matching may be omitted. The one fixed intra prediction mode may be a planner mode, a DC mode, or a directional intra mode. If the candidate prediction mode includes multiple intra prediction modes, the step of selecting a prediction mode according to the template cost may be performed.

[0456] If the current block is not in intra mode, it may be determined to be in IBC mode, and the video decoder (2000) may acquire an IBC merge flag (S3206). The video decoder (2000) may selectively apply an IBC merge mode or an IBC mode (indicating a mode that directly signals the BV) according to the acquired IBC merge flag (S3207, S3208, S3209).

[0457] When the IBC merge mode is applied, the image decoder (2000) can determine whether to use (or apply) the intra blending mode (S3209). When the intra blending mode is applied, the image decoder (2000) performs a prediction according to the intra blending mode (S3210, S3211), and when not, performs a prediction according to the IBC merge mode (S3210, S3212). At this time, the BV prediction mode may be used in the intra blending mode branched to the IBC merge mode. In this regard, the embodiments described above in FIGS. 23 to 30 may be applied, and here, redundant descriptions are omitted.

[0458] FIG. 33 is a flowchart illustrating a prediction mode determination process according to one embodiment.

[0459] According to one embodiment of the present disclosure, in addition to the embodiment described in FIG. 32 above, the IntraTMP mode can also be designed to signal based on the IBC mode. An intra mode that performs prediction using BV can be distinguished as an IBC mode. By clearly distinguishing between a mode using BV and a general intra prediction mode, a more efficient implementation can be achieved.

[0460] If the current block is not in intra mode, it may be determined to be in IBC mode, and the image decoder (2000) may acquire an IBC merge flag (S3301). The image decoder (2000) may selectively apply an IBC merge mode or an IBC mode (indicating a mode that directly signals the BV) according to the acquired IBC merge flag (S3302, S3303, S3304).

[0461] When the IBC merge mode is applied, the image decoder (2000) can determine whether to use (or apply) the intra blending mode (S3304). When the intra blending mode is applied, the image decoder (2000) performs a prediction according to the intra blending mode (S3305, S3306), and when not, can obtain an IntraTMP flag (S3305, S3307).

[0462] When the IntraTMP mode is applied, the image decoder (2000) can perform prediction by obtaining BV through template matching according to the IntraTMP mode (S3308, S3309), and when not, can perform prediction according to the IBC merge mode (S3308, S3310). At this time, the BV prediction mode may be used in the intra blending mode branched into the IBC merge mode. In this regard, the embodiments described above in FIGS. 23 to 30 may be applied, and here, redundant descriptions are omitted.

[0463] In one embodiment, in computer-generated video where many repetitive patterns exist, the IBC skip mode may generally be selected with high frequency. On the other hand, in camera-captured video, since such patterns are rare, the IBC skip mode tends not to be selected. Taking this into consideration, an activation flag determining whether to allow IBC skip can be signaled in the sequence parameter set, image parameter set, picture header, slice header, etc. If it is set to allow IBC but not IBC skip, the process of selecting a prediction mode can be configured to be performed starting from the step of determining the mode of the current block.

[0464] FIG. 34 is a flowchart of an image decoding method according to one embodiment.

[0465] The video decoding device (2000) can determine whether an intra-blending mode is applied to the current block (S3410).

[0466] When an intra-blending mode is applied to the current block, the video decoding device (2000) can induce a first prediction mode of the current block based on a region decoded prior to the current block (S3420).

[0467] The image decoding device (2000) can generate a first prediction block of the current block by performing intra prediction on the current block using a first prediction mode (S3430).

[0468] The image decoding device (2000) can select a second prediction mode of the current block from among a plurality of candidate prediction modes based on the template cost (S3440).

[0469] In one embodiment, a plurality of candidate prediction modes may include at least one of an intra-mode based prediction mode, a block vector based prediction mode, or a motion vector based prediction mode.

[0470] In one embodiment, the intra-mode based prediction mode may represent a mode that performs intra-prediction according to a fixed intra-mode. The block vector based prediction mode may represent a mode that performs prediction based on a first reference block specified by the block vector of the current block within the current picture to which the current block belongs. The motion vector based prediction mode may represent a mode that performs prediction based on a second reference block specified by the motion vector of the current block within the reference picture of the current block.

[0471] In one embodiment, the fixed intra mode may include at least one of a planar mode or a DC mode. Alternatively, the fixed intra mode may include a directional intra prediction mode. The embodiments described above in FIGS. 23 to 30 may be applied in the same way. Here, redundant descriptions are omitted.

[0472] In one embodiment, the image decoding device (2000) determines a candidate prediction mode according to the slice type of the slice to which the current block belongs, and among the determined candidate prediction modes, determines a prediction mode to be applied to the current block using a template cost. The image decoding device (2000) can check whether the current slice to which the current block belongs is an intra-slice.

[0473] If the current slice is an intra-slice, the second prediction mode may be selected from an intra-mode based prediction mode and a block vector based prediction mode according to the template cost. If the current slice is not an intra-slice, the second prediction mode may be selected from an intra-mode based prediction mode, a block vector based prediction mode, and a motion vector based prediction mode according to the template cost. The embodiments described above in FIGS. 27 to 30 may be applied in the same way. Here, redundant descriptions are omitted.

[0474] In one embodiment, the template cost can be obtained based on the difference between the predicted sample and the restored sample of the template area of ​​the current block. In this case, the template area may be defined as an area of ​​a predetermined size adjacent to the left and upper sides of the block. The embodiments described above in FIGS. 23 to 29 may be applied in the same way. Here, redundant descriptions are omitted.

[0475] Additionally, a cost function for calculating the template cost may be defined. The cost function may include at least one of SAD (sum of absolute difference), SSD (sum of squared difference), SATD (sum of Absolute Transformed Difference), SSE (sum of squared error), or MR-SAD (mean removed SAD). The difference between the predicted sample and the restored sample in the template region of the current block can be calculated using the cost function described above.

[0476] In one embodiment, the prediction sample of the template area for the intra-mode-based prediction mode may be obtained using a reference sample adjacent to the template area of ​​the current block according to the intra-mode fixed for the intra-mode-based prediction mode. The embodiment described above in FIG. 24 and FIG. 25 may be applied in the same way. Here, redundant descriptions are omitted.

[0477] In one embodiment, the prediction sample of the template region for the block vector-based prediction mode may be obtained from the restoration sample of the template region of the first reference block in the current picture. The embodiment described above in FIG. 26 may be applied in the same way. Here, redundant descriptions are omitted.

[0478] In one embodiment, a prediction sample of the template region for the motion vector-based prediction mode may be obtained from a restoration sample of the template region of the second reference block within the reference picture. The embodiment described above in FIGS. 28 and 29 may be applied in the same way. Here, redundant descriptions are omitted.

[0479] In one embodiment, if the current slice is not an intra-slice and neither the IBC mode nor the IntraTMP mode is applied to the current block, the second prediction mode may be selected from the intra-mode based prediction mode and the motion vector based prediction mode according to the template cost. The embodiment described in FIG. 30 above may be applied in the same way. Here, redundant descriptions are omitted.

[0480] The image decoding device (2000) can generate a second prediction block of the current block by performing a prediction corresponding to the second prediction mode for the current block using the second prediction mode (S3450).

[0481] The image decoding device (2000) can generate a final prediction block of the current block by combining a first prediction block and a second prediction block (S3460). In one embodiment, the image decoding device (2000) can generate a prediction block of the current block by performing a weighted sum of a first reference block and a second reference block. The weights used for the weighted sum can be determined in various ways. The embodiments described above in FIGS. 23 and 27 can be applied in the same way. Here, redundant descriptions are omitted.

[0482] In one embodiment, the image decoder (2000) can restore the current block using the final prediction block. The image decoder (2000) can determine the final prediction block as the restored block. Alternatively, the image decoder (2000) can generate the restored current block by combining residual data obtained from the bitstream with the final prediction block.

[0483] FIG. 35 is a block diagram showing the configuration of an image encoding device according to one embodiment of the present disclosure.

[0484] Referring to FIG. 35, the image encoding device (3500) may include a processor (3510) and a memory (3520).

[0485] In one embodiment of the present disclosure, the processor (3510) may include processing circuits and / or multiple processors. For example, the processor (3510) may include various processing circuits including at least one processor, and at least one of the at least one processor may be configured to perform the various functions described in the present disclosure individually and / or collectively in a distributed manner.

[0486] In one embodiment of the present disclosure, the memory (3520) may include one or more storage media that store at least one instruction. The processor (3510) may control the image encoding device (3500) by executing the instruction stored in the memory (3520). For example, the processor (3510) may control the image encoding device (3500) to perform an operation by executing the instruction stored in the memory (3520) individually or collectively. In one embodiment of the present disclosure, the operation performed by the image encoding device (3500) may be an operation performed by the processor (3510) of the image decoding device (2000).

[0487] In one embodiment of the present disclosure, the image encoding device (3500) may correspond to the image encoding device (200) shown in FIG. 2 and / or the encoding unit (1910) shown in FIG. 19.

[0488] The video encoding device (3500) can determine the prediction mode of the current block. The current block may include at least one of a maximum encoding unit, an encoding unit, a conversion unit, or a prediction unit divided from the current image to be encoded.

[0489] In one embodiment, the image can be partitioned into tiles, slices, subpictures, Coding Tree Units (CTU), Coding Tree Blocks (CTB), Coding Units (CU), Coding Blocks (CB), Prediction Units (PU), Prediction Blocks (PB), Transform Units (TU), and Transform Blocks (TB) for various purposes. Subsequently, detailed processes in encoding and decoding can be performed in block units.

[0490] In one embodiment, prediction can be classified into intra prediction, inter prediction (or motion compensation prediction), and combined prediction utilizing both intra prediction and inter prediction (Combined Intra / Inter Prediction or Multi-hypothesis Prediction). Intra prediction refers to the process of generating a predicted image using restored samples (or pixels, images) within the current picture. Inter prediction refers to the process of generating a predicted image using motion information from a reference picture. Combined prediction refers to the process of generating a predicted image by combining the two types of prediction—intra prediction and inter prediction—in various forms.

[0491] After performing a prediction of the block, the image encoding device (3500) can transmit the residual (or difference) of the original image and the predicted image to the image decoding device (2000). A conversion and quantization process can be performed on the residual. The quantized residual data can be output as a bitstream through an entropy coding process and transmitted to the image decoding device (2000).

[0492] In one embodiment, the image encoding device (3500) may apply a deblocking filter to reduce blocking artifacts, a sample adaptive offset (SAO) to remove various noises in the image or to improve subjective image quality, a bilateral filter (BIF), an adaptive loop filter (ALF), a neural network based loop filter (NN), etc., in the same way as the image decoding device (2000).

[0493] In one embodiment, in-loop filtering may be performed on a block, tile, slice, or picture basis. The video encoding device (3500) may optionally perform post-processing at the output stage of the picture and then perform output. The video encoding device (3500) may additionally perform a luminance mapping and / or chroma residual scaling process. Luminance mapping and chroma residual scaling may be applied as tone mapping techniques to improve coding efficiency.

[0494] In one embodiment, the intra prediction mode may be selected from a plurality of intra prediction modes. The plurality of intra prediction modes may include a non-directional intra prediction mode and a directional intra prediction mode. An intra prediction mode according to one embodiment of the present disclosure is as described with reference to FIG. 21, wherein redundant descriptions are omitted.

[0495] In one embodiment, the block copy mode may include an intra block copy mode (IBC). In one embodiment, the block copy mode may include an intra block copy mode. In one embodiment, the intra block copy mode may be a sub-mode of the intra mode, but is not limited thereto, and may represent a mode distinct from the intra mode.

[0496] In one embodiment, when the prediction mode of the current block is a template matching prediction mode, the image encoding device (3500) can restore the current block using a reference block. The image encoding device (3500) can obtain information regarding whether to use the template matching prediction mode. The image encoding device (3500) can determine whether to use the template matching prediction mode based on the obtained information. The reference block may be determined based on at least one of the region included in the current image or the region included in the previously decoded image.

[0497] In one embodiment, the intermode can perform prediction using motion information. For example, the motion information may include a prediction direction, a reference picture index, and a motion vector. Within the reference picture, a reference block may be identified by the motion vector.

[0498] The video encoding device (3500) can restore the current block by performing a prediction according to the prediction mode for the current block according to the prediction mode of the current block.

[0499] In one embodiment, the image encoding device (3500) can signal information regarding the prediction mode of the current block through a bitstream. For example, the image encoding device (3500) can signal index information indicating the prediction mode of the current block through a bitstream.

[0500] In a prediction mode (e.g., intra mode) utilizing reference samples included in the current image, a prediction block of the current block can be generated based on the surrounding samples of the current block according to the prediction mode, under the assumption that there is continuity between the surrounding samples of the current block and the samples within the current block. An image encoding device (3500) according to one embodiment of the present disclosure may utilize spatial reference samples included in the current image, as well as the surrounding samples of the current block included in the current image, for intra prediction. When using samples restored before the current block, the size of residual data may be reduced by predicting the samples of the current block using not only samples immediately adjacent to the current block but also samples far from the current block.

[0501] In a prediction mode (e.g., inter mode) that utilizes reference samples included in a reference image rather than the current image, a prediction block of the current block can be generated based on a reference block (or reference sample) of the reference image according to the prediction mode, under the assumption that there is continuity between the current image and the reference image. An image encoding device (3500) according to one embodiment of the present disclosure can improve compression efficiency by increasing the efficiency of intra prediction.

[0502] The image encoding device (3500) can improve prediction accuracy by considering together the reference block (or reference sample) included in the current image and the reference block (or reference sample) included in an image other than the current image. The image encoding device (3500) according to one embodiment of the present disclosure can improve prediction accuracy by considering both the current image and the image other than the current image.

[0503] The video encoding device (3500) can perform deblocking filtering. The deblocking filter can improve image quality by smoothing the edges between blocks.

[0504] The image encoding device (3500) can perform filtering on samples of the current block where deblocking filtering has been performed using a Sample Adaptive Offset (SAO) filter and / or a Bilateral Filter (BIF). The SAO filter and BIF can improve image quality by reducing the error between the restored image and the original image. The SAO filter and BIF can perform filtering on a sample-by-sample basis.

[0505] The image encoding device (3500) can perform filtering using an Adaptive Loop Filter (ALF). The ALF can improve image quality by reducing the error between the restored image and the original image. The ALF can perform filtering in block units.

[0506] In one embodiment of the present disclosure, encoding of the current block may mean a process of generating information that enables an image decoding device (2000) to restore the current block. The information generated through encoding may be included in a bitstream.

[0507] In one embodiment of the present disclosure, the image encoding device (3500) may generate residual data corresponding to the difference between the prediction block and the current block. If the prediction block is determined to be the current block, residual data may not be generated.

[0508] The video encoding device (3500) can generate a bitstream containing the encoding result of the video. The bitstream may contain the encoding result for the current block.

[0509] In one embodiment, the image encoding device (3500) may generate a bitstream containing residual data. The residual data may contain information regarding the difference between the original image (or original sample) and the predicted image (or predicted sample). In one embodiment, the image encoding device (3500) may generate a bitstream containing a transformation coefficient of a residual block corresponding to a transformation unit. In one embodiment, the image encoding device (3500) may obtain a transformation coefficient of a residual block based on a residual sample of a residual block. For example, the image encoding device (3500) may obtain a transformation coefficient of a residual block by performing at least one of transformation or quantization on a residual sample of a residual block.

[0510] In one embodiment, the image encoding device (3500) can determine the conversion coefficient of a residual block using residual samples of a conversion unit. If the size of the residual block corresponding to the conversion unit is larger than the size of the conversion unit, the image encoding device (3500) can determine a portion of the residual samples of the residual block as residual samples of the conversion unit. Alternatively, the image encoding device (3500) can determine the residual samples of the residual block as residual samples of the conversion unit after filtering has been performed. The image encoding device (3500) can generate a bitstream containing residual samples of the conversion unit.

[0511] In one embodiment of the present disclosure, the image encoding device (3500) can generate a bitstream containing information about a block vector representing a reference block when the prediction mode of the current block is a block copy mode.

[0512] In one embodiment of the present disclosure, the image encoding device (3500) can transmit the bitstream to the image decoding device (2000) through a network.

[0513] In one embodiment of the present disclosure, the image encoding device (3500) can store a bitstream in a data storage medium comprising at least one of a magnetic medium such as a hard disk, a floppy disk and a magnetic tape, an optical recording medium such as a CD-ROM and a DVD, or a magneto-optical medium such as a floptical disk.

[0514] The video encoding device (3500) can generate a bitstream containing syntax elements generated through video encoding. Values ​​corresponding to the syntax elements can be included in the bitstream according to the hierarchical structure of the video.

[0515] FIG. 36 is a flowchart illustrating an image encoding method according to one embodiment of the present disclosure.

[0516] Although the embodiments of FIGS. 22 to 30 described above based on the decoding process, they can be substantially applied to the encoding process as well. For example, information indicating whether to apply an intra-blending mode can be signaled from the image encoding device (3500) to the image decoding device (2000).

[0517] After it is determined that an intra-blending mode is applied, a series of prediction processes, including an intra-prediction mode induction process and a template matching-based prediction mode selection process, can be performed identically in the image encoding device (3500) and the image decoding device (2000).

[0518] The video encoding device (3500) can determine whether to apply an intra-blending mode to the current block (S3610). In one embodiment, information indicating whether an intra-blending mode is applied to the current block can be signaled through a bitstream.

[0519] The video encoding device (3500) can induce a first prediction mode of the current block based on a region decoded prior to the current block (S3620).

[0520] The video encoding device (3500) can generate a first prediction block of the current block by performing intra prediction on the current block using a first prediction mode (S3630).

[0521] The video encoding device (3500) can select a second prediction mode of the current block from among a plurality of candidate prediction modes based on the template cost (S3640).

[0522] In one embodiment, a plurality of candidate prediction modes may include at least one of an intra-mode based prediction mode, a block vector based prediction mode, or a motion vector based prediction mode.

[0523] In one embodiment, the intra-mode based prediction mode may represent a mode that performs intra-prediction according to a fixed intra-mode. The block vector based prediction mode may represent a mode that performs prediction based on a first reference block specified by the block vector of the current block within the current picture to which the current block belongs. The motion vector based prediction mode may represent a mode that performs prediction based on a second reference block specified by the motion vector of the current block within the reference picture of the current block.

[0524] In one embodiment, the fixed intra mode may include at least one of a planar mode or a DC mode. Alternatively, the fixed intra mode may include a directional intra prediction mode. The embodiments described above in FIGS. 23 to 30 may be applied in the same way. Here, redundant descriptions are omitted.

[0525] In one embodiment, the image encoding device (3500) determines a candidate prediction mode according to the slice type of the slice to which the current block belongs, and among the determined candidate prediction modes, determines a prediction mode to be applied to the current block using a template cost. The image encoding device (3500) can check whether the current slice to which the current block belongs is an intra-slice.

[0526] If the current slice is an intra-slice, the second prediction mode may be selected from an intra-mode based prediction mode and a block vector based prediction mode according to the template cost. If the current slice is not an intra-slice, the second prediction mode may be selected from an intra-mode based prediction mode, a block vector based prediction mode, and a motion vector based prediction mode according to the template cost. The embodiments described above in FIGS. 27 to 30 may be applied in the same way. Here, redundant descriptions are omitted.

[0527] In one embodiment, the template cost can be obtained based on the difference between the predicted sample and the restored sample of the template area of ​​the current block. In this case, the template area may be defined as an area of ​​a predetermined size adjacent to the left and upper sides of the block. The embodiments described above in FIGS. 23 to 29 may be applied in the same way. Here, redundant descriptions are omitted.

[0528] Additionally, a cost function for calculating the template cost may be defined. The cost function may include at least one of SAD (sum of absolute difference), SSD (sum of squared difference), SATD (sum of Absolute Transformed Difference), SSE (sum of squared error), or MR-SAD (mean removed SAD). The difference between the predicted sample and the restored sample in the template region of the current block can be calculated using the cost function described above.

[0529] In one embodiment, the prediction sample of the template area for the intra-mode-based prediction mode may be obtained using a reference sample adjacent to the template area of ​​the current block according to the intra-mode fixed for the intra-mode-based prediction mode. The embodiment described above in FIG. 24 and FIG. 25 may be applied in the same way. Here, redundant descriptions are omitted.

[0530] In one embodiment, the prediction sample of the template region for the block vector-based prediction mode may be obtained from the restoration sample of the template region of the first reference block in the current picture. The embodiment described above in FIG. 26 may be applied in the same way. Here, redundant descriptions are omitted.

[0531] In one embodiment, a prediction sample of the template region for the motion vector-based prediction mode may be obtained from a restoration sample of the template region of the second reference block within the reference picture. The embodiment described above in FIGS. 28 and 29 may be applied in the same way. Here, redundant descriptions are omitted.

[0532] In one embodiment, if the current slice is not an intra-slice and neither the IBC mode nor the IntraTMP mode is applied to the current block, the second prediction mode may be selected from the intra-mode based prediction mode and the motion vector based prediction mode according to the template cost. The embodiment described in FIG. 30 above may be applied in the same way. Here, redundant descriptions are omitted.

[0533] The video encoding device (3500) can generate a second prediction block of the current block by performing a prediction corresponding to the second prediction mode for the current block using the second prediction mode (S3650).

[0534] The image encoding device (3500) can generate a final prediction block of the current block by combining the first prediction block and the second prediction block (S3660). In one embodiment, the image encoding device (3500) can generate a prediction block of the current block by performing a weighted sum of the first reference block and the second reference block. The weights used for the weighted sum can be determined in various ways. The embodiments described above in FIG. 23 and FIG. 27 can be applied in the same way. Here, redundant descriptions are omitted.

[0535] In one embodiment, the image encoding device (3500) can restore the current block using the final prediction block. The image encoding device (3500) can determine the final prediction block as the restored block. Alternatively, the image encoding device (3500) can signal information about the residual data obtained by subtracting the final prediction block from the original image to the image decoding device (2000) via a bitstream.

[0536] An image encoding method and apparatus (3500) and an image decoding method and apparatus (2000) according to one embodiment have the objective of improving the performance of predictive encoding and predictive decoding for a current block.

[0537] An image encoding method and apparatus (3500) and an image decoding method and apparatus (2000) according to one embodiment have the objective of reducing the amount of data required for signaling in an intra-prediction mode.

[0538] An image encoding method and apparatus (3500) and an image decoding method and apparatus (2000) according to one embodiment have the objective of reducing the bit rate of a bitstream.

[0539] The technical problems to be solved by the present disclosure are not limited to those mentioned above, and other unmentioned technical problems will be clearly understood by those skilled in the art to which the present disclosure belongs from the description below.

[0540] A method for decoding an image according to one embodiment may include a step of determining whether an intra-blending mode is applied to the current block.

[0541] A method for decoding an image according to one embodiment may include the step of deriving a first prediction mode of the current block based on a region decoded prior to the current block when an intra-blending mode is applied to the current block.

[0542] A decoding method for an image according to one embodiment may include the step of generating a first prediction block of a current block by performing intra prediction on a current block using a first prediction mode.

[0543] A decoding method for an image according to one embodiment may include the step of selecting a second prediction mode of the current block among a plurality of candidate prediction modes based on a template cost.

[0544] In one embodiment, a plurality of candidate prediction modes may include at least one of an intra-mode based prediction mode, a block vector based prediction mode, or a motion vector based prediction mode.

[0545] A decoding method for an image according to one embodiment may include the step of generating a second prediction block of a current block by performing a prediction corresponding to the second prediction mode for the current block using a second prediction mode.

[0546] A decoding method for an image according to one embodiment may include the step of generating a final prediction block of the current block by combining a first prediction block and a second prediction block.

[0547] A method for decoding an image according to one embodiment may include a step of restoring the current block using the final predicted block of the current block.

[0548] In one embodiment, the intra-mode based prediction mode may represent a mode that performs intra-prediction according to a fixed intra-mode.

[0549] In one embodiment, the fixed intra mode may include at least one of a planar mode or a DC mode.

[0550] A method for decoding an image according to one embodiment may further include a step of checking whether the current slice to which the current block belongs is an intra-slice.

[0551] In one embodiment, when the current slice is an intra-slice, the second prediction mode may be selected from an intra-mode based prediction mode and a block vector based prediction mode according to the template cost.

[0552] In one embodiment, if the current slice is not an intra-slice, the second prediction mode may be selected from an intra-mode based prediction mode, a block vector based prediction mode, and a motion vector based prediction mode according to the template cost.

[0553] In one embodiment, the template cost can be obtained based on the difference between the predicted sample and the restored sample of the template area of ​​the current block.

[0554] In one embodiment, the template area may be defined as an area of ​​a predetermined size adjacent to the left and upper sides of the block.

[0555] In one embodiment, the difference may be calculated based on at least one of SAD (sum of absolute difference), SSD (sum of squared difference), SATD (sum of Absolute Transformed Difference), SSE (sum of squared error), or MR-SAD (mean removed SAD).

[0556] In one embodiment, a prediction sample of the template area for an intra-mode based prediction mode can be obtained using a reference sample adjacent to the template area of ​​the current block according to the intra-mode fixed for the intra-mode based prediction mode.

[0557] In one embodiment, the block vector-based prediction mode may represent a mode that performs a prediction based on a first reference block specified by the block vector of the current block within the current picture to which the current block belongs.

[0558] In one embodiment, a prediction sample of the template region for a block vector-based prediction mode can be obtained from a restoration sample of the template region of a first reference block in the current picture.

[0559] In one embodiment, the motion vector-based prediction mode may represent a mode that performs prediction based on a second reference block specified by the motion vector of the current block within the reference picture of the current block.

[0560] In one embodiment, a prediction sample of the template area for a motion vector-based prediction mode can be obtained from a restoration sample of the template area of ​​a second reference block within the reference picture.

[0561] A method for decoding an image according to one embodiment may further include the step of checking whether the current slice to which the current block belongs is an intra slice, and the step of checking whether an Intra Block Copy (IBC) mode and an Intra Template Matching based Prediction (IntraTMP) mode are applied to the current block.

[0562] In one embodiment, if the current slice is not an intra-slice and neither the IBC mode nor the IntraTMP mode is applied to the current block, the second prediction mode may be selected from an intra-mode based prediction mode and a motion vector based prediction mode according to the template cost.

[0563] An image decoding device according to one embodiment may include at least one memory storing at least one instruction; and at least one processor operating according to at least one instruction.

[0564] In one embodiment, at least one processor can determine whether an intra-blending mode is applied to the current block.

[0565] In one embodiment, at least one processor can derive a first prediction mode of the current block based on a region decoded prior to the current block when an intra-blending mode is applied to the current block.

[0566] In one embodiment, at least one processor can generate a first prediction block of the current block by performing intra prediction on the current block using a first prediction mode.

[0567] In one embodiment, at least one processor can select a second prediction mode of the current block from a plurality of candidate prediction modes based on a template cost.

[0568] In one embodiment, a plurality of candidate prediction modes may include at least one of an intra-mode based prediction mode, a block vector based prediction mode, or a motion vector based prediction mode.

[0569] In one embodiment, at least one processor can generate a second prediction block of the current block by performing a prediction corresponding to the second prediction mode for the current block using the second prediction mode.

[0570] In one embodiment, at least one processor can generate a final prediction block of the current block by combining a first prediction block and a second prediction block.

[0571] A method for encoding an image according to one embodiment may include a step of determining whether to apply an intra-blending mode to the current block.

[0572] A method for encoding an image according to one embodiment may include a step of inducing a first prediction mode of the current block based on a region decoded prior to the current block.

[0573] An image encoding method according to one embodiment may include the step of generating a first prediction block of a current block by performing intra prediction on a current block using a first prediction mode.

[0574] An image encoding method according to one embodiment may include the step of selecting a second prediction mode of the current block among a plurality of candidate prediction modes based on a template cost.

[0575] In one embodiment, a plurality of candidate prediction modes may include at least one of an intra-mode based prediction mode, a block vector based prediction mode, or a motion vector based prediction mode.

[0576] An image encoding method according to one embodiment may include the step of generating a second prediction block of a current block by performing a prediction corresponding to the second prediction mode for the current block using a second prediction mode.

[0577] An image encoding method according to one embodiment may include the step of generating a final prediction block of a current block by combining a first prediction block and a second prediction block.

[0578] An image encoding method according to one embodiment may include the step of encoding the current block using the final predicted block of the current block.

[0579] An image encoding device according to one embodiment may include at least one memory for storing at least one instruction; and at least one processor for operating according to at least one instruction.

[0580] In one embodiment, at least one processor can determine whether to apply an intra-blending mode to the current block.

[0581] In one embodiment, at least one processor can derive a first prediction mode of the current block based on a region decoded prior to the current block.

[0582] In one embodiment, at least one processor can generate a first prediction block of the current block by performing intra prediction on the current block using a first prediction mode.

[0583] In one embodiment, at least one processor can select a second prediction mode of the current block from a plurality of candidate prediction modes based on a template cost.

[0584] In one embodiment, a plurality of candidate prediction modes may include at least one of an intra-mode based prediction mode, a block vector based prediction mode, or a motion vector based prediction mode.

[0585] In one embodiment, at least one processor can generate a second prediction block of the current block by performing a prediction corresponding to the second prediction mode for the current block using the second prediction mode.

[0586] In one embodiment, at least one processor can generate a final prediction block of the current block by combining a first prediction block and a second prediction block.

[0587] In one embodiment, at least one processor can encode the current block using the final prediction block of the current block.

[0588] In a computer-readable recording medium that records a bitstream according to one embodiment, the bitstream may include the encoding result of the current block.

[0589] In one embodiment, the encoding result of the current block can be generated by determining whether to apply an intra-blending mode to the current block.

[0590] In one embodiment, the encoding result of the current block can be generated by deriving a first prediction mode of the current block based on a region decoded prior to the current block.

[0591] In one embodiment, the encoding result of the current block can be generated by generating a first prediction block of the current block by performing intra prediction on the current block using a first prediction mode.

[0592] In one embodiment, the encoding result of the current block can be generated by selecting a second prediction mode of the current block from a plurality of candidate prediction modes based on a template cost.

[0593] In one embodiment, a plurality of candidate prediction modes may include at least one of an intra-mode based prediction mode, a block vector based prediction mode, or a motion vector based prediction mode.

[0594] In one embodiment, the encoding result of the current block can be generated by generating a second prediction block of the current block by performing a prediction corresponding to the second prediction mode on the current block using the second prediction mode.

[0595] In one embodiment, the encoding result of the current block can be generated by combining the first prediction block and the second prediction block to generate the final prediction block of the current block.

[0596] In one embodiment, the encoding result of the current block can be generated by encoding the current block using the final predicted block of the current block.

[0597] An image encoding method and apparatus (2800) and an image decoding method and apparatus (2000) according to one embodiment can improve the performance of predictive encoding and predictive decoding for a current block.

[0598] An image encoding method and apparatus (2800) and an image decoding method and apparatus (2000) according to one embodiment can reduce the amount of data required for signaling in an intra-prediction mode.

[0599] An image encoding method and apparatus (2800) and an image decoding method and apparatus (2000) according to one embodiment can reduce the bit rate of a bitstream.

[0600] The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present disclosure belongs from the description below.

[0601] Meanwhile, the embodiments of the present disclosure described above can be written as a program that can be executed on a computer, and the written program can be stored on a storage medium that can be read by a device.

[0602] A device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory storage medium' simply means that it is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily. For example, a 'non-transitory storage medium' may include a buffer in which data is stored temporarily.

[0603] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., downloadable app) may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

Claims

1. In a method for decoding an image, A step (S3410) for determining whether an intra blending mode is applied to the current block; When the intra-blending mode is applied to the current block, a step (S3420) of inducing a first prediction mode of the current block based on a region decoded prior to the current block; A step of generating a first prediction block of the current block by performing an intra prediction on the current block using the first prediction mode (S3430); A step (S3440) of selecting a second prediction mode of the current block from among a plurality of candidate prediction modes based on a template cost, wherein the plurality of candidate prediction modes includes at least one of an intra-mode based prediction mode, a block vector based prediction mode, or a motion vector based prediction mode; A step of generating a second prediction block of the current block by performing a prediction corresponding to the second prediction mode for the current block using the second prediction mode (S3450); and A method for decoding an image, comprising the step (S3460) of generating a final prediction block of the current block by combining the first prediction block and the second prediction block.

2. In Paragraph 1, The above intra-mode based prediction mode represents a mode that performs intra-prediction according to a fixed intra-mode, and A method for decoding an image, wherein the fixed intra mode includes at least one of a planar mode or a DC mode.

3. In Paragraph 1, A method for decoding an image, further comprising the step of determining whether the current slice to which the current block belongs is an intra-slice.

4. In Paragraph 3, A method for decoding an image, wherein, when the current slice is an intra-slice, the second prediction mode is selected according to the template cost among the intra-mode-based prediction mode and the block vector-based prediction mode.

5. In Paragraph 3, A method for decoding an image in which, when the current slice is not an intra-slice, the second prediction mode is selected according to the template cost among the intra-mode-based prediction mode, the block vector-based prediction mode, and the motion vector-based prediction mode.

6. In Paragraph 1, The above template cost is obtained based on the difference between the predicted sample and the restored sample of the template area of ​​the current block, and A method for decoding an image, wherein the above template area is defined as an area of ​​a predetermined size adjacent to the left and upper sides of the above current block.

7. In Paragraph 6, A method for decoding an image, wherein the difference is calculated based on at least one of SAD (sum of absolute difference), SSD (sum of squared difference), SATD (sum of Absolute Transformed Difference), SSE (sum of squared error), or MR-SAD (mean removed SAD).

8. In Paragraph 6, A method for decoding an image, wherein the prediction sample of the template region for the intra-mode-based prediction mode is obtained using a reference sample adjacent to the template region of the current block according to the intra-mode fixed for the intra-mode-based prediction mode.

9. In Paragraph 6, The above block vector-based prediction mode represents a mode that performs a prediction based on a first reference block specified by the block vector of the current block within the current picture to which the current block belongs, and A method for decoding an image, wherein the prediction sample of the template region for the block vector-based prediction mode is obtained from a restoration sample of the template region of the first reference block in the current picture.

10. In Paragraph 6, The above motion vector-based prediction mode represents a mode that performs prediction based on a second reference block specified by the motion vector of the current block within the reference picture of the current block, and A method for decoding an image, wherein the prediction sample of the template region for the above motion vector-based prediction mode is obtained from a restoration sample of the template region of the second reference block within the above reference picture.

11. In Paragraph 1, A step of determining whether the current slice to which the above current block belongs is an intra-slice; and The method further includes a step of checking whether IBC (Intra Block Copy) mode and IntraTMP (Intra Template Matching based Prediction) mode are applied to the current block. A method for decoding an image in which, when the current slice is not an intra slice and neither the IBC mode nor the IntraTMP mode is applied to the current block, the second prediction mode is selected according to the template cost among the intra mode-based prediction mode and the motion vector-based prediction mode.

12. In an image decoding device, At least one memory storing at least one instruction; and It includes at least one processor that operates according to the above at least one instruction, and The above-mentioned at least one processor is, Determine whether intra-blending mode is applied to the current block, and When the intra-blending mode is applied to the current block, a first prediction mode of the current block is derived based on a region decoded prior to the current block, and By performing intra prediction on the current block using the first prediction mode, a first prediction block of the current block is generated, and A second prediction mode of the current block is selected from a plurality of candidate prediction modes based on template cost, wherein the plurality of candidate prediction modes includes at least one of an intra-mode based prediction mode, a block vector based prediction mode, or a motion vector based prediction mode. By using the second prediction mode, a prediction corresponding to the second prediction mode is performed on the current block, thereby generating a second prediction block of the current block, and An image decoding device that generates a final prediction block of the current block by combining the first prediction block and the second prediction block.

13. In a method for encoding images, A step (S3610) for determining whether to apply an intra-blending mode to the current block; Step (S3620) of inducing a first prediction mode of the current block based on a region decrypted prior to the current block; A step of generating a first prediction block of the current block by performing an intra prediction on the current block using the first prediction mode (S3630); A step (S3640) of selecting a second prediction mode of the current block from among a plurality of candidate prediction modes based on a template cost, wherein the plurality of candidate prediction modes includes at least one of an intra-mode based prediction mode, a block vector based prediction mode, or a motion vector based prediction mode; A step of generating a second prediction block of the current block by performing a prediction corresponding to the second prediction mode for the current block using the second prediction mode (S3650); and A method for encoding an image, comprising the step (S3660) of generating a final prediction block of the current block by combining the first prediction block and the second prediction block.

14. A method for transmitting a bitstream generated by the image encoding method of paragraph 13.

15. In a computer-readable recording medium that records a bitstream, The above bitstream includes the encoding result of the current block, and The encoding result of the current block above is, Determine whether to apply an intra-blending mode to the current block above, and Inducing a first prediction mode of the current block based on a region decrypted prior to the current block, and By performing intra prediction on the current block using the first prediction mode, a first prediction block of the current block is generated, and A second prediction mode of the current block is selected from a plurality of candidate prediction modes based on template cost, wherein the plurality of candidate prediction modes includes at least one of an intra-mode based prediction mode, a block vector based prediction mode, or a motion vector based prediction mode. By using the second prediction mode, a prediction corresponding to the second prediction mode is performed on the current block, thereby generating a second prediction block of the current block, and By combining the first prediction block and the second prediction block, the final prediction block of the current block is generated, and A recording medium generated by encoding the current block using the final prediction block.

Images