Image encoding method, image encoding device, image decoding method, and image decoding device

Abstract

Provided are an image decoding method, an image decoding device, an image encoding method, and an image encoding device, the image decoding method comprising the steps of: identifying a plurality of candidate positions for a significant sub-block including at least one non-zero transform coefficient within a current block; determining a cost for each of the plurality of candidate positions on the basis of at least one of a variation between a plurality of neighboring samples at a boundary of the significant sub-block and a variation between a plurality of neighboring samples at a boundary of the current block; determining rankings for the plurality of candidate positions on the basis of the cost for each of the plurality of candidate positions; determining the significant sub-block's position for reconstruction of the current block from among the plurality of candidate positions, on the basis of a ranking indicated by index information acquired from a bitstream; determining positions of a plurality of sub-blocks included in the current block on the basis of the significant sub-block's position for reconstruction of the current block; and determining a boundary for applying filtering, on the basis of the determined positions of the plurality of sub-blocks.

Description

Image encoding method, encoding device, decoding method and decoding device

[0001] The present disclosure relates to the field of image encoding and decoding. More specifically, it relates to an encoding and decoding method and apparatus for predicting samples of an image.

[0002] In video encoding and decoding, the video is divided into blocks, and each block is predictively encoded and predictedly decoded through inter-prediction or intra-prediction.

[0003] Inter-prediction is a technique that compresses images by eliminating temporal redundancy between images. In inter-prediction, blocks of the current image are predicted using a reference image. The reference block most similar to the current block can be searched within a predetermined search range in the reference image. The current block is predicted based on the reference block, and a residual block is generated by subtracting the predicted block resulting from the prediction result from the current block.

[0004] Intra prediction is a technique that compresses images by eliminating spatial redundancy within the image. Depending on the prediction mode, intra prediction generates a prediction block based on the surrounding pixels of the current block. Then, it generates a residual block by subtracting the prediction block from the current block.

[0005] Residual blocks generated through inter-prediction or intra-prediction are passed to a decoder after undergoing transformation and quantization. The decoder inversely quantizes and inversely transforms the residual blocks, and reconstructs the current block by combining the prediction block of the current block with the residual blocks. The decoder can filter the reconstructed current block to remove artifacts within it.

[0006] In one embodiment of the present disclosure, an image decoding method is provided. The image decoding method may include the step of identifying a plurality of candidate locations for an effective sub-block that includes at least one non-zero transform factor within the current block. The image decoding method may include the step of determining a cost for each of the plurality of candidate locations based on at least one of the change amount between a plurality of surrounding samples of the boundary of the effective sub-block and the change amount between a plurality of surrounding samples of the boundary of the current block. The image decoding method may include the step of determining a rank for the plurality of candidate locations based on the cost for each of the plurality of candidate locations. The image decoding method may include the step of determining the location of the effective sub-block for the restoration of the current block among the plurality of candidate locations based on the rank indicated by index information obtained from the bitstream. The image decoding method may include the step of determining the location of a plurality of sub-blocks included in the current block based on the location of the effective sub-block for the restoration of the current block. The image decoding method may include the step of determining a boundary for applying filtering based on the determined location of the plurality of sub-blocks.

[0007] In one embodiment of the present disclosure, an image decoder is provided. The image decoder may include at least one processor comprising a processing circuit and a memory comprising one or more storage media for storing instructions. By executing instructions individually or collectively by at least one processor, the image decoder may identify a plurality of candidate locations for a valid sub-block comprising at least one non-zero transform factor within a current block. By executing instructions individually or collectively by at least one processor, the image decoder may determine a cost for each of the plurality of candidate locations based on at least one of the change amount between a plurality of surrounding samples at the boundary of a valid sub-block and the change amount between a plurality of surrounding samples at the boundary of a current block. By executing instructions individually or collectively by at least one processor, the image decoder may determine a ranking for the plurality of candidate locations based on the cost for each of the plurality of candidate locations. By executing instructions individually or collectively by at least one processor, the image decoder can determine the location of an effective sub-block for the restoration of the current block among a plurality of candidate locations based on the rank indicated by index information obtained from the bitstream. By executing instructions individually or collectively by at least one processor, the image decoder can determine the location of a plurality of sub-blocks included in the current block based on the location of the effective sub-block for the restoration of the current block. By executing instructions individually or collectively by at least one processor, the image decoder can determine a boundary for applying filtering based on the determined location of a plurality of sub-blocks.

[0008] In one embodiment of the present disclosure, an image encoding method is provided. The image encoding method may include the step of identifying a plurality of candidate locations for an effective sub-block comprising at least one non-zero transform factor within a current block. The image encoding method may include the step of determining a cost for each of the plurality of candidate locations based on at least one of the change amount between a plurality of surrounding samples of the boundary of the effective sub-block and the change amount between a plurality of surrounding samples of the boundary of the current block. The image encoding method may include the step of determining a rank for the plurality of candidate locations based on the cost for each of the plurality of candidate locations. The image encoding method may include the step of determining a location of an effective sub-block for the restoration of the current block among the plurality of candidate locations. The image encoding method may include the step of determining a boundary for applying filtering based on the determined location of the plurality of sub-blocks. The image encoding method may include the step of generating a bitstream including index information representing the rank of the effective sub-block for restoration.

[0009] In one embodiment of the present disclosure, a method for transmitting a bitstream is provided. The method for transmitting a bitstream may include the step of identifying a plurality of candidate locations for an effective sub-block comprising at least one non-zero transformation factor within a current block. The method for transmitting a bitstream may include the step of determining a cost for each of the plurality of candidate locations based on at least one of the change amount between a plurality of surrounding samples of the boundary of the effective sub-block and the change amount between a plurality of surrounding samples of the boundary of the current block. The method for transmitting a bitstream may include the step of determining a rank for the plurality of candidate locations based on the cost for each of the plurality of candidate locations. The method for transmitting a bitstream may include the step of determining a location of an effective sub-block for the restoration of the current block among the plurality of candidate locations. The method for transmitting a bitstream may include the step of determining a boundary for applying filtering based on the determined location of the plurality of sub-blocks. The method for transmitting a bitstream may include the step of generating a bitstream including index information representing the rank of the effective sub-block for restoration. The method for transmitting a bitstream may include the step of transmitting the bitstream to an image decoder.

[0010] FIG. 1 is a block diagram of an image decoding device according to one embodiment of the present disclosure.

[0011] FIG. 2 is a block diagram of an image encoding device according to one embodiment of the present disclosure.

[0012] FIG. 3 illustrates a process of determining at least one encoding unit by dividing a current encoding unit according to one embodiment of the present disclosure.

[0013] FIG. 4 illustrates a process of determining at least one encoding unit by dividing a encoding unit that is in the shape of a non-square according to one embodiment of the present disclosure.

[0014] FIG. 5 illustrates a process of dividing a encoding unit based on at least one of block shape information and division shape mode information according to one embodiment of the present disclosure.

[0015] FIG. 6 illustrates a method for determining a predetermined encoding unit among an odd number of encoding units according to one embodiment of the present disclosure.

[0016] FIG. 7 illustrates the order in which a plurality of encoding units are processed when a current encoding unit is divided to determine a plurality of encoding units according to one embodiment of the present disclosure.

[0017] FIG. 8 illustrates a process for determining that, in accordance with one embodiment of the present disclosure, when the encoding unit cannot be processed in a predetermined order, the current encoding unit is divided into an odd number of encoding units.

[0018] FIG. 9 illustrates a process of determining at least one encoding unit by dividing a first encoding unit according to one embodiment of the present disclosure.

[0019] FIG. 10 illustrates that, according to one embodiment of the present disclosure, the shape that can be divided is limited when a second encoding unit of a non-square shape determined by dividing a first encoding unit satisfies a predetermined condition.

[0020] FIG. 11 illustrates a process of dividing square-shaped encoding units when the divided shape mode information cannot represent division into four square-shaped encoding units according to one embodiment of the present disclosure.

[0021] FIG. 12 illustrates that, according to one embodiment of the present disclosure, the processing order between a plurality of encoding units may vary depending on the division process of the encoding unit.

[0022] FIG. 13 illustrates a process in which, according to one embodiment of the present disclosure, a encoding unit is recursively divided to determine a plurality of encoding units, and the depth of the encoding unit is determined as the shape and size of the encoding unit change.

[0023] FIG. 14 illustrates a depth and part index (hereinafter PID) for distinguishing between coding units that can be determined according to the shape and size of the coding units according to one embodiment of the present disclosure.

[0024] FIG. 15 illustrates that a plurality of encoding units are determined according to a plurality of predetermined data units included in a picture according to one embodiment of the present disclosure.

[0025] FIG. 16 illustrates encoding units that can be determined for each picture when the combination of forms in which encoding units can be divided according to one embodiment of the present disclosure is different for each picture.

[0026] FIG. 17 illustrates various forms of encoding units that can be determined based on partitioned form mode information expressed in binary code according to one embodiment of the present disclosure.

[0027] FIG. 18 illustrates another form of encoding unit that can be determined based on partitioned form mode information represented by binary code according to one embodiment of the present disclosure.

[0028] FIG. 19 is a block diagram of an image encoding and decoding system that performs loop filtering according to one embodiment of the present disclosure.

[0029] FIG. 20 is a block diagram showing the configuration of an image decoding device according to one embodiment of the present disclosure.

[0030] FIG. 21 is a drawing for explaining the process of determining an effective sub-block according to one embodiment of the present disclosure.

[0031] FIG. 22 is a drawing for explaining the process of determining an effective sub-block according to one embodiment of the present disclosure.

[0032] FIG. 23 is a drawing for explaining the process of determining an effective sub-block according to one embodiment of the present disclosure.

[0033] FIG. 24 is a flowchart illustrating an image decoding method according to one embodiment of the present disclosure.

[0034] FIG. 25 is a drawing for explaining the process of determining the location of an effective sub-block based on cost according to one embodiment of the present disclosure.

[0035] FIG. 26 is a drawing for explaining the process of determining the location of an effective sub-block based on cost according to one embodiment of the present disclosure.

[0036] FIG. 27a is a drawing for explaining a method for determining a cost according to one embodiment of the present disclosure.

[0037] FIG. 27b is a drawing for explaining a method for determining a cost according to one embodiment of the present disclosure.

[0038] FIG. 28 is a drawing for explaining a method for determining a cost according to one embodiment of the present disclosure.

[0039] FIG. 29a is a drawing for explaining a binarization method of index information according to one embodiment of the present disclosure.

[0040] FIG. 29b is a diagram illustrating a binarization method of index information according to one embodiment of the present disclosure.

[0041] FIG. 30 is a drawing for explaining a binarization method of index information according to one embodiment of the present disclosure.

[0042] FIG. 31a is a drawing for explaining the process of applying filtering according to one embodiment of the present disclosure.

[0043] FIG. 31b is a drawing for explaining the process of determining a boundary for applying filtering according to one embodiment of the present disclosure.

[0044] FIG. 32 is a drawing for explaining the process of obtaining information regarding an effective sub-block according to one embodiment of the present disclosure.

[0045] FIG. 33 is a drawing for explaining the process of obtaining information regarding an effective sub-block according to one embodiment of the present disclosure.

[0046] FIG. 34 is a drawing for explaining the process of obtaining information regarding an effective sub-block according to one embodiment of the present disclosure.

[0047] FIG. 35 is a drawing for explaining the process of obtaining information regarding an effective sub-block according to one embodiment of the present disclosure.

[0048] FIG. 36 is a drawing for explaining the process of obtaining information regarding an effective sub-block according to one embodiment of the present disclosure.

[0049] FIG. 37 is a drawing for explaining the process of obtaining information regarding an effective sub-block according to one embodiment of the present disclosure.

[0050] FIG. 38 is a drawing for explaining the process of obtaining information regarding an effective sub-block according to one embodiment of the present disclosure.

[0051] FIG. 39 is a block diagram showing the configuration of an image encoding device according to one embodiment of the present disclosure.

[0052] FIG. 40 is a flowchart illustrating an image encoding method according to one embodiment of the present disclosure.

[0053] In the present disclosure, the expression “at least one of a, b, or c” may refer to “a”, “b”, “c”, “a and b”, “a and c”, “b and c”, or “all of a, b, and c”.

[0054] In the present disclosure, the expression "a, b and / or c" may be replaced with "at least one of a, b, or c." That is, the expression "a, b and / or c" may refer to "a," "b," "c," "a and b," "a and c," "b and c," or "all of a, b, and c."

[0055] The present disclosure is capable of various modifications and may have various embodiments, and embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the embodiments of the present disclosure, and the present disclosure may include all modifications, equivalents, and substitutions that fall within the spirit and technical scope of the various embodiments.

[0056] The terms used in this disclosure have been selected to be as widely used and general as possible, taking into account their functions within this disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been selected at the applicant's discretion, and in such cases, their meaning can be understood through the relevant explanatory sections. Therefore, the terms used in this disclosure should be defined not merely by their names, but based on their meanings and the content throughout this disclosure.

[0057] In this disclosure, singular expressions may include plural expressions unless the context clearly indicates otherwise. In describing the embodiments, detailed descriptions of related prior art are omitted if it is determined that such detailed descriptions would unnecessarily obscure the essence of this disclosure. Additionally, numbers used in the description of the embodiments (e.g., first, second, etc.) correspond to identification symbols for distinguishing one component from another.

[0058] In the present disclosure, when one component is described as being "connected" or "connected" to another component, the one component may be directly connected to or directly connected to another component, but unless specifically stated otherwise, it may be connected or connected through another component in between.

[0059] In the present disclosure, when a part is described as "comprising" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. Components expressed in the present disclosure as "part (unit)," "module," etc., may consist of two or more components combined into a single component, or a single component may be divided into two or more more subdivided components. Furthermore, each component described below may additionally perform some or all of the functions of other components in addition to its own primary function, and some of the primary functions of each component may be exclusively performed by other components.

[0060] As used in this disclosure, the expression “configured to” may be replaced, depending on the context, with, for example, “suitable for,” “having the capacity to,” “designed to,” “adapted to,” “made to,” or “capable of.” The term “configured to” may not necessarily mean only “specifically designed to” in hardware. Alternatively, in some situations, the expression “system configured to” may include that the system is “capable of” in conjunction with other devices or components. For example, the phrase “processor configured to perform A, B, and C” may include a dedicated processor for performing the said operations (e.g., an embedded processor), or a generic-purpose processor (e.g., a CPU or an application processor) capable of performing said operations by executing one or more software programs stored in memory.

[0061] At least one processor according to an embodiment of the present disclosure may include various processing circuits and / or multiple processors. For example, the term “processor” as used in the present disclosure, including in the claims, may include various processing circuits including at least one processor, and at least one of the at least one processor is configured to perform the various functions described herein individually and / or collectively in a distributed manner. Where “processor,” “at least one processor,” and “one or more processors” are described as being configured to perform various functions as used in the specification, these terms may include, for example, without limitation, situations in which a single processor performs. There may be situations in which some of the cited functions and other processor(s) perform other functions among the cited functions, and also situations in which a single processor can perform all the cited functions. Additionally, at least one processor may include a combination of processors that perform the various enumerated / disclosed functions, for example, in a distributed manner. At least one processor may execute program instructions to achieve or perform the various functions.

[0062] In the present disclosure, 'image' may include a picture, a still image, a frame, a video composed of a plurality of consecutive still images, or a video.

[0063] In the present disclosure, 'sample' may include data to be processed as data assigned to a sampling location of an image. For example, a sample may include pixels within a frame in a spatial region. A block may refer to a unit comprising a plurality of samples.

[0064] Hereinafter, with reference to FIGS. 1 to 19, an image encoding method and apparatus based on a tree structure encoding unit and a conversion unit according to one embodiment of the present disclosure, an image decoding method and apparatus are disclosed.

[0065] FIG. 1 illustrates a block diagram of an image decoding device (100) according to one embodiment of the present disclosure.

[0066] The video decoding device (100) may include a bitstream acquisition unit (110) and a decoding unit (120). The bitstream acquisition unit (110) and the decoding unit (120) may include at least one processor. Additionally, the bitstream acquisition unit (110) and the decoding unit (120) may include a memory that stores instructions to be executed by at least one processor.

[0067] The bitstream acquisition unit (110) can receive a bitstream. The bitstream contains information in which an image is encoded by an image encoding device (200) described later. Additionally, the bitstream can be transmitted from the image encoding device (200). The image encoding device (200) and the image decoding device (100) can be connected via wired or wireless connection, and the bitstream acquisition unit (110) can receive the bitstream via wired or wireless connection. The bitstream acquisition unit (110) can receive the bitstream from a storage medium such as an optical medium or a hard disk. The decoding unit (120) can restore the image based on information obtained from the received bitstream. The decoding unit (120) can obtain syntax elements for restoring the image from the bitstream. The decoding unit (120) can restore the image based on the syntax elements.

[0068] To explain in detail the operation of the video decoding device (100), the bitstream acquisition unit (110) can receive a bitstream.

[0069] The image decoder (100) can perform an operation of obtaining an empty string corresponding to a partitioning mode of the encoding unit from a bitstream. The image decoder (100) can also perform an operation of determining a partitioning rule for the encoding unit. Additionally, the image decoder (100) can perform an operation of partitioning the encoding unit into a plurality of encoding units based on at least one of the empty string corresponding to the partitioning mode and the partitioning rule. To determine the partitioning rule, the image decoder (100) can determine an allowable first range of the size of the encoding unit according to the ratio of the width and height of the encoding unit. To determine the partitioning rule, the image decoder (100) can determine an allowable second range of the size of the encoding unit according to the partitioning mode of the encoding unit.

[0070] In the following, the division of a encoding unit according to one embodiment of the present disclosure will be described in detail.

[0071] First, a picture may be divided into one or more slices or one or more tiles. A slice or a tile may be a sequence of one or more Coding Tree Units (CTUs). Depending on the embodiment, a slice may include one or more tiles, and a slice may include one or more Coding Tree Units. A slice containing one or more tiles may be determined within the picture.

[0072] In contrast to the Max Coding Unit (CTU), there is the Max Coding Tree Block (CTB). A Max Coding Tree Block (CTB) refers to an NxN block containing NxN samples (where N is an integer). Each color component can be divided into one or more Max Coding Tree Blocks.

[0073] When a picture has three sample arrays (sample arrays for Y, Cr, and Cb components), the maximum encoding unit (CTU) is a unit that includes a maximum encoding block for luminance samples and two corresponding maximum encoding blocks for chroma samples, and syntax structures used to encode the luminance samples and chroma samples. When a picture is a monochrome picture, the maximum encoding unit is a unit that includes a maximum encoding block for monochrome samples and syntax structures used to encode the monochrome samples. When a picture is encoded into color planes separated by color components, the maximum encoding unit is a unit that includes syntax structures used to encode the samples of the picture and the image.

[0074] A single maximum coding block (CTB) can be divided into MxN coding blocks containing MxN samples (M and N are integers). In one embodiment, a coding block may be referred to as a coding unit.

[0075] When a picture has sample arrays for Y, Cr, and Cb components, a coding unit (CU) is a unit that includes a coding block for luminance samples and two coding blocks for corresponding chroma samples, and syntax structures used to encode the luminance samples and chroma samples. When a picture is a monochrome picture, a coding unit is a unit that includes a coding block for monochrome samples and syntax structures used to encode the monochrome samples. When a picture is a picture encoded in color planes separated by color components, a coding unit is a unit that includes syntax structures used to encode the corresponding picture and image samples.

[0076] As explained above, the maximum encoding block and the maximum encoding unit are distinct concepts, and the encoding block and the encoding unit are distinct concepts. That is, the (maximum) encoding unit refers to a data structure that includes the (maximum) encoding block containing the corresponding sample and the corresponding syntax structure. However, since a person skilled in the art can understand that the (maximum) encoding unit or the (maximum) encoding block refers to a block of a predetermined size containing a predetermined number of samples, the maximum encoding block and the maximum encoding unit, or the encoding block and the encoding unit, are referred to without distinction in the following specification unless there are special circumstances.

[0077] The image can be divided into Coding Tree Units (CTUs). The size of the CTU can be determined based on information obtained from the bitstream. The shape of the CTU can be a square of equal size, but it is not limited to this.

[0078] For example, information about the maximum size of a luma-encoded block can be obtained from a bitstream. For example, the maximum size of a luma-encoded block indicated by the information about the maximum size of a luma-encoded block may be one of 4x4, 8x8, 16x16, 32x32, 64x64, 128x128, or 256x256.

[0079] For example, information regarding the maximum size of a two-divisionable luma encoding block and the difference in luma block size can be obtained from a bitstream. The information regarding the difference in luma block size may indicate the size difference between the maximum luma encoding unit and the maximum two-divisionable luma encoding block. Therefore, by combining the information regarding the maximum size of the two-divisionable luma encoding block obtained from the bitstream with the information regarding the difference in luma block size, the size of the maximum luma encoding unit can be determined. Using the size of the maximum luma encoding unit, the size of the maximum chroma encoding unit can also be determined. For example, if the Y:Cb:Cr ratio according to the color format is 4:2:0, the size of the chroma block may be half the size of the luma block, and similarly, the size of the maximum chroma encoding unit may be half the size of the maximum luma encoding unit.

[0080] According to one embodiment, information regarding the maximum size of a binary splittable luminous encoding block is obtained from a bitstream, so the maximum size of the binary splittable luminous encoding block can be determined variably. Alternatively, the maximum size of a ternary splittable luminous encoding block can be fixed. For example, the maximum size of a ternary splittable luminous encoding block in image I may be 32x32, and the maximum size of a ternary splittable luminous encoding block in picture P or image B may be 64x64.

[0081] Additionally, the maximum encoding unit can be hierarchically divided into encoding units based on splitting mode information obtained from the bitstream. As splitting mode information, at least one of information indicating whether it is a quad split, information indicating whether it is a multi-split, splitting direction information, and splitting type information can be obtained from the bitstream.

[0082] For example, information indicating whether quad splitting is performed can indicate whether the current encoding unit will be quad split or not.

[0083] If the current encoding unit is not quad-splitting, information indicating multi-splitting can indicate whether the current encoding unit will not be split further (NO_SPLIT) or whether it will be binary / ternary split.

[0084] If the current encoding unit is binary or binary split, the splitting direction information indicates that the current encoding unit is split in either the horizontal or vertical direction.

[0085] If the current encoding unit is split horizontally or vertically, the split type information indicates that the current encoding unit is split into binary or binary splits.

[0086] The splitting mode of the current encoding unit can be determined based on the splitting direction information and the splitting type information. The splitting mode when the current encoding unit is binary split in the horizontal direction can be determined as binary horizontal splitting (SPLIT_BT_HOR), when it is territorial split in the horizontal direction as territorial horizontal splitting (SPLIT_TT_HOR), when it is binary split in the vertical direction as binary vertical splitting (SPLIT_BT_VER), and when it is territorial split in the vertical direction as territorial vertical splitting (SPLIT_TT_VER).

[0087] The image decoding device (100) can obtain splitting mode information from a bitstream from a single empty string. The form of the bitstream received by the image decoding device (100) may include a fixed-length binary code, a unary code, a truncated unary code, a predetermined binary code, etc. The empty string represents information as a sequence of binary numbers. The empty string may consist of at least one bit. The image decoding device (100) can obtain splitting mode information corresponding to the empty string based on a splitting rule. Based on the single empty string, the image decoding device (100) can determine whether to quad split the encoding unit, whether not to split it, or the splitting direction and splitting type.

[0088] A coding unit may be smaller than or equal to a maximum coding unit. For example, since the maximum coding unit is a coding unit having the maximum size, it is also a coding unit. If the segmentation mode information for the maximum coding unit indicates that it is not segmented, the coding unit determined from the maximum coding unit has the same size as the maximum coding unit. If the segmentation mode information for the maximum coding unit indicates that it is segmented, the maximum coding unit may be segmented into coding units. Additionally, if the segmentation mode information for a coding unit indicates segmentation, the coding units may be segmented into coding units of smaller size. However, the segmentation of the image is not limited to this, and the maximum coding unit and the coding unit may not be distinguished. The segmentation of coding units is explained in more detail in FIGS. 3 through 16.

[0089] Additionally, one or more prediction blocks for prediction may be determined from the coding unit. The prediction blocks may be equal to or smaller than the coding unit. Additionally, one or more transformation blocks for transformation may be determined from the coding unit. The transformation blocks may be equal to or smaller than the coding unit. The transformation blocks may be referred to as the transformation unit.

[0090] The shape and size of the transformation block and the prediction block may not be related to each other.

[0091] In another embodiment, prediction can be performed using the encoding unit as a prediction block. Additionally, conversion can be performed using the encoding unit as a conversion block.

[0092] The division of the encoding unit is described in more detail in FIGS. 3 through 16. The current block and neighbor block of the present disclosure may represent one of the maximum encoding unit, the encoding unit, the prediction block, and the transformation block. Additionally, the current block or the current encoding unit is a block currently undergoing decoding or encoding, or a block currently undergoing division. The neighbor block may be a block restored prior to the current block. The neighbor block may be spatially or temporally adjacent to the current block. The neighbor block may be located on one of the lower-left, left, upper-left, upper, upper-right, right, or lower-right sides of the current block.

[0093] The above-described embodiment explains the operation related to the image decoding method performed by the image decoding device (100). Below, the operation of the image encoding device (200) that performs an image encoding method corresponding to the reverse process of the image decoding method will be explained through an embodiment of the present disclosure.

[0094] FIG. 2 illustrates a block diagram of an image encoding device (200) capable of encoding an image based on at least one of block shape information and segmented shape mode information according to one embodiment of the present disclosure.

[0095] The video encoding device (200) may include an encoding unit (220) and a bitstream generation unit (210). The encoding unit (220) may receive an input video and encode the input video. The encoding unit (220) may encode the input video to obtain at least one syntax element. The syntax element may include at least one of a skip flag, a prediction mode, a motion vector difference, a motion vector prediction method (or index), a transform quantized coefficient, a coded block pattern, a coded block flag, an intra prediction mode, a direct flag, a merge flag, a delta QP, a reference index, a prediction direction, and a transform index. The encoding unit (220) may determine a context model based on block shape information including at least one of the shape, direction, width, and height ratio or size of the encoding unit.

[0096] The bitstream generation unit (210) can generate a bitstream based on an encoded input image. For example, the bitstream generation unit (210) can generate a bitstream by entropy encoding syntax elements based on a context model. In one embodiment of the present disclosure, the image encoding device (100) can generate a bitstream according to an image encoding method to be described later. The image encoding device (100) can store the bitstream in a computer-readable storage medium.

[0097] In one embodiment of the present disclosure, the image encoding device (200) can transmit a bitstream to the image decoding device (100). For example, the image encoding device (200) can transmit a bitstream generated by an image encoding method.

[0098] According to one embodiment of the present disclosure, the encoding unit (220) of the image encoding device (200) can determine the shape of the encoding unit. For example, the encoding unit may be square or non-square in shape, and information indicating such shape may be included in block shape information.

[0099] According to one embodiment of the present disclosure, the encoding unit (220) may determine how the encoding unit will be divided. The encoding unit (220) may determine the form of at least one encoding unit included in the encoding unit, and the bitstream generation unit (210) may generate a bitstream including division form mode information including information about the form of such encoding unit.

[0100] According to one embodiment of the present disclosure, the encoding unit (220) may determine whether the encoding unit is divided or not. If the encoding unit (220) determines that only one encoding unit is included in the encoding unit or that the encoding unit is not divided, the bitstream generation unit (210) may generate a bitstream including division type mode information indicating that the encoding unit is not divided. Additionally, the encoding unit (220) may divide the encoding unit into a plurality of encoding units, and the bitstream generation unit (210) may generate a bitstream including division type mode information indicating that the encoding unit is divided into a plurality of encoding units.

[0101] According to one embodiment of the present disclosure, information indicating how many encoding units to divide or in which direction to divide may be included in the division type mode information. For example, the division type mode information may indicate dividing in at least one of the vertical direction and the horizontal direction, or not dividing.

[0102] The video encoding device (200) determines information regarding the segmentation mode based on the segmentation mode of the encoding unit. The video encoding device (200) determines a context model based on at least one of the ratio or size of the shape, direction, width, and height of the encoding unit. Then, the video encoding device (200) generates information regarding the segmentation mode for segmenting the encoding unit based on the context model as a bitstream.

[0103] The video encoding device (200) may obtain an array for matching at least one of the ratio or size of the shape, direction, width, and height of an encoding unit with an index for the context model in order to determine the context model. The video encoding device (200) may obtain an index for the context model based on at least one of the ratio or size of the shape, direction, width, and height of an encoding unit in the array. The video encoding device (200) may determine the context model based on the index for the context model.

[0104] The video encoding device (200) may determine a context model based further on block shape information including at least one of the ratio or size of the shape, direction, width, and height of a surrounding encoding unit adjacent to the encoding unit, in order to determine the context model. Additionally, the surrounding encoding unit may include at least one of the encoding units located on the lower left, left, upper left, upper, upper right, right, or lower right side of the encoding unit.

[0105] Additionally, the video encoding device (200) can compare the width of an upper peripheral encoding unit with the width of an encoding unit in order to determine a context model. Additionally, the video encoding device (200) can compare the height of left and right peripheral encoding units with the height of an encoding unit. Additionally, the video encoding device (200) can determine a context model based on the comparison results.

[0106] The operation of the video encoding device (200) includes content similar to the operation of the video decoding device (100) described in FIGS. 3 to 19, so a detailed description is omitted.

[0107] FIG. 3 illustrates a process in which an image decoding device (100) divides a current encoding unit to determine at least one encoding unit according to one embodiment of the present disclosure.

[0108] The block shape may include 4Nx4N, 4Nx2N, 2Nx4N, 4NxN, Nx4N, 32NxN, Nx32N, 16NxN, Nx16N, 8NxN, or Nx8N. Here, N may be a positive integer. Block shape information is information indicating at least one of the shape, orientation, width, and height ratio or size of the encoding unit.

[0109] The shape of the encoding unit may include square and non-square. When the width and height of the encoding unit are the same (i.e., when the block shape of the encoding unit is 4Nx4N), the image decoder (100) may determine the block shape information of the encoding unit as square. The image decoder (100) may determine the shape of the encoding unit as non-square.

[0110] When the width and height of the encoding unit are different (i.e., when the block shape of the encoding unit is 4Nx2N, 2Nx4N, 4NxN, Nx4N, 32NxN, Nx32N, 16NxN, Nx16N, 8NxN, or Nx8N), the image decoder (100) can determine the block shape information of the encoding unit as non-square. When the shape of the encoding unit is non-square, the image decoder (100) can determine the ratio of the width and height among the block shape information of the encoding unit as at least one of 1:2, 2:1, 1:4, 4:1, 1:8, 8:1, 1:16, 16:1, 1:32, or 32:1. Additionally, based on the width and height of the encoding unit, the image decoding device (100) can determine whether the encoding unit is in a horizontal direction or a vertical direction. Additionally, based on at least one of the width, height, or width of the encoding unit, the image decoding device (100) can determine the size of the encoding unit.

[0111] According to one embodiment of the present disclosure, the image decoding device (100) can determine the form of the encoding unit using block form information and can determine how the encoding unit is divided using division form mode information. That is, depending on what block form the block form information used by the image decoding device (100) represents, the method of dividing the encoding unit represented by the division form mode information can be determined.

[0112] The video decoder (100) can obtain split-form mode information from the bitstream. However, it is not limited thereto, and the video decoder (100) and the video encoding device (200) can determine pre-agreed split-form mode information based on block form information. The video decoder (100) can determine pre-agreed split-form mode information for a maximum encoding unit or a minimum encoding unit. For example, the video decoder (100) can determine the split-form mode information for the maximum encoding unit as quad split. Additionally, the video decoder (100) can determine the split-form mode information for the minimum encoding unit as "not split." Specifically, the video decoder (100) can determine the size of the maximum encoding unit to be 256x256. The video decoder (100) can determine the pre-agreed split-form mode information as quad split. Quad split is a split-form mode that divides both the width and height of the encoding unit into two equal parts. The image decoder (100) can obtain a 128x128 size encoding unit from a 256x256 size encoding unit based on the division mode information. Additionally, the image decoder (100) can determine the size of the minimum encoding unit to be 4x4. The image decoder (100) can obtain division mode information indicating "not divided" for the minimum encoding unit.

[0113] According to one embodiment of the present disclosure, the image decoding device (100) may use block shape information indicating that the current encoding unit is in the shape of a square. For example, the image decoding device (100) may determine whether to not divide the square encoding unit, to divide it vertically, to divide it horizontally, or to divide it into four encoding units, etc., according to the division shape mode information. Referring to FIG. 3, when the block shape information of the current encoding unit (300) indicates a square shape, the decoding unit (120) may not divide the encoding unit (310a) having the same size as the current encoding unit (300) according to the division shape mode information indicating that it is not divided, or may determine the divided encoding units (310b, 310c, 310d, 310e, 310f, etc.) based on the division shape mode information indicating a predetermined division method.

[0114] Referring to FIG. 3, the image decoding device (100) can determine two encoding units (310b) that divide the current encoding unit (300) in the vertical direction based on splitting form mode information indicating that it is divided in the vertical direction according to one embodiment of the present disclosure. The image decoding device (100) can determine two encoding units (310c) that divide the current encoding unit (300) in the horizontal direction based on splitting form mode information indicating that it is divided in the horizontal direction. The image decoding device (100) can determine four encoding units (310d) that divide the current encoding unit (300) in the vertical direction and the horizontal direction based on splitting form mode information indicating that it is divided in the vertical direction and the horizontal direction. The image decoding device (100) may determine three encoding units (310e) that divide the current encoding unit (300) in the vertical direction based on division form mode information indicating ternary division in the vertical direction according to one embodiment of the present disclosure. The image decoding device (100) may determine three encoding units (310f) that divide the current encoding unit (300) in the horizontal direction based on division form mode information indicating ternary division in the horizontal direction. However, the division form in which the square encoding unit can be divided should not be interpreted as being limited to the form described above, and may include various forms that the division form mode information can represent. The predetermined division forms in which the square encoding unit is divided will be specifically described below through one embodiment of the present disclosure.

[0115] FIG. 4 illustrates a process in which an image decoding device (100) determines at least one encoding unit by dividing an encoding unit that is in the shape of a non-square according to one embodiment of the present disclosure.

[0116] According to one embodiment of the present disclosure, the image decoding device (100) may use block shape information indicating that the current encoding unit is in a non-square shape. The image decoding device (100) may determine whether to not divide the current encoding unit of the non-square shape or to divide it in a predetermined way according to the division shape mode information. Referring to FIG. 4, when the block shape information of the current encoding unit (400 or 450) indicates a non-square shape, the image decoding device (100) may determine an encoding unit (410 or 460) having the same size as the current encoding unit (400 or 450) according to division shape mode information indicating that it is not divided, or determine divided encoding units (420a, 420b, 430a, 430b, 430c, 470a, 470b, 480a, 480b, 480c) based on division shape mode information indicating a predetermined division method. A predetermined division method in which a non-square encoding unit is divided will be specifically described below through an embodiment of the present disclosure.

[0117] According to one embodiment of the present disclosure, the image decoding device (100) can determine the form in which the encoding unit is divided using the division form mode information, and in this case, the division form mode information may indicate the number of at least one encoding unit generated by dividing the encoding unit. Referring to FIG. 4, when the division form mode information indicates that the current encoding unit (400 or 450) is divided into two encoding units, the image decoding device (100) can determine two encoding units (420a, 420b, or 470a, 470b) included in the current encoding unit by dividing the current encoding unit (400 or 450) based on the division form mode information.

[0118] According to one embodiment of the present disclosure, when an image decoding device (100) divides a current encoding unit (400 or 450) in a non-square shape based on division shape mode information, the image decoding device (100) may divide the current encoding unit by considering the position of the long side of the current encoding unit (400 or 450) in a non-square shape. For example, the image decoding device (100) may determine a plurality of encoding units by dividing the current encoding unit (400 or 450) in a direction that divides the long side of the current encoding unit (400 or 450) by considering the shape of the current encoding unit (400 or 450).

[0119] According to one embodiment of the present disclosure, when the segmentation mode information indicates that the encoding unit is divided into an odd number of blocks (terminal segmentation), the image decoder (100) can determine an odd number of encoding units included in the current encoding unit (400 or 450). For example, when the segmentation mode information indicates that the current encoding unit (400 or 450) is divided into three encoding units, the image decoder (100) can divide the current encoding unit (400 or 450) into three encoding units (430a, 430b, 430c, 480a, 480b, 480c).

[0120] According to one embodiment of the present disclosure, the ratio of the width to the height of the current encoding unit (400 or 450) may be 4:1 or 1:4. When the ratio of the width to the height is 4:1, the block shape information may be in the horizontal direction because the width is longer than the height. When the ratio of the width to the height is 1:4, the block shape information may be in the vertical direction because the width is shorter than the height. The image decoder (100) may determine to divide the current encoding unit into an odd number of blocks based on the division shape mode information. Additionally, the image decoder (100) may determine the division direction of the current encoding unit (400 or 450) based on the block shape information of the current encoding unit (400 or 450). For example, if the current encoding unit (400) is in a vertical direction, the image decoding device (100) can determine the encoding units (430a, 430b, 430c) by dividing the current encoding unit (400) in a horizontal direction. Also, if the current encoding unit (450) is in a horizontal direction, the image decoding device (100) can determine the encoding units (480a, 480b, 480c) by dividing the current encoding unit (450) in a vertical direction.

[0121] According to one embodiment of the present disclosure, an image decoding device (100) may determine an odd number of encoding units included in a current encoding unit (400 or 450), and the sizes of the determined encoding units may not all be the same. For example, among the determined odd number of encoding units (430a, 430b, 430c, 480a, 480b, 480c), the size of a certain encoding unit (430b or 480b) may have a different size from the other encoding units (430a, 430c, 480a, 480c). That is, the current encoding unit (400 or 450) can be divided and determined as a encoding unit, and the encoding unit can have multiple types of sizes, and in some cases, an odd number of encoding units (430a, 430b, 430c, 480a, 480b, 480c) may each have different sizes.

[0122] According to one embodiment of the present disclosure, when the segmentation mode information indicates that the encoding unit is divided into an odd number of blocks, the image decoding device (100) can determine the odd number of encoding units included in the current encoding unit (400 or 450), and furthermore, the image decoding device (100) can impose a predetermined limit on at least one encoding unit among the odd number of encoding units generated by the segmentation. Referring to FIG. 4, the image decoding device (100) can perform the decoding process for the central encoding unit (430b, 480b) among the three encoding units (430a, 430b, 430c, 480a, 480b, 480c) generated by the segmentation of the current encoding unit (400 or 450) differently from the other encoding units (430a, 430c, 480a, 480c). For example, the video decoding device (100) may restrict the centrally located encoding unit (430b, 480b) from being further divided unlike other encoding units (430a, 430c, 480a, 480c), or restrict it to being divided only a predetermined number of times.

[0123] FIG. 5 illustrates a process in which an image decoding device (100) divides a encoding unit based on at least one of block form information and division form mode information according to one embodiment of the present disclosure.

[0124] According to one embodiment of the present disclosure, an image decoding device (100) may determine whether to divide a square-shaped first encoding unit (500) into encoding units or not to divide it based on at least one of block shape information and division shape mode information. According to one embodiment of the present disclosure, if the division shape mode information indicates that the first encoding unit (500) is divided in a horizontal direction, the image decoding device (100) may divide the first encoding unit (500) in a horizontal direction to determine a second encoding unit (510). The first encoding unit, the second encoding unit, and the third encoding unit used according to one embodiment of the present disclosure are terms used to understand the relationship before and after division between the encoding units. For example, if the first encoding unit is divided, the second encoding unit may be determined, and if the second encoding unit is divided, the third encoding unit may be determined. In the following, the relationship between the first encoding unit, the second encoding unit, and the third encoding unit used may be understood as following the features described above.

[0125] According to one embodiment of the present disclosure, the image decoding device (100) may determine whether to divide the determined second encoding unit (510) into encoding units or not to divide it based on the division shape mode information. Referring to FIG. 5, the image decoding device (100) may divide the determined non-square second encoding unit (510) into at least one third encoding unit (520a, 520b, 520c, 520d, etc.) by dividing the first encoding unit (500) based on the division shape mode information, or may not divide the second encoding unit (510). The image decoding device (100) can obtain split-form mode information, and the image decoding device (100) can divide a first encoding unit (500) based on the obtained split-form mode information to divide a plurality of second encoding units (e.g., 510) of various forms, and the second encoding units (510) can be divided according to the method in which the first encoding unit (500) was divided based on the split-form mode information. According to one embodiment of the present disclosure, when the first encoding unit (500) is divided into a second encoding unit (510) based on the split-form mode information for the first encoding unit (500), the second encoding unit (510) can also be divided into a third encoding unit (e.g., 520a, 520b, 520c, 520d, etc.) based on the split-form mode information for the second encoding unit (510). That is, the encoding unit can be recursively partitioned based on partitioning mode information associated with each encoding unit. Thus, a square encoding unit can be determined from a non-square encoding unit, and a non-square encoding unit can be determined by recursively partitioning this square encoding unit.

[0126] Referring to FIG. 5, among the odd number of third encoding units (520b, 520c, 520d) determined by dividing a second encoding unit (510) of a non-square shape, a predetermined encoding unit (e.g., a central encoding unit or a square encoding unit) may be recursively divided. According to one embodiment of the present disclosure, a third encoding unit (520b) of a non-square shape, which is one of the odd number of third encoding units (520b, 520c, 520d), may be divided horizontally into a plurality of fourth encoding units. A fourth encoding unit (530b or 530d) of a non-square shape, which is one of the plurality of fourth encoding units (530a, 530b, 530c, 530d), may again be divided into a plurality of encoding units. For example, a non-square fourth encoding unit (530b or 530d) may be further subdivided into an odd number of encoding units. A method that can be used for the recursive subdivision of encoding units will be described later through an embodiment of the present disclosure.

[0127] According to one embodiment of the present disclosure, the image decoding device (100) may divide each of the third encoding units (520a, 520b, 520c, 520d, etc.) into encoding units based on the division shape mode information. Additionally, the image decoding device (100) may decide not to divide the second encoding unit (510) based on the division shape mode information. According to one embodiment of the present disclosure, the image decoding device (100) may divide the non-square second encoding unit (510) into an odd number of third encoding units (520b, 520c, 520d). The image decoding device (100) may place a certain limit on a certain third encoding unit among the odd number of third encoding units (520b, 520c, 520d). For example, the video decoding device (100) may limit the encoding unit (520c) located in the middle of the odd number of third encoding units (520b, 520c, 520d) so that it is not further divided or is limited to being divided a set number of times.

[0128] Referring to FIG. 5, the image decoding device (100) may limit the middle encoding unit (520c) among the odd number of third encoding units (520b, 520c, 520d) included in the second encoding unit (510) of a non-square shape to no longer be divided, to be divided into a predetermined division form (e.g., divided into only 4 encoding units or divided into a form corresponding to the divided form of the second encoding unit (510)), or to be divided only a predetermined number of times (e.g., divided only n times, n > 0). However, the above limitation on the middle encoding unit (520c) is merely a simple example and should not be interpreted as being limited to the above-described examples, but should be interpreted as including various limitations that allow the middle encoding unit (520c) to be decoded differently from the other encoding units (520b, 520d).

[0129] According to one embodiment of the present disclosure, an image decoding device (100) can obtain splitting form mode information used to split a current encoding unit at a predetermined location within the current encoding unit.

[0130] FIG. 6 illustrates a method for an image decoding device (100) to determine a predetermined encoding unit among an odd number of encoding units according to one embodiment of the present disclosure.

[0131] Referring to FIG. 6, the segmentation mode information of the current encoding unit (600, 650) can be obtained from a sample at a specific location among a plurality of samples included in the current encoding unit (600, 650) (e.g., a sample located in the center (640, 690)). However, the specific location within the current encoding unit (600) where at least one of such segmentation mode information can be obtained should not be interpreted as being limited to the center location shown in FIG. 6, and should be interpreted as including various locations within the current encoding unit (600) (e.g., top, bottom, left, right, top-left, bottom-left, top-right, or bottom-right, etc.). The image decoding device (100) can obtain the segmentation mode information obtained from the specific location and decide whether to divide the current encoding unit into encoding units of various shapes and sizes or not to divide it.

[0132] According to one embodiment of the present disclosure, the image decoding device (100) may select one of the encoding units when the current encoding unit is divided into a predetermined number of encoding units. There may be various methods for selecting one of the plurality of encoding units, and such methods will be described later through one embodiment of the present disclosure.

[0133] According to one embodiment of the present disclosure, the image decoding device (100) can divide the current encoding unit into a plurality of encoding units and determine the encoding unit at a predetermined position.

[0134] According to one embodiment of the present disclosure, an image decoding device (100) may use information indicating the position of each of the odd number of encoding units to determine the encoding unit located in the middle among the odd number of encoding units. Referring to FIG. 6, the image decoding device (100) may divide the current encoding unit (600) or the current encoding unit (650) to determine the odd number of encoding units (620a, 620b, 620c) or the odd number of encoding units (660a, 660b, 660c). The image decoding device (100) may determine the middle encoding unit (620b) or the middle encoding unit (660b) by using information regarding the positions of the odd number of encoding units (620a, 620b, 620c) or the odd number of encoding units (660a, 660b, 660c). For example, the image decoding device (100) can determine the centrally located encoding unit (620b) by determining the positions of the encoding units (620a, 620b, 620c) based on information indicating the positions of a predetermined sample included in the encoding units (620a, 620b, 620c). Specifically, the image decoding device (100) can determine the centrally located encoding unit (620b) by determining the positions of the encoding units (620a, 620b, 620c) based on information indicating the positions of the upper-left samples (630a, 630b, 630c) of the encoding units (620a, 620b, 620c).

[0135] According to one embodiment of the present disclosure, information indicating the location of the upper-left sample (630a, 630b, 630c) included in each of the encoding units (620a, 620b, 620c) may include information regarding the location or coordinates within the picture of the encoding units (620a, 620b, 620c). According to one embodiment of the present disclosure, information indicating the location of the upper-left sample (630a, 630b, 630c) included in each of the encoding units (620a, 620b, 620c) may include information indicating the width or height of the encoding units (620a, 620b, 620c) included in the current encoding unit (600), and such width or height may correspond to information indicating the difference between coordinates within the picture of the encoding units (620a, 620b, 620c). That is, the image decoding device (100) can determine the centrally located encoding unit (620b) by directly using information about the position or coordinates of the encoding units (620a, 620b, 620c) within the picture, or by using information about the width or height of the encoding unit corresponding to the difference value between the coordinates.

[0136] According to one embodiment of the present disclosure, information indicating the location of a sample (630a) at the upper left of an upper encoding unit (620a) may be represented by the (xa, ya) coordinates, information indicating the location of a sample (530b) at the upper left of a middle encoding unit (620b) may be represented by the (xb, yb) coordinates, and information indicating the location of a sample (630c) at the upper left of a lower encoding unit (620c) may be represented by the (xc, yc) coordinates. An image decoding device (100) may determine the middle encoding unit (620b) using the coordinates of the upper left samples (630a, 630b, 630c) included in each of the encoding units (620a, 620b, 620c). For example, when the coordinates of the upper-left samples (630a, 630b, 630c) are sorted in ascending or descending order, the encoding unit (620b) containing the coordinates (xb, yb) of the sample (630b) located in the middle can be determined as the encoding unit located in the middle among the encoding units (620a, 620b, 620c) determined by dividing the current encoding unit (600). However, the coordinates indicating the position of the upper-left samples (630a, 630b, 630c) may represent absolute positions within the picture, and furthermore, based on the position of the upper-left sample (630a) of the upper-left of the upper-left of the middle encoding unit (620b), the (dxb, dyb) coordinates, which represent the relative position of the upper-left sample (630b) of the middle encoding unit (620b), and the (dxc, dyc) coordinates, which represent the relative position of the upper-left sample (630c) of the lower encoding unit (620c) may also be used. In addition, the method of determining the encoding unit of a predetermined position by using the coordinates of the corresponding sample as information indicating the position of the sample included in the encoding unit should not be interpreted as being limited to the method described above, but should be interpreted as various arithmetic methods that can utilize the coordinates of the sample.

[0137] According to one embodiment of the present disclosure, the image decoding device (100) can divide the current encoding unit (600) into a plurality of encoding units (620a, 620b, 620c) and select an encoding unit among the encoding units (620a, 620b, 620c) according to a predetermined criterion. For example, the image decoding device (100) can select an encoding unit (620b) of a different size among the encoding units (620a, 620b, 620c).

[0138] According to one embodiment of the present disclosure, an image decoding device (100) can determine the width or height of each of the encoding units (620a, 620b, 620c) using the (xa, ya) coordinates, which are information indicating the location of a sample (630a) at the top left of the upper encoding unit (620a); the (xb, yb) coordinates, which are information indicating the location of a sample (630b) at the top left of the middle encoding unit (620b); and the (xc, yc) coordinates, which are information indicating the location of a sample (630c) at the top left of the lower encoding unit (620c). The image decoding device (100) can determine the size of each of the encoding units (620a, 620b, 620c) using coordinates (xa, ya), (xb, yb), and (xc, yc) which represent the positions of the encoding units (620a, 620b, 620c). According to one embodiment of the present disclosure, the image decoding device (100) can determine the width of the upper encoding unit (620a) as the width of the current encoding unit (600). The image decoding device (100) can determine the height of the upper encoding unit (620a) as yb-ya. According to one embodiment of the present disclosure, the image decoding device (100) can determine the width of the middle encoding unit (620b) as the width of the current encoding unit (600). The image decoding device (100) can determine the height of the middle encoding unit (620b) as yc-yb. According to one embodiment of the present disclosure, the image decoding device (100) can determine the width or height of the lower encoding unit using the width or height of the current encoding unit and the width and height of the upper encoding unit (620a) and the middle encoding unit (620b). The image decoding device (100) can determine an encoding unit having a different size from other encoding units based on the width and height of the determined encoding units (620a, 620b, 620c).Referring to FIG. 6, the image decoding device (100) can determine a middle encoding unit (620b) having a size different from that of the upper encoding unit (620a) and the lower encoding unit (620c) as the encoding unit of a predetermined position. However, since the process of the image decoding device (100) described above determining the encoding unit having a size different from other encoding units is merely one embodiment of determining the encoding unit of a predetermined position using the size of the encoding unit determined based on sample coordinates, various processes of determining the encoding unit of a predetermined position by comparing the size of the encoding unit determined according to the predetermined sample coordinates may be used.

[0139] The image decoding device (100) can determine the width or height of each of the encoding units (660a, 660b, 660c) using the (xd, yd) coordinates, which are information indicating the location of the upper-left sample (670a) of the left encoding unit (660a), the (xe, ye) coordinates, which are information indicating the location of the upper-left sample (670b) of the middle encoding unit (660b), and the (xf, yf) coordinates, which are information indicating the location of the upper-left sample (670c) of the right encoding unit (660c). The image decoding device (100) can determine the size of each of the encoding units (660a, 660b, 660c) using the (xd, yd), (xe, ye), and (xf, yf) coordinates, which are information indicating the location of the encoding units (660a, 660b, 660c).

[0140] According to one embodiment of the present disclosure, the image decoding device (100) may determine the width of the left encoding unit (660a) as xe-xd. The image decoding device (100) may determine the height of the left encoding unit (660a) as the height of the current encoding unit (650). According to one embodiment of the present disclosure, the image decoding device (100) may determine the width of the middle encoding unit (660b) as xf-xe. The image decoding device (100) may determine the height of the middle encoding unit (660b) as the height of the current encoding unit (600). According to one embodiment of the present disclosure, the image decoding device (100) may determine the width or height of the right encoding unit (660c) using the width or height of the current encoding unit (650) and the width and height of the left encoding unit (660a) and the middle encoding unit (660b). The image decoding device (100) can determine a encoding unit having a different size from other encoding units based on the width and height of the determined encoding units (660a, 660b, 660c). Referring to FIG. 6, the image decoding device (100) can determine a middle encoding unit (660b) having a different size from the left encoding unit (660a) and the right encoding unit (660c) as the encoding unit of a predetermined position. However, since the process of the image decoding device (100) described above determining a encoding unit having a different size from other encoding units is merely one embodiment of determining the encoding unit of a predetermined position using the size of the encoding unit determined based on sample coordinates, various processes of determining the encoding unit of a predetermined position by comparing the size of the encoding unit determined according to the predetermined sample coordinates may be used.

[0141] However, the sample location considered to determine the location of the encoding unit should not be interpreted as being limited to the upper left corner described above, and can be interpreted as allowing the use of information regarding the location of any sample included in the encoding unit.

[0142] According to one embodiment of the present disclosure, the image decoding device (100) may select an encoding unit at a predetermined position among an odd number of encoding units determined by dividing the current encoding unit, taking into consideration the shape of the current encoding unit. For example, if the current encoding unit is a non-square shape where the width is longer than the height, the image decoding device (100) may determine an encoding unit at a predetermined position according to the horizontal direction. That is, the image decoding device (100) may determine one of the encoding units at a different position in the horizontal direction and impose a restriction on that encoding unit. If the current encoding unit is a non-square shape where the height is longer than the width, the image decoding device (100) may determine an encoding unit at a predetermined position according to the vertical direction. That is, the image decoding device (100) may determine one of the encoding units at a different position in the vertical direction and impose a restriction on that encoding unit.

[0143] According to one embodiment of the present disclosure, an image decoding device (100) may use information indicating the location of each of an even number of encoding units to determine an encoding unit at a predetermined location among an even number of encoding units. The image decoding device (100) may determine an even number of encoding units by dividing (binary division) the current encoding unit and may determine an encoding unit at a predetermined location using information regarding the locations of the even number of encoding units. A specific process for this may be omitted as it may correspond to the process of determining an encoding unit at a predetermined location (e.g., a middle location) among an odd number of encoding units described above in FIG. 6.

[0144] According to one embodiment of the present disclosure, when a current encoding unit of a non-square shape is divided into a plurality of encoding units, certain information regarding the encoding unit at a certain position may be used during the division process to determine the encoding unit at a certain position among the plurality of encoding units. For example, an image decoding device (100) may use at least one of block shape information and division shape mode information stored in a sample included in the middle encoding unit during the division process to determine the encoding unit located in the middle among the encoding units into which the current encoding unit is divided into a plurality of encoding units.

[0145] Referring to FIG. 6, the image decoding device (100) can divide the current encoding unit (600) into a plurality of encoding units (620a, 620b, 620c) based on the division form mode information, and can determine the encoding unit (620b) located in the middle among the plurality of encoding units (620a, 620b, 620c). Furthermore, the image decoding device (100) can determine the encoding unit (620b) located in the middle by considering the location where the division form mode information is obtained. That is, the segmentation mode information of the current encoding unit (600) can be obtained from a sample (640) located in the middle of the current encoding unit (600), and based on the segmentation mode information, if the current encoding unit (600) is divided into a plurality of encoding units (620a, 620b, 620c), the encoding unit (620b) containing the sample (640) can be determined as the encoding unit located in the middle. However, the information used to determine the encoding unit located in the middle should not be interpreted as being limited to segmentation mode information, and various types of information may be used in the process of determining the encoding unit located in the middle.

[0146] According to one embodiment of the present disclosure, a predetermined information for identifying a coding unit at a predetermined location may be obtained from a predetermined sample included in the coding unit to be determined. Referring to FIG. 6, an image decoding device (100) may use a segmentation mode information obtained from a sample at a predetermined location within the current coding unit (600) (for example, a sample at the center of the current coding unit (600)) to determine a coding unit at a predetermined location (for example, a coding unit located in the center of the multiple divided coding units) among a plurality of coding units (620a, 620b, 620c) determined by dividing the current coding unit (600). That is, the image decoding device (100) can determine a sample at a predetermined location by considering the block shape of the current encoding unit (600), and the image decoding device (100) can determine a encoding unit (620b) containing a sample from which certain information (e.g., division shape mode information) can be obtained among a plurality of encoding units (620a, 620b, 620c) from which the current encoding unit (600) is divided and determined, and can impose a certain limit. Referring to FIG. 6, according to one embodiment of the present disclosure, the image decoding device (100) can determine a sample (640) located in the middle of the current encoding unit (600) as a sample from which certain information can be obtained, and the image decoding device (100) can impose a certain limit on the encoding unit (620b) containing such a sample (640) during the decoding process. However, the location of the sample from which the specified information can be obtained should not be interpreted as being limited to the location described above, but can be interpreted as samples at any location included in the encoding unit (620b) to be determined for the purpose of imposing a limit.

[0147] According to one embodiment of the present disclosure, the location of a sample from which a predetermined information can be obtained may be determined according to the shape of the current encoding unit (600). According to one embodiment of the present disclosure, block shape information may determine whether the shape of the current encoding unit is square or non-square, and may determine the location of a sample from which a predetermined information can be obtained according to the shape. For example, the image decoding device (100) may determine a sample located on a boundary that divides at least one of the width and height of the current encoding unit in half using at least one of the information regarding the width and height of the current encoding unit as a sample from which a predetermined information can be obtained. As another example, if the block shape information related to the current encoding unit indicates that it is a non-square shape, the image decoding device (100) may determine one of the samples adjacent to the boundary that divides the long side of the current encoding unit in half as a sample from which a predetermined information can be obtained.

[0148] According to one embodiment of the present disclosure, when the image decoding device (100) divides the current encoding unit into a plurality of encoding units, it may use division form mode information to determine the encoding unit at a predetermined position among the plurality of encoding units. According to one embodiment of the present disclosure, the image decoding device (100) may obtain division form mode information from a sample at a predetermined position included in the encoding unit, and the image decoding device (100) may divide the plurality of encoding units generated by dividing the current encoding unit using the division form mode information obtained from a sample at a predetermined position included in each of the plurality of encoding units. That is, the encoding unit may be recursively divided using the division form mode information obtained from a sample at a predetermined position included in each of the encoding units. Since the recursive division process of the encoding unit has been described in detail through FIG. 5, a detailed explanation will be omitted.

[0149] According to one embodiment of the present disclosure, the image decoding device (100) can determine at least one encoding unit by dividing the current encoding unit, and can determine the order in which the at least one encoding unit is decoded according to a predetermined block (e.g., the current encoding unit).

[0150] FIG. 7 illustrates the order in which a plurality of encoding units are processed when an image decoding device (100) determines a plurality of encoding units by dividing a current encoding unit according to one embodiment of the present disclosure.

[0151] According to one embodiment of the present disclosure, the image decoding device (100) may determine a second encoding unit (710a, 710b) by dividing a first encoding unit (700) in a vertical direction according to the splitting form mode information, determine a second encoding unit (730a, 730b) by dividing the first encoding unit (700) in a horizontal direction, or determine a second encoding unit (750a, 750b, 750c, 750d) by dividing the first encoding unit (700) in both a vertical and a horizontal direction.

[0152] Referring to FIG. 7, the image decoding device (100) can determine the order of processing the determined second encoding units (710a, 710b) in the horizontal direction (710c) by dividing the first encoding unit (700) in the vertical direction. The image decoding device (100) can determine the processing order of the determined second encoding units (730a, 730b) in the vertical direction (730c) by dividing the first encoding unit (700) in the horizontal direction. The image decoding device (100) can determine the second encoding unit (750a, 750b, 750c, 750d) determined by dividing the first encoding unit (700) into vertical and horizontal directions, according to a predetermined order in which encoding units located in one row are processed and then encoding units located in the next row are processed (e.g., raster scan order or z scan order (750e), etc.).

[0153] According to one embodiment of the present disclosure, the image decoding device (100) can recursively divide the encoding units. Referring to FIG. 7, the image decoding device (100) can divide the first encoding unit (700) to determine a plurality of encoding units (710a, 710b, 730a, 730b, 750a, 750b, 750c, 750d), and can recursively divide each of the determined plurality of encoding units (710a, 710b, 730a, 730b, 750a, 750b, 750c, 750d). A method of dividing multiple encoding units (710a, 710b, 730a, 730b, 750a, 750b, 750c, 750d) may be a method corresponding to a method of dividing the first encoding unit (700). Accordingly, the multiple encoding units (710a, 710b, 730a, 730b, 750a, 750b, 750c, 750d) may each be independently divided into multiple encoding units. Referring to FIG. 7, the image decoding device (100) may determine the second encoding units (710a, 710b) by dividing the first encoding unit (700) in a vertical direction, and furthermore, may determine whether to independently divide or not divide each of the second encoding units (710a, 710b).

[0154] According to one embodiment of the present disclosure, the image decoding device (100) may divide the second encoding unit (710a) on the left side horizontally into third encoding units (720a, 720b), and may not divide the second encoding unit (710b) on the right side.

[0155] According to one embodiment of the present disclosure, the processing order of the encoding units may be determined based on the process of dividing the encoding units. In other words, the processing order of the divided encoding units may be determined based on the processing order of the encoding units immediately before they are divided. The image decoding device (100) may determine the processing order of the third encoding units (720a, 720b), which are determined by dividing the second encoding unit (710a) on the left, independently of the second encoding unit (710b) on the right. Since the third encoding units (720a, 720b) ​​are determined by dividing the second encoding unit (710a) on the left in a horizontal direction, the third encoding units (720a, 720b) ​​may be processed in a vertical direction (720c). In addition, since the processing order of the second encoding unit (710a) on the left and the second encoding unit (710b) on the right corresponds to the horizontal direction (710c), the third encoding unit (720a, 720b) ​​included in the second encoding unit (710a) on the left can be processed in the vertical direction (720c) before the right encoding unit (710b) is processed. The above description is intended to explain the process in which the processing order of the encoding units is determined according to the encoding unit before division, and therefore should not be interpreted as being limited to the above-described embodiment, but should be interpreted as being used in various ways in which encoding units determined by division in various forms can be processed independently according to a predetermined order.

[0156] FIG. 8 illustrates a process in which, according to one embodiment of the present disclosure, an image decoding device (100) determines that the current encoding unit is divided into an odd number of encoding units when the encoding unit cannot be processed in a predetermined order.

[0157] According to one embodiment of the present disclosure, the image decoding device (100) may determine that the current encoding unit is divided into an odd number of encoding units based on acquired segmentation mode information. Referring to FIG. 8, a square-shaped first encoding unit (800) may be divided into non-square-shaped second encoding units (810a, 810b), and the second encoding units (810a, 810b) may each be independently divided into third encoding units (820a, 820b, 820c, 820d, 820e). According to one embodiment of the present disclosure, the image decoding device (100) can determine a plurality of third encoding units (820a, 820b) by dividing the left encoding unit (810a) among the second encoding units in a horizontal direction, and the right encoding unit (810b) can be divided into an odd number of third encoding units (820c, 820d, 820e).

[0158] According to one embodiment of the present disclosure, the image decoding device (100) can determine whether there exists an odd number of divided encoding units by determining whether the third encoding units (820a, 820b, 820c, 820d, 820e) can be processed in a predetermined order. Referring to FIG. 8, the image decoding device (100) can determine the third encoding units (820a, 820b, 820c, 820d, 820e) by recursively dividing the first encoding unit (800). The video decoding device (100) can determine whether the first encoding unit (800), the second encoding unit (810a, 810b), or the third encoding unit (820a, 820b, 820c, 820d, 820e) are divided into an odd number of encoding units based on at least one of block form information and division form mode information. For example, the encoding unit located on the right among the second encoding units (810a, 810b) may be divided into an odd number of third encoding units (820c, 820d, 820e). The order in which a plurality of encoding units included in the first encoding unit (800) are processed may be a predetermined order (e.g., z-scan order (830)), and the image decoding device (100) may determine whether the right second encoding unit (810b) is divided into an odd number of determined third encoding units (820c, 820d, 820e) can be processed according to the predetermined order.

[0159] According to one embodiment of the present disclosure, an image decoding device (100) can determine whether a third encoding unit (820a, 820b, 820c, 820d, 820e) included in a first encoding unit (800) satisfies a condition that the third encoding unit (820a, 820b, 820c, 820d, 820e) can be processed in a predetermined order, and the condition relates to whether at least one of the width and height of the second encoding unit (810a, 810b) is divided in half according to the boundary of the third encoding unit (820a, 820b, 820c, 820d, 820e). For example, the third encoding unit (820a, 820b) determined by dividing the height of the left second encoding unit (810a) in a non-square shape in half may satisfy the condition. Since the boundaries of the third encoding units (820c, 820d, 820e), which are determined by dividing the right second encoding unit (810b) into three encoding units, do not divide the width or height of the right second encoding unit (810b) in half, the third encoding units (820c, 820d, 820e) may be determined not to satisfy the condition. In the case of such non-satisfaction of the condition, the image decoding device (100) determines that there is a disconnection in the scan order, and based on the result of the determination, the right second encoding unit (810b) may be determined to be divided into an odd number of encoding units. According to one embodiment of the present disclosure, when the image decoding device (100) is divided into an odd number of encoding units, it may place a certain restriction on the encoding unit at a certain position among the divided encoding units, and since the details of such restriction or the certain position, etc. have been described in detail through one embodiment of the present disclosure, a detailed explanation will be omitted.

[0160] FIG. 9 illustrates a process in which an image decoding device (100) divides a first encoding unit (900) to determine at least one encoding unit according to one embodiment of the present disclosure.

[0161] According to one embodiment of the present disclosure, an image decoding device (100) may divide a first encoding unit (900) based on division shape mode information obtained through a bitstream acquisition unit (110). A square-shaped first encoding unit (900) may be divided into four square-shaped encoding units or into a plurality of non-square-shaped encoding units. For example, referring to FIG. 9, if the first encoding unit (900) is square and the division shape mode information indicates that it is divided into non-square encoding units, the image decoding device (100) may divide the first encoding unit (900) into a plurality of non-square encoding units. Specifically, when the splitting mode information indicates that the first encoding unit (900) is divided in a horizontal or vertical direction to determine an odd number of encoding units, the image decoding device (100) can divide the square-shaped first encoding unit (900) into an odd number of encoding units, such as a second encoding unit (910a, 910b, 910c) determined by dividing in a vertical direction or a second encoding unit (920a, 920b, 920c) determined by dividing in a horizontal direction.

[0162] According to one embodiment of the present disclosure, an image decoding device (100) can determine whether a second encoding unit (910a, 910b, 910c, 920a, 920b, 920c) included in a first encoding unit (900) satisfies a condition that the second encoding unit (910a, 910b, 910c, 920a, 920b, 920c) can be processed in a predetermined order, and the condition relates to whether at least one of the width and height of the first encoding unit (900) is divided in half according to the boundary of the second encoding unit (910a, 910b, 910c, 920a, 920b, 920c). Referring to FIG. 9, the boundaries of the second encoding units (910a, 910b, 910c), which are determined by dividing the square-shaped first encoding unit (900) in the vertical direction, do not divide the width of the first encoding unit (900) in half, so the first encoding unit (900) may be determined not to satisfy the condition of being processed in a predetermined order. Additionally, the boundaries of the second encoding units (920a, 920b, 920c), which are determined by dividing the square-shaped first encoding unit (900) in the horizontal direction, do not divide the height of the first encoding unit (900) in half, so the first encoding unit (900) may be determined not to satisfy the condition of being processed in a predetermined order. The image decoding device (100) determines that if these conditions are not satisfied, there is a disconnection in the scan order, and based on the result of the determination, the first encoding unit (900) may be divided into an odd number of encoding units. According to one embodiment of the present disclosure, when the image decoding device (100) is divided into an odd number of encoding units, it may place a certain restriction on the encoding unit at a certain position among the divided encoding units. Since the details of such restriction or the certain position, etc., have been described in detail through one embodiment of the present disclosure, a detailed explanation will be omitted.

[0163] According to one embodiment of the present disclosure, the image decoding device (100) can divide the first encoding unit to determine various forms of encoding units.

[0164] Referring to FIG. 9, the image decoding device (100) can divide a square-shaped first encoding unit (900) and a non-square-shaped first encoding unit (930 or 950) into various types of encoding units.

[0165] FIG. 10 illustrates that, according to one embodiment of the present disclosure, when a second encoding unit in a non-square shape determined by dividing a first encoding unit (1000) satisfies a predetermined condition, the shape in which the second encoding unit can be divided is limited.

[0166] According to one embodiment of the present disclosure, an image decoding device (100) may decide to divide a square-shaped first encoding unit (1000) into non-square-shaped second encoding units (1010a, 1010b, 1020a, 1020b) based on division shape mode information obtained through a bitstream acquisition unit (110). The second encoding units (1010a, 1010b, 1020a, 1020b) may be divided independently. Accordingly, the image decoding device (100) may decide to divide into a plurality of encoding units or not divide based on division shape mode information related to each of the second encoding units (1010a, 1010b, 1020a, 1020b). According to one embodiment of the present disclosure, the image decoding device (100) may determine a third encoding unit (1012a, 1012b) by dividing a left second encoding unit (1010a), which is a non-square shape determined by dividing a first encoding unit (1000) in a vertical direction, in a horizontal direction. However, when the image decoding device (100) divides the left second encoding unit (1010a) in a horizontal direction, the right second encoding unit (1010b) may be restricted so that it cannot be divided in the same horizontal direction as the left second encoding unit (1010a) was divided. If the right second encoding unit (1010b) is divided in the same direction to determine the third encoding unit (1014a, 1014b), the left second encoding unit (1010a) and the right second encoding unit (1010b) may be divided independently in the horizontal direction to determine the third encoding unit (1012a, 1012b, 1014a, 1014b). However, this is the same result as the image decoding device (100) dividing the first encoding unit (1000) into four square-shaped second encoding units (1030a, 1030b, 1030c, 1030d) based on the division shape mode information, and this may be inefficient in terms of image decoding.

[0167] According to one embodiment of the present disclosure, the image decoding device (100) may determine a third encoding unit (1022a, 1022b, 1024a, 1024b) by dividing a first encoding unit (1000) in a horizontal direction and a second encoding unit (1020a or 1020b) in a non-square shape in a vertical direction. However, if the image decoding device (100) divides one of the second encoding units (e.g., the upper second encoding unit (1020a)) in a vertical direction, it may restrict the other second encoding unit (e.g., the lower encoding unit (1020b)) from being divided in the same vertical direction as the upper second encoding unit (1020a) for the reasons described above.

[0168] FIG. 11 illustrates the process of a video decoder (100) dividing square-shaped encoding units when the divided shape mode information cannot be divided into four square-shaped encoding units according to one embodiment of the present disclosure.

[0169] According to one embodiment of the present disclosure, an image decoding device (100) can determine a second encoding unit (1110a, 1110b, 1120a, 1120b, etc.) by dividing a first encoding unit (1100) based on division form mode information. The division form mode information may include information on various forms in which the encoding unit can be divided, but the information on various forms may not include information for dividing into four square-shaped encoding units. According to such division form mode information, the image decoding device (100) cannot divide the square-shaped first encoding unit (1100) into four square-shaped second encoding units (1130a, 1130b, 1130c, 1130d). Based on the segmented form mode information, the image decoding device (100) can determine a non-square second encoding unit (1110a, 1110b, 1120a, 1120b, etc.).

[0170] According to one embodiment of the present disclosure, the image decoding device (100) can independently divide each of the second encoding units (1110a, 1110b, 1120a, 1120b, etc.) in a non-square shape. Each of the second encoding units (1110a, 1110b, 1120a, 1120b, etc.) can be divided in a predetermined order through a recursive method, and this may be a division method corresponding to the method in which the first encoding unit (1100) is divided based on division shape mode information.

[0171] For example, the image decoding device (100) can determine a square-shaped third encoding unit (1112a, 1112b) by dividing the left second encoding unit (1110a) in a horizontal direction, and can determine a square-shaped third encoding unit (1114a, 1114b) by dividing the right second encoding unit (1110b) in a horizontal direction. Furthermore, the image decoding device (100) can determine a square-shaped third encoding unit (1116a, 1116b, 1116c, 1116d) by dividing both the left second encoding unit (1110a) and the right second encoding unit (1110b) in a horizontal direction. In this case, the encoding unit can be determined in the same form as the first encoding unit (1100) being divided into four square-shaped second encoding units (1130a, 1130b, 1130c, 1130d).

[0172] As another example, the video decoding device (100) may determine a square-shaped third encoding unit (1122a, 1122b) by dividing the upper second encoding unit (1120a) in a vertical direction, and determine a square-shaped third encoding unit (1124a, 1124b) by dividing the lower second encoding unit (1120b) in a vertical direction. Furthermore, the video decoding device (100) may determine a square-shaped third encoding unit (1126a, 1126b, 1126a, 1126b) by dividing both the upper second encoding unit (1120a) and the lower second encoding unit (1120b) in a vertical direction. In this case, the encoding unit can be determined in the same form as the first encoding unit (1100) being divided into four square-shaped second encoding units (1130a, 1130b, 1130c, 1130d).

[0173] FIG. 12 illustrates that, according to one embodiment of the present disclosure, the processing order between a plurality of encoding units may vary depending on the division process of the encoding unit.

[0174] According to one embodiment of the present disclosure, an image decoding device (100) may divide a first encoding unit (1200) based on division shape mode information. When the block shape is square and the division shape mode information indicates that the first encoding unit (1200) is divided in at least one of a horizontal direction and a vertical direction, the image decoding device (100) may divide the first encoding unit (1200) to determine a second encoding unit (e.g., 1210a, 1210b, 1220a, 1220b, etc.). Referring to FIG. 12, the second encoding units (1210a, 1210b, 1220a, 1220b) in a non-square shape determined by dividing the first encoding unit (1200) only in a horizontal direction or a vertical direction may be divided independently based on the division shape mode information for each. For example, the video decoding device (100) can determine the third encoding unit (1216a, 1216b, 1216c, 1216d) by dividing the second encoding unit (1210a, 1210b) generated by dividing the first encoding unit (1200) in the vertical direction, and can determine the third encoding unit (1226a, 1226b, 1226c, 1226d) by dividing the second encoding unit (1220a, 1220b) generated by dividing the first encoding unit (1200) in the horizontal direction, respectively, in the vertical direction. Since the process of dividing these second encoding units (1210a, 1210b, 1220a, 1220b) has been described in detail in relation to FIG. 11, a detailed explanation will be omitted.

[0175] According to one embodiment of the present disclosure, the image decoding device (100) can process encoding units in a predetermined order. Since the characteristics of processing encoding units in a predetermined order have been described in detail in relation to FIG. 7, a detailed description will be omitted. Referring to FIG. 12, the image decoding device (100) can divide a square-shaped first encoding unit (1200) to determine four square-shaped third encoding units (1216a, 1216b, 1216c, 1216d, 1226a, 1226b, 1226c, 1226d). According to one embodiment of the present disclosure, the image decoding device (100) can determine the processing order of the third encoding unit (1216a, 1216b, 1216c, 1216d, 1226a, 1226b, 1226c, 1226d) according to the form in which the first encoding unit (1200) is divided.

[0176] According to one embodiment of the present disclosure, the image decoding device (100) can determine the third encoding unit (1216a, 1216b, 1216c, 1216d) by dividing the second encoding unit (1210a, 1210b) generated by dividing in a vertical direction into the horizontal direction, and the image decoding device (100) can process the third encoding unit (1216a, 1216c) included in the left second encoding unit (1210a) first in a vertical direction, and then process the third encoding unit (1216b, 1216d) included in the right second encoding unit (1210b) in a vertical direction according to the order (1217).

[0177] According to one embodiment of the present disclosure, the image decoding device (100) can determine the third encoding unit (1226a, 1226b, 1226c, 1226d) by dividing the second encoding unit (1220a, 1220b) generated by dividing in a horizontal direction into the vertical direction, and the image decoding device (100) can process the third encoding unit (1226a, 1226b, 1226c, 1226d) according to the order (1227) of first processing the third encoding unit (1226a, 1226b) included in the upper second encoding unit (1220a) in a horizontal direction, and then processing the third encoding unit (1226c, 1226d) included in the lower second encoding unit (1220b) in a horizontal direction.

[0178] Referring to FIG. 12, the second encoding units (1210a, 1210b, 1220a, 1220b) are each divided to determine the square-shaped third encoding units (1216a, 1216b, 1216c, 1216d, 1226a, 1226b, 1226c, 1226d). The second encoding unit (1210a, 1210b) determined by dividing in the vertical direction and the second encoding unit (1220a, 1220b) determined by dividing in the horizontal direction are divided into different forms, but according to the third encoding unit (1216a, 1216b, 1216c, 1216d, 1226a, 1226b, 1226c, 1226d) determined thereafter, the result is that the first encoding unit (1200) is divided into encoding units of the same form. Accordingly, the image decoding device (100) recursively divides the encoding unit through different processes based on the division form mode information, so that even if the encoding units of the same form are determined as a result, the multiple encoding units determined in the same form can be processed in different orders.

[0179] FIG. 13 illustrates a process in which, according to one embodiment of the present disclosure, a encoding unit is recursively divided to determine a plurality of encoding units, and the depth of the encoding unit is determined as the shape and size of the encoding unit change.

[0180] According to one embodiment of the present disclosure, the image decoding device (100) may determine the depth of an encoding unit according to a predetermined standard. For example, the predetermined standard may be the length of the long side of the encoding unit. The image decoding device (100) may determine that if the length of the long side of the current encoding unit is divided by 2n (n>0) times the length of the long side of the encoding unit before division, the depth of the current encoding unit is increased by n compared to the depth of the encoding unit before division. In the following, the encoding unit with increased depth is expressed as an encoding unit of lower depth.

[0181] Referring to FIG. 13, according to one embodiment of the present disclosure, based on block shape information indicating that it is a square shape (for example, the block shape information may indicate '0: SQUARE'), an image decoding device (100) can determine a second encoding unit (1302), a third encoding unit (1304), etc. of a lower depth by dividing a first encoding unit (1300) that is square in shape. If the size of the first encoding unit (1300) that is square in shape is 2Nx2N, the second encoding unit (1302), which is determined by dividing the width and height of the first encoding unit (1300) by half, may have a size of NxN. Furthermore, the third encoding unit (1304), which is determined by dividing the width and height of the second encoding unit (1302) by half, may have a size of N / 2xN / 2. In this case, the width and height of the third encoding unit (1304) correspond to 1 / 4 times the width and height of the first encoding unit (1300). When the depth of the first encoding unit (1300) is D, the depth of the second encoding unit (1302), which is 1 / 2 times the width and height of the first encoding unit (1300), may be D+1, and the depth of the third encoding unit (1304), which is 1 / 4 times the width and height of the first encoding unit (1300), may be D+2.

[0182] According to one embodiment of the present disclosure, based on block shape information representing a non-square shape (for example, the block shape information may represent '1: NS_VER' indicating that the height is longer than the width, or '2: NS_HOR' indicating that the width is longer than the height), an image decoding device (100) may divide a first encoding unit (1310 or 1320) that is a non-square shape to determine a second encoding unit (1312 or 1322), a third encoding unit (1314 or 1324), etc. of a lower depth.

[0183] The image decoding device (100) can determine a second encoding unit (e.g., 1302, 1312, 1322, etc.) by dividing at least one of the width and height of a first encoding unit (1310) of size Nx2N. That is, the image decoding device (100) can determine a second encoding unit (1302) of size NxN or a second encoding unit (1322) of size NxN / 2 by dividing the first encoding unit (1310) in a horizontal direction, and can also determine a second encoding unit (1312) of size N / 2xN by dividing it in a horizontal direction and a vertical direction.

[0184] According to one embodiment of the present disclosure, the image decoding device (100) may determine a second encoding unit (e.g., 1302, 1312, 1322, etc.) by dividing at least one of the width and height of a first encoding unit (1320) of size 2NxN. That is, the image decoding device (100) may determine a second encoding unit (1302) of size NxN or a second encoding unit (1312) of size N / 2xN by dividing the first encoding unit (1320) in the vertical direction, and may determine a second encoding unit (1322) of size NxN / 2 by dividing it in the horizontal and vertical directions.

[0185] According to one embodiment of the present disclosure, the image decoding device (100) may determine a third encoding unit (e.g., 1304, 1314, 1324, etc.) by dividing at least one of the width and height of a second encoding unit (1302) of size NxN. That is, the image decoding device (100) may determine a third encoding unit (1304) of size N / 2xN / 2 by dividing the second encoding unit (1302) in a vertical direction and a horizontal direction, or determine a third encoding unit (1314) of size N / 4xN / 2, or determine a third encoding unit (1324) of size N / 2xN / 4.

[0186] According to one embodiment of the present disclosure, the image decoding device (100) may determine a third encoding unit (e.g., 1304, 1314, 1324, etc.) by dividing at least one of the width and height of a second encoding unit (1312) of size N / 2xN. That is, the image decoding device (100) may determine a third encoding unit (1304) of size N / 2xN / 2 or a third encoding unit (1324) of size N / 2xN / 4 by dividing the second encoding unit (1312) in a horizontal direction, or determine a third encoding unit (1314) of size N / 4xN / 2 by dividing it in a vertical direction and a horizontal direction.

[0187] According to one embodiment of the present disclosure, the image decoding device (100) may determine a third encoding unit (e.g., 1304, 1314, 1324, etc.) by dividing at least one of the width and height of a second encoding unit (1322) of size NxN / 2. That is, the image decoding device (100) may determine a third encoding unit (1304) of size N / 2xN / 2 or a third encoding unit (1314) of size N / 4xN / 2 by dividing the second encoding unit (1322) in the vertical direction, or determine a third encoding unit (1324) of size N / 2xN / 4 by dividing it in the vertical and horizontal directions.

[0188] According to one embodiment of the present disclosure, an image decoding device (100) may divide square-shaped encoding units (e.g., 1300, 1302, 1304) in a horizontal or vertical direction. For example, a first encoding unit (1300) of size 2Nx2N may be divided in a vertical direction to determine a first encoding unit (1310) of size Nx2N, or divided in a horizontal direction to determine a first encoding unit (1320) of size 2NxN. According to one embodiment of the present disclosure, when the depth is determined based on the length of the longest side of the encoding unit, the depth of the encoding unit determined by dividing the first encoding unit (1300) of size 2Nx2N in a horizontal or vertical direction may be the same as the depth of the first encoding unit (1300).

[0189] According to one embodiment of the present disclosure, the width and height of the third encoding unit (1314 or 1324) may correspond to 1 / 4 times the width and height of the first encoding unit (1310 or 1320). If the depth of the first encoding unit (1310 or 1320) is D, the depth of the second encoding unit (1312 or 1322), which is 1 / 2 times the width and height of the first encoding unit (1310 or 1320), may be D+1, and the depth of the third encoding unit (1314 or 1324), which is 1 / 4 times the width and height of the first encoding unit (1310 or 1320), may be D+2.

[0190] FIG. 14 illustrates a depth and part index (hereinafter PID) for distinguishing between coding units that can be determined according to the shape and size of the coding units according to one embodiment of the present disclosure.

[0191] According to one embodiment of the present disclosure, an image decoding device (100) can determine various shapes of second encoding units by dividing a square-shaped first encoding unit (1400). Referring to FIG. 14, the image decoding device (100) can determine second encoding units (1402a, 1402b, 1404a, 1404b, 1406a, 1406b, 1406c, 1406d) by dividing the first encoding unit (1400) in at least one of a vertical direction and a horizontal direction according to the division shape mode information. That is, the image decoding device (100) can determine the second encoding unit (1402a, 1402b, 1404a, 1404b, 1406a, 1406b, 1406c, 1406d) based on the segmented form mode information for the first encoding unit (1400).

[0192] According to one embodiment of the present disclosure, the depth of the second encoding unit (1402a, 1402b, 1404a, 1404b, 1406a, 1406b, 1406c, 1406d), which is determined according to the segmented shape mode information for the first encoding unit (1400) in a square shape, can be determined based on the length of the longer side. For example, since the length of one side of the first encoding unit (1400) in a square shape and the length of the longer side of the second encoding unit (1402a, 1402b, 1404a, 1404b) in a non-square shape are the same, the depth of the first encoding unit (1400) and the second encoding unit (1402a, 1402b, 1404a, 1404b) in a non-square shape can be considered to be the same as D. In contrast, when the video decoding device (100) divides the first encoding unit (1400) into four square-shaped second encoding units (1406a, 1406b, 1406c, 1406d) based on the division shape mode information, since the length of one side of the square-shaped second encoding units (1406a, 1406b, 1406c, 1406d) is half the length of one side of the first encoding unit (1400), the depth of the second encoding units (1406a, 1406b, 1406c, 1406d) may be a depth of D+1, which is one depth lower than the depth D of the first encoding unit (1400).

[0193] According to one embodiment of the present disclosure, an image decoding device (100) may divide a first encoding unit (1410), in which the height is longer than the width, into a plurality of second encoding units (1412a, 1412b, 1414a, 1414b, 1414c) by dividing it in a horizontal direction according to the division shape mode information. According to one embodiment of the present disclosure, an image decoding device (100) may divide a first encoding unit (1420), in which the width is longer than the height, into a plurality of second encoding units (1422a, 1422b, 1424a, 1424b, 1424c) by dividing it in a vertical direction according to the division shape mode information.

[0194] According to one embodiment of the present disclosure, a second encoding unit (1412a, 1412b, 1414a, 1414b, 1414c, 1422a, 1422b, 1424a, 1424b, 1424c) determined according to the segmented shape mode information for a first encoding unit (1410 or 1420) of a non-square shape may have its depth determined based on the length of the longer side. For example, since the length of one side of the square-shaped second encoding unit (1412a, 1412b) is half the length of one side of the non-square-shaped first encoding unit (1410) in which the height is longer than the width, the depth of the square-shaped second encoding unit (1412a, 1412b) is D+1, which is one depth lower than the depth D of the non-square-shaped first encoding unit (1410).

[0195] Furthermore, the image decoding device (100) may divide a first encoding unit (1410) in a non-square shape into an odd number of second encoding units (1414a, 1414b, 1414c) based on the division shape mode information. The odd number of second encoding units (1414a, 1414b, 1414c) may include a second encoding unit (1414a, 1414c) in a non-square shape and a second encoding unit (1414b) in a square shape. In this case, since the length of the longer side of the non-square second encoding unit (1414a, 1414c) and the length of one side of the square second encoding unit (1414b) are half the length of one side of the first encoding unit (1410), the depth of the second encoding unit (1414a, 1414b, 1414c) may be a depth of D+1, which is one depth lower than the depth D of the first encoding unit (1410). The image decoding device (100) may determine the depth of the encoding units associated with the non-square first encoding unit (1420), in a manner corresponding to the above method of determining the depth of the encoding units associated with the first encoding unit (1410).

[0196] According to one embodiment of the present disclosure, an image decoding device (100) may determine an index (PID) for distinguishing divided encoding units, and when the odd number of divided encoding units are not of the same size, the index may be determined based on the size ratio between the encoding units. Referring to FIG. 14, among the odd number of divided encoding units (1414a, 1414b, 1414c), the encoding unit (1414b) located in the middle may have the same width as the other encoding units (1414a, 1414c) but may have a height twice that of the other encoding units (1414a, 1414c). That is, in this case, the encoding unit (1414b) located in the middle may include two of the other encoding units (1414a, 1414c). Accordingly, if the index (PID) of the encoding unit (1414b) located in the middle according to the scan order is 1, the encoding unit (1414c) located in the next order may have an index of 3, which is an increase of 2. That is, there may be a discontinuity in the value of the index. According to one embodiment of the present disclosure, the image decoding device (100) may determine whether the encoding units divided into an odd number are not of the same size based on whether there is a discontinuity in the index for distinguishing between these divided encoding units.

[0197] According to one embodiment of the present disclosure, an image decoding device (100) may determine whether a plurality of encoding units determined by dividing from a current encoding unit are divided into a specific division form based on the value of an index for distinguishing the plurality of encoding units. Referring to FIG. 14, the image decoding device (100) may divide a first encoding unit (1410) in the shape of a rectangle whose height is greater than its width to determine an even number of encoding units (1412a, 1412b) or an odd number of encoding units (1414a, 1414b, 1414c). The image decoding device (100) may use an index (PID) representing each encoding unit to distinguish each of the plurality of encoding units. According to one embodiment of the present disclosure, the PID may be obtained from a sample at a predetermined position of each encoding unit (e.g., the upper left sample).

[0198] According to one embodiment of the present disclosure, an image decoding device (100) may determine a coding unit at a predetermined position among the coding units determined by division using an index for distinguishing coding units. According to one embodiment of the present disclosure, if the division shape mode information for a first coding unit (1410) in the form of a rectangle whose height is longer than its width indicates that it is divided into three coding units, the image decoding device (100) may divide the first coding unit (1410) into three coding units (1414a, 1414b, 1414c). The image decoding device (100) may assign an index to each of the three coding units (1414a, 1414b, 1414c). The image decoding device (100) may compare the indices for each coding unit to determine the middle coding unit among the odd number of divided coding units. The image decoding device (100) may determine a encoding unit (1414b) having an index corresponding to the middle value among the indices based on the indices of the encoding units, as the encoding unit at the middle position among the encoding units determined by dividing the first encoding unit (1410). According to one embodiment of the present disclosure, when determining an index for distinguishing the divided encoding units, the image decoding device (100) may determine the index based on the size ratio between the encoding units if the encoding units are not of the same size. Referring to FIG. 14, the encoding unit (1414b) generated by dividing the first encoding unit (1410) may have the same width as the other encoding units (1414a, 1414c) but may be twice the height of the encoding units (1414a, 1414c) which have different heights. In this case, if the index (PID) of the encoding unit (1414b) located in the middle is 1, the encoding unit (1414c) located in the next order may have an index of 3, which is increased by 2.In cases where the index increases uniformly and then the rate of increase changes, such as in this case, the image decoding device (100) may determine that the current encoding unit is divided into a plurality of encoding units, including encoding units having different sizes from other encoding units. In the case where the division type mode information indicates that the current encoding unit is divided into an odd number of encoding units according to one embodiment of the present disclosure, the image decoding device (100) may divide the current encoding unit in such a way that the encoding unit at a predetermined position among the odd number of encoding units (e.g., the middle encoding unit) has a different size from other encoding units. In this case, the image decoding device (100) may determine the middle encoding unit having a different size by using an index (PID) for the encoding unit. However, the aforementioned index, the size or position of the encoding unit at a predetermined position to be determined, is specific for the purpose of explaining one embodiment and should not be interpreted as being limited thereto, and should be interpreted as allowing various indices, positions, and sizes of encoding units to be used.

[0199] According to one embodiment of the present disclosure, an image decoding device (100) may use a predetermined data unit in which recursive division of the encoding unit begins.

[0200] FIG. 15 illustrates that a plurality of encoding units are determined according to a plurality of predetermined data units included in a picture according to one embodiment of the present disclosure.

[0201] According to one embodiment of the present disclosure, a predetermined data unit may be defined as a data unit in which an encoding unit begins to recursively divide using partitioning mode information. That is, it may correspond to the highest depth encoding unit used in the process of determining a plurality of encoding units that divide the current picture. For convenience of explanation, such a predetermined data unit will be referred to as a reference data unit below.

[0202] According to one embodiment of the present disclosure, a reference data unit may have a predetermined size and shape. According to one embodiment of the present disclosure, the reference data unit may include MxN samples. Here, M and N may be the same and may be integers expressed as powers of 2. That is, the reference data unit may have a square or non-square shape and may subsequently be divided into an integer number of encoding units.

[0203] According to one embodiment of the present disclosure, the image decoding device (100) can divide the current picture into a plurality of reference data units. According to one embodiment of the present disclosure, the image decoding device (100) can divide the current picture into a plurality of reference data units using division form mode information for each reference data unit. This division process of reference data units may correspond to a division process using a quad-tree structure.

[0204] According to one embodiment of the present disclosure, the image decoding device (100) can predetermine the minimum size that a reference data unit included in the current picture may have. Accordingly, the image decoding device (100) can determine reference data units of various sizes having a size greater than or equal to the minimum size, and can determine at least one encoding unit using segmented form mode information based on the determined reference data unit.

[0205] Referring to FIG. 15, the image decoding device (100) may use a square-shaped reference encoding unit (1500) or a non-square-shaped reference encoding unit (1502). According to one embodiment of the present disclosure, the shape and size of the reference encoding unit may be determined according to various data units (e.g., sequence, picture, slice, slice segment, tile, tile group, maximum encoding unit, etc.) that may include at least one reference encoding unit.

[0206] According to one embodiment of the present disclosure, the bitstream acquisition unit (110) of the image decoding device (100) can acquire at least one of information regarding the shape of the reference encoding unit and information regarding the size of the reference encoding unit from the bitstream for each of the various data units. The process of determining at least one encoding unit included in the square-shaped reference encoding unit (1500) has been described in detail through the process of dividing the current encoding unit (300) of FIG. 3, and the process of determining at least one encoding unit included in the non-square-shaped reference encoding unit (1502) has been described in detail through the process of dividing the current encoding unit (400 or 450) of FIG. 4, so a detailed explanation is omitted.

[0207] According to one embodiment of the present disclosure, an image decoding device (100) may use an index to identify the size and shape of a reference coding unit in order to determine the size and shape of a reference coding unit according to a portion of data units that are predetermined based on a predetermined condition. That is, a bitstream acquisition unit (110) may acquire only an index for identifying the size and shape of a reference coding unit for each slice, slice segment, tile, tile group, maximum coding unit, etc., among the various data units (e.g., sequence, picture, slice, slice segment, tile, tile group, maximum coding unit, etc.) from the bitstream, as a data unit that satisfies a predetermined condition (e.g., a data unit having a size smaller than or equal to a slice). By using the index, the image decoding device (100) may determine the size and shape of a reference data unit for each data unit that satisfies the predetermined condition. When information regarding the form of the reference encoding unit and information regarding the size of the reference encoding unit are obtained from the bitstream for each data unit of a relatively small size and used, the utilization efficiency of the bitstream may be poor; therefore, instead of directly obtaining information regarding the form of the reference encoding unit and information regarding the size of the reference encoding unit, only the index may be obtained and used. In this case, at least one of the size and form of the reference encoding unit corresponding to the index representing the size and form of the reference encoding unit may be predetermined. That is, the image decoding device (100) can determine at least one of the size and form of the reference encoding unit included in the data unit that serves as the basis for obtaining the index by selecting at least one of the predetermined size and form of the reference encoding unit according to the index.

[0208] According to one embodiment of the present disclosure, the image decoding device (100) may utilize at least one reference encoding unit included in one maximum encoding unit (1510). That is, the maximum encoding unit (1510) that divides the image may include at least one reference encoding unit, and the encoding unit may be determined through a recursive division process of each reference encoding unit. According to one embodiment of the present disclosure, at least one of the width and height of the maximum encoding unit (1510) may correspond to an integer multiple of at least one of the width and height of the reference encoding unit. According to one embodiment of the present disclosure, the size of the reference encoding unit may be the size obtained by dividing the maximum encoding unit (1510) n times according to a quad tree structure. That is, the image decoding device (100) may determine the reference encoding unit by dividing the maximum encoding unit (1510) n times according to a quad tree structure, and according to one embodiment of the present disclosure, the reference encoding unit may be divided based on at least one of block shape information and division shape mode information.

[0209] According to one embodiment of the present disclosure, an image decoding device (100) may obtain and use block form information indicating the form of a current encoding unit or division form mode information indicating a method of dividing a current encoding unit from a bitstream. The division form mode information may be included in bitstreams associated with various data units. For example, the image decoding device (100) may use division form mode information included in a sequence parameter set (SPS), a picture parameter set (PPS), a video parameter set (VPS), a slice header, a slice segment header, a tile header, and a tile group header. Furthermore, the image decoding device (100) may obtain and use syntax elements corresponding to the block form information or division form mode information from the bitstream for each maximum encoding unit and reference encoding unit.

[0210] A method for determining a division rule according to one embodiment of the present disclosure will be described in detail below.

[0211] The image decoding device (100) can determine the segmentation rule of the image. The segmentation rule may be predetermined between the image decoding device (100) and the image encoding device (200). The image decoding device (100) can determine the segmentation rule of the image based on information obtained from a bitstream. The image decoding device (100) can determine the segmentation rule based on information obtained from at least one of a sequence parameter set, a picture parameter set, a video parameter set, a slice header, a slice segment header, a tile header, and a tile group header. The image decoding device (100) can determine the segmentation rule differently depending on the frame, slice, tile, temporal layer, maximum encoding unit, or encoding unit.

[0212] The image decoding device (100) can determine a partitioning rule based on the block shape of the encoding unit. The block shape may include the size, shape, ratio of width and height, and orientation of the encoding unit. The image encoding device (200) and the image decoding device (100) may predetermine to determine a partitioning rule based on the block shape of the encoding unit. However, they are not limited thereto. The image decoding device (100) can determine a partitioning rule based on information obtained from a bitstream received from the image encoding device (200).

[0213] The shape of the encoding unit may include square and non-square. If the width and height of the encoding unit are the same, the image decoder (100) may determine the shape of the encoding unit as square. Additionally, if the width and height of the encoding unit are not the same, the image decoder (100) may determine the shape of the encoding unit as non-square.

[0214] The size of the encoding unit may include various sizes such as 4x4, 8x4, 4x8, 8x8, 16x4, 16x8, ..., 256x256. The size of the encoding unit may be classified according to the length of the long side, the length of the short side, or the width of the encoding unit. The image decoding device (100) may apply the same partitioning rule to encoding units classified into the same group. For example, the image decoding device (100) may classify encoding units having the same long side length into the same size. Additionally, the image decoding device (100) may apply the same partitioning rule to encoding units having the same long side length.

[0215] The ratio of width to height of a encoding unit may include 1:2, 2:1, 1:4, 4:1, 1:8, 8:1, 1:16, 16:1, 32:1, or 1:32, etc. Additionally, the direction of the encoding unit may include a horizontal direction and a vertical direction. The horizontal direction may indicate a case where the width of the encoding unit is longer than the height. The vertical direction may indicate a case where the width of the encoding unit is shorter than the height.

[0216] The image decoder (100) can adaptively determine a splitting rule based on the size of the encoding unit. The image decoder (100) can determine different acceptable splitting mode types based on the size of the encoding unit. For example, the image decoder (100) can determine whether splitting is allowed based on the size of the encoding unit. The image decoder (100) can determine the splitting direction according to the size of the encoding unit. The image decoder (100) can determine an acceptable splitting type according to the size of the encoding unit.

[0217] Determining the division rule based on the size of the encoding unit may be a division rule predetermined between the image encoding device (200) and the image decoding device (100). Additionally, the image decoding device (100) may determine the division rule based on information obtained from the bitstream.

[0218] The image decoder (100) can adaptively determine a partitioning rule based on the position of the encoding unit. The image decoder (100) can adaptively determine a partitioning rule based on the position occupied by the encoding unit in the image.

[0219] Additionally, the video decoding device (100) can determine a splitting rule so that encoding units generated by different splitting paths do not have the same block shape. However, this is not limited thereto, and encoding units generated by different splitting paths may have the same block shape. Encoding units generated by different splitting paths may have different decoding processing orders. Since the decoding processing order has been explained together with FIG. 12, a detailed explanation is omitted.

[0220] FIG. 16 illustrates encoding units that can be determined for each picture when the combination of forms in which encoding units can be divided according to one embodiment of the present disclosure is different for each picture.

[0221] Referring to FIG. 16, the image decoding device (100) may determine different combinations of division forms in which the encoding unit can be divided for each picture. For example, the image decoding device (100) may decode an image using a picture (1600) that can be divided into four encoding units, a picture (1610) that can be divided into two or four encoding units, and a picture (1620) that can be divided into two, three, or four encoding units among at least one picture included in the image. To divide the picture (1600) into multiple encoding units, the image decoding device (100) may use only division form information indicating that it is divided into four square encoding units. To divide the picture (1610), the image decoding device (100) may use only division form information indicating that it is divided into two or four encoding units. The image decoding device (100) may use only division form information indicating that the picture (1620) is divided into two, three, or four encoding units in order to divide the picture. Since the combination of division forms described above is merely an example for explaining the operation of the image decoding device (100), the combination of division forms described above should not be interpreted as being limited to the above example, but should be interpreted as allowing various combinations of division forms to be used for each predetermined data unit.

[0222] According to one embodiment of the present disclosure, a bitstream acquisition unit (110) of an image decoding device (100) may acquire a bitstream including an index representing a combination of division form information for each predetermined data unit (e.g., sequence, picture, slice, slice segment, tile, or tile group, etc.). For example, the bitstream acquisition unit (110) may acquire an index representing a combination of division form information from a sequence parameter set, a picture parameter set, a slice header, a tile header, or a tile group header. The image decoding device (100) may determine a combination of division forms in which the encoding unit can be divided for each predetermined data unit using the acquired index, and accordingly, different combinations of division forms may be used for each predetermined data unit.

[0223] FIG. 17 illustrates various forms of encoding units that can be determined based on partitioned form mode information that can be expressed as binary code according to one embodiment of the present disclosure.

[0224] According to one embodiment of the present disclosure, the image decoding device (100) can divide the encoding unit into various forms using block form information and division form mode information obtained through the bitstream acquisition unit (110). The forms of the encoding unit that can be divided may correspond to various forms including the forms described through the embodiments described above.

[0225] Referring to FIG. 17, the image decoding device (100) can divide a square-shaped encoding unit into at least one of a horizontal direction and a vertical direction based on the divided shape mode information, and can divide a non-square-shaped encoding unit into a horizontal direction or a vertical direction.

[0226] According to one embodiment of the present disclosure, if an image decoding device (100) can divide a square-shaped encoding unit into four square encoding units by dividing it in the horizontal and vertical directions, there may be four types of division forms that can be represented by the division form mode information for the square encoding unit. According to one embodiment of the present disclosure, the division form mode information may be expressed as a two-digit binary code, and a binary code may be assigned to each division form. For example, if the encoding unit is not divided, the division form mode information may be expressed as (00)b; if the encoding unit is divided in the horizontal and vertical directions, the division form mode information may be expressed as (01)b; if the encoding unit is divided in the horizontal direction, the division form mode information may be expressed as (10)b; and if the encoding unit is divided in the vertical direction, the division form mode information may be expressed as (11)b.

[0227] According to one embodiment of the present disclosure, when the image decoding device (100) divides a non-square encoding unit in a horizontal or vertical direction, the type of division form that can be represented by the division form mode information may be determined by how many encoding units are divided. Referring to FIG. 17, the image decoding device (100) may divide a non-square encoding unit into up to three units according to one embodiment of the present disclosure. The image decoding device (100) may divide the encoding unit into two encoding units, in which case the division form mode information may be expressed as (10)b. The image decoding device (100) may divide the encoding unit into three encoding units, in which case the division form mode information may be expressed as (11)b. The image decoding device (100) may decide not to divide the encoding unit, in which case the division form mode information may be expressed as (0)b. That is, the image decoding device (100) can use variable length coding (VLC) rather than fixed length coding (FLC) to use binary code representing segmented form mode information.

[0228] Referring to FIG. 17 according to one embodiment of the present disclosure, the binary code of the division form mode information indicating that the encoding unit is not divided can be represented as (0)b. If the binary code of the division form mode information indicating that the encoding unit is not divided is set to (00)b, then all 2 bits of the binary code of the division form mode information must be used even though there is no division form mode information set to (01)b. However, as illustrated in FIG. 17, if three division forms for a non-square encoding unit are used, the image decoding device (100) can determine that the encoding unit is not divided even if a 1-bit binary code (0)b is used as the division form mode information, thus allowing for efficient use of the bitstream. However, the division form of the non-square encoding unit indicated by the division form mode information should not be interpreted as being limited only to the three forms illustrated in FIG. 17, but should be interpreted as various forms including the embodiments described above.

[0229] FIG. 18 illustrates another form of a encoding unit that can be determined based on partitioned form mode information that can be represented as a binary code according to one embodiment of the present disclosure.

[0230] Referring to FIG. 18, the image decoding device (100) can divide a square-shaped encoding unit in a horizontal or vertical direction based on the division shape mode information, and can divide a non-square-shaped encoding unit in a horizontal or vertical direction. That is, the division shape mode information can indicate that a square-shaped encoding unit is divided in one direction. In this case, the binary code of the division shape mode information indicating that the square-shaped encoding unit is not divided can be represented as (0)b. If the binary code of the division shape mode information indicating that the encoding unit is not divided is set to (00)b, then all 2 bits of the binary code of the division shape mode information must be used even though there is no division shape mode information set to (01)b. However, as illustrated in FIG. 18, if three division forms for a square-shaped encoding unit are used, the image decoding device (100) can determine that the encoding unit is not divided even if a 1-bit binary code (0)b is used as division form mode information, thus allowing the bitstream to be used efficiently. However, the division form of the square-shaped encoding unit indicated by the division form mode information should not be interpreted as being limited only to the three forms illustrated in FIG. 18, but should be interpreted as various forms including the embodiments described above.

[0231] According to one embodiment of the present disclosure, block form information or partition form mode information can be represented using binary code, and such information can be immediately generated as a bitstream. Additionally, block form information or partition form mode information that can be represented by binary code may not be immediately generated as a bitstream but may be used as binary code input in CABAC (context adaptive binary arithmetic coding).

[0232] According to one embodiment of the present disclosure, an image decoding device (100) describes a process of obtaining syntax for block form information or partition form mode information through CABAC. A bitstream including a binary code for the syntax can be obtained through a bitstream acquisition unit (110). The image decoding device (100) can detect a syntax element representing block form information or partition form mode information by debinding a bin string included in the obtained bitstream. According to one embodiment of the present disclosure, the image decoding device (100) obtains a set of binary bin strings corresponding to the syntax element to be decoded, and can decode each bin using probability information, and the image decoding device (100) can repeat this process until the bin string composed of these decoded bins becomes equal to one of the previously obtained bin strings. The image decoding device (100) can determine syntax elements by performing inverse binarization of an empty string.

[0233] According to one embodiment of the present disclosure, an image decoding device (100) can determine the syntax for a bin string by performing a decoding process of adaptive binary arithmetic coding, and the image decoding device (100) can update a probability model for bins obtained through a bitstream acquisition unit (110). Referring to FIG. 17, the bitstream acquisition unit (110) of the image decoding device (100) can acquire a bitstream representing a binary code representing partitioned mode information according to one embodiment of the present disclosure. Using the acquired binary code having a size of 1 bit or 2 bits, the image decoding device (100) can determine the syntax for the partitioned mode information. To determine the syntax for the partitioned mode information, the image decoding device (100) can update the probability for each bit of the 2-bit binary code. That is, the image decoding device (100) can update the probability of having a value of 0 or 1 when decoding the next bin, depending on whether the value of the first bin of the 2-bit binary code is 0 or 1.

[0234] According to one embodiment of the present disclosure, the image decoding device (100) can update the probability for the bins used in the process of decoding the bins of the empty string for the syntax during the process of determining the syntax, and the image decoding device (100) can determine that the probability is not updated for certain bits of the empty string and has the same probability.

[0235] Referring to FIG. 17, in the process of determining syntax using an empty string representing segmentation mode information for a non-square type encoding unit, the image decoder (100) can determine syntax for the segmentation mode information using one bin having a value of 0 when the non-square type encoding unit is not segmented. That is, when block type information indicates that the current encoding unit is a non-square type, the first bin of the empty string for the segmentation mode information may be 0 when the non-square type encoding unit is not segmented, and 1 when it is segmented into two or three encoding units. Accordingly, the probability that the first bin of the empty string for the segmentation mode information for a non-square type encoding unit is 0 may be 1 / 3, and the probability that it is 1 may be 2 / 3. As described above, since the image decoding device (100) can represent only a 1-bit empty string having a value of 0 as the segmentation form mode information indicating that a non-square type encoding unit is not segmented, the image decoding device (100) can determine the syntax for the segmentation form mode information by determining whether the second bin is 0 or 1 only when the first bin of the segmentation form mode information is 1. According to one embodiment of the present disclosure, the image decoding device (100) can decode the bin by considering that when the first bin of the segmentation form mode information is 1, the probability that the second bin is 0 or 1 is equal to the probability.

[0236] According to one embodiment of the present disclosure, the image decoding device (100) may use various probabilities for each bin in the process of determining the bins of the bin string for the segmented form mode information. According to one embodiment of the present disclosure, the image decoding device (100) may determine the probability of the bins for the segmented form mode information differently depending on the direction of the non-square block. According to one embodiment of the present disclosure, the image decoding device (100) may determine the probability of the bins for the segmented form mode information differently depending on the width or the length of the long side of the current encoding unit. According to one embodiment of the present disclosure, the image decoding device (100) may determine the probability of the bins for the segmented form mode information differently depending on at least one of the shape of the current encoding unit and the length of the long side.

[0237] According to one embodiment of the present disclosure, the image decoding device (100) may determine that the probability of bins for segmented form mode information is the same for encoding units of a predetermined size or larger. For example, based on the length of the long side of the encoding unit, the probability of bins for segmented form mode information is determined to be the same for encoding units of a size of 64 samples or larger.

[0238] According to one embodiment of the present disclosure, the image decoding device (100) may determine the initial probability for the bins constituting the empty string of the segmented form mode information based on the slice type (e.g., I slice, P slice, or B slice).

[0239] Figure 19 is a block diagram of an image encoding and decoding system that performs loop filtering.

[0240] The encoding unit (1910) of the video encoding and decoding system (1900) transmits an encoded bitstream of the video, and the decoding unit (1950) receives the bitstream and decodes it to output a restored video. Here, the encoding unit (1910) may have a configuration similar to the video encoding device (200) described later, and the decoding unit (1950) may have a configuration similar to the video decoding device (100).

[0241] In the encoding unit (1910), the prediction encoding unit (1915) outputs prediction data through inter-prediction and intra-prediction, and the transformation and quantization unit (1920) outputs quantized transformed coefficients (or quantized coefficients) of residual data between the prediction data and the current input image. For example, the transform coefficients may be generated using a transformation kernel that includes at least one of the Discrete Cosine Transform (DT), Discrete Sine Transform (DST), Hadamard transform, Karhunen-Loeve Transform (KLT), or Wavelet Transform. The residual data may have its information compressed as the transformation is performed. For example, the residual data may be represented using a small number of frequencies as the transformation is performed. In one embodiment, the conversion and quantization unit (1920) may omit the conversion of residual data between the prediction data and the current input image. For example, if the distribution of residual data makes it inefficient to perform the conversion, the conversion process for residual data may be omitted and the quantization process may be performed. In one embodiment, information regarding whether the conversion is omitted (or whether the conversion is performed) may be transmitted to the decoding unit (1950) via a bitstream. The entropy encoding unit (1925) encodes and converts the quantized conversion coefficients and outputs them as a bitstream. The quantized conversion coefficients may be restored to spatial domain data through the inverse quantization and inverse conversion unit (1930). The inverse quantization and inverse conversion unit (1930) may perform inverse quantization on the quantized conversion coefficients and determine residual data by applying a conversion kernel. In one embodiment, the inverse quantization and inverse conversion unit (1930) may not perform inverse conversion.For example, if the transformation for residual data is omitted, the inverse quantization and inverse transformation unit (1930) may not perform the inverse transformation. The inverse quantization and inverse transformation unit (1930) can determine residual data by performing inverse quantization on the quantized transformation coefficients. The data in the restored spatial domain is output as a restored image after passing through the deblocking filtering unit (1935) and the loop filtering unit (1940). The restored image can be used as a reference image for the next input image after passing through the predictive encoding unit (1915).

[0242] The encoded image data among the bitstreams received by the decoding unit (1950) is restored into spatial domain residual data through the entropy decoding unit (1955) and the inverse quantization and inverse transform unit (1960). The predicted data and residual data output from the prediction decoding unit (1975) are combined to form spatial domain image data, and the deblocking filtering unit (1965) and the loop filtering unit (1970) can perform filtering on the spatial domain image data to output a restored image for the current original image. The restored image can be used as a reference image for the next original image by the prediction decoding unit (1975).

[0243] The loop filtering unit (1940) of the encoding unit (1910) performs loop filtering using filter information input according to user input or system settings. The filter information used by the loop filtering unit (1940) is output to the entropy encoding unit (1925) and transmitted to the decoding unit (1950) along with the encoded image data. The loop filtering unit (1970) of the decoding unit (1950) can perform loop filtering based on the filter information input from the decoding unit (1950).

[0244] FIG. 20 is a block diagram showing the configuration of an image decoding device according to one embodiment of the present disclosure.

[0245] Referring to FIG. 20, the image decoding device (2000) may include a processor (2010) and a memory (2020).

[0246] In one embodiment of the present disclosure, the processor (2010) may include processing circuits and / or multiple processors. For example, the processor (2010) may include various processing circuits including at least one processor, and at least one of the at least one processor may be configured to perform the various functions described in the present disclosure individually and / or collectively in a distributed manner.

[0247] In one embodiment of the present disclosure, the memory (2020) may include one or more storage media that store at least one instruction. The processor (2010) may control the image decoding device (2000) by executing the instruction stored in the memory (2020). For example, the processor (2010) may control the image decoding device (2000) to perform an operation by executing the instruction stored in the memory (2020) individually or collectively. In one embodiment of the present disclosure, the operation performed by the image decoding device (2000) may be an operation performed by the processor (2010) of the image decoding device (2000).

[0248] In one embodiment of the present disclosure, the image decoding device (2000) may correspond to the image decoding device (100) shown in FIG. 1 and / or the decoding unit (1950) shown in FIG. 19.

[0249] The image decoding device (2000) can obtain a bitstream generated as a result of encoding an image. The bitstream may include the encoding result for the current block. In one embodiment of the present disclosure, the image decoding device (2000) can receive the bitstream from the image encoding device via a network. In one embodiment of the present disclosure, the image decoding device (2000) can obtain the bitstream from a data storage medium comprising at least one of a magnetic medium such as a hard disk, a floppy disk, and a magnetic tape, an optical recording medium such as a CD-ROM and a DVD, or a magneto-optical medium such as a floptical disk.

[0250] The image decoding device (2000) can obtain syntax elements for decoding an image from a bitstream. Values ​​corresponding to the syntax elements may be included in the bitstream according to the hierarchical structure of the image. In one embodiment of the present disclosure, the image decoding device (2000) can obtain syntax elements by entropy decoding bins included in the bitstream.

[0251] In one embodiment of the present disclosure, the bitstream may include information regarding the prediction mode of the current block within the current image. The current block may include at least one of a maximum encoding unit, an encoding unit, a transform unit, or a prediction unit divided from the current image to be decoded. In one embodiment of the present disclosure, the prediction mode of the current block may include at least one of an intra mode, an inter mode, a combined mode, a geometric partitioning mode (GPM), a block copy mode, or a template matching prediction mode.

[0252] In one embodiment of the present disclosure, an intra mode may include a non-directional Planar mode 0 (or Intra_Planar mode), a non-directional DC mode 1 (or Intra_DC mode), and directional Angular modes 2 through 66 (or Intra_directional modes) (e.g., (Intra_Angular2.. Intra_Angular66). In one embodiment of the present disclosure, an intra-planar mode may refer to a mode that determines a predicted sample based on a weighted average value according to the distance of a left reference sample, an upper reference sample, a lower-left sample of the current block, and an upper-right sample. In one embodiment of the present disclosure, an intra-DC mode may refer to a mode that determines the average value of reference samples as the predicted sample. In one embodiment of the present disclosure, in the intra-directional modes, the locations of reference samples for generating predicted samples of samples within the current block may be identified by considering the direction indicated by the intra-directional modes. For example, in mode 34, located at a 45-degree upper-left direction relative to the samples within the current block. Reference samples may be identified. In one embodiment of the present disclosure, the intra mode may include -14 through -1 and 67 through 80 Wide-Angular modes (Intra_Wide_Angular). Wide-Angular modes may be used to identify reference samples of samples within a non-square current block. In one embodiment of the present disclosure, the image decoder (2000) may determine one of the Wide-Angular modes as the intra prediction mode of the non-square current block. The image decoder (2000) may determine the Wide-Angular mode based on the width and height of the current block. If the width of the current block is greater than the height, the image decoder (2000) may replace the lower-left directional mode with the upper-right extended directional mode.For example, the image decoding device (2000) may replace the index value "predModeIntra" of the intra prediction mode, which is 2 or more and less than or equal to a predetermined value, with "predModeIntra + 65". If the height of the current block is greater than the width, the image decoding device (2000) may replace the upper right directional mode with the lower left extended directional mode. For example, the image decoding device (2000) may replace the index value "predModeIntra" of the intra prediction mode, which is 66 or more and less than or equal to a predetermined value, with "predModeIntra - 67". The number and types of intra prediction modes available in the intra mode of the image decoding device (2000) according to one embodiment of the present disclosure may be set in various ways. For example, the image decoding device (2000) can determine a Wide-Angular mode using a predetermined method based on the ratio of the height and width of the block (e.g., 16, 8, 4, 2, 1 / 2, 1 / 4, 1 / 8, or 1 / 16).

[0253] In one embodiment of the present disclosure, the image decoder (2000) may determine an intra prediction mode using most probable modes (MPM). The image decoder (2000) may determine whether to use the MPM. The image decoder (2000) may obtain information related to whether to use the MPM from a bitstream. If the MPM is used, the image decoder (2000) may determine an MPM list. In one embodiment of the present disclosure, the image decoder (2000) may determine an MPM list using the surrounding blocks of the current block. The image decoder (2000) may determine an MPM list based on the intra mode of the upper block of the current block and the intra mode of the left block. If the surrounding blocks of the current block cannot be used (e.g., if the intra prediction mode of the surrounding blocks is not determined), the intra prediction mode of the unavailable surrounding blocks may be set to a predetermined mode (e.g., Planar mode). The image decoding device (2000) can determine one of the MPM lists as the intra prediction mode of the current block. The image decoding device (2000) can obtain information (e.g., index information) indicating the intra prediction mode of the current block among the MPM lists from the bitstream.

[0254] In one embodiment of the present disclosure, an image decoder (2000) may determine an intra-prediction mode using a template. The image decoder (2000) may determine a template of the current block. The template of the current block may include a left sample, an upper-left sample, and / or an upper sample of the current block. For example, the template of the current block may include a reconstructed sample or prediction sample of the surrounding L-shape of the current block. The image decoder (2000) may determine a surrounding sample of the template of the current block. The surrounding sample of the template may include a left sample, an upper-left sample, and / or an upper sample of the template. The image decoder (2000) may perform a prediction on the template using the surrounding sample of the template as a reference sample. In one embodiment of the present disclosure, the process by which the image decoder (2000) determines an intra-prediction mode by performing a prediction on the template may be referred to as Template-based intra-mode derivation (TIMD). In one embodiment of the present disclosure, the template shape of the block may be applied to prediction modes using a template other than TIMD.

[0255] In one embodiment of the present disclosure, the image decoder (2000) can infer an intra prediction mode of the current block using surrounding samples of the current block. The image decoder (2000) can determine a gradient using surrounding samples of the current block. The image decoder (2000) can determine a plurality of 3 x 3 blocks adjacent to the current block. The image decoder (2000) can obtain horizontal change amounts and vertical change amounts of samples included in each determined 3 x 3 block. However, the present disclosure is not limited to 3 x 3 blocks, and the gradient can be determined using N x N blocks. The image decoder (2000) can determine the gradient based on the horizontal change amounts and vertical change amounts. The image decoder (2000) can determine the horizontal change amounts and vertical change amounts using a Sobel filter. The image decoder (2000) can determine an intra prediction mode corresponding to the gradient. The image decoder (2000) can determine the intra prediction mode of the current block based on a plurality of intra prediction modes determined for a plurality of 3 x 3 blocks. The image decoder (2000) can obtain gradient statistical information. For example, the image decoder (2000) can obtain a Histogram of Gradients (HoG) based on a plurality of intra prediction modes determined for a plurality of 3 x 3 blocks. HoG may mean a representation of a plurality of gradient information in the form of a histogram. In one embodiment of the present disclosure, the image decoder (2000) can determine the most frequently determined intra prediction mode as the intra prediction mode of the current block. In one embodiment of the present disclosure, the image decoder (2000) can determine the amplitude based on the horizontal change amount and the vertical change amount. The image decoder (2000) can determine the intra prediction mode of the current block based on the amplitude.The image decoder (2000) can determine the weight of the intra prediction mode corresponding to the slope as a magnitude. For example, the image decoder (2000) can increase the weight of the intra prediction mode determined based on the slope as the horizontal change amount and the vertical change amount increase. The image decoder (2000) can determine the intra prediction mode of the current block based on the result reflecting the weight determined according to the magnitude. In one embodiment of the present disclosure, the process of the image decoder (2000) determining the intra prediction mode based on the slope may be referred to as decoder-side intra mode derivation (DIMD).

[0256] In one embodiment of the present disclosure, the block copy mode may include an intra-block copy mode. In one embodiment, the block copy mode may include an intra-block copy mode. In one embodiment, the intra-block copy mode may be a sub-mode of the intra mode, but is not limited thereto, and may represent a mode distinct from the intra mode. In one embodiment, the template matching prediction mode may include a template matching intra-prediction mode. The combination mode may include a Combined Inter-Intra Prediction (CIIP) mode that performs prediction by combining prediction according to the intra mode and prediction according to the inter mode. The geometric partitioning mode may include a mode that partitions to include directionality within the block. The geometric partitioning mode may perform prediction for each partitioned region using inter-prediction or intra-prediction.

[0257] The video decoding device (2000) can restore the current block by performing a prediction according to the prediction mode for the current block according to the prediction mode of the current block.

[0258] In one embodiment of the present disclosure, the image decoder (2000) can obtain information regarding the prediction mode of the current block from a bitstream. For example, the image decoder (2000) can obtain index information indicating the prediction mode of the current block from a bitstream.

[0259] In one embodiment of the present disclosure, when the prediction mode of the current block is CIIP mode, the image decoder (2000) can restore the current block by combining inter prediction and intra prediction. For example, the image decoder (2000) can perform intra prediction according to Planar mode. For example, the image decoder (2000) can perform inter prediction using a Motion Vector (MV). The image decoder (2000) can restore the current block using a weighted sum of the prediction block according to inter prediction and the prediction block according to intra prediction. The weights can be determined based on whether the blocks adjacent to the current block have been intra predicted.

[0260] In one embodiment of the present disclosure, when the prediction mode of the current block is a geometric partitioning mode, the image decoder (2000) can perform prediction by partitioning the current block. The image decoder (2000) can obtain a partitioning angle and a partitioning distance for the boundary where partitioning is performed within the current block. The image decoder (2000) can partition the current block based on the partitioning angle and the partitioning distance. The image decoder (2000) can restore the current block by performing inter-prediction or intra-prediction on each of the partitioned regions within the current block. The image decoder (2000) can (i) perform intra-prediction on both partitioned regions, (ii) perform inter-prediction on one region and intra-prediction on the other region, or (iii) perform inter-prediction on both partitioned regions.

[0261] In one embodiment of the present disclosure, when the prediction mode of the current block is a block copy mode, the image decoder (2000) can restore the current block based on a reference block included in the current image. In one embodiment of the present disclosure, when the prediction mode of the current block is a block copy mode, the image decoder (2000) can determine information regarding a Block Vector (BV) representing a reference block included in the current image. In one embodiment of the present disclosure, the image decoder (2000) can determine a prediction block based on a reference block. For example, the image decoder (2000) can determine a prediction block by making it identical to the reference block or by performing filtering on the reference block.

[0262] In one embodiment of the present disclosure, when the prediction mode of the current block is a template matching prediction mode, the image decoder (2000) can restore the current block using a reference block. The image decoder (2000) can obtain information regarding whether to use the template matching prediction mode. The image decoder (2000) can determine whether to use the template matching prediction mode based on the obtained information. The reference block may be determined based on at least one of a region included in the current image or a region included in a previously decoded image. In one embodiment of the present disclosure, the image decoder (2000) can determine the reference block using a cost function. The cost function may include at least one of SAD (sum of absolute difference), SSD (sum of squared difference), SATD (sum of Absolute Transformed Difference), SSE (sum of squared error), or MR-SAD (mean removed SAD). For example, the image decoder (2000) can determine the error between templates using at least one cost function among SAD, SATD, SSE, and MR-SAD. When the cost function is SAD, the image decoder (2000) can determine the error based on the sum of the absolute values ​​of the differences between each sample of the template of the current block and each sample of the template of the candidate block. When the cost function is SSD, the image decoder (2000) can determine the error based on the sum of the squares of the differences between each sample of the template of the current block and each sample of the template of the candidate block. The cost function may include a function representing the number of identical samples. The image decoder (2000) can determine the block with the smaller error among the candidate blocks as the reference block.An image decoding device (2000) can determine a prediction block by performing a template matching intra prediction on the current image. In the present disclosure, the process of determining a reference block for the current block using a template may be referred to as Template Matching (TM). In the present disclosure, performing a prediction for the current block based on Template Matching may be referred to as Template Matching Prediction (TMP) or Intra Template Matching Prediction (IntraTMP).

[0263] In one embodiment of the present disclosure, the image decoder (2000) can restore (or predict) the current block using a Matrix-based Intra Prediction (MIP) mode. The image decoder (2000) can determine a left boundary sample and an upper boundary sample using a left reference sample and an upper reference sample of the current block. In one embodiment of the present disclosure, the image decoder (2000) can determine the average of the left reference samples as the left boundary sample and the average of the upper reference samples as the upper boundary sample. For example, one left boundary sample may be determined by the average of a plurality of left reference samples, and one upper boundary sample may be determined by the average of a plurality of upper reference samples. In one embodiment of the present disclosure, the number of left boundary samples and the number of upper boundary samples may be determined based on the size and shape of the current block. The image decoder (2000) can determine a boundary vector using the left boundary sample and the upper boundary sample. In one embodiment of the present disclosure, the boundary vector may refer to a one-dimensional vector that includes both left boundary samples and upper boundary samples. For example, if the current block is a 4 x 4 block, the number of left boundary samples and the number of upper boundary samples are each 2, and the boundary vector may be a vector containing 4 samples. Additionally, for example, if the current block is not a 4 x 4 block, the boundary vector may be a vector containing 8 samples. The image decoding device (2000) may determine at least one of a matrix or an offset vector based on at least one of the width, height, or intra prediction mode of the current block. In one embodiment of the present disclosure, the intra prediction mode may be referred to as a matrix index. The image decoding device (2000) may predict some samples of the current block using the matrix, the offset vector, and the boundary vector.The image decoder (2000) can predict the remaining samples of the current block using a matrix and a boundary vector, a predicted sample, a left reference sample, and an upper boundary sample. In one embodiment of the present disclosure, the image decoder (2000) can obtain the remaining samples by performing interpolation in the vertical direction and the horizontal direction. The image decoder (2000) can obtain the vertical samples of the predicted sample by performing vertical interpolation on the predicted sample and the upper boundary sample. The image decoder (2000) can obtain the remaining samples by performing horizontal interpolation using the left reference sample, the predicted sample, and the vertical samples obtained by interpolation.

[0264] In one embodiment of the present disclosure, when the prediction mode of the current block is Occurrence-based intra coding (OBIC), the image decoder (2000) can perform prediction based on the intra prediction mode of the surrounding blocks. For example, the image decoder (2000) can obtain an intra prediction mode from the surrounding blocks of the current block (e.g., adjacent blocks and / or non-adjacent blocks). The image decoder (2000) can select a prediction mode based on the frequency of the intra prediction mode of the surrounding blocks. The image decoder (2000) can obtain a prediction block using the selected prediction mode.

[0265] In one embodiment of the present disclosure, when the prediction mode of the current block is extrapolation filter-based intra prediction (EIP), the image decoder (2000) can obtain a prediction sample based on surrounding samples and an EIP filter. For example, the EIP filter may include 15 filter coefficients corresponding to 15 samples among a 4 x 4 block, an 8 x 2 block, or a 2 x 8 block.

[0266] In one embodiment of the present disclosure, when the prediction mode of the current block is Intra Sub Partitions (ISP), the image decoder (2000) can acquire prediction samples based on sub-blocks. The image decoder (2000) can divide the current block into a plurality of sub-blocks along a horizontal or vertical direction. The image decoder (2000) can acquire prediction samples for each sub-block.

[0267] In one embodiment of the present disclosure, when the prediction mode of the current block is a Cross-Component Linear Model (CCLM), the current block of the chroma component (e.g., chroma block) can be obtained based on the current block of the lumina component (e.g., lumina block). For example, the image decoder (2000) can obtain chroma samples by applying linear transformation coefficients to lumina samples. The image decoder (2000) can determine or update parameters (e.g., linear transformation coefficients) based on samples of the lumina block and samples of the chroma block.

[0268] In one embodiment of the present disclosure, when the prediction mode of the current block is a convolutional cross-component model (CCCM), the image decoder (2000) can acquire a current block of chroma components based on a current block of luminance components. For example, the image decoder (2000) can acquire chroma samples by applying a convolution filter to luminance samples. The image decoder (2000) can determine or update parameters (e.g., convolution filter) based on samples of the luminance block and samples of the chroma block.

[0269] In one embodiment of the present disclosure, when the prediction mode of the current block is a block differential pulse coded modulation (BDPCM) mode, the image decoder (2000) can recover the current block based on sample difference values. For example, the image decoder (2000) can obtain a prediction sample using the sum of sample difference values ​​in the vertical or horizontal direction. The image decoder (2000) can obtain a BDPCM enable flag through the SPS. The BDPCM enable flag may mean a flag indicating whether BDPCM can be used in the SPS. The BDPCM enable flag may be obtained when the transform skip mode is enabled in the SPS. If BDPCM can be used, the image decoder (2000) can obtain a BDPCM flag indicating whether BDPCM is used. For example, if the size of the encoding unit is less than or equal to a predetermined value (e.g., the maximum size of a block for which a conversion skip is allowed) and is in an intra-prediction mode, the image decoder (2000) can acquire a BDPCM flag. The BDPCM flag can be transmitted at the encoding unit level. For example, the BDPCM flag can indicate whether BDPCM is used in the encoding unit. The image decoder (2000) can acquire information regarding the direction of the BDPCM. For example, the image decoder (2000) can acquire a BDPCM direction flag indicating whether the prediction is in a vertical or horizontal direction. The image decoder (2000) can acquire prediction samples along the vertical or horizontal direction. For example, the prediction value of the i-th sample can be determined as the sum of the BDPCM values ​​from the first sample to the i-th sample.

[0270] The image decoding device (2000) can generate a restored current block using a prediction block. In one embodiment of the present disclosure, the image decoding device (2000) can determine the prediction block as the restored current block. In one embodiment of the present disclosure, the image decoding device (2000) can generate a restored current block by combining residual data obtained from a bitstream by the image decoding device (2000) with the prediction block. The restored current block can be used as a reference block for the next block.

[0271] In one embodiment, the image decoder (2000) may obtain residual data from a bitstream. The residual data may include information regarding the difference between the original image (or original sample) and the predicted image (or predicted sample). In one embodiment, the image decoder (2000) may obtain the transform coefficients of a residual block corresponding to a transform unit from the bitstream. In one embodiment, the image decoder (2000) may obtain residual samples of a residual block based on the transform coefficients of a residual block. For example, the image decoder (2000) may obtain residual samples of a residual block by performing at least one of inverse quantization or inverse transform on the transform coefficients of a residual block. In one embodiment, the image decoder (2000) may determine residual samples of the encoding unit using at least a portion of the residual samples of the residual block. If the size of the residual block is larger than the size of the transform unit, the image decoder (2000) may determine a portion of the residual samples of the residual block as residual samples of the encoding unit. Alternatively, the image decoder (2000) may determine a portion of the residual samples of the residual block, after filtering has been performed, as residual samples of the encoding unit. The image decoder (2000) may restore samples of the encoding unit based on the residual samples of the encoding unit.

[0272] In a prediction mode (e.g., intra mode) utilizing reference samples included in the current image, a prediction block of the current block can be generated based on the surrounding samples of the current block according to the prediction mode, under the assumption that there is continuity between the surrounding samples of the current block and the samples within the current block. An image decoding device (2000) according to one embodiment of the present disclosure may utilize spatial reference samples included in the current image, as well as surrounding samples of the current block included in the current image, for intra prediction. When using samples restored before the current block, the size of residual data may be reduced by predicting the samples of the current block using samples immediately adjacent to the current block as well as samples far from the current block. In one embodiment of the present disclosure, the image decoding device (2000) may increase the range of the area that can be determined as a reference block by performing intra prediction using a reference block containing unrestored samples. An image decoding device (2000) according to one embodiment of the present disclosure may improve compression efficiency by increasing the efficiency of intra prediction.

[0273] In a prediction mode (e.g., inter mode) that utilizes reference samples included in a reference image rather than the current image, a prediction block of the current block can be generated based on a reference block (or reference sample) of the reference image according to the prediction mode, under the assumption that there is continuity between the current image and the reference image. An image decoding device (2000) according to one embodiment of the present disclosure can improve compression efficiency by increasing the efficiency of intra prediction.

[0274] The image decoding device (2000) can improve prediction accuracy by considering together the reference block (or reference sample) included in the current image and the reference block (or reference sample) included in an image other than the current image. The image decoding device (2000) according to one embodiment of the present disclosure can improve prediction accuracy by considering both the current image and an image other than the current image.

[0275] The image decoding device (2000) can perform deblocking filtering. The deblocking filter can improve image quality by smoothing the edges between blocks.

[0276] The image decoding device (2000) can perform filtering on samples of the current block that have undergone deblocking filtering using a Sample Adaptive Offset (SAO) filter and / or a Bilateral Filter (BIF). The SAO filter and BIF can improve image quality by reducing the error between the restored image and the original image. The SAO filter and BIF can perform filtering on a sample basis.

[0277] The image decoding device (2000) can perform filtering using an Adaptive Loop Filter (ALF). The ALF can improve image quality by reducing the error between the restored image and the original image. The ALF can perform filtering in block units.

[0278] FIG. 21 is a drawing for explaining the process of determining an effective sub-block according to one embodiment of the present disclosure.

[0279] In one embodiment of the present disclosure, a coding unit may be divided into one or more transformation units. When a sub-block transform is applied to a coding unit, the coding unit may be divided into a plurality of transformation units. Each transformation unit of the coding unit to which a sub-block is transformed may be referred to as a sub-block. Among the plurality of transformation units included in the coding unit to which the sub-block transform is applied, one transformation unit may include a non-zero transformation coefficient. A sub-block including a non-zero transformation coefficient may be referred to as an effective sub-block. Among the plurality of sub-blocks of the coding unit, other sub-blocks excluding the effective sub-block may be determined not to have a transformation coefficient. That is, the residual data of the other sub-blocks may be determined to be zero.

[0280] In one embodiment of the present disclosure, the image decoder (2000) may obtain information regarding whether a sub-block transformation is applied (e.g., cu_sbt_flag) through a bitstream. The image decoder (2000) may obtain at least one of information regarding the division size of a encoding unit, information regarding the division direction of a encoding unit, or information regarding the location of an effective sub-block within a encoding unit from the bitstream. For example, when a sub-block transformation is applied, the image decoder (2000) may obtain at least one of information regarding the division size of a encoding unit, information regarding the division direction of a encoding unit, or information regarding the location of an effective sub-block within a encoding unit from the bitstream. In one embodiment, the image decoder (2000) may obtain information regarding whether a sub-block transformation is applied from the bitstream when the encoding unit is predicted using inter-prediction.

[0281] In one embodiment of the present disclosure, information regarding the division size (e.g., cu_sbt_quad_flag) may indicate the size of an effective sub-block. For example, an image decoder (2000) may determine, based on the information regarding the division size, whether the size of an effective sub-block corresponds to 1 / 2 (e.g., cu_sbt_quad_flag == 0) or 1 / 4 (e.g., cu_sbt_quad_flag == 1) the width of an encoding unit.

[0282] In one embodiment of the present disclosure, the image decoding device (2000) may determine the size of an effective sub-block without obtaining information regarding the division size. The image decoding device (2000) may determine the division size based on at least one of the height or width of the encoding unit. For example, the image decoding device (2000) may determine the size of an effective sub-block to be 1 / 4 of the encoding unit when the height of the encoding unit in the horizontal division is greater than or equal to a predetermined value. For example, the image decoding device (2000) may determine the size of an effective sub-block to be 1 / 2 of the encoding unit when the height of the encoding unit in the horizontal division is smaller than a predetermined value. In one embodiment, the effective sub-block may be an encoding unit divided into N equal parts (e.g., N is a power of 2).

[0283] In one embodiment of the present disclosure, information regarding the division direction (cu_sbt_horizontal_flag) may indicate the division direction of the encoding unit. For example, the image decoding device (2000) may determine whether to divide the encoding unit in a horizontal direction (e.g., cu_sbt_horizontal_flag == 0) or in a vertical direction (e.g., cu_sbt_horizontal_flag == 1) based on the information regarding the division direction.

[0284] In one embodiment of the present disclosure, information regarding the location of an effective sub-block (cu_sbt_pos_flag) may indicate the location of an effective sub-block in a divided encoding unit. For example, an image decoder (2000) may determine that the location of an effective sub-block is one of the upper side, lower side, left side, or right side based on the information regarding the location of the effective sub-block.

[0285] Referring to FIG. 21, an effective sub-block can be determined based on information regarding the division size (cu_sbt_quad_flag), information regarding the division direction (cu_sbt_horizontal_flag), and information regarding the position of the effective sub-block (cu_sbt_pos_flag). In one embodiment, the shaded area of ​​the encoding unit in FIG. 21 may represent an effective sub-block. The square area outside the shaded area may represent the encoding unit. Although the shape of the encoding unit in FIG. 21 according to one embodiment is depicted as a square, it is not limited thereto, and the shape of the encoding unit may be a rectangle.

[0286] Referring to FIG. 21, in one embodiment, when the effective sub-block is half of the encoding unit (cu_sbt_quad_flag==0), the encoding unit is horizontally divided (cu_sbt_horizontal_flag==1), and the effective sub-block is located below the encoding unit (cu_sbt_pos_flag==1), the image decoder (2000) determines that the effective sub-block is located below the encoding unit and can perform decoding of the residual data of the effective sub-block. For example, the image decoder (2000) can perform decoding of information regarding the non-zero transform coefficients of the effective sub-block.

[0287] In one embodiment of the present disclosure, the image decoder (2000) may apply an inverse transform kernel to an effective sub-block. For example, the image decoder (2000) may obtain residual samples by applying an inverse transform kernel to transform coefficients. The inverse transform kernel may include at least one of DCT 8, DST 7, or DCT 1. The inverse transform kernel refers to a kernel used to perform an inverse transform to obtain residual samples based on transform coefficients, and may simply be referred to as a transform kernel. In one embodiment of the present disclosure, the image decoder (2000) may obtain a restored sample based on the sum of the residual sample and the predicted sample.

[0288] FIG. 22 is a drawing for explaining the process of determining an effective sub-block according to one embodiment of the present disclosure.

[0289] The video decoding device (2000) can determine the valid sub-blocks included in the encoding unit based on information obtained from the bitstream.

[0290] In one embodiment, the image decoding device (2000) may obtain at least one of information related to the division size of the encoding unit, information regarding the division direction of the encoding unit, or information regarding the location of a sub-block within the encoding unit from the bitstream. For convenience of explanation, the information obtained from the bitstream described with reference to FIG. 21 is omitted.

[0291] In one embodiment, information regarding the location of an effective sub-block (cu_sbt_pos_flag) may indicate the location of an effective sub-block in a divided encoding unit. For example, an image decoding device (2000) may determine, based on the information regarding the location of an effective sub-block, that the location of the effective sub-block is at least one of the upper side, middle side, lower side, left side, or right side. In one embodiment, the information regarding the location of an effective sub-block may be represented as index information, such as cu_sbt_pos_idx.

[0292] Referring to FIG. 22, a conversion unit can be determined based on information regarding the division size (cu_sbt_quad_flag), information regarding the division direction (cu_sbt_horizontal_flag), and information regarding the position of the valid sub-block (cu_sbt_pos_flag). In one embodiment, the shaded area of ​​the encoding unit in FIG. 22 may represent the valid sub-block.

[0293] Referring to FIG. 22, in one embodiment, when the size of the effective sub-block is half the size of the encoding unit (cu_sbt_quad_flag==0), the encoding unit is horizontally divided (cu_sbt_horizontal_flag==1), and the effective sub-block is located in the middle of the encoding unit (cu_sbt_pos_flag==1), the image decoding device (2000) can perform decoding for the effective sub-block, which has the same width as the encoding unit located in the middle area of ​​the encoding unit and half the height.

[0294] Referring to FIG. 22, in one embodiment, when the size of the effective sub-block is 1 / 4 of the size of the encoding unit (cu_sbt_quad_flag==1), the encoding unit is vertically divided (cu_sbt_horizontal_flag==0), and the effective sub-block is located at the right edge of the encoding unit (cu_sbt_pos_flag==3), the image decoding device (2000) can perform decoding for the effective sub-block, which has the same height as the encoding unit located in the right edge area of ​​the encoding unit and has a width of 1 / 4.

[0295] In one embodiment, the transform tree syntax structure can be represented as shown in Table 1. In the transform tree syntax, the transform unit can be determined based on information related to the division size of the encoding unit (cu_sbt_quad_flag), information regarding the division direction of the encoding unit (cu_sbt_horizontal_flag), and information regarding the location of the effective sub-block within the encoding unit (cu_sbt_pos_flag). For example, when cu_sbt_horizontal_flag == 0 and cu_sbt_pos_flag == 1, the encoding unit is divided into three transform units, and the width ratio of each transform unit is 1:2:1.

[0296] transform_tree( x0, y0, tbWidth, tbHeight, treeType, chType ) {DescriptorInferTuCbfLuma = 1if( IntraSubPartitionsSplitType = = ISP_NO_SPLIT && !cu_sbt_flag ) {...}} else if( cu_sbt_flag ) {if( !cu_sbt_horizontal_flag ) {if( cu_sbt_pos_flag = = 0 ) {...} else if( cu_sbt_pos_flag = = 1 ) {if( cu_sbt_quad_flag = = 0 ) {transform_unit( x0, y0, tbWidth / 4, tbHeight, treeType, 0, 0, 1 )transform_unit( x0 + tbWidth / 4, y0, tbWidth / 2, tbHeight, treeType, 1, 0, 0 )transform_unit( x0 + tbWidth * 3 / 4, y0, tbWidth / 4, tbHeight, treeType, 2, 0, 1 )} else { / ( cu sbt_quad_flag = = 1 )transform_unit( x0, y0, tbWidth / 4, tbHeight, treeType, 0, 0, 1 )transform_unit( x0 + tbWidth / 4, y0, tbWidth / 4, tbHeight, treeType, 1, 0, 0 )transform_unit( x0 + tbWidth / 2, y0, tbWidth / 2, tbHeight, treeType, 2, 0, 1 )}} else if( cu_sbt_pos_flag = = 2 ) {...}} else { / ( cu_sbt_pos_flag = = 3 )...}}} else { / horizontal split cases}...}

[0297] In one embodiment, information regarding the location of an effective sub-block (cu_sbt_pos_flag) may indicate that the effective sub-block is located on the upper, lower, left, and right sides of the encoding unit, as shown in FIG. 21. In this example, information regarding the location of the effective sub-block may be represented by 1 bit. In one embodiment, information regarding the location of an effective sub-block (cu_sbt_pos_flag) may indicate that the effective sub-block is located on at least one of the upper, middle, lower, left, and right sides of the encoding unit, as shown in FIG. 22. In this example, information regarding the location of the effective sub-block may be represented by 2 bits.

[0298] In one embodiment, information regarding the location of an effective sub-block may be included in the header of the bitstream (e.g., Sequence Parameter Set, Video Parameter Set, Picture Header, Slice Header, CTU (coding tree unit) unit, CU (coding unit) unit, PU (prediction unit), TU (transform unit) unit) as to whether the information regarding the location of an effective sub-block indicates that the transformation unit is located in the intermediate region of the coding unit. For a specific unit, it may be determined whether the information regarding the location of an effective sub-block indicates that the transformation unit is located in the intermediate region of the coding unit.

[0299] In one embodiment, information regarding the location of an effective sub-block is represented by 1 bit, and an extension flag indicating whether the conversion unit represents the middle region and / or both edge regions of the encoding unit may be used. For example, when the extension flag is 0, the image decoder (2000) can determine an effective sub-block as in FIG. 21. For example, when the extension flag is 1, the image decoder (2000) can determine an effective sub-block that is not represented according to FIG. 21 in FIG. 22.

[0300] FIG. 23 is a drawing for explaining the process of determining an effective sub-block according to one embodiment of the present disclosure.

[0301] In one embodiment of the present disclosure, the height and width of the effective sub-block may be smaller than the height and width of the encoding unit, respectively. Referring to FIG. 21 and FIG. 22, the width or height of the effective sub-block according to one embodiment is equal to the encoding unit, but as shown in FIG. 23, the width of the effective sub-block according to one embodiment may be smaller than the width of the encoding unit, and the height of the effective sub-block may be smaller than the height of the encoding unit.

[0302] In one embodiment of the present disclosure, the width of an effective subblock may be 1 / N of a encoding unit, and the height of an effective subblock may be 1 / M of a encoding unit. For example, the width and height of an effective subblock may each be half the width and height of a encoding unit (e.g., N = 2, M = 2). FIG. 23 is an example of a case where the width and height of an effective subblock according to one embodiment of the present disclosure are each half the width and height of a encoding unit, and the shaded area may represent an effective subblock.

[0303] In one embodiment of the present disclosure, information regarding the division size may include information regarding the division ratio of the height of the encoding unit and the division ratio of the width of the encoding unit. For example, information regarding the division size may include information indicating the values ​​of N and M (e.g., N is the ratio of the width of the encoding unit to the width of the encoding unit, and M is the ratio of the height). In one embodiment of the present disclosure, an image decoder (2000) may obtain information indicating whether the height of an effective sub-block is smaller than the height of the encoding unit and whether the width of an effective sub-block is smaller than the width of the encoding unit. For example, the image decoder (2000) may obtain information (e.g., sbt_corner_flag) indicating whether an effective sub-block is located at a corner or in the middle of the encoding unit. If an effective sub-block is located at a corner or in the middle of the encoding unit, it may mean that the height and width of the effective sub-block are smaller than the height and width of the encoding unit, respectively.

[0304] In one embodiment, the image decoding device (2000) may omit information regarding the division direction and determine the conversion unit based on information regarding the location of the effective sub-block. The information regarding the location of the effective sub-block may indicate one of the predetermined locations of the conversion unit. For example, with reference to FIG. 23, the information regarding the location of the effective sub-block may indicate one of the candidate locations of five effective sub-blocks (2310-2350). In one embodiment, the effective sub-blocks (2310, 2320, 2330, 2340) may be adjacent to two edges of the encoding unit. In one embodiment, the effective sub-block (2350) may not be adjacent to an edge. However, not limited thereto, in one embodiment, the effective sub-block may be adjacent to one edge of the encoding unit. For example, the effective sub-block may be adjacent to only one of the upper, lower, left, or right boundaries of the encoding unit.

[0305] Information regarding the location of an effective sub-block described through FIGS. 21 to 23 may be randomly encoded. For example, information regarding the location of an effective sub-block in FIGS. 21 to 23 may indicate a predetermined location within an encoding unit. In one embodiment of the present disclosure, an image decoding device (2000) may determine a candidate location of an effective sub-block based on the shape of the effective sub-block and determine the location of the effective sub-block based on continuity according to the candidate location of the effective sub-block. An effective sub-block at a location with high recovery performance may have continuity with an adjacent recovery sample. For example, the amount of change between surrounding samples at the boundary of an effective sub-block at a location with high recovery performance may be small. In one embodiment of the present disclosure, the image decoding device (2000) may determine a cost regarding the amount of change between recovery samples based on at least one of the boundary of the effective sub-block and the boundary of the current block, and determine a ranking among the effective sub-blocks based on the cost. If information regarding the location of an effective sub-block indicates the ranking of a candidate location, there will be a large amount of information indicating a low ranking, so the compression efficiency may be increased.

[0306] FIG. 24 is a flowchart illustrating an image decoding method according to one embodiment of the present disclosure.

[0307] In one embodiment of the present disclosure, the image decoding method may be performed by an image decoding device (2000). For example, the image decoding device (2000) may perform each step of the image decoding method by having the processor of the image decoding device (2000) execute at least one instruction contained in memory.

[0308] In step S2410, the image decoder (2000) can identify a plurality of candidate locations for a significant sub-block containing at least one non-zero transformation coefficient within the current block.

[0309] In one embodiment of the present disclosure, the current block may be divided into a plurality of sub-blocks. For example, a sub-block transform may be applied to the current block. The current block may be divided into a plurality of transform units based on the application of the sub-block transform. Each transform unit of the current block to which the sub-block transform is applied may be referred to as a sub-block.

[0310] In one embodiment of the present disclosure, an image decoding device (2000) can determine the shape of an effective sub-block included in the current block. The image decoding device (2000) can obtain information related to the shape of the effective sub-block of the current block. For example, the image decoding device (2000) can obtain information related to the shape of the effective sub-block of the current block from a bitstream. In one embodiment of the present disclosure, information related to the shape of the effective sub-block may include at least one of ratio information indicating the ratio between the size of the current block and the size of the effective sub-block, division direction information indicating whether the current block is divided vertically, or corner information indicating whether the effective sub-block is located at the corner or center of the current block. For example, the ratio information may indicate whether the size ratio between the current block and the effective sub-block is 1 / 2 or 1 / 4. In one embodiment, the ratio information may be referred to as size information. For example, the division direction information may indicate whether the current block is divided vertically or horizontally.

[0311] In one embodiment of the present disclosure, the image decoding device (2000) can obtain information related to the shape of an effective sub-block of the current block from the encoding unit syntax structure of the bitstream.

[0312] In one embodiment of the present disclosure, the image decoding device (2000) may determine candidate locations of an effective sub-block based on information related to the shape of the sub-block. The image decoding device (2000) may determine the height and width of the sub-block based on information related to the shape of the sub-block obtained from a bitstream. The image decoding device (2000) may identify a plurality of candidate locations corresponding to the shape of the effective sub-block. For example, if the sub-block has a size of 1 / 4 of the encoding unit of vertical division, the candidate locations may include four locations where cu_sbt_quad_flag of FIG. 22 is 1 and cu_sbt_horizontal is 0. However, not limited thereto, the candidate locations may include predetermined locations among locations that may exist within the current block according to the shape of the sub-block, or locations determined based on the height and width of the current block.

[0313] In step S2420, the image decoder (2000) can determine the cost for each of the plurality of candidate locations based on at least one of the amount of change between the plurality of surrounding samples at the boundary of the valid sub-block and the amount of change between the plurality of surrounding samples at the boundary of the current block.

[0314] In one embodiment of the present disclosure, the amount of change between neighboring samples may refer to the continuity or discontinuity of the neighboring samples. Neighboring samples may include at least one of adjacent samples or non-adjacent samples along a boundary. For example, neighboring samples may include at least one of adjacent samples or non-adjacent samples along the vertical direction, horizontal direction, or slope direction corresponding to the intra-prediction mode of the boundary of the valid sub-block or the current block. In one embodiment of the present disclosure, the cost may include the sum of the amounts of change between a plurality of reconstructed samples adjacent to at least one of the boundary of the valid sub-block and the boundary of the current block. The cost according to one embodiment of the present disclosure is described with reference to FIGS. 27a, 27b, and 28.

[0315] In one embodiment of the present disclosure, the surrounding samples may be reconstructed samples. An image decoder (2000) may obtain a plurality of candidate reconstructed blocks based on applying an inverse transform kernel to a sub-block corresponding to each of a plurality of candidate locations. An image decoder (2000) may obtain a plurality of candidate reconstructed blocks based on performing inverse quantization and inverse transform on an effective sub-block corresponding to each of a plurality of candidate locations. For example, the image decoder (2000) may obtain residual blocks for each candidate location by performing inverse quantization and inverse transform on an effective sub-block. In one embodiment, the image decoder (2000) may determine quantization parameters applied to an effective sub-block corresponding to each of a plurality of candidate locations and perform inverse quantization based on the quantization parameters. The image decoder (2000) may perform inverse transform on the coefficients for which inverse quantization was performed.

[0316] The image decoder (2000) can obtain residual blocks for each candidate location based on applying an inverse transform kernel to a valid sub-block corresponding to each of a plurality of candidate locations. In one embodiment of the present disclosure, the image decoder (2000) can obtain residual blocks by applying at least one of a vertical inverse transform kernel and a horizontal inverse transform kernel to a valid sub-block located at a candidate location. In one embodiment of the present disclosure, the image decoder (2000) can obtain information regarding the transform coefficients of the valid sub-block from a bitstream. The image decoder (2000) can obtain residual blocks by applying an inverse transform kernel to the transform coefficients of the valid sub-block.

[0317] The image decoding device (2000) can obtain a plurality of candidate restoration blocks based on adding a prediction block for the current block to a residual block for each candidate position. For example, the image decoding device (2000) can obtain a prediction block using inter-prediction, obtain a first candidate restoration block by adding the prediction block to a residual block corresponding to a first candidate position, and obtain a second candidate restoration block by adding the prediction block to a residual block corresponding to a second candidate position.

[0318] In step S2430, the image decoding device (2000) can determine the ranking of the multiple candidate locations based on the cost for each of the multiple candidate locations.

[0319] In one embodiment of the present disclosure, the ranking may be determined according to the order of the cost of the candidate locations. For example, the ranking of the candidate location with the smallest cost is determined as 0, the ranking of the next smallest candidate location is determined as 1, and the ranking may increase as the cost increases. However, the lowest ranking is not limited to 0, and the lowest ranking may be set to 1. Furthermore, the embodiments are not limited thereto, and the ranking of the candidate location with a small cost may be set to have a large value.

[0320] In step S2440, the image decoding device (2000) can determine the location of an effective sub-block for restoring the current block among a plurality of candidate locations based on the ranking indicated by index information obtained from the bitstream.

[0321] In one embodiment of the present disclosure, index information may include information indicating a candidate location for restoring the current block among a plurality of candidate locations. An image decoding device (2000) may determine a candidate location corresponding to the rank indicated by the index information as a location for an effective sub-block for restoration. For example, if the index information indicates 1, an effective sub-block of a candidate location with a rank of 1 may be used for restoring the current block. When the rank is determined based on cost, the probability of selecting a candidate location with a lower cost rank increases, so the encoding efficiency may be increased.

[0322] In one embodiment of the present disclosure, the image decoding device (2000) can determine the location of an effective sub-block for restoring the current block by performing arithmetic decoding on index information. For example, the image decoding device (2000) can obtain an empty string by performing arithmetic decoding on index information using a context model. An empty string may mean that the value of a syntax element has been binarized.

[0323] In one embodiment of the present disclosure, the empty string may be encoded in binary for the rank indicated by the index information. For example, the empty string "01" may correspond to rank "1", and the empty string "11" may correspond to rank "3". In one embodiment of the present disclosure, the process of binarization based on fixed-length (FL) binarization may be described with reference to FIG. 29a or FIG. 29b.

[0324] In one embodiment of the present disclosure, a bean string may be encoded according to a truncated unary (TU) method. The rank represented by the index information may be determined based on the number of 1s or 0s contained in the bean string. For example, the bean string "1" may correspond to rank "0" because it does not contain 0s. For example, the bean string "001" may correspond to rank "2" because it contains 2 0s. In one embodiment of the present disclosure, the process of binarization based on the truncated unary may be described with reference to FIG. 30.

[0325] In one embodiment of the present disclosure, the image decoding device (2000) can obtain a rank indicated by index information based on a bean string. The image decoding device (2000) can determine the location of an effective sub-block for restoring the current block corresponding to the rank indicated by index information among a plurality of candidate locations.

[0326] In one embodiment of the present disclosure, index information may be obtained from a encoding unit syntax structure of a bitstream containing information regarding the current block. In one embodiment of the present disclosure, information regarding an effective sub-block may be obtained first in the bitstream among information regarding a plurality of sub-blocks of the current block. For example, if the current block is divided into four parts including an effective sub-block, a first sub-block, a second sub-block, and a third sub-block, information regarding the effective sub-block may be included in the bitstream to be obtained before information regarding the first sub-block, the second sub-block, and the third sub-block. For example, referring to Table 1, information regarding the effective sub-block may be obtained through a conversion unit syntax structure where cu_sbt_pos_flag is 0, but the embodiments are not limited thereto.

[0327] In one embodiment of the present disclosure, information regarding a valid sub-block may be stored ahead of information regarding another sub-block of the same shape as the valid sub-block in the bitstream. For example, information regarding a valid sub-block may be obtained before information regarding a first sub-block of the same shape as the valid sub-block included in the current block. For example, referring to Table 1, information regarding a valid sub-block may be obtained through a conversion unit syntax structure where cu_sbt_pos_flag is smaller than other sub-blocks of the same shape, but the embodiments are not limited thereto. Information regarding a valid sub-block may include information regarding conversion coefficients included in the valid sub-block.

[0328] In step S2450, the image decoding device (2000) can determine the locations of a plurality of sub-blocks included in the current block based on the locations of valid sub-blocks for the restoration of the current block.

[0329] In one embodiment of the present disclosure, the image decoding device (2000) can determine the locations of all sub-blocks included in the current block based on the locations of effective sub-blocks included in the current block. For example, the locations of sub-blocks included in the current block can be determined based on the size of a plurality of sub-blocks included in the current block and the locations of effective sub-blocks.

[0330] In step S2460, the image decoder (2000) can determine a boundary for applying filtering based on the locations of a plurality of determined sub-blocks.

[0331] The image decoding device (2000) can determine boundaries for applying deblocking filtering. The image decoding device (2000) can apply filtering to the boundaries of the encoding unit and the boundaries of the conversion unit.

[0332] In one embodiment of the present disclosure, the boundaries of the encoding unit and the boundaries of the conversion unit (e.g., sub-block) may be stored in memory. The image decoder (2000) may obtain a location in memory corresponding to each of the plurality of sub-blocks. For example, index information indicating a sub-block divided from the encoding unit may be obtained from memory. The image decoder (2000) may change the location obtained in memory based on the determined plurality of sub-block locations. For example, the location of the sub-block stored in memory may be set to a predetermined value. The image decoder (2000) may change the location obtained in memory to the plurality of sub-block locations determined in step S2450. The image decoder (2000) may store the changed location in memory. The image decoder (2000) may determine a boundary for applying filtering that includes the boundaries of the determined plurality of sub-blocks.

[0333] FIG. 25 is a drawing for explaining the process of determining the location of an effective sub-block based on cost according to one embodiment of the present disclosure.

[0334] In one embodiment of the present disclosure, an image decoder (2000) may obtain information regarding an effective sub-block from a bitstream. For example, the image decoder (2000) may obtain information regarding whether a sub-block transformation is applied (e.g., cu_sbt_flag). If the obtained information indicates that a sub-block transformation is applied, the image decoder (2000) may obtain information regarding the shape of the effective sub-block. For example, the image decoder (2000) may obtain information regarding the size of the effective sub-block (e.g., cu_sbt_quad_flag) and information regarding the orientation of the effective sub-block (e.g., cu_sbt_horizontal_flag). The image decoder (2000) may determine the shape of the effective sub-block based on the information regarding the shape of the effective sub-block. For example, referring to FIG. 25, information regarding the shape of the effective sub-block may mean that the size of the effective sub-block is 1 / 4 of the current block (e.g., encoding unit) and that it is a vertical division. Based on the information regarding the shape of the effective sub-block, the image decoder (2000) may determine that the height of the effective sub-block is the same as the height of the current block and the width of the effective sub-block is 1 / 4 of the width of the current block.

[0335] In one embodiment of the present disclosure, the image decoding device (2000) may determine candidate locations of an effective subblock based on the shape of the effective subblock. For example, the image decoding device (2000) may determine a first candidate location (2510) where the effective subblock is adjacent to the left side of the current block and a second candidate location (2515) where the effective subblock is adjacent to the right side of the current block. The first candidate location (2510) and the second candidate location (2515) may be referred to as Pos 0 and Pos 1, respectively.

[0336] The image decoder (2000) can obtain residual data from the bitstream. For example, the image decoder (2000) can obtain information related to the transform coefficients. The image decoder (2000) can perform inverse quantization (IQ) on the transform coefficients. The image decoder (2000) can perform inverse transformation (IT) on the transform coefficients to which inverse quantization has been applied. For example, the image decoder (2000) can apply an inverse transformation kernel to the transform coefficients (or the transform coefficients to which inverse quantization has been applied). In one embodiment of the present disclosure, the image decoder (2000) can determine that the transform coefficients for regions that are not valid sub-blocks of the current block are 0.

[0337] In one embodiment of the present disclosure, the image decoder (2000) can obtain a first residual block (2520) by applying an inverse transform kernel to an effective sub-block of a first candidate location (2510). For example, the image decoder (2000) can obtain the first residual block (2520) by applying DCT 8 as a horizontal transform kernel and DST 7 as a vertical transform kernel. In one embodiment of the present disclosure, the image decoder (2000) can obtain a second residual block (2525) by applying an inverse transform kernel to an effective sub-block of a second candidate location (2515). For example, the image decoder (2000) can obtain the second residual block (2525) by applying DST 7 as a horizontal transform kernel and a vertical transform kernel.

[0338] In one embodiment of the present disclosure, the image decoder (2000) may obtain a prediction block (2530) for the current block. For example, the image decoder (2000) may obtain the prediction block (2530) using inter-prediction. The image decoder (2000) may obtain a restoration block based on the prediction block and the residual block. For example, the image decoder (2000) may obtain a first restoration block (2540) based on the prediction block (2530) and the first residual block (2520), and obtain a second restoration block (2545) based on the prediction block (2530) and the second residual block (2525).

[0339] In one embodiment of the present disclosure, the image decoding device (2000) may determine a cost for a restoration block. The image decoding device (2000) may determine a cost based on the amount of change between samples adjacent to at least one of the boundary of an effective sub-block and the boundary of a current block. The image decoding device (2000) may determine a first cost (2550) based on a first restoration block (2540). For example, the image decoding device (2000) may determine a first cost (2550) based on the amount of change between samples surrounding the boundary of an effective sub-block and the amount of change between samples surrounding the boundary of a current block in the first restoration block (2540). The image decoding device (2000) may determine a second cost (2555) based on a second restoration block (2545). For example, the image decoding device (2000) can determine the second cost (2555) based on the amount of change between the surrounding samples of the boundary of the valid sub-block in the second recovery block (2545) and the amount of change between the surrounding samples of the boundary of the current block. Since the first candidate location (2510) and the second candidate location (2515) differ from each other in the left side of the current block and the right side of the current block, the first cost (2550) and the second cost (2555) differ in the recovery samples for determining the amount of change.

[0340] In one embodiment of the present disclosure, the image decoding device (2000) can determine the ranking of candidate locations. For example, the image decoding device (2000) can determine the ranking of the first candidate location (2510) and the second candidate location (2515) by comparing the first cost (2550) of the first candidate location (2510) and the second cost (2555) of the second candidate location (2515). The image decoding device (2000) can determine that the ranking is lower for candidate locations with lower costs. For example, if the second cost (2555) is smaller than the first cost (2550), the ranking (2575) corresponding to the second candidate location (2515) can be determined as "0", and the ranking (2570) corresponding to the first candidate location (2510) can be determined as "1".

[0341] In one embodiment of the present disclosure, an image decoding device (2000) may obtain index information (2580) from a bitstream. The index information (2580) may indicate the rank of the location of an effective sub-block used for the restoration of the current block. For example, if the index information (2580) indicates rank "1", it may mean that a first candidate location (2510) corresponding to rank "1" is used for the restoration of the current block. The image decoding device (2000) may determine the location (2590) of an effective sub-block for restoring the current block based on the index information (2580). By obtaining information indicating a rank determined based on cost instead of information directly indicating the location of the effective sub-block, the image decoding device (2000) may increase the compression ratio of the information and increase the decoding efficiency.

[0342] FIG. 26 is a drawing for explaining the process of determining the location of an effective sub-block based on cost according to one embodiment of the present disclosure.

[0343] In one embodiment of the present disclosure, an image decoder (2000) may obtain information regarding an effective sub-block from a bitstream. For example, the image decoder (2000) may obtain information regarding whether a sub-block transformation is applied. If the obtained information indicates that a sub-block transformation is applied, the image decoder (2000) may obtain information regarding the shape of the effective sub-block. For example, the image decoder (2000) may obtain information regarding the size of the effective sub-block (e.g., cu_sbt_quad_flag) and information regarding the orientation of the effective sub-block (e.g., cu_sbt_corner_flag). The information regarding the orientation of the effective sub-block may indicate whether the effective sub-block is located at the corner of the current block. For example, if cu_sbt_corner_flag is 1, it indicates that the valid sub-block is located at the corner of the current encoding unit, and if cu_sbt_corner_flag is 0, it indicates that the valid sub-block is located at the center of the current encoding unit.

[0344] The image decoder (2000) can determine the shape of an effective sub-block based on information regarding the shape of the effective sub-block. For example, referring to FIG. 26, the information regarding the shape of the effective sub-block may mean that the size of the effective sub-block is 1 / 4 of the current block (e.g., encoding unit) and is located at a corner. Based on the information regarding the shape of the effective sub-block, the image decoder (2000) can determine that the height of the effective sub-block is 1 / 2 of the height of the current block and the width of the effective sub-block is 1 / 2 of the width of the current block.

[0345] In one embodiment of the present disclosure, the image decoding device (2000) may determine candidate locations of an effective sub-block based on the shape of the effective sub-block. For example, the image decoding device (2000) may determine a first candidate location (2610) where the effective sub-block is adjacent to the upper-left side of the current block, a second candidate location (2612) where the effective sub-block is adjacent to the upper-right side of the current block, a third candidate location (2614) where the effective sub-block is adjacent to the lower-left side of the current block, and a fourth candidate location (2616) where the effective sub-block is adjacent to the lower-right side of the current block. The candidate locations (2610, 2612, 2614, 2616) may be referred to as Pos 0, Pos 1, Pos 2, and Pos 3, respectively.

[0346] The image decoder (2000) can obtain residual data from the bitstream. For example, the image decoder (2000) can obtain information related to the transform coefficients. The image decoder (2000) can perform inverse quantization (IQ) on the transform coefficients. The image decoder (2000) can perform inverse transformation (IT) on the transform coefficients to which inverse quantization has been applied. For example, the image decoder (2000) can apply an inverse transformation kernel to the transform coefficients (or, the transform coefficients to which inverse quantization has been applied). In one embodiment of the present disclosure, the image decoder (2000) can determine that the transform coefficients for regions that are not valid sub-blocks of the current block are 0.

[0347] In one embodiment of the present disclosure, the image decoder (2000) can obtain a first residual block (2620) by applying an inverse transform kernel to an effective sub-block of a first candidate location (2610). For example, the image decoder (2000) can obtain the first residual block (2620) by applying DCT 8 as a horizontal transform kernel and a vertical transform kernel. In one embodiment of the present disclosure, the image decoder (2000) can obtain a second residual block (2522) by applying an inverse transform kernel to an effective sub-block of a second candidate location (2612). For example, the image decoder (2000) can obtain the second residual block (2522) by applying DST 7 as a horizontal transform kernel and DCT 8 as a vertical transform kernel. In one embodiment of the present disclosure, the image decoder (2000) may obtain a third residual block (2624) by applying an inverse transform kernel to the valid sub-block of the third candidate location (2614). For example, the image decoder (2000) may obtain the third residual block (2624) by applying DCT 8 as a horizontal transform kernel and DST 7 as a vertical transform kernel. In one embodiment of the present disclosure, the image decoder (2000) may obtain a fourth residual block (2526) by applying an inverse transform kernel to the valid sub-block of the fourth candidate location (2616). For example, the image decoder (2000) may obtain the fourth residual block (2526) by applying DST 7 as a horizontal transform kernel and a vertical transform kernel.

[0348] In one embodiment of the present disclosure, the image decoder (2000) may obtain a prediction block (2630) for the current block. For example, the image decoder (2000) may obtain the prediction block (2630) using inter-prediction. The image decoder (2000) may obtain a restoration block based on the prediction block and the residual block. For example, the image decoder (2000) may obtain a first restoration block (2640) based on the prediction block (2630) and the first residual block (2620), and obtain a second restoration block (2642) based on the prediction block (2630) and the second residual block (2622). The image decoding device (2000) can obtain a third restoration block (2644) based on the prediction block (2630) and the third residual block (2624), and obtain a fourth restoration block (2646) based on the prediction block (2630) and the fourth residual block (2626).

[0349] In one embodiment of the present disclosure, the image decoding device (2000) may determine a cost for a restored block. The image decoding device (2000) may determine a cost based on the amount of change between samples adjacent to at least one of the boundary of an effective sub-block and the boundary of a current block. The image decoding device (2000) may determine a cost (2650, 2652, 2654, 2656) corresponding to a restored block based on the restored blocks (2640, 2642, 2644, 2646). For example, the image decoding device (2000) can determine a cost (2650, 2652, 2654, 2656) corresponding to the restoration block based on the amount of change between surrounding samples of the boundary of the valid sub-block in the restoration blocks (2640, 2642, 2644, 2646) and the amount of change between surrounding samples of the boundary of the current block.

[0350] In one embodiment of the present disclosure, the image decoding device (2000) may determine a ranking for candidate locations. For example, the image decoding device (2000) may determine a ranking (2670, 2672, 2674, 2676) for a plurality of candidate locations (2610, 2612, 2614, 2616) by comparing costs (2650, 2652, 2654, 2656) corresponding to a plurality of candidate locations (2610, 2612, 2614, 2616). The image decoding device (2000) may determine that a candidate location with a low cost has a lower ranking.

[0351] In one embodiment of the present disclosure, an image decoding device (2000) may obtain index information (2680) from a bitstream. The index information (2680) may indicate the ranking of the positions of effective sub-blocks used for the restoration of the current block. The image decoding device (2000) may determine the positions (2690) of effective sub-blocks for the restoration of the current block based on the index information (2680). By obtaining information indicating a ranking determined based on cost instead of information directly indicating the positions of effective sub-blocks, the image decoding device (2000) may increase the compression ratio of the information and thus increase the decoding efficiency.

[0352] FIG. 27a is a drawing for explaining a method for determining a cost according to one embodiment of the present disclosure.

[0353] In one embodiment of the present disclosure, the image decoding device (2000) can determine the cost for a candidate location based on the amount of change between a plurality of surrounding samples at the boundary of an effective sub-block and the amount of change between a plurality of surrounding samples at the boundary of a current block. In one embodiment, the surrounding samples may be reconstructed samples.

[0354] For example, referring to FIG. 27a, the effective sub-block (2720) has half the size of the encoding unit and may be located to the left of the encoding unit. The image decoder (2000) may determine the cost based on at least one surrounding sample, which is either the boundary of the effective sub-block (2720) or the boundary of the current block (2710). The surrounding samples may include adjacent samples and / or non-adjacent samples.

[0355] The image decoding device (2000) can determine the cost using the amount of change between surrounding samples of a first boundary (2730), which is the upper boundary of the current block (2710). The image decoding device (2000) can determine the cost using the amount of change between surrounding samples of a second boundary (2740), which is the left boundary of the current block (2710) or the left boundary of the effective sub-block (2720). The image decoding device (2000) can determine the cost using the amount of change between surrounding samples of a third boundary (2750), which is the right boundary of the effective sub-block (2720). In one embodiment of the present disclosure, the image decoding device (2000) can determine the cost based on surrounding samples of the boundary of the effective sub-block (2720) and surrounding samples of the boundary of the current block (2710). For example, the image decoding device (2000) can determine the cost as the sum of the amounts of change between the surrounding samples of the first boundary (2730), the second boundary (2740), and the third boundary (2750). For example, the cost can be determined as in Equation 1 and Equation 2.

[0356]

[0357]

[0358] Here, costbk represents the sum of the changes in the surrounding samples of the k-th boundary, and cost corresponds to the sum of the changes in all boundaries. size(k) may represent the size (e.g., height or width) of the k-th boundary. Rx, y may represent the value of the restored sample located at (x, y) relative to the top-left side of the current block.

[0359] In one embodiment of the present disclosure, as shown in Equation 2, the image decoder (2000) can determine the cost using the difference of surrounding samples of the boundary. The image decoder (2000) can determine the cost using the sum of the absolute values ​​of the differences of surrounding samples of the boundary. The image decoder (2000) can determine the boundary for determining the cost using the boundary of the current block (2710) and the boundary of the effective sub-block (2720).

[0360] Referring to FIG. 27a, the image decoding device (2000) can determine the cost for the left position of the effective sub-block (2720) as in Equation 3.

[0361]

[0362] FIG. 27b is a drawing for explaining a method for determining a cost according to one embodiment of the present disclosure.

[0363] In one embodiment of the present disclosure, the image decoding device (2000) can determine the cost for a candidate location based on the amount of change between a plurality of reconstructed samples adjacent to the boundary of an effective sub-block and the amount of change between a plurality of reconstructed samples adjacent to the boundary of a current block. Although the cost is determined using two reconstructed samples in FIG. 27a, it is not limited thereto, and the cost can be determined using three or more reconstructed samples as in FIG. 27b. For example, referring to FIG. 27b, the effective sub-block (2720) has half the size of the encoding unit and can be located to the left of the encoding unit.

[0364] The image decoder (2000) can determine the cost based on at least one surrounding sample of the boundary of the effective sub-block (2720) or the boundary of the current block (2710). The image decoder (2000) can determine the cost using the amount of change between surrounding samples of the first boundary (2730), which is the upper boundary of the current block (8710). The image decoder (2000) can determine the cost using the amount of change between surrounding samples of the second boundary (2740), which is the left boundary of the current block (2710) or the left boundary of the effective sub-block (2720). The image decoder (2000) can determine the cost using the amount of change between surrounding samples of the third boundary (2750), which is the right boundary of the effective sub-block (2720). In one embodiment of the present disclosure, the image decoder (2000) may determine the cost based on the surrounding samples of the boundary of the effective sub-block (2720) and the surrounding samples of the boundary of the current block (2710). For example, the image decoder (2000) may determine the cost as the sum of the amounts of change between the surrounding samples of the first boundary (2730), the second boundary (2740), and the third boundary (2750). For example, the cost may be determined by referring to Equation 1 and Equation 4.

[0365]

[0366] Here, costbk represents the sum of the changes in the surrounding samples of the k-th boundary, and cost corresponds to the sum of the changes in all boundaries. size(k) may represent the size (e.g., height or width) of the k-th boundary. Rx, y may represent the value of the restored sample located at (x, y) relative to the top-left side of the current block.

[0367] In one embodiment of the present disclosure, the image decoding device (2000) may determine the number of surrounding samples of a boundary included in the valid sub-block (2720) or the current block (2710) to be one less than the number of restored samples outside the current block (2720). Referring to FIG. 27b, the cost may be determined by using one restored sample of the valid sub-block (2720) and two restored samples outside the current block (2710), but is not limited thereto.

[0368] In one embodiment of the present disclosure, as shown in Equation 4, the image decoder (2000) can determine the cost using the difference of surrounding samples of the boundary. The image decoder (2000) can determine the cost using the sum of the absolute values ​​of the difference of surrounding samples of the boundary. The image decoder (2000) can determine the boundary for determining the cost using the boundary of the current block (2710) and the boundary of the effective sub-block (2720).

[0369] Referring to FIG. 27b, the image decoding device (2000) can determine the cost for the left position of the effective sub-block (2720) as in Equation 5.

[0370]

[0371] In one embodiment of the present disclosure, the image decoder (2000) may determine surrounding samples of a boundary based on an intra prediction mode. For example, as in the example described in FIG. 27a, in the case of an intra prediction mode indicating an upper-left 45-degree direction, the image decoder (2000) may determine surrounding samples of the current block (2700) located at (x0-1, y0-1) and (x0-2, y0-2) as adjacent reconstruction samples corresponding to the first reconstruction sample (x0, y0) of the current block (2700). As described in FIG. 27a, the image decoder (2000) may determine a cost using three or more reconstruction samples according to the intra prediction mode.

[0372] FIG. 28 is a drawing for explaining a method for determining a cost according to one embodiment of the present disclosure.

[0373] In one embodiment of the present disclosure, the image decoding device (2000) can determine the cost for a candidate location based on the amount of change between a plurality of reconstructed samples adjacent to the boundary of the current block (2800) and the amount of change between a plurality of reconstructed samples adjacent to the boundary of the effective sub-block (2830).

[0374] The image decoder (2000) can determine the cost using the amount of change between surrounding samples of the first boundary (2830), which is the upper boundary of the current block (2800), the amount of change between surrounding samples of the second boundary (2840), which is the left boundary of the current block (2800), and the third boundary (2850), which is the boundary of the effective sub-block (2810). For example, the image decoder (2000) can determine the cost as the sum of the amounts of change between surrounding samples of the first boundary (2830), the second boundary (2840), and the third boundary (2850). The image decoder (2000) can determine the surrounding samples of the boundary based on an intra prediction mode. The image decoder (2000) can determine the surrounding samples for determining the cost based on the angle of the intra prediction mode. For example, in the case of an intra prediction mode indicating a 45-degree direction to the upper left, the image decoder (2000) can determine a recovery sample around the current block (2800) located at (x0-1, y0-1) as an adjacent recovery sample corresponding to the first recovery sample (x0, y0) of the current block (2800). For example, the cost can be determined as in Equation 6.

[0375]

[0376] Here, cost is the sum of the changes of all boundaries and can correspond to cost. R x, ycan mean the value of a restoration sample located at (x, y) relative to the upper-left side of the current block (2800).

[0377] In one embodiment of the present disclosure, as shown in Equation 10, the image decoder (2000) may determine the cost using the difference of surrounding samples of the boundary. However, the adjacent reconstructed samples are not limited to being adjacent in the horizontal or vertical direction, and may be determined based on the direction of the intra prediction mode. The intra prediction mode may be determined through a bitstream or derived using DIMD or TIMD. For example, the intra prediction mode may be determined by performing DIMD or TIMD on the template of the current block or the template of the inter-reference block of the current block. The image decoder (2000) may determine the cost using the sum of the absolute values ​​of the differences of surrounding samples of the boundary.

[0378] FIG. 29a is a drawing for explaining a binarization method of index information according to one embodiment of the present disclosure.

[0379] In one embodiment of the present disclosure, the image decoding device (2000) can determine a ranking based on the cost for candidate positions. For example, if the cost is lower in the order of the third candidate position (Pos2), the first candidate position (Pos0), the second candidate position (Pos1), and the fourth candidate position (Pos3), the rankings for the third candidate position (Pos2), the first candidate position (Pos0), the second candidate position (Pos1), and the fourth candidate position (Pos3) can be determined as 0, 1, 2, and 3, respectively.

[0380] In one embodiment of the present disclosure, an image decoding device (2000) can obtain index information from a bitstream. The image decoding device (2000) can obtain an empty string by performing arithmetic encoding on the index information obtained from the bitstream. The image decoding device (2000) can obtain the rank represented by the index information by performing inverse binary conversion on the empty string.

[0381] In one embodiment of the present disclosure, the empty string may be binarized based on fixed-length (FL) binarization. Fixed-length binarization may mean that binarization is performed for a fixed length. For example, referring to FIG. 29a, the empty string for the candidate positions may all be represented by 2 bits.

[0382] The empty string corresponding to the third candidate position (pos2) with a rank of 0 can be represented as "00". Similarly, the empty strings representing the first candidate position (pos0), second candidate position (pos1), and fourth candidate position (pos3) with ranks of 1, 2, and 3 can be represented as "01", "10", and "11", respectively. Since there is a high probability that index information with a lower rank according to cost will be signaled, the image decoding method according to one embodiment of the present disclosure can improve the efficiency of arithmetic encoding.

[0383] FIG. 29b is a diagram illustrating a binarization method of index information according to one embodiment of the present disclosure.

[0384] In one embodiment of the present disclosure, the image decoding device (2000) can determine the ranking based on the cost for candidate locations. For example, if the costs are lower in the order of the third candidate location (Pos2), the first candidate location (Pos0), the second candidate location (Pos1), and the fourth candidate location (Pos3), the third candidate location (Pos2) can be determined as the lowest ranked location.

[0385] In one embodiment of the present disclosure, the image decoding device (2000) can obtain index information from a bitstream. The image decoding device (2000) can obtain an empty string by performing arithmetic encoding on the index information obtained from the bitstream. The image decoding device (2000) can determine a location for restoring the current block based on the empty string.

[0386] The first candidate position (Pos0), the second candidate position (Pos1), the third candidate position (Pos2), and the fourth candidate position (Pos3) may correspond to the binary numbers "00", "01", "10", and "11" for 0, 1, 2, and 3, respectively. In one embodiment, the empty string may represent the difference from the lowest-ranked position based on cost. For example, if the lowest-ranked position based on cost is the third candidate position (Pos2) and the empty string is "01", the position for restoring the current block may be "11", which differs from "01" by "10" corresponding to the third candidate position. The difference in positions may be determined based on exclusive OR.

[0387] If the third candidate position with the lowest rank (pos2) is used for the restoration of the current block, the empty string may be represented as "00". Similarly, if the first candidate position (pos0), the second candidate position (pos1), or the fourth candidate position (pos3) is used for the restoration of the current block, the empty string may be represented as "10", "11", or "01", respectively.

[0388] FIG. 30 is a drawing for explaining a binarization method of index information according to one embodiment of the present disclosure.

[0389] In one embodiment of the present disclosure, the image decoding device (2000) can determine a ranking based on the cost for candidate positions. For example, if the cost is lower in the order of the third candidate position (Pos2), the first candidate position (Pos0), the second candidate position (Pos1), and the fourth candidate position (Pos3), the rankings for the third candidate position (Pos2), the first candidate position (Pos0), the second candidate position (Pos1), and the fourth candidate position (Pos3) can be determined as 0, 1, 2, and 3, respectively.

[0390] In one embodiment of the present disclosure, an image decoding device (2000) can obtain index information from a bitstream. The image decoding device (2000) can obtain an empty string by performing arithmetic encoding on the index information obtained from the bitstream. The image decoding device (2000) can obtain the rank represented by the index information by performing inverse binary conversion on the empty string.

[0391] In one embodiment of the present disclosure, a bean string may be binarized based on truncated unary (TU) binarization. Truncated unary binarization may mean performing binarization using the number of 1s or the number of 0s contained in the bean string. For example, referring to FIG. 30, the bean strings for candidate positions may be determined based on the number of 0s. For example, a third candidate position (pos2) for rank "0" may correspond to a bean string "1" which has 0 0s. A first candidate position (pos0), a second candidate position (pos1), and a fourth candidate position (pos3) for ranks "1", "2", and "3" may correspond to bean strings "01", "001", and "000" which have 1, 2, and 3 0s, respectively. In one embodiment of the present disclosure, the last bean string in the truncated unary binarization may not contain 1s. For example, as the rank increases, the number of leading zeros is increased by one each time, but the empty string of the last rank may be represented only as "000" and may not include 1. However, not limited thereto, the empty strings for candidate positions may be determined based on the number of 1s. Since there is a high probability that index information with a lower rank according to cost will be signaled, the image decoding method according to one embodiment of the present disclosure can improve the efficiency of arithmetic encoding.

[0392] FIG. 31a is a drawing for explaining the process of applying filtering according to one embodiment of the present disclosure.

[0393] Referring to FIG. 31a, the boundary (3100) may mean a boundary to which filtering is applied. Samples q1, q2, q3, and q4 may mean reconstructed samples included in the current block (3110). Samples p1, p2, p3, and p4 may mean reconstructed samples of the surrounding block (3120). In one embodiment of the present disclosure, filtering may be applied to the boundaries of the encoding unit and / or the conversion unit.

[0394] The image decoder (2000) can obtain a reconstructed sample based on residual samples and prediction samples. The image decoder (2000) can generate a reconstructed sample with applied filtering by applying filtering to the reconstructed sample. The image decoder (2000) can apply filtering along the vertical and horizontal boundaries of a block. For example, the image decoder (2000) can apply filtering along the vertical boundary of a block and then apply filtering along the horizontal boundary.

[0395] The image decoding device (2000) can determine the number of samples to which the transformation is applied based on the length of the block. For example, if the length of the block is greater than a first threshold and less than a second threshold, the number of samples to which the filter is applied can be determined to be 3.

[0396] In one embodiment of the present disclosure, the image decoder (2000) may determine a filtering strength based on the prediction mode of the current block (3110) and the surrounding block (3120). For example, the image decoder (2000) may determine a filtering strength based on whether the prediction mode of the current block (3110) and / or the surrounding block (3120) is an intra mode.

[0397] In one embodiment of the present disclosure, the image decoder (2000) may determine the filtering strength based on the slice type of the current block (3110) and the surrounding block (3120). For example, the image decoder (2000) may determine the filtering strength based on whether the slice type of the current block (3110) and the surrounding block (3120) is a P slice or a B slice.

[0398] In one embodiment of the present disclosure, the filtering length and filtering strength applied to the current block (3110) may be determined for every 4 x 4 blocks. The image decoder (2000) may determine the filtering strength and filtering length for every 4 x 4 blocks along the boundary of the current block (3110). The image decoder (2000) may obtain a reconstructed sample with filtering applied by modifying the reconstructed sample based on the filtering strength and filtering length.

[0399] Although Figure 31a is illustrated as having horizontal filtering applied, it is not limited thereto, and vertical filtering can also be applied in a similar manner to that described in Figure 31a, using a restoration sample of the block to be filtered along the vertical boundary and a restoration sample of the surrounding block to determine the filtering intensity.

[0400] In one embodiment of the present disclosure, the image decoding device (2000) may obtain information regarding a conversion unit from a bitstream. For example, the image decoding device (2000) may obtain information regarding a plurality of sub-blocks included in the encoding unit through a conversion unit syntax. In one embodiment of the present disclosure, the encoding unit may be divided into various forms including a plurality of sub-blocks. In one embodiment of the present disclosure, the image decoding device (2000) does not directly obtain information indicating the location for the restoration of an effective sub-block, but rather obtains the order of the location for restoration. Since the actual location of the effective sub-block is not determined, the image decoding device (2000) obtains information regarding the effective sub-block through a conversion unit syntax of a predetermined order of the bitstream, rather than a conversion unit syntax of the actual location. The image decoding device (2000) applies filtering using the location of the effective sub-block. Therefore, information regarding the location of a plurality of sub-blocks included in the current block is required for filtering to be applied. The image decoding device (2000) can store or modify information regarding the location (or boundary) of a plurality of sub-blocks included in the current block to apply filtering. With reference to FIGS. 32 to 38, a process for obtaining information regarding valid sub-blocks included in the current block is described.

[0401] FIG. 31b is a drawing for explaining the process of determining a boundary for applying filtering according to one embodiment of the present disclosure.

[0402] Referring to FIG. 31b, the encoding unit may be divided into one or more conversion units. For example, the first encoding unit may be divided into two conversion units TU0 and TU1, the second encoding unit may be divided into two conversion units TU2 and TU3, the third encoding unit may be divided into one conversion unit TU4, and the fourth encoding unit may be divided into three conversion units TU5, TU6, and TU7. FIG. 31b is illustrated as an example where TU1 and TU3 are each valid sub-blocks for convenience of explanation, but is not limited thereto and can be applied in the same way to sub-blocks of other shapes and positions.

[0403] The first block (3140) indicates the location and type of conversion unit that encodes an image in the image encoding device (3900). For example, a 1 / 2 size effective sub-block may be located below the first encoding block, and a 1 / 4 size effective sub-block may be located to the right of the second encoding block.

[0404] The second block (3142) indicates the location and type of conversion unit included in the bitstream. The bitstream may contain information (e.g., encoding unit flag, information regarding the sign of the coefficient, and information regarding the magnitude of the coefficient) in the order of TU0 -> TU1 -> ... -> TU7. Information regarding the valid sub-block may be transmitted as the first conversion unit information of each encoding unit. For example, information regarding TU1 of the first block (3140) may be included in the order of TU0 of the second block (3142). For example, information regarding TU3 of the first block (3140) may be included in the third position, which is the order of TU2 of the second block (3142), rather than being included in the fourth position of the bitstream.

[0405] The third block (3144) indicates the location and type of the conversion unit after the image decoder (2000) adjusts the location of the conversion unit based on information regarding the location of the valid sub-block. As described above, the image decoder (2000) can adjust the location of the conversion unit, such as the second block (3142), to the third block (3144) based on the ranking of a plurality of candidate locations and the ranking indicated by index information obtained from the bitstream.

[0406] The fourth block (3146) represents the unit to which the deblocking filter is applied in the state of the second block (3142) before adjustment, as with the third block (3144). The deblocking filter can be applied in N x N units (e.g., 2 x 2, 4 x 4, or 8 x 8, etc.). Since the position of the effective sub-blocks in the fourth block (3146) is not adjusted differently from the first block (3140), the result is different from applying deblocking to the first block (3140). For example, in the first block (3140), TU1 and TU3 are effective sub-blocks, so the coefficients of TU0 and TU2 are 0, but in the fourth block (3146), TU0 and TU2 are effective sub-blocks, so the coefficients may not be 0. Therefore, the intensity and result of applying the deblocking filter may differ.

[0407] The first filter boundary (3150) may represent a filter boundary corresponding to the first block (3142). For example, the first filter boundary (3150) may be determined by the image decoder (2000) based on information regarding the size of the conversion unit obtained from the bitstream. The first filter boundary (3150) may be stored in memory. The image decoder (2000) may change the first filter boundary (3150) to the second filter boundary (3152), such as adjusting the second block (3142) to the third block (3144). For example, the image decoder (2000) may adjust the first filter boundary (3150) stored in memory based on index information or the third block (3144). The image decoder (2000) may apply filtering in the same way as the encoded block by adjusting the position of the conversion unit.

[0408] FIG. 32 is a drawing for explaining the process of obtaining information regarding an effective sub-block according to one embodiment of the present disclosure.

[0409] Referring to FIG. 32, the current block can be divided into two sub-blocks of the same size. For example, the current block can be divided in a vertical direction or a horizontal direction.

[0410] The valid sub-block (3210) represents the left block into which the current block is vertically divided, and the valid sub-block (3220) represents the right block into which the current block is vertically divided. For example, if the current block is divided in the vertical direction, the position of the left block may be referred to as the position corresponding to index 0 (e.g., Pos 0), and the position of the right block may be referred to as the position corresponding to index 1 (e.g., Pos 1). M' (e.g., 0' or 1') shown in FIGS. 32 to 38 represents an index indicating the actual position of the valid sub-block, and N (e.g., 0 or 1) may represent the order included in the bitstream. For example, 1' = 0 in FIG. 32 may mean that the actual position of the valid sub-block is pos1, but the information in the bitstream is included in the order of pos0.

[0411] In one embodiment of the present disclosure, the image decoding device (2000) can obtain information regarding an effective sub-block from a bitstream. The image decoding device (2000) can first obtain information regarding a left block through a conversion unit syntax structure corresponding to index 0, and then obtain information regarding a right block through a conversion unit syntax structure corresponding to index 1. However, the image decoding device (2000) according to one embodiment of the present disclosure can determine the location of the effective sub-block using the image decoding method of FIG. 24 after obtaining information regarding the effective sub-block. Accordingly, the image decoding device (2000) can obtain information regarding an effective sub-block through a conversion unit syntax structure in a predetermined order of the bitstream. For example, the effective sub-block (3210) corresponding to the left block and the effective sub-block (3220) corresponding to the right block can both obtain information through a conversion unit syntax corresponding to index 0.

[0412] In this case, the valid sub-block (3210) obtains information through a conversion unit syntax structure identical to the actual location, but the valid sub-block (3220) obtains information through a conversion unit syntax structure different from the actual location.

[0413] In one embodiment of the present disclosure, the image decoding device (2000) must determine the location of the sub-block to apply filtering. The strength of the filtering or the length of the filtering is determined based on the size of the sub-block. Accordingly, modification of information regarding the location of the previously stored conversion unit is required. If the location of the effective sub-block (3220) is not changed, the information of the effective sub-block (3220), which is the right block, may be used for filtering the left block. FIG. 32 shows sub-blocks of the same size, but the size of the sub-blocks is not limited thereto and the sizes of the sub-blocks may differ as shown in FIG. 33, FIG. 34, and FIG. 38 described later.

[0414] In one embodiment of the present disclosure, the image decoding device (2000) may determine the locations of sub-blocks included in the current block based on the location of an effective sub-block. For example, the image decoding device (2000) may determine the index of the effective sub-block (3220) to 1, and determine the index of the sub-block from which information is obtained through the conversion unit syntax structure of index 1 to 0. The image decoding device (2000) may store the locations of a plurality of conversion units of the current block in memory. The image decoding device (2000) may change and store the locations of the conversion units stored in memory to match the actual locations.

[0415] The valid sub-block (3230) represents the upper block to which the current block is horizontally divided, and the valid sub-block (3240) represents the lower block to which the current block is horizontally divided. For example, if the current block is divided horizontally, the location of the upper block may be referred to as the location corresponding to index 0 (e.g., Pos 0), and the location of the lower block may be referred to as the location corresponding to index 1 (e.g., Pos 1). Just like the vertically divided sub-block, the information in the horizontally divided sub-block must be modified and stored to match the location of the sub-block in order to apply filtering. Although the valid sub-block (3230, 3240) differs from the valid sub-block (3210, 3220) in that it is horizontally divided, the process of the image decoding device (2000) acquiring information through the bitstream and determining the location can be applied in the same way.

[0416] FIG. 33 is a drawing for explaining the process of obtaining information regarding an effective sub-block according to one embodiment of the present disclosure.

[0417] Referring to FIG. 33, the current block can be divided into two sub-blocks of different sizes. For example, an effective sub-block can have a size of 1 / 4 of the current block. The current block can be divided in a vertical or horizontal direction.

[0418] The valid sub-block (3310) represents the left block into which the current block is vertically divided, and the valid sub-block (3320) represents the right block into which the current block is vertically divided. For example, if the current block is divided in the vertical direction, the location of the left block may be referred to as the location corresponding to index 0 (e.g., Pos 0) and the location of the right block may be referred to as the location corresponding to index 1 (e.g., Pos 1).

[0419] In one embodiment of the present disclosure, the image decoding device (2000) can obtain information regarding an effective sub-block from a bitstream. The image decoding device (2000) can first obtain information regarding a left block through a conversion unit syntax structure corresponding to index 0, and then obtain information regarding a right block through a conversion unit syntax structure corresponding to index 1. However, the image decoding device (2000) according to one embodiment of the present disclosure can determine the location of the effective sub-block using the image decoding method of FIG. 24 after obtaining information regarding the effective sub-block. Accordingly, the image decoding device (2000) can obtain information regarding an effective sub-block through a conversion unit syntax structure in a predetermined order of the bitstream. For example, the effective sub-block (3310) corresponding to the left block and the effective sub-block (3320) corresponding to the right block can both obtain information through a conversion unit syntax corresponding to index 0.

[0420] In this case, the valid sub-block (3310) obtains information through a conversion unit syntax structure identical to the actual location, but the valid sub-block (3320) obtains information through a conversion unit syntax structure different from the actual location.

[0421] In one embodiment of the present disclosure, the image decoding device (2000) must determine the location of a sub-block to apply filtering. The strength of the filtering or the length of the filtering is determined based on the size of the sub-block. Accordingly, modification of information regarding the location of a previously stored conversion unit is required. If the location of the effective sub-block (3320) is not changed, the filtering length may be determined based on the size of the effective sub-block (3320), which is the right block, for filtering of the left block.

[0422] In one embodiment of the present disclosure, the image decoding device (2000) may determine the locations of sub-blocks included in the current block based on the location of an effective sub-block. For example, the image decoding device (2000) may determine the index of the effective sub-block (3320) to 1, and determine the index of the sub-block from which information is obtained through the conversion unit syntax structure of index 1 to 0. The image decoding device (2000) may store the locations of a plurality of conversion units of the current block in memory. The image decoding device (2000) may change and store the locations of the conversion units stored in memory to match the actual locations.

[0423] The valid sub-block (3330) represents the upper block to which the current block is horizontally divided, and the valid sub-block (3340) represents the lower block to which the current block is horizontally divided. For example, if the current block is divided horizontally, the location of the upper block may be referred to as the location corresponding to index 0 (e.g., Pos 0), and the location of the lower block may be referred to as the location corresponding to index 1 (e.g., Pos 1). Just like the vertically divided sub-block, the information in the horizontally divided sub-block must be modified and stored to match the location of the sub-block in order to apply filtering. Although the valid sub-block (3330, 3340) differs from the valid sub-block (3310, 3320) in that it is horizontally divided, the process of the image decoding device (2000) acquiring information through the bitstream and determining the location can be applied in the same way.

[0424] FIG. 34 is a drawing for explaining the process of obtaining information regarding an effective sub-block according to one embodiment of the present disclosure.

[0425] Referring to FIG. 34, the current block can be divided into three sub-blocks. For example, the effective sub-block may have half the size of the current block, and the remaining sub-block may have one-fourth the size of the current block. The current block may be divided in a vertical or horizontal direction.

[0426] The valid sub-block (3410) may represent the left block into which the current block is vertically divided, the valid sub-block (3420) may represent the middle block into which the current block is vertically divided, and the valid sub-block (3430) may represent the right block into which the current block is vertically divided. For example, when the current block is divided in the vertical direction, the location of the left block may be referred to as the location corresponding to index 0 (e.g., Pos 0), the location of the middle block may be referred to as the location corresponding to index 1 (e.g., Pos 1), and the location of the right block may be referred to as the location corresponding to index 2 (e.g., Pos 2).

[0427] In one embodiment of the present disclosure, the image decoding device (2000) can obtain information regarding an effective sub-block from a bitstream. The image decoding device (2000) can first obtain information regarding a left block through a conversion unit syntax structure corresponding to index 0, and then sequentially obtain information regarding an intermediate block and a right block through conversion unit syntax structures corresponding to index 1 and index 2. However, the image decoding device (2000) according to one embodiment of the present disclosure can determine the location of the effective sub-block using the image decoding method of FIG. 24 after obtaining information regarding the effective sub-block. Accordingly, the image decoding device (2000) can obtain information regarding the effective sub-block through a conversion unit syntax structure in a predetermined order of the bitstream. For example, the valid sub-block (3410) corresponding to the left block, the valid sub-block (3420) corresponding to the middle block, and the valid sub-block (3430) corresponding to the right block can all obtain information through the conversion unit syntax corresponding to index 0.

[0428] In this case, the valid sub-block (3410) obtains information through a conversion unit syntax structure identical to the actual location, but the valid sub-blocks (3420, 3430) obtain information through a conversion unit syntax structure different from the actual location.

[0429] In one embodiment of the present disclosure, the image decoding device (2000) must determine the location of a sub-block to apply filtering. The strength of the filtering or the length of the filtering is determined based on the size of the sub-block. Accordingly, modification of information regarding the location of a previously stored conversion unit is required. For example, if the location of the effective sub-block (3420) is not changed, the filtering length may be determined based on the size of the effective sub-block (3420), which is an intermediate block, for filtering the left block.

[0430] In one embodiment of the present disclosure, the image decoding device (2000) may determine the locations of sub-blocks included in the current block based on the location of an effective sub-block. For example, the image decoding device (2000) may determine the index of an effective sub-block (3420) to 1, and determine the index of a sub-block from which information is obtained through the conversion unit syntax structure of index 1 to 0. The image decoding device (2000) may determine or maintain the index of a sub-block with index 2 as 2, the same as before. The image decoding device (2000) may store the locations of a plurality of conversion units of the current block in memory. The image decoding device (2000) may change and store the locations of the conversion units stored in memory to match the actual locations.

[0431] The valid sub-block (3440) may represent the upper block to which the current block is horizontally divided, the valid sub-block (3450) may represent the middle block to which the current block is horizontally divided, and the valid sub-block (3460) may represent the lower block to which the current block is horizontally divided. For example, if the current block is divided horizontally, the location of the upper block may be referred to as the location corresponding to index 0 (e.g., Pos 0), the location of the middle block may be referred to as the location corresponding to index 1 (e.g., Pos 1), and the location of the lower block may be referred to as the location corresponding to index 2 (e.g., Pos 2). Just like the vertically divided sub-block, the information of the horizontally divided sub-block must be modified and stored according to the location of the sub-block in order to apply filtering. Although the effective sub-blocks (3440, 3450, 3460) differ from the effective sub-blocks (3410, 3420, 3430) in that they are horizontally divided, the process of the image decoding device (2000) acquiring information through the bitstream and determining the location can be applied in the same way.

[0432] FIG. 35 is a drawing for explaining the process of obtaining information regarding an effective sub-block according to one embodiment of the present disclosure.

[0433] Referring to FIG. 35, the current block can be divided into four sub-blocks of equal size. The current block can be divided in a vertical or horizontal direction.

[0434] The valid sub-block (3510) represents the left block in which the current block is vertically divided, and the valid sub-block (3520) may represent the middle block adjacent to the left block in which the current block is vertically divided. The valid sub-block (3530) may represent the middle block adjacent to the right block in which the current block is vertically divided, and the valid sub-block (3540) may represent the right block in which the current block is vertically divided. For example, when the current block is divided in the vertical direction, the location of the left block may be referred to as the location corresponding to index 0 (e.g., Pos 0), the location of the middle block adjacent to the left block may be referred to as the location corresponding to index 1 (e.g., Pos 1), the location of the middle block adjacent to the right block may be referred to as the location corresponding to index 2 (e.g., Pos 2), and the location of the right block may be referred to as the location corresponding to index 3 (e.g., Pos 3).

[0435] In one embodiment of the present disclosure, the image decoding device (2000) can obtain information regarding an effective sub-block from a bitstream. The image decoding device (2000) can first obtain information regarding a left block through a conversion unit syntax structure corresponding to index 0, and then sequentially obtain information regarding an intermediate block and a right block through conversion unit syntax structures corresponding to index 1, index 2, and index 3. However, the image decoding device (2000) according to one embodiment of the present disclosure can determine the location of the effective sub-block using the image decoding method of FIG. 24 after obtaining information regarding the effective sub-block. Accordingly, the image decoding device (2000) can obtain information regarding the effective sub-block through a conversion unit syntax structure in a predetermined order of the bitstream. For example, all valid sub-blocks (3510, 3520, 3530, 3540) at all locations of the current block can obtain information through a conversion unit syntax corresponding to index 0. In this case, the valid sub-block (3510) obtains information through a conversion unit syntax structure identical to the actual location, but the valid sub-blocks (3520, 3530, 3540) obtain information through a conversion unit syntax structure different from the actual location.

[0436] In one embodiment of the present disclosure, the image decoding device (2000) may determine the location of a sub-block to apply filtering. The strength of the filtering or the length of the filtering is determined based on the size of the sub-block. Accordingly, modification of information regarding the location of a previously stored conversion unit is required.

[0437] In one embodiment of the present disclosure, the image decoding device (2000) may determine the locations of sub-blocks included in the current block based on the location of the effective sub-block. The image decoding device (2000) may exchange the index corresponding to the actual location of the effective sub-block with index 0. The image decoding device (2000) may determine or maintain the remaining indices, excluding the index corresponding to the actual location of the effective sub-block and index 0, as they were previously. The image decoding device (2000) may store the locations of a plurality of conversion units of the current block in memory. The image decoding device (2000) may change and store the locations of the conversion units stored in memory to match the actual locations.

[0438] The valid sub-block (3550) represents the upper block to which the current block is horizontally divided, and the valid sub-block (3560) may represent the middle block adjacent to the upper block to which the current block is horizontally divided. The valid sub-block (3570) represents the middle block adjacent to the lower block to which the current block is horizontally divided, and the valid sub-block (3580) may represent the lower block to which the current block is horizontally divided. Just like the vertically divided sub-block, the information in the horizontally divided sub-block must be modified and stored according to the position of the sub-block in order to apply filtering. Although the valid sub-blocks (3550, 3560, 3570, 3580) differ from the valid sub-blocks (3510, 3520, 3530, 3540) in that they are horizontally divided, the process of the image decoding device (2000) acquiring information through the bitstream and determining the position can be applied in the same way.

[0439] FIG. 36 is a drawing for explaining the process of obtaining information regarding an effective sub-block according to one embodiment of the present disclosure.

[0440] Referring to FIG. 36, the current block can be divided into four sub-blocks of equal size. The current block can be divided in the vertical and horizontal directions. For example, the width and height of the sub-blocks can each be half the width and height of the current block.

[0441] The valid sub-block (3610) represents the upper-left sub-block of the current block, and the valid sub-block (3620) may represent the upper-right sub-block of the current block. The valid sub-block (3630) represents the lower-left sub-block of the current block, and the valid sub-block (3640) may represent the lower-right sub-block of the current block. For example, the upper-left sub-block, the upper-right sub-block, the lower-left sub-block, and the lower-right sub-block may correspond to index 0, index 1, index 2, and index 3, respectively.

[0442] In one embodiment of the present disclosure, the image decoding device (2000) can obtain information regarding an effective sub-block from a bitstream. The image decoding device (2000) can first obtain information regarding a left block through a conversion unit syntax structure corresponding to index 0, and then sequentially obtain information regarding sub-blocks through conversion unit syntax structures corresponding to index 1, index 2, and index 3. However, the image decoding device (2000) according to one embodiment of the present disclosure can determine the location of the effective sub-block using the image decoding method of FIG. 24 after obtaining information regarding the effective sub-block. Accordingly, the image decoding device (2000) can obtain information regarding the effective sub-block through a conversion unit syntax structure in a predetermined order of the bitstream. For example, all valid sub-blocks (3610, 3620, 3630, 3640) of the current block can obtain information through a conversion unit syntax corresponding to index 0. In this case, the valid sub-block (3610) obtains information through a conversion unit syntax structure identical to the actual location, but the valid sub-blocks (3620, 3630, 3640) obtain information through a conversion unit syntax structure different from the actual location.

[0443] In one embodiment of the present disclosure, the image decoding device (2000) may determine the location of a sub-block to apply filtering. The strength of the filtering or the length of the filtering is determined based on the size of the sub-block. Accordingly, modification of information regarding the location of a previously stored conversion unit is required.

[0444] In one embodiment of the present disclosure, the image decoding device (2000) may determine the locations of sub-blocks included in the current block based on the location of the effective sub-block. The image decoding device (2000) may exchange the index corresponding to the actual location of the effective sub-block with index 0. The image decoding device (2000) may determine or maintain the remaining indices, excluding the index corresponding to the actual location of the effective sub-block and index 0, as they were previously. The image decoding device (2000) may store the locations of a plurality of conversion units of the current block in memory. The image decoding device (2000) may change and store the locations of the conversion units stored in memory to match the actual locations.

[0445] FIG. 37 is a drawing for explaining the process of obtaining information regarding an effective sub-block according to one embodiment of the present disclosure.

[0446] Referring to FIG. 37, the current block can be divided into 16 sub-blocks of the same size. The current block can be divided in the vertical and horizontal directions. For example, the width and height of the sub-blocks can each be 1 / 4 of the width and height of the current block.

[0447] The valid sub-block (3710) represents the upper-left sub-block of the current block, and the valid sub-block (3720) may represent the upper-right sub-block of the current block. The valid sub-block (3730) represents the lower-left sub-block of the current block, and the valid sub-block (3740) may represent the lower-right sub-block of the current block. For example, the upper-left sub-block, the upper-right sub-block, the lower-left sub-block, and the lower-right sub-block may correspond to index 0, index 3, index 12, and index 15, respectively. In one embodiment, the valid sub-block may be located at four candidate positions at the corners of the current block.

[0448] In one embodiment of the present disclosure, an image decoding device (2000) can obtain information regarding an effective sub-block from a bitstream. The image decoding device (2000) can first obtain information regarding the left block through a conversion unit syntax structure corresponding to index 0, and then sequentially obtain information regarding the sub-block through conversion unit syntax structures corresponding to index 1 through index 15. However, the image decoding device (2000) according to one embodiment of the present disclosure can determine the location of the effective sub-block using the image decoding method of FIG. 24 after obtaining information regarding the effective sub-block. Accordingly, the image decoding device (2000) can obtain information regarding the effective sub-block through a conversion unit syntax structure in a predetermined order of the bitstream. For example, information regarding all effective sub-blocks (3710, 3720, 3730, 3740) at all locations of the current block can all be obtained through a conversion unit syntax corresponding to index 0. In this case, the valid sub-block (3710) obtains information through a conversion unit syntax structure identical to the actual location, but the valid sub-blocks (3720, 3730, 3740) obtain information through a conversion unit syntax structure different from the actual location.

[0449] In one embodiment of the present disclosure, the image decoding device (2000) may determine the location of a sub-block to apply filtering. The strength of the filtering or the length of the filtering is determined based on the size of the sub-block. Accordingly, modification of information regarding the location of a previously stored conversion unit is required.

[0450] In one embodiment of the present disclosure, the image decoding device (2000) may determine the locations of sub-blocks included in the current block based on the location of the effective sub-block. The image decoding device (2000) may exchange the index corresponding to the actual location of the effective sub-block with index 0. The image decoding device (2000) may determine or maintain the remaining indices, excluding the index corresponding to the actual location of the effective sub-block and index 0, as they were previously.

[0451] The image decoding device (2000) can obtain index information indicating the actual location of an effective sub-block through a bitstream. In one embodiment of the present disclosure, the index information indicating the actual location of an effective sub-block may be information indicating one of four candidate locations of the effective sub-block. Referring to FIG. 37, the index information indicating the actual location of an effective sub-block may indicate one of pos 0 to pos 3. The index information may indicate one of the candidate location vectors. For example, if the index information is 3, it means that it indicates 15 among the candidate location vectors composed of [0, 3, 12, 15], and may mean that the order of TU0 and TU15 is reversed. That is, the bitstream may contain information in the order of TU15 -> TU1 -> ... -> TU14 -> TU0. The image decoding device (2000) obtains information by setting the order as such as TU0 -> TU1 -> ... -> TU14 -> TU15 before obtaining index information, and after obtaining index information, determines that the order of TU0 and TU15 has been changed and can modify information related to TU0 and TU15. For example, the image decoding device (2000) can modify the location of the valid sub-block and the boundary for applying filtering.

[0452] The image decoding device (2000) can store the locations of multiple conversion units of the current block in memory. The image decoding device (2000) can change the locations of the conversion units stored in memory to match the actual locations and store them.

[0453] FIG. 38 is a drawing for explaining the process of obtaining information regarding an effective sub-block according to one embodiment of the present disclosure.

[0454] Referring to FIG. 38, the current block can be divided into one 1 / 4 size sub-block and 12 1 / 16 size sub-blocks.

[0455] The valid sub-block (3810) represents the upper-left sub-block of the current block, and the valid sub-block (3820) may represent the upper-right sub-block of the current block. The valid sub-block (3830) represents the lower-left sub-block of the current block, and the valid sub-block (3840) may represent the lower-right sub-block of the current block. The valid sub-block (3850) represents the upper sub-block of the current block, and the valid sub-block (3860) may represent the lower sub-block of the current block. The valid sub-block (3870) represents the left sub-block of the current block, and the valid sub-block (3880) may represent the right sub-block of the current block. For example, the upper-left sub-block, the upper-right sub-block, the lower-left sub-block, and the lower-right sub-block may correspond to index 0, index 4, index 8, and index 12, respectively. Additionally, for example, the upper sub-block, lower sub-block, left sub-block, and right sub-block may correspond to index 2, index 10, index 4, and index 12, respectively. In one embodiment, the valid sub-block may be located at eight candidate positions adjacent to the boundary of the current block.

[0456] In one embodiment of the present disclosure, the image decoding device (2000) can obtain information regarding an effective sub-block from a bitstream. The image decoding device (2000) can first obtain information regarding a left block through a conversion unit syntax structure corresponding to index 0, and then sequentially obtain information regarding sub-blocks through conversion unit syntax structures corresponding to index 1 through index 13. However, the image decoding device (2000) according to one embodiment of the present disclosure can determine the location of the effective sub-block using the image decoding method of FIG. 24 after obtaining information regarding the effective sub-block. Accordingly, the image decoding device (2000) can obtain information regarding the effective sub-block through a conversion unit syntax structure in a predetermined order of the bitstream. For example, all valid sub-blocks (3810, 3820, 3830, 3840, 3850, 3860, 3870, 3880) of the current block can obtain information through a conversion unit syntax corresponding to index 0. In this case, the valid sub-block (3810) obtains information through a conversion unit syntax structure identical to the actual location, but the valid sub-blocks (3820, 3830, 3840, 3850, 3860, 3870, 3880) obtain information through a conversion unit syntax structure different from the actual location.

[0457] In one embodiment of the present disclosure, the image decoding device (2000) may determine the location of a sub-block to apply filtering. The strength of the filtering or the length of the filtering is determined based on the size of the sub-block. Accordingly, modification of information regarding the location of a previously stored conversion unit is required.

[0458] In one embodiment of the present disclosure, the image decoding device (2000) may determine the locations of sub-blocks included in the current block based on the location of the effective sub-block. The image decoding device (2000) may exchange the index corresponding to the actual location of the effective sub-block with index 0. The image decoding device (2000) may determine or maintain the remaining indices, excluding the index corresponding to the actual location of the effective sub-block and index 0, as they were previously.

[0459] The image decoding device (2000) can obtain index information indicating the actual location of an effective sub-block through a bitstream. In one embodiment of the present disclosure, the index information indicating the actual location of an effective sub-block may be information indicating one of eight candidate locations of an effective sub-block. Referring to FIG. 38, the index information indicating the actual location of an effective sub-block may indicate one of pos 0 to pos 7. The index information may indicate one of the candidate location vectors. For example, if the index information is 4, it means that it indicates 2 among the candidate location vectors composed of [0, 4, 8, 12, 2, 10, 4, 8], and may mean that the order of TU0 and TU2 is reversed. That is, in the bitstream, information may be included in the order of TU2 -> TU1 -> TU0 -> TU 3 -> TU4, ..., TU12. The image decoding device (2000) obtains information by setting the order as TU0 -> TU1 -> TU2 -> TU3 -> TU4, ..., TU12 before obtaining index information, and after obtaining index information, determines that the order of TU0 and TU2 has been changed and can modify information related to TU0 and TU2. For example, the image decoding device (2000) can modify the location of the valid sub-block and the boundary for applying filtering.

[0460] The image decoding device (2000) can store the locations of multiple conversion units of the current block in memory. The image decoding device (2000) can change the locations of the conversion units stored in memory to match the actual locations and store them.

[0461] FIG. 39 is a block diagram showing the configuration of an image encoding device according to one embodiment of the present disclosure.

[0462] Referring to FIG. 39, the image encoding device (3900) may include a processor (3910) and a memory (3920).

[0463] In one embodiment of the present disclosure, the processor (3910) may include processing circuits and / or multiple processors. For example, the processor (3910) may include various processing circuits including at least one processor, and at least one of the at least one processor may be configured to perform the various functions described in the present disclosure individually and / or collectively in a distributed manner.

[0464] In one embodiment of the present disclosure, the memory (3920) may include one or more storage media that store at least one instruction. The processor (3910) may control the image encoding device (3900) by executing the instruction stored in the memory (3920). For example, the processor (3910) may control the image encoding device (3900) to perform an operation by executing the instruction stored in the memory (3920) individually or collectively. In one embodiment of the present disclosure, the operation performed by the image encoding device (3900) may be an operation performed by the processor (3910) of the image decoding device (3900).

[0465] In one embodiment of the present disclosure, the image encoding device (3900) may correspond to the image encoding device (200) shown in FIG. 2 and / or the encoding unit (1910) shown in FIG. 19.

[0466] The image encoding device (3900) can determine the prediction mode of the current block. The current block may include at least one of a maximum encoding unit, an encoding unit, a transform unit, or a prediction unit divided from the current image to be encoded. In one embodiment of the present disclosure, the prediction mode of the current block may include at least one of an intra mode, an inter mode, a combined mode, a geometric partitioning mode, a block copy mode, or a template matching prediction mode.

[0467] In one embodiment of the present disclosure, an intra mode may include a non-directional Planar mode 0 (or Intra_Planar mode), a non-directional DC mode 1 (or Intra_DC mode), and directional Angular modes 2 through 66 (or Intra_directional modes) (e.g., (Intra_Angular2.. Intra_Angular66). In one embodiment of the present disclosure, an intra-planar mode may refer to a mode that determines a predicted sample based on a weighted average value according to the distance of a left reference sample, an upper reference sample, a lower-left sample of the current block, and an upper-right sample. In one embodiment of the present disclosure, an intra-DC mode may refer to a mode that determines the average value of reference samples as the predicted sample. In one embodiment of the present disclosure, in the intra-directional modes, the locations of reference samples for generating predicted samples of samples within the current block may be identified by considering the direction indicated by the intra-directional modes. For example, in mode 34, located at a 45-degree upper-left direction relative to the samples within the current block. Reference samples may be identified. In one embodiment of the present disclosure, the intra mode may include -14 through -1 and 67 through 80 Wide-Angular modes (Intra_Wide_Angular). Wide-Angular modes may be used to identify reference samples of samples within a non-square current block. In one embodiment of the present disclosure, the image encoding device (3900) may determine one of the Wide-Angular modes as the intra prediction mode of the non-square current block. The image encoding device (3900) may determine the Wide-Angular mode based on the width and height of the current block. If the width of the current block is greater than the height, the image encoding device (3900) may replace the lower-left directional mode with the upper-right extended directional mode.For example, the video encoding device (3900) may replace the index value "predModeIntra" of the intra prediction mode, which is 2 or more and less than or equal to a predetermined value, with "predModeIntra + 65". If the height of the current block is greater than the width, the video encoding device (3900) may replace the upper right directional mode with the lower left extended directional mode. For example, the video encoding device (3900) may replace the index value "predModeIntra" of the intra prediction mode, which is 66 or more and less than or equal to a predetermined value, with "predModeIntra - 67". The number and types of intra prediction modes available in the intra mode of the video encoding device (3900) according to one embodiment of the present disclosure may be set in various ways. For example, the video encoding device (3900) can determine a Wide-Angular mode using a predetermined method based on the ratio of the height and width of the block (e.g., 16, 8, 4, 2, 1 / 2, 1 / 4, 1 / 8, or 1 / 16).

[0468] In one embodiment of the present disclosure, the image encoding device (3900) may determine an intra prediction mode using most probable modes (MPM). The image encoding device (3900) may determine whether to use the MPM. The image encoding device (3900) may generate a bitstream containing information regarding whether to use the MPM. If the MPM is used, the image encoding device (3900) may determine an MPM list. In one embodiment of the present disclosure, the image encoding device (3900) may determine an MPM list using the surrounding blocks of the current block. The image encoding device (3900) may determine an MPM list based on the intra mode of the upper block of the current block and the intra mode of the left block. If the surrounding blocks of the current block are not available (e.g., if the intra prediction mode of the surrounding blocks is not determined), the intra prediction mode of the unavailable surrounding blocks may be set to a predetermined mode (e.g., Planar mode). The video encoding device (3900) can determine one of the MPM lists as the intra prediction mode of the current block. The video encoding device (3900) can generate a bitstream containing information (e.g., index information) indicating the intra prediction mode of the current block among the MPM lists.

[0469] In one embodiment of the present disclosure, the image encoding device (3900) can determine an intra prediction mode using a template. The image encoding device (3900) can determine a template of the current block. The template of the current block may include a left sample, an upper-left sample, and / or an upper sample of the current block. The image encoding device (3900) can determine a peripheral sample of the template of the current block. The peripheral sample of the template may include a left sample, an upper-left sample, and / or an upper sample of the template. The image encoding device (3900) can perform a prediction for the template using the peripheral sample of the template as a reference sample. The image encoding device (3900) can determine an intra mode for the reference block by comparing the predicted template with the restored template of the current block. The image encoding device (3900) can determine the intra mode with the smallest error between the predicted template and the restored template of the current block as the intra mode for the reference block. In one embodiment of the present disclosure, the process of determining an intra prediction mode by the image encoding device (3900) performing a prediction on a template may be referred to as Template-based intra mode derivation (TIMD).

[0470] In one embodiment of the present disclosure, the image encoding device (3900) can infer the intra prediction mode of the current block using surrounding samples of the current block. The image encoding device (3900) can determine the slope using surrounding samples of the current block. The image encoding device (3900) can determine a plurality of 3 x 3 blocks adjacent to the current block. The image encoding device (3900) can obtain the horizontal change amount and the vertical change amount of samples included in each determined 3 x 3 block. The image encoding device (3900) can determine the slope based on the horizontal change amount and the vertical change amount. The image encoding device (3900) can determine the horizontal change amount and the vertical change amount using a Sobel filter. The image encoding de...

Claims

1. In a video decoding method, A step (S2410) of identifying a plurality of candidate locations for a significant sub-block containing at least one non-zero transformation coefficient within the current block; A step (S2420) of determining a cost for each of the plurality of candidate locations based on at least one of the amount of change between a plurality of surrounding samples at the boundary of the valid sub-block and the amount of change between a plurality of surrounding samples at the boundary of the current block; A step of determining the ranking of the plurality of candidate locations based on the cost for each of the plurality of candidate locations (S2430); A step (S2440) of determining the location of an effective sub-block for restoring the current block among the plurality of candidate locations based on the ranking indicated by the index information obtained from the bitstream; A step (S2450) of determining the location of a plurality of sub-blocks included in the current block based on the location of an effective sub-block for restoring the current block; and A video decoding method comprising the step (S2460) of determining a boundary for applying filtering based on the positions of a plurality of sub-blocks determined above.

2. In Paragraph 1, The step of identifying the above plurality of candidate locations (S2410) is, A step of obtaining information related to the shape of the above-mentioned valid sub-block; A step of determining the shape of the effective sub-block based on information related to the shape of the effective sub-block; and The method includes the step of identifying a plurality of candidate locations corresponding to the shape of the above valid sub-block, and An image decoding method comprising at least one of the following: information related to the shape of the valid sub-block, ratio information indicating the ratio between the size of the current block and the size of the valid sub-block, division direction information indicating whether the current block is divided vertically, or corner information indicating whether the valid sub-block is located at the corner of the current block.

3. In Paragraph 2, The above bitstream includes information regarding the valid sub-block before the information of other sub-blocks of the current block, and An image decoding method characterized in that the information regarding the above-mentioned effective sub-block includes information regarding the transformation coefficients included in the above-mentioned effective sub-block.

4. In any one of paragraphs 1 through 3, A step of obtaining residual blocks for each candidate location based on applying a transformation kernel to valid sub-blocks corresponding to each of the plurality of candidate locations; and An image decoding method characterized by further including the step of obtaining a plurality of candidate restoration blocks based on the sum of the prediction blocks for the current block and the residual blocks for each candidate location.

5. In any one of paragraphs 1 through 4, An image decoding method characterized in that the above ranking is determined according to the order of the size of the costs for the above candidate locations.

6. In any one of paragraphs 1 through 5, The step (S2440) of determining the location of an effective sub-block for restoring the current block is, A step of obtaining an empty string by performing arithmetic decoding on the index information using a context model; A step of obtaining the rank represented by the index information based on the above empty string; and A video decoding method comprising the step of determining the location of an effective sub-block for restoring the current block corresponding to the rank indicated by the index information among the plurality of candidate locations.

7. In Paragraph 6, The above empty string is encoded using a truncated unary, and An image decoding method characterized by determining the rank represented by the index information based on the number of 1s or 0s included in the empty string.

8. In Paragraph 6, An image decoding method characterized in that the above empty string represents a rank in which the index information is encoded in binary.

9. In any one of paragraphs 1 through 8, An image decoding method in which the above cost includes the sum of the change amounts between a plurality of restoration samples located around at least one of the boundary of the effective sub-block and the boundary of the current block.

10. In any one of paragraphs 1 through 9, The step (S2460) of determining a boundary for applying filtering based on the locations of the plurality of sub-blocks determined above is, A step of obtaining a location in memory corresponding to each of the plurality of sub-blocks; A step of changing a location obtained in the memory based on a plurality of sub-block locations determined above; The step of storing the above-mentioned changed location in the memory; and An image decoding method comprising the step of determining a boundary for applying filtering including the boundary of a plurality of sub-blocks determined above.

11. In any one of paragraphs 1 through 10, The above index information is obtained from the encoding unit syntax structure of the bitstream containing information regarding the current block, and An image decoding method characterized in that information regarding the valid sub-block is first obtained in the bitstream among information regarding a plurality of sub-blocks of the current block.

12. In any one of paragraphs 1 through 11, A plurality of surrounding samples of the boundary of the effective sub-block include adjacent samples along the vertical direction, horizontal direction, or slope direction corresponding to the intra-prediction mode of the boundary of the effective sub-block, and A method for decoding images, wherein a plurality of surrounding samples of the boundary of the current block include adjacent samples along the vertical direction, horizontal direction, or gradient direction corresponding to the intra-prediction mode of the boundary of the current block.

13. In a video decoding device, At least one processor including a processing circuit; and The memory includes one or more storage media for storing instructions, and By executing the above instructions individually or collectively by the at least one processor, the image decoder, Identifying multiple candidate locations for a significant sub-block containing at least one non-zero transformation coefficient within the current block, and A cost for each of the plurality of candidate locations is determined based on at least one of the amount of change between a plurality of surrounding samples at the boundary of the valid sub-block and the amount of change between a plurality of surrounding samples at the boundary of the current block. Determining the ranking of the plurality of candidate locations based on the cost for each of the plurality of candidate locations, and Based on the ranking indicated by the index information obtained from the bitstream, the location of an effective sub-block for restoring the current block is determined among the plurality of candidate locations, and Based on the location of an effective sub-block for restoring the current block, the location of a plurality of sub-blocks included in the current block is determined, and An image decoding device that determines a boundary for applying filtering based on the positions of a plurality of sub-blocks determined above.

14. In a video encoding method, A step (S4010) of identifying a plurality of candidate locations for a significant sub-block containing at least one non-zero transformation coefficient within the current block; A step (S4020) of determining a cost for each of the plurality of candidate locations based on at least one of the amount of change between a plurality of surrounding samples at the boundary of the valid sub-block and the amount of change between a plurality of surrounding samples at the boundary of the current block; A step of determining the ranking of the plurality of candidate locations based on the cost for each of the plurality of candidate locations (S4030); A step (S4040) of determining the location of an effective sub-block for restoring the current block among the plurality of candidate locations; A step of determining a boundary for applying filtering based on the positions of the plurality of sub-blocks determined above (S4060); and A video encoding method comprising the step (S4070) of generating a bitstream including index information indicating the rank of an effective sub-block for the above restoration.

15. Regarding the method of transmitting a bitstream, A step of identifying a plurality of candidate locations for a significant sub-block containing at least one non-zero transformation coefficient within the current block; A step of determining a cost for each of the plurality of candidate locations based on at least one of the amount of change between a plurality of surrounding samples at the boundary of the valid sub-block and the amount of change between a plurality of surrounding samples at the boundary of the current block; A step of determining the ranking of the plurality of candidate locations based on the cost for each of the plurality of candidate locations; A step of determining the location of an effective sub-block for restoring the current block among the plurality of candidate locations; A step of determining a boundary for applying filtering based on the locations of the plurality of sub-blocks determined above; A step of generating a bitstream including index information indicating the rank of a valid sub-block for the above restoration; and A method comprising the step of transmitting the bitstream to a video decoder.

Images