Video decoding method and device using sub-partition-based derived intra prediction mode, and video encoding method and device using sub-partition-based derived intra prediction mode

Abstract

A video decoding method is provided, the method comprising the steps of: obtaining, from a bitstream, first information indicating whether an intra prediction mode derived for each sub-partition is used for a current block; when the intra prediction mode derived for each sub-partition according to the first information is used, determining an intra prediction mode of a first sub-partition among sub-partitions of the current block on the basis of surrounding information of the first sub-partition; obtaining reconstructed samples of the first sub-partition by using prediction samples obtained by performing intra prediction on the basis of the intra prediction mode of the first sub-partition; determining an intra prediction mode of a second sub-partition adjacent to the first sub-partition among the sub-partitions of the current block by using a prediction result of the first sub-partition and surrounding information of the second sub-partition; obtaining reconstructed samples of the second sub-partition by using prediction samples obtained by performing intra prediction on the basis of the intra prediction mode of the second sub-partition; and reconstructing the current block on the basis of the reconstructed samples of the first sub-partition and the reconstructed samples of the second sub-partition.

Description

A video decoding method and apparatus using a subpartition-based induced intra prediction mode, a video encoding method and apparatus using a subpartition-based induced intra prediction mode

[0001] The present disclosure relates to the field of video encoding and decoding. More specifically, it relates to an apparatus and method for video encoding and decoding using an intra-prediction mode.

[0002] In image encoding and decoding, the image is divided into blocks, and each block is predictively encoded and predictedly decoded through inter-prediction or intra-prediction.

[0003] Inter-prediction is a technique that compresses images by eliminating temporal redundancy between images. In inter-prediction, blocks of the current image are predicted using a reference image. The reference block most similar to the current block can be searched within a predetermined search range in the reference image. The current block is predicted based on the reference block, and a residual block is generated by subtracting the predicted block resulting from the prediction result from the current block.

[0004] Intra prediction is a technique that compresses images by eliminating spatial redundancy within the image. Depending on the prediction mode, intra prediction generates a prediction block based on the surrounding pixels of the current block. Then, it generates a residual block by subtracting the prediction block from the current block.

[0005] Residual blocks generated through inter-prediction or intra-prediction are passed to a decoder after undergoing transformation and quantization. The decoder inversely quantizes and inversely transforms the residual blocks, and reconstructs the current block by combining the prediction block of the current block with the residual blocks. In certain cases, the decoder can filter the reconstructed current block to remove artifacts within it.

[0006] According to one embodiment of the present disclosure, a video decoding method is provided. A video decoding method according to one embodiment may include a step of obtaining first information from a bitstream indicating whether an intra prediction mode derived per sub-partition in a current block is utilized. The video decoding method may include a step of determining the intra prediction mode of a first sub-partition among the sub-partitions of the current block based on surrounding information of the first sub-partition when the intra prediction mode derived per sub-partition according to the first information is utilized. The video decoding method may include a step of obtaining restoration samples of the first sub-partition using prediction samples obtained by performing intra prediction based on the intra prediction mode of the first sub-partition. The video decoding method may include a step of determining the intra prediction mode of a second sub-partition adjacent to the first sub-partition among the sub-partitions of the current block using the prediction result of the first sub-partition and surrounding information of the second sub-partition. The video decoding method may include a step of obtaining recovery samples of the second subpartition using prediction samples obtained by performing intra prediction based on an intra prediction mode of the second subpartition. The video decoding method may include a step of recovering the current block based on the recovery samples of the first subpartition and the recovery samples of the second subpartition.

[0007] FIG. 1 is a block diagram of an image decoding device according to one embodiment of the present disclosure.

[0008] FIG. 2 is a block diagram of an image encoding device according to one embodiment of the present disclosure.

[0009] FIG. 3 illustrates a process of determining at least one encoding unit by dividing a current encoding unit according to one embodiment of the present disclosure.

[0010] FIG. 4 illustrates a process of determining at least one encoding unit by dividing a encoding unit that is in the shape of a non-square according to one embodiment of the present disclosure.

[0011] FIG. 5 illustrates a process of dividing a encoding unit based on at least one of block shape information and division shape mode information according to one embodiment of the present disclosure.

[0012] FIG. 6 illustrates a method for determining a predetermined encoding unit among an odd number of encoding units according to one embodiment of the present disclosure.

[0013] FIG. 7 illustrates the order in which a plurality of encoding units are processed when a current encoding unit is divided to determine a plurality of encoding units according to one embodiment of the present disclosure.

[0014] FIG. 8 illustrates a process for determining that, in accordance with one embodiment of the present disclosure, when the encoding unit cannot be processed in a predetermined order, the current encoding unit is divided into an odd number of encoding units.

[0015] FIG. 9 illustrates a process of determining at least one encoding unit by dividing a first encoding unit according to one embodiment of the present disclosure.

[0016] FIG. 10 illustrates that, according to one embodiment of the present disclosure, the shape that can be divided is limited when a second encoding unit of a non-square shape determined by dividing a first encoding unit satisfies a predetermined condition.

[0017] FIG. 11 illustrates a process of dividing square-shaped encoding units when the divided shape mode information cannot represent division into four square-shaped encoding units according to one embodiment of the present disclosure.

[0018] FIG. 12 illustrates that, according to one embodiment of the present disclosure, the processing order between a plurality of encoding units may vary depending on the division process of the encoding unit.

[0019] FIG. 13 illustrates a process in which, according to one embodiment of the present disclosure, a encoding unit is recursively divided to determine a plurality of encoding units, and the depth of the encoding unit is determined as the shape and size of the encoding unit change.

[0020] FIG. 14 illustrates a depth and part index (hereinafter PID) for distinguishing between coding units that can be determined according to the shape and size of the coding units according to one embodiment of the present disclosure.

[0021] FIG. 15 illustrates that a plurality of encoding units are determined according to a plurality of predetermined data units included in a picture according to one embodiment of the present disclosure.

[0022] FIG. 16 illustrates encoding units that can be determined for each picture when the combination of forms in which encoding units can be divided according to one embodiment of the present disclosure is different for each picture.

[0023] FIG. 17 illustrates various forms of encoding units that can be determined based on partitioned form mode information expressed in binary code according to one embodiment of the present disclosure.

[0024] FIG. 18 illustrates another form of encoding unit that can be determined based on partitioned form mode information represented by binary code according to one embodiment of the present disclosure.

[0025] FIG. 19 is a block diagram of an image encoding and decoding system according to one embodiment of the present disclosure.

[0026] FIG. 20 is a block diagram showing the configuration of a video decoding device according to one embodiment.

[0027] Figure 21 illustrates the directionality of the intra prediction mode.

[0028] Figure 22 illustrates an intra prediction method according to the ISP (intra sub partition) mode.

[0029] FIG. 23 is a diagram illustrating a DIMD (Decoder-side Intra Sub Partition) method for inducing an intra prediction mode in a video decoding device according to one embodiment.

[0030] FIG. 24 is a diagram illustrating a Template-based Intra Mode Derivation (TIMD) method for inducing an intra prediction mode in a video decoding device according to one embodiment.

[0031] FIGS. 25, 26a and 26b are drawings illustrating a Cross Line Prediction (CLP) method for inducing an intra prediction mode in a video decoding device according to one embodiment.

[0032] FIG. 27 illustrates a flowchart of a video decoding method according to one embodiment.

[0033] FIGS. 28 and 29 illustrate a method for inducing an intra-prediction mode per sub-partition according to one embodiment.

[0034] FIGS. 30 and 31 illustrate an intra prediction mode derived per sub-partition according to one embodiment.

[0035] FIG. 32 illustrates a case in which an intra prediction mode induced per sub-partition is limited according to one embodiment.

[0036] FIG. 33 illustrates a method for inducing an intra prediction mode for each sub-partition using reference information located at a distance according to one embodiment.

[0037] FIGS. 34, 35, 36, 37, 38, 39 and 40 illustrate a method of transmitting information indicating whether to induce an intra prediction mode per sub-partition according to various embodiments.

[0038] FIG. 41 illustrates a case where a method for inducing an intra prediction mode for each sub-partition is determined individually according to one embodiment.

[0039] FIGS. 42a and 42b illustrate various forms of sub-partitions that can be determined for intra-prediction from a single CU according to one embodiment.

[0040] FIG. 43 is a block diagram showing the configuration of a video encoding device according to one embodiment.

[0041] FIG. 44 illustrates a flowchart of a video encoding method according to one embodiment.

[0042] According to one embodiment of the present disclosure, a video decoding method is provided. A video decoding method according to one embodiment may include a step of obtaining first information from a bitstream indicating whether an intra prediction mode derived per sub-partition in a current block is utilized. The video decoding method may include a step of determining the intra prediction mode of a first sub-partition among the sub-partitions of the current block based on surrounding information of the first sub-partition when the intra prediction mode derived per sub-partition according to the first information is utilized. The video decoding method may include a step of obtaining restoration samples of the first sub-partition using prediction samples obtained by performing intra prediction based on the intra prediction mode of the first sub-partition. The video decoding method may include a step of determining the intra prediction mode of a second sub-partition adjacent to the first sub-partition among the sub-partitions of the current block using the prediction result of the first sub-partition and surrounding information of the second sub-partition. The video decoding method may include a step of obtaining recovery samples of the second subpartition using prediction samples obtained by performing intra prediction based on an intra prediction mode of the second subpartition. The video decoding method may include a step of recovering the current block based on the recovery samples of the first subpartition and the recovery samples of the second subpartition.

[0043] According to one embodiment of the present disclosure, a video encoding method is provided. A video encoding method according to one embodiment may include a step of generating first information indicating whether an intra prediction mode derived per sub-partition in a current block is utilized. The video encoding method may include a step of determining, when the intra prediction mode derived per sub-partition is utilized, an intra prediction mode of a first sub-partition among the sub-partitions of the current block based on surrounding information of the first sub-partition. The video encoding method may include a step of obtaining residual samples of the first sub-partition using prediction samples obtained by performing intra prediction based on the intra prediction mode of the first sub-partition. The video encoding method may include a step of determining an intra prediction mode of a second sub-partition adjacent to the first sub-partition among the sub-partitions of the current block using a prediction result of the first sub-partition and surrounding information of the second sub-partition. The video encoding method may include the step of obtaining residual samples of the second subpartition using prediction samples obtained by performing intra prediction based on the intra prediction mode of the second subpartition. The video encoding method may include the step of generating transformation coefficients of the residual samples of the first subpartition and transformation coefficients of the residual samples of the second subpartition.

[0044] According to one embodiment of the present disclosure, a method for transmitting a bitstream is provided. According to one embodiment, a method for transmitting a bitstream generated by encoding video may include the step of generating first information indicating whether an intra prediction mode derived per sub-partition in a current block is utilized. The method for transmitting the bitstream may include the step of determining the intra prediction mode of a first sub-partition among the sub-partitions of the current block based on surrounding information of the first sub-partition when the intra prediction mode derived per sub-partition is utilized. The method for transmitting the bitstream may include the step of obtaining residual samples of the first sub-partition using prediction samples obtained by performing intra prediction based on the intra prediction mode of the first sub-partition. The method for transmitting the bitstream may include the step of determining the intra prediction mode of a second sub-partition adjacent to the first sub-partition among the sub-partitions of the current block using the prediction result of the first sub-partition and surrounding information of the second sub-partition. A method for transmitting a bitstream may include the step of obtaining residual samples of the second sub-partition using prediction samples obtained by performing intra prediction based on an intra prediction mode of the second sub-partition. A method for transmitting a bitstream may include the step of outputting a bitstream comprising the first information, the transformation coefficients of the residual samples of the first sub-partition, and the transformation coefficients of the residual samples of the second sub-partition.

[0045] In one embodiment of the present disclosure, a video decoding device is provided. The video decoding device may include a memory for storing instructions; and at least one processor operably coupled to the memory and comprising a processing circuit. The device for decoding the video by the at least one processor executing the instructions alone or in cooperation may: obtain first information from a bitstream indicating whether an intra prediction mode derived per sub-partition in the current block is utilized. The device for decoding the video may: when an intra prediction mode derived per sub-partition is utilized according to the first information, determine the intra prediction mode of a first sub-partition among the sub-partitions of the current block based on surrounding information of the first sub-partition. The device for decoding the video may: obtain recovery samples of the first sub-partition using prediction samples obtained by performing intra prediction based on the intra prediction mode of the first sub-partition. A device for decoding a video can determine the intra prediction mode of a second sub-partition adjacent to the first sub-partition among the sub-partitions of the current block using the prediction result of the first sub-partition and surrounding information of the second sub-partition. A device for decoding a video can obtain restoration samples of the second sub-partition using prediction samples obtained by performing intra prediction based on the intra prediction mode of the second sub-partition. A device for decoding a video can restore the current block based on the restoration samples of the first sub-partition and the restoration samples of the second sub-partition.

[0046] In one embodiment of the present disclosure, a video encoding device is provided. The video encoding device may include a memory for storing instructions; and at least one processor operably coupled to the memory and comprising a processing circuit. By the at least one processor executing the instructions alone or in cooperation, the video encoding device may: generate first information indicating whether an intra prediction mode derived per sub-partition is utilized in the current block. When the intra prediction mode derived per sub-partition is utilized, the video encoding device may determine the intra prediction mode of a first sub-partition among the sub-partitions of the current block based on the surrounding information of the first sub-partition. The video encoding device may obtain residual samples of the first sub-partition using prediction samples obtained by performing intra prediction based on the intra prediction mode of the first sub-partition. The video encoding device may determine the intra prediction mode of a second sub-partition adjacent to the first sub-partition among the sub-partitions of the current block using the prediction result of the first sub-partition and the surrounding information of the second sub-partition. A video encoding device can obtain residual samples of the second sub-partition using prediction samples obtained by performing intra prediction based on an intra prediction mode of the second sub-partition. The video encoding device can generate transformation coefficients of residual samples of the first sub-partition and transformation coefficients of residual samples of the second sub-partition.

[0047] In the present disclosure, the expression “at least one of a, b, or c” may refer to “a”, “b”, “c”, “a and b”, “a and c”, “b and c”, “a, b, and c all”, or variations thereof.

[0048] In the present disclosure, the expression "a, b and / or c" may be replaced with "at least one of a, b or c." That is, the expression "a, b and / or c" may refer to "a," "b," "c," "a and b," "a and c," "b and c," "all of a, b and c," or variations thereof.

[0049] The present disclosure is capable of various modifications and may have various embodiments, and embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the embodiments of the present disclosure, and the present disclosure may include all modifications, equivalents, and substitutions that fall within the spirit and technical scope of the various embodiments.

[0050] The terms used in this disclosure have been selected to be as widely used and general as possible, taking into account their functions within this disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been arbitrarily selected by the applicant, the meaning of which can be understood through the relevant explanatory sections. Therefore, the terms used in this disclosure should be defined not merely by their names, but based on their meanings and the overall content of this disclosure.

[0051] In this disclosure, singular expressions may include plural expressions unless the context clearly indicates otherwise. In describing the embodiments, detailed descriptions of related prior art are omitted if it is determined that such detailed descriptions would unnecessarily obscure the essence of this disclosure. Additionally, numbers used in the description of the embodiments (e.g., first, second, etc.) correspond to identification symbols for distinguishing one component from another.

[0052] In the present disclosure, when one component is described as being "connected" or "connected" to another component, the one component may be directly connected to or directly connected to another component, but unless specifically stated otherwise, it may be connected or connected through another component in between.

[0053] In the present disclosure, when a part is described as "comprising" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. Components expressed in the present disclosure as "part (unit)," "module," etc., may consist of two or more components combined into a single component, or a single component may be divided into two or more more subdivided components. Furthermore, each component described below may additionally perform some or all of the functions of other components in addition to its own primary function, and some of the primary functions of each component may be exclusively performed by other components.

[0054] As used in this disclosure, the expression “configured to” may be replaced, depending on the context, with, for example, “suitable for,” “having the capacity to,” “designed to,” “adapted to,” “made to,” or “capable of.” The term “configured to” may not necessarily mean only “specifically designed to” in hardware. Alternatively, in some situations, the expression “system configured to” may include that the system is “capable of” in conjunction with other devices or components. For example, the phrase “processor configured to perform A, B, and C” may include a dedicated processor for performing the said operations (e.g., an embedded processor), or a generic-purpose processor (e.g., a CPU or an application processor) capable of performing said operations by executing one or more software programs stored in memory.

[0055] At least one processor according to an embodiment of the present disclosure may include various processing circuits and / or multiple processors. For example, the term “processor” as used in the present disclosure, including in the claims, may include various processing circuits including at least one processor, and at least one of the at least one processor is configured to perform the various functions described herein individually and / or collectively in a distributed manner. Where “processor,” “at least one processor,” and “one or more processors” are described as being configured to perform various functions as used in the specification, these terms may include, for example, without limitation, situations in which a single processor performs. There may be situations in which some of the cited functions and other processor(s) perform other functions among the cited functions, and also situations in which a single processor can perform all the cited functions. Additionally, at least one processor may include a combination of processors that perform the various enumerated / disclosed functions, for example, in a distributed manner. At least one processor may execute program instructions to achieve or perform the various functions.

[0056] In the present disclosure, 'image' may include a picture, a still image, a frame, a video composed of a plurality of consecutive still images, or a video.

[0057] In the present disclosure, 'sample' may include data to be processed as data assigned to a sampling location of an image. For example, a sample may include pixels within a frame in a spatial region. A block may refer to a unit comprising a plurality of samples.

[0058] Hereinafter, with reference to FIGS. 1 to 19, an image encoding method and apparatus based on a tree structure encoding unit and a conversion unit according to one embodiment of the present disclosure, an image decoding method and apparatus are disclosed.

[0059] FIG. 1 illustrates a block diagram of an image decoding device (100) according to one embodiment of the present disclosure.

[0060] The video decoding device (100) may include a bitstream acquisition unit (110) and a decoding unit (120). The bitstream acquisition unit (110) and the decoding unit (120) may include at least one processor. Additionally, the bitstream acquisition unit (110) and the decoding unit (120) may include a memory that stores instructions to be executed by at least one processor.

[0061] The bitstream acquisition unit (110) can receive a bitstream. The bitstream contains information in which an image is encoded by an image encoding device (200) described later. Additionally, the bitstream can be transmitted from the image encoding device (200). The image encoding device (200) and the image decoding device (100) can be connected via wired or wireless connection, and the bitstream acquisition unit (110) can receive the bitstream via wired or wireless connection. The bitstream acquisition unit (110) can receive the bitstream from a storage medium such as an optical medium or a hard disk. The decoding unit (120) can restore the image based on information obtained from the received bitstream. The decoding unit (120) can obtain syntax elements for restoring the image from the bitstream. The decoding unit (120) can restore the image based on the syntax elements.

[0062] To explain in detail the operation of the video decoding device (100), the bitstream acquisition unit (110) can receive a bitstream.

[0063] The image decoder (100) can perform an operation of obtaining an empty string corresponding to a partitioning mode of the encoding unit from a bitstream. The image decoder (100) can also perform an operation of determining a partitioning rule for the encoding unit. Additionally, the image decoder (100) can perform an operation of partitioning the encoding unit into a plurality of encoding units based on at least one of the empty string corresponding to the partitioning mode and the partitioning rule. To determine the partitioning rule, the image decoder (100) can determine an allowable first range of the size of the encoding unit according to the ratio of the width and height of the encoding unit. To determine the partitioning rule, the image decoder (100) can determine an allowable second range of the size of the encoding unit according to the partitioning mode of the encoding unit.

[0064] In the following, the division of a encoding unit according to one embodiment of the present disclosure will be described in detail.

[0065] First, a picture may be divided into one or more slices or one or more tiles. A slice or a tile may be a sequence of one or more Coding Tree Units (CTUs). Depending on the embodiment, a slice may include one or more tiles, and a slice may include one or more Coding Tree Units. A slice containing one or more tiles may be determined within the picture.

[0066] In contrast to the Max Coding Unit (CTU), there is the Max Coding Tree Block (CTB). A Max Coding Tree Block (CTB) refers to an NxN block containing NxN samples (where N is an integer). Each color component can be divided into one or more Max Coding Tree Blocks.

[0067] When a picture has three sample arrays (sample arrays for Y, Cr, and Cb components), the maximum encoding unit (CTU) is a unit that includes a maximum encoding block for luminance samples and two corresponding maximum encoding blocks for chroma samples, and syntax structures used to encode the luminance samples and chroma samples. When a picture is a monochrome picture, the maximum encoding unit is a unit that includes a maximum encoding block for monochrome samples and syntax structures used to encode the monochrome samples. When a picture is encoded into color planes separated by color components, the maximum encoding unit is a unit that includes syntax structures used to encode the samples of the picture and the image.

[0068] A single maximum coding block (CTB) can be divided into MxN coding blocks containing MxN samples (M and N are integers). In one embodiment, a coding block may be referred to as a coding unit.

[0069] When a picture has sample arrays for Y, Cr, and Cb components, a coding unit (CU) is a unit that includes a coding block for luminance samples and two coding blocks for corresponding chroma samples, and syntax structures used to encode the luminance samples and chroma samples. When a picture is a monochrome picture, a coding unit is a unit that includes a coding block for monochrome samples and syntax structures used to encode the monochrome samples. When a picture is a picture encoded in color planes separated by color components, a coding unit is a unit that includes syntax structures used to encode the corresponding picture and image samples.

[0070] As explained above, the maximum encoding block and the maximum encoding unit are distinct concepts, and the encoding block and the encoding unit are distinct concepts. That is, the (maximum) encoding unit refers to a data structure that includes the (maximum) encoding block containing the corresponding sample and the corresponding syntax structure. However, since a person skilled in the art can understand that the (maximum) encoding unit or the (maximum) encoding block refers to a block of a predetermined size containing a predetermined number of samples, the maximum encoding block and the maximum encoding unit, or the encoding block and the encoding unit, are referred to without distinction in the following specification unless there are special circumstances.

[0071] The image can be divided into Coding Tree Units (CTUs). The size of the CTU can be determined based on information obtained from the bitstream. The shape of the CTU can be a square of equal size, but it is not limited to this.

[0072] For example, information about the maximum size of a luma-encoded block can be obtained from a bitstream. For example, the maximum size of a luma-encoded block indicated by the information about the maximum size of a luma-encoded block may be one of 4x4, 8x8, 16x16, 32x32, 64x64, 128x128, or 256x256.

[0073] For example, information regarding the maximum size of a two-divisionable luma encoding block and the difference in luma block size can be obtained from a bitstream. The information regarding the difference in luma block size may indicate the size difference between the maximum luma encoding unit and the maximum two-divisionable luma encoding block. Therefore, by combining the information regarding the maximum size of the two-divisionable luma encoding block obtained from the bitstream with the information regarding the difference in luma block size, the size of the maximum luma encoding unit can be determined. Using the size of the maximum luma encoding unit, the size of the maximum chroma encoding unit can also be determined. For example, if the Y:Cb:Cr ratio according to the color format is 4:2:0, the size of the chroma block may be half the size of the luma block, and similarly, the size of the maximum chroma encoding unit may be half the size of the maximum luma encoding unit.

[0074] According to one embodiment, information regarding the maximum size of a binary splittable luminous encoding block is obtained from a bitstream, so the maximum size of the binary splittable luminous encoding block can be determined variably. Alternatively, the maximum size of a ternary splittable luminous encoding block can be fixed. For example, the maximum size of a ternary splittable luminous encoding block in image I may be 32x32, and the maximum size of a ternary splittable luminous encoding block in picture P or image B may be 64x64.

[0075] Additionally, the maximum encoding unit can be hierarchically divided into encoding units based on splitting mode information obtained from the bitstream. As splitting mode information, at least one of information indicating whether it is a quad split, information indicating whether it is a multi-split, splitting direction information, and splitting type information can be obtained from the bitstream.

[0076] For example, information indicating whether quad splitting is performed can indicate whether the current encoding unit will be quad split or not.

[0077] If the current encoding unit is not quad-splitting, information indicating multi-splitting can indicate whether the current encoding unit will no longer be split (NO_SPLIT) or whether it will be binary / ternary split.

[0078] If the current encoding unit is binary or binary split, the splitting direction information indicates that the current encoding unit is split in either the horizontal or vertical direction.

[0079] If the current encoding unit is split horizontally or vertically, the split type information indicates that the current encoding unit is split into binary or binary splits.

[0080] The splitting mode of the current encoding unit can be determined based on the splitting direction information and the splitting type information. The splitting mode when the current encoding unit is binary split in the horizontal direction can be determined as binary horizontal splitting (SPLIT_BT_HOR), when it is territorial split in the horizontal direction as territorial horizontal splitting (SPLIT_TT_HOR), when it is binary split in the vertical direction as binary vertical splitting (SPLIT_BT_VER), and when it is territorial split in the vertical direction as territorial vertical splitting (SPLIT_TT_VER).

[0081] The image decoding device (100) can obtain splitting mode information from a bitstream from a single empty string. The form of the bitstream received by the image decoding device (100) may include a fixed-length binary code, a unary code, a truncated unary code, a predetermined binary code, etc. The empty string represents information as a sequence of binary numbers. The empty string may consist of at least one bit. The image decoding device (100) can obtain splitting mode information corresponding to the empty string based on a splitting rule. Based on the single empty string, the image decoding device (100) can determine whether to quad split the encoding unit, whether not to split it, or the splitting direction and splitting type.

[0082] A coding unit may be smaller than or equal to a maximum coding unit. For example, since the maximum coding unit is a coding unit having the maximum size, it is also a coding unit. If the segmentation mode information for the maximum coding unit indicates that it is not segmented, the coding unit determined from the maximum coding unit has the same size as the maximum coding unit. If the segmentation mode information for the maximum coding unit indicates that it is segmented, the maximum coding unit may be segmented into coding units. Additionally, if the segmentation mode information for a coding unit indicates segmentation, the coding units may be segmented into coding units of smaller size. However, the segmentation of the image is not limited to this, and the maximum coding unit and the coding unit may not be distinguished. The segmentation of coding units is explained in more detail in FIGS. 3 through 16.

[0083] Additionally, one or more prediction blocks for prediction may be determined from the coding unit. The prediction blocks may be equal to or smaller than the coding unit. Additionally, one or more transformation blocks for transformation may be determined from the coding unit. The transformation blocks may be equal to or smaller than the coding unit. The transformation blocks may be referred to as the transformation unit.

[0084] The shape and size of the transformation block and the prediction block may not be related to each other.

[0085] In another embodiment, prediction can be performed using the encoding unit as a prediction block. Additionally, conversion can be performed using the encoding unit as a conversion block.

[0086] The division of the encoding unit is described in more detail in FIGS. 3 through 16. The current block and neighbor block of the present disclosure may represent one of the maximum encoding unit, the encoding unit, the prediction block, and the transformation block. Additionally, the current block or the current encoding unit is a block currently undergoing decoding or encoding, or a block currently undergoing division. The neighbor block may be a block restored prior to the current block. The neighbor block may be spatially or temporally adjacent to the current block. The neighbor block may be located on one of the lower-left, left, upper-left, upper, upper-right, right, or lower-right sides of the current block.

[0087] The above-described embodiment explains the operation related to the image decoding method performed by the image decoding device (100). Below, the operation of the image encoding device (200) that performs an image encoding method corresponding to the reverse process of the image decoding method will be explained through an embodiment of the present disclosure.

[0088] FIG. 2 illustrates a block diagram of an image encoding device (200) capable of encoding an image based on at least one of block shape information and segmented shape mode information according to one embodiment of the present disclosure.

[0089] The video encoding device (200) may include an encoding unit (220) and a bitstream generation unit (210). The encoding unit (220) may receive an input video and encode the input video. The encoding unit (220) may encode the input video to obtain at least one syntax element. The syntax element may include at least one of a skip flag, a prediction mode, a motion vector difference, a motion vector prediction method (or index), a transform quantized coefficient, a coded block pattern, a coded block flag, an intra prediction mode, a direct flag, a merge flag, a delta QP, a reference index, a prediction direction, and a transform index. The encoding unit (220) may determine a context model based on block shape information including at least one of the shape, direction, width, and height ratio or size of the encoding unit.

[0090] The bitstream generation unit (210) can generate a bitstream based on an encoded input image. For example, the bitstream generation unit (210) can generate a bitstream by entropy encoding syntax elements based on a context model. In one embodiment of the present disclosure, the image encoding device (100) can generate a bitstream according to an image encoding method to be described later. The image encoding device (100) can store the bitstream in a computer-readable storage medium.

[0091] In one embodiment of the present disclosure, the image encoding device (200) can transmit a bitstream to the image decoding device (100). For example, the image encoding device (200) can transmit a bitstream generated by an image encoding method.

[0092] According to one embodiment of the present disclosure, the encoding unit (220) of the image encoding device (200) can determine the shape of the encoding unit. For example, the encoding unit may be square or non-square in shape, and information indicating such shape may be included in block shape information.

[0093] According to one embodiment of the present disclosure, the encoding unit (220) may determine how the encoding unit will be divided. The encoding unit (220) may determine the form of at least one encoding unit included in the encoding unit, and the bitstream generation unit (210) may generate a bitstream including division form mode information including information about the form of such encoding unit.

[0094] According to one embodiment of the present disclosure, the encoding unit (220) may determine whether the encoding unit is divided or not. If the encoding unit (220) determines that only one encoding unit is included in the encoding unit or that the encoding unit is not divided, the bitstream generation unit (210) may generate a bitstream including division type mode information indicating that the encoding unit is not divided. Additionally, the encoding unit (220) may divide the encoding unit into a plurality of encoding units, and the bitstream generation unit (210) may generate a bitstream including division type mode information indicating that the encoding unit is divided into a plurality of encoding units.

[0095] According to one embodiment of the present disclosure, information indicating how many encoding units to divide or in which direction to divide may be included in the division type mode information. For example, the division type mode information may indicate dividing in at least one of the vertical direction and the horizontal direction, or not dividing.

[0096] The video encoding device (200) determines information regarding the segmentation mode based on the segmentation mode of the encoding unit. The video encoding device (200) determines a context model based on at least one of the ratio or size of the shape, direction, width, and height of the encoding unit. Then, the video encoding device (200) generates information regarding the segmentation mode for segmenting the encoding unit based on the context model as a bitstream.

[0097] The video encoding device (200) may obtain an array for matching at least one of the ratio or size of the shape, direction, width, and height of an encoding unit with an index for the context model in order to determine the context model. The video encoding device (200) may obtain an index for the context model based on at least one of the ratio or size of the shape, direction, width, and height of an encoding unit in the array. The video encoding device (200) may determine the context model based on the index for the context model.

[0098] The video encoding device (200) may determine a context model based further on block shape information including at least one of the ratio or size of the shape, direction, width, and height of a surrounding encoding unit adjacent to the encoding unit, in order to determine the context model. Additionally, the surrounding encoding unit may include at least one of the encoding units located on the lower left, left, upper left, upper, upper right, right, or lower right side of the encoding unit.

[0099] Additionally, the video encoding device (200) can compare the width of an upper peripheral encoding unit with the width of an encoding unit in order to determine a context model. Additionally, the video encoding device (200) can compare the height of left and right peripheral encoding units with the height of an encoding unit. Additionally, the video encoding device (200) can determine a context model based on the comparison results.

[0100] The operation of the video encoding device (200) includes content similar to the operation of the video decoding device (100) described in FIGS. 3 to 19, so a detailed description is omitted.

[0101] FIG. 3 illustrates a process in which an image decoding device (100) divides a current encoding unit to determine at least one encoding unit according to one embodiment of the present disclosure.

[0102] The block shape may include 4Nx4N, 4Nx2N, 2Nx4N, 4NxN, Nx4N, 32NxN, Nx32N, 16NxN, Nx16N, 8NxN, or Nx8N. Here, N may be a positive integer. Block shape information is information indicating at least one of the shape, orientation, width, and height ratio or size of the encoding unit.

[0103] The shape of the encoding unit may include square and non-square. When the width and height of the encoding unit are the same (i.e., when the block shape of the encoding unit is 4Nx4N), the image decoder (100) may determine the block shape information of the encoding unit as square. The image decoder (100) may determine the shape of the encoding unit as non-square.

[0104] When the width and height of the encoding unit are different (i.e., when the block shape of the encoding unit is 4Nx2N, 2Nx4N, 4NxN, Nx4N, 32NxN, Nx32N, 16NxN, Nx16N, 8NxN, or Nx8N), the image decoder (100) can determine the block shape information of the encoding unit as non-square. When the shape of the encoding unit is non-square, the image decoder (100) can determine the ratio of the width and height among the block shape information of the encoding unit as at least one of 1:2, 2:1, 1:4, 4:1, 1:8, 8:1, 1:16, 16:1, 1:32, or 32:1. Additionally, based on the width and height of the encoding unit, the image decoding device (100) can determine whether the encoding unit is in a horizontal direction or a vertical direction. Additionally, based on at least one of the width, height, or width of the encoding unit, the image decoding device (100) can determine the size of the encoding unit.

[0105] According to one embodiment of the present disclosure, the image decoding device (100) can determine the form of the encoding unit using block form information and can determine how the encoding unit is divided using division form mode information. That is, depending on what block form the block form information used by the image decoding device (100) represents, the method of dividing the encoding unit represented by the division form mode information can be determined.

[0106] The video decoder (100) can obtain split-form mode information from the bitstream. However, it is not limited thereto, and the video decoder (100) and the video encoding device (200) can determine pre-agreed split-form mode information based on block form information. The video decoder (100) can determine pre-agreed split-form mode information for a maximum encoding unit or a minimum encoding unit. For example, the video decoder (100) can determine the split-form mode information for the maximum encoding unit as quad split. Additionally, the video decoder (100) can determine the split-form mode information for the minimum encoding unit as "not split." Specifically, the video decoder (100) can determine the size of the maximum encoding unit to be 256x256. The video decoder (100) can determine the pre-agreed split-form mode information as quad split. Quad split is a split-form mode that divides both the width and height of the encoding unit into two equal parts. The image decoder (100) can obtain a 128x128 size encoding unit from a 256x256 size encoding unit based on the division mode information. Additionally, the image decoder (100) can determine the size of the minimum encoding unit to be 4x4. The image decoder (100) can obtain division mode information indicating "not divided" for the minimum encoding unit.

[0107] According to one embodiment of the present disclosure, the image decoding device (100) may use block shape information indicating that the current encoding unit is in the shape of a square. For example, the image decoding device (100) may determine whether to not divide the square encoding unit, to divide it vertically, to divide it horizontally, or to divide it into four encoding units, etc., according to the division shape mode information. Referring to FIG. 3, when the block shape information of the current encoding unit (300) indicates a square shape, the decoding unit (120) may not divide the encoding unit (310a) having the same size as the current encoding unit (300) according to the division shape mode information indicating that it is not divided, or may determine the divided encoding units (310b, 310c, 310d, 310e, 310f, etc.) based on the division shape mode information indicating a predetermined division method.

[0108] Referring to FIG. 3, the image decoding device (100) can determine two encoding units (310b) that divide the current encoding unit (300) in the vertical direction based on splitting form mode information indicating that it is divided in the vertical direction according to one embodiment of the present disclosure. The image decoding device (100) can determine two encoding units (310c) that divide the current encoding unit (300) in the horizontal direction based on splitting form mode information indicating that it is divided in the horizontal direction. The image decoding device (100) can determine four encoding units (310d) that divide the current encoding unit (300) in the vertical direction and the horizontal direction based on splitting form mode information indicating that it is divided in the vertical direction and the horizontal direction. The image decoding device (100) may determine three encoding units (310e) that divide the current encoding unit (300) in the vertical direction based on division form mode information indicating ternary division in the vertical direction according to one embodiment of the present disclosure. The image decoding device (100) may determine three encoding units (310f) that divide the current encoding unit (300) in the horizontal direction based on division form mode information indicating ternary division in the horizontal direction. However, the division form in which the square encoding unit can be divided should not be interpreted as being limited to the form described above, and may include various forms that the division form mode information can represent. The predetermined division forms in which the square encoding unit is divided will be specifically described below through one embodiment of the present disclosure.

[0109] FIG. 4 illustrates a process in which an image decoding device (100) determines at least one encoding unit by dividing an encoding unit that is in the shape of a non-square according to one embodiment of the present disclosure.

[0110] According to one embodiment of the present disclosure, the image decoding device (100) may use block shape information indicating that the current encoding unit is in a non-square shape. The image decoding device (100) may determine whether to not divide the current encoding unit of the non-square shape or to divide it in a predetermined way according to the division shape mode information. Referring to FIG. 4, when the block shape information of the current encoding unit (400 or 450) indicates a non-square shape, the image decoding device (100) may determine an encoding unit (410 or 460) having the same size as the current encoding unit (400 or 450) according to division shape mode information indicating that it is not divided, or determine divided encoding units (420a, 420b, 430a, 430b, 430c, 470a, 470b, 480a, 480b, 480c) based on division shape mode information indicating a predetermined division method. A predetermined division method in which a non-square encoding unit is divided will be specifically described below through an embodiment of the present disclosure.

[0111] According to one embodiment of the present disclosure, the image decoding device (100) can determine the form in which the encoding unit is divided using the division form mode information, and in this case, the division form mode information may indicate the number of at least one encoding unit generated by dividing the encoding unit. Referring to FIG. 4, when the division form mode information indicates that the current encoding unit (400 or 450) is divided into two encoding units, the image decoding device (100) can determine two encoding units (420a, 420b, or 470a, 470b) included in the current encoding unit by dividing the current encoding unit (400 or 450) based on the division form mode information.

[0112] According to one embodiment of the present disclosure, when an image decoding device (100) divides a current encoding unit (400 or 450) in a non-square shape based on division shape mode information, the image decoding device (100) may divide the current encoding unit by considering the position of the long side of the current encoding unit (400 or 450) in a non-square shape. For example, the image decoding device (100) may determine a plurality of encoding units by dividing the current encoding unit (400 or 450) in a direction that divides the long side of the current encoding unit (400 or 450) by considering the shape of the current encoding unit (400 or 450).

[0113] According to one embodiment of the present disclosure, when the segmentation mode information indicates that the encoding unit is divided into an odd number of blocks (terminal segmentation), the image decoder (100) can determine an odd number of encoding units included in the current encoding unit (400 or 450). For example, when the segmentation mode information indicates that the current encoding unit (400 or 450) is divided into three encoding units, the image decoder (100) can divide the current encoding unit (400 or 450) into three encoding units (430a, 430b, 430c, 480a, 480b, 480c).

[0114] According to one embodiment of the present disclosure, the ratio of the width to the height of the current encoding unit (400 or 450) may be 4:1 or 1:4. When the ratio of the width to the height is 4:1, the block shape information may be in the horizontal direction because the width is longer than the height. When the ratio of the width to the height is 1:4, the block shape information may be in the vertical direction because the width is shorter than the height. The image decoder (100) may determine to divide the current encoding unit into an odd number of blocks based on the division shape mode information. Additionally, the image decoder (100) may determine the division direction of the current encoding unit (400 or 450) based on the block shape information of the current encoding unit (400 or 450). For example, if the current encoding unit (400) is in a vertical direction, the image decoding device (100) can determine the encoding units (430a, 430b, 430c) by dividing the current encoding unit (400) in a horizontal direction. Also, if the current encoding unit (450) is in a horizontal direction, the image decoding device (100) can determine the encoding units (480a, 480b, 480c) by dividing the current encoding unit (450) in a vertical direction.

[0115] According to one embodiment of the present disclosure, an image decoding device (100) may determine an odd number of encoding units included in a current encoding unit (400 or 450), and the sizes of the determined encoding units may not all be the same. For example, among the determined odd number of encoding units (430a, 430b, 430c, 480a, 480b, 480c), the size of a certain encoding unit (430b or 480b) may have a different size from the other encoding units (430a, 430c, 480a, 480c). That is, the current encoding unit (400 or 450) can be divided and determined as a encoding unit, and the encoding unit can have multiple types of sizes, and in some cases, an odd number of encoding units (430a, 430b, 430c, 480a, 480b, 480c) may each have different sizes.

[0116] According to one embodiment of the present disclosure, when the segmentation mode information indicates that the encoding unit is divided into an odd number of blocks, the image decoding device (100) can determine the odd number of encoding units included in the current encoding unit (400 or 450), and furthermore, the image decoding device (100) can impose a predetermined limit on at least one encoding unit among the odd number of encoding units generated by the segmentation. Referring to FIG. 4, the image decoding device (100) can perform the decoding process for the central encoding unit (430b, 480b) among the three encoding units (430a, 430b, 430c, 480a, 480b, 480c) generated by the segmentation of the current encoding unit (400 or 450) differently from the other encoding units (430a, 430c, 480a, 480c). For example, the video decoding device (100) may restrict the centrally located encoding unit (430b, 480b) from being further divided unlike other encoding units (430a, 430c, 480a, 480c), or restrict it to being divided only a predetermined number of times.

[0117] FIG. 5 illustrates a process in which an image decoding device (100) divides a encoding unit based on at least one of block form information and division form mode information according to one embodiment of the present disclosure.

[0118] According to one embodiment of the present disclosure, an image decoding device (100) may determine whether to divide a square-shaped first encoding unit (500) into encoding units or not to divide it based on at least one of block shape information and division shape mode information. According to one embodiment of the present disclosure, if the division shape mode information indicates that the first encoding unit (500) is divided in a horizontal direction, the image decoding device (100) may divide the first encoding unit (500) in a horizontal direction to determine a second encoding unit (510). The first encoding unit, the second encoding unit, and the third encoding unit used according to one embodiment of the present disclosure are terms used to understand the relationship before and after division between the encoding units. For example, if the first encoding unit is divided, the second encoding unit may be determined, and if the second encoding unit is divided, the third encoding unit may be determined. In the following, the relationship between the first encoding unit, the second encoding unit, and the third encoding unit used may be understood as following the features described above.

[0119] According to one embodiment of the present disclosure, the image decoding device (100) may determine whether to divide the determined second encoding unit (510) into encoding units or not to divide it based on the division shape mode information. Referring to FIG. 5, the image decoding device (100) may divide the determined non-square second encoding unit (510) into at least one third encoding unit (520a, 520b, 520c, 520d, etc.) by dividing the first encoding unit (500) based on the division shape mode information, or may not divide the second encoding unit (510). The image decoding device (100) can obtain split-form mode information, and the image decoding device (100) can divide a first encoding unit (500) based on the obtained split-form mode information to divide a plurality of second encoding units (e.g., 510) of various forms, and the second encoding units (510) can be divided according to the method in which the first encoding unit (500) was divided based on the split-form mode information. According to one embodiment of the present disclosure, when the first encoding unit (500) is divided into a second encoding unit (510) based on the split-form mode information for the first encoding unit (500), the second encoding unit (510) can also be divided into a third encoding unit (e.g., 520a, 520b, 520c, 520d, etc.) based on the split-form mode information for the second encoding unit (510). That is, the encoding unit can be recursively partitioned based on partitioning mode information associated with each encoding unit. Thus, a square encoding unit can be determined from a non-square encoding unit, and a non-square encoding unit can be determined by recursively partitioning this square encoding unit.

[0120] Referring to FIG. 5, among the odd number of third encoding units (520b, 520c, 520d) determined by dividing a second encoding unit (510) of a non-square shape, a predetermined encoding unit (e.g., a central encoding unit or a square encoding unit) may be recursively divided. According to one embodiment of the present disclosure, a third encoding unit (520b) of a non-square shape, which is one of the odd number of third encoding units (520b, 520c, 520d), may be divided horizontally into a plurality of fourth encoding units. A fourth encoding unit (530b or 530d) of a non-square shape, which is one of the plurality of fourth encoding units (530a, 530b, 530c, 530d), may again be divided into a plurality of encoding units. For example, a non-square fourth encoding unit (530b or 530d) may be further subdivided into an odd number of encoding units. A method that can be used for the recursive subdivision of encoding units will be described later through an embodiment of the present disclosure.

[0121] According to one embodiment of the present disclosure, the image decoding device (100) may divide each of the third encoding units (520a, 520b, 520c, 520d, etc.) into encoding units based on the division shape mode information. Additionally, the image decoding device (100) may decide not to divide the second encoding unit (510) based on the division shape mode information. According to one embodiment of the present disclosure, the image decoding device (100) may divide the non-square second encoding unit (510) into an odd number of third encoding units (520b, 520c, 520d). The image decoding device (100) may place a certain limit on a certain third encoding unit among the odd number of third encoding units (520b, 520c, 520d). For example, the video decoding device (100) may limit the encoding unit (520c) located in the middle of the odd number of third encoding units (520b, 520c, 520d) so that it is not further divided or is limited to being divided a set number of times.

[0122] Referring to FIG. 5, the image decoding device (100) may limit the middle encoding unit (520c) among the odd number of third encoding units (520b, 520c, 520d) included in the second encoding unit (510) of a non-square shape to no longer be divided, to be divided into a predetermined division form (e.g., divided into only 4 encoding units or divided into a form corresponding to the divided form of the second encoding unit (510)), or to be divided only a predetermined number of times (e.g., divided only n times, n > 0). However, the above limitation on the middle encoding unit (520c) is merely a simple example and should not be interpreted as being limited to the above-described examples, but should be interpreted as including various limitations that allow the middle encoding unit (520c) to be decoded differently from the other encoding units (520b, 520d).

[0123] According to one embodiment of the present disclosure, an image decoding device (100) can obtain splitting form mode information used to split a current encoding unit at a predetermined location within the current encoding unit.

[0124] FIG. 6 illustrates a method for an image decoding device (100) to determine a predetermined encoding unit among an odd number of encoding units according to one embodiment of the present disclosure.

[0125] Referring to FIG. 6, the segmentation mode information of the current encoding unit (600, 650) can be obtained from a sample at a specific location among a plurality of samples included in the current encoding unit (600, 650) (e.g., a sample located in the center (640, 690)). However, the specific location within the current encoding unit (600) where at least one of such segmentation mode information can be obtained should not be interpreted as being limited to the center location shown in FIG. 6, and should be interpreted as including various locations within the current encoding unit (600) (e.g., top, bottom, left, right, top-left, bottom-left, top-right, or bottom-right, etc.). The image decoding device (100) can obtain the segmentation mode information obtained from the specific location and decide whether to divide the current encoding unit into encoding units of various shapes and sizes or not to divide it.

[0126] According to one embodiment of the present disclosure, the image decoding device (100) may select one of the encoding units when the current encoding unit is divided into a predetermined number of encoding units. There may be various methods for selecting one of the plurality of encoding units, and such methods will be described later through one embodiment of the present disclosure.

[0127] According to one embodiment of the present disclosure, the image decoding device (100) can divide the current encoding unit into a plurality of encoding units and determine the encoding unit at a predetermined position.

[0128] According to one embodiment of the present disclosure, an image decoding device (100) may use information indicating the position of each of the odd number of encoding units to determine the encoding unit located in the middle among the odd number of encoding units. Referring to FIG. 6, the image decoding device (100) may divide the current encoding unit (600) or the current encoding unit (650) to determine the odd number of encoding units (620a, 620b, 620c) or the odd number of encoding units (660a, 660b, 660c). The image decoding device (100) may determine the middle encoding unit (620b) or the middle encoding unit (660b) by using information regarding the positions of the odd number of encoding units (620a, 620b, 620c) or the odd number of encoding units (660a, 660b, 660c). For example, the image decoding device (100) can determine the centrally located encoding unit (620b) by determining the positions of the encoding units (620a, 620b, 620c) based on information indicating the positions of a predetermined sample included in the encoding units (620a, 620b, 620c). Specifically, the image decoding device (100) can determine the centrally located encoding unit (620b) by determining the positions of the encoding units (620a, 620b, 620c) based on information indicating the positions of the upper-left samples (630a, 630b, 630c) of the encoding units (620a, 620b, 620c).

[0129] According to one embodiment of the present disclosure, information indicating the location of the upper-left sample (630a, 630b, 630c) included in each of the encoding units (620a, 620b, 620c) may include information regarding the location or coordinates within the picture of the encoding units (620a, 620b, 620c). According to one embodiment of the present disclosure, information indicating the location of the upper-left sample (630a, 630b, 630c) included in each of the encoding units (620a, 620b, 620c) may include information indicating the width or height of the encoding units (620a, 620b, 620c) included in the current encoding unit (600), and such width or height may correspond to information indicating the difference between coordinates within the picture of the encoding units (620a, 620b, 620c). That is, the image decoding device (100) can determine the centrally located encoding unit (620b) by directly using information about the position or coordinates of the encoding units (620a, 620b, 620c) within the picture, or by using information about the width or height of the encoding unit corresponding to the difference value between the coordinates.

[0130] According to one embodiment of the present disclosure, information indicating the location of a sample (630a) at the upper left of an upper encoding unit (620a) may be represented by the (xa, ya) coordinates, information indicating the location of a sample (530b) at the upper left of a middle encoding unit (620b) may be represented by the (xb, yb) coordinates, and information indicating the location of a sample (630c) at the upper left of a lower encoding unit (620c) may be represented by the (xc, yc) coordinates. An image decoding device (100) may determine the middle encoding unit (620b) using the coordinates of the upper left samples (630a, 630b, 630c) included in each of the encoding units (620a, 620b, 620c). For example, when the coordinates of the upper-left samples (630a, 630b, 630c) are sorted in ascending or descending order, the encoding unit (620b) containing the coordinates (xb, yb) of the sample (630b) located in the middle can be determined as the encoding unit located in the middle among the encoding units (620a, 620b, 620c) determined by dividing the current encoding unit (600). However, the coordinates indicating the position of the upper-left samples (630a, 630b, 630c) may represent absolute positions within the picture, and furthermore, based on the position of the upper-left sample (630a) of the upper-left of the upper-left of the middle encoding unit (620b), the (dxb, dyb) coordinates, which represent the relative position of the upper-left sample (630b) of the middle encoding unit (620b), and the (dxc, dyc) coordinates, which represent the relative position of the upper-left sample (630c) of the lower encoding unit (620c) may also be used. In addition, the method of determining the encoding unit of a predetermined position by using the coordinates of the corresponding sample as information indicating the position of the sample included in the encoding unit should not be interpreted as being limited to the method described above, but should be interpreted as various arithmetic methods that can utilize the coordinates of the sample.

[0131] According to one embodiment of the present disclosure, the image decoding device (100) can divide the current encoding unit (600) into a plurality of encoding units (620a, 620b, 620c) and select an encoding unit among the encoding units (620a, 620b, 620c) according to a predetermined criterion. For example, the image decoding device (100) can select an encoding unit (620b) of a different size among the encoding units (620a, 620b, 620c).

[0132] According to one embodiment of the present disclosure, an image decoding device (100) can determine the width or height of each of the encoding units (620a, 620b, 620c) using the (xa, ya) coordinates, which are information indicating the location of a sample (630a) at the top left of the upper encoding unit (620a); the (xb, yb) coordinates, which are information indicating the location of a sample (630b) at the top left of the middle encoding unit (620b); and the (xc, yc) coordinates, which are information indicating the location of a sample (630c) at the top left of the lower encoding unit (620c). The image decoding device (100) can determine the size of each of the encoding units (620a, 620b, 620c) using coordinates (xa, ya), (xb, yb), and (xc, yc) which represent the positions of the encoding units (620a, 620b, 620c). According to one embodiment of the present disclosure, the image decoding device (100) can determine the width of the upper encoding unit (620a) as the width of the current encoding unit (600). The image decoding device (100) can determine the height of the upper encoding unit (620a) as yb-ya. According to one embodiment of the present disclosure, the image decoding device (100) can determine the width of the middle encoding unit (620b) as the width of the current encoding unit (600). The image decoding device (100) can determine the height of the middle encoding unit (620b) as yc-yb. According to one embodiment of the present disclosure, the image decoding device (100) can determine the width or height of the lower encoding unit using the width or height of the current encoding unit and the width and height of the upper encoding unit (620a) and the middle encoding unit (620b). The image decoding device (100) can determine an encoding unit having a different size from other encoding units based on the width and height of the determined encoding units (620a, 620b, 620c).Referring to FIG. 6, the image decoding device (100) can determine a middle encoding unit (620b) having a size different from that of the upper encoding unit (620a) and the lower encoding unit (620c) as the encoding unit of a predetermined position. However, since the process of the image decoding device (100) described above determining the encoding unit having a size different from other encoding units is merely one embodiment of determining the encoding unit of a predetermined position using the size of the encoding unit determined based on sample coordinates, various processes of determining the encoding unit of a predetermined position by comparing the size of the encoding unit determined according to the predetermined sample coordinates may be used.

[0133] The image decoding device (100) can determine the width or height of each of the encoding units (660a, 660b, 660c) using the (xd, yd) coordinates, which are information indicating the location of the upper-left sample (670a) of the left encoding unit (660a), the (xe, ye) coordinates, which are information indicating the location of the upper-left sample (670b) of the middle encoding unit (660b), and the (xf, yf) coordinates, which are information indicating the location of the upper-left sample (670c) of the right encoding unit (660c). The image decoding device (100) can determine the size of each of the encoding units (660a, 660b, 660c) using the (xd, yd), (xe, ye), and (xf, yf) coordinates, which are information indicating the location of the encoding units (660a, 660b, 660c).

[0134] According to one embodiment of the present disclosure, the image decoding device (100) may determine the width of the left encoding unit (660a) as xe-xd. The image decoding device (100) may determine the height of the left encoding unit (660a) as the height of the current encoding unit (650). According to one embodiment of the present disclosure, the image decoding device (100) may determine the width of the middle encoding unit (660b) as xf-xe. The image decoding device (100) may determine the height of the middle encoding unit (660b) as the height of the current encoding unit (600). According to one embodiment of the present disclosure, the image decoding device (100) may determine the width or height of the right encoding unit (660c) using the width or height of the current encoding unit (650) and the width and height of the left encoding unit (660a) and the middle encoding unit (660b). The image decoding device (100) can determine a encoding unit having a different size from other encoding units based on the width and height of the determined encoding units (660a, 660b, 660c). Referring to FIG. 6, the image decoding device (100) can determine a middle encoding unit (660b) having a different size from the left encoding unit (660a) and the right encoding unit (660c) as the encoding unit of a predetermined position. However, since the process of the image decoding device (100) described above determining a encoding unit having a different size from other encoding units is merely one embodiment of determining the encoding unit of a predetermined position using the size of the encoding unit determined based on sample coordinates, various processes of determining the encoding unit of a predetermined position by comparing the size of the encoding unit determined according to the predetermined sample coordinates may be used.

[0135] However, the sample location considered to determine the location of the encoding unit should not be interpreted as being limited to the upper left corner described above, and can be interpreted as allowing the use of information regarding the location of any sample included in the encoding unit.

[0136] According to one embodiment of the present disclosure, the image decoding device (100) may select an encoding unit at a predetermined position among an odd number of encoding units determined by dividing the current encoding unit, taking into consideration the shape of the current encoding unit. For example, if the current encoding unit is a non-square shape where the width is longer than the height, the image decoding device (100) may determine an encoding unit at a predetermined position according to the horizontal direction. That is, the image decoding device (100) may determine one of the encoding units at a different position in the horizontal direction and impose a restriction on that encoding unit. If the current encoding unit is a non-square shape where the height is longer than the width, the image decoding device (100) may determine an encoding unit at a predetermined position according to the vertical direction. That is, the image decoding device (100) may determine one of the encoding units at a different position in the vertical direction and impose a restriction on that encoding unit.

[0137] According to one embodiment of the present disclosure, an image decoding device (100) may use information indicating the location of each of an even number of encoding units to determine an encoding unit at a predetermined location among an even number of encoding units. The image decoding device (100) may determine an even number of encoding units by dividing (binary division) the current encoding unit and may determine an encoding unit at a predetermined location using information regarding the locations of the even number of encoding units. A specific process for this may be omitted as it may correspond to the process of determining an encoding unit at a predetermined location (e.g., a middle location) among an odd number of encoding units described above in FIG. 6.

[0138] According to one embodiment of the present disclosure, when a current encoding unit of a non-square shape is divided into a plurality of encoding units, certain information regarding the encoding unit at a certain position may be used during the division process to determine the encoding unit at a certain position among the plurality of encoding units. For example, an image decoding device (100) may use at least one of block shape information and division shape mode information stored in a sample included in the middle encoding unit during the division process to determine the encoding unit located in the middle among the encoding units into which the current encoding unit is divided into a plurality of encoding units.

[0139] Referring to FIG. 6, the image decoding device (100) can divide the current encoding unit (600) into a plurality of encoding units (620a, 620b, 620c) based on the division form mode information, and can determine the encoding unit (620b) located in the middle among the plurality of encoding units (620a, 620b, 620c). Furthermore, the image decoding device (100) can determine the encoding unit (620b) located in the middle by considering the location where the division form mode information is obtained. That is, the segmentation mode information of the current encoding unit (600) can be obtained from a sample (640) located in the middle of the current encoding unit (600), and based on the segmentation mode information, if the current encoding unit (600) is divided into a plurality of encoding units (620a, 620b, 620c), the encoding unit (620b) containing the sample (640) can be determined as the encoding unit located in the middle. However, the information used to determine the encoding unit located in the middle should not be interpreted as being limited to segmentation mode information, and various types of information may be used in the process of determining the encoding unit located in the middle.

[0140] According to one embodiment of the present disclosure, a predetermined information for identifying a coding unit at a predetermined location may be obtained from a predetermined sample included in the coding unit to be determined. Referring to FIG. 6, an image decoding device (100) may use a segmentation mode information obtained from a sample at a predetermined location within the current coding unit (600) (for example, a sample at the center of the current coding unit (600)) to determine a coding unit at a predetermined location (for example, a coding unit located in the center of the multiple divided coding units) among a plurality of coding units (620a, 620b, 620c) determined by dividing the current coding unit (600). That is, the image decoding device (100) can determine a sample at a predetermined location by considering the block shape of the current encoding unit (600), and the image decoding device (100) can determine a encoding unit (620b) containing a sample from which certain information (e.g., division shape mode information) can be obtained among a plurality of encoding units (620a, 620b, 620c) from which the current encoding unit (600) is divided and determined, and can impose a certain limit. Referring to FIG. 6, according to one embodiment of the present disclosure, the image decoding device (100) can determine a sample (640) located in the middle of the current encoding unit (600) as a sample from which certain information can be obtained, and the image decoding device (100) can impose a certain limit on the encoding unit (620b) containing such a sample (640) during the decoding process. However, the location of the sample from which the specified information can be obtained should not be interpreted as being limited to the location described above, but can be interpreted as samples at any location included in the encoding unit (620b) to be determined for the purpose of imposing a limit.

[0141] According to one embodiment of the present disclosure, the location of a sample from which a predetermined information can be obtained may be determined according to the shape of the current encoding unit (600). According to one embodiment of the present disclosure, block shape information may determine whether the shape of the current encoding unit is square or non-square, and may determine the location of a sample from which a predetermined information can be obtained according to the shape. For example, the image decoding device (100) may determine a sample located on a boundary that divides at least one of the width and height of the current encoding unit in half using at least one of the information regarding the width and height of the current encoding unit as a sample from which a predetermined information can be obtained. As another example, if the block shape information related to the current encoding unit indicates that it is a non-square shape, the image decoding device (100) may determine one of the samples adjacent to the boundary that divides the long side of the current encoding unit in half as a sample from which a predetermined information can be obtained.

[0142] According to one embodiment of the present disclosure, when the image decoding device (100) divides the current encoding unit into a plurality of encoding units, it may use division form mode information to determine the encoding unit at a predetermined position among the plurality of encoding units. According to one embodiment of the present disclosure, the image decoding device (100) may obtain division form mode information from a sample at a predetermined position included in the encoding unit, and the image decoding device (100) may divide the plurality of encoding units generated by dividing the current encoding unit using the division form mode information obtained from a sample at a predetermined position included in each of the plurality of encoding units. That is, the encoding unit may be recursively divided using the division form mode information obtained from a sample at a predetermined position included in each of the encoding units. Since the recursive division process of the encoding unit has been described in detail through FIG. 5, a detailed explanation will be omitted.

[0143] According to one embodiment of the present disclosure, the image decoding device (100) can determine at least one encoding unit by dividing the current encoding unit, and can determine the order in which the at least one encoding unit is decoded according to a predetermined block (e.g., the current encoding unit).

[0144] FIG. 7 illustrates the order in which a plurality of encoding units are processed when an image decoding device (100) determines a plurality of encoding units by dividing a current encoding unit according to one embodiment of the present disclosure.

[0145] According to one embodiment of the present disclosure, the image decoding device (100) may determine a second encoding unit (710a, 710b) by dividing a first encoding unit (700) in a vertical direction according to the splitting form mode information, determine a second encoding unit (730a, 730b) by dividing the first encoding unit (700) in a horizontal direction, or determine a second encoding unit (750a, 750b, 750c, 750d) by dividing the first encoding unit (700) in both a vertical and a horizontal direction.

[0146] Referring to FIG. 7, the image decoding device (100) can determine the order of processing the determined second encoding units (710a, 710b) in the horizontal direction (710c) by dividing the first encoding unit (700) in the vertical direction. The image decoding device (100) can determine the processing order of the determined second encoding units (730a, 730b) in the vertical direction (730c) by dividing the first encoding unit (700) in the horizontal direction. The image decoding device (100) can determine the second encoding unit (750a, 750b, 750c, 750d) determined by dividing the first encoding unit (700) into vertical and horizontal directions, according to a predetermined order in which encoding units located in one row are processed and then encoding units located in the next row are processed (e.g., raster scan order or z scan order (750e), etc.).

[0147] According to one embodiment of the present disclosure, the image decoding device (100) can recursively divide the encoding units. Referring to FIG. 7, the image decoding device (100) can divide the first encoding unit (700) to determine a plurality of encoding units (710a, 710b, 730a, 730b, 750a, 750b, 750c, 750d), and can recursively divide each of the determined plurality of encoding units (710a, 710b, 730a, 730b, 750a, 750b, 750c, 750d). A method of dividing multiple encoding units (710a, 710b, 730a, 730b, 750a, 750b, 750c, 750d) may be a method corresponding to a method of dividing the first encoding unit (700). Accordingly, the multiple encoding units (710a, 710b, 730a, 730b, 750a, 750b, 750c, 750d) may each be independently divided into multiple encoding units. Referring to FIG. 7, the image decoding device (100) may determine the second encoding units (710a, 710b) by dividing the first encoding unit (700) in a vertical direction, and furthermore, may determine whether to independently divide or not divide each of the second encoding units (710a, 710b).

[0148] According to one embodiment of the present disclosure, the image decoding device (100) may divide the second encoding unit (710a) on the left side horizontally into third encoding units (720a, 720b), and may not divide the second encoding unit (710b) on the right side.

[0149] According to one embodiment of the present disclosure, the processing order of the encoding units may be determined based on the process of dividing the encoding units. In other words, the processing order of the divided encoding units may be determined based on the processing order of the encoding units immediately before they are divided. The image decoding device (100) may determine the processing order of the third encoding units (720a, 720b), which are determined by dividing the second encoding unit (710a) on the left, independently of the second encoding unit (710b) on the right. Since the third encoding units (720a, 720b) ​​are determined by dividing the second encoding unit (710a) on the left in a horizontal direction, the third encoding units (720a, 720b) ​​may be processed in a vertical direction (720c). In addition, since the processing order of the second encoding unit (710a) on the left and the second encoding unit (710b) on the right corresponds to the horizontal direction (710c), the third encoding unit (720a, 720b) ​​included in the second encoding unit (710a) on the left can be processed in the vertical direction (720c) before the right encoding unit (710b) is processed. The above description is intended to explain the process in which the processing order of the encoding units is determined according to the encoding unit before division, and therefore should not be interpreted as being limited to the above-described embodiment, but should be interpreted as being used in various ways in which encoding units determined by division in various forms can be processed independently according to a predetermined order.

[0150] FIG. 8 illustrates a process in which, according to one embodiment of the present disclosure, an image decoding device (100) determines that the current encoding unit is divided into an odd number of encoding units when the encoding unit cannot be processed in a predetermined order.

[0151] According to one embodiment of the present disclosure, the image decoding device (100) may determine that the current encoding unit is divided into an odd number of encoding units based on acquired segmentation mode information. Referring to FIG. 8, a square-shaped first encoding unit (800) may be divided into non-square-shaped second encoding units (810a, 810b), and the second encoding units (810a, 810b) may each be independently divided into third encoding units (820a, 820b, 820c, 820d, 820e). According to one embodiment of the present disclosure, the image decoding device (100) can determine a plurality of third encoding units (820a, 820b) by dividing the left encoding unit (810a) among the second encoding units in a horizontal direction, and the right encoding unit (810b) can be divided into an odd number of third encoding units (820c, 820d, 820e).

[0152] According to one embodiment of the present disclosure, the image decoding device (100) can determine whether there exists an odd number of divided encoding units by determining whether the third encoding units (820a, 820b, 820c, 820d, 820e) can be processed in a predetermined order. Referring to FIG. 8, the image decoding device (100) can determine the third encoding units (820a, 820b, 820c, 820d, 820e) by recursively dividing the first encoding unit (800). The video decoding device (100) can determine whether the first encoding unit (800), the second encoding unit (810a, 810b), or the third encoding unit (820a, 820b, 820c, 820d, 820e) are divided into an odd number of encoding units based on at least one of block form information and division form mode information. For example, the encoding unit located on the right among the second encoding units (810a, 810b) may be divided into an odd number of third encoding units (820c, 820d, 820e). The order in which a plurality of encoding units included in the first encoding unit (800) are processed may be a predetermined order (e.g., z-scan order (830)), and the image decoding device (100) may determine whether the right second encoding unit (810b) is divided into an odd number of determined third encoding units (820c, 820d, 820e) can be processed according to the predetermined order.

[0153] According to one embodiment of the present disclosure, an image decoding device (100) can determine whether a third encoding unit (820a, 820b, 820c, 820d, 820e) included in a first encoding unit (800) satisfies a condition that the third encoding unit (820a, 820b, 820c, 820d, 820e) can be processed in a predetermined order, and the condition relates to whether at least one of the width and height of the second encoding unit (810a, 810b) is divided in half according to the boundary of the third encoding unit (820a, 820b, 820c, 820d, 820e). For example, the third encoding unit (820a, 820b) determined by dividing the height of the left second encoding unit (810a) in a non-square shape in half may satisfy the condition. Since the boundaries of the third encoding units (820c, 820d, 820e), which are determined by dividing the right second encoding unit (810b) into three encoding units, do not divide the width or height of the right second encoding unit (810b) in half, the third encoding units (820c, 820d, 820e) may be determined not to satisfy the condition. In the case of such non-satisfaction of the condition, the image decoding device (100) determines that there is a disconnection in the scan order, and based on the result of the determination, the right second encoding unit (810b) may be determined to be divided into an odd number of encoding units. According to one embodiment of the present disclosure, when the image decoding device (100) is divided into an odd number of encoding units, it may place a certain restriction on the encoding unit at a certain position among the divided encoding units, and since the details of such restriction or the certain position, etc. have been described in detail through one embodiment of the present disclosure, a detailed explanation will be omitted.

[0154] FIG. 9 illustrates a process in which an image decoding device (100) divides a first encoding unit (900) to determine at least one encoding unit according to one embodiment of the present disclosure.

[0155] According to one embodiment of the present disclosure, an image decoding device (100) may divide a first encoding unit (900) based on division shape mode information obtained through a bitstream acquisition unit (110). A square-shaped first encoding unit (900) may be divided into four square-shaped encoding units or into a plurality of non-square-shaped encoding units. For example, referring to FIG. 9, if the first encoding unit (900) is square and the division shape mode information indicates that it is divided into non-square encoding units, the image decoding device (100) may divide the first encoding unit (900) into a plurality of non-square encoding units. Specifically, when the splitting mode information indicates that the first encoding unit (900) is divided in a horizontal or vertical direction to determine an odd number of encoding units, the image decoding device (100) can divide the square-shaped first encoding unit (900) into an odd number of encoding units, such as a second encoding unit (910a, 910b, 910c) determined by dividing in a vertical direction or a second encoding unit (920a, 920b, 920c) determined by dividing in a horizontal direction.

[0156] According to one embodiment of the present disclosure, an image decoding device (100) can determine whether a second encoding unit (910a, 910b, 910c, 920a, 920b, 920c) included in a first encoding unit (900) satisfies a condition that the second encoding unit (910a, 910b, 910c, 920a, 920b, 920c) can be processed in a predetermined order, and the condition relates to whether at least one of the width and height of the first encoding unit (900) is divided in half according to the boundary of the second encoding unit (910a, 910b, 910c, 920a, 920b, 920c). Referring to FIG. 9, the boundaries of the second encoding units (910a, 910b, 910c), which are determined by dividing the square-shaped first encoding unit (900) in the vertical direction, do not divide the width of the first encoding unit (900) in half, so the first encoding unit (900) may be determined not to satisfy the condition of being processed in a predetermined order. Additionally, the boundaries of the second encoding units (920a, 920b, 920c), which are determined by dividing the square-shaped first encoding unit (900) in the horizontal direction, do not divide the height of the first encoding unit (900) in half, so the first encoding unit (900) may be determined not to satisfy the condition of being processed in a predetermined order. The image decoding device (100) determines that if these conditions are not satisfied, there is a disconnection in the scan order, and based on the result of the determination, the first encoding unit (900) may be divided into an odd number of encoding units. According to one embodiment of the present disclosure, when the image decoding device (100) is divided into an odd number of encoding units, it may place a certain restriction on the encoding unit at a certain position among the divided encoding units. Since the details of such restriction or the certain position, etc., have been described in detail through one embodiment of the present disclosure, a detailed explanation will be omitted.

[0157] According to one embodiment of the present disclosure, the image decoding device (100) can divide the first encoding unit to determine various forms of encoding units.

[0158] Referring to FIG. 9, the image decoding device (100) can divide a square-shaped first encoding unit (900) and a non-square-shaped first encoding unit (930 or 950) into various types of encoding units.

[0159] FIG. 10 illustrates that, according to one embodiment of the present disclosure, when a second encoding unit in a non-square shape determined by dividing a first encoding unit (1000) satisfies a predetermined condition, the shape in which the second encoding unit can be divided is limited.

[0160] According to one embodiment of the present disclosure, an image decoding device (100) may decide to divide a square-shaped first encoding unit (1000) into non-square-shaped second encoding units (1010a, 1010b, 1020a, 1020b) based on division shape mode information obtained through a bitstream acquisition unit (110). The second encoding units (1010a, 1010b, 1020a, 1020b) may be divided independently. Accordingly, the image decoding device (100) may decide to divide into a plurality of encoding units or not divide based on division shape mode information related to each of the second encoding units (1010a, 1010b, 1020a, 1020b). According to one embodiment of the present disclosure, the image decoding device (100) may determine a third encoding unit (1012a, 1012b) by dividing a left second encoding unit (1010a), which is a non-square shape determined by dividing a first encoding unit (1000) in a vertical direction, in a horizontal direction. However, when the image decoding device (100) divides the left second encoding unit (1010a) in a horizontal direction, the right second encoding unit (1010b) may be restricted so that it cannot be divided in the same horizontal direction as the left second encoding unit (1010a) was divided. If the right second encoding unit (1010b) is divided in the same direction to determine the third encoding unit (1014a, 1014b), the left second encoding unit (1010a) and the right second encoding unit (1010b) may be divided independently in the horizontal direction to determine the third encoding unit (1012a, 1012b, 1014a, 1014b). However, this is the same result as the image decoding device (100) dividing the first encoding unit (1000) into four square-shaped second encoding units (1030a, 1030b, 1030c, 1030d) based on the division shape mode information, and this may be inefficient in terms of image decoding.

[0161] According to one embodiment of the present disclosure, the image decoding device (100) may determine a third encoding unit (1022a, 1022b, 1024a, 1024b) by dividing a first encoding unit (1000) in a horizontal direction and a second encoding unit (1020a or 1020b) in a non-square shape in a vertical direction. However, if the image decoding device (100) divides one of the second encoding units (e.g., the upper second encoding unit (1020a)) in a vertical direction, it may restrict the other second encoding unit (e.g., the lower encoding unit (1020b)) from being divided in the same vertical direction as the upper second encoding unit (1020a) for the reasons described above.

[0162] FIG. 11 illustrates the process of a video decoder (100) dividing square-shaped encoding units when the divided shape mode information cannot be divided into four square-shaped encoding units according to one embodiment of the present disclosure.

[0163] According to one embodiment of the present disclosure, an image decoding device (100) can determine a second encoding unit (1110a, 1110b, 1120a, 1120b, etc.) by dividing a first encoding unit (1100) based on division form mode information. The division form mode information may include information on various forms in which the encoding unit can be divided, but the information on various forms may not include information for dividing into four square-shaped encoding units. According to such division form mode information, the image decoding device (100) cannot divide the square-shaped first encoding unit (1100) into four square-shaped second encoding units (1130a, 1130b, 1130c, 1130d). Based on the segmented form mode information, the image decoding device (100) can determine a non-square second encoding unit (1110a, 1110b, 1120a, 1120b, etc.).

[0164] According to one embodiment of the present disclosure, the image decoding device (100) can independently divide each of the second encoding units (1110a, 1110b, 1120a, 1120b, etc.) in a non-square shape. Each of the second encoding units (1110a, 1110b, 1120a, 1120b, etc.) can be divided in a predetermined order through a recursive method, and this may be a division method corresponding to the method in which the first encoding unit (1100) is divided based on division shape mode information.

[0165] For example, the image decoding device (100) can determine a square-shaped third encoding unit (1112a, 1112b) by dividing the left second encoding unit (1110a) in a horizontal direction, and can determine a square-shaped third encoding unit (1114a, 1114b) by dividing the right second encoding unit (1110b) in a horizontal direction. Furthermore, the image decoding device (100) can determine a square-shaped third encoding unit (1116a, 1116b, 1116c, 1116d) by dividing both the left second encoding unit (1110a) and the right second encoding unit (1110b) in a horizontal direction. In this case, the encoding unit can be determined in the same form as the first encoding unit (1100) being divided into four square-shaped second encoding units (1130a, 1130b, 1130c, 1130d).

[0166] As another example, the video decoding device (100) may determine a square-shaped third encoding unit (1122a, 1122b) by dividing the upper second encoding unit (1120a) in a vertical direction, and determine a square-shaped third encoding unit (1124a, 1124b) by dividing the lower second encoding unit (1120b) in a vertical direction. Furthermore, the video decoding device (100) may determine a square-shaped third encoding unit (1126a, 1126b, 1126a, 1126b) by dividing both the upper second encoding unit (1120a) and the lower second encoding unit (1120b) in a vertical direction. In this case, the encoding unit can be determined in the same form as the first encoding unit (1100) being divided into four square-shaped second encoding units (1130a, 1130b, 1130c, 1130d).

[0167] FIG. 12 illustrates that, according to one embodiment of the present disclosure, the processing order between a plurality of encoding units may vary depending on the division process of the encoding unit.

[0168] According to one embodiment of the present disclosure, an image decoding device (100) may divide a first encoding unit (1200) based on division shape mode information. When the block shape is square and the division shape mode information indicates that the first encoding unit (1200) is divided in at least one of a horizontal direction and a vertical direction, the image decoding device (100) may divide the first encoding unit (1200) to determine a second encoding unit (e.g., 1210a, 1210b, 1220a, 1220b, etc.). Referring to FIG. 12, the second encoding units (1210a, 1210b, 1220a, 1220b) in a non-square shape determined by dividing the first encoding unit (1200) only in a horizontal direction or a vertical direction may be divided independently based on the division shape mode information for each. For example, the video decoding device (100) can determine the third encoding unit (1216a, 1216b, 1216c, 1216d) by dividing the second encoding unit (1210a, 1210b) generated by dividing the first encoding unit (1200) in the vertical direction, and can determine the third encoding unit (1226a, 1226b, 1226c, 1226d) by dividing the second encoding unit (1220a, 1220b) generated by dividing the first encoding unit (1200) in the horizontal direction, respectively, in the vertical direction. Since the process of dividing these second encoding units (1210a, 1210b, 1220a, 1220b) has been described in detail in relation to FIG. 11, a detailed explanation will be omitted.

[0169] According to one embodiment of the present disclosure, the image decoding device (100) can process encoding units in a predetermined order. Since the characteristics of processing encoding units in a predetermined order have been described in detail in relation to FIG. 7, a detailed description will be omitted. Referring to FIG. 12, the image decoding device (100) can divide a square-shaped first encoding unit (1200) to determine four square-shaped third encoding units (1216a, 1216b, 1216c, 1216d, 1226a, 1226b, 1226c, 1226d). According to one embodiment of the present disclosure, the image decoding device (100) can determine the processing order of the third encoding unit (1216a, 1216b, 1216c, 1216d, 1226a, 1226b, 1226c, 1226d) according to the form in which the first encoding unit (1200) is divided.

[0170] According to one embodiment of the present disclosure, the image decoding device (100) can determine the third encoding unit (1216a, 1216b, 1216c, 1216d) by dividing the second encoding unit (1210a, 1210b) generated by dividing in a vertical direction into the horizontal direction, and the image decoding device (100) can process the third encoding unit (1216a, 1216c) included in the left second encoding unit (1210a) first in a vertical direction, and then process the third encoding unit (1216b, 1216d) included in the right second encoding unit (1210b) in a vertical direction according to the order (1217).

[0171] According to one embodiment of the present disclosure, the image decoding device (100) can determine the third encoding unit (1226a, 1226b, 1226c, 1226d) by dividing the second encoding unit (1220a, 1220b) generated by dividing in a horizontal direction into the vertical direction, and the image decoding device (100) can process the third encoding unit (1226a, 1226b, 1226c, 1226d) according to the order (1227) of first processing the third encoding unit (1226a, 1226b) included in the upper second encoding unit (1220a) in a horizontal direction, and then processing the third encoding unit (1226c, 1226d) included in the lower second encoding unit (1220b) in a horizontal direction.

[0172] Referring to FIG. 12, the second encoding units (1210a, 1210b, 1220a, 1220b) are each divided to determine the square-shaped third encoding units (1216a, 1216b, 1216c, 1216d, 1226a, 1226b, 1226c, 1226d). The second encoding unit (1210a, 1210b) determined by dividing in the vertical direction and the second encoding unit (1220a, 1220b) determined by dividing in the horizontal direction are divided into different forms, but according to the third encoding unit (1216a, 1216b, 1216c, 1216d, 1226a, 1226b, 1226c, 1226d) determined thereafter, the result is that the first encoding unit (1200) is divided into encoding units of the same form. Accordingly, the image decoding device (100) recursively divides the encoding unit through different processes based on the division form mode information, so that even if the encoding units of the same form are determined as a result, the multiple encoding units determined in the same form can be processed in different orders.

[0173] FIG. 13 illustrates a process in which, according to one embodiment of the present disclosure, a encoding unit is recursively divided to determine a plurality of encoding units, and the depth of the encoding unit is determined as the shape and size of the encoding unit change.

[0174] According to one embodiment of the present disclosure, the image decoding device (100) may determine the depth of an encoding unit according to a predetermined standard. For example, the predetermined standard may be the length of the long side of the encoding unit. The image decoding device (100) may determine that if the length of the long side of the current encoding unit is divided by 2n (n>0) times the length of the long side of the encoding unit before division, the depth of the current encoding unit is increased by n compared to the depth of the encoding unit before division. In the following, the encoding unit with increased depth is expressed as an encoding unit of lower depth.

[0175] Referring to FIG. 13, according to one embodiment of the present disclosure, based on block shape information indicating that it is a square shape (for example, the block shape information may indicate '0: SQUARE'), an image decoding device (100) can determine a second encoding unit (1302), a third encoding unit (1304), etc. of a lower depth by dividing a first encoding unit (1300) that is square in shape. If the size of the first encoding unit (1300) that is square in shape is 2Nx2N, the second encoding unit (1302), which is determined by dividing the width and height of the first encoding unit (1300) by half, may have a size of NxN. Furthermore, the third encoding unit (1304), which is determined by dividing the width and height of the second encoding unit (1302) by half, may have a size of N / 2xN / 2. In this case, the width and height of the third encoding unit (1304) correspond to 1 / 4 times the width and height of the first encoding unit (1300). When the depth of the first encoding unit (1300) is D, the depth of the second encoding unit (1302), which is 1 / 2 times the width and height of the first encoding unit (1300), may be D+1, and the depth of the third encoding unit (1304), which is 1 / 4 times the width and height of the first encoding unit (1300), may be D+2.

[0176] According to one embodiment of the present disclosure, based on block shape information representing a non-square shape (for example, the block shape information may represent '1: NS_VER' indicating that the height is longer than the width, or '2: NS_HOR' indicating that the width is longer than the height), an image decoding device (100) may divide a first encoding unit (1310 or 1320) that is a non-square shape to determine a second encoding unit (1312 or 1322), a third encoding unit (1314 or 1324), etc. of a lower depth.

[0177] The image decoding device (100) can determine a second encoding unit (e.g., 1302, 1312, 1322, etc.) by dividing at least one of the width and height of a first encoding unit (1310) of size Nx2N. That is, the image decoding device (100) can determine a second encoding unit (1302) of size NxN or a second encoding unit (1322) of size NxN / 2 by dividing the first encoding unit (1310) in a horizontal direction, and can also determine a second encoding unit (1312) of size N / 2xN by dividing it in a horizontal direction and a vertical direction.

[0178] According to one embodiment of the present disclosure, the image decoding device (100) may determine a second encoding unit (e.g., 1302, 1312, 1322, etc.) by dividing at least one of the width and height of a first encoding unit (1320) of size 2NxN. That is, the image decoding device (100) may determine a second encoding unit (1302) of size NxN or a second encoding unit (1312) of size N / 2xN by dividing the first encoding unit (1320) in the vertical direction, and may determine a second encoding unit (1322) of size NxN / 2 by dividing it in the horizontal and vertical directions.

[0179] According to one embodiment of the present disclosure, the image decoding device (100) may determine a third encoding unit (e.g., 1304, 1314, 1324, etc.) by dividing at least one of the width and height of a second encoding unit (1302) of size NxN. That is, the image decoding device (100) may determine a third encoding unit (1304) of size N / 2xN / 2 by dividing the second encoding unit (1302) in a vertical direction and a horizontal direction, or determine a third encoding unit (1314) of size N / 4xN / 2, or determine a third encoding unit (1324) of size N / 2xN / 4.

[0180] According to one embodiment of the present disclosure, the image decoding device (100) may determine a third encoding unit (e.g., 1304, 1314, 1324, etc.) by dividing at least one of the width and height of a second encoding unit (1312) of size N / 2xN. That is, the image decoding device (100) may determine a third encoding unit (1304) of size N / 2xN / 2 or a third encoding unit (1324) of size N / 2xN / 4 by dividing the second encoding unit (1312) in a horizontal direction, or determine a third encoding unit (1314) of size N / 4xN / 2 by dividing it in a vertical direction and a horizontal direction.

[0181] According to one embodiment of the present disclosure, the image decoding device (100) may determine a third encoding unit (e.g., 1304, 1314, 1324, etc.) by dividing at least one of the width and height of a second encoding unit (1322) of size NxN / 2. That is, the image decoding device (100) may determine a third encoding unit (1304) of size N / 2xN / 2 or a third encoding unit (1314) of size N / 4xN / 2 by dividing the second encoding unit (1322) in the vertical direction, or determine a third encoding unit (1324) of size N / 2xN / 4 by dividing it in the vertical and horizontal directions.

[0182] According to one embodiment of the present disclosure, an image decoding device (100) may divide square-shaped encoding units (e.g., 1300, 1302, 1304) in a horizontal or vertical direction. For example, a first encoding unit (1300) of size 2Nx2N may be divided in a vertical direction to determine a first encoding unit (1310) of size Nx2N, or divided in a horizontal direction to determine a first encoding unit (1320) of size 2NxN. According to one embodiment of the present disclosure, when the depth is determined based on the length of the longest side of the encoding unit, the depth of the encoding unit determined by dividing the first encoding unit (1300) of size 2Nx2N in a horizontal or vertical direction may be the same as the depth of the first encoding unit (1300).

[0183] According to one embodiment of the present disclosure, the width and height of the third encoding unit (1314 or 1324) may correspond to 1 / 4 times the width and height of the first encoding unit (1310 or 1320). If the depth of the first encoding unit (1310 or 1320) is D, the depth of the second encoding unit (1312 or 1322), which is 1 / 2 times the width and height of the first encoding unit (1310 or 1320), may be D+1, and the depth of the third encoding unit (1314 or 1324), which is 1 / 4 times the width and height of the first encoding unit (1310 or 1320), may be D+2.

[0184] FIG. 14 illustrates a depth and part index (hereinafter PID) for distinguishing between coding units that can be determined according to the shape and size of the coding units according to one embodiment of the present disclosure.

[0185] According to one embodiment of the present disclosure, an image decoding device (100) can determine various shapes of second encoding units by dividing a square-shaped first encoding unit (1400). Referring to FIG. 14, the image decoding device (100) can determine second encoding units (1402a, 1402b, 1404a, 1404b, 1406a, 1406b, 1406c, 1406d) by dividing the first encoding unit (1400) in at least one of a vertical direction and a horizontal direction according to the division shape mode information. That is, the image decoding device (100) can determine the second encoding unit (1402a, 1402b, 1404a, 1404b, 1406a, 1406b, 1406c, 1406d) based on the segmented form mode information for the first encoding unit (1400).

[0186] According to one embodiment of the present disclosure, the depth of the second encoding unit (1402a, 1402b, 1404a, 1404b, 1406a, 1406b, 1406c, 1406d), which is determined according to the segmented shape mode information for the first encoding unit (1400) in a square shape, can be determined based on the length of the longer side. For example, since the length of one side of the first encoding unit (1400) in a square shape and the length of the longer side of the second encoding unit (1402a, 1402b, 1404a, 1404b) in a non-square shape are the same, the depth of the first encoding unit (1400) and the second encoding unit (1402a, 1402b, 1404a, 1404b) in a non-square shape can be considered to be the same as D. In contrast, when the video decoding device (100) divides the first encoding unit (1400) into four square-shaped second encoding units (1406a, 1406b, 1406c, 1406d) based on the division shape mode information, since the length of one side of the square-shaped second encoding units (1406a, 1406b, 1406c, 1406d) is half the length of one side of the first encoding unit (1400), the depth of the second encoding units (1406a, 1406b, 1406c, 1406d) may be a depth of D+1, which is one depth lower than the depth D of the first encoding unit (1400).

[0187] According to one embodiment of the present disclosure, an image decoding device (100) may divide a first encoding unit (1410), in which the height is longer than the width, into a plurality of second encoding units (1412a, 1412b, 1414a, 1414b, 1414c) by dividing it in a horizontal direction according to the division shape mode information. According to one embodiment of the present disclosure, an image decoding device (100) may divide a first encoding unit (1420), in which the width is longer than the height, into a plurality of second encoding units (1422a, 1422b, 1424a, 1424b, 1424c) by dividing it in a vertical direction according to the division shape mode information.

[0188] According to one embodiment of the present disclosure, a second encoding unit (1412a, 1412b, 1414a, 1414b, 1414c, 1422a, 1422b, 1424a, 1424b, 1424c) determined according to the segmented shape mode information for a first encoding unit (1410 or 1420) of a non-square shape may have its depth determined based on the length of the longer side. For example, since the length of one side of the square-shaped second encoding unit (1412a, 1412b) is half the length of one side of the non-square-shaped first encoding unit (1410) in which the height is longer than the width, the depth of the square-shaped second encoding unit (1412a, 1412b) is D+1, which is one depth lower than the depth D of the non-square-shaped first encoding unit (1410).

[0189] Furthermore, the image decoding device (100) may divide a first encoding unit (1410) in a non-square shape into an odd number of second encoding units (1414a, 1414b, 1414c) based on the division shape mode information. The odd number of second encoding units (1414a, 1414b, 1414c) may include a second encoding unit (1414a, 1414c) in a non-square shape and a second encoding unit (1414b) in a square shape. In this case, since the length of the longer side of the non-square second encoding unit (1414a, 1414c) and the length of one side of the square second encoding unit (1414b) are half the length of one side of the first encoding unit (1410), the depth of the second encoding unit (1414a, 1414b, 1414c) may be a depth of D+1, which is one depth lower than the depth D of the first encoding unit (1410). The image decoding device (100) may determine the depth of the encoding units associated with the non-square first encoding unit (1420), in a manner corresponding to the above method of determining the depth of the encoding units associated with the first encoding unit (1410).

[0190] According to one embodiment of the present disclosure, an image decoding device (100) may determine an index (PID) for distinguishing divided encoding units, and when the odd number of divided encoding units are not of the same size, the index may be determined based on the size ratio between the encoding units. Referring to FIG. 14, among the odd number of divided encoding units (1414a, 1414b, 1414c), the encoding unit (1414b) located in the middle may have the same width as the other encoding units (1414a, 1414c) but may have a height twice that of the other encoding units (1414a, 1414c). That is, in this case, the encoding unit (1414b) located in the middle may include two of the other encoding units (1414a, 1414c). Accordingly, if the index (PID) of the encoding unit (1414b) located in the middle according to the scan order is 1, the encoding unit (1414c) located in the next order may have an index of 3, which is an increase of 2. That is, there may be a discontinuity in the value of the index. According to one embodiment of the present disclosure, the image decoding device (100) may determine whether the encoding units divided into an odd number are not of the same size based on whether there is a discontinuity in the index for distinguishing between these divided encoding units.

[0191] According to one embodiment of the present disclosure, an image decoding device (100) may determine whether a plurality of encoding units determined by dividing from a current encoding unit are divided into a specific division form based on the value of an index for distinguishing the plurality of encoding units. Referring to FIG. 14, the image decoding device (100) may divide a first encoding unit (1410) in the shape of a rectangle whose height is greater than its width to determine an even number of encoding units (1412a, 1412b) or an odd number of encoding units (1414a, 1414b, 1414c). The image decoding device (100) may use an index (PID) representing each encoding unit to distinguish each of the plurality of encoding units. According to one embodiment of the present disclosure, the PID may be obtained from a sample at a predetermined position of each encoding unit (e.g., the upper left sample).

[0192] According to one embodiment of the present disclosure, an image decoding device (100) may determine a coding unit at a predetermined position among the coding units determined by division using an index for distinguishing coding units. According to one embodiment of the present disclosure, if the division shape mode information for a first coding unit (1410) in the form of a rectangle whose height is longer than its width indicates that it is divided into three coding units, the image decoding device (100) may divide the first coding unit (1410) into three coding units (1414a, 1414b, 1414c). The image decoding device (100) may assign an index to each of the three coding units (1414a, 1414b, 1414c). The image decoding device (100) may compare the indices for each coding unit to determine the middle coding unit among the odd number of divided coding units. The image decoding device (100) may determine a encoding unit (1414b) having an index corresponding to the middle value among the indices based on the indices of the encoding units, as the encoding unit at the middle position among the encoding units determined by dividing the first encoding unit (1410). According to one embodiment of the present disclosure, when determining an index for distinguishing the divided encoding units, the image decoding device (100) may determine the index based on the size ratio between the encoding units if the encoding units are not of the same size. Referring to FIG. 14, the encoding unit (1414b) generated by dividing the first encoding unit (1410) may have the same width as the other encoding units (1414a, 1414c) but may be twice the height of the encoding units (1414a, 1414c) which have different heights. In this case, if the index (PID) of the encoding unit (1414b) located in the middle is 1, the encoding unit (1414c) located in the next order may have an index of 3, which is increased by 2.In cases where the index increases uniformly and then the rate of increase changes, such as in this case, the image decoding device (100) may determine that the current encoding unit is divided into a plurality of encoding units, including encoding units having different sizes from other encoding units. In the case where the division type mode information indicates that the current encoding unit is divided into an odd number of encoding units according to one embodiment of the present disclosure, the image decoding device (100) may divide the current encoding unit in such a way that the encoding unit at a predetermined position among the odd number of encoding units (e.g., the middle encoding unit) has a different size from other encoding units. In this case, the image decoding device (100) may determine the middle encoding unit having a different size by using an index (PID) for the encoding unit. However, the aforementioned index, the size or position of the encoding unit at a predetermined position to be determined, is specific for the purpose of explaining one embodiment and should not be interpreted as being limited thereto, and should be interpreted as allowing various indices, positions, and sizes of encoding units to be used.

[0193] According to one embodiment of the present disclosure, an image decoding device (100) may use a predetermined data unit in which recursive division of the encoding unit begins.

[0194] FIG. 15 illustrates that a plurality of encoding units are determined according to a plurality of predetermined data units included in a picture according to one embodiment of the present disclosure.

[0195] According to one embodiment of the present disclosure, a predetermined data unit may be defined as a data unit in which an encoding unit begins to recursively divide using partitioning mode information. That is, it may correspond to the highest depth encoding unit used in the process of determining a plurality of encoding units that divide the current picture. For convenience of explanation, such a predetermined data unit will be referred to as a reference data unit below.

[0196] According to one embodiment of the present disclosure, a reference data unit may have a predetermined size and shape. According to one embodiment of the present disclosure, the reference data unit may include MxN samples. Here, M and N may be the same and may be integers expressed as powers of 2. That is, the reference data unit may have a square or non-square shape and may subsequently be divided into an integer number of encoding units.

[0197] According to one embodiment of the present disclosure, the image decoding device (100) can divide the current picture into a plurality of reference data units. According to one embodiment of the present disclosure, the image decoding device (100) can divide the current picture into a plurality of reference data units using division form mode information for each reference data unit. This division process of reference data units may correspond to a division process using a quad-tree structure.

[0198] According to one embodiment of the present disclosure, the image decoding device (100) can predetermine the minimum size that a reference data unit included in the current picture may have. Accordingly, the image decoding device (100) can determine reference data units of various sizes having a size greater than or equal to the minimum size, and can determine at least one encoding unit using segmented form mode information based on the determined reference data unit.

[0199] Referring to FIG. 15, the image decoding device (100) may use a square-shaped reference encoding unit (1500) or a non-square-shaped reference encoding unit (1502). According to one embodiment of the present disclosure, the shape and size of the reference encoding unit may be determined according to various data units (e.g., sequence, picture, slice, slice segment, tile, tile group, maximum encoding unit, etc.) that may include at least one reference encoding unit.

[0200] According to one embodiment of the present disclosure, the bitstream acquisition unit (110) of the image decoding device (100) can acquire at least one of information regarding the shape of the reference encoding unit and information regarding the size of the reference encoding unit from the bitstream for each of the various data units. The process of determining at least one encoding unit included in the square-shaped reference encoding unit (1500) has been described in detail through the process of dividing the current encoding unit (300) of FIG. 3, and the process of determining at least one encoding unit included in the non-square-shaped reference encoding unit (1502) has been described in detail through the process of dividing the current encoding unit (400 or 450) of FIG. 4, so a detailed explanation is omitted.

[0201] According to one embodiment of the present disclosure, an image decoding device (100) may use an index to identify the size and shape of a reference coding unit in order to determine the size and shape of a reference coding unit according to a portion of data units that are predetermined based on a predetermined condition. That is, a bitstream acquisition unit (110) may acquire only an index for identifying the size and shape of a reference coding unit for each slice, slice segment, tile, tile group, maximum coding unit, etc., among the various data units (e.g., sequence, picture, slice, slice segment, tile, tile group, maximum coding unit, etc.) from the bitstream, as a data unit that satisfies a predetermined condition (e.g., a data unit having a size smaller than or equal to a slice). By using the index, the image decoding device (100) may determine the size and shape of a reference data unit for each data unit that satisfies the predetermined condition. When information regarding the form of the reference encoding unit and information regarding the size of the reference encoding unit are obtained from the bitstream for each data unit of a relatively small size and used, the utilization efficiency of the bitstream may be poor; therefore, instead of directly obtaining information regarding the form of the reference encoding unit and information regarding the size of the reference encoding unit, only the index may be obtained and used. In this case, at least one of the size and form of the reference encoding unit corresponding to the index representing the size and form of the reference encoding unit may be predetermined. That is, the image decoding device (100) can determine at least one of the size and form of the reference encoding unit included in the data unit that serves as the basis for obtaining the index by selecting at least one of the predetermined size and form of the reference encoding unit according to the index.

[0202] According to one embodiment of the present disclosure, the image decoding device (100) may utilize at least one reference encoding unit included in one maximum encoding unit (1510). That is, the maximum encoding unit (1510) that divides the image may include at least one reference encoding unit, and the encoding unit may be determined through a recursive division process of each reference encoding unit. According to one embodiment of the present disclosure, at least one of the width and height of the maximum encoding unit (1510) may correspond to an integer multiple of at least one of the width and height of the reference encoding unit. According to one embodiment of the present disclosure, the size of the reference encoding unit may be the size obtained by dividing the maximum encoding unit (1510) n times according to a quad tree structure. That is, the image decoding device (100) may determine the reference encoding unit by dividing the maximum encoding unit (1510) n times according to a quad tree structure, and according to one embodiment of the present disclosure, the reference encoding unit may be divided based on at least one of block shape information and division shape mode information.

[0203] According to one embodiment of the present disclosure, an image decoding device (100) may obtain and use block form information indicating the form of a current encoding unit or division form mode information indicating a method of dividing a current encoding unit from a bitstream. The division form mode information may be included in a bitstream associated with various data units. For example, the image decoding device (100) may use the division form mode information included in a sequence parameter set, a picture parameter set, a video parameter set, a slice header, a slice segment header, a tile header, and a tile group header. Furthermore, the image decoding device (100) may obtain and use syntax elements corresponding to the block form information or division form mode information from the bitstream for each maximum encoding unit and reference encoding unit.

[0204] A method for determining a division rule according to one embodiment of the present disclosure will be described in detail below.

[0205] The image decoding device (100) can determine the segmentation rule of the image. The segmentation rule may be predetermined between the image decoding device (100) and the image encoding device (200). The image decoding device (100) can determine the segmentation rule of the image based on information obtained from a bitstream. The image decoding device (100) can determine the segmentation rule based on information obtained from at least one of a sequence parameter set, a picture parameter set, a video parameter set, a slice header, a slice segment header, a tile header, and a tile group header. The image decoding device (100) can determine the segmentation rule differently depending on the frame, slice, tile, temporal layer, maximum encoding unit, or encoding unit.

[0206] The image decoding device (100) can determine a partitioning rule based on the block shape of the encoding unit. The block shape may include the size, shape, ratio of width and height, and orientation of the encoding unit. The image encoding device (200) and the image decoding device (100) may predetermine to determine a partitioning rule based on the block shape of the encoding unit. However, they are not limited thereto. The image decoding device (100) can determine a partitioning rule based on information obtained from a bitstream received from the image encoding device (200).

[0207] The shape of the encoding unit may include square and non-square. If the width and height of the encoding unit are the same, the image decoder (100) may determine the shape of the encoding unit as square. Additionally, if the width and height of the encoding unit are not the same, the image decoder (100) may determine the shape of the encoding unit as non-square.

[0208] The size of the encoding unit may include various sizes such as 4x4, 8x4, 4x8, 8x8, 16x4, 16x8, ..., 256x256. The size of the encoding unit may be classified according to the length of the long side, the length of the short side, or the width of the encoding unit. The image decoding device (100) may apply the same partitioning rule to encoding units classified into the same group. For example, the image decoding device (100) may classify encoding units having the same long side length into the same size. Additionally, the image decoding device (100) may apply the same partitioning rule to encoding units having the same long side length.

[0209] The ratio of width to height of a encoding unit may include 1:2, 2:1, 1:4, 4:1, 1:8, 8:1, 1:16, 16:1, 32:1, or 1:32, etc. Additionally, the direction of the encoding unit may include a horizontal direction and a vertical direction. The horizontal direction may indicate a case where the width of the encoding unit is longer than the height. The vertical direction may indicate a case where the width of the encoding unit is shorter than the height.

[0210] The image decoder (100) can adaptively determine a splitting rule based on the size of the encoding unit. The image decoder (100) can determine different acceptable splitting mode types based on the size of the encoding unit. For example, the image decoder (100) can determine whether splitting is allowed based on the size of the encoding unit. The image decoder (100) can determine the splitting direction according to the size of the encoding unit. The image decoder (100) can determine an acceptable splitting type according to the size of the encoding unit.

[0211] Determining the division rule based on the size of the encoding unit may be a division rule predetermined between the image encoding device (200) and the image decoding device (100). Additionally, the image decoding device (100) may determine the division rule based on information obtained from the bitstream.

[0212] The image decoder (100) can adaptively determine a partitioning rule based on the position of the encoding unit. The image decoder (100) can adaptively determine a partitioning rule based on the position occupied by the encoding unit in the image.

[0213] Additionally, the video decoding device (100) can determine a splitting rule so that encoding units generated by different splitting paths do not have the same block shape. However, this is not limited thereto, and encoding units generated by different splitting paths may have the same block shape. Encoding units generated by different splitting paths may have different decoding processing orders. Since the decoding processing order has been explained together with FIG. 12, a detailed explanation is omitted.

[0214] FIG. 16 illustrates encoding units that can be determined for each picture when the combination of forms in which encoding units can be divided according to one embodiment of the present disclosure is different for each picture.

[0215] Referring to FIG. 16, the image decoding device (100) may determine different combinations of division forms in which the encoding unit can be divided for each picture. For example, the image decoding device (100) may decode an image using a picture (1600) that can be divided into four encoding units, a picture (1610) that can be divided into two or four encoding units, and a picture (1620) that can be divided into two, three, or four encoding units among at least one picture included in the image. To divide the picture (1600) into multiple encoding units, the image decoding device (100) may use only division form information indicating that it is divided into four square encoding units. To divide the picture (1610), the image decoding device (100) may use only division form information indicating that it is divided into two or four encoding units. The image decoding device (100) may use only division form information indicating that the picture (1620) is divided into two, three, or four encoding units in order to divide the picture. Since the combination of division forms described above is merely an example for explaining the operation of the image decoding device (100), the combination of division forms described above should not be interpreted as being limited to the above example, but should be interpreted as allowing various combinations of division forms to be used for each predetermined data unit.

[0216] According to one embodiment of the present disclosure, a bitstream acquisition unit (110) of an image decoding device (100) may acquire a bitstream including an index representing a combination of division form information for each predetermined data unit (e.g., sequence, picture, slice, slice segment, tile, or tile group, etc.). For example, the bitstream acquisition unit (110) may acquire an index representing a combination of division form information from a sequence parameter set, a picture parameter set, a slice header, a tile header, or a tile group header. The image decoding device (100) may determine a combination of division forms in which the encoding unit can be divided for each predetermined data unit using the acquired index, and accordingly, different combinations of division forms may be used for each predetermined data unit.

[0217] FIG. 17 illustrates various forms of encoding units that can be determined based on partitioned form mode information that can be expressed as binary code according to one embodiment of the present disclosure.

[0218] According to one embodiment of the present disclosure, the image decoding device (100) can divide the encoding unit into various forms using block form information and division form mode information obtained through the bitstream acquisition unit (110). The forms of the encoding unit that can be divided may correspond to various forms including the forms described through the embodiments described above.

[0219] Referring to FIG. 17, the image decoding device (100) can divide a square-shaped encoding unit into at least one of a horizontal direction and a vertical direction based on the divided shape mode information, and can divide a non-square-shaped encoding unit into a horizontal direction or a vertical direction.

[0220] According to one embodiment of the present disclosure, if an image decoding device (100) can divide a square-shaped encoding unit into four square encoding units by dividing it in the horizontal and vertical directions, there may be four types of division forms that can be represented by the division form mode information for the square encoding unit. According to one embodiment of the present disclosure, the division form mode information may be expressed as a two-digit binary code, and a binary code may be assigned to each division form. For example, if the encoding unit is not divided, the division form mode information may be expressed as (00)b; if the encoding unit is divided in the horizontal and vertical directions, the division form mode information may be expressed as (01)b; if the encoding unit is divided in the horizontal direction, the division form mode information may be expressed as (10)b; and if the encoding unit is divided in the vertical direction, the division form mode information may be expressed as (11)b.

[0221] According to one embodiment of the present disclosure, when the image decoding device (100) divides a non-square encoding unit in a horizontal or vertical direction, the type of division form that can be represented by the division form mode information may be determined by how many encoding units are divided. Referring to FIG. 17, the image decoding device (100) may divide a non-square encoding unit into up to three units according to one embodiment of the present disclosure. The image decoding device (100) may divide the encoding unit into two encoding units, in which case the division form mode information may be expressed as (10)b. The image decoding device (100) may divide the encoding unit into three encoding units, in which case the division form mode information may be expressed as (11)b. The image decoding device (100) may decide not to divide the encoding unit, in which case the division form mode information may be expressed as (0)b. That is, the video decoding device (100) can use variable length coding (VLC) rather than fixed length coding (FLC) to use binary code representing segmented form mode information.

[0222] Referring to FIG. 17 according to one embodiment of the present disclosure, the binary code of the division form mode information indicating that the encoding unit is not divided can be represented as (0)b. If the binary code of the division form mode information indicating that the encoding unit is not divided is set to (00)b, then all 2 bits of the binary code of the division form mode information must be used even though there is no division form mode information set to (01)b. However, as illustrated in FIG. 17, if three division forms for a non-square encoding unit are used, the image decoding device (100) can determine that the encoding unit is not divided even if a 1-bit binary code (0)b is used as the division form mode information, thus allowing for efficient use of the bitstream. However, the division form of the non-square encoding unit indicated by the division form mode information should not be interpreted as being limited only to the three forms illustrated in FIG. 17, but should be interpreted as various forms including the embodiments described above.

[0223] FIG. 18 illustrates another form of encoding unit that can be determined based on partitioned form mode information that can be represented as a binary code according to one embodiment of the present disclosure.

[0224] Referring to FIG. 18, the image decoding device (100) can divide a square-shaped encoding unit in a horizontal or vertical direction based on the division shape mode information, and can divide a non-square-shaped encoding unit in a horizontal or vertical direction. That is, the division shape mode information can indicate that a square-shaped encoding unit is divided in one direction. In this case, the binary code of the division shape mode information indicating that the square-shaped encoding unit is not divided can be represented as (0)b. If the binary code of the division shape mode information indicating that the encoding unit is not divided is set to (00)b, then all 2 bits of the binary code of the division shape mode information must be used even though there is no division shape mode information set to (01)b. However, as illustrated in FIG. 18, if three division forms for a square-shaped encoding unit are used, the image decoding device (100) can determine that the encoding unit is not divided even if a 1-bit binary code (0)b is used as division form mode information, thus allowing the bitstream to be used efficiently. However, the division form of the square-shaped encoding unit indicated by the division form mode information should not be interpreted as being limited only to the three forms illustrated in FIG. 18, but should be interpreted as various forms including the embodiments described above.

[0225] According to one embodiment of the present disclosure, block form information or partition form mode information can be represented using binary code, and such information can be immediately generated as a bitstream. Additionally, block form information or partition form mode information that can be represented by binary code may not be immediately generated as a bitstream but may be used as binary code input in CABAC (context adaptive binary arithmetic coding).

[0226] According to one embodiment of the present disclosure, the image decoding device (100) describes a process of obtaining syntax for block form information or partition form mode information through CABAC. A bitstream including a binary code for the syntax can be obtained through a bitstream acquisition unit (110). The image decoding device (100) can detect a syntax element representing block form information or partition form mode information by debinding a bin string included in the obtained bitstream. According to one embodiment of the present disclosure, the image decoding device (100) obtains a set of binary bin strings corresponding to the syntax element to be decoded, and can decode each bin using probability information, and the image decoding device (100) can repeat this process until the bin string composed of these decoded bins becomes equal to one of the previously obtained bin strings. The image decoding device (100) can determine syntax elements by performing inverse binarization of an empty string.

[0227] According to one embodiment of the present disclosure, an image decoding device (100) can determine the syntax for a bin string by performing a decoding process of adaptive binary arithmetic coding, and the image decoding device (100) can update a probability model for bins obtained through a bitstream acquisition unit (110). Referring to FIG. 17, the bitstream acquisition unit (110) of the image decoding device (100) can acquire a bitstream representing a binary code representing partitioned mode information according to one embodiment of the present disclosure. Using the acquired binary code having a size of 1 bit or 2 bits, the image decoding device (100) can determine the syntax for the partitioned mode information. To determine the syntax for the partitioned mode information, the image decoding device (100) can update the probability for each bit of the 2-bit binary code. That is, the image decoding device (100) can update the probability of having a value of 0 or 1 when decoding the next bin, depending on whether the value of the first bin of the 2-bit binary code is 0 or 1.

[0228] According to one embodiment of the present disclosure, the image decoding device (100) can update the probability for the bins used in the process of decoding the bins of the empty string for the syntax during the process of determining the syntax, and the image decoding device (100) can determine that the probability is not updated for certain bits of the empty string and has the same probability.

[0229] Referring to FIG. 17, in the process of determining syntax using an empty string representing segmentation mode information for a non-square type encoding unit, the image decoder (100) can determine syntax for the segmentation mode information using one bin having a value of 0 when the non-square type encoding unit is not segmented. That is, when block type information indicates that the current encoding unit is a non-square type, the first bin of the empty string for the segmentation mode information may be 0 when the non-square type encoding unit is not segmented, and 1 when it is segmented into two or three encoding units. Accordingly, the probability that the first bin of the empty string for the segmentation mode information for a non-square type encoding unit is 0 may be 1 / 3, and the probability that it is 1 may be 2 / 3. As described above, since the image decoding device (100) can represent only a 1-bit empty string having a value of 0 as the segmentation form mode information indicating that a non-square type encoding unit is not segmented, the image decoding device (100) can determine the syntax for the segmentation form mode information by determining whether the second bin is 0 or 1 only when the first bin of the segmentation form mode information is 1. According to one embodiment of the present disclosure, the image decoding device (100) can decode the bin by considering that when the first bin of the segmentation form mode information is 1, the probability that the second bin is 0 or 1 is equal to the probability.

[0230] According to one embodiment of the present disclosure, the image decoding device (100) may use various probabilities for each bin in the process of determining the bins of the bin string for the segmented form mode information. According to one embodiment of the present disclosure, the image decoding device (100) may determine the probability of the bins for the segmented form mode information differently depending on the direction of the non-square block. According to one embodiment of the present disclosure, the image decoding device (100) may determine the probability of the bins for the segmented form mode information differently depending on the width or the length of the long side of the current encoding unit. According to one embodiment of the present disclosure, the image decoding device (100) may determine the probability of the bins for the segmented form mode information differently depending on at least one of the shape of the current encoding unit and the length of the long side.

[0231] According to one embodiment of the present disclosure, the image decoding device (100) may determine that the probability of bins for segmented form mode information is the same for encoding units of a predetermined size or larger. For example, based on the length of the long side of the encoding unit, the probability of bins for segmented form mode information is determined to be the same for encoding units of a size of 64 samples or larger.

[0232] According to one embodiment of the present disclosure, the image decoding device (100) may determine the initial probability for the bins constituting the empty string of the segmented form mode information based on the slice type (e.g., I slice, P slice, or B slice).

[0233] Figure 19 is a block diagram of an image encoding and decoding system.

[0234] The encoding unit (1910) of the video encoding and decoding system (1900) transmits an encoded bitstream of the video, and the decoding unit (1950) receives the bitstream and decodes it to output a restored video. Here, the encoding unit (1910) may have a configuration similar to the video encoding device (200) described later, and the decoding unit (1950) may have a configuration similar to the video decoding device (100).

[0235] In the encoding unit (1910), the prediction encoding unit (1915) outputs prediction data through inter-prediction and intra-prediction, and the transformation and quantization unit (1920) outputs quantized transformed coefficients (or quantized coefficients) of residual data between the prediction data and the current input image. For example, the transform coefficients may be generated using a transformation kernel that includes at least one of the Discrete Cosine Transform (DT), Discrete Sine Transform (DST), Hadamard transform, Karhunen-Loeve Transform (KLT), or Wavelet Transform. The residual data may have its information compressed as the transformation is performed. For example, the residual data may be represented using a small number of frequencies as the transformation is performed. In one embodiment, the conversion and quantization unit (1920) may omit the conversion of residual data between the prediction data and the current input image. For example, if the distribution of residual data makes it inefficient to perform the conversion, the conversion process for residual data may be omitted and the quantization process may be performed. In one embodiment, information regarding whether the conversion is omitted (or whether the conversion is performed) may be transmitted to the decoding unit (1950) via a bitstream. The entropy encoding unit (1925) encodes and converts the quantized conversion coefficients and outputs them as a bitstream. The quantized conversion coefficients may be restored to spatial domain data through the inverse quantization and inverse conversion unit (1930). The inverse quantization and inverse conversion unit (1930) may perform inverse quantization on the quantized conversion coefficients and determine residual data by applying a conversion kernel. In one embodiment, the inverse quantization and inverse conversion unit (1930) may not perform inverse conversion.For example, if the transformation for residual data is omitted, the inverse quantization and inverse transformation unit (1930) may not perform the inverse transformation. The inverse quantization and inverse transformation unit (1930) can determine residual data by performing inverse quantization on the quantized transformation coefficients. The data in the restored spatial domain is output as a restored image after passing through the deblocking filtering unit (1935) and the loop filtering unit (1940). The restored image can be used as a reference image for the next input image after passing through the predictive encoding unit (1915).

[0236] The encoded image data among the bitstreams received by the decoding unit (1950) is restored into spatial domain residual data through the entropy decoding unit (1955) and the inverse quantization and inverse transform unit (1960). The predicted data and residual data output from the prediction decoding unit (1975) are combined to form spatial domain image data, and the deblocking filtering unit (1965) and the loop filtering unit (1970) can perform filtering on the spatial domain image data to output a restored image for the current original image. The restored image can be used as a reference image for the next original image by the prediction decoding unit (1975).

[0237] The loop filtering unit (1940) of the encoding unit (1910) performs loop filtering using filter information input according to user input or system settings. The filter information used by the loop filtering unit (1940) is output to the entropy encoding unit (1925) and transmitted to the decoding unit (1950) along with the encoded image data. The loop filtering unit (1970) of the decoding unit (1950) can perform loop filtering based on the filter information input from the decoding unit (1950).

[0238] FIG. 20 is a block diagram showing the configuration of a video decoding device according to one embodiment.

[0239] Referring to FIG. 20, the video decoding device (2000) may include a processor (2010) and a memory (2020).

[0240] In one embodiment of the present disclosure, the processor (2010) may include processing circuits and / or multiple processors. For example, the processor (2010) may include various processing circuits including at least one processor, and at least one of the at least one processor may be configured to perform the various functions described in the present disclosure individually and / or collectively in a distributed manner.

[0241] In one embodiment of the present disclosure, the memory (2020) may include one or more storage media that store at least one instruction. A processor (2010) may control a video decoder (2000) by executing an instruction stored in the memory (2020). For example, the processor (2010) may control the video decoder (2000) to perform an operation by executing the instructions stored in the memory (2020) individually or collectively. In one embodiment of the present disclosure, the operation performed by the video decoder (2000) may be an operation performed by the processor (2010) of the video decoder (2000).

[0242] In one embodiment of the present disclosure, the video decoding device (2000) may correspond to the image decoding device (100) shown in FIG. 1 and / or the decoding unit (1950) shown in FIG. 19.

[0243] The video decoder (2000) can obtain a bitstream generated as a result of encoding by the video encoding device (3430). The bitstream may contain the result of encoding for the current block. The configuration and operation of the video encoding device (3430) will be described later through FIGS. 43 and 44.

[0244] In one embodiment of the present disclosure, a video decoding device (2000) may receive a bitstream from a video decoding device (3400) via a network. In one embodiment of the present disclosure, the video decoding device (2000) may obtain a bitstream from a data storage medium comprising at least one of a magnetic medium such as a hard disk, a floppy disk, and a magnetic tape, an optical recording medium such as a CD-ROM and a DVD, or a magneto-optical medium such as a floptical disk.

[0245] A video decoding device (2000) can obtain syntax elements for decoding an image from a bitstream. Values ​​corresponding to the syntax elements may be included in the bitstream according to the hierarchical structure of the image. In one embodiment of the present disclosure, the video decoding device (2000) can obtain empty strings by entropy decoding bit sequences included in the bitstream, and obtain syntax elements corresponding to the empty strings.

[0246] In one embodiment of the present disclosure, the bitstream may include information regarding the prediction mode of the current block within the current image. The current block may include at least one of a maximum encoding unit, an encoding unit, a transform unit, or a prediction unit divided from the current image to be decoded. In one embodiment of the present disclosure, the prediction mode of the current block may include at least one of an intra mode, an inter mode, a combined mode, a geometric partitioning mode, a block copy mode, or a template matching prediction mode.

[0247] The video decoder (2000) can determine a prediction mode based on information regarding the prediction mode of the current block. The video decoder (2000) can generate a prediction block of the current block by performing a prediction according to the prediction mode.

[0248] A video decoding device (2000) according to one embodiment can determine prediction samples of a coding unit for each of two or more sub-partitions generated by dividing a coding unit. After determining prediction samples for each sub-partition, the video decoding device (2000) can collect the prediction samples of the sub-partitions to determine a prediction block of a coding unit.

[0249] The video decoder (2000) can generate a restored current block using a prediction block. In one embodiment of the present disclosure, the video decoder (2000) can determine the prediction block as the restored current block. In one embodiment of the present disclosure, the video decoder (2000) can generate a restored current block by combining the prediction block with residual data obtained from the bitstream by the video decoder (2000). The restored current block can be used as a reference block for the next block.

[0250] In one embodiment, the video decoder (2000) may obtain residual data from a bitstream. The residual data may include information regarding the difference between the original image (or original sample) and the predicted image (or predicted sample). In one embodiment, the video decoder (2000) may obtain a transformation coefficient of a residual block corresponding to a transformation unit from the bitstream. In one embodiment, the video decoder (2000) may obtain a residual sample of a residual block based on the transformation coefficient of the residual block. For example, the video decoder (2000) may obtain a residual sample of a residual block by performing at least one of inverse quantization or inverse transformation on the transformation coefficient of the residual block. In one embodiment, the video decoder (2000) may determine a residual sample of the encoding unit using at least a portion of a portion of the residual samples of the residual block. If the size of the residual block is larger than the size of the conversion unit, the video decoder (2000) may determine a portion of the residual samples of the residual block as residual samples of the encoding unit. Alternatively, the video decoder (2000) may determine a portion of the residual samples of the residual block, after filtering has been performed, as residual samples of the encoding unit. The video decoder (2000) may restore samples of the encoding unit based on the residual samples of the encoding unit.

[0251] A video decoding device (2000) according to one embodiment may determine whether to separately use an intra prediction mode for each sub-partition for each encoding unit. Additionally, an intra prediction mode may be induced for each sub-partition without obtaining information regarding the intra prediction modes of the sub-partitions from the beast stream. At this time, the method of inducing the intra prediction mode for each sub-partition may be determined individually.

[0252] A video decoding device (2000) according to one embodiment can obtain first information from a bitstream indicating whether an intra prediction mode induced by sub-partitions in the current block is used.

[0253] A video decoding device (2000) according to one embodiment can determine the intra prediction mode of a first sub-partition among the sub-partitions of a current block based on surrounding information of the first sub-partition when an intra prediction mode derived per sub-partition according to first information is used.

[0254] A video decoding device (2000) according to one embodiment can obtain recovery samples of the first sub-partition using prediction samples obtained by performing intra prediction based on an intra prediction mode of the first sub-partition.

[0255] A video decoding device (2000) according to one embodiment can determine the intra prediction mode of a second sub-partition adjacent to a first sub-partition among the sub-partitions of the current block using the prediction result of the first sub-partition and surrounding information of the second sub-partition.

[0256] A video decoding device (2000) according to one embodiment can obtain recovery samples of the second sub-partition using prediction samples obtained by performing intra prediction based on an intra prediction mode of the second sub-partition.

[0257] A video decoding device (2000) according to one embodiment can restore the current block based on the restoration samples of the first sub-partition and the restoration samples of the second sub-partition.

[0258] The video decoder (2000) can perform intra prediction for the current block of the current picture by referring to reference samples included in the current picture or reference picture.

[0259] When the size of the block is large or the texture or features within the block are heterogeneous, the accuracy of the prediction result may decrease when the intra prediction mode is derived only from surrounding reference samples or reference templates surrounding the block. In contrast, the video decoding device (2000) according to one embodiment performs prediction according to an individual intra prediction mode for each sub-partition divided from a block or encoding unit, so that the intra prediction can be performed by considering the unique texture or features of each sub-partition, and thus a more accurate prediction result can be derived.

[0260] Figure 21 illustrates the directionality of the intra prediction mode.

[0261] Referring to FIG. 21, in one embodiment, a plurality of intra prediction modes may include 0 Intra_Planar mode which is non-directional, 1 Intra_DC mode which is non-directional, 2 to 66 Intra_Angular modes which are directional, and -14 to -1 and 67 to 80 Intra_Wide_Angular modes.

[0262] Intra_Planar mode may refer to a mode that determines predicted samples based on the weighted average value according to the distance of the left reference sample, the upper reference sample, the lower-left sample of the current block, and the upper-right sample.

[0263] Intra_DC mode may refer to a mode that determines the average value of reference samples as the prediction sample.

[0264] In Intra_Angular modes, the locations of reference samples for generating predicted samples of samples within the current block can be identified by considering the direction indicated by the Intra_Angular modes. For example, in mode 34, reference samples located at a 45-degree upper-left direction relative to the samples within the current block can be identified.

[0265] Intra_Wide_Angular modes may be used to identify reference samples of samples within a non-square current block. In one embodiment, the video decoder (2000) may determine one of the Intra_Wide_Angular modes as the intra prediction mode of the non-square current block. In one embodiment, the video decoder (2000) may determine one of the Intra_Wide_Angular modes as the improved intra prediction mode of the square current block. For example, the video decoder (2000) may determine one of the Intra_Wide_Angular modes as the second intra prediction mode based on a first intra prediction mode which is one of the Intra_Angular modes.

[0266] A video decoding device (2000) according to one embodiment of the present disclosure can determine an intra prediction mode of the current block to perform intra prediction for the current block.

[0267] In one embodiment, the video decoding device (2000) may obtain information regarding the intra prediction mode of the current block from the bitstream when the prediction mode of the current block is an intra mode. The information regarding the intra prediction mode may include one of information indicating the intra prediction mode or information indicating a range that can be determined as an intra prediction mode. For example, the information regarding the intra prediction mode may include index information indicating the 64th intra prediction mode (e.g., predModeIntra == 64). For example, the information regarding the intra prediction mode may include information indicating a predetermined range (e.g., 2 to 34).

[0268] The intra prediction mode of the current block may be any one of a plurality of intra prediction modes. In one embodiment, the plurality of intra prediction modes may include directional prediction modes and non-directional prediction modes.

[0269] The intra prediction mode illustrated in FIG. 21 is an example, and the number and types of intra prediction modes available in the intra mode of the video decoding device (2000) according to one embodiment can be set in various ways.

[0270] Figure 22 illustrates an intra prediction method according to the ISP (intra sub partition) mode.

[0271] In intra-subpartition mode or ISP mode, a block (or upper block) is divided into subpartitions, and intra-prediction can be performed independently for each of the subpartitions. The upper block may be divided vertically or horizontally. In one embodiment, the upper block may be a maximum encoding unit, encoding unit, conversion unit, or prediction unit divided from a picture.

[0272] In intra-subpartition mode, the intra prediction mode determined for the parent block can be applied to all subpartitions. For example, if the intra prediction mode of the parent block is directional mode 34, the subpartitions can be intra-predicted according to directional mode 34.

[0273] According to one embodiment, information indicating whether an intra-subpartition mode is applied and information indicating the partitioning form of an upper block may be included in the bitstream. The decoder determines whether the intra-subpartition mode is applied to the upper block based on the information obtained from the bitstream, and if the intra-subpartition mode is applied to the upper block, it can determine, based on the information, in what form the upper block should be partitioned.

[0274] Referring to FIG. 22, as the upper block (2200) is divided into four vertical sections, intra prediction can be performed sequentially for each of the four sub-partitions (2210, 2220, 2230, 2240). Intra prediction for the sub-partitions (2210, 2220, 2230, 2240) can be performed using the original reference samples (2260) surrounding the upper block.

[0275] In the case where the decoding order of the leftmost first sub-partition (2210) among the four sub-partitions (2210, 2220, 2230, 2240) is the fastest, intra prediction for the first sub-partition (2210) can be performed using reference samples located in the prediction direction (2250) indicated by the intra prediction mode among the previously restored reference samples (2260).

[0276] The previously restored reference samples (2260) may be samples included in a block that was encoded or decoded before the upper block (2200). The previously restored reference samples (2260) may include samples located to the left of the upper block (2200) and samples located above the upper block (2200).

[0277] When the restoration of the first sub-partition (2210) is completed through intra prediction for the first sub-partition (2210), an intra prediction mode for the second sub-partition (2220) can be performed according to the prediction direction (2250). Since the restoration of the first sub-partition (2210) is completed, for the intra prediction for the second sub-partition (2220), a newly restored reference sample (2270) located on the left side of the second sub-partition (2220) and a reference sample located on the upper side of the second sub-partition (2220) among the previously restored reference samples (2260) may be used. The newly restored reference sample (2270) located on the left side of the second sub-partition (2220) may be included in the first sub-partition (2210).

[0278] When the restoration of the first sub-partition (2220) is completed through intra prediction for the second sub-partition (2210), an intra prediction mode for the third sub-partition (2230) can be performed according to the prediction direction (2250). Since the restoration of the second sub-partition (2220) is completed, for the intra prediction for the third sub-partition (2230), a newly restored reference sample (2270) located on the left side of the third sub-partition (2230) and a reference sample located on the upper side of the third sub-partition (2230) among the previously restored reference samples (2260) may be used. The newly restored reference sample (2270) located on the left side of the third sub-partition (2230) may be included in the second sub-partition (2220).

[0279] Likewise, an intra-prediction mode for the fourth sub-partition (2240) can be performed according to the prediction direction (2250). For the intra-prediction for the fourth sub-partition (2240), a newly restored reference sample (2270) located to the left of the fourth sub-partition (2240) and a reference sample located above the fourth sub-partition (2240) among the previously restored reference samples (2260) may be used. The newly restored reference sample (2270) located to the left of the fourth sub-partition (2240) may be included in the third sub-partition (2230).

[0280] Even when the upper block (2100) is divided in a horizontal direction, the first sub-partition (2215) located at the top among the sub-partitions (2215, 2225, 2235, 2245) can be intra-predicted from previously restored reference samples (2260) adjacent to the upper block (2200). Additionally, among the sub-partitions divided from the upper block (2200), sub-partitions (2225, 2235, 2245) other than the first sub-partition (2215) can be intra-predicted using samples (2270) included in the previously restored sub-partitions as reference samples. In this case, the prediction direction (2250) of the intra-prediction mode applied to the sub-partitions (2215, 2225, 2235, 2245) can all be the same.

[0281] In one embodiment, when the intra-subpartition mode is applied to a very small block, the encoding / decoding efficiency may be poor, so the application of the intra-subpartition mode may be limited based on the size of the upper block (2200). For example, if the size of the upper block (2200) is less than or equal to a predetermined size, the intra-subpartition mode may not be applied to the upper block (2200), and information indicating whether the intra-subpartition mode is applied may be omitted from the bitstream.

[0282] FIG. 22 illustrates an intra-prediction for sub-partitions (2210, 2220, 2230, 2240, 2215, 2225, 2235, 2245) determined by a horizontal or vertical division of an upper block (2200), and in one embodiment, an intra-prediction for a plurality of sub-partitions determined by a quad division of the upper block (2200) may be performed.

[0283] In one embodiment, the minimum size (e.g., minimum width, minimum height, or minimum width) that a sub-partition can have in an intra-sub-partition mode may be preset. In this case, the size of the sub-partitions (2210, 2220, 2230, 2240, 2215, 2225, 2235, 2245) obtained through partitioning of the upper block (2200) may be equal to or greater than the minimum size.

[0284] The video decoding device (2000) can obtain information from the bitstream for determining the intra prediction mode of the current block or the sub-partitions of the current block. Additionally, according to one embodiment of the present disclosure, if the information regarding the intra prediction mode of the current block or the sub-partitions is not included in the bitstream in order to reduce the amount of data of the bitstream, the video decoding device (2000) can directly derive the intra prediction mode of the sub-partitions by referring to the surrounding information of the current block.

[0285] The video decoder (2000) can obtain an intra_subpartitions_mode_flag indicating whether an ISP mode is applied and an intra_subpartitions_split_flag indicating a sub-partitioning method from the bitstream. The IntraSubPartitionsSplitType according to the intra_subpartitions_mode_flag and intra_subpartitions_split_flag is as shown in and operates as follows.

[0286] IntraSubPartitionsSplitTypeName of IntraSubPartitionsSplitType0ISP_NO_SPLIT1ISP_HOR_SPLIT2ISP_VER_SPLIT

[0287] If intra_subpartitions_mode_flag is 0, IntraSubPartitionsSplitType is set to 0; in this case, sub-block splitting is not performed and ISP mode is not applied. If intra_subpartitions_mode_flag is not 0, IntraSubPartitionsSplitType is set to 1 + intra_subpartitions_split_flag, and sub-block splitting may be performed according to the set IntraSubPartitionsSplitType. That is, if IntraSubPartitionsSplitType=1, sub-block splitting may be performed horizontally, and if IntraSubPartitionsSplitType=2, sub-block splitting may be performed vertically. Therefore, intra-prediction can be performed based on sub-blocks split horizontally or vertically.

[0288] Hereinafter, with reference to FIGS. 23 to 26b, methods for a video decoding device (2000) to induce an intra prediction mode by referring to surrounding information of a block are described in detail.

[0289] FIG. 23 is a diagram illustrating a DIMD (Decoder-side Intra Sub Partition) method for inducing an intra prediction mode in a video decoding device according to one embodiment.

[0290] A video decoding device (2000) according to one embodiment can directly derive an intra prediction mode using surrounding information of a block without information about the intra prediction mode according to the DIMD method. According to the DIMD method, the video decoding device (2000) can derive two directional intra prediction modes in a reconstructed area around the current block (2300). The two directional intra prediction modes can be derived from a histogram of gradient (HoG) constructed by collecting gradients at all locations in the reconstructed area. A Sobel filter may be used as a method to obtain the HoG.

[0291] Specifically, according to the DIMD method, a gradient can be calculated using pixels belonging to the surrounding area of ​​the current block (2300). In one embodiment, a template (2310) for generating gradient information can be defined within the restored area surrounding the current block (2300).

[0292] The template (2310) may include the left, upper-left, and upper regions of the current block (2300). As an example, the template (2310) may be defined as an L-shaped region of three pixel lines adjacent to the current block (2300), as shown in FIG. 23. A gradient may be calculated using pixels belonging to the template (2310) among the surrounding regions of the current block (2300). In one embodiment, the number of pixel lines that can be used as the template (2310) may vary.

[0293] In one embodiment, a gradient may be calculated through filtering the pixels belonging to the template (2310). In one embodiment, filtering may be performed on the pixels within a 3x3 pixel area belonging to the template (2310). The pixel area to which filtering is applied may be referred to as a window.

[0294] As an example, a 3x3 Sobel filter may be applied to the filtering kernel (2320) applied to the template (2310). In this disclosure, the case where the filtering kernel (2320) is 3x3 is described primarily, but this is merely an example and is not limited thereto. For example, the filtering kernel (2320) may be defined with a size of 2x2, 4x4, 5x5, etc.

[0295] In one embodiment, a filtering kernel (2320) may be determined around a pixel belonging to the central pixel line within a template (2310) of three pixel lines, and a gradient may be calculated by applying filtering to the filtering kernel (2320).

[0296] The gradient is the horizontal gradient G hor or vertical gradient G ver It may include at least one of the following. Vertical gradient G ver is generated by applying vertical Sobel filtering (2323) to the template (2310), and the horizontal gradient G hor This can be generated by applying horizontal Sobel filtering (2325) to a template (2310). An intra prediction mode can be determined based on the calculated gradient. An angle (or prediction direction) (2340) can be calculated based on the gradient, and the calculated angle (2340) can be mapped to an intra prediction mode having the closest or most similar angle. In other words, an intra prediction mode can be calculated from the gradient. Gradient information can be constructed using the intra prediction mode calculated from the gradient.

[0297] The angle (or prediction direction) (2340) calculated through filtering on the filtering kernel (2320) can be mapped (or transformed) to one of the predefined intra prediction modes. The angle derived from the gradient can be the texture angle or prediction direction of the window (2320). As an example, an intra prediction mode as described above in FIG. 21 can be derived from the gradient. That is, the angle (2340) calculated through filtering using the filtering kernel (2320) can be mapped to one of 65 directional intra prediction modes.

[0298] In one embodiment, the intra prediction mode corresponding to the current filtering kernel (2320) can be derived according to the description of the following mathematical formula 1.

[0299]

[0300] In mathematical equation 1, G hor represents the horizontal gradient of the current filtering kernel (2320), and G ver represents the vertical gradient of the current filtering kernel (2320). According to Equation 1, the angle (2340) of the current filtering kernel (2320) can be calculated using the horizontal gradient and the vertical gradient.

[0301] In one embodiment, an intra-prediction mode corresponding to an angle adjacent to the angle (2340) calculated using Equation 1 can be derived (or mapped) to the intra-prediction mode of the current window (2320).

[0302] In one embodiment, the intensity (or amplitude) of the intra-prediction mode corresponding to the current gradient can be calculated based on the following mathematical formula 2.

[0303]

[0304] Referring to Equation 2, the intensity I of the intra-prediction mode θIt can be calculated as the sum of the absolute values ​​of the horizontal gradient and the vertical gradient of the current window (2320), respectively.

[0305] In one embodiment, the HoG (2350) of the gradient may be generated (or updated) based on an intra prediction mode and the intensity of the said intra prediction mode. The HoG (2350) of the gradient may be generated by accumulating the intra prediction mode calculated from the gradient and the amplitude (or intensity) corresponding to the said intra prediction mode. In other words, the HoG (2350) of the gradient may be generated by summing the amplitudes for each intra prediction mode collected from the template (2310) based on the intra prediction mode.

[0306] The video decoder (2000) can generate a final predictor using the result of a weighted sum of up to five intra prediction modes selected in order of largest value in the gradient HoG (2350) and predictors generated in planar mode. The video decoder (2000) can determine the intra prediction mode of the block through the final predictor.

[0307] FIG. 24 is a diagram illustrating a Template-based Intra Mode Derivation (TIMD) method for inducing an intra prediction mode in a video decoding device according to one embodiment.

[0308] The video decoder (2000) can identify one or two intra prediction modes with the lowest cost from a list of Most Probable Modes (MPM) composed of certain modes that are highly likely to be selected. The template (2410) may be composed of an area adjacent to the upper and left sides of the current block (2400). A predictor may be generated using reference samples (2420) of the template (2410) using the identified intra prediction modes. The video decoder (2000) may determine a reference sample from the reference samples (2420) according to each intra prediction mode and perform intra prediction on the template (2410) using the determined reference sample. The Sum of Absolute Transformed Difference (SATD) may be obtained using the difference between the already reconstructed value and the predicted value of the template (2410). One or two minimum SATD values ​​among all SATD values ​​of the intra prediction modes are selected, and the prediction samples of the current block (2420) can be determined using the corresponding intra prediction modes.

[0309] FIGS. 25, 26a and 26b are drawings illustrating a Cross Line Prediction (CLP) method for inducing an intra prediction mode in a video decoding device according to one embodiment.

[0310] The video decoding device (2000) can generate prediction samples (2530) of the current block by performing intra prediction for the current block using the intra prediction mode of the current block (2500) and reference samples within the surrounding restored area (2510) of the current block (2500).

[0311] An MPM may be used for efficient encoding / decoding of an intra prediction mode. A video decoder (2000) may use an intra prediction mode derived based on the difference between a predetermined set of samples within a surrounding restored region (2510) of the current block (2500) as an MPM. The method of deriving an intra prediction mode based on the difference between a predetermined set of samples within a surrounding restored region is referred to as the Cross Line Prediction (CLP) method, but it should be noted that the content of this disclosure is not limited to this name.

[0312] The video decoder (2000) can determine a second sample set including samples indicated by the prediction direction (2520) of the first directional mode from the first sample among the samples included in the surrounding restored region (2510). By using the second sample set as a reference sample set, a prediction sample set corresponding to the first sample can be generated.

[0313] To generate a prediction sample of any one sample in a prediction sample set, one or more samples in a second sample set located on the prediction direction (2520) relative to the sample may be used.

[0314] The prediction decoder (2030) can calculate a cost value of a first directional mode corresponding to a prediction direction (2520) by comparing a prediction sample set with a first sample set including a first sample. The cost value of the first directional mode may be based on the difference values ​​between the sample values ​​in the prediction sample set and the sample values ​​in the first sample set.

[0315] In one embodiment, the more similar the predicted sample set is to the first sample set, the smaller the cost value may be calculated.

[0316] In one embodiment, the cost value corresponding to the comparison result between the prediction sample set and the first sample set may include SAD (sum of absolute difference), SSE (sum of squared error), MR-SAD (mean removed SAD), or MAD (mean of absolute difference) between the sample values ​​in the prediction sample set and the sample values ​​in the first sample set.

[0317] The intra-prediction mode of the current block (2500) can be determined using a prediction mode corresponding to the prediction direction with the lowest cost. Based on the determined intra-prediction mode, the prediction samples (2530) of the current block can be determined.

[0318] Based on the relationship (2600) between the conventional intra prediction mode and the prediction direction, a prediction mode corresponding to the prediction direction (2520) that generates the minimum cost can be determined. According to the table (2620), the intra prediction mode can be corresponding to the angle of the prediction direction (2520).

[0319] With reference to FIGS. 23 to 26b, an existing sub-block-based intra prediction method has been described in detail, and a method for determining an intra prediction mode at the decoder stage has been described in detail.

[0320] However, in the ISP mode of Fig. 23, a single Intra Prediction Mode (IPM) is used to predict the current block. However, to reduce signaling overhead, sub-partitions divided within a single encoding unit all have the same prediction mode.

[0321] In addition, according to the method for deriving an intra prediction mode in a decoder described in detail in FIGS. 24 to 26b, a single IPM capable of representing the current block can be determined by deriving an intra prediction mode through a global search of the surrounding reference samples, reference templates, and reference multiple lines of the current block. However, if the block size is large or the texture or image characteristics within the block are heterogeneous, there is a high probability that the prediction accuracy of the intra prediction mode derived solely from surrounding reference samples or surrounding reference templates will decrease.

[0322] Accordingly, the present disclosure proposes a method for rapidly and accurately inducing an intra prediction mode for each sub-partition. Hereinafter, with reference to FIGS. 27 to 42, a method in which a video decoding device (2000) according to one embodiment induces an intra prediction mode separately for each sub-partition of a block and performs intra prediction is described in detail.

[0323] FIG. 27 illustrates a flowchart of a video decoding method according to one embodiment.

[0324] In step 2710, a video decoding device (2000) according to one embodiment may obtain first information from a bitstream indicating whether an intra prediction mode derived per sub-partition is used in the current block. The current block may be an encoding unit, a maximum encoding unit, a prediction unit, or a conversion unit.

[0325] In step 2720, the video decoding device (2000) according to one embodiment can determine the intra prediction mode of the first sub-partition among the sub-partitions of the current block based on the surrounding information of the first sub-partition when an intra prediction mode derived per sub-partition according to the first information is used. The first sub-partition may be the left, upper, right, or lower sub-partition closest to the first restored surrounding blocks adjacent to the current block among the plurality of sub-partitions included in the current block. For example, if the coding order between blocks follows a Z-shaped direction proceeding from left to right or from upper to lower, the left sub-partition or the right sub-partition may be the first sub-partition. As another example, if the coding order between blocks proceeds from right to left, the right sub-partition may be the first sub-partition. As another example, if the coding order between blocks proceeds from lower to upper, the lower sub-partition may be the first sub-partition.

[0326] In step 2730, a video decoding device (2000) according to one embodiment can obtain recovery samples of the first sub-partition using prediction samples obtained by performing intra prediction based on an intra prediction mode of the first sub-partition.

[0327] A video decoding device (2000) according to one embodiment determines samples of a surrounding recovery block adjacent to the first sub-partition as at least one reference sample based on an intra prediction mode of the first sub-partition, and can determine prediction samples of the first sub-partition using the reference sample. If there are residual samples corresponding to the first sub-partition, the recovery samples of the first sub-partition can be determined by combining the prediction samples and the residual samples.

[0328] In step 2740, a video decoding device (2000) according to one embodiment may determine the intra prediction mode of a second sub-partition adjacent to a first sub-partition among the sub-partitions of the current block using the prediction result of the first sub-partition and the surrounding information of the second sub-partition. The prediction result of the first sub-partition may include the intra prediction mode of the first sub-partition, prediction samples, or information used to determine the intra prediction mode of the first sub-partition. The surrounding information of the second sub-partition may be information related to samples adjacent to the second sub-partition among the information of a previously restored surrounding block adjacent to the current block and restoration samples. Accordingly, the intra prediction mode of the second sub-partition may be determined by considering the restoration information of the first sub-partition located inside the current block and the restoration information of the surrounding block located outside the current block.

[0329] The surrounding information of the first sub-partition used to determine the intra prediction mode of the first sub-partition may include not only reference samples or templates of surrounding blocks adjacent to the first sub-partition, but also reference samples or reference templates located at a distance greater than a predetermined distance from the first sub-partition. The predetermined distance may be a distance of 10 to 20 pixels.

[0330] Similarly, the surrounding information of the second sub-partition used to determine the intra-prediction mode of the second sub-partition may also include a reference sample or reference template located at a distance greater than a predetermined distance from the second sub-partition.

[0331] In step 2750, a video decoding device (2000) according to one embodiment can obtain recovery samples of the second sub-partition using prediction samples obtained by performing intra prediction based on an intra prediction mode of the second sub-partition.

[0332] A video decoding device (2000) according to one embodiment determines at least one reference sample for surrounding reconstructed samples adjacent to the second sub-partition based on an intra prediction mode of the second sub-partition, and can determine prediction samples of the first sub-partition using the reference sample. At this time, the surrounding reconstructed samples used for intra prediction of the second sub-partition may include the reconstructed samples of the first sub-partition and the reconstructed samples of surrounding blocks adjacent to the outside of the current block in the second sub-partition. If there are residual samples corresponding to the second sub-partition, the reconstructed samples of the second sub-partition may be determined by combining the prediction samples and the residual samples.

[0333] In step 2760, a video decoding device (2000) according to one embodiment can restore the current block based on the restoration samples of the first sub-partition and the restoration samples of the second sub-partition.

[0334] Although FIG. 27 describes only the first and second sub-partitions, i.e., two sub-partitions, the intra prediction mode of the sub-partitions can be induced in the same way even when the current block includes three or more sub-partitions. For example, if the current block includes multiple sub-partitions and a third sub-partition adjacent to the second sub-partition exists, the video decoding device (2000) may use at least one of the intra prediction mode of the second sub-partition, the prediction sample and the recovery sample, and the information of the surrounding block to induce the intra prediction mode of the third sub-partition. The video decoding device (2000) may obtain prediction samples by performing intra prediction on the third sub-partition using the induced intra prediction mode of the third sub-partition. When residual samples corresponding to the third sub-partition are decoded, recovery samples may be obtained by combining the prediction samples of the third sub-partition and the residual samples.

[0335] The video decoding device (2000) can obtain recovery samples of the current block that include all recovery samples of the sub-partitions included in the current block. In-loop filtering, such as deblocking filtering, can be performed on the recovery samples of the current block.

[0336] A video decoding device (2000) according to one embodiment can individually determine an intra prediction mode induction type for inducing an intra prediction mode for each sub-partition. The intra prediction mode induction type may include types according to the DIMD, TIMD, and CLP methods described above with reference to FIGS. 24 to 26b.

[0337] The video decoding device (2000) may separately determine a first intra prediction mode induction type for determining the intra prediction mode of the first sub-partition based on surrounding information of the first sub-partition, and a second intra prediction mode induction type for determining the intra prediction mode of the second sub-partition based on the prediction result of the first sub-partition and surrounding information of the second sub-partition. Accordingly, the first intra prediction mode induction type of the first sub-partition and the second intra prediction mode induction type of the second sub-partition may be the same or different.

[0338] A video decoding device (2000) according to one embodiment may use a set of intra prediction induction type candidates including two or more intra prediction induction type candidates. Among the set of intra prediction induction type candidates, a first intra prediction mode induction type is determined, and according to the first intra prediction mode induction type, an intra prediction mode of a first sub-partition may be determined based on surrounding information of the first sub-partition. Additionally, among the set of intra prediction induction type candidates, a second intra prediction mode induction type is determined, and according to the second intra prediction mode induction type, an intra prediction mode of a second sub-partition may be determined using the prediction result of the first sub-partition and surrounding information of the second sub-partition.

[0339] A video decoding device (2000) according to one embodiment may acquire first information indicating whether an intra prediction mode induced per sub-partition in the current block is utilized after determining whether intra prediction is performed according to an ISP mode in the current block. The ISP mode is a method in which intra prediction is performed on conversion block sub-partitions divided from the current block, and in this case, intra prediction may be performed according to the same intra prediction mode for each conversion block sub-partition. Since the first information is acquired after determining the ISP mode, the video decoding device (2000) may determine whether an intra prediction mode induced per sub-partition is utilized according to the first information after determining whether intra prediction is performed according to the same intra prediction mode for each conversion block sub-partition according to the ISP mode.

[0340] A video decoding device (2000) according to one embodiment may obtain second information indicating whether an intra prediction mode derived per sub-partition is used for a chroma block of the current block. Based on the second information, the video decoding device (2000) may determine whether an intra prediction mode derived per sub-partition is used for a chroma block of the current block.

[0341] A video decoding device (2000) according to one embodiment may determine whether an intra prediction mode derived for each sub-partition of the current block is used, and then determine whether to determine an intra prediction mode of the chroma block using the intra prediction mode of the lumina block of the current block. The intra prediction mode of the chroma block may be determined using the intra prediction mode of the lumina block or may be determined individually.

[0342] A video decoding device (2000) according to one embodiment may determine whether the intra prediction mode of the chroma block of the current block is determined based on the intra prediction mode of the lumina block, and then determine whether the intra prediction mode derived for each sub-partition for the chroma block is used.

[0343] A video decoding device (2000) according to one embodiment may determine that an intra prediction mode derived per sub-partition of the current block is not used when the difference in sample values ​​of surrounding samples of the current block is smaller than a predetermined value. In this case, the first information indicating whether an intra prediction mode derived per sub-partition is used may not be obtained from the bitstream.

[0344] A video decoding device (2000) according to one embodiment can individually utilize a derivation method for an intra prediction mode by using a surrounding reconstructed reference sample or reference template for each sub-partition, thereby reducing the amount of computation required to deriv the intra prediction mode and increasing the accuracy of the prediction result. Since intra prediction is performed according to the intra prediction mode derived for each sub-partition divided from the current block, multiple intra prediction modes individually determined for each of the multiple sub-partitions within a single encoding unit can be utilized, thereby enabling more accurate prediction.

[0345] FIGS. 28 and 29 illustrate a method for inducing an intra-prediction mode per sub-partition according to one embodiment.

[0346] A video decoding device (2000) according to one embodiment can divide the current block (2800) into sub-partitions SP1, SP2, SP3, SP4 and sequentially induce an intra prediction mode for the sub-partitions SP1, SP2, SP3, SP4. By using a reference sample or reference template located near each sub-partition SP1, SP2, SP3, SP4, an intra prediction mode determined individually for each sub-partition can be determined.

[0347] In FIG. 28, samples (2810, 2820) adjacent to SP1 among surrounding samples that are adjacent to the current block (2800) and restored before the current block can be used as reference samples or reference templates for determining the intra prediction mode of sub-partition SP1. The video decoder (2000) can derive the intra prediction mode of sub-partition SP1 using the surrounding restored samples.

[0348] The intra prediction mode of sub-partition SP2 can be determined by using samples (2840) adjacent to SP2 among surrounding samples that are adjacent to the current block (2800) and restored before the current block (2800), as reference samples or reference templates. Additionally, the video decoder (2000) can induce the intra prediction mode of sub-partition SP2 by using samples (2830) adjacent to sub-partition SP2 among samples restored through the prediction of sub-partition SP1. Accordingly, the video decoder (2000) can induce the intra prediction mode of sub-partition SP2 by using samples (2840) restored before the current block (2800) and restored samples (2830) of sub-partition SP1 adjacent to sub-partition SP2.

[0349] The intra prediction mode of sub-partition SP3 can be determined by using samples (2860) adjacent to SP3 among surrounding samples that are adjacent to the current block (2800) and restored before the current block (2800) as reference samples or reference templates. Additionally, the video decoder (2000) can induce the intra prediction mode of sub-partition SP3 by using samples (2850) adjacent to sub-partition SP3 among samples restored through the prediction of sub-partition SP2. Accordingly, the video decoder (2000) can induce the intra prediction mode of sub-partition SP3 by using samples (2860) restored before the current block (2800) and restored samples (2850) of sub-partition SP2 adjacent to sub-partition SP3.

[0350] The intra prediction mode of sub-partition SP4 can be determined by using samples (2880) adjacent to SP4 among surrounding samples that are adjacent to the current block (2800) and restored before the current block (2800) as reference samples or reference templates. Additionally, the video decoder (2000) can induce the intra prediction mode of sub-partition SP4 by using samples (2870) adjacent to sub-partition SP4 among samples restored through the prediction of sub-partition SP3. Accordingly, the video decoder (2000) can induce the intra prediction mode of sub-partition SP4 by using samples (2880) restored before the current block (2800) and restored samples (2870) of sub-partition SP3 adjacent to sub-partition SP4.

[0351] Accordingly, the video decoding device (2000) may use the intra prediction mode of the sub-partition that is derived first among the sub-partitions or the recovery sample generated as a prediction result to derive the intra prediction mode of the sub-partitions. Accordingly, the intra prediction mode derived first may be used to derive the intra prediction mode of the next sub-partition.

[0352] The prediction results of the previously restored sub-partition can be used not only when inducing the intra prediction mode, but also when performing the intra prediction. Among the surrounding samples restored before the current block (2800) that are adjacent to sub-partition SP1 (2810, 2820), reference samples can be determined according to the induced intra prediction mode of sub-partition SP1. Predicted samples of sub-partition SP1 are determined using the reference samples, and restored samples of sub-partition SP1 can also be determined based on the predicted samples.

[0353] Likewise, among the surrounding samples restored before the current block (2800) and the samples (2830, 2840) adjacent to sub-partition SP2 among the restored samples of sub-partition SP1, reference samples can be determined according to the induced intra-prediction mode of sub-partition SP2. Predicted samples of sub-partition SP2 are determined using the reference samples, and restored samples of sub-partition SP2 can also be determined based on the predicted samples.

[0354] Additionally, among the surrounding samples restored before the current block (2800) and the samples (2850, 2860) adjacent to sub-partition SP3 among the restored samples of sub-partition SP2, reference samples can be determined according to the induced intra-prediction mode of sub-partition SP3. Predicted samples of sub-partition SP3 are determined using the reference samples, and restored samples of sub-partition SP3 can also be determined based on the predicted samples.

[0355] Additionally, among the surrounding samples restored before the current block (2800) and the samples (2870, 2880) adjacent to sub-partition SP4 among the restored samples of sub-partition SP3, reference samples can be determined according to the induced intra prediction mode of sub-partition SP4. Predicted samples of sub-partition SP4 are determined using the reference samples, and restored samples of sub-partition SP4 can also be determined based on the predicted samples.

[0356] According to the example of FIG. 29, a method for deriving an intra prediction mode of a sub-partition using globally referenced surrounding information is illustrated. Depending on the texture or characteristics of the image, an intra prediction mode can be accurately derived for each sub-partition by using all globally available reference samples or reference templates. All globally available reference samples or reference templates may include not only the reconstructed samples adjacent to the current sub-partition, but also all samples located within a predetermined range among the previously reconstructed samples, even if they are not adjacent.

[0357] As a reference sample or reference template for determining the intra prediction mode of sub-partition SP1, global surrounding samples (2910, 2920) that surround the current block (2900) or are located within a predetermined range from the current block (2900) among surrounding samples restored before the current block (2900) may be used. The video decoder (2000) can induce the intra prediction mode of sub-partition SP1 using the global surrounding samples (2910, 2920).

[0358] As a reference sample or reference template for determining the intra prediction mode of sub-partition SP2, samples (2930) adjacent to sub-partition SP2 among the samples restored through prediction of sub-partition SP1, and global surrounding samples restored before the current block (2900) may be used. In particular, among the global surrounding samples restored before the current block (2900), global surrounding samples (2940, 2945) excluding the region adjacent to sub-partition SP1 may be utilized to induce the intra prediction mode of sub-partition SP2. The video decoding device (2000) can induce the intra prediction mode of sub-partition SP2 by using the restored samples (2930) of sub-partition SP1 adjacent to sub-partition SP2 and the global surrounding samples (2940, 2945).

[0359] Among the global surrounding samples restored before the current block (2900), global surrounding samples (2945, 2960), excluding the regions adjacent to sub-partitions SP1 and SP2, can be used to induce an intra prediction mode of sub-partition SP3. The video decoding device (2000) can induce an intra prediction mode of sub-partition SP3 using the restored samples (2950) of sub-partition SP2 adjacent to sub-partition SP3 and the global surrounding samples (2945, 2960).

[0360] In a similar manner, among the global surrounding samples restored before the current block (2900), global surrounding samples (2945, 2980) excluding the regions adjacent to sub-partitions SP1, SP2, and SP3 can be utilized to induce an intra prediction mode of sub-partition SP4. The video decoder (2000) can induce an intra prediction mode of sub-partition SP4 using the restored samples (2970) of sub-partition SP3 adjacent to sub-partition SP4 and the global surrounding samples (2945, 2980).

[0361] The sample values ​​of the reference sample or reference template can be used as is, or the encoding information of the reference sample or reference template can be used to derive the intra prediction modes of SP1, SP2, SP3, and SP4.

[0362] FIGS. 30 and 31 illustrate an intra prediction mode derived per sub-partition according to one embodiment.

[0363] According to FIG. 30, a video decoding device (2000) according to one embodiment may divide the current block (3000) into sub-blocks (3010, 3030, 3050, 3070) used according to the ISP mode, even when an ISP mode is applied to the current block (3000), and may separately induce an intra-prediction mode for each sub-block. At this time, the sub-block may be an encoding unit or a conversion unit.

[0364] Specifically, prediction can be performed first on the first sub-block (3010) located furthest to the left in the decoding order, and prediction can be performed moving one by one to the right from the first sub-block (3010) to the fourth sub-block (3070). The first intra-prediction mode of the first sub-block (3010) can be sequentially induced to correspond to the first prediction direction (3020), the second intra-prediction mode of the second sub-block (3030) to correspond to the second prediction direction (3040), the third intra-prediction mode of the third sub-block (3050) to correspond to the third prediction direction (3060), and the fourth intra-prediction mode of the fourth sub-block (3070) to correspond to the fourth prediction direction (3080).

[0365] As described above in FIGS. 28 and 29, surrounding restoration samples (3090) of the current block (3000) may be used as a reference sample or reference template for inducing the first intra prediction mode of the first sub-block (3010). The surrounding restoration samples (3090) may be samples located globally among samples restored before the current block (3000).

[0366] As reference samples or reference templates for inducing the second intra prediction mode of the second sub-block (3030), surrounding restoration samples (3090) of the current block (3000) and restoration samples of the first sub-block (3010) may be used. As reference samples or reference templates for inducing the third intra prediction mode of the third sub-block (3050), surrounding restoration samples (3090) of the current block (3000) and restoration samples of the second sub-block (3030) may be used. As reference samples or reference templates for inducing the fourth intra prediction mode of the fourth sub-block (3070), surrounding restoration samples (3090) of the current block (3000) and restoration samples of the third sub-block (3050) may be used.

[0367] Since the first, second, third, and fourth intra prediction modes are all determined separately, multiple intra prediction modes can be determined within the current block (3000).

[0368] A video decoding device (2000) according to one embodiment can sequentially perform intra prediction of sub-blocks (3010, 3030, 3050, 3070). First, based on the first intra prediction mode of the first sub-block (3010), prediction samples of the first sub-block (3010) can be determined using a reference sample corresponding to the first prediction direction (3020) among the surrounding restoration samples (3090). If there are residual samples of the first sub-block (3010), the restoration samples of the first sub-block (3010) can also be determined by combining the prediction samples of the first sub-block (3010) with the residual samples.

[0369] Based on the second intra-prediction mode of the second sub-block (3030), prediction samples of the second sub-block (3030) can be determined using a reference sample corresponding to the second prediction direction (3040) among the surrounding restoration samples (3090). If the restoration samples of the first sub-block (3010) are also located in the second prediction direction (3040), the prediction samples of the second sub-block (3030) can be determined using reference samples located in the second prediction direction (3040) among the surrounding restoration samples (3090) and the restoration samples of the first sub-block (3010). If there are residual samples of the second sub-block (3030), the restoration samples of the second sub-block (3030) can also be determined by combining the prediction samples of the second sub-block (3030) with the residual samples.

[0370] Based on the third intra-prediction mode of the third sub-block (3050), prediction samples of the third sub-block (3050) can be determined using a reference sample corresponding to the third prediction direction (3060) among the surrounding restoration samples (3090). In this case, if there are no restoration samples of the first sub-block (3010) or the second sub-block (3030) in the third prediction direction (3060), the prediction samples of the third sub-block (3050) can be determined by considering only the surrounding restoration samples (3090). If there are residual samples of the third sub-block (3050), the restoration samples of the third sub-block (3050) can also be determined by combining the prediction samples of the third sub-block (3050) with the residual samples.

[0371] Similarly, if there are no restoration samples of the first sub-block (3010), second sub-block (3030), and third sub-block (3050) in the fourth prediction direction (3080) corresponding to the fourth intra prediction mode of the fourth sub-block (3070), the prediction samples of the fourth sub-block (3070) can be determined by considering only the surrounding restoration samples (3090). If there are residual samples of the fourth sub-block (3070), the restoration samples of the fourth sub-block (3070) can also be determined by combining the prediction samples of the fourth sub-block (3070) with the residual samples.

[0372] A video decoding device (2000) according to one embodiment can individually determine an intra prediction mode for subblocks created in various division forms, as well as subblocks divided in the horizontal direction or subblocks divided in the vertical direction.

[0373] According to FIG. 31, the current block (3100) may be divided into a plurality of sub-blocks (3110, 3120, 3130, 3140). The video decoder (2000) may determine an intra prediction mode for each sub-block. The video decoder (2000) may determine an intra prediction mode (3115) based on a first reference region (3160) of a first sub-block (3110). The video decoder (2000) may obtain prediction samples by performing a prediction on the first sub-block (3110) based on the determined intra prediction mode. The video decoder (2000) may determine restored samples using the prediction samples of the first sub-block (3110).

[0374] In one embodiment, the video decoder (2000) may determine an intra prediction mode (3125) based on a second reference region (3170) of a second sub-block (3120). The second reference region (3170) may include restored samples of a first sub-block (3110). The video decoder (2000) may determine an intra prediction mode of the second sub-block (3120) based on the intra prediction mode (3110) of the first sub-block (3110) or the restored samples. The video decoder (2000) may obtain predicted samples by performing a prediction on the second sub-block (3120) based on the determined intra prediction mode (3125). The video decoder (2000) may obtain restored samples using the predicted samples of the second sub-block (3120).

[0375] In one embodiment, the video decoding device (2000) can determine an intra prediction mode (3135, 3145) for each of the third sub-block (3130) and the fourth sub-block (3140), just as it does for the first sub-block (3110) and the second sub-block (3120), perform prediction, and perform restoration.

[0376] In one embodiment, the video decoder (2000) may individually determine reference samples or reference templates for each sub-block. The video decoder (2000) may determine reference samples of the first sub-block (3110) based on the surrounding restoration region (3160) of the first sub-block (3110). The video decoder (2000) may also determine a template region similar to the surrounding restoration region (3160). The video decoder (2000) may determine a reference block corresponding to the similar template region. The video decoder (2000) may perform a prediction for the first sub-block (3110) based on the determined reference block. For example, the video decoder (2000) may determine the value of the first sub-block to be equal to the value of the reference block. The video decoder (2000) may restore the first sub-block (3110).

[0377] In one embodiment, the video decoding device (2000) may determine a reference sample of the second sub-block (3120) based on the surrounding restoration area (3170) of the second sub-block (3120). The surrounding restoration area (3170) of the second sub-block (3120) may include a restoration area outside the current block (3100) as well as a restoration area of ​​the first sub-block (3110).

[0378] In a similar manner, the video decoder (2000) can determine a reference sample of the third sub-block (3130) based on the surrounding restoration area (3180) of the third sub-block (3130). The surrounding restoration area (3180) of the third sub-block (3130) may include a restoration area outside the current block (3100), as well as a restoration area of ​​the first sub-block (3110) and a restoration area of ​​the second sub-block (3120). Accordingly, the video decoder (2000) can determine an intra prediction mode (3135) of the third sub-block (3130) using the restoration area of ​​the first sub-block (3110) and the restoration area of ​​the second sub-block (3120), and can also obtain prediction samples according to the intra prediction mode (3135).

[0379] In a similar manner, the video decoding device (2000) can determine a reference sample of the fourth sub-block (3140) based on the surrounding restoration area (3190) of the fourth sub-block (3140). The surrounding restoration area (3190) of the fourth sub-block (3140) may include a restoration area outside the current block (3100), as well as a restoration area of ​​the first sub-block (3110), a restoration area of ​​the second sub-block (3120), and a restoration area of ​​the third sub-block (3130). Accordingly, the video decoding device (2000) can determine an intra prediction mode (3145) of the fourth sub-block (3140) and obtain prediction samples according to the intra prediction mode (3145) by using the restoration area of ​​the first sub-block (3110), the restoration area of ​​the second sub-block (3120), and the restoration area of ​​the third sub-block (3130).

[0380] A video decoding device (2000) according to one embodiment can induce an intra prediction mode of a current sub-partition (sub-block) by using information of a sub-partition (sub-block) restored before the current sub-partition (sub-block) and information of the current sub-partition (sub-block). At this time, the information of the current sub-partition (sub-block) that can be used to induce an intra prediction mode of the sub-partition may include the channel, height, size, aspect ratio, and position of the sub-partition (sub-block). In addition, the information of the current sub-partition (sub-block) that can be used to induce an intra prediction mode of the sub-partition may include an intra prediction mode of a block / coding unit containing the sub-partition (sub-block), a type of frame / slice to which the block / coding unit containing the sub-partition (sub-block) belongs, a conversion type (MTS / LFNST / NSPT), a prediction method of the current block, a final predictor sample value, and information used to generate the final predictor. Specifically, examples of information used to generate the final predictor that can be referenced are as follows.

[0381] If the final predictor intra prediction mode is determined according to DIMD / TIMD / OBIC, the intra prediction modes and their weights used in the weighted sum for generating the final predictor may be information that can be referenced to derive the intra prediction mode of the current subpartition.

[0382] If the final predictor's intra prediction mode follows the Spatial Geometric Prediction Mode (SGPM), the orientation of the partition used to generate the final predictor and the partition's intra prediction mode may be information that can be referenced to derive the current sub-partition's intra prediction mode.

[0383] If the final predictor's intra prediction mode follows the Intra TMP (Intra Template Matching Prediction) mode or the IBC (Intra Block Copy) mode, the block vector (BV) used to generate the final predictor and the region pointed to by the BV may be information that can be referenced to derive the intra prediction mode of the current subpartition.

[0384] When the final predictor's intra prediction mode is performed according to the Enhanced Intra Prediction (EIP) mode, the filter shape, filter size, filter type, and filter coefficients used for prediction may be referenceable information to derive the intra prediction mode of the current subpartition.

[0385] When the final predictor intra prediction mode is performed according to the ISP mode, the partitioning direction (Hor / Ver) of the sub-block, the size and number of partitioned sub-blocks, etc., may be information that can be referenced to induce the intra prediction mode of the current sub-partition.

[0386] In the case of a final predictor generated by exceptionally performing inter-prediction on the current subpartition, information such as the motion vector (MV), the region pointed to by the MV, the reference picture index, whether uni-inter prediction is performed, the prediction direction when bi-interpreting is performed, the BCW (Bi-prediction CU weighting) index, and the AMVR precision (Adaptive Motion Vector Resolution precision) may be information that can be referenced to induce the intra-prediction mode of the current subpartition.

[0387] FIG. 32 illustrates a case in which an intra prediction mode induced per sub-partition is limited according to one embodiment.

[0388] In a video decoding device (2000) according to one embodiment, when an intra prediction mode is determined individually for each sub-partition, additional operations must be performed for each sub-partition compared to when the sub-partitions use the same prediction mode. Therefore, it is desirable that the intra prediction mode be determined for each sub-partition only when there are heterogeneous texts or characteristics for each sub-partition.

[0389] To this end, the video decoding device (2000) can analyze the consistency of reference samples adjacent to the current block (3200) in order to determine in advance whether there are heterogeneous texts or characteristics even if sub-partitions are distinguished within the current block (3200). A value representing the consistency of reference samples can be defined as a difference value between reference samples abs(max - min).

[0390] The difference value abs(max - min) between reference samples refers to the difference between the maximum sample value max and the minimum sample value min among the samples included in the reference range.

[0391] For example, if the difference value abs(maxTop - minTop) between reference samples in the upper reference area (3210) of the current block (3200) and the difference value abs(maxTop - minTop) between reference samples in the left reference area (3220) are both smaller than a predetermined threshold FLAT_THRESHOLD, the current block (3200) can be determined to be flat. In the case of a flat block, since it does not have heterogeneous text or characteristics for each sub-partition, there is no need to determine individual intra-prediction modes for each sub-partition.

[0392] The threshold for the difference value abs(maxTop - minTop) between reference samples in the upper reference area (3210) and the threshold for the difference value abs(maxTop - minTop) between reference samples in the left reference area (3220) may be different or the same.

[0393] Accordingly, when the video decoding device (2000) according to one embodiment determines that the current block (3200) is a flat block, it is not necessary to obtain first information from the bitstream indicating whether an intra prediction mode derived per sub-partition is used, and accordingly, the same intra prediction mode can be used without determining an intra prediction mode per sub-partition.

[0394] FIG. 33 illustrates a method for inducing an intra prediction mode for each sub-partition using reference information located at a distance according to one embodiment.

[0395] A video decoding device (2000) according to one embodiment can determine an intra prediction mode for each sub-partition using all reference samples or reference templates located at a distance. This is because, depending on the texture or features of the image, it may be more advantageous to use information from a distance to accurately derive the intra prediction mode.

[0396] Among the surrounding samples restored before the current block (3300), restored samples (3310) located at a certain distance in the vertical direction from the upper boundary of the current block (3300) and restored samples (3320) located at a certain distance in the horizontal direction from the left boundary of sub-partition SP1 can be used as reference samples or reference templates for determining an intra prediction mode for sub-partition SP1. The video decoding device (2000) can induce an intra prediction mode of sub-partition SP1 using restored samples (3310, 3320) located at a distance from the current block (3300). At this time, the distance can be defined as a distance of 10, 12, 15, 20 pixels, etc. from the upper boundary or the left boundary of the current block (3300).

[0397] Additionally, as reference samples or reference templates for inducing the intra prediction mode of sub-partition SP2, restoration samples (3310) located at a predetermined distance in the vertical direction from the upper boundary of the current block (3300) and restoration samples (3330) located at a predetermined distance in the horizontal direction from the left boundary of sub-partition SP2 may be used.

[0398] Similarly, as reference samples or reference templates for inducing the intra prediction mode of sub-partition SP3, restoration samples (3310) located at a distant upper side and restoration samples (3340) located at a predetermined distance in the horizontal direction from the left boundary of sub-partition SP3 may be used.

[0399] Similarly, as reference samples or reference templates for inducing the intra prediction mode of sub-partition SP4, restoration samples (3310) located at a distant upper side and restoration samples (3350) located at a predetermined distance in the horizontal direction from the left boundary of sub-partition SP4 may be used.

[0400] The restoration samples (3310) located on the upper side of the current block (3300) can be defined as a predetermined number of samples located at a predetermined distance in the vertical direction from the upper boundary of the current block (3300) and adjacent to each other on the left and right.

[0401] The restoration samples (3320, 3330, 3340, 3350), which are located to the left of the current block (3300) and are used as reference samples or reference templates to induce intra prediction modes of sub-partitions, can be defined as a predetermined number of samples located at a predetermined distance in the horizontal direction from the left boundary of each sub-partition SP1, SP2, SP3, SP4 and adjacent to each other vertically.

[0402] In FIG. 33, when performing intra prediction sequentially for sub-partitions SP1, SP2, SP3, and SP4 and generating recovery samples, recovery samples of sub-partitions that were recovered before the current sub-partition among sub-partitions SP1, SP2, SP3, and SP4 may also be included in the surrounding samples used for intra prediction. For example, if sub-partition SP1 or SP2 exists in the prediction direction of the intra prediction mode induced for sub-partition SP3, a prediction sample of sub-partition SP can be obtained by referring to the recovery sample of sub-partition SP1 or SP2.

[0403] A method of using an intra prediction mode derived per sub-partition according to one embodiment may be referred to as a SPIMD (Sub partition-derived Intra prediction mode derivation) method. With reference to FIGS. 34 to 40, a method of signaling information indicating whether a video decoding device (2000) according to one embodiment uses an intra prediction mode derived per sub-partition in the current block, that is, whether a SPIMD mode is applied, will be described in detail below.

[0404] FIGS. 34, 35, 36, 37, 38, 39 and 40 illustrate a method of transmitting information indicating whether to induce an intra prediction mode per sub-partition according to various embodiments.

[0405] In FIGS. 34, 35, 36, 37, 38, 39 and 40, the syntax element Spimd_flag may correspond to information indicating whether the intra prediction mode derived per sub-partition is used in the current block, that is, whether the SPIMD mode is applied. If Spimd_flag is 1, the video decoder (2000) may determine that the intra prediction mode derived per sub-partition is used in the current block, and if Spimd_flag is 0, it may determine that the intra prediction mode derived per sub-partition is not used in the current block.

[0406] The syntax (3400) of FIG. 34 represents an embodiment in which the syntax element Spimd_flag is acquired before the syntax element intra_mip_flag is acquired. The syntax element intra_mip_flag indicates whether the Matrix-based Intra Prediction (MIP) mode is used among the intra prediction modes. Accordingly, according to the syntax (3400), the video decoder (2000) can first acquire the syntax element Spimd_flag to perform intra prediction for the current block, and if Spimd_flag is 1 (3410), determine the intra prediction mode derived per sub-partition of the current block and perform intra prediction sequentially. If Spimd_flag is 0 (3420), the video decoder (2000) can acquire the syntax element intra_mip_flag and determine whether to perform intra prediction according to the MIP mode. Therefore, in this case, intra prediction according to SPIMD mode may be possible regardless of whether the current block's intra prediction follows MIP mode or not.

[0407] Syntax (3500) of FIG. 35 represents an embodiment in which the syntax element Spimd_flag is acquired after the syntax element intra_mip_flag is acquired and before the syntax element intra_luma_ref_idx is acquired. The syntax element intra_luma_ref_idx represents information about a reference index indicating the prediction direction during intra prediction. Accordingly, according to the syntax (3500), the video decoder (2000) first acquires the syntax element intra_mip_flag to perform intra prediction for the current block, and if intra_mip_flag is 1 (3510), it can perform intra prediction according to the MIP mode. If intra_mip_flag is 0 (3520), the video decoder (2000) can acquire the syntax element Spimd_flag, and if Spimd_flag is 1, it can determine the intra prediction mode derived for each sub-partition of the current block and perform intra prediction sequentially. If Spimd_flag is 0 (3530), the video decoder (2000) can acquire the syntax element intra_luma_ref_idx and perform intra prediction based on the prediction direction indicated by intra_luma_ref_idx. Therefore, in this case, if the intra prediction of the current block does not follow the MIP mode, intra prediction according to the SPIMD mode is possible, and the intra prediction mode can be derived regardless of the prediction direction indicated by intra_luma_ref_idx. intra_luma_ref_idx may also point to a distant reference sample such as that shown in FIG. 33. However, according to the syntax (3500), since the application of SPIMD is determined before the acquisition of intra_luma_ref_idx, a distant reference sample cannot be used in SPIMD.

[0408] Syntax (3600) of FIG. 36 represents an embodiment in which the syntax element Spimd_flag is acquired after the syntax element intra_luma_ref_idx is acquired and before the syntax element intra_subpartitions_mode_flag is acquired. The syntax element intra_subpartitions_mode_flag indicates whether intra prediction is performed on a subpartition basis during intra prediction, that is, whether intra prediction is performed according to the ISP mode. Accordingly, according to the syntax (3600), the video decoder (2000) first acquires the syntax element intra_mip_flag to perform intra prediction for the current block, and if intra_mip_flag is 1 (3610), intra prediction can be performed according to the MIP mode. If intra_mip_flag is 0 (3620), the video decoder (2000) can obtain the syntax element intra_luma_ref_idx, determine the prediction direction indicated by intra_luma_ref_idx, and obtain the syntax element Spimd_flag. If Spimd_flag is 1, the video decoder (2000) can induce an intra prediction mode for each subpartition of the current block and sequentially perform intra prediction according to the intra prediction mode induced for the subpartitions. If Spimd_flag is 0 (3630), the video decoder (2000) can obtain the syntax element intra_subpartitions_mode_flag and determine whether to perform intra prediction according to the ISP mode on the current block according to intra_subpartitions_mode_flag. Therefore, according to the syntax (3600), since the decision to refer to a distant surrounding restoration sample is determined by intra_luma_ref_idx, the intra prediction mode of the sub-partition can be derived using a distant restoration sample.However, if the current block's intra prediction follows SPIMD mode, ISP mode cannot be applied.

[0409] Syntax (3700) of FIG. 37 represents an embodiment in which the syntax element Spimd_flag is acquired after the syntax element intra_subpartitions_mode_flag is acquired and before the syntax element intra_luma_mpm_flag is acquired. The syntax element intra_luma_mpm_flag indicates whether the lumina intra prediction mode is selected in the MPM list during intra prediction. Accordingly, according to the syntax (3700), the video decoder (2000) can first acquire the syntax element intra_mip_flag to perform intra prediction for the current block, and if intra_mip_flag is 1 (3710), it can perform intra prediction according to the MIP mode. If intra_mip_flag is 0 (3720), the video decoder (2000) can acquire the syntax element intra_luma_ref_idx and determine the prediction direction indicated by intra_luma_ref_idx, acquire the syntax element intra_subpartitions_mode_flag and determine whether to perform intra prediction according to the ISP mode on the current block according to the intra_subpartitions_mode_flag, and if the ISP mode is applied, determine the partitioning shape of the subpartition. Additionally, the video decoder (2000) can acquire the syntax element Spimd_flag after determining whether the ISP mode is applied. If Spimd_flag is 1, the video decoder (2000) can induce an intra prediction mode for each subpartition of the current block and sequentially perform intra prediction according to the intra prediction mode induced for the subpartitions. If Spimd_flag is 0 (3730), the video decoder (2000) can perform intra prediction of the current block based on the intra prediction direction indicated by intra_luma_ref_idx or the intra prediction mode determined based on the MPM list.However, if the current block performs intra prediction according to the ISP mode, it is divided into sub-partitions according to the ISP mode, and an intra prediction mode can be induced for each sub-partition according to the Spimd mode. Additionally, the intra prediction mode for each sub-partition can be determined by considering not only the nearby reconstructed samples pointed to by intra_luma_ref_idx but also the distant nearby reconstructed samples.

[0410] Syntax (3800) of FIG. 38 represents an embodiment in which the syntax element Spimd_flag is acquired instead of the syntax element intra_luma_mpm_remainder being acquired after the syntax element intra_luma_mpm_flag is acquired. Accordingly, according to the syntax (3800), the video decoder (2000) can first acquire the syntax element intra_mip_flag to perform intra prediction for the current block, and if intra_mip_flag is 1 (3810), the intra prediction can be performed according to the MIP mode. If intra_mip_flag is 0, the video decoder (2000) obtains the syntax element intra_luma_ref_idx and determines the prediction direction indicated by intra_luma_ref_idx, obtains the syntax element intra_subpartitions_mode_flag and determines whether to perform intra prediction according to the ISP mode on the current block according to the intra_subpartitions_mode_flag, and if the ISP mode is applied, determines the partitioning form of the subpartition. Additionally, after determining whether the ISP mode is applied, the video decoder (2000) obtains the syntax element intra_luma_mpm_flag and determines whether to determine the intra prediction mode using the MPM list (3820). If intra_luma_mpm_flag is 0 (3830), the video decoder (2000) obtains the syntax element Spimd_flag, and if Spimd_flag is 1, it induces an intra prediction mode for each sub-partition of the current block, and can sequentially perform intra prediction according to the intra prediction mode induced for the sub-partitions.If Spimd_flag is 0, the video decoder (2000) can obtain intra_luma_mpm_remainder and perform intra prediction of the current block according to the intra prediction mode determined from the MPM list using intra_luma_mpm_idx and intra_luma_mpm_remainder.

[0411] The syntax of FIG. 39 and the syntax of FIG. 40 include syntax elements for determining a method for determining the intra-prediction mode of the chroma block of the current block.

[0412] According to FIG. 39, the video decoding device (2000) can acquire the Spimd_flag for the chroma block of the current block before the cclm_mode_flag is acquired. If Spimd_flag is 1, the cclm_mode_flag is not acquired, and a chroma intra prediction mode can be derived for each sub-partition separately from the luminance intra prediction mode of the current block. If Spimd_flag is 0, the cclm_mode_flag is acquired, and if cclm_mode_flag is 1, the cclm_mode_idx is acquired. Therefore, the chroma intra prediction mode of the current block can be determined using the luminance intra prediction mode according to the method indicated by the cclm_mode_idx.

[0413] According to FIG. 40, the video decoder (2000) can acquire Spimd_flag instead of intra_chroma_pred_mode after acquiring cclm_mode_flag for the chroma block of the current block. Thus, when cclm_mode_flag is 1, the chroma intra prediction mode is determined by referencing the lumina intra prediction mode according to the CCLM (Cross-Component Linear Model) mode, so there is no need to directly derive the chroma intra prediction mode for each sub-partition. When cclm_mode_flag is 0, Spimd_flag is acquired, so the chroma intra prediction mode is determined separately from the lumina intra prediction mode of the current block, and the intra prediction mode for each sub-partition can be individually determined according to Spimd_flag.

[0414] In Figures 39 and 40, when both cclm_mode_flag and Spimd_flag are 0, the chroma intra-pred mode of the current block can be determined according to intra_chroma_pred_mode.

[0415] Methods for generating a final predictor to predict intra-prediction modes may include general methods and non-unidirectional methods.

[0416] According to a general method, a final predictor can be generated using the intra prediction mode indicated by the information regarding the intra prediction mode.

[0417] Non-standard methods can be classified into decoder-side induction methods, SGPM methods, and MIP methods.

[0418] According to the SGPM method, the current block is divided into two partitions, and an intra prediction mode final predictor is generated using an intra prediction mode determined for each partition. The two partitions divided from the current block may include partitions divided in the vertical and horizontal directions, as well as in the event direction.

[0419] According to the MIP method, the final predictor in intra-prediction mode is generated using a learning-based matrix and reference samples.

[0420] The decoder-side derivation method may include TIMD, DIMD, OBIC (Occurrence-based intra coding), intra TMP (Intra template matching prediction), and EIP methods.

[0421] According to the TIMD method, a final predictor of an intra prediction mode can be generated by weighting the predictors of up to two directional intra prediction modes and up to one non-directional intra prediction mode.

[0422] According to the DIMD method, a final predictor of the intra prediction mode can be generated by weighting up to five directional IPMs and non-directional IPMs or BV predictors.

[0423] The OBIC method is a technique that improves compression efficiency by analyzing occurrence patterns in advance during intra prediction and incorporating that pattern information into the prediction. Unlike general intra prediction, which only references pixel values ​​around the current block, the OBIC method allows for the utilization of pattern frequency information from previously occurring blocks. According to the OBIC method, a final predictor for the intra prediction mode can be generated by weighting up to five directional IPMs and non-directional IPMs or BV predictors.

[0424] The Intra TMP method is a technique that searches for templates in adjacent regions to find the most similar block for the prediction of the current block, and performs the prediction based on that block. According to the Intra TMP method, the final predictor of the Intra prediction mode can be generated by using the block having the template most similar to the template of the current block.

[0425] According to the EIP (Enhanced Intra Prediction) method, it is a technique that combines learning surrounding block patterns, template matching, filtering, and multi-directional prediction to achieve more precise predictions than Planar, DC, or directional modes. An intra prediction mode with an adjustable prediction direction, rather than a fixed direction, can be determined. Under the EIP method, a final predictor for the intra prediction mode can be generated using reconstructed information and filters.

[0426] In the case where a video decoding device (2000) according to one embodiment uses an intra prediction mode derived per sub-partition for the current block, the intra prediction mode can be determined according to a different derivation method for each sub-partition. A detailed explanation thereof is described below with reference to FIG. 41.

[0427] FIG. 41 illustrates a case where a method for inducing an intra prediction mode for each sub-partition is determined individually according to one embodiment.

[0428] A video decoding device (2000) according to one embodiment may include a method of directly deriving an intra prediction mode for each sub-partition, such as the DIMD, TIMD, and CLP types described above with reference to FIGS. 23 to 26b. However, the derivation method proposed in this disclosure is not limited to only the DIMD, TIMD, and CLP types. For convenience, the method of directly deriving an intra prediction mode for each sub-partition by the video decoding device (2000) is referred to as the 'Decoder-side IPM derivation method'.

[0429] The video decoding device (2000) can compare costs based on the results of determining an intra prediction mode sequentially according to various decoder-side IPM derivation methods for the sub-partitions SP1, SP2, SP3, and SP4 of the current block (4100). As a result, the decoder-side IPM derivation method that generates the lowest cost to determine the intra prediction mode can be determined. When the video decoding device (2000) according to one embodiment compares the costs generated by applying various combinations of decoder-side IPM derivation methods to the sub-partitions SP1, SP2, SP3, and SP4, the DIMD method can be selected for SP1, the TIMD method for SP2, and the CLP method for SP3 and SP4.

[0430] A video decoding device (2000) according to one embodiment may configure a SPIMD method candidate set including a plurality of decoder-side IPM derivation method candidates to be used for each sub-partition. The video decoding device (2000) may determine the Sum of Absolute Differences (SAD) or Sum of Absolute Transformed Differences (SATD) that occur when applying each derivation method candidate included in the SPIMD method candidate set to the current sub-partition to determine the intra prediction mode. By comparing the SAD or SATD of the derivation method candidates, the derivation method candidate that generates the lowest value may be determined as the derivation method of the current sub-partition. Additionally, the final predictor determined according to the derivation method may be determined as the intra prediction mode of the current sub-partition. By performing an operation to compare the SAD or SATD according to the derivation methods based on the candidate set for each sub-partition, the decoder-side IPM derivation method of each sub-partition is determined, and the final predictor determined accordingly may be determined as the intra prediction mode of each sub-partition.

[0431] In addition, the SPIMD method candidate set, which includes decoder-side IPM derivation method candidates, may be determined per picture or slice, per maximum coding unit, per coding unit, or per sub-partition. That is, the same SPIMD method candidate set may be used within a single picture or slice, within a single maximum coding unit, or within a single coding unit. Alternatively, an individual SPIMD method candidate set may be used for each sub-partition.

[0432] According to one embodiment, a video decoding device (2000) can configure a SPIMD method candidate set even without obtaining information from a bitstream indicating a method for determining a SPIMD method candidate set that includes decoder-side IPM derivation method candidates available per sub-partition. For example, the SPIMD method candidate set may be (DIMD, TIMD, CLP, OBIC, EIP).

[0433] Additionally, the video decoder (2000) can form a set of MPM (Most Probable Mode Candidate Pair Set) within the SPIMD mode candidate set. By comparing the SAD / SATD costs of candidate derivation methods included in the SPIMD mode candidate set, an MPM mode pair set composed of two candidate derivation methods that generate the lowest cost can be determined. The MPM mode pair set can be determined for a maximum encoding unit or for each encoding unit. Alternatively, the MPM mode pair set can be determined for each sub-partition. The video decoder (2000) can determine the decoder-side IPMD derivation method to be applied to the current sub-partition from the MPM mode pair set. For example, the combinable MPM mode pair sets may be ((DIMD,TIMD), (DIMD,CLP), (TIMD,CLP), (OBIC,CLP),…).

[0434] An MPM method pair set can be determined based on the presence or absence of a template, information regarding the intra prediction mode of a nearby previously restored block, and basic information of the current block. An MPM method pair set for a sub-partition can be configured by using information of a previously restored sub-partition (sub-block) and information of the current sub-partition (sub-block) that can be referenced to induce the intra prediction mode of the current sub-partition described above with reference to FIGS. 30 and 31.

[0435] Two embodiments for determining a decoder-side IPM derivation method applicable to each sub-partition, so that the video decoding device (2000) determines an intra prediction mode for each sub-partition, are described below. According to the first embodiment, the video decoding device (2000) may determine a derivation method applied to a sub-partition by configuring a plurality of decoder-side IPM derivation method candidates and by additionally obtaining information from the bitstream for selecting at least one of the decoder-side IPM derivation method candidates for each sub-partition. According to the second embodiment, the video decoding device (2000) may determine a derivation method applied to each sub-partition by additionally obtaining information from the bitstream for selecting at least one of the decoder-side IPM derivation method candidates to be applied to each sub-partition by each sub-partition or encoding unit.

[0436] First, the first embodiment will be described in detail.

[0437] In one embodiment, the video decoding device (2000) may determine whether to determine the decoder-side IPM derivation method for each sub-partition based on surrounding information without signaling additional information. Information regarding whether to determine the decoder-side IPM derivation method for each sub-partition without signaling additional information is also not obtained from the bitstream.

[0438] A video decoding device (2000) according to one embodiment can identify the candidate method with the lowest SAD / SATD cost generated to generate the final predictor of the intra prediction mode by applying candidate methods included in the SPIMD method candidate set for each sub-partition, and determine the identified candidate method as the decoder-side IPM derivation method of the corresponding sub-partition. In this case, the decoder-side IPMD derivation method can be determined individually for each sub-partition.

[0439] A video decoding device (2000) according to one embodiment can identify the candidate method with the lowest SAD / SATD cost generated to generate a final predictor of an intra prediction mode by applying candidate methods included in a SPIMD method candidate set to all sub-partitions included in one encoding unit or one block, and can determine the identified candidate method as the decoder-side IPM derivation method of the sub-partitions. In this case, the same decoder-side IPM derivation method can be determined for all sub-partitions included in one encoding unit or one block.

[0440] Additionally, the video decoding device (2000) can identify a candidate method that generates the lowest cost in generating a final predictor by applying candidate methods included in the MPM method pair set to each sub-partition. The identified candidate method can be determined as the decoder-side IPMD derivation method to be applied to the current sub-partition.

[0441] The following second embodiment is described in detail.

[0442] A video decoding device (2000) according to one embodiment can obtain information from a bitstream indicating whether to determine a decoder-side IPM derivation method for each sub-partition. That is, information indicating whether to determine a decoder-side IPM derivation method for each sub-partition may exist in the bitstream. Based on the information obtained from the bitstream, whether to determine a decoder-side IPM derivation method for each sub-partition can be identified. Additionally, information indicating whether to determine a decoder-side IPM derivation method for each sub-partition exists in the bitstream can also be obtained from the bitstream. Accordingly, based on the obtained information, whether to obtain information regarding whether to determine a decoder-side IPM derivation method for each sub-partition from the bitstream, or to determine based on information from surrounding blocks or information from the current block, can also be identified.

[0443] A video decoding device (2000) according to one embodiment may configure a SPIMD method candidate set and obtain information from a bitstream indicating one of the candidate methods included in the SPIMD method candidate set for each sub-partition. The obtained information may be a flag or an index. A flag is information for indicating one of the two, and an index is information for indicating one of more than three candidates. Accordingly, based on the obtained information, a candidate method for generating a final predictor of an intra-prediction mode may be identified among the candidate set, and the identified candidate method may be determined as the decoder-side IPM derivation method for the corresponding sub-partition. In this case, the decoder-side IPMD derivation method may be determined individually for each sub-partition.

[0444] A video decoding device (2000) according to one embodiment may configure a SPIMD method candidate set and, for all sub-partitions included in one encoding unit or one block, obtain information (flag or index) indicating one of the candidate methods included in the SPIMD method candidate set from a bitstream. Accordingly, based on the obtained information, a candidate method for generating a final predictor of an intra-prediction mode may be identified from the candidate set, and the identified candidate method may be determined as the decoder-side IPM derivation method for the sub-partitions included in the corresponding encoding unit or block. In this case, the same decoder-side IPM derivation method may be determined for all sub-partitions included in one encoding unit or one block.

[0445] Additionally, the video decoding device (2000) obtains information (flag or index) from the bitstream pointing to one of the candidate methods included in the MPM method pair set, and can identify the candidate method pointed to by the obtained information among the MPM method pair set. The identified candidate method can be determined to be the decoder-side IPMD derivation method to be applied to the current sub-partition.

[0446] According to the embodiments described above, the video decoding device (2000) can determine a decoder-side IPM derivation method to be used for each sub-partition. The video decoding device (2000) can also determine an intra prediction mode for each sub-partition according to the decoder-side IPM derivation method determined for the sub-partitions. Accordingly, by performing intra prediction on the intra prediction mode determined for each sub-partition, prediction samples of the sub-partition are generated, and recovery samples can also be generated using the prediction samples.

[0447] FIGS. 42a and 42b illustrate various forms of sub-partitions that can be determined for intra-prediction from a single CU according to one embodiment.

[0448] When a block is divided into sub-partitions, not only are sub-partitions created by dividing in a vertical or horizontal direction possible, but various forms of sub-partitions shown in FIG. 42a and 42b are also possible.

[0449] Within a single encoding unit, sub-partitions may have the same shape or different sizes. Additionally, symmetric and asymmetric partitioning are possible. Information regarding the number of sub-partitions to be divided into may also be signaled via the bitstream.

[0450] Above, a method for a video decoding device (2000) to utilize an intra prediction mode derived per sub-partition, and in particular to utilize a decoder-side IPM derivation method per sub-partition, has been described in detail. A video encoding method for generating a bitstream containing information that enables a decoder to utilize an intra prediction mode derived per sub-partition and to utilize a decoder-side IPM derivation method per sub-partition is described in detail below with reference to FIGS. 43 and 44.

[0451] FIG. 43 is a block diagram showing the configuration of a video encoding device according to one embodiment.

[0452] Referring to FIG. 43, the video encoding device (4300) may include a processor (4310) and a memory (4320).

[0453] In one embodiment of the present disclosure, the processor (4310) may include processing circuits and / or multiple processors. For example, the processor (4310) may include various processing circuits including at least one processor, and at least one of the at least one processor may be configured to perform the various functions described in the present disclosure individually and / or collectively in a distributed manner.

[0454] In one embodiment of the present disclosure, the memory (4320) may include one or more storage media that store at least one instruction. The processor (4310) may control the video encoding device (4300) by executing the instruction stored in the memory (4320). For example, the processor (4310) may control the video encoding device (4300) to perform an operation by executing the instruction stored in the memory (4320) individually or collectively. In one embodiment of the present disclosure, the operation performed by the video encoding device (4300) may be an operation performed by the processor (4310) of the video encoding device (4300).

[0455] In one embodiment of the present disclosure, the video encoding device (4300) may correspond to the image encoding device (200) shown in FIG. 2 and / or the encoding unit (1910) shown in FIG. 19.

[0456] The video encoding device (4300) can determine the prediction mode of the current block. The current block may be a maximum encoding unit, encoding unit, conversion unit, or prediction unit divided from the current picture to be encoded.

[0457] In one embodiment, the prediction mode of the current block may be determined to be an intra mode or an inter mode.

[0458] In one embodiment, the video encoding device (4300) can determine the intra prediction mode of the current block when the prediction mode of the current block is an intra mode.

[0459] The intra prediction mode of the current block may be any one of a plurality of intra prediction modes. As described with reference to FIG. 21, the plurality of intra prediction modes may include a non-directional intra mode and a directional intra mode.

[0460] In one embodiment, the video encoding device (4300) performs intra prediction on the current block according to the intra prediction mode of the current block, and can encode the current block using the prediction block generated as a result of performing the intra prediction.

[0461] In one embodiment, encoding of the current block may mean the process of generating information that enables the video encoding device (4300) to restore the current block. The information generated through encoding may be included in a bitstream.

[0462] In one embodiment, the video encoding device (4300) may generate residual samples corresponding to the difference between the predicted samples of the prediction block and the original samples of the current block. If the prediction block is determined to be the current block, residual data may not be generated. The video encoding device (4300) may generate quantized transformation coefficients by performing transformation and quantization on the residual samples, which are the difference between the predicted samples generated from the intra-prediction result and the residual samples. The video encoding device (4300) may encode encoding information corresponding to the encoding methods used to generate the quantized transformation coefficients. The video encoding device (4300) may generate bit sequences by performing entropy encoding on the transformation coefficients and encoding information, and generate a bitstream containing the bit sequences according to the order of the syntax elements.

[0463] In order to restore an image using an intra prediction mode derived per sub-partition in a video decoding device (2000) according to one embodiment, the video encoding device (4300) must first perform intra prediction in the same manner as the video decoding device (2000) and generate a bitstream containing residual data and encoding information using the prediction samples generated as a result. Accordingly, the video encoding device (4300) according to one embodiment can also derive an intra prediction mode per sub-partition and perform intra prediction using the intra prediction mode derived per sub-partition. In addition, the intra prediction mode may be determined by determining the decoder-side IPM derivation method per sub-partition. Hereinafter, while describing a method for encoding an image using an intra prediction mode derived per sub-partition in a video encoding device (4300), descriptions that overlap with previously described content or concepts are omitted to explain the operation performed in the video decoding device (2000), but it should be noted that they are identical to the previously described technology and concepts.

[0464] A video encoding device (4300) according to one embodiment can generate first information indicating whether an intra prediction mode induced by sub-partitions in the current block is used.

[0465] A video encoding device (4300) according to one embodiment can determine the intra prediction mode of a first sub-partition among the sub-partitions of the current block based on surrounding information of the first sub-partition when an intra prediction mode derived per sub-partition is used.

[0466] The video encoding device (4300) can obtain residual samples of the first sub-partition using prediction samples obtained by performing intra prediction based on the intra prediction mode of the first sub-partition.

[0467] The video encoding device (4300) can determine the intra prediction mode of a second sub-partition adjacent to a first sub-partition among the sub-partitions of the current block using the prediction result of the first sub-partition and the surrounding information of the second sub-partition.

[0468] The video encoding device (4300) can obtain residual samples of the second sub-partition using prediction samples obtained by performing intra prediction based on the intra prediction mode of the second sub-partition.

[0469] The video encoding device (4300) can generate conversion coefficients for residual samples of a first sub-partition and conversion coefficients for residual samples of a second sub-partition. The video encoding device (4300) can generate a bitstream including the generated conversion coefficients and the first information. The generated bitstream can be transmitted over a network or stored in a storage medium.

[0470] When the size of a block is large or the textures or features within the block are heterogeneous, the accuracy of the prediction result may decrease when the intra prediction mode is derived only from surrounding reference samples or reference templates surrounding the block. In contrast, the video encoding device (4300) according to one embodiment performs prediction according to an individual intra prediction mode for each sub-partition divided from a block or encoding unit, so that the intra prediction can be performed by considering the unique textures or features of each sub-partition, thereby yielding a more accurate prediction result. In addition, since there is no need to include information for determining the intra prediction mode for each sub-partition in the bitstream, there is an effect of reducing the amount of data that must be transmitted through the bitstream.

[0471] FIG. 44 illustrates a flowchart of a video encoding method according to one embodiment.

[0472] In step 4410, a video encoding device (4300) according to one embodiment may generate first information indicating whether an intra prediction mode derived per sub-partition is used in the current block. The current block may be an encoding unit, a maximum encoding unit, a prediction unit, or a conversion unit.

[0473] In step 4420, the video encoding device (4300) according to one embodiment can determine the intra prediction mode of the first sub-partition among the sub-partitions of the current block based on the surrounding information of the first sub-partition when the intra prediction mode derived per sub-partition is used. The first sub-partition may be the left, upper, right, or lower sub-partition closest to the first restored surrounding blocks adjacent to the current block among the plurality of sub-partitions included in the current block.

[0474] In step 4430, a video encoding device (4300) according to one embodiment can obtain residual samples of the first sub-partition using prediction samples obtained by performing intra prediction based on an intra prediction mode of the first sub-partition.

[0475] A video encoding device (4300) according to one embodiment determines samples of a surrounding restoration block adjacent to the first sub-partition as at least one reference sample based on an intra prediction mode of the first sub-partition, and can determine prediction samples of the first sub-partition using the reference sample. Difference samples between prediction samples corresponding to the original samples of the first sub-partition may be residual samples.

[0476] In step 4440, a video encoding device (4300) according to one embodiment may determine the intra prediction mode of a second sub-partition adjacent to a first sub-partition among the sub-partitions of the current block using the prediction result of the first sub-partition and the surrounding information of the second sub-partition. The prediction result of the first sub-partition may include the intra prediction mode of the first sub-partition, prediction samples, or information used to determine the intra prediction mode of the first sub-partition. The surrounding information of the second sub-partition may be information related to samples adjacent to the second sub-partition among the information of the first restored surrounding block adjacent to the current block and the restored samples. Accordingly, the intra prediction mode of the second sub-partition may be determined by considering the restored information of the first sub-partition located inside the current block and the restored information of the surrounding block located outside the current block.

[0477] The surrounding information of the first sub-partition used to determine the intra prediction mode of the first sub-partition may include not only reference samples or templates of surrounding blocks adjacent to the first sub-partition, but also reference samples or reference templates located at a distance greater than a predetermined distance from the first sub-partition. The predetermined distance may be a distance of 10 to 20 pixels.

[0478] Similarly, the surrounding information of the second sub-partition used to determine the intra-prediction mode of the second sub-partition may also include a reference sample or reference template located at a distance greater than a predetermined distance from the second sub-partition.

[0479] In step 4450, a video encoding device (4300) according to one embodiment can obtain residual samples of the second sub-partition using prediction samples obtained by performing intra prediction based on an intra prediction mode of the second sub-partition.

[0480] A video encoding device (4300) according to one embodiment can determine at least one reference sample for surrounding reconstructed samples adjacent to the second sub-partition based on an intra prediction mode of the second sub-partition, and can determine prediction samples of the first sub-partition using the reference sample. At this time, the surrounding reconstructed samples used for intra prediction of the second sub-partition may include the reconstructed samples of the first sub-partition and the reconstructed samples of surrounding blocks adjacent to the outside of the current block in the second sub-partition. Difference samples between the prediction samples corresponding to the original samples of the second sub-partition may be obtained as residual samples.

[0481] In step 4460, a video encoding device (4300) according to one embodiment can generate transformation coefficients for residual samples of a first sub-partition and transformation coefficients for residual samples of a second sub-partition.

[0482] In the case where the current block includes multiple sub-partitions and a third sub-partition adjacent to the second sub-partition exists, the video encoding device (4300) may use at least one of the intra prediction mode of the second sub-partition, prediction samples and recovery samples, and information of surrounding blocks to induce an intra prediction mode of the third sub-partition. The video encoding device (4300) may obtain prediction samples by performing intra prediction on the third sub-partition using the induced intra prediction mode of the third sub-partition. Residual samples, which are the difference values ​​between the original samples and the prediction samples corresponding to the third sub-partition, may be generated.

[0483] The video encoding device (4300) can obtain residual samples of the current block that include all residual samples of the sub-partitions included in the current block. Since the current block can be divided into transformation blocks, the residual samples of the current block can also be divided according to the transformation blocks. Transformation and quantization can be performed on the residual samples included in each transformation block.

[0484] A video encoding device (4300) according to one embodiment can individually determine an intra prediction mode induction type for inducing an intra prediction mode for each sub-partition. The intra prediction mode induction type may include types according to the previously described DIMD, TIMD, CLP, and EIP methods.

[0485] The video encoding device (4300) may separately determine a first intra prediction mode induction type for determining the intra prediction mode of the first sub-partition based on surrounding information of the first sub-partition, and a second intra prediction mode induction type for determining the intra prediction mode of the second sub-partition based on the prediction result of the first sub-partition and surrounding information of the second sub-partition. Accordingly, the first intra prediction mode induction type of the first sub-partition and the second intra prediction mode induction type of the second sub-partition may be the same or different.

[0486] A video encoding device (4300) according to one embodiment may use a set of intra prediction induction type candidates including two or more intra prediction induction type candidates. Among the set of intra prediction induction type candidates, a first intra prediction mode induction type is determined, and according to the first intra prediction mode induction type, the intra prediction mode of the first sub-partition may be determined based on the surrounding information of the first sub-partition. Additionally, among the set of intra prediction induction type candidates, a second intra prediction mode induction type is determined, and according to the second intra prediction mode induction type, the intra prediction mode of the second sub-partition may be determined using the prediction result of the first sub-partition and the surrounding information of the second sub-partition.

[0487] After determining whether to perform intra prediction according to the same intra prediction mode for each conversion block subpartition according to the ISP mode, the video encoding device (4300) can determine whether to use the intra prediction mode derived per subpartition according to the first information. Accordingly, the video encoding device (4300) according to one embodiment can encode the first information indicating whether the intra prediction mode derived per subpartition is used in the current block after encoding the information regarding whether intra prediction is performed according to the ISP mode in the current block.

[0488] A video encoding device (4300) according to one embodiment can determine whether to use an intra prediction mode derived per sub-partition for the chroma block of the current block. The video encoding device (4300) can encode second information indicating whether an intra prediction mode derived per sub-partition for the chroma block of the current block is used.

[0489] A video encoding device (4300) according to one embodiment may determine whether an intra prediction mode derived for each sub-partition of the current block is used, and then determine whether to determine an intra prediction mode of the chroma block using the intra prediction mode of the lumina block of the current block. The intra prediction mode of the chroma block may be determined using the intra prediction mode of the lumina block or may be determined individually.

[0490] A video encoding device (4300) according to one embodiment may determine whether the intra prediction mode of the chroma block of the current block is determined based on the intra prediction mode of the lumina block, and then determine whether the intra prediction mode derived for each sub-partition for the chroma block is used.

[0491] A video encoding device (4300) according to one embodiment may determine that an intra prediction mode derived per sub-partition of the current block is not used when the difference in sample values ​​of surrounding samples of the current block is smaller than a predetermined value. In this case, the first information may not be generated either.

[0492] A video encoding device (4300) according to one embodiment can individually utilize a method for deriving an intra prediction mode by using a reference sample or reference template, etc., which is reconstructed around a sub-partition, thereby reducing the amount of computation required to derive the intra prediction mode and increasing the accuracy of the prediction result. Since intra prediction is performed according to the intra prediction mode derived for each sub-partition divided from the current block, multiple intra prediction modes individually determined for each of the multiple sub-partitions within a single encoding unit can be utilized, thereby enabling more accurate prediction.

[0493] The operation of a video encoding device (4300) for implementing the embodiments of FIGS. 28 to 42 is described in detail below. If the video encoding device (4300) performs the same operation as the video decoding device (2000), the description may be omitted to avoid redundant description.

[0494] In FIG. 28, a video encoding device (4300) according to one embodiment can divide the current block (2800) into sub-partitions SP1, SP2, SP3, SP4 and sequentially induce an intra prediction mode for the sub-partitions SP1, SP2, SP3, SP4. By using a reference sample or reference template located near each sub-partition SP1, SP2, SP3, SP4, an intra prediction mode determined individually for each sub-partition can be determined.

[0495] The video encoding device (4300) may use the intra prediction mode of the sub-partition that is derived first among the sub-partitions or the reconstructed sample generated as a prediction result to derive the intra prediction mode of the sub-partitions. Thus, the intra prediction mode derived first can be used to derive the intra prediction mode of the next sub-partition.

[0496] First, the video encoding device (4300) can induce an intra prediction mode of sub-partition SP1 using samples (2810, 2820) adjacent to SP1 among surrounding samples that are adjacent to the current block (2800) and restored earlier than the current block.

[0497] A video encoding device (4300) according to one embodiment can induce an intra prediction mode of sub-partition SP2 using samples (2840) restored before the current block (2800) and samples (2830) restored from sub-partition SP1 adjacent to sub-partition SP2.

[0498] A video encoding device (4300) according to one embodiment can induce an intra prediction mode of sub-partition SP3 using samples (2860) restored before the current block (2800) and samples (2850) restored from sub-partition SP2 adjacent to sub-partition SP3.

[0499] A video encoding device (4300) according to one embodiment can induce an intra prediction mode of sub-partition SP4 using samples (2880) restored before the current block (2800) and samples (2870) restored from sub-partition SP3 adjacent to sub-partition SP4.

[0500] In a video encoding device (4300) according to one embodiment, reference samples can be determined according to the induced intra-prediction mode of sub-partition SP1. Prediction samples of sub-partition SP1 are determined using the reference samples, and residual samples of sub-partition SP1 can also be determined based on the prediction samples.

[0501] Among the surrounding samples restored before the current block (2800) and the samples (2830, 2840) adjacent to sub-partition SP2 among the restored samples of sub-partition SP1, reference samples can be determined according to the induced intra prediction mode of sub-partition SP2. Predicted samples of sub-partition SP2 are determined using the reference samples, and based on the predicted samples, residual samples of sub-partition SP2 can also be determined.

[0502] Among the surrounding samples restored before the current block (2800) and the samples (2850, 2860) adjacent to sub-partition SP3 among the restored samples of sub-partition SP2, reference samples can be determined according to the induced intra-prediction mode of sub-partition SP3. Predicted samples of sub-partition SP3 are determined using the reference samples, and based on the predicted samples, residual samples of sub-partition SP3 can also be determined.

[0503] Additionally, among the surrounding samples restored before the current block (2800) and the samples (2870, 2880) adjacent to sub-partition SP4 among the restored samples of sub-partition SP3, reference samples can be determined according to the induced intra-prediction mode of sub-partition SP4. Predicted samples of sub-partition SP4 are determined using the reference samples, and based on the predicted samples, residual samples of sub-partition SP4 can also be determined.

[0504] In FIG. 29, a video encoding device (4300) according to one embodiment illustrates a method of inducing an intra-prediction mode of a sub-partition using globally referenceable surrounding information.

[0505] A video encoding device (4300) according to one embodiment can induce an intra prediction mode of sub-partition SP1 using global peripheral samples (2910, 2920). In addition,

[0506] Among the global surrounding samples restored before the current block (2900), global surrounding samples (2940, 2945), excluding the region adjacent to sub-partition SP1, can be used to induce an intra prediction mode of sub-partition SP2. The video encoding device (4300) can induce an intra prediction mode of sub-partition SP2 using the restored samples (2930) of sub-partition SP1 adjacent to sub-partition SP2 and global surrounding samples (2940, 2945).

[0507] Among the global surrounding samples restored before the current block (2900), global surrounding samples (2945, 2960), excluding the regions adjacent to sub-partitions SP1 and SP2, can be used to induce an intra prediction mode of sub-partition SP3. The video encoding device (4300) can induce an intra prediction mode of sub-partition SP3 using the restored samples (2950) of sub-partition SP2 adjacent to sub-partition SP3 and global surrounding samples (2945, 2960).

[0508] In a similar manner, among the global surrounding samples restored before the current block (2900), global surrounding samples (2945, 2980) excluding the regions adjacent to sub-partitions SP1, SP2, and SP3 can be used to induce an intra prediction mode of sub-partition SP4. The video encoding device (4300) can induce an intra prediction mode of sub-partition SP4 using the restored samples (2970) of sub-partition SP3 adjacent to sub-partition SP4 and global surrounding samples (2945, 2980).

[0509] According to FIG. 30, a video encoding device (4300) according to one embodiment can divide the current block (3000) into sub-blocks (3010, 3030, 3050, 3070) used according to the ISP mode, even when an ISP mode is applied to the current block (3000), and can sequentially induce an intra prediction mode for each sub-block. At this time, the sub-block may be an encoding unit or a conversion unit. Since the first, second, third, and fourth intra prediction modes are all determined separately, multiple intra prediction modes may be determined even within the current block (3000). A video encoding device (4300) according to one embodiment can also sequentially perform intra prediction of the sub-blocks (3010, 3030, 3050, 3070).

[0510] A video encoding device (4300) according to one embodiment can individually determine an intra prediction mode for sub-blocks created in various division forms, as well as sub-blocks divided in the horizontal direction or sub-blocks divided in the vertical direction.

[0511] According to FIG. 31, the current block (3100) may be divided into a plurality of sub-blocks (3110, 3120, 3130, 3140). The video encoding device (4300) may determine an intra prediction mode for each sub-block. The video encoding device (4300) may determine an intra prediction mode (3115) based on a first reference region (3160) of a first sub-block (3110). The video encoding device (4300) may obtain prediction samples by performing a prediction for the first sub-block (3110) based on the determined intra prediction mode. The video encoding device (4300) may determine residual samples, which are the difference between the prediction samples of the first sub-block (3110) and the original samples.

[0512] In one embodiment, the video encoding device (4300) may determine an intra prediction mode (3125) based on a second reference region (3170) of a second sub-block (3120). The second reference region (3170) may include restored samples of a first sub-block (3110). The video encoding device (4300) may determine an intra prediction mode of the second sub-block (3120) based on the intra prediction mode (3110) of the first sub-block (3110) or the restored samples. The video encoding device (4300) may obtain predicted samples by performing a prediction for the second sub-block (3120) based on the determined intra prediction mode (3125). The video encoding device (4300) may obtain residual samples, which are the difference between the predicted samples of the second sub-block (3120) and the original samples.

[0513] In one embodiment, the video encoding device (4300) can determine an intra prediction mode (3135, 3145) for each of the third sub-block (3130) and the fourth sub-block (3140), just as it does for the first sub-block (3110) and the second sub-block (3120), perform a prediction, and obtain prediction samples and residual samples.

[0514] A video encoding device (4300) according to one embodiment can induce an intra prediction mode of a current sub-partition (sub-block) by using information of a sub-partition (sub-block) restored before the current sub-partition (sub-block) and information of the current sub-partition (sub-block). At this time, the information of the current sub-partition (sub-block) that can be used to induce an intra prediction mode of the sub-partition may include the channel, height, size, aspect ratio, and location of the sub-partition (sub-block). In addition, information regarding the current subpartition (subblock) that can be used to derive the intra prediction mode of the subpartition may include the intra prediction mode of the block / coding unit containing the subpartition (subblock), the type of the frame / slice to which the block / coding unit containing the subpartition (subblock) belongs, the transformation type (MTS: Multiple Transform Selection / LFNST: Low-Frequency Non-Separable Transform / NSPT: Non-Separable Secondary Transform), the prediction method of the current block, the final predictor sample value, and the information used to generate the final predictor. Specific examples of the information used to generate the referenceable final predictor have been described above and are therefore not listed redundantly.

[0515] As shown in FIG. 32, when the difference value abs(maxTop - minTop) between reference samples in the upper reference area (3210) of the current block (3200) and the difference value abs(maxTop - minTop) between reference samples in the left reference area (3220) are both smaller than a predetermined threshold FLAT_THRESHOLD, the video encoding device (4300) does not induce and use an intra prediction mode individually for each sub-partition for the current block (3200). In this case, the same intra prediction mode can be used without determining an intra prediction mode for each sub-partition, and there is no need to encode first information indicating whether the intra prediction mode induced for each sub-partition is being used.

[0516] In FIG. 33, the video encoding device (4300) can determine an intra-prediction mode per sub-partition using all reference samples or reference templates located at a distance.

[0517] The video encoding device (4300) can induce an intra prediction mode of sub-partition SP1 using reconstructed samples (3310, 3320) located at a distance from the current block (3300). At this time, the distance may be defined as a distance of 10, 12, 15, 20 pixels, etc. from the upper boundary or left boundary of the current block (3300). As reference samples or reference templates for inducing an intra prediction mode of sub-partition SP2, reconstructed samples (3310) located at a predetermined distance in the vertical direction from the upper boundary of the current block (3300) and reconstructed samples (3330) located at a predetermined distance in the horizontal direction from the left boundary of sub-partition SP2 may be used.

[0518] Similarly, as reference samples or reference templates for inducing the intra prediction mode of sub-partition SP3, restoration samples (3310) located at a distant upper side and restoration samples (3340) located at a predetermined distance in the horizontal direction from the left boundary of sub-partition SP3 may be used.

[0519] Similarly, as reference samples or reference templates for inducing the intra prediction mode of sub-partition SP4, restoration samples (3310) located at a distant upper side and restoration samples (3350) located at a predetermined distance in the horizontal direction from the left boundary of sub-partition SP4 may be used.

[0520] Referring to FIGS. 34, 35, 36, 37, 38, 39 and 40, various forms of syntax for transmitting information indicating whether a video encoding device (4300) induces an intra prediction mode per sub-partition are described.

[0521] Syntax (3400) of FIG. 34 represents an embodiment in which the syntax element Spimd_flag is transmitted before the syntax element intra_mip_flag is transmitted. Accordingly, according to the syntax (3400), the video encoding device (4300) may first transmit the syntax element Spimd_flag to determine whether to determine the intra prediction mode induced per sub-partition of the current block in order to perform intra prediction for the current block. If Spimd_flag is 1 (3410), the intra prediction mode induced per sub-partition can be performed sequentially. If Spimd_flag is 0 (3420), the syntax element intra_mip_flag is transmitted, and whether to perform intra prediction according to the MIP mode can be indicated. Therefore, in this case, intra prediction according to the SPIMD mode may be possible regardless of whether the intra prediction of the current block follows the MIP mode.

[0522] Syntax (3500) of FIG. 35 illustrates an embodiment in which the syntax element Spimd_flag is transmitted after the syntax element intra_mip_flag is transmitted and before the syntax element intra_luma_ref_idx is transmitted. Accordingly, according to the syntax (3500), the video encoding device (4300) can first transmit the syntax element intra_mip_flag to perform intra prediction for the current block, and if intra_mip_flag is 1 (3510), it can be indicated to perform intra prediction according to the MIP mode. If intra_mip_flag is 0 (3520), the syntax element Spimd_flag is transmitted, and if Spimd_flag is 1, it can be indicated to determine the intra prediction mode derived per sub-partition of the current block and perform intra prediction sequentially. If Spimd_flag is 0 (3530), the video encoding device (4300) can transmit the syntax element intra_luma_ref_idx and indicate that intra prediction should be performed based on the prediction direction indicated by intra_luma_ref_idx. Thus, in this case, intra prediction according to SPIMD mode is possible when the intra prediction of the current block does not follow MIP mode, and the intra prediction mode can be induced regardless of the prediction direction indicated by intra_luma_ref_idx. intra_luma_ref_idx may point to a distant reference sample such as in FIG. 33. However, according to the syntax (3500), since whether SPIMD is applied is determined before the transmission of intra_luma_ref_idx, a distant reference sample cannot be used in SPIMD.

[0523] Syntax (3600) of FIG. 36 illustrates an embodiment in which the syntax element Spimd_flag is transmitted after the syntax element intra_luma_ref_idx is transmitted and before the syntax element intra_subpartitions_mode_flag is transmitted. Accordingly, according to the syntax (3600), the video encoding device (4300) first transmits the syntax element intra_mip_flag to perform intra prediction for the current block, so that if intra_mip_flag is 1 (3610), it indicates that intra prediction can be performed according to the MIP mode. If intra_mip_flag is 0 (3620), the video encoding device (4300) can transmit the syntax element Spimd_flag after transmitting the syntax element intra_luma_ref_idx. If Spimd_flag is 1, the syntax (3600) may indicate that an intra prediction mode is induced for each subpartition of the current block, and that intra prediction is performed sequentially according to the intra prediction mode induced for the subpartitions. If Spimd_flag is 0 (3630), the syntax element intra_subpartitions_mode_flag is transmitted, and whether to perform intra prediction according to the ISP mode on the current block may be determined according to the intra_subpartitions_mode_flag. Thus, according to the syntax (3600), an intra_luma_ref_idx indicating whether to refer to a distant surrounding recovery sample is transmitted, so that the intra prediction mode of the subpartition can be induced using a distant recovery sample. However, if the intra prediction of the current block follows the SPIMD mode, the ISP mode cannot be applied.

[0524] Syntax (3700) of FIG. 37 illustrates an embodiment in which the syntax element Spimd_flag is transmitted after the syntax element intra_subpartitions_mode_flag is transmitted and before the syntax element intra_luma_mpm_flag is transmitted. According to the syntax (3700), the video encoding device (4300) can first transmit the syntax element intra_mip_flag to perform intra prediction for the current block, and if intra_mip_flag is 1 (3710), it can be indicated to perform intra prediction according to the MIP mode. If intra_mip_flag is 0 (3720), the syntax element intra_luma_ref_idx is transmitted to determine the prediction direction indicated by intra_luma_ref_idx, and the syntax element intra_subpartitions_mode_flag is transmitted to determine whether to perform intra prediction according to the ISP mode on the current block according to intra_subpartitions_mode_flag, and if the ISP mode is applied, the partitioning form of the subpartition can be determined. Additionally, the video encoding device (4300) can transmit the syntax element Spimd_flag after determining whether to apply the ISP mode. The syntax (3700) can indicate that if Spimd_flag is 1, the intra prediction mode is induced for each subpartition of the current block, and intra prediction is performed sequentially according to the intra prediction mode induced for the subpartitions. If Spimd_flag is 0 (3730), intra prediction of the current block may be performed based on the intra prediction direction pointed to by intra_luma_ref_idx or the intra prediction mode determined based on the MPM list. However, if the current block performs intra prediction according to the ISP mode, it may be divided into sub-partitions according to the ISP mode, and an intra prediction mode may be induced for each sub-partition according to the Spimd mode.In addition, the intra prediction mode can be determined per subpartition by considering not only the nearby reconstructed samples pointed to by intra_luma_ref_idx but also the distant nearby reconstructed samples.

[0525] Syntax (3800) of FIG. 38 represents an embodiment in which the syntax element Spimd_flag is transmitted instead of the syntax element intra_luma_mpm_remainder being transmitted after the syntax element intra_luma_mpm_flag is transmitted. Thus, according to the syntax (3800), the video encoding device (4300) can first transmit the syntax element intra_mip_flag to perform intra prediction for the current block, so that if intra_mip_flag is 1 (3810), intra prediction is performed according to the MIP mode. Syntax (3800) may indicate that if intra_mip_flag is 0, the syntax element intra_luma_ref_idx is transmitted to determine the prediction direction indicated by intra_luma_ref_idx, the syntax element intra_subpartitions_mode_flag is transmitted to determine whether to perform intra prediction according to the ISP mode on the current block according to intra_subpartitions_mode_flag, and if the ISP mode is applied, the subpartition splitting form is determined. Additionally, the video encoding device (4300) may determine whether to apply the ISP mode and then transmit the syntax element intra_luma_mpm_flag to determine whether to determine the intra prediction mode using the MPM list (3820). If intra_luma_mpm_flag is 0 through the syntax (3800) (3830), the syntax element Spimd_flag is transmitted, and if Spimd_flag is 1, an intra prediction mode is induced for each sub-partition of the current block, and intra prediction can be performed sequentially according to the intra prediction mode induced for the sub-partitions.If Spimd_flag is 0, intra_luma_mpm_remainder is transmitted, and intra prediction of the current block can be performed according to the intra prediction mode determined from the MPM list using intra_luma_mpm_idx and intra_luma_mpm_remainder.

[0526] According to FIG. 39, the video encoding device (4300) may transmit a Spimd_flag for the chroma block of the current block before the cclm_mode_flag is transmitted. If Spimd_flag is 1, the cclm_mode_flag is not transmitted, and a chroma intra prediction mode may be derived for each sub-partition separately from the luminance intra prediction mode of the current block. If Spimd_flag is 0, the cclm_mode_flag is transmitted, and if the cclm_mode_flag is 1, the cclm_mode_idx is transmitted. In this case, the cclm_mode_idx may be encoded to indicate how the chroma intra prediction mode of the current block is determined using the luminance intra prediction mode.

[0527] According to FIG. 40, the video encoding device (4300) can transmit Spimd_flag instead of intra_chroma_pred_mode after transmitting cclm_mode_flag for the chroma block of the current block. Thus, when cclm_mode_flag is 1, the chroma intra prediction mode is determined by referencing the lumina intra prediction mode according to the CCLM (Cross-Component Linear Model) mode, so there is no need to directly derive the chroma intra prediction mode for each sub-partition. When cclm_mode_flag is 0, Spimd_flag is transmitted, and the chroma intra prediction mode is determined separately from the lumina intra prediction mode of the current block, and the intra prediction mode for each sub-partition can be determined individually according to Spimd_flag.

[0528] According to FIG. 41, the video encoding device (4300) can determine a method for directly deriving an intra prediction mode for each sub-partition. However, the derivation method proposed in this disclosure is not limited to only DIMD, TIMD, and CLP types. The video encoding device (4300) can compare the costs resulting from determining the intra prediction mode according to various decoder-side IPM derivation methods sequentially for the sub-partitions SP1, SP2, SP3, and SP4 of the current block (4100). As a result, the decoder-side IPM derivation method that generates the lowest cost to determine the intra prediction mode can be determined. When the video encoding device (4300) according to one embodiment compares the costs generated by applying various combinations of decoder-side IPM derivation methods to the sub-partitions SP1, SP2, SP3, and SP4, the DIMD method can be selected for SP1, the TIMD method for SP2, and the CLP method for SP3 and SP4.

[0529] A video encoding device (4300) according to one embodiment may configure a SPIMD method candidate set including a plurality of decoder-side IPM derivation method candidates to be used for each sub-partition. By performing an operation to compare SAD or SATD according to derivation methods based on the SPIMD method candidate set for each sub-partition, the decoder-side IPM derivation method of each sub-partition is determined, and the final predictor accordingly can be determined as the intra prediction mode of each sub-partition.

[0530] Additionally, the video encoding device (4300) can form a predetermined MPM method pair set within the SPIMD method candidate set. By comparing the SAD / SATD costs of the candidate derivation methods included in the SPIMD method candidate set, an MPM method pair set composed of two candidate derivation methods that generate the lowest cost can be determined. The video encoding device (4300) can determine the decoder-side IPMD derivation method to be applied to the current sub-partition from the MPM method pair set...

Claims

1. In a video decoding method, A step of obtaining first information from a bitstream (2710) indicating whether an intra prediction mode induced by sub-partitions in the current block is being used; When an intra prediction mode derived for each sub-partition according to the first information is used, a step of determining (2720) the intra prediction mode of the first sub-partition among the sub-partitions of the current block based on the surrounding information of the first sub-partition; A step of obtaining restoration samples of the first subpartition (2730) using prediction samples obtained by performing intra prediction based on the intra prediction mode of the first subpartition; A step of determining (2740) the intra prediction mode of a second sub-partition adjacent to the first sub-partition among the sub-partitions of the current block above using the prediction result of the first sub-partition and surrounding information of the second sub-partition; A step of obtaining restoration samples of the second subpartition (2750) using prediction samples obtained by performing intra prediction based on the intra prediction mode of the second subpartition; and A video decoding method characterized by including the step of restoring the current block (2760) based on the restoration samples of the first sub-partition and the restoration samples of the second sub-partition.

2. In Paragraph 1, A video decoding method characterized by separately determining a first intra prediction mode induction type for determining the intra prediction mode of the first sub-partition based on surrounding information of the first sub-partition, and a second intra prediction mode induction type for determining the intra prediction mode of the second sub-partition based on the prediction result of the first sub-partition and surrounding information of the second sub-partition.

3. In either Paragraph 1 or Paragraph 2, A set of intra-predicted-derived type candidates including two or more intra-predicted-derived type candidates is determined, and Among the above set of intra prediction induction type candidates, the first intra prediction mode induction type is determined, and according to the first intra prediction mode induction type, the intra prediction mode of the first subpartition is determined based on the surrounding information of the first subpartition, and A video decoding method characterized in that, among the above set of intra prediction induction type candidates, the second intra prediction mode induction type is determined, and according to the second intra prediction mode induction type, the intra prediction mode of the second sub-partition is determined using the prediction result of the first sub-partition and the surrounding information of the second sub-partition.

4. In any one of paragraphs 1 through 3, The surrounding information of the first subpartition used to determine the intra prediction mode of the first subpartition includes a reference sample or reference template located at a distance greater than a predetermined distance from the first subpartition, and A video decoding method characterized in that the surrounding information of the second sub-partition used to determine the intra prediction mode of the second sub-partition includes a reference sample or reference template located at a distance greater than a predetermined distance from the second sub-partition.

5. In any one of paragraphs 1 through 4, A video decoding method characterized in that the first information is obtained after it is determined whether intra prediction is performed on the transformation block subpartitions divided from the current block.

6. In any one of paragraphs 1 to 5, the video decoding method is, A video decoding method characterized by further including the step of obtaining second information indicating whether an intra prediction mode derived per sub-partition is used for the chroma block of the current block.

7. In any one of claims 1 to 6, the step of obtaining the information from the bitstream is, A video decoding method characterized by including a step of determining that an intra prediction mode derived per sub-partition of the current block is not used when the difference in sample values ​​of surrounding samples of the current block is smaller than a predetermined value.

8. In a video encoding method, A step of generating first information (4410) indicating whether an intra prediction mode induced by sub-partitions in the current block is being used; When the intra prediction mode derived for each of the above sub-partitions is used, the step of determining (4420) the intra prediction mode of the first sub-partition among the sub-partitions of the current block based on the surrounding information of the first sub-partition; A step of obtaining residual samples of the first sub-partition (4430) using prediction samples obtained by performing intra prediction based on the intra prediction mode of the first sub-partition; A step of determining (4440) the intra prediction mode of a second sub-partition adjacent to the first sub-partition among the sub-partitions of the current block using the prediction result of the first sub-partition and surrounding information of the second sub-partition; A step of obtaining residual samples of the second sub-partition (4450) using prediction samples obtained by performing intra prediction based on the intra prediction mode of the second sub-partition; and A video encoding method characterized by including the step of generating (4460) conversion coefficients of residual samples of the first sub-partition and conversion coefficients of residual samples of the second sub-partition.

9. In Paragraph 8, A video encoding method characterized by separately determining a first intra prediction mode induction type for determining the intra prediction mode of the first sub-partition based on surrounding information of the first sub-partition, and a second intra prediction mode induction type for determining the intra prediction mode of the second sub-partition based on the prediction result of the first sub-partition and surrounding information of the second sub-partition.

10. In either paragraph 8 or paragraph 9, A set of intra-predicted-derived type candidates including two or more intra-predicted-derived type candidates is determined, and Among the above set of intra prediction induction type candidates, the first intra prediction mode induction type is determined, and according to the first intra prediction mode induction type, the intra prediction mode of the first subpartition is determined based on the surrounding information of the first subpartition, and A video encoding method characterized in that, among the above set of intra prediction induction type candidates, the second intra prediction mode induction type is determined, and according to the second intra prediction mode induction type, the intra prediction mode of the second sub-partition is determined using the prediction result of the first sub-partition and the surrounding information of the second sub-partition.

11. In any one of paragraphs 8 through 10, The surrounding information of the first subpartition used to determine the intra prediction mode of the first subpartition includes a reference sample or reference template located at a distance greater than a predetermined distance from the first subpartition, and A video encoding method characterized in that the surrounding information of the second subpartition used to determine the intra-prediction mode of the second subpartition includes a reference sample or reference template located at a distance greater than a predetermined distance from the second subpartition.

12. In any one of paragraphs 8 through 11, A video encoding method characterized in that the first information is encoded after it is determined whether intra prediction is performed on the transformation block subpartitions divided from the current block.

13. In any one of paragraphs 8 to 12, the video encoding method is, A video encoding method characterized by further including a step of encoding second information indicating whether an intra prediction mode derived per sub-partition is used for the chroma block of the current block.

14. In any one of paragraphs 8 to 13, the step of generating the first information is A video encoding method characterized by including a step of determining that an intra prediction mode derived per sub-partition of the current block is not used when the difference in sample values ​​of surrounding samples of the current block is smaller than a predetermined value.

15. A method for transmitting a bitstream generated by encoding video, A step of generating first information (4410) indicating whether an intra prediction mode induced by sub-partitions in the current block is being used; When the intra prediction mode derived for each of the above sub-partitions is used, the step of determining (4420) the intra prediction mode of the first sub-partition among the sub-partitions of the current block based on the surrounding information of the first sub-partition; A step of obtaining residual samples of the first sub-partition (4430) using prediction samples obtained by performing intra prediction based on the intra prediction mode of the first sub-partition; A step of determining (4440) the intra prediction mode of a second sub-partition adjacent to the first sub-partition among the sub-partitions of the current block using the prediction result of the first sub-partition and surrounding information of the second sub-partition; A step of obtaining residual samples of the second sub-partition (4450) using prediction samples obtained by performing intra prediction based on the intra prediction mode of the second sub-partition; and A method characterized by including the step of outputting a bitstream comprising the first information, the transformation coefficients of the residual samples of the first sub-partition, and the transformation coefficients of the residual samples of the second sub-partition.

Images