VIDEO DECODING METHODS, VIDEO ENCRYPTION METHODS, AND COMPUTER-READABLE INVARIABLE RECORDING MATERIALS

Abstract

The invention relates to a method for decoding video, a method for encoding video, and a computer-readable, invariant recording medium, such that the method for decoding video comprises the following steps: receiving from a bit stream information that enables the first interstructure prediction mode indicating whether the first interstructure prediction mode for predicting motion vectors is enabled for the data unit above; when the first interstructure prediction mode is enabled for the data unit above based on the information enabling the first interstructure prediction mode, receiving, from a bit stream, information on the correction distance range indicating one of several candidate sets of correction distances; when the first interstructure prediction mode is enabled for the data unit above, determining whether the first interstructure prediction mode is applied to the current block contained in the data unit above; receiving from a bit stream information on the correction distance and information on the correction direction;Determine the current correction distance using information about the correction distance and the correction distance range; determine the current correction direction using information about the correction direction; and determine the current block's motion vector by correcting the base motion vector according to the current correction distance and current correction direction.

Description

Encoding method and device thereof, decoding method and device thereof

[0001] The present disclosure relates to a method for encoding and decoding a video, and more particularly, to a method for efficiently encoding and decoding information about a motion vector.

[0002] High-quality video requires a large amount of data to encode. However, the bandwidth available for transmitting video data is limited, which can limit the data rate applied during video data transmission. Therefore, to efficiently transmit video data, a method for encoding and decoding video data that increases compression ratio while minimizing image quality degradation is needed.

[0003] Video data can be compressed by removing spatial and temporal redundancy between pixels. Because adjacent pixels typically share common characteristics, encoding information is transmitted as data units composed of pixels to eliminate redundancy between adjacent pixels.

[0004] The pixel values ​​of the pixels contained in a data unit are not transmitted directly, but rather the method required to obtain the pixel values ​​is transmitted. A prediction method that predicts pixel values ​​similar to the original values ​​is determined for each data unit, and encoding information for the prediction method is transmitted from the encoder to the decoder. Furthermore, because the predicted values ​​are not completely identical to the original values, residual data regarding the difference between the original and predicted values ​​is transmitted from the encoder to the decoder.

[0005] As prediction accuracy increases, the encoding information required to specify the prediction method increases, but the size of the residual data decreases. Therefore, the prediction method is determined by considering both the encoding information and the size of the residual data. In particular, data units partitioned from a picture vary in size. Larger data units are more likely to decrease prediction accuracy, but also reduce encoding information. Therefore, block sizes are determined based on the characteristics of the picture.

[0006] Prediction methods also include intraprediction and interprediction. Intraprediction predicts pixels within a block from the pixels surrounding the block. Interprediction predicts pixels by referencing pixels in other pictures referenced by the picture containing the block. Therefore, intraprediction removes spatial redundancy, and interprediction removes temporal redundancy.

[0007] As the number of prediction methods increases, the amount of coding information required to represent the prediction methods also increases. Therefore, the coding information applied to a block can also be predicted from other blocks, reducing its size.

[0008] Since loss of video data is acceptable to the extent that human vision cannot perceive it, the amount of residual data can be reduced by lossy compression through the conversion and quantization process of residual data.

[0009] A video encoding method and a video encoding device for performing inter prediction according to the UMVE mode are disclosed. In addition, a video decoding method and a video decoding device for performing inter prediction according to the UMVE mode are disclosed. In addition, a computer-readable recording medium recording a program for executing the video encoding method and the video decoding method according to an embodiment of the present disclosure on a computer is disclosed.

[0010] In the present disclosure, a video decoding method is provided, including the steps of: determining whether an upper data unit including a current block is allowed to have an UMVE mode; determining whether the UMVE mode is applied to the current block if the UMVE mode is allowed to the upper data unit; determining a basic motion vector of the current block if the UMVE mode is applied to the current block; determining a correction size and a correction direction for correcting the basic motion vector; determining a motion vector of the current block by correcting the basic motion vector according to the correction size and the correction direction; and restoring the current block based on the motion vector of the current block.

[0011] In the present disclosure, a video decoding device is provided, including a processor that determines whether an upper data unit including a current block is allowed to have an UMVE mode, and if the UMVE mode is allowed for the upper data unit, determines whether the UMVE mode is applied to the current block, determines a basic motion vector of the current block if the UMVE mode is applied to the current block, determines a correction magnitude and a correction direction for correction of the basic motion vector, corrects the basic motion vector according to the correction magnitude and the correction direction to determine a motion vector of the current block, and restores the current block based on the motion vector of the current block.

[0012] A video encoding method is provided, comprising: a step of determining whether an upper data unit including a current block allows UMVE mode; a step of determining a motion vector for the current block; a step of determining whether the UMVE mode is applied to the current block based on the motion vector if the UMVE mode is allowed for the upper data unit; a step of determining a base motion vector of the current block and a correction size and a correction direction for correction of the base motion vector based on the motion vector; and a step of outputting a bitstream including information about the UMVE mode of the current block.

[0013] A video encoding device is provided, comprising a processor that determines whether an upper data unit including a current block allows UMVE mode, determines a motion vector for the current block, determines whether the UMVE mode is applied to the current block based on the motion vector if the UMVE mode is allowed for the upper data unit, determines a base motion vector of the current block and a correction size and a correction direction for correction of the base motion vector based on the motion vector, and outputs a bitstream including encoding information according to the UMVE mode of the current block.

[0014] A computer-recordable recording medium is provided, in which a program for performing the above video encoding method and video decoding method is recorded.

[0015] The technical tasks to be achieved by this embodiment are not limited to the technical tasks described above, and other technical tasks can be inferred from the following embodiments.

[0016] By inter-predicting blocks included in an image according to the UMVE mode, the encoding rate of the image can be improved.

[0017] FIG. 1A illustrates a block diagram of an image encoding device based on encoding units according to a tree structure according to one embodiment of the present disclosure.

[0018] FIG. 1b illustrates a block diagram of an image decoding device based on encoding units according to a tree structure according to one embodiment.

[0019] FIG. 2 illustrates a process in which a current encoding unit is divided to determine at least one encoding unit according to one embodiment.

[0020] FIG. 3 illustrates a process in which a non-square coding unit is divided to determine at least one coding unit according to one embodiment.

[0021] FIG. 4 illustrates a process in which an encoding unit is divided based on at least one of block shape information and division shape information according to one embodiment.

[0022] FIG. 5 illustrates a method for determining a predetermined encoding unit among an odd number of encoding units according to one embodiment.

[0023] FIG. 6 illustrates the order in which multiple encoding units are processed when a current encoding unit is divided and multiple encoding units are determined according to one embodiment.

[0024] FIG. 7 illustrates a process in which, when encoding units cannot be processed in a predetermined order according to one embodiment, it is determined that the current encoding unit is divided into an odd number of encoding units.

[0025] FIG. 8 illustrates a process in which a first encoding unit is divided to determine at least one encoding unit according to one embodiment.

[0026] FIG. 9 illustrates that, according to one embodiment, when a non-square shape of a second encoding unit determined by splitting a first encoding unit satisfies a predetermined condition, the shapes into which the second encoding unit can be split are limited.

[0027] FIG. 10 illustrates a process of splitting a square-shaped encoding unit when the split shape information cannot indicate splitting into four square-shaped encoding units according to one embodiment.

[0028] FIG. 11 illustrates that, according to one embodiment, the processing order between multiple encoding units may vary depending on the process of dividing the encoding units.

[0029] FIG. 12 illustrates a process in which the depth of an encoding unit is determined as the shape and size of the encoding unit change when the encoding unit is recursively divided to determine a plurality of encoding units according to one embodiment.

[0030] FIG. 13 illustrates an index (part index, hereinafter referred to as PID) for depth and encoding unit distinction that can be determined according to the shape and size of encoding units according to one embodiment.

[0031] FIG. 14 illustrates that a plurality of coding units are determined according to a plurality of predetermined data units included in a picture according to one embodiment.

[0032] FIG. 15 illustrates a processing block that serves as a criterion for determining the order of determination of reference coding units included in a picture according to one embodiment.

[0033] Figure 16 describes an inter prediction method according to UMVE mode.

[0034] Fig. 17 illustrates a block diagram of a video decoding device that performs decoding according to the UMVE mode.

[0035] Figure 18 illustrates motion vector candidates of UMVE mode having a diamond-shaped distribution based on the base motion vector.

[0036] Figure 19 illustrates motion vector candidates of UMVE mode having a rectangular distribution based on the base motion vector.

[0037] Figure 20 illustrates an embodiment of the UMVE mode with different numbers of motion vector candidates for each group.

[0038] Figure 21 illustrates an example of UMVE mode in which the distribution shape of motion vector candidates is different for each group.

[0039] Fig. 22 shows an embodiment of a method for correcting two basic motion vectors according to pair prediction.

[0040] Figure 23 illustrates how to select an inter-mode tool for the current block in skip mode.

[0041] Fig. 24 discloses a method for selecting an inter mode tool of a current block in direct mode when the slice type of the current slice including the current block is P type.

[0042] Fig. 25 discloses a method for selecting an inter mode tool of a current block in direct mode when the slice type of the current slice including the current block is type B.

[0043] FIG. 26 illustrates a flowchart of a decryption method according to one embodiment of the UMVE mode.

[0044] FIG. 27 illustrates a block diagram of a video encoding device performing inter prediction according to one embodiment of the UMVE mode.

[0045] Fig. 28 illustrates a flowchart of an encoding method according to one embodiment of the UMVE mode.

[0046] A video decoding method is provided, comprising: a step of determining whether an upper data unit including a current block allows UMVE mode; a step of determining whether the UMVE mode is applied to the current block if the UMVE mode is allowed to the upper data unit; a step of determining a basic motion vector of the current block if the UMVE mode is applied to the current block; a step of determining a correction magnitude and a correction direction for correction of the basic motion vector; a step of determining a motion vector of the current block by correcting the basic motion vector according to the correction magnitude and the correction direction; and a step of restoring the current block based on the motion vector of the current block.

[0047] The advantages and features of the disclosed embodiments, and the methods for achieving them, will become clearer with reference to the embodiments described below together with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms. These embodiments are provided solely to ensure the complete disclosure of the present disclosure and to fully inform those skilled in the art of the present disclosure of the scope of the invention.

[0048] The terms used in this specification will be briefly explained, and the disclosed embodiments will be described in detail.

[0049] The terms used in this specification have been selected from widely used, current terms, taking into account the functions of the present disclosure. However, these terms may vary depending on the intentions of engineers working in the relevant fields, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this disclosure should not be defined simply as names, but rather based on the meanings of the terms and the overall content of the present disclosure.

[0050] In this specification, singular expressions include plural expressions unless the context clearly specifies that they are singular.

[0051] When a part of the specification is said to "include" a component, unless otherwise stated, this does not exclude other components but rather may include other components. Furthermore, the term "component" as used throughout the specification refers to software or hardware components such as FPGAs or ASICs, and the "component" performs certain functions. However, the "component" is not limited to software or hardware. The "component" may be configured to reside on an addressable storage medium or configured to execute one or more processors. Thus, by way of example, the "component" may include components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. The functionality provided within the components and "components" may be combined into a smaller number of components and "components" or further separated into additional components and "components."

[0052] "Current block" refers to one of the coding unit, prediction unit, and transformation unit currently being encoded or decoded. If, for convenience of explanation, it is necessary to distinguish other types of blocks, such as prediction units and transformation units, "current coding block," "current prediction block," and "current transformation block" may be used. Furthermore, "sub-block" refers to a data unit divided from the "current block." And "upper block" refers to a data unit that includes the "current block."

[0053] Hereinafter, "sample" refers to data assigned to a sampling location in an image and is the subject of processing. For example, pixel values ​​in a spatial domain image or transformation coefficients in a transformation domain may be samples. A unit containing at least one such sample may be defined as a block.

[0054] Below, with reference to the attached drawings, a detailed description of the embodiments is provided so that those skilled in the art can easily implement the present disclosure. Furthermore, in order to clearly illustrate the present disclosure, portions irrelevant to the description are omitted from the drawings.

[0055] FIG. 1a illustrates a block diagram of an image encoding device (100) based on encoding units according to a tree structure according to one embodiment of the present disclosure.

[0056] The video encoding device (100) includes an encoding unit (110) and a bitstream generation unit (120).

[0057] The encoding unit (110) divides a picture or a slice included in a picture into a plurality of maximum encoding units according to the size of the maximum encoding unit. The maximum encoding unit is a data unit having a size of 32x32, 64x64, 128x128, 256x256, etc., and may be a square data unit whose width and height are squares to the power of 2. The encoding unit (110) may provide maximum encoding unit size information indicating the size of the maximum encoding unit to the bitstream generation unit (120). In addition, the bitstream generation unit (120) may include the maximum encoding unit size information in the bitstream.

[0058] The encoding unit (110) determines the encoding unit by splitting the maximum encoding unit. Whether or not the encoding unit is split is determined based on whether the splitting of the encoding unit is efficient through rate-distortion optimization. In addition, splitting information indicating whether the encoding unit has been split can be generated. The splitting information can be expressed in the form of a flag.

[0059] A coding unit can be split in various ways. For example, a square coding unit can be split into four square coding units, each with half the width and half the height. A square coding unit can be split into two rectangular coding units, each with half the width. A square coding unit can be split into two rectangular coding units, each with half the height. A square coding unit can be split into three coding units by splitting the width or height in a 1:2:1 ratio.

[0060] A rectangular coding unit whose width is twice its height can be split into two square coding units. A rectangular coding unit whose width is twice its height can be split into two rectangular coding units whose width is four times its height. A rectangular coding unit whose width is twice its height can be split into two rectangular coding units and one square coding unit by splitting the widths in a ratio of 1:2:1.

[0061] Similarly, a rectangular coding unit whose height is twice its width can be split into two square coding units. Also, a rectangular coding unit whose height is twice its width can be split into two rectangular coding units whose height is four times its width. Similarly, a rectangular coding unit whose height is twice its width can be split into two rectangular coding units and one square coding unit by splitting the heights in a ratio of 1:2:1.

[0062] When two or more segmentation methods are available in the video encoding device (100), information about a segmentation method that can be used for an encoding unit among the segmentation methods available in the video encoding device (100) can be determined for each picture. Accordingly, only specific segmentation methods can be determined to be used for each picture. If the video encoding device (100) uses only one segmentation method, information about a segmentation method that can be used for an encoding unit is not separately determined.

[0063] For a coding unit of a specific size, it can be split using a specific splitting method. For example, if the coding unit size is 256x265, the coding unit can be set to be split only into four square coding units with half the width and half the height.

[0064] When the segmentation information of a coding unit indicates that the coding unit is segmented, segmentation shape information indicating the segmentation method of the coding unit may be generated. If there is only one segmentation method that can be used in the picture to which the coding unit belongs, the segmentation shape information may not be generated. If the segmentation method is adaptively determined based on the coding information surrounding the coding unit, the segmentation shape information may not be generated.

[0065] As described above, the image data of the current picture is divided into maximum coding units according to the maximum size of the coding unit. The maximum coding unit may include coding units hierarchically divided from the maximum coding unit. The shape and position of the lower coding unit may be determined according to the division shape of the upper coding unit. In addition, the minimum size of the coding unit that limits the division of the coding unit may be preset.

[0066] The encoding unit (110) compares the encoding efficiency when the encoding unit is hierarchically divided with the encoding efficiency when the encoding unit is not divided. Then, the encoding unit (110) determines whether to divide the encoding unit based on the comparison result. If it is determined that division of the encoding unit is more efficient, the encoding unit (110) hierarchically divides the encoding unit. If it is determined that not to divide the encoding unit is more efficient based on the comparison result, the encoding unit is not divided. Whether to divide the encoding unit can be determined independently of whether to divide other adjacent encoding units.

[0067] Finally, the segmented coding units can be predicted by intra-prediction or inter-prediction. Intra-prediction is a method of predicting samples of a prediction unit using reference samples surrounding the prediction unit. Inter-prediction is a method of predicting samples of a prediction unit by obtaining reference samples from the reference picture referenced by the current picture.

[0068] The encoding unit (110) can apply multiple intra prediction methods to prediction units for intra prediction, thereby selecting the most efficient intra prediction method. Intra prediction methods include directional modes such as DC mode, planar mode, vertical mode, and horizontal mode.

[0069] Intra prediction can be performed for each prediction unit when the reconstructed samples around the coding unit are used as reference samples. However, when the reconstructed samples within the coding unit are used as reference samples, the reconstruction of the reference samples within the coding unit must have priority over the prediction, so the prediction order of the prediction unit may depend on the transformation order of the transformation unit. Therefore, when the reconstructed samples within the coding unit are used as reference samples, only the intra prediction method for the transformation units corresponding to the prediction unit is determined for the prediction unit, and the actual intra prediction can be performed for each transformation unit.

[0070] The encoding unit (110) can select the most efficient inter prediction method by determining the optimal motion vector and reference picture. The encoding unit determination unit (120) can determine a plurality of motion vector candidates from coding units that are spatially and temporally neighboring from the current coding unit for inter prediction, and determine the most efficient motion vector among them as a motion vector. Similarly, a plurality of reference picture candidates can be determined from coding units that are spatially and temporally neighboring from the current coding unit, and determine the most efficient reference picture among them. According to an embodiment, the reference picture can be determined from among reference picture lists determined in advance for the current picture. According to an embodiment, for the accuracy of prediction, the most efficient motion vector among the plurality of motion vector candidates can be determined as a predicted motion vector, and the motion vector can be determined by correcting the predicted motion vector. Inter prediction can be performed in parallel for each prediction unit within the coding unit.

[0071] The encoding unit (110) can restore the encoding unit by obtaining only information indicating motion vectors and reference pictures according to the skip mode. According to the skip mode, all encoding information, including the residual signal, is omitted except for the information indicating the motion vectors and reference pictures. Since the residual signal is omitted, the skip mode can be used when the prediction accuracy is very high.

[0072] Depending on the prediction method for the prediction unit, the partition mode used may be limited. For example, only the partition mode for the prediction unit of size 2Nx2N and NxN may be applied to intra prediction, whereas the partition mode for the prediction unit of size 2Nx2N, 2NxN, Nx2N and NxN may be applied to inter prediction. In addition, only the partition mode for the prediction unit of size 2Nx2N may be applied to the skip mode of inter prediction. The partition mode allowed for each prediction method in the video encoding device (100) may be changed according to the encoding efficiency.

[0073] The image encoding device (100) can perform conversion based on encoding units. The image encoding device (100) can convert residual data, which is the difference between the original value and the predicted value for pixels included in the encoding unit, through a predetermined process. For example, the image encoding device (100) can perform lossy compression on the residual data through quantization and DCT / DST conversion. Alternatively, the image encoding device (100) can perform lossless compression on the residual data without quantization.

[0074] In conclusion, the encoder (110) determines the most efficient prediction method for the current encoding unit among a plurality of intra prediction methods and inter prediction methods. Then, the encoder (110) determines the prediction method of the current encoding unit according to the encoding efficiency according to the prediction result. Similarly, the encoder (110) can determine the transformation method according to the encoding efficiency according to the transformation result. The encoding efficiency of the encoding unit is ultimately determined according to the most efficient prediction method and transformation method determination method of the encoding unit. The encoder (110) finally determines the hierarchical structure of the maximum encoding unit according to the encoding efficiency of the divided encoding units.

[0075] The encoding unit (110) can measure the encoding efficiency of the encoding unit, the prediction efficiency of the prediction methods, etc. using a rate-distortion optimization technique based on a Lagrangian multiplier.

[0076] The encoding unit (110) can generate partition information indicating whether or not the encoding unit is split according to the hierarchical structure of the determined maximum encoding unit. In addition, the encoding unit (110) can generate partition mode information for determining a prediction unit and transformation unit split information for determining a transformation unit for the encoding unit for which splitting has been completed. In addition, when there are two or more splitting methods for the encoding unit, the encoding unit (110) can generate partition shape information indicating the splitting method together with the splitting information. In addition, the encoding unit (110) can generate information regarding the prediction method and the transformation method used for the prediction unit and the transformation unit.

[0077] The bitstream generation unit (120) can output the information generated by the encoding unit (110) in the form of a bitstream according to the hierarchical structure of the maximum encoding unit.

[0078] The method of determining encoding units, prediction units, and transformation units according to the tree structure of the maximum encoding unit according to one embodiment will be described in detail later with reference to FIGS. 3 to 12.

[0079] FIG. 1b illustrates a block diagram of an image decoding device (150) based on encoding units according to a tree structure according to one embodiment.

[0080] The image decoding device (150) includes a receiving unit (160) and a decoding unit (170).

[0081] The definitions of various terms, such as encoding units, prediction units, transformation units, and various segmentation information, for the decoding operation of the image decoding device (150) according to one embodiment are the same as those described above with reference to FIG. 1 and the image encoding device (100). In addition, since the purpose of the image decoding device (150) is to restore image data, various encoding methods used in the image encoding device (100) can be applied to the image decoding device (150).

[0082] The receiving unit (160) receives and parses a bitstream for an encoded video. The decoding unit (170) extracts information necessary for decoding for each maximum encoding unit from the parsed bitstream and provides the information to the decoding unit (170). The decoding unit (170) can extract information regarding the maximum size of the encoding unit of the current picture from the header, sequence parameter set, or picture parameter set for the current picture.

[0083] In addition, the decoding unit (170) extracts segmentation information for encoding units according to a tree structure for each maximum encoding unit from the parsed bitstream. The extracted segmentation information is output to the decoding unit (170). The decoding unit (170) can determine the tree structure of the maximum encoding unit by segmenting the maximum encoding unit according to the extracted segmentation information.

[0084] The segmentation information extracted by the decoding unit (170) is segmentation information for a tree structure determined by the image encoding device (100) to generate the minimum encoding error. Accordingly, the image decoding device (150) can restore the image by decoding data according to the encoding method that generates the minimum encoding error.

[0085] The decoding unit (170) can extract partition information for data units, such as prediction units and transformation units, included in the encoding unit. For example, the decoding unit (170) can extract information regarding the most efficient partition mode for the prediction unit. In addition, the decoding unit (170) can extract transformation partition information regarding the most efficient tree structure for the transformation unit.

[0086] Additionally, the decoding unit (170) can obtain information on the most efficient prediction method for prediction units divided from an encoding unit. In addition, the decoding unit (170) can obtain information on the most efficient transformation method for transformation units divided from an encoding unit.

[0087] The decoding unit (170) extracts information from the bitstream according to the method of configuring the bitstream in the bitstream generation unit (120) of the video encoding device (100).

[0088] The decoding unit (170) can divide the maximum encoding unit into encoding units having the most efficient tree structure based on the partition information. In addition, the decoding unit (170) can divide the encoding unit into prediction units based on information about the partition mode. The decoding unit (170) can divide the encoding unit into transformation units based on the transformation division information.

[0089] The decoding unit (170) can predict a prediction unit based on information about a prediction method. In addition, the decoding unit (170) can inversely quantize and inversely transform residual data corresponding to the difference between the original value of a pixel and the predicted value based on information about a transformation method. In addition, the decoding unit (170) can restore pixels of an encoding unit based on the prediction result of the prediction unit and the transformation result of the transformation unit.

[0090] FIG. 2 illustrates a process in which an image decoding device (150) divides a current encoding unit to determine at least one encoding unit according to one embodiment.

[0091] According to one embodiment, the image decoding device (150) can determine the shape of an encoding unit using block shape information, and can determine the shape into which the encoding unit is divided using split shape information. That is, the splitting method of the encoding unit indicated by the split shape information can be determined depending on which block shape the block shape information used by the image decoding device (150) indicates.

[0092] According to one embodiment, the image decoding device (150) may use block shape information indicating that the current encoding unit is a square shape. For example, the image decoding device (150) may determine whether to not split a square encoding unit, to split it vertically, to split it horizontally, to split it into four encoding units, etc., based on the split shape information. Referring to FIG. 2, when the block shape information of the current encoding unit (200) indicates a square shape, the decoding unit (180) may not split an encoding unit (210a) having the same size as the current encoding unit (200) based on the split shape information indicating that it is not split, or may determine a split encoding unit (210b, 210c, 210d, etc.) based on the split shape information indicating a predetermined splitting method.

[0093] Referring to FIG. 2, the image decoding device (150) may determine two coding units (210b) into which the current coding unit (200) is vertically divided based on split shape information indicating that the current coding unit (200) is vertically divided according to one embodiment. The image decoding device (150) may determine two coding units (210c) into which the current coding unit (200) is horizontally divided based on split shape information indicating that the current coding unit (200) is horizontally divided. The image decoding device (150) may determine four coding units (210d) into which the current coding unit (200) is vertically and horizontally divided based on split shape information indicating that the current coding unit (200) is vertically and horizontally divided. However, the split shapes into which a square coding unit may be split should not be interpreted as being limited to the above-described shapes, and may include various shapes that the split shape information may indicate. Specified split shapes into which a square coding unit is split will be described in detail through various embodiments below.

[0094] FIG. 3 illustrates a process in which an image decoding device (150) divides a non-square coding unit to determine at least one coding unit according to one embodiment.

[0095] According to one embodiment, the image decoding device (150) may utilize block shape information indicating that the current encoding unit is non-square. The image decoding device (150) may determine whether to not split the current non-square encoding unit or to split it using a predetermined method based on the split shape information. Referring to FIG. 3, when the block shape information of the current encoding unit (300 or 350) indicates a non-square shape, the image decoding device (150) may not split the encoding unit (310 or 360) having the same size as the current encoding unit (300 or 350) according to the split shape information indicating that it is not split, or may determine the split encoding units (320a, 320b, 330a, 330b, 330c, 370a, 370b, 380a, 380b, 380c) based on the split shape information indicating a predetermined splitting method. The predetermined splitting method by which a non-square encoding unit is split will be described in detail through various embodiments below.

[0096] According to one embodiment, the image decoding device (150) may determine a form in which an encoding unit is split using split shape information, and in this case, the split shape information may indicate the number of at least one encoding unit generated by splitting the encoding unit. Referring to FIG. 3, when the split shape information indicates that the current encoding unit (300 or 350) is split into two encoding units, the image decoding device (150) may split the current encoding unit (300 or 350) based on the split shape information to determine two encoding units (320a, 320b, or 370a, 370b) included in the current encoding unit.

[0097] According to one embodiment, when the image decoding device (150) splits a current encoding unit (300 or 350) having a non-square shape based on split shape information, the current encoding unit may be split by considering the position of the long side of the non-square current encoding unit (300 or 350). For example, the image decoding device (150) may split the current encoding unit (300 or 350) in a direction that splits the long side of the current encoding unit (300 or 350) by considering the shape of the current encoding unit (300 or 350) to determine a plurality of encoding units.

[0098] According to one embodiment, when the split shape information indicates that the coding unit is split into an odd number of blocks, the image decoding device (150) may determine an odd number of coding units included in the current coding unit (300 or 350). For example, when the split shape information indicates that the current coding unit (300 or 350) is split into three coding units, the image decoding device (150) may split the current coding unit (300 or 350) into three coding units (330a, 330b, 330c, 380a, 380b, 380c). According to one embodiment, the image decoding device (150) may determine an odd number of coding units included in the current coding unit (300 or 350), and the sizes of the determined coding units may not all be the same. For example, among the determined odd number of coding units (330a, 330b, 330c, 380a, 380b, 380c), the size of a given coding unit (330b or 380b) may have a different size from the other coding units (330a, 330c, 380a, 380c). That is, the coding units that can be determined by dividing the current coding unit (300 or 350) may have multiple types of sizes.

[0099] According to one embodiment, when the split shape information indicates that the coding unit is split into an odd number of blocks, the image decoding device (150) can determine an odd number of coding units included in the current coding unit (300 or 350), and further, the image decoding device (150) can place a predetermined restriction on at least one coding unit among the odd number of coding units generated by splitting. Referring to FIG. 3, the image decoding device (150) can perform a decoding process for a coding unit (330b, 380b) located in the center among three coding units (330a, 330b, 330c, 380a, 380b, 380c) generated by splitting the current coding unit (300 or 350) differently from the decoding process for other coding units (330a, 330c, 380a, 380c). For example, the image decoding device (150) can restrict the encoding unit (330b, 380b) located in the center from being split any further, unlike other encoding units (330a, 330c, 380a, 380c), or can restrict the splitting to be limited to a predetermined number of times.

[0100] FIG. 4 illustrates a process in which an image decoding device (150) divides an encoding unit based on at least one of block shape information and division shape information according to one embodiment.

[0101] According to one embodiment, the image decoding device (150) may determine whether to split or not to split the first coding unit (400) having a square shape into coding units based on at least one of block shape information and split shape information. According to one embodiment, when the split shape information indicates splitting the first coding unit (400) in the horizontal direction, the image decoding device (150) may split the first coding unit (400) in the horizontal direction to determine the second coding unit (410). The first coding unit, the second coding unit, and the third coding unit used according to one embodiment are terms used to understand the relationship before and after splitting between coding units. For example, when the first coding unit is split, the second coding unit may be determined, and when the second coding unit is split, the third coding unit may be determined. Hereinafter, the relationship between the first coding unit, the second coding unit, and the third coding unit used may be understood to follow the above-described characteristics.

[0102] According to one embodiment, the image decoding device (150) may determine to split or not split the determined second encoding unit (410) into encoding units based on at least one of block shape information and split shape information. Referring to FIG. 4, the image decoding device (150) may split the first encoding unit (400) based on at least one of block shape information and split shape information to split the determined second encoding unit (410) of a non-square shape into at least one third encoding unit (420a, 420b, 420c, 420d, etc.) or may not split the second encoding unit (410). The image decoding device (150) can obtain at least one of block shape information and segmentation shape information, and the image decoding device (150) can divide the first encoding unit (400) based on at least one of the obtained block shape information and segmentation shape information to divide the first encoding unit (400) into a plurality of second encoding units (e.g., 410) of various shapes, and the second encoding unit (410) can be divided according to the method in which the first encoding unit (400) is divided based on at least one of the block shape information and the segmentation shape information. According to one embodiment, when the first coding unit (400) is split into second coding units (410) based on at least one of block shape information and split shape information for the first coding unit (400), the second coding unit (410) may also be split into third coding units (e.g., 420a, 420b, 420c, 420d, etc.) based on at least one of block shape information and split shape information for the second coding unit (410). That is, the coding unit may be split recursively based on at least one of split shape information and block shape information related to each coding unit. A method that can be used for recursive splitting of the coding unit will be described later through various embodiments.

[0103] According to one embodiment, the image decoding device (150) may determine to split each of the third encoding units (420a, 420b, 420c, 420d, etc.) into encoding units or not to split the second encoding unit (410) based on at least one of the block shape information and the split shape information. According to one embodiment, the image decoding device (150) may split the second encoding unit (410) having a non-square shape into an odd number of third encoding units (420b, 420c, 420d). The image decoding device (150) may place a predetermined restriction on a predetermined third encoding unit among the odd number of third encoding units (420b, 420c, 420d). For example, the image decoding device (150) may limit the encoding unit (420c) located in the middle among the odd number of third encoding units (420b, 420c, 420d) to not be split any further, or to be split a settable number of times. Referring to FIG. 4, the image decoding device (150) may limit the encoding unit (420c) located in the middle among the odd number of third encoding units (420b, 420c, 420d) included in the non-square second encoding unit (410) to not be split any further, or to be split in a predetermined split form (for example, to be split only into four encoding units or to be split in a form corresponding to the split form of the second encoding unit (410), or to be split only a predetermined number of times (for example, to be split only n times, n>0). However, the above limitations on the encoding unit (420c) located in the middle are merely simple examples and should not be interpreted as being limited to the above-described examples, but should be interpreted as including various limitations that allow the encoding unit (420c) located in the middle to be decoded differently from the other encoding units (420b, 420d).

[0104] According to one embodiment, the image decoding device (150) can obtain at least one of block shape information and segmentation shape information used to segment the current encoding unit from a predetermined location within the current encoding unit.

[0105] According to one embodiment, the image decoding device (150) may select one of the coding units when the current coding unit is divided into a predetermined number of coding units. There may be various methods for selecting one of the multiple coding units, and descriptions of such methods will be provided later through various embodiments.

[0106] According to one embodiment, the image decoding device (150) can divide the current encoding unit into a plurality of encoding units and determine an encoding unit at a predetermined position.

[0107] FIG. 5 illustrates a method for determining an encoding unit at a predetermined position among an odd number of encoding units by an image decoding device (150) according to one embodiment.

[0108] According to one embodiment, the image decoding device (150) may use information indicating the positions of each of the odd-numbered coding units to determine the coding unit located in the middle of the odd-numbered coding units. Referring to FIG. 5, the image decoding device (150) may divide the current coding unit (500) to determine odd-numbered coding units (520a, 520b, 520c). The image decoding device (150) may use information about the positions of the odd-numbered coding units (520a, 520b, 520c) to determine the middle coding unit (520b). For example, the image decoding device (150) can determine the coding unit (520b) located in the center by determining the positions of the coding units (520a, 520b, 520c) based on information indicating the positions of predetermined samples included in the coding units (520a, 520b, 520c). Specifically, the image decoding device (150) can determine the coding unit (520b) located in the center by determining the positions of the coding units (520a, 520b, 520c) based on information indicating the positions of samples (530a, 530b, 530c) at the upper left of the coding units (520a, 520b, 520c).

[0109] According to one embodiment, information indicating the position of the upper left samples (530a, 530b, 530c) included in each of the coding units (520a, 520b, 520c) may include information on the position or coordinates of the coding units (520a, 520b, 520c) within the picture. According to one embodiment, information indicating the position of the upper left samples (530a, 530b, 530c) included in each of the coding units (520a, 520b, 520c) may include information indicating the width or height of the coding units (520a, 520b, 520c) included in the current coding unit (500), and this width or height may correspond to information indicating the difference between the coordinates within the picture of the coding units (520a, 520b, 520c). That is, the image decoding device (150) can determine the encoding unit (520b) located in the center by directly using information about the position or coordinates of the encoding units (520a, 520b, 520c) within the picture or by using information about the width or height of the encoding unit corresponding to the difference between the coordinates.

[0110] According to one embodiment, information indicating the position of the sample (530a) at the upper left of the upper encoding unit (520a) may represent (xa, ya) coordinates, information indicating the position of the sample (530b) at the upper left of the middle encoding unit (520b) may represent (xb, yb) coordinates, and information indicating the position of the sample (530c) at the upper left of the lower encoding unit (520c) may represent (xc, yc) coordinates. The image decoding device (150) may determine the middle encoding unit (520b) using the coordinates of the samples (530a, 530b, 530c) at the upper left included in the encoding units (520a, 520b, 520c), respectively. For example, when the coordinates of the samples (530a, 530b, 530c) at the upper left are sorted in ascending or descending order, the encoding unit (520b) including the coordinates (xb, yb) of the sample (530b) located in the center can be determined as the encoding unit located in the center among the encoding units (520a, 520b, 520c) determined by dividing the current encoding unit (500). However, the coordinates indicating the position of the upper left samples (530a, 530b, 530c) can indicate coordinates indicating the absolute position within the picture, and further, based on the position of the upper left sample (530a) of the upper coding unit (520a), the (dxb, dyb) coordinates, which are information indicating the relative position of the sample (530b) at the upper left of the middle coding unit (520b), and the (dxc, dyc) coordinates, which are information indicating the relative position of the sample (530c) at the upper left of the lower coding unit (520c), can be used. In addition, the method of determining the coding unit of a given position by using the coordinates of the corresponding sample as information indicating the position of the sample included in the coding unit should not be interpreted as being limited to the above-described method, but should be interpreted as various arithmetic methods that can utilize the coordinates of the sample.

[0111] According to one embodiment, the image decoding device (150) can divide the current encoding unit (500) into a plurality of encoding units (520a, 520b, 520c), and select an encoding unit among the encoding units (520a, 520b, 520c) according to a predetermined criterion. For example, the image decoding device (150) can select an encoding unit (520b) having a different size among the encoding units (520a, 520b, 520c).

[0112] According to one embodiment, the image decoding device (150) may determine the width or height of each of the encoding units (520a, 520b, 520c) by using the (xa, ya) coordinate, which is information indicating the position of the sample (530a) at the upper left of the upper encoding unit (520a), the (xb, yb) coordinate, which is information indicating the position of the sample (530b) at the upper left of the middle encoding unit (520b), and the (xc, yc) coordinate, which is information indicating the position of the sample (530c) at the upper left of the lower encoding unit (520c). The image decoding device (150) may determine the size of each of the encoding units (520a, 520b, 520c) by using the (xa, ya), (xb, yb), (xc, yc), which are coordinates indicating the positions of the encoding units (520a, 520b, 520c).

[0113] According to one embodiment, the image decoding device (150) may determine the width of the upper coding unit (520a) as xb-xa and the height as yb-ya. According to one embodiment, the image decoding device (150) may determine the width of the middle coding unit (520b) as xc-xb and the height as yc-yb. According to one embodiment, the image decoding device (150) may determine the width or height of the lower coding unit using the width or height of the current coding unit and the widths and heights of the upper coding unit (520a) and the middle coding unit (520b). The image decoding device (150) may determine a coding unit having a different size from other coding units based on the widths and heights of the determined coding units (520a, 520b, 520c). Referring to FIG. 5, the image decoding device (150) can determine the middle encoding unit (520b) having a different size from the sizes of the upper encoding unit (520a) and the lower encoding unit (520c) as the encoding unit of a predetermined position. However, the process by which the image decoding device (150) described above determines the encoding unit having a different size from other encoding units is merely an embodiment of determining the encoding unit of a predetermined position using the size of the encoding unit determined based on sample coordinates, and therefore, various processes of determining the encoding unit of a predetermined position by comparing the sizes of the encoding units determined according to predetermined sample coordinates can be used.

[0114] However, the location of the sample considered for determining the location of the encoding unit should not be interpreted as being limited to the upper left corner described above, and it can be interpreted that information on the location of any sample included in the encoding unit can be used.

[0115] According to one embodiment, the image decoding device (150) may select an encoding unit at a predetermined position from among an odd number of encoding units determined by splitting the current encoding unit, taking into consideration the shape of the current encoding unit. For example, if the current encoding unit has a non-square shape in which the width is longer than the height, the image decoding device (150) may determine an encoding unit at a predetermined position in the horizontal direction. That is, the image decoding device (150) may determine one of the encoding units whose positions vary in the horizontal direction and place a restriction on the corresponding encoding unit. If the current encoding unit has a non-square shape in which the height is longer than the width, the image decoding device (150) may determine an encoding unit at a predetermined position in the vertical direction. That is, the image decoding device (150) may determine one of the encoding units whose positions vary in the vertical direction and place a restriction on the corresponding encoding unit.

[0116] According to one embodiment, the image decoding device (150) may use information indicating the positions of each of the even-numbered coding units to determine the coding unit at a predetermined position among the even-numbered coding units. The image decoding device (150) may determine the even-numbered coding units by dividing the current coding unit, and may use information about the positions of the even-numbered coding units to determine the coding unit at a predetermined position. A specific process for this may correspond to a process of determining a coding unit at a predetermined position (e.g., a center position) among the odd-numbered coding units described above in FIG. 5, and thus will be omitted.

[0117] According to one embodiment, when a current encoding unit having a non-square shape is split into a plurality of encoding units, in order to determine an encoding unit at a predetermined position among the plurality of encoding units, predetermined information about the encoding unit at a predetermined position may be used during the splitting process. For example, the image decoding device (150) may use at least one of block shape information and split shape information stored in a sample included in a middle encoding unit during the splitting process in order to determine an encoding unit located in the middle among the encoding units into which the current encoding unit is split.

[0118] Referring to FIG. 5, the image decoding device (150) can divide the current encoding unit (500) into a plurality of encoding units (520a, 520b, 520c) based on at least one of block shape information and split shape information, and can determine an encoding unit (520b) located in the middle among the plurality of encoding units (520a, 520b, 520c). Furthermore, the image decoding device (150) can determine an encoding unit (520b) located in the middle by considering a position where at least one of the block shape information and the split shape information is acquired. That is, at least one of the block shape information and the split shape information of the current encoding unit (500) can be obtained from a sample (540) located in the center of the current encoding unit (500), and when the current encoding unit (500) is split into a plurality of encoding units (520a, 520b, 520c) based on at least one of the block shape information and the split shape information, the encoding unit (520b) including the sample (540) can be determined as the encoding unit located in the center. However, the information used to determine the encoding unit located in the center should not be interpreted as being limited to at least one of the block shape information and the split shape information, and various types of information can be used in the process of determining the encoding unit located in the center.

[0119] According to one embodiment, predetermined information for identifying a coding unit at a predetermined position may be obtained from a predetermined sample included in the coding unit to be determined. Referring to FIG. 5, the image decoding device (150) may use at least one of block shape information and split shape information obtained from a sample at a predetermined position within the current coding unit (500) (e.g., a sample located at the center of the current coding unit (500)) to determine a coding unit at a predetermined position among a plurality of coding units (520a, 520b, 520c) determined by splitting the current coding unit (500) (e.g., a coding unit located at the center of the coding units split into multiple units). That is, the image decoding device (150) can determine the sample at the predetermined position by considering the block shape of the current encoding unit (500), and the image decoding device (150) can determine an encoding unit (520b) that includes a sample from which predetermined information (for example, at least one of block shape information and division shape information) can be obtained, among a plurality of encoding units (520a, 520b, 520c) determined by dividing the current encoding unit (500), and can set a predetermined restriction. Referring to FIG. 5, according to one embodiment, the image decoding device (150) can determine a sample (540) located at the center of the current encoding unit (500) as a sample from which predetermined information can be obtained, and the image decoding device (150) can set a predetermined restriction on the encoding unit (520b) that includes such a sample (540) during the decoding process. However, the location of the sample from which certain information can be obtained should not be interpreted as being limited to the above-described location, but may be interpreted as samples at any location included in the encoding unit (520b) to be determined in order to set a limitation.

[0120] According to one embodiment, the position of a sample from which predetermined information can be obtained may be determined according to the shape of the current encoding unit (500). According to one embodiment, the block shape information may determine whether the shape of the current encoding unit is square or non-square, and may determine the position of a sample from which predetermined information can be obtained according to the shape. For example, the image decoding device (150) may use at least one of information about the width and information about the height of the current encoding unit to determine a sample located on a boundary that divides at least one of the width and the height of the current encoding unit in half as a sample from which predetermined information can be obtained. As another example, when the block shape information related to the current encoding unit indicates that the shape is non-square, the image decoding device (150) may determine one of the samples adjacent to the boundary that divides the long side of the current encoding unit in half as a sample from which predetermined information can be obtained.

[0121] According to one embodiment, when the image decoding device (150) divides the current encoding unit into a plurality of encoding units, the image decoding device (150) may use at least one of block shape information and split shape information to determine an encoding unit at a predetermined position among the plurality of encoding units. According to one embodiment, the image decoding device (150) may obtain at least one of the block shape information and the split shape information from a sample at a predetermined position included in the encoding unit, and the image decoding device (150) may divide the plurality of encoding units generated by dividing the current encoding unit by using at least one of the split shape information and the block shape information obtained from a sample at a predetermined position included in each of the plurality of encoding units. That is, the encoding unit may be recursively divided by using at least one of the block shape information and the split shape information obtained from a sample at a predetermined position included in each of the encoding units. Since the recursive division process of the encoding unit has been described above with reference to FIG. 4, a detailed description thereof will be omitted.

[0122] According to one embodiment, the image decoding device (150) can determine at least one encoding unit by dividing the current encoding unit, and can determine the order in which the at least one encoding unit is decoded according to a predetermined block (e.g., the current encoding unit).

[0123] FIG. 6 illustrates the order in which multiple encoding units are processed when, according to one embodiment, an image decoding device (150) divides a current encoding unit to determine multiple encoding units.

[0124] According to one embodiment, the image decoding device (150) may determine second encoding units (610a, 610b) by vertically dividing the first encoding unit (600) according to block shape information and split shape information, determine second encoding units (630a, 630b) by horizontally dividing the first encoding unit (600), or determine second encoding units (650a, 650b, 650c, 650d) by vertically and horizontally dividing the first encoding unit (600).

[0125] Referring to FIG. 6, the image decoding device (150) can determine the order in which the second encoding units (610a, 610b) determined by vertically dividing the first encoding unit (600) are processed in the horizontal direction (610c). The image decoding device (150) can determine the order in which the second encoding units (630a, 630b) determined by horizontally dividing the first encoding unit (600) are processed in the vertical direction (630c). The image decoding device (150) can determine the second encoding units (650a, 650b, 650c, 650d) determined by dividing the first encoding unit (600) in the vertical and horizontal directions according to a predetermined order (e.g., raster scan order or z scan order (650e), etc.) in which encoding units located in one row are processed and then encoding units located in the next row are processed.

[0126] According to one embodiment, the image decoding device (150) can recursively split the encoding units. Referring to FIG. 6, the image decoding device (150) can split the first encoding unit (600) to determine a plurality of encoding units (610a, 610b, 630a, 630b, 650a, 650b, 650c, 650d), and can recursively split each of the determined plurality of encoding units (610a, 610b, 630a, 630b, 650a, 650b, 650c, 650d). A method for splitting a plurality of coding units (610a, 610b, 630a, 630b, 650a, 650b, 650c, 650d) may be a method corresponding to a method for splitting the first coding unit (600). Accordingly, the plurality of coding units (610a, 610b, 630a, 630b, 650a, 650b, 650c, 650d) may each be independently split into a plurality of coding units. Referring to FIG. 6, the image decoding device (150) may split the first coding unit (600) in the vertical direction to determine the second coding units (610a, 610b), and further may determine to independently split or not split each of the second coding units (610a, 610b).

[0127] According to one embodiment, the image decoding device (150) may horizontally divide the second encoding unit (610a) on the left into third encoding units (620a, 620b), and may not divide the second encoding unit (610b) on the right.

[0128] According to one embodiment, the processing order of the coding units may be determined based on the splitting process of the coding units. In other words, the processing order of the split coding units may be determined based on the processing order of the coding units immediately before splitting. The image decoding device (150) may determine the processing order of the third coding units (620a, 620b) determined by splitting the second coding unit (610a) on the left, independently from the second coding unit (610b) on the right. Since the second coding unit (610a) on the left is split horizontally and the third coding units (620a, 620b) are determined, the third coding units (620a, 620b) may be processed in the vertical direction (620c). In addition, since the order in which the second encoding unit (610a) on the left and the second encoding unit (610b) on the right are processed corresponds to the horizontal direction (610c), the right encoding unit (610b) can be processed after the third encoding units (620a, 620b) included in the second encoding unit (610a) on the left are processed in the vertical direction (620c). Since the above-described content is intended to explain the process in which the processing order of the encoding units is determined according to the encoding units before splitting, it should not be interpreted as being limited to the above-described embodiment, but should be interpreted as being used in various ways in which encoding units that are split and determined in various forms can be independently processed according to a predetermined order.

[0129] FIG. 7 illustrates a process in which, according to one embodiment, an image decoding device (150) determines that a current encoding unit is divided into an odd number of encoding units when the encoding units cannot be processed in a predetermined order.

[0130] According to one embodiment, the image decoding device (150) may determine that the current encoding unit is split into an odd number of encoding units based on the acquired block shape information and split shape information. Referring to FIG. 7, a first encoding unit (700) having a square shape may be split into second encoding units (710a, 710b) having a non-square shape, and the second encoding units (710a, 710b) may each be independently split into third encoding units (720a, 720b, 720c, 720d, 720e). According to one embodiment, the image decoding device (150) can determine a plurality of third encoding units (720a, 720b) ​​by horizontally dividing the left encoding unit (710a) among the second encoding units, and can divide the right encoding unit (710b) into an odd number of third encoding units (720c, 720d, 720e).

[0131] According to one embodiment, the image decoding device (150) can determine whether there is an odd number of split encoding units by determining whether the third encoding units (720a, 720b, 720c, 720d, 720e) can be processed in a predetermined order. Referring to FIG. 7, the image decoding device (150) can determine the third encoding units (720a, 720b, 720c, 720d, 720e) by recursively splitting the first encoding unit (700). The image decoding device (150) can determine whether the first encoding unit (700), the second encoding unit (710a, 710b), or the third encoding unit (720a, 720b, 720c, 720d, 720e) is divided into an odd number of encoding units based on at least one of the block shape information and the split shape information. For example, the encoding unit located on the right side of the second encoding unit (710a, 710b) can be split into an odd number of third encoding units (720c, 720d, 720e). The order in which the plurality of encoding units included in the first encoding unit (700) are processed can be a predetermined order (e.g., z-scan order (730)), and the image decoding device (150) can determine whether the third encoding unit (720c, 720d, 720e) determined by dividing the second encoding unit (710b) on the right into odd numbers satisfies the condition that it can be processed according to the predetermined order.

[0132] According to one embodiment, the image decoding device (150) can determine whether the third encoding units (720a, 720b, 720c, 720d, 720e) included in the first encoding unit (700) satisfy a condition that they can be processed in a predetermined order, and the condition is related to whether at least one of the width and height of the second encoding unit (710a, 710b) is split in half according to the boundary of the third encoding unit (720a, 720b, 720c, 720d, 720e). For example, the third coding unit (720a, 720b) ​​determined by dividing the height of the left second coding unit (710a) of a non-square shape in half satisfies the condition, but the boundaries of the third coding units (720c, 720d, 720e) determined by dividing the right second coding unit (710b) into three coding units do not divide the width or height of the right second coding unit (710b) in half, so the third coding units (720c, 720d, 720e) may be determined not to satisfy the condition, and the image decoding device (150) may determine that the scan order is disconnected in the case of such condition dissatisfaction, and may determine that the right second coding unit (710b) is divided into an odd number of coding units based on the determination result. According to one embodiment, when the image decoding device (150) is divided into an odd number of coding units, a predetermined restriction may be placed on the coding unit at a predetermined position among the divided coding units. Since the contents of such restrictions or predetermined positions, etc. have been described above through various embodiments, a detailed description thereof will be omitted.

[0133] FIG. 8 illustrates a process in which an image decoding device (150) divides a first encoding unit (800) to determine at least one encoding unit, according to one embodiment. According to one embodiment, the image decoding device (150) may divide the first encoding unit (800) based on at least one of block shape information and split shape information obtained through the receiver (160). The first encoding unit (800) having a square shape may be divided into four square-shaped encoding units or may be divided into a plurality of non-square encoding units. For example, referring to FIG. 8, when the block shape information indicates that the first encoding unit (800) is square and the split shape information indicates that the first encoding unit (800) is divided into non-square encoding units, the image decoding device (150) may divide the first encoding unit (800) into a plurality of non-square encoding units. Specifically, when the split shape information indicates that the first encoding unit (800) is split in the horizontal or vertical direction to determine an odd number of encoding units, the image decoding device (150) can split the square-shaped first encoding unit (800) into second encoding units (810a, 810b, 810c) determined by splitting them in the vertical direction into an odd number of encoding units or second encoding units (820a, 820b, 820c) determined by splitting them in the horizontal direction.

[0134] According to one embodiment, the image decoding device (150) can determine whether the second encoding units (810a, 810b, 810c, 820a, 820b, 820c) included in the first encoding unit (800) satisfy a condition that they can be processed in a predetermined order, and the condition is related to whether at least one of the width and height of the first encoding unit (800) is split in half according to the boundary of the second encoding unit (810a, 810b, 810c, 820a, 820b, 820c). Referring to FIG. 8, since the boundaries of the second encoding units (810a, 810b, 810c) determined by vertically dividing the first encoding unit (800) having a square shape do not divide the width of the first encoding unit (800) in half, it may be determined that the first encoding unit (800) does not satisfy the condition that it can be processed in a predetermined order. In addition, since the boundaries of the second encoding units (820a, 820b, 820c) determined by horizontally dividing the first encoding unit (800) having a square shape do not divide the width of the first encoding unit (800) in half, it may be determined that the first encoding unit (800) does not satisfy the condition that it can be processed in a predetermined order. If such a condition is not satisfied, the image decoding device (150) may determine that the scan order is disconnected, and based on the determination result, may determine that the first encoding unit (800) is divided into an odd number of encoding units. According to one embodiment, the image decoding device (150) may place a predetermined restriction on an encoding unit at a predetermined position among the divided encoding units when the encoding unit is divided into an odd number of encoding units. Since the contents of such restrictions or predetermined positions have been described above through various embodiments, a detailed description thereof will be omitted.

[0135] According to one embodiment, the image decoding device (150) can divide the first encoding unit to determine encoding units of various shapes.

[0136] Referring to FIG. 8, the image decoding device (150) can divide the first encoding unit (800) having a square shape and the first encoding unit (830 or 850) having a non-square shape into encoding units of various shapes.

[0137] FIG. 9 illustrates that, according to one embodiment, a video decoding device (150) limits the shapes into which a second encoding unit can be divided when a non-square shape of a second encoding unit determined by dividing a first encoding unit (900) satisfies a predetermined condition.

[0138] According to one embodiment, the image decoding device (150) may determine to split the first coding unit (900) having a square shape into second coding units (910a, 910b, 920a, 920b) having a non-square shape based on at least one of the block shape information and the split shape information obtained through the receiver (160). The second coding units (910a, 910b, 920a, 920b) may be split independently. Accordingly, the image decoding device (150) may determine to split or not split into a plurality of coding units based on at least one of the block shape information and the split shape information related to each of the second coding units (910a, 910b, 920a, 920b). According to one embodiment, the image decoding device (150) may determine third encoding units (912a, 912b) by horizontally dividing the left second encoding unit (910a) having a non-square shape determined by vertically dividing the first encoding unit (900). However, when the image decoding device (150) horizontally divides the left second encoding unit (910a), the right second encoding unit (910b) may be restricted from being horizontally divided in the same direction as the direction in which the left second encoding unit (910a) is divided. If the second encoding unit on the right (910b) is split in the same direction to determine the third encoding unit (914a, 914b), the second encoding unit on the left (910a) and the second encoding unit on the right (910b) may be independently split in the horizontal direction to determine the third encoding unit (912a, 912b, 914a, 914b). However, this is the same result as the image decoding device (150) splitting the first encoding unit (900) into four square-shaped second encoding units (930a, 930b, 930c, 930d) based on at least one of the block shape information and the split shape information, which may be inefficient in terms of image decoding.

[0139] According to one embodiment, the image decoding device (150) may determine third coding units (922a, 922b, 924a, 924b) by vertically dividing a second coding unit (920a or 920b) having a non-square shape determined by dividing the first coding unit (330) in the horizontal direction. However, when the image decoding device (150) vertically divides one of the second coding units (e.g., the upper second coding unit (920a)), the other second coding units (e.g., the lower coding unit (920b)) may be restricted from being vertically divided in the same direction as the direction in which the upper second coding unit (920a) is divided, for the reasons described above.

[0140] FIG. 10 illustrates a process in which an image decoding device (150) divides a square-shaped encoding unit when the segmentation shape information cannot indicate that the encoding unit is divided into four square-shaped encoding units according to one embodiment.

[0141] According to one embodiment, the image decoding device (150) may split the first encoding unit (1000) based on at least one of block shape information and split shape information to determine second encoding units (1010a, 1010b, 1020a, 1020b, etc.). The split shape information may include information about various shapes into which the encoding unit may be split, but the information about various shapes may not include information for splitting into four encoding units in a square shape. According to such split shape information, the image decoding device (150) cannot split the first encoding unit (1000) in a square shape into four second encoding units in a square shape (1030a, 1030b, 1030c, 1030d). Based on the segmentation shape information, the image decoding device (150) can determine a second encoding unit (1010a, 1010b, 1020a, 1020b, etc.) of a non-square shape.

[0142] According to one embodiment, the image decoding device (150) can independently split the second encoding units (1010a, 1010b, 1020a, 1020b, etc.) having a non-square shape. Each of the second encoding units (1010a, 1010b, 1020a, 1020b, etc.) can be split in a predetermined order using a recursive method, which may be a splitting method corresponding to a method in which the first encoding unit (1000) is split based on at least one of block shape information and split shape information.

[0143] For example, the image decoding device (150) can determine a third coding unit (1012a, 1012b) having a square shape by splitting the left second coding unit (1010a) in the horizontal direction, and can determine a third coding unit (1014a, 1014b) having a square shape by splitting the right second coding unit (1010b) in the horizontal direction. Furthermore, the image decoding device (150) can also determine a third coding unit (1016a, 1016b, 1016c, 1016d) having a square shape by splitting both the left second coding unit (1010a) and the right second coding unit (1010b) in the horizontal direction. In this case, the encoding unit can be determined in the same form as the first encoding unit (1000) being divided into four square-shaped second encoding units (1030a, 1030b, 1030c, 1030d).

[0144] As another example, the image decoding device (150) may determine third coding units (1022a, 1022b) having a square shape by vertically splitting the upper second coding unit (1020a), and may determine third coding units (1024a, 1024b) having a square shape by vertically splitting the lower second coding unit (1020b). Furthermore, the image decoding device (150) may determine third coding units (1022a, 1022b, 1024a, 1024b) having a square shape by vertically splitting both the upper second coding unit (1020a) and the lower second coding unit (1020b). In this case, the encoding unit can be determined in the same form as the first encoding unit (1000) being divided into four square-shaped second encoding units (1030a, 1030b, 1030c, 1030d).

[0145] FIG. 11 illustrates that, according to one embodiment, the processing order between multiple encoding units may vary depending on the division process of the encoding units.

[0146] According to one embodiment, the image decoding device (150) may split the first encoding unit (1100) based on block shape information and split shape information. If the block shape information indicates a square shape and the split shape information indicates that the first encoding unit (1100) is split in at least one of the horizontal direction and the vertical direction, the image decoding device (150) may split the first encoding unit (1100) to determine second encoding units (e.g., 1110a, 1110b, 1120a, 1120b, 1130a, 1130b, 1130c, 1130d, etc.). Referring to FIG. 11, the first encoding unit 1100) is divided only in the horizontal or vertical direction, and the second encoding units (1110a, 1110b, 1120a, 1120b) of non-square shape determined can be independently divided based on block shape information and division shape information for each. For example, the image decoding device (150) can determine third encoding units (1116a, 1116b, 1116c, 1116d) by horizontally dividing second encoding units (1110a, 1110b) generated by vertically dividing the first encoding unit (1100), and can determine third encoding units (1126a, 1126b, 1126c, 1126d) by horizontally dividing second encoding units (1120a, 1120b) generated by horizontally dividing the first encoding unit (1100). Since the process of dividing the second encoding units (1110a, 1110b, 1120a, 1120b) has been described above with reference to FIG. 9, a detailed description thereof will be omitted.

[0147] According to one embodiment, the image decoding device (150) can process encoding units according to a predetermined order. Since the characteristics of processing encoding units according to a predetermined order have been described above with reference to FIG. 6, a detailed description thereof will be omitted. Referring to FIG. 11, the image decoding device (150) can divide a first encoding unit (1100) having a square shape and determine four third encoding units (1116a, 1116b, 1116c, 1116d, 1126a, 1126b, 1126c, 1126d) having a square shape. According to one embodiment, the image decoding device (150) can determine the processing order of the third encoding units (1116a, 1116b, 1116c, 1116d, 1126a, 1126b, 1126c, 1126d) depending on the form in which the first encoding unit (1100) is divided.

[0148] According to one embodiment, the image decoding device (150) may determine third encoding units (1116a, 1116b, 1116c, 1116d) by horizontally dividing the second encoding units (1110a, 1110b) generated by vertically dividing them, and the image decoding device (150) may process the third encoding units (1116a, 1116b, 1116c, 1116d) according to an order (1117) of first processing the third encoding units (1116a, 1116b) included in the left second encoding unit (1110a) in the vertical direction and then processing the third encoding units (1116c, 1116d) included in the right second encoding unit (1110b) in the vertical direction.

[0149] According to one embodiment, the image decoding device (150) may determine third encoding units (1126a, 1126b, 1126c, 1126d) by vertically dividing the second encoding units (1120a, 1120b) generated by being divided in the horizontal direction, and the image decoding device (150) may process the third encoding units (1126a, 1126b, 1126c, 1126d) according to an order (1127) of first processing the third encoding units (1126a, 1126b) included in the upper second encoding unit (1120a) in the horizontal direction and then processing the third encoding units (1126c, 1126d) included in the lower second encoding unit (1120b) in the horizontal direction.

[0150] Referring to FIG. 11, the second encoding units (1110a, 1110b, 1120a, 1120b) can be divided into third encoding units (1116a, 1116b, 1116c, 1116d, 1126a, 1126b, 1126c, 1126d) having a square shape. The second encoding units (1110a, 1110b) determined by being vertically divided and the second encoding units (1120a, 1120b) determined by being horizontally divided are divided into different shapes, but the third encoding units (1116a, 1116b, 1116c, 1116d, 1126a, 1126b, 1126c, 1126d) determined later ultimately result in the first encoding unit (1100) being divided into encoding units of the same shape. Accordingly, even if the image decoding device (150) determines encoding units of the same shape as a result by recursively dividing the encoding units through different processes based on at least one of the block shape information and the split shape information, it can process a plurality of encoding units determined to have the same shape in different orders.

[0151] FIG. 12 illustrates a process in which, according to one embodiment, a coding unit is recursively divided to determine multiple coding units, and the coding unit depth is determined as the shape and size of the coding unit change.

[0152] According to one embodiment, the image decoding device (150) may determine the depth of an encoding unit according to a predetermined criterion. For example, the predetermined criterion may be the length of a long side of the encoding unit. If the length of the long side of the current encoding unit is split into 2n (n>0) times the length of the long side of the encoding unit before splitting, the image decoding device (150) may determine that the depth of the current encoding unit is increased by n compared to the depth of the encoding unit before splitting. Hereinafter, an encoding unit with an increased depth is expressed as an encoding unit of a lower depth.

[0153] Referring to FIG. 12, according to one embodiment, based on block shape information indicating a square shape (for example, the block shape information may indicate '0: SQUARE'), the image decoding device (150) may divide a first coding unit (1200) having a square shape to determine a second coding unit (1202), a third coding unit (1204), etc. of a lower depth. If the size of the first coding unit (1200) having a square shape is 2Nx2N, the second coding unit (1202) determined by dividing the width and height of the first coding unit (1200) by 1 / 21 may have a size of NxN. Furthermore, the third coding unit (1204) determined by dividing the width and height of the second coding unit (1202) by 1 / 2 may have a size of N / 2xN / 2. In this case, the width and height of the third encoding unit (1204) correspond to 1 / 22 times that of the first encoding unit (1200). When the depth of the first encoding unit (1200) is D, the depth of the second encoding unit (1202), which is 1 / 21 times that of the width and height of the first encoding unit (1200), may be D+1, and the depth of the third encoding unit (1204), which is 1 / 22 times that of the width and height of the first encoding unit (1200), may be D+2.

[0154] In one embodiment, based on block shape information indicating a non-square shape (for example, the block shape information may indicate '1: NS_VER' indicating a non-square shape in which the height is longer than the width or '2: NS_HOR' indicating a non-square shape in which the width is longer than the height), the image decoding device (150) may split a first coding unit (1210 or 1220) having a non-square shape to determine a second coding unit (1212 or 1222), a third coding unit (1214 or 1224) of a lower depth, etc.

[0155] The image decoding device (150) can determine a second encoding unit (e.g., 1202, 1212, 1222, etc.) by splitting at least one of the width and height of the first encoding unit (1210) having a size of Nx2N. That is, the image decoding device (150) can split the first encoding unit (1210) in the horizontal direction to determine a second encoding unit (1202) having a size of NxN or a second encoding unit (1222) having a size of NxN / 2, and can also split the first encoding unit (1210) in the horizontal direction and the vertical direction to determine a second encoding unit (1212) having a size of N / 2xN.

[0156] According to one embodiment, the image decoding device (150) may determine a second coding unit (e.g., 1202, 1212, 1222, etc.) by splitting at least one of the width and the height of the first coding unit (1220) having a size of 2NxN. That is, the image decoding device (150) may split the first coding unit (1220) in the vertical direction to determine a second coding unit (1202) having a size of NxN or a second coding unit (1212) having a size of N / 2xN, and may split the first coding unit (1220) in the horizontal direction and the vertical direction to determine a second coding unit (1222) having a size of NxN / 2.

[0157] According to one embodiment, the image decoding device (150) may determine a third encoding unit (e.g., 1204, 1214, 1224, etc.) by splitting at least one of the width and the height of the second encoding unit (1202) having a size of NxN. That is, the image decoding device (150) may split the second encoding unit (1202) in the vertical direction and the horizontal direction to determine a third encoding unit (1204) having a size of N / 2xN / 2, or a third encoding unit (1214) having a size of N / 2xN / 2, or a third encoding unit (1224) having a size of N / 2xN / 2.

[0158] According to one embodiment, the image decoding device (150) may determine a third coding unit (e.g., 1204, 1214, 1224, etc.) by splitting at least one of the width and the height of the second coding unit (1212) having a size of N / 2xN. That is, the image decoding device (150) may split the second coding unit (1212) in the horizontal direction to determine a third coding unit (1204) having a size of N / 2xN / 2 or a third coding unit (1224) having a size of N / 2xN / 2, or may split the second coding unit (1212) in the vertical direction and the horizontal direction to determine a third coding unit (1214) having a size of N / 2xN / 2.

[0159] According to one embodiment, the image decoding device (150) may determine a third coding unit (e.g., 1204, 1214, 1224, etc.) by splitting at least one of the width and the height of the second coding unit (1214) having a size of NxN / 2. That is, the image decoding device (150) may split the second coding unit (1212) in the vertical direction to determine a third coding unit (1204) having a size of N / 2xN / 2 or a third coding unit (1214) having a size of N / 2xN / 2, or split the second coding unit (1212) in the vertical direction and the horizontal direction to determine a third coding unit (1224) having a size of N / 2xN / 2.

[0160] According to one embodiment, the image decoding device (150) may split a square-shaped encoding unit (e.g., 1200, 1202, 1204) in a horizontal direction or a vertical direction. For example, a first encoding unit (1200) having a size of 2Nx2N may be split in the vertical direction to determine a first encoding unit (1210) having a size of Nx2N, or may be split in the horizontal direction to determine a first encoding unit (1220) having a size of 2NxN. According to one embodiment, when the depth is determined based on the length of the longest side of the encoding unit, the depth of the encoding unit determined by splitting the first encoding unit (1200, 1202, or 1204) having a size of 2Nx2N in the horizontal direction or the vertical direction may be the same as the depth of the first encoding unit (1200, 1202, or 1204).

[0161] According to one embodiment, the width and height of the third coding unit (1214 or 1224) may be half the width and height of the first coding unit (1210 or 1220). When the depth of the first coding unit (1210 or 1220) is D, the depth of the second coding unit (1212 or 1214), which is half the width and height of the first coding unit (1210 or 1220), may be D+1, and the depth of the third coding unit (1214 or 1224), which is half the width and height of the first coding unit (1210 or 1220), may be D+2.

[0162] FIG. 13 illustrates an index (part index, hereinafter referred to as PID) for depth and encoding unit distinction that can be determined according to the shape and size of encoding units according to one embodiment.

[0163] According to one embodiment, the image decoding device (150) may split a first encoding unit (1300) having a square shape to determine second encoding units of various shapes. Referring to FIG. 13, the image decoding device (150) may split the first encoding unit (1300) in at least one of a vertical direction and a horizontal direction according to split shape information to determine second encoding units (1302a, 1302b, 1304a, 1304b, 1306a, 1306b, 1306c, 1306d). That is, the image decoding device (150) can determine the second encoding units (1302a, 1302b, 1304a, 1304b, 1306a, 1306b, 1306c, 1306d) based on the segmentation shape information for the first encoding unit (1300).

[0164] According to one embodiment, the depth of the second coding units (1302a, 1302b, 1304a, 1304b, 1306a, 1306b, 1306c, 1306d) determined based on the split shape information for the first coding unit (1300) having a square shape may be determined based on the length of the long side. For example, since the length of one side of the first coding unit (1300) having a square shape and the length of the long side of the second coding unit (1302a, 1302b, 1304a, 1304b) having a non-square shape are the same, the depths of the first coding unit (1300) and the second coding units (1302a, 1302b, 1304a, 1304b) having a non-square shape may be considered to be the same as D. In contrast, when the image decoding device (150) divides the first encoding unit (1300) into four square-shaped second encoding units (1306a, 1306b, 1306c, 1306d) based on the division shape information, the length of one side of the square-shaped second encoding units (1306a, 1306b, 1306c, 1306d) is half the length of one side of the first encoding unit (1300), so the depth of the second encoding units (1306a, 1306b, 1306c, 1306d) may be a depth of D+1, which is one depth lower than D, the depth of the first encoding unit (1300).

[0165] According to one embodiment, the image decoding device (150) can horizontally split a first encoding unit (1310) having a height longer than width according to split shape information into a plurality of second encoding units (1312a, 1312b, 1314a, 1314b, 1314c). According to one embodiment, the image decoding device (150) can vertically split a first encoding unit (1320) having a width longer than height according to split shape information into a plurality of second encoding units (1322a, 1322b, 1324a, 1324b, 1324c).

[0166] According to one embodiment, the depth of the second encoding unit (1312a, 1312b, 1314a, 1314b, 1316a, 1316b, 1316c, 1316d) determined based on the segmentation shape information for the first encoding unit (1310 or 1320) of a non-square shape may be determined based on the length of the long side. For example, since the length of one side of the second encoding unit (1312a, 1312b) in the shape of a square is half the length of one side of the first encoding unit (1310) in the shape of a non-square in which the height is longer than the width, the depth of the second encoding unit (1302a, 1302b, 1304a, 1304b) in the shape of a square is D+1, which is one depth lower than the depth D of the first encoding unit (1310) in the shape of a non-square.

[0167] Furthermore, the image decoding device (150) can split the non-square first encoding unit (1310) into an odd number of second encoding units (1314a, 1314b, 1314c) based on the split shape information. The odd number of second encoding units (1314a, 1314b, 1314c) can include the non-square second encoding units (1314a, 1314c) and the square second encoding unit (1314b). In this case, since the length of the long side of the second encoding unit (1314a, 1314c) of a non-square shape and the length of one side of the second encoding unit (1314b) of a square shape are half the length of one side of the first encoding unit (1310), the depth of the second encoding unit (1314a, 1314b, 1314c) may be a depth of D+1, which is one depth lower than D, the depth of the first encoding unit (1310). The image decoding device (150) may determine the depth of the encoding units associated with the first encoding unit (1320) of a non-square shape, in which the width is longer than the height, in a manner corresponding to the above method of determining the depth of the encoding units associated with the first encoding unit (1310).

[0168] According to one embodiment, when determining an index (PID) for distinguishing divided coding units, the image decoding device (150) may determine the index based on a size ratio between the coding units when the coding units divided into an odd number are not of the same size. Referring to FIG. 13, among the coding units (1314a, 1314b, 1314c) divided into an odd number, the coding unit (1314b) located in the middle may have the same width as the other coding units (1314a, 1314c) but may have a height that is twice the height of the coding units (1314a, 1314c) that are different in height. That is, in this case, the coding unit (1314b) located in the middle may include two of the other coding units (1314a, 1314c). Accordingly, if the index (PID) of the encoding unit (1314b) located in the middle according to the scanning order is 1, the index of the encoding unit (1314c) located in the next order may be 3, which is an increase of 2. In other words, there may be a discontinuity in the index value. According to one embodiment, the image decoding device (150) may determine whether the encoding units divided into an odd number are not of the same size based on the presence or absence of discontinuity in the index for distinguishing between the divided encoding units.

[0169] According to one embodiment, the image decoding device (150) may determine whether the image is divided into a specific split shape based on the value of an index for distinguishing a plurality of coding units determined by splitting from the current coding unit. Referring to FIG. 13, the image decoding device (150) may divide a first coding unit (1310) having a rectangular shape in which the height is longer than the width, to determine an even number of coding units (1312a, 1312b) or an odd number of coding units (1314a, 1314b, 1314c). The image decoding device (150) may use an index (PID) indicating each coding unit to distinguish each of the plurality of coding units. According to one embodiment, the PID may be obtained from a sample (for example, an upper left sample) at a predetermined position of each coding unit.

[0170] According to one embodiment, the image decoding device (150) may determine an coding unit at a predetermined position among the coding units that are divided and determined using an index for distinguishing the coding units. According to one embodiment, when the split shape information for the first coding unit (1310) having a rectangular shape with a height longer than the width indicates that the first coding unit (1310) is divided into three coding units, the image decoding device (150) may divide the first coding unit (1310) into three coding units (1314a, 1314b, 1314c). The image decoding device (150) may assign an index to each of the three coding units (1314a, 1314b, 1314c). The image decoding device (150) may compare the indexes for each coding unit to determine the middle coding unit among the coding units divided into an odd number of coding units. The image decoding device (150) may determine a coding unit (1314b) having an index corresponding to a middle value among the indices of the coding units as a coding unit at a middle position among the coding units determined by splitting the first coding unit (1310). According to an embodiment, when determining an index for distinguishing the split coding units, the image decoding device (150) may determine the index based on a size ratio between the coding units when the coding units do not have the same size. Referring to FIG. 13, the coding unit (1314b) generated by splitting the first coding unit (1310) may have the same width as other coding units (1314a, 1314c) but may be twice the height of the coding units (1314a, 1314c) that are different in height. In this case, if the index (PID) of the encoding unit (1314b) located in the middle is 1, the index of the encoding unit (1314c) located in the next order may be 3, which is an increase of 2.In cases like this where the index increases uniformly and then the increase amount changes, the image decoding device (150) may determine that the current encoding unit is divided into a plurality of encoding units including encoding units having different sizes from other encoding units. According to one embodiment, when the split type information indicates that the current encoding unit is divided into an odd number of encoding units, the image decoding device (150) may divide the current encoding unit into a form in which an encoding unit at a predetermined position among the odd number of encoding units (for example, a middle encoding unit) has a different size from the other encoding units. In this case, the image decoding device (150) may determine the middle encoding unit having a different size using an index (PID) for the encoding unit. However, the above-described index, the size or position of the encoding unit at the predetermined position to be determined are specific for explaining one embodiment and should not be interpreted as being limited thereto, and it should be interpreted that various indexes, positions and sizes of encoding units can be used.

[0171] According to one embodiment, the image decoding device (150) may utilize a predetermined data unit from which recursive division of the encoding unit begins.

[0172] FIG. 14 illustrates that a plurality of coding units are determined according to a plurality of predetermined data units included in a picture according to one embodiment.

[0173] According to one embodiment, a predetermined data unit may be defined as a data unit from which a coding unit begins to be recursively divided using at least one of block shape information and split shape information. That is, it may correspond to a coding unit of the highest depth used in the process of determining multiple coding units for dividing the current picture. For convenience of explanation, this predetermined data unit will be referred to as a reference data unit hereinafter.

[0174] In one embodiment, the reference data unit may exhibit a predetermined size and shape. In one embodiment, the reference coding unit may include MxN samples, where M and N may be the same and may be integers expressed as powers of 2. That is, the reference data unit may exhibit a square or non-square shape, and may be subsequently divided into an integer number of coding units.

[0175] According to one embodiment, the video decoding device (150) may divide the current picture into a plurality of reference data units. According to one embodiment, the video decoding device (150) may divide the plurality of reference data units into which the current picture is divided using division information for each reference data unit. This division process of the reference data units may correspond to a division process using a quad-tree structure.

[0176] According to one embodiment, the image decoding device (150) may determine in advance the minimum size that a reference data unit included in the current picture may have. Accordingly, the image decoding device (150) may determine reference data units of various sizes having a size greater than or equal to the minimum size, and may determine at least one encoding unit using block shape information and segmentation shape information based on the determined reference data unit.

[0177] Referring to FIG. 14, the image decoding device (150) may use a reference coding unit (1400) having a square shape, or may use a reference coding unit (1402) having a non-square shape. According to one embodiment, the shape and size of the reference coding unit may be determined according to various data units (e.g., sequence, picture, slice, slice segment, maximum coding unit, etc.) that may include at least one reference coding unit.

[0178] According to one embodiment, the receiving unit (160) of the image decoding device (150) can obtain at least one of information on the shape of the reference coding unit and information on the size of the reference coding unit from the bitstream for each of the various data units. The process of determining at least one coding unit included in the square-shaped reference coding unit (1400) has been described above through the process of splitting the current coding unit (300) of FIG. 10, and the process of determining at least one coding unit included in the non-square-shaped reference coding unit (1400) has been described above through the process of splitting the current coding unit (1100 or 1150) of FIG. 11, so a detailed description thereof will be omitted.

[0179] According to one embodiment, the image decoding device (150) may use an index for identifying the size and shape of the reference coding unit to determine the size and shape of the reference coding unit according to some data units that are predetermined based on a predetermined condition. That is, the receiving unit (160) may obtain only an index for identifying the size and shape of the reference coding unit for each slice, slice segment, maximum coding unit, etc., among the various data units (e.g., sequences, pictures, slices, slice segments, maximum coding units, etc.) from the bitstream, as a data unit that satisfies a predetermined condition (e.g., a data unit having a size smaller than a slice). The image decoding device (150) may determine the size and shape of the reference data unit for each data unit that satisfies the predetermined condition by using the index. When information about the shape of the reference coding unit and information about the size of the reference coding unit are obtained and used from the bitstream for each relatively small-sized data unit, the efficiency of bitstream use may not be good. Therefore, instead of directly obtaining information about the shape of the reference coding unit and information about the size of the reference coding unit, only the index may be obtained and used. In this case, at least one of the size and shape of the reference coding unit corresponding to the index indicating the size and shape of the reference coding unit may be determined in advance. That is, the image decoding device (150) can determine at least one of the size and shape of the reference coding unit included in the data unit that serves as the basis for obtaining the index by selecting at least one of the sizes and shapes of the predetermined reference coding units according to the index.

[0180] According to one embodiment, the image decoding device (150) may use at least one reference coding unit included in one maximum coding unit. That is, a maximum coding unit for dividing an image may include at least one reference coding unit, and a coding unit may be determined through a recursive splitting process of each reference coding unit. According to one embodiment, at least one of the width and the height of the maximum coding unit may correspond to an integer multiple of at least one of the width and the height of the reference coding unit. According to one embodiment, the size of the reference coding unit may be a size obtained by splitting the maximum coding unit n times according to a quad tree structure. That is, the image decoding device (150) may determine the reference coding unit by splitting the maximum coding unit n times according to the quad tree structure, and may split the reference coding unit based on at least one of block shape information and split shape information according to various embodiments.

[0181] FIG. 15 illustrates a processing block that serves as a criterion for determining the order of determination of reference encoding units included in a picture (1500) according to one embodiment.

[0182] According to one embodiment, the image decoding device (150) may determine at least one processing block for dividing a picture. The processing block is a data unit including at least one reference coding unit for dividing an image, and at least one reference coding unit included in the processing block may be determined in a specific order. That is, the determination order of at least one reference coding unit determined in each processing block may correspond to one of various types of orders in which the reference coding unit may be determined, and the determination order of the reference coding unit determined in each processing block may be different for each processing block. The determination order of the reference coding unit determined for each processing block may be one of various orders such as raster scan, Z-scan, N-scan, up-right diagonal scan, horizontal scan, and vertical scan, but the order in which it may be determined should not be interpreted as being limited to the above scan orders.

[0183] According to one embodiment, the image decoding device (150) can obtain information about the size of a processing block to determine the size of at least one processing block included in an image. The image decoding device (150) can obtain information about the size of a processing block from a bitstream to determine the size of at least one processing block included in an image. The size of the processing block may be a predetermined size of a data unit indicated by the information about the size of the processing block.

[0184] According to one embodiment, the receiving unit (160) of the image decoding device (150) can obtain information about the size of a processing block from a bitstream for each specific data unit. For example, the information about the size of a processing block can be obtained from the bitstream for each data unit such as an image, a sequence, a picture, a slice, a slice segment, etc. That is, the receiving unit (160) can obtain information about the size of a processing block from the bitstream for each of the above-described multiple data units, and the image decoding device (150) can use the obtained information about the size of the processing block to determine the size of at least one processing block for dividing a picture, and the size of the processing block can be an integer multiple of a reference coding unit.

[0185] According to one embodiment, the video decoding device (150) may determine the size of the processing blocks (1502, 1512) included in the picture (1500). For example, the video decoding device (150) may determine the size of the processing block based on information about the size of the processing block obtained from the bitstream. Referring to FIG. 15, the video decoding device (150) may determine the horizontal size of the processing blocks (1502, 1512) to be four times the horizontal size of the reference coding unit, and the vertical size of the processing blocks (1502, 1512) to be four times the vertical size of the reference coding unit, according to one embodiment. The video decoding device (150) may determine the order in which at least one reference coding unit is determined within at least one processing block.

[0186] According to one embodiment, the video decoding device (150) may determine each processing block (1502, 1512) included in the picture (1500) based on the size of the processing block, and may determine the determination order of at least one reference coding unit included in the processing block (1502, 1512). According to one embodiment, the determination of the reference coding unit may include determining the size of the reference coding unit.

[0187] According to one embodiment, the image decoding device (150) can obtain information about a determination order of at least one reference coding unit included in at least one processing block from a bitstream, and can determine an order in which at least one reference coding unit is determined based on the obtained information about the determination order. The information about the determination order can be defined as an order or direction in which the reference coding units are determined within the processing block. That is, the order in which the reference coding units are determined can be independently determined for each processing block.

[0188] According to one embodiment, the image decoding device (150) can obtain information about the determination order of the reference coding unit for each specific data unit from the bitstream. For example, the receiving unit (160) can obtain information about the determination order of the reference coding unit for each data unit such as an image, sequence, picture, slice, slice segment, processing block, etc. from the bitstream. Since the information about the determination order of the reference coding unit indicates the determination order of the reference coding unit within the processing block, the information about the determination order can be obtained for each specific data unit including an integer number of processing blocks.

[0189] The image decoding device (150) can determine at least one reference encoding unit based on an order determined according to one embodiment.

[0190] According to one embodiment, the receiver (160) may obtain information about the reference coding unit determination order as information related to the processing blocks (1502, 1512) from the bitstream, and the image decoding device (150) may determine the order in which at least one reference coding unit included in the processing blocks (1502, 1512) is determined and determine at least one reference coding unit included in the picture (1500) according to the determination order of the coding units. Referring to FIG. 15, the image decoding device (150) may determine the determination order (1504, 1514) of at least one reference coding unit related to each processing block (1502, 1512). For example, when information about the determination order of the reference coding unit is obtained for each processing block, the determination order of the reference coding unit related to each processing block (1502, 1512) may be different for each processing block. If the reference encoding unit determination order (1504) related to the processing block (1502) is a raster scan order, the reference encoding unit included in the processing block (1502) can be determined according to the raster scan order. In contrast, if the reference encoding unit determination order (1514) related to another processing block (1512) is a reverse order of the raster scan order, the reference encoding unit included in the processing block (1512) can be determined according to the reverse order of the raster scan order.

[0191] In Figures 1 to 15, a method for dividing an image into maximum coding units and dividing the maximum coding units into coding units of a hierarchical tree structure is described. In Figures 16 to 28, an Ultimate Motion Vector Expression (UMVE) mode for deriving a motion vector for inter prediction is described.

[0192] Inter prediction is a prediction method that predicts the current block from a reference block similar to the current block obtained from the reference picture of the current picture. To inter-predict the current block, a motion vector indicating the spatial difference between the current block and the reference block and a reference picture referenced by the current picture are determined. Then, the current block is predicted with reference to the reference block determined based on the motion vector and the reference picture. The UMVE mode introduced in the present disclosure is an inter-prediction mode for efficiently encoding and / or decoding motion vectors.

[0193] In UMVE mode, a UMVE candidate list is determined from blocks spatially adjacent to the current block or blocks temporally adjacent to the current block. Then, the reference picture and base motion vector of the current block are determined from the UMVE candidate of the current block selected from the UMVE candidate list. The motion vector of the current block is generated by correcting the base motion vector of the current block according to the correction size and direction.

[0194] In Fig. 16, inter prediction according to the UMVE mode is described in detail. According to Fig. 16, the base motion vector (1600) obtained from the UMVE candidate of the current block is (5, -4). However, if the reference block indicated by the base motion vector (1600) is inaccurate, the encoding efficiency of the current block may be reduced. Therefore, in the UMVE mode, the base motion vector (1600) can be corrected according to the correction size (1602) and the correction direction (1604).

[0195] For example, when the correction size (1602) is 2 and the correction direction (1604) is in the +x direction, the correction motion vector (1606) for correcting the base motion vector (1600) is determined as (2,0). Then, the sum vector (7,-4) of the base motion vector (1600) and the correction motion vector (1606) is determined as the motion vector (1608) of the current block. Therefore, the current block can be predicted by the motion vector (1608) indicating the exact reference block. Therefore, in the UMVE mode, by correcting the base motion vector (1600), the accuracy of prediction can be improved.

[0196] In order to reduce the size of information required for correction of the basic motion vector (1600) in UMVE mode, the correction size (1602) and correction direction (1604) have a limited number of candidates. For example, when the correction size (1602) is determined from among eight correction size candidates, the correction size information representing the correction size (1602) can be expressed with a minimum of 1 bit and a maximum of 7 bits according to a truncated unary code. Similarly, when the correction direction (1604) is determined from among four correction direction candidates (+x, -x, +y, -y), the correction direction information representing the correction direction (1604) can be expressed with 2 bits. Therefore, the number of bits required to determine the correction motion vector (1606) is limited to 9 bits. Therefore, compression efficiency can be increased by limiting the number of bits required for determining the correction motion vector (1606) in UMVE mode to a certain size or less.

[0197] Fig. 17 illustrates a block diagram of a video decoding device (1700) that performs decoding according to the UMVE mode.

[0198] The video decoding device (1700) includes an upper parameter determination unit (1710), an encoding mode determination unit (1720), and a decoding unit (1730). In FIG. 17, the upper parameter determination unit (1710), the encoding mode determination unit (1720), and the decoding unit (1730) are expressed as separate configuration units, but depending on the embodiment, the upper parameter determination unit (1710), the encoding mode determination unit (1720), and the decoding unit (1730) may be combined and implemented as a single configuration unit.

[0199] In FIG. 17, the upper parameter determination unit (1710), the encoding mode determination unit (1720), and the decoding unit (1730) are expressed as a configuration unit located in one device, but the devices responsible for each function of the upper parameter determination unit (1710), the encoding mode determination unit (1720), and the decoding unit (1730) do not necessarily need to be physically adjacent. Therefore, depending on the embodiment, the upper parameter determination unit (1710), the encoding mode determination unit (1720), and the decoding unit (1730) may be distributed.

[0200] The upper parameter determination unit (1710), the encoding mode determination unit (1720), and the decoding unit (1730) may be implemented by one processor according to an embodiment. Alternatively, they may be implemented by multiple processors according to an embodiment.

[0201] The upper parameter determination unit (1710) can determine whether the UMVE mode is allowed for the upper data unit of the current block. The upper data unit may be a sequence, picture, slice, or slice segment of the current block. For example, the upper parameter determination unit (1710) can determine whether the UMVE mode is allowed for each picture.

[0202] The upper parameter determination unit (1710) can determine whether the UMVE mode is allowed for each upper data unit. For example, the upper parameter determination unit (1710) can determine whether the UMVE mode is allowed for the current sequence. In addition, the upper parameter determination unit (1710) can determine whether the UMVE mode is allowed for the current picture included in the current sequence. Similarly, the upper parameter determination unit (1710) can determine whether the UMVE mode is allowed for the current slice included in the current picture.

[0203] If the UMVE mode is not allowed for the current sequence, the upper parameter determination unit (1710) may determine that the UMVE mode is not allowed for all pictures included in the current sequence. Similarly, if the UMVE mode is not allowed for the current picture, the upper parameter determination unit (1710) may determine that the UMVE mode is not allowed for all slices included in the current picture.

[0204] The upper parameter determination unit (1710) obtains a UMVE enabled flag indicating whether the UMVE mode is allowed for the upper data unit of the current block from the bitstream. If the UMVE enabled flag indicates that the UMVE mode is allowed for the upper data unit, the encoding mode determination unit (1720) determines whether the UMVE mode is applied to all blocks included in the upper data unit. Conversely, if the UMVE enabled flag indicates that the UMVE mode is not allowed for the upper data unit, the encoding mode determination unit (1720) determines that the UMVE mode is not applied to all blocks included in the upper data unit.

[0205] When the UMVE Allowed flag indicates 1, the UMVE Allowed flag can be interpreted as allowing UMVE mode in the upper data unit. Conversely, when the UMVE Allowed flag indicates 0, the UMVE Allowed flag can be interpreted as not allowing UMVE mode in the upper data unit. Depending on the embodiment, the meaning of the UMVE Allowed flag can be interpreted in the opposite way.

[0206] The upper parameter determination unit (1710) can obtain a UMVE permission flag for each upper data unit. If the permission of the UMVE mode is determined on a picture-by-picture basis, the upper parameter determination unit (1710) can obtain a UMVE permission flag on a picture-by-picture basis. However, if the permission of the UMVE mode is determined hierarchically in the order of sequence unit, picture unit, and slice unit, the upper parameter determination unit (1710) can obtain a UMVE permission flag for the sequence unit, a UMVE permission flag for the picture unit, and a UMVE permission flag for the slice unit.

[0207] In another embodiment, the UMVE Allowed flag may indicate whether the default setting for the UMVE mode is applied to the upper data unit. For example, when the UMVE Allowed flag indicates 0, the UMVE Allowed flag may be interpreted as applying the default setting for the UMVE mode to the upper data unit. Conversely, when the UMVE Allowed flag indicates 1, the UMVE Allowed flag may be interpreted as not applying the default setting for the UMVE mode to the upper data unit. If the UMVE Allowed flag is not acquired, the upper parameter determination unit (1710) may apply the default setting for the UMVE mode to the upper data unit.

[0208] The default setting for UMVE mode may be to disallow UMVE mode for upper data units. Alternatively, the default setting for UMVE mode may be to only apply UMVE mode to upper data units under certain conditions. The default setting for UMVE mode may be set differently for each sequence unit, picture unit, or slice unit.

[0209] The upper parameter determination unit (1710) can determine whether to allow the UMVE mode of the upper data unit without the UMVE allow flag according to specific conditions. For example, if the current picture is the last picture of a GoP (Group of Pictures), the upper parameter determination unit (1710) can apply the default setting for the UMVE mode to the upper data unit without the UMVE allow flag. As another example, the upper parameter determination unit (1710) can apply the default setting for the UMVE mode to the upper data unit without the UMVE allow flag according to the temporal layer depth of the current picture.

[0210] The upper parameter decision unit (1710) can determine whether to allow other inter prediction modes depending on whether the UMVE mode is allowed in the upper data unit. For example, if the UMVE mode is allowed according to the UMVE permission flag, certain inter prediction modes may not be allowed in the upper data unit. The certain inter prediction modes may include the DMVD (Decoder-side Motion Vector Derivation) mode, the DMVR (Decoder-side Motion Vector Refinement) mode, the Skip mode, the Direct mode, the Merge mode, etc.

[0211] For example, in UMVE mode, information about motion vectors is obtained from the bitstream, whereas in DMVD mode or DMVR mode, motion vectors are derived without obtaining information about motion vectors from the bitstream. Therefore, according to one embodiment, the upper parameter determination unit (1710) may determine that if the UMVE mode is allowed for the upper data unit, the DMVD mode or DMVR mode is not allowed for the upper data unit.

[0212] As another example, if the UMVE mode is allowed in the upper data unit, the upper parameter determination unit (1710) may determine that the Skip mode, Direct mode, and Merge mode, which omit information about the motion vector, are not allowed.

[0213] According to another embodiment, the upper parameter determination unit (1710) may obtain an allowance flag for the predetermined inter prediction mode from the bitstream for the upper data unit if the UMVE mode is allowed for the upper data unit. According to another embodiment, the upper parameter determination unit (1710) may apply a default setting for the predetermined inter prediction mode to the upper data unit without obtaining an allowance flag for the predetermined inter prediction mode if the UMVE mode is allowed for the upper data unit. The predetermined inter prediction mode may include a DMVD (Decoder-side Motion Vector Derivation) mode, a DMVR (Decoder-side Motion Vector Refinement) mode, a Skip mode, a Direct mode, a Merge mode, an OBMC (overlapped block motion compensation) mode, an IC (Illumination Compensation) mode, an Affine mode, etc.

[0214] The upper parameter determination unit (1710) can additionally obtain setting information regarding the UMVE mode applied to the upper data unit if the UMVE mode is permitted for the upper data unit. Below, additional information regarding the UMVE mode obtained by the upper parameter determination unit (1710) is described.

[0215] The upper parameter determination unit (1710) can obtain UMVE candidate count information indicating the number of UMVE candidates of the UMVE mode allowed for the upper data unit from the bitstream. Then, the decoding unit (1730) determines as many UMVE candidates as the number indicated by the UMVE candidate count information.

[0216] Additionally, the upper parameter determination unit (1710) can obtain correction size candidate count information indicating the number of correction size candidates allowed for the upper data unit from the bitstream. Then, the decoding unit (1730) determines as many correction size candidates as the number indicated by the correction size candidate count information.

[0217] Additionally, the upper parameter determination unit (1710) can obtain correction direction candidate number information indicating the number of correction direction candidates allowed for the upper data unit from the bitstream. Then, the decoding unit (1730) determines as many correction direction candidates as the number indicated by the correction direction candidate number information.

[0218] The UMVE candidate count information can indicate the difference between the minimum number of UMVE candidates and the number of UMVE candidates used in the upper data unit. For example, if the minimum number of UMVE candidates is 4 and the number of UMVE candidates used in the upper data unit is 5, the UMVE candidate count information can indicate 1. The correction size candidate count information and the correction direction candidate count information, like the UMVE candidate count information, can also indicate the difference between the minimum number and the number of candidates used in the upper data unit.

[0219] The upper parameter determination unit (1710) can obtain UMVE candidate list configuration information indicating a method of configuring the UMVE candidate list from a bitstream. The UMVE candidate list configuration information can indicate that the UMVE candidate list of the UMVE mode is configured based on a motion vector candidate list used in the Merge mode.

[0220] Alternatively, the UMVE candidate list configuration information may indicate that a motion vector candidate list exclusively for the UMVE mode is configured. For example, a motion vector candidate list exclusively for the UMVE mode may be configured by statistically analyzing the motion vector information used to decode the current picture. If the left block and the upper block of the current block have a high probability of being selected as UMVE candidates, the left block and the upper block of the current block may be preferentially included in the UMVE candidate list.

[0221] Alternatively, the UMVE candidate list configuration information may indicate that the motion vector candidate list is generated by mixing the motion vector candidate list used in the Merge mode and the motion vector candidate list exclusive to the UMVE mode.

[0222] The upper parameter determination unit (1710) selects a method for configuring a UMVE candidate list according to default settings when it is unable to obtain UMVE candidate list configuration information or when the UMVE candidate list is determined implicitly.

[0223] The upper parameter determination unit (1710) can obtain correction size range information for a plurality of blocks of the upper data unit from the bitstream. Then, the decoding unit (1730) determines the correction size of the basic motion vector based on the correction size range indicated by the correction size range information.

[0224] For example, the correction size range information may indicate the minimum value of the correction size candidates. If the minimum value of the correction size candidates indicated by the correction size range information is 1, and the number of correction size candidates indicated by the correction size number candidate information is 4, the correction size candidates may be determined as {1,2,4,8}.

[0225] Additionally, the correction size range information can indicate the size difference between the correction size candidates. For example, the correction size range information can indicate whether the values ​​of the correction size candidates increase arithmetically or geometrically. If the values ​​of the correction size candidates increase arithmetically, the correction size candidates are determined as {1,2,3,4,5,쪋}. And if the values ​​of the correction size candidates increase geometrically, the correction size candidates are determined as {1,2,4,8,16,쪋}.

[0226] Additionally, the correction size range information may indicate one of a plurality of predetermined correction size candidate sets. For example, the correction size range information may indicate one of the first correction size candidate set {1,2,4,8}, the second correction size candidate set {1,2,3,4}, and the third correction size candidate set {4,8,16,32}.

[0227] The upper parameter determination unit (1710) can obtain correction size change information for a plurality of blocks of the upper data unit from the bitstream. The correction size change information indicates whether the range of the correction size candidate changes depending on the temporal distance between the current picture and the reference picture. Therefore, when the correction size change information indicates a change in the range of the correction size candidate, the decoding unit (1730) can determine the range of the correction size candidate depending on the temporal distance between the current picture and the reference picture.

[0228] In general, if the current block references a reference block in a reference picture that is temporally distant from the current picture, the error in the base motion vector is likely to be large. Conversely, if the current block references a reference block in a reference picture that is temporally close to the current picture, the error in the base motion vector is likely to be small. Therefore, determining the set of compensation size candidates in proportion to the temporal distance between the current picture and the reference picture can improve the motion vector prediction accuracy.

[0229] For example, if the temporal distance between the current picture and the reference picture is small, the minimum value of the correction size candidate can be determined as 1 / 2. And if the temporal distance between the current picture and the reference picture is medium, the minimum value of the correction size candidate can be determined as 1. And if the temporal distance between the current picture and the reference picture is large, the minimum value of the correction size candidate can be determined as 2.

[0230] The temporal distance between the current picture and the reference picture can be determined based on one or more thresholds. For example, if there are three thresholds, the temporal distance between the current picture and the reference picture can be compared with the three thresholds to determine the minimum value of the compensation size candidate.

[0231] The upper parameter determination unit (1710) can determine the minimum value of the correction size candidate of the upper data unit based on the minimum value of the correction size candidate of the correction size range information and the temporal distance between the current picture and the reference picture. Specifically, the minimum value of the correction size candidate of the correction size range information can be determined by changing according to the temporal distance between the current picture and the reference picture.

[0232] The upper parameter determination unit (1710) determines whether to change the correction size according to the default settings when it is not possible to obtain correction size change information or when whether to change the correction size is determined internally.

[0233] The upper parameter determination unit (1710) can obtain prediction direction change permission information for a plurality of blocks of the upper data unit from the bitstream. The current block obtains information about the prediction direction of the reference picture together with information about the base motion vector from the UMVE candidate selected from the UMVE candidate list. If the change in the prediction direction is not permitted according to the prediction direction change permission information, the prediction direction of the UMVE candidate is applied to the current block. However, conversely, if the change in the prediction direction is permitted according to the prediction direction change permission information, a prediction direction different from that of the UMVE candidate can be applied to the current block.

[0234] The above prediction direction indicates one of uni-prediction of list 0, uni-prediction of list 1, and bi-prediction using both list 0 and list 1. In the case of uni-prediction, only one reference block of list 0 or list 1 is used to predict the current block, but in the case of bi-prediction, both reference blocks of list 0 and list 1 are used. Therefore, the prediction accuracy of bi-prediction is likely to be higher than that of uni-prediction. Therefore, if a change in the prediction direction is allowed, even if the UMVE candidate indicates uni-prediction, the prediction direction of the UMVE mode can be determined as bi-prediction, so that the encoding efficiency according to the UMVE mode can be increased.

[0235] Therefore, if a change in the prediction direction is allowed based on the prediction direction change permission information, the decoding unit (1730) can change the prediction direction of the current block. For example, if a change in the prediction direction is allowed, even if the prediction direction of the UMVE candidate is list 0 uniprediction, the prediction direction of the current block can be determined as list 1 uniprediction or biprediction.

[0236] The upper parameter determination unit (1710) can obtain primary prediction direction information for the upper data unit from the bitstream. The primary prediction direction information indicates whether there is a primary prediction direction for the upper data unit. In addition, the primary prediction direction information indicates what the primary prediction direction of the upper data unit is, if there is a primary prediction direction for the upper data unit. When the primary prediction direction of the upper data unit is set, blocks to which the UMVE mode is applied included in the upper data unit are predicted according to the primary prediction direction. For example, when the primary prediction direction is determined as bi-prediction, blocks to which the UMVE mode is applied can be bi-predicted. Therefore, even when the prediction direction of the UMVE candidate is uni-prediction, the prediction direction of the current block can be determined as bi-prediction.

[0237] The upper parameter determination unit (1710) can obtain both prediction direction change permission information and main prediction direction information. If the prediction direction change permission information indicates that prediction direction change is permitted and the main prediction direction information indicates that the main prediction direction is set, the prediction direction information obtained by the decoding unit (1730) indicates whether the prediction direction of the current block is the main prediction direction. For example, if the main prediction direction is bi-prediction and the prediction direction information indicates that the prediction direction of the current block is the main prediction direction, the prediction direction of the current block is determined as bi-prediction. If the main prediction direction is bi-prediction and the prediction direction information indicates that the prediction direction of the current block is not the main prediction direction, the prediction direction of the current block is determined as list 0 uni-prediction or list 1 uni-prediction (auxiliary prediction direction) according to the prediction direction information.

[0238] The upper parameter determination unit (1710) can obtain multiple correction allowance information indicating whether the corrected basic motion vector from the bitstream can be corrected multiple times. If the basic motion vector can be corrected multiple times according to the multiple correction allowance information, the decoding unit (1730) can correct the basic motion vector multiple times.

[0239] The encoding mode decision unit (1720) determines whether the UMVE mode is applied to the current block when the UMVE mode is allowed for the upper data unit. The encoding mode decision unit (1720) can obtain a UMVE flag indicating whether the UMVE mode is applied to the current block from the bitstream. For example, the encoding mode decision unit (1720) can determine the prediction mode of the current block as the UMVE mode when the UMVE flag indicates that the UMVE mode is applied to the current block. Conversely, the encoding mode decision unit (1720) does not determine the prediction mode of the current block as the UMVE mode when the UMVE flag indicates that the UMVE mode is not applied to the current block.

[0240] The encoding mode decision unit (1720) may determine that, when the UMVE mode is applied to the current block, other inter prediction modes are not applied to the current block. For example, when the UMVE mode is applied to the current block according to the UMVE flag, certain inter prediction modes may not be applied to the current block. The certain inter prediction modes may include DMVD mode, DMVR mode, Skip mode, Direct mode, Merge mode, OBMC mode, IC mode, Affine mode, etc. Conversely, the encoding mode decision unit (1720) may determine that, when the UMVE mode is applied to the current block, a specific prediction mode is applied to the current block.

[0241] Additionally, the encoding mode decision unit (1720) may determine that a different intra prediction mode is to be applied to the current block if the UMVE mode is not applied to the current block. For example, the intra prediction mode may include DMVD mode, DMVR mode, Merge mode, OBMC mode, IC mode, Affine mode, etc.

[0242] When the UMVE mode is applied to the current block, the decoding unit (1730) can determine the basic motion vector of the current block from the UMVE candidate. The decoding unit (1730) can determine the UMVE candidate list according to the UMVE candidate list construction method based on the UMVE candidate list construction information. The number of UMVE candidates included in the UMVE candidate list can be determined according to the UMVE candidate number information.

[0243] The decoding unit (1730) can obtain a UMVE index indicating a UMVE candidate including the base motion vector and reference picture of the current block from the UMVE candidate list from the bitstream. Then, the decoding unit (1730) can determine the base motion vector and reference picture of the current block according to the UMVE index.

[0244] The decoding unit (1730) can determine the correction size and direction for correction of the basic motion vector. In the UMVE mode, the basic motion vector is corrected for the accuracy of prediction. Therefore, in order to determine the correction motion vector for correction of the basic motion vector, the decoding unit (1730) determines the correction size indicating the size of the correction motion vector and the correction direction indicating the direction of the correction motion vector.

[0245] The decoding unit (1730) can determine the correction size candidates according to the number of correction size candidates, the correction size range information, and the correction size change information for the upper data unit. The decoding unit (1730) can determine the number of correction size candidates according to the number of correction size candidates information. In addition, the decoding unit (1730) can determine the minimum value of the correction size candidates and the size difference between the correction size candidates according to the correction size range information. In addition, when the correction size change information indicates a change in the range of the correction size candidates, the decoding unit (1730) can determine the range of the correction size candidates according to the distance between the current picture and the reference picture. For example, the minimum value of the correction size candidates can be changed by comparing the distance between the current picture and the reference picture with a predetermined threshold value. Therefore, the range of the correction size candidates can be determined according to the changed minimum value.

[0246] The decoding unit (1730) can obtain correction size information indicating the correction size of the basic motion vector from the bitstream. The correction size information indicates a correction size candidate corresponding to the correction size of the basic motion vector among a predetermined number of correction size candidates. The decoding unit (1730) can determine the correction size of the basic motion vector based on the correction size candidate indicated by the correction size information.

[0247] The decoding unit (1730) can determine correction direction candidates based on the correction size direction number information for the upper data unit. Then, the decoding unit (1730) determines the correction direction from a predetermined number of correction direction candidates.

[0248] The decoding unit (1730) can obtain correction direction information indicating the correction direction of the basic motion vector from the bitstream. The correction direction information indicates a correction direction candidate corresponding to the correction direction of the basic motion vector among a predetermined number of correction direction candidates. The decoding unit (1730) can determine the correction direction of the basic motion vector based on the correction direction candidate indicated by the correction direction information.

[0249] The decoding unit (1730) can determine the motion vector of the current block by correcting the basic motion vector according to the determined correction size and correction direction. For example, if the correction size is 2 and the correction direction is +x, a motion vector whose x value is increased by 2 from the basic motion vector is used for prediction of the current block.

[0250] Figures 18 to 21 are diagrams for explaining the process of determining the motion vector of the current block in UMVE mode. In Figures 18 and 21, the coordinates indicated by the basic motion vector are set to (base_x, base_y). In addition, the correction motion vector according to the correction size and correction direction of the UMVE mode is explained based on (base_x, base_y).

[0251] Referring to FIG. 18, the decoding unit (1730) can determine motion vector candidates having a diamond-shaped distribution based on the basic motion vector.

[0252] The decoding unit (1730) can determine motion vector candidates ((base_x+1 / 4, base_y)(1802), (base_x-1 / 4, base_y)(1803), (base_x, base_y+1 / 4)(1804), (base_x, base_y-1 / 4)(1805)) located at a distance of 1 / 4 pixel from the base motion vector (base_x, base_y)(1801) as the first candidate group.

[0253] The decoding unit (1730) can determine motion vector candidates ((base_x+1 / 2, base_y)(1806), (base_x-1 / 2, base_y)(1807), (base_x, base_y+1 / 2)(1808), (base_x, base_y-1 / 2)(1809)) located at a distance of 1 / 2 pixel from the base motion vector (base_x, base_y)(1801) as the second candidate group.

[0254] The decoding unit (1730) can determine motion vector candidates ((base_x+1, base_y)(1810), (base_x-1, base_y)(1811), (base_x, base_y+1)(1812), (base_x, base_y-1)(1813)) at a distance of 1 pixel from the base motion vector (base_x, base_y)(1801) as the third candidate group.

[0255] The decoding unit (1730) can select one of the first to third candidate groups based on the correction distance. Furthermore, the decoding unit (1730) can determine a motion vector candidate from one of the candidate groups based on the correction direction as the motion vector of the current block.

[0256] Referring to FIG. 19, the decoding unit (1730) can determine predicted motion vector candidates having a rectangular distribution based on the basic motion vector.

[0257] Similarly, the decoding unit (1730) can determine motion vector candidates (base_x+1 / 4, base_y+1 / 4)(1902), (base_x+1 / 4, base_y-1 / 4)(1903), (base_x-1 / 4, base_y+1 / 4)(1904), (base_x-1 / 4, base_y-1 / 4)(1905) located at a distance of about 1 / 4 pixel from the base motion vector (base_x, base_y)(1901) as the first candidate group.

[0258] The decoding unit (1730) can determine motion vector candidates (base_x+1 / 2, base_y+1 / 2)(1906), (base_x+1 / 2, base_y-1 / 2)(1907), (base_x-1 / 2, base_y+1 / 2)(1908), (base_x-1 / 2, base_y-1 / 2)(1909) located at a distance of about 1 / 2 pixel from the base motion vector (base_x, base_y)(1901) as the second candidate group.

[0259] The decoding unit (1730) can determine motion vector candidates ((base_x+1, base_y+1)(1910), (base_x+1, base_y-1)(1911), (base_x-1, base_y+1)(1912), (base_x-1, base_y-1)(1913)) located at a distance of about 1 pixel from the base motion vector (base_x, base_y)(1901) as a third candidate group.

[0260] Referring to Fig. 20, the video decoding device (1730) can determine each group as a different number of motion vector candidates. Although the interval between each pixel is 1 / 4 pixel distance, the component values ​​of the vector candidates are displayed by scaling them by 4 for convenience.

[0261] Specifically, the decoding unit (1730) can determine eight motion vector candidates (base_x+1 / 4, base_y / 4)(2002), (base_x-1 / 4, base_y)(2003), (base_x, base_y+1 / 4)(2004), (base_x, base_y-1 / 4)(2005), (base_x+1 / 4, base_y+1 / 4)(2006), (base_x+1 / 4, base_y-1 / 4)(2007), (base_x-1 / 4, base_y+1 / 4)(2008), (base_x-1 / 4, base_y-1 / 4)(2009) at a distance of about 1 / 4 pixel from the base motion vector as the first candidate group.

[0262] In addition, the decoding unit (1730) can determine eight motion vector candidates (base_x+1 / 2, base_y)(2010), (base_x-1 / 2, base_y)(2011), (base_x, base_y+1 / 2)(2012), (base_x, base_y-1 / 2)(2013), (base_x+1 / 2, base_y+1 / 2)(2014), (base_x+1 / 2, base_y-1 / 2)(2015), (base_x-1 / 2, base_y+1 / 2)(2016), (base_x-1 / 2, base_y-1 / 2)(2017) at a distance of about 1 / 2 pixel from the base motion vector as a second candidate group.

[0263] The decoding unit (1730) can determine eight motion vector candidates (base_x+1, base_y)(2018), (base_x-1, base_y)(2019), (base_x, base_y+1)(2020), (base_x, base_y-1)(2021), (base_x+1, base_y+1)(2022), (base_x+1, base_y-1)(2023), (base_x-1, base_y+1)(2024), (base_x-1, base_y-1)(2025) at a distance of about 1 pixel from the base motion vector as a third candidate group.

[0264] Referring to FIG. 21, the decoding unit (1730) can determine the distribution form of the motion vector candidates included in each candidate group in various ways for each candidate group. Specifically, the decoding unit (1730) can determine motion vector candidates (2102, 2103, 2104, 2105) having a diamond-shaped distribution based on the basic motion vector (2101) as a first candidate group. In addition, the decoding unit (1730) can determine motion vector candidates (2106, 2107, 2108, 2109) having a square-shaped distribution based on the basic motion vector (2101) as a second candidate group. In addition, the decoding unit (1730) can determine motion vector candidates (2110, 2111, 2112, 2113) having a diamond-shaped distribution based on the basic motion vector (2101) as a third candidate group. As illustrated in Fig. 21, the distribution of motion vector candidates of each candidate group can be determined in various distribution forms other than the distribution form illustrated in Fig. 21.

[0265] The decoding unit (1730) can determine one or more base motion vectors. If there are two base motion vectors, motion vector candidates can be generated using each of the base motion vectors.

[0266] The decoding unit (1730) can perform pair prediction. If the base motion vector is performed as pair prediction using reference pictures in List 0 and List 1, the motion vectors in each prediction direction can be corrected according to the temporal distance between the current picture and the reference picture.

[0267] For example, if the reference pictures of list 0 and list 1 are located in the same direction from the current picture, the correction direction of the base motion vector for list 0 and the base motion vector for list 1 are determined to be the same. Conversely, if the reference pictures of list 0 and list 1 are located in opposite directions from the current picture, the correction direction of the base motion vector of list 0 is determined to be the opposite of the correction direction of the base motion vector of list 1.

[0268] In addition, the correction size of the base motion vector of list 0 and the correction size of the base motion vector of list 1 are determined to be proportional to the temporal distance between the reference picture of list 0 and the current picture and the temporal distance between the reference picture of list 1 and the current picture, respectively. Therefore, if the temporal distance between the reference picture of list 0 and the current picture is twice the temporal distance between the reference picture of list 1 and the current picture, the correction size of the base motion vector of list 0 is determined to be twice the correction size of the base motion vector of list 0.

[0269] The correction size and correction direction indicated by the correction size information and correction direction information obtained from the bitstream can be applied to the basic motion vector of list 0.

[0270] The correction size and correction direction used for correction of the basic motion vector of list 0 can be changed by considering the distance between the reference picture of list 0 and the current picture and the distance between the reference picture of list 1 and the current picture. And the changed correction size and correction direction can be applied to the basic motion vector of list 1.

[0271] In an embodiment, the correction size and correction direction indicated by the correction size information and the correction direction information may be applied to the basic motion vector of list 1, and the changed correction size and correction direction may be applied to the basic motion vector of list 0.

[0272] Fig. 22 illustrates an embodiment of a method for correcting two basic motion vectors according to pair prediction. In Fig. 22, the basic motion vector (2202) of list 0 represents (0, -1). And the basic motion vector (2212) of list 1 represents (0, 2).

[0273] In UMVE mode, the correction magnitude and correction direction are applied to both the base motion vector (2202) of list 0 and the base motion vector (2212) of list 1. When the correction magnitude for the base motion vector (2202) in the list 0 direction is 1 and the correction direction is -y, the motion vector (2204) of list 0 becomes (0, -2).

[0274] The reference pictures of list 0 and list 1 are located in opposite directions from the current picture. Therefore, the correction direction of the base motion vector (2212) of list 1 is determined to be the opposite of the correction direction of the base motion vector (2202) in the direction of list 0. Therefore, the correction direction of the base motion vector (2212) of list 1 is determined to be +y.

[0275] The reference pictures of List 0 and List 1 are the same temporal distance from the current picture. Therefore, the correction size of the base motion vector (2212) of List 1 is determined to be the same as the correction size for the base motion vector (2202) in the direction of List 0. Therefore, the correction size of the base motion vector (2212) of List 1 is determined to be 1.

[0276] Therefore, the motion vector (2214) of list 1 is determined to be (0,3), which has a y component that is 1 larger than the basic motion vector (2212) of list 1.

[0277] If the correction size for the basic motion vector (2202) in the list 0 direction is 2, the motion vector (2206) in the list 0 direction is determined as (0, -3), and the motion vector (2216) in the list 1 direction is determined as (0, 4). In addition, if the correction size for the basic motion vector (2202) in the list 0 direction is 4, the motion vector (2208) in the list 0 direction is determined as (0, -5), and the motion vector (2218) in the list 1 direction is determined as (0, 6).

[0278] The decoding unit (1730) can obtain prediction direction information from the bitstream and change the prediction direction of the current block according to the prediction direction information if a change in the prediction direction is allowed according to the prediction direction change permission information. The prediction direction information can indicate whether or not to change the prediction direction. If the prediction direction information does not indicate a change in the prediction direction, the current block is predicted according to the prediction direction of the UMVE candidate. However, if the prediction direction information indicates a change in the prediction direction, the current block is predicted according to a prediction direction different from that of the UMVE candidate.

[0279] Additionally, the prediction direction information can indicate the changing prediction direction. For example, if the prediction direction of a UMVE candidate is List 0 uniprediction, the prediction direction information can indicate either List 1 uniprediction or biprediction. Therefore, the prediction direction information can indicate whether the prediction direction has changed and the changed prediction direction.

[0280] If there is no change in the prediction direction, the prediction direction information can be set to 0. If there is a change in the prediction direction, the prediction direction information can be set to 10 or 11 depending on the changed prediction direction. For example, if the prediction direction of the UMVE candidate is list 0 single prediction, the prediction direction information can be set to 0 when the prediction direction of the current block is list 0 single prediction.

[0281] If the prediction direction of the current block is bi-prediction or list 1 uni-prediction, the prediction direction information may be set to 10 or 11. In an embodiment, if the prediction direction of the UMVE candidate is bi-prediction and the prediction direction of the current block is bi-prediction, the prediction direction information may be set to 10. In an embodiment, if the prediction direction of the UMVE candidate is bi-prediction and the prediction direction of the current block is bi-prediction in a different direction, the prediction direction information may be set to 11.

[0282] As another example, if the prediction direction of the UMVE candidate is list 1 single prediction, when the prediction direction of the current block is list 1 single prediction, the prediction direction information can be set to 0.

[0283] If the prediction direction of the current block is bi-prediction or list 0 uni-prediction, the prediction direction information may be set to 10 or 11. In an embodiment, if the prediction direction of the UMVE candidate is bi-prediction and the prediction direction of the current block is bi-prediction, the prediction direction information may be set to 10. In an embodiment, if the prediction direction of the UMVE candidate is bi-prediction and the prediction direction of the current block is bi-prediction in a different direction, the prediction direction information may be set to 11.

[0284] As another example, if the prediction direction of the UMVE candidate is bi-prediction, the prediction direction information can be set to 0 when the prediction direction of the current block is bi-prediction.

[0285] If the prediction direction of the current block is list 0 uni-prediction or list 1 uni-prediction, the prediction direction information may be set to 10 or 11. In an embodiment, if the prediction direction of the UMVE candidate is bi-prediction and the prediction direction of the current block is list 0 uni-prediction, the prediction direction information may be set to 10. In an embodiment, if the prediction direction of the UMVE candidate is bi-prediction and the prediction direction of the current block is list 1 uni-prediction, the prediction direction information may be set to 11.

[0286] If the prediction direction information indicates a change in the prediction direction, the decoding unit (1730) can change the motion vector of the UMVE candidate according to the changed prediction direction. For example, if the prediction direction of the UMVE candidate is list 0 uniprediction and the prediction direction of the current block is list 1 uniprediction, the motion vector of list 0 is modified to be suitable for prediction according to the reference picture of list 1. Specifically, the motion vector of list 0 can be modified by being scaled according to the ratio of the temporal distance between the reference picture of list 0 and the current picture and the temporal distance between the reference picture of list 1 and the current picture.

[0287] For example, when the motion vector of list 0 is (8,6), and the ratio of the temporal distance between the reference picture of list 0 and the current picture and the temporal distance between the reference picture of list 1 and the current picture is 2:1, the motion vector of list 1 can be determined as (4,3). If the ratio of the temporal distances is 2:-1, that is, when the reference picture of list 1 and the reference picture of list 0 are in different directions from the current picture, the motion vector of list 1 can be determined as (-4,-3).

[0288] If the prediction direction of the UMVE candidate is uni-prediction of list 0 and the prediction direction of the current block is bi-prediction, the motion vector of list 1 can be obtained based on the motion vector of list 0 of the UMVE candidate. The motion vector of list 1 can be obtained by scaling the motion vector for the reference picture of list 0 according to the ratio of the temporal distance between the reference picture of list 0 and the current picture and the temporal distance between the reference picture of list 1 and the current picture. And the reference picture of list 1 can be determined as a picture located in a specific order of list 1. And the reference picture and motion vector of list 0 and the reference picture and motion vector of list 1 can both be used for prediction of the current block.

[0289] Similarly, even when the predicted direction of the UMVE candidate is a single prediction in List1 and the predicted direction of the current block is a paired prediction, the motion vector of List0 can be obtained from the motion vector of List1. And the reference picture of List0 can be determined as the picture located in a specific order of List0. And both the reference picture and motion vector of List0 and the reference picture and motion vector of List1 can be used for the prediction of the current block.

[0290] If the predicted direction of the UMVE candidate is a paired prediction and the predicted direction of the current block is a single prediction in List0, the current block is predicted using only the motion vector and reference picture of List0 among the motion vector and reference picture of List0 and the motion vector and reference picture of List1. Conversely, if the predicted direction before change is a paired prediction and the changed predicted direction is a single prediction in List1, the current block is predicted using only the motion vector and reference picture of List1 among the motion vector and reference picture of List0 and the motion vector and reference picture of List1.

[0291] When the main prediction direction information is obtained, the decoding unit (1730) can change the motion vector of the UMVE candidate according to the main prediction direction indicated by the main prediction direction information. If the main prediction direction of the current picture is the same as the reference "nun sleeve" of the UMVE candidate, the current block is predicted according to the reference picture and motion vector of the UMVE candidate. However, if the main prediction direction of the current picture is different from the reference "nun sleeve" of the UMVE candidate, the motion vector of the UMVE candidate is changed according to the main prediction direction. The change of the motion vector and reference picture of the UMVE candidate according to the main prediction direction is performed in the same way as the change of the motion vector and reference picture of the UMVE candidate according to the prediction direction information.

[0292] According to one embodiment, the decoding unit (1730) may change the prediction direction of the current block according to the prediction direction information when the main prediction direction information is acquired and a change in the prediction direction is allowed according to the prediction direction change permission information. When the main prediction direction information is set, the prediction direction information indicates the prediction direction of the current block among the main prediction direction and two preliminary prediction directions. If the prediction direction information indicates the main prediction direction, the current block is predicted according to the main prediction direction. However, if the prediction direction information indicates the preliminary prediction direction, the current block is predicted according to the prediction direction indicated by the prediction direction information among the two preliminary prediction directions. When the main prediction direction information is set, 0 may be assigned to the main prediction direction, 10 may be assigned to the first preliminary prediction direction, and 11 may be assigned to the second preliminary prediction direction in the prediction direction information. In addition, the two preliminary prediction directions for the main prediction direction may be determined in various ways according to the embodiment.

[0293] If the decoding unit (1730) can correct the basic motion vector multiple times according to the multiple correction allowance information, the corrected basic motion vector can be corrected once more. Accordingly, the decoding unit (1730) can obtain additional correction size information and additional correction direction information from the bitstream. In addition, the decoding unit (1730) can correct the basic motion vector once more based on the additional correction size determined according to the additional correction size information and the additional correction direction determined according to the additional correction direction information.

[0294] When the correction size is greater than or equal to a threshold, the decoding unit (1730) can acquire additional correction size information and additional correction direction information. Furthermore, the additional correction size and additional correction direction indicated by the additional correction size information and additional correction direction information may be limited depending on the correction size and the correction direction. For example, the additional correction size may be determined to be smaller than the correction size. Alternatively, the additional correction direction may be determined to be different from the correction direction.

[0295] The decoding unit (1730) can reconstruct the current block based on the motion vector and reference picture of the current block. When the prediction direction of the current block is list 0 uni-prediction or list 1 uni-prediction, the decoding unit (1730) can predict the current block using one motion vector and reference picture. When the prediction direction of the current block is bi-prediction, the decoding unit (1730) can predict the current block using the motion vector and reference picture of list 0 and the motion vector and reference picture of list 1.

[0296] The decryption unit (1730) can restore the current block based on the prediction result of the current block. The decryption unit (1730) can restore the current block based on the predicted block of the current block and the residual block of the current block. The residual block of the current block represents the difference value between the restored block of the current block and the predicted block.

[0297] Information on the number of UMVE candidates to be decoded in the video decoding device (1700), information on the number of correction size candidates, information on the number of correction direction candidates, information on the composition of the UMVE candidate list, information on the range of correction sizes, information on the main prediction direction, information on the UMVE index, information on the correction size and the correction direction, information on the prediction direction, information on the additional correction size and the additional correction direction, etc. may be binarized according to fixed length coding or truncated unary coding.

[0298] The functions of the upper parameter determination unit (1710), the encoding mode determination unit (1720), and the decoding unit (1730) can be performed in the decoding unit (170) of FIG. 1b.

[0299] Figures 23 to 25 illustrate an embodiment of a method for selecting an inter-mode tool for a reference picture and a motion vector of a current block. Figures 23 to 25 commonly illustrate a syntax structure of a coding unit. According to the syntax structure according to the embodiment, when the current block is applied with skip mode or direct mode, the UMVE mode can be selected. ae(v) in Figures 23 to 25 indicates that the syntax elements in the same row are entropy decoded. Accordingly, cu_skip_flag[ x0 ][ y0 ], cu_skip_umve[ x0 ][ y0 ] of Fig. 23, cu_direct[ x0 ][ y0 ], affine_dir[ x0 ][ y0 ] of Fig. 24, and cu_direct[ x0 ][ y0 ], cu_direct_umve[ x0 ][ y0 ] of Fig. 25 are entropy decrypted.

[0300] Figure 23 discloses a method for selecting an inter-mode tool for the current block in skip mode. In skip mode, most of the encoding information required for restoring the current block is omitted, and the current block is restored using only a portion of the encoding information. Therefore, by omitting the encoding information required for restoring the current block, encoding efficiency can be increased.

[0301] According to an embodiment of FIG. 23, in skip mode, an inter mode tool can be used. In one embodiment, in skip mode, umve mode, affine mode, and merge mode can be used. In one embodiment, cu_skip_flag[ x0 ][ y0 ] is acquired, which indicates whether skip mode is applied to the current block. If skip mode is applied to the current block (if( cu_skip_flag[ x0 ][ y0 ])), cu_skip_umve[ x0 ][ y0 ] is acquired, which indicates whether the inter mode tool of the current block is umve mode. If the umve mode is applied to the current block (if( cu_skip_umve[ x0 ][ y0 ] ), cu_mode = SKIP_UMVE), information about the base motion vector of the current picture and the adjacent blocks containing the reference picture, as well as the correction size information and correction direction information for the correction of the base motion vector, are obtained (mvp_idx_umve( x0, y0, identical_list0_list1 flag )). If the umve mode is not applied to the current block, the affine mode (cu_mode = SKIP_AFFINE) or merge mode (cu_mode = SKIP) can be applied to the current block.

[0302] In the embodiment of Fig. 23, the application of the umve mode, affine mode, and merge mode was determined in that order. However, a person skilled in the art can easily change the type of inter-mode tool applied in skip mode. Furthermore, a person skilled in the art can change the order in which the inter-mode tools applied in skip mode are determined.

[0303] Figure 24 discloses a method for selecting an inter-mode tool for a current block in direct mode when the slice type of the current slice containing the current block is type P. In direct mode, some of the encoding information required to determine the motion vector of the current block is omitted. Therefore, encoding efficiency can be increased by reducing the encoding information required to determine the motion vector of the current block.

[0304] According to the embodiment of FIG. 24, when the slice type of the current slice is P type, the umve mode and the affine mode can be used in the direct mode. According to one embodiment, cu_direct[ x0 ][ y0 ], which indicates whether the direct mode is applied to the current block, is obtained. If the direct mode is applied to the current block (if( cu_direct[ x0 ][ y0 ])), affine_dir[ x0 ][ y0 ], which indicates whether the inter mode tool of the current block is the affine mode, is obtained. If the affine mode is not applied to the current block (if( !affine_dir[ x0 ][ y0 ])), it is determined that the umve mode is applied to the current block (cu_mode = DIR_UMVE). And, when the umve mode is applied to the current block, information about the adjacent block including the base motion vector and reference picture of the current picture, correction size information about the correction of the base motion vector, and correction direction information are obtained (mvp_idx_umve( x0, y0, identical_list0_list1 flag )).

[0305] In the embodiment of Fig. 24, the application of affine mode and then umve mode was determined in that order. However, a person skilled in the art can easily change the type of inter-mode tool applied in direct mode. Furthermore, a person skilled in the art can change the order in which inter-mode tools are determined to be applied in direct mode.

[0306] FIG. 25 discloses a method for selecting an inter mode tool of a current block in direct mode when the slice type of the current slice including the current block is type B.

[0307] According to the embodiment of Fig. 25, when the slice type of the current slice is type B, umve mode, affine mode, and merge mode can be used. According to one embodiment, cu_direct[ x0 ][ y0 ] is obtained, which indicates whether direct mode is applied to the current block. If direct mode is applied to the current block (if( cu_direct[ x0 ][ y0 ])), cu_direct_umve[ x0 ][ y0 ] is obtained, which indicates whether the inter mode tool of the current block is umve mode. If the umve mode is applied to the current block (if( cu_direct_umve[ x0 ][ y0 ] ), cu_mode = DIR_UMVE), information about the base motion vector of the current picture and the adjacent blocks containing the reference picture, as well as the correction size information and correction direction information for the correction of the base motion vector, are obtained (mvp_idx_umve( x0, y0, identical_list0_list1 flag )). If the umve mode is not applied to the current block, the affine mode (cu_mode = DIR_AFFINE) or merge mode (cu_mode = DIR) can be applied to the current block.

[0308] In the embodiment of Fig. 25, the application of the umve mode, affine mode, and merge mode was determined in that order. However, a person skilled in the art can easily change the type of inter-mode tool applied in direct mode. Furthermore, a person skilled in the art can change the order in which the inter-mode tool applied in direct mode is determined.

[0309] FIG. 26 illustrates a flowchart of a decoding method according to the UMVE mode of the video decoding device (1700) of FIG. 17.

[0310] In step 2610, it is determined whether the upper data unit containing the current block allows UMVE mode. Whether the UMVE mode is allowed for the upper data unit may be determined based on the UMVE permission flag obtained from the bitstream.

[0311] When the UMVE mode is allowed in the upper data unit, UMVE candidate count information indicating the number of UMVE candidates of the UMVE mode allowed in the upper data unit and UMVE candidate list configuration information indicating a method of configuring a motion vector candidate list to be referenced by multiple blocks of the upper data unit can be obtained from the bitstream.

[0312] Information on the number of correction size candidates, which indicates the number of correction size candidates, can be obtained from a bitstream. Information on the range of correction sizes for multiple blocks of an upper data unit can be obtained. In addition, information on changing correction sizes for multiple blocks of an upper data unit can be obtained.

[0313] Information on the number of correction direction candidates, which indicates the number of correction direction candidates, can be obtained from the bitstream.

[0314] Prediction direction change permission information for multiple blocks of an upper data unit can be obtained from a bitstream. In addition, main prediction direction information indicating a main prediction direction of an upper data unit can be obtained from the bitstream.

[0315] In step 2620, if the upper data unit allows UMVE mode, it is determined whether the UMVE mode is applied to the current block. Whether the UMVE mode is applied to the current block can be determined based on the UMVE flag obtained from the bitstream. If the UMVE mode is not applied to the current block, an inter prediction mode other than the UMVE mode can be applied to the current block.

[0316] The number of UMVE candidates for the current block indicated by the UMVE candidate count information can be determined. Furthermore, the method for constructing the UMVE candidate list can be determined based on the UMVE candidate list composition information. The UMVE candidate list for the current block can be constructed based on the number of UMVE candidates and the method for constructing the UMVE candidate list.

[0317] In addition, the number of correction size candidates of the current block indicated by the correction size candidate count information can be determined. The correction size candidates can be determined based on the correction size range indicated by the correction size range information. In addition, when the correction size change information indicates that the correction size of the base motion vector is adaptively changed according to the distance between the current picture of the current block and the reference picture, the correction size range can be changed according to the distance between the current picture of the current block and the reference picture.

[0318] Additionally, the number of correction direction candidates of the current block indicated by the correction direction candidate number information can be determined.

[0319] When the UMVE mode is applied to the current block and the prediction direction change permission information indicates that a change in the prediction direction is allowed, prediction direction information can be obtained from the bitstream.

[0320] In step 2630, when the UMVE mode is applied to the current block, the base motion vector of the current block is determined. An UMVE candidate is determined from the determined UMVE candidate list. Then, a reference picture and a base motion vector can be extracted from an adjacent block of the current block indicated by the UMVE candidate.

[0321] In step 2640, the correction size and direction for correcting the basic motion vector are determined. The correction size of the basic motion vector can be determined among the correction size candidates based on the correction size information obtained from the bitstream. The correction direction of the basic motion vector can be determined among the correction direction candidates based on the correction direction information obtained from the bitstream.

[0322] The corrected basic motion vector may be further corrected according to an additional correction size and an additional correction direction. In an embodiment, when the correction size is greater than a threshold, additional correction of the basic motion vector may be performed. The additional correction size may be determined to be smaller than the correction size.

[0323] In step 2650, the motion vector of the current block is determined by correcting the base motion vector according to the correction size and the correction direction. In addition, the motion vector of the current block and the reference picture may be changed according to the prediction direction indicated by the prediction direction information. If the main prediction direction is determined for the upper data unit, the prediction direction information may indicate the prediction direction of the current block among the main prediction direction and two preliminary prediction directions. If the main prediction direction is not determined for the upper data unit, the prediction direction information may indicate the prediction direction of the UMVE candidate and the prediction direction of the current block among the remaining two prediction directions.

[0324] At step 2660, the current block is restored based on the motion vector of the current block.

[0325] The inter prediction method according to the UMVE mode performed in the video decoding device (1700) of FIG. 17 can be applied to the decoding method according to the UMVE mode of FIG. 26.

[0326] FIG. 27 illustrates a block diagram of a video encoding device (2800) that performs inter prediction according to the UMVE mode.

[0327] The video encoding device (2700) includes an encoding parameter determination unit (2710), a motion vector determination unit (2720), an encoding information determination unit (2730), and an output unit (2740). In FIG. 27, the encoding parameter determination unit (2710), the motion vector determination unit (2720), the encoding information determination unit (2730), and the output unit (2740) are expressed as separate configuration units, but according to an embodiment, the encoding parameter determination unit (2710), the motion vector determination unit (2720), the encoding information determination unit (2730), and the output unit (2740) may be combined and implemented as a single configuration unit.

[0328] In FIG. 27, the encoding parameter determination unit (2710), the motion vector determination unit (2720), the encoding information determination unit (2730), and the output unit (2740) are expressed as components located in one device, but the devices responsible for each function of the encoding parameter determination unit (2710), the motion vector determination unit (2720), the encoding information determination unit (2730), and the output unit (2740) do not necessarily need to be physically adjacent. Therefore, depending on the embodiment, the encoding parameter determination unit (2710), the motion vector determination unit (2720), the encoding information determination unit (2730), and the output unit (2740) may be distributed.

[0329] The encoding parameter determination unit (2710), the motion vector determination unit (2720), the encoding information determination unit (2730), and the output unit (2740) may be implemented by one processor according to an embodiment. Additionally, they may be implemented by multiple processors according to an embodiment.

[0330] The encoding parameter determination unit (2710) determines whether the UMVE mode is allowed for the upper data unit including the current block. The encoding parameter determination unit (2710) may determine whether the UMVE mode is allowed based on the frequency with which the UMVE mode is used in the upper data unit. Alternatively, the encoding parameter determination unit (2710) may determine whether the UMVE mode is allowed based on the characteristics of the image.

[0331] The motion vector determination unit (2720) determines the motion vector of the current block. The motion vector determination unit (2720) can obtain multiple motion vectors from surrounding blocks of the current block. In addition, the motion vector determination unit (2720) can determine the motion vector of the current block by scanning the surroundings of the multiple motion vectors. The motion vector determination unit (2720) can determine the optimal motion vector as the motion vector of the current block according to rate-distortion optimization.

[0332] The encoding information determination unit (2730) determines whether the UMVE mode is applied to the current block based on the motion vector if the UMVE mode is allowed for the upper data unit. The encoding information determination unit (2730) can determine whether the motion vector of the current block determined by the motion vector determination unit (2720) can be derived by the UMVE mode. If the motion vector of the current block can be derived by the UMVE mode, the encoding information determination unit (2730) can determine to apply the UMVE mode to the current block. Alternatively, if the minimum error between the motion vector of the current block and the motion vector according to the UMVE mode is less than or equal to a threshold, the encoding information determination unit (2730) can determine to apply the UMVE mode to the current block.

[0333] When the UMVE mode is applied to the current block, the encoding information determination unit (2730) determines the basic motion vector of the current block and the correction size and correction direction for correction of the basic motion vector according to the motion vector of the current block. The encoding information determination unit (2730) can determine the correction size and correction direction according to the difference between the motion vector of the current block and the basic motion vector of the current block.

[0334] The output unit (2740) outputs a bitstream including encoding information according to the UMVE mode of the current block. The encoding information may include a UMVE permission flag, a UMVE flag, information on the number of UMVE candidates, information on the number of correction size candidates, information on the number of correction direction candidates, information on the composition of a UMVE candidate list, information on the range of correction size, information on changing the correction size, information on allowing a change in the prediction direction, information on the main prediction direction, information on allowing multiple corrections, etc.

[0335] The functions of the encoding parameter determination unit (2710), the motion vector determination unit (2720), and the encoding information determination unit (2730) can be performed in the encoding unit (110) of Fig. 1a. The function of the output unit (2740) can be performed in the bitstream generation unit (120) of Fig. 1a.

[0336] FIG. 28 illustrates a flowchart of an encoding method according to the UMVE mode of the video encoding device (2800) of FIG. 27.

[0337] In step 2810, it is determined whether the upper data unit containing the current block allows UMVE mode. Whether UMVE mode is allowed may be determined based on the frequency with which UMVE mode is used in the upper data unit. Alternatively, whether UMVE mode is allowed may be determined based on the characteristics of the image.

[0338] In step 2820, a motion vector for the current block is determined. Multiple motion vectors can be acquired from neighboring blocks of the current block. By scanning the neighborhood of the multiple motion vectors, the motion vector of the current block can be determined. Based on rate-distortion optimization, the optimal motion vector can be determined as the motion vector of the current block.

[0339] In step 2830, if the UMVE mode is allowed for the upper data unit, it is determined whether the UMVE mode is applied to the current block based on the motion vector. It can be determined whether the motion vector of the current block can be derived by the UMVE mode. If the motion vector of the current block can be derived by the UMVE mode, it can be determined that the UMVE mode is applied to the current block. Alternatively, if the minimum error between the motion vector of the current block and the motion vector according to the UMVE mode is less than or equal to a threshold, it can be determined that the UMVE mode is applied to the current block.

[0340] In step 2840, the base motion vector of the current block and the correction size and correction direction for correction of the base motion vector are determined based on the motion vector. The correction size and correction direction can be determined based on the difference between the motion vector of the current block and the base motion vector of the current block.

[0341] At step 2850, a bitstream containing information about the UMVE mode of the current block is output.

[0342] The inter prediction method according to the UMVE mode performed in the video encoding device (2700) of FIG. 27 can be applied to the encoding method according to the UMVE mode of FIG. 28.

[0343] According to the video encoding technique based on the coding units of the tree structure described above with reference to FIGS. 1 to 28, image data of a spatial domain is encoded for each coding unit of the tree structure, and the image data of the spatial domain is restored while decoding is performed for each maximum coding unit according to the video decoding technique based on the coding units of the tree structure, so that a video which is a picture and a picture sequence can be restored. The restored video can be played back by a playback device, stored in a storage medium, or transmitted through a network.

[0344] Meanwhile, the embodiments of the present disclosure described above can be written as a program that can be executed on a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable recording medium.

[0345] While this disclosure has been described with reference to specific preferred embodiments, alternatives, modifications, and variations of this disclosure will be apparent to those skilled in the art in light of the foregoing description. Therefore, the claims are intended to encompass all such alternatives, modifications, and variations. Therefore, all matters set forth in this specification and drawings are to be construed in an illustrative and non-limiting sense.

Claims

1. A step of determining whether UMVE mode is allowed for the upper data unit containing the current block; A step of determining whether the UMVE mode is applied to the current block when the UMVE mode is allowed in the above upper data unit; When the UMVE mode is applied to the current block, a step of determining a basic motion vector of the current block; A step of determining a correction size and a correction direction for correction of the above basic motion vector; A step of determining a motion vector of the current block by correcting the basic motion vector according to the correction size and the correction direction; A video decoding method comprising a step of restoring the current block based on a motion vector of the current block.

2. In paragraph 1, A video decoding method, characterized in that the above upper data unit is one of a sequence, a picture, and a slice.

3. In paragraph 1, The step of determining whether the above UMVE mode is allowed is: A step of obtaining a UMVE permission flag from a bitstream; and A step of determining whether the UMVE mode is allowed in the upper data unit according to the UMVE permission flag, The step of determining whether UMVE mode is applied to the current block is: A step of obtaining a UMVE flag indicating whether the UMVE mode is applied to the current block from the bitstream; and Including a step of determining whether the UMVE mode is applied to the current block according to the UMVE flag; The step of determining the basic motion vector of the current block is: A step of obtaining a UMVE index indicating a UMVE candidate including a base motion vector and a reference picture of the current block from the bitstream; and A step of determining a base motion vector and a reference picture of the current block according to the UMVE index, The step of determining the above correction size and the above correction direction is: A step of obtaining correction size information indicating a correction size of the basic motion vector and correction direction information indicating a correction direction of the basic motion vector from the bitstream; and A video decoding method characterized by comprising a step of determining a correction size and a correction direction of the basic motion vector according to the correction size information and the correction direction information.

4. In paragraph 1, The step of determining whether the above UMVE mode is applied is: A video decoding method, characterized in that it includes a step of applying a first inter prediction mode other than the UMVE mode to the current block when the UMVE mode is not applied to the current block.

5. In paragraph 1, The above video decryption method is, When the UMVE mode is allowed in the upper data unit, a step of obtaining UMVE candidate count information indicating the number of UMVE candidates of the UMVE mode allowed in the upper data unit from the bitstream, correction size candidate count information indicating the number of correction size candidates, and correction direction candidate count information indicating the number of correction direction candidates; and When the UMVE mode is applied to the current block, the step of determining the reference picture candidate of the current block, the correction size candidate, and the correction direction candidate is further included according to the number of UMVE candidates indicated by the UMVE candidate number information, the number of correction size candidates indicated by the correction size candidate number information, and the number of correction direction candidates indicated by the correction direction candidate number information. The step of determining the above basic motion vector is: Determine the basic motion vector of the current block based on the basic motion vector selected from the above UMVE candidates, The step of determining the above correction size and the above correction direction is: A video decoding method characterized by determining a correction size from the above correction size candidate and determining a correction direction from the above correction direction candidate.

6. In paragraph 1, The above video decryption method is, If the UMVE mode is allowed in the upper data unit, the method further includes obtaining UMVE candidate list configuration information indicating a method of configuring a motion vector candidate list to be referenced by a plurality of blocks of the upper data unit from the bitstream, The step of determining the above basic motion vector is: A step of configuring a UMVE candidate list of the current block according to the UMVE candidate list configuration information; and A video decoding method, characterized by comprising a step of determining a base motion vector and a reference picture of a current block from the above UMVE candidate list.

7. In paragraph 6, The above UMVE candidate list configuration information is: A video decoding method characterized by indicating a first list determination method for determining a UMVE candidate list according to a merge motion vector candidate list or a second list determination method for determining a UMVE candidate list based on motion vectors frequently used in a current picture.

8. In paragraph 1, The above video decryption method is, If the UMVE mode is allowed in the upper data unit, the step of obtaining compensation size range information for a plurality of blocks of the upper data unit from the bitstream is further included. The step of determining the above correction size and the above correction direction is: A video decoding method characterized by comprising a step of determining a correction size of the basic motion vector based on a correction size range indicated by the correction size range information.

9. In paragraph 1, The above video decryption method is, If the UMVE mode is allowed in the upper data unit, the step of obtaining correction size change information for a plurality of blocks of the upper data unit from the bitstream is further included. The step of determining the above correction size and the above correction direction is: A video decoding method, characterized in that it includes a step of changing the correction size of the basic motion vector according to the distance between the current picture of the current block and the reference picture, when the correction size change information indicates that the correction size of the basic motion vector is adaptively changed according to the distance between the current picture of the current block and the reference picture.

10. In paragraph 1, The above video decryption method is, When the UMVE mode is allowed in the upper data unit, a step of obtaining prediction direction change permission information for a plurality of blocks of the upper data unit from the bitstream; and A step of obtaining prediction direction information from the bitstream when the UMVE mode is applied to the current block and the prediction direction change permission information indicates that a change in the prediction direction is permitted; The step of determining the above motion vector is: A video decoding method characterized by comprising a step of changing a motion vector of the current block according to a prediction direction indicated by the prediction direction information.

11. In paragraph 10, The above video decryption method includes: If the UMVE mode is allowed in the above upper data unit, a step of determining the main prediction direction of the UMVE mode; and The step of determining the above motion vector is: a step of determining the motion vector based on the main prediction direction; and When the above prediction direction change information indicates that the prediction direction is changed, a step of changing the basic motion vector according to an auxiliary prediction direction other than the main prediction direction is included. The step of determining the above correction size and the above correction direction is: A step of determining the correction size and the correction direction based on the main prediction direction; and A video decoding method, characterized in that it includes a step of changing the correction size and the correction direction according to a preliminary prediction direction other than the main prediction direction when the prediction direction change information indicates that the prediction direction is changed.

12. In paragraph 1, The above video decryption method is, When the above correction size is greater than or equal to a threshold, the step of determining an additional correction size and an additional correction direction for additional correction of the basic motion vector is further included. The motion vector determination step of the current block above. Determine the motion vector of the current block according to the above correction size, the correction direction, the additional correction size, and the additional correction direction, A video decoding method characterized in that the additional correction size is smaller than the correction size.

13. Determine whether UMVE mode is allowed for the upper data unit containing the current block, If the UMVE mode is allowed in the above upper data unit, determine whether the UMVE mode is applied to the current block, When the UMVE mode is applied to the current block, the basic motion vector of the current block is determined, and the correction size and correction direction for correction of the basic motion vector are determined. Determine the motion vector of the current block by correcting the basic motion vector according to the correction size and the correction direction, A video decoding device comprising a processor for restoring the current block based on a motion vector of the current block.

14. A step for determining whether UMVE mode is allowed for the upper data unit containing the current block; A step of determining a motion vector for the current block; A step of determining whether the UMVE mode is applied to the current block according to the motion vector, when the UMVE mode is allowed in the upper data unit; A step of determining a basic motion vector of the current block and a correction size and a correction direction for correction of the basic motion vector according to the above motion vector; and A video encoding method comprising the step of outputting a bitstream including information about the UMVE mode of the current block.

15. Determine whether UMVE mode is allowed for the upper data unit containing the current block, Determine the motion vector for the current block, If the UMVE mode is allowed in the above upper data unit, it is determined whether the UMVE mode is applied to the current block according to the motion vector, Determine the basic motion vector of the current block and the correction size and correction direction for correction of the basic motion vector according to the above motion vector, A video encoding device comprising a processor that outputs a bitstream including encoding information according to the UMVE mode of the current block.

Images