Encoding method and device thereof, and decoding method and device thereof

Abstract

To provide a video decoding method.SOLUTION: A video decoding method includes: determining whether a UMVE mode is allowed for an upper data unit including a current block; when the UMVE mode is allowed for the upper data unit, determining whether the UMVE mode is applied to the current block; when the UMVE mode is applied to the current block, determining a base motion vector of the current block; determining a correction distance and a correction direction for correction of the base motion vector; determining a motion vector of the current block by correcting the base motion vector according to the correction distance and the correction direction; and reconstructing the current block on the basis of the motion vector of the current block.SELECTED DRAWING: Figure 28

Description

[Technical field]

[0001] The present invention relates to video encoding and decoding methods, and more particularly to a method for efficiently encoding and decoding information related to motion vectors. [Background technology]

[0002] High quality video requires a large amount of data when it is encoded. However, the bandwidth allowed for transmitting video data is limited, and the data rate applied when transmitting the video data is restricted. Therefore, for efficient video data transmission, a method for encoding and decoding video data with an increased compression rate while minimizing degradation of image quality is required.

[0003] Video data is also compressed by removing spatial and temporal redundancy between pixels. Since adjacent pixels typically have common characteristics, coding information is transmitted in data units made up of pixels to remove the redundancy between adjacent pixels.

[0004] The pixel values ​​of the pixels included in the data unit are not directly transmitted, but a method required to obtain the pixel values ​​is transmitted. A prediction method for predicting the pixel value similar to the original value is determined for each data unit, and coding information related to the prediction method is transmitted from the encoder to the decoder. Also, since the predicted value is not completely the same as the original value, residual data related to the difference between the original value and the predicted value is transmitted from the encoder to the decoder.

[0005] As prediction becomes more accurate, the coding information required to specify a prediction method increases, but the size of residual data is reduced. Thus, a prediction method is determined taking into consideration the sizes of the coding information and the residual data. In particular, data units divided by a picture have various sizes, and as the data unit becomes larger, the accuracy of prediction is more likely to decrease, but the coding information is reduced. Thus, the size of a block is determined to match the picture characteristics.

[0006] Prediction methods include intra prediction and inter prediction. Intra prediction is a method of predicting pixels of a block from pixels surrounding the block. Inter prediction is a method of predicting pixels by referring to pixels of other pictures referenced by the picture containing the block. Thus, spatial redundancy is eliminated by intra prediction, and temporal redundancy is eliminated by inter prediction.

[0007] As the number of prediction methods increases, the amount of coding information to indicate the prediction methods increases. Therefore, the coding information applied to a block can also be predicted from other blocks to reduce the size of the coding information.

[0008] Although video data loss is permitted to the extent that it cannot be perceived by human vision, the residual data can be lossily compressed through conversion and quantization processes to reduce the amount of residual data. Summary of the Invention [Problem to be solved by the invention]

[0009] A video encoding method and a video encoding device that perform inter prediction in an ultimate motion vector expression (UMVE) mode are disclosed. Also, a video decoding method and a video decoding device that perform inter prediction in an UMVE mode are disclosed. Also, a computer-readable recording medium having a program recorded thereon for causing a computer to execute the video encoding method and the video decoding method according to an embodiment of the present disclosure is disclosed. [Means for solving the problem]

[0010] In the present disclosure, a video decoding method is provided, including a step of determining whether UMVE mode is allowed for an upper data unit including a current block, and if UMVE mode is allowed for the upper data unit, a step of determining whether UMVE mode is applied to the current block, and if UMVE mode is applied to the current block, a step of determining a base motion vector of the current block, a step of determining a correction size and a correction direction for correcting the base motion vector, a step of correcting the base motion vector according to the correction size and the correction direction and determining a motion vector of the current block, and a step of restoring the current block based on the motion vector of the current block.

[0011] In the present disclosure, a video decoding device is provided that includes a processor that determines whether UMVE mode is allowed for an upper data unit including a current block, and if UMVE mode is allowed for the upper data unit, determines whether UMVE mode is applied to the current block, and if UMVE mode is applied to the current block, determines a base motion vector of the current block, determines a correction size and a correction direction for correcting the base motion vector, corrects the base motion vector based on the correction size and the correction direction, determines a motion vector of the current block, and restores the current block based on the motion vector of the current block.

[0012] A video encoding method is provided, comprising: a step of determining whether UMVE mode is allowed for an upper data unit including a current block; a step of determining a motion vector for the current block; if UMVE mode is allowed for the upper data unit, a step of determining whether UMVE mode is applied to the current block based on the motion vector; a step of determining a base motion vector of the current block and a correction size and correction direction for correcting the base motion vector based on the motion vector; and a step of outputting a bitstream including information related to the UMVE mode of the current block.

[0013] A video encoding device is provided that includes a processor that determines whether UMVE mode is allowed for an upper data unit including a current block, determines a motion vector related to the current block, and if UMVE mode is allowed for the upper data unit, determines whether UMVE mode is applied to the current block based on the motion vector, determines a base motion vector of the current block and a correction size and correction direction for correcting the base motion vector based on the motion vector, and outputs a bitstream including encoding information according to the UMVE mode of the current block.

[0014] A computer-recordable recording medium having a program for performing the video encoding method and the video decoding method recorded thereon is provided.

[0015] The technical problems to be solved by this embodiment are not limited to those described above, and other technical problems can be inferred from the following embodiments. Effect of the Invention

[0016] The coding rate of a video can be improved by performing inter prediction on blocks included in the video in the UMVE mode. [Brief description of the drawings]

[0017] [Figure 1A] 1 is a block diagram of a video encoding device based on a coding unit having a tree structure according to an embodiment of the present disclosure. [Figure 1B] 1 is a block diagram of a video decoding device based on a coding unit having a tree structure according to an embodiment; [Diagram 2] 1 is a diagram illustrating a process in which a current coding unit is divided and at least one coding unit is determined, according to one embodiment. [Diagram 3] 1 illustrates a process in which a non-square coding unit is divided to determine at least one coding unit, according to one embodiment. [Figure 4] 1 is a diagram illustrating a process of dividing a coding unit based on at least one of block configuration information and division configuration information according to an embodiment; [Diagram 5] 1 is a diagram illustrating a method for determining a predetermined coding unit among an odd number of coding units according to one embodiment. [Figure 6] 10 is a diagram illustrating an order in which a plurality of coding units are processed when a current coding unit is divided and a plurality of coding units are determined, according to one embodiment. [Figure 7] 11 is a diagram illustrating a process of determining that a current coding unit is to be divided into an odd number of coding units when coding units cannot be processed in a predetermined order, according to one embodiment. [Figure 8] 1 illustrates a process in which a first coding unit is divided to determine at least one coding unit according to one embodiment. [Figure 9] FIG. 13 is a diagram illustrating that, according to one embodiment, when a non-square second coding unit determined by dividing a first coding unit satisfies a certain condition, the manner in which the second coding unit may be divided is restricted. [Figure 10] 11 is a diagram illustrating a process of dividing a square-shaped coding unit when division form information does not indicate division into four square-shaped coding units according to one embodiment. FIG. [Figure 11] 1 is a diagram illustrating that a processing order among a plurality of coding units may differ depending on a division process of the coding units, according to an embodiment. [Figure 12] 11 is a diagram illustrating a process of determining the depth of a coding unit according to changes in the shape and size of the coding unit when the coding unit is recursively divided to determine multiple coding units according to one embodiment. [Figure 13] 1 is a diagram illustrating a depth determined according to the type and size of a coding unit and an index (PID: part index) for a coding unit partition according to an embodiment. [Figure 14]1 is a diagram illustrating a plurality of coding units being determined according to a plurality of predetermined data units included in a picture, according to an embodiment; [Figure 15] 1 is a diagram illustrating processing blocks that are used to determine the order in which reference coding units included in a picture are determined according to one embodiment. [Figure 16] FIG. 1 is a diagram illustrating an inter prediction method in UMVE mode. [Figure 17] FIG. 2 is a block diagram of a video decoding device that performs decoding in UMVE mode. [Figure 18] 1 is a diagram illustrating UMVE mode motion vector candidates having a diamond-shaped distribution based on a base motion vector; [Figure 19] 1 is a diagram illustrating UMVE mode motion vector candidates having a rectangular distribution based on a base motion vector. [Figure 20] A figure illustrating an embodiment of a UMVE mode having a different number of motion vector candidates for each group. [Figure 21] 1 is a diagram illustrating an embodiment of a UMVE mode in which the distribution pattern of motion vector candidates differs for each group; [Figure 22] FIG. 1 is a diagram showing an embodiment of a method for correcting two basic motion vectors by bi-prediction. [Diagram 23] 13A-13C are diagrams illustrating a method for selecting an inter mode tool for a current block in skip mode. [Figure 24] When the slice type of the current slice in which the current block is included is P type, a method of selecting an inter mode tool for the current block in direct mode is disclosed. [Diagram 25] When the slice type of the current slice in which the current block is included is type B, a method of selecting an inter mode tool for the current block in direct mode is disclosed. [Figure 26] 1 is a flowchart of a decoding method according to one embodiment of the UMVE mode. [Figure 27]FIG. 2 is a block diagram of a video encoding device with inter prediction according to one embodiment of the UMVE mode. [Figure 28] 1 is a flowchart of an encoding method according to one embodiment of the UMVE mode. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] A video decoding method is provided, including a step of determining whether an ultimate motion vector expression (UMVE) mode is allowed for an upper data unit including a current block, a step of determining whether an UMVE mode is applied to the current block if the UMVE mode is allowed for the upper data unit, a step of determining a base motion vector of the current block if the UMVE mode is applied to the current block, a step of determining a correction size and a correction direction for correcting the base motion vector, a step of correcting the base motion vector according to the correction size and the correction direction and determining a motion vector of the current block, and a step of restoring the current block based on the motion vector of the current block.

[0019] The features of the disclosed embodiments and methods for achieving them will become clear by referring to the following embodiments in conjunction with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below, and may be embodied in various different forms, and the present embodiments are provided only to complete the disclosure of the present disclosure and to fully inform those skilled in the art of the scope of the invention.

[0020] The terms used in this specification are briefly explained, and the disclosed embodiments are specifically explained.

[0021] The terms used in this specification are currently common terms that are used as much as possible while considering the function in this disclosure, but they may vary depending on the intention of the technical personnel in the relevant field, precedents, or the emergence of new technology. In addition, in certain cases, the applicant may arbitrarily select a term, and in such cases, the meaning of the term will be described in detail in the description of the invention. Therefore, the terms used in this disclosure must be defined based on the meaning of the term and the overall content of this disclosure, rather than simply the name of the term.

[0022] In this specification, the singular expression includes the plural expression unless the context clearly specifies otherwise.

[0023] Throughout the specification, when a part "includes" a certain component, it does not mean excluding other components, and may further include other components, unless otherwise specified to the contrary. In addition, the term "module" used in the specification means a software, a hardware component such as an FPGA or an ASIC, and a "module" performs a certain function. However, the meaning of "module" is not limited to software or hardware. A "module" may also be configured to be in an addressable recording medium and to execute one or more processors. Thus, by way of example, a "module" includes components such as software components, object-oriented software components, class components, and task components; as well as processes, functions, attributes, procedures, subroutines, program code segments, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The components and functions provided in the "module" may be combined into fewer components and "modules" or may be further separated into additional components and "modules".

[0024] The term "current block" refers to one of a coding unit, a prediction unit, and a transform unit that is currently being coded or decoded. If it is necessary to distinguish other types of blocks such as a prediction unit and a transform unit for convenience of explanation, the terms "current coding block", "current prediction block", and "current transform block" may also be used. In addition, the term "lower block" refers to a data unit divided from the "current block". And the term "upper block" refers to a data unit including the "current block".

[0025] Hereinafter, a "sample" refers to data assigned to a sampling position of an image and to be processed. For example, in a spatial domain image, a pixel value and a transform coefficient in a transform domain are also samples. A unit including at least one such sample can be defined as a block.

[0026] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present disclosure. In the drawings, portions that are not relevant to the description will be omitted in order to clearly explain the present disclosure.

[0027] FIG. 1A illustrates a block diagram of a video coding device 100 based on a tree-structured coding unit, according to one embodiment of the present disclosure.

[0028] The video encoding device 100 includes an encoding unit 110 and a bitstream generation unit 120 .

[0029] The encoding unit 110 divides a picture or a slice included in a picture into a plurality of maximum coding units according to the size of the maximum coding unit. The maximum coding unit is a data unit having a size such as 32x32, 64x64, 128x128, or 256x256, and is also a square data unit whose vertical and horizontal sizes are a power of 2. The encoding unit 110 may provide maximum coding unit size information indicating the size of the maximum coding unit to the bitstream generation unit 120. Then, the bitstream generation unit 120 may include the maximum coding unit size information in the bitstream.

[0030] The encoding unit 110 divides the maximum coding unit to determine the coding unit. Whether or not to divide the coding unit is determined depending on whether or not the division of the coding unit is efficient through rate-distortion optimization. Then, division information indicating whether or not the coding unit is divided is generated. The division information may also be expressed in the form of a flag.

[0031] Coding units are also split in various ways. For example, a square coding unit is also split into four square coding units with half the width and half the height. A square coding unit is also split into two rectangular coding units with half the width. A square coding unit is also split into two rectangular coding units with half the height. A square coding unit is also split into three coding units by splitting the width or height 1:2:1.

[0032] A rectangular coding unit whose width is twice its height is also split into two square coding units. A rectangular coding unit whose width is twice its height is also split into two rectangular coding units whose width is four times its height. A rectangular coding unit whose width is twice its height is also split into two rectangular coding units and one square coding unit by splitting the width 1:2:1.

[0033] Similarly, a rectangular coding unit whose height is twice its width is also split into two square coding units, a rectangular coding unit whose height is twice its width is also split into two rectangular coding units whose height is four times its width, and a rectangular coding unit whose height is twice its width is also split into two rectangular coding units and one square coding unit by splitting the height 1:2:1.

[0034] In the case where two or more partition methods are available in the video encoding device 100, information related to a partition method that can be used for a coding unit among the partition methods available in the video encoding device 100 is also determined for each picture. Thus, it is also determined that only a specific partition method is used for each picture. If the video encoding device 100 uses only one partition method, information related to a partition method that can be used for a coding unit is not separately determined.

[0035] For coding units of a certain size, they are also divided according to a certain division method, for example, if the coding unit size is 256x265, the coding unit is also set to be divided only into four square coding units with half the width and height.

[0036] When the partition information of the coding unit indicates that the coding unit is partitioned, partition form information indicating a partitioning method of the coding unit is also generated. If there is only one partitioning method that can be used in the picture to which the coding unit belongs, the partition form information is not generated. If the partitioning method is adaptively determined based on the coding information around the coding unit, the partition form information is not generated.

[0037] As described above, the video data of the current picture is divided into maximum coding units according to the maximum size of the coding unit. The maximum coding unit may include coding units hierarchically divided from the maximum coding unit. The type and position of the lower coding units may be determined according to the division type of the higher coding unit. A minimum size of the coding unit that limits the division of the coding units may also be preset.

[0038] The encoding unit 110 compares the coding efficiency when the coding unit is hierarchically divided with the coding efficiency when the coding unit is not divided. Then, the encoding unit 110 determines whether to divide the coding unit based on the comparison result. If it is determined that dividing the coding unit is more efficient, the encoding unit 110 divides the coding unit hierarchically. If it is determined that it is more efficient not to divide the coding unit based on the comparison result, the encoding unit 110 does not divide the coding unit. Whether to divide the coding unit is determined independently of whether other adjacent coding units are divided.

[0039] The final divided coding unit is predicted by intra prediction or inter prediction. The intra prediction is a method of predicting samples of the prediction unit using reference samples around the prediction unit. The inter prediction is a method of predicting samples of the prediction unit by obtaining reference samples from a reference picture referred to by the current picture.

[0040] For intra prediction, the encoding unit 110 may apply a plurality of intra prediction methods to a prediction unit and select the most efficient intra prediction method, which may include a DC mode, a planar mode, and a directional mode such as a vertical mode and a horizontal mode.

[0041] The intra prediction is also performed for each prediction unit when the reconstructed samples around the coding unit are used as reference samples. However, when the reconstructed samples inside the coding unit are used as reference samples, the reconstruction of the reference samples inside the coding unit must take precedence over prediction, so the prediction order of the prediction unit belongs to the transformation order of the transformation unit. Therefore, when the reconstructed samples inside the coding unit are used as reference samples, only the intra prediction method for the transformation unit corresponding to the prediction unit is determined for the prediction unit, and the actual intra prediction is also performed for each transformation unit.

[0042] The encoding unit 110 may select the most efficient inter prediction method by determining an optimal motion vector and a reference picture. The coding unit determination unit 120 may determine a plurality of motion vector candidates from coding units spatially and temporally adjacent to the current coding unit for inter prediction, and determine the most efficient motion vector among them as a motion vector. Similarly, a plurality of reference picture candidates may be determined from coding units spatially and temporally adjacent to the current coding unit, and determine the most efficient reference picture among them. According to an embodiment, the reference picture may also be determined from a reference picture list previously determined for the current picture. According to an embodiment, for the accuracy of prediction, the most efficient motion vector among the plurality of motion vector candidates may be determined as a predicted motion vector, and the predicted motion vector may be corrected to determine a motion vector. Inter prediction may also be performed in parallel for each prediction unit within a coding unit.

[0043] The encoding unit 110 can obtain only information indicating a motion vector and a reference picture and restore a coding unit by using the skip mode. According to the skip mode, all coding information including a residual signal is omitted except for information indicating a motion vector and a reference picture. Since the residual signal is omitted, the skip mode can be used when the accuracy of prediction is very high.

[0044] The partition mode used may be restricted depending on the prediction method for the prediction unit. For example, only partition modes related to prediction units of 2Nx2N and NxN sizes are applied to intra prediction, while partition modes related to prediction units of 2Nx2N, 2NxN, Nx2N, and NxN sizes are applied to inter prediction. Also, only partition modes related to prediction units of 2Nx2N sizes are applied to the skip mode of inter prediction. In the video encoding device 100, the partition modes allowed for each prediction method are also changed depending on the encoding efficiency.

[0045] The video encoding device 100 may perform conversion based on a coding unit. The video encoding device 100 may convert residual data, which is a difference between an original value and a predicted value of a pixel included in a coding unit, through a predetermined process. For example, the video encoding device 100 may perform lossy compression on the residual data through quantization and DCT / DST transformation. Alternatively, the video encoding device 100 may perform lossless compression on the residual data without quantization.

[0046] In conclusion, the encoding unit 110 determines the most efficient prediction method for the current coding unit from among a plurality of intra prediction methods and inter prediction methods. Then, the encoding unit 110 determines the prediction method for the current coding unit according to the coding efficiency according to the prediction result. Similarly, the encoding unit 110 can determine the conversion method according to the coding efficiency according to the conversion result. The coding efficiency of the coding unit is finally determined according to the method of determining the most efficient prediction method and conversion method for the coding unit. The encoding unit 110 determines the hierarchical structure of the maximum coding unit according to the coding efficiency of the finally divided coding unit.

[0047] The encoding unit 110 can measure the coding efficiency of a coding unit and the prediction efficiency of a prediction method by using a rate-distortion optimization technique based on a Lagrangian multiplier.

[0048] The encoding unit 110 may generate partition information indicating whether the coding unit is partitioned according to the determined hierarchical structure of the maximum coding unit. Then, the encoding unit 110 may generate partition mode information for determining a prediction unit and transform unit partition information for determining a transform unit for the coding unit for which partitioning has been completed. Also, when there are two or more partition methods for the coding unit, the encoding unit 110 may generate partition mode information indicating the partition method together with the partition information. Then, the encoding unit 110 may generate information related to the prediction method and transform method used for the prediction unit and the transform unit.

[0049] The bitstream generating unit 120 can output the information generated by the encoding unit 110 in the form of a bitstream according to the hierarchical structure of the maximum coding unit.

[0050] A method for determining a coding unit, a prediction unit, and a transform unit based on a tree structure of a maximum coding unit according to an embodiment will be described in detail below with reference to FIGS.

[0051] FIG. 1B illustrates a block diagram of a video decoder 150 based on a tree-structured coding unit according to one embodiment.

[0052] The video decoding device 150 includes a receiving unit 160 and a decoding unit 170 .

[0053] Definitions of various terms such as a coding unit, a prediction unit, a transform unit, and various partition information for a decoding operation of the video decoder 150 according to an embodiment are the same as those described with reference to FIG 1 and the video encoder 100. In addition, although the purpose of the video decoder 150 is to restore video data, various encoding methods used in the video encoder 100 are also applicable to the video decoder 150.

[0054] The receiving unit 160 receives and parses a bitstream related to the coded video. The decoding unit 170 extracts information required for decoding for each maximum coding unit from the parsed bitstream and provides the extracted information to the decoding unit 170. The decoding unit 170 may extract information related to the maximum size of the coding unit of the current picture from a header, a sequence parameter set, or a picture parameter set related to the current picture.

[0055] In addition, the decoding unit 170 extracts partition information related to the coding unit according to a tree structure for each maximum coding unit from the parsed bitstream. The extracted partition information is output to the decoding unit 170. The decoding unit 170 may divide the maximum coding unit according to the extracted partition information and determine the tree structure of the maximum coding unit.

[0056] The partition information extracted by the decoding unit 170 is partition information related to a tree structure determined to generate the minimum coding error by the video encoding device 100. Therefore, the video decoding device 150 can decode the data by the coding method that generates the minimum coding error, and restore the video.

[0057] The decoding unit 170 may extract partition information related to a data unit, such as a prediction unit and a transform unit, included in a coding unit. For example, the decoding unit 170 may extract information related to a most efficient partition mode related to a prediction unit. And, the decoding unit 170 may extract transform partition information related to a most efficient tree structure in a transform unit.

[0058] In addition, the decoding unit 170 may obtain information related to the most efficient prediction method for a prediction unit divided from a coding unit, and may obtain information related to the most efficient transform method for a transform unit divided from a coding unit.

[0059] The decoding unit 170 extracts information from the bitstream using a method for constructing the bitstream in the bitstream generation unit 120 of the video encoding device 100.

[0060] The decoding unit 170 can divide the maximum coding unit into coding units having the most efficient tree structure based on the division information. The decoding unit 170 can then divide the coding unit into prediction units based on the information related to the partition mode. The decoding unit 170 can divide the coding unit into transform units based on the transform division information.

[0061] The decoding unit 170 can predict the prediction unit based on information related to the prediction method. The decoding unit 170 can inverse quantize and inverse transform residual data corresponding to the difference between the original value and the predicted value of a pixel based on the information related to the transformation method of the transformation unit. The decoding unit 170 can also restore the pixel of the coding unit based on the prediction result of the prediction unit and the transformation result of the transformation unit.

[0062] FIG. 2 illustrates a process in which a video decoder 150 divides a current coding unit to determine at least one coding unit, according to one embodiment.

[0063] According to an embodiment, the video decoding device 150 may determine the type of the coding unit using the block type information, and may determine the type of the coding unit to be divided using the partition type information. That is, depending on the block type indicated by the block type information used by the video decoding device 150, a partitioning method of the coding unit indicated by the partition type information may be determined.

[0064] According to an embodiment, the video decoding device 150 may use block shape information indicating that the current coding unit is square. For example, the video decoding device 150 may determine whether to not divide the square coding unit, to divide it vertically, to divide it horizontally, or to divide it into four coding units, etc., according to the division shape information. Referring to FIG. 2, when the block shape information of the current coding unit 200 indicates a square shape, the decoder 180 may determine whether to not divide the coding unit 210a having the same size as the current coding unit 200 according to the division shape information indicating no division, or to divide the coding unit 210b, 210c, 210d, etc. according to the division shape information indicating a predetermined division method.

[0065] Referring to FIG. 2, the video decoding device 150 may determine two coding units 210b obtained by dividing the current coding unit 200 in the vertical direction based on the division type information indicating division in the vertical direction, according to an embodiment. The video decoding device 150 may determine two coding units 210c obtained by dividing the current coding unit 200 in the horizontal direction based on the division type information indicating division in the horizontal direction. The video decoding device 150 may determine four coding units 210d obtained by dividing the current coding unit 200 in the vertical and horizontal directions based on the division type information indicating division in the vertical and horizontal directions. However, the division type in which the square coding unit may be divided is not limited to the above-mentioned type, and may include various types that can be indicated by the division type information. Predetermined division types in which the square coding unit is divided will be described in detail below through various embodiments.

[0066] FIG. 3 illustrates a process by which video decoder 150 may partition a non-square shaped coding unit to determine at least one coding unit, according to one embodiment.

[0067] According to an embodiment, the video decoding device 150 may use block shape information indicating that the current coding unit is non-square. The video decoding device 150 may determine whether to not divide the non-square current coding unit or to divide it in a predetermined manner according to the division shape information. Referring to FIG. 3, when the block shape information of the current coding unit 300 or 350 indicates a non-square shape, the video decoding device 150 may not divide the coding unit 310 or 360 having the same size as the current coding unit 300 or 350 according to the division shape information indicating no division, or may determine the divided coding units 320a, 320b, 330a, 330b, 330c, 370a, 370b, 380a, 380b, and 380c according to the division shape information indicating a predetermined division method. The predetermined division method for dividing the non-square coding unit will be described in detail below through various embodiments.

[0068] According to an embodiment, the video decoder 150 may determine a manner in which the coding unit is divided using the division type information, and in this case, the division type information may indicate the number of at least one coding unit to be generated by dividing the coding unit. Referring to FIG 3, when the division type information indicates that the current coding unit 300 or 350 is divided into two coding units, the video decoder 150 may divide the current coding unit 300 or 350 based on the division type information and determine two coding units 320a, 320b, or 370a, 370b included in the current coding unit.

[0069] According to an embodiment, when the video decoding device 150 divides the non-square shaped current coding unit 300 or 350 based on the division shape information, the video decoding device 150 may divide the current coding unit in consideration of the position of a long side of the non-square current coding unit 300 or 350. For example, the video decoding device 150 may determine a plurality of coding units by dividing the current coding unit 300 or 350 in a direction that divides the long side of the current coding unit 300 or 350 in consideration of the shape of the current coding unit 300 or 350.

[0070] According to an embodiment, when the partitioning format information indicates that the coding unit is to be partitioned into an odd number of blocks, the video decoding device 150 may determine an odd number of coding units included in the current coding unit 300 or 350. For example, when the partitioning format information indicates that the current coding unit 300 or 350 is to be partitioned into three coding units, the video decoding device 150 may partition the current coding unit 300 or 350 into three coding units 330a, 330b, 330c, 380a, 380b, and 380c. According to an embodiment, the video decoding device 150 may determine an odd number of coding units included in the current coding unit 300 or 350, and the determined coding units may not all have the same size. For example, among the determined odd number of coding units 330a, 330b, 330c, 380a, 380b, 380c, the size of a given coding unit 330b or 380b may be different from the sizes of the other coding units 330a, 330c, 380a, 380c. That is, the coding units that may be determined by dividing the current coding unit 300 or 350 may have a variety of sizes.

[0071] According to an embodiment, when the partitioning type information indicates that a coding unit is partitioned into an odd number of blocks, the video decoding device 150 may determine an odd number of coding units included in the current coding unit 300 or 350, and may further set a predetermined restriction on at least one coding unit among the odd number of coding units generated by partitioning. Referring to FIG 3, the video decoding device 150 may perform a decoding process for the coding units 330b and 380b located in the middle of the three coding units 330a, 330b, 330c, 380a, 380b, and 380c generated by partitioning the current coding unit 300 or 350 differently from the other coding units 330a, 330c, 380a, and 380c. For example, the video decoding device 150 may limit the centrally located coding units 330b and 380b from being further divided, or may limit the number of divisions to a predetermined number, unlike the other coding units 330a, 330c, 380a, and 380c.

[0072] FIG. 4 illustrates a process in which a video decoding device 150 divides a coding unit based on at least one of block format information and partition format information, according to an embodiment.

[0073] According to an embodiment, the video decoding device 150 may determine whether or not to divide the square-shaped first coding unit 400 into coding units based on at least one of the block shape information and the division shape information. According to an embodiment, if the division shape information indicates that the first coding unit 400 is to be divided in the horizontal direction, the video decoding device 150 may divide the first coding unit 400 in the horizontal direction to determine the second coding unit 410. According to an embodiment, the first coding unit, the second coding unit, and the third coding unit used are terms used to understand the division context between the coding units. For example, if the first coding unit is divided, the second coding unit may be determined, and if the second coding unit is divided, the third coding unit may be determined. In the following, the relationship between the first coding unit, the second coding unit, and the third coding unit used may also be understood to be according to the above-mentioned characteristics.

[0074] According to an embodiment, the video decoder 150 may determine whether to divide the determined second coding unit 410 into coding units based on at least one of block shape information and partition shape information. Referring to FIG 4, the video decoder 150 may divide the first coding unit 400 based on at least one of block shape information and partition shape information, and may divide the determined non-square shaped second coding unit 410 into at least one third coding unit 420a, 420b, 420c, 420d, etc., or may not divide the second coding unit 410. The video decoding device 150 may acquire at least one of block configuration information and partition configuration information, and may divide the first coding unit 400 based on at least one of the acquired block configuration information and partition configuration information, for example, into a plurality of second coding units 410 of various types, and the second coding units 410 may also be divided according to the manner in which the first coding unit 400 was divided, based on at least one of the block configuration information and partition configuration information related to the first coding unit 400. According to an embodiment, when the first coding unit 400 is divided into the second coding units 410 based on at least one of the block configuration information and partition configuration information related to the first coding unit 400, the second coding units 410 may also be divided into, for example, third coding units 420a, 420b, 420c, 420d, etc., based on at least one of the block configuration information and partition configuration information related to the second coding unit 410. That is, the coding units are also recursively divided based on at least one of the partitioning pattern information and the block pattern information associated with each coding unit. Methods that can be used for recursively dividing the coding units will be described below in various exemplary embodiments.

[0075] According to an embodiment, the video decoder 150 may determine whether to divide each of the third coding units 420a, 420b, 420c, 420d, etc. into coding units or not to divide the second coding unit 410 based on at least one of the block configuration information and the division configuration information. According to an embodiment, the video decoder 150 may divide the non-square second coding unit 410 into an odd number of third coding units 420b, 420c, 420d. The video decoder 150 may set a predetermined restriction on a certain third coding unit among the odd number of third coding units 420b, 420c, 420d. For example, the video decoder 150 may restrict the coding unit 420c located in the middle of the odd number of third coding units 420b, 420c, 420d so that it is not further divided or that it must be divided a configurable number of times. 4, the video decoding device 150 may restrict the coding unit 420c located in the middle of the odd number of third coding units 420b, 420c, and 420d included in the non-square second coding unit 410 to not be further divided, or to be divided into a predetermined division form (e.g., to be divided into only four coding units, or to be divided into a form corresponding to the division form of the second coding unit 410), or to be divided a predetermined number of times (e.g., to be divided only n times; n>0). However, the above restriction on the coding unit 420c located in the middle is merely an embodiment, and should not be construed as being limited to the above embodiment, but should be construed as including various restrictions that the coding unit 420c located in the middle may be decoded differently from the other coding units 420b and 420d.

[0076] According to an embodiment, the video decoding device 150 may obtain at least one of block configuration information and partition configuration information used to partition the current coding unit at a predetermined position within the current coding unit.

[0077] According to an embodiment, when a current coding unit is divided into a predetermined number of coding units, the video decoding device 150 may select one of the coding units. There are various methods for selecting one of the multiple coding units, and such methods will be described below through various embodiments.

[0078] According to an embodiment, the video decoder 150 may divide a current coding unit into a plurality of coding units and determine a coding unit at a predetermined position.

[0079] FIG. 5 illustrates a method by which video decoder 150 determines a coding unit at a given position among an odd number of coding units, according to one embodiment.

[0080] According to an embodiment, the video decoder 150 may use information indicating the positions of each of the odd-numbered coding units to determine a coding unit located in the middle among the odd-numbered coding units. Referring to FIG. 5, the video decoder 150 may divide the current coding unit 500 to determine odd-numbered coding units 520a, 520b, and 520c. The video decoder 150 may determine the middle coding unit 520b using information related to the positions of the odd-numbered coding units 520a, 520b, and 520c. For example, the video decoder 150 may determine the positions of the coding units 520a, 520b, and 520c based on information indicating the positions of certain samples included in the coding units 520a, 520b, and 520c to determine the middle coding unit 520b. Specifically, the video decoding device 150 can determine the positions of the coding units 520a, 520b, and 520c based on information indicating the positions of the upper left samples 530a, 530b, and 530c of the coding units 520a, 520b, and 520c, thereby determining the coding unit 520b located in the middle.

[0081] According to an embodiment, the information indicating the positions of the top left samples 530a, 530b, and 530c included in the coding units 520a, 520b, and 520c, respectively, may include information related to the positions or coordinates of the coding units 520a, 520b, and 520c in a picture. According to an embodiment, the information indicating the positions of the top left samples 530a, 530b, and 530c included in the coding units 520a, 520b, and 520c, respectively, may include information indicating the width or height of the coding units 520a, 520b, and 520c included in the current coding unit 500, and such width or height also corresponds to information indicating the difference between the coordinates of the coding units 520a, 520b, and 520c in a picture. That is, the video decoding device 150 can determine the coding unit 520b located in the middle by directly using information related to the positions or coordinates of the coding units 520a, 520b, and 520c within the picture, or by using information related to the width or height of the coding unit corresponding to the difference between the coordinates.

[0082] According to an embodiment, information indicating a position of the top left sample 530a of the top coding unit 520a may indicate (xa, ya) coordinates, information indicating a position of the top left sample 530b of the middle coding unit 520b may indicate (xb, yb) coordinates, and information indicating a position of the top left sample 530c of the bottom coding unit 520c may indicate (xc, yc) coordinates. The video decoding device 150 may determine the middle coding unit 520b using the coordinates of the top left samples 530a, 530b, and 530c included in the coding units 520a, 520b, and 520c, respectively. For example, when the coordinates of the top left samples 530a, 530b, and 530c are sorted in ascending or descending order, the coding unit 520b including the coordinates (xb, yb) of the middle sample 530b may be determined as the middle coding unit among the coding units 520a, 520b, and 520c determined by dividing the current coding unit 500. However, the coordinates indicating the positions of the top left samples 530a, 530b, and 530c may indicate coordinates indicating absolute positions within a picture, and further, the (dxb, dyb) coordinates indicating the relative position of the top left sample 530b of the middle coding unit 520b and the (dxc, dyc) coordinates indicating the relative position of the top left sample 530c of the bottom coding unit 520c may be used. In addition, the method of determining a coding unit at a predetermined position by using the coordinates of a sample as information indicating the position of the sample included in the coding unit should not be interpreted as being limited to the above-mentioned method, but should be interpreted as various arithmetic methods that can use the coordinates of the sample.

[0083] According to an embodiment, the video decoder 150 may divide the current coding unit 500 into a plurality of coding units 520a, 520b, and 520c, and may select a coding unit from among the coding units 520a, 520b, and 520c according to a predetermined criterion. For example, the video decoder 150 may select a coding unit 520b having a different size from among the coding units 520a, 520b, and 520c.

[0084] According to an embodiment, the video decoder 150 may determine the width or height of each of the coding units 520a, 520b, and 520c using the (xa, ya) coordinates indicating the position of the top left sample 530a of the top coding unit 520a, the (xb, yb) coordinates indicating the position of the top left sample 530b of the middle coding unit 520b, and the (xc, yc) coordinates indicating the position of the top left sample 530c of the bottom coding unit 520c. The video decoder 150 may determine the size of each of the coding units 520a, 520b, and 520c using the (xa, ya), (xb, yb), and (xc, yc) coordinates indicating the positions of the coding units 520a, 520b, and 520c.

[0085] According to an embodiment, the video decoder 150 may determine the width of the top coding unit 520a as xb-xa and the height as yb-ya. According to an embodiment, the video decoder 150 may determine the width of the middle coding unit 520b as xc-xb and the height as yc-yb. According to an embodiment, the video decoder 150 may determine the width or height of the bottom coding unit using the width or height of the current coding unit and the widths and heights of the top coding unit 520a and the middle coding unit 520b. The video decoder 150 may determine a coding unit having a different size from other coding units based on the determined widths and heights of the coding units 520a, 520b, and 520c. Referring to FIG. 5, the video decoder 150 may determine the middle coding unit 520b having a different size from the top coding unit 520a and the bottom coding unit 520c as a coding unit at a predetermined position. However, the above-mentioned process in which the video decoding device 150 determines coding units having different coding units and different sizes is merely one embodiment of determining a coding unit at a predetermined position using the size of the coding unit determined based on sample coordinates, and various other processes of comparing the size of the coding unit determined based on predetermined sample coordinates and determining a coding unit at a predetermined position may also be used.

[0086] However, the position of the sample considered to determine the position of the coding unit is not to be interpreted as being limited to the upper left end as mentioned above, but rather, it is also to be interpreted that information related to the position of any sample included in the coding unit may be used.

[0087] According to an embodiment, the video decoding device 150 may select a coding unit at a predetermined position from among an odd number of coding units determined by dividing the current coding unit, taking into consideration the shape of the current coding unit. For example, if the current coding unit is a non-square shape in which the width is greater than the height, the video decoding device 150 may determine a coding unit at a predetermined position along the horizontal direction. That is, the video decoding device 150 may determine one of the coding units at different positions in the horizontal direction and set a restriction on the coding unit. If the current coding unit is a non-square shape in which the height is greater than the width, the video decoding device 150 may determine a coding unit at a predetermined position along the vertical direction. That is, the video decoding device 150 may determine one of the coding units at different positions in the vertical direction and set a restriction on the coding unit.

[0088] According to an embodiment, the video decoding device 150 may use information indicating the positions of each of the even number of coding units to determine a coding unit at a predetermined position among the even number of coding units. The video decoding device 150 may divide the current coding unit to determine the even number of coding units, and may determine a coding unit at a predetermined position using information regarding the positions of the even number of coding units. A detailed process related thereto corresponds to the process of determining a coding unit at a predetermined position (e.g., a center position) among the odd number of coding units described in FIG. 5, and therefore will not be described in detail.

[0089] According to an embodiment, when a non-square current coding unit is divided into a plurality of coding units, in order to determine a coding unit at a predetermined position among the plurality of coding units, the video decoding device 150 may use predetermined information related to the coding unit at a predetermined position during the division process. For example, in order to determine a coding unit at a center among the plurality of coding units into which the current coding unit is divided, the video decoding device 150 may use at least one of block shape information and partition shape information stored in a sample included in the center coding unit during the division process.

[0090] 5, the video decoding device 150 may divide the current coding unit 500 into a plurality of coding units 520a, 520b, and 520c based on at least one of the block configuration information and the partition configuration information, and may determine the coding unit 520b located in the middle of the plurality of coding units 520a, 520b, and 520c. Furthermore, the video decoding device 150 may determine the coding unit 520b located in the middle by considering a position where at least one of the block configuration information and the partition configuration information is obtained. That is, when at least one of the block configuration information and the partition configuration information of the current coding unit 500 is also obtained in a sample 540 located in the middle of the current coding unit 500, and the current coding unit 500 is divided into a plurality of coding units 520a, 520b, and 520c based on at least one of the block configuration information and the partition configuration information, the coding unit 520b including the sample 540 may be determined as the coding unit located in the middle. However, the information used to determine the coding unit located in the middle is not interpreted as being limited to at least one of block shape information and partition shape information, and various types of information are also used in the process of determining the coding unit located in the middle.

[0091] According to an embodiment, the predetermined information for identifying a coding unit at a predetermined position is also obtained from a predetermined sample included in the coding unit to be determined. Referring to FIG. 5, the video decoding device 150 may use at least one of block shape information and partition shape information obtained from a sample at a predetermined position in the current coding unit 500 (e.g., a sample at a middle position in the current coding unit 500) in order to determine a coding unit at a predetermined position (e.g., a coding unit at a middle position among a plurality of coding units 520a, 520b, and 520c determined by dividing the current coding unit 500). That is, the video decoding device 150 may determine the sample at the predetermined position by considering the block shape of the current coding unit 500, and may determine a coding unit 520b including a sample from which the predetermined information (e.g., at least one of block shape information and partition shape information) can be obtained among a plurality of coding units 520a, 520b, and 520c determined by dividing the current coding unit 500, and may set a predetermined restriction. 5, according to an embodiment, the video decoder 150 may determine a sample 540 located in the middle of the current coding unit 500 as a sample from which certain information may be acquired, and may set a certain restriction in the decoding process of the coding unit 520b including such sample 540. However, the position of the sample from which certain information may be acquired is not limited to the above position, but may be a sample at any position included in the coding unit 520b determined to set the restriction.

[0092] According to an embodiment, the position of the sample from which the predetermined information can be obtained is also determined according to the shape of the current coding unit 500. According to an embodiment, the block shape information may determine whether the shape of the current coding unit is square or non-square, and may determine the position of the sample from which the predetermined information can be obtained according to the shape. For example, the video decoding apparatus 150 may determine, using at least one of information related to the width and information related to the height of the current coding unit, a sample located on a boundary that divides at least one of the width and height of the current coding unit in half as a sample from which the predetermined information can be obtained. For another example, when the block shape information related to the current coding unit indicates that the current coding unit is non-square, the video decoding apparatus 150 may determine one of the samples adjacent to a boundary that divides the long side of the current coding unit in half as a sample from which the predetermined information can be obtained.

[0093] According to an embodiment, when the current coding unit is divided into a plurality of coding units, the video decoding device 150 may use at least one of block shape information and partition shape information to determine a coding unit at a predetermined position among the plurality of coding units. According to an embodiment, the video decoding device 150 may acquire at least one of block shape information and partition shape information at a sample at a predetermined position included in the coding unit, and may divide a plurality of coding units generated by dividing the current coding unit by using at least one of partition shape information and block shape information acquired from a sample at a predetermined position included in each of the plurality of coding units. That is, the coding units are also divided recursively using at least one of block shape information and partition shape information acquired at a sample at a predetermined position included in each of the coding units. The recursive division process of the coding unit has been described with reference to FIG. 4, and a detailed description thereof will be omitted.

[0094] In one embodiment, the video decoding device 150 can divide the current coding unit and determine at least one coding unit, and can determine the order in which such at least one coding unit is decoded by a predetermined block (e.g., the current coding unit).

[0095] FIG. 6 illustrates an order in which multiple coding units are processed when the video decoder 150 divides a current coding unit to determine multiple coding units, according to one embodiment.

[0096] According to one embodiment, the video decoding device 150 may, based on the block format information and the partition format information, vertically divide the first coding unit 600 to determine the second coding units 610a and 610b, or horizontally divide the first coding unit 600 to determine the second coding units 630a and 630b, or vertically and horizontally divide the first coding unit 600 to determine the second coding units 650a, 650b, 650c, and 650d.

[0097] 6, the video decoding device 150 may determine an order in which the second coding units 610a and 610b determined by dividing the first coding unit 600 in the vertical direction are processed in the horizontal direction 610c. The video decoding device 150 may determine a processing order of the second coding units 630a and 630b determined by dividing the first coding unit 600 in the horizontal direction as the vertical direction 630c. The video decoding device 150 may determine the second coding units 650a, 650b, 650c, and 650d determined by dividing the first coding unit 600 in the vertical and horizontal directions according to a predetermined order (650e) (e.g., raster scan order or z scan order) in which a coding unit located in one row is processed and then a coding unit located in the next row is processed.

[0098] According to an embodiment, the video decoder 150 may recursively divide the coding units. Referring to FIG. 6, the video decoder 150 may divide the first coding unit 600 to determine a plurality of coding units 610a, 610b, 630a, 630b, 650a, 650b, 650c, and 650d, and may recursively divide each of the determined coding units 610a, 610b, 630a, 630b, 650a, 650b, 650c, and 650d. A method of dividing the coding units 610a, 610b, 630a, 630b, 650a, 650b, 650c, and 650d corresponds to a method of dividing the first coding unit 600. As a result, the multiple coding units 610a, 610b, 630a, 630b, 650a, 650b, 650c, and 650d are also divided into multiple coding units independently. Referring to FIG 6, the video decoder 150 may divide the first coding unit 600 vertically to determine the second coding units 610a and 610b, and may further determine whether or not to divide each of the second coding units 610a and 610b independently.

[0099] According to one embodiment, the video decoder 150 may horizontally split the second coding unit 610a on the left side into third coding units 620a and 620b, while not splitting the second coding unit 610b on the right side.

[0100] According to an embodiment, the processing order of the coding units is also determined based on the division process of the coding units. In other words, the processing order of the divided coding units is also determined based on the processing order of the coding units immediately before the division. The video decoding device 150 may determine the processing order of the third coding units 620a and 620b determined by dividing the second coding unit 610a on the left side, independently of the second coding unit 610b on the right side. Since the second coding unit 610a on the left side is divided in the horizontal direction and the third coding units 620a and 620b are determined, the third coding units 620a and 620b are also processed in the vertical direction 620c. In addition, since the processing order of the second coding unit 610a on the left side and the second coding unit 610b on the right side corresponds to the horizontal direction 610c, the third coding units 620a and 620b included in the second coding unit 610a on the left side may be processed in the vertical direction 620c before the right coding unit 610b is processed. The above content is intended to explain the process in which the processing order of coding units is determined based on the coding units before they are divided, and therefore should not be interpreted as being limited to the above embodiment, but should be interpreted as meaning that coding units determined by being divided into various forms are used in various methods in which they can be processed independently in a predetermined order.

[0101] FIG. 7 illustrates a process in which a video decoder 150 determines that a current coding unit is to be divided into an odd number of coding units when a predetermined-order coding unit cannot be processed, according to an embodiment.

[0102] According to an embodiment, the video decoder 150 may determine that the current coding unit is divided into an odd number of coding units based on the acquired block configuration information and division configuration information. Referring to FIG. 7, a square-shaped first coding unit 700 is divided into non-square-shaped second coding units 710a and 710b, and the second coding units 710a and 710b are also independently divided into third coding units 720a, 720b, 720c, 720d, and 720e. According to an embodiment, the video decoder 150 may divide the left coding unit 710a of the second coding unit in the horizontal direction to determine a plurality of third coding units 720a and 720b, and may divide the right coding unit 710b into an odd number of third coding units 720c, 720d, and 720e.

[0103] According to an embodiment, the video decoder 150 may determine whether the third coding units 720a, 720b, 720c, 720d, and 720e are processed in a predetermined order, and may determine whether there is an odd number of coding units. Referring to FIG. 7, the video decoder 150 may recursively divide the first coding unit 700 to determine the third coding units 720a, 720b, 720c, 720d, and 720e. The video decoder 150 may determine whether the first coding unit 700, the second coding unit 710a, and 710b, or the third coding units 720a, 720b, 720c, 720d, and 720e are divided into an odd number of coding units according to at least one of block configuration information and partition configuration information. For example, in the second coding units 710a and 710b, the coding units located on the right side are also divided into an odd number of third coding units 720c, 720d, and 720e. The order in which the coding units included in the first coding unit 700 are processed may be a predetermined order (730) (e.g., z scan order), and the video decoding device 150 may determine whether the third coding units 720c, 720d, and 720e determined by dividing the right second coding unit 710b into an odd number of coding units satisfy a condition that they can be processed in the predetermined order.

[0104] According to one embodiment, the video decoding device 150 may determine whether the third coding units 720a, 720b, 720c, 720d, and 720e included in the first coding unit 700 satisfy a condition that they may be processed in a predetermined order, the condition being related to whether at least one of the width and height of the second coding units 710a, 710b is divided in half by the boundary of the third coding units 720a, 720b, 720c, 720d, and 720e. For example, the third coding units 720a and 720b determined by dividing the height of the non-square left second coding unit 710a in half satisfy the condition, but the boundaries of the third coding units 720c, 720d, and 720e determined by dividing the right second coding unit 710b into three coding units do not divide the width or height of the right second coding unit 710b in half, so it is determined that the third coding units 720c, 720d, and 720e do not satisfy the condition. If such a condition is not satisfied, the video decoding device 150 may determine that there is a disconnection in the scan order, and may determine that the right second coding unit 710b is divided into an odd number of coding units based on the determination result. According to an embodiment, the video decoding device 150 may set a predetermined restriction on a coding unit at a predetermined position among the divided coding units when the coding units are divided into an odd number of coding units, and detailed description of such a restriction or a predetermined position has been described through various embodiments, and will not be repeated.

[0105] FIG. 8 illustrates a process in which the video decoding device 150 divides the first coding unit 800 and determines at least one coding unit, according to an embodiment. According to an embodiment, the video decoding device 150 may divide the first coding unit 800 based on at least one of block shape information and division shape information acquired via the receiving unit 160. The square-shaped first coding unit 800 may be divided into four square-shaped coding units or into a plurality of non-square-shaped coding units. For example, referring to FIG. 8, when the block shape information indicates that the first coding unit 800 is a square and the division shape information indicates that the first coding unit 800 is to be divided into non-square coding units, the video decoding device 150 may divide the first coding unit 800 into a plurality of non-square coding units. Specifically, when the division form information indicates that the first coding unit 800 is to be divided horizontally or vertically to determine an odd number of coding units, the video decoding device 150 can divide the square-shaped first coding unit 800 into second coding units 810a, 810b, and 810c determined by dividing the first coding unit 800 vertically as an odd number of coding units, or into second coding units 820a, 820b, and 820c determined by dividing the first coding unit 800 horizontally.

[0106] According to an embodiment, the video decoding device 150 may determine whether the second coding units 810a, 810b, 810c, 820a, 820b, and 820c included in the first coding unit 800 satisfy a condition that they can be processed in a predetermined order, and the condition is related to whether at least one of the width and height of the first coding unit 800 is divided in half by the boundaries of the second coding units 810a, 810b, 810c, 820a, 820b, and 820c. Referring to FIG 8, the boundaries of the second coding units 810a, 810b, and 810c determined by dividing the square-shaped first coding unit 800 in the vertical direction do not divide the width of the first coding unit 800 in half, so it is also determined that the first coding unit 800 does not satisfy the condition that it can be processed in a predetermined order. In addition, since the boundaries of the second coding units 820a, 820b, and 820c determined by dividing the square first coding unit 800 in the horizontal direction do not divide the width of the first coding unit 800 in half, it is determined that the first coding unit 800 does not satisfy the condition that it can be processed according to a predetermined order. If such a condition is not satisfied, the video decoding device 150 may determine that the scanning order is broken, and may determine that the first coding unit 800 is divided into an odd number of coding units based on the determination result. According to an embodiment, when the coding unit is divided into an odd number of coding units, the video decoding device 150 may set a predetermined restriction on a coding unit at a predetermined position among the divided coding units, and the content of such a restriction or the predetermined position has been described through various embodiments, and detailed description thereof will be omitted.

[0107] According to an embodiment, the video decoding device 150 may divide the first coding unit and determine various types of coding units.

[0108] 8, the video decoding apparatus 150 may divide a square-shaped first coding unit 800, a non-square-shaped first coding unit 830, or a non-square-shaped first coding unit 850 into various types of coding units.

[0109] FIG. 9 illustrates that, according to one embodiment, the video decoding device 150 restricts the manner in which the second coding unit may be divided when the non-square second coding unit determined by dividing the first coding unit (900) satisfies certain conditions.

[0110] According to an embodiment, the video decoding device 150 may determine to divide the square-shaped first coding unit 900 into non-square-shaped second coding units 910a, 910b, 920a, and 920b based on at least one of block shape information and partition shape information acquired via the receiving unit 160. The second coding units 910a, 910b, 920a, and 920b may be divided independently. Thus, the video decoding device 150 may determine whether to divide the second coding units 910a, 910b, 920a, and 920b into a plurality of coding units or not to divide the coding units based on at least one of block shape information and partition shape information related to each of the second coding units 910a, 910b, 920a, and 920b. According to an embodiment, the video decoding device 150 may determine the third coding units 912a and 912b by dividing the left non-square-shaped second coding unit 910a, which is determined by dividing the first coding unit 900 vertically, in the horizontal direction. However, when the video decoding device 150 divides the left second coding unit 910a in the horizontal direction, the right second coding unit 910b may be restricted so that it is not divided horizontally in the same direction as the division direction of the left second coding unit 910a. If the right second coding unit 910b is divided in the same direction and the third coding units 914a and 914b are determined, the left second coding unit 910a and the right second coding unit 910b may be divided independently in the horizontal direction to determine the third coding units 912a, 912b, 914a, and 914b. However, this is the same result as when the video decoding device 150 divides the first coding unit 900 into four square second coding units 930a, 930b, 930c, and 930d based on at least one of the block shape information and the division shape information, which is inefficient in terms of video decoding.

[0111] According to an embodiment, the video decoding device 150 may vertically divide the non-square second coding unit 920a or 920b, which is determined by dividing the first coding unit 330 in the horizontal direction, to determine the third coding units 922a, 922b, 924a, and 924b. However, when the video decoding device 150 divides one of the second coding units (e.g., the top second coding unit 920a) in the vertical direction, the video decoding device 150 may restrict the other second coding units (e.g., the bottom coding unit 920b) from being divided vertically in the same direction as the top second coding unit 920a, for the above-mentioned reasons.

[0112] FIG. 10 illustrates a process in which the video decoder 150 divides a square coding unit when the division form information does not indicate division into four square coding units, according to an embodiment.

[0113] According to an embodiment, the video decoding device 150 may divide the first coding unit 1000 based on at least one of the block shape information and the division shape information, and determine the second coding units 1010a, 1010b, 1020a, 1020b, etc. The division shape information may include information related to various shapes into which the coding unit may be divided, but the information related to the various shapes may not include information for dividing into four square-shaped coding units. According to such division shape information, the video decoding device 150 may not divide the square-shaped first coding unit 1000 into four square-shaped second coding units 1030a, 1030b, 1030c, 1030d. Based on the division shape information, the video decoding device 150 may determine the non-square-shaped second coding units 1010a, 1010b, 1020a, 1020b, etc.

[0114] According to an embodiment, the video decoding device 150 may independently divide the non-square second coding units 1010a, 1010b, 1020a, 1020b, etc., into each of the second coding units 1010a, 1010b, 1020a, 1020b, etc. in a predetermined order through a recursive method, which is also a division method corresponding to a method of dividing the first coding unit 1000 based on at least one of the block shape information and the division shape information.

[0115] For example, the video decoder 150 may horizontally divide the left-side second coding unit 1010a to determine square-shaped third coding units 1012a and 1012b, and may horizontally divide the right-side second coding unit 1010b to determine square-shaped third coding units 1014a and 1014b. Furthermore, the video decoder 150 may horizontally divide both the left-side second coding unit 1010a and the right-side second coding unit 1010b to determine square-shaped third coding units 1016a, 1016b, 1016c, and 1016d. In this case, the coding units may be determined in the same form as when the first coding unit 1000 is divided into four square-shaped second coding units 1030a, 1030b, 1030c, and 1030d.

[0116] As another example, the video decoding device 150 may vertically divide the top second coding unit 1020a to determine square third coding units 1022a and 1022b, and may vertically divide the bottom second coding unit 1020b to determine square third coding units 1024a and 1024b. Furthermore, the video decoding device 150 may vertically divide both the top second coding unit 1020a and the bottom second coding unit 1020b to determine square third coding units 1022a, 1022b, 1024a, and 1024b. In this case, the coding units may be determined in the same form as when the first coding unit 1000 is divided into four square second coding units 1030a, 1030b, 1030c, and 1030d.

[0117] FIG. 11 illustrates that a processing order among multiple coding units may vary depending on a division process of the coding units, according to an embodiment.

[0118] According to an embodiment, the video decoding device 150 may divide the first coding unit 1100 based on the block shape information and the division shape information. When the block shape information indicates a square shape and the division shape information indicates that the first coding unit 1100 is divided in at least one of the horizontal direction and the vertical direction, the video decoding device 150 may divide the first coding unit 1100 to determine, for example, second coding units 1110a, 1110b, 1120a, 1120b, 1130a, 1130b, 1130c, 1130d, etc. Referring to FIG. 11, the non-square second coding units 1110a, 1110b, 1120a, 1120b, determined by dividing the first coding unit 1100 only in the horizontal direction or the vertical direction are also divided independently based on the block shape information and the division shape information associated therewith. For example, the video decoder 150 may horizontally divide the second coding units 1110a and 1110b, which are generated by vertically dividing the first coding unit 1100, into third coding units 1116a, 1116b, 1116c, and 1116d, and may horizontally divide the second coding units 1120a and 1120b, which are generated by horizontally dividing the first coding unit 1100, into third coding units 1126a, 1126b, 1126c, and 1126d. The process of dividing the second coding units 1110a, 1110b, 1120a, and 1120b has been described with reference to FIG. 9, and a detailed description thereof will be omitted.

[0119] According to an embodiment, the video decoder 150 may process the coding units in a predetermined order. The characteristics related to the processing of the coding units in a predetermined order have been described with reference to FIG. 6, and therefore detailed description thereof will be omitted. Referring to FIG. 11, the video decoder 150 may divide the square-shaped first coding unit 1100 to determine four square-shaped third coding units 1116a, 1116b, 1116c, 1116d, 1126a, 1126b, 1126c, and 1126d. According to an embodiment, the video decoder 150 may determine a processing order of the third coding units 1116a, 1116b, 1116c, 1116d, 1126a, 1126b, 1126c, and 1126d according to the manner in which the first coding unit 1100 is divided.

[0120] According to one embodiment, the video decoder 150 can horizontally divide the second coding units 1110a, 1110b generated by vertical division to determine the third coding units 1116a, 1116b, 1116c, and 1116d, and the video decoder 150 can process the third coding units 1116a, 1116b, 1116c, and 1116d in an order (1117) of first vertically processing the third coding units 1116a, 1116b included in the left-side second coding unit 1110a, and then vertically processing the third coding units 1116c, 1116d included in the right-side second coding unit 1110b.

[0121] According to one embodiment, the video decoding device 150 can vertically divide the second coding units 1120a, 1120b generated by dividing them horizontally to determine the third coding units 1126a, 1126b, 1126c, and 1126d, and the video decoding device 150 can process the third coding units 1126a, 1126b, 1126c, and 1126d in an order (1127) of first horizontally processing the third coding units 1126a, 1126b included in the upper second coding unit 1120a, and then horizontally processing the third coding units 1126c, 1126d included in the lower second coding unit 1120b.

[0122] 11, the second coding units 1110a, 1110b, 1120a, and 1120b are each divided to determine square-shaped third coding units 1116a, 1116b, 1116c, 1116d, 1126a, 1126b, 1126c, and 1126d. The second coding units 1110a and 1110b determined by dividing in the vertical direction and the second coding units 1120a and 1120b determined by dividing in the horizontal direction are divided in different shapes, but the third coding units 1116a, 1116b, 1116c, 1116d, 1126a, 1126b, 1126c, and 1126d determined thereafter ultimately result in the first coding unit 1100 being divided into coding units of the same shape. As a result, the video decoding device 150 can recursively divide coding units through different processes based on at least one of block type information and partition type information, determine coding units of the same type as a result, and process multiple coding units of the same type in different orders.

[0123] FIG. 12 illustrates a process of determining a coding unit depth according to changes in the shape and size of a coding unit when a coding unit is recursively divided to determine a plurality of coding units, according to one embodiment.

[0124] According to an embodiment, the video decoder 150 may determine the depth of the coding unit according to a predetermined criterion. For example, the predetermined criterion may be the long side length of the coding unit. The video decoder 150 may determine the depth of the coding unit according to a predetermined criterion. For example, the long side length of the current coding unit may be 2 times the long side length of the coding unit before the division. n When the coding unit is divided by (n>0), it may be determined that the depth of the current coding unit is increased by n from the depth of the coding unit before the division. Hereinafter, the coding unit with the increased depth will be referred to as a coding unit of a lower depth.

[0125] 12, according to an embodiment, the video decoding device 150 may divide the square-shaped first coding unit 1200 based on block shape information indicating a square shape (for example, the block shape information may indicate "0:SQUARE"), and determine a second coding unit 1202, a third coding unit 1204, etc., of a lower depth. If the size of the square-shaped first coding unit 1200 is 2Nx2N, the width and height of the first coding unit 1200 may be divided into 1 / 2. 1 The second coding unit 1202 determined by dividing the width and height of the second coding unit 1202 by 1 / 2 may have a size of NxN. Furthermore, the third coding unit 1204 determined by dividing the width and height of the second coding unit 1202 by 1 / 2 may have a size of N / 2xN / 2. In this case, the width and height of the third coding unit 1204 may be 1 / 2 that of the first coding unit 1200. 2 When the depth of the first coding unit 1200 is D, the width and height of the first coding unit 1200 are halved. 1 The depth of the second coding unit 1202 is also D+1, and is 1 / 2 the width and height of the first coding unit 1200. 2 The depth of the third coding unit 1204, which is double the depth, is also D+2.

[0126] According to one embodiment, based on block shape information indicating a non-square shape (e.g., the block shape information may indicate "1:NS_VER" indicating a non-square shape in which the height is greater than the width, or "2:NS_HOR" indicating a non-square shape in which the width is greater than the height), the video decoding device 150 may divide the first coding unit 1210 or 1220, which is non-square, and determine a second coding unit 1212 or 1222, a third coding unit 1214 or 1224, etc., of a lower depth.

[0127] The video decoder 150 may divide at least one of the width and height of the first coding unit 1210 having a size of Nx2N to determine, for example, the second coding units 1202, 1212, 1222, etc. That is, the video decoder 150 may divide the first coding unit 1210 in the horizontal direction to determine the second coding unit 1202 having a size of NxN or the second coding unit 1222 having a size of NxN / 2, or may divide the first coding unit 1210 in the horizontal and vertical directions to determine the second coding unit 1212 having a size of N / 2xN.

[0128] According to an embodiment, the video decoder 150 may divide at least one of the width and height of the first coding unit 1220 having a size of 2NxN to determine, for example, the second coding units 1202, 1212, 1222, etc. That is, the video decoder 150 may divide the first coding unit 1220 vertically to determine the second coding unit 1202 having a size of NxN or the second coding unit 1212 having a size of N / 2xN, and may divide the first coding unit 1220 horizontally and vertically to determine the second coding unit 1222 having a size of NxN / 2.

[0129] According to an embodiment, the video decoder 150 may divide at least one of the width and height of the second coding unit 1202 having a size of NxN to determine, for example, the third coding units 1204, 1214, 1224, etc. That is, the video decoder 150 may divide the second coding unit 1202 in the vertical and horizontal directions to determine the third coding unit 1204 having a size of N / 2xN / 2, the third coding unit 1214 having a size of N / 2xN / 2, or the third coding unit 1224 having a size of N / 2xN / 2.

[0130] According to an embodiment, the video decoder 150 may divide at least one of the width and height of the second coding unit 1212 having a size of N / 2xN to determine, for example, the third coding units 1204, 1214, 1224, etc. That is, the video decoder 150 may divide the second coding unit 1212 horizontally to determine the third coding unit 1204 having a size of N / 2xN / 2 or the third coding unit 1224 having a size of N / 2xN / 2, or may divide the second coding unit 1212 vertically and horizontally to determine the third coding unit 1214 having a size of N / 2xN / 2.

[0131] According to an embodiment, the video decoder 150 may divide at least one of the width and height of the second coding unit 1214 having a size of NxN / 2 to determine, for example, the third coding units 1204, 1214, 1224, etc. That is, the video decoder 150 may divide the second coding unit 1212 vertically to determine the third coding unit 1204 having a size of N / 2xN / 2 or the third coding unit 1214 having a size of N / 2xN / 2, or may divide the second coding unit 1212 vertically and horizontally to determine the third coding unit 1224 having a size of N / 2xN / 2.

[0132] According to an embodiment, the video decoding device 150 may divide, for example, the square-shaped coding units 1200, 1202, and 1204 in the horizontal or vertical direction. For example, the first coding unit 1200 having a size of 2Nx2N may be divided vertically to determine the first coding unit 1210 having a size of Nx2N, or may be divided horizontally to determine the first coding unit 1220 having a size of 2NxN. According to an embodiment, when the depth is determined based on the longest side length of the coding unit, the depth of the coding unit determined by dividing the first coding unit 1200, 1202, or 1204 having a size of 2Nx2N in the horizontal or vertical direction may be the same as the depth of the first coding unit 1200, 1202, or 1204.

[0133] According to an embodiment, the width and height of the third coding unit 1214 or 1224 also correspond to 1 / 2 times that of the first coding unit 1210 or 1220. When the depth of the first coding unit 1210 or 1220 is D, the depth of the second coding unit 1212 or 1214, which is 1 / 2 times the width and height of the first coding unit 1210 or 1220, is also D+1, and the depth of the third coding unit 1214 or 1224, which is 1 / 2 times the width and height of the first coding unit 1210 or 1220, is also D+2.

[0134] FIG. 13 illustrates a depth determined according to the type and size of a coding unit, and an index (PID: part index) for coding unit partitioning, according to one embodiment.

[0135] According to an embodiment, the video decoder 150 may divide the square-shaped first coding unit 1300 to determine various types of second coding units. Referring to FIG. 13, the video decoder 150 may divide the first coding unit 1300 in at least one of the vertical and horizontal directions according to the division type information to determine the second coding units 1302a, 1302b, 1304a, 1304b, 1306a, 1306b, 1306c, and 1306d. That is, the video decoder 150 may determine the second coding units 1302a, 1302b, 1304a, 1304b, 1306a, 1306b, 1306c, and 1306d based on the division type information related to the first coding unit 1300.

[0136] According to an embodiment, the depths of the second coding units 1302a, 1302b, 1304a, 1304b, 1306a, 1306b, 1306c, and 1306d determined by the division form information related to the square-shaped first coding unit 1300 are also determined based on the long side length. For example, since the length of one side of the square-shaped first coding unit 1300 is the same as the long side length of the non-square-shaped second coding units 1302a, 1302b, 1304a, and 1304b, the depths of the first coding unit 1300 and the non-square-shaped second coding units 1302a, 1302b, 1304a, and 1304b can be regarded as being the same as D. In contrast, when the video decoding device 150 divides the first coding unit 1300 into four square-shaped second coding units 1306a, 1306b, 1306c, and 1306d based on the division form information, the length of one side of the square-shaped second coding units 1306a, 1306b, 1306c, and 1306d is half the length of one side of the first coding unit 1300, so the depth of the second coding units 1306a, 1306b, 1306c, and 1306d is also a depth of D+1, which is one depth lower than the depth D of the first coding unit 1300.

[0137] According to an embodiment, the video decoder 150 may divide the first coding unit 1310, whose height is greater than its width, in the horizontal direction according to the division format information, and divide the first coding unit 1310 into a plurality of second coding units 1312a, 1312b, 1314a, 1314b, and 1314c. According to an embodiment, the video decoder 150 may divide the first coding unit 1320, whose width is greater than its height, in the vertical direction according to the division format information, and divide the first coding unit 1320 into a plurality of second coding units 1322a, 1322b, 1324a, 1324b, and 1324c.

[0138] According to an embodiment, the depths of the second coding units 1312a, 1312b, 1314a, 1314b, 1316a, 1316b, 1316c, and 1316d determined by the division form information related to the non-square first coding unit 1310 or 1320 are also determined based on the long side length. For example, the length of one side of the square second coding units 1312a and 1312b is ½ times the length of one side of the non-square first coding unit 1310 whose height is greater than its width, so the depths of the square second coding units 1302a, 1302b, 1304a, and 1304b are D+1, which is one depth lower than the depth D of the non-square first coding unit 1310.

[0139] Furthermore, the video decoding device 150 can divide the non-square first coding unit 1310 into an odd number of second coding units 1314a, 1314b, and 1314c based on the division form information. The odd number of second coding units 1314a, 1314b, and 1314c may include the non-square second coding units 1314a and 1314c and the square second coding unit 1314b. In this case, the long side length of the non-square second coding units 1314a and 1314c and the side length of the square second coding unit 1314b are 1 / 2 times the side length of the first coding unit 1310, so the depths of the second coding units 1314a, 1314b, and 1314c are also a depth of D+1, which is one depth lower than D, which is the depth of the first coding unit 1310. The video decoding device 150 may determine the depth of a coding unit associated with a non-square first coding unit 1320, whose width is greater than its height, in a manner corresponding to the above-mentioned manner for determining the depth of a coding unit associated with the first coding unit 1310.

[0140] According to an embodiment, the video decoder 150 may determine an index (PID) for a divided coding unit partition based on a size ratio between the coding units when the coding units divided into an odd number of coding units are not the same size. Referring to FIG. 13, the coding unit 1314b located in the middle of the coding units 1314a, 1314b, and 1314c divided into an odd number of coding units has the same width as the other coding units 1314a and 1314c, but is twice as tall as the other coding units 1314a and 1314c. That is, in this case, the coding unit 1314b located in the middle may include the other coding units 1314a and 1314c. Therefore, if the index (PID) of the coding unit 1314b located in the middle according to the scanning order is 1, the coding unit 1314c located next in the order is 3, which is an index increased by 2. That is, discontinuity of index values ​​occurs. According to one embodiment, the video decoder 150 can determine whether coding units divided into an odd number of parts are not the same size as each other based on whether there is a discontinuity in the indexes for the partitions between the coding units divided in this way.

[0141] According to an embodiment, the video decoder 150 may determine whether the current coding unit is divided into a specific division type based on an index value for distinguishing the plurality of coding units determined by dividing the current coding unit. Referring to FIG. 13, the video decoder 150 may divide a rectangular first coding unit 1310 whose height is greater than its width to determine an even number of coding units 1312a, 1312b or an odd number of coding units 1314a, 1314b, 1314c. The video decoder 150 may use an index (PID) indicating each coding unit to distinguish each of the plurality of coding units. According to an embodiment, the PID is also obtained at a sample at a predetermined position (e.g., the top left sample) of each coding unit.

[0142] According to an embodiment, the video decoder 150 may determine a coding unit at a predetermined position among the coding units determined by dividing the coding units using an index for dividing the coding units. According to an embodiment, when division type information related to a rectangular first coding unit 1310 having a height greater than a width indicates that the first coding unit 1310 is divided into three coding units, the video decoder 150 may divide the first coding unit 1310 into three coding units 1314a, 1314b, and 1314c. The video decoder 150 may assign an index related to each of the three coding units 1314a, 1314b, and 1314c. The video decoder 150 may compare the indexes related to each coding unit to determine a middle coding unit among the odd number of divided coding units. The video decoder 150 may determine the coding unit 1314b having an index corresponding to a middle value among the indexes based on the indexes of the coding units as a coding unit at a middle position among the coding units determined by dividing the first coding unit 1310. According to an embodiment, the video decoder 150 may determine the index for the divided coding unit partition based on the size ratio between the coding units when the coding units are not the same size. Referring to FIG. 13, a coding unit 1314b generated by dividing a first coding unit 1310 has the same width as other coding units 1314a and 1314c, but is twice as high as other coding units 1314a and 1314c, which have different heights. In this case, if the index (PID) of the coding unit 1314b located in the middle is 1, the next coding unit 1314c is 3, which is an index increased by 2. In such a case, when the indexes are increased uniformly and the increase widths are different, the video decoder 150 may determine that the coding unit has been divided into a plurality of coding units including a coding unit having a different size from other coding units.According to an embodiment, when the division type information indicates that the current coding unit is divided into an odd number of coding units, the video decoding device 150 may divide the current coding unit into a type in which a coding unit at a predetermined position (e.g., a middle coding unit) among the odd number of coding units has a different size from other coding units. In this case, the video decoding device 150 may determine the middle coding unit having a different size using an index (PID) related to the coding unit. However, the above index and the size or position of the coding unit at the predetermined position are specified for describing an embodiment, and should not be construed as being limited thereto, and various indexes, positions and sizes of coding units may be used.

[0143] According to one embodiment, the video decoder 150 can utilize a predetermined data unit from which the recursive division of the coding unit begins.

[0144] FIG. 14 illustrates a number of coding units being determined according to a number of predetermined data units included in a picture, according to one embodiment.

[0145] According to an embodiment, the predetermined data unit is also defined as a data unit into which the coding unit starts to be recursively divided using at least one of block configuration information and division configuration information. That is, it also corresponds to a coding unit of the highest depth used in a process of determining a plurality of coding units into which the current picture is divided. Hereinafter, for convenience of explanation, such a predetermined data unit will be referred to as a reference data unit.

[0146] According to an embodiment, the reference data unit may have a predetermined size and shape. According to an embodiment, the reference coding unit may include MxN samples, where M and N may be equal to each other or may be integers expressed as powers of 2. That is, the reference data unit may have a square or non-square shape, and then be divided into an integer number of coding units.

[0147] According to an embodiment, the video decoder 150 may divide the current picture into a plurality of reference data units. According to an embodiment, the video decoder 150 may divide the reference data units into which the current picture is divided, using division information related to each of the reference data units. The division process of the reference data units corresponds to a division process using a quad-tree structure.

[0148] According to an embodiment, the video decoding device 150 may determine in advance a minimum size that a reference data unit included in a current picture may have, and may determine reference data units of various sizes that are equal to or larger than the minimum size, and may determine at least one coding unit based on the determined reference data unit using block configuration information and partition configuration information.

[0149] 14, the video decoding apparatus 150 may use a square-shaped reference coding unit 1400 or a non-square-shaped reference coding unit 1402. According to an embodiment, the shape and size of the reference coding unit may be determined according to various data units (e.g., a sequence, a picture, a slice, a slice segment, a maximum coding unit, etc.) including at least one reference coding unit.

[0150] According to an embodiment, the receiving unit 160 of the video decoding device 150 may obtain at least one of information on the type of the base coding unit and information on the size of the base coding unit from the bitstream for each of the various data units. The process of determining at least one coding unit included in the square-shaped base coding unit 1400 has been described through the process of dividing the current coding unit 300 in Fig. 10, and the process of determining at least one coding unit included in the non-square-shaped base coding unit 1400 has been described through the process of dividing the current coding unit 1100 or 1150 in Fig. 11, so detailed description thereof will be omitted.

[0151] According to an embodiment, the video decoding apparatus 150 may use an index for identifying the size and type of the reference coding unit in order to determine the size and type of the reference coding unit according to a certain data unit that is determined in advance based on a predetermined condition. That is, the receiving unit 160 may acquire only an index for identifying the size and type of the reference coding unit for each slice, slice segment, maximum coding unit, etc. as a data unit that satisfies a predetermined condition (e.g., a data unit having a size equal to or smaller than a slice) among the various data units (e.g., a sequence, a picture, a slice, a slice segment, a maximum coding unit, etc.) from the bitstream. The video decoding apparatus 150 may determine the size and type of the reference data unit for each data unit that satisfies the predetermined condition by using the index. If information related to the type of the reference coding unit and information related to the size of the reference coding unit are acquired from the bitstream for each data unit of a relatively small size and used, the bitstream utilization efficiency is not good. Therefore, instead of directly acquiring information related to the type of the reference coding unit and information related to the size of the reference coding unit, only the index may be acquired and used. In this case, at least one of the size and type of the reference coding unit corresponding to the index indicating the size and type of the reference coding unit is also determined in advance. That is, the video decoding device 150 may determine at least one of the size and type of the reference coding unit included in the data unit serving as a reference for index acquisition by selecting at least one of the size and type of the reference coding unit determined in advance according to the index.

[0152] According to an embodiment, the video decoding device 150 may use at least one reference coding unit included in one largest coding unit. That is, the largest coding unit into which a video is divided includes at least one reference coding unit, and coding units may be determined through a recursive division process of each reference coding unit. According to an embodiment, at least one of the width and height of the largest coding unit corresponds to at least one integer multiple of the width and height of the reference coding unit. According to an embodiment, the size of the reference coding unit is a size obtained by dividing the largest coding unit n times according to a quadtree structure. That is, the video decoding device 150 may determine the reference coding unit by dividing the largest coding unit n times according to a quadtree structure, and may divide the reference coding unit based on at least one of block shape information and division shape information according to various embodiments.

[0153] FIG. 15 illustrates processing blocks that are responsible for determining a determination order of reference coding units included in a picture 1500, according to one embodiment.

[0154] According to an embodiment, the video decoding device 150 may determine at least one processing block for dividing a picture. The processing block is a data unit including at least one reference coding unit for dividing a picture, and the at least one reference coding unit included in the processing block may be determined in a specific order. That is, the determination order of the at least one reference coding unit determined in each processing block may correspond to one of various orders in which the reference coding unit may be determined, and the determination order of the reference coding unit determined in each processing block may differ for each processing block. The determination order of the reference coding unit determined for each processing block may be one of various orders such as a raster scan, a Z scan, an N scan, an up-right diagonal scan, a horizontal scan, and a vertical scan, but the order that may be determined is not limited to the scan order.

[0155] According to an embodiment, the video decoder 150 may obtain information on the size of the processing block and determine the size of at least one processing block included in the image. The video decoder 150 may obtain information on the size of the processing block from a bitstream and determine the size of at least one processing block included in the image. The size of such a processing block is also a predetermined size of a data unit indicated by the information on the size of the processing block.

[0156] According to an embodiment, the receiving unit 160 of the video decoding device 150 may obtain information on the size of a processing block for each specific data unit from a bitstream. For example, the information on the size of a processing block may be obtained from the bitstream for each data unit such as an image, a sequence, a picture, a slice, a slice segment, etc. That is, the receiving unit 160 may obtain information on the size of a processing block from the bitstream for each of the many data units, and the video decoding device 150 may determine the size of at least one processing block for dividing a picture using the obtained information on the size of the processing block, and the size of such a processing block may be an integer multiple of a reference coding unit.

[0157] According to an embodiment, the video decoder 150 may determine the size of the processing blocks 1502 and 1512 included in the picture 1500. For example, the video decoder 150 may determine the size of the processing blocks based on information related to the size of the processing blocks acquired from a bitstream. Referring to FIG. 15, according to an embodiment, the video decoder 150 may determine the horizontal size of the processing blocks 1502 and 1512 to be four times the horizontal size of a reference coding unit and the vertical size of the processing blocks 1502 and 1512 to be four times the vertical size of the reference coding unit. The video decoder 150 may determine an order in which at least one reference coding unit is determined within at least one processing block.

[0158] According to one embodiment, the video decoding device 150 may determine each of the processing blocks 1502, 1512 included in the picture 1500 based on the size of the processing block, and may determine a determination order of at least one reference coding unit included in the processing blocks 1502, 1512. According to one embodiment, the determination of the reference coding unit may include determining the size of the reference coding unit.

[0159] According to an embodiment, the video decoding device 150 may acquire information on a determination order of at least one reference coding unit included in at least one processing block from a bitstream, and may determine an order in which at least one reference coding unit is determined based on the acquired information on the determination order. The information on the determination order may also be defined as an order or direction in which a reference coding unit is determined within a processing block. That is, the order in which a reference coding unit is determined may be determined independently for each processing block.

[0160] According to an embodiment, the video decoding apparatus 150 may obtain information regarding a determination order of the reference coding units for each specific data unit from a bitstream. For example, the receiving unit 160 may obtain information regarding a determination order of the reference coding units from a bitstream for each data unit path such as an image, a sequence, a picture, a slice, a slice segment, a processing block, etc. Since the information regarding the determination order of the reference coding units indicates a determination order of the reference coding units within a processing block, the information regarding the determination order is also obtained for each specific data unit including an integer number of processing blocks.

[0161] The video decoder 150 may determine at least one reference coding unit based on the order determined according to an embodiment.

[0162] According to an embodiment, the receiving unit 160 may acquire information related to a base coding unit determination order as information related to the processing blocks 1502 and 1512 from a bitstream, and the video decoding device 150 may determine an order for determining at least one base coding unit included in the processing blocks 1502 and 1512, and may determine at least one base coding unit included in the picture 1500 according to the determination order of the coding units. Referring to FIG. 15, the video decoding device 150 may determine a determination order (1504, 1514) of at least one base coding unit associated with each of the processing blocks 1502 and 1512. For example, when information related to a determination order of the base coding units is acquired for each processing block, the determination order of the base coding units associated with each of the processing blocks 1502 and 1512 may differ for each processing block. When the reference coding unit determination order (1504) associated with the processing block 1502 is a raster scan order, the reference coding units included in the processing block 1502 are also determined in the raster scan order. On the other hand, when the reference coding unit determination order (1514) associated with another processing block 1512 is a reverse order of the raster scan order, the reference coding units included in the processing block 1512 are also determined in the reverse order of the raster scan order.

[0163] A method for dividing an image into maximum coding units and dividing the maximum coding units into coding units of a hierarchical tree structure has been described in Figures 1 to 15. An ultimate motion vector expression (UMVE) mode for deriving a motion vector for inter prediction is described in Figures 16 to 28.

[0164] Inter prediction is a prediction method for predicting a current block from a reference block similar to the current block obtained from a reference picture of the current picture. To inter predict the current block, a motion vector indicating a spatial difference between the current block and the reference block and a reference picture to which the current picture refers are determined. Then, the current block is predicted by referring to the reference block determined by the motion vector and the reference picture. The UMVE mode introduced in this disclosure is an inter prediction mode for efficiently encoding and / or decoding a motion vector.

[0165] In the UMVE mode, a UMVE candidate list is determined from blocks spatially adjacent to the current block or blocks temporally adjacent to the current block. Then, a reference picture and a basic motion vector of the current block are determined from the UMVE candidate of the current block selected from the UMVE candidate list. The motion vector of the current block is generated by correcting the basic motion vector of the current block according to the correction size and correction direction.

[0166] Inter prediction according to UMVE mode will be described in detail in FIG. 16. According to FIG. 16, a base motion vector 1600 obtained from a UMVE candidate of a current block is (5,-4). However, if the reference block indicated by the base motion vector 1600 is inaccurate, the coding efficiency of the current block will be reduced. Therefore, in the UMVE mode, the base motion vector 1600 is also corrected by a correction size 1602 and a correction direction 1604.

[0167] For example, when the correction size 1602 is 2 and the correction direction 1604 is the +x direction, a correction motion vector 1606 for correcting the base motion vector 1600 is determined to be (2, 0). Then, the sum vector (7, -4) of the base motion vector 1600 and the correction motion vector 1606 is determined to be the motion vector 1608 of the current block. Therefore, the current block is also predicted by the motion vector 1608 indicating the accurate reference block. Therefore, in the UMVE mode, the accuracy of prediction can be improved by correcting the base motion vector 1600.

[0168] In the UMVE mode, in order to reduce the size of information required for correcting the basic motion vector 1600, the correction size 1602 and the correction direction 1604 have a limited number of candidates. For example, when the correction size 1602 is determined from among eight correction size candidates, the correction size information indicating the correction size 1602 is expressed from a minimum of 1 bit to a maximum of 7 bits according to a truncated unary code. Similarly, when the correction direction 1604 is determined from four correction direction candidates (+x, -x, +y, -y), the correction direction information indicating the correction direction 1604 is also expressed by 2 bits. Therefore, the number of bits required to determine the correction motion vector 1606 is limited to 9 bits. Therefore, in the UMVE mode, the number of bits required to determine the correction motion vector 1606 is limited to a determined size or less, thereby increasing the compression efficiency.

[0169] FIG. 17 illustrates a block diagram of a video decoding device 1700 for decoding according to the UMVE mode.

[0170] The video decoding apparatus 1700 includes a high-level parameter determination unit 1710, a coding mode determination unit 1720, and a decoding unit 1730. In FIG. 17, the high-level parameter determination unit 1710, the coding mode determination unit 1720, and the decoding unit 1730 are expressed as separate components, but according to an embodiment, the high-level parameter determination unit 1710, the coding mode determination unit 1720, and the decoding unit 1730 may be combined into one component.

[0171] 17, the higher-level parameter determination unit 1710, the coding mode determination unit 1720, and the decoding unit 1730 are expressed as a structural unit located in one device, but the devices responsible for each function of the higher-level parameter determination unit 1710, the coding mode determination unit 1720, and the decoding unit 1730 do not necessarily need to be physically adjacent to each other. Therefore, depending on the embodiment, the higher-level parameter determination unit 1710, the coding mode determination unit 1720, and the decoding unit 1730 may be distributed.

[0172] Depending on the embodiment, the upper level parameter determining unit 1710, the coding mode determining unit 1720, and the decoding unit 1730 may be implemented by one processor or by a plurality of processors.

[0173] The upper parameter determination unit 1710 may determine whether the UMVE mode is allowed for an upper data unit of the current block. The upper data unit may be a sequence, a picture, a slice, or a slice segment of the current block. For example, the upper parameter determination unit 1710 may determine whether the UMVE mode is allowed for a picture unit.

[0174] The upper parameter determination unit 1710 may determine whether the UMVE mode is permitted for each upper data unit. For example, the upper parameter determination unit 1710 may determine whether the UMVE mode is permitted for a current sequence. The upper parameter determination unit 1710 may then determine whether the UMVE mode is permitted for a current picture included in the current sequence. Similarly, the upper parameter determination unit 1710 may determine whether the UMVE mode is permitted for a current slice included in the current picture.

[0175] If the UMVE mode is not allowed for the current sequence, the upper parameter determination unit 1710 may determine that the UMVE mode is not allowed for all pictures included in the current sequence. Similarly, if the UMVE mode is not allowed for the current picture, the upper parameter determination unit 1710 may determine that the UMVE mode is not allowed for all slices included in the current picture.

[0176] The upper parameter determination unit 1710 acquires a UMVE enabled flag indicating whether the UMVE mode is enabled for the upper data unit of the current block from the bitstream. If the UMVE enabled flag indicates that the UMVE mode is enabled for the upper data unit, the coding mode determination unit 1720 determines whether the UMVE mode is applied to all blocks included in the upper data unit. On the other hand, if the UMVE enabled flag indicates that the UMVE mode is not enabled for the upper data unit, the coding mode determination unit 1720 determines that the UMVE mode is not applied to all blocks included in the upper data unit.

[0177] When the UMVE allow flag indicates 1, the UMVE allow flag is also interpreted as allowing the UMVE mode for the upper data unit. Conversely, when the UMVE allow flag indicates 0, the UMVE allow flag is also interpreted as not allowing the UMVE mode for the upper data unit. Depending on the embodiment, the meaning of the UMVE allow flag is also interpreted in the opposite way.

[0178] The upper parameter determination unit 1710 may acquire a UMVE permission flag for each upper data unit. If the permission of the UMVE mode is determined on a picture-by-picture basis, the upper parameter determination unit 1710 may acquire a UMVE permission flag on a picture-by-picture basis. However, if the permission of the UMVE mode is determined hierarchically in the order of a sequence unit, a picture unit, and a slice unit, the upper parameter determination unit 1710 may acquire a UMVE permission flag for a sequence unit, a UMVE permission flag for a picture unit, and a UMVE permission flag for a slice unit.

[0179] In another embodiment, the UMVE enable flag may indicate whether or not a default setting related to the UMVE mode is applied to the upper data unit. For example, when the UMVE enable flag indicates 0, the UMVE enable flag may be interpreted as applying the default setting related to the UMVE mode to the upper data unit. Conversely, when the UMVE enable flag indicates 1, the UMVE enable flag may be interpreted as not applying the default setting related to the UMVE mode to the upper data unit. If the UMVE enable flag is not acquired, the upper parameter determination unit 1710 may apply the default setting related to the UMVE mode to the upper data unit.

[0180] The basic settings for the UMVE mode may not allow the UMVE mode for the upper data unit, or may apply only the UMVE mode to the upper data unit under a specific condition. The basic settings for the UMVE mode may be set differently for each sequence unit, each picture unit, and each slice unit.

[0181] The upper parameter determination unit 1710 may determine whether the upper data unit is allowed to use the UMVE mode without the UMVE allowed flag according to a specific condition. For example, if the current picture is the last picture of a group of pictures (GoP), the upper parameter determination unit 1710 may apply a basic setting related to the UMVE mode to the upper data unit without the UMVE allowed flag. As another example, the upper parameter determination unit 1710 may apply a basic setting related to the UMVE mode to the upper data unit without the UMVE allowed flag according to a temporal layer depth of the current picture.

[0182] The upper parameter determination unit 1710 may determine whether to allow other inter prediction modes depending on whether the UMVE mode is allowed for the upper data unit. For example, if the UMVE mode is allowed by the UMVE allowance flag, a predetermined inter prediction mode is not allowed for the upper data unit. The predetermined inter prediction mode may include a decoder-side motion vector derivation (DMVD) mode, a decoder-side motion vector refinement (DMVR) mode, a skip mode, a direct mode, a merge mode, etc.

[0183] For example, in the UMVE mode, information related to a motion vector is obtained from a bitstream, whereas in the DMVD mode or DMVR mode, information related to a motion vector is not obtained from the bitstream and a motion vector is derived. Therefore, according to an embodiment, when the UMVE mode is allowed for the upper data unit, the upper parameter determination unit 1710 may determine that the DMVD mode or the DMVR mode is not allowed for the upper data unit.

[0184] As another example, when the UMVE mode is allowed for the upper data unit, the upper parameter determination unit 1710 may determine that the skip mode, direct mode, and merge mode that omit information related to a motion vector are not allowed.

[0185] According to another embodiment, the upper parameter determination unit 1710 may acquire an allowance flag for the predetermined inter prediction mode from the bitstream for the upper data unit when the UMVE mode is allowed for the upper data unit. According to yet another embodiment, the upper parameter determination unit 1710 may apply a basic setting for the predetermined inter prediction mode to the upper data unit without acquiring an allowance flag for the predetermined inter prediction mode when the UMVE mode is allowed for the upper data unit. The predetermined inter prediction mode may include a DMVD mode, a DMVR mode, a skip mode, a direct mode, a merge mode, an overlapped block motion compensation (OBMC) mode, an illumination compensation (IC) mode, an affine mode, and the like.

[0186] If the UMVE mode is allowed for the upper data unit, the upper parameter determination unit 1710 may additionally acquire setting information related to the UMVE mode to be applied to the upper data unit. Hereinafter, the additional information related to the UMVE mode acquired by the upper parameter determination unit 1710 will be described.

[0187] The higher-level parameter determination unit 1710 may obtain UMVE candidate number information indicating the number of UMVE candidates of the UMVE mode permitted for the higher-level data unit from the bitstream. Then, the decoding unit 1730 determines UMVE candidates as many as the number indicated by the UMVE candidate number information.

[0188] In addition, the higher-level parameter determination unit 1710 may acquire correction size candidate number information indicating the number of correction size candidates permitted for the higher-level data unit from the bitstream. Then, the decoding unit 1730 determines as many correction size candidates as the number indicated by the correction size candidate number information.

[0189] In addition, the higher-level parameter determination unit 1710 may acquire, from the bitstream, information on the number of correction direction candidates indicating the number of correction direction candidates permitted for the higher-level data unit. Then, the decoding unit 1730 determines as many correction direction candidates as the number indicated by the information on the number of correction direction candidates.

[0190] The UMVE candidate number information may indicate a difference between the minimum number of UMVE candidates and the number of UMVE candidates used in the upper data unit. For example, if the minimum number of UMVE candidates is 4 and the number of UMVE candidates used in the upper data unit is 5, the UMVE candidate number information may indicate 1. Similar to the UMVE candidate number information, the correction size candidate number information and the correction direction candidate number information may also indicate a difference between the minimum number and the number of candidates used in the upper data unit.

[0191] The high-level parameter determination unit 1710 may acquire UMVE candidate list configuration information indicating a method for configuring a UMVE candidate list from a bitstream. The UMVE candidate list configuration information may indicate that a UMVE candidate list in the UMVE mode is configured based on a motion vector candidate list used in the merge mode.

[0192] Alternatively, the UMVE candidate list configuration information may indicate that a motion vector candidate list dedicated to the UMVE mode is configured. For example, the motion vector candidate list dedicated to the UMVE mode may also be configured by statistically analyzing motion vector information used in decoding the current picture. If the left block and the upper block of the current block are highly likely to be selected as UMVE candidates, the left block and the upper block of the current block may be preferentially included in the UMVE candidate list.

[0193] Alternatively, the UMVE candidate list configuration information may indicate that a motion vector candidate list is configured by mixing a motion vector candidate list used in the merge mode and a motion vector candidate list dedicated to the UMVE mode.

[0194] If the upper parameter determination unit 1710 cannot obtain UMVE candidate list configuration information or if the UMVE candidate list is determined implicitly, it selects a UMVE candidate list configuration method based on basic settings.

[0195] The higher level parameter determination unit 1710 may obtain correction size range information for a plurality of blocks of the higher level data unit from the bitstream, and the decoding unit 1730 determines the correction size of the base motion vector based on the correction size range indicated by the correction size range information.

[0196] For example, the correction size range information may indicate a minimum value of the correction size candidates. If the minimum value of the correction size candidates indicated by the correction size range information is 1 and the number of correction size candidates indicated by the correction size number candidate information is 4, the correction size candidates are also determined to be {1, 2, 4, 8}.

[0197] Also, the correction size range information may indicate a size difference between correction size candidates. For example, the correction size range information may indicate whether the values ​​of the correction size candidates increase arithmetically or geometrically. If the values ​​of the correction size candidates increase arithmetically, the correction size candidates are determined as {1, 2, 3, 4, 5, ...}. If the values ​​of the correction size candidates increase geometrically, the correction size candidates are determined as {1, 2, 4, 8, 16, ...}.

[0198] Also, the correction size range information may indicate one of a plurality of predetermined correction size candidate sets. For example, the correction size range information may indicate one of a first correction size candidate set {1,2,4,8}, a second correction size candidate set {1,2,3,4}, and a third correction size candidate set {4,8,16,32}.

[0199] The higher level parameter determination unit 1710 may obtain correction size change information for a plurality of blocks of the higher level data unit from the bitstream. The correction size change information indicates whether a range of correction size candidates is changed according to a temporal distance between the current picture and the reference picture. Therefore, when the correction size change information indicates a change in the range of correction size candidates, the decoding unit 1730 may determine a range of correction size candidates according to a temporal distance between the current picture and the reference picture.

[0200] In general, when a current block refers to a reference block of a reference picture that is far away from the current picture in time, the probability that the error of the basic motion vector is large is high. On the other hand, when a current block refers to a reference block of a reference picture that is close to the current picture in time, the probability that the error of the basic motion vector is small is high. Therefore, when a correction size candidate set is determined in proportion to the temporal distance between the current picture and the reference picture, the prediction accuracy of the motion vector can be improved.

[0201] For example, when the temporal distance between the current picture and the reference picture is short, the minimum value of the correction size candidate is determined to be 1 / 2. When the temporal distance between the current picture and the reference picture is intermediate, the minimum value of the correction size candidate is determined to be 1. When the temporal distance between the current picture and the reference picture is long, the minimum value of the correction size candidate is determined to be 2.

[0202] The temporal distance between the current picture and the reference picture may also be determined by one or more threshold values. For example, if the threshold value is 3, the temporal distance between the current picture and the reference picture may be compared with the three threshold values ​​to determine the minimum correction size candidate.

[0203] The higher-level parameter determination unit 1710 may determine a minimum value of correction size candidates for the higher-level data unit based on a correction size candidate minimum value of the correction size range information and a temporal distance between the current picture and the reference picture. Specifically, the correction size candidate minimum value of the correction size range information may be changed based on a temporal distance between the current picture and the reference picture, thereby determining a minimum value of correction size candidates for the higher-level data unit.

[0204] If the upper parameter determination unit 1710 cannot obtain the compensation size change information or if the compensation size change is determined internally, the upper parameter determination unit 1710 determines whether to change the compensation size according to the basic setting.

[0205] The upper parameter determination unit 1710 may obtain prediction direction change permission information for a plurality of blocks of the upper data unit from the bitstream. The current block obtains information related to the prediction direction of the reference picture together with information related to the base motion vector from a UMVE candidate selected from the UMVE candidate list. If the prediction direction change permission information does not permit a change in the prediction direction, the prediction direction of the UMVE candidate is applied to the current block. However, on the other hand, if the prediction direction change permission information permits a change in the prediction direction, a prediction direction different from that of the UMVE candidate may be applied to the current block.

[0206] The prediction direction indicates one of list 0 uni-prediction, list 1 uni-prediction, and bi-prediction using both list 0 and list 1. In the case of uni-prediction, only one reference block of list 0 or list 1 is used to predict the current block, but in the case of bi-prediction, the reference block of list 0 and the reference block of list 1 are both used. Therefore, the prediction accuracy of bi-prediction is likely to be higher than that of uni-prediction. Therefore, when a change in the prediction direction is allowed, even if the UMVE candidate indicates uni-prediction, the prediction direction of the UMVE mode is also determined as bi-prediction, and the coding efficiency of the UMVE mode is increased.

[0207] Therefore, when the prediction direction change permission information allows a change in the prediction direction, the decoding unit 1730 can change the prediction direction of the current block. For example, when a change in the prediction direction is allowed, even if the prediction direction of the UMVE candidate is list 0 uni-prediction, the prediction direction of the current block is also determined to be list 1 uni-prediction or bi-prediction.

[0208] The upper parameter determination unit 1710 may obtain primary prediction direction information related to the upper data unit from the bitstream. The primary prediction direction information indicates whether there is a primary prediction direction related to the upper data unit. In addition, the primary prediction direction information indicates what the primary prediction direction of the upper data unit is if there is a primary prediction direction for the upper data unit. If the primary prediction direction of the upper data unit is set, the block to which the UMVE mode is applied, which is included in the upper data unit, is predicted according to the primary prediction direction. For example, if the primary prediction direction is determined to be bi-predictive, the block to which the UMVE mode is applied may be bi-predicted. Therefore, even if the prediction direction of the UMVE candidate is uni-predictive, the prediction direction of the current block is also determined to be bi-predictive.

[0209] The higher-level parameter determination unit 1710 may acquire both prediction direction change permission information and primary prediction direction information. When the prediction direction change permission information indicates permission of prediction direction change and the primary prediction direction information indicates that the primary prediction direction is set, the prediction direction information acquired by the decoding unit 1730 indicates whether the prediction direction of the current block is the primary prediction direction. For example, when the primary prediction direction is bi-prediction and the prediction direction information indicates that the prediction direction of the current block is the primary prediction direction, the prediction direction of the current block is determined to be bi-prediction. If the primary prediction direction is bi-prediction and the prediction direction information indicates that the prediction direction of the current block is not the primary prediction direction, the prediction direction of the current block is determined to be list 0 uni-prediction or list 1 uni-prediction, which is a preliminary prediction direction, according to the prediction direction information.

[0210] The higher level parameter determination unit 1710 can obtain multiple correction allowance information indicating how many times the corrected base motion vector is corrected from the bitstream. If the multiple correction allowance information allows the base motion vector to be corrected several times, the decoding unit 1730 can correct the base motion vector several times.

[0211] The coding mode determination unit 1720 determines whether the UMVE mode is applied to the current block when the UMVE mode is allowed for the higher data unit. The coding mode determination unit 1720 may acquire a UMVE flag indicating whether the UMVE mode is applied to the current block from the bitstream. For example, when the UMVE flag indicates that the UMVE mode is applied to the current block, the coding mode determination unit 1720 may determine the prediction mode of the current block to be the UMVE mode. Conversely, when the UMVE flag indicates that the UMVE mode is not applied to the current block, the coding mode determination unit 1720 does not determine the prediction mode of the current block to be the UMVE mode.

[0212] The coding mode determination unit 1720 may determine that, when the UMVE mode is applied to the current block, another inter prediction mode is not applied to the current block. For example, when the UMVE mode is applied to the current block due to the UMVE flag, a predetermined inter prediction mode is not applied to the current block. The predetermined inter prediction mode may include a DMVD mode, a DMVR mode, a skip mode, a direct mode, a merge mode, an OBMC mode, an IC mode, an affine mode, etc. Conversely, the coding mode determination unit 1720 may determine that a specific prediction mode is applied to the current block when the UMVE mode is applied to the current block.

[0213] In addition, when the UMVE mode is not applied to the current block, the coding mode determination unit 1720 may determine that another intra prediction mode is applied to the current block. For example, the intra prediction mode may include a DMVD mode, a DMVR mode, a merge mode, an OBMC mode, an IC mode, an affine mode, etc.

[0214] When the UMVE mode is applied to the current block, the decoder 1730 may determine a basic motion vector of the current block from the UMVE candidates. The decoder 1730 may determine a UMVE candidate list by a UMVE candidate list configuration method according to the UMVE candidate list configuration information. The number of UMVE candidates included in the UMVE candidate list is also determined by the UMVE candidate number information.

[0215] The decoder 1730 may obtain a UMVE index indicating a UMVE candidate including a base motion vector and a reference picture of the current block from the UMVE candidate list from the bitstream. Then, the decoder 1730 may determine the base motion vector and the reference picture of the current block according to the UMVE index.

[0216] The decoder 1730 may determine a correction size and a correction direction for correcting the base motion vector. In the UMVE mode, the base motion vector is corrected for prediction accuracy. Therefore, in order to determine a correction motion vector for correcting the base motion vector, the decoder 1730 determines a correction size indicating the magnitude of the correction motion vector and a correction direction indicating the direction of the correction motion vector.

[0217] The decoding unit 1730 may determine correction size candidates according to the correction size candidate number information, the correction size range information, and the correction size change information related to the higher data unit. The decoding unit 1730 may determine the number of correction size candidates according to the correction size candidate number information. The decoding unit 1730 may determine a minimum value of the correction size candidates and a size difference between the correction size candidates according to the correction size range information. In addition, the decoding unit 1730 may determine the range of the correction size candidates according to a distance between the current picture and the reference picture when the correction size change information indicates a change in the range of the correction size candidates. For example, the minimum value of the correction size candidates may be changed by comparing the distance between the current picture and the reference picture with a predetermined threshold value. Thus, the range of the correction size candidates may be determined according to the changed minimum value.

[0218] The decoding unit 1730 may obtain correction size information indicating a correction size of the base motion vector from the bitstream. The correction size information indicates a correction size candidate corresponding to the correction size of the base motion vector among a predetermined number of correction size candidates. The decoding unit 1730 may determine the correction size of the base motion vector according to the correction size candidate indicated by the correction size information.

[0219] The decoding unit 1730 can determine the correction direction candidate based on the correction size direction number information related to the higher-order data unit. Then, the decoding unit 1730 determines the correction direction from a predetermined number of correction direction candidates.

[0220] The decoding unit 1730 may obtain correction direction information indicating a correction direction of the base motion vector from the bitstream. The correction direction information indicates a correction direction candidate corresponding to the correction direction of the base motion vector among a predetermined number of correction direction candidates. The decoding unit 1730 may determine the correction direction of the base motion vector according to the correction direction candidate indicated by the correction direction information.

[0221] The decoding unit 1730 may determine a motion vector of a current block by correcting a base motion vector according to the determined correction size and correction direction. For example, if the correction size is 2 and the correction direction is +x, a motion vector with an x ​​value increased by 2 from the base motion vector is used for predicting the current block.

[0222] 18 to 21 are diagrams for explaining a process of determining a motion vector of a current block in UMVE mode. In FIG. 18 and FIG. 21, the coordinates indicated by the base motion vector are set to (base_x, base_y). In addition, the correction motion vector according to the correction size and correction direction in UMVE mode will be explained based on (base_x, base_y).

[0223] 18, the decoding unit 1730 may determine motion vector candidates having a diamond-shaped distribution based on a base motion vector.

[0224] The decoding unit 1730 can determine the motion vector candidates ((base_x+1 / 4,base_y) 1802, (base_x-1 / 4,base_y) 1803, (base_x,base_y+1 / 4) 1804, (base_x,base_y-1 / 4) 1805) located 1 / 4 pixel away from the base motion vector (base_x,base_y) 1801 as the first candidate group.

[0225] The decoding unit 1730 can determine the motion vector candidates ((base_x+1 / 2,base_y) 1806, (base_x-1 / 2,base_y) 1807, (base_x,base_y+1 / 2) 1808, (base_x,base_y-1 / 2) 1809) located 1 / 2 pixel away from the base motion vector (base_x,base_y) 1801 as a second candidate group.

[0226] The decoding unit 1730 can determine the motion vector candidates ((base_x+1, base_y) 1810, (base_x-1, base_y) 1811, (base_x, base_y+1) 1812, (base_x, base_y-1) 1813) that are one pixel away from the base motion vector (base_x, base_y) 1801 as a third candidate group.

[0227] The decoding unit 1730 may select one of the first to third candidate groups according to the correction distance, and may determine one motion vector candidate from the candidate group as the motion vector of the current block according to the correction direction.

[0228] 19, the decoding unit 1730 may determine motion vector predictor candidates having a rectangular distribution based on a base motion vector.

[0229] Similarly, the decoding unit 1730 can determine that the motion vector candidates ((base_x+1 / 4,base_y+1 / 4) 1902, (base_x+1 / 4,base_y-1 / 4) 1903, (base_x-1 / 4,base_y+1 / 4) 1904, (base_x-1 / 4,base_y-1 / 4) 1905) located approximately 1 / 4 pixel away from the base motion vector (base_x,base_y) 1901 are the first candidate group.

[0230] The decoding unit 1730 can determine the motion vector candidates ((base_x+1 / 2,base_y+1 / 2) 1906, (base_x+1 / 2,base_y-1 / 2) 1907, (base_x-1 / 2,base_y+1 / 2) 1908, (base_x-1 / 2,base_y-1 / 2) 1909) located approximately 1 / 2 pixel away from the base motion vector (base_x,base_y) 1901 as a second candidate group.

[0231] The decoding unit 1730 can determine the motion vector candidates ((base_x+1,base_y+1) 1910, (base_x+1,base_y-1) 1911, (base_x-1,base_y+1) 1912, (base_x-1,base_y-1) 1913) located approximately one pixel away from the base motion vector (base_x,base_y) 1901 as a third candidate group.

[0232] 20, the video decoding device 1730 may determine each group as a different number of motion vector candidates. Although the interval between each pixel is 1 / 4 pixel, the component values ​​of the vector candidates are scaled by 4 times for convenience.

[0233] Specifically, the decoding unit 1730 can determine eight motion vector candidates ((base_x+1 / 4,base_y / 4) 2002, (base_x-1 / 4,base_y) 2003, (base_x,base_y+1 / 4) 2004, (base_x,base_y-1 / 4) 2005, (base_x+1 / 4,base_y+1 / 4) 2006, (base_x+1 / 4,base_y-1 / 4) 2007, (base_x-1 / 4,base_y+1 / 4) 2008, (base_x-1 / 4,base_y-1 / 4) 2009) located at a distance of approximately 1 / 4 pixel from the base motion vector as a first candidate group.

[0234] In addition, the decoding unit 1730 can determine eight motion vector candidates ((base_x+1 / 2,base_y) 2010, (base_x-1 / 2,base_y) 2011, (base_x,base_y+1 / 2) 2012, (base_x,base_y-1 / 2) 2013, (base_x+1 / 2,base_y+1 / 2) 2014, (base_x+1 / 2,base_y-1 / 2) 2015, (base_x-1 / 2,base_y+1 / 2) 2016, (base_x-1 / 2,base_y-1 / 2) 2017) located at a distance of approximately 1 / 2 pixel from the base motion vector as a second candidate group.

[0235] The decoding unit 1730 can determine eight motion vector candidates ((base_x+1,base_y) 2018, (base_x-1,base_y) 2019, (base_x,base_y+1) 2020, (base_x,base_y-1) 2021, (base_x+1,base_y+1) 2022, (base_x+1,base_y-1) 2023, (base_x-1,base_y+1) 2024, (base_x-1,base_y-1) 2025) that are located approximately one pixel away from the base motion vector as a third candidate group.

[0236] 21, the decoding unit 1730 may determine various distribution patterns of motion vector candidates included in each candidate group for each candidate group. Specifically, the decoding unit 1730 may determine motion vector candidates 2102, 2103, 2104, and 2105 having a diamond-shaped distribution based on a base motion vector 2101 as a first candidate group. Also, the decoding unit 1730 may determine motion vector candidates 2106, 2107, 2108, and 2109 having a square-shaped distribution based on the base motion vector 2101 as a second candidate group. Also, the decoding unit 1730 may determine motion vector candidates 2110, 2111, 2112, and 2113 having a diamond-shaped distribution based on the base motion vector 2101 as a third candidate group. As shown in FIG. 21, the distribution of motion vector candidates of each candidate group may be determined to have various distribution patterns other than the distribution patterns shown in FIG. 21.

[0237] The decoder 1730 may determine one or more base motion vectors. If there are two base motion vectors, motion vector candidates may also be generated using each of the base motion vectors.

[0238] The decoder 1730 may perform bi-prediction. If the base motion vector is performed as bi-prediction using the reference pictures in list 0 and list 1, the motion vector of each prediction direction is also corrected by the temporal distance between the current picture and the reference picture.

[0239] For example, if the reference pictures of list 0 and the reference pictures of list 1 are located in the same direction from the current picture, the correction directions of the base motion vectors of list 0 and list 1 are determined to be the same. Conversely, if the reference pictures of list 0 and the reference pictures of list 1 are located in opposite directions from the current picture, the correction direction of the base motion vectors of list 0 is determined to be opposite to the correction direction of the base motion vectors of list 1.

[0240] Furthermore, the correction size of the base motion vector of list 0 and the correction size of the base motion vector of list 1 are determined to be proportional to the temporal distance between the reference picture of list 0 and the current picture, and the temporal distance between the reference picture of list 1 and the current picture, respectively. Therefore, when the temporal distance between the reference picture of list 0 and the current picture is twice the temporal distance between the reference picture of list 1 and the current picture, the correction size of the base motion vector of list 0 is determined to be twice the correction size of the base motion vector of list 0.

[0241] The correction size and correction direction indicated by the correction size information and correction direction information obtained from the bitstream are also applied to the base motion vectors of list 0.

[0242] The correction size and correction direction used for correcting the base motion vector of list 0 are also changed taking into account the distance between the reference picture of list 0 and the current picture and the distance between the reference picture of list 1 and the current picture, and the changed correction size and correction direction are then applied to the base motion vector of list 1.

[0243] In an embodiment, conversely, the correction size and correction direction indicated by the correction size information and correction direction information are applied to the basic motion vectors of list 1, and the changed correction size and correction direction are also applied to the basic motion vectors of list 0.

[0244] Figure 22 shows an embodiment of a method for correcting two basic motion vectors by bi-prediction. In Figure 22, a basic motion vector 2202 of list 0 indicates (0, -1), and a basic motion vector 2212 of list 1 indicates (0, 2).

[0245] In UMVE mode, the correction size and correction direction are applied to both the base motion vector 2202 of list 0 and the base motion vector 2212 of list 1. When the correction size for the base motion vector 2202 in the list 0 direction is 1 and the correction direction is −y, the motion vector 2204 in list 0 becomes (0, −2).

[0246] The reference pictures of list 0 and the reference pictures of list 1 are located in opposite directions from the current picture, so the correction direction of the base motion vector 2212 of list 1 is determined to be the opposite of the correction direction of the base motion vector 2202 of list 0. Therefore, the correction direction of the base motion vector 2212 of list 1 is determined to be +y.

[0247] The reference pictures of list 0 and the reference pictures of list 1 are the same temporal distance away from the current picture. Therefore, the correction size of the base motion vector 2212 of list 1 is determined to be the same as the correction size of the base motion vector 2202 in the list 0 direction. Therefore, the correction size of the base motion vector 2212 of list 1 is determined to be 1.

[0248] Therefore, the motion vector 2214 of list 1 is determined to be (0, 3), which is larger by 1 in y component than the basic motion vector 2212 of list 1 .

[0249] If the correction size associated with the basic motion vector 2202 in the list 0 direction is 2, the motion vector 2206 in the list 0 direction is determined to be (0, -3), and the motion vector 2216 in the list 1 direction is determined to be (0, 4). If the correction size associated with the basic motion vector 2202 in the list 0 direction is 4, the motion vector 2208 in the list 0 direction is determined to be (0, -5), and the motion vector 2218 in the list 1 direction is determined to be (0, 6).

[0250] If the prediction direction change permission information indicates that a change in the prediction direction is permitted, the decoder 1730 may obtain prediction direction information from the bitstream and change the prediction direction of the current block according to the prediction direction information. The prediction direction information may indicate whether or not to change the prediction direction. If the prediction direction information does not indicate a change in the prediction direction, the current block is predicted according to the prediction direction of the UMVE candidate. However, if the prediction direction information indicates a change in the prediction direction, the current block is predicted according to a prediction direction different from that of the UMVE candidate.

[0251] Additionally, the prediction direction information may indicate a prediction direction to be changed. For example, if the prediction direction of the UMVE candidate is list 0 uni-prediction, the prediction direction information may indicate one of list 1 uni-prediction and bi-prediction. Thus, the prediction direction information may indicate whether the prediction direction is changed and the changed prediction direction.

[0252] If there is no change in the prediction direction, the prediction direction information is also set to 0. If there is a change in the prediction direction, the prediction direction information is also set to 10 or 11 according to the changed prediction direction. For example, if the prediction direction of the UMVE candidate is list 0 uni-prediction, the prediction direction information is also set to 0 when the prediction direction of the current block is list 0 uni-prediction.

[0253] If the prediction direction of the current block is bi-predictive or list 1 uni-predictive, the prediction direction information is also set to 10 or 11. According to an embodiment, if the prediction direction of the UMVE candidate is uni-predictive and the prediction direction of the current block is bi-predictive, the prediction direction information is also set to 10. According to an embodiment, if the prediction direction of the UMVE candidate is uni-predictive and the prediction direction of the current block is uni-predictive of the other direction, the prediction direction information is also set to 11.

[0254] As another example, if the prediction direction of a UMVE candidate is list 1 uni-prediction, the prediction direction information is also set to 0 when the prediction direction of the current block is list 1 uni-prediction.

[0255] If the prediction direction of the current block is bi-predictive or list 0 uni-predictive, the prediction direction information is also set to 10 or 11. According to an embodiment, if the prediction direction of the UMVE candidate is uni-predictive and the prediction direction of the current block is bi-predictive, the prediction direction information is also set to 10. According to an embodiment, if the prediction direction of the UMVE candidate is uni-predictive and the prediction direction of the current block is uni-predictive in the other direction, the prediction direction information is also set to 11.

[0256] As another example, if the prediction direction of the UMVE candidate is bi-predictive, the prediction direction information may also be set to 0 when the prediction direction of the current block is bi-predictive.

[0257] If the prediction direction of the current block is list 0 uni-predictive or list 1 uni-predictive, the prediction direction information is also set to 10 or 11. According to an embodiment, if the prediction direction of the UMVE candidate is bi-predictive and the prediction direction of the current block is list 0 uni-predictive, the prediction direction information is also set to 10. According to an embodiment, if the prediction direction of the UMVE candidate is bi-predictive and the prediction direction of the current block is list 1 uni-predictive, the prediction direction information is also set to 11.

[0258] If the prediction direction information indicates a change in the prediction direction, the decoder 1730 may modify the motion vector of the UMVE candidate according to the modified prediction direction. For example, if the prediction direction of the UMVE candidate is list 0 uni-prediction and the prediction direction of the current block is list 1 uni-prediction, the motion vector of list 0 is modified to be suitable for prediction by the reference picture of list 1. Specifically, the motion vector of list 0 is modified to be scaled according to the ratio of the temporal distance between the reference picture of list 0 and the current picture and the temporal distance between the reference picture of list 1 and the current picture.

[0259] For example, when the motion vector of list 0 is (8,6) and the ratio of the temporal distance between the reference picture of list 0 and the current picture and the temporal distance between the reference picture of list 1 and the current picture is 2:1, the motion vector of list 1 is also determined to be (4,3). If the ratio of the temporal distance is 2:-1, that is, the reference picture of list 1 and the reference picture of list 0 are in different directions from the current picture, the motion vector of list 1 is also determined to be (-4,-3).

[0260] If the prediction direction of the UMVE candidate is list 0 uni-prediction and the prediction direction of the current block is bi-prediction, the motion vector of list 1 can be obtained based on the motion vector of list 0 of the UMVE candidate. The motion vector of list 1 can also be obtained by scaling the motion vector related to the reference picture of list 0 according to the ratio of the temporal distance between the reference picture of list 0 and the current picture and the temporal distance between the reference picture of list 1 and the current picture. The reference picture of list 1 is also determined as a picture located in a specific order of list 1. The reference picture and motion vector of list 0 and the reference picture and motion vector of list 1 can both be used to predict the current block.

[0261] Similarly, when the prediction direction of a UMVE candidate is list 1 uni-prediction and the prediction direction of a current block is bi-prediction, a motion vector of list 0 may be obtained from a motion vector of list 1. A reference picture of list 0 may also be determined to be a picture located in a specific order of list 0. A reference picture and motion vector of list 0 and a reference picture and motion vector of list 1 may both be used to predict a current block.

[0262] If the prediction direction of the UMVE candidate is bi-predictive and the prediction direction of the current block is list 0 uni-predictive, the current block is predicted using only the motion vectors and reference pictures of list 0 among the motion vectors and reference pictures of list 0 and the motion vectors and reference pictures of list 1. Conversely, if the changed prediction direction is bi-predictive and the changed prediction direction is list 1 uni-predictive, the current block is predicted using only the motion vectors and reference pictures of list 1 among the motion vectors and reference pictures of list 0 and the motion vectors and reference pictures of list 1.

[0263] When the primary prediction direction information is acquired, the decoding unit 1730 may change the motion vector of the UMVE candidate according to the primary prediction direction indicated by the primary prediction direction information. If the primary prediction direction of the current picture and the reference prediction direction of the UMVE candidate are the same, the current block is predicted by the reference picture and motion vector of the UMVE candidate. However, if the primary prediction direction of the current picture and the reference prediction direction of the UMVE candidate are different, the motion vector of the UMVE candidate is changed according to the primary prediction direction. The change of the motion vector and reference picture of the UMVE candidate according to the primary prediction direction is performed in the same manner as the change of the motion vector and reference picture of the UMVE candidate according to the prediction direction information.

[0264] According to an embodiment, the decoding unit 1730 may change the prediction direction of the current block according to the prediction direction information when the main prediction direction information is acquired and the prediction direction change permission information allows a change in the prediction direction. When the main prediction direction information is set, the prediction direction information indicates a prediction direction of the current block among the main prediction direction and two auxiliary prediction directions. If the prediction direction information indicates the main prediction direction, the current block is predicted according to the main prediction direction. However, if the prediction direction information indicates an auxiliary prediction direction, the current block is predicted according to the prediction direction indicated by the prediction direction information among the two auxiliary prediction directions. When the main prediction direction information is set, 0 may be assigned to the main prediction direction, 10 to the first auxiliary prediction direction, and 11 to the second auxiliary prediction direction in the prediction direction information. The two auxiliary prediction directions related to the main prediction direction may be determined by various methods according to an embodiment.

[0265] When the base motion vector can be corrected several times according to the multiple correction allowance information, the decoding unit 1730 can correct the corrected base motion vector once more. Thus, the decoding unit 1730 can obtain the additional correction size information and the additional correction direction information from the bitstream. Then, the decoding unit 1730 can correct the base motion vector once more based on the additional correction size determined by the additional correction size information and the additional correction direction determined by the additional correction direction information.

[0266] When the correction size is equal to or greater than the threshold value, the decoder 1730 may obtain the additional correction size information and the additional correction direction information. The additional correction size and the additional correction direction indicated by the additional correction size information and the additional correction direction information are also restricted by the correction size and the correction direction. For example, the additional correction size may be determined to be smaller than the correction size. Alternatively, the additional correction direction may be determined to be different from the correction direction.

[0267] The decoding unit 1730 may reconstruct the current block based on the motion vector and the reference picture of the current block. When the prediction direction of the current block is list 0 uni-prediction or list 1 uni-prediction, the decoding unit 1730 may predict the current block using one motion vector and reference picture. When the prediction direction of the current block is bi-prediction, the decoding unit 1730 may predict the current block using the motion vector and reference picture of list 0 and the motion vector and reference picture of list 1.

[0268] The decoding unit 1730 may reconstruct the current block based on the prediction result of the current block. The decoding unit 1730 may reconstruct the current block based on the prediction block of the current block and the residual block of the current block. The residual block of the current block indicates a difference between the reconstructed block of the current block and the prediction block.

[0269] The UMVE candidate number information, correction size candidate number information, correction direction candidate number information, UMVE candidate list configuration information, correction size range information, main prediction direction information, UMVE index, correction size information and correction direction information, prediction direction information, additional correction size information, and additional correction direction information decoded by the video decoding device 1700 are also binarized by fixed length coding or truncated unary coding.

[0270] The functions of the high-level parameter determination unit 1710, the coding mode determination unit 1720 and the decoding unit 1730 are also performed in the decoding unit 170 of FIG. 1B.

[0271] 23 to 25 show an example of a method for selecting an inter mode tool for a reference picture and a motion vector of a current block. FIG. 23 to 25 commonly show a syntax structure of a coding unit. According to the syntax structure according to the embodiment, when the current block is applied to the skip mode or the direct mode, the UMVE mode may be selected. In FIG. 23 to 25, ae(v) means that the syntax element in the same row is entropy decoded. Therefore, cu_skip_flag[x0][y0], cu_skip_umve[x0][y0] in FIG. 23, cu_direct[x0][y0], affine_dir[x0][y0] in FIG. 24, and cu_direct[x0][y0], cu_direct_umve[x0][y0] in FIG. 25 are entropy decoded.

[0272] Fig. 23 discloses a method for selecting an inter mode tool for a current block in skip mode. In skip mode, most of the coding information required for restoring the current block is omitted, and the current block is restored by only a part of the coding information. Therefore, the coding efficiency is increased by omitting the coding information required for restoring the current block.

[0273] According to the embodiment of FIG. 23, in the skip mode, an inter mode tool may be used. According to an embodiment, in the skip mode, an UMVE mode, an affine mode, or a merge mode may be used. According to an embodiment, cu_skip_flag[x0][y0] indicating whether the skip mode is applied to the current block is acquired. If the skip mode is applied to the current block (if(cu_skip_flag[x0][y0])), cu_skip_umve[x0][y0] indicating whether the inter mode tool of the current block is the UMVE mode is acquired. If the UMVE mode is applied to the current block (if(cu_skip_umve[x0][y0]), cu_mode=skip_UMVE), information related to the basic motion vector of the current picture and neighboring blocks included in the reference picture, and correction size information and correction direction information related to correction of the basic motion vector are acquired (mvp_idx_umve(x0, y0, identical_list0_list1 flag)). If the UMVE mode is not applied to the current block, the affine mode (cu_mode=skip_AFFINE) or the merge mode (cu_mode=SKIP) can be applied to the current block.

[0274] In the embodiment of Fig. 23, the order of application of the UMVE mode, the affine mode, and the merge mode is determined. However, a person skilled in the art could easily change the type of inter mode tool applied in the skip mode. Also, a person skilled in the art could easily change the order of determination of the inter mode tools applied in the skip mode.

[0275] 24 discloses a method for selecting an inter mode tool for a current block in direct mode when the slice type of the current slice including the current block is P type. In direct mode, a part of the coding information required for determining the motion vector of the current block is omitted. Therefore, the coding efficiency is increased by reducing the coding information required for determining the motion vector of the current block.

[0276] According to the embodiment of FIG. 24, when the slice type of the current slice is the P type, the UMVE mode and the affine mode may be used in the direct mode. According to an embodiment, cu_direct[x0][y0] indicating whether the direct mode is applied to the current block is obtained. If the direct mode is applied to the current block (if(cu_direct[x0][y0])), affine_dir[x0][y0] indicating whether the inter mode tool of the current block is the affine mode is obtained. If the affine mode is not applied to the current block (if(affine_dir[x0][y0])), it is determined that the UMVE mode is applied to the current block (cu_mode=DIR_UMVE). Then, if the UMVE mode is applied to the current block, information related to neighboring blocks including the base motion vector and the reference picture of the current picture, and correction size information and correction direction information related to correction of the base motion vector are obtained (mvp_idx_umve(x0, y0, identical_list0_list1 flag)).

[0277] In the embodiment of Fig. 24, the order of application of the affine mode and the UMVE mode is determined. However, a person skilled in the art could easily change the type of intermode tool applied in the direct mode. Also, a person skilled in the art could easily change the order of determination of the intermode tools applied in the direct mode.

[0278] FIG. 25 discloses a method for selecting an inter mode tool for a current block in direct mode when the slice type of the current slice including the current block is type B.

[0279] According to the embodiment of FIG. 25, when the slice type of the current slice is type B, the UMVE mode, the affine mode, and the merge mode may be used. According to an embodiment, cu_direct[x0][y0] indicating whether the direct mode is applied to the current block is obtained. If the direct mode is applied to the current block (if(cu_direct[x0][y0])), cu_direct_umve[x0][y0] indicating whether the inter mode tool of the current block is the UMVE mode is obtained. If the UMVE mode is applied to the current block (if(cu_direct_umve[x0][y0]), cu_mode=DIR_UMVE), information related to neighboring blocks including the base motion vector of the current picture and the reference picture, and correction size information and correction direction information related to correction of the base motion vector are obtained (mvp_idx_umve(x0, y0, identical_list0_list1 flag)). If the UMVE mode is not applied to the current block, the affine mode (cu_mode=DIR_AFFINE) or the merge mode (cu_mode=DIR) may be applied to the current block.

[0280] In the embodiment of Fig. 25, the order of application of the UMVE mode, the affine mode, and the merge mode is determined. However, a person skilled in the art could easily change the type of intermode tool applied in the direct mode. Also, a person skilled in the art could easily change the order of determination of the intermode tools applied in the direct mode.

[0281] FIG. 26 illustrates a flowchart of a decoding method according to the UMVE mode of the video decoding device 1700 of FIG.

[0282] In step 2610, it is determined whether the UMVE mode is allowed for the upper data unit including the current block. Whether the UMVE mode is allowed for the upper data unit can be determined according to a UMVE allowable flag obtained from the bitstream.

[0283] When the UMVE mode is allowed for the upper data unit, UMVE candidate number information indicating the number of UMVE candidates for the UMVE mode allowed for the upper data unit and UMVE candidate list configuration information indicating how to construct a motion vector candidate list referenced by multiple blocks of the upper data unit can be obtained from the bitstream.

[0284] Correction size candidate number information indicating the number of correction size candidates may be obtained from the bitstream, correction size range information related to a plurality of blocks of the upper data unit may be obtained, and correction size change information related to a plurality of blocks of the upper data unit may be obtained.

[0285] Correction direction candidate number information indicating the number of correction direction candidates may be acquired from the bitstream.

[0286] Prediction direction change permission information for a plurality of blocks of the higher level data unit may be obtained from the bitstream, and main prediction direction information indicating a main prediction direction of the higher level data unit may be obtained from the bitstream.

[0287] In step 2620, if the UMVE mode is allowed for the higher data unit, it is determined whether the UMVE mode is applied to the current block. Whether the UMVE mode is applied to the current block may be determined according to a UMVE flag obtained from the bitstream. If the UMVE mode is not applied to the current block, another inter prediction mode other than the UMVE mode may be applied to the current block.

[0288] The number of UMVE candidates of the current block indicated by the UMVE candidate number information may be determined. In addition, a method for constructing a UMVE candidate list may be determined by the UMVE candidate list configuration information. A UMVE candidate list of the current block may be constructed according to the number of UMVE candidates and the method for constructing the UMVE candidate list.

[0289] In addition, the number of correction size candidates for the current block indicated by the correction size candidate number information may be determined. Correction size candidates may be determined based on a correction size range indicated by the correction size range information. In addition, when the correction size change information indicates that the correction size of the base motion vector is adaptively changed according to a distance between the current picture of the current block and the reference picture, the correction size range may be changed according to a distance between the current picture of the current block and the reference picture.

[0290] In addition, the number of correction direction candidates for the current block indicated by the correction direction candidate number information can be determined.

[0291] When the UMVE mode is applied to the current block and the prediction direction change permission information indicates that a change in the prediction direction is permitted, prediction direction information can be obtained from the bitstream.

[0292] In step 2630, if the UMVE mode is applied to the current block, a base motion vector of the current block is determined. A UMVE candidate is determined from the determined UMVE candidate list. Then, a reference picture and a base motion vector can be extracted from a neighboring block of the current block indicated by the UMVE candidate.

[0293] In step 2640, a correction size and a correction direction for correcting the base motion vector are determined. A correction size of the base motion vector among correction size candidates may be determined according to correction size information acquired from the bitstream. A correction direction of the base motion vector among correction direction candidates may be determined according to correction direction information acquired from the bitstream.

[0294] The corrected basic motion vector is further corrected according to an additional correction size and an additional correction direction. According to an embodiment, when the correction size is equal to or greater than a threshold value, a further correction of the basic motion vector is performed. The additional correction size is also determined to be smaller than the correction size.

[0295] In step 2650, a motion vector of a current block is determined by correcting a base motion vector according to the correction size and the correction direction. Then, the motion vector of the current block and a reference picture may be changed according to a prediction direction indicated by the prediction direction information. If a primary prediction direction is determined for the higher-level data unit, the prediction direction information may indicate a prediction direction of the current block in the primary prediction direction and two auxiliary prediction directions. If a primary prediction direction is not determined for the higher-level data unit, the prediction direction information may indicate a prediction direction of the current block in the prediction direction of a UMVE candidate and the remaining two prediction directions.

[0296] In step 2660, the current block is reconstructed based on the motion vector of the current block.

[0297] The inter prediction method according to the UMVE mode performed in the video decoding device 1700 of FIG. 17 is also applied to the decoding method according to the UMVE mode of FIG.

[0298] FIG. 27 illustrates a block diagram of a video encoding device 2800 that performs inter prediction according to the UMVE mode.

[0299] The video encoding device 2700 includes an encoding parameter determination unit 2710, a motion vector determination unit 2720, an encoding information determination unit 2730, and an output unit 2740. In FIG. 27, the encoding parameter determination unit 2710, the motion vector determination unit 2720, the encoding information determination unit 2730, and the output unit 2740 are expressed as separate components, but according to an embodiment, the encoding parameter determination unit 2710, the motion vector determination unit 2720, the encoding information determination unit 2730, and the output unit 2740 may be combined into one component.

[0300] 27, the coding parameter determination unit 2710, the motion vector determination unit 2720, the coding information determination unit 2730, and the output unit 2740 are expressed as a structural unit located in one device, but the devices responsible for each function of the coding parameter determination unit 2710, the motion vector determination unit 2720, the coding information determination unit 2730, and the output unit 2740 do not necessarily need to be physically adjacent to each other. Therefore, depending on the embodiment, the coding parameter determination unit 2710, the motion vector determination unit 2720, the coding information determination unit 2730, and the output unit 2740 may be distributed.

[0301] The coding parameter determination unit 2710, the motion vector determination unit 2720, the coding information determination unit 2730, and the output unit 2740 may be implemented by one processor or by a plurality of processors according to an embodiment.

[0302] The encoding parameter determination unit 2710 determines whether the UMVE mode is allowed for the upper data unit including the current block. The encoding parameter determination unit 2710 may determine whether the UMVE mode is allowed depending on the frequency with which the UMVE mode is used in the upper data unit. Alternatively, the encoding parameter determination unit 2710 may determine whether the UMVE mode is allowed depending on the characteristics of the image.

[0303] The motion vector determination unit 2720 determines a motion vector of a current block. The motion vector determination unit 2720 may obtain a plurality of motion vectors from neighboring blocks of the current block. The motion vector determination unit 2720 may then determine the motion vector of the current block by scanning the neighboring blocks of the plurality of motion vectors. The motion vector determination unit 2720 may determine an optimal motion vector as the motion vector of the current block through rate-distortion optimization.

[0304] When the UMVE mode is allowed for the higher data unit, the coding information decision unit 2730 determines whether the UMVE mode is applied to the current block according to the motion vector. The coding information decision unit 2730 may determine whether the motion vector of the current block determined by the motion vector decision unit 2720 is derived in the UMVE mode. If the motion vector of the current block is derived in the UMVE mode, the coding information decision unit 2730 may decide to apply the UMVE mode to the current block. Alternatively, if the minimum error between the motion vector of the current block and the motion vector according to the UMVE mode is equal to or less than a threshold value, the coding information decision unit 2730 may decide to apply the UMVE mode to the current block.

[0305] When the UMVE mode is applied to the current block, the coding information determination unit 2730 determines a base motion vector of the current block and a correction size and correction direction for correcting the base motion vector according to the motion vector of the current block. The coding information determination unit 2730 may determine the correction size and correction direction according to a difference between the motion vector of the current block and the base motion vector of the current block.

[0306] The output unit 2740 outputs a bitstream including coding information according to the UMVE mode of the current block. The coding information may include a UMVE permission flag, a UMVE flag, UMVE candidate number information, correction size candidate number information, correction direction candidate number information, UMVE candidate list configuration information, correction size range information, correction size change information, prediction direction change permission information, main prediction direction information, multiple correction permission information, etc.

[0307] The functions of the coding parameter determination unit 2710, the motion vector determination unit 2720, and the coding information determination unit 2730 are also performed in the coding unit 110 of Fig. 1A. The function of the output unit 2740 is also performed in the bitstream generation unit 120 of Fig. 1A.

[0308] FIG. 28 illustrates a flowchart of an encoding method according to the UMVE mode of the video encoding device 2800 of FIG.

[0309] In step 2810, it is determined whether the UMVE mode is allowed for the upper data unit including the current block. Whether the UMVE mode is allowed may be determined based on the frequency with which the UMVE mode is used in the upper data unit. Alternatively, whether the UMVE mode is allowed may be determined based on image characteristics.

[0310] In operation 2820, a motion vector for a current block is determined. A plurality of motion vectors may be obtained from neighboring blocks of the current block. Then, the motion vector for the current block may be determined by scanning the neighboring motion vectors. An optimal motion vector may be determined as the motion vector for the current block through rate-distortion optimization.

[0311] In step 2830, if the UMVE mode is allowed for the upper data unit, it is determined whether the UMVE mode is applied to the current block according to the motion vector. It can be determined whether the motion vector of the current block can be derived in the UMVE mode. If the motion vector of the current block can be derived in the UMVE mode, it is also determined that the UMVE mode is applied to the current block. Alternatively, if the minimum error between the motion vector of the current block and the motion vector according to the UMVE mode is less than a threshold value, it is also determined that the UMVE mode is applied to the current block.

[0312] In step 2840, a base motion vector of the current block and a correction size and correction direction for correcting the base motion vector are determined according to the motion vector. The correction size and correction direction can be determined according to the difference between the motion vector of the current block and the base motion vector of the current block.

[0313] In step 2850, a bitstream including information related to the UMVE mode of the current block is output.

[0314] The inter prediction method according to the UMVE mode performed in the video encoding device 2700 of FIG. 27 is also applied to the encoding method according to the UMVE mode of FIG.

[0315] According to the video encoding technique based on the coding unit of the tree structure described with reference to Figures 1 to 28, the video data of the spatial domain is encoded for each coding unit of the tree structure, and according to the video decoding technique based on the coding unit of the tree structure, the video data of the spatial domain is restored while decoding is performed for each maximum coding unit, and a video, which is a picture and a picture sequence, may be restored. The restored video may be played back by a playback device, stored in a recording medium, or transmitted via a network.

[0316] Meanwhile, the above-described embodiments of the present disclosure can be written into a program that can be executed by a computer, and can also be embodied in a general-purpose digital computer that runs the program using a computer-readable recording medium.

[0317] Although the present disclosure has been described in relation to certain best embodiments, other alternatives, modifications, and alterations to the present disclosure will be apparent to those skilled in the art in light of the foregoing description. That is, the claims are intended to include all such alternatives, modifications, and alterations. Accordingly, all contents described in the specification and drawings are intended to be illustrative and not limiting.

[0318] The means taught by the present application are listed below as examples.

[0319] (Appendix 1) determining whether an ultimate motion vector expression (UMVE) mode is allowed for an upper data unit including a current block; determining whether a UMVE mode is applied to the current block if the UMVE mode is allowed for the upper data unit; determining a base motion vector of the current block when a UMVE mode is applied to the current block; determining a correction size and a correction direction for correcting the basic motion vector; determining a motion vector of the current block by correcting the base motion vector according to the correction size and the correction direction; reconstructing the current block based on the motion vector of the current block. (Appendix 2) The video decoding method according to claim 1, wherein the higher level data unit is one of a sequence, a picture, and a slice. (Appendix 3) The step of determining whether the UMVE mode is permitted comprises: obtaining a UMVE enable flag from the bitstream; determining whether a UMVE mode is permitted for the upper data unit according to the UMVE permission flag; The step of determining whether a UMVE mode is applied to the current block includes: obtaining an UMVE flag indicating whether an UMVE mode is applied to the current block from the bitstream; determining whether a UMVE mode is applied to the current block according to the UMVE flag; The step of determining a base motion vector of the current block comprises: obtaining a UMVE index indicating a UMVE candidate including a base motion vector and a reference picture of the current block from the bitstream; determining a base motion vector and a reference picture of the current block according to the UMVE index; The step of determining the correction size and the correction direction comprises: obtaining correction size information indicating a correction size of the base motion vector and correction direction information indicating a correction direction of the base motion vector from the bitstream; and determining a correction size and a correction direction of the base motion vector according to the correction size information and the correction direction information. (Appendix 4) The step of determining whether the UMVE mode is applied includes: The video decoding method of claim 1, further comprising the step of applying a first inter prediction mode other than the UMVE mode to the current block if the UMVE mode is not applied to the current block. (Appendix 5) The video decoding method includes: When the UMVE mode is allowed for the upper data unit, acquiring UMVE candidate number information indicating the number of UMVE candidates of the UMVE mode allowed for the upper data unit, correction size candidate number information indicating the number of correction size candidates, and correction direction candidate number information indicating the number of correction direction candidates from a bitstream; When the UMVE mode is applied to the current block, determining reference picture candidates, the correction size candidates, and the correction direction candidates of the current block based on a number of UMVE candidates indicated by the UMVE candidate number information, a number of correction size candidates indicated by the correction size candidate number information, and a number of correction direction candidates indicated by the correction direction candidate number information, The step of determining the base motion vector comprises: determining a base motion vector of the current block based on the base motion vector selected from the UMVE candidates; The step of determining the correction size and the correction direction comprises: The video decoding method according to claim 1, further comprising: determining a correction size from the correction size candidates; and determining a correction direction from the correction direction candidates. (Appendix 6) The video decoding method includes: If the UMVE mode is allowed for the high-level data unit, the method further includes obtaining UMVE candidate list configuration information from a bitstream, the UMVE candidate list configuration information indicating a method of configuring a motion vector candidate list to which a plurality of blocks of the high-level data unit refer; The step of determining the base motion vector comprises: constructing a UMVE candidate list for the current block according to the UMVE candidate list configuration information; and determining a base motion vector and a reference picture for a current block from the UMVE candidate list. (Appendix 7) The UMVE candidate list configuration information is The video decoding method according to claim 6, characterized in that it shows a first list determination method for determining a UMVE candidate list using a merged motion vector candidate list, or a second list determination method for determining a UMVE candidate list based on a frequently used motion vector in a current picture. (Appendix 8) The video decoding method includes: If the UMVE mode is allowed for the upper data unit, the method further includes obtaining correction size range information for a plurality of blocks of the upper data unit from a bitstream; The step of determining the correction size and the correction direction comprises: 2. The video decoding method of claim 1, further comprising a step of determining a correction size of the base motion vector based on a correction size range indicated by the correction size range information. (Appendix 9) The video decoding method includes: If the UMVE mode is allowed for the upper data unit, the method further includes obtaining correction size change information for a plurality of blocks of the upper data unit from a bitstream; The step of determining the correction size and the correction direction comprises: The video decoding method described in Appendix 1, characterized in that when the correction size change information indicates that the correction size of the base motion vector is adaptively changed depending on the distance between the current picture of the current block and the reference picture, the method includes a step of changing the correction size of the base motion vector depending on the distance between the current picture of the current block and the reference picture. (Appendix 10) The video decoding method includes: obtaining prediction direction change permission information for a plurality of blocks of the upper data unit from a bitstream if the upper data unit is allowed to use the UMVE mode; When a UMVE mode is applied to the current block and the prediction direction change permission information indicates that a change in the prediction direction is permitted, obtaining prediction direction information from the bitstream; The step of determining a motion vector comprises: 2. The video decoding method of claim 1, further comprising a step of changing a motion vector of the current block according to a prediction direction indicated by the prediction direction information. (Appendix 11) The video decoding method includes: determining a primary prediction direction of the UMVE mode when the UMVE mode is allowed for the upper data unit; The step of determining a motion vector comprises: determining the motion vector based on the primary prediction direction; When the prediction direction change information indicates that the prediction direction is changed, changing the base motion vector according to an auxiliary prediction direction other than the primary prediction direction; The step of determining the correction size and the correction direction comprises: determining the correction size and the correction direction based on the primary prediction direction; and when the prediction direction change information indicates that the prediction direction is changed, changing the correction size and the correction direction by a preliminary prediction direction other than the primary prediction direction. (Appendix 12) The video decoding method includes: When the correction size is equal to or greater than a threshold value, determining an additional correction size and an additional correction direction for further correcting the base motion vector, The motion vector determining step of the current block includes: determining a motion vector of the current block according to the correction size, the correction direction, and the additional correction size and additional correction direction; 2. The video decoding method of claim 1, wherein the additional correction size is smaller than the correction size. (Appendix 13) Determine whether the ultimate motion vector expression (UMVE) mode is allowed for the upper data unit including the current block, If the UMVE mode is allowed for the upper data unit, determining whether the UMVE mode is applied to the current block; If the UMVE mode is applied to the current block, a base motion vector of the current block is determined, and a correction size and a correction direction for correcting the base motion vector are determined; correcting the base motion vector according to the correction size and the correction direction to determine a motion vector of the current block; A video decoding apparatus comprising: a processor for reconstructing the current block based on a motion vector of the current block. (Appendix 14) determining whether an ultimate motion vector expression (UMVE) mode is allowed for an upper data unit including a current block; determining a motion vector for the current block; determining whether a UMVE mode is applied to the current block according to the motion vector when the UMVE mode is allowed for the upper data unit; determining a base motion vector of the current block and a correction size and a correction direction for correcting the base motion vector according to the motion vector; and outputting a bitstream including information related to the UMVE mode of the current block. (Appendix 15) Determine whether the ultimate motion vector expression (UMVE) mode is allowed for the upper data unit including the current block, determining a motion vector for the current block; If the upper data unit is allowed to use the UMVE mode, it is determined whether the UMVE mode is applied to the current block according to the motion vector; determining a base motion vector of the current block and a correction size and a correction direction for correcting the base motion vector according to the motion vector; A video encoding apparatus comprising: a processor that outputs a bitstream including encoding information of the current block according to the UMVE mode.

Claims

1. obtaining first MV prediction mode allowance information indicating whether a first MV prediction mode for predicting a motion vector is allowed in the upper data unit from the bitstream; obtaining correction size range information indicating one of a plurality of correction size candidate sets from the bitstream when the first MV prediction mode is permitted for the higher data unit based on the first MV prediction mode permit information; determining whether the first MV prediction mode is applied to a current block included in the upper data unit when the first MV prediction mode is allowed in the upper data unit; obtaining correction size information and correction direction information from the bitstream when the first MV prediction mode is applied to the current block; determining a correction size using the correction size range information and the correction size information; and correcting a base motion vector according to the correction size and the correction direction information to determine a motion vector of the current block, Among the plurality of correction size candidate sets, one set includes a plurality of candidates, One correction size candidate set is selected from the plurality of correction size candidate sets according to the correction size range information; The correction size is determined to be one of the correction size candidates included in the selected correction size candidate set that is indicated by the correction size information; A video decoding method, characterized in that, among the multiple correction size candidate sets, a first correction size candidate set includes the correction size candidate {1,2}, and a second correction size candidate set does not include the correction size candidate {1,2}.

2. Acquire first MV prediction mode allowance information indicating whether a first MV prediction mode for predicting a motion vector is allowed in a higher data unit from the bitstream; When the first MV prediction mode is permitted for the higher data unit based on the first MV prediction mode permit information, correction size range information indicating one of a plurality of correction size candidate sets is obtained from the bitstream; determining whether the first MV prediction mode is applied to a current block included in the upper data unit when the first MV prediction mode is allowed in the upper data unit; When the first MV prediction mode is applied to the current block, correction size information and correction direction information are obtained from the bitstream; determining a correction size using the correction size range information and the correction size information; a processor for performing an operation of correcting a base motion vector according to the correction size and the correction direction information and determining a motion vector of the current block; one of the plurality of correction size candidate sets includes a plurality of candidates; One correction size candidate set is selected from the plurality of correction size candidate sets according to the correction size range information; The correction size is determined to be one of the correction size candidates included in the selected correction size candidate set that is indicated by the correction size information; A video decoding device characterized in that, among the multiple correction size candidate sets, a first correction size candidate set includes the correction size candidate {1,2}, and a second correction size candidate set does not include the correction size candidate {1,2}.

3. generating first MV prediction mode allowance information indicating whether a first MV prediction mode for predicting a motion vector is allowed in the higher data unit; generating correction size range information indicating one of a plurality of correction size candidate sets when the first MV prediction mode is permitted in the higher level data unit; determining whether the first MV prediction mode is applied to a current block included in the upper data unit when the first MV prediction mode is allowed in the upper data unit; determining a correction size and a correction direction based on a difference between a motion vector of the current block and a base motion vector when the first MV prediction mode is applied to the current block; generating correction size information for indicating the correction size selected within a correction size range; generating correction direction information for indicating the correction direction; and outputting a bitstream including the correction size range information, the correction size information, and the correction direction information, Among the plurality of correction size candidate sets, one set includes a plurality of candidates, the correction size range information indicates one correction size candidate set among the plurality of correction size candidate sets, The correction size information indicates the correction size among the correction size candidates included in the correction size candidate set, A video encoding method, characterized in that, among the multiple correction size candidate sets, a first correction size candidate set includes the correction size candidate {1,2}, and a second correction size candidate set does not include the correction size candidate {1,2}.

4. 1. A method for transmitting a bitstream generated by encoding video, comprising: generating first MV prediction mode allowance information indicating whether a first MV prediction mode for predicting a motion vector is allowed in the higher data unit; generating correction size range information indicating one of a plurality of correction size candidate sets when the first MV prediction mode is permitted for the higher level data unit; determining whether the first MV prediction mode is applied to a current block included in the upper data unit when the first MV prediction mode is allowed in the upper data unit; determining a correction size and a correction direction based on a difference between a motion vector of the current block and a base motion vector when the first MV prediction mode is applied to the current block; generating correction size information for indicating the correction size selected within a correction size range; generating correction direction information for indicating the correction direction; outputting the bitstream including the first MV prediction mode allowance information, the correction size range information, the correction size information, and the correction direction information; Among the plurality of correction size candidate sets, one set includes a plurality of candidates, the correction size range information indicates one correction size candidate set among the plurality of correction size candidate sets, The correction size information indicates the correction size among the correction size candidates included in the correction size candidate set, A method according to claim 1, wherein, among the plurality of correction size candidate sets, a first correction size candidate set includes the correction size candidate {1,2}, and a second correction size candidate set does not include the correction size candidate {1,2}.

Images