Coding method and device thereof, and decoding method and device thereof

Abstract

A video decoding method is provided, comprising the steps of: determining whether a UMVE mode is allowed for an upper layer data unit including a current block; if the UMVE mode is allowed for the upper layer data unit, determining whether the UMVE mode is applied to the current block; if the UMVE mode is applied to the current block, determining a basic motion vector of the current block; determining a correction size and a correction direction for correcting the basic motion vector; determining a motion vector of the current block by correcting the basic motion vector according to the correction size and the correction direction; and reconstructing the current block based on the motion vector of the current block.

Description

[0001] This application is a divisional application of the invention patent application with the application date of July 13, 2018, application number "201880090461.9", and title "Encoding method and device thereof and decoding method and device thereof". Technical Field

[0002] The present disclosure relates to a video encoding method and a video decoding method, and more particularly, to a method for efficiently encoding and decoding information about a motion vector. Background Art

[0003] When encoding high-quality video, a large amount of data is required. However, because the bandwidth allowed to send video data is limited, the data rate applied to send video data may be limited. Therefore, in order to effectively send video data, video data encoding and decoding methods with minimal image quality degradation and increased compression rate are required.

[0004] Video data can be compressed by removing spatial and temporal redundancy between pixels. Since adjacent pixels generally have common characteristics, encoding information is transmitted as a data unit consisting of pixels in order to remove redundancy between adjacent pixels.

[0005] Instead of directly transmitting the pixel values ​​of the pixels included in the data unit, information about the method of obtaining the pixel values ​​is transmitted. A prediction method for predicting a pixel value similar to the original value is determined for each data unit, and encoding information about the prediction method is transmitted from the encoder to the decoder. Because the predicted value is not exactly the same as the original value, residual data of the difference between the original value and the predicted value is transmitted from the encoder to the decoder.

[0006] As the accuracy of prediction increases, the coding information required to specify the prediction method increases, but the size of the residual data decreases. Therefore, the prediction method is determined in consideration of the size of the residual data and the coding information. In particular, the data units divided from the picture have various sizes, and as the size of the data unit increases, the possibility of reduced prediction accuracy increases, while the coding information decreases. Therefore, the size of the block is determined according to the characteristics of the picture.

[0007] In addition, prediction methods include intra prediction and inter prediction. Intra prediction is a method of predicting pixels of a block from neighboring pixels around the block. Inter prediction is a method of predicting pixels by referring to pixels of another picture referenced by a picture including the block. Therefore, spatial redundancy is removed in intra prediction, and temporal redundancy is removed in inter prediction.

[0008] As the number of prediction methods increases, the amount of encoding information for indicating the prediction method increases. Therefore, when encoding information applied to a block is predicted from another block, the size of the encoding information can be reduced.

[0009] Since the loss of video data is allowed to the extent that the human eye cannot recognize the loss, the amount of residual data can be reduced by lossily compressing the residual data according to the transform and quantization processes. Summary of the invention

[0010] Technical issues

[0011] A video encoding method and a video encoding device for performing inter-frame prediction according to a final motion vector expression (UMVE) mode are provided. A video decoding method and a video decoding device for performing inter-frame prediction according to a UMVE mode are provided. A computer-readable recording medium having a program for executing the video encoding method and the video decoding method according to an embodiment of the present disclosure on a computer is also provided.

[0012] Problem Solution

[0013] The present disclosure provides a video decoding method, including: determining whether a final motion vector expression (UMVE) mode is allowed for an upper layer data unit including a current block; when the UMVE mode is allowed for the upper layer data unit, determining whether the UMVE mode is applied to the current block; when the UMVE mode is applied to the current block, determining a basic motion vector of the current block; determining a correction distance and a correction direction for correcting the basic motion vector; determining a motion vector of the current block by correcting the basic motion vector according to the correction distance and the correction direction; and reconstructing the current block based on the motion vector of the current block.

[0014] The present disclosure provides a video decoding apparatus including a processor, wherein the processor is configured to perform the following operations: determine whether a final motion vector expression (UMVE) mode is allowed for an upper layer data unit including a current block; when the UMVE mode is allowed for the upper layer data unit, determine whether the UMVE mode is applied to the current block; when the UMVE mode is applied to the current block, determine a basic motion vector of the current block and determine a correction distance and a correction direction for correcting the basic motion vector; determine a motion vector of the current block by correcting the basic motion vector according to the correction distance and the correction direction; and reconstruct the current block based on the motion vector of the current block.

[0015] The present disclosure provides a video encoding method, including: determining whether a final motion vector expression (UMVE) mode is allowed for an upper layer data unit including a current block; determining a motion vector of the current block; when the UMVE mode is allowed for the upper layer data unit, determining whether the UMVE mode is applied to the current block according to the motion vector; determining a basic motion vector of the current block and a correction distance and a correction direction for correcting the basic motion vector according to the motion vector; and outputting a bitstream including information about the UMVE mode of the current block.

[0016] The present disclosure provides a video encoding apparatus including a processor, wherein the processor is configured to perform the following operations: determine whether a final motion vector expression (UMVE) mode is allowed for an upper layer data unit including a current block; determine a motion vector of the current block; when the UMVE mode is allowed for the upper layer data unit, determine whether the UMVE mode is applied to the current block according to the motion vector; determine a basic motion vector of the current block and a correction distance and a correction direction for correcting the basic motion vector according to the motion vector; and output a bitstream including encoding information according to the UMVE mode of the current block.

[0017] The present disclosure provides a computer-readable recording medium having recorded thereon a program for executing the video encoding method and the video decoding method.

[0018] The technical problems of the present embodiment are not limited to the aforementioned technical problems, and other unstated technical problems can be inferred from the following embodiments.

[0019] Advantageous Effects of the Present Disclosure

[0020] Since blocks included in an image are inter-predicted according to a final motion vector expression (UMVE) mode, the encoding rate of the image can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1A is a block diagram of an image encoding apparatus based on coding units having a tree structure according to an embodiment of the present disclosure.

[0022] Figure 1B is a block diagram of an image decoding apparatus based on coding units having a tree structure according to an embodiment.

[0023] Figure 2 A process of determining at least one coding unit by splitting a current coding unit according to an embodiment is illustrated.

[0024] Figure 3 A process of determining at least one coding unit by dividing non-square coding units according to an embodiment is illustrated.

[0025] Figure 4 A process of splitting a coding unit based on at least one of block shape information and partition shape information according to an embodiment is illustrated.

[0026] Figure 5 A method of determining a specific coding unit from among an odd number of coding units according to an embodiment is illustrated.

[0027] Figure 6 An order in which a plurality of coding units are processed when a plurality of coding units are determined by splitting a current coding unit according to an embodiment is illustrated.

[0028] Figure 7 A process of determining that a current coding unit is to be split into an odd number of coding units when coding units may not be processed in a specific order according to an embodiment is illustrated.

[0029] Figure 8 A process of determining at least one coding unit by splitting a first coding unit according to an embodiment is illustrated.

[0030] Fig. 9 It is shown that shapes into which the second coding unit may be divided are restricted when a non-square second coding unit determined by splitting the first coding unit satisfies a specific condition according to an embodiment.

[0031] Fig.10 A process of splitting a square coding unit when division shape information indicates that the square coding unit is not to be split into four square coding units according to an embodiment is illustrated.

[0032] Fig.11 It is shown that a processing order among a plurality of coding units may be changed depending on a process of dividing the coding units according to an embodiment.

[0033] Fig.12 A process of determining a depth of a coding unit as a shape and a size of a coding unit change when a coding unit is recursively split so that a plurality of coding units are determined according to an embodiment is illustrated.

[0034] Fig.13 Depth that may be determined based on a shape and a size of a coding unit and a partial index (PID) for distinguishing coding units according to an embodiment are illustrated.

[0035] Fig.14 It is illustrated that a plurality of coding units are determined based on a plurality of specific data units included in a picture according to an embodiment.

[0036] Fig.15 A processing block serving as a unit for determining an order of reference coding units included in a picture according to an embodiment is illustrated.

[0037] Fig.16 is a diagram for describing an inter prediction method according to a final motion vector expression (UMVE) mode.

[0038] Fig.17 is a block diagram of a video decoding apparatus for performing decoding according to a UMVE mode.

[0039] Fig.18 Motion vector candidates of a UMVE pattern with a diamond-shaped distribution based on a basic motion vector are shown.

[0040] Fig.19 Motion vector candidates based on a basic motion vector in a UMVE pattern with a rectangular distribution are shown.

[0041] Fig. 20 is a diagram illustrating a UMVE mode having a different number of motion vector candidates for each group according to an embodiment.

[0042] Fig.21 is a diagram showing a UMVE pattern of motion vector candidates having different distribution shapes for each group.

[0043] Fig. 22 A method of correcting two basic motion vectors according to bidirectional prediction according to an embodiment is shown.

[0044] Fig.23 A method of selecting an inter-mode tool of a current block in skip mode is shown.

[0045] Fig.24 A method of selecting an inter mode tool of a current block in a direct mode when a slice type of a current slice including the current block is a P type is shown.

[0046] Fig.25 A method of selecting an inter mode tool of a current block in a direct mode when a slice type of a current slice including the current block is type B is shown.

[0047] Fig.26 is a flowchart of a decoding method according to the UMVE mode.

[0048] Fig. 27 is a block diagram of a video encoding apparatus for performing inter-frame prediction according to a UMVE mode.

[0049] Fig.28 A flow chart showing an encoding method according to the UMVE mode.

[0050] Best Mode

[0051] A video decoding method is provided, comprising: determining whether a final motion vector expression (UMVE) mode is allowed for an upper layer data unit including a current block; when the UMVE mode is allowed for the upper layer data unit, determining whether the UMVE mode is applied to the current block; when the UMVE mode is applied to the current block, determining a basic motion vector of the current block; determining a correction distance and a correction direction for correcting the basic motion vector; determining a motion vector of the current block by correcting the basic motion vector according to the correction distance and the correction direction; and reconstructing the current block based on the motion vector of the current block. DETAILED DESCRIPTION

[0052] Public Mode

[0053] The advantages and features of the present disclosure and methods of achieving the same will be described more fully with reference to the accompanying drawings, in which embodiments of the present invention are shown. However, the present disclosure may be implemented in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that the present disclosure will be thorough and complete and will fully convey the concepts of the present disclosure to those of ordinary skill in the art.

[0054] The terms used herein will be briefly described, and the disclosed embodiments will be described in detail.

[0055] The terms used herein are those common terms currently widely used in the art in view of the functions in the present disclosure, but the terms may vary according to the intention of a person of ordinary skill in the art, precedents, or new technologies in the art. In addition, some of the terms used herein may be arbitrarily selected by the applicant, and in such cases, these terms are defined in detail below. Therefore, specific terms used herein should be defined based on their unique meanings and the entire context of the present disclosure.

[0056] It is to be understood that singular forms include plural referents unless the context clearly dictates otherwise.

[0057] It should be understood that, unless the context clearly specifies otherwise, when a specific part "includes" a specific component, the part does not exclude another component, but may further include another component. The term "~ unit" used herein refers to a software component or a hardware component (such as a field programmable gate array (FPGA) or an application-specific integrated circuit (ASIC)) that performs a specific task. However, the term "~ unit" is not limited to software or hardware. "~ unit" can be configured to be in an addressable storage medium or configured to operate one or more processors. Therefore, as an example, "~ unit" may include components such as software components, object-oriented software components, class components and task components, processes, functions, attributes, procedures, subroutines, program code segments, drivers, firmware, microcodes, circuits, data, databases, data structures, tables, arrays and variables. The functions provided by components and "~ units" can be combined into fewer components and "~ units" or further separated into other components and "~ units".

[0058] The term "current block" refers to one of a coding unit, a prediction unit, and a transformation unit that is currently encoded or decoded. For ease of explanation, when other types of blocks such as a prediction unit and a transformation unit need to be distinguished from each other, "current coding unit", "current prediction unit", and "current transformation unit" may be used. In addition, the term "lower layer block" refers to a data unit divided from a "current block". The term "upper layer block" refers to a data unit including a "current block".

[0059] In addition, the term "sample" used herein refers to data assigned to a sampling position of an image and to be processed. For example, a pixel value in an image in a spatial domain or a transform coefficient in a transform domain may be a sample. A unit including at least one sample may be defined as a block.

[0060] The present disclosure will now be described more fully with reference to the accompanying drawings so that those skilled in the art can perform the present disclosure without any difficulty. In addition, in order to clearly describe the present disclosure, parts irrelevant to the description of the present disclosure will be omitted in the accompanying drawings.

[0061] Figure 1A is a block diagram of an image encoding apparatus 100 based on coding units having a tree structure according to an embodiment of the present disclosure.

[0062] The image encoding apparatus 100 includes an encoder 110 and a bitstream generator 120 .

[0063] The encoder 110 divides a picture or a slice included in the picture into a plurality of maximum coding units according to the size of the maximum coding unit. The maximum coding unit may be a data unit having a size of 32×32, 64×64, 128×128, 256×256, etc., wherein the shape of the data unit is a square whose width and length are powers of 2. The encoder 110 may provide maximum coding unit size information indicating the size of the maximum coding unit to the bitstream generator 120. The bitstream generator 120 may cause the maximum coding unit size information to be included in the bitstream.

[0064] The encoder 110 determines the coding unit by dividing the maximum coding unit. Whether to divide the coding unit is determined according to whether it is effective to divide the coding unit based on rate-distortion optimization. Then, division information indicating whether the coding unit is divided may be generated. The division information may be indicated by using a flag.

[0065] The coding unit may be divided by using various methods. For example, a square coding unit may be divided into four square coding units whose width and height are half of the width and height of the square coding unit. A square coding unit may be divided into two rectangular coding units whose width is half of the width of the square coding unit. A square coding unit may be divided into two rectangular coding units whose height is half of the height of the square coding unit. A square coding unit may be divided into three coding units by dividing the width or height according to 1:2:1.

[0066] A rectangular coding unit whose width is twice the height may be divided into two square coding units. A rectangular coding unit whose width is twice the height may be divided into two rectangular coding units whose width is four times the height. A rectangular coding unit whose width is twice the height may be divided into two rectangular coding units and one square coding unit by dividing the width in 1:2:1.

[0067] Likewise, a rectangular coding unit whose height is twice as large as its width may be divided into two square coding units. In addition, a rectangular coding unit whose height is twice as large as its width may be divided into two rectangular coding units whose height is four times as large as its width. Likewise, a rectangular coding unit whose height is twice as large as its width may be divided into two rectangular coding units and one square coding unit by dividing the height in 1:2:1.

[0068] When the image encoding device 100 can use two or more division methods, information about a division method available for a coding unit among the division methods that can be used by the image encoding device 100 may be determined for each picture. Therefore, only a specific division method may be specifically determined to be used for each picture. When the image encoding device 100 uses only one division method, information about a division method available for a coding unit is not separately determined.

[0069] A coding unit having a specific size may be divided by using a specific division method. For example, when the size of the coding unit is 256×265, the coding unit may be set to be divided into only four square units whose width and height are half of the width and height of the coding unit.

[0070] When the split information of the coding unit indicates that the coding unit is split, the split shape information indicating the split method of the coding unit may be generated. When there is only one split method that can be used in the picture to which the coding unit belongs, the split shape information may not be generated. When the split method is adaptively determined with respect to the encoding information around the coding unit, the split shape information may not be generated.

[0071] As described above, the image data of the current picture is divided into maximum coding units according to the maximum size of the coding unit. Each of the maximum coding units may include coding units hierarchically divided from the maximum coding unit. The shape and position of the lower layer coding unit may be determined according to the division shape of the upper layer coding unit. The minimum size of the coding unit that limits the division of the coding unit may be preset.

[0072] The encoder 110 compares the encoding efficiency when the coding unit is hierarchically divided with the encoding efficiency when the coding unit is not divided. Then, the encoder 110 determines whether to divide the coding unit according to the comparison result. When it is determined that it is more effective to divide the coding unit, the encoder 110 divides the coding unit hierarchically. When it is determined that it is effective not to divide the coding unit according to the comparison result, the encoder 110 does not divide the coding unit. Whether to divide the coding unit can be determined regardless of whether the adjacent coding unit is divided.

[0073] The final divided coding unit may be predicted by using intra prediction or inter prediction. Intra prediction is a method of predicting samples of a prediction unit by using reference samples around the prediction unit. Inter prediction is a method of predicting samples of a prediction unit by obtaining reference samples from a reference picture referenced by a current picture.

[0074] For intra prediction, the encoder 110 may select the most efficient intra prediction method by applying a plurality of intra prediction methods to a prediction unit. The intra prediction methods include a DC mode, a planar mode, and a directional mode such as a vertical mode or a horizontal mode.

[0075] When reconstructed samples around a coding unit are used as reference samples, intra prediction may be performed for each prediction unit. However, when reconstructed samples in a coding unit are used as reference samples, reconstruction of the reference samples in the coding unit must precede prediction, and thus the prediction order of the prediction unit may depend on the transformation order of the transformation unit. Therefore, when reconstructed samples in a coding unit are used as reference samples, only the intra prediction method for the transformation unit corresponding to the prediction unit may be determined, and actual intra prediction may be performed for each transformation unit.

[0076] The encoder 110 may select the most effective inter-frame prediction method by determining the best motion vector and the best reference picture. For inter-frame prediction, the encoder 110 may determine a plurality of motion vector candidates from coding units that are spatially and temporally adjacent to the current coding unit, and may determine the most effective motion vector from the motion vector candidates as the motion vector. Similarly, the encoder 110 may determine a plurality of reference picture candidates from coding units that are spatially and temporally adjacent to the current coding unit, and may determine the most effective reference picture from the reference picture candidates. According to an embodiment, the reference picture may be determined based on a reference picture list predetermined for the current picture. According to this embodiment, for the accuracy of prediction, the most effective motion vector among a plurality of motion vector candidates may be determined as a predicted motion vector, and the motion vector may be determined by correcting the predicted motion vector. Inter-frame prediction may be performed in parallel for each prediction unit in the coding unit.

[0077] The encoder 110 may reconstruct the coding unit by obtaining only information indicating a motion vector and a reference picture according to the skip mode. According to the skip mode, all encoding information including the residual signal is skipped except for the information indicating the motion vector and the reference picture. Since the residual signal is skipped, the skip mode may be used when the prediction accuracy is very high.

[0078] The partition mode to be used may be limited according to the prediction method for the prediction unit. For example, a partition mode only for a prediction unit having a size of 2N×2N or N×N may be applied to intra prediction, and a partition mode for a prediction unit having a size of 2N×2N, 2N×N, N×2N, or N×N may be applied to inter prediction. In addition, a partition mode only for a prediction unit having a size of 2N×2N may be applied to a skip mode of inter prediction. The partition mode for each prediction method in the image encoding device 100 may vary according to encoding efficiency.

[0079] The image encoding device 100 may perform transformation based on the coding unit. The image encoding device 100 may transform the residual data by a specific process, wherein the residual data is the difference between the original value and the predicted value for the pixel included in the coding unit. For example, the image encoding device 100 may perform lossy compression on the residual data by quantization and discrete cosine transform (DCT) / discrete sine transform (DST). Alternatively, the image encoding device 100 may perform lossless compression on the residual data without quantization.

[0080] In summary, the encoder 110 determines the most effective prediction method for the current prediction unit from a plurality of intra prediction methods and inter prediction methods. Then, the encoder 110 determines the prediction method of the current coding unit according to the coding efficiency based on the prediction result. Similarly, the encoder 110 determines the transformation method according to the coding efficiency based on the transformation result. The coding efficiency of the coding unit is finally determined according to the most effective coding unit prediction method and transformation method determination scheme. The encoder 110 finally determines the hierarchical structure of the maximum coding unit according to the coding efficiency of the finally divided coding unit.

[0081] The encoder 110 may measure the encoding efficiency of a coding unit, the prediction efficiency of a prediction method, etc. by using rate-distortion optimization based on Lagrange multipliers.

[0082] The encoder 110 may generate split information indicating whether to split the coding unit according to the determined hierarchical structure of the maximum coding unit. Then, the encoder 110 may generate partition mode information for determining the prediction unit and transform unit split information for determining the transform unit for the split coding unit. In addition, when the coding unit can be split by using at least two split methods, the encoder 110 may generate both split information indicating the split method and split shape information. The encoder 110 may generate information about the prediction method and the transform method for the prediction unit and the transform unit.

[0083] The bitstream generator 120 may output a plurality of pieces of information generated by the encoder 110 in a bitstream according to the hierarchical structure of the maximum coding unit.

[0084] The following will refer to Figures 3 to 12 A method of determining a coding unit, a prediction unit, and a transformation unit according to a tree structure of a maximum coding unit according to an embodiment is described.

[0085] Figure 1B is a block diagram of an image decoding apparatus 150 based on coding units having a tree structure according to an embodiment.

[0086] The image decoding device 150 includes a receiver 160 and a decoder 170 .

[0087] Definitions of terms including coding units, prediction units, transform units, various partition information, and the like used for a decoding operation performed by the image decoding device 150 are the same as those described above with reference to FIG. 1 and the image encoding device 100. In addition, since the image decoding device 150 is designed to reconstruct image data, various encoding methods used by the image encoding device 100 may be applied to the image decoding device 150.

[0088] The receiver 160 receives and parses a bitstream about the encoded video. The decoder 170 extracts a plurality of pieces of information for decoding the maximum coding unit from the parsed bitstream and provides the information to the decoder 170. The decoder 170 may extract information about the maximum size of the coding unit of the current picture from a header of the current picture, a sequence parameter set, or a picture parameter set.

[0089] The decoder 170 extracts split information of the coding unit having a tree structure from the parsed bitstream according to each maximum coding unit. The extracted split information is output to the decoder 170. The decoder 170 may split the maximum coding unit according to the extracted split information to determine the tree structure of the maximum coding unit.

[0090] The split information extracted by the decoder 170 is split information of a tree structure determined by the image encoding device 100 to generate a minimum encoding error. Therefore, the image decoding device 150 can reconstruct an image by decoding data according to a decoding method that generates a minimum encoding error.

[0091] The decoder 170 may extract partition information of a data unit (such as a prediction unit and a transformation unit) included in a coding unit. For example, the decoder 170 may extract information about the most effective partition mode of the prediction unit. The decoder 170 may extract transformation partition information of the most effective tree structure for the transformation unit.

[0092] Also, the decoder 170 may obtain information about the most efficient prediction method of the prediction unit split from the coding unit. Then, the decoder 170 may obtain information about the most efficient transformation method of the transformation unit split from the coding unit.

[0093] The decoder 170 extracts information from the bitstream according to the method in which the bitstream generator 120 of the image encoding apparatus 100 constructs the bitstream.

[0094] The decoder 170 may split the maximum coding unit into coding units having the most efficient tree structure based on the split information. Then, the decoder 170 may split the coding unit into prediction units according to the information about the partition mode. The decoder 170 may split the coding unit into transformation units according to the transformation split information.

[0095] The decoder 170 may predict the prediction unit according to the information about the prediction method. The decoder 170 may perform inverse quantization and inverse transformation on the residual data which is the difference between the original value and the prediction value of the pixel according to the information about the method of transforming the transform unit. The decoder 170 may reconstruct the pixels of the coding unit according to the prediction result of the prediction unit and the transformation result of the transform unit.

[0096] Figure 2 A process of determining at least one coding unit by splitting a current coding unit, performed by the image decoding device 150 according to an embodiment, is illustrated.

[0097] According to an embodiment, the image decoding device 150 may determine the shape of the coding unit by using the block shape information, and may determine the shape into which the coding unit will be divided according to the division shape information by using the division shape information. That is, the coding unit division method indicated by the division shape information may be determined according to which block shape is indicated by the block shape information used by the image decoding device 150.

[0098] According to an embodiment, the image decoding device 150 may use block shape information indicating that the current coding unit has a square shape. For example, the image decoding device 150 may determine whether to not divide the square coding unit, whether to divide the square coding unit vertically, whether to divide the square coding unit horizontally, or whether to divide the square coding unit into four coding units according to the division shape information. Figure 2 , when the block shape information of the current coding unit 200 indicates a square shape, the decoder 180 may not divide the coding unit 210a having the same size as the current coding unit 200 according to the division shape information indicating that division is not performed, or may determine the coding units 210b, 210c and 210d divided based on the division shape information indicating a specific division method.

[0099] Reference Figure 2 According to an embodiment, the image decoding device 150 may determine two coding units 210b obtained by vertically dividing the current coding unit 200 based on the division shape information indicating that the division is performed vertically. The image decoding device 150 may determine two coding units 210c obtained by horizontally dividing the current coding unit 200 based on the division shape information indicating that the division is performed horizontally. The image decoding device 150 may determine four coding units 210d obtained by vertically and horizontally dividing the current coding unit 200 based on the division shape information indicating that the division is performed vertically and horizontally. However, the division shape for dividing the coding unit of the square may not be limited to the above-mentioned shape, and may include various shapes that may be indicated by the division shape information. The division shape for dividing the coding unit of the square will be described in detail through various embodiments below.

[0100] Figure 3 A process of determining at least one coding unit by dividing non-square coding units, performed by the image decoding device 150 according to an embodiment, is illustrated.

[0101] According to an embodiment, the image decoding device 150 may use block shape information indicating that the current coding unit has a non-square shape. The image decoding device 150 may determine whether to not split the current non-square coding unit or to split the non-square current coding unit by using a specific method according to the split shape information. Figure 3 , when the block shape information of the current coding unit 300 or 350 indicates a non-square shape, the image decoding device 150 may not divide the coding unit 310 or 360 having the same size as the current coding unit 300 or 350 according to the division shape information indicating that division is not performed, or may determine the coding units 320a, 320b, 330a, 330b, 330c, 370a, 370b, 380a, 380b, and 380c divided according to the division shape information indicating a specific division method. A specific division method for dividing non-square coding units will be described in detail below through various embodiments.

[0102] According to an embodiment, the image decoding device 150 may determine a shape of splitting a coding unit according to the split shape information by using the split shape information, and in this case, the split shape information may indicate the number of at least one coding unit generated when the coding unit is split. Figure 3 When the division shape information indicates that the current coding unit 300 or 350 is divided into two coding units, the image decoding device 150 can determine the two coding units 320a and 320b or the coding units 370a and 370b respectively included in the current coding unit 300 or 350 by dividing the current coding unit 300 or 350 based on the division shape information.

[0103] According to an embodiment, when the image decoding device 150 divides the current coding unit 300 or 350 having a non-square shape based on the division shape information, the image decoding device 150 may divide the current coding unit 300 or 350 having a non-square shape in consideration of the position of the long side of the current coding unit 300 or 350. For example, the image decoding device 150 may determine a plurality of coding units by dividing the current coding unit 300 or 350 in a direction in which the long side of the current coding unit 300 or 350 is divided, in consideration of the shape of the current coding unit 300 or 350.

[0104] According to an embodiment, when the division shape information indicates that the coding unit is divided into an odd number of blocks, the image decoding device 150 may determine the odd number of coding units included in the current coding unit 300 or 350. For example, when the division shape information indicates that the current coding unit 300 or 350 is divided into three coding units, the image decoding device 150 may divide the current coding unit 300 or 350 into three coding units 330a, 330b, and 330c or coding units 380a, 380b, and 380c. According to an embodiment, the image decoding device 150 may determine the odd number of coding units included in the current coding unit 300 or 350, and the sizes of the determined coding units may be different. For example, the size of the coding unit 330b or 380b among the odd number of coding units 330a, 330b, and 330c or the coding units 380a, 380b, and 380c may be different from the size of the coding units 330a and 330c or the coding units 380a and 380c. That is, coding units that may be determined when the current coding unit 300 or 350 is split may have a plurality of sizes.

[0105] According to an embodiment, when the division shape information indicates that the coding unit is divided into an odd number of blocks, the image decoding device 150 may determine an odd number of coding units included in the current coding unit 300 or 350, and may impose a restriction on at least one of the odd number of coding units generated by dividing the current coding unit 300 or 350. Figure 3 , the image decoding device 150 may decode the coding unit 330b or 380b at the center of the three coding units 330a, 330b, and 330c or the coding units 380a, 380b, and 380c generated when the current coding unit 300 or 350 is divided in a different manner from the coding units 330a and 330c or the coding units 380a and 380c. For example, unlike the coding units 330a and 330c or the coding units 380a and 380c, the image decoding device 150 may restrict the coding unit 330b or 380b at the center from being further divided or being divided only a specific number of times.

[0106] Figure 4 A process of splitting coding units based on at least one of block shape information and division shape information, performed by the image decoding device 150 according to an embodiment, is illustrated.

[0107] According to an embodiment, the image decoding device 150 may determine whether to divide the square first coding unit 400 into coding units or not to divide the square first coding unit 400 based on at least one of the block shape information and the division shape information. According to an embodiment, when the division shape information indicates that the first coding unit 400 is divided in the horizontal direction, the image decoding device 150 may determine the second coding unit 410 by dividing the first coding unit 400 in the horizontal direction. The first coding unit, the second coding unit, and the third coding unit used according to the embodiment are terms for understanding the relationship before and after dividing the coding unit. For example, the second coding unit may be determined by dividing the first coding unit, and the third coding unit may be determined by dividing the second coding unit. It will be understood that the relationship between the first coding unit, the second coding unit, and the third coding unit is applicable to the following description.

[0108] According to an embodiment, the image decoding device 150 may determine whether to split the determined second coding unit 410 into coding units or not to split the determined second coding unit 410 based on at least one of the block shape information and the division shape information. Figure 4 , the image decoding device 150 may divide the non-square second coding unit 410 determined by dividing the first coding unit 400 into one or more third coding units 420a or third coding units 420b, 420c, and 420d based on at least one of the block shape information and the division shape information, or may not divide the non-square second coding unit 410. The image decoding device 150 may obtain at least one of the block shape information and the division shape information, and may divide a plurality of second coding units (for example, 410) of various shapes by dividing the first coding unit 400 based on at least one of the obtained block shape information and the division shape information, and may divide the second coding unit 410 by using the division method of the first coding unit 400 based on at least one of the block shape information and the division shape information. According to an embodiment, when the first coding unit 400 is divided into the second coding unit 410 based on at least one of the block shape information and the division shape information of the first coding unit 400, the second coding unit 410 may be further divided into the third coding unit 420a or the third coding units 420b, 420c, and 420d based on at least one of the block shape information and the division shape information of the second coding unit 410. That is, the coding unit may be recursively divided based on at least one of the block shape information and the division shape information of each coding unit. A method for recursively dividing the coding unit will be described below through various embodiments.

[0109] According to an embodiment, the image decoding device 150 may divide the third coding unit 420a or each of the third coding units 420b, 420c, and 420d into coding units based on at least one of the block shape information and the division shape information, or may determine not to divide the second coding unit 410 based on at least one of the block shape information and the division shape information. According to an embodiment, the image decoding device 150 may divide the non-square second coding unit 410 into an odd number of third coding units 420b, 420c, and 420d. The image decoding device 150 may impose specific restrictions on the third coding units among the odd number of third coding units 420b, 420c, and 420d. For example, the image decoding device 150 may limit the third coding unit 420c located at the center position among the odd number of third coding units 420b, 420c, and 420d to no longer be divided or to be divided a set number of times. Refer to Figure 4 , the image decoding device 150 may limit the third coding unit 420c located at the center position among the odd third coding units 420b, 420c, and 420d included in the non-square second coding unit 410 to no longer be divided, to be divided by using a specific division method (for example, divided only into four coding units or divided into a shape corresponding to the shape into which the second coding unit 410 is divided), or to be divided only a specific number of times (for example, divided only n times (where n>0)). However, the restriction on the third coding unit 420c at the center position is not limited to the above example, and may include various restrictions for decoding the third coding unit 420c at the center position that are different from the other third coding units 420b and 420d.

[0110] According to an embodiment, the image decoding device 150 may obtain at least one of block shape information and partition shape information for partitioning the current coding unit from a specific position in the current coding unit.

[0111] According to an embodiment, when the current coding unit is split into a specific number of coding units, the image decoding device 150 may select one of the coding units. Various methods that may be used to select one of a plurality of coding units will be described below through various embodiments.

[0112] According to an embodiment, the image decoding device 150 may split a current coding unit into a plurality of coding units, and may determine a coding unit at a specific position.

[0113] Figure 5 A method of determining a coding unit of a specific position from among odd-numbered coding units, performed by the image decoding device 150 according to an embodiment, is illustrated.

[0114] According to an embodiment, the image decoding device 150 may use information indicating the positions of odd-numbered coding units to determine a coding unit at a center position from among the odd-numbered coding units. Figure 5 , the image decoding device 150 may determine the odd-numbered coding units 520a, 520b, and 520c by dividing the current coding unit 500. The image decoding device 150 may determine the coding unit 520b at the center position by using information about the positions of the odd-numbered coding units 520a, 520b, and 520c. For example, the image decoding device 150 may determine the coding unit 520b at the center position by determining the positions of the coding units 520a, 520b, and 520c based on information indicating the positions of specific samples included in the coding units 520a, 520b, and 520c. In detail, the image decoding device 150 may determine the coding unit 520b at the center position by determining the positions of the coding units 520a, 520b, and 520c based on information indicating the positions of the upper left samples 530a, 530b, and 530c of the coding units 520a, 520b, and 520c.

[0115] According to an embodiment, the information indicating the positions of the upper left sample points 530a, 530b, and 530c respectively included in the coding units 520a, 520b, and 520c may include information about the positions or coordinates of the coding units 520a, 520b, and 520c in the picture. According to an embodiment, the information indicating the positions of the upper left sample points 530a, 530b, and 530c respectively included in the coding units 520a, 520b, and 520c may include information indicating the width or height of the coding units 520a, 520b, and 520c included in the current coding unit 500, and the width or height may correspond to the information indicating the difference between the coordinates of the coding units 520a, 520b, and 520c in the picture. That is, the image decoding device 150 can determine the coding unit 520b at the center position by directly using the information about the positions or coordinates of the coding units 520a, 520b, and 520c in the picture, or by using the information about the width or height of the coding unit corresponding to the difference between the coordinates.

[0116] According to an embodiment, information indicating the position of the upper left sample point 530a of the upper coding unit 520a may include coordinates (xa, ya), information indicating the position of the upper left sample point 530b of the middle coding unit 520b may include coordinates (xb, yb), and information indicating the position of the upper left sample point 530c of the lower coding unit 520c may include coordinates (xc, yc). The image decoding device 150 may determine the middle coding unit 520b by using the coordinates of the upper left sample points 530a, 530b, and 530c respectively included in the coding units 520a, 520b, and 520c. For example, when the coordinates of the upper left sample points 530a, 530b, and 530c are sorted in ascending or descending order, the coding unit 520b including the coordinates (xb, yb) of the sample point 530b at the center position may be determined as the coding unit at the center position among the coding units 520a, 520b, and 520c determined by dividing the current coding unit 500. However, the coordinates indicating the positions of the upper left sample points 530a, 530b, and 530c may include coordinates indicating absolute positions in the picture, or coordinates (dxb, dyb) indicating the relative position of the upper left sample point 530b of the middle coding unit 520b relative to the position of the upper left sample point 530a of the upper coding unit 520a and coordinates (dxc, dyc) indicating the relative position of the upper left sample point 530c of the lower coding unit 520c relative to the position of the upper left sample point 530a of the upper coding unit 520a may be used. In addition, the method of determining the coding unit at a specific position by using the coordinates of the sample points included in the coding unit as information indicating the positions of the sample points is not limited to the above method, and may include various arithmetic methods capable of using the coordinates of the sample points.

[0117] According to an embodiment, the image decoding device 150 may divide the current coding unit 500 into a plurality of coding units 520a, 520b, and 520c, and may select one of the coding units 520a, 520b, and 520c based on a specific criterion. For example, the image decoding device 150 may select a coding unit 520b having a size different from that of other coding units from among the coding units 520a, 520b, and 520c.

[0118] According to an embodiment, the image decoding device 150 may determine the width or height of the coding units 520a, 520b, and 520c by using the coordinates (xa, ya) indicating the position of the upper left sample point 530a of the upper coding unit 520a, the coordinates (xb, yb) indicating the position of the upper left sample point 530b of the middle coding unit 520b, and the coordinates (xc, yc) indicating the position of the upper left sample point 530c of the lower coding unit 520c. The image decoding device 150 may determine the sizes of the coding units 520a, 520b, and 520c by using the coordinates (xa, ya), (xb, yb), and (xc, yc) indicating the positions of the coding units 520a, 520b, and 520c.

[0119] According to an embodiment, the image decoding device 150 may determine the width of the upper coding unit 520a as xb-xa, and determine the height of the upper coding unit 520a as yb-ya. According to an embodiment, the image decoding device 150 may determine the width of the intermediate coding unit 520b as xc-xb, and determine the height of the intermediate coding unit 520b as yc-yb. According to an embodiment, the image decoding device 150 may determine the width or height of the lower coding unit 520c by using the width or height of the current coding unit 500 and the width or height of the upper coding unit 520a and the intermediate coding unit 520b. The image decoding device 150 may determine a coding unit having a size different from that of other coding units based on the determined width and height of the coding units 520a to 520c. Referring to Figure 5 , the image decoding device 150 may determine the intermediate coding unit 520b having a size different from the sizes of the upper coding unit 520a and the lower coding unit 520c as the coding unit at a specific position. However, the above-mentioned method of determining the coding unit having a size different from the sizes of other coding units performed by the image decoding device 150 only corresponds to an example of determining the coding unit at a specific position by using the size of the coding unit determined based on the coordinates of the sample points, and thus various methods of determining the coding unit at a specific position by comparing the size of the coding unit determined based on the coordinates of the specific sample points may be used.

[0120] However, the position of the sample considered to determine the position of the coding unit is not limited to the above-mentioned upper left position, and information on arbitrary positions of the samples included in the coding unit may be used.

[0121] According to an embodiment, the image decoding device 150 may select a coding unit at a specific position from an odd number of coding units determined by dividing the current coding unit, taking into account the shape of the current coding unit. For example, when the current coding unit has a non-square shape in which the width is greater than the height, the image decoding device 150 may determine a coding unit at a specific position in the horizontal direction. That is, the image decoding device 150 may determine one of the coding units at different positions in the horizontal direction and impose restrictions on the coding unit. When the current coding unit has a non-square shape in which the height is greater than the width, the image decoding device 150 may determine a coding unit at a specific position in the vertical direction. That is, the image decoding device 150 may determine one of the coding units at different positions in the vertical direction and may impose restrictions on the coding unit.

[0122] According to an embodiment, the image decoding device 150 may use information indicating respective positions of even-numbered coding units to determine a coding unit at a specific position among the even-numbered coding units. The image decoding device 150 may determine the even-numbered coding units by dividing the current coding unit, and may determine the coding unit at the specific position by using information about the positions of the even-numbered coding units. Operations related thereto may be similar to those described above with reference to Figure 5 The operation correspondence of determining a coding unit at a specific position (eg, a center position) among odd-numbered coding units has been described in detail, and thus a detailed description thereof is not provided here.

[0123] According to an embodiment, when a non-square current coding unit is split into a plurality of coding units, specific information about a coding unit at a specific position may be used in a split operation to determine a coding unit at a specific position among the plurality of coding units. For example, the image decoding device 150 may use at least one of block shape information and split shape information stored in samples included in a coding unit at a center position to determine a coding unit at a center position among a plurality of coding units determined by splitting the current coding unit in a split operation.

[0124] Reference Figure 5, the image decoding device 150 may divide the current coding unit 500 into a plurality of coding units 520a, 520b, and 520c based on at least one of the block shape information and the division shape information, and may determine the coding unit 520b at the center position among the plurality of coding units 520a, 520b, and 520c. In addition, the image decoding device 150 may determine the coding unit 520b at the center position in consideration of the position where at least one of the block shape information and the division shape information is obtained. That is, at least one of the block shape information and the division shape information of the current coding unit 500 may be obtained from a sample point 540 at the center position of the current coding unit 500, and when the current coding unit 500 is divided into a plurality of coding units 520a, 520b, and 520c based on at least one of the block shape information and the division shape information, the coding unit 520b including the sample point 540 may be determined as the coding unit at the center position. However, information for determining the coding unit at the center position is not limited to at least one of the block shape information and the division shape information, and various types of information may be used to determine the coding unit at the center position.

[0125] According to an embodiment, specific information for identifying a coding unit at a specific position may be obtained from a specific sample point included in a coding unit to be determined. Figure 5 , the image decoding device 150 may determine a coding unit at a specific position (e.g., a coding unit at a center position of the multiple divided coding units) among the multiple coding units 520a, 520b, and 520c determined by dividing the current coding unit 500, using at least one of the block shape information and the division shape information obtained from the sample at a specific position in the current coding unit 500 (e.g., a sample at a center position of the multiple divided coding units). That is, the image decoding device 150 may determine the sample at a specific position by considering the block shape of the current coding unit 500, may determine the coding unit 520b including the sample for which specific information (e.g., at least one of the block shape information and the division shape information) may be obtained from the multiple coding units 520a, 520b, and 520c determined by dividing the current coding unit 500, and may impose specific restrictions on the coding unit 520b. Referring to Figure 5 According to an embodiment, the image decoding device 150 may determine a sample 540 at the center position of the current coding unit 500 as a sample from which specific information may be obtained, and may impose a specific restriction on the coding unit 520b including the sample 540 in a decoding operation. However, the position of the sample from which specific information may be obtained is not limited to the above-mentioned position, and may include any sample position included in the coding unit 520b to be determined for restriction.

[0126] According to an embodiment, the position of the sample at which specific information can be obtained may be determined based on the shape of the current coding unit 500. According to an embodiment, the block shape information may indicate whether the current coding unit has a square shape or a non-square shape, and the position of the sample at which specific information can be obtained may be determined based on the shape. For example, the image decoding device 150 may determine a sample located on a boundary for dividing at least one of the width and the height of the current coding unit in half as a sample at which specific information can be obtained by using at least one of the information about the width of the current coding unit and the information about the height of the current coding unit. As another example, when the block shape information of the current coding unit indicates a non-square shape, the image decoding device 150 may determine one of the samples adjacent to the boundary for dividing the long side of the current coding unit in half as a sample at which specific information can be obtained.

[0127] According to an embodiment, when the current coding unit is divided into a plurality of coding units, the image decoding device 150 may use at least one of block shape information and division shape information to determine a coding unit at a specific position in the plurality of coding units. According to an embodiment, the image decoding device 150 may obtain at least one of block shape information and division shape information from a sample at a specific position in the coding unit, and may divide the plurality of coding units by using at least one of block shape information and division shape information obtained from a sample at a specific position in each of the plurality of coding units generated by dividing the current coding unit. That is, the coding unit may be recursively divided based on at least one of block shape information and division shape information obtained from a sample at a specific position in each coding unit. As has been described above with reference to Figure 4 An operation of recursively splitting the coding unit is described, and thus a detailed description thereof will not be provided here.

[0128] According to an embodiment, the image decoding device 150 may determine one or more coding units by splitting a current coding unit, and may determine an order in which to decode the one or more coding units based on a specific block (for example, the current coding unit).

[0129] Figure 6 An order in which a plurality of coding units are processed when the image decoding device 150 determines a plurality of coding units by splitting a current coding unit according to an embodiment is illustrated.

[0130] According to an embodiment, the image decoding device 150 may determine the second coding units 610a and 610b by dividing the first coding unit 600 in a vertical direction, determine the second coding units 630a and 630b by dividing the first coding unit 600 in a horizontal direction, or determine the second coding units 650a to 650d by dividing the first coding unit 600 in vertical and horizontal directions, based on block shape information and division shape information.

[0131] Reference Figure 6 , the image decoding device 150 may determine to process the second coding units 610a and 610b determined by dividing the first coding unit 600 in the vertical direction in a horizontal direction order 610c. The image decoding device 150 may determine to process the second coding units 630a and 630b determined by dividing the first coding unit 600 in the horizontal direction in a vertical direction order 630c. The image decoding device 150 may determine to process the second coding units 650a to 650d determined by dividing the first coding unit 600 in the vertical and horizontal directions in a specific order (for example, in a raster scan order or a zigzag scan order 650e), wherein the specific order is used to process the coding units in one row and then process the coding units in the next row.

[0132] According to an embodiment, the image decoding device 150 may recursively divide the coding unit. Figure 6 , the image decoding device 150 may determine a plurality of second coding units 610a, 610b, 630a, 630b, 650a, 650b, 650c, and 650d by dividing the first coding unit 600, and may recursively divide each of the determined plurality of second coding units 610a, 610b, 630a, 630b, 650a, 650b, 650c, and 650d. The division method of the plurality of second coding units 610a, 610b, 630a, 630b, 650a, 650b, 650c, and 650d may correspond to the division method of the first coding unit 600. In this way, each of the plurality of second coding units 610a, 610b, 630a, 630b, 650a, 650b, 650c, and 650d may be independently divided into a plurality of coding units. Referring to Figure 6 , the image decoding device 150 may determine the second coding units 610a and 610b by splitting the first coding unit 600 in a vertical direction, and may determine whether to split or not split each of the second coding units 610a and 610b independently.

[0133] According to an embodiment, the image decoding device 150 may determine the third coding units 620a and 620b by splitting the left second coding unit 610a in a horizontal direction, and may not split the right second coding unit 610b.

[0134] According to an embodiment, the processing order of the coding units may be determined based on the operation of dividing the coding units. In other words, the processing order of the divided coding units may be determined based on the processing order of the coding units immediately before being divided. The image decoding device 150 may determine the processing order of the third coding units 620a and 620b determined by dividing the left second coding unit 610a independently of the right second coding unit 610b. Since the third coding units 620a and 620b are determined by dividing the left second coding unit 610a in the horizontal direction, the third coding units 620a and 620b may be processed in a vertical direction order 620c. Since the left second coding unit 610a and the right second coding unit 610b are processed in a horizontal direction order 610c, the right second coding unit 610b may be processed after the third coding units 620a and 620b included in the left second coding unit 610a are processed in a vertical direction order 620c. The operation of determining the processing order of coding units based on the coding units before being split is not limited to the above example, and coding units split and determined into various shapes may be independently processed in a specific order using various methods.

[0135] Figure 7 A process of determining that a current coding unit is to be split into an odd number of coding units when coding units cannot be processed in a specific order, performed by the image decoding device 150 according to an embodiment, is illustrated.

[0136] According to an embodiment, the image decoding device 150 may determine whether the current coding unit is split into an odd number of coding units based on the obtained block shape information and the division shape information. Figure 7 , the square first coding unit 700 may be divided into non-square second coding units 710a and 710b, and the second coding units 710a and 710b may be independently divided into third coding units 720a and 720b and third coding units 720c to 720e. According to an embodiment, the image decoding device 150 may determine a plurality of third coding units 720a and 720b by dividing the left second coding unit 710a in the horizontal direction, and may divide the right second coding unit 710b into an odd number of third coding units 720c to 720e.

[0137] According to an embodiment, the image decoding device 150 may determine whether to split any coding unit into an odd number of coding units by determining whether the third coding units 720a and 720b and the third coding units 720c to 720e can be processed in a specific order. Figure 7 , the image decoding device 150 may determine the third coding units 720a and 720b and the third coding units 720c to 720e by recursively dividing the first coding unit 700. The image decoding device 150 may determine whether any one of the first coding unit 700, the second coding units 710a and 710b, the third coding units 720a and 720b, and the third coding units 720c, 720d, and 720e is divided into an odd number of coding units based on at least one of the block shape information and the division shape information. For example, the right second coding unit 710b may be divided into an odd number of third coding units 720c, 720d, and 720e. The processing order of the multiple coding units included in the first coding unit 700 can be a specific order (for example, a zigzag scanning order 730), and the image decoding device 150 can determine whether the third coding units 720c, 720d and 720e determined by dividing the right second coding unit 710b into an odd number of coding units meet the conditions for processing in a specific order.

[0138] According to an embodiment, the image decoding device 150 may determine whether the third coding units 720a and 720b and the third coding units 720c, 720d, and 720e included in the first coding unit 700 satisfy a condition for being processed in a specific order, and the condition is related to whether at least one of the width and height of the second coding units 710a and 710b is divided in half along the boundary of the third coding units 720a and 720b and the third coding units 720c, 720d, and 720e. For example, the third coding units 720a and 720b determined by dividing the height of the non-square left second coding unit 710a in half satisfy the condition, because the boundary of the third coding units 720c, 720d, and 720e determined by dividing the right second coding unit 710b into three coding units does not divide the width or height of the right second coding unit 710b in half, so it can be determined that the third coding units 720c, 720d, and 720e do not satisfy the condition. When the condition is not satisfied as described above, the image decoding device 150 may determine that the scanning order is discontinuous, and determine that the right second coding unit 710b is divided into an odd number of coding units based on the result of the determination. According to an embodiment, when the coding unit is divided into an odd number of coding units, the image decoding device 150 may impose specific restrictions on the coding units at specific positions in the divided coding units, and the restrictions or specific positions have been described above through various embodiments, so a detailed description thereof will not be provided here.

[0139] Figure 8 1 shows a process of determining at least one coding unit by dividing the first coding unit 800, which is performed by the image decoding device 150 according to an embodiment. According to an embodiment, the image decoding device 150 may divide the first coding unit 800 based on at least one of the block shape information and the division shape information obtained by the receiver 160. The square first coding unit 800 may be divided into four square coding units, or may be divided into a plurality of non-square coding units. For example, referring to Figure 8 , when the block shape information indicates that the first coding unit 800 has a square shape and the division shape information indicates that the first coding unit 800 is divided into non-square coding units, the image decoding device 150 may divide the first coding unit 800 into a plurality of non-square coding units. In detail, when the division shape information indicates that an odd number of coding units is determined by dividing the first coding unit 800 in a horizontal direction or a vertical direction, the image decoding device 150 may divide the square first coding unit 800 into an odd number of coding units, for example, second coding units 810a, 810b, and 810c determined by dividing the square first coding unit 800 in a vertical direction or second coding units 820a, 820b, and 820c determined by dividing the square first coding unit 800 in a horizontal direction.

[0140] According to an embodiment, the image decoding device 150 may determine whether the second coding units 810a, 810b, 810c, 820a, 820b, and 820c included in the first coding unit 800 satisfy a condition for being processed in a specific order, and the condition is related to whether at least one of the width and height of the first coding unit 800 is divided in half along the boundary of the second coding units 810a, 810b, 810c, 820a, 820b, and 820c. Figure 8, because the boundaries of the second coding units 810a, 810b, and 810c determined by dividing the square first coding unit 800 in the vertical direction do not divide the width of the first coding unit 800 in half, it can be determined that the first coding unit 800 does not meet the condition for processing in a specific order. In addition, because the boundaries of the second coding units 820a, 820b, and 820c determined by dividing the square first coding unit 800 in the horizontal direction do not divide the width of the first coding unit 800 in half, it can be determined that the first coding unit 800 does not meet the condition for processing in a specific order. When the condition is not satisfied as described above, the image decoding device 150 may decide that the scanning order is not continuous, and may determine that the first coding unit 800 is divided into an odd number of coding units based on the result of the decision. According to an embodiment, when the coding unit is divided into an odd number of coding units, the image decoding device 150 may impose a specific restriction on the coding unit at a specific position in the divided coding unit, and the restriction or the specific position has been described above through various embodiments, so a detailed description thereof will not be provided here.

[0141] According to an embodiment, the image decoding device 150 may determine coding units of various shapes by splitting the first coding unit.

[0142] Reference Figure 8 , the image decoding device 150 may divide the square first coding unit 800 or the non-square first coding unit 830 or 850 into coding units of various shapes.

[0143] Fig. 9 It is shown that when a non-square second coding unit determined by splitting the first coding unit 900 satisfies a specific condition according to an embodiment, shapes into which the second coding unit may be split by the image decoding device 150 are limited.

[0144] According to an embodiment, the image decoding device 150 may determine whether to divide the square first coding unit 900 into non-square second coding units 910a, 910b, 920a, and 920b based on at least one of the block shape information and the division shape information obtained by the receiver 160. The second coding units 910a, 910b, 920a, and 920b may be independently divided. Therefore, the image decoding device 150 may determine whether to divide the first coding unit 900 into a plurality of coding units or not to divide the first coding unit 900 based on at least one of the block shape information and the division shape information of each of the second coding units 910a, 910b, 920a, and 920b. According to an embodiment, the image decoding device 150 may determine the third coding units 912a and 912b by dividing the non-square left second coding unit 910a determined by dividing the first coding unit 900 in the vertical direction in the horizontal direction. However, when the left second coding unit 910a is divided in the horizontal direction, the image decoding device 150 may limit the right second coding unit 910b to not be divided in the horizontal direction in which the left second coding unit 910a is divided. When the third coding units 914a and 914b are determined by dividing the right second coding unit 910b in the same direction, since the left second coding unit 910a and the right second coding unit 910b are independently divided in the horizontal direction, the third coding units 912a, 912b, 914a, and 914b may be determined. However, this case is equivalent to the case in which the image decoding device 150 divides the first coding unit 900 into four square second coding units 930a, 930b, 930c, and 930d based on at least one of the block shape information and the division shape information, and may be inefficient in terms of image decoding.

[0145] According to an embodiment, the image decoding device 150 may determine the third coding units 922a, 922b, 924a, and 924b by vertically splitting the non-square second coding unit 920a or 920b determined by splitting the first coding unit 900 in the horizontal direction. However, when the second coding unit (e.g., the upper second coding unit 920a) is split in the vertical direction, due to the above reasons, the image decoding device 150 may limit another second coding unit (e.g., the lower second coding unit 920b) to not be split in the vertical direction in which the upper second coding unit 920a is split.

[0146] Fig.10 A process of splitting a square coding unit when division shape information indicates that the square coding unit is not to be split into four square coding units, performed by the image decoding device 150 according to an embodiment, is illustrated.

[0147] According to an embodiment, the image decoding device 150 may determine the second coding units 1010a, 1010b, 1020a, 1020b, etc. by dividing the first coding unit 1000 based on at least one of the block shape information and the division shape information. The division shape information may include information about various methods of dividing the coding unit, but the information about the various division methods may not include information for dividing the coding unit into four square coding units. According to such division shape information, the image decoding device 150 may not divide the square first coding unit 1000 into four square second coding units 1030a, 1030b, 1030c, and 1030d. The image decoding device 150 may determine the non-square second coding units 1010a, 1010b, 1020a, 1020b, etc. based on the division shape information.

[0148] According to an embodiment, the image decoding device 150 may independently divide the non-square second coding units 1010a, 1010b, 1020a, 1020b, etc. Each of the second coding units 1010a, 1010b, 1020a, 1020b, etc. may be recursively divided in a specific order, and this division method may correspond to a method of dividing the first coding unit 1000 based on at least one of block shape information and division shape information.

[0149] For example, the image decoding device 150 may determine the square third coding units 1012a and 1012b by dividing the left second coding unit 1010a in the horizontal direction, and may determine the square third coding units 1014a and 1014b by dividing the right second coding unit 1010b in the horizontal direction. In addition, the image decoding device 150 may determine the square third coding units 1016a, 1016b, 1016c, and 1016d by dividing both the left second coding unit 1010a and the right second coding unit 1010b in the horizontal direction. In this case, a coding unit having the same shape as the four square second coding units 1030a, 1030b, 1030c, and 1030d divided from the first coding unit 1000 may be determined.

[0150] As another example, the image decoding device 150 may determine the third coding units 1022a and 1022b of the square shape by dividing the upper second coding unit 1020a in the vertical direction, and may determine the third coding units 1024a and 1024b of the square shape by dividing the lower second coding unit 1020b in the vertical direction. In addition, the image decoding device 150 may determine the third coding units 1026a, 1026b, 1026c, and 1026d of the square shape by dividing both the upper second coding unit 1020a and the lower second coding unit 1020b in the vertical direction. In this case, the coding units having the same shape as the four square second coding units 1030a, 1030b, 1030c, and 1030d divided from the first coding unit 1000 may be determined.

[0151] Fig.11 It is shown that a processing order among a plurality of coding units may be changed depending on a process of dividing the coding units according to an embodiment.

[0152] According to an embodiment, the image decoding device 150 may divide the first coding unit 1100 based on the block shape information and the division shape information. When the block shape information indicates a square shape and the division shape information indicates that the first coding unit 1100 is divided in at least one of the horizontal direction and the vertical direction, the image decoding device 150 may determine the second coding units 1110a, 1110b, 1120a, 1120b, etc. by dividing the first coding unit 1100. Fig.11 , the non-square second coding units 1110a, 1110b, 1120a, and 1120b determined by dividing the first coding unit 1100 only in the horizontal direction or the vertical direction may be independently divided based on the block shape information and the division shape information of each coding unit. For example, the image decoding device 150 may determine the third coding units 1116a, 1116b, 1116c, and 1116d by dividing the second coding units 1110a and 1110b generated by dividing the first coding unit 1100 in the vertical direction in the horizontal direction, and may determine the third coding units 1126a, 1126b, 1126c, and 1126d by dividing the second coding units 1120a and 1120b generated by dividing the first coding unit 1100 in the horizontal direction in the horizontal direction. As has been mentioned above, Fig. 9 The operation of dividing the second coding units 1110a, 1110b, 1120a, and 1120b is described, and thus a detailed description thereof will not be provided here.

[0153] According to an embodiment, the image decoding device 150 may process the coding units in a specific order. Figure 6The operations of processing the coding units in a specific order are described, and thus a detailed description thereof will not be provided here. Fig.11 , the image decoding device 150 may determine four square third coding units 1116a, 1116b, 1116c, and 1116d and third coding units 1126a, 1126b, 1126c, and 1126d by dividing the square first coding unit 1100. According to an embodiment, the image decoding device 150 may determine the processing order of the third coding units 1116a, 1116b, 1116c, and 1116d and the third coding units 1126a, 1126b, 1126c, and 1126d based on the division method of the first coding unit 1100.

[0154] According to an embodiment, the image decoding device 150 may determine third coding units 1116a, 1116b, 1116c, and 1116d by dividing the second coding units 1110a and 1110b generated by dividing the first coding unit 1100 in the vertical direction in the horizontal direction, and may process the third coding units 1116a, 1116b, 1116c, and 1116d in a processing order 1117, wherein the processing order 1117 is: first, the third coding units 1116a and 1116c included in the left second coding unit 1110a are processed in the vertical direction, and then the third coding units 1116b and 1116d included in the right second coding unit 1110b are processed in the vertical direction.

[0155] According to an embodiment, the image decoding device 150 may determine third coding units 1126a, 1126b, 1126c, and 1126d by dividing the second coding units 1120a and 1120b generated by dividing the first coding unit 1100 in the horizontal direction in a vertical direction, and may process the third coding units 1126a, 1126b, 1126c, and 1126d in a processing order 1127, wherein the processing order 1127 is: first, the third coding units 1126a and 1126b included in the upper second coding unit 1120a are processed in the horizontal direction, and then the third coding units 1126c and 1126d included in the lower second coding unit 1120b are processed in the horizontal direction.

[0156] Reference Fig.11, square third coding units 1116a, 1116b, 1116c and 1116d and third coding units 1126a, 1126b, 1126c and 1126d can be determined by dividing the second coding units 1110a, 1110b, 1120a and 1120b respectively. Although the second coding units 1110a and 1110b determined by splitting the first coding unit 1100 in the vertical direction are different from the second coding units 1120a and 1120b determined by splitting the first coding unit 1100 in the horizontal direction, the third coding units 1116a, 1116b, 1116c, and 1116d and the third coding units 1126a, 1126b, 1126c, and 1126d split from the second coding units 1110a and 1110b and the second coding units 1120a and 1120b ultimately show the same shape of coding units split from the first coding unit 1100. In this way, by recursively splitting the coding units in different ways based on at least one of the block shape information and the split shape information, even if the coding units are ultimately determined to be the same shape, the image decoding device 150 can process a plurality of coding units in different orders.

[0157] Fig.12 A process of determining a depth of a coding unit as a shape and a size of a coding unit change when a coding unit is recursively split so that a plurality of coding units are determined according to an embodiment is illustrated.

[0158] According to an embodiment, the image decoding device 150 may determine the depth of the coding unit based on a specific criterion. For example, the specific criterion may be the length of the long side of the coding unit. When the length of the long side of the coding unit before being divided is 2n times (n>0) the length of the long side of the divided current coding unit, the image decoding device 150 may determine that the depth of the current coding unit is increased by n from the depth of the coding unit before being divided. In the following description, a coding unit with an increased depth is represented as a coding unit with a deeper depth.

[0159] Reference Fig.12According to an embodiment, the image decoding device 150 may determine a second coding unit 1202, a third coding unit 1204, and the like having a deeper depth by dividing the first coding unit 1200 of the square based on block shape information indicating a square shape (for example, the block shape information may be represented as "0: SQUARE"). Assuming that the size of the first coding unit 1200 of the square is 2N×2N, the second coding unit 1202 determined by dividing the width and height of the first coding unit 1200 to 1 / 21 may have a size of N×N. In addition, the third coding unit 1204 determined by dividing the width and height of the second coding unit 1202 to 1 / 2 may have a size of N / 2×N / 2. In this case, the width and height of the third coding unit 1204 are 1 / 22 times the width and height of the first coding unit 1200. When the depth of the first coding unit 1200 is D, the depth of the second coding unit 1202 whose width and height are 1 / 21 times the width and height of the first coding unit 1200 may be D+1, and the depth of the third coding unit 1204 whose width and height are 1 / 22 times the width and height of the first coding unit 1200 may be D+2.

[0160] According to an embodiment, the image decoding device 150 can determine a second coding unit 1212 or 1222, a third coding unit 1214 or 1224, and the like having a deeper depth by dividing the non-square first coding unit 1210 or 1220 based on block shape information indicating a non-square shape (for example, the block shape information may be represented as "1:NS_VER" indicating a non-square shape in which the height is greater than the width, or as "2:NS_HOR" indicating a non-square shape in which the width is greater than the height).

[0161] The image decoding device 150 may determine the second coding unit 1202, 1212, or 1222 by dividing at least one of the width and the height of the first coding unit 1210 having a size of N×2N. That is, the image decoding device 150 may determine the second coding unit 1202 having a size of N×N or the second coding unit 1222 having a size of N×N / 2 by dividing the first coding unit 1210 in the horizontal direction, or may determine the second coding unit 1212 having a size of N / 2×N by dividing the first coding unit 1210 in the horizontal direction and the vertical direction.

[0162] According to an embodiment, the image decoding device 150 may determine the second coding unit 1202, 1212, or 1222 by dividing at least one of the width and the height of the first coding unit 1220 having a size of 2N×N. That is, the image decoding device 150 may determine the second coding unit 1202 having a size of N×N or the second coding unit 1212 having a size of N / 2×N by dividing the first coding unit 1220 in a vertical direction, or may determine the second coding unit 1222 having a size of N×N / 2 by dividing the first coding unit 1220 in horizontal and vertical directions.

[0163] According to an embodiment, the image decoding device 150 may determine the third coding unit 1204, 1214, or 1224 by dividing at least one of the width and the height of the second coding unit 1202 having a size of N×N. That is, the image decoding device 150 may determine the third coding unit 1204 having a size of N / 2×N / 2, the third coding unit 1214 having a size of N / 2×N / 2, or the third coding unit 1224 having a size of N / 2×N / 2 by dividing the second coding unit 1202 in the vertical direction and the horizontal direction.

[0164] According to an embodiment, the image decoding device 150 may determine the third coding unit 1204, 1214, or 1224 by dividing at least one of the width and the height of the second coding unit 1212 having a size of N / 2×N. That is, the image decoding device 150 may determine the third coding unit 1204 having a size of N / 2×N / 2 or the third coding unit 1224 having a size of N / 2×N / 2 by dividing the second coding unit 1212 in the horizontal direction, or may determine the third coding unit 1214 having a size of N / 2×N / 2 by dividing the second coding unit 1212 in the vertical and horizontal directions.

[0165] According to an embodiment, the image decoding device 150 may determine the third coding unit 1204, 1214, or 1224 by dividing at least one of the width and the height of the second coding unit 1222 having a size of N×N / 2. That is, the image decoding device 150 may determine the third coding unit 1204 having a size of N / 2×N / 2 or the third coding unit 1214 having a size of N / 2×N / 2 by dividing the second coding unit 1222 in the vertical direction, or may determine the third coding unit 1224 having a size of N / 2×N / 2 by dividing the second coding unit 1222 in the vertical direction and the horizontal direction.

[0166] According to an embodiment, the image decoding device 150 may divide a square coding unit (e.g., 1200, 1202, or 1204) in a horizontal direction or a vertical direction. For example, the image decoding device 150 may determine a first coding unit 1210 having a size of N×2N by dividing a first coding unit 1200 having a size of 2N×2N in a vertical direction, or may determine a first coding unit 1220 having a size of 2N×N by dividing the first coding unit 1200 in a horizontal direction. According to an embodiment, when the depth is determined based on the length of the longest side of the coding unit, the depth of the coding unit determined by dividing the first coding unit 1200, 1202, or 1204 having a size of 2N×2N in a horizontal direction or a vertical direction may be the same as the depth of the first coding unit 1200, 1202, or 1204.

[0167] According to an embodiment, the width and height of the third coding unit 1214 or 1224 may be 1 / 2 times the width and height of the first coding unit 1210 or 1220. When the depth of the first coding unit 1210 or 1220 is D, the depth of the second coding unit 1212 or 1214 whose width and height are 1 / 2 times the width and height of the first coding unit 1210 or 1220 may be D+1, and the depth of the third coding unit 1214 or 1224 whose width and height are 1 / 2 times the width and height of the first coding unit 1210 or 1220 may be D+2.

[0168] Fig.13 Depth that may be determined based on a shape and a size of a coding unit and a partial index (PID) for distinguishing coding units according to an embodiment are illustrated.

[0169] According to an embodiment, the image decoding device 150 may determine second coding units of various shapes by dividing the square first coding unit 1300. Fig.13 , the image decoding device 150 may determine the second coding units 1302a and 1302b, the second coding units 1304a and 1304b, and the second coding units 1306a, 1306b, 1306c, and 1306d by dividing the first coding unit 1300 in at least one of the vertical direction and the horizontal direction based on the division shape information. That is, the image decoding device 150 may determine the second coding units 1302a and 1302b, the second coding units 1304a and 1304b, and the second coding units 1306a, 1306b, 1306c, and 1306d based on the division shape information of the first coding unit 1300.

[0170] According to an embodiment, the depths of the second coding units 1302a and 1302b, the second coding units 1304a and 1304b, and the second coding units 1306a, 1306b, 1306c, and 1306d determined based on the division shape information of the square first coding unit 1300 may be determined based on the length of their long sides. For example, since the side length of the square first coding unit 1300 is equal to the length of the long sides of the non-square second coding units 1302a and 1302b and the second coding units 1304a and 1304b, the first coding unit 1300 and the non-square second coding units 1302a and 1302b and the second coding units 1304a and 1304b may have the same depth, for example, D. However, when the image decoding device 150 divides the first coding unit 1300 into four square second coding units 1306a, 1306b, 1306c and 1306d based on the division shape information, because the side length of the square second coding units 1306a, 1306b, 1306c and 1306d is 1 / 2 times the side length of the first coding unit 1300, the depth of the second coding units 1306a, 1306b, 1306c and 1306d can be D+1 which is 1 deeper than the depth D of the first coding unit 1300.

[0171] According to an embodiment, the image decoding device 150 may determine a plurality of second coding units 1312a and 1312b and second coding units 1313a, 1313b, and 1313c by dividing the first coding unit 1310 having a height greater than a width in a horizontal direction based on the division shape information. According to an embodiment, the image decoding device 150 may determine a plurality of second coding units 1322a and 1322b and second coding units 1324a, 1324b, and 1324c by dividing the first coding unit 1320 having a width greater than a height in a vertical direction based on the division shape information.

[0172] According to an embodiment, the depth of the second coding units 1312a, 1312b, 1314a, 1314b, 1316a, 1316b, 1316c, and 1316d determined based on the division shape information of the non-square first coding unit 1310 or 1320 may be determined based on the length of its long side. For example, since the length of the side of the square second coding units 1312a and 1312b is 1 / 2 times the length of the long side of the first coding unit 1310 of the non-square shape having a height greater than a width, the depth of the square second coding units 1302a, 1302b, 1304a, and 1304b is D+1 which is 1 deeper than the depth D of the non-square first coding unit 1310.

[0173] In addition, the image decoding device 150 may divide the non-square first coding unit 1310 into an odd number of second coding units 1314a, 1314b, and 1314c based on the division shape information. The odd number of second coding units 1314a, 1314b, and 1314c may include the non-square second coding units 1314a and 1314c and the square second coding unit 1314b. In this case, since the length of the long side of the non-square second coding units 1314a and 1314c and the length of the side of the square second coding unit 1314b are 1 / 2 times the length of the long side of the first coding unit 1310, the depth of the second coding units 1314a, 1314b, and 1314c may be D+1 which is 1 deeper than the depth D of the first coding unit 1310. The image decoding device 150 may determine the depth of the coding unit divided from the non-square first coding unit 1320 whose width is greater than the height by using the above-mentioned method of determining the depth of the coding unit divided from the first coding unit 1310.

[0174] According to an embodiment, when the divided odd-numbered coding units do not have equal sizes, the image decoding device 150 may determine a PID for identifying the divided coding units based on a size ratio between the coding units. Fig.13 , the coding unit 1314b at the center position among the divided odd-numbered coding units 1314a, 1314b, and 1314c may have a width equal to that of the other coding units 1314a and 1314c and a height twice the height of the other coding units 1314a and 1314c. That is, in this case, the coding unit 1314b at the center position may include two other coding units 1314a or 1314c. Therefore, when the PID of the coding unit 1314b at the center position based on the scanning order is 1, the PID of the coding unit 1314c located next to the coding unit 1314b may increase by 2 and thus may be 3. That is, there may be discontinuity in the PID value. According to an embodiment, the image decoding device 150 may determine whether the divided odd-numbered coding units do not have equal sizes based on whether there is discontinuity in the PID for identifying the divided coding units.

[0175] According to an embodiment, the image decoding device 150 may determine whether to use a specific splitting method based on a PID value for identifying a plurality of coding units determined by splitting a current coding unit. Fig.13, the image decoding device 150 may determine the even number of coding units 1312a and 1312b or the odd number of coding units 1314a, 1314b, and 1314c by dividing the first coding unit 1310 having a rectangular shape whose height is greater than its width. The image decoding device 150 may use the PID to identify each coding unit. According to an embodiment, the PID may be obtained from a sample at a specific position of each coding unit (e.g., a top left sample).

[0176] According to an embodiment, the image decoding device 150 may determine a coding unit at a specific position among the divided coding units by using a PID for distinguishing coding units. According to an embodiment, when the division shape information of the first coding unit 1310 having a rectangular shape with a height greater than a width indicates that the coding unit is divided into three coding units, the image decoding device 150 may divide the first coding unit 1310 into three coding units 1314a, 1314b, and 1314c. The image decoding device 150 may assign a PID to each of the three coding units 1314a, 1314b, and 1314c. The image decoding device 150 may compare the PIDs of the odd-numbered divided coding units to determine a coding unit at a center position among the coding units. The image decoding device 150 may determine the coding unit 1314b having a PID corresponding to a middle value among the PIDs of the coding units as a coding unit at a center position among the coding units determined by dividing the first coding unit 1310. According to an embodiment, when the divided coding units do not have equal sizes, the image decoding device 150 may determine a PID for distinguishing the divided coding units based on a size ratio between the coding units. Fig.13, the width of the coding unit 1314b generated by dividing the first coding unit 1310 may be equal to the width of the other coding units 1314a and 1314c, and the height may be twice the height of the other coding units 1314a and 1314c. In this case, when the PID of the coding unit 1314b at the center position is 1, the PID of the coding unit 1314c located next to the coding unit 1314b may increase by 2 and thus may be 3. When the PID does not increase uniformly as described above, the image decoding device 150 may determine to divide the coding unit into a plurality of coding units, wherein the plurality of coding units include coding units having sizes different from those of the other coding units. According to an embodiment, when the division shape information indicates that the coding unit is divided into an odd number of coding units, the image decoding device 150 may divide the current coding unit in such a manner that the coding unit at a specific position (e.g., the coding unit at the center position) among the odd number of coding units has a size different from that of the other coding units. In this case, the image decoding device 150 may determine the coding units at the center position having different sizes by using the PIDs of the coding units. However, the PID and the size or the position of the coding unit of the specific position to be determined are not limited to the above-mentioned examples, and various PIDs and various positions and sizes of the coding unit may be used.

[0177] According to an embodiment, the image decoding device 150 may use a specific data unit in which a coding unit starts to be recursively split.

[0178] Fig.14 It is illustrated that a plurality of coding units are determined based on a plurality of specific data units included in a picture according to an embodiment.

[0179] According to an embodiment, a specific data unit may be defined as a data unit that starts recursively splitting a coding unit by using at least one of block shape information and division shape information. That is, the specific data unit may correspond to a coding unit for determining a highest depth of a plurality of coding units split from a current picture. In the following description, for ease of explanation, the specific data unit is referred to as a reference data unit.

[0180] According to an embodiment, the reference data unit may have a specific size and a specific shape. According to an embodiment, the reference data unit may include M×N samples. Herein, M and N may be equal to each other and may be an integer expressed as a power of 2. That is, the reference data unit may have a square shape or a non-square shape and may be divided into an integer number of coding units.

[0181] According to an embodiment, the image decoding device 150 may divide the current picture into a plurality of reference data units. According to an embodiment, the image decoding device 150 may divide the plurality of reference data units divided from the current picture by using the division shape information of each reference data unit. The operation of dividing the reference data unit may correspond to the division operation using a quadtree structure.

[0182] According to an embodiment, the image decoding device 150 may predetermine a minimum size allowed for a reference data unit included in the current picture. Therefore, the image decoding device 150 may determine various reference data units having a size equal to or larger than the minimum size, and may determine one or more coding units by using the division shape information and block shape information of the reference data unit determined with reference.

[0183] Reference Fig.14 , the image decoding device 150 may use a square reference coding unit 1400 or a non-square reference coding unit 1402. According to an embodiment, the shape and size of the reference coding unit may be determined based on various data units (e.g., a sequence, a picture, a slice, a slice segment, a maximum coding unit, etc.) that can include one or more reference coding units.

[0184] According to an embodiment, the receiver 160 of the image decoding device 150 may obtain at least one of reference coding unit shape information and reference coding unit size information about each of various data units from a bitstream. Fig.10 The operation of dividing the current coding unit 300 describes the operation of determining one or more coding units included in the square reference coding unit 1400, and the above has been described with respect to Fig.11 The operation of splitting the current coding unit 1100 or 1150 describes an operation of determining one or more coding units included in the non-square reference coding unit 1402, and therefore, a detailed description thereof will not be provided here.

[0185] According to an embodiment, the image decoding device 150 may use a PID for identifying the size and shape of a reference coding unit to determine the size and shape of the reference coding unit according to some data units previously determined based on a specific condition. That is, the receiver 160 may obtain only a PID for identifying the size and shape of a reference coding unit for each slice, slice segment, or maximum coding unit from a bitstream, wherein each slice, slice segment, or maximum coding unit is a data unit (e.g., a data unit having a size equal to or smaller than a slice) among various data units (e.g., a sequence, a picture, a slice, a slice segment, a maximum coding unit, etc.) that satisfies a specific condition. The image decoding device 150 may determine the size and shape of a reference data unit for each data unit that satisfies a specific condition by using a PID. When the reference coding unit shape information and the reference coding unit size information are obtained and used from a bitstream according to each data unit having a relatively small size, the efficiency of using the bitstream may not be high, and therefore, only the PID may be obtained and used instead of directly obtaining the reference coding unit shape information and the reference coding unit size information. In this case, at least one of the size and shape of the reference coding unit corresponding to the PID for identifying the size and shape of the reference coding unit may be predetermined. That is, the image decoding device 150 may determine at least one of the size and shape of a reference coding unit included in a data unit serving as a unit for obtaining a PID by selecting at least one of the size and shape of a predetermined reference coding unit based on the PID.

[0186] According to an embodiment, the image decoding device 150 may use one or more reference coding units included in one maximum coding unit. That is, the maximum coding unit divided from the image may include one or more reference coding units, and the coding unit may be determined by recursively dividing each reference coding unit. According to an embodiment, at least one of the width and height of the maximum coding unit may be an integer multiple of at least one of the width and height of the reference coding unit. According to an embodiment, the size of the reference coding unit may be obtained by dividing the maximum coding unit n times based on a quadtree structure. That is, according to various embodiments, the image decoding device 150 may determine the reference coding unit by dividing the maximum coding unit n times based on a quadtree structure, and may divide the reference coding unit based on at least one of the block shape information and the division shape information.

[0187] Fig.15 A processing block serving as a unit for determining an order of reference coding units included in the picture 1500 according to an embodiment is illustrated.

[0188] According to an embodiment, the image decoding device 150 may determine one or more processing blocks divided from the picture. A processing block is a data unit including one or more reference coding units divided from the image, and one or more reference coding units included in the processing block may be determined according to a specific order. That is, the determination order of the one or more reference coding units determined in each processing block may correspond to one of the various types of orders for determining the reference coding units, and may vary according to the processing block. The determination order of the reference coding units determined for each processing block may be one of various orders (e.g., raster scanning, zigzag scanning, N-shaped scanning, upper right diagonal scanning, horizontal scanning, and vertical scanning), but is not limited to the above-mentioned scanning order.

[0189] According to an embodiment, the image decoding device 150 may obtain processing block size information and may determine the size of one or more processing blocks included in the image. The image decoding device 150 may obtain processing block size information from a bitstream and may determine the size of one or more processing blocks included in the image. The size of the processing block may be a specific size of a data unit indicated by the processing block size information.

[0190] According to an embodiment, the receiver 160 of the image decoding device 150 may obtain processing block size information from the bitstream according to each specific data unit. For example, the processing block size information may be obtained from the bitstream according to a data unit such as an image, a sequence, a picture, a slice, or a slice segment. That is, the receiver 160 may obtain processing block size information from the bitstream according to each of the various data units, and the image decoding device 150 may determine the size of one or more processing blocks divided from the picture by using the obtained processing block size information, and the size of the processing block may be an integer multiple of the size of the reference coding unit.

[0191] According to an embodiment, the image decoding device 150 may determine the size of the processing blocks 1502 and 1512 included in the picture 1500. For example, the image decoding device 150 may determine the size of the processing block based on the processing block size information obtained from the bitstream. Fig.15 According to an embodiment, the image decoding device 150 may determine the width of the processing blocks 1502 and 1512 to be four times the width of the reference coding unit, and may determine the height of the processing blocks 1502 and 1512 to be four times the height of the reference coding unit. The image decoding device 150 may determine the determination order of one or more reference coding units in one or more processing blocks.

[0192] According to an embodiment, the image decoding device 150 may determine the processing blocks 1502 and 1512 included in the picture 1500 based on the size of the processing blocks, and may determine the determination order of one or more reference coding units included in the processing blocks 1502 and 1512. According to an embodiment, the operation of determining the reference coding unit may include determining the size of the reference coding unit.

[0193] According to an embodiment, the image decoding device 150 may obtain determination order information of one or more reference coding units included in one or more processing blocks from a bitstream, and may determine a determination order for one or more reference coding units based on the obtained determination order information. The determination order information may be defined as an order or direction for determining the reference coding units in the processing block. That is, the determination order of the reference coding units may be determined independently for each processing block.

[0194] According to an embodiment, the image decoding device 150 may obtain the determination order information of the reference coding unit from the bitstream according to each specific data unit. For example, the receiver 160 may obtain the determination order information of the reference coding unit from the bitstream according to each data unit (such as an image, a sequence, a picture, a slice, a slice segment, or a processing block). Because the determination order information of the reference coding unit indicates the order of the reference coding unit in the processing block, the determination order information may be obtained for each specific data unit including an integer number of processing blocks.

[0195] According to an embodiment, the image decoding device 150 may determine one or more reference coding units based on the determined determination order.

[0196] According to an embodiment, the receiver 160 may obtain the determination order information of the reference coding units from the bitstream as the information related to the processing blocks 1502 and 1512, and the image decoding device 150 may determine the determination order of one or more reference coding units included in the processing blocks 1502 and 1512, and determine one or more reference coding units included in the picture 1500 based on the determination order. Fig.15, the image decoding device 150 may determine determination orders 1504 and 1514 of one or more reference coding units in the processing blocks 1502 and 1512, respectively. For example, when determination order information of the reference coding units is obtained for each processing block, determination order information of different kinds of reference coding units may be obtained for the processing blocks 1502 and 1512. When the determination order 1504 of the reference coding units in the processing block 1502 is a raster scan order, the reference coding units included in the processing block 1502 may be determined according to the raster scan order. On the contrary, when the determination order 1514 of the reference coding units in another processing block 1512 is a backward raster scan order, the reference coding units included in the processing block 1512 may be determined according to the backward raster scan order.

[0197] Referring to Figures 1 to Fig.15 A method of dividing an image into maximum coding units and dividing the maximum coding units into coding units having a hierarchical tree structure is described. Figures 16 to 28 The ultimate motion vector expression (UMVE) mode for deriving motion vectors for inter prediction is described.

[0198] Inter prediction is a prediction method for predicting a current block from a reference block that is similar to the current block and obtained from a reference picture of the current picture. In order to inter-predict the current block, a motion vector indicating a spatial difference between the current block and the reference block and a reference picture referenced by the current picture are determined. The current block is predicted by referring to the reference block determined based on the motion vector and the reference picture. The UMVE mode of the present disclosure is an inter prediction mode for efficiently encoding and / or decoding a motion vector.

[0199] In the UMVE mode, a UMVE candidate list is determined from blocks that are spatially adjacent to the current block or blocks that are temporally adjacent to the current block. A base motion vector and a reference picture of the current block are determined according to a UMVE candidate of the current block selected from the UMVE candidate list. A motion vector of the current block is generated by correcting the base motion vector of the current block according to a correction distance and a correction direction.

[0200] Will refer to Fig.16 The inter-frame prediction according to the UMVE mode is described in detail. Fig.16 , the basic motion vector 1600 obtained from the UMVE candidate of the current block is (5, -4). However, when the reference block indicated by the basic motion vector 1600 is inaccurate, the encoding efficiency of the current block may be reduced. Therefore, in the UMVE mode, the basic motion vector 1600 may be corrected according to the correction distance 1602 and the correction direction 1604.

[0201] For example, when the correction distance 1602 is 2 and the correction direction 1604 is +x, the correction motion vector 1606 for correcting the basic motion vector 1600 is determined to be (2, 0). The result vector (7, -4) of the basic motion vector 1600 and the correction motion vector 1606 is determined as the motion vector 1608 of the current block. Therefore, the current block can be predicted by using the motion vector 1608 indicating an accurate reference block. Therefore, since the basic motion vector 1600 is corrected in the UMVE mode, the accuracy of prediction can be improved.

[0202] In the UMVE mode, in order to reduce the size of information required to correct the basic motion vector 1600, the correction distance 1602 and the correction direction 1604 have a limited number of candidates. For example, when the correction distance 1602 is determined from eight correction distance candidates, the correction distance information indicating the correction distance 1602 can be represented by using bits ranging from 1 bit to 7 bits according to the truncated unary code. Similarly, when the correction direction 1604 is determined from four correction direction candidates +x, -x, +y, and -y, the correction direction information indicating the correction direction 1604 can be represented by using 2 bits. Therefore, the number of bits required to determine the correction motion vector 1606 is limited to 9 bits. Therefore, in the UMVE mode, since the number of bits required to determine the correction motion vector 1606 is limited to a specific number or less, the compression efficiency can be improved.

[0203] Fig.17 is a block diagram of a video decoding apparatus 1700 that performs decoding according to the UMVE mode.

[0204] The video decoding apparatus 1700 includes an upper layer parameter determiner 1710, a coding mode determiner 1720, and a decoder 1730. Fig.17 The upper layer parameter determiner 1710, the encoding mode determiner 1720, and the decoder 1730 are separate elements, but according to an embodiment, the upper layer parameter determiner 1710, the encoding mode determiner 1720, and the decoder 1730 may be implemented as one element.

[0205] Although the upper layer parameter determiner 1710, the encoding mode determiner 1720, and the decoder 1730 are located Fig.17 However, the devices responsible for the functions of the upper layer parameter determiner 1710, the encoding mode determiner 1720, and the decoder 1730 do not need to be physically adjacent to each other. Therefore, according to an embodiment, the upper layer parameter determiner 1710, the encoding mode determiner 1720, and the decoder 1730 may be separated.

[0206] The upper layer parameter determiner 1710, the encoding mode determiner 1720, and the decoder 1730 may be implemented by using one processor according to an embodiment. Also, according to an embodiment, the upper layer parameter determiner 1710, the encoding mode determiner 1720, and the decoder 1730 may be implemented by using a plurality of processors.

[0207] The upper layer parameter determiner 1710 may determine whether the UMVE mode is allowed for an upper layer data unit of the current block. Examples of the upper layer data unit may include a sequence, a picture, a slice, and a slice segment of the current block. For example, the upper layer parameter determiner 1710 may determine whether the UMVE mode is allowed for a picture unit.

[0208] The upper layer parameter determiner 1710 may determine whether the UMVE mode is allowed for each upper layer data unit. For example, the upper layer parameter determiner 1710 may determine whether the UMVE mode is allowed for the current sequence. The upper layer parameter determiner 1710 may determine whether the UMVE mode is allowed for the current picture included in the current sequence. Likewise, the upper layer parameter determiner 1710 may determine whether the UMVE mode is allowed for the current slice included in the current picture.

[0209] When the UMVE mode is not allowed for the current sequence, the upper layer parameter determiner 1710 may determine that the UMVE mode is not allowed for all pictures included in the current sequence. Likewise, when the UMVE mode is not allowed for the current picture, the upper layer parameter determiner 1710 may determine that the UMVE mode is not allowed for all slices included in the current picture.

[0210] The upper layer parameter determiner 1710 obtains a UMVE enable flag indicating whether the UMVE mode is allowed for the upper layer data unit of the current block from the bitstream. When the UMVE enable flag indicates that the UMVE mode is allowed for the upper layer data unit, the coding mode determiner 1720 determines whether the UMVE mode is applied to all blocks included in the upper layer data unit. On the contrary, when the UMVE enable flag indicates that the UMVE mode is not allowed for the upper layer data unit, the coding mode determiner 1720 determines that the UMVE mode is not applied to all blocks included in the upper layer data unit.

[0211] When the UMVE enable flag is 1, the UMVE enable flag may be interpreted as allowing the UMVE mode to be used for the upper layer data unit. Conversely, when the UMVE enable flag is 0, the UMVE enable flag may be interpreted as not allowing the UMVE mode to be used for the upper layer data unit. According to an embodiment, the meaning of the UMVE enable flag may be interpreted in an opposite manner.

[0212] The upper layer parameter determiner 1710 may obtain a UMVE enable flag for each upper layer data unit. When determining whether the UMVE mode is allowed for each picture unit, the upper layer parameter determiner 1710 may obtain a UMVE enable flag for each picture unit. However, when determining whether the UMVE mode is allowed hierarchically in the order of a sequence unit, a picture unit, and a slice unit, the upper layer parameter determiner 1710 may obtain a UMVE enable flag for a sequence unit, a UMVE enable flag for a picture unit, and a UMVE enable flag for a slice unit.

[0213] In another embodiment, the UMVE enable flag may indicate whether the default setting of the UMVE mode is applied to the upper layer data unit. For example, when the UMVE enable flag is 0, the UMVE enable flag may be interpreted as applying the default setting of the UMVE mode to the upper layer data unit. On the contrary, when the UMVE enable flag is 1, the UMVE enable flag may be interpreted as not applying the default setting of the UMVE mode to the upper layer data unit. When the UMVE enable flag is not obtained, the upper layer parameter determiner 1710 may apply the default setting of the UMVE mode to the upper layer data unit.

[0214] The default setting of the UMVE mode may be to not allow the UMVE mode to be used for the upper layer data unit. Optionally, the default setting of the UMVE mode may be to apply the UMVE mode to the upper layer data unit only under certain conditions. The default setting of the UMVE mode may be set differently according to each sequence unit, picture unit, and slice unit.

[0215] The upper layer parameter determiner 1710 may determine whether to allow the UMVE mode of the upper layer data unit without the UMVE enable flag according to a specific condition. For example, when the current picture is the last picture of a group of pictures (GoP), the upper layer parameter determiner 1710 may apply a default setting of the UMVE mode to the upper layer data unit without the UMVE enable flag. As another example, the upper layer parameter determiner 1710 may apply a default setting of the UMVE mode to the upper layer data unit without the UMVE enable flag according to the temporal layer depth of the current picture.

[0216] The upper layer parameter determiner 1710 may determine whether to allow another inter prediction mode according to whether the UMVE mode is allowed for the upper layer data unit. For example, when the UMVE mode is allowed according to the UMVE enable flag, a specific inter prediction mode may not be allowed for the upper layer data unit. The specific inter prediction mode may include a decoder side motion vector derivation (DMVD) mode, a decoder side motion vector refinement (DMVR) mode, a skip mode, a direct mode, and a merge mode.

[0217] For example, in the UMVE mode, information about the motion vector is obtained from the bitstream, while in the DMVD mode or the DMVR mode, the motion vector is derived without obtaining information about the motion vector from the bitstream. Therefore, according to an embodiment, when the UMVE mode is allowed for the upper layer data unit, the upper layer parameter determiner 1710 may determine that the DMVD mode or the DMVR mode is not allowed for the upper layer data unit.

[0218] As another example, when the UMVE mode is allowed for the higher layer data unit, the higher layer parameter determiner 1710 may determine that the skip mode, the direct mode, and the merge mode for skipping information about the motion vector are not allowed.

[0219] According to another embodiment, when the UMVE mode is allowed for the upper layer data unit, the upper layer parameter determiner 1710 may obtain an enable flag of a specific inter prediction mode from a bitstream for the upper layer data unit. According to another embodiment, when the UMVE mode is allowed for the upper layer data unit, the upper layer parameter determiner 1710 may not obtain an enable flag of a specific inter prediction mode, and may apply a default setting of the specific inter prediction mode to the upper layer data unit. The specific inter prediction mode may include a decoder-side motion vector derivation (DMVD) mode, a decoder-side motion vector refinement (DMVR) mode, a skip mode, a direct mode, a merge mode, an overlapped block motion compensation (OBMC) mode, an illumination compensation (IC) mode, and an affine mode.

[0220] When the UMVE mode is allowed for the upper layer data unit, the upper layer parameter determiner 1710 may additionally obtain construction information about the UMVE mode applied to the upper layer data unit. The additional information about the UMVE mode obtained by the upper layer parameter determiner 1710 will now be described.

[0221] The upper layer parameter determiner 1710 may obtain UMVE candidate number information indicating the number of UMVE candidates allowed for a UMVE mode for an upper layer data unit from a bitstream. The decoder 1730 determines as many UMVE candidates as indicated by the UMVE candidate number information.

[0222] In addition, the higher layer parameter determiner 1710 may obtain correction distance candidate number information indicating the number of correction distance candidates allowed for the higher layer data unit from the bitstream. The decoder 1730 determines as many correction distance candidates as the number indicated by the correction distance candidate number information.

[0223] In addition, the higher layer parameter determiner 1710 may obtain correction direction candidate number information indicating the number of correction direction candidates allowed for the higher layer data unit from the bitstream. The decoder 1730 determines as many correction direction candidates as the number indicated by the correction direction candidate number information.

[0224] The UMVE candidate number information may indicate the difference between the minimum UMVE candidate number and the UMVE candidate number used in the upper layer data unit. For example, when the minimum UMVE candidate number is 4 and the UMVE candidate number used in the upper layer data unit is 5, the UMVE candidate number information may indicate 1. Similar to the UMVE candidate number information, the correction distance candidate number information and the correction direction candidate number information may also indicate the difference between the minimum number and the candidate number used in the upper layer data unit.

[0225] The upper layer parameter determiner 1710 may obtain UMVE candidate list construction information indicating a method of constructing a UMVE candidate list from a bitstream. The UMVE candidate list construction information may indicate that a UMVE candidate list of a UMVE mode is constructed based on a motion vector candidate list used in a merge mode.

[0226] Optionally, the UMVE candidate list construction information may indicate the construction of a motion vector candidate list used only for the UMVE mode. For example, the motion vector candidate list used only for the UMVE mode may be constructed by statistically analyzing the motion vector information used to decode the current picture. When the left block and the upper block of the current block may be selected as UMVE candidates, the left block and the upper block of the current block may be preferentially included in the UMVE candidate list.

[0227] Alternatively, the UMVE candidate list construction information may indicate construction of a motion vector candidate list generated by combining a motion vector candidate list used in a merge mode with a motion vector candidate list used only in a UMVE mode.

[0228] When the UMVE candidate list construction information cannot be obtained or the UMVE candidate list is inherently determined, the upper layer parameter determiner 1710 selects a UMVE candidate list construction method according to a default setting.

[0229] The higher layer parameter determiner 1710 may obtain correction distance range information of a plurality of blocks of the higher layer data unit from a bitstream. The decoder 1730 determines a correction distance of a base motion vector based on a correction distance range indicated by the correction distance range information.

[0230] For example, the correction distance range information may indicate a minimum value of the correction distance candidates. When the minimum value of the correction distance candidates indicated by the correction distance range information is 1 and the number of correction distance candidates indicated by the correction distance number candidate information is 4, the correction distance candidates may be determined as {1, 2, 4, 8}.

[0231] In addition, the correction distance range information may indicate the size difference between the correction distance candidates. For example, the correction distance range information may indicate whether the value of the correction distance candidate increases arithmetically or exponentially. When the value of the correction distance candidate increases arithmetically, the correction distance candidate may be determined as {1, 2, 3, 4, 5, ...}. When the value of the correction distance candidate increases exponentially, the correction distance candidate is determined as {1, 2, 4, 8, 16, ...}.

[0232] In addition, the correction distance range information may indicate one of a plurality of predetermined correction distance candidate sets. For example, the correction distance range information may indicate one of the first correction distance candidate set {1,2,4,8}, the second correction distance candidate set {1,2,3,4} and the third correction distance candidate set {4,8,16,32}.

[0233] The upper layer parameter determiner 1710 may obtain correction distance change information of a plurality of blocks of the upper layer data unit from the bitstream. The correction distance change information indicates whether the range of the correction distance candidate is changed according to the temporal distance between the current picture and the reference picture. Therefore, when the correction distance change information indicates that the range of the correction distance candidate is changed, the decoder 1730 may determine the range of the correction distance candidate according to the temporal distance between the current picture and the reference picture.

[0234] Generally speaking, when the current block references a reference block of a reference picture that is temporally far away from the current picture, the error of the basic motion vector may be high. Conversely, when the current block references a reference block of a reference picture that is temporally close to the current picture, the error of the basic motion vector may be low. Therefore, when the correction distance candidate set is determined in proportion to the temporal distance between the current picture and the reference picture, the prediction accuracy of the motion vector can be improved.

[0235] For example, when the temporal distance between the current picture and the reference picture is small, the minimum value of the correction distance candidate may be determined as 1 / 2. When the temporal distance between the current picture and the reference picture is medium, the minimum value of the correction distance candidate may be determined as 1. When the temporal distance between the current picture and the reference picture is large, the minimum value of the correction distance candidate may be determined as 2.

[0236] The temporal distance between the current picture and the reference picture may be determined according to one or more thresholds. For example, when the number of thresholds is 3, the minimum value of the correction distance candidate may be determined by comparing the temporal distance between the current picture and the reference picture with three thresholds.

[0237] The upper layer parameter determiner 1710 may determine the minimum value of the correction distance candidate of the upper layer data unit according to the temporal distance between the current picture and the reference picture and the correction distance candidate minimum value of the correction distance range information. In detail, since the correction distance candidate minimum value of the correction distance range information changes according to the temporal distance between the current picture and the reference picture, the minimum value of the correction distance candidate of the upper layer data unit may be determined.

[0238] When the correction distance change information cannot be obtained or whether the correction distance is changed is inherently determined, the upper layer parameter determiner 1710 determines whether the correction distance is changed according to a default setting.

[0239] The upper layer parameter determiner 1710 may obtain prediction direction change permission information of a plurality of blocks of an upper layer data unit from a bitstream. The current block obtains information on a prediction direction of a reference picture and information on a basic motion vector from a UMVE candidate selected from a UMVE candidate list. When the prediction direction is not allowed to be changed according to the prediction direction change permission information, the prediction direction of the UMVE candidate is applied to the current block. However, on the contrary, when the prediction direction is allowed to be changed according to the prediction direction change permission information, a prediction direction different from the prediction direction of the UMVE candidate may be applied to the current block.

[0240] The prediction direction indicates one of the following: list 0 unidirectional prediction, list 1 unidirectional prediction, and bidirectional prediction using both list 0 and list 1. Although only one reference block of list 0 or list 1 is used to predict the current block in unidirectional prediction, both the reference block of list 0 and the reference block of list 1 are used in bidirectional prediction. Therefore, the accuracy of bidirectional prediction may be higher than that of unidirectional prediction. Therefore, when the prediction direction is allowed to be changed, even if the UMVE candidate indicates unidirectional prediction, the prediction direction of the UMVE mode may be determined as bidirectional prediction, thereby improving the encoding efficiency according to the UMVE mode.

[0241] Therefore, when the prediction direction is allowed to be changed according to the prediction direction change permission information, the decoder 1730 may change the prediction direction of the current block. For example, when the prediction direction is allowed to be changed, even if the prediction direction of the UMVE candidate is list 0 unidirectional prediction, the prediction direction of the current block may be determined as list 1 unidirectional prediction or bidirectional prediction.

[0242] The upper layer parameter determiner 1710 may obtain main prediction direction information of the upper layer data unit from the bitstream. The main prediction direction information indicates whether there is a main prediction direction of the upper layer data unit. In addition, when there is a main prediction direction of the upper layer data unit, the main prediction direction information indicates the main prediction direction of the upper layer data unit. When the main prediction direction of the upper layer data unit is set, a block included in the upper layer data unit and to which the UMVE mode is applied is predicted according to the main prediction direction. For example, when the main prediction direction is determined to be a bidirectional prediction, a block to which the UMVE mode is applied may be bidirectionally predicted. Therefore, even when the prediction direction of the UMVE candidate is a unidirectional prediction, the prediction direction of the current block may be determined to be a bidirectional prediction.

[0243] The upper layer parameter determiner 1710 may obtain both the prediction direction change permission information and the main prediction direction information. When the prediction direction change permission information indicates that the prediction direction is allowed to be changed and the main prediction direction information indicates that the main prediction direction is set, the prediction direction information obtained by the decoder 1730 indicates whether the prediction direction of the current block is the main prediction direction. For example, when the main prediction direction is a bidirectional prediction and the prediction direction information indicates that the prediction direction of the current block is the main prediction direction, the prediction direction of the current block is determined to be a bidirectional prediction. When the main prediction direction is a bidirectional prediction and the prediction direction information indicates that the prediction direction of the current block is not the main prediction direction, the prediction direction of the current block is determined as a list 0 unidirectional prediction or a list 1 unidirectional prediction as an auxiliary prediction direction according to the prediction direction information.

[0244] The upper layer parameter determiner 1710 may obtain multiple correction permission information indicating whether the corrected basic motion vector may be corrected multiple times from the bitstream. When the basic motion vector may be corrected multiple times according to the multiple correction permission information, the decoder 1730 may correct the basic motion vector multiple times.

[0245] When the UMVE mode is allowed for the upper layer data unit, the coding mode determiner 1720 determines whether the UMVE mode is applied to the current block. The coding mode determiner 1720 may obtain a UMVE flag indicating whether the UMVE mode is applied to the current block from the bitstream. For example, when the UMVE flag indicates that the UMVE mode is applied to the current block, the coding mode determiner 1720 may determine that the prediction mode of the current block is the UMVE mode. On the contrary, when the UMVE flag indicates that the UMVE mode is not applied to the current block, the coding mode determiner 1720 determines that the prediction mode of the current block is not the UMVE mode.

[0246] When the UMVE mode is applied to the current block, the encoding mode determiner 1720 may determine that another inter prediction mode is not applied to the current block. For example, when the UMVE mode is applied to the current block according to the UMVE flag, a specific inter prediction mode may not be applied to the current block. The specific inter prediction mode may include a DMVD mode, a DMVR mode, a skip mode, a direct mode, a merge mode, an OBMC mode, an IC mode, and an affine mode. Conversely, when the UMVE mode is applied to the current block, the encoding mode determiner 1720 may determine that a specific prediction mode is applied to the current block.

[0247] In addition, when the UMVE mode is not applied to the current block, the encoding mode determiner 1720 may determine that another intra prediction mode is to be applied to the current block. Examples of another intra prediction mode may include DMVD mode, DMVR mode, merge mode, OBMC mode, IC mode, and affine mode.

[0248] When the UMVE mode is applied to the current block, the decoder 1730 may determine a basic motion vector of the current block from the UMVE candidates. The decoder 1730 may determine a UMVE candidate list according to a UMVE candidate list construction method according to UMVE candidate list construction information. The number of UMVE candidates included in the UMVE candidate list may be determined according to the UMVE candidate number information.

[0249] The decoder 1730 may obtain a UMVE index from the bitstream, the UMVE index indicating a UMVE candidate in the UMVE candidate list including a reference picture and a base motion vector of the current block. The decoder 1730 may determine a reference picture and a base motion vector of the current block according to the UMVE index.

[0250] The decoder 1730 may determine a correction distance and a correction direction for correction of a basic motion vector. In the UMVE mode, the basic motion vector is corrected for prediction accuracy. Therefore, in order to determine a correction motion vector for correcting the basic motion vector, the decoder 1730 determines a correction distance indicating the size of the correction motion vector and a correction direction indicating the direction of the correction motion vector.

[0251] The decoder 1730 may determine the correction distance candidates based on the correction distance candidate quantity information, the correction distance range information, and the correction distance change information of the upper layer data unit. The decoder 1730 may determine the number of correction distance candidates based on the correction distance candidate quantity information. The decoder 1730 may determine the size difference between the correction distance candidates and the minimum value of the correction distance candidates based on the correction distance range information. In addition, when the correction distance change information indicates a change in the range of the correction distance candidates, the decoder 1730 may determine the range of the correction distance candidates based on the distance between the current picture and the reference picture. For example, the minimum value of the correction distance candidates may be changed by comparing the distance between the current picture and the reference picture with a specific threshold. Therefore, the range of the correction distance candidates may be determined based on the changed minimum value.

[0252] The decoder 1730 may obtain correction distance information indicating the correction distance of the basic motion vector from the bitstream. The correction distance information indicates a correction distance candidate corresponding to the correction distance of the basic motion vector among a specific number of correction distance candidates. The decoder 1730 may determine the correction distance of the basic motion vector according to the correction distance candidate indicated by the correction distance information.

[0253] The decoder 1730 may determine the correction direction candidate according to the correction direction candidate number information of the higher layer data unit. The decoder 1730 determines the correction direction from a specific number of correction direction candidates.

[0254] The decoder 1730 may obtain correction direction information indicating a correction direction of a basic motion vector from a bitstream. The correction direction information indicates a correction direction candidate corresponding to the correction direction of the basic motion vector among a specific number of correction direction candidates. The decoder 1730 may determine the correction direction of the basic motion vector according to the correction direction candidate indicated by the correction direction information.

[0255] The decoder 1730 may determine the motion vector of the current block by correcting the basic motion vector according to the determined correction distance and the determined correction direction. For example, when the correction distance is 2 and the correction direction is +x, the motion vector obtained by increasing the x value from the basic motion vector by 2 is used for prediction of the current block.

[0256] Figures 18 to 21 is a diagram for describing a process of determining a motion vector of a current block in UMVE mode. Fig.18 and Fig.21 In the example, the coordinates of the basic motion vector are set to (base_x, base_y). The correction motion vector describing the correction distance and correction direction according to the UMVE mode is based on (base_x, base_y).

[0257] Reference Fig.18, the decoder 1730 may determine a motion vector candidate having a diamond distribution based on the basic motion vector.

[0258] The decoder 1730 may determine the motion vector candidates (base_x+1 / 4, base_y) 1802, (base_x-1 / 4, base_y) 1803, (base_x, base_y+1 / 4) 1804, and (base_x, base_y-1 / 4) 1805 which are 1 / 4 pixel away from the basic motion vector (base_x, base_y) 1801 as the first candidate group.

[0259] The decoder 1730 may determine the motion vector candidates (base_x+1 / 2, base_y) 1806, (base_x-1 / 2, base_y) 1807, (base_x, base_y+1 / 2) 1808, and (base_x, base_y-1 / 2) 1809 which are 1 / 2 pixel away from the basic motion vector (base_x, base_y) 1801 as the second candidate group.

[0260] The decoder 1730 may determine the motion vector candidates (base_x+1, base_y) 1810, (base_x-1, base_y) 1811, (base_x, base_y+1) 1812, and (base_x, base_y-1) 1813 which are 1 pixel away from the basic motion vector (base_x, base_y) 1801 as the third candidate group.

[0261] The decoder 1730 may select one candidate group from the first candidate group to the third candidate group according to the correction distance. The decoder 1730 may determine one motion vector candidate from the candidate group as the motion vector of the current block according to the correction direction.

[0262] Reference Fig.19 , the decoder 1730 may determine prediction motion vector candidates having a quadrilateral distribution based on the basic motion vector.

[0263] Similarly, the decoder 1730 may determine the motion vector candidates (base_x+1 / 4, base_y+1 / 4)1902, (base_x+1 / 4, base_y-1 / 4)1903, (base_x-1 / 4, base_y+1 / 4)1904 and (base_x-1 / 4, base_y-1 / 4)1905 which are 1 / 4 pixel away from the basic motion vector (base_x, base_y) 1901 as the first candidate group.

[0264] The decoder 1730 may determine the motion vector candidates (base_x+1 / 2, base_y+1 / 2)1906, (base_x+1 / 2, base_y-1 / 2)1907, (base_x-1 / 2, base_y+1 / 2)1908 and (base_x-1 / 2, base_y-1 / 2)1909 which are 1 / 2 pixel away from the basic motion vector (base_x, base_y) 1901 as the second candidate group.

[0265] The decoder 1730 may determine the motion vector candidates (base_x+1, base_y+1)1910, (base_x+1, base_y-1)1911, (base_x-1, base_y+1)1912 and (base_x-1, base_y-1)1913 which are 1 pixel away from the basic motion vector (base_x, base_y) 1901 as the third candidate group.

[0266] Reference Fig. 20 , the decoder 1730 can determine each group by using a different number of motion vector candidates. Although the interval between pixels is 1 / 4 pixel distance, for convenience, the component values ​​of the vector candidates are scaled four times.

[0267] In detail, the decoder 1730 can determine eight motion vector candidates (base_x+1 / 4, base_y / 4)2002, (base_x-1 / 4, base_y)2003, (base_x, base_y+1 / 4)2004, (base_x, base_y-1 / 4)2005, (base_x+1 / 4, base_y+1 / 4)2006, (base_x+1 / 4, base_y-1 / 4)2007, (base_x-1 / 4, base_y+1 / 4)2008 and (base_x-1 / 4, base_y-1 / 4)2009 which are 1 / 4 pixel away from the basic motion vector as the first candidate group.

[0268] In addition, the decoder 1730 can determine eight motion vector candidates (base_x+1 / 2, base_y)2010, (base_x-1 / 2, base_y)2011, (base_x, base_y+1 / 2)2012, (base_x, base_y-1 / 2)2013, (base_x+1 / 2, base_y+1 / 2)2014, (base_x+1 / 2, base_y-1 / 2)2015, (base_x-1 / 2, base_y+1 / 2)2016 and (base_x-1 / 2, base_y-1 / 2)2017 which are 1 / 2 pixel away from the basic motion vector as the second candidate group.

[0269] The decoder 1730 can determine the eight motion vector candidates (base_x+1, base_y)2018, (base_x-1, base_y)2019, (base_x, base_y+1)2020, (base_x, base_y-1)2021, (base_x+1, base_y+1)2022, (base_x+1, base_y-1)2023, (base_x-1, base_y+1)2024 and (base_x-1, base_y-1)2025 which are 1 pixel away from the basic motion vector as the third candidate group.

[0270] Reference Fig.21 , the decoder 1730 may determine the distribution shape of the motion vector candidates included in the candidate group according to the candidate group in various ways. In detail, the decoder 1730 may determine the motion vector candidates 2102, 2103, 2104, and 2105 having a diamond distribution based on the basic motion vector 2101 as the first candidate group. In addition, the decoder 1730 may determine the motion vector candidates 2106, 2107, 2108, and 2109 having a quadrilateral distribution based on the basic motion vector 2101 as the second candidate group. In addition, the decoder 1730 may determine the motion vector candidates 2110, 2111, 2112, and 2113 having a diamond distribution based on the basic motion vector 2101 as the third candidate group. Fig.21 As shown, except Fig.21 In addition to the distribution shape of , the distribution shape of the motion vector candidates of each candidate group can be determined as any distribution shape among various distribution shapes.

[0271] The decoder 1730 may determine one or more basic motion vectors. When there are two basic motion vectors, a motion vector candidate may be generated by using each basic motion vector.

[0272] The decoder 1730 may perform bidirectional prediction. When bidirectional prediction is performed on a basic motion vector by using reference pictures of list 0 and list 1, a motion vector of each prediction direction may be corrected according to a temporal distance between a current picture and a reference picture.

[0273] For example, when the reference picture of list 0 and the reference picture of list 1 are located in the same direction as the current picture, the correction direction of the basic motion vector of list 0 and the basic motion vector of list 1 are determined to be the same. Conversely, when the reference picture of list 0 and the reference picture of list 1 are located in the opposite direction to the current picture, the correction direction of the basic motion vector of list 0 is determined to be opposite to the correction direction of the basic motion vector of list 1.

[0274] Furthermore, the correction distance of the basic motion vector of list 0 and the correction distance of the basic motion vector of list 1 are determined in proportion to the temporal distance between the current picture and the reference picture of list 0 and the temporal distance between the current picture and the reference picture of list 1. Therefore, when the temporal distance between the current picture and the reference picture of list 0 is twice the temporal distance between the current picture and the reference picture of list 1, the correction distance of the basic motion vector of list 0 is determined to be twice the correction distance of the basic motion vector of list 0.

[0275] The correction distance and correction direction indicated by the correction distance information and correction direction information obtained from the bitstream may be applied to the basic motion vector of list 0.

[0276] The correction distance and correction direction for correcting the basic motion vector of list 0 may be changed in consideration of the distance between the current picture and the reference picture of list 0 and the distance between the current picture and the reference picture of list 1. The changed correction distance and the changed correction direction may be applied to the basic motion vector of list 1.

[0277] According to an embodiment, instead, the correction distance and the correction direction indicated by the correction distance information and the correction direction information may be applied to the basic motion vector of list 1, and the changed correction distance and the changed correction direction may be applied to the basic motion vector of list 0.

[0278] Fig. 22 A method for correcting two basic motion vectors according to bidirectional prediction according to an embodiment is shown. Fig. 22 In the example, the basic motion vector 2202 of list 0 is (0, -1). The basic motion vector 2212 of list 1 is (0, 2).

[0279] The correction distance and correction direction in the UMVE mode are applied to both the basic motion vector 2202 of list 0 and the basic motion vector 2212 of list 1. When the correction distance of the basic motion vector 2202 of list 0 is 1 and the correction direction of the basic motion vector 2202 is -y, the motion vector 2204 of list 0 becomes (0, -2).

[0280] The reference picture of list 0 and the reference picture of list 1 are located in the opposite direction to the current picture. Therefore, the correction direction of the basic motion vector 2212 of list 1 is determined to be opposite to the correction direction of the basic motion vector 2202 of list 0. Therefore, the correction direction of the basic motion vector 2212 of list 1 is determined to be +y.

[0281] The reference picture of list 0 and the reference picture of list 1 are located at the same temporal distance from the current picture. Therefore, the correction distance of the basic motion vector 2212 of list 1 is determined to be the same as the correction distance of the basic motion vector 2202 of list 0. Therefore, the correction distance of the basic motion vector 2212 of list 1 is determined to be 1.

[0282] Therefore, the motion vector 2214 of list 1 is determined to be (0, 3) whose y component is 1 greater than the y component of the basic motion vector 2212 of list 1.

[0283] When the correction distance of the basic motion vector 2202 of list 0 is 2, the motion vector 2206 of list 0 is determined to be (0, -3), and the motion vector 2216 of list 1 is determined to be (0, 4). In addition, when the correction distance of the basic motion vector 2202 of list 0 is 4, the motion vector 2208 of list 0 is determined to be (0, -5), and the motion vector 2218 of list 1 is determined to be (0, 6).

[0284] When the prediction direction is allowed to be changed according to the prediction direction change permission information, the decoder 1730 may obtain the prediction direction information from the bitstream, and may change the prediction direction of the current block according to the prediction direction information. The prediction direction information may indicate whether to change the prediction direction. When the prediction direction information does not indicate changing the prediction direction, the current block is predicted according to the prediction direction of the UMVE candidate. However, when the prediction direction information indicates changing the prediction direction, the current block is predicted according to a prediction direction different from the prediction direction of the UMVE candidate.

[0285] In addition, the prediction direction information may indicate the changed prediction direction. For example, when the prediction direction of the UMVE candidate is list 0 unidirectional prediction, the prediction direction information may indicate one of list 1 unidirectional prediction and bidirectional prediction. Therefore, the prediction direction information may indicate whether the prediction direction is changed and the changed prediction direction.

[0286] When the prediction direction does not change, the prediction direction information may be set to 0. When the prediction direction changes, the prediction direction information may be set to 10 or 11 according to the changed prediction direction. For example, when the prediction direction of the UMVE candidate is list 0 unidirectional prediction and the prediction direction of the current block is list 0 unidirectional prediction, the prediction direction information may be set to 0.

[0287] When the prediction direction of the current block is bidirectional prediction or list 1 unidirectional prediction, the prediction direction information may be set to 10 or 11. According to an embodiment, when the prediction direction of the UMVE candidate is unidirectional prediction and the prediction direction of the current block is bidirectional prediction, the prediction direction information may be set to 10. According to an embodiment, when the prediction direction of the UMVE candidate is unidirectional prediction and the prediction direction of the current block is unidirectional prediction of another direction, the prediction direction information may be set to 11.

[0288] As another example, when the prediction direction of the UMVE candidate is list 1 unidirectional prediction and the prediction direction of the current block is list 1 unidirectional prediction, the prediction direction information may be set to 0.

[0289] When the prediction direction of the current block is bidirectional prediction or list 0 unidirectional prediction, the prediction direction information may be set to 10 or 11. According to an embodiment, when the prediction direction of the UMVE candidate is unidirectional prediction and the prediction direction of the current block is bidirectional prediction, the prediction direction information may be set to 10. According to an embodiment, when the prediction direction of the UMVE candidate is unidirectional prediction and the prediction direction of the current block is unidirectional prediction of another direction, the prediction direction information may be set to 11.

[0290] As another example, when the prediction direction of the UMVE candidate is bi-directional prediction and the prediction direction of the current block is bi-directional prediction, the prediction direction information may be set to 0.

[0291] When the prediction direction of the current block is list 0 unidirectional prediction or list 1 unidirectional prediction, the prediction direction information may be set to 10 or 11. According to an embodiment, when the prediction direction of the UMVE candidate is bidirectional prediction and the prediction direction of the current block is list 0 unidirectional prediction, the prediction direction information may be set to 10. According to an embodiment, when the prediction direction of the UMVE candidate is bidirectional prediction and the prediction direction of the current block is list 1 unidirectional prediction, the prediction direction information may be set to 11.

[0292] When the prediction direction information indicates that the prediction direction is changed, the decoder 1730 may change the motion vector of the UMVE candidate according to the changed prediction direction. For example, when the prediction direction of the UMVE candidate is list 0 unidirectional prediction and the prediction direction of the current block is list 1 unidirectional prediction, the motion vector of list 0 is corrected to be suitable for prediction according to the reference picture of list 1. In detail, the motion vector of list 0 may be modified by scaling the motion vector of list 0 according to the ratio between the temporal distance between the current picture and the reference picture of list 0 and the temporal distance between the current picture and the reference picture of list 1.

[0293] For example, when the motion vector of list 0 is (8, 6) and the ratio between the temporal distance between the current picture and the reference picture of list 0 and the temporal distance between the current picture and the reference picture of list 0 is 2: 1, the motion vector of list 1 may be determined as (4, 3). When the ratio between the temporal distances is 2: -1, that is, when the reference picture of list 1 and the reference picture of list 0 are located in a different direction from the current picture, the motion vector of list 1 may be determined as (-4, -3).

[0294] When the prediction direction of the UMVE candidate is list 0 unidirectional prediction and the prediction direction of the current block is bidirectional prediction, the motion vector of list 1 may be obtained based on the motion vector of list 0 of the UMVE candidate. The motion vector of list 1 may be obtained by scaling the motion vector of the reference picture of list 0 according to a ratio between a temporal distance between the current picture and the reference picture of list 0 and a temporal distance between the current picture and the reference picture of list 1. The reference picture of list 1 may be determined as a picture located at a specific order of list 1. The motion vector and reference picture of list 0 and the motion vector and reference picture of list 1 may all be used for prediction of the current block.

[0295] Likewise, even when the prediction direction of the UMVE candidate is list 1 unidirectional prediction and the prediction direction of the current block is bidirectional prediction, the motion vector of list 0 may be obtained from the motion vector of list 1. The reference picture of list 0 may be determined as a picture located at a specific order of list 0. The motion vector and reference picture of list 0 and the motion vector and reference picture of list 1 may all be used for prediction of the current block.

[0296] When the prediction direction of the UMVE candidate is bidirectional prediction and the prediction direction of the current block is list 0 unidirectional prediction, the current block is predicted by using only the motion vector and reference picture of list 0 and the motion vector and reference picture of list 1. On the contrary, when the prediction direction before the change is bidirectional prediction and the prediction direction after the change is list 1 unidirectional prediction, the current block is predicted by using only the motion vector and reference picture of list 0 and the motion vector and reference picture of list 1.

[0297] When the main prediction direction information is obtained, the decoder 1730 may change the motion vector of the UMVE candidate according to the main prediction indication indicated by the main prediction direction information. When the main prediction direction of the current picture and the reference prediction direction of the UMVE candidate are the same, the current block is predicted according to the motion vector and the reference picture of the UMVE candidate. However, when the main prediction direction of the current picture and the reference prediction direction of the UMVE candidate are different from each other, the motion vector of the UMVE candidate is changed according to the main prediction direction. Changing the motion vector and the reference picture of the UMVE candidate according to the main prediction direction is performed in the same manner as changing the motion vector and the reference picture of the UMVE candidate according to the prediction direction information.

[0298] According to an embodiment, when the main prediction direction information is obtained and the prediction direction is allowed to be changed according to the prediction direction change permission information, the decoder 1730 may change the prediction direction of the current block according to the prediction direction information. When the main prediction direction information is set, the prediction direction information indicates the prediction direction of the current block from the main prediction direction and the two auxiliary prediction directions. When the prediction direction information indicates the main prediction direction, the current block is predicted according to the main prediction direction. However, when the prediction direction information indicates the auxiliary prediction direction, the current block is predicted according to the prediction direction indicated by the prediction direction information in the two auxiliary prediction directions. When the main prediction direction information is set, in the prediction direction information, 0 may be assigned to the main prediction direction, 10 may be applied to the first auxiliary prediction direction, and 11 may be assigned to the second auxiliary prediction direction. The two auxiliary prediction directions for the main prediction direction may be determined by using various methods according to an embodiment.

[0299] When the basic motion vector can be corrected multiple times according to the multi-correction permission information, the decoder 1730 can correct the corrected basic motion vector again. Therefore, the decoder 1730 can obtain the additional correction distance information and the additional correction direction information from the bitstream. The decoder 1730 can correct the basic motion vector again based on the additional correction distance determined according to the additional correction distance information and the additional correction direction determined according to the additional correction direction information.

[0300] When the correction distance is equal to or greater than the threshold, the decoder 1730 may obtain additional correction distance information and additional correction direction information. The additional correction distance and the additional correction direction indicated by the additional correction distance information and the additional correction direction information, respectively, may be limited according to the correction distance and the correction direction. For example, the additional correction distance may be determined to be less than the correction distance. Alternatively, the additional correction direction may be determined to be different from the correction direction.

[0301] The decoder 1730 may reconstruct the current block based on the motion vector and the reference picture of the current block. When the prediction direction of the current block is list 0 unidirectional prediction or list 1 unidirectional prediction, the decoder 1730 may predict the current block by using one motion vector and one reference picture. When the prediction direction of the current block is bidirectional prediction, the decoder 1730 may predict the current block by using the motion vector and the reference picture of list 0 and the motion vector and the reference picture of list 1.

[0302] The decoder 1730 may reconstruct the current block according to the prediction result of the current block. The decoder 1730 may reconstruct the current block according to the residual block and the prediction block of the current block. The residual block of the current block indicates the difference between the prediction block and the reconstructed block of the current block.

[0303] UMVE candidate number information, correction distance candidate number information, correction direction candidate number information, UMVE candidate list construction information, correction distance range information, main prediction direction information, UMVE index, correction distance information and correction direction information, prediction direction information, additional correction distance information, and additional correction direction information of the UMVE candidates decoded by the video decoding device 1700 can be binarized according to fixed-length coding or truncated unary coding.

[0304] can be Figure 1B The decoder 170 performs functions of a higher layer parameter determiner 1710 , a coding mode determiner 1720 , and a decoder 1730 .

[0305] Figure 23 to Figure 25 A method of selecting an inter mode tool for a reference picture and a motion vector of a current block according to an embodiment is illustrated. Figure 23 to Figure 25 Each shows a syntax structure of a coding unit. According to the syntax structure based on an embodiment, when the skip mode or the direct mode is applied to the current block, the UMVE mode may be selected. Figure 23 to Figure 25 ae(v) in represents entropy decoding of syntax elements in the same row. Therefore, Fig.23 cu_skip_flag[x0][y0] and cu_skip_umve[x0][y0], Fig.24 cu_direct[x0][y0] and affine_dir[x0][y0] and Fig.25cu_direct[x0][y0] and cu_direct_umve[x0][y0] are entropy decoded.

[0306] Fig.23 A method of selecting an inter-mode tool for a current block in a skip mode is shown in FIG. In the skip mode, most pieces of encoding information required to reconstruct the current block are skipped, and the current block is reconstructed by using only a few pieces of encoding information. Therefore, since the encoding information required to reconstruct the current block is skipped, encoding efficiency can be improved.

[0307] according to Fig.23 According to an embodiment, an inter-mode tool may be used in skip mode. According to an embodiment, in skip mode, umve mode, affine mode, and merge mode may be used. According to an embodiment, cu_skip_flag[x0][y0] indicating whether sip mode is applied to the current block is obtained. When skip mode is applied to the current block (if(cu_skip_flag[x0][y0])), cu_skip_umve[x0][y0] indicating whether the inter-mode tool of the current block is umve mode is obtained. When umve mode is applied to the current block (if(cu_skip_umve[x0][y0]), cu_mode=SKIP_UMVE), information about neighboring blocks including a reference picture and a basic motion vector of the current picture and correction distance information and correction direction information about correcting the basic motion vector are obtained (mvp_idx_umve(x0,y0,identical_list0_list1 flag)). When the umve mode is not applied to the current block, the affine mode (cu_mode=SKIP_AFFINE) or the merge mode (cu_mode=SKIP) may be applied to the current block.

[0308] exist Fig.23 In the embodiment of the invention, whether to apply the mode is determined in the order of umve mode, affine mode and merge mode. However, a person skilled in the art can easily change the type of inter-frame mode tool used in skip mode. In addition, a person skilled in the art can change the order of determining the inter-frame mode tools used in skip mode.

[0309] Fig.24 A method of selecting an inter-mode tool of a current block in a direct mode when a slice type of a current slice including the current block is a P type is shown. In the direct mode, several pieces of encoding information required to determine a motion vector of the current block are skipped. Therefore, since the encoding information required to determine the motion vector of the current block is reduced, encoding efficiency can be improved.

[0310] according to Fig.24In an embodiment, when the slice type of the current slice is the P type, the umve mode and the affine mode may be used for the direct mode. According to an embodiment, cu_direct[x0][y0] indicating whether the direct mode is applied to the current block is obtained. When the direct mode is applied to the current block (if(cu_direct[x0][y0])), affine_dir[x0][y0] indicating whether the inter mode tool of the current block is the affine mode is obtained. When the affine mode is not applied to the current block (if(!affine_dir[x0][y0])), it is determined that the umve mode is applied to the current block (cu_mode=DIR_UMVE). When the umve mode is applied to the current block, information about the neighboring blocks including the reference picture and the basic motion vector of the current picture and the correction distance information and the correction direction information about the correction basic motion vector are obtained (mvp_idx_umve(x0, y0, identical_list0_list1 flag)).

[0311] exist Fig.24 In the embodiment of the present invention, whether to apply the mode is determined in the order of the affine mode and the umve mode. However, a person skilled in the art can easily change the type of inter-frame mode tools used in the direct mode. In addition, a person skilled in the art can change the order in which the inter-frame mode tools used in the direct mode are determined.

[0312] Fig.25 A method of selecting an inter mode tool of a current block in a direct mode when a slice type of a current slice including the current block is type B is shown.

[0313] according to Fig.25In an embodiment, when the slice type of the current slice is type B, the umve mode, the affine mode, and the merge mode may be used. According to an embodiment, cu_direct[x0][y0] indicating whether the direct mode is applied to the current block is obtained. When the direct mode is applied to the current block (if (cu_direct[x0][y0])), cu_direct_umve[x0][y0] indicating whether the inter mode tool of the current block is the umve mode is obtained. When the umve mode is applied to the current block (if (cu_direct_umve[x0][y0]), cu_mode=DIR_UMVE), information about neighboring blocks including a reference picture and a basic motion vector of the current picture and correction distance information and correction direction information about correcting the basic motion vector are obtained (mvp_idx_umve(x0, y0, identical_list0_list1flag)). When the umve mode is not applied to the current block, the affine mode (cu_mode=DIR_AFFINE) or the merge mode (cu_mode=DIR) may be applied to the current block.

[0314] exist Fig.25 In the embodiment of the present invention, whether to apply the mode is determined in the order of umve mode, affine mode and merge mode. However, a person skilled in the art can easily change the type of inter-frame mode tools used in direct mode. In addition, a person skilled in the art can change the order in which the inter-frame mode tools used in direct mode are determined.

[0315] Fig.26 is based on Fig.17 Flowchart of a decoding method of a UMVE mode of a video decoding apparatus 1700 .

[0316] In operation 2610, it is determined whether the UMVE mode is allowed for the upper layer data unit including the current block. Whether the UMVE mode is allowed for the upper layer data unit may be determined according to a UMVE enable flag obtained from a bitstream.

[0317] When the UMVE mode is allowed to be used for the upper layer data unit, UMVE candidate number information and UMVE candidate list construction information can be obtained, wherein the UMVE candidate number information indicates the number of UMVE candidates of the UMVE mode allowed to be used for the upper layer data unit, and the UMVE candidate list construction information indicates a method of constructing a motion vector candidate list referenced by multiple blocks of the upper layer data unit.

[0318] Correction distance candidate number information indicating the number of correction distance candidates may be obtained from the bitstream. Correction distance range information of a plurality of blocks of the upper layer data unit may be obtained. Furthermore, correction distance change information of a plurality of blocks of the upper layer data unit may be obtained.

[0319] Correction direction candidate number information indicating the number of correction direction candidates may be obtained from a bitstream.

[0320] Prediction direction change permission information of a plurality of blocks of the higher layer data unit may be obtained from a bitstream. Main prediction direction information indicating a main prediction direction of the higher layer data unit may be obtained from a bitstream.

[0321] In operation 2620, when the UMVE mode is allowed for the upper layer data unit, it is determined whether the UMVE mode is applied to the current block. Whether the UMVE mode is applied to the current block may be determined according to a UMVE flag obtained from a bitstream. When the UMVE mode is not applied to the current block, an inter prediction mode other than the UMVE mode may be applied to the current block.

[0322] The number of UMVE candidates of the current block indicated by the UMVE candidate number information may be determined. A method for constructing the UMVE candidate list may be determined according to the UMVE candidate list construction information. The UMVE candidate list of the current block may be constructed according to the number of UMVE candidates and the method for constructing the UMVE candidate list.

[0323] In addition, the number of correction distance candidates of the current block indicated by the correction distance candidate number information may be determined. The correction distance candidates may be determined based on the correction distance range indicated by the correction distance range information. In addition, when the correction distance change information indicates that the correction distance of the basic motion vector is adaptively changed according to the distance between the current picture of the current block and the reference picture, the correction distance range may be changed according to the distance between the current picture of the current block and the reference picture.

[0324] Also, the number of correction direction candidates for the current block indicated by the correction direction candidate number information may be determined.

[0325] When the UMVE mode is applied to the current block and the prediction direction change permission information indicates that changing the prediction direction is allowed, the prediction direction information may be obtained from a bitstream.

[0326] In operation 2630, when the UMVE mode is applied to the current block, a base motion vector of the current block is determined. A UMVE candidate is determined in the determined UMVE candidate list. A reference picture and a base motion vector may be extracted from a neighboring block of the current block indicated by the UMVE candidate.

[0327] In operation 2640, a correction distance and a correction direction for correcting the basic motion vector are determined. The correction distance of the basic motion vector may be determined from the correction distance candidates according to the correction distance information obtained from the bitstream. The correction direction of the basic motion vector may be determined from the correction direction candidates according to the correction direction information obtained from the bitstream.

[0328] The corrected basic motion vector may be further corrected according to the additional correction distance and the additional correction direction. According to an embodiment, when the correction distance is equal to or greater than a threshold, additional correction of the basic motion vector may be performed. The additional correction distance may be determined to be less than the correction distance.

[0329] In operation 2650, a motion vector of the current block is determined by correcting the basic motion vector according to the correction distance and the correction direction. The motion vector and the reference picture of the current block may be changed according to the prediction direction indicated by the prediction direction information. When a main prediction direction is determined for the upper layer data unit, the prediction direction information may indicate a prediction direction of the current block from the main prediction direction and two auxiliary prediction directions. When a main prediction direction is not determined for the upper layer data unit, the prediction direction information may indicate a prediction direction of the current block from the prediction direction of the UMVE candidate and the remaining two prediction directions.

[0330] In operation 2660, the current block is reconstructed based on the motion vector of the current block.

[0331] Depend on Fig.17 The inter-frame prediction method according to the UMVE mode performed by the video decoding apparatus 1700 may be applied to Fig.26 The decoding method according to the UMVE mode.

[0332] Fig. 27 is a block diagram of a video encoding apparatus 2700 for performing inter-frame prediction according to a UMVE mode.

[0333] The video encoding apparatus 2700 includes an encoding parameter determiner 2710, a motion vector determiner 2720, an encoding information determiner 2730, and an output unit 2740. Fig. 27 The encoding parameter determiner 2710, the motion vector determiner 2720, the encoding information determiner 2730 and the output unit 2740 are separate elements, but according to an embodiment, the encoding parameter determiner 2710, the motion vector determiner 2720, the encoding information determiner 2730 and the output unit 2740 may be implemented as one element.

[0334] Despite Fig. 27 The encoding parameter determiner 2710, the motion vector determiner 2720, the encoding information determiner 2730, and the output unit 2740 are elements in one device, but the devices responsible for the functions of the encoding parameter determiner 2710, the motion vector determiner 2720, the encoding information determiner 2730, and the output unit 2740 do not need to be physically adjacent to each other. Therefore, according to an embodiment, the encoding parameter determiner 2710, the motion vector determiner 2720, the encoding information determiner 2730, and the output unit 2740 may be separated.

[0335] According to an embodiment, the encoding parameter determiner 2710, the motion vector determiner 2720, the encoding information determiner 2730, and the output unit 2740 may be implemented by using one processor. In addition, according to an embodiment, the encoding parameter determiner 2710, the motion vector determiner 2720, the encoding information determiner 2730, and the output unit 2740 may be implemented by using a plurality of processors.

[0336] The coding parameter determiner 2710 determines whether the UMVE mode is allowed for the upper layer data unit including the current block. The coding parameter determiner 2710 may determine whether the UMVE mode is allowed according to the frequency of using the UMVE mode in the upper layer data unit. Alternatively, the coding parameter determiner 2710 may determine whether the UMVE mode is allowed according to the characteristics of the image.

[0337] The motion vector determiner 2720 determines the motion vector of the current block. The motion vector determiner 2720 may obtain a plurality of motion vectors from neighboring blocks around the current block. The motion vector determiner 2720 may determine the motion vector of the current block by scanning around the plurality of motion vectors. The motion vector determiner 2720 may determine the optimal motion vector as the motion vector of the current block according to rate-distortion optimization.

[0338] When the UMVE mode is allowed for the upper layer data unit, the encoding information determiner 2730 determines whether the UMVE mode is applied to the current block according to the motion vector. The encoding information determiner 2730 may determine whether the motion vector of the current block determined by the motion vector determiner 2720 may be derived by using the UMVE mode. When the motion vector of the current block can be derived by using the UMVE mode, the encoding information determiner 2730 may determine to apply the UMVE mode to the current block. Alternatively, when the minimum error between the motion vector of the current block and the motion vector according to the UMVE mode is equal to or less than a threshold value, the encoding information determiner 2730 may determine to apply the UMVE mode to the current block.

[0339] When the UMVE mode is applied to the current block, the encoding information determiner 2730 determines a basic motion vector of the current block and a correction distance and a correction direction for correcting the basic motion vector according to the motion vector of the current block. The encoding information determiner 2730 may determine the correction distance and the correction direction according to a difference between the motion vector of the current block and the basic motion vector of the current block.

[0340] The output unit 2740 outputs a bitstream including encoding information according to the UMVE mode of the current block. The encoding information may include a UMVE enable flag, a UMVE flag, UMVE candidate number information, correction distance candidate number information, correction direction candidate number information, UMVE candidate list construction information, correction distance range information, correction distance change information, prediction direction change permission information, main prediction direction information, and multiple correction permission information.

[0341] can be Figure 1A The encoder 110 performs the functions of the encoding parameter determiner 2710, the motion vector determiner 2720, and the encoding information determiner 2730. Figure 1A The bitstream generator 120 performs the function of the output unit 2740.

[0342] Fig.28 Show according to Fig. 27 Flowchart of an encoding method of a UMVE mode of a video encoding apparatus 2700 .

[0343] In operation 2810, it is determined whether the UMVE mode is allowed for the upper layer data unit including the current block. Whether the UMVE mode is allowed may be determined according to the frequency of using the UMVE mode in the upper layer data unit. Alternatively, whether the UMVE mode is allowed may be determined according to characteristics of the image.

[0344] In operation 2820, a motion vector of the current block is determined. A plurality of motion vectors may be obtained from neighboring blocks around the current block. The motion vector of the current block may be determined by scanning around the plurality of motion vectors. The optimal motion vector may be determined as the motion vector of the current block according to rate-distortion optimization.

[0345] In operation 2830, when the UMVE mode is allowed for the upper layer data unit, it is determined whether the UMVE mode is applied to the current block according to the motion vector. It may be determined whether the motion vector of the current block can be derived by using the UMVE mode. When the motion vector of the current block can be derived by using the UMVE mode, it may be determined that the UMVE mode is applied to the current block. Alternatively, when a minimum error between the motion vector of the current block and the motion vector according to the UMVE mode is equal to or less than a threshold value, it may be determined that the UMVE mode is applied to the current block.

[0346] In operation 2840, a basic motion vector of the current block and a correction distance and a correction direction for correcting the basic motion vector are determined according to the motion vector. The correction distance and the correction direction may be determined according to a difference between the motion vector of the current block and the basic motion vector of the current block.

[0347] In operation 2850, a bitstream including information on a UMVE mode of the current block is output.

[0348] Depend on Fig. 27 The inter-frame prediction method according to the UMVE mode performed by the video encoding apparatus 2700 may be applied to Fig.28 The encoding method according to the UMVE mode.

[0349] According to reference 1 to Fig.28 The video encoding method based on the coding unit with a tree structure is described, and the image data in the spatial domain is encoded according to the coding unit with a tree structure, and according to the video decoding method based on the coding unit with a tree structure, decoding is performed according to the maximum coding unit, and the image data in the spatial domain is reconstructed, so that the video (such as a picture and a picture sequence) can be reconstructed. The reconstructed video can be played by a playback device, can be stored in a storage medium, and can be transmitted through a network.

[0350] In addition, the embodiments of the present disclosure can be realized as a computer-executable program, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable recording medium.

[0351] Those skilled in the art will appreciate that, although the preferred embodiments of the present disclosure have been described, various substitutions, modifications, and amendments may be made to one or more embodiments. That is, substitutions, modifications, and amendments do not depart from the scope of the present disclosure and are included in the present disclosure. Therefore, the embodiments should be considered only in a descriptive sense and not for the purpose of limitation.

Claims

1. A video decoding device, comprising: The receiver is configured as: obtaining first motion vector MV expression mode permission information from a bitstream, wherein the first MV expression mode permission information indicates whether a first MV expression mode for predicting a motion vector is allowed to be used for an upper layer data unit, When the first MV expression mode is permitted for the upper layer data unit based on the first MV expression mode permission information, obtaining correction distance range information from the bitstream, When the first MV expression mode is applied to the current block, information about the motion vector is obtained from the bitstream, and correction distance information and correction direction information for correcting the basic motion vector are obtained from the bitstream; and The decoder is configured as: when a first MV expression mode is allowed for the upper layer data unit, determining whether the first MV expression mode is applied to a current block included in the upper layer data unit, when the first MV expression mode is applied to the current block, determining the basic motion vector indicated by the information about the motion vector from the motion vector candidates of the current block, determining a correction distance by using the correction distance information and the correction distance range information, and determining the motion vector of the current block by correcting the basic motion vector by using the correction distance information and the correction direction information, wherein the motion vector candidates of the current block are included in the motion vector candidate list of the current block, and reconstructing the current block based on the motion vector of the current block, wherein one of the plurality of correction distance candidate sets includes a plurality of correction distance candidates, The correction distance range information indicates one of the plurality of correction distance candidate sets. wherein, in one set selected from the plurality of correction distance candidate sets according to the correction distance range information, the basic motion vector is corrected by using the correction distance indicated by the correction distance information.

2. A video encoding method, comprising: generating first motion vector MV expression mode permission information, wherein the first MV expression mode permission information indicates whether a first MV expression mode for predicting a motion vector is allowed to be used for an upper layer data unit; When the first MV expression mode is allowed to be used for the upper layer data unit, generating correction distance range information; When a first MV expression mode is allowed for the upper layer data unit, determining whether the first MV expression mode is applied to a current block included in the upper layer data unit; When the first MV expression mode is applied to the current block, determining a motion vector of the current block, determining a basic motion vector corresponding to the motion vector from motion vector candidates included in a motion vector candidate list of the current block, and determining a correction distance and a correction direction between the motion vector and the basic motion vector; generating information about the motion vector indicating the basic motion vector in the motion vector candidate list of the current block, and generating information about the correction distance and information about the correction direction when a first MV expression mode is applied to the current block; and generating a bit stream including the first MV expression mode permission information, the correction distance range information, the information about the motion vector, the information about the correction distance, and the information about the correction direction, wherein one of the plurality of correction distance candidate sets includes a plurality of correction distance candidates, and the correction distance range information indicates one of the plurality of correction distance candidate sets, The correction distance information indicates the correction distance in a set selected from the plurality of correction distance candidate sets according to the correction distance range information.

Images