Video Encoding Method and Apparatus, and Video Decoding Method and Apparatus

Abstract

A video decoding method includes: when performing inter prediction of a current block in an affine merge mode, generating an affine merge candidate list including affine merge candidates based on control points corresponding to control point motion vectors, wherein the control point motion vectors are determined by using motion vectors of neighboring blocks included in a group of blocks of representative neighboring blocks according to the corners of the current block; and determining an affine motion vector of the current block by using a control point motion vector corresponding to a merge candidate selected from the affine merge candidate list, and obtaining predicted samples of the current block by using the affine motion vector of the current block, wherein when a right block of the current block is available, a representative neighboring block adjacent to the lower right corner of the current block is a block that obtains available motion information from neighboring blocks adjacent to the lower right corner of the current block and the right boundary of the current block and neighboring blocks diagonally adjacent to the lower right corner of the current block.

Description

Technical Field

[0001] The present disclosure relates to the field of image encoding and decoding. Specifically, the present disclosure relates to a method and apparatus for encoding and decoding video by dividing an image into blocks of various shapes. Background Art

[0002] In compression methods according to the prior art, in a process of determining the size of a coding unit included in a picture, it is determined whether to divide the coding unit, and then a square coding unit is determined through a recursive division process of uniformly dividing the coding unit into four coding units having the same size. However, recently, degradation in the quality of a reconstructed image caused by using a uniform square coding unit for a high-resolution image has been a problem. Accordingly, methods and apparatuses for dividing a high-resolution image into coding units of various shapes have been proposed. Summary of the Invention

[0003] Technical Problem

[0004] The present disclosure relates to a video decoding method and apparatus and a video encoding method and apparatus, and provides a method for determining a reference neighboring block when performing inter prediction in an affine mode in an environment where the coding order between adjacent coding units can be changed.

[0005] Solution to the Problem

[0006] A video decoding method according to an embodiment of the present disclosure includes: when performing inter prediction of a current block in an affine merge mode, generating an affine merge candidate list including affine merge candidates corresponding to control point motion vectors, wherein the control point motion vectors are determined by using motion vectors of neighboring blocks included in a block group of representative neighboring blocks according to the corners of the current block; and determining an affine motion vector of the current block by using a control point motion vector corresponding to a merge candidate selected from the affine merge candidate list, and obtaining predicted sample points of the current block by using the affine motion vector of the current block, wherein when a right block of the current block is available, a representative neighboring block adjacent to the lower right corner of the current block is a block that obtains available motion information from neighboring blocks adjacent to the lower right corner of the current block and the right boundary of the current block and a neighboring block diagonally adjacent to the lower right corner of the current block, and when the right block of the current block is not available, a representative neighboring block adjacent to the lower right corner of the current block is a co-located block located at a point diagonally adjacent to the lower right corner of the current block, and the co-located block is included in a co-located picture. Brief Description of the Drawings

[0007] To better understand the drawings cited herein, a brief description of each drawing is provided.

[0008] Figure 1 It is a block diagram of an image decoding device according to an embodiment.

[0009] Figure 2 It is a flowchart of an image decoding method according to an embodiment.

[0010] Figure 3 It shows the process of an image decoding device according to an embodiment for determining at least one coding unit by dividing a current coding unit.

[0011] Figure 4 It shows the process of an image decoding device according to an embodiment for determining at least one coding unit by dividing a non-square coding unit.

[0012] Figure 5 It shows the process of an image decoding device according to an embodiment for dividing a coding unit based on at least one of block shape information and division shape mode information.

[0013] Figure 6 It shows a method for an image decoding device according to an embodiment to determine a specific coding unit from an odd number of coding units.

[0014] Figure 7 It shows the order of processing multiple coding units when an image decoding device determines multiple coding units by dividing a current coding unit according to an embodiment.

[0015] Figure 8 It shows the process of an image decoding device according to an embodiment for determining that a current coding unit will be divided into an odd number of coding units when the coding units cannot be processed in a specific order.

[0016] Figure 9 It shows the process of an image decoding device according to an embodiment for determining at least one coding unit by dividing a first coding unit.

[0017] Figure 10 It shows the shapes into which a second coding unit having a non-square shape determined when an image decoding device divides a first coding unit is restricted when the second coding unit satisfies a specific condition according to an embodiment.

[0018] Figure 11 It shows the process of an image decoding device according to an embodiment for dividing a square coding unit when the division shape mode information cannot indicate that a square coding unit is divided into four square coding units.

[0019] Figure 12 It shows that the processing order between multiple coding units can be changed according to the process of dividing coding units according to an embodiment.

[0020] Figure 13Shows the process of determining the depth of a coding unit when the shape and size of the coding unit change, according to an embodiment, when the coding unit is recursively divided such that multiple coding units are determined.

[0021] Figure 14 Shows the depth that can be determined based on the shape and size of a coding unit and a partial index for differentiating coding units (PID), according to an embodiment.

[0022] Figure 15 Shows determining multiple coding units based on multiple specific data units included in a picture, according to an embodiment.

[0023] Figure 16 Is a block diagram of an image encoding and decoding system.

[0024] Figure 17 Is a block diagram of a video decoding device according to an embodiment.

[0025] Figure 18 Is a flowchart of a video decoding method according to an embodiment.

[0026] Figure 19 Is a block diagram of a video encoding device according to an embodiment.

[0027] Figure 20 Is a flowchart of a video encoding method according to an embodiment.

[0028] Figure 21 Shows changing the encoding order of a coding unit according to the partition unit coding order (SUCO) method according to various methods.

[0029] Figure 22 Shows a method of deriving a motion vector for samples applied to a current block in an affine mode.

[0030] Figure 23 Shows a method of determining an affine motion vector of a current block in an affine mode.

[0031] Figure 24 Shows a method of determining a representative neighboring block adjacent to a corner of a current block and a control point motion vector derived from the representative neighboring block, according to an embodiment.

[0032] Figure 25 Shows a method of determining a representative neighboring block adjacent to a corner of a current block and a control point motion vector derived from the representative neighboring block, according to another embodiment.

[0033] Figure 26 Shows a method of determining a representative neighboring block adjacent to a corner of a current block and a control point motion vector derived from the representative neighboring block, according to another embodiment.

[0034] Figure 27 A method for determining a representative neighboring block adjacent to a corner of a current block and a control point motion vector derived from the representative neighboring block according to another embodiment is shown.

[0035] Best mode

[0036] A video decoding method according to an embodiment of the present disclosure includes: when performing inter prediction of a current block in an affine merge mode, generating an affine merge candidate list including affine merge candidates based on control points corresponding to control point motion vectors, where the control point motion vectors are determined by using motion vectors of neighboring blocks included in a block group of representative neighboring blocks according to a corner of the current block; and determining an affine motion vector of the current block by using a control point motion vector corresponding to a merge candidate selected from the affine merge candidate list, and obtaining predicted samples of the current block by using the affine motion vector of the current block, where when a right block of the current block is available, a representative neighboring block adjacent to the lower right corner of the current block is a block that obtains available motion information from neighboring blocks adjacent to the lower right corner of the current block and the right boundary of the current block and a neighboring block diagonally adjacent to the lower right corner of the current block, and when the right block of the current block is not available, a representative neighboring block adjacent to the lower right corner of the current block is a co-located block located at a point diagonally adjacent to the lower right corner of the current block, and the co-located block is included in a co-located picture.

[0037] According to an embodiment, the step of generating the affine merge candidate list including the affine merge candidates based on the control points may include: determining the availability of motion information of a first representative neighboring block adjacent to the upper left corner of the current block, the availability of motion information of a second representative neighboring block adjacent to the upper right corner of the current block, the availability of motion information of a third representative neighboring block adjacent to the lower left corner of the current block, and the availability of motion information of a fourth representative neighboring block adjacent to the lower right corner of the current block; and determining an affine merge candidate based on the control point corresponding to a block group including a plurality of representative neighboring blocks based on the availability of motion information of at least one of the first representative neighboring block, the second representative neighboring block, the third representative neighboring block, and the fourth representative neighboring block.

[0038] According to an embodiment, the step of determining the control point-based affine merge candidate may include: when the availability of the motion information of the first representative neighboring block adjacent to the upper left corner of the current block, the availability of the motion information of the second representative neighboring block adjacent to the upper right corner of the current block, and the availability of the motion information of the third representative neighboring block adjacent to the lower left corner of the current block respectively indicate availability, determining a first control point-based affine merge candidate corresponding to a block group including the first representative neighboring block, the second representative neighboring block, and the third representative neighboring block; when the availability of the motion information of the first representative neighboring block adjacent to the upper left corner of the current block, the availability of the motion information of the second representative neighboring block adjacent to the upper right corner of the current block, and the availability of the motion information of the fourth representative neighboring block adjacent to the lower right corner of the current block respectively indicate availability, determining a second control point-based affine merge candidate corresponding to a block group including the first representative neighboring block, the second representative neighboring block, and the fourth representative neighboring block; when the availability of the motion information of the first representative neighboring block adjacent to the upper left corner of the current block, the availability of the motion information of the third representative neighboring block adjacent to the lower left corner of the current block, and the availability of the motion information of the fourth representative neighboring block adjacent to the lower right corner of the current block respectively indicate availability, determining a third control point-based affine merge candidate corresponding to a block group including the first representative neighboring block, the third representative neighboring block, and the fourth representative neighboring block; when the availability of the motion information of the second representative neighboring block adjacent to the upper right corner of the current block, the availability of the motion information of the third representative neighboring block adjacent to the lower left corner of the current block, and the availability of the motion information of the fourth representative neighboring block adjacent to the lower right corner of the current block respectively indicate availability, determining a fourth control point-based affine merge candidate corresponding to a block group including the second representative neighboring block, the third representative neighboring block, and the fourth representative neighboring block; when the availability of the motion information of the first representative neighboring block adjacent to the upper left corner of the current block and the availability of the motion information of the second representative neighboring block adjacent to the upper right corner of the current block respectively indicate availability, determining a fifth control point-based affine merge candidate corresponding to a block group including the first representative neighboring block and the second representative neighboring block; and when the availability of the motion information of the first representative neighboring block adjacent to the upper left corner of the current block and the availability of the motion information of the third representative neighboring block adjacent to the lower left corner of the current block respectively indicate availability, determining a sixth control point-based affine merge candidate corresponding to a block group including the first representative neighboring block and the third representative neighboring block.

[0039] According to an embodiment, when the left block of the current block is available, the representative neighboring block adjacent to the lower left corner of the current block is a block that obtains available motion information from neighboring blocks diagonally adjacent to the lower left corner of the current block and neighboring blocks adjacent to the lower left corner of the current block and the left boundary of the current block, and when the left block of the current block is not available, the representative neighboring block adjacent to the lower left corner of the current block is a co-located block located at a point diagonally adjacent to the lower left corner of the current block, and the co-located block is included in a co-located picture.

[0040] According to an embodiment, the representative neighboring block adjacent to the upper left corner of the current block is a block that obtains available motion information from neighboring blocks diagonally adjacent to the upper left corner of the current block, neighboring blocks adjacent to the upper left corner of the current block and the upper boundary of the current block, and neighboring blocks adjacent to the upper left corner of the current block and the left boundary of the current block.

[0041] According to an embodiment, the representative neighboring block adjacent to the upper right corner of the current block is a block that obtains available motion information from neighboring blocks diagonally adjacent to the upper right corner of the current block, neighboring blocks adjacent to the upper right corner of the current block and the upper boundary of the current block, and neighboring blocks adjacent to the upper right corner of the current block and the right boundary of the current block.

[0042] According to an embodiment, the step of generating the affine merge candidate list may include: obtaining a merge mode flag from a bitstream indicating whether the inter prediction mode of the current block is a merge mode; when the merge mode flag indicates the merge mode, obtaining an affine flag from the bitstream indicating whether to perform motion compensation based on an affine model to generate prediction samples of the current block; when the affine flag indicates to perform motion compensation based on the affine model, generating an affine merge candidate list including affine merge candidates determined based on available neighboring blocks among neighboring blocks at a specific position adjacent to the current block; and when the number of the affine merge candidates is less than a predetermined number, adding affine merge candidates based on control points to the affine merge candidate list.

[0043] According to an embodiment, the step of determining the affine motion vector of the current block may include: determining a horizontal change amount of the motion vector, a vertical change amount of the motion vector, and a basic motion vector by using control point motion vectors.

[0044] According to an embodiment, when selecting a control point-based affine merge candidate from the affine merge candidate list, the reference index, prediction direction, and control point motion vector corresponding to the affine merge candidate are determined by using the reference index, prediction direction, and motion vector of a representative neighboring block adjacent to a corner of the current block belonging to a block group corresponding to the control point-based affine merge candidate, and the affine motion vector of the current block is determined by using the reference index, prediction direction, and control point motion vector corresponding to the affine merge candidate.

[0045] A video decoding device according to an embodiment of the present disclosure includes: an affine merge candidate list determiner configured to generate an affine merge candidate list including control point-based affine merge candidates corresponding to control point motion vectors when performing inter prediction of a current block in an affine merge mode, where the control point motion vector is determined by using motion vectors of neighboring blocks included in a block group of representative neighboring blocks according to a corner of the current block; and an affine mode predictor configured to determine the affine motion vector of the current block by using the control point motion vector corresponding to a merge candidate selected from the affine merge candidate list, and obtain predicted samples of the current block by using the affine motion vector of the current block, where when a right block of the current block is available, a representative neighboring block adjacent to the lower right corner of the current block is a block that obtains available motion information from neighboring blocks adjacent to the lower right corner of the current block and the right boundary of the current block and a neighboring block diagonally adjacent to the lower right corner of the current block, and when the right block of the current block is not available, a representative neighboring block adjacent to the lower right corner of the current block is a co-located block located at a point diagonally adjacent to the lower right corner of the current block, and the co-located block is included in a co-located picture.

[0046] A video coding method according to an embodiment of the present disclosure includes: when performing inter prediction of a current block in an affine merge mode, generating an affine merge candidate list including affine merge candidates based on control points corresponding to control point motion vectors, wherein the control point motion vectors are determined by using motion vectors of neighboring blocks included in a group of blocks of representative neighboring blocks according to the corners of the current block; and encoding a merge index of a merge candidate indicating a control point motion vector for determining inter prediction of the current block in an affine mode from the affine merge candidate list, wherein when a right block of the current block is available, a representative neighboring block adjacent to the lower right corner of the current block is a block that obtains available motion information from neighboring blocks adjacent to the lower right corner and the right boundary of the current block and neighboring blocks diagonally adjacent to the lower right corner of the current block, and when the right block of the current block is not available, a representative neighboring block adjacent to the lower right corner of the current block is a co-located block located at a point diagonally adjacent to the lower right corner of the current block, and the co-located block is included in a co-located picture.

[0047] According to an embodiment, the video coding method may further include: encoding a merge mode flag indicating whether the inter prediction mode of the current block is a merge mode; and encoding an affine flag indicating whether to perform motion compensation based on an affine model to generate prediction samples of the current block.

[0048] According to an embodiment, a horizontal change amount of a motion vector, a vertical change amount of the motion vector, and a basic motion vector may be determined by using a control point motion vector, and an affine motion vector of the current block may be predicted by using the horizontal change amount of the motion vector, the vertical change amount of the motion vector, and the basic motion vector.

[0049] A computer-readable recording medium having recorded thereon a program for executing a video decoding method according to an embodiment of the present disclosure by using a computer is provided.

[0050] A computer-readable recording medium having recorded thereon a program for executing a video coding method according to an embodiment of the present disclosure by using a computer is provided. Detailed Description of the Embodiment

[0051] Since the present disclosure allows various changes and many examples, specific embodiments will be shown in the drawings and described in detail in the written description. However, this is not intended to limit the present disclosure to a specific implementation, and it should be understood that all changes, equivalents, and substitutions that do not depart from the spirit and technical scope of various embodiments are included in the present disclosure.

[0052] In the description of the embodiments, specific detailed explanations of related technologies are omitted when it is considered that they may unnecessarily obscure the essence of the present disclosure. In addition, the numbers (e.g., first, second, etc.) used in the description of the specification are merely identifier codes for distinguishing one element from another element.

[0053] In addition, in this specification, it should be understood that when elements are "connected" or "coupled" to each other, the elements may be directly connected or coupled to each other, but may optionally be connected or coupled to each other through intermediate elements therebetween, unless otherwise specified.

[0054] In this specification, with respect to elements represented as "units" or "modules", two or more elements may be combined into one element, or one element may be divided into two or more elements according to the subdivision of functions. In addition, each element described below may additionally perform some or all of the functions performed by another element in addition to its own main function, and some main functions of each element may be entirely performed by another component.

[0055] In addition, in this specification, an "image" or "picture" may represent a still image or a moving image of a video (i.e., the video itself).

[0056] In addition, in this specification, a "sample point" represents data assigned to a sampling position of an image (i.e., data to be processed). For example, the pixel value of an image in the spatial domain and the transform coefficient on the transform region may be sample points. A unit including at least one such sample point may be defined as a block.

[0057] In addition, in this specification, a "current block" may represent a block of the largest coding unit, coding unit, prediction unit, or transform unit of the current picture to be encoded or decoded.

[0058] In addition, in this specification, the motion vector in the list 0 direction may represent a motion vector for indicating a block in a reference picture included in list 0, and the motion vector in the list 1 direction may represent a motion vector for indicating a block in a reference picture included in list 1. In addition, a unidirectional motion vector may represent a motion vector for indicating a block in a reference picture included in list 0 or list 1, and a bidirectional motion vector may represent a motion vector including the motion vector in the list 0 direction and the motion vector in the list 1 direction.

[0059] In addition, in this specification, the term "binary partitioning" refers to partitioning a block into two sub-blocks whose width or height is half of the width or height of the block. Specifically, when performing "binary vertical partitioning" on the current block, since partitioning is performed vertically at a point corresponding to half of the width of the current block, two sub-blocks with a width equal to half of the width of the current block and a height equal to the height of the current block can be generated. When performing "binary horizontal partitioning" on the current block, since partitioning is performed horizontally at a point corresponding to half of the height of the current block, two sub-blocks with a height equal to half of the height of the current block and a width equal to the width of the current block can be generated.

[0060] In addition, in this specification, the term "ternary partitioning" refers to partitioning the width or height of a block at a ratio of 1:2:1 to generate three sub-blocks. Specifically, when performing "ternary vertical partitioning" on the current block, since partitioning is performed vertically at a point corresponding to the 1:2:1 ratio of the width of the current block, two sub-blocks with a width equal to 1 / 4 of the width of the current block and a height equal to the height of the current block and one sub-block with a width equal to 2 / 4 of the width of the current block and a height equal to the height of the current block can be generated. When performing "ternary horizontal partitioning" on the current block, since partitioning is performed horizontally at a point corresponding to the 1:2:1 ratio of the height of the current block, two sub-blocks with a height equal to 1 / 4 of the height of the current block and a width equal to the width of the current block and one sub-block with a height equal to 2 / 4 of the height of the current block and a width equal to the width of the current block can be generated.

[0061] In addition, in this specification, the term "quaternary partitioning" refers to partitioning the width and height of a block at a ratio of 1:1 to generate four sub-blocks. Specifically, when performing "quaternary partitioning" on the current block, since partitioning is performed vertically at a point corresponding to half of the width of the current block and horizontally at a point corresponding to half of the height of the current block, four sub-blocks with a width equal to 1 / 2 of the width of the current block and a height equal to 1 / 2 of the height of the current block can be generated.

[0062] Hereinafter, reference will be made to Figures 1 to 16 describe in detail an image encoding device, an image decoding device, an image encoding method, and an image decoding method according to an embodiment. Reference will be made to Figures 3 to 16 describe a method for determining a data unit of an image according to an embodiment, and reference will be made to Figures 17 to 27 describe a video encoding / decoding method according to an embodiment.

[0063] Hereinafter, reference will be made to Figure 1 and Figure 2 describe in detail a method and a device for adaptively selecting coding units based on various shapes according to an embodiment of the present disclosure.

[0064] Figure 1 It is a block diagram of an image decoding device according to an embodiment.

[0065] The image decoding device 100 may include a receiver 110 and a decoder 120. The receiver 110 and the decoder 120 may include at least one processor. In addition, the receiver 110 and the decoder 120 may include a memory that stores instructions to be executed by the at least one processor.

[0066] The receiver 110 may receive a bitstream. The bitstream includes information of an image encoded by an image encoding device 2200 described later. In addition, the bitstream may be sent from the image encoding device 2200. The image encoding device 2200 and the image decoding device 100 may be connected by a wired or wireless connection, and the receiver 110 may receive the bitstream by wire or wirelessly. The receiver 110 may receive the bitstream from a storage medium (such as an optical medium or a hard disk). The decoder 120 may reconstruct an image based on the information obtained from the received bitstream. The decoder 120 may obtain syntax elements for reconstructing the image from the bitstream. The decoder 120 may reconstruct the image based on the syntax elements.

[0067] Reference will be made to Figure 2 describe the operation of the image decoding device 100 in detail.

[0068] Figure 2 It is a flowchart of an image decoding method according to an embodiment.

[0069] According to an embodiment of the present disclosure, the receiver 110 receives a bitstream.

[0070] The image decoding device 100 obtains a binary bitstring corresponding to a partitioning shape mode of a coding unit from the bitstream (operation 210). The image decoding device 100 determines a partitioning rule of the coding unit (operation 220). In addition, the image decoding device 100 divides the coding unit into a plurality of coding units based on at least one of the binary bitstring corresponding to the partitioning shape mode and the partitioning rule (operation 230). The image decoding device 100 may determine a first allowable range of the size of the coding unit according to the aspect ratio of the coding unit in order to determine the partitioning rule. The image decoding device 100 may determine a second allowable range of the size of the coding unit according to the partitioning shape mode of the coding unit in order to determine the partitioning rule.

[0071] Hereinafter, the partitioning of the coding unit will be described in detail according to an embodiment of the present disclosure.

[0072] First, a picture can be divided into one or more stripes or one or more parallel blocks. A stripe or a parallel block can be a sequence of one or more largest coding units (Coding Tree Units (CTUs)). Conceptually, there is a largest coding block (Coding Tree Block (CTB)) compared to the largest coding unit (CTU).

[0073] The largest coding block (CTB) represents an N×N block including N×N samples (N is an integer). Each color component can be divided into one or more largest coding blocks.

[0074] When the picture has three sample arrays (sample arrays for Y, Cr, and Cb components), the largest coding unit (CTU) includes the largest coding block of luminance samples, two corresponding largest coding blocks of chrominance samples, and syntax structures for encoding the luminance samples and chrominance samples. When the picture is a monochrome picture, the largest coding unit includes the largest coding block of monochrome samples and syntax structures for encoding the monochrome samples. When the picture is a picture encoded in color planes separated according to color components, the largest coding unit includes syntax structures for encoding the picture and the samples of the picture.

[0075] A largest coding block (CTB) can be divided into M×N coding blocks including M×N samples (M and N are integers).

[0076] When the picture has sample arrays for Y, Cr, and Cb components, the coding unit (CU) includes coding blocks of luminance samples, two corresponding coding blocks of chrominance samples, and syntax structures for encoding the luminance samples and chrominance samples. When the picture is a monochrome picture, the coding unit includes coding blocks of monochrome samples and syntax structures for encoding the monochrome samples. When the picture is a picture encoded in color planes separated according to color components, the coding unit includes syntax structures for encoding the picture and the samples of the picture.

[0077] As described above, the largest coding block and the largest coding unit are conceptually distinguished from each other, and the coding block and the coding unit are conceptually distinguished from each other. That is, the (largest) coding unit refers to a data structure including the (largest) coding block including corresponding samples and the syntax structure corresponding to the (largest) coding block. However, since those of ordinary skill in the art understand that the (largest) coding unit or the (largest) coding block refers to a block of a specific size including a specific number of samples, unless otherwise specified, the largest coding block and the largest coding unit or the coding block and the coding unit are not distinguished in the following description.

[0078] An image can be divided into Coding Tree Units (CTUs). The size of each Coding Tree Unit can be determined based on information obtained from the bitstream. The shape of each Coding Tree Unit can be a square shape of the same size. However, the embodiments are not limited thereto.

[0079] For example, information about the maximum size of a luminance coding block can be obtained from the bitstream. For example, the maximum size of the luminance coding block indicated by the information about the maximum size of the luminance coding block can be one of 4×4, 8×8, 16×16, 32×32, 64×64, 128×128, and 256×256.

[0080] For example, information about the luminance block size difference and the maximum size of the luminance coding block that can be divided into two can be obtained from the bitstream. The information about the luminance block size difference can refer to the size difference between the largest luminance coding unit and the largest luminance coding block that can be divided into two. Accordingly, when the information about the maximum size of the luminance coding block that can be divided into two and the information about the luminance block size difference obtained from the bitstream are combined with each other, the size of the largest luminance coding unit can be determined. The size of the chrominance largest coding unit can also be determined by using the size of the largest luminance coding unit. For example, when the Y:Cb:Cr ratio is 4:2:0 according to the color format, the size of the chrominance block can be half of the size of the luminance block, and the size of the chrominance largest coding unit can be half of the size of the largest luminance coding unit.

[0081] According to an embodiment, since information about the maximum size of the binary-divisible luminance coding block is obtained from the bitstream, the maximum size of the binary-divisible luminance coding block can be variably determined. On the contrary, the maximum size of the ternary-divisible luminance coding block can be fixed. For example, the maximum size of the ternary-divisible luminance coding block in an I picture can be 32×32, and the maximum size of the ternary-divisible luminance coding block in a P picture or a B picture can be 64×64.

[0082] In addition, the largest coding unit can be hierarchically divided into coding units based on the partition shape mode information obtained from the bitstream. At least one of information indicating whether to perform quaternary partitioning, information indicating whether to perform multi-way partitioning, partitioning direction information, and partitioning type information can be obtained from the bitstream as the partition shape mode information.

[0083] For example, the information indicating whether to perform quaternary partitioning can indicate whether the current coding unit is quaternary partitioned (QUAD_SPLIT).

[0084] When the current coding unit is not quaternary partitioned, the information indicating whether to perform multi-way partitioning can indicate that the current coding unit is no longer partitioned (NO_SPLIT) or is binary / ternary partitioned.

[0085] When the current coding unit is binary partitioned or ternary partitioned, the partition direction information indicates that the current coding unit is partitioned in one of the horizontal direction and the vertical direction.

[0086] When the current coding unit is partitioned in the horizontal direction or the vertical direction, the partition type information indicates that the current coding unit is binary partitioned or ternary partitioned.

[0087] The partition mode of the current coding unit can be determined according to the partition direction information and the partition type information. The partition mode when the current coding unit is a binary partition in the horizontal direction can be determined as the binary horizontal partition mode (SPLIT_BT_HOR), the partition mode when the current coding unit is a ternary partition in the horizontal direction can be determined as the ternary horizontal partition mode (SPLIT_TT_HOR), the partition mode when the current coding unit is a binary partition in the vertical direction can be determined as the binary vertical partition mode (SPLIT_BT_VER), and the partition mode when the current coding unit is a ternary partition in the vertical direction can be determined as the ternary vertical partition mode (SPLIT_BT_VERT).

[0088] The image decoding device 100 can obtain partition shape mode information from a binary bit string from a bitstream. The form of the bitstream received by the image decoding device 100 can include a fixed-length binary code, a unary code, a truncated unary code, a predetermined binary code, etc. The binary bit string is information of binary numbers. The binary bit string can include at least one bit. The image decoding device 100 can obtain the partition shape mode information corresponding to the binary bit string based on a partitioning rule. The image decoding device 100 can determine whether to perform a quaternary partition on a coding unit, whether not to partition the coding unit, the partition direction, and the partition type based on a binary bit string.

[0089] The coding unit can be less than or equal to the maximum coding unit. For example, since the maximum coding unit is the coding unit with the largest size, the maximum coding unit is one of the coding units. When the partition shape mode information about the maximum coding unit indicates no partition, the coding unit determined in the maximum coding unit has the same size as the maximum coding unit. When the partition shape mode information about the maximum coding unit indicates performing a partition, the maximum coding unit can be partitioned into multiple coding units. In addition, when the partition shape mode information about a coding unit indicates performing a partition, the coding unit can be partitioned into smaller coding units. However, the partitioning of the image is not limited to this, and the maximum coding unit and the coding unit may not be distinguished. The partitioning of the coding unit will be described in more detail with reference to Figures 3 to 16 Describe the partitioning of the coding unit in more detail.

[0090] In addition, one or more prediction blocks for prediction can be determined from a coding unit. The prediction blocks can be the same as or smaller than the coding unit. In addition, one or more transform blocks for transformation can be determined from the coding unit. The transform blocks can be the same as or smaller than the coding unit.

[0091] The shapes and sizes of the transform blocks and the prediction blocks may be independent of each other.

[0092] In another embodiment, prediction can be performed by using the coding unit as a prediction block. In addition, transformation can be performed by using the coding unit as a transform block.

[0093] Reference will be made to Figures 3 to 16 The partitioning of the coding unit will be described in more detail. The current block and neighboring blocks of the present invention can indicate one of a maximum coding unit, a coding unit, a prediction block, and a transform block. In addition, the current block or the current coding unit is the block that is currently being decoded or encoded or the block that is currently being partitioned. The neighboring block can be a block that has been reconstructed before the current block. The neighboring block can be adjacent to the current block spatially or temporally. The neighboring block can be located at one of the lower left, left, upper left, upper, upper right, right, and lower right of the current block.

[0094] Figure 3 An image decoding device according to an embodiment is shown that determines at least one coding unit by partitioning a current coding unit.

[0095] The block shape can include 4N×4N, 4N×2N, 2N×4N, 4N×N, N×4N, 32N×N, N×32N, 16N×N, N×16N, 8N×N, or N×8N. Here, N can be a positive integer. The block shape information is information indicating at least one of the shape, direction, aspect ratio, or size of the coding unit.

[0096] The shape of the coding unit can include a square and a non-square. When the lengths of the width and height of the coding unit are the same (i.e., when the block shape of the coding unit is 4N×4N), the image decoding device 100 can determine the block shape information of the coding unit as a square. The image decoding device 100 can determine the shape of the coding unit as a non-square.

[0097] When the lengths of the width and height of a coding unit are different from each other (i.e., when the block shape of the coding unit is 4N×2N, 2N×4N, 4N×N, N×4N, 32N×N, N×32N, 16N×N, N×16N, 8N×N, or N×8N), the image decoding device 100 may determine the block shape information of the coding unit as non-square. When the shape of the coding unit is non-square, the image decoding device 100 may determine the aspect ratio in the block shape information of the coding unit as at least one of 1:2, 2:1, 1:4, 4:1, 1:8, 8:1, 1:16, 16:1, 1:32, and 32:1. In addition, the image decoding device 100 may determine whether the coding unit is in the horizontal direction or the vertical direction based on the lengths of the width and height of the coding unit. In addition, the image decoding device 100 may determine the size of the coding unit based on at least one of the length of the width, the length of the height, or the area of the coding unit.

[0098] According to an embodiment, the image decoding device 100 may determine the shape of the coding unit by using the block shape information, and may determine the division method of the coding unit by using the division shape pattern information. That is, the coding unit division method indicated by the division shape pattern information may be determined based on the block shape indicated by the block shape information used by the image decoding device 100.

[0099] The image decoding device 100 may obtain the division shape pattern information from the bitstream. However, the embodiment is not limited thereto, and the image decoding device 100 and the image encoding device 2200 may determine the pre-agreed division shape pattern information based on the block shape information. The image decoding device 100 may determine the pre-agreed division shape pattern information for the largest coding unit or the smallest coding unit. For example, the image decoding device 100 may determine the division shape pattern information regarding the largest coding unit as quadtree division. In addition, the image decoding device 100 may determine the division shape pattern information regarding the smallest coding unit as "no division is performed". Specifically, the image decoding device 100 may determine the size of the largest coding unit as 256×256. The image decoding device 100 may determine the pre-agreed division shape pattern information as quadtree division. Quadtree division is a division shape pattern in which both the width and height of the coding unit are bisected. The image decoding device 100 may obtain a coding unit with a size of 128×128 from the largest coding unit with a size of 256×256 based on the division shape pattern information. In addition, the image decoding device 100 may determine the size of the smallest coding unit as 4×4. The image decoding device 100 may obtain the division shape pattern information indicating "no division is performed" for the smallest coding unit.

[0100] According to an embodiment, the image decoding device 100 may use block shape information indicating that the current coding unit has a square shape. For example, the image decoding device 100 may determine whether to not divide the square coding unit, whether to vertically divide the square coding unit, whether to horizontally divide the square coding unit, or whether to divide the square coding unit into four coding units based on the partitioning shape mode information. Refer to Figure 3 , when the block shape information of the current coding unit 300 indicates a square shape, the decoder 120 may not divide the coding unit 310a having the same size as the current coding unit 300 based on the partitioning shape mode information indicating no partitioning, or may determine the divided coding units 310b, 310c, 310d, 310e, or 310f based on the partitioning shape mode information indicating a specific partitioning method.

[0101] Refer to Figure 3 , according to an embodiment, the image decoding device 100 may determine two coding units 310b obtained by vertically dividing the current coding unit 300 based on the partitioning shape mode information indicating partitioning in the vertical direction. The image decoding device 100 may determine two coding units 310c obtained by horizontally dividing the current coding unit 300 based on the partitioning shape mode information indicating partitioning in the horizontal direction. The image decoding device 100 may determine four coding units 310d obtained by vertically and horizontally dividing the current coding unit 300 based on the partitioning shape mode information indicating partitioning in the vertical and horizontal directions. According to an embodiment, the image decoding device 100 may determine three coding units 310e obtained by vertically dividing the current coding unit 300 based on the partitioning shape mode information indicating ternary partitioning in the vertical direction. The image decoding device 100 may determine three coding units 310f obtained by horizontally dividing the current coding unit 300 based on the partitioning shape mode information indicating ternary partitioning in the horizontal direction. However, the method of dividing the square coding unit is not limited to the above methods, and the partitioning shape mode information may indicate various methods. Specific partitioning methods for dividing the square coding unit will be described in detail regarding various embodiments below.

[0102] Figure 4 Illustrates the process of an image decoding device according to an embodiment determining at least one coding unit by dividing a non-square coding unit.

[0103] According to an embodiment, the image decoding device 100 may use block shape information indicating that the current coding unit has a non-square shape. The image decoding device 100 may determine whether to not divide the non-square current coding unit or whether to divide the non-square current coding unit by using a specific partitioning method based on the partitioning shape mode information. Refer to Figure 4, when the block shape information of the current coding unit 400 or 450 indicates a non-square shape, the image decoding device 100 may determine a coding unit 410 or 460 having the same size as the current coding unit 400 or 450 based on the partitioning shape mode information indicating not to perform partitioning, or may determine the partitioned coding units 420a and 420b, 430a to 430c, 470a and 470b, or 480a to 480c based on the partitioning shape mode information indicating a specific partitioning method. A specific partitioning method for partitioning a non-square coding unit will be described in detail with respect to various embodiments below.

[0104] According to an embodiment, the image decoding device 100 may determine a partitioning method of a coding unit by using the partitioning shape mode information, and in this case, the partitioning shape mode information may indicate the number of one or more coding units generated by partitioning the coding unit. Refer to Figure 4 , when the partitioning shape mode information indicates that the current coding unit 400 or 450 is partitioned into two coding units, the image decoding device 100 may determine two coding units 420a and 420b or 470a and 470b included in the current coding unit 400 or 450 by partitioning the current coding unit 400 or 450 based on the partitioning shape mode information.

[0105] According to an embodiment, when the image decoding device 100 partitions the non-square current coding unit 400 or 450 based on the partitioning shape mode information, the image decoding device 100 may consider the position of the long side of the non-square current coding unit 400 or 450 to partition the current coding unit. For example, the image decoding device 100 may consider the shape of the current coding unit 400 or 450 and determine a plurality of coding units by partitioning the long side of the current coding unit 400 or 450.

[0106] According to an embodiment, when the partitioning shape mode information indicates that the coding unit is partitioned (ternary partitioning) into an odd number of blocks, the image decoding device 100 may determine an odd number of coding units included in the current coding unit 400 or 450. For example, when the partitioning shape mode information indicates that the current coding unit 400 or 450 is partitioned into three coding units, the image decoding device 100 may partition the current coding unit 400 or 450 into three coding units 430a, 430b, and 430c or 480a, 480b, and 480c.

[0107] According to an embodiment, the aspect ratio of the current coding unit 400 or 450 may be 4:1 or 1:4. Since the length of the width is longer than the length of the height, when the aspect ratio is 4:1, the block shape information may be in the horizontal direction. Since the length of the width is shorter than the length of the height, when the aspect ratio is 1:4, the block shape information may be in the vertical direction. The image decoding device 100 may determine to divide the current coding unit into an odd number of blocks based on the partitioning shape mode information. In addition, the image decoding device 100 may determine the partitioning direction of the current coding unit 400 or 450 based on the block shape information of the current coding unit 400 or 450. For example, when the current coding unit 400 is in the vertical direction, the image decoding device 100 may determine coding units 430a to 430c by partitioning the current coding unit 400 in the horizontal direction. In addition, when the current coding unit 450 is in the horizontal direction, the image decoding device 100 may determine coding units 480a to 480c by partitioning the current coding unit 450 in the vertical direction.

[0108] According to an embodiment, the image decoding device 100 may determine an odd number of coding units included in the current coding unit 400 or 450, and not all of the determined coding units may have the same size. For example, a specific coding unit 430b or 480b among the determined odd number of coding units 430a, 430b, and 430c or 480a, 480b, and 480c may have a size different from that of the other coding units 430a and 430c or 480a and 480c. That is, the coding units that can be determined by partitioning the current coding unit 400 or 450 may have multiple sizes, and in some cases, all of the odd number of coding units 430a, 430b, and 430c or 480a, 480b, and 480c may have different sizes.

[0109] According to an embodiment, when the partitioning shape mode information indicates that the coding unit is to be divided into an odd number of blocks, the image decoding device 100 may determine an odd number of coding units included in the current coding unit 400 or 450, and in addition, a specific restriction may be imposed on at least one of the odd number of coding units generated by partitioning the current coding unit 400 or 450. Referring to Figure 4 , the image decoding device 100 may set the decoding process of the coding unit 430b or 480b located at the center among the three coding units 430a, 430b, and 430c or 480a, 480b, and 480c generated when the current coding unit 400 or 450 is divided to be different from the decoding processes of the other coding units 430a and 430c or 480a or 480c. For example, different from the other coding units 430a and 430c or 480a and 480c, the image decoding device 100 may restrict the coding unit 430b or 480b at the center position from being further divided or only divided a specific number of times.

[0110] Figure 5 Shows the process in which an image decoding device according to an embodiment divides a coding unit based on at least one of block shape information and partitioning shape mode information.

[0111] According to an embodiment, the image decoding device 100 may determine whether to divide the square first coding unit 500 into coding units based on at least one of block shape information and partitioning shape mode information. According to an embodiment, when the partitioning shape mode information indicates that the first coding unit 500 is divided in the horizontal direction, the image decoding device 100 may determine the second coding unit 510 by dividing the first coding unit 500 in the horizontal direction. The first coding unit, the second coding unit, and the third coding unit used according to an embodiment are terms for understanding the relationship before and after dividing the coding unit. For example, the second coding unit may be determined by dividing the first coding unit, and the third coding unit may be determined by dividing the second coding unit. It should be understood that the structures of the first coding unit, the second coding unit, and the third coding unit follow the above description.

[0112] According to an embodiment, the image decoding device 100 may determine whether to divide the determined second coding unit 510 into a plurality of coding units based on the partitioning shape mode information. Referring to Figure 5 , the image decoding device 100 may divide the non-square second coding unit 510 determined by dividing the first coding unit 500 into one or more third coding units 520a or 520b, 520c, and 520d based on the partitioning shape mode information, or may not divide the non-square second coding unit 510 into one or more third coding units 520a or 520b, 520c, and 520d. The image decoding device 100 may obtain the partitioning shape mode information, and may obtain a plurality of second coding units (e.g., 510) of various shapes by dividing the first coding unit 500 based on the obtained partitioning shape mode information, and may divide the second coding unit 510 by using the dividing method of the first coding unit 500 based on the partitioning shape mode information. According to an embodiment, when the first coding unit 500 is divided into the second coding unit 510 based on the partitioning shape mode information of the first coding unit 500, the second coding unit 510 may also be divided into the third coding units 520a or 520b, 520c, and 520d based on the partitioning shape mode information of the second coding unit 510. That is, the coding unit may be recursively divided based on the partitioning shape mode information of each coding unit. Accordingly, a square coding unit may be determined by dividing a non-square coding unit, and a non-square coding unit may be determined by recursively dividing a square coding unit.

[0113] Referring to Figure 5, a specific coding unit (e.g., a coding unit at the center position or a square coding unit) among an odd number of third coding units 520b, 520c, and 520d determined by recursively partitioning a non-square second coding unit 510 can be partitioned. According to an embodiment, a square third coding unit 520b among the odd number of third coding units 520b, 520c, and 520d can be horizontally partitioned into a plurality of fourth coding units. A non-square fourth coding unit 530b or 530d among the plurality of fourth coding units 530a, 530b, 530c, and 530d can be further partitioned into a plurality of coding units. For example, the non-square fourth coding unit 530b or 530d can be further partitioned into an odd number of coding units. Methods for recursively partitioning coding units will be described below with respect to various embodiments.

[0114] According to an embodiment, the image decoding device 100 can partition each of the third coding units 520a or 520b, 520c, and 520d based on the partition shape mode information. In addition, the image decoding device 100 can determine not to partition the second coding unit 510 based on the partition shape mode information. According to an embodiment, the image decoding device 100 can partition the non-square second coding unit 510 into an odd number of third coding units 520b, 520c, and 520d. The image decoding device 100 can impose specific restrictions on a specific third coding unit among the odd number of third coding units 520b, 520c, and 520d. For example, the image decoding device 100 can restrict the third coding unit 520c at the center position among the odd number of third coding units 520b, 520c, and 520d from being further partitioned or from being partitioned a set number of times.

[0115] Referring to Figure 5 , the image decoding device 100 can restrict the third coding unit 520c at the center position among the odd number of third coding units 520b, 520c, and 520d included in the non-square second coding unit 510 from being further partitioned, from being partitioned by using a specific partitioning method (e.g., being partitioned only into four coding units or being partitioned by using the partitioning method of the second coding unit 510), or from being partitioned only a specific number of times (e.g., being partitioned only n times (where n > 0)). However, the restrictions on the third coding unit 520c at the center position are not limited to the above examples and can include various restrictions for decoding the third coding unit 520c at the center position differently from the other third coding units 520b and 520d.

[0116] According to an embodiment, the image decoding device 100 can obtain partition shape mode information for partitioning the current coding unit from a specific position in the current coding unit.

[0117] Figure 6Disclosed is a method for an image decoding device according to an embodiment to determine a specific coding unit from an odd number of coding units.

[0118] Referring to Figure 6 , the partitioning shape mode information of the current coding unit 600 or 650 can be obtained from the sample at a specific position among the multiple samples included in the current coding unit 600 or 650 (e.g., the sample 640 or 690 at the center position). However, the specific position in the current coding unit 600 where at least one of the partitioning shape mode information can be obtained is not limited to Figure 6 the center position in

[0119] According to an embodiment, when the current coding unit is partitioned into a specific number of coding units, the image decoding device 100 can select one of the coding units. As will be described regarding various embodiments below, various methods can be used to select one of the multiple coding units.

[0120] According to an embodiment, the image decoding device 100 can partition the current coding unit into multiple coding units and can determine the coding unit at a specific position.

[0121] According to an embodiment, the image decoding device 100 can use the information indicating the positions of an odd number of coding units to determine the coding unit at the center position among the odd number of coding units. Referring to Figure 6 , the image decoding device 100 can determine an odd number of coding units 620a, 620b, and 620c or an odd number of coding units 660a, 660b, and 660c by partitioning the current coding unit 600 or the current coding unit 650. The image decoding device 100 can determine the middle coding unit 620b or the middle coding unit 660b by using the information about the positions of the odd number of coding units 620a, 620b, and 620c or the odd number of coding units 660a, 660b, and 660c. For example, the image decoding device 100 can determine the coding unit 620b at the center position by determining the positions of the coding units 620a, 620b, and 620c based on the information indicating the positions of the specific samples included in the coding units 620a, 620b, and 620c. Specifically, the image decoding device 100 can determine the coding unit 620b at the center position by determining the positions of the coding units 620a, 620b, and 620c based on the information indicating the positions of the upper left samples 630a, 630b, and 630c of the coding units 620a, 620b, and 620c.

[0122] According to an embodiment, the information indicating the positions of the upper-left samples 630a, 630b, and 630c included in the coding units 620a, 620b, and 620c, respectively, may include information about the positions or coordinates of the coding units 620a, 620b, and 620c in the picture. According to an embodiment, the information indicating the positions of the upper-left samples 630a, 630b, and 630c included in the coding units 620a, 620b, and 620c, respectively, may include information indicating the widths or heights of the coding units 620a, 620b, and 620c included in the current coding unit 600, and the width or height may correspond to the information indicating the difference between the coordinates of the coding units 620a, 620b, and 620c in the picture. That is, the image decoding device 100 may determine the coding unit 620b at the center position by directly using the information about the positions or coordinates of the coding units 620a, 620b, and 620c in the picture, or by using the information about the width or height of the coding unit corresponding to the difference between the coordinates.

[0123] According to an embodiment, the information indicating the position of the upper-left sample 630a of the upper coding unit 620a may include the coordinates (xa, ya), the information indicating the position of the upper-left sample 630b of the middle coding unit 620b may include the coordinates (xb, yb), and the information indicating the position of the upper-left sample 630c of the lower coding unit 620c may include the coordinates (xc, yc). The image decoding device 100 may determine the middle coding unit 620b by using the coordinates of the upper-left samples 630a, 630b, and 630c included in the coding units 620a, 620b, and 620c, respectively. For example, when the coordinates of the upper-left samples 630a, 630b, and 630c are sorted in ascending or descending order, the coding unit 620b including the coordinates (xb, yb) of the sample 630b at the center position may be determined as the coding unit at the center position among the coding units 620a, 620b, and 620c determined by dividing the current coding unit 600. However, the coordinates indicating the positions of the upper-left samples 630a, 630b, and 630c may include the coordinates indicating the absolute positions in the picture, or may use the coordinates (dxb, dyb) indicating the relative position of the upper-left sample 630b of the middle coding unit 620b with respect to the upper-left sample 630a of the upper coding unit 620a and the coordinates (dxc, dyc) indicating the relative position of the upper-left sample 630c of the lower coding unit 620c with respect to the upper-left sample 630a of the upper coding unit 620a. The method of determining the coding unit at a specific position by using the coordinates of the samples included in the coding unit as the information indicating the positions of the samples is not limited to the above method, and may include various arithmetic methods capable of using the coordinates of the samples.

[0124] According to an embodiment, the image decoding device 100 may divide the current coding unit 600 into a plurality of coding units 620a, 620b, and 620c, and may select one of the coding units 620a, 620b, and 620c based on a specific criterion. For example, the image decoding device 100 may select the coding unit 620b whose size is different from the sizes of the other coding units from among the coding units 620a, 620b, and 620c.

[0125] According to an embodiment, the image decoding device 100 may determine the width or height of each of the coding units 620a, 620b, and 620c by using the coordinates (xa, ya) which are information indicating the position of the upper left sample 630a of the upper coding unit 620a, the coordinates (xb, yb) which are information indicating the position of the upper left sample 630b of the middle coding unit 620b, and the coordinates (xc, yc) which are information indicating the position of the upper left sample 630c of the lower coding unit 620c. The image decoding device 100 may determine the respective sizes of the coding units 620a, 620b, and 620c by using the coordinates (xa, ya), (xb, yb), and (xc, yc) indicating the positions of the coding units 620a, 620b, and 620c. According to an embodiment, the image decoding device 100 may determine the width of the upper coding unit 620a as the width of the current coding unit 600. The image decoding device 100 may determine the height of the upper coding unit 620a as yb - ya. According to an embodiment, the image decoding device 100 may determine the width of the middle coding unit 620b as the width of the current coding unit 600. The image decoding device 100 may determine the height of the middle coding unit 620b as yc - yb. According to an embodiment, the image decoding device 100 may determine the width or height of the lower coding unit 620c by using the width or height of the current coding unit 600 or the width or height of the upper coding unit 620a and the middle coding unit 620b. The image decoding device 100 may determine a coding unit whose size is different from the sizes of the other coding units based on the determined widths and heights of the coding units 620a to 620c. Referring to Figure 6 , the image decoding device 100 may determine the middle coding unit 620b whose size is different from the sizes of the upper coding unit 620a and the lower coding unit 620c as the coding unit at a specific position. However, the above method by which the image decoding device 100 determines a coding unit whose size is different from the sizes of the other coding units only corresponds to an example of determining a coding unit at a specific position by using the sizes of the coding units determined based on the coordinates of the samples, and thus various methods of determining a coding unit at a specific position by comparing the sizes of the coding units determined based on the coordinates of specific samples may be used.

[0126] The image decoding device 100 can determine the width or height of each of the coding units 660a, 660b, and 660c by using the coordinates (xd, yd) as the information indicating the position of the upper-left sample 670a of the left coding unit 660a, the coordinates (xe, ye) as the information indicating the position of the upper-left sample 670b of the middle coding unit 660b, and the coordinates (xf, yf) as the information indicating the position of the upper-left sample 670c of the right coding unit 660c. The image decoding device 100 can determine the respective sizes of the coding units 660a, 660b, and 660c by using the coordinates (xd, yd), (xe, ye), and (xf, yf) indicating the positions of the coding units 660a, 660b, and 660c.

[0127] According to an embodiment, the image decoding device 100 can determine the width of the left coding unit 660a as xe - xd. The image decoding device 100 can determine the height of the left coding unit 660a as the height of the current coding unit 650. According to an embodiment, the image decoding device 100 can determine the width of the middle coding unit 660b as xf - xe. The image decoding device 100 can determine the height of the middle coding unit 660b as the height of the current coding unit 600. According to an embodiment, the image decoding device 100 can determine the width or height of the right coding unit 660c by using the width or height of the current coding unit 650 or the width or height of the left coding unit 660a and the middle coding unit 660b. The image decoding device 100 can determine a coding unit having a size different from that of other coding units based on the determined widths and heights of the coding units 660a to 660c. Referring to Figure 6 , the image decoding device 100 can determine the middle coding unit 660b having a size different from that of the left coding unit 660a and the right coding unit 660c as a coding unit at a specific position. However, the above method by which the image decoding device 100 determines a coding unit having a size different from that of other coding units only corresponds to an example of determining a coding unit at a specific position by using the sizes of coding units determined based on the coordinates of samples, and thus various methods of determining a coding unit at a specific position by comparing the sizes of coding units determined based on the coordinates of specific samples can be used.

[0128] However, the positions of the samples considered to determine the positions of the coding units are not limited to the above upper-left positions, and information on any position of the samples included in the coding units can be used.

[0129] According to an embodiment, the image decoding device 100 may select a coding unit at a specific position from an odd number of coding units determined by dividing a current coding unit in consideration of the shape of the current coding unit. For example, when the current coding unit has a non-square shape with a width greater than its height, the image decoding device 100 may determine a coding unit at a specific position in the horizontal direction. That is, the image decoding device 100 may determine one of the coding units at different positions in the horizontal direction and restrict the coding unit. When the current coding unit has a non-square shape with a height longer than its width, the image decoding device 100 may determine a coding unit at a specific position in the vertical direction. That is, the image decoding device 100 may determine one of the coding units at different positions in the vertical direction and may restrict the coding unit.

[0130] According to an embodiment, the image decoding device 100 may use information indicating corresponding positions of an even number of coding units to determine a coding unit at a specific position from the even number of coding units. The image decoding device 100 may determine the even number of coding units by dividing the current coding unit (binary division) and may determine a coding unit at a specific position by using information about the positions of the even number of coding units. Operations related thereto may correspond to the operations of determining a coding unit at a specific position (e.g., the center position) from an odd number of coding units described in detail above, and thus detailed descriptions thereof are not provided here. Figure 6 Detailed description of the operation of determining a coding unit at a specific position (e.g., the center position) from an odd number of coding units is provided above, and thus detailed descriptions thereof are not provided here.

[0131] According to an embodiment, when a non-square current coding unit is divided into a plurality of coding units, specific information about a coding unit at a specific position may be used in the division operation to determine a coding unit at a specific position from the plurality of coding units. For example, the image decoding device 100 may use at least one of block shape information and division shape mode information stored in samples included in an intermediate coding unit in the division operation to determine a coding unit located at the center position from the plurality of coding units determined by dividing the current coding unit.

[0132] Refer to Figure 6, the image decoding device 100 may divide the current coding unit 600 into multiple coding units 620a, 620b, and 620c based on the partitioning shape mode information, and may determine the coding unit 620b located at the central position from among the multiple coding units 620a, 620b, and 620c. In addition, the image decoding device 100 may determine the coding unit 620b located at the central position in consideration of the position where the partitioning shape mode information is obtained. That is, the partitioning shape mode information of the current coding unit 600 may be obtained from the sample point 640 at the central position of the current coding unit 600, and when the current coding unit 600 is divided into multiple coding units 620a, 620b, and 620c based on the partitioning shape mode information, the coding unit 620b including the sample point 640 may be determined as the coding unit at the central position. However, the information for determining the coding unit at the central position is not limited to the partitioning shape mode information, and various types of information may be used to determine the coding unit at the central position.

[0133] According to an embodiment, specific information for identifying a coding unit at a specific position may be obtained from a specific sample point included in the coding unit to be determined. Refer to Figure 6 , the image decoding device 100 may use the partitioning shape mode information obtained from the sample point at a specific position in the current coding unit 600 (e.g., the sample point at the central position of the current coding unit 600) to determine the coding unit at a specific position (e.g., the coding unit at the central position among the multiple partitioned coding units) among the multiple coding units 620a, 620b, and 620c determined by partitioning the current coding unit 600. That is, the image decoding device 100 may determine the sample point at a specific position by considering the block shape of the current coding unit 600, may determine the coding unit 620b including the sample point from which specific information (e.g., the partitioning shape mode information) can be obtained from among the multiple coding units 620a, 620b, and 620c determined by partitioning the current coding unit 600, and may impose specific restrictions on the coding unit 620b. Refer to Figure 6 , according to an embodiment, the image decoding device 100 may determine the sample point 640 at the central position of the current coding unit 600 as the sample point from which specific information can be obtained, and may impose specific restrictions on the coding unit 620b including the sample point 640 during the decoding operation. However, the position of the sample point from which specific information can be obtained is not limited to the above position, and may include any position of the sample points included in the coding unit 620b for restriction.

[0134] According to an embodiment, the positions of sample points from which specific information can be obtained may be determined based on the shape of the current coding unit 600. According to an embodiment, the block shape information may indicate whether the current coding unit has a square shape or a non-square shape, and the positions of sample points from which specific information can be obtained may be determined based on the shape. For example, the image decoding device 100 may determine, by using at least one of the information on the width of the current coding unit and the information on the height of the current coding unit, a sample point located on a boundary that divides at least one of the width and the height of the current coding unit in half as a sample point from which specific information can be obtained. In another example, when the block shape information of the current coding unit indicates a non-square shape, the image decoding device 100 may determine one of the sample points included in the sample points of the boundary that divides the long side of the current coding unit in half as a sample point from which preset information can be obtained.

[0135] According to an embodiment, when the current coding unit is divided into a plurality of coding units, the image decoding device 100 may use the partitioning shape mode information to determine a coding unit at a specific position among the plurality of coding units. According to an embodiment, the image decoding device 100 may obtain the partitioning shape mode information from a sample point at a specific position in the coding unit, and may divide the plurality of coding units generated by dividing the current coding unit by using the partitioning shape mode information obtained from the sample points at specific positions in each of the plurality of coding units. That is, the coding unit may be recursively divided based on the partitioning shape mode information obtained from the sample points at specific positions in each coding unit. The operation of recursively dividing the coding unit has been described above, and thus a detailed description thereof will not be provided here. Figure 5 The operation of recursively dividing the coding unit has been described above, and thus a detailed description thereof will not be provided here.

[0136] According to an embodiment, the image decoding device 100 may determine one or more coding units by dividing the current coding unit, and may determine the decoding order of the one or more coding units based on a specific block (e.g., the current coding unit).

[0137] Figure 7 FIG. shows the order of processing a plurality of coding units when the image decoding device determines a plurality of coding units by dividing the current coding unit according to an embodiment.

[0138] According to an embodiment, the image decoding device 100 may determine second coding units 710a and 710b by dividing a first coding unit 700 in a vertical direction based on the partitioning shape mode information, determine second coding units 730a and 730b by dividing the first coding unit 700 in a horizontal direction, or determine second coding units 750a to 750d by dividing the first coding unit 700 in both the vertical direction and the horizontal direction.

[0139] Refer to Figure 7, the image decoding device 100 may determine to process the second coding units 710a and 710b determined by dividing the first coding unit 700 in the vertical direction in a horizontal direction order 710c. The image decoding device 100 may determine to process the second coding units 730a and 730b determined by dividing the first coding unit 700 in the horizontal direction in a vertical direction order 730c. The image decoding device 100 may determine to process the second coding units 750a to 750d determined by dividing the first coding unit 700 in both the vertical and horizontal directions in a specific order for processing coding units in one row and then processing coding units in the next row (e.g., in a raster scan order or a zigzag scan order 750e).

[0140] According to an embodiment, the image decoding device 100 may recursively divide coding units. Referring to Figure 7 , the image decoding device 100 may determine a plurality of coding units 710a and 710b, 730a and 730b, or 750a to 750d by dividing the first coding unit 700, and recursively divide each of the determined plurality of coding units 710a and 710b, 730a and 730b, or 750a to 750d. The method of dividing the plurality of coding units 710a and 710b, 730a and 730b, or 750a to 750d may correspond to the method of dividing the first coding unit 700. In this way, each of the plurality of coding units 710a and 710b, 730a and 730b, or 750a to 750d may be independently divided into a plurality of coding units. Referring to Figure 7 , the image decoding device 100 may determine the second coding units 710a and 710b by dividing the first coding unit 700 in the vertical direction, and may determine to independently divide or not divide each of the second coding units 710a and 710b.

[0141] According to an embodiment, the image decoding device 100 may determine third coding units 720a and 720b by dividing the left second coding unit 710a in the horizontal direction, and may not divide the right second coding unit 710b.

[0142] According to an embodiment, the processing order of a coding unit may be determined based on an operation of dividing the coding unit. In other words, the processing order of the divided coding unit may be determined based on the processing order of the coding unit immediately before being divided. The image decoding device 100 may determine the processing order of the third coding units 720a and 720b determined by dividing the left second coding unit 710a independently of the right second coding unit 710b. Since the third coding units 720a and 720b are determined by dividing the left second coding unit 710a in the horizontal direction, the third coding units 720a and 720b may be processed in the vertical direction order 720c. Since the left second coding unit 710a and the right second coding unit 710b are processed in the horizontal direction order 710c, the right second coding unit 710b may be processed after the third coding units 720a and 720b included in the left second coding unit 710a are processed in the vertical direction order 720c. The operation of determining the processing order of a coding unit based on the coding unit before being divided is not limited to the above example, and various methods may be used to independently process the divided coding units determined to have various shapes in a specific order.

[0143] Figure 8 Illustrated is the processing in which the image decoding device determines that the current coding unit is to be divided into an odd number of coding units when the coding units cannot be processed in a specific order according to an embodiment.

[0144] According to an embodiment, the image decoding device 100 may determine that the current coding unit is divided into an odd number of coding units based on the obtained division shape mode information. Refer to Figure 8 , the square first coding unit 800 may be divided into non-square second coding units 810a and 810b, and the second coding units 810a and 810b may be independently divided into third coding units 820a and 820b and 820c to 820e. According to an embodiment, the image decoding device 100 may determine a plurality of third coding units 820a and 820b by dividing the left second coding unit 810a in the horizontal direction, and may divide the right second coding unit 810b into an odd number of third coding units 820c to 820e.

[0145] According to an embodiment, the image decoding device 100 may determine whether any coding unit is divided into an odd number of coding units by determining whether the third coding units 820a and 820b and 820c to 820e can be processed in a specific order. Refer to Figure 8, the image decoding device 100 may determine the third coding units 820a, 820b, and 820c to 820e by recursively dividing the first coding unit 800. The image decoding device 100 may determine whether any one of the first coding unit 800, the second coding units 810a and 810b, and the third coding units 820a, 820b, and 820c to 820e is divided into an odd number of coding units based on at least one of the block shape information and the division shape mode information. For example, the right second coding unit 810b among the second coding units 810a and 810b may be divided into an odd number of third coding units 820c, 820d, and 820e. The processing order of the multiple coding units included in the first coding unit 800 may be a specific order (e.g., the zigzag scan order 830), and the image decoding device 100 may determine whether the third coding units 820c, 820d, and 820e determined by dividing the right second coding unit 810b into an odd number of coding units satisfy the condition for processing in a specific order.

[0146] According to an embodiment, the image decoding device 100 may determine whether the third coding units 820a, 820b, and 820c to 820e included in the first coding unit 800 satisfy the condition for processing in a specific order, and this condition is related to whether at least one of the width and height of the second coding units 810a and 810b is halved along the boundaries of the third coding units 820a, 820b, and 820c to 820e. For example, the third coding units 820a and 820b determined when the height of the left second coding unit 810a with a non-square shape is halved may satisfy this condition. Since the boundaries of the third coding units 820c to 820e determined when the right second coding unit 810b is divided into three coding units cannot halve the width or height of the right second coding unit 810b, it may be determined that the third coding units 820c to 820e do not satisfy this condition. When the condition is not satisfied as described above, the image decoding device 100 may determine the break of the scan order, and may determine that the right second coding unit 810b is divided into an odd number of coding units based on the determined result. According to an embodiment, when a coding unit is divided into an odd number of coding units, the image decoding device 100 may impose specific restrictions on the coding units at specific positions among the divided coding units. The restrictions or specific positions have been described above with respect to various embodiments, and thus their detailed descriptions will not be provided here.

[0147] Figure 9 Shows the processing of an image decoding device according to an embodiment for determining at least one coding unit by dividing a first coding unit.

[0148] According to an embodiment, the image decoding device 100 may divide the first coding unit 900 based on the partition shape mode information obtained through the receiver 110. The square first coding unit 900 may be divided into four square coding units, or may be divided into a plurality of non-square coding units. For example, referring to Figure 9 , when the partition shape mode information indicates that the first coding unit 900 is to be divided into non-square coding units, the image decoding device 100 may divide the first coding unit 900 into a plurality of non-square coding units. Specifically, when the partition shape mode information indicates that an odd number of coding units are to be determined by dividing the first coding unit 900 in the horizontal or vertical direction, the image decoding device 100 may divide the square first coding unit 900 into an odd number of coding units (e.g., the second coding units 910a, 910b, and 910c determined by dividing the square first coding unit 900 in the vertical direction or the second coding units 920a, 920b, and 920c determined by dividing the square first coding unit 900 in the horizontal direction).

[0149] According to an embodiment, the image decoding device 100 may determine whether the second coding units 910a, 910b, 910c, 920a, 920b, and 920c included in the first coding unit 900 satisfy the condition of being processed in a specific order, and the condition is related to whether at least one of the width and height of the first coding unit 900 is halved along the boundaries of the second coding units 910a, 910b, 910c, 920a, 920b, and 920c. Referring to Figure 9 , since the boundaries of the second coding units 910a, 910b, and 910c determined by dividing the square first coding unit 900 in the vertical direction do not halve the width of the first coding unit 900, it may be determined that the first coding unit 900 does not satisfy the condition of being processed in a specific order. In addition, since the boundaries of the second coding units 920a, 920b, and 920c determined by dividing the square first coding unit 900 in the horizontal direction do not halve the width of the first coding unit 900, it may be determined that the first coding unit 900 does not satisfy the condition of being processed in a specific order. When the condition is not satisfied as described above, the image decoding device 100 may determine a break in the scanning order, and may determine that the first coding unit 900 is divided into an odd number of coding units based on the determined result. According to an embodiment, when a coding unit is divided into an odd number of coding units, the image decoding device 100 may impose a specific restriction on the coding unit at a specific position among the divided coding units. The restrictions or specific positions have been described above with respect to various embodiments, and thus their detailed descriptions will not be provided here.

[0150] According to an embodiment, the image decoding device 100 may determine coding units of various shapes by dividing the first coding unit.

[0151] Refer to Figure 9 As shown in , the image decoding device 100 may divide the square first coding unit 900 or the non-square first coding units 930 or 950 into coding units of various shapes.

[0152] Figure 10 Illustrates that when the second coding unit having a non-square shape determined when dividing the first coding unit in the image decoding device satisfies a specific condition according to an embodiment, the shapes into which the second coding unit can be divided are restricted.

[0153] According to an embodiment, the image decoding device 100 may determine to divide the square first coding unit 1000 into non-square second coding units 1010a and 1010b or 1020a and 1020b based on the division shape mode information obtained by the receiver 110. The second coding units 1010a and 1010b or 1020a and 1020b can be divided independently. Therefore, the image decoding device 100 may determine whether to divide each of the second coding units 1010a and 1010b or 1020a and 1020b into multiple coding units based on the division shape mode information of each of the second coding units 1010a and 1010b or 1020a and 1020b. According to an embodiment, the image decoding device 100 may determine the third coding units 1012a and 1012b by dividing the non-square left second coding unit 1010a determined by dividing the first coding unit 1000 in the vertical direction in the horizontal direction. However, when the left second coding unit 1010a is divided in the horizontal direction, the image decoding device 100 may restrict the right second coding unit 1010b from being divided in the horizontal direction in which the left second coding unit 1010a is divided. When determining the third coding units 1014a and 1014b by dividing the right second coding unit 1010b along the same direction, since the left second coding unit 1010a and the right second coding unit 1010b are divided independently in the horizontal direction, the third coding units 1012a and 1012b or 1014a and 1014b can be determined. However, this situation also serves as a situation where the image decoding device 100 divides the first coding unit 1000 into four square second coding units 1030a, 1030b, 1030c, and 1030d based on the division shape mode information, and may be inefficient in terms of image decoding.

[0154] According to an embodiment, the image decoding device 100 may determine third coding units 1022a and 1022b or 1024a and 1024b by dividing non-square second coding units 1020a or 1020b determined by dividing a first coding unit 1000 in a horizontal direction in a vertical direction. However, when a second coding unit (e.g., the upper second coding unit 1020a) is divided in the vertical direction, for the above reasons, the image decoding device 100 may restrict another second coding unit (e.g., the lower second coding unit 1020b) from being divided in the vertical direction in which the upper second coding unit 1020a is divided.

[0155] Figure 11 FIG. shows a process in which an image decoding device divides a square coding unit when division shape mode information cannot indicate that the square coding unit is divided into four square coding units according to an embodiment.

[0156] According to an embodiment, the image decoding device 100 may determine second coding units 1110a and 1110b or 1120a and 1120b, etc. by dividing a first coding unit 1100 based on division shape mode information. The division shape mode information may include information on various methods of dividing a coding unit, but the information on various division methods may not include information for dividing a coding unit into four square coding units. According to such division shape mode information, the image decoding device 100 may not divide the square first coding unit 1100 into four square second coding units 1130a, 1130b, 1130c, and 1130d. The image decoding device 100 may determine non-square second coding units 1110a and 1110b or 1120a and 1120b, etc. based on the division shape mode information.

[0157] According to an embodiment, the image decoding device 100 may independently divide non-square second coding units 1110a and 1110b or 1120a and 1120b, etc. Each of the second coding units 1110a and 1110b or 1120a and 1120b, etc. may be recursively divided in a specific order, and the division method may correspond to the method of dividing the first coding unit 1100 based on the division shape mode information.

[0158] For example, the image decoding device 100 may determine square third coding units 1112a and 1112b by dividing the second left coding unit 1110a in the horizontal direction, and may determine square third coding units 1114a and 1114b by dividing the second right coding unit 1110b in the horizontal direction. In addition, the image decoding device 100 may determine square third coding units 1116a, 1116b, 1116c, and 1116d by dividing both the second left coding unit 1110a and the second right coding unit 1110b in the horizontal direction. In this case, coding units having the same shape as the four square second coding units 1130a, 1130b, 1130c, and 1130d divided from the first coding unit 1100 may be determined.

[0159] In another example, the image decoding device 100 may determine square third coding units 1122a and 1122b by dividing the second upper coding unit 1120a in the vertical direction, and may determine square third coding units 1124a and 1124b by dividing the second lower coding unit 1120b in the vertical direction. In addition, the image decoding device 100 may determine square third coding units 1126a, 1126b, 1126c, and 1126d by dividing both the second upper coding unit 1120a and the second lower coding unit 1120b in the vertical direction. In this case, coding units having the same shape as the four square second coding units 1130a, 1130b, 1130c, and 1130d divided from the first coding unit 1100 may be determined.

[0160] Figure 12 It is shown that according to an embodiment, the processing order between multiple coding units may be changed based on the processing of dividing coding units.

[0161] According to an embodiment, the image decoding device 100 may divide the first coding unit 1200 based on division shape mode information. When the block shape is a square shape and the division shape mode information indicates dividing the first coding unit 1200 in at least one of the horizontal direction and the vertical direction, the image decoding device 100 may determine the second coding units 1210a and 1210b or 1220a and 1220b, etc. by dividing the first coding unit 1200. Refer to Figure 12, the non-square second coding units 1210a and 1210b or 1220a and 1220b determined by dividing the first coding unit 1200 only in the horizontal or vertical direction can be independently divided based on the division shape mode information of each coding unit. For example, the image decoding device 100 can determine the third coding units 1216a, 1216b, 1216c, and 1216d by dividing the second coding units 1210a and 1210b generated by dividing the first coding unit 1200 in the vertical direction in the horizontal direction, and can determine the third coding units 1226a, 1226b, 1226c, and 1226d by dividing the second coding units 1220a and 1220b generated by dividing the first coding unit 1200 in the horizontal direction in the horizontal direction. The operation of dividing the second coding units 1210a and 1210b or 1220a and 1220b has been described above with respect to Figure 11 , so a detailed description thereof will not be provided here.

[0162] According to an embodiment, the image decoding device 100 can process coding units in a specific order. The operation of processing coding units in a specific order has been described above with respect to Figure 7 , so a detailed description thereof will not be provided herein. Referring to Figure 12 , the image decoding device 100 can determine four square third coding units 1216a, 1216b, 1216c, and 1216d and 1226a, 1226b, 1226c, and 1226d by dividing the square first coding unit 1200. According to an embodiment, the image decoding device 100 can determine the processing order of the third coding units 1216a, 1216b, 1216c, and 1216d and 1226a, 1226b, 1226c, and 1226d based on the division method of the first coding unit 1200.

[0163] According to an embodiment, the image decoding device 100 can determine the third coding units 1216a, 1216b, 1216c, and 1216d by dividing the second coding units 1210a and 1210b generated by dividing the first coding unit 1200 in the vertical direction in the horizontal direction, and can process the third coding units 1216a, 1216b, 1216c, and 1216d in the processing order 1217 so as to first process the third coding units 1216a and 1216c included in the left second coding unit 1210a in the vertical direction, and then process the third coding units 1216b and 1216d included in the right second coding unit 1210b in the vertical direction.

[0164] According to an embodiment, the image decoding device 100 may determine third coding units 1226a, 1226b, 1226c, and 1226d by dividing second coding units 1220a and 1220b generated by dividing a first coding unit 1200 in a horizontal direction in both the horizontal and vertical directions, and may process the third coding units 1226a, 1226b, 1226c, and 1226d in processing order 1227 so as to process, first in the horizontal direction, the third coding units 1226a and 1226b included in the upper second coding unit 1220a, and then process the third coding units 1226c and 1226d included in the lower second coding unit 1220b in the horizontal direction.

[0165] Referring Figure 12 , square third coding units 1216a, 1216b, 1216c, and 1216d and 1226a, 1226b, 1226c, and 1226d may be determined by dividing second coding units 1210a and 1210b and 1220a and 1220b, respectively. Although the second coding units 1210a and 1210b are determined by dividing the first coding unit 1200 in a vertical direction differently from the second coding units 1220a and 1220b determined by dividing the first coding unit 1200 in a horizontal direction, the divided third coding units 1216a, 1216b, 1216c, and 1216d and 1226a, 1226b, 1226c, and 1226d ultimately show coding units of the same shape divided from the first coding unit 1200. Accordingly, by recursively dividing coding units in different ways based on the division shape pattern information, even when the coding units are ultimately determined to be of the same shape, the image decoding device 100 may process multiple coding units in different orders.

[0166] Figure 13 Shows the process of determining the depth of a coding unit when the shape and size of the coding unit change when the coding unit is recursively divided such that multiple coding units are determined according to an embodiment.

[0167] According to an embodiment, the image decoding device 100 may determine the depth of a coding unit based on a specific criterion. For example, the specific criterion may be the length of the long side of the coding unit. When the length of the long side of the coding unit before being divided is 2n times (n > 0) the length of the long side of the current coding unit being divided, the image decoding device 100 may determine that the depth of the current coding unit increases by n from the depth of the coding unit before being divided. In the following description, a coding unit with an increased depth is represented as a coding unit with a lower depth.

[0168] Referring Figure 13, according to an embodiment, the image decoding device 100 may determine a second coding unit 1302 and a third coding unit 1304 of a lower depth by dividing a square first coding unit 1300 based on block shape information indicating a square shape (e.g., the block shape information may be expressed as "0:SQUARE"). Assuming that the size of the square first coding unit 1300 is 2N×2N, the second coding unit 1302 determined by dividing the width and height of the first coding unit 1300 into 1 / 2 may have a size of N×N. In addition, the third coding unit 1304 determined by dividing the width and height of the second coding unit 1302 into 1 / 2 may have a size of N / 2×N / 2. In this case, the width and height of the third coding unit 1304 are 1 / 4 times the width and height of the first coding unit 1300. When the depth of the first coding unit 1300 is D, the depth of the second coding unit 1302, whose width and height are 1 / 2 times the width and height of the first coding unit 1300, may be D+1, and the depth of the third coding unit 1304, whose width and height are 1 / 4 times the width and height of the first coding unit 1300, may be D+2.

[0169] According to an embodiment, the image decoding device 100 may determine a second coding unit 1312 or 1322 and a third coding unit 1314 or 1324 of a lower depth by dividing a non-square first coding unit 1310 or 1320 based on block shape information indicating a non-square shape (e.g., the block shape information may be expressed as "1:NS_VER" indicating a non-square shape with a height longer than the width, or as "2:NS_HOR" indicating a non-square shape with a width longer than the height).

[0170] The image decoding device 100 may determine the second coding unit 1302, 1312, or 1322 by dividing at least one of the width and height of a first coding unit 1310 having a size of N×2N. That is, the image decoding device 100 may determine the second coding unit 1302 having a size of N×N or the second coding unit 1322 having a size of N×N / 2 by dividing the first coding unit 1310 in the horizontal direction, or may determine the second coding unit 1312 having a size of N / 2×N by dividing the first coding unit 1310 in both the horizontal and vertical directions.

[0171] According to an embodiment, the image decoding device 100 may determine a second coding unit 1302, 1312, or 1322 by dividing at least one of the width and height of a first coding unit 1320 having a size of 2N×N. That is, the image decoding device 100 may determine a second coding unit 1302 having a size of N×N or a second coding unit 1312 having a size of N / 2×N by dividing the first coding unit 1320 in the vertical direction, or may determine a second coding unit 1322 having a size of N×N / 2 by dividing the first coding unit 1320 in both the horizontal and vertical directions.

[0172] According to an embodiment, the image decoding device 100 may determine a third coding unit 1304, 1314, or 1324 by dividing at least one of the width and height of a second coding unit 1302 having a size of N×N. That is, the image decoding device 100 may determine a third coding unit 1304 having a size of N / 2×N / 2, a third coding unit 1314 having a size of N / 4×N / 2, or a third coding unit 1324 having a size of N / 2×N / 4 by dividing the second coding unit 1302 in both the vertical and horizontal directions.

[0173] According to an embodiment, the image decoding device 100 may determine a third coding unit 1304, 1314, or 1324 by dividing at least one of the width and height of a second coding unit 1312 having a size of N / 2×N. That is, the image decoding device 100 may determine a third coding unit 1304 having a size of N / 2×N / 2 or a third coding unit 1324 having a size of N / 2×N / 4 by dividing the second coding unit 1312 in the horizontal direction, or may determine a third coding unit 1314 having a size of N / 4×N / 2 by dividing the second coding unit 1312 in both the vertical and horizontal directions.

[0174] According to an embodiment, the image decoding device 100 may determine a third coding unit 1304, 1314, or 1324 by dividing at least one of the width and height of a second coding unit 1322 having a size of N×N / 2. That is, the image decoding device 100 may determine a third coding unit 1304 having a size of N / 2×N / 2 or a third coding unit 1314 having a size of N / 4×N / 2 by dividing the second coding unit 1322 in the vertical direction, or may determine a third coding unit 1324 having a size of N / 2×N / 4 by dividing the second coding unit 1322 in both the vertical and horizontal directions.

[0175] According to an embodiment, the image decoding device 100 may divide the square coding units 1300, 1302, or 1304 in the horizontal or vertical direction. For example, the image decoding device 100 may determine a first coding unit 1310 with a size of N×2N by dividing a first coding unit 1300 with a size of 2N×2N in the vertical direction, or may determine a first coding unit 1320 with a size of 2N×N by dividing the first coding unit 1300 in the horizontal direction. According to an embodiment, when determining the depth based on the length of the longest side of the coding unit, the depth of the coding unit determined by dividing the first coding unit 1300 with a size of 2N×2N in the horizontal or vertical direction may be the same as the depth of the first coding unit 1300.

[0176] According to an embodiment, the width and height of the third coding units 1314 or 1324 may be 1 / 4 times the width and height of the first coding units 1310 or 1320. When the depth of the first coding units 1310 or 1320 is D, the depth of the second coding units 1312 or 1322 with a width and height that are 1 / 2 times the width and height of the first coding units 1310 or 1320 may be D + 1, and the depth of the third coding units 1314 or 1324 with a width and height that are 1 / 4 times the width and height of the first coding units 1310 or 1320 may be D + 2.

[0177] Figure 14 Shows the depth that can be determined based on the shape and size of the coding unit and the partial index (PID) used to distinguish the coding unit according to an embodiment.

[0178] According to an embodiment, the image decoding device 100 may determine second coding units of various shapes by dividing the square first coding unit 1400. Refer to Figure 14 , the image decoding device 100 may determine second coding units 1402a and 1402b, 1404a and 1404b, and 1406a, 1406b, 1406c, and 1406d by dividing the first coding unit 1400 in at least one of the vertical and horizontal directions based on the division shape pattern information. That is, the image decoding device 100 may determine the second coding units 1402a and 1402b, 1404a and 1404b, and 1406a, 1406b, 1406c, and 1406d based on the division shape pattern information of the first coding unit 1400.

[0179] According to an embodiment, the depths of second coding units 1402a and 1402b, 1404a and 1404b, and 1406a, 1406b, 1406c, and 1406d that can be determined based on the division shape pattern information of the square first coding unit 1400 can be determined based on the length of their long sides. For example, since the length of the side of the square first coding unit 1400 is equal to the length of the long sides of the non-square second coding units 1402a and 1402b, and 1404a and 1404b, the first coding unit 2100 and the non-square second coding units 1402a and 1402b, and 1404a and 1404b can have the same depth (e.g., D). However, when the image decoding device 100 divides the first coding unit 1400 into four square second coding units 1406a, 1406b, 1406c, and 1406d based on the division shape pattern information, since the length of the side of the square second coding units 1406a, 1406b, 1406c, and 1406d is 1 / 2 times the length of the side of the first coding unit 1400, the depth of the second coding units 1406a, 1406b, 1406c, and 1406d can be D + 1, which is 1 deeper than the depth D of the first coding unit 1400.

[0180] According to an embodiment, the image decoding device 100 can determine a plurality of second coding units 1412a and 1412b, and 1414a, 1414b, and 1414c by dividing a first coding unit 1410 having a height longer than its width in the horizontal direction based on the division shape pattern information. According to an embodiment, the image decoding device 100 can determine a plurality of second coding units 1422a and 1422b, and 1424a, 1424b, and 1424c by dividing a first coding unit 1420 having a width longer than its height in the vertical direction based on the division shape pattern information.

[0181] According to an embodiment, the depths of second coding units 1412a and 1412b, and 1414a, 1414b, and 1414c or 1422a and 1422b, and 1424a, 1424b, and 1424c that are determined based on the division shape pattern information of the non-square first coding unit 1410 or 1420 can be determined based on the length of their long sides. For example, since the length of the side of the square second coding units 1412a and 1412b is 1 / 2 times the length of the long side of the non-square first coding unit 1410 having a height longer than its width, the depth of the square second coding units 1412a and 1412b is D + 1, which is 1 deeper than the depth D of the non-square first coding unit 1410.

[0182] In addition, the image decoding device 100 may divide the non-square first coding unit 1410 into an odd number of second coding units 1414a, 1414b, and 1414c based on the division shape pattern information. The odd number of second coding units 1414a, 1414b, and 1414c may include non-square second coding units 1414a and 1414c and a square second coding unit 1414b. In this case, since the lengths of the long sides of the non-square second coding units 1414a and 1414c and the length of the side of the square second coding unit 1414b are 1 / 2 times the length of the long side of the first coding unit 1410, the depth of the second coding units 1414a, 1414b, and 1414c may be D + 1 which is 1 deeper than the depth D of the non-square first coding unit 1410. The image decoding device 100 may determine the depth of the coding units divided from the first coding unit 1420 having a non-square shape with a width longer than the height by using the method of determining the depth of the coding units divided from the first coding unit 1410 as described above.

[0183] According to an embodiment, when the odd number of divided coding units do not have equal sizes, the image decoding device 100 may determine the PID for identifying the divided coding units based on the size ratio between the coding units. Refer to Figure 14 , among the odd number of divided coding units 1414a, 1414b, and 1414c, the coding unit 1414b at the central position may have a width equal to the widths of the other coding units 1414a and 1414c and a height twice the height of the other coding units 1414a and 1414c. That is, in this case, the coding unit 1414b at the central position may include two of the other coding units 1414a or 1414c. Accordingly, when the PID of the coding unit 1414b at the central position is 1 based on the scanning order, the PID of the coding unit 1414c adjacent to the coding unit 1414b may increase by 2 and thus may be 3. That is, there may be a discontinuity in the PID. According to an embodiment, the image decoding device 100 may determine whether the odd number of divided coding units do not have equal sizes based on whether there is a discontinuity in the PID for identifying the divided coding units.

[0184] According to an embodiment, the image decoding device 100 may determine whether to use a specific division method based on the PID for identifying a plurality of coding units determined by dividing the current coding unit. Refer to Figure 14, the image decoding device 100 may determine an even number of coded units 1412a and 1412b or an odd number of coded units 1414a, 1414b, and 1414c by dividing a first coded unit 1410 having a rectangular shape with a height longer than the width. The image decoding device 100 may use a PID indicating each coded unit to identify each coded unit. According to an embodiment, the PID may be obtained from samples at a specific position (e.g., the upper left sample) of each coded unit.

[0185] According to an embodiment, the image decoding device 100 may determine a coded unit at a specific position from the divided coded units by using a PID for differentiating the coded units. According to an embodiment, when the division shape mode information of the first coded unit 1410 having a rectangular shape with a height longer than the width indicates that the coded unit is divided into three coded units, the image decoding device 100 may divide the first coded unit 1410 into three coded units 1414a, 1414b, and 1414c. The image decoding device 100 may assign a PID to each of the three coded units 1414a, 1414b, and 1414c. The image decoding device 100 may compare the PIDs of the odd number of divided coded units to determine a coded unit located at the central position from the coded units. The image decoding device 100 may determine the coded unit 1414b having a PID corresponding to the median value among the PIDs of the coded units as the coded unit at the central position among the coded units determined by dividing the first coded unit 1410. According to an embodiment, when the divided coded units do not have equal sizes, the image decoding device 100 may determine a PID for differentiating the divided coded units based on the size ratio between the coded units. Refer to Figure 14, the coding unit 1414b generated by dividing the first coding unit 1410 may have a width equal to the widths of the other coding units 1414a and 1414c and a height twice the height of the other coding units 1414a and 1414c. In this case, when the PID of the coding unit 1414b at the center position is 1, the PID of the coding unit 1414c adjacent to the coding unit 1414b may be increased by 2 and thus may be 3. When the PID is not increased uniformly as described above, the image decoding device 100 may determine that the coding unit is divided into a plurality of coding units including a coding unit having a size different from the sizes of the other coding units. According to an embodiment, when the division shape mode information indicates that the coding unit is divided into an odd number of coding units, the image decoding device 100 may divide the current coding unit in such a manner that a coding unit at a specific position (e.g., the coding unit at the center position) among the odd number of coding units has a size different from the sizes of the other coding units. In this case, the image decoding device 100 may determine the coding unit at the center position having a different size by using the PID of the coding unit. However, the PID and the size or position of the coding unit at the specific position are not limited to the above examples, and various PIDs and various positions and sizes of the coding units may be used.

[0186] According to an embodiment, the image decoding device 100 may use a specific data unit that starts recursively dividing the coding unit.

[0187] Figure 15 FIG. shows determining a plurality of coding units based on a plurality of specific data units included in a picture according to an embodiment.

[0188] According to an embodiment, the specific data unit may be defined as a data unit that starts recursively dividing the coding unit by using the division shape mode information. That is, the specific data unit may correspond to the coding unit for determining the highest depth of the plurality of coding units divided from the current picture. In the following description, for convenience of description, the specific data unit will be referred to as a reference data unit.

[0189] According to an embodiment, the reference data unit may have a specific size and a specific shape. According to an embodiment, the reference data unit may include M×N samples. Here, M and N may be equal to each other and may be integers represented as a power of 2. That is, the reference data unit may have a square or non-square shape and may be divided into an integer number of coding units.

[0190] According to an embodiment, the image decoding device 100 may divide a current picture into a plurality of reference data units. According to an embodiment, the image decoding device 100 may divide the plurality of reference data units divided from the current picture by using the division shape mode information of each reference data unit. The operation of dividing the reference data unit may correspond to the division operation using a quadtree structure.

[0191] According to an embodiment, the image decoding device 100 may pre-determine the minimum size allowed for a reference data unit included in the current picture. Thus, the image decoding device 100 may determine various reference data units having a size equal to or greater than the minimum size, and may determine one or more coding units by referring to the determined reference data units and using the division shape mode information.

[0192] Refer to Figure 15 , the image decoding device 100 may use a square reference coding unit 1500 or a non-square reference coding unit 1502. According to an embodiment, the shape and size of the reference coding unit may be determined based on various data units capable of including one or more reference coding units (e.g., sequence, picture, slice, slice segment, parallel block, parallel block group, largest coding unit, etc.).

[0193] According to an embodiment, the receiver 110 of the image decoding device 100 may obtain at least one of the reference coding unit shape information and the reference coding unit size information for each of the various data units from the bitstream. The operation of dividing the square reference coding unit 1500 into one or more coding units has been described above with respect to the operation of dividing the current coding unit 300 of Figure 3 , and the operation of dividing the non-square reference coding unit 1502 into one or more coding units has been described above with respect to the operation of dividing the current coding unit 400 or 450 of Figure 4 . Therefore, a detailed description thereof will not be provided here.

[0194] According to an embodiment, the image decoding device 100 may use a PID for identifying the size and shape of a reference coding unit to determine the size and shape of the reference coding unit based on some data units predetermined according to specific conditions. That is, the receiver 110 may obtain from the bitstream only the PID for identifying the size and shape of the reference coding unit for each slice, slice segment, parallel block, parallel block group, maximum coding unit, etc. among various data units (e.g., sequence, picture, slice, slice segment, parallel block, parallel block group, maximum coding unit, etc.) that are data units satisfying specific conditions (e.g., data units having a size equal to or smaller than a slice). The image decoding device 100 may determine the size and shape of the reference data unit for each data unit satisfying the specific conditions by using the PID. When the reference coding unit shape information and the reference coding unit size information are obtained from and used with respect to each data unit having a relatively small size, the efficiency of using the bitstream may not be high. Thus, only the PID may be obtained and used instead of directly obtaining the reference coding unit shape information and the reference coding unit size information. In this case, at least one of the size and shape of the reference coding unit corresponding to the PID for identifying the size and shape of the reference coding unit may be predetermined. That is, the image decoding device 100 may determine at least one of the size and shape of the reference coding unit included in the data unit used as the unit for obtaining the PID by selecting at least one of the previously determined size and shape of the reference coding unit based on the PID.

[0195] According to an embodiment, the image decoding device 100 may use one or more reference coding units included in a maximum coding unit. That is, the maximum coding unit divided from a picture may include one or more reference coding units, and coding units may be determined by recursively dividing each reference coding unit. According to an embodiment, at least one of the width and height of the maximum coding unit may be an integer multiple of at least one of the width and height of the reference coding unit. According to an embodiment, the size of the reference coding unit may be obtained by dividing the maximum coding unit n times based on a quadtree structure. That is, according to various embodiments, the image decoding device 100 may determine the reference coding unit by dividing the maximum coding unit n times based on a quadtree structure, and may divide the reference coding unit based on at least one of block shape information and division shape pattern information.

[0196] According to an embodiment, the image decoding device 100 may obtain block shape information indicating the shape of a current coding unit or partitioning shape mode information indicating a partitioning method of the current coding unit from a bitstream, and may use the obtained information. The partitioning shape mode information may be included in the bitstream related to various data units. For example, the image decoding device 100 may use the partitioning shape mode information included in a sequence parameter set, a picture parameter set, a video parameter set, a slice header, a slice segment header, a parallel block header, or a parallel block group header. In addition, the image decoding device 100 may obtain a syntax element corresponding to the block shape information or the partitioning shape mode information from the bitstream according to each maximum coding unit, each reference coding unit, or each processing block, and may use the obtained syntax element.

[0197] Hereinafter, a method for determining a partitioning rule according to an embodiment of the present disclosure will be described in detail.

[0198] The image decoding device 100 may determine a partitioning rule of an image. The partitioning rule may be predetermined between the image decoding device 100 and the image encoding device 2200. The image decoding device 100 may determine the partitioning rule of the image based on the information obtained from the bitstream. The image decoding device 100 may determine the partitioning rule based on the information obtained from at least one of a sequence parameter set, a picture parameter set, a video parameter set, a slice header, a slice segment header, a parallel block header, or a parallel block group header. The image decoding device 100 may determine the partitioning rule differently according to a frame, a slice, a parallel block, a temporal layer, a maximum coding unit, or a coding unit.

[0199] The image decoding device 100 may determine the partitioning rule based on the block shape of a coding unit. The block shape may include the size, shape, aspect ratio, and orientation of the coding unit. The image decoding device 100 may predetermine based on the block shape of the coding unit to determine the partitioning rule. However, the embodiment is not limited thereto. The image decoding device 100 may determine the partitioning rule of the image based on the information obtained from the received bitstream.

[0200] The shape of a coding unit may include a square and a non-square. When the lengths of the width and height of the coding unit are the same, the image decoding device 100 may determine the shape of the coding unit as a square. In addition, when the lengths of the width and height of the coding unit are different, the image decoding device 100 may determine the shape of the coding unit as a non-square.

[0201] The size of a coding unit may include various sizes (such as 4×4, 8×4, 4×8, 8×8, 16×4, 16×8, …, and 256×256). The size of the coding unit may be classified based on the length of the long side, the length of the short side, or the area of the coding unit. The image decoding device 100 may apply the same partitioning rule to coding units classified into the same group. For example, the image decoding device 100 may classify coding units having the same long side length as having the same size. In addition, the image decoding device 100 may apply the same partitioning rule to coding units having the same long side length.

[0202] The aspect ratio of the coding unit may include 1:2, 2:1, 1:4, 4:1, 1:8, 8:1, 1:16, 16:1, 32:1, 1:32, etc. In addition, the direction of the coding unit may include a horizontal direction and a vertical direction. The horizontal direction may indicate a case where the length of the width of the coding unit is longer than the length of its height. The vertical direction may indicate a case where the length of the width of the coding unit is shorter than the length of its height.

[0203] The image decoding device 100 may adaptively determine a partitioning rule based on the size of the coding unit. The image decoding device 100 may differently determine an allowed partitioning shape mode based on the size of the coding unit. For example, the image decoding device 100 may determine whether partitioning is allowed based on the size of the coding unit. The image decoding device 100 may determine a partitioning direction according to the size of the coding unit. The image decoding device 100 may determine an allowed partitioning type according to the size of the coding unit.

[0204] The partitioning rule determined based on the size of the coding unit may be a partitioning rule predetermined in the image decoding device 100. In addition, the image decoding device 100 may determine a partitioning rule based on information obtained from a bitstream.

[0205] The image decoding device 100 may adaptively determine a partitioning rule based on the position of the coding unit. The image decoding device 100 may adaptively determine a partitioning rule based on the position of the coding unit in the image.

[0206] In addition, the image decoding device 100 may determine a partitioning rule such that coding units generated via different partitioning paths do not have the same block shape. However, the embodiment is not limited thereto, and coding units generated via different partitioning paths have the same block shape. Coding units generated via different partitioning paths may have different decoding processing orders. Since the decoding processing order has been described above with reference to Figure 12 it, a detailed description thereof will not be provided again.

[0207] Figure 16 is a block diagram of an image encoding and decoding system.

[0208] The encoder 1610 of the image encoding and decoding system 1600 transmits an encoded bitstream of an image, and the decoder 1650 receives the bitstream, decodes the bitstream, and outputs a reconstructed image. The decoder 1650 may have a configuration similar to that of the image decoding device 100.

[0209] In the encoder 1610, when the prediction mode of the current block is an inter prediction mode, the inter predictor 1605 generates motion information of the current block indicating a reference block of a reference picture temporally adjacent to the current picture. The inter predictor 1605 may generate prediction samples of the current block by using samples of the reference block. The intra predictor 1610 may determine intra prediction information indicating a direction in which neighboring samples similar to the current block are located or a method of determining prediction samples such that prediction samples of the current block are determined by using neighboring samples spatially adjacent to the current block. The inter predictor 1605 may determine reference samples to be used for prediction of the current block from previously reconstructed samples stored in a decoded picture buffer (DPB) 1648.

[0210] The transformer 1620 outputs transform coefficients by performing a transform on residual sample values obtained by subtracting prediction samples generated by the inter predictor 1605 or the intra predictor 1610 from original samples of the current block. The quantizer 1625 outputs quantized transform coefficients by quantizing the transform coefficients output from the transformer 1620. The entropy encoder 1630 may output a bitstream by encoding the quantized transform coefficients into residual syntax elements including level values.

[0211] The inverse quantizer 1633 and the inverse transformer 1635 may inverse quantize and inverse transform the quantized transform coefficients output from the quantizer 1625, and may generate residual sample values again.

[0212] The adder 1615 outputs reconstructed sample values obtained by adding the residual sample values and the prediction sample values. The post - reconstruction filter 1640 may perform post - reconstruction filtering on the reconstructed samples, and the reconstructed sample values updated by the post - reconstruction filtering may be used as reference sample values for intra prediction to be performed by the intra predictor 1610. The post - reconstruction filter 1640 may perform Hadamard transform domain filtering or bilateral filtering on the reconstructed sample values.

[0213] The loop filter 1645 may perform at least one of deblocking filtering and adaptive loop filtering on the reconstructed samples updated by the post - reconstruction filtering. The reconstructed sample values filtered by the loop filter 1645 may be stored in the DPB 1648, and may be used as reference sample values for inter prediction to be performed by the inter predictor 1605.

[0214] The entropy decoder 1655 of the decoder 1650 may perform entropy decoding on the received bitstream and may parse the residual syntax elements including the level values. The entropy decoder 1655 may reconstruct the quantized transform coefficients from the residual syntax elements. The inverse quantizer 1660 may output the transform coefficients by performing inverse quantization on the quantized transform coefficients, and the inverse transformer 1665 may output the residual sample values by performing inverse transformation on the transform coefficients.

[0215] The inter-predictor 1670 of the decoder 1650 may determine the reference picture temporally adjacent to the current picture by using the motion information of the current block parsed by the entropy decoder 1655 and may determine the reference block in the reference picture. The inter-predictor 1670 may determine the predicted sample of the current block by using the samples of the reference block. The intra-predictor 1675 of the decoder 1650 may determine the reference samples spatially adjacent to the current block by using the intra-prediction information by using the motion information of the current block parsed by the entropy decoder 1655 and may determine the predicted sample of the current block by using the determined neighboring samples. The inter-predictor 1670 may determine the reference samples to be used for the prediction of the current block from the previously reconstructed samples stored in the DPB 1690.

[0216] The adder 1695 of the decoder 1650 outputs the reconstructed sample values obtained by adding the residual sample values and the predicted sample values. The post-reconstruction filter 1680 of the decoder 1650 may perform Hadamard transform domain filtering or bilateral filtering on the reconstructed sample values. The reconstructed sample values updated by the filtering of the post-reconstruction filter 1680 may be used as the reference sample values for the intra-prediction to be performed by the intra-predictor 1675.

[0217] The loop filter 1685 of the decoder 1650 may perform at least one of deblocking filtering and adaptive loop filtering on the reconstructed samples updated by the post-reconstruction filtering. The reconstructed sample values updated by the filtering of the loop filter 1685 may be stored in the DPB 1690 and may be used as the reference sample values for the inter-prediction to be performed by the inter-predictor 1670.

[0218] A video encoding method, a decoding method, a video encoding device, and a decoding device according to an embodiment propose a method of performing prediction based on an affine model by considering the encoding order of encoding units. Hereinafter, reference will be made to Figures 17 to 27 A method and a device for encoding or decoding a video by performing post-reconstruction filtering according to an embodiment of the present disclosure will be described in detail.

[0219] Hereinafter, the term "maximum size of an encoding unit" refers to the maximum size of the long side among the width and height of the encoding unit, and the term "minimum size of an encoding unit" refers to the minimum size of the long side among the width and height of the encoding unit.

[0220] Hereinafter, the term "tree structure" may refer to a hierarchical structure of one or more coding units formed according to whether the partitioning pattern of the coding units is a quaternary partitioning pattern, a binary partitioning pattern, a ternary partitioning pattern, or a non-partitioning pattern. For example, the hierarchical structure of blocks generated from a current coding unit according to Figure 5 the partitioning process is referred to as a tree structure.

[0221] Hereinafter, the term "availability of a block" refers to whether the block has been encoded or decoded and whether the information of the block is available. Specifically, when the current block has been encoded in the encoding process, the neighboring blocks can be encoded by using the encoding information of the current block, so the current block can be marked as available. When the current block has not been encoded, the current block can be marked as unavailable. Similarly, when the current block has been decoded in the decoding process, since the neighboring blocks can be decoded by using the encoding information of the current block, the current block can be marked as available. When the current block has not been decoded, the current block can be marked as unavailable.

[0222] Hereinafter, the term "availability of motion information of a block" refers to whether motion prediction (prediction other than prediction according to the intra mode or intra block copy mode) is performed on the block and whether the motion information of the block (motion vector, prediction direction (L0-pred, L1-pred, or Bi-pred), and reference picture index) is available. Specifically, when motion prediction has been performed on the current block in the encoding process and there is motion information of the current block, the motion prediction of the neighboring blocks can be performed by using the motion information of the current block, so the motion information of the current block can be marked as available. When motion prediction is not performed on the current block in the encoding process, the motion information of the current block can be marked as unavailable. Similarly, when motion prediction has been performed on the current block in the decoding process and there is motion information of the current block, the motion prediction of the neighboring blocks can be performed by using the motion information of the current block, and thus the motion information of the current block can be marked as available. When motion prediction is not performed on the current block in the decoding process, the motion information of the current block can be marked as unavailable.

[0223] Hereinafter, the term "affine merge candidate" may correspond to a control point vector corresponding to a neighboring block of a current block or a block group. Since the control point vector is determined from the motion vectors of neighboring blocks or the control point vector is determined based on the motion vectors of neighboring blocks belonging to the block group, each control point vector may correspond to a neighboring block or a block group. Accordingly, in this specification, for ease of description, the term "affine merge candidate" may correspond to a control point vector determined from a neighboring block or a block group, or may correspond to a neighboring block or a block group, and there is no difference in the meaning of the two expressions. Figure 17 is a block diagram of a video decoding device according to an embodiment.

[0224] Referring to Figure 17 , the video decoding device 1700 according to an embodiment may include an affine merge candidate list determiner 1710 and an affine mode predictor 1720.

[0225] The video decoding device 1700 may obtain a bitstream generated as a result of image encoding, may identify positions of blocks divided from a picture based on information included in the bitstream, and may decode blocks (such as a largest coding unit and a coding unit).

[0226] The video decoding device 1700 according to an embodiment may include an intermediate processor (not shown) for controlling the affine merge candidate list determiner 1710 and the affine mode predictor 1720. Optionally, the affine merge candidate list determiner 1710 and the affine mode predictor 1720 may be operated by their own processors (not shown), and the processors (not shown) may be systematically operated with each other to operate the video decoding device 1700. Optionally, the affine merge candidate list determiner 1710 and the affine mode predictor 1720 may be controlled under the control of an external processor (not shown) of the video decoding device 1700.

[0227] The video decoding device 1700 may include one or more data memories (not shown) storing input / output data of the affine merge candidate list determiner 1710 and the affine mode predictor 1720. The video decoding device 1700 may include a memory controller (not shown) for controlling data input and output of the data memory (not shown).

[0228] The video decoding device 1700 may perform an image decoding operation including prediction by operating in connection with an internal video decoding processor or an external video decoding processor, so as to reconstruct an image via image decoding. The internal video decoding processor of the video decoding device 1700 according to an embodiment may perform a basic graphic decoding operation not only in a manner of a separate processor but also in a manner of an image decoding processing module included in an intermediate processing device or a graphic processing device.

[0229] The video decoding device 1700 may be included in the image decoding device 100. For example, the affine merge candidate list determiner 1710 and the affine mode predictor 1720 may correspond to a receiver 110 and a decoder 120 of the image decoding device 100, respectively. The video decoding device 1700 may correspond to the decoder 1650 of the image encoding and decoding system described with reference to Figure 16 . For example, the affine merge candidate list determiner 1710 and the affine mode predictor 1720 may correspond to an entropy decoder 1655 and an inter-frame predictor 1670 of the decoder 1650, respectively.

[0230] The video decoding device 1700 receives a bitstream generated as a result of image encoding. The bitstream may include information about the current picture. The picture may include one or more largest coding units. The video decoding device 1700 may determine the position of the current block in the picture based on the information obtained from the bitstream. The current block, which is a block generated by partitioning the picture according to a tree structure, may correspond to, for example, a largest coding unit or a coding unit. The video decoding device 1700 may determine whether to further partition the current block into sub-blocks of a lower depth, and may determine the tree structure of the current block. The lower depth can be determined by the number of times of performing partitioning from the current block to the sub-blocks by increasing the depth compared to the current depth of the current block. The blocks located in the leaves of the blocks constituting the tree structure included in the current picture are no longer partitioned. Accordingly, the video decoding device 1700 may perform inverse quantization, inverse transformation, and prediction on one or more blocks that are no longer partitioned to decode these blocks.

[0231] The video decoding device 1700 may generate predicted samples of the current block by performing prediction on the current block. The video decoding device 1700 may generate residual samples of the current block by performing inverse transformation on the current block. The reconstructor 1920 may generate reconstructed samples of the current block by using the predicted samples of the current block and the residual samples of the current block. The video decoding device 1700 may reconstruct the current picture by reconstructing the samples of each block.

[0232] For example, when the prediction mode of the current block is the intra mode, the video decoding device 1700 may determine reference samples from the samples of spatially adjacent blocks in the intra prediction direction by using the intra prediction information of the current block, and may determine the predicted samples corresponding to the current block by using the reference samples.

[0233] For example, when the prediction mode of the current block is the inter mode, the video decoding device 1700 may reconstruct the current block by using the motion vector of the current block. The video decoding device 1700 may determine a reference block in the reference picture by using the motion vector of the current block, and may determine the predicted samples corresponding to the current block from the reference samples included in the reference block. The video decoding device 1700 may reconstruct the transform coefficients by using the transform coefficient levels obtained from the bitstream, and may reconstruct the residual samples by performing inverse quantization and inverse transformation on the transform coefficients. The video decoding device 1700 may determine the reconstructed samples of the current block by combining the predicted samples corresponding to the current block and the residual samples.

[0234] When predicting the current block in the skip mode, the video decoding device 1700 does not need to parse the transform coefficients of the current block from the bitstream. The video decoding device 1700 may determine the reconstructed samples of the current block by using the predicted samples of the current block.

[0235] Specifically, when performing inter prediction of a current block based on an affine model, the affine merge candidate list determiner 1710 may be called. The video decoding device 1700 according to an embodiment may obtain from a bitstream a merge mode flag indicating whether the inter prediction mode of the current block is a merge mode, and when the merge mode flag indicates the merge mode, the video decoding device 1700 may obtain from the bitstream an affine flag indicating whether to perform motion compensation based on an affine model to generate prediction samples of the current block. When the affine flag indicates performing motion compensation based on an affine model, the video decoding device 1700 according to an embodiment may call the affine merge candidate list determiner 1710, and the affine merge candidate list determiner 1710 according to an embodiment may generate an affine merge candidate list including control point-based affine merge candidates corresponding to control point motion vectors, where the control point motion vectors are determined by using motion vectors of neighboring blocks included in a block group of representative neighboring blocks according to the corners of the current block.

[0236] Specifically, when performing inter prediction of a current block in the affine merge mode, before configuring control point-based affine merge candidates, the affine merge candidate list determiner 1710 according to an embodiment may generate an affine merge candidate list including model-based affine merge candidates. The model-based affine merge candidates are merge candidates corresponding to control point motion vectors determined according to affine motion vectors of available neighboring blocks in neighboring blocks at a specific position adjacent to the current block. That is, neighboring blocks decoded before the current block may be determined as model-based affine merge candidates. When the motion information of a neighboring block is available, the model-based affine merge candidate corresponding to the neighboring block is also set to be available. In contrast, when the motion information of a neighboring block is not available, the model-based affine merge candidate corresponding to the neighboring block is also set to be unavailable. However, when the number of available model-based affine merge candidates is less than a specific number, the affine merge candidate list determiner 1710 may determine control point-based affine merge candidates corresponding to a block group of representative neighboring blocks according to the corners of the current block, and may add the control point-based affine merge candidates to the affine merge candidate list. The specific number is the maximum number of affine merge candidates allowed to be included in the affine merge candidate list, and may be set to, for example, 5.

[0237] Accordingly, the model-based affine merge candidates may be corresponding merge candidates derived from one neighboring block, and the control point-based affine merge candidates may be merge candidates derived from a combination of affine models of two or more neighboring blocks. The available model-based affine merge candidates may be included in the affine merge candidate list of the current block, and when the number of valid affine merge candidates of the current block is less than the specific number, as many control point-based affine merge candidates as the insufficient number of affine merge candidates may be added to the affine merge candidate list.

[0238] The affine mode predictor 1720 according to an embodiment may determine an affine motion vector of a current block by using control point motion vectors corresponding to merge candidates selected from an affine merge candidate list. By using an affine merge index obtained from a bitstream, the affine mode predictor 1720 may determine control point motion vectors of merge candidates indicated by the affine merge index from the affine merge candidate list. The affine mode predictor 1720 may determine affine motion model parameters of the current block by using the control point motion vectors, and may determine an affine motion vector of the current block by using the affine motion model parameters of the current block. The affine mode predictor 1720 may obtain a predicted sample of the current block by using reference samples indicated by the affine motion vector of the current block.

[0239] When a model-based affine merge candidate is selected from the affine merge candidate list, a control point motion vector corresponding to the model-based affine merge candidate may be determined by using an affine model of one neighboring block, and the affine mode predictor 1720 according to an embodiment may determine an affine motion vector of the current block by using the control point motion vector corresponding to the model-based affine merge candidate.

[0240] When a control point-based affine merge candidate is selected from the affine merge candidate list, a control point motion vector corresponding to the control point-based affine merge candidate may be determined by using control points of the current block or neighboring blocks included in a corresponding block group, and the affine mode predictor 1720 according to an embodiment may determine an affine motion vector of the current block by using the control point motion vector corresponding to the control point-based affine merge candidate.

[0241] Specifically, when a model-based affine merge candidate is selected from the affine merge candidate list, a control point motion vector corresponding to control points of the current block may be determined by using a vertical change amount, a horizontal change amount, and a motion vector of one neighboring block corresponding to the model-based affine merge candidate. More specifically, a reference index, a prediction direction (L0-pred, L1-pred, or Bi-pred), and a control point motion vector corresponding to the model-based affine merge candidate may be determined by using a reference index, a prediction direction, and a motion vector of the neighboring block, respectively. A reference index, a prediction direction, and an affine motion vector of the current block may be determined by using the reference index, the prediction direction, and the control point motion vector corresponding to the affine merge candidate, respectively.

[0242] Specifically, when selecting a control point-based affine merge candidate from the affine merge candidate list, the control point motion vector can be determined by using the motion vectors of neighboring blocks adjacent to the control points of the current block, and the affine motion vector of the current block can be determined by using the control point motion vector according to the selected affine merge candidate. More specifically, the reference index, prediction direction, and control point motion vector corresponding to the affine merge candidate can be determined by using the reference index, prediction direction, and motion vector of the representative neighboring blocks belonging to the block group, respectively. The reference index, prediction direction, and affine motion vector of the current block can be determined by using the reference index, prediction direction, and control point motion vector corresponding to the affine merge candidate, respectively.

[0243] The video decoding device 1700 according to an embodiment can change the coding order between horizontally adjacent coding units according to the slice unit coding order (SUCO) method. For example, the video decoding device 1700 can obtain a coding order flag indicating the direction of the coding order from the bitstream. When the coding order flag indicates the direction from left to right, among the horizontally adjacent sub-coding units generated by dividing the current coding unit, the left sub-coding unit can be decoded first, and then the right sub-coding unit can be decoded. When the coding order flag indicates the direction from right to left, the right sub-coding unit can be decoded first, and then the left sub-coding unit can be decoded.

[0244] For example, when the right block of the current block is initially decoded and available, the representative neighboring block adjacent to the lower right corner of the current block can be determined as the neighboring block that obtains available motion information from the neighboring blocks adjacent to the lower right corner and the right boundary of the current block and the neighboring block diagonally adjacent to the lower right corner of the current block. It can be checked whether the available motion information of the block is obtained in the order of neighboring block C1 and neighboring block C0, and the initially available block can be determined as the BR representative neighboring block.

[0245] However, when the right block of the current block is not available, the representative neighboring block adjacent to the lower right corner of the current block can be a co-located block at the point diagonally adjacent to the lower right corner of the current block. The co-located block is a block included in the co-located picture and can be included in the affine merge candidate list as a temporal merge candidate.

[0246] In another example, when the right block of the current block has been previously decoded and is available, the BR representative neighboring block adjacent to the lower right corner of the current block can be determined as the block where motion information is initially available from neighboring block C1 adjacent to the lower right corner of the current block and the right boundary of the current block, neighboring block C0 diagonally adjacent to the lower right corner of the current block, and co-located block Col located at a point diagonally adjacent to the lower right corner of the current block. It can be checked whether the available motion information of the blocks is obtained in the order of neighboring block C1, neighboring block C0, and co-located block Col, and the initially available block can be determined as the BR representative neighboring block.

[0247] The co-located block is a block included in the co-located picture and can be included in the affine merge candidate list as a temporal merge candidate. The co-located block adjacent to the lower right corner of the current block can be the block located at the coordinates corresponding to the lower right corner of the current block in the co-located picture.

[0248] The video decoding device 1700 according to an embodiment needs to check the availability of the motion information of neighboring blocks in order to include valid merge candidates in the affine merge candidate list. This is because valid merge candidates can be determined from neighboring blocks that perform motion prediction and have available motion information. Generally, neighboring blocks that have previously performed motion prediction before the current block can be used for the prediction of the current block. Specifically, when motion prediction has been performed on the left neighboring block of the current block before the current block, the motion information of the left neighboring block is available. When motion prediction has been performed on the right neighboring block of the current block before the current block, the motion information of the right neighboring block is available. On the contrary, when motion prediction has not been performed on the left neighboring block of the current block, the motion information of the left neighboring block is not available. When motion prediction has not been performed on the right neighboring block of the current block, the motion information of the right neighboring block is not available.

[0249] Since the right neighboring block of the neighboring block can be decoded first according to the coding order instead of the left neighboring block, the video decoding device 1700 according to an embodiment needs to determine the affine merge candidate list by considering the coding order.

[0250] Hereinafter, with reference to Figure 18 A video decoding method that performs prediction based on an affine model by generating an affine merge candidate list by considering the coding order will be described in detail.

[0251] Figure 18 is a flowchart of a video decoding method according to an embodiment.

[0252] In operation 1810, when performing inter prediction of the current block in affine merge mode, the affine merge candidate list determiner 1710 according to an embodiment may generate an affine merge candidate list including affine merge candidates based on control points corresponding to control point motion vectors, where the control point motion vectors are determined by using motion vectors of neighboring blocks included in a block group of representative neighboring blocks according to the corners of the current block.

[0253] Specifically, when the right block of the current block is available, the representative neighboring block adjacent to the lower right corner of the current block may be determined as the block that obtains available motion information from neighboring blocks adjacent to the lower right corner and the right boundary of the current block and neighboring blocks diagonally adjacent to the lower right corner of the current block. However, when the right block of the current block is not available, the representative neighboring block adjacent to the lower right corner of the current block may be a co-located block at a point diagonally adjacent to the lower right corner of the current block.

[0254] Specifically, the affine merge candidate list determiner 1710 according to an embodiment may determine the availability of motion information of a first representative neighboring block adjacent to the upper left corner of the current block, the availability of motion information of a second representative neighboring block adjacent to the upper right corner of the current block, the availability of motion information of a third representative neighboring block adjacent to the lower left corner of the current block, and the availability of motion information of a fourth representative neighboring block adjacent to the lower right corner of the current block.

[0255] The affine merge candidate list determiner 1710 according to an embodiment may determine affine merge candidates based on control points corresponding to a block group including a plurality of representative neighboring blocks based on the availability of at least one of the motion information of the first representative neighboring block, the motion information of the second representative neighboring block, the motion information of the third representative neighboring block, and the motion information of the fourth representative neighboring block.

[0256] For example, the affine merge candidate list determiner 1710 may determine six affine merge candidates based on control points. The groups of representative neighboring blocks corresponding to the affine merge candidates based on control points may be different from each other. The affine merge candidate list determiner 1710 may include only available candidates among the affine merge candidates based on control points in the affine merge candidate list.

[0257] The affine merge candidate list determiner 1710 according to an embodiment may determine a first control point-based affine merge candidate corresponding to a block group including a first representative neighboring block, a second representative neighboring block, and a third representative neighboring block. Specifically, when the availability of the motion information of the first representative neighboring block adjacent to the upper left corner of the current block, the availability of the motion information of the second representative neighboring block adjacent to the upper right corner of the current block, and the availability of the motion information of the third representative neighboring block adjacent to the lower left corner of the current block respectively indicate availability, the affine merge candidate list determiner 1710 may determine that the first control point-based affine merge candidate corresponding to the block group including the first representative neighboring block, the second representative neighboring block, and the third representative neighboring block is available. When any one of the motion information of the first representative neighboring block, the motion information of the second representative neighboring block, and the motion information of the third representative neighboring block is unavailable, the first control point-based affine merge candidate may be set as unavailable.

[0258] The affine merge candidate list determiner 1710 according to an embodiment may determine a second control point-based affine merge candidate corresponding to a block group including a first representative neighboring block, a second representative neighboring block, and a fourth representative neighboring block. Specifically, when the availability of the motion information of the first representative neighboring block adjacent to the upper left corner of the current block, the availability of the motion information of the second representative neighboring block adjacent to the upper right corner of the current block, and the availability of the motion information of the fourth representative neighboring block adjacent to the lower right corner of the current block respectively indicate availability, the affine merge candidate list determiner 1710 may determine that the second control point-based affine merge candidate corresponding to the block group including the first representative neighboring block, the second representative neighboring block, and the fourth representative neighboring block is available. When any one of the motion information of the first representative neighboring block, the motion information of the second representative neighboring block, and the motion information of the fourth representative neighboring block is unavailable, the second control point-based affine merge candidate may be set as unavailable.

[0259] The affine merge candidate list determiner 1710 according to an embodiment may determine a third control point-based affine merge candidate corresponding to the first representative neighboring block, the third representative neighboring block, and the fourth representative neighboring block. Specifically, when the availability of the motion information of the first representative neighboring block adjacent to the upper left corner of the current block, the availability of the motion information of the third representative neighboring block adjacent to the lower left corner of the current block, and the availability of the motion information of the fourth representative neighboring block adjacent to the lower right corner of the current block respectively indicate availability, the affine merge candidate list determiner 1710 may determine that the third control point-based affine merge candidate corresponding to the block group including the first representative neighboring block, the third representative neighboring block, and the fourth representative neighboring block is available. When any one of the motion information of the first representative neighboring block, the motion information of the third representative neighboring block, and the motion information of the fourth representative neighboring block is unavailable, the third control point-based affine merge candidate may be set as unavailable.

[0260] The affine merge candidate list determiner 1710 according to an embodiment may determine a fourth control point-based affine merge candidate corresponding to the block group including the second representative neighboring block, the third representative neighboring block, and the fourth representative neighboring block. Specifically, when the availability of the motion information of the second representative neighboring block adjacent to the upper right corner of the current block, the availability of the motion information of the third representative neighboring block adjacent to the lower left corner of the current block, and the availability of the motion information of the fourth representative neighboring block adjacent to the lower right corner of the current block respectively indicate availability, the affine merge candidate list determiner 1710 may determine that the fourth control point-based affine merge candidate corresponding to the block group including the second representative neighboring block, the third representative neighboring block, and the fourth representative neighboring block is available. When any one of the motion information of the second representative neighboring block, the motion information of the third representative neighboring block, and the motion information of the fourth representative neighboring block is unavailable, the fourth control point-based affine merge candidate may be set as unavailable.

[0261] The affine merge candidate list determiner 1710 according to an embodiment may determine a fifth control point-based affine merge candidate corresponding to the block group including the first representative neighboring block and the second representative neighboring block. Specifically, when the availability of the motion information of the first representative neighboring block adjacent to the upper left corner of the current block and the availability of the motion information of the second representative neighboring block adjacent to the upper right corner of the current block respectively indicate availability, the affine merge candidate list determiner 1710 may determine that the fifth control point-based affine merge candidate corresponding to the block group including the first representative neighboring block and the second representative neighboring block is available. When any one of the motion information of the first representative neighboring block and the motion information of the second representative neighboring block is unavailable, the fifth control point-based affine merge candidate may be set as unavailable.

[0262] The affine merge candidate list determiner 1710 according to an embodiment may determine a sixth control point-based affine merge candidate corresponding to a block group including a first representative neighboring block and a third representative neighboring block. When the availability of the motion information of the first representative neighboring block adjacent to the upper left corner of the current block and the availability of the motion information of the third representative neighboring block adjacent to the lower left corner of the current block respectively indicate availability, the affine merge candidate list determiner 1710 may determine that the sixth control point-based affine merge candidate corresponding to the block group including the first representative neighboring block and the third representative neighboring block is available. When any one of the motion information of the first representative neighboring block and the motion information of the third representative neighboring block is unavailable, the sixth control point-based affine merge candidate may be set to unavailable.

[0263] According to an embodiment, the first representative neighboring block adjacent to the upper left corner of the current block may be determined as a block having available motion information from among neighboring blocks diagonally adjacent to the upper left corner of the current block, neighboring blocks adjacent to the upper left corner of the current block and the upper boundary of the current block, and neighboring blocks adjacent to the upper left corner of the current block and the left boundary of the current block.

[0264] According to an embodiment, the second representative neighboring block adjacent to the upper right corner of the current block may be determined as a block having obtained available motion information from among neighboring blocks diagonally adjacent to the upper right corner of the current block, neighboring blocks adjacent to the upper right corner of the current block and the upper boundary of the current block, and neighboring blocks adjacent to the upper right corner of the current block and the right boundary of the current block.

[0265] According to an embodiment, when the left block of the current block is available, the third representative neighboring block adjacent to the lower left corner of the current block may be determined as a block having obtained available motion information from among neighboring blocks diagonally adjacent to the lower left corner of the current block and neighboring blocks adjacent to the lower left corner of the current block and the left boundary of the current block. However, when the left block of the current block is unavailable, the third representative neighboring block may be a co-located block at a point diagonally adjacent to the lower left corner of the current block. The co-located block adjacent to the lower left corner of the current block may be a block at coordinates corresponding to the lower left corner of the current block in a co-located picture.

[0266] According to an embodiment, when the right block of the current block is available, the fourth representative neighboring block adjacent to the lower right corner of the current block may be determined as a block having obtained available motion information from among neighboring blocks adjacent to the lower right corner of the current block and the right boundary of the current block and neighboring blocks diagonally adjacent to the lower right corner of the current block. However, when the right block of the current block is unavailable, the fourth representative neighboring block may be a co-located block at a point diagonally adjacent to the lower right corner of the current block.

[0267] At operation 1820, the affine mode predictor 1720 according to an embodiment may determine an affine motion vector of a current block by using control point motion vectors corresponding to merge candidates selected from an affine merge candidate list, and may obtain predicted samples of the current block by using the affine motion vector of the current block.

[0268] When performing interpolation filtering to determine reference samples by using motion vectors generated according to affine prediction, affine motion model parameters are required. The affine motion model parameters may include a horizontal change amount of the motion vector, a vertical change amount of the motion vector, and a basic motion vector. The affine motion model parameters may be determined from the control point motion vectors. The affine mode predictor 1720 according to an embodiment may determine an affine motion vector of the current block by using the horizontal change amount of the motion vector, the vertical change amount of the motion vector, and the basic motion vector, and may obtain predicted samples of the current block by using samples of a reference block indicated by the affine motion vector of the current block.

[0269] When selecting a control point-based affine merge candidate from the affine merge candidate list, the affine mode predictor 1720 according to an embodiment may determine a control point motion vector by using motion vectors of representative neighboring blocks adjacent to a corner of the current block belonging to a neighboring block group corresponding to the control point-based affine merge candidate.

[0270] First, control point motion information (motion vector, prediction direction, and reference picture index) may be determined according to the upper left control point, upper right control point, lower left control point, and lower right control point of the current block by using motion information (motion vector, prediction direction, and reference picture index) of representative neighboring blocks.

[0271] The control point motion information of the upper left control point may be determined by using motion information of a representative neighboring block (first representative neighboring block) in a neighboring block of the upper left control point. The control point motion information of the upper right control point may be determined by using motion information of a representative neighboring block (second representative neighboring block) in a neighboring block of the upper right control point. The control point motion information of the lower left control point may be determined by using motion information of a representative neighboring block (third representative neighboring block) in a neighboring block of the lower left control point. The control point motion information of the lower right control point may be determined by using motion information of a representative neighboring block (fourth representative neighboring block) in a neighboring block of the lower right control point.

[0272] Next, by using the control point motion information of control points belonging to the affine merge candidate (i.e., representative neighboring blocks adjacent to a corner of the current block belonging to a neighboring block group corresponding to the control point-based affine merge candidate), a plurality of control point motion information corresponding to the affine merge candidate may be determined.

[0273] For example, when selecting a first control point-based affine merge candidate from affine control point merge candidates, the control point motion vectors of a first representative neighboring block, a second representative neighboring block, and a third representative neighboring block belonging to a second control point-based affine merge candidate can be respectively determined as a first control point motion vector, a second control point motion vector, and a third control point motion vector corresponding to the first affine merge candidate.

[0274] For example, when selecting a second control point-based affine merge candidate from affine control point merge candidates, the control point motion vectors of a first representative neighboring block and a second representative neighboring block belonging to the second control point-based affine merge candidate can be respectively determined as a first control point motion vector and a second control motion vector corresponding to the second control point-based affine merge candidate, and a third control point motion vector corresponding to the second control point-based affine merge candidate can be determined by using the control point motion vectors of the first representative neighboring block, the second representative neighboring block, and a fourth representative neighboring block.

[0275] For example, when selecting a third control point-based affine merge candidate from affine control point merge candidates, the control point motion vectors of a first representative neighboring block and a third representative neighboring block belonging to the third control point-based affine merge candidate can be respectively determined as a first control point motion vector and a third control point motion vector corresponding to the third control point-based affine merge candidate, and a second control point motion vector corresponding to the third control point-based affine merge candidate can be determined by using the control point motion vectors of the first representative neighboring block, the third representative neighboring block, and a fourth representative neighboring block.

[0276] For example, when selecting a fourth control point-based affine merge candidate from affine control point merge candidates, the control point motion vectors of a second representative neighboring block and a third representative neighboring block belonging to the fourth control point-based affine merge candidate can be respectively determined as a second control point motion vector and a third control point motion vector corresponding to the fourth control point-based affine merge candidate, and a first control point motion vector corresponding to the fourth control point-based affine merge candidate can be determined by using the control point motion vectors of the second representative neighboring block, the third representative neighboring block, and a fourth representative neighboring block.

[0277] For example, when selecting a fifth control point-based affine merge candidate from affine control point merge candidates, the control point motion vectors of the first representative neighboring block and the second representative neighboring block belonging to the fifth control point-based affine merge candidate may be respectively determined as the first control point motion vector and the second control point motion vector corresponding to the fifth control point-based affine merge candidate.

[0278] For example, when selecting a sixth control point-based affine merge candidate from affine control point merge candidates, the control point motion vector of the first representative neighboring block belonging to the sixth control point-based affine merge candidate may be determined as the first control point motion vector corresponding to the sixth control point-based affine merge candidate, and the second control point motion vector corresponding to the sixth control point-based affine merge candidate may be determined by using the control point motion vector of the first representative neighboring block and the control point motion vector of the second representative neighboring block.

[0279] Reference will be made to Figure 22 and Figure 23 to describe a method of predicting an affine motion vector of a current block by using a control point motion vector according to an affine mode.

[0280] The video decoding device 1700 according to an embodiment may include neighboring blocks adjacent to the right or lower right of a current block in a control point-based affine merge candidate by considering the encoding order of coding units. Accordingly, when a right block of the current block is available, inter prediction based on an affine model of the current block is possible by using control point motion vectors derived from neighboring blocks adjacent to the right or lower right of the current block. In addition, since no new control point-based affine merge candidates are added for neighboring blocks adjacent to the right or lower right, there is no need to change the existing process of generating a merge list by using control point-based affine merge candidates. Since neighboring blocks adjacent to the right or lower right are conditionally added to existing control point-based affine merge candidates, an affine merge candidate list can be effectively generated when considering the possibility of changing the encoding order.

[0281] Figure 19 is a block diagram of a video encoding device according to an embodiment.

[0282] Reference is made to Figure 19 According to an embodiment, the video encoding device 1900 may include an affine mode predictor 1910 and an encoder 1920.

[0283] According to an embodiment, the information encoder 1910 may generate limited prediction information indicating whether a limited intra prediction mode is activated, and may perform entropy coding on the limited prediction information to output it as a bitstream.

[0284] The video encoding device 1900 according to an embodiment may divide a picture into one or more luminance encoding units and may encode the encoding units.

[0285] The video encoding device 1900 according to an embodiment may include an intermediate processor (not shown) for controlling the affine mode predictor 1910 and the encoder 1920. Optionally, the affine mode predictor 1910 and the encoder 1920 may be operated by their own processors (not shown), and the processors (not shown) may operate systematically with each other to operate the video encoding device 1900. Optionally, the affine mode predictor 1910 and the encoder 1920 may be controlled under the control of an external processor (not shown) of the video encoding device 1900.

[0286] The video encoding device 1900 may include one or more data memories (not shown) for storing the input / output data of the affine mode predictor 1910 and the encoder 1920. The video encoding device 1900 may include a memory controller (not shown) for controlling the data input and output of the data memory (not shown).

[0287] The video encoding device 1900 may perform an image encoding operation including prediction by operating in connection with an internal video encoding processor or an external video encoding processor to encode an image. The internal video encoding processor of the video encoding device 1900 according to an embodiment may perform a basic image encoding operation not only in the manner of a separate processor but also in the manner of an image encoding processing module included in an intermediate processing device or a graphics processing device.

[0288] The video encoding device 1900 may correspond to the encoder 1600 of the image encoding and decoding system described with reference to Figure 16 For example, the encoder 1920 may correspond to the entropy encoder 1630 of the encoder 1600. The affine mode predictor 1910 may correspond to the inter-frame predictor 1605 of the encoder 1600.

[0289] The information encoder 2110 according to an embodiment may divide a picture into a plurality of largest coding units and may divide and encode each largest coding unit into blocks of various sizes and various shapes.

[0290] For example, when the prediction mode of the current block is an intra-frame mode, the video encoding device 1900 may determine a reference sample point from the sample points of the spatial neighboring blocks in the intra-frame prediction direction of the current block and may determine a prediction sample point corresponding to the current block by using the reference sample point.

[0291] For example, when predicting a current block in skip mode, the video encoding device 1900 may determine a motion vector for predicting the current block. The video encoding device 1900 may determine a reference block of the current block in a reference picture, and may determine, from the current block, a motion vector indicating the reference block. In skip mode, there is no need to encode the residual block.

[0292] For example, when the prediction mode of the current block is an inter prediction mode, the video encoding device 1900 may determine a motion vector for predicting the current block. The video encoding device 1900 may determine a reference block of the current block in a reference picture, and may determine, from the current block, a motion vector indicating the reference block. The video encoding device 1900 may determine residual samples of the current block from reference samples included in the reference block, and may perform transformation and quantization on the residual samples based on a transform unit to generate quantized transform coefficients.

[0293] The current block, which is a block generated by partitioning an image according to a tree structure, may correspond to, for example, a largest coding unit, a coding unit, or a transform unit. The video encoding device 1900 may encode the blocks included in a picture in the encoding order.

[0294] In particular, when performing inter prediction of a current block based on an affine model, the affine mode predictor 1910 may be called. The affine mode predictor 1910 may generate an affine merge candidate list including affine merge candidates based on control points corresponding to control point motion vectors, where the control point motion vectors are determined by using motion vectors of neighboring blocks included in a block group of representative neighboring blocks according to the corners of the current block.

[0295] Specifically, when performing inter prediction of a current block in affine merge mode, before configuring affine merge candidates based on control points, the affine mode predictor 1910 according to an embodiment may generate an affine merge candidate list including affine merge candidates based on models. When motion prediction is performed on a neighboring block before the current block and motion information of the neighboring block is available, the affine merge candidates based on models corresponding to the neighboring block may be included in the affine merge candidate list. When motion information of the neighboring block is available, the affine merge candidates based on models corresponding to the neighboring block are also set to be available. In contrast, when motion information of the neighboring block is not available, the affine merge candidates based on models corresponding to the neighboring block are also set to be unavailable. However, when the number of available affine merge candidates based on models is less than a specific number, the affine mode predictor 1910 may determine affine merge candidates based on control points corresponding to a group of representative neighboring blocks according to the corners of the current block, and may add the affine merge candidates based on control points to the affine merge candidate list. The specific number may be the maximum number of affine merge candidates allowed to be included in the affine merge candidate list, and may be set to, for example, 5.

[0296] Accordingly, available model-based affine merge candidates may be included in the affine merge candidate list of the current block, and when the number of valid affine merge candidates of the current block is less than a specific number, as many control point-based affine merge candidates as the insufficient number of affine merge candidates may be added to the affine merge candidate list.

[0297] The affine mode predictor 1910 according to an embodiment may select a merge candidate having the minimum error from among the merge candidates included in the affine merge candidate list to represent the affine motion vector of the current block. The affine mode predictor 1910 may determine the affine motion vector of the current block by using the control point motion vectors corresponding to the merge candidates selected from the affine merge candidate list. The affine mode predictor 1910 may determine the affine motion model parameters of the current block by using the control point motion vectors, and may determine the affine motion vector of the current block by using the affine motion model parameters of the current block. The predicted sample point of the current block may be determined by using the reference sample point indicated by the affine motion vector of the current block.

[0298] When a model-based affine merge candidate is selected from the affine merge candidate list, the control point motion vectors corresponding to the model-based affine merge candidate may be determined by using the affine model of one neighboring block. The affine motion vector of the current block may be determined by using the control point motion vectors corresponding to the model-based affine merge candidate.

[0299] When a control point-based affine merge candidate is selected from the affine merge candidate list, the control point motion vectors corresponding to the control point-based affine merge candidate may be determined by using the control points of the current block or neighboring blocks included in the corresponding block group. The affine motion vector of the current block may be determined by using the control point motion vectors corresponding to the control point-based affine merge candidate. Specifically, when a model-based affine merge candidate is selected from the affine merge candidate list, the control point motion vectors corresponding to the control points of the current block may be determined by using the vertical change amount, horizontal change amount, and motion vector of one neighboring block corresponding to the model-based affine merge candidate. More specifically, the reference index, prediction direction, and control point motion vectors corresponding to the model-based affine merge candidate may be determined by using the reference index, prediction direction, and motion vector of the neighboring block, respectively. The reference index, prediction direction, and affine motion vector of the current block may be determined by using the reference index, prediction direction, and control point motion vectors corresponding to the affine merge candidate, respectively.

[0300] Specifically, when selecting a control point-based affine merge candidate from the affine merge candidate list, the motion vector of a neighboring block adjacent to the control point of the current block may be determined as the control point motion vector, and the affine motion vector of the current block may be determined by using the control point motion vector according to the affine model of the current block. More specifically, the reference index, prediction direction, and control point motion vector corresponding to the affine merge candidate may be determined by using the reference index, prediction direction, and motion vector of the representative neighboring blocks belonging to the block group, respectively. The reference index, prediction direction, and affine motion vector of the current block may be determined by using the reference index, prediction direction, and control point motion vector corresponding to the affine merge candidate, respectively. The video coding device 1900 according to an embodiment may change the coding order between horizontally adjacent coding units according to the SUCO method. The encoder 1920 may encode a coding order flag indicating the direction of the coding order. When encoding the left sub-coding unit first and then the right sub-coding unit among the horizontally adjacent sub-coding units generated by dividing the current coding unit, the coding order flag may be encoded to indicate the left-to-right direction. When encoding the right sub-coding unit first and then the left sub-coding unit, the coding order flag may be encoded to indicate the right-to-left direction.

[0301] For example, when the right block of the current block is initially encoded and available, the representative neighboring block adjacent to the bottom-right corner of the current block may be the block that obtains available motion information from the neighboring blocks adjacent to the bottom-right corner and the right boundary of the current block and the neighboring block diagonally adjacent to the bottom-right corner of the current block. It may be checked whether the available motion information is obtained in the order of neighboring block C1 and neighboring block C0, and the block that first obtains the available motion information may be determined as the BR representative neighboring block. However, when the right block of the current block is not available, the representative neighboring block adjacent to the bottom-right corner of the current block is the co-located block at the point diagonally adjacent to the bottom-right corner of the current block. The co-located block is a block included in the co-located picture and may be included in the affine merge candidate list as a temporal merge candidate.

[0302] In another example, when the right block of the current block is initially decoded and available, the BR representative neighboring block adjacent to the bottom-right corner of the current block may be the block that obtains available motion information from the neighboring block C1 adjacent to the bottom-right corner and the right boundary of the current block, the neighboring block C0 diagonally adjacent to the bottom-right corner of the current block, and the co-located block Col at the point diagonally adjacent to the bottom-right corner of the current block. It may be checked whether the available motion information is obtained in the order of neighboring block C1, neighboring block C0, and co-located block Col, and the block that first obtains the available motion information may be determined as the BR representative neighboring block.

[0303] A collocated block is a block included in a collocated picture and can be included in an affine merge candidate list as a temporal merge candidate. A collocated block adjacent to the lower right corner of the current block can be a block located at the coordinates corresponding to the lower right corner of the current block in the collocated picture.

[0304] The video encoding device 1900 according to an embodiment needs to check the availability of motion information of neighboring blocks in order to include valid merge candidates in the affine merge candidate list. This is because valid merge candidates can be determined from neighboring blocks for which motion prediction has been performed and available motion information has been obtained. Generally, neighboring blocks for which motion prediction has been performed before the current block can be used for prediction of the current block. Specifically, when motion prediction has been performed on the left neighboring block of the current block before the current block, the motion information of the left neighboring block is available. When motion prediction has been performed on the right neighboring block of the current block before the current block, the motion information of the right neighboring block is available. Conversely, when motion prediction has not been performed on the left neighboring block of the current block, the motion information of the left neighboring block is not available. When motion prediction has not been performed on the right neighboring block of the current block, the motion information of the right neighboring block is not available.

[0305] Since the right neighboring block of a neighboring block can be encoded first according to the coding order rather than the left neighboring block, the video encoding device 1900 according to an embodiment needs to determine the affine merge candidate list by considering the coding order.

[0306] The encoder 1920 according to an embodiment can encode a merge mode flag indicating whether the inter-frame prediction mode of the current block is a merge mode, and can encode an affine flag indicating whether motion compensation based on an affine model is performed to generate prediction samples of the current block. The encoder 1920 according to an embodiment can encode an affine merge index indicating an affine merge candidate from the affine merge candidate list.

[0307] Hereinafter, with reference to Figure 20 A video encoding method for performing prediction based on an affine model by generating an affine merge candidate list by considering the coding order will be described in detail.

[0308] Figure 20 is a flowchart of a video encoding method according to an embodiment.

[0309] In operation 2010, when performing inter-frame prediction of the current block in affine merge mode, the affine mode predictor 1910 can generate an affine merge candidate list including control point-based affine merge candidates corresponding to control point motion vectors, where the control point motion vectors are determined by using motion vectors of neighboring blocks included in a block group of representative neighboring blocks according to the corners of the current block.

[0310] Specifically, when the right block of the current block is available, the representative neighboring block adjacent to the lower right corner of the current block can be a block that obtains available motion information from the neighboring blocks adjacent to the lower right corner and the right boundary of the current block and the neighboring block diagonally adjacent to the lower right corner of the current block. However, when the right block of the current block is not available, the representative neighboring block adjacent to the lower right corner of the current block can be a co-located block at a point diagonally adjacent to the lower right corner of the current block.

[0311] Specifically, the affine mode predictor 1910 according to an embodiment can determine the availability of the motion information of the first representative neighboring block adjacent to the upper left corner of the current block, the availability of the motion information of the second representative neighboring block adjacent to the upper right corner of the current block, the availability of the motion information of the third representative neighboring block adjacent to the lower left corner of the current block, and the availability of the motion information of the fourth representative neighboring block adjacent to the lower right corner of the current block.

[0312] The affine mode predictor 1910 according to an embodiment can determine a control point-based affine merge candidate corresponding to a block group including a plurality of representative neighboring blocks based on the availability of at least one of the motion information of the first representative neighboring block, the motion information of the second representative neighboring block, the motion information of the third representative neighboring block, and the motion information of the fourth representative neighboring block.

[0313] For example, the affine mode predictor 1910 can determine six control point-based affine merge candidates. The groups of representative neighboring blocks corresponding to the control point-based affine merge candidates can be different from each other. The affine mode predictor 1910 can include only the available candidates among the control point-based affine merge candidates in the affine merge candidate list.

[0314] The affine mode predictor 1910 according to an embodiment can determine a first control point-based affine merge candidate corresponding to a block group including the first representative neighboring block, the second representative neighboring block, and the third representative neighboring block. Specifically, when the availability of the motion information of the first representative neighboring block adjacent to the upper left corner of the current block, the availability of the motion information of the second representative neighboring block adjacent to the upper right corner of the current block, and the availability of the motion information of the third representative neighboring block adjacent to the lower left corner of the current block respectively indicate availability, the affine mode predictor 1910 can determine that the first control point-based affine merge candidate corresponding to the block group including the first representative neighboring block, the second representative neighboring block, and the third representative neighboring block is available. When any one of the motion information of the first representative neighboring block, the motion information of the second representative neighboring block, and the motion information of the third representative neighboring block is not available, the first control point-based affine merge candidate can be set as not available.

[0315] The affine mode predictor 1910 according to an embodiment may include a second control point-based affine merge candidate corresponding to a block group including a first representative neighboring block, a second representative neighboring block, and a fourth representative neighboring block. Specifically, when the availability of the motion information of the first representative neighboring block adjacent to the upper left corner of the current block, the availability of the motion information of the second representative neighboring block adjacent to the upper right corner of the current block, and the availability of the motion information of the fourth representative neighboring block adjacent to the lower right corner of the current block respectively indicate availability, the affine mode predictor 1910 may determine that the second control point-based affine merge candidate corresponding to the block group including the first representative neighboring block, the second representative neighboring block, and the fourth representative neighboring block is available. When any one of the motion information of the first representative neighboring block, the motion information of the second representative neighboring block, and the motion information of the fourth representative neighboring block is unavailable, the second control point-based affine merge candidate may be set as unavailable.

[0316] The affine mode predictor 1910 according to an embodiment may determine a third control point-based affine merge candidate corresponding to a block group including a first representative neighboring block, a third representative neighboring block, and a fourth representative neighboring block. Specifically, when the availability of the motion information of the first representative neighboring block adjacent to the upper left corner of the current block, the availability of the motion information of the third representative neighboring block adjacent to the lower left corner of the current block, and the availability of the motion information of the fourth representative neighboring block adjacent to the lower right corner of the current block respectively indicate availability, the affine mode predictor 1910 may determine that the third control point-based affine merge candidate corresponding to the block group including the first representative neighboring block, the third representative neighboring block, and the fourth representative neighboring block is available. When any one of the motion information of the first representative neighboring block, the motion information of the third representative neighboring block, and the motion information of the fourth representative neighboring block is unavailable, the third control point-based affine merge candidate may be set as unavailable.

[0317] The affine mode predictor 1910 according to an embodiment may determine a fourth control point-based affine merge candidate including a second representative neighboring block, a third representative neighboring block, and a fourth representative neighboring block. Specifically, when the availability of the motion information of the second representative neighboring block adjacent to the upper right corner of the current block, the availability of the motion information of the third representative neighboring block adjacent to the lower left corner of the current block, and the availability of the motion information of the fourth representative neighboring block adjacent to the lower right corner of the current block respectively indicate availability, the affine mode predictor 1910 may determine that the fourth control point-based affine merge candidate corresponding to the block group including the second representative neighboring block, the third representative neighboring block, and the fourth representative neighboring block is available. When any one of the motion information of the second representative neighboring block, the motion information of the third representative neighboring block, and the motion information of the fourth representative neighboring block is unavailable, the fourth control point-based affine merge candidate may be set as unavailable.

[0318] The affine mode predictor 1910 according to an embodiment may determine a fifth control point-based affine merge candidate corresponding to a block group including a first representative neighboring block and a second representative neighboring block. Specifically, when the availability of the motion information of the first representative neighboring block adjacent to the upper left corner of the current block and the availability of the motion information of the second representative neighboring block adjacent to the upper right corner of the current block indicate availability, the affine mode predictor 1910 may determine that the fifth control point-based affine merge candidate corresponding to the block group including the first representative neighboring block and the second representative neighboring block is available. When any one of the motion information of the first representative neighboring block and the motion information of the second representative neighboring block is unavailable, the fifth control point-based affine merge candidate may be set as unavailable.

[0319] The affine mode predictor 1910 according to an embodiment may determine a sixth control point-based affine merge candidate corresponding to the first representative neighboring block and the third representative neighboring block. When the availability of the motion information of the first representative neighboring block adjacent to the upper left corner of the current block and the availability of the motion information of the third representative neighboring block adjacent to the lower left corner of the current block indicate availability, the affine mode predictor 1910 may determine that the sixth control point-based affine merge candidate corresponding to the block group including the first representative neighboring block and the third representative neighboring block is available. When any one of the motion information of the first representative neighboring block and the motion information of the third representative neighboring block is unavailable, the sixth control point-based affine merge candidate may be set as unavailable.

[0320] According to an embodiment, the first representative neighboring block adjacent to the upper left corner of the current block may be the block that first obtains available motion information from among the neighboring blocks diagonally adjacent to the upper left corner of the current block, the neighboring blocks adjacent to the upper left corner of the current block and the upper boundary of the current block, and the neighboring blocks adjacent to the upper left corner of the current block and the left boundary of the current block.

[0321] According to an embodiment, the second representative neighboring block adjacent to the upper right corner of the current block may be the block that first obtains available motion information from among the neighboring blocks diagonally adjacent to the upper right corner of the current block, the neighboring blocks adjacent to the upper right corner of the current block and the upper boundary of the current block, and the neighboring blocks adjacent to the upper right corner of the current block and the right boundary of the current block.

[0322] According to an embodiment, when the left block of the current block is available, the third representative neighboring block adjacent to the lower left corner of the current block may be the block that first obtains available motion information from the neighboring blocks diagonally adjacent to the lower left corner of the current block and the neighboring blocks adjacent to the lower left corner of the current block and the left boundary of the current block. However, when the left block of the current block is not available, the third representative neighboring block may be a co-located block located at a point diagonally adjacent to the lower left corner of the current block. The co-located block adjacent to the lower left corner of the current block may be a block located at the coordinates corresponding to the lower left corner of the current block in the co-located picture.

[0323] According to an embodiment, when the right block of the current block is available, the fourth representative neighboring block adjacent to the lower right corner of the current block may be the block that first obtains available motion information from the neighboring blocks adjacent to the lower right corner of the current block and the right boundary of the current block and the neighboring blocks diagonally adjacent to the lower right corner of the current block. However, when the right block of the current block is not available, the fourth representative neighboring block may be a co-located block located at a point diagonally adjacent to the lower right corner of the current block.

[0324] In operation 2020, the encoder 1920 may encode a merge index in the affine merge candidate list that indicates a merge candidate for determining a control point motion vector for performing inter prediction on the current block in the affine mode.

[0325] The affine mode predictor 1910 may determine affine motion model parameters by using the control point motion vectors. The affine motion model parameters may include a horizontal change amount of the motion vector, a vertical change amount of the motion vector, and a basic motion vector. The affine mode predictor 1910 according to an embodiment may determine the affine motion vector of the current block by using the horizontal change amount of the motion vector, the vertical change amount of the motion vector, and the basic motion vector. The predicted samples of the current block may be obtained by using the samples of the reference block indicated by the affine motion vector of the current block.

[0326] When selecting a control point-based affine merge candidate from the affine merge candidate list, the affine mode predictor 1910 according to an embodiment may determine the control point motion vector by using the motion vectors of the representative neighboring blocks adjacent to the corners of the current block that belong to the neighboring block group corresponding to the control point-based affine merge candidate. First, the control point motion information (motion vector, prediction direction, and reference picture index) may be determined based on the upper left control point, upper right control point, lower left control point, and lower right control point of the current block by using the motion information (motion vector, prediction direction, and reference picture index) of the representative neighboring blocks.

[0327] The control point motion information of the upper left control point can be determined by using the motion information of the representative neighboring block (the first representative neighboring block) in the neighboring blocks of the upper left control point. The control point motion information of the upper right control point can be determined by using the motion information of the representative neighboring block (the second representative neighboring block) in the neighboring blocks of the upper right control point. The control point motion information of the lower left control point can be determined by using the motion information of the representative neighboring block (the third representative neighboring block) in the neighboring blocks of the lower left control point. The control point motion information of the lower right control point can be determined by using the motion information of the representative neighboring block (the fourth representative neighboring block) in the neighboring blocks of the lower right control point.

[0328] Next, by using the control point motion information of the control points belonging to the affine merge candidate (i.e., the representative neighboring block adjacent to the corner of the current block belonging to the neighboring block group corresponding to the control point-based affine merge candidate), a plurality of control point motion information corresponding to the affine merge candidate can be determined.

[0329] For example, when the first control point-based affine merge candidate is selected from the affine control point merge candidates, the control point motion vectors of the first representative neighboring block, the second representative neighboring block, and the third representative neighboring block belonging to the second control point-based affine merge candidate can be determined as the first control point motion vector, the second control point motion vector, and the third control point motion vector corresponding to the first affine merge candidate, respectively.

[0330] For example, when the second control point-based affine merge candidate is selected from the affine control point merge candidates, the control point motion vectors of the first representative neighboring block and the second representative neighboring block belonging to the second control point-based affine merge candidate can be determined as the first control point motion vector and the second control point motion vector corresponding to the second control point-based affine merge candidate, respectively, and the third control point motion vector corresponding to the second control point-based affine merge candidate can be determined by using the control point motion vectors of the first representative neighboring block, the second representative neighboring block, and the fourth representative neighboring block.

[0331] For example, when selecting a third control-point-based affine merge candidate from affine control-point merge candidates, the control-point motion vectors of the first representative neighboring block and the third representative neighboring block belonging to the third control-point-based affine merge candidate may be respectively determined as the first control-point motion vector and the third control-point motion vector corresponding to the third control-point-based affine merge candidate, and the second control-point motion vector corresponding to the third control-point-based affine merge candidate may be determined by using the control-point motion vectors of the first representative neighboring block, the third representative neighboring block, and the fourth representative neighboring block.

[0332] For example, when selecting a fourth control-point-based affine merge candidate from affine control-point merge candidates, the control-point motion vectors of the second representative neighboring block and the third representative neighboring block belonging to the fourth control-point-based affine merge candidate may be respectively determined as the second control-point motion vector and the third control motion vector, and the first control-point motion vector corresponding to the fourth control-point-based affine merge candidate may be determined by using the control-point motion vectors of the second representative neighboring block, the third representative neighboring block, and the fourth representative neighboring block.

[0333] For example, when selecting a fifth control-point-based affine merge candidate from affine control-point merge candidates, the control-point motion vectors of the first representative neighboring block and the second representative neighboring block belonging to the fifth control-point-based affine merge candidate may be respectively determined as the first control-point motion vector and the second control-point motion vector corresponding to the fifth control-point-based affine merge candidate.

[0334] For example, when selecting a sixth control-point-based affine merge candidate from affine control-point merge candidates, the control-point motion vector of the first representative neighboring block belonging to the sixth control-point-based affine merge candidate may be determined as the first control-point motion vector corresponding to the sixth control-point-based affine merge candidate, and the second control-point motion vector corresponding to the sixth control-point-based affine merge candidate may be determined by using the control-point motion vectors of the first representative neighboring block and the second representative neighboring block.

[0335] Reference will be made to Figure 22 and Figure 23 to describe a method for predicting an affine motion vector of a current block by using control-point motion vectors according to an affine mode.

[0336] The video encoding device 1900 according to an embodiment may include neighboring blocks adjacent to the right or lower right of a current block in affine merge candidates based on control points by considering the encoding order of the encoding units. Accordingly, when a right block of the current block is available, inter prediction of the current block based on an affine model is possible by using control point motion vectors derived from neighboring blocks adjacent to the right or lower right of the current block. In addition, since no new affine merge candidates based on control points for neighboring blocks adjacent to the right or lower right are added, there is no need to change the existing process of generating a merge list by using affine merge candidates based on control points. Since neighboring blocks adjacent to the right or lower right are conditionally added to existing affine merge candidates based on control points, an affine merge candidate list can be effectively generated when considering the possibility of changing the encoding order.

[0337] Figure 21 Shows changing the encoding order of an encoding unit according to a partitioning unit coding order (SUCO) method according to various embodiments.

[0338] When an encoding unit 2100 is partitioned, an encoding order flag indicating a direction of an encoding order of sub-encoding units partitioned from the encoding unit 2100 may be signaled.

[0339] For example, sub-encoding units 2111 and 2112 generated by a binary vertical partitioning 2110 of the encoding unit 2100 may include a left encoding unit 2111 and a right encoding unit 2112. Sub-encoding units 2121, 2122, and 2123 generated by a ternary vertical partitioning 2120 of the encoding unit 2100 may include a left encoding unit 2121, a middle encoding unit 2122, and a right encoding unit 2123. Sub-encoding units 2131 and 2132 generated by a binary horizontal partitioning 2130 of the encoding unit 2100 may include an upper encoding unit 2131 and a lower encoding unit 2132. Sub-encoding units 2141, 2142, and 2143 generated by a ternary horizontal partitioning 2140 of the encoding unit 2100 may include an upper encoding unit 2141, a middle encoding unit 2142, and a lower encoding unit 2143. Sub-encoding units 2151, 2152, 2153, and 2154 generated by a quadtree partitioning 2150 of the encoding unit 2100 may include an upper left encoding unit 2151, an upper right encoding unit 2152, a lower left encoding unit 2153, and a lower right encoding unit 2154.

[0340] When the coding order flag indicates the left-to-right direction, the left coding unit 2111 among the sub-coding units 2111 and 2112 generated by the binary vertical division 2110 can be decoded before the right coding unit 2112. In this case, although the information of the left coding unit 2111 can be used for the prediction of the right coding unit 2112, the information of the right coding unit 2112 cannot be used for the prediction of the left coding unit 2111. On the contrary, when the coding order flag indicates the right-to-left direction, the right coding unit 2112 can be decoded before the left coding unit 2111. In this case, although the information of the left coding unit 2111 cannot be used for the prediction of the right coding unit 2112, the information of the right coding unit 2112 can be used for the prediction of the left coding unit 2111.

[0341] Similarly, when the coding order flag indicates the left-to-right direction, the left coding unit 2121 among the sub-coding units 2121, 2122, and 2123 generated by the ternary vertical division 2120 can be decoded before the middle coding unit 2122, and the middle coding unit 2122 can be decoded before the right coding unit 2123. In this case, although the information of the left coding unit 2121 can be used for the prediction of the middle coding unit 2122, the information of the middle coding unit 2122 cannot be used for the prediction of the left coding unit 2121. Although the information of the right coding unit 2123 cannot be used for the prediction of the middle coding unit 222, the information of the middle coding unit 2122 can be used for the prediction of the right coding unit 2123. On the contrary, when the coding order flag indicates the right-to-left direction, the middle coding unit 2122 can be decoded before the left coding unit 2121, and the right coding unit 2123 can be decoded before the middle coding unit 2122. In this case, although the information of the left coding unit 2121 cannot be used for the prediction of the middle coding unit 2122, the information of the middle coding unit 2122 can be used for the prediction of the left coding unit 2121. Although the information of the right coding unit 2123 can be used for the prediction of the middle coding unit 2122, the information of the middle coding unit 2122 cannot be used for the prediction of the right coding unit 2123.

[0342] The coding order flag does not affect the coding order of the sub-coding units 2131 and 2132, and 2141, 2142, and 2143 generated by the binary horizontal division 2130 and the ternary horizontal division 2140 of the coding unit 2100.

[0343] When performing quadtree partitioning on coding unit 2100 by quadtree partitioning 2150, the coding order indicated by the coding order flag can simultaneously affect the coding order between the upper left coding unit 2151 and the upper right coding unit 2152 and the coding order between the lower left coding unit 2153 and the lower right coding unit 2154. Specifically, when the coding order flag indicates the left-to-right direction, the upper left coding unit 2151 can be decoded before the upper right coding unit 2152, and the lower left coding unit 2153 can be decoded before the lower right coding unit 2154. In this case, although the information of the upper left coding unit 2151 can be used for the prediction of the upper right coding unit 2152, the information of the upper right coding unit 2152 cannot be used for the prediction of the upper left coding unit 2151. Although the information of the lower left coding unit 2153 can be used for the prediction of the lower right coding unit 2154, the information of the lower right coding unit 2154 cannot be used for the prediction of the lower left coding unit 2153. On the contrary, when the coding order flag indicates the right-to-left direction, the upper right coding unit 2152 can be decoded before the upper left coding unit 2151, and the lower right coding unit 2154 can be decoded before the lower left coding unit 2153. In this case, although the information of the upper left coding unit 2151 cannot be used for the prediction of the upper right coding unit 2152, the information of the upper right coding unit 2152 can be used for the prediction of the upper left coding unit 2151. Although the information of the lower left coding unit 2153 cannot be used for the prediction of the lower right coding unit 2154, the information of the lower right coding unit 2154 can be used for the prediction of the lower left coding unit 2153.

[0344] As referred to Figure 21 As described, it is possible to determine whether a neighboring block is available for the prediction of the current block according to the coding order flag. Accordingly, a video decoding device 1700 and a video encoding device 1900 according to an embodiment propose a method by which, when performing inter prediction according to an affine mode, the availability of neighboring blocks according to the coding order is checked, and a representative neighboring block included in an affine merge candidate based on control points is determined.

[0345] Figure 22 Shows a method of deriving a motion vector of a sample applied to a current block in an affine mode.

[0346] In the affine mode, in order to derive a motion vector of a sample of the current block 2200, at least three affine parameters are required. Specifically, the affine mode may include a 6-parameter affine mode and a 4-parameter affine mode. Hereinafter, a method of deriving a motion vector of a sample of the current block 2200 according to each affine mode will be described.

[0347] In the 6-parameter affine mode, the affine mode predictor 1910 and the affine mode predictor 1720 may obtain three motion vectors 2202, 2204, and 2206 from neighboring samples (2201, 2203, and 2205) of the current block 2200. The first motion vector 2202 may be obtained from neighboring samples of the top-left coordinate 2201 of the current block 2200. The second motion vector 2204 may be obtained from neighboring samples of the top-right coordinate 2203 of the current block 2200. The third motion vector 2206 may be obtained from neighboring samples of the bottom-left coordinate 2205 of the current block 2200. Although the third motion vector 2206 is obtained based on the bottom-left coordinate 2205 of the current block 2200 in Figure 22 , according to an embodiment, the third motion vector 2206 may be obtained based on the bottom-right coordinate 2207 of the current block 2200. The affine mode predictor 1910 and the affine mode predictor 1720 may determine the x and y components of the first motion vector 2202, the x and y components of the second motion vector 2204, and the x and y components of the third motion vector 2206 as parameters.

[0348] According to an embodiment, the first motion vector 2202 may be determined as the average of the motion vectors of a plurality of neighboring blocks adjacent to the top-left coordinate 2201 of the current block 2200. Similarly, the second motion vector 2204 may be determined as the average of the motion vectors of a plurality of neighboring blocks adjacent to the top-right coordinate 2203 of the current block 2200. In addition, the third motion vector 2206 may be determined as the average of the motion vectors of a plurality of neighboring blocks adjacent to the bottom-left coordinate 2205 or the bottom-right coordinate 2207 of the current block 2200.

[0349] The motion vector 2210 of the sample 2208 of the current block 2200 may be determined by using Equations 1 to 3 based on the first motion vector 2202, the second motion vector 2204, and the third motion vector 2206.

[0350] In Equations 1 to 3, x represents the horizontal distance difference between the top-left coordinate 2201 of the current block 2200 and the sample 2208 of the current block 2200, and y represents the vertical distance difference between the top-left coordinate 2201 of the current block 2200 and the sample 2208 of the current block 2200. MV0 represents the first motion vector 2202, MV1 represents the second motion vector 2204, and MV2 represents the third motion vector 2206. MV represents the motion vector 2210 of the sample 2208 of the current block. w represents the width of the current block 2200, and h represents the height of the current block 2200. dMV x represents the horizontal change rate of the motion vector 2210, while dMV y represents the vertical change rate of the motion vector 2210.

[0351] [Equation 1]

[0352] dMV x = (MV1 - MV0) / w

[0353] [Equation 2]

[0354] dMV y = (MV2 - MV0) / h

[0355] [Equation 3]

[0356] MV = MV0 + x.dMV x + y.dMV y

[0357] Equation 1 shows a method of obtaining the horizontal change rate dMV of the motion vector 2210 x . According to Equation 1, the value obtained by dividing the value obtained by subtracting the first motion vector 2202 from the second motion vector 2204 by the width of the current block 2200 is determined as the horizontal change rate of the motion vector 2210.

[0358] Equation 2 shows a method of obtaining the vertical change rate dMV of the motion vector 2210 y . According to Equation 2, the value obtained by dividing the value obtained by subtracting the first motion vector 2202 from the third motion vector 2206 by the height of the current block 2200 is determined as the vertical change rate of the motion vector 2210.

[0359] Equation 3 shows a method of obtaining the motion vector 2210. According to Equation 2, the motion vector 2210 is determined as the value obtained by adding the inner product value of (x, y), which is the coordinate of the sample point 2208 of the current block 2200 with respect to the upper left coordinate 2201 of the current block 2200, and (dMVx, dMVy), which indicates the vertical change rate and the horizontal change rate, to the first motion vector 2202 MV0.

[0360] According to Equations 1 to 3, the motion vectors of all sample points or sub - blocks included in the current block 2200 can be determined. According to Equations 1 to 3, the motion vectors of the sample points can be determined differently according to the positions of the sample points. When the vertical components of the coordinates of the first motion vector 2202 and the second motion vector 2204 extracted are the same and the horizontal components of the coordinates of the first motion vector 2202 and the third motion vector 2206 extracted are the same, Equations 1 and 2 can be applied. Accordingly, the general equation for determining the motion vector of the current block 2200 will be described below with reference to Figure 22 describe the general equation for determining the motion vector of the current block 2200.

[0361] In the 6 - parameter affine mode, since the motion vector 2210 is determined by three motion vectors, the reference block of the current block 2200 can be scaled, rotated, and sheared from the current block 2200.

[0362] In the 4-parameter affine mode, the affine mode predictor 1910 and the affine mode predictor 1720 can obtain two motion vectors 2202 and 2204 from neighboring samples of the current block 2200. Similar to the 6-parameter affine mode, the first motion vector 2202 can be obtained from neighboring samples of the upper left coordinates of the current block 2200. Similarly, the second motion vector 2204 can be obtained from neighboring samples of the upper right coordinates of the current block 2200. The affine mode predictor 1910 and the affine mode predictor 1720 can determine the x and y components of the first motion vector 2202 and the x and y components of the second motion vector 2204 as affine parameters.

[0363] In the 4-parameter affine mode, the third motion vector 2206 is not determined from the lower left or lower right coordinates of the current block 2200, but is determined by combining the first motion vector 2202 and the second motion vector 2204.

[0364] Equations 4 and 5 show the method of determining the third motion vector 2206 by combining the first motion vector 2202 and the second motion vector 2204. In Equations 4 and 5, x represents the horizontal component of the motion vector, and y represents the vertical component of the motion vector. MV0 represents the first motion vector 2202, MV1 represents the second motion vector 2204, and MV2 represents the third motion vector 2206. w represents the width of the current block 2200, and h represents the height of the current block 2200.

[0365] [Equation 4]

[0366]

[0367] [Equation 5]

[0368]

[0369] According to Equation 4, the horizontal coordinate value (MV2[x]) of the third motion vector 2206 is determined as the value obtained by adding the horizontal coordinate value (MV0[x]) of the first motion vector 2202 to the product of the value obtained by subtracting the vertical coordinate value of the first motion vector 2202 from the vertical coordinate value of the second motion vector 2204 (MV1[y] - MV0[y]) and the value of the width of the current block divided by the height of the current block (w / h). .

[0370] According to Equation 5, the vertical coordinate value (MV2[y]) of the third motion vector 2206 is determined as the value obtained by adding the vertical coordinate value (MV0[y]) of the first motion vector 2202 to the product of the value obtained by subtracting the horizontal coordinate value of the second motion vector 2204 from the horizontal coordinate value of the first motion vector 2202 (MV0[x] - MV1[x]) and the value obtained by dividing the width of the current block by the height of the current block (w / h). 。

[0371] In the 4-parameter affine mode, the x and y components of the third motion vector 2206 are derived from the first motion vector 2202 and the second motion vector 2204. Accordingly, different from the 6-parameter affine mode, in the 4-parameter affine mode, the reference block of the current block 2200 can be scaled and rotated based on the first motion vector 2202 and the second motion vector 2204 for the current block 2200. That is, in the 4-parameter affine mode, the current block 2200 is not sheared.

[0372] Figure 23 Shows a method for determining the affine motion vector of the current block in the affine mode.

[0373] Equations 6 to 8 show a method for determining the affine motion vector of the current block according to Figure 23 the motion vector extraction positions 2800, 2810, and 2820.

[0374] In Equations 6 and 7, w represents the horizontal distance between the first motion vector extraction position 2800 and the second motion vector extraction position 2810. In addition, h represents the vertical distance between the first motion vector extraction position 2800 and the third motion vector extraction position 2820. In addition, x represents the horizontal distance between the first motion vector extraction position 2800 and the third motion vector extraction position 2820. In addition, y represents the vertical distance between the first motion vector extraction position 2800 and the second motion vector extraction position 2810.

[0375] P0 represents the first motion vector, P1 represents the second motion vector, and P2 represents the third motion vector. In addition, dx and dy represent the horizontal change amount and the vertical change amount, respectively.

[0376] [Equation 6]

[0377]

[0378] [Equation 7]

[0379]

[0380] Determine the horizontal change amount according to Equation 6, and determine the vertical change amount according to Equation 7. According to Equation 8, determine the motion vector of sample 2830 of the current block according to the horizontal change amount and the vertical change amount. In Equation 8, P a represents the motion vector of sample 2830 of the current block. In addition, i represents the horizontal distance between the first motion vector extraction position 2800 and sample 2830 of the current block, and j represents the vertical distance between the first motion vector extraction position 2800 and sample 2830 of the current block.

[0381] [Equation 8]

[0382] P a = P0 + idx + jdy

[0383] When three motion vectors and the extraction position of each motion vector are provided according to Equations 6 to 8, the motion vectors of the samples included in the current block can be determined. Accordingly, even when the extraction positions of the motion vectors are not aligned, the motion vectors of the samples included in the current block can be determined.

[0384] Hereinafter, various methods for generating affine merge candidates based on control points will be described in detail with reference to Figures 24 to 27 Detailed description of various methods for generating affine merge candidates based on control points.

[0385] According to an embodiment, in the 6-parameter affine mode, three control point motion vectors are obtained. According to an embodiment, in order to obtain three motion vectors, first determine whether the left block and the right block of the current block are decoded. In Figure 23 the methods for determining affine parameters when the following cases occur will be described sequentially: 1) only the left block of the current block is decoded, 2) only the right block of the current block is decoded, 3) neither the left block nor the right block of the current block is decoded, and 4) both the left block and the right block of the current block are decoded.

[0386] When only the left block of the current block is decoded, obtain the first motion vector from the neighboring samples at the upper left coordinates of the current block. According to an embodiment, the first motion vector can be determined as the motion vector of the block corresponding to one of the neighboring samples. In addition, the neighboring samples can be scanned in a specific order, and when an inter-predicted neighboring block is found, stop scanning and extract the first motion vector from the inter-predicted neighboring block. In addition, according to an embodiment, the first motion vector can be determined as the average value of the motion vectors obtained from multiple blocks corresponding to the neighboring samples.

[0387] A second motion vector is obtained from neighboring samples at the upper right coordinates of the current block. According to an embodiment, the second motion vector may be determined as the motion vector of a block corresponding to one of the neighboring samples. Additionally, the neighboring samples may be scanned in a specific order, and when an inter-predicted neighboring block is found, the scanning is stopped, and the second motion vector is extracted from the inter-predicted neighboring block. Further, according to an embodiment, the second motion vector may be determined as the average of motion vectors obtained from multiple blocks corresponding to the neighboring samples.

[0388] In addition, a third motion vector is obtained from neighboring samples at the lower left coordinates of the current block. According to an embodiment, the third motion vector may be determined as the motion vector of a block corresponding to one of the neighboring samples. Additionally, the neighboring samples may be scanned in a specific order, and when an inter-predicted neighboring block is found, the scanning is stopped, and the third motion vector is extracted from the inter-predicted neighboring block. Further, according to an embodiment, the third motion vector may be determined as the average of motion vectors obtained from multiple blocks corresponding to the neighboring samples.

[0389] In addition, the horizontal change rate is determined as a value obtained by dividing the difference between the first motion vector and the second motion vector by the width of the current block. The vertical change rate is determined as a value obtained by dividing the difference between the first motion vector and the third motion vector by the height of the current block.

[0390] When only the right block of the current block is decoded, similar to when only the left block of the current block is decoded, a first motion vector is obtained from neighboring samples at the upper left coordinates of the current block. Additionally, a second motion vector is obtained from neighboring samples at the upper right coordinates of the current block.

[0391] However, since the left block of the current block is not decoded, a third motion vector is obtained from neighboring samples at the lower right coordinates of the current block. According to an embodiment, the third motion vector may be determined as the motion vector of a block corresponding to one of the neighboring samples. Additionally, according to an embodiment, the third motion vector may be determined as the average of motion vectors obtained from multiple blocks corresponding to the neighboring samples.

[0392] The horizontal change rate is determined as a value obtained by dividing the difference between the first motion vector and the second motion vector by the width of the current block. The vertical change rate is determined as a value obtained by dividing the difference between the second motion vector and the third motion vector by the height of the current block.

[0393] When neither the left block nor the right block of the current block is decoded, similar to when only the left block of the current block is decoded, a first motion vector is obtained from neighboring samples at the upper left coordinates of the current block. Additionally, a second motion vector is obtained from neighboring samples at the upper right coordinates of the current block.

[0394] However, since the left and right blocks of the current block are not decoded, a third motion vector is determined from the first and second motion vectors. Accordingly, when the 6-parameter affine mode is applied to the current block and neither the left nor the right block of the current block is decoded, the current block is decoded substantially according to the 4-parameter affine mode.

[0395] When the left and right blocks of the current block are decoded, one of the method of determining a motion vector when only the left block of the current block is decoded and the method of determining a motion vector when only the right block of the current block is decoded can be selected. Thus, a first motion vector and a second motion vector can be determined based on the neighboring samples of the top-left coordinates and the top-right coordinates of the current block, respectively. In addition, a third motion vector can be determined based on the bottom-left motion vector or the bottom-right motion vector. The bottom-left motion vector is determined from the neighboring samples of the top-left coordinates of the current block, and the bottom-right motion vector is determined from the neighboring samples of the top-left coordinates of the current block.

[0396] The method of determining a motion vector can be determined from a higher level of the current block. For example, the processor 1700 can determine the default motion vector determination method for a higher level of the current block. When the default motion vector determination method is the method of determining a motion vector when only the left block of the current block is decoded, the third motion vector can be determined according to the bottom-left motion vector of the current block.

[0397] Optionally, for the current block, the processor 1700 can obtain motion vector information indicating one of the method of determining a motion vector when only the left block of the current block is decoded and the method of determining a motion vector when only the right block of the current block is decoded. Then, the method of determining the motion vector of the current block can be selected according to the motion vector information.

[0398] Figure 24 A method of determining a representative neighboring block adjacent to a corner of a current block and a control point motion vector derived from the representative neighboring block according to an embodiment is shown.

[0399] According to an embodiment, a control point motion vector MV0 of the upper right corner of the current block 2400 can be determined based on the control point motion vector of the TL representative neighboring block 2410 adjacent to the upper left corner of the current block 2400. The TL representative neighboring block 2410 can be the block that first obtains available motion information from the neighboring block 2411 diagonally adjacent to the upper left corner of the current block 2400, the neighboring block 2412 adjacent to the upper left corner of the current block and the upper boundary of the current block, and the neighboring block 2413 adjacent to the upper left corner of the current block and the left boundary of the current block. That is, the availability of motion information can be checked in the order of the neighboring blocks 2411, 2412, and 2413 adjacent to the upper left corner of the current block 2400, and the neighboring block in which the available motion information is first detected can be determined as the TL representative neighboring block 2410.

[0400] According to an embodiment, the control point motion vector MV1 of the upper right corner of the current block 2400 may be determined based on the motion vector of the TR representative neighboring block 2420 adjacent to the upper right corner of the current block 2400. The TR representative neighboring block 2420 may be the block that first obtains available motion information from among the neighboring blocks 2421 diagonally adjacent to the upper right corner of the current block, the neighboring block 2422 adjacent to both the upper right corner of the current block and the upper boundary of the current block, and the neighboring block 2423 adjacent to both the upper right corner of the current block and the right boundary of the current block. That is, the availability of the motion information may be checked in the order of the neighboring blocks 2421, 2422, and 2423 adjacent to the upper right corner of the current block 2400, and the neighboring block from which the available motion information is first detected may be determined as the TR representative neighboring block 2420.

[0401] According to an embodiment, the control point motion vector MV2 of the lower left corner of the current block 2400 may be determined based on the motion vector of the BL representative neighboring block 2430 adjacent to the lower left corner of the current block. The BL representative neighboring block 2430 may be the block that first obtains available motion information from among the neighboring block 2431 diagonally adjacent to the lower left corner of the current block and the neighboring block 2432 adjacent to both the lower left corner of the current block and the left boundary of the current block. That is, the availability of the motion information may be checked in the order of the neighboring blocks 2431 and 2432 adjacent to the lower left corner of the current block, and the neighboring block from which the available motion information is first detected may be determined as the BL representative neighboring block 2430.

[0402] According to an embodiment, the control point motion vector MV3 of the lower right corner of the current block 2400 may be determined based on the motion vector of the BR representative neighboring block 2440 adjacent to the lower right corner of the current block 2400. When the right neighboring block of the current block is available, the BR representative neighboring block 2440 may be the block that first obtains available motion information from among the neighboring block 2441 adjacent to both the lower right corner of the current block and the right boundary of the current block and the neighboring block 2442 diagonally adjacent to the lower right corner of the current block. However, when the right neighboring block of the current block is not available, the BR representative neighboring block 2440 may be the co-located block 2443 located at the point diagonally adjacent to the lower right corner of the current block.

[0403] When the availability of the motion information of the TL representative neighboring block 2410, the availability of the motion information of the TR representative neighboring block 2420, and the availability of the motion information of the BL representative neighboring block 2430 respectively indicate availability, the video decoding device 1700 and the video encoding device 1900 according to an embodiment may determine that the first control point-based affine merge candidate Const1 corresponding to the block group including the TL representative neighboring block 2410, the TR representative neighboring block 2420, and the BL representative neighboring block 2430 is available.

[0404] When the availability of the motion information of the TL representative neighboring block 2410, the availability of the motion information of the TR representative neighboring block 2420, and the availability of the motion information of the BR representative neighboring block 2440 respectively indicate availability, the video decoding device 1700 and the video encoding device 1900 according to an embodiment may determine that the second control point-based affine merge candidate Const2 corresponding to the block group including the TL representative neighboring block 2410, the TR representative neighboring block 2420, and the BR representative neighboring block 2440 is available.

[0405] When the availability of the motion information of the TL representative neighboring block 2410, the availability of the motion information of the BL representative neighboring block 2430, and the availability of the motion information of the BR representative neighboring block 2440 respectively indicate availability, the video decoding device 1700 and the video encoding device 1900 according to an embodiment may determine that the third control point-based affine merge candidate Const3 corresponding to the block group including the TL representative neighboring block 2410, the BL representative neighboring block 2430, and the BR representative neighboring block 2440 is available.

[0406] When the availability of the motion information of the TR representative neighboring block 2420 adjacent to the upper right corner of the current block, the availability of the motion information of the BL representative neighboring block 2430 adjacent to the lower left corner of the current block, and the availability of the motion information of the BR representative neighboring block 2440 adjacent to the lower right corner of the current block respectively indicate availability, the video decoding device 1700 and the video encoding device 1900 according to an embodiment may determine that the fourth control point-based affine merge candidate Const4 corresponding to the block group including the TR representative neighboring block 2420, the BL representative neighboring block 2430, and the BR representative neighboring block 2440 is available.

[0407] When the availability of the motion information of the TL representative neighboring block 2410 adjacent to the upper left corner of the current block and the availability of the motion information of the TR representative neighboring block 2420 adjacent to the upper right corner of the current block respectively indicate availability, the video decoding device 1700 and the video encoding device 1900 according to an embodiment may determine that the fifth control point-based affine merge candidate Const5 corresponding to the block group including the TL representative neighboring block 2410 and the TR representative neighboring block 2420 is available.

[0408] When the availability of the motion information of the TL representative neighboring block 2410 adjacent to the upper left corner of the current block and the availability of the motion information of the BL representative neighboring block 2420 adjacent to the lower left corner of the current block respectively indicate availability, the video decoding device 1700 and the video encoding device 1900 according to an embodiment may determine that the sixth control point-based affine merge candidate Const6 corresponding to the block group including the TL representative neighboring block 2410 and the BL representative neighboring block 2420 is available.

[0409] The video decoding device 1700 and the video encoding device 1900 according to an embodiment may check the availability of the control point-based affine merge candidates Const1, Const2, Const3, Const4, Const5, and Const6 in order, and may include the available control point-based affine merge candidates in the affine merge candidate list in order. However, when the number of merge candidates included in the affine merge candidate list is 5, the control point-based affine merge candidates may no longer be added to the affine merge candidate list.

[0410] According to an embodiment, an affine merge index indicating a merge candidate selected from the affine merge candidate list may be signaled between the video decoding device 1700 and the video encoding device 1900. The affine motion vector of the current block 2400 may be determined based on the control point motion vectors of the merge candidates selected from the affine merge candidate list.

[0411] Figure 25 A method for determining a representative neighboring block adjacent to a corner of a current block and a control point motion vector derived from the representative neighboring block according to another embodiment is shown.

[0412] Referring to Figure 25 the embodiment of, the video decoding device 1700 and the video encoding device 1900 may determine a TL representative neighboring block 2410, a TR representative neighboring block 2420, a BL representative neighboring block 2530, and a BR representative neighboring block 2440. That is, there is a difference in the method for determining the BL representative neighboring block 2530 as compared with the embodiment of Figure 24 .

[0413] Specifically, according to Figure 25 the embodiment of, the control point motion vector MV2 at the lower left corner of the current block 2400 may be determined to be the same as the motion vector of the BL representative neighboring block 2530 adjacent to the lower left corner of the current block 2400. When the left block of the current block is available, the BL representative neighboring block 2530 may be the block that first obtains available motion information from the neighboring block 2431 diagonally adjacent to the lower left corner of the current block and the neighboring block 2432 adjacent to the lower left corner of the current block and the left boundary of the current block. However, when the left block of the current block is not available, the BL representative neighboring block 2530 may be the co-located block 2533 at the point diagonally adjacent to the lower left corner of the current block.

[0414] The group of representative neighboring blocks included in the control point-based affine merge candidates Const1, Const2, Const3, Const4, Const5, and Const6 determined according to Figure 25 the embodiment of and referring toFigure 24 The groups of described representative neighboring blocks are the same. However, the BL representative neighboring block 2530 instead of the BL representative neighboring block 2430 may be included in Const1, Const3, Const4, and Const6.

[0415] Figure 26 A method for determining representative neighboring blocks adjacent to a corner of a current block and control point motion vectors derived from the representative neighboring blocks according to another embodiment is shown.

[0416] According to Figure 26 an embodiment of, the video decoding device 1700 and the video encoding device 1900 may determine the TL representative neighboring block 2410, the TR representative neighboring block 2620, the BL representative neighboring block 2430, and the BR representative neighboring block 2640. That is, when compared with Figure 24 an embodiment of, there are differences in the method for determining the TR representative neighboring block 2620 and the BR representative neighboring block 2640.

[0417] Specifically, according to Figure 26 an embodiment of, the control point motion vector MV1 of the upper right corner of the current block 2400 may be determined to be the same as the motion vector of the TR representative neighboring block 2620 adjacent to the upper right corner of the current block 2400. The TR representative neighboring block 2620 may be a block from which available motion information is initially obtained from the neighboring block 2421 diagonally adjacent to the upper right corner of the current block and the neighboring block 2422 adjacent to the upper right corner of the current block and the upper boundary of the current block. That is, the availability of motion information may be checked in the order of the neighboring blocks 2421 and 2422 adjacent to the upper right corner of the current block 2400, and the neighboring block in which available motion information acquisition is first detected may be determined as the TR representative neighboring block 2620. That is, when compared with Figure 24 an embodiment of, the neighboring block 2423 adjacent to the upper right corner and the right boundary of the current block 2400 is excluded from the TR representative neighboring block 2620.

[0418] In addition, according to Figure 26 an embodiment of, the control point motion vector MV3 of the lower right corner of the current block 2400 and the motion vector of the BR representative neighboring block 2640 adjacent to the upper right corner of the current block 2400 may be determined to be the same. The BR representative neighboring block 2640 may be the co-located block 2443 located at a point diagonally adjacent to the lower right corner of the current block. That is, when compared with Figure 24 an embodiment of, the neighboring blocks 2441 and 2442 adjacent to the lower right corner and the right boundary of the current block 2400 may be excluded from the BR representative neighboring block 2640, and only the co-located block 2443 may be determined as the BR representative neighboring block 2640.

[0419] Included in accordance with Figure 26 the group of representative neighboring blocks among the affine merge candidates Const1, Const2, Const3, Const4, Const5, and Const6 based on control points determined in the embodiment of Figure 24 is the same as the group of representative neighboring blocks described with reference to Figure 24 . However, the TR representative neighboring block 2620 and the BR representative neighboring block 2640, rather than the TR representative neighboring block 2420 and the BR representative neighboring block 2440, may be included in Const1, Const2, Const3, Const4, and Const5.

[0420] Figure 27 FIG. shows a method for determining representative neighboring blocks adjacent to the corners of a current block and control point motion vectors derived from the representative neighboring blocks according to another embodiment.

[0421] In accordance with Figure 27 the embodiment of Figure 27 , the video decoding device 1700 and the video encoding device 1900 may determine the TL representative neighboring block 2410, the TR representative neighboring block 2420, the BL representative neighboring block 2430, and the BR representative neighboring block 2740. That is, there is a difference in the method for determining the BR representative neighboring block 2740 when compared with the embodiment of Figure 24 .

[0422] Specifically, in accordance with Figure 27 the embodiment of Figure 27 , it is possible to determine that the control point motion vector MV3 at the lower right corner of the current block 2400 is the same as the motion vector of the BR representative neighboring block 2740 adjacent to the lower right corner of the current block 2400. The BR representative neighboring block 2740 may be the block that first obtains available motion information from the neighboring block 2441 adjacent to the lower right corner and the right side boundary of the current block and the neighboring block 2442 diagonally adjacent to the lower right corner of the current block.

[0423] Included in accordance with Figure 27 the group of representative neighboring blocks among the affine merge candidates Const1, Const2, Const3, Const4, Const5, and Const6 based on control points determined in the embodiment of Figure 24 is the same as the group of representative neighboring blocks described with reference to Figure 24 . However, the BR representative neighboring block 2740, rather than the BR representative neighboring block 2440, may be included in Const2, Const3, and Const4.

[0424] In addition, the embodiments of the present disclosure described above may be written as computer-executable programs, and the computer-executable programs may be stored in a medium.

[0425] The medium can continuously store computer-executable programs or temporarily store computer-executable programs for execution or downloading. In addition, the medium can be any of various recording media or storage media that combine single or multiple pieces of hardware, and the medium is not limited to the medium directly connected to the computer system but can be distributed on the network. Examples of the medium include magnetic media (such as hard disks, floppy disks, and magnetic tapes) configured to store program instructions, optical recording media (such as CD-ROMs and DVDs), magneto-optical media (such as optical floppy disks and ROMs), random access memories (RAMs), and flash memories. The machine-readable storage medium can be provided as a non-transitory storage medium. Here, the term "non-transitory" means that the storage medium is a tangible device and does not include signals (e.g., electromagnetic waves), but does not distinguish whether the data is stored semi-permanently or temporarily in the storage medium. For example, the "non-transitory storage medium" can include buffers that temporarily store data.

[0426] In addition, other examples of the medium can include recording media and storage media managed by an app store that publishes applications or by websites, servers, etc. that provide or publish various other types of software.

[0427] According to an embodiment of the present disclosure, a method according to various embodiments of the present disclosure can be provided in a computer program product. The computer program product is a product that can be traded between a seller and a purchaser. The computer program product can be distributed in a machine-readable recording medium (e.g., CD-ROM), or can be distributed online (e.g., downloaded or uploaded) via an app store (e.g., PlayStoreTM) or directly between two user devices (e.g., smart phones) (e.g., downloaded or uploaded). When distributed online, at least part of the computer program product (e.g., a downloadable application) can be generated temporarily or at least temporarily stored in a machine-readable storage medium (such as the memory of a manufacturer's server, an app store's server, or a forwarding server).

[0428] Although one or more embodiments of the present disclosure have been described with reference to the accompanying drawings, those of ordinary skill in the art will understand that various changes in form and detail can be made therein without departing from the spirit and scope defined by the appended claims.

Claims

1. A video decoding method, comprising: Determining whether a right block of the current block is available according to a flag indicating an encoding order of blocks including the current block and the right block; Wherein, when the right block of the current block is available, obtaining a first control point motion vector as an affine merge candidate by using a motion vector of a first block in the right block, wherein the first block is adjacent to the lower right corner of the current block, and When the right block of the current block is not available, obtaining a first control point motion vector as an affine merge candidate by using a motion vector of a second block included in a first reference picture, wherein the second block is a reference block located at a point diagonally adjacent to the lower right corner of the current block; When the left block of the current block is available, obtaining a second control point motion vector as an affine merge candidate by using a motion vector of a third block in the left block, wherein the third block is adjacent to the lower left corner of the current block; When the left block of the current block is not available, obtaining a second control point motion vector as an affine merge candidate by using a motion vector of a fourth block included in a second reference picture, wherein the fourth block is a reference block located at a point diagonally adjacent to the lower left corner of the current block; and Obtaining predicted sample points of the current block by using motion vectors included in an affine merge candidate list including the affine merge candidates.

2. A video encoding method, comprising: Determining whether a right block of the current block is available according to a flag indicating an encoding order of blocks including the current block and the right block; When the right block of the current block is available, obtaining a first control point motion vector as an affine merge candidate by using a motion vector of a first block in the right block, wherein the first block is adjacent to the lower right corner of the current block; When the right block of the current block is not available, obtaining a first control point motion vector as an affine merge candidate by using a motion vector of a second block included in a first reference picture, wherein the second block is a reference block located at a point diagonally adjacent to the lower right corner of the current block; When the left block of the current block is available, obtaining a second control point motion vector as an affine merge candidate by using a motion vector of a third block in the left block, wherein the third block is adjacent to the lower left corner of the current block; When the left block of the current block is not available, obtaining a second control point motion vector as an affine merge candidate by using a motion vector of a fourth block included in a second reference picture, wherein the fourth block is a reference block located at a point diagonally adjacent to the lower left corner of the current block; Obtaining predicted sample points of the current block by using motion vectors included in an affine merge candidate list including the affine merge candidates; and Generating a bitstream, the bitstream including a flag indicating an encoding order of blocks including the current block and the right block.

3. A method for sending a bitstream, the bitstream being generated by the video encoding method according to claim 2.

Images