Apparatus and method for encoding and decoding image by using reference block

Abstract

An image decoding method according to an embodiment may comprise the steps of: determining a first reference block in a first reference image and a second reference block in a second reference image for bidirectional prediction of a current block; determining weight information for combining the first reference block and the second reference block based on a current reference template of the current block, a first reference template of the first reference block and a second reference template of the second reference block; using the weight information to combine the first reference block and the second reference block to generate a prediction block for the current block; and reconstructing the current block using the prediction block.

Description

TECHNICAL FIELD

[0001] The disclosure relates to the field of image encoding and decoding, and more particularly, to an apparatus and method for encoding and decoding an image by using a reference block included in at least one of a current image or a previous image. BACKGROUND

[0002] In image encoding and decoding, an image is divided into blocks, and each block is predictively encoded and predictively decoded via inter prediction.

[0003] Inter prediction refers to a technique for compressing an image by removing temporal redundancy between images. In inter prediction, a block of a current image is predicted by using a reference image. A reference block most similar to the current block can be searched within a preset search range within the reference image. The current block is predicted based on the reference block, and a residual block is generated by subtracting a prediction block generated as a result of prediction from the current block.

[0004] In codecs such as H.264 Advanced Video Coding (AVC) and High Efficiency Video Coding (HEVC), in order to predict a motion vector of a current block, a motion vector of a previously encoded block adjacent to the current block included in a previously encoded image is used as a motion vector predictor of the current block. A motion vector difference that is a difference between the motion vector of the current block and the motion vector predictor is signaled to a decoder by using a preset scheme.

[0005] The residual block generated via inter prediction is transferred to a decoder via transformation and quantization. The decoder performs inverse quantization and inverse transformation on the residual block, and reconstructs the current block by combining the prediction block of the current block and the residual block. The decoder can remove artifacts within the reconstructed current block by filtering the reconstructed current block in a regular manner. SUMMARY

[0006] TECHNICAL PROBLEM

[0007] According to an embodiment, an image encoding method and apparatus and an image decoding method and apparatus are provided to improve the performance of predictively encoding and predictively decoding a current block.

[0008] According to an embodiment, an image encoding method and apparatus and an image decoding method and apparatus are provided to reduce the amount of data required to signal an inter prediction mode.

[0009] According to an embodiment, an image encoding method and apparatus and an image decoding method and apparatus are provided to reduce the bit rate of a bitstream.

[0010] The technical features intended to be implemented in the disclosure are not limited to the above-described technical features, and other un-stated technical features will be clearly understood by those of ordinary skill in the art in view of the following description.

[0011] Solution to the problem

[0012] In an embodiment of the disclosure, an image decoding method can include determining a first reference block in a first reference picture and a second reference block in a second reference picture for bi-prediction of a current block.

[0013] In an embodiment of the disclosure, the image decoding method can include determining weight information for combination of the first reference block and the second reference block based on a current reference template of the current block, a first reference template of the first reference block, and a second reference template of the second reference block.

[0014] In an embodiment of the disclosure, the image decoding method can include generating a prediction block of the current block by combining the first reference block and the second reference block using the weight information.

[0015] In an embodiment of the disclosure, the image decoding method can include reconstructing the current block by using the prediction block.

[0016] In an embodiment of the disclosure, an image decoding apparatus 2000 can include at least one memory storing at least one instruction, and at least one processor configured to operate according to the at least one instruction.

[0017] In an embodiment of the disclosure, the at least one processor can be configured to determine a first reference block in a first reference picture and a second reference block in a second reference picture for bi-prediction of a current block.

[0018] In an embodiment of the disclosure, the at least one processor can be configured to determine weight information for combination of the first reference block and the second reference block based on a current reference template of the current block, a first reference template of the first reference block, and a second reference template of the second reference block.

[0019] In an embodiment of the disclosure, the at least one processor can be configured to generate a prediction block of the current block by combining the first reference block and the second reference block using the weight information.

[0020] In an embodiment of the disclosure, the at least one processor can be configured to reconstruct the current block by using the prediction block.

[0021] In an embodiment of the disclosure, an image encoding method can include determining a first reference block in a first reference picture and a second reference block in a second reference picture for bi-prediction of a current block.

[0022] In an embodiment of the disclosure, the image encoding method can include determining weight information for combination of the first reference block and the second reference block based on a current reference template of the current block, a first reference template of the first reference block, and a second reference template of the second reference block.

[0023] In an embodiment of the disclosure, the image encoding method can include generating a prediction block of the current block by combining the first reference block and the second reference block using the weight information.

[0024] In an embodiment of the disclosure, the image encoding method can include encoding the current block using the prediction block.

[0025] In an embodiment of the disclosure, the image encoding apparatus 3200 can include at least one memory storing at least one instruction, and at least one processor configured to operate according to the at least one instruction.

[0026] In an embodiment of the disclosure, the at least one processor can be configured to determine a first reference block in a first reference picture and a second reference block in a second reference picture for bi-prediction of a current block.

[0027] In an embodiment of the disclosure, the at least one processor can be configured to determine weight information for combination of the first reference block and the second reference block based on a current reference template of the current block, a first reference template of the first reference block, and a second reference template of the second reference block.

[0028] In an embodiment of the disclosure, the at least one processor can be configured to generate a prediction block of the current block by combining the first reference block and the second reference block using the weight information.

[0029] In an embodiment of the disclosure, the at least one processor can be configured to encode the current block using the prediction block.

[0030] In an embodiment of the disclosure, there is provided a computer-readable recording medium having stored therein a bitstream generated by an image encoding method, wherein the bitstream can include motion information of a current block.

[0031] In embodiments of the disclosure, motion information of a current block can be generated by determining a first reference block in a first reference picture and a second reference block in a second reference picture for bi-prediction of the current block, obtaining a current reference template indicating a reference template of the current block, a first reference template indicating a reference template of the first reference block, and a second reference template indicating a reference template of the second reference block, determining weight information for a combination of the first reference block and the second reference block based on the current reference template, the first reference template, and the second reference template, generating a prediction block of the current block by combining the first reference block and the second reference block using the weight information, and encoding the current block using the prediction block.

[0032] Advantages of Invention

[0033] According to embodiments, the image encoding method and apparatus and the image decoding method and apparatus can improve performance of prediction encoding and prediction decoding for a current block.

[0034] According to embodiments, the image encoding method and apparatus and the image decoding method and apparatus can reduce an amount of data required to signal an inter prediction mode.

[0035] According to embodiments, the image encoding method and apparatus and the image decoding method and apparatus can reduce a bit rate of a bitstream.

[0036] Technical features intended to be implemented in the present disclosure are not limited to the above-described technical features, and other technical features not stated will be clearly understood by those of ordinary skill in the art in view of the following description. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 is a block diagram of an image decoding apparatus according to an embodiment.

[0038] Figure 2 is a block diagram of an image encoding apparatus according to an embodiment.

[0039] Figure 3 shows a process of determining at least one coding unit by partitioning a current coding unit according to an embodiment.

[0040] Figure 4 shows a process of determining at least one coding unit by partitioning a non-square coding unit according to an embodiment.

[0041] Figure 5 shows a process of partitioning a coding unit based on at least one of block shape information or partition shape mode information according to an embodiment.

[0042] Figure 6 shows a method of determining a preset coding unit from an odd number of coding units according to an embodiment.

[0043] Figure 7 A process of determining a plurality of coding units when the current coding unit is divided is shown according to an embodiment.

[0044] Figure 8 A process of determining that the current coding unit will be divided into an odd number of coding units when the coding units cannot be processed in a preset order is shown according to an embodiment.

[0045] Figure 9 A process of determining at least one coding unit by dividing a first coding unit is shown according to an embodiment.

[0046] Figure 10 A shape to which a second coding unit determined when the first coding unit is divided can be limited when the second coding unit having a non-square shape satisfies a preset condition is shown according to an embodiment.

[0047] Figure 11 A process of dividing a square coding unit when the division shape mode information cannot indicate that the square coding unit is divided into four square coding units is shown according to an embodiment.

[0048] Figure 12 A process of changing a processing order between a plurality of coding units according to a process of dividing the coding units is shown according to an embodiment.

[0049] Figure 13 A process of determining a depth of a coding unit as a shape and a size of the coding unit change when the coding unit is recursively divided such that a plurality of coding units are determined is shown according to an embodiment.

[0050] Figure 14 A depth of a coding unit that can be determined based on a shape and a size of the coding unit and a part index (PID) for distinguishing the coding unit is shown according to an embodiment.

[0051] Figure 15 A plurality of coding units is determined based on a plurality of preset data units included in a picture according to an embodiment.

[0052] Figure 16 A coding unit that can be determined in each picture when a combination of shapes to which the coding unit can be divided is different for each picture is shown according to an embodiment.

[0053] Figure 17 Various shapes of a coding unit that can be determined based on division shape mode information that can be expressed in a binary code are shown according to an embodiment.

[0054] Figure 18 Other shapes of a coding unit that can be determined based on division shape mode information that can be expressed in a binary code are shown according to an embodiment.

[0055] Figure 19 is a block diagram of an image encoding and decoding system according to an embodiment performing loop filtering.

[0056] Figure 20 is a block diagram illustrating a configuration of an image decoding apparatus according to an embodiment.

[0057] Figure 21 is a diagram for describing bi-prediction of a current block according to an embodiment.

[0058] Figure 22 is a diagram illustrating a block related to a current block in time and / or space.

[0059] Figure 23 is a diagram for describing a template of a current block or a reference block according to an embodiment.

[0060] Figure 24 is a diagram for describing an operation of determining a reference block based on template matching according to an embodiment.

[0061] Figure 25 is a diagram for describing a process of determining weight information according to an embodiment.

[0062] Figure 26 is a diagram for describing an operation of determining a weight filter according to an embodiment.

[0063] Figure 27 is a diagram for describing an operation of generating a prediction block by using weight information according to an embodiment.

[0064] Figure 28 is a diagram for describing a detailed process of determining weight information according to an embodiment.

[0065] Figure 29 is a diagram for describing a detailed process of determining a weight filter according to an embodiment.

[0066] Figure 30 is a diagram for describing a detailed process of determining weight information according to an embodiment.

[0067] Figure 31 is a flowchart of an image decoding method according to an embodiment.

[0068] Figure 32 is a block diagram illustrating a configuration of an image encoding apparatus according to an embodiment.

[0069] Figure 33 is a flowchart of an image encoding method according to an embodiment. DETAILED DESCRIPTION

[0070] Since the present disclosure allows various changes and numerous embodiments, particular embodiments will be shown in the drawings and described in detail in the written description. However, this is not intended to limit the present disclosure to a particular mode of practice, and it is to be understood that all changes, equivalents, and substitutes not departing from the spirit and technical scope of the present disclosure are encompassed in the present disclosure.

[0071] In the description of the embodiments, detailed explanation of related technology can be omitted when it is deemed to unnecessarily obscure the essence of the present disclosure. Furthermore, numbers (for example, "first", "second", and the like) in the description of the embodiments are used only to distinguish one element from another element.

[0072] Throughout the present disclosure, the expression "at least one of a, b, or c" indicates only a, only b, only c, both a and b, both a and c, both b and c, all of a, b, and c, or variations thereof.

[0073] Throughout the specification, when an element is referred to as being "connected to" another element or "coupled with" another element, it can be directly connected to the other element or directly coupled with the other element, or it can be indirectly connected to the other element or indirectly coupled with the other element with intermediate elements interposed therebetween.

[0074] In the present specification, as to an element denoted as "unit" or "module", two or more elements can be combined into one element, or one element can be divided into two or more elements according to a subdivided function. In addition, each element described below can additionally perform some or all of the functions performed by another element, and some main functions of each element can be completely performed by another element, in addition to its own main function.

[0075] In the present disclosure, "image" can refer to a picture, a still image, a frame, a moving picture including a plurality of consecutive still images, or a video.

[0076] In the present disclosure, "sample" can refer to data assigned to a sampling position of an image, that is, data to be processed. For example, a pixel within a frame of a spatial domain can correspond to a sample. A unit including a plurality of samples can be defined as a block.

[0077] Hereinafter, referring to Figures 1 to 19 , an image encoding method and apparatus and an image decoding method and apparatus based on a coding unit and a transform unit of a tree structure according to an embodiment are provided.

[0078] Figure 1 A block diagram of an image decoding apparatus 100 according to an embodiment is shown.

[0079] The image decoding apparatus 100 can include a bitstream obtainer 110 and a decoder 120. The bitstream obtainer 110 and the decoder 120 can include at least one processor. Also, the bitstream obtainer 110 and the decoder 120 can include a memory storing instructions to be executed by the at least one processor.

[0080] The bitstream obtainer 110 can receive a bitstream. The bitstream includes information about an image encoded by an image encoding apparatus 200 to be described below. Also, the bitstream can be transmitted from the image encoding apparatus 200. The image encoding apparatus 200 and the image decoding apparatus 100 can be connected by a wired or wireless way, and the bitstream obtainer 110 can receive the bitstream by a wired or wireless way. The bitstream obtainer 110 can receive the bitstream from a storage medium such as an optical medium, a hard disk, or the like. The decoder 120 can reconstruct an image based on information obtained from the received bitstream. The decoder 120 can obtain a syntax element for reconstructing an image from the bitstream. The decoder 120 can reconstruct an image based on the syntax element.

[0081] According to a detailed description of operations of the image decoding apparatus 100, the bitstream obtainer 110 can receive a bitstream.

[0082] The image decoding apparatus 100 can perform an operation of obtaining a bin string corresponding to a division shape mode of a coding unit from a bitstream. The image decoding apparatus 100 can perform an operation of determining a division rule of the coding unit. Also, the image decoding apparatus 100 can perform an operation of dividing the coding unit into a plurality of coding units based on at least one of the bin string corresponding to the division shape mode or the division rule. The image decoding apparatus 100 can determine a first allowable range of a size of the coding unit according to an aspect ratio of the coding unit in order to determine the division rule. The image decoding apparatus 100 can determine a second allowable range of the size of the coding unit according to the division shape mode of the coding unit in order to determine the division rule.

[0083] Hereinafter, a division of a coding unit will be described in detail according to embodiments of the disclosure.

[0084] First, one picture can be divided into one or more slices or one or more tiles. One slice or one tile can be a sequence of one or more largest coding units (coding tree units (CTUs)). According to an embodiment, one slice can include one or more tiles, or one slice can include one or more CTUs. A slice including one or more tiles can be determined within a picture.

[0085] There is a maximum coding block (coding tree block (CTB)) in concept compared to a maximum coding tree unit (CTU). The maximum coding block (CTB) indicates an NxN block including NxN samples (where N is an integer). Each color component can be divided into one or more maximum coding blocks.

[0086] The maximum coding unit (CTU) in the case of a picture including three sample arrays (sample arrays for Y, Cr, and Cb components) is a unit including a maximum coding block of luma samples, two corresponding maximum coding blocks of chroma samples, and a syntax structure for coding the luma and chroma samples. The maximum coding unit in the case of a picture being a monochrome picture is a unit including a maximum coding block of monochrome samples and a syntax structure for coding the monochrome samples. The maximum coding unit in the case of a picture being a picture coded in a color plane separated according to color components is a unit including a syntax structure for coding the picture and samples of the picture.

[0087] One maximum coding block (CTB) can be divided into MxN coding blocks including MxN samples (where M and N are integers).

[0088] The coding unit (CU) in the case of a picture having sample arrays for Y, Cr, and Cb components is a unit including a coding block of luma samples, two corresponding coding blocks of chroma samples, and a syntax structure for coding the luma and chroma samples. The coding unit in the case of a picture being a monochrome picture is a unit including a coding block of monochrome samples and a syntax structure for coding the monochrome samples. The coding unit in the case of a picture being a picture coded in a color plane separated according to color components is a unit including a syntax structure for coding the picture and samples of the picture.

[0089] As described above, the maximum coding block and the maximum coding unit are distinguished from each other in concept, and the coding block and the coding unit are distinguished from each other in concept. That is, the (maximum) coding unit refers to a data structure including a (maximum) coding block including corresponding samples and a syntax structure corresponding to the (maximum) coding block. However, since it is understood by one of ordinary skill in the art that the (maximum) coding unit or the (maximum) coding block refers to a block of a predetermined size including a predetermined number of samples, unless otherwise specified, the maximum coding block and the maximum coding unit, or the coding block and the coding unit are referred to without distinction in the following specification.

[0090] An image can be divided into maximum coding units (CTUs). The size of each maximum coding unit can be determined based on information obtained from a bitstream. The shape of each maximum coding unit can be a square shape of the same size. However, the disclosure is not limited thereto.

[0091] For example, information about a maximum size of a luma coding block can be obtained from the bitstream. For example, the maximum size of the luma coding block indicated by the information about the maximum size of the luma coding block can be one of 4x4, 8x8, 16x16, 32x32, 64x64, 128x128, and 256x256.

[0092] For example, information about a luma block size difference and a maximum size of a luma coding block that can be divided into two can be obtained from the bitstream. The information about the luma block size difference can indicate a size difference between a luma maximum coding unit and a maximum luma coding block that can be divided into two. Accordingly, when the information about the maximum size of the luma coding block that can be divided into two obtained from the bitstream and the information about the luma block size difference are combined with each other, a size of the luma maximum coding unit can be determined. A size of a chroma maximum coding unit can be determined by using the size of the luma maximum coding unit. For example, when a Y:Cb:Cr ratio is 4:2:0 according to a color format, a size of a chroma block can be half of a size of a luma block, and a size of the chroma maximum coding unit can be half of a size of the luma maximum coding unit.

[0093] According to an embodiment, because information about a maximum size of a luma coding block that can be divided into two is obtained from a bitstream, the maximum size of the luma coding block that can be divided into two can be variably determined. In contrast, a maximum size of a luma coding block that can be divided into three can be fixed. For example, the maximum size of the luma coding block that can be divided into three in an I picture can be 32x32, and the maximum size of the luma coding block that can be divided into three in a P picture or a B picture can be 64x64.

[0094] Further, a maximum coding unit can be hierarchically divided into coding units based on division shape mode information obtained from a bitstream. At least one of information indicating whether quad division is performed, information indicating whether multi-division is performed, division direction information, or division type information can be obtained from the bitstream as the division shape mode information.

[0095] For example, the information indicating whether quad division is performed can indicate whether a current coding unit is to be quad divided (QUAD_SPLIT).

[0096] When the current coding unit is not to be quad divided, the information indicating whether multi-division is performed can indicate whether the current coding unit is to be no more divided (NO_SPLIT) or is to be bi / tri divided.

[0097] When the current coding unit is to be bi divided or tri divided, the division direction information indicates that the current coding unit is to be divided in one of a horizontal direction and a vertical direction.

[0098] When the current coding unit is split in the horizontal direction or the vertical direction, the split type information indicates that the current coding unit is bi-split or tri-split.

[0099] A split mode of the current coding unit can be determined according to the split direction information and the split type information. The split mode when the current coding unit is bi-split in the horizontal direction can be determined as a bi-horizontal split mode (SPLIT_BT_HOR), the split mode when the current coding unit is tri-split in the horizontal direction can be determined as a tri-horizontal split mode (SPLIT_TT_HOR), the split mode when the current coding unit is bi-split in the vertical direction can be determined as a bi-vertical split mode (SPLIT_BT_VER), and the split mode when the current coding unit is tri-split in the vertical direction can be determined as a tri-vertical split mode SPLIT_TT_VER.

[0100] The image decoding apparatus 100 can obtain a bin string of the split shape mode information from a bitstream. A form of the bitstream received by the image decoding apparatus 100 can include a fixed length bin, a unary code, a truncated unary code, a predetermined bin, etc. The bin string is information of a binary number. The bin string can include at least one bit. The image decoding apparatus 100 can obtain the split shape mode information corresponding to the bin string based on a split rule. The image decoding apparatus 100 can determine, based on one bin string, whether to perform quad-split on a coding unit, whether to not perform split on the coding unit, a split direction, and a split type.

[0101] A coding unit can be smaller than or equal to a maximum coding unit. For example, because the maximum coding unit is a coding unit having the largest size, the maximum coding unit is one of the coding units. When the split shape mode information about the maximum coding unit indicates that split is not performed, a coding unit determined in the maximum coding unit has the same size as that of the maximum coding unit. When the split shape mode information about the maximum coding unit indicates that split is performed, the maximum coding unit can be split into coding units. Also, when the split shape mode information about a coding unit indicates that split is performed, the coding unit can be split into smaller coding units. However, the split of the image is not limited thereto, and can not distinguish between the maximum coding unit and the coding unit. It will be described in detail with reference to Figures 3 to 16 The split of the coding unit is described in detail.

[0102] Also, one or more prediction blocks for prediction can be determined from the coding unit. The prediction block can be equal to or smaller than the coding unit. Also, one or more transform blocks for transform can be determined from the coding unit. The transform block can be equal to or smaller than the coding unit.

[0103] The shapes and sizes of the transform block and the prediction block can be irrelevant to each other.

[0104] In another embodiment, prediction can be performed by using a coding unit as a prediction unit. In addition, transform can be performed by using a coding unit as a transform block.

[0105] Reference will be made to Figures 3 to 16 The partitioning of a coding unit is described. The current block and the neighboring block of the present disclosure can indicate one of a maximum coding unit, a coding unit, a prediction block, and a transform block. In addition, the current block of the current coding unit is a block that is currently being decoded or coded or a block that is currently being partitioned. The neighboring block can be a block that is reconstructed before the current block. The neighboring block can be spatially or temporally adjacent to the current block. The neighboring block can be located one of below the left, the left, the upper left, the upper, the upper right, the right, the lower right of the current block.

[0106] Figure 3 A process of determining at least one coding unit by partitioning a current coding unit, performed by the image decoding apparatus 100 according to an embodiment, is illustrated.

[0107] The block shape can include 4N×4N, 4N×2N, 2N×4N, 4N×N, N×4N, 32N×N, N×32N, 16N×N, N×16N, 8N×N, or N×8N. Here, N can be a positive integer. The block shape information is information indicating at least one of a shape, a direction, an aspect ratio, or a size of a coding unit.

[0108] The shape of the coding unit can include a square and a non-square. When the lengths of the width and the height of the coding unit are the same (i.e., when the block shape of the coding unit is 4N×4N), the image decoding apparatus 100 can determine the block shape information of the coding unit as a square. The image decoding apparatus 100 can determine the shape of the coding unit as a non-square.

[0109] When the width and the height of the coding unit are not the same as each other (i.e., when the block shape of the coding unit is 4N×2N, 2N×4N, 4N×N, N×4N, 32N×N, N×32N, 16N×N, N×16N, 8N×N, or N×8N), the image decoding apparatus 100 can determine the block shape information of the coding unit as a non-square shape. When the shape of the coding unit is a non-square, the image decoding apparatus 100 can determine the aspect ratio in the block shape information of the coding unit as at least one of 1:2, 2:1, 1:4, 4:1, 1:8, 8:1, 1:16, 16:1, 1:32, or 32:1. In addition, the image decoding apparatus 100 can determine whether the coding unit is in a horizontal direction or a vertical direction based on the length of the width and the length of the height of the coding unit. In addition, the image decoding apparatus 100 can determine the size of the coding unit based on at least one of the length of the width, the length of the height, or the area of the coding unit.

[0110] According to an embodiment, the image decoding apparatus 100 can determine a shape of a coding unit by using the block shape information, and can determine a partitioning method of the coding unit by using the partition shape mode information. That is, the partitioning method of the coding unit indicated by the partition shape mode information can be determined based on the block shape indicated by the block shape information used by the image decoding apparatus 100.

[0111] The image decoding apparatus 100 can obtain the partition shape mode information from the bitstream. However, embodiments are not limited thereto, and the image decoding apparatus 100 and the image encoding apparatus 200 can determine the pre-agreed partition shape mode information based on the block shape information. The image decoding apparatus 100 can determine the pre-agreed partition shape mode information with respect to a maximum coding unit or a minimum coding unit. For example, the image decoding apparatus 100 can determine the partition shape mode information for the maximum coding unit as a four partition. Also, the image decoding apparatus 100 can determine the partition shape mode information with respect to the minimum coding unit as "no partition". Specifically, the image decoding apparatus 100 can determine the size of the maximum coding unit as 256x256. The image decoding apparatus 100 can determine the pre-agreed partition shape mode information as a four partition. The four partition is a partition shape mode in which both the width and the height of the coding unit are bisected. The image decoding apparatus 100 can obtain a coding unit of 128x128 size from the maximum coding unit of 256x256 size based on the partition shape mode information. Also, the image decoding apparatus 100 can determine the size of the minimum coding unit as 4x4. The image decoding apparatus 100 can obtain the partition shape mode information indicating "no partition" with respect to the minimum coding unit.

[0112] According to an embodiment, the image decoding apparatus 100 can use the block shape information indicating that the current coding unit has a square shape. For example, the image decoding apparatus 100 can determine whether to not partition the square coding unit, whether to vertically partition the square coding unit, whether to horizontally partition the square coding unit, or whether to partition the square coding unit into four coding units based on the partition shape mode information. Referring to Figure 3 When the block shape information of the current coding unit 300 indicates a square shape, the decoder 120 can determine that the coding unit 310a having the same size as the current coding unit 300 is not partitioned based on the partition shape mode information indicating no partition, or can determine the coding units 310b, 310c, 310d, 310e, 310f, etc. that are partitioned based on the partition shape mode information indicating a pre-set partitioning method.

[0113] Referring to Figure 3According to embodiments, the image decoding apparatus 100 can determine two coding units 310b obtained by dividing the current coding unit 300 in the vertical direction, based on the division shape mode information indicating that division is performed in the vertical direction. The image decoding apparatus 100 can determine two coding units 310c obtained by dividing the current coding unit 300 in the horizontal direction, based on the division shape mode information indicating that division is performed in the horizontal direction. The image decoding apparatus 100 can determine four coding units 310d obtained by dividing the current coding unit 300 in the vertical and horizontal directions, based on the division shape mode information indicating that division is performed in the vertical and horizontal directions. According to embodiments, the image decoding apparatus 100 can determine three coding units 310e obtained by dividing the current coding unit 300 in the vertical direction, based on the division shape mode information indicating that three divisions are performed in the vertical direction. The image decoding apparatus 100 can determine three coding units 310f obtained by dividing the current coding unit 300 in the horizontal direction, based on the division shape mode information indicating that three divisions are performed in the horizontal direction. However, the division method of a square coding unit is not limited to the above-described methods, and the division shape mode information can indicate various methods. The preset division methods of dividing a square coding unit will be described in detail with respect to various embodiments below.

[0114] Figure 4 A process of determining at least one coding unit by dividing a non-square coding unit, performed by the image decoding apparatus 100 according to embodiments, is shown.

[0115] According to embodiments, the image decoding apparatus 100 can use block shape information indicating that the current coding unit has a non-square shape. The image decoding apparatus 100 can determine whether to not divide the non-square current coding unit or to divide the non-square current coding unit by using a preset division method, based on the division shape mode information. Referring to Figure 4 When the block shape information of the current coding unit 400 or 450 indicates a non-square shape, the image decoding apparatus 100 can determine that the coding unit 410 or 460 having the same size as the current coding unit 400 or 450 is not divided, based on the division shape mode information indicating no division, or can determine the coding units 420a, 420b, 430a, 430b, 430c, 470a, 470b, 480a, 480b, and 480c divided based on the division shape mode information indicating a preset division method. The preset division methods of dividing a non-square coding unit will be described in detail with respect to various embodiments below.

[0116] According to an embodiment, the image decoding apparatus 100 can determine a partitioning method of a coding unit by using the partition shape mode information, and in this case, the partition shape mode information can indicate a number of one or more coding units generated by partitioning the coding unit. Referring to Figure 4 When the partition shape mode information indicates that the current coding unit 400 or 450 is partitioned into two coding units, the image decoding apparatus 100 can determine two coding units 420a and 420b or 470a and 470b included in the current coding unit 400 or 450 by partitioning the current coding unit 400 or 450 based on the partition shape mode information.

[0117] According to an embodiment, when the image decoding apparatus 100 partitions the non-square current coding unit 400 or 450 based on the partition shape mode information, the image decoding apparatus 100 can consider a position of a long side of the non-square current coding unit 400 or 450 in order to partition the current coding unit. For example, the image decoding apparatus 100 can determine a plurality of coding units by partitioning the current coding unit 400 or 450 in a direction in which a long side of the current coding unit 400 or 450 is partitioned, considering the shape of the current coding unit 400 or 450.

[0118] According to an embodiment, when the partition shape mode information indicates that the coding unit is partitioned (tri-partitioned) into an odd number of blocks, the image decoding apparatus 100 can determine an odd number of coding units included in the current coding unit 400 or 450. For example, when the partition shape mode information indicates that the current coding unit 400 or 450 is partitioned into three coding units, the image decoding apparatus 100 can partition the current coding unit 400 or 450 into three coding units 430a, 430b, and 430c or 480a, 480b, and 480c.

[0119] According to an embodiment, the aspect ratio of the current coding unit 400 or 450 can be 4:1 or 1:4. When the aspect ratio is 4:1, because the length of the width is longer than the length of the height, the block shape information can be horizontal. When the aspect ratio is 1:4, because the length of the width is shorter than the length of the height, the block shape information can be vertical. The image decoding apparatus 100 can determine to divide the current coding unit into an odd number of blocks based on the division shape mode information. In addition, the image decoding apparatus 100 can determine a division direction of the current coding unit 400 or 450 based on the block shape information of the current coding unit 400 or 450. For example, when the current coding unit 400 is in a vertical direction, the image decoding apparatus 100 can determine the coding units 430a, 430b, and 430c by dividing the current coding unit 400 in a horizontal direction. In addition, when the current coding unit 450 is in a horizontal direction, the image decoding apparatus 100 can determine the coding units 480a, 480b, and 480c by dividing the current coding unit 450 in a vertical direction.

[0120] According to an embodiment, the image decoding apparatus 100 can determine an odd number of coding units included in the current coding unit 400 or 450, and not all of the determined coding units can have the same size. For example, a predetermined coding unit 430b or 480b among the determined odd number of coding units 430a, 430b, and 430c or 480a, 480b, and 480c can have a size different from the sizes of the other coding units 430a and 430c or 480a and 480c. That is, the coding units that can be determined by dividing the current coding unit 400 or 450 can have various sizes, and in some cases, the odd number of coding units 430a, 430b, and 430c or 480a, 480b, and 480c can all have different sizes.

[0121] According to an embodiment, when the division shape mode information indicates to divide a coding unit into an odd number of blocks, the image decoding apparatus 100 can determine an odd number of coding units included in the current coding unit 400 or 450, and in addition, can apply a predetermined restriction to at least one coding unit among the odd number of coding units generated by dividing the current coding unit 400 or 450. Referring to Figure 4, the image decoding apparatus 100 can set a decoding process for the coding unit 430b or 480b, which is located at the center among three coding units 430a, 430b, and 430c or 480a, 480b, and 480c generated as the current coding unit 400 or 450 is divided, to be different from a decoding process for other coding units 430a and 430c or 480a and 480c. For example, unlike other coding units 430a and 430c or 480a and 480c, the image decoding apparatus 100 can limit the coding unit 430b or 480b at the center position not to be divided again or to be divided only a predetermined number of times.

[0122] Figure 5 A process of dividing a coding unit based on at least one of block shape information or division shape mode information, which is performed by the image decoding apparatus 100 according to an embodiment, is illustrated.

[0123] According to an embodiment, the image decoding apparatus 100 can determine to divide or not to divide the first coding unit 500, which is a square, into coding units based on at least one of the block shape information or the division shape mode information. According to an embodiment, when the division shape mode information indicates to divide the first coding unit 500 in a horizontal direction, the image decoding apparatus 100 can determine a second coding unit 510 by dividing the first coding unit 500 in the horizontal direction. The first, second, and third coding units used according to an embodiment are terms used to understand a relationship before and after a coding unit is divided. For example, a second coding unit can be determined by dividing a first coding unit, and a third coding unit can be determined by dividing the second coding unit. Hereinafter, it will be understood that the structure of the first, second, and third coding units follows the above description.

[0124] According to an embodiment, the image decoding apparatus 100 can determine to divide or not to divide the determined second coding unit 510 into coding units based on the division shape mode information. Referring to Figure 5, the image decoding apparatus 100 can divide the non-square second coding unit 510 determined by dividing the first coding unit 500 into one or more third coding units 520a or 520b, 520c and 520d based on the division shape mode information, or can not divide the non-square second coding unit 510 determined by dividing the first coding unit 500 into one or more third coding units 520a or 520b, 520c and 520d. The image decoding apparatus 100 can obtain the division shape mode information, and can obtain a plurality of second coding units (e.g., the second coding unit 510) of various shapes by dividing the first coding unit 500 based on the obtained division shape mode information, and can divide the second coding unit 510 by using a division method of the first coding unit 500 based on the division shape mode information. According to an embodiment, when the first coding unit 500 is divided into the second coding unit 510 based on the division shape mode information of the first coding unit 500, the second coding unit 510 can also be divided into third coding units (e.g., 520a, or 520b, 520c and 520d) based on the division shape mode information of the second coding unit 510. That is, the coding units can be recursively divided based on the division shape mode information of each coding unit. Accordingly, a square coding unit can be determined by dividing a non-square coding unit, and a non-square coding unit can be determined by recursively dividing a square coding unit.

[0125] Referring to Figure 5 A predetermined coding unit (e.g., a coding unit at a center position or a square coding unit) among the odd number of third coding units 520b, 520c and 520d determined by dividing the non-square second coding unit 510 can be recursively divided. According to an embodiment, the square third coding unit 520b among the odd number of third coding units 520b, 520c and 520d can be divided into a plurality of fourth coding units in a horizontal direction. A non-square fourth coding unit 530b or 530d among the plurality of fourth coding units 530a, 530b, 530c and 530d can be again divided into a plurality of coding units. For example, the non-square fourth coding unit 530b or 530d can be again divided into an odd number of coding units. Methods that can be used to recursively divide coding units will be described below with respect to various embodiments.

[0126] According to an embodiment, the image decoding apparatus 100 can divide each of the third coding units 520a or 520b, 520c, and 520d into coding units based on the division shape mode information. In addition, the image decoding apparatus 100 can determine not to divide the second coding unit 510 based on the division shape mode information. According to an embodiment, the image decoding apparatus 100 can divide the second coding unit 510, which is not a square, into an odd number of third coding units 520b, 520c, and 520d. The image decoding apparatus 100 can apply a predetermined restriction to a predetermined third coding unit among the odd number of third coding units 520b, 520c, and 520d. For example, the image decoding apparatus 100 can restrict the third coding unit 520c, which is located at a center position, among the odd number of third coding units 520b, 520c, and 520d, to not be divided again or to be divided a settable number of times.

[0127] Referring to Figure 5 , the image decoding apparatus 100 can restrict the third coding unit 520c, which is located at a center position, among the odd number of third coding units 520b, 520c, and 520d included in the second coding unit 510, which is not a square, to not be divided again, to be divided by using a predetermined division method (for example, to be divided into only four coding units or to be divided by using the division method of the second coding unit 510), or to be divided only a predetermined number of times (for example, to be divided only n times (where n > 0)). However, the restriction on the third coding unit 520c at the center position is not limited to the above examples and can include various restrictions for decoding the third coding unit 520c at the center position differently from the other third coding units 520b and 520d.

[0128] According to an embodiment, the image decoding apparatus 100 can obtain the division shape mode information for dividing the current coding unit from a predetermined position in the current coding unit.

[0129] Figure 6 A method of determining a predetermined coding unit from among odd number of coding units, performed by the image decoding apparatus 100 according to an embodiment, is illustrated.

[0130] Referring to Figure 6 , the division shape mode information of the current coding unit 600 or 650 can be obtained from a sample at a predetermined position (for example, the sample 640 or 690 at a center position) among a plurality of samples included in the current coding unit 600 or 650. However, the predetermined position in the current coding unit 600 from which at least one piece of division shape mode information can be obtained is not limited to Figure 6The center position in the current coding unit 600 and can include various positions (e.g., top, bottom, left, right, top-left, top-right, bottom-left, and bottom-right positions) included in the current coding unit 600. The image decoding apparatus 100 can obtain the partition shape mode information from the preset positions, and can determine whether to partition the current coding unit into coding units of various shapes and various sizes.

[0131] According to an embodiment, when the current coding unit is partitioned into a preset number of coding units, the image decoding apparatus 100 can select one of the coding units. One of the plurality of coding units can be selected using various methods, and descriptions of the methods will be described with respect to various embodiments below.

[0132] According to an embodiment, the image decoding apparatus 100 can partition the current coding unit into a plurality of coding units, and can determine a coding unit at a preset position.

[0133] According to an embodiment, the image decoding apparatus 100 can use information indicating a position of an odd number of coding units in order to determine a coding unit located at a center position from among the odd number of coding units. Referring to Figure 6 , the image decoding apparatus 100 can determine the odd number of coding units 620a, 620b, and 620c or the odd number of coding units 660a, 660b, and 660c by partitioning the current coding unit 600 or the current coding unit 650. The image decoding apparatus 100 can determine the middle coding unit 620b or the middle coding unit 660b by using information about positions of the odd number of coding units 620a, 620b, and 620c or the odd number of coding units 660a, 660b, and 660c. For example, the image decoding apparatus 100 can determine the coding unit 620b at the center position by determining positions of the coding units 620a, 620b, and 620c based on information indicating positions of preset samples included in the coding units 620a, 620b, and 620c. In detail, the image decoding apparatus 100 can determine the coding unit 620b at the center position by determining positions of the coding units 620a, 620b, and 620c based on information indicating positions of top-left samples 630a, 630b, and 630c of the coding units 620a, 620b, and 620c.

[0134] According to an embodiment, the information indicating the positions of the top-left samples 630a, 630b, and 630c included in the coding units 620a, 620b, and 620c, respectively, can include information on positions or coordinates of the coding units 620a, 620b, and 620c in the picture. According to an embodiment, the information indicating the positions of the top-left samples 630a, 630b, and 630c included in the coding units 620a, 620b, and 620c, respectively, can include information indicating widths or heights of the coding units 620a, 620b, and 620c included in the current coding unit 600, and the widths or heights can correspond to information indicating differences between the coordinates of the coding units 620a, 620b, and 620c in the picture. That is, the image decoding apparatus 100 can determine the coding unit 620b at the center position by directly using the information on the positions or coordinates of the coding units 620a, 620b, and 620c in the picture, or by using the information on the widths or heights of the coding units corresponding to the differences between the coordinates.

[0135] According to an embodiment, the information indicating the position of the top-left sample 630a of the upper coding unit 620a can include a coordinate (xa, ya), the information indicating the position of the top-left sample 630b of the middle coding unit 620b can include a coordinate (xb, yb), and the information indicating the position of the top-left sample 630c of the lower coding unit 620c can include a coordinate (xc, yc). The image decoding apparatus 100 can determine the middle coding unit 620b by using the coordinates of the top-left samples 630a, 630b, and 630c included in the coding units 620a, 620b, and 620c, respectively. For example, when the coordinates of the top-left samples 630a, 630b, and 630c are classified in ascending order or descending order, the coding unit 620b including the coordinate (xb, yb) of the sample 630b at the center position can be determined as the coding unit at the center position among the coding units 620a, 620b, and 620c determined by dividing the current coding unit 600. However, the coordinates indicating the positions of the top-left samples 630a, 630b, and 630c can include coordinates indicating absolute positions in the picture, or can use a coordinate (dxb, dyb) indicating a relative position of the top-left sample 630b of the middle coding unit 620b and a coordinate (dxc, dyc) indicating a relative position of the top-left sample 630c of the lower coding unit 620c with respect to the position of the top-left sample 630a of the upper coding unit 620a. The method of determining a coding unit at a predetermined position by using the coordinates of a sample included in a coding unit as information indicating the position of the sample is not limited to the above-described method, and can include various arithmetic methods using the coordinates of the sample.

[0136] According to an embodiment, the image decoding apparatus 100 can divide the current coding unit 600 into a plurality of coding units 620a, 620b, and 620c, and can select one of the coding units 620a, 620b, and 620c based on a preset criterion. For example, the image decoding apparatus 100 can select the coding unit 620b having a size different from those of the other coding units from among the coding units 620a, 620b, and 620c.

[0137] According to an embodiment, the image decoding apparatus 100 can determine the width or height of each of the coding units 620a, 620b, and 620c by using a coordinate (xa, ya) that is information indicating the position of the top-left sample 630a of the top coding unit 620a, a coordinate (xb, yb) that is information indicating the position of the top-left sample 630b of the middle coding unit 620b, and a coordinate (xc, yc) that is information indicating the position of the top-left sample 630c of the bottom coding unit 620c. The image decoding apparatus 100 can determine the size of each of the coding units 620a, 620b, and 620c by using the coordinates (xa, ya), (xb, yb), and (xc, yc) indicating the positions of the coding units 620a, 620b, and 620c. According to an embodiment, the image decoding apparatus 100 can determine the width of the top coding unit 620a as the width of the current coding unit 600. The image decoding apparatus 100 can determine the height of the top coding unit 620a as yb-ya. According to an embodiment, the image decoding apparatus 100 can determine the width of the middle coding unit 620b as the width of the current coding unit 600. The image decoding apparatus 100 can determine the height of the middle coding unit 620b as yc-yb. According to an embodiment, the image decoding apparatus 100 can determine the width or height of the bottom coding unit by using the width or height of the current coding unit or the width or height of the top coding unit 620a and the middle coding unit 620b. The image decoding apparatus 100 can determine a coding unit having a size different from those of the other coding units based on the determined widths and heights of the coding units 620a, 620b, and 620c. Referring to Figure 6 , the image decoding apparatus 100 can determine the middle coding unit 620b having a size different from those of the top coding unit 620a and the bottom coding unit 620c as the coding unit at the preset position. However, the above-described method of determining a coding unit having a size different from those of the other coding units performed by the image decoding apparatus 100 corresponds only to an example of determining a coding unit at a preset position by using the sizes of the coding units determined based on the coordinates of the samples, and thus various methods of determining a coding unit at a preset position by comparing the sizes of the coding units determined based on the coordinates of the preset samples can be used.

[0138] The image decoding apparatus 100 can determine the width or height of each of the coding units 660a, 660b, and 660c by using the coordinates (xd, yd) that are information indicating the position of the top-left sample 670a of the left coding unit 660a, the coordinates (xe, ye) that are information indicating the position of the top-left sample 670b of the middle coding unit 660b, and the coordinates (xf, yf) that are information indicating the position of the top-left sample 670c of the right coding unit 660c. The image decoding apparatus 100 can determine the size of each of the coding units 660a, 660b, and 660c by using the coordinates (xd, yd), (xe, ye), and (xf, yf) indicating the positions of the coding units 660a, 660b, and 660c.

[0139] According to an embodiment, the image decoding apparatus 100 can determine the width of the left coding unit 660a as xe-xd. The image decoding apparatus 100 can determine the height of the left coding unit 660a as the height of the current coding unit 650. According to an embodiment, the image decoding apparatus 100 can determine the width of the middle coding unit 660b as xf-xe. The image decoding apparatus 100 can determine the height of the middle coding unit 660b as the height of the current coding unit 600. According to an embodiment, the image decoding apparatus 100 can determine the width or height of the right coding unit 660c by using the width or height of the current coding unit 650 or the width or height of the left coding unit 660a and the middle coding unit 660b. The image decoding apparatus 100 can determine a coding unit having a size different from the sizes of the other coding units based on the determined widths and heights of the coding units 660a, 660b, and 660c. Referring to Figure 6 , the image decoding apparatus 100 can determine the middle coding unit 660b having a size different from the sizes of the left coding unit 660a and the right coding unit 660c as a coding unit at a certain position. However, the above-described method of determining a coding unit having a size different from the sizes of the other coding units performed by the image decoding apparatus 100 corresponds only to an example of determining a coding unit at a predetermined position by using the sizes of the coding units determined based on the coordinates of the samples, and thus various methods of determining a coding unit at a predetermined position by comparing the sizes of the coding units determined based on the coordinates of certain samples can be used.

[0140] However, the position of the sample considered for determining the position of the coding unit is not limited to the above-described top-left position, and information on an arbitrary position of a sample included in the coding unit can be used.

[0141] According to an embodiment, the image decoding apparatus 100 can select a coding unit at a preset position from among the odd number of coding units determined by partitioning the current coding unit, by considering a shape of the current coding unit. For example, when the current coding unit has a non-square shape with a width longer than a height, the image decoding apparatus 100 can determine a coding unit at a preset position in a horizontal direction. That is, the image decoding apparatus 100 can determine one of the coding units at different positions in the horizontal direction, and can impose a restriction on the coding unit. When the current coding unit has a non-square shape with a height longer than a width, the image decoding apparatus 100 can determine a coding unit at a preset position in a vertical direction. That is, the image decoding apparatus 100 can determine one of the coding units at different positions in the vertical direction, and can impose a restriction on the coding unit.

[0142] According to an embodiment, the image decoding apparatus 100 can use information indicating respective positions of the even number of coding units, in order to determine a coding unit at a preset position from among the even number of coding units. The image decoding apparatus 100 can determine the even number of coding units by partitioning (bi-partitioning) the current coding unit, and can determine a coding unit at a preset position by using information about positions of the even number of coding units. Operations related thereto can correspond to operations of determining a coding unit at a preset position (e.g., a center position) from among the odd number of coding units, which have been described in detail above with respect to FIGS. 1 to 3, and thus a detailed description thereof will not be provided here. Figure 6

[0143] According to an embodiment, when a current coding unit that is not square is partitioned into a plurality of coding units, a preset information about a coding unit at a preset position can be used in a partitioning operation to determine a coding unit at the preset position from among the plurality of coding units. For example, the image decoding apparatus 100 can use at least one of block shape information or partition shape mode information stored in a sample included in an intermediate coding unit in a partitioning operation to determine a coding unit at a center position from among a plurality of coding units determined by partitioning the current coding unit.

[0144] Reference will now be made to Figure 6 ​, the image decoding apparatus 100 can divide the current coding unit 600 into the plurality of coding units 620a, 620b, and 620c based on the division shape mode information, and can determine the coding unit 620b at the center position among the plurality of coding units 620a, 620b, and 620c. Also, the image decoding apparatus 100 can determine the coding unit 620b at the center position in consideration of a position at which the division shape mode information is obtained. That is, the division shape mode information of the current coding unit 600 can be obtained from the sample 640 at the center position of the current coding unit 600, and when the current coding unit 600 is divided into the plurality of coding units 620a, 620b, and 620c based on the division shape mode information, the coding unit 620b including the sample 640 can be determined as the coding unit at the center position. However, information used to determine the coding unit at the center position is not limited to the division shape mode information, and various types of information can be used to determine the coding unit at the center position.

[0145] According to an embodiment, the preset information used to identify the coding unit at the preset position can be obtained from a preset sample included in the coding unit to be determined. Referring to Figure 6 , the image decoding apparatus 100 can determine the coding unit at the preset position (e.g., the coding unit at the center position among the divided plurality of coding units) among the plurality of coding units 620a, 620b, and 620c determined by dividing the current coding unit 600 using the division shape mode information obtained from the sample at the preset position (e.g., the sample at the center position of the current coding unit 600) in the current coding unit 600. That is, the image decoding apparatus 100 can determine the sample at the preset position by considering the block shape of the current coding unit 600, can determine the coding unit 620b including the sample from which specific information (e.g., the division shape mode information) can be obtained from among the plurality of coding units 620a, 620b, and 620c determined by dividing the current coding unit 600, and can apply a preset restriction to the coding unit 620b. Referring to Figure 5 , according to an embodiment, in the decoding operation, the image decoding apparatus 100 can determine the sample 640 at the center position of the current coding unit 600 as the sample from which the preset information can be obtained, and can apply a preset restriction to the coding unit 620b including the sample 640. However, the position of the sample from which the preset information can be obtained is not limited to the above-described position, and can include any position of the sample included in the coding unit 620b to be determined for the restriction.

[0146] According to an embodiment, the position of the sample from which the preset information can be obtained can be determined based on the shape of the current coding unit 600. According to an embodiment, the block shape information can indicate whether the current coding unit has a square shape or a non-square shape, and the position of the sample from which the preset information can be obtained can be determined based on the shape. For example, the image decoding apparatus 100 can determine a sample located on a boundary that divides at least one of the width or the height of the current coding unit in half as a sample from which the preset information can be obtained by using at least one of information about the width of the current coding unit or information about the height of the current coding unit. As another example, when the block shape information of the current coding unit indicates a non-square shape, the image decoding apparatus 100 can determine one of samples adjacent to a boundary for dividing a long side of the current coding unit in half as a sample from which the preset information can be obtained.

[0147] According to an embodiment, when the current coding unit is divided into a plurality of coding units, the image decoding apparatus 100 can determine a coding unit at a preset position among the plurality of coding units using the division shape mode information. According to an embodiment, the image decoding apparatus 100 can obtain the division shape mode information from a sample at the preset position in the coding unit, and can divide the plurality of coding units by using the division shape mode information obtained from the sample at the preset position in each of the plurality of coding units generated by dividing the current coding unit. That is, the coding units can be recursively divided based on the division shape mode information obtained from the sample at the preset position in each coding unit. The operation of recursively dividing the coding units has been described above with respect to FIGS. 6A to 6D, and thus a detailed description thereof is not provided here. Figure 7 The operation of recursively dividing the coding units is described above, and thus a detailed description thereof is not provided here.

[0148] According to an embodiment, the image decoding apparatus 100 can determine one or more coding units by dividing the current coding unit, and can determine an order in which the one or more coding units are decoded based on a preset block (e.g., the current coding unit).

[0149] Figure 7 An order in which the plurality of coding units are processed when the image decoding apparatus 100 determines the plurality of coding units by dividing the current coding unit is shown according to an embodiment.

[0150] According to an embodiment, based on the division shape mode information, the image decoding apparatus 100 can determine the second coding units 710a and 710b by dividing the first coding unit 700 in a vertical direction, can determine the second coding units 730a and 730b by dividing the first coding unit 700 in a horizontal direction, or can determine the second coding units 750a, 750b, 750c, and 750d by dividing the first coding unit 700 in the vertical direction and the horizontal direction.

[0151] Referring to Figure 7 , the image decoding apparatus 100 can determine to process the second coding units 710a and 710b determined by dividing the first coding unit 700 in the vertical direction in the horizontal direction order 710c. The image decoding apparatus 100 can determine to process the second coding units 730a and 730b determined by dividing the first coding unit 700 in the horizontal direction in the vertical direction order 730c. The image decoding apparatus 100 can determine to process the second coding units 750a, 750b, 750c, and 750d determined by dividing the first coding unit 700 in the vertical direction and the horizontal direction in a predetermined order for processing coding units in one row and then processing coding units in the next row (e.g., in a raster scan order or a zigzag scan order 750e).

[0152] According to an embodiment, the image decoding apparatus 100 can recursively divide coding units. Referring to Figure 7 , the image decoding apparatus 100 can determine a plurality of coding units 710a, 710b, 730a, 730b, 750a, 750b, 750c, and 750d by dividing the first coding unit 700, and can recursively divide each of the determined plurality of coding units 710a, 710b, 730a, 730b, 750a, 750b, 750c, and 750d. The division method of the plurality of coding units 710a, 710b, 730a, 730b, 750a, 750b, 750c, and 750d can correspond to the division method of the first coding unit 700. Accordingly, each of the plurality of coding units 710a, 710b, 730a, 730b, 750a, 750b, 750c, and 750d can be independently divided into a plurality of coding units. Referring to Figure 8 , the image decoding apparatus 100 can determine the second coding units 710a and 710b by dividing the first coding unit 700 in the vertical direction, and can determine to independently divide or not to divide each of the second coding units 710a and 710b.

[0153] According to an embodiment, the image decoding apparatus 100 can determine the third coding units 720a and 720b by dividing the left second coding unit 710a in the horizontal direction, and can not divide the right second coding unit 710b.

[0154] According to an embodiment, the processing order of a coding unit can be determined based on the operation of dividing the coding unit. In other words, the processing order of a divided coding unit can be determined based on the processing order of a coding unit immediately before being divided. The image decoding apparatus 100 can determine the processing order of the third coding units 720a and 720b determined by dividing the left second coding unit 710a independently of the right second coding unit 710b. Because the third coding units 720a and 720b are determined by dividing the left second coding unit 710a in the horizontal direction, the third coding units 720a and 720b can be processed in the vertical direction order 720c. Since the left second coding unit 710a and the right second coding unit 710b are processed in the horizontal direction order 710c, the right second coding unit 710b can be processed after the third coding units 720a and 720b included in the left second coding unit 710a are processed in the vertical direction order 720c. The operation of determining the processing order of a coding unit based on a coding unit before being divided is not limited to the above example, and it is understood that various methods can be used to independently process coding units divided and determined to be various shapes in a preset order.

[0155] Figure 8 A process in which the image decoding apparatus 100 determines that a current coding unit is to be divided into an odd number of coding units when the coding units cannot be processed in a preset order according to an embodiment is illustrated.

[0156] According to an embodiment, the image decoding apparatus 100 can determine whether a current coding unit is divided into an odd number of coding units based on the obtained division shape mode information. Referring to Figure 8 , the first coding unit 800, which is a square, can be divided into the second coding units 810a and 810b, which are not squares, and the second coding units 810a and 810b can be independently divided into the third coding units 820a and 820b and 820c, 820d, and 820e. According to an embodiment, the image decoding apparatus 100 can determine the plurality of third coding units 820a and 820b by dividing the left second coding unit 810a in the horizontal direction, and can divide the right second coding unit 810b into an odd number of third coding units 820c through 820e.

[0157] According to an embodiment, the image decoding apparatus 100 can determine whether the third coding units are arbitrary coding units divided into an odd number of coding units by determining whether the third coding units 820a and 820b and 820c, 820d, and 820e can be processed in a preset order. Referring to Figure 9The image decoding apparatus 100 can determine the third coding units 820a and 820b and 820c, 820d, and 820e by recursively dividing the first coding unit 800. The image decoding apparatus 100 can determine whether any one of the first coding unit 800, the second coding units 810a and 810b, and the third coding units 820a and 820b and 820c, 820d, and 820e is divided into an odd number of coding units based on at least one of the block shape information or the division shape mode information. For example, the right second coding unit 810b among the second coding units 810a and 810b can be divided into an odd number of third coding units 820c, 820d, and 820e. The processing order of the plurality of coding units included in the first coding unit 800 can be a preset order (e.g., a zigzag scan order 830), and the image decoding apparatus 100 can determine whether the third coding units 820c, 820d, and 820e determined by dividing the right second coding unit 810b into an odd number of coding units satisfy a condition for being processed in the preset order.

[0158] According to an embodiment, the image decoding apparatus 100 can determine whether the third coding units 820a and 820b and 820c, 820d, and 820e included in the first coding unit 800 satisfy a condition for being processed in a preset order, which involves whether at least one of the widths or the heights of the second coding units 810a and 810b is halved along the boundaries of the third coding units 820a and 820b and 820c, 820d, and 820e. For example, the third coding units 820a and 820b determined when the height of the left second coding unit 810a of a non-square shape is halved can satisfy the condition. Since the boundaries of the third coding units 820c, 820d, and 820e determined when the right second coding unit 810b is divided into three coding units cannot halve the width or the height of the right second coding unit 810b, it can be determined that the third coding units 820c, 820d, and 820e do not satisfy the condition. When the condition is not satisfied as described above, the image decoding apparatus 100 can determine that the scan order is discontinuous, and can determine that the right second coding unit 810b is divided into an odd number of coding units based on the determined result. According to an embodiment, when a coding unit is divided into an odd number of coding units, the image decoding apparatus 100 can impose a preset restriction on a coding unit at a preset position among the divided coding units, and the restriction or the preset position is described above with respect to various embodiments, and thus a detailed description thereof is not provided herein.

[0159] Figure 9 A process of determining at least one coding unit by dividing a first coding unit 900 performed by the image decoding apparatus 100 according to an embodiment is illustrated.

[0160] According to an embodiment, the image decoding apparatus 100 can partition the first coding unit 900 based on the partition shape mode information obtained via the bitstream obtainer 110. The square first coding unit 900 can be partitioned into four square coding units, or can be partitioned into a plurality of non-square coding units. For example, referring to FIG. 9, the square first coding unit 900 can be partitioned into the second coding units 910a, 910b, 910c, 920a, 920b, and 920c. Figure 9 When the partition shape mode information indicates that the first coding unit 900 is partitioned into non-square coding units, the image decoding apparatus 100 can partition the first coding unit 900 into a plurality of non-square coding units. In detail, when the partition shape mode information indicates that an odd number of coding units are determined by partitioning the first coding unit 900 in a horizontal direction or a vertical direction, the image decoding apparatus 100 can partition the square first coding unit 900 into the odd number of coding units, which are the second coding units 910a, 910b, and 910c determined by partitioning the square first coding unit 900 in the vertical direction or the second coding units 920a, 920b, and 920c determined by partitioning the square first coding unit 900 in the horizontal direction.

[0161] According to an embodiment, the image decoding apparatus 100 can determine whether the second coding units 910a, 910b, 910c, 920a, 920b, and 920c included in the first coding unit 900 satisfy a condition for processing in a preset order, which involves whether at least one of the width or the height of the first coding unit 900 is bisected along the boundaries of the second coding units 910a, 910b, 910c, 920a, 920b, and 920c. Referring to FIG. 9, Figure 9 Since the boundaries of the second coding units 910a, 910b, and 910c determined by partitioning the square first coding unit 900 in the vertical direction do not bisect the width of the first coding unit 900 in half, it can be determined that the first coding unit 900 does not satisfy the condition for processing in the preset order. Also, since the boundaries of the second coding units 920a, 920b, and 920c determined by partitioning the square first coding unit 900 in the horizontal direction do not bisect the height of the first coding unit 900 in half, it can be determined that the first coding unit 900 does not satisfy the condition for processing in the preset order. When the condition is not satisfied as described above, the image decoding apparatus 100 can determine that the scan order is discontinuous, and can determine, based on the result of the determination, that the first coding unit 900 is partitioned into an odd number of coding units. According to an embodiment, when a coding unit is partitioned into an odd number of coding units, the image decoding apparatus 100 can impose a preset restriction on a coding unit at a preset position among the partitioned coding units, and the restriction or the preset position is described above with respect to various embodiments, and thus a detailed description thereof is not provided here.

[0162] According to an embodiment, the image decoding apparatus 100 can determine various shapes of coding units by partitioning the first coding unit.

[0163] Referring to Figure 10 , the image decoding apparatus 100 can partition the square first coding unit 900 or the non-square first coding unit 930 or 950 into various shapes of coding units.

[0164] Figure 11 It is shown that, when the second coding units having non-square shapes determined when the first coding unit 1000 is partitioned by the image decoding apparatus 100 satisfy a preset condition, shapes into which the second coding units can be partitioned are limited according to an embodiment.

[0165] According to an embodiment, the image decoding apparatus 100 can determine to partition the square first coding unit 1000 into the non-square second coding units 1010a, 1010b, 1020a, and 1020b based on the partition shape mode information obtained via the bitstream obtainer 110. The second coding units 1010a, 1010b, 1020a, and 1020b can be independently partitioned. Thus, the image decoding apparatus 100 can determine whether to partition each of the second coding units 1010a, 1010b, 1020a, and 1020b into a plurality of coding units based on the partition shape mode information of each of the second coding units 1010a, 1010b, 1020a, and 1020b. According to an embodiment, the image decoding apparatus 100 can determine the third coding units 1012a and 1012b by partitioning the left second coding unit 1010a determined by partitioning the first coding unit 1000 in the vertical direction in the horizontal direction. However, when the left second coding unit 1010a is partitioned in the horizontal direction, the image decoding apparatus 100 can limit the right second coding unit 1010b from being partitioned in the horizontal direction in which the left second coding unit 1010a is partitioned. When the third coding units 1014a and 1014b are determined by partitioning the right second coding unit 1010b in the same direction, the third coding units 1012a and 1012b or 1014a and 1014b can be determined in a manner that the left second coding unit 1010a and the right second coding unit 1010b are independently partitioned in the horizontal direction. However, this case serves as a case in which the image decoding apparatus 100 partitions the first coding unit 1000 into four square second coding units 1030a, 1030b, 1030c, and 1030d based on the partition shape mode information as well, and can be inefficient in terms of image decoding.

[0166] According to an embodiment, the image decoding apparatus 100 can determine the third coding units 1022a and 1022b or 1024a and 1024b by dividing the non-square second coding units 1020a or 1020b determined by dividing the first coding unit 1000 in the horizontal direction in the vertical direction. However, when a second coding unit (e.g., the upper second coding unit 1020a) is divided in the vertical direction, the image decoding apparatus 100 can restrict another second coding unit (e.g., the lower second coding unit 1020b) from being divided in the vertical direction in which the upper second coding unit 1020a is divided, for the above-described reason.

[0167] Figure 12 A process of dividing a square coding unit when division shape mode information cannot indicate that the square coding unit is divided into four square coding units, performed by the image decoding apparatus 100 according to an embodiment, is illustrated.

[0168] According to an embodiment, the image decoding apparatus 100 can determine the second coding units 1110a, 1110b, 1120a, 1120b, etc. by dividing the first coding unit 1100 based on the division shape mode information. The division shape mode information can include information on various methods of dividing a coding unit, but the information on the various division methods can not include information for dividing a coding unit into four square coding units. According to such division shape mode information, the image decoding apparatus 100 can not divide a square first coding unit 1100 into four square second coding units 1130a, 1130b, 1130c, and 1130d. Based on the division shape mode information, the image decoding apparatus 100 can determine the non-square second coding units 1110a, 1110b, 1120a, 1120b, etc.

[0169] According to an embodiment, the image decoding apparatus 100 can independently divide the non-square second coding units 1110a, 1110b, 1120a, 1120b, etc. Each of the second coding units 1110a, 1110b, 1120a, 1120b, etc. can be recursively divided in a preset order, and the division method can correspond to the method of dividing the first coding unit 1100 based on the division shape mode information.

[0170] For example, the image decoding apparatus 100 can determine the square third coding units 1112a and 1112b by dividing the left second coding unit 1110a in the horizontal direction, and can determine the square third coding units 1114a and 1114b by dividing the right second coding unit 1110b in the horizontal direction. Also, the image decoding apparatus 100 can determine the square third coding units 1116a, 1116b, 1116c, and 1116d by dividing both the left second coding unit 1110a and the right second coding unit 1110b in the horizontal direction. In this case, coding units having the same shape as the four square second coding units 1130a, 1130b, 1130c, and 1130d divided from the first coding unit 1100 can be determined.

[0171] As another example, the image decoding apparatus 100 can determine the square third coding units 1122a and 1122b by dividing the upper second coding unit 1120a in the vertical direction, and can determine the square third coding units 1124a and 1124b by dividing the lower second coding unit 1120b in the vertical direction. Also, the image decoding apparatus 100 can determine the square third coding units 1126a, 1126b, 1126c, and 1126d by dividing both the upper second coding unit 1120a and the lower second coding unit 1120b in the vertical direction. In this case, coding units having the same shape as the four square second coding units 1130a, 1130b, 1130c, and 1130d divided from the first coding unit 1100 can be determined.

[0172] Figure 12 It is shown that the processing order between a plurality of coding units can be changed in accordance with a process of dividing a coding unit according to an embodiment.

[0173] According to an embodiment, the image decoding apparatus 100 can divide the first coding unit 1200 based on the division shape mode information. When the block shape indicates a square shape and the division shape mode information indicates that the first coding unit 1200 is divided in at least one of a horizontal direction or a vertical direction, the image decoding apparatus 100 can determine second coding units (e.g., second coding units 1210a, 1210b, 1220a, 1220b, etc.) by dividing the first coding unit 1200. Referring to Figure 11, the second coding units 1210a, 1210b, 1220a, and 1220b, which are non-square determined by dividing the first coding unit 1200 in only the horizontal direction or the vertical direction, can be independently divided based on the division shape mode information of each coding unit. For example, the image decoding apparatus 100 can determine third coding units 1216a, 1216b, 1216c, and 1216d by dividing the second coding units 1210a and 1210b generated by dividing the first coding unit 1200 in the vertical direction in the horizontal direction, and can determine third coding units 1226a, 1226b, 1226c, and 1226d by dividing the second coding units 1220a and 1220b generated by dividing the first coding unit 1200 in the horizontal direction in the vertical direction. The above is described with reference to FIG. 12, and thus a detailed description thereof will not be provided here. Figure 7 The operation of dividing the second coding units 1210a, 1210b, 1220a, and 1220b is described, and thus a detailed description thereof will not be provided here.

[0174] According to an embodiment, the image decoding apparatus 100 can process the coding units in a predetermined order. The above is described with reference to FIG. 11, and thus a detailed description thereof will not be provided here. Figure 12 The operation of processing the coding units in the predetermined order is described, and thus a detailed description thereof will not be provided here. Reference is made to Figure 12 , the image decoding apparatus 100 can determine four square third coding units 1216a, 1216b, 1216c, and 1216d and 1226a, 1226b, 1226c, and 1226d by dividing the square first coding unit 1200. According to an embodiment, the image decoding apparatus 100 can determine a processing order of the third coding units 1216a, 1216b, 1216c, and 1216d and 1226a, 1226b, 1226c, and 1226d based on a division method of the first coding unit 1200.

[0175] According to an embodiment, the image decoding apparatus 100 can determine third coding units 1216a, 1216b, 1216c, and 1216d by dividing the second coding units 1210a and 1210b generated by dividing the first coding unit 1200 in the vertical direction in the horizontal direction, and can process the third coding units 1216a, 1216b, 1216c, and 1216d in the following processing order 1217: first processing the third coding units 1216a and 1216c included in the left second coding unit 1210a in the vertical direction, and then processing the third coding units 1216b and 1216d included in the right second coding unit 1210b in the vertical direction.

[0176] According to an embodiment, the image decoding apparatus 100 can determine the third coding units 1226a, 1226b, 1226c, and 1226d by dividing the second coding units 1220a and 1220b generated by dividing the first coding unit 1200 in the horizontal direction in the vertical direction, and can process the third coding units 1226a, 1226b, 1226c, and 1226d in the following processing order 1227: first, the third coding units 1226a and 1226b included in the upper second coding unit 1220a are processed in the horizontal direction, and then the third coding units 1226c and 1226d included in the lower second coding unit 1220b are processed in the horizontal direction.

[0177] Referring to Figure 13 The square third coding units 1216a, 1216b, 1216c, and 1216d and 1226a, 1226b, 1226c, and 1226d can be determined by dividing the second coding units 1210a and 1210b and 1220a and 1220b, respectively. Although the second coding units 1210a and 1210b determined by dividing the first coding unit 1200 in the vertical direction are different from the second coding units 1220a and 1220b determined by dividing the first coding unit 1200 in the horizontal direction, the third coding units 1216a, 1216b, 1216c, and 1216d divided from the second coding units 1210a and 1210b and the third coding units 1226a, 1226b, 1226c, and 1226d ultimately show the same shape of coding units divided from the first coding unit 1200. Accordingly, by recursively dividing coding units differently based on the division shape mode information, even when the coding units are ultimately determined to be the same shape, the image decoding apparatus 100 can process the plurality of coding units in different orders.

[0178] Figure 13 A process of determining depths of coding units as shapes and sizes of the coding units change when the coding units are recursively divided such that a plurality of coding units are determined according to an embodiment is shown.

[0179] According to an embodiment, the image decoding apparatus 100 can determine depths of coding units based on a preset criterion. For example, the preset criterion can be a length of a long side of a coding unit. When a length of a long side of a coding unit before being divided is 2n times (n > 0) a length of a long side of a current coding unit divided, the image decoding apparatus 100 can determine that a depth of the current coding unit is increased by n from a depth of the coding unit before being divided. Hereinafter, a coding unit having an increased depth is denoted as a coding unit of a deeper depth.

[0180] Referring toFigure 14 According to an embodiment, the image decoding apparatus 100 can determine the second coding unit 1302 and the third coding unit 1304 of a deeper depth by dividing the first coding unit 1300 of a square shape based on the block shape information indicating the square shape (e.g., the block shape information can be represented as "0: square"). Assuming that the size of the first coding unit 1300 of a square shape is 2Nx2N, the second coding unit 1302 determined by dividing the width and height of the first coding unit 1300 by 1 / 2 can have a size of NxN. Also, the third coding unit 1304 determined by dividing the width and height of the second coding unit 1302 by 1 / 2 can have a size of N / 2xN / 2. In this case, the width and height of the third coding unit 1304 are 1 / 4 times the width and height of the first coding unit 1300. When the depth of the first coding unit 1300 is D, the depth of the second coding unit 1302 having a width and height that are 1 / 2 times the width and height of the first coding unit 1300 can be D+1, and the depth of the third coding unit 1304 having a width and height that are 1 / 4 times the width and height of the first coding unit 1300 can be D+2.

[0181] According to an embodiment, the image decoding apparatus 100 can determine the second coding unit 1312 or 1322 and the third coding unit 1314 or 1324 of a deeper depth by dividing the first coding unit 1310 or 1320 of a non-square shape based on the block shape information indicating the non-square shape (e.g., the block shape information can be represented as "1: NS_VER" indicating a non-square shape in which the height is longer than the width, or as "2: NS_HOR" indicating a non-square shape in which the width is longer than the height).

[0182] The image decoding apparatus 100 can determine the second coding unit 1302, 1312, or 1322 by dividing at least one of the width or height of the first coding unit 1310 having a size of Nx2N. That is, the image decoding apparatus 100 can determine the second coding unit 1302 having a size of NxN or the second coding unit 1322 having a size of NxN / 2 by dividing the first coding unit 1310 in the horizontal direction, or can determine the second coding unit 1312 having a size of N / 2xN by dividing the first coding unit 1310 in the horizontal and vertical directions.

[0183] According to an embodiment, the image decoding apparatus 100 can determine a second coding unit (e.g., the second coding unit 1302, 1312, 1322, etc.) by dividing at least one of the width or the height of the first coding unit 1320 having a size of 2N×N. That is, the image decoding apparatus 100 can determine the second coding unit 1302 having a size of N×N or the second coding unit 1312 having a size of N / 2×N by dividing the first coding unit 1320 in a vertical direction, or can determine the second coding unit 1322 having a size of N×N / 2 by dividing the first coding unit 1320 in a horizontal direction and a vertical direction.

[0184] According to an embodiment, the image decoding apparatus 100 can determine a third coding unit 1304, 1314, or 1324 by dividing at least one of the width or the height of the second coding unit 1302 having a size of N×N. That is, the image decoding apparatus 100 can determine the third coding unit 1304 having a size of N / 2×N / 2, the third coding unit 1314 having a size of N / 4×N / 2, or the third coding unit 1324 having a size of N / 2×N / 4 by dividing the second coding unit 1302 in a vertical direction and a horizontal direction.

[0185] According to an embodiment, the image decoding apparatus 100 can determine a third coding unit (e.g., the third coding unit 1304, 1314, 1324, etc.) by dividing at least one of the width or the height of the second coding unit 1312 having a size of N / 2×N. That is, the image decoding apparatus 100 can determine the third coding unit 1304 having a size of N / 2×N / 2 or the third coding unit 1324 having a size of N / 2×N / 4 by dividing the second coding unit 1312 in a horizontal direction, or can determine the third coding unit 1314 having a size of N / 4×N / 2 by dividing the second coding unit 1312 in a vertical direction and a horizontal direction.

[0186] According to an embodiment, the image decoding apparatus 100 can determine a third coding unit (e.g., the third coding unit 1304, 1314, 1324, etc.) by dividing at least one of the width or the height of the second coding unit 1322 having a size of N×N / 2. That is, the image decoding apparatus 100 can determine the third coding unit 1304 having a size of N / 2×N / 2 or the third coding unit 1314 having a size of N / 4×N / 2 by dividing the second coding unit 1322 in a vertical direction, or can determine the third coding unit 1324 having a size of N / 2×N / 4 by dividing the second coding unit 1322 in a vertical direction and a horizontal direction.

[0187] According to an embodiment, the image decoding apparatus 100 can divide the square coding unit 1300, 1302, or 1304 in a horizontal direction or a vertical direction. For example, the image decoding apparatus 100 can determine a first coding unit 1310 having a size of N×2N by dividing the first coding unit 1300 having a size of 2N×2N in a vertical direction, or can determine a first coding unit 1320 having a size of 2N×N by dividing the first coding unit 1300 in a horizontal direction. According to an embodiment, when a depth is determined based on a length of a longest side of a coding unit, the depth of a coding unit determined by dividing the first coding unit 1300 having a size of 2N×2N in a horizontal direction or a vertical direction can be the same as the depth of the first coding unit 1300.

[0188] According to an embodiment, the width and height of the third coding unit 1314 or 1324 can be 1 / 4 times the width and height of the first coding unit 1310 or 1320. When the depth of the first coding unit 1310 or 1320 is D, the depth of the second coding unit 1312 or 1322 having a width and height of 1 / 2 times the width and height of the first coding unit 1310 or 1320 can be D+1, and the depth of the third coding unit 1314 or 1324 having a width and height of 1 / 4 times the width and height of the first coding unit 1310 or 1320 can be D+2.

[0189] Figure 14 A depth that can be determined based on a shape and size of a coding unit and a part index (PID) for distinguishing a coding unit according to an embodiment are shown.

[0190] According to an embodiment, the image decoding apparatus 100 can determine second coding units of various shapes by dividing the square first coding unit 1400. Referring to Figure 14 , the image decoding apparatus 100 can determine the second coding units 1402a and 1402b, 1404a and 1404b, and 1406a, 1406b, 1406c, and 1406d by dividing the first coding unit 1400 in at least one of a vertical direction or a horizontal direction based on the division shape mode information. That is, the image decoding apparatus 100 can determine the second coding units 1402a and 1402b, 1404a and 1404b, and 1406a, 1406b, 1406c, and 1406d based on the division shape mode information of the first coding unit 1400.

[0191] According to an embodiment, depths of the second coding units 1402a and 1402b, 1404a and 1404b, and 1406a, 1406b, 1406c, and 1406d determined based on the partition shape mode information of the square-shaped first coding unit 1400 can be determined based on lengths of long sides thereof. For example, because a length of a side of the square-shaped first coding unit 1400 is equal to lengths of long sides of the non-square-shaped second coding units 1402a and 1402b and 1404a and 1404b, the first coding unit 1400 and the non-square-shaped second coding units 1402a and 1402b and 1404a and 1404b can have the same depth (e.g., D). However, when the image decoding apparatus 100 partitions the first coding unit 1400 into four square-shaped second coding units 1406a, 1406b, 1406c, and 1406d based on the partition shape mode information, because lengths of sides of the square-shaped second coding units 1406a, 1406b, 1406c, and 1406d are 1 / 2 times lengths of sides of the first coding unit 1400, depths of the second coding units 1406a, 1406b, 1406c, and 1406d can be D + 1 deeper than the depth D of the first coding unit 1400.

[0192] According to an embodiment, the image decoding apparatus 100 can determine the plurality of second coding units 1412a and 1412b and 1414a, 1414b, and 1414c by partitioning the first coding unit 1410 having a height longer than a width in a horizontal direction based on the partition shape mode information. According to an embodiment, the image decoding apparatus 100 can determine the plurality of second coding units 1422a and 1422b and 1424a, 1424b, and 1424c by partitioning the first coding unit 1420 having a width longer than a height in a vertical direction based on the partition shape mode information.

[0193] According to an embodiment, depths of the second coding units 1412a and 1412b and 1414a, 1414b, and 1414c or 1422a and 1422b and 1424a, 1424b, and 1424c determined based on the partition shape mode information of the non-square-shaped first coding unit 1410 or 1420 can be determined based on lengths of long sides thereof. For example, because lengths of sides of the square-shaped second coding units 1412a and 1412b are 1 / 2 times lengths of long sides of the first coding unit 1410 having a non-square shape (having a height longer than a width), depths of the square-shaped second coding units 1412a and 1412b are D + 1, which is 1 deeper than a depth D of the non-square-shaped first coding unit 1410.

[0194] Also, the image decoding apparatus 100 can divide the first coding unit 1410, which is not square, into an odd number of second coding units 1414a, 1414b, and 1414c based on the division shape mode information. The odd number of second coding units 1414a, 1414b, and 1414c can include the second coding units 1414a and 1414c, which are not square, and the second coding unit 1414b, which is square. In this case, because the length of the long side of the second coding units 1414a and 1414c, which are not square, and the length of the side of the second coding unit 1414b, which is square, are 1 / 2 times the length of the long side of the first coding unit 1410, the depth of the second coding units 1414a, 1414b, and 1414c can be D+1, where D+1 is 1 deeper than the depth D of the first coding unit 1410, which is not square. The image decoding apparatus 100 can determine the depth of the coding units divided from the first coding unit 1420 having a non-square shape in which the width is longer than the height, by using the above-described method of determining the depth of the coding units divided from the first coding unit 1410.

[0195] According to an embodiment, when the divided odd number of coding units do not have equal sizes, the image decoding apparatus 100 can determine the PIDs for identifying the divided coding units based on a size ratio between the coding units. Referring to Figure 14 , the coding unit 1414b at the center position among the odd number of divided coding units 1414a, 1414b, and 1414c can have a width equal to the widths of the other coding units 1414a and 1414c and a height twice the heights of the other coding units 1414a and 1414c. That is, in this case, the coding unit 1414b at the center position can include two of the other coding units 1414a or 1414c. Accordingly, when the PID of the coding unit 1414b at the center position based on the scanning order is 1, the PID of the coding unit 1414c adjacent to the coding unit 1414b can increase by 2, and thus can be 3. That is, there can be a discontinuity in the PID values. According to an embodiment, the image decoding apparatus 100 can determine whether the odd number of divided coding units do not have equal sizes based on whether there is a discontinuity in the PIDs for identifying the divided coding units.

[0196] According to an embodiment, the image decoding apparatus 100 can determine whether to use a specific division method based on the PID values for identifying a plurality of coding units determined by dividing a current coding unit. Referring to Figure 14, the image decoding apparatus 100 can determine an even number of coding units 1412a and 1412b or an odd number of coding units 1414a, 1414b, and 1414c by dividing the first coding unit 1410 having a rectangular shape with a height longer than a width. The image decoding apparatus 100 can use a PID indicating each coding unit in order to identify each coding unit. According to an embodiment, the PID can be obtained from a sample at a preset position (e.g., a top-left sample) of each coding unit.

[0197] According to an embodiment, the image decoding apparatus 100 can determine a coding unit at a preset position among divided coding units by using a PID for distinguishing the coding units. According to an embodiment, when the division shape mode information of the first coding unit 1410 having a rectangular shape with a height longer than a width indicates that the coding unit is divided into three coding units, the image decoding apparatus 100 can divide the first coding unit 1410 into three coding units 1414a, 1414b, and 1414c. The image decoding apparatus 100 can assign a PID to each of the three coding units 1414a, 1414b, and 1414c. The image decoding apparatus 100 can compare the PIDs of the divided odd number of coding units in order to determine a coding unit at a center position among the coding units. The image decoding apparatus 100 can determine the coding unit 1414b having a PID corresponding to an intermediate value among the PIDs of the coding units as a coding unit at a preset position among the coding units determined by dividing the first coding unit 1410. According to an embodiment, when the divided coding units do not have equal sizes, the image decoding apparatus 100 can determine a PID for distinguishing the divided coding units based on a size ratio between the coding units. Referring to Figure 15The width of the coding unit 1414b generated by dividing the first coding unit 1410 can be equal to the widths of the other coding units 1414a and 1414c, and the height of the coding unit 1414b can be twice the heights of the other coding units 1414a and 1414c. In this case, when the PID of the coding unit 1414b at the center position is 1, the PID of the coding unit 1414c located next to the coding unit 1414b can increase by 2, and thus can be 3. When the PIDs are not uniformly increased as described above, the image decoding apparatus 100 can determine that the coding unit is divided into a plurality of coding units including a coding unit having a size different from those of the other coding units. According to an embodiment, when the division shape mode information indicates that the coding unit is divided into an odd number of coding units, the image decoding apparatus 100 can divide the current coding unit in such a manner that a coding unit at a predetermined position (e.g., a coding unit at a center position) among the odd number of coding units has a size different from those of the other coding units. In this case, the image decoding apparatus 100 can determine the coding unit at the center position having the different size by using the PIDs of the coding units. However, the PIDs of the predetermined positions and the sizes or positions of the coding units are not limited to the above examples, and various PIDs and various positions and sizes of the coding units can be used.

[0198] According to an embodiment, the image decoding apparatus 100 can use a predetermined data unit in which the coding unit starts to be recursively divided.

[0199] Figure 15 A determination of a plurality of coding units based on a plurality of predetermined data units included in a picture according to an embodiment is illustrated.

[0200] According to an embodiment, a specific data unit can be defined as a data unit in which the coding unit starts to be recursively divided by using the division shape mode information. That is, the predetermined data unit can correspond to a coding unit used to determine an uppermost depth of a plurality of coding units divided from a current picture. In the following description, the predetermined data unit is referred to as a reference data unit for convenience of description.

[0201] According to an embodiment, the reference data unit can have a predetermined size and a predetermined size shape. According to an embodiment, the reference data unit can include MxN samples. Here, M and N can be equal to each other, and can be an integer expressed as a power of 2. That is, the reference data unit can have a square or a non-square shape, and can be divided into an integer number of coding units.

[0202] According to an embodiment, the image decoding apparatus 100 can divide the current picture into a plurality of reference data units. According to an embodiment, the image decoding apparatus 100 can divide the plurality of reference data units divided from the current picture by using the division shape mode information of each of the reference data units. The operation of dividing the reference data units can correspond to the division operation using a quad-tree structure.

[0203] According to an embodiment, the image decoding apparatus 100 can previously determine a minimum size allowed for the reference data units included in the current picture. Accordingly, the image decoding apparatus 100 can determine various reference data units having a size equal to or greater than the minimum size, and can determine one or more coding units by using the division shape mode information with reference to the determined reference data units.

[0204] Referring to FIG. 15, Figure 3 , the image decoding apparatus 100 can use a square reference coding unit 1500 or a non-square reference coding unit 1502. According to an embodiment, the shape and size of the reference coding unit can be determined based on various data units (e.g., a sequence, a picture, a slice, a slice segment, a tile, a tile group, a largest coding unit, etc.) capable of including one or more reference coding units.

[0205] According to an embodiment, the bitstream obtainer 110 of the image decoding apparatus 100 can obtain at least one of reference coding unit shape information or reference coding unit size information for each of various data units from a bitstream. The above-described operation of dividing the current coding unit 300 describes an operation of dividing the square reference coding unit 1500 into one or more coding units, and the above-described operation of dividing the current coding unit 400 or 450 describes an operation of dividing the non-square reference coding unit 1502 into one or more coding units. Accordingly, a detailed description thereof is not provided here. Figure 4 Figure 12 According to an embodiment, the bitstream obtainer 110 of the image decoding apparatus 100 can obtain at least one of reference coding unit shape information or reference coding unit size information for each of various data units from a bitstream. The above-described operation of dividing the current coding unit 300 describes an operation of dividing the square reference coding unit 1500 into one or more coding units, and the above-described operation of dividing the current coding unit 400 or 450 describes an operation of dividing the non-square reference coding unit 1502 into one or more coding units. Accordingly, a detailed description thereof is not provided here.

[0206] ​According to an embodiment, the image decoding apparatus 100 can use the PID for identifying the size and shape of the reference coding unit to determine the size and shape of the reference coding unit from some data units previously determined based on a preset condition. That is, the bitstream obtainer 110 can obtain from the bitstream only the PID for identifying the size and shape of the reference coding unit for each slice, slice segment, tile, tile group, or largest coding unit that is a data unit (e.g., a data unit having a size equal to or smaller than a slice) among various data units (e.g., sequence, picture, slice, slice segment, tile, tile group, largest coding unit, etc.) satisfying the preset condition. The image decoding apparatus 100 can determine the size and shape of the reference data unit for each data unit satisfying the preset condition by using the PID. When the reference coding unit shape information and the reference coding unit size information are obtained from the bitstream according to each data unit having a relatively small size and used, efficiency of using the bitstream can not be high, and thus, the PID can be obtained and used only, instead of directly obtaining the reference coding unit shape information and the reference coding unit size information. In this case, at least one of the size or the shape of the reference coding unit corresponding to the PID for identifying the size and shape of the reference coding unit can be previously determined. That is, the image decoding apparatus 100 can determine at least one of the size or the shape of the reference coding unit included in the data unit used as a unit for obtaining the PID by selecting at least one of the size or the shape of the reference coding unit previously determined based on the PID.

[0207] According to an embodiment, the image decoding apparatus 100 can use one or more reference coding units included in the largest coding unit 1510. That is, the largest coding unit partitioned from a picture can include one or more reference coding units, and the coding units can be determined by recursively partitioning each reference coding unit. According to an embodiment, at least one of the width or the height of the largest coding unit can be an integer multiple of at least one of the width or the height of the reference coding unit. According to an embodiment, the size of the reference coding unit can be obtained by partitioning the largest coding unit n times based on a quad-tree structure. That is, according to various embodiments, the image decoding apparatus 100 can determine the reference coding unit by n times partitioning of the largest coding unit based on a quad-tree structure, and can partition the reference coding unit based on at least one of the block shape information or the partition shape mode information.

[0208] According to an embodiment, the image decoding apparatus 100 can obtain block shape information indicating a shape of a current coding unit or partition shape mode information indicating a partition method of the current coding unit from a bitstream, and can use the obtained information. The partition shape mode information can be included in the bitstream in relation to various data units. For example, the image decoding apparatus 100 can use the partition shape mode information included in a sequence parameter set, a picture parameter set, a video parameter set, a slice header, a slice segment header, a tile header, or a tile group header. Also, the image decoding apparatus 100 can obtain syntax elements corresponding to the block shape information or the partition shape mode information from the bitstream according to each largest coding unit or each reference coding unit, and can use the obtained syntax elements.

[0209] Hereinafter, a method of determining a partition rule according to an embodiment of the disclosure will be described in detail.

[0210] The image decoding apparatus 100 can determine a partition rule of an image. The partition rule can be determined in advance between the image decoding apparatus 100 and the image encoding apparatus 200. The image decoding apparatus 100 can determine the partition rule of the image based on information obtained from a bitstream. The image decoding apparatus 100 can determine the partition rule based on information obtained from at least one of a sequence parameter set, a picture parameter set, a video parameter set, a slice header, a slice segment header, a tile header, or a tile group header. The image decoding apparatus 100 can determine the partition rule differently according to a frame, a slice, a tile, a temporal layer, a largest coding unit, or a coding unit.

[0211] The image decoding apparatus 100 can determine a partition rule based on a block shape of a coding unit. The block shape can include a size, a shape, an aspect ratio, and a direction of the coding unit. The image encoding apparatus 200 and the image decoding apparatus 100 can determine the partition rule based on the block shape information of the coding unit determined in advance. However, the disclosure is not limited thereto. The image decoding apparatus 100 can determine the partition rule based on information obtained from a bitstream received from the image encoding apparatus 200.

[0212] A shape of a coding unit can include a square and a non-square. When lengths of a width and a height of the coding unit are the same, the image decoding apparatus 100 can determine the shape of the coding unit as a square. Also, when the lengths of the width and the height of the coding unit are not the same, the image decoding apparatus 100 can determine the shape of the coding unit as a non-square.

[0213] The size of the coding unit can include various sizes such as 4x4, 8x4, 4x8, 8x8, 16x4, 16x8, …, 256x256. The size of the coding unit can be classified based on the length of the long side, the length of the short side, or the area of the coding unit. The image decoding apparatus 100 can apply the same division rule to the coding units classified into the same group. For example, the image decoding apparatus 100 can classify the coding units having the same length of the long side as having the same size. Also, the image decoding apparatus 100 can apply the same division rule to the coding units having the same length of the long side.

[0214] The aspect ratio of the coding unit can include 1:2, 2:1, 1:4, 4:1, 1:8, 8:1, 1:16, 16:1, 32:1, 1:32, etc. Also, the direction of the coding unit can include a horizontal direction and a vertical direction. The horizontal direction can indicate a case where the length of the width of the coding unit is longer than the length of the height thereof. The vertical direction can indicate a case where the length of the width of the coding unit is shorter than the length of the height thereof.

[0215] The image decoding apparatus 100 can adaptively determine the division rule based on the size of the coding unit. The image decoding apparatus 100 can differently determine the allowable division shape mode based on the size of the coding unit. For example, the image decoding apparatus 100 can determine whether to allow division based on the size of the coding unit. The image decoding apparatus 100 can determine the division direction according to the size of the coding unit. The image decoding apparatus 100 can determine the allowable division type according to the size of the coding unit.

[0216] Determining the division rule based on the size of the coding unit can be a division rule determined in advance between the image encoding apparatus 200 and the image decoding apparatus 100. Also, the image decoding apparatus 100 can determine the division rule based on information obtained from the bitstream.

[0217] The image decoding apparatus 100 can adaptively determine the division rule based on the position of the coding unit. The image decoding apparatus 100 can adaptively determine the division rule based on the position of the coding unit in the image.

[0218] Also, the image decoding apparatus 100 can determine the division rule such that the coding units generated via different division paths do not have the same block shape. However, the disclosure is not limited thereto, and the coding units generated via different division paths have the same block shape. The coding units generated via different division paths can have different decoding processing orders. Since the above is described with reference to Figure 16 The decoding processing order is described, and thus details thereof are not provided here.

[0219] Figure 16It is shown that combinations of shapes into which a coding unit can be divided are determined differently in each picture according to an embodiment.

[0220] Referring to Figure 17 , the image decoding apparatus 100 can determine combinations of division shapes into which a coding unit can be divided to be different in each picture. For example, the image decoding apparatus 100 can decode an image by using a picture 1600 that can be divided into four coding units, a picture 1610 that can be divided into two or four coding units, and a picture 1620 that can be divided into two, three, or four coding units among one or more pictures included in the image. To divide the picture 1600 into a plurality of coding units, the image decoding apparatus 100 can use only division shape information indicating division into four square coding units. To divide the picture 1610, the image decoding apparatus 100 can use only division shape information indicating division into two or four coding units. To divide the picture 1620, the image decoding apparatus 100 can use only division shape information indicating division into two, three, or four coding units. The combinations of division shapes are only an embodiment for describing the operation of the image decoding apparatus 100, and thus should not be interpreted only for this embodiment, but should be interpreted as combinations of various division shapes that can be used in each preset data unit.

[0221] According to an embodiment, the bitstream obtainer 110 of the image decoding apparatus 100 can obtain a bitstream including an index indicating a combination of division shape information for each preset data unit (e.g., each sequence, each picture, each slice, each slice segment, each tile, each tile group, etc.). For example, the bitstream obtainer 110 can obtain an index indicating a combination of division shape information from a sequence parameter set, a picture parameter set, a slice header, a tile header, or a tile group header. The image decoding apparatus 100 can determine a combination of division shapes into which a coding unit can be divided in each preset data unit by using the obtained index, and thus a combination of different division shapes can be used in each preset data unit.

[0222] Figure 17 It is shown that various shapes of a coding unit that can be determined based on division shape mode information that can be expressed in binary code according to an embodiment.

[0223] According to an embodiment, the image decoding apparatus 100 can divide a coding unit into various shapes by using block shape information and division shape mode information obtained via the bitstream obtainer 110. Shapes into which a coding unit can be divided can correspond to various shapes including the shapes described above in the embodiments.

[0224] Referring to Figure 17, the image decoding apparatus 100 can divide the square coding unit in at least one of a horizontal direction and a vertical direction based on the division shape mode information, and can divide the non-square coding unit in the horizontal direction or the vertical direction.

[0225] According to an embodiment, when the image decoding apparatus 100 can divide the square coding unit into four square coding units in the horizontal direction and the vertical direction, the division shapes of the square coding unit that can be indicated by the division shape mode information can be four shapes. According to an embodiment, the division shape mode information can be expressed in a 2-bit binary code, and the binary code can be assigned to each division shape. For example, when the coding unit is not divided, the division shape mode information can be expressed as (00)b, when the coding unit is divided in the horizontal direction and the vertical direction, the division shape mode information can be expressed as (01)b, when the coding unit is divided in the horizontal direction, the division shape mode information can be expressed as (10)b, and when the coding unit is divided in the vertical direction, the division shape mode information can be expressed as (11)b.

[0226] According to an embodiment, when the image decoding apparatus 100 divides the non-square coding unit in the horizontal direction or the vertical direction, the division shape type that can be indicated by the division shape mode information can be determined based on the number of divisions into which the coding unit is divided. Referring to Figure 17 According to an embodiment, the image decoding apparatus 100 can divide the non-square coding unit into three coding units. The image decoding apparatus 100 can divide the coding unit into two coding units, and in this case, the division shape mode information can be expressed as (10)b. The image decoding apparatus 100 can divide the coding unit into three coding units, and in this case, the division shape mode information can be expressed as (11)b. The image decoding apparatus 100 can determine not to divide the coding unit, and in this case, the division shape mode information can be expressed as (0)b. That is, the image decoding apparatus 100 can not use a fixed length coding (FLC), but can use a variable length coding (VLC) in order to use the binary code indicating the division shape mode information.

[0227] According to an embodiment, referring to Figure 17 The binary code of the division shape mode information indicating that the coding unit is not divided can be expressed as (0)b. In the case where the binary code of the division shape mode information indicating that the coding unit is not divided is set in (00)b, even when there is no division shape mode information set in (01)b, the binary code of the 2-bit division shape mode information must be used in its entirety. However, as Figure 17As shown in FIG. 10, in the case of using three division shapes for the non-square coding unit, even when a 1-bit binary code (0)b is used as the division shape mode information, the image decoding apparatus 100 can determine not to divide the coding unit, and can effectively use the bitstream. However, it should not be interpreted that the division shape of the non-square coding unit indicated by the division shape mode information is limited to the three shapes shown in FIG. 10, but should be interpreted that the division shape corresponds to various shapes including the above-described embodiments. Figure 18

[0228] Figure 18 FIG. 11 shows other shapes of the coding unit that can be determined based on the division shape mode information that can be expressed in a binary code according to an embodiment.

[0229] Referring to FIG. 12, Figure 18 , the image decoding apparatus 100 can divide the square coding unit in the horizontal direction or the vertical direction based on the division shape mode information, and can divide the non-square coding unit in the horizontal direction or the vertical direction. That is, the division shape mode information can indicate that the square coding unit is divided in one direction. In this case, a binary code of the division shape mode information indicating not to divide the square coding unit can be expressed as (0)b. In the case where a binary code of the division shape mode information indicating not to divide the coding unit is set in (00)b, even when there is no division shape mode information set in (01)b, a binary code of 2-bit division shape mode information must be used in total. However, as shown in FIG. 12, in the case of using three division shapes for the square coding unit, even when a 1-bit binary code (0)b is used as the division shape mode information, the image decoding apparatus 100 can determine not to divide the coding unit, and can effectively use the bitstream. However, it should not be interpreted that the division shape of the square coding unit indicated by the division shape mode information is limited to the three shapes shown in FIG. 12, but should be interpreted that the division shape corresponds to various shapes including the above-described embodiments. Figure 18 Figure 17

[0230] According to an embodiment, the block shape information or the division shape mode information can be expressed by using a binary code, and such information can be immediately generated as a bitstream. Alternatively, the block shape information or the division shape mode information that can be expressed in a binary code can not be immediately generated as a bitstream, and can be used as a binary code to be input to a context adaptive binary arithmetic coding (CABAC).

[0231] ​​​According to an embodiment, a process in which the image decoding apparatus 100 obtains a syntax of the block shape information or the partition shape mode information via CABAC will now be described. The bitstream including the binary code for the syntax can be obtained via the bitstream obtainer 110. The image decoding apparatus 100 can detect the syntax element indicating the block shape information or the partition shape mode information by performing inverse binarization on the bin string included in the obtained bitstream. According to an embodiment, the image decoding apparatus 100 can calculate a set of binary bin strings corresponding to the syntax element to be decoded, and can decode each bin by using the probability information. The image decoding apparatus 100 can repeat the decoding until the bin string configured with such decoded bins becomes equal to one of the pre-calculated bin strings. The image decoding apparatus 100 can determine the syntax element by performing inverse binarization on the bin string.

[0232] According to an embodiment, the image decoding apparatus 100 can determine the syntax of the bin string by performing a decoding process of adaptive binary arithmetic coding. The image decoding apparatus 100 can update the probability model of the bins obtained via the bitstream obtainer 110. Referring to Figure 17 , according to an embodiment, the bitstream obtainer 110 of the image decoding apparatus 100 can obtain a bitstream indicating a binary code, wherein the binary code indicates the partition shape mode information. The image decoding apparatus 100 can determine the syntax of the partition shape mode information by using the binary code having 1 bit or 2 bits. To determine the syntax of the partition shape mode information, the image decoding apparatus 100 can update the probability of each bit in the 2-bit binary code. That is, depending on whether the value of the first bin in the 2-bit binary code is 0 or 1, the image decoding apparatus 100 can update the probability that the next bit has a value of 0 or 1 at the time of decoding.

[0233] According to an embodiment, in the process of determining the syntax, the image decoding apparatus 100 can update the probability of the bins used in the process of decoding the bin string of the syntax, the image decoding apparatus 100 can determine that the probability is not updated in a specific bit of the bin string and the specific bit has the same probability.

[0234] Referring to Figure 19In the process of determining the syntax by using the bin string indicating the division shape mode information on the non-square coding unit, when the non-square coding unit is not divided, the image decoding apparatus 100 can determine the syntax of the division shape mode information by using one bin having a value of 0. That is, in the case where the block shape information indicates that the current coding unit has a non-square coding unit, the first bin of the bin string on the division shape mode information can be 0 when the non-square coding unit is not divided, and can be 1 when the non-square coding unit is divided into two or three coding units. Accordingly, the probability that the first bin of the bin string on the division shape mode information of the non-square coding unit is 0 can be 1 / 3, and the probability that the first bin is 1 can be 2 / 3. As described above, the division shape mode information indicating that the non-square coding unit is not divided can be represented only by a bin string having a zero value 1 bit, and only when the first bin of the division shape mode information is 1, the image decoding apparatus 100 can determine the syntax of the division shape mode information by determining whether the second bin is 0 or 1. According to an embodiment, when the first bin of the division shape mode information is 1, the image decoding apparatus 100 can decode the bin by determining that the probability that the second bin is 0 or 1 is the same.

[0235] According to an embodiment, the image decoding apparatus 100 can use various probabilities of each bin in the process of determining the bins of the bin string on the division shape mode information. According to an embodiment, the image decoding apparatus 100 can determine that the probabilities of the bins on the division shape mode information are different according to the direction of the non-square block. According to an embodiment, the image decoding apparatus 100 can determine that the probabilities of the bins on the division shape mode information are different according to the area or the length of the long side of the current coding unit. According to an embodiment, the image decoding apparatus 100 can determine that the probabilities of the bins on the division shape mode information are different according to at least one of the shape of the current coding unit or the length of the long side.

[0236] According to an embodiment, the image decoding apparatus 100 can determine that the probabilities of the bins on the division shape mode information are equal for coding units having a preset size or greater. For example, it can be determined that the probabilities of the bins on the division shape mode information are equal for coding units having a size of 64 samples or greater.

[0237] According to an embodiment, the image decoding apparatus 100 can determine the initial probabilities of the bins constituting the bin string of the division shape mode information based on a slice type (e.g., I slice, P slice, or B slice).

[0238] Figure 2 is a block diagram of an image encoding and decoding system that performs loop filtering.

[0239] The encoding end 1910 of the image encoding and decoding system 1900 transmits an encoded bitstream of an image, and the decoding end 1950 outputs a reconstructed image by receiving and decoding the bitstream. Here, the encoding end 1910 can have a configuration similar to that of the image encoding apparatus 200, and the decoding end 1950 can have a configuration similar to that of the image decoding apparatus 100.

[0240] At the encoding end 1910, the prediction encoder 1915 outputs a reference image via inter prediction and intra prediction, and the transformer and quantizer 1920 outputs quantized transform coefficients of residual data between predicted data and a current input image. The entropy encoder 1925 transforms the quantized transform coefficients by encoding the quantized transform coefficients, and outputs the transformed quantized transform coefficients as a bitstream. The quantized transform coefficients are reconstructed as spatial domain data via the dequantizer and inverse transformer 1930, and the spatial domain data is output as a reconstructed image via the deblocking filter 1935 and the loop filter 1940. The reconstructed image can be used as a reference image for a next input image via the prediction encoder 1915.

[0241] The encoded image data in the bitstream received by the decoding end 1950 is reconstructed as spatial domain residual data via the entropy decoder 1955 and the dequantizer and inverse transformer 1960. When the reference image output from the prediction decoder 1975 and the residual data are combined, the spatial domain image data is configured, and the deblocking filter 1965 and the loop filter 1970 can output a reconstructed image with respect to a current original image by performing filtering on the spatial domain image data. The reconstructed image can be used as a reference image for a next original image by the prediction decoder 1975.

[0242] The loop filter 1940 of the encoding end 1910 performs loop filtering by using filter information input according to a user input or a system setting. The filter information used by the loop filter 1940 is output to the entropy encoder 1925, and then transmitted to the decoding end 1950 together with the encoded image data. The loop filter 1970 of the decoding end 1950 can perform loop filtering based on the filter information input from the decoding end 1950.

[0243] In the various embodiments described above, operations related to the image decoding method performed by the image decoding apparatus 100 are described. Hereinafter, operations of the image encoding apparatus 200 performing an image encoding method corresponding to inverse processing of the image decoding method will now be described in various embodiments.

[0244] Figures 3 to 19A block diagram of an image encoding apparatus 200 capable of encoding an image based on at least one of block shape information or partition shape mode information according to an embodiment is shown.

[0245] The image encoding apparatus 200 can include an encoder 220 and a bitstream generator 210. The encoder 220 can receive an input image and then can encode the input image. The encoder 220 can obtain at least one syntax element by encoding the input image. The syntax element can include at least one of a skip flag, a prediction mode, a motion vector difference, a motion vector prediction method (or index), a transform quantization coefficient, an encoding block mode, an encoding block flag, an intra prediction mode, a direct flag, a merge flag, a delta QP, a reference index, a prediction direction, or a transform index. The encoder 220 can determine a context model based on block shape information including at least one of a shape, a direction, an aspect ratio, or a size of an encoding unit.

[0246] The bitstream generator 210 can generate a bitstream based on the encoded input image. For example, the bitstream generator 210 can generate a bitstream by entropy encoding the syntax element based on the context model. In addition, the image encoding apparatus 200 can transmit the bitstream to the image decoding apparatus 100.

[0247] According to an embodiment, the encoder 220 of the image encoding apparatus 200 can determine a shape of an encoding unit. For example, the encoding unit can have a square shape or a non-square shape, and information indicating the shape can be included in the block shape information.

[0248] According to an embodiment, the encoder 220 can determine which shape the encoding unit will be partitioned into. The encoder 220 can determine a shape of at least one encoding unit included in the encoding unit, and the bitstream generator 210 can generate a bitstream including partition shape mode information including information about the shape of the encoding unit.

[0249] According to an embodiment, the encoder 220 can determine whether the encoding unit will be partitioned or will not be partitioned. When the encoder 220 determines that only one encoding unit is included in the encoding unit or that the encoding unit will not be partitioned, the bitstream generator 210 can generate a bitstream including partition shape mode information indicating that the encoding unit will not be partitioned. In addition, the encoder 220 can partition the encoding unit into a plurality of encoding units included in the encoding unit, and the bitstream generator 210 can generate a bitstream including partition shape mode information indicating that the encoding unit will be partitioned into the plurality of encoding units.

[0250] According to an embodiment, information indicating how many coding units a coding unit is divided into or a direction in which the coding unit is divided can be included in the division shape mode information. For example, the division shape mode information can indicate a division in at least one of a vertical direction and a horizontal direction, or can indicate no division.

[0251] The image encoding apparatus 200 can determine the division shape mode information based on the division shape mode of the coding unit. The image encoding apparatus 200 determines a context model based on at least one of a shape, a direction, an aspect ratio, or a size of the coding unit. Then, the image encoding apparatus 200 generates the division shape mode information for dividing the coding unit as a bitstream based on the context model.

[0252] To determine the context model, the image encoding apparatus 200 can obtain an array for at least one of a shape, a direction, an aspect ratio, or a size of the coding unit corresponding to the at least one, and an index for the context model. The image encoding apparatus 200 can obtain the index for the context model from the array based on at least one of a shape, a direction, an aspect ratio, or a size of the coding unit. The image encoding apparatus 200 can determine the context model based on the index for the context model.

[0253] To determine the context model, the image encoding apparatus 200 can further determine the context model based on block shape information including at least one of a shape, a direction, an aspect ratio, or a size of a neighboring coding unit adjacent to the coding unit. Also, the neighboring coding unit can include at least one of a coding unit located below, to the left, above, to the right, or to the upper right of the coding unit.

[0254] Also, to determine the context model, the image encoding apparatus 200 can compare a width length of an upper neighboring coding unit with a width length of the coding unit. Also, the image encoding apparatus 200 can compare a height length of a left neighboring coding unit and a right neighboring coding unit with a height length of the coding unit. Also, the image encoding apparatus 200 can determine the context model based on a result of the comparison.

[0255] Since the operation of the image encoding apparatus 200 includes a process of referring to a table, a description thereof will not be provided here. Figure 20 The operation of the image decoding apparatus 100 described above is similar to that of the image encoding apparatus 200, and thus a detailed description thereof will not be provided here.

[0256] Figure 20 is a block diagram illustrating a configuration of an image decoding apparatus 2000 according to an embodiment.

[0257] Referring to Figure 20 The image decoding apparatus 2000 can include an obtainer 2010 and a prediction decoder 2030. Figure 1The bitstream obtainer 2010 shown in FIG. 2 can correspond to the bitstream obtainer 110 shown in FIG. 1. Figure 1 The prediction decoder 2030 shown in FIG. 2 can correspond to the prediction decoder 2030 shown in FIG. 1. Figure 19 The decoder 120 shown in FIG. 2 can correspond to the decoder 120 shown in FIG. 1. In addition, the obtainer 2010 can correspond to the entropy decoder 1955 shown in FIG. 1, and the prediction decoder 2030 can correspond to the prediction decoder 1975 shown in FIG. 1. Figure 19 The decoder 120 shown in FIG. 2 can correspond to the decoder 120 shown in FIG. 1. In addition, the obtainer 2010 can correspond to the entropy decoder 1955 shown in FIG. 1, and the prediction decoder 2030 can correspond to the prediction decoder 1975 shown in FIG. 1. Figures 24 to 29 The decoder 120 shown in FIG. 2 can correspond to the decoder 120 shown in FIG. 1. In addition, the obtainer 2010 can correspond to the entropy decoder 1955 shown in FIG. 1, and the prediction decoder 2030 can correspond to the prediction decoder 1975 shown in FIG. 1.

[0258] In an embodiment, the obtainer 2010 and the prediction decoder 2030 can be implemented as at least one processor. In an embodiment, the obtainer 2010 and the prediction decoder 2030 can operate according to instructions stored in a memory.

[0259] In an embodiment, the image encoding apparatus 2000 can include a memory that stores input / output data of the obtainer 2010 and the prediction decoder 2030. In addition, the image encoding apparatus 2000 can include a memory controller configured to control data input / output to / from the memory.

[0260] In an embodiment, the obtainer 2010 can obtain a bitstream generated as an encoding result of an image.

[0261] In an embodiment, the bitstream can include an encoding result of a current block. The bitstream can include a plurality of pieces of information used to reconstruct the current block. The current block can be a largest coding unit, a coding unit, a transform unit, or a prediction unit, which is divided from a current picture to be decoded.

[0262] In an embodiment, the prediction decoder 2150 can determine a current block based on block shape information and / or a split shape mode corresponding to at least one level among a sequence parameter set, a picture parameter set, a video parameter set, a slice header, and a slice segment header, which are included in the bitstream.

[0263] In an embodiment, the obtainer 2010 can receive a bitstream from an image encoding apparatus via a network.

[0264] In an embodiment, the obtainer 2010 can obtain a bitstream from a data storage medium including a magnetic medium such as a hard disk, a floppy disk, and a magnetic tape, an optical recording medium such as a compact disc read only memory (CD-ROM) and a digital versatile disc (DVD), a magneto-optical medium such as a floptical disc, or the like.

[0265] In an embodiment, the obtainer 2010 can obtain, from a bitstream, syntax elements used to decode an image. According to a layered structure of an image, values corresponding to the syntax elements can be included in the bitstream.

[0266] In an embodiment, the obtainer 2010 can obtain the syntax elements by entropy-decoding binary bits included in the bitstream.

[0267] In an embodiment, the bitstream can include information indicating a prediction mode of a current block in a current picture. The prediction mode of the current block can include an inter mode. The inter mode refers to a mode in which the current block is predicted or reconstructed based on a reference picture to reduce temporal redundancy between pictures.

[0268] In an embodiment, the prediction decoder 2030 can generate a prediction block of the current block by performing inter prediction on the current block based on the prediction mode of the current block, and can reconstruct the current block by using the prediction block.

[0269] In an embodiment, when the prediction decoder 2030 reconstructs the current block based on a reference picture, the prediction decoder 2030 can use one reference picture (e.g., uni-prediction), or can use two reference pictures (e.g., bi-prediction). Whether the current block is uni-predicted or bi-predicted can be determined according to explicit information included in the bitstream, or can be implicitly determined from a prediction mode of a neighboring block related to the current block.

[0270] In an embodiment, when the current block is bi-predicted, the prediction decoder 2030 can determine a reference block of the current block using motion information included in the bitstream, or can perform implicit determination from a reference block of a neighboring block related to the current block. The motion information of the current block can include at least one of a reference picture index, a motion vector, a differential motion vector, or a reference direction, and can include all information for a motion vector used to predict the current block.

[0271] In an embodiment, the prediction decoder 2030 can use a motion vector of a block included in a previously decoded block neighboring the current block or a block included in a previously decoded picture as a motion vector predictor of the current block, or can use a motion vector difference that is a difference between a motion vector of the current block and the motion vector predictor, and thus can determine a first motion vector and a second motion vector.

[0272] In an embodiment, when the current block is bi-predicted, the prediction decoder 2030 can determine a first reference block in a first reference picture and a second reference block in a second reference picture for bi-prediction of the current block. For example, the prediction decoder 2030 can determine the first reference block and the second reference block for bi-prediction of the current block by using motion information. The prediction decoder 2030 can determine the first reference block and the second reference block for bi-prediction of the current block based on template matching.

[0273] In an embodiment, the prediction decoder 2030 can determine a motion vector by using motion information or a reference block of a neighboring block, and can determine a block indicated by the determined motion vector as at least one of the first reference block or the second reference block.

[0274] In an embodiment, the prediction decoder 2030 can correct a motion vector by comparing a current reference template with a template in a preset region including a block indicated by the determined motion vector, and thus, can determine at least one of the first reference block or the second reference block of the current block. The current reference template can indicate a template of the current block. That is, the reference block can be determined based on only the motion information, or the reference block can be determined based on the motion information and the template matching. When the prediction decoder 2030 is capable of defining templates of the first reference block and the second reference block, the prediction decoder 2030 can determine the reference block regardless of whether the motion vector is corrected by performing the template matching. In an embodiment, the prediction decoder 2030 can determine weight information for combining the reference blocks to generate a prediction block of the current block based on the templates of the reference blocks and the current reference template.

[0275] In an embodiment, the weight candidate list usage information is information indicating whether to use a weight candidate list to determine the weight. When the weight candidate list usage information indicates that the weight candidate list is used, the prediction decoder 2030 can obtain the weight information for determining the reference block based on a weight candidate value determined from the weight candidate list by using a weight index obtained from the bitstream or a reference block of a neighboring block. Alternatively, when the weight candidate list usage information indicates that the weight candidate list is not used, the prediction decoder 2030 can obtain information indicating whether to determine the weight information by using the motion information or the reference block of the neighboring block, without the weight candidate list and the weight index.

[0276] In an embodiment, the weight candidate list usage information can be included in a sequence parameter set, a picture parameter set, a slice header, or slice data of the bitstream.

[0277] In an embodiment, the weight candidate list usage information can not be included in the bitstream. When the weight candidate list usage information is not included in the bitstream, the prediction decoder 2030 can obtain the weight candidate list and the weight index according to a predetermined method, or can determine the weight information by using the motion information or the reference block of the neighboring block, without the weight candidate list and the weight index.

[0278] In an embodiment, the prediction decoder 2030 can determine whether to use the weight candidate list based on information indicated by information obtained from the bitstream. For example, the weight candidate list usage information can include a flag or an index indicating to use the weight candidate list and / or the weight index or indicating not to use the weight candidate list and / or the weight index.

[0279] In an embodiment, the prediction decoder 2030 can obtain the weight candidate list usage information by using at least one neighboring block and a reference block of the neighboring block. The prediction decoder 2030 can determine the weight candidate list and / or the weight index of the reference block of the current block by using at least one of the weight candidate list and / or the weight index of the reference block of the neighboring block. For example, the preset weight candidate list can be a list including -2, 3, 4, 5, and 10 as elements. Alternatively, the preset weight candidate list can be a list including 1, 3, 4, 5, 7, 2, and 6 as elements. The weight candidate list is not limited to the disclosed examples. The prediction decoder can determine the weight index of the reference block of the current block to have the same value as the weight index of the reference block of the neighboring block. The prediction decoder can determine the weight index of the reference block of the current block to have a different value from the weight index of the reference block by additionally obtaining a delta value of the weight index from the bitstream or performing template matching.

[0280] In an embodiment, there can be a plurality of methods of representing the weight information. For example, the weight information can be determined by using a single weight (e.g., using a constant W), can be determined by using a left weight and an upper weight (using a constant W corresponding to the left weight and a constant W corresponding to the upper weight and a constant W corresponding to the upper weight ), or can be determined by using a weight filter (using a pixel-by-pixel W having weight values corresponding to samples in the block, respectively).

[0281] In an embodiment, the information indicating the weight information representation method can be included in a sequence parameter set, a picture parameter set, a slice header, or slice data of the bitstream.

[0282] In an embodiment, the information indicating the weight information representation method can not be included in the bitstream, and in this case, the prediction decoder 2030 can determine the weight information of the current block according to a predetermined method.

[0283] In an embodiment, the information indicating the weight information representation method indicates using information obtained from the bitstream, the prediction decoder 2030 can determine the weight information according to the weight information representation method indicated by the information obtained from the bitstream among a plurality of weight information representation methods. For example, the information indicating the weight information representation method can include a flag or an index indicating any one of a plurality of weight information representation methods.

[0284] In an embodiment, the prediction decoder 2030 can obtain information indicating a weight information representation method by using at least one neighboring block. In this case, even when information indicating the weight information representation method is not obtained from the bitstream, the representation method of the weight information can be determined by using at least one neighboring block. For example, when the neighboring block determines the weight information by using a single weight, as in the neighboring block, the weight information representation method of the current block can be determined to use a single weight.

[0285] Hereinafter, a description will be given of Figures 21 to 22 an operation of determining weight information for combining a reference block.

[0286] In an embodiment, the prediction decoder 2030 can generate a prediction block of the current block by combining the reference block using the determined weight information.

[0287] In an embodiment, the prediction decoder 2030 can determine the prediction block as the reconstructed current block.

[0288] In an embodiment, the prediction decoder 2030 can generate the reconstructed current block by combining the prediction block with residual data obtained by the obtainer 2010 from the bitstream.

[0289] According to a standard such as High Efficiency Video Coding (HEVC), Versatile Video Coding (VVC), etc., a prediction block is generated based on weight information obtained from a bitstream, and this can cause an increase in the amount of data requested in signaling of an inter prediction mode. Accordingly, in an embodiment, the prediction decoder 2030 can determine a reference block, and can determine weight information by using a template of the reference block, thereby reducing the amount of data requested in signaling.

[0290] Hereinafter, a description will be given of Figure 21 Now, a description will be given of bi-prediction performed on a current block.

[0291] Figure 22 is a diagram for describing bi-prediction of a current block according to an embodiment.

[0292] In an embodiment, a current block 2114 included in a current picture 2110 can be uni-predicted using a first reference picture 2120 included in List 0 or a second reference picture 2130 included in List 1, or can be bi-predicted by using the first reference picture 2120 included in List 0 and the second reference picture 2130 included in List 1.

[0293] In an embodiment, the prediction decoder 2030 can determine a first reference picture 2120 and a second reference picture 2130 to be referenced by the current block for bi-prediction of the current block, and can determine a first motion vector 2122 indicating a first reference block 2124 in the first reference picture 2120 and a second motion vector 2132 indicating a second reference block 2134 in the second reference picture 2130. Accordingly, the prediction decoder 2030 can determine the first reference block 2124 based on the first motion vector 2122, and can determine the second reference block 2134 based on the second motion vector 2132.

[0294] For example, the prediction decoder 2030 can determine the first reference picture 2120 and the second reference picture 2130 as the reference pictures to be referenced by the current block based on information included in the bitstream, or can determine the first reference picture 2120 and the second reference picture 2130 as the reference pictures to be referenced by the current block 2114, considering pictures referenced by neighboring blocks related to the current block 2114.

[0295] In an embodiment, to determine the first motion vector 2122 and the second motion vector 2132, the prediction decoder 2030 can generate a motion vector candidate list by using motion vectors of temporal blocks related to the current block in time and spatial blocks related to the current block in space. The prediction decoder 2030 can determine the first motion vector 2122 and the second motion vector 2132 by using a motion vector candidate among a plurality of motion vector candidates included in the motion vector candidate list, wherein the motion vector candidate is indicated by information included in the bitstream.

[0296] Figure 22 is a diagram illustrating blocks related to the current block 2114 in time and / or space.

[0297] Referring to Figure 22 , the temporal block can include a collocated block (hereinafter, referred to as a "col block") Col located at a point corresponding to the current block 2114 and in a reference picture having a picture order count (POC) different from that of the current block 2114, and at least one block Br adjacent to the col block Col in space. The block Br can be located right below the current block 2114 and the col block Col. The collocated block can be determined as any of blocks in a collocated picture and having a position and a size corresponding to the current block, or blocks adjacent to the blocks having the position and the size corresponding to the current block.

[0298] The spatial block related to the current block 2114 in space can include at least one of a left-bottom-external block A0, a left-bottom block A1, a right-top-external block B0, a right-top block B1, or a left-top-external block B2.

[0299] Figure 21 The positions of the time block and the space block shown in the middle are one example, and in other embodiments, the number of the time block and the space block and their positions can vary.

[0300] In an embodiment, the prediction decoder 2030 can directly search for the first reference block 2124 and the second reference block 2134 to be used in the reconstruction of the current block 2114 in the first reference picture 2120 and the second reference picture 2130, or can identify via other methods to determine them. In this case, the prediction decoder 2030 can identify or determine the first reference block 2124 and the second reference block 2134 by using the same method performed by the image encoding apparatus 3100.

[0301] Referring back to Figure 26 In an embodiment, when the first reference block 2124 in the first reference picture 2120 and the second reference block 2134 in the second reference picture 2130 are determined, the prediction decoder 2030 combines the first reference block 2124 and the second reference block 2134 and reconstructs the current block 2114 based on the result of the combination. In this regard, the combination of the first reference block 2124 and the second reference block 2134 can indicate that the samples included in the first reference block 2124 and the samples included in the second reference block 2134 are linearly combined.

[0302] In an embodiment, the combination of the first reference block 2124 and the second reference block 2134 can be performed differently according to the weight candidate list usage information.

[0303] In an embodiment, the weight candidate list can include weight candidate values used in the bi-prediction of the current block. In addition, weight indices corresponding to the weight candidates used in the bi-prediction of the current block, respectively, can be assigned. The weight candidate values can correspond to five values or three values defined in the VVC standard. For example, the weight candidate list can include weight candidate values of -2, 3, 4, 5, and 10, or can include weight candidate values of 3, 4, and 5. In addition, weight indices of 0 to 4 can be assigned to the weight candidate values, respectively. However, the type / number of weight candidate values and the weight indices assigned to the candidate values, respectively, are only one example, and can vary within a range obvious to those skilled in the art.

[0304] For example, the weight candidate list can include weight candidate values of -4, -3, -2, -1, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, and 12, or can include weight candidate values of 1, 2, 3, 4, 5, 6, and 7.

[0305] In an embodiment, the prediction decoder 2030 can determine the weight information by using the weight candidate list and the weight index. The weight index can be obtained from the bitstream or can be obtained from a neighboring block.

[0306] In an embodiment, when the weight information is determined by using the weight candidate list, a linear combination of samples included in the first reference block 2124 and samples included in the second reference block 2134 can be performed by using Equation 1 below.

[0307] [Equation 1]

[0308] pbSamples[x][y] =

[0309] (w0 predSamplesL0[x][y]+w1 predSamplesL1[x][y]+offset) >> (shift)

[0310] In Equation 1, pbSamples[x][y] indicates a result value of a combination of a sample at an x, y position in the first reference block 2124 and a sample at an x, y position in the second reference block 2134, predSamplesL0[x][y] indicates a sample value at an x, y position of the first reference block 2124 included in the first reference picture 2120 in List 0, and predSamplesL1[x][y] indicates a sample value at an x, y position of the second reference block 2134 included in the second reference picture 2130 in List 1. Further, offset and shift are predetermined values.

[0311] In Equation 1, w0 indicates a weight candidate value corresponding to the weight index, and w1 indicates a pair value of w0. The pair value indicates a value obtained by applying the weight candidate value corresponding to the weight index to a prediction calculation formula, and for example, w1 can have a value of 8-w0. In other embodiments, w1 can be the weight candidate value corresponding to the weight index, and w0 can be the pair value of w1.

[0312] As described above, when the weight candidate value indicated by the weight index among the weight candidate values included in the weight candidate list is identified, the weight candidate value is used as a weight for a combination of the first reference block and the second reference block. That is, in an embodiment, when the first reference block and the second reference block are combined for bi-prediction of the current block, the weight candidate value from which the most similar combination result for the current block is derived is selected, so that the size of residual data included in the bitstream can be reduced.

[0313] Reference will be made to Figure 23The detailed description describes an operation of determining weight information and combining the first reference block 2124 and the second reference block 2134 by using motion information or a reference block of a neighboring block without using a weight candidate list and a weight index.

[0314] Hereinafter, reference will be made to Figure 23 Now, a template of a current block or a reference block will be described.

[0315] Figure 23 is a diagram for describing a template of a current block or a reference block according to an embodiment.

[0316] In an embodiment, the prediction decoder 2030 can identify neighboring samples adjacent to the current block 2310 in order to perform bi-prediction for the current block 2310. The neighboring samples adjacent to the current block 2310 can be used as a template for generating a prediction block of the current block 2310, and can be referred to as a current reference template 2320. Also, the first reference template and the second reference template can be templates of the first reference block and the second reference block, which are determined based on motion information or template matching, and a method of determining a reference block is not limited to examples of the present specification.

[0317] In an embodiment, the prediction decoder 2030 can identify or determine the current reference template 2320 of the current block 2310. The current reference template 2320 can include at least one of the upper samples 2330 of the current block 2310 and the left samples 2340 of the current block.

[0318] Referring to Figure 23 , although 16 upper samples 2330 and 20 left samples 2340 are shown as the current reference template 2320 for a case in which the size of the current block 2310 is 8x8, the number of the upper samples 2330 and the number of the left samples 2340 used in bi-prediction of the current block 2310 can be differently set. The left samples 2340 can or can not include samples of an upper left outer region 2350 in contact with a top corner of the current block 2310.

[0319] For example, the prediction decoder 2030 can identify p n upper samples 2330 (where p is a constant, and n is the width of the current block 2310), and can identify m q or (m + p) q left samples 2340 (where q is a constant, and m is the height of the current block 2310).

[0320] In an embodiment, although the upper samples 2330 are in Figure 24The middle samples are arranged in two rows, but the upper samples 2330 can be arranged in 0 or 1 row, or can be arranged in three or more rows. In an embodiment, although the left samples 2340 are arranged as two columns, the left samples 2340 can be arranged as 0 or 1 column, or can be arranged as three or more columns. In addition, the left samples 2340 can exclude the samples of the upper left outer region 2350, or can indicate only the samples of the upper left outer region 2350.

[0321] In an embodiment, the prediction decoder 2030 can obtain the same or similar shape template in the same or similar manner as the current reference template 2320 with respect to the reference block of the current block 2310, and the obtained template can be referred to as a template of the reference block or a reference template. When the current block 2310 is bi-predicted, a first reference template of a first reference block and a second reference template of a second reference block can be identified or determined.

[0322] In an embodiment, when the current reference template 2320, the first reference template, and the second reference template are identified or determined, a prediction block of the current block 2310 can be generated by using the current reference template 2320, the first reference template, and the second reference template in the case where the current block 2310 is bi-predicted.

[0323] In an embodiment, the prediction decoder 2030 can determine weight information for a combination of the first reference block and the second reference block based on the current reference template 2320, the first reference template, and the second reference template. The prediction decoder 2030 can generate a prediction block of the current block by combining the first reference block and the second reference block by using the determined weight information.

[0324] Figures 21 to 22 is a diagram for describing an operation of determining a reference block based on template matching according to an embodiment.

[0325] In an embodiment, in order to perform bi-prediction with respect to the current block 2410, the prediction decoder 2030 can determine at least one of a first reference block 2435 of a first reference image 2420 or a second reference block 2455 of a second reference image 2440. For example, the prediction decoder 2030 can determine a motion vector by using motion information or a reference block of a neighboring block, and can determine a block indicated by the determined motion vector as at least one of the first reference block or the second reference block. Alternatively, the prediction decoder 2030 can correct a motion vector by comparing a current reference template with a template in a preset region including the block indicated by the determined motion vector, and thus, can determine at least one of the first reference block or the second reference block of the current block.

[0326] Reference Figure 23The operation of the prediction decoder 2030 to determine a reference block by using motion information or a reference block of a neighboring block to determine a motion vector will be described in detail, and thus, the operation of the prediction decoder 2030 to determine at least one of the first reference block or the second reference block based on template matching will now be described below. That is, the operation of the prediction decoder 2030 to correct a motion vector by comparing a current reference template with a template in a preset area including a block indicated by the determined motion vector, thereby determining at least one of the first reference block or the second reference block of the current block will now be described in detail.

[0327] In an embodiment, the first reference picture 2420 and the second reference picture 2440 can be pictures reconstructed before the current picture.

[0328] In template matching, a neighboring sample set reconstructed before the current block can be used as a template. In an embodiment, the current reference template 2415, which is a template of the current block, can include at least some of the neighboring samples illustrated in FIG. 24. Figure 24

[0329] In an embodiment, the prediction decoder 2030 can search for a template most similar to the current reference template 2415 in the first reference picture 2420 by using the current reference template 2415, and can determine a block neighboring the first reference template 2430, which is the most similar template, as the first reference block 2435 according to a result of the search. Referring to FIG. 24, Figure 24 When the current reference template 2415 is located at the upper left of the current block 2410, a block located at the lower right of the first reference template 2430, which is the most similar template in the first reference picture 2420, according to a result of the search can be determined as the first reference block 2435.

[0330] In an embodiment, the prediction decoder 2030 can search for a template most similar to the current reference template 2415 in the second reference picture 2440 by using the current reference template 2415, and can determine a block neighboring the second reference template 2450, which is the most similar template, as the second reference block 2455 according to a result of the search. Referring to FIG. 24, Figure 21 When the current reference template 2415 is located at the upper left of the current block 2410, a block located at the lower right of the second reference template 2450, which is the most similar template in the second reference picture 2440, according to a result of the search can be determined as the second reference block 2455.

[0331] ​In an embodiment, to search for a template similar to the current reference template 2415, a difference between sample values can be used. For example, a similar template including a sample value most similar to a sample value included in the current reference template 2415 can be determined within the first reference image 2420 and / or the second reference image 2440. In an embodiment, a template having a minimum sum of absolute difference (SAD) value can be determined as the most similar template by using a difference between sample values, however, this is merely an example, and thus, a sum of absolute transformed difference (SATD) or a histogram of oriented gradients (HoG) can be used, and this can vary within a range apparent to those skilled in the art.

[0332] In an embodiment, the prediction decoder 2030 can determine a base motion vector for determining each of the reference blocks in order to perform a search for a similar template. For example, a predetermined motion vector (e.g., a zero vector, a vector based on information obtained from a bitstream) or a motion vector of a block at a predetermined position can be determined as the base motion vector. For example, the base motion vector for determining the first reference block 2435 can be the first motion vector 2122 of Figure 21 , and the base motion vector for determining the second reference block 2455 can be the second motion vector 2132 of Figure 25 .

[0333] In an embodiment, the prediction decoder 2030 can search for a template most similar to the current reference template 2415 within a preset range around a point indicated by the base motion vector. Further, a block adjacent to the first reference template 2430 in the first reference image 2420 can be determined as the first reference block 2435, in which the first reference template 2430 is a template most similar to the current reference template 2415. The prediction decoder 2030 can determine a block adjacent to the second reference template 2450 in the second reference image 2440 as the second reference block 2455, in which the second reference template 2450 is a template most similar to the current reference template 2415.

[0334] Figure 26 is a diagram for describing a process of determining weight information according to an embodiment.

[0335] In an embodiment, the prediction decoder 2030 can determine weight information for a combination of the first reference block 2510 and the second reference block 2550 based on the current reference template 2590, the first reference template 2510, and the second reference template 2550. The prediction decoder can determine the weight information by using the current reference template 2590, the first reference template 2510, and the second reference template 2550 without using the weight candidate list and the weight index.

[0336] In an embodiment, the weight information can include information on weights for a linear combination of the first reference block and the second reference block. For example, the weight information can include a weight to be applied to the first reference block and a weight to be applied to the second reference block.

[0337] In an embodiment, the operation of determining the weight information can include an operation of determining the first weight. Also, the first weight can be determined as one of a single weight 2530, a left weight 2532 and an above weight 2534, or a weight filter.

[0338] In an embodiment, the first weight, which is a weight to be applied to the first reference block, can be at least one of a constant or a matrix. For example, the first weight can be one weight to be commonly applied to the first reference block 2510, and in this case, the first weight can be a single weight 2530 having a value The first weight can include a left weight 2532 having a value obtained based on at least one left sample of each template and an above weight 2534 having a value obtained based on at least one above sample of each template. When the first weight is expressed as a matrix, the first weight can be a weight filter having a value The elements of the weight filter can each be independently determined, and can indicate weights corresponding to pixels or samples, respectively. The matrix in the present disclosure indicates that the element values of the matrix are used to indicate the respective sample values included in the block, other configurations for indicating the respective sample values included in the block can be used, and the present disclosure is not limited thereto.

[0339] In an embodiment, when a weight to be applied to the first reference block (hereinafter, a first weight) is determined, a weight to be applied to the second reference block (hereinafter, a second weight) can be determined according to the determined first weight. For example, when the first weight is determined as , the second weight can be determined as (wherein, a, b, and c are constants or a constant matrix having element values of ).

[0340] In another embodiment, when the second weight is determined, the first weight can be determined according to the determined second weight.

[0341] In an embodiment, the information indicating the weight information representation method can be information indicating whether the first weight is a single weight, a left weight and an above weight, or a weight filter. In this regard, the method of representing or determining the weight of the first weight is merely an example, and can vary within a range obvious to one skilled in the art.

[0342] ​In an embodiment, the second weight can be determined based on the first weight. For example, when the first weight is determined as the single weight 2530, the second weight can be determined as the paired single weight 2570, which is one weight corresponding to the single weight 2530. When the first weight is determined as the left weight 2532 and the upper weight 2534, the second weight can be determined as the paired left weight 2572 and the paired upper weight 2574, which respectively correspond to the left weight 2532 and the upper weight 2534. When the first weight is determined as the weight filter, the second weight can be determined as the paired weight filter.

[0343] In an embodiment, the first weight included in the weight information can be determined by using Equation 2.

[0344] [Equation 2]

[0345]

[0346] In Equation 2, may indicate the first reference template 2510, may indicate the second reference template 2550, and may indicate the current reference template 2590. may indicate the first weight, and may indicate the second weight. In addition, Equation 2 can be commonly used for the single weight 2530, the left weight 2532, and the upper weight 2534 or the weight filter.

[0347] In an embodiment, the prediction decoder 2030 can determine the single weight 2530 to be applied to the first reference block based on at least one sample of the first reference template 2510, at least one sample of the second reference template 2550, and at least one sample of the current reference template 2590.

[0348] For example, the prediction decoder 2030 can determine the single weight 2530 based on values of samples included in the first reference template 2510 and the second reference template 2550 at positions corresponding to the samples included in the current reference template 2590. The prediction decoder 2030 can determine the w value by applying values of a center upper sample 2514-1 among upper samples of the first reference template 2510, a center upper sample 2554-1 among upper samples of the second reference template 2550, and a center upper sample 2594-1 among upper samples of the current reference template 2590 to the above Equation 2. However, the positions of the samples used to determine the w value are not limited to the disclosed examples.

[0349] For example, the prediction decoder 2030 can determine a single weight 2530 based on values of samples included in the first reference template 2510 and samples included in the second reference template 2550 at positions corresponding to two or more samples included in the current reference template 2590. When the first weight is determined by using a plurality of samples included in each template, the single weight 2530 can be determined by using Equation 3.

[0350] [Equation 3]

[0351]

[0352] In Equation 3, , and may respectively indicate sample values of samples corresponding to the same positions in the current reference template 2590, the first reference template 2510, and the second reference template 2550. The single weight 2530 obtained using Equation 3 can be a value determined by performing linear regression analysis on a plurality of samples included in each template to minimize a difference between a value of the current reference template 2590 and a value obtained by combining the first reference template 2510 and the second reference template 2550. In detail, w can be a value for minimizing a function value obtained using Equation 4.

[0353] [Equation 4]

[0354]

[0355] In Equation 4, , and may respectively indicate values of samples corresponding to the same positions in the current reference template 2590, the first reference template 2510, and the second reference template 2550.

[0356] According to Equation 3, the prediction decoder 2030 can determine a w value by applying values of first samples 2512-1 and 2514-1 included in the first reference template 2510, second samples 2552-1 and 2554-1 included in the second reference template 2550, and current samples 2592-1 and 2594-1 included in the current reference template 2590 to Equation 3 above. Positions of samples used to obtain the w value are not limited to the respective positions in the respective templates, and two or more samples can be used in each template, or all samples in the templates can be used for determination.

[0357] For example, when the left-side weight 2532 and the above-side weight 2534 are determined, a single weight 2530 can be determined by using the left-side weight 2532 and the above-side weight 2534. The single weight 2530 can be determined as an average of the left-side weight 2532 and the above-side weight 2534, or can be determined as a weighted average proportional to the number of samples of the left-side weight 2532 and the number of samples of the above-side weight 2534. In an embodiment, when each template includes only the left-side samples of the block or only the above-side samples of the block, the single weight 2530 can be equal to the left-side weight 2532 or equal to the above-side weight 2534.

[0358] Hereinafter, an operation of determining the left-side weight 2532 and the above-side weight will be described.

[0359] In an embodiment, the prediction decoder 2030 can determine a weight to be applied to the first reference block as the left-side weight 2532 based on at least one sample of the first reference template 2510, at least one sample of the second reference template 2550, and at least one sample of the current reference template 2590 and a weight to be applied to the first reference block as the above-side weight 2534 . The left-side weight 2532 can be a weight to be applied to a left-lower sample region of the first reference block. The above-side weight 2534 can be a weight to be applied to a right-upper sample region of the first reference block. In order to apply a weight to a specific region, a block or a sample can indicate that a value of the weight can be used to generate a prediction block of the specific region.

[0360] In an embodiment, the left-lower sample region of the first reference block can indicate a left-lower triangular region obtained by dividing the first reference block according to a diagonal line connecting a left-upper vertex and a right-lower vertex of the first reference block. The right-upper sample region of the first reference block can indicate a right-upper triangular region obtained by dividing the first reference block according to a diagonal line connecting a left-upper vertex and a right-lower vertex of the first reference block.

[0361] In an embodiment, the left-side weight 2532 can be determined based on at least one sample among the left-side samples of the first reference template 2510, at least one sample among the left-side samples of the second reference template 2550, and at least one sample among the left-side samples of the current reference template 2590. The above-side weight 2534 can be determined based on at least one sample among the above-side samples of the first reference template 2510, at least one sample among the above-side samples of the second reference template 2550, and at least one sample among the above-side samples of the current reference template 2590.

[0362] For example, a left weight 2532 can be determined based on a sample from the left side of the first reference template 2510, a sample from the left side of the second reference template 2550, and a sample from the left side of the current reference template 2590. An upper weight 2534 can be determined based on a sample from the upper side of the first reference template 2510, a sample from the upper side of the second reference template 2550, and a sample from the upper side of the current reference template 2590. For example, this can be determined by applying the upper left sample 2512-1 from the left side of the first reference template 2510, the upper left sample 2552-1 from the left side of the second reference template 2550, and the upper left sample 2592-1 from the left side of the current reference template 2590 to Equation 2. The left-side weight 2532 is determined using the same method as described in Equation 2. This can be determined by applying the center upper sample 2514-1 from the upper sample of the first reference template 2510, the center upper sample 2554-1 from the upper sample of the second reference template 2550, and the center upper sample 2594-1 from the upper sample of the current reference template 2590 to Equation 2. The same method is used to determine the upper weight 2534. However, the positions of the samples used to determine the left weight 2532 and the positions of the samples used to determine the upper weight 2534 are not limited to the examples above.

[0363] For example, the predictive decoder 2030 can determine the left weight 2532 and the top weight 2534 based on the values ​​of the left and top samples of the first reference template 2510 at positions corresponding to two or more samples in the left samples and two or more samples in the top samples of the current reference template 2590, respectively, and the values ​​of the left and top samples of the second reference template 2550.

[0364] For example, the predictive decoder 2030 can determine the left weight 2532 based on the values ​​of multiple samples included in a row of every 2, 4, 8, or K rows in the left samples of the current reference template 2590, and the values ​​of multiple samples from the first reference template and the second reference template corresponding to the multiple samples included in a row of every 2, 4, 8, or K rows in the left samples. Similarly, the predictive decoder 2030 can determine the upper weight 2534 based on the values ​​of multiple samples included in a column of every 2, 4, 8, or K columns in the upper samples of the current reference template 2590, and the values ​​of multiple samples from the first reference template and the second reference template corresponding to the multiple samples included in a column of every 2, 4, 8, or K columns in the upper samples. K can be a preset value or can be determined based on the size of the current block.

[0365] For example, the prediction decoder 2030 can determine the left side weight 2532 by performing a linear regression analysis on a plurality of samples among the left side samples of the first reference template 2510, a plurality of samples among the left side samples of the second reference template 2550, and a plurality of samples among the left side samples of the current reference template 2590. The linear regression analysis can be performed to determine a value that minimizes a difference between a combined value of the left side samples of the first reference template 2510 and the left side samples of the second reference template 2550 and a value of the left side samples of the current reference template 2590. In this case, the value of the left side weight 2532 can be determined by using the same method as a method of determining the value of the left side weight 2532 of Equation 3. .

[0366] For example, the prediction decoder 2030 can determine the upper side weight 2534 by performing a linear regression analysis on a plurality of samples among the upper side samples of the first reference template 2510, a plurality of samples among the upper side samples of the second reference template 2550, and a plurality of samples among the upper side samples of the current reference template 2590. The linear regression analysis can be performed to determine a value that minimizes a difference between a combined value of the upper side samples of the first reference template 2510 and the upper side samples of the second reference template 2550 and a value of the upper side samples of the current reference template 2590. In this case, the value of the upper side weight 2534 can be determined by using the same method as a method of determining the value of the upper side weight 2534 of Equation 3. .

[0367] For example, the prediction decoder 2030 can determine the left side weight 2532 by performing a linear regression analysis on odd-numbered row samples among the left side samples of the first reference template 2510, odd-numbered row samples among the left side samples of the second reference template 2550, and odd-numbered row samples among the left side samples of the current reference template 2590. The positions of the samples to be used to determine the left side weight 2532 can be determined based on the size of the template, and the number or positions of the samples to be used to determine the left side weight 2532 are not limited to the above example. In one example, the number or positions of the samples to be used to determine the upper side weight 2534 can also be determined based on the size of the template.

[0368] ​​In an embodiment, the prediction decoder 2030 can obtain a left weight matrix corresponding to the left samples of the first reference template 2510 based on the left samples of the first reference template 2510, the left samples of the second reference template 2550, and the left samples of the current reference template 2590. The prediction decoder 2030 can obtain an up weight matrix corresponding to the up samples of the first reference template 2510 based on the up samples of the first reference template 2510, the up samples of the second reference template 2550, and the up samples of the current reference template 2590. For example, the prediction decoder 2030 can determine the values of all elements of the left weight matrix by applying the respective values of the left samples of the current reference template 2590, the left samples of the first reference template 2510, and the left samples of the second reference template 2550 to Equation 2. The prediction decoder 2030 can determine the values of all elements of the up weight matrix by applying the respective values of the up samples of the current reference template 2590, the up samples of the first reference template 2510, and the up samples of the second reference template 2550 to Equation 2.

[0369] Hereinafter, an operation of determining weight information by using the left weight matrix and the up weight matrix will be described.

[0370] For example, the prediction decoder 2030 can determine an average value of all or some elements of the left weight matrix as the left weight 2532. The prediction decoder 2030 can determine an average value of all or some elements of the up weight matrix as the up weight 2534. Also, the prediction decoder 2030 can determine an average value of all or some elements of the left weight matrix or the up weight matrix as a single matrix.

[0371] In an embodiment, the prediction decoder 2030 can determine a weight filter by using at least one of the left weight matrix or the up weight matrix, wherein the weight filter indicates weights corresponding to and to be applied to samples included in the first reference block, respectively.

[0372] Hereinafter, the operation of determining the weight filter based on the left weight matrix or the up weight matrix will be described with reference to Figure 26 Hereinafter, the operation of determining the weight filter based on the left weight matrix or the up weight matrix will be described with reference to

[0373] Figure 25 is a diagram for describing an operation of determining a weight filter according to an embodiment.

[0374] In an embodiment, the prediction decoder 2030 can determine weight information for a combination of the first reference block and the second reference block based on the current reference template, the first reference template, and the second reference template. For example, the prediction decoder 2030 can determine the weight information by using the left weight matrix or the up weight matrix. Figure 27 The operation of determining the weight information can include an operation of determining a first weight, and the first weight can be the weight filter 2600.

[0375] In an embodiment, the prediction decoder 2030 can obtain a left weight matrix or an above weight matrix based on the plurality of samples of the first reference template, the plurality of samples of the second reference template, and the plurality of samples of the current reference template. For example, the above weight matrix can be obtained based on the above samples of the first reference template, the above samples of the second reference template, and the above samples of the current reference template. The above weight matrix can be a matrix determined to correspond to the above samples of the first reference template. For example, when the prediction decoder 2030 identifies p x n above samples of the first reference template (where p is a constant and n is a width of the weight filter 2600), the prediction decoder 2030 can obtain the above weight matrix as an r x n matrix (where r is a natural number equal to or smaller than p).

[0376] In an embodiment, the prediction decoder 2030 can obtain a left weight matrix based on the left samples of the first reference template, the left samples of the second reference template, and the left samples of the current reference template. The left weight matrix can be a matrix determined to correspond to the left samples of the first reference template. For example, when the prediction decoder identifies m x q left samples of the first reference template (where p is a constant and m is a height of the weight filter 2600), the prediction decoder 2030 can obtain the left weight matrix as an m x s matrix (where s is a natural number equal to or smaller than q). However, the size of each template, the left weight matrix, the above weight matrix, and the weight filter is not limited to the above example.

[0377] In an embodiment, the prediction decoder 2030 can determine the weight filter 2600 to be applied to each sample included in the first reference block by using at least one of the left weight matrix or the above weight matrix. The weight filter 2600 can indicate information about weights corresponding to the samples included in the first reference block, respectively.

[0378] In an embodiment, the weight filter 2600 can be determined based on at least one column included in the left weight matrix and / or at least one row included in the above weight matrix. One sample value included in the weight filter 2600 can be determined based on at least one element included in a column of the above weight matrix corresponding to a position of the sample value and at least one element included in a row of the left weight matrix corresponding to the position of the sample value.

[0379] In an embodiment, the weight filter 2600 can be determined based on at least one column included in the left weight matrix and / or at least one row included in the above weight matrix. One sample value included in the weight filter 2600 can be determined based on at least one element included in a column of the above weight matrix corresponding to a position of the sample value and at least one element included in a row of the left weight matrix corresponding to the position of the sample value.

[0380] ​In an embodiment, the weight filter 2600 can be determined based on a column included in the left weight matrix and a row included in the above weight matrix. For example, the column included in the left weight matrix can be a column 2630 of the left weight matrix that is most adjacent to the weight filter 2600. The row included in the above weight matrix can be a column 2620 of the above weight matrix that is most adjacent to the weight filter 2600. For example, a value of a first sample 2610 can be determined based on a first left element 2631 and a first above element 2621. The first left element 2631 can be an element of the left weight matrix that is included in the column 2630 of the left weight matrix that is most adjacent to the weight filter 2600 and corresponds to a position of the first sample 2610. The first above element 2621 can be an element of the above weight matrix that is included in the column 2620 of the above weight matrix that is most adjacent to the weight filter 2600 and corresponds to the position of the first sample 2610. The value of the first sample 2610 can be determined as an average of the first left element 2631 and the first above element 2621, or can be determined by considering a ratio of a distance between the first sample 2610 and the first left element 2631 to a distance between the first sample 2610 and the first above element 2621. For example, when the distance between the first sample 2610 and the first left element 2631 is twice the distance between the first sample 2610 and the first above element 2621, the prediction decoder 2030 can determine, as the value of the first sample 2610, a value obtained by multiplying a value of the first left element 2631 by one third and a value obtained by multiplying a value of the first above element 2621 by two thirds. However, a method of determining the value of the first sample 2610 based on the first left element 2631 and the first above element 2621 is not limited to the above example. Also, values of samples in the weight filter 2600 can be determined by using the same or different method as that of determining the value of the first sample 2610.

[0381] In an embodiment, values of respective samples included in the weight filter 2600 can be determined based on at least one element included in each column of the above weight matrix and at least one element included in each row of the left weight matrix. That is, the weight filter 2600 can be determined based on at least one element included in each column of the above weight matrix and at least one element included in each row of the left weight matrix.

[0382] For example, a value of the first sample 2610 included in the weight filter 2600 can be determined as an average of a representative value of a first upper column corresponding to a column included in the upper weight matrix and a representative value of a first left column corresponding to a row included in the left weight matrix, which correspond to a position of the first sample 2610. The representative value of the first upper column can be determined based on at least one element included in the first upper column. The representative value of the first left column can be determined based on at least one element included in the first left row. The representative value of the first upper column can be an average of at least one element included in the first upper column, and the representative value of the first left column can be an average of at least one element included in the first left row. When values of all samples included in the weight filter 2600 are determined by the same method as that of determining the value of the first sample 2610, the weight filter 2600 can be determined based on an average of at least one element included in each row of the left weight matrix and an average of at least one element included in each column of the upper weight matrix. Values of samples in the weight filter 2600 can be determined by using the same or different method from that of determining the value of the first sample 2610.

[0383] For example, a value of the first sample 2610 included in the weight filter 2600 can be determined by multiplying a ratio of a distance between the first sample 2610 and the upper weight matrix and a distance between the first sample 2610 and the left weight matrix by the representative value of the first upper column and the representative value of the first left column. When a ratio of the distance between the first sample 2610 and the upper weight matrix and the distance between the first sample 2610 and the left weight matrix is 2:3, a value obtained by multiplying the representative value of the first upper column by 0.4 and a value obtained by multiplying the representative value of the first left column by 0.6 are determined as the value of the first sample 2610. A method of determining the value of the first sample 2610 based on the representative value of the first upper column and the representative value of the first left column is not limited to the above example, and values of samples in the weight filter 2600 can be determined by using the same or different method from that of determining the value of the first sample 2610.

[0384] In an embodiment, the weight filter 2600 can be determined by applying a predetermined ratio to at least one element included in each row of the left weight matrix and at least one element included in each column of the upper weight matrix. For example, a value of the first sample 2610 can be determined from a representative value of a first upper column and a representative value of a first left column.

[0385] The representative value of the first upper column can be determined from the value of at least one of a first upper element 2621 and a second upper element 2622, the first upper element 2621 being an element included in the first upper column included in the upper weight matrix. The representative value of the first upper column can be determined as the value of the first upper element 2621 or the value of the second upper element 2622, or can be determined as the average of the first upper element 2621 and the second upper element 2622. The representative value of the first upper column can be determined by applying different ratios to the first upper element 2621 and the second upper element 2622. The different ratios can be preset. A sample at a position closer to the first sample 2610 can be determined by applying a higher ratio thereto. For example, the representative value of the first upper column can be a value obtained by adding values obtained by multiplying each of the first upper element 2621 and the second upper element 2622 by 0.5, or can be a value obtained by adding values obtained by multiplying the first upper element 2621 at a position closer to the first sample 2610 by 0.7 and the second upper element 2622 by 0.3. However, the method of determining the representative value of the first upper column is not limited to the above examples.

[0386] The representative value of the first left column can be determined from the value of at least one of a first left element 2631 and a second left element 2632, the first left element 2631 being an element included in the first left column included in the left weight matrix. The representative value of the first left column can be determined as the value of the first left element 2631 or the value of the second left element 2632, or can be determined as the average of the first left element 2631 and the second left element 2632. The representative value of the first left column can be determined by applying different ratios to the first left element 2631 and the second left element 2632. The different ratios can be preset. For example, a sample at a position closer to the first sample 2610 can be determined by applying a higher ratio thereto. For example, the representative value of the first left column can be a value obtained by adding values obtained by multiplying each of the first left element 2631 and the second left element 2632 by 0.5, or can be a value obtained by adding values obtained by multiplying the first left element 2631 at a position closer to the first sample 2610 by 0.7 and the second left element 2632 by 0.3. However, the method of determining the representative value of the first left column is not limited to the above examples.

[0387] In an embodiment, when each template includes only an above sample, the weight filter 2600 can be obtained based on an above weight matrix, based on at least one element included in a column of the above weight matrix, a value of a sample in the weight filter 2600 with respect to a collocated column of the above weight matrix can be determined. For example, when each template includes only an above weight sample, the above weight matrix can be obtained. A value of the first sample 2610 can be determined based on a value of at least one of a first above element 2621 and a second above element 2622 of the above weight matrix which are located in the same column as the first sample 2610. For example, the prediction decoder 2030 can determine a value of the first above element 2621 or a value of the second above element 2622 as a value of all samples in the weight filter 2600 which are located in the same column as the first sample 2610. A value of the samples in the weight filter 2600 which are located in the same column as the first sample 2610 can be determined as an average of the first above element 2621 and the second above element 2622. A value of the samples in the weight filter 2600 which are located in the same column as the first sample 2610 can be determined by applying different ratios to the first above element 2621 and the second above element 2622.

[0388] In an embodiment, when each template includes only a left sample, the weight filter 2600 can be obtained based on a left weight matrix, based on at least one element included in a row of the left weight matrix, a value of a sample in the weight filter 2600 with respect to a collocated row of the left weight matrix can be determined. For example, when each template includes only a left weight sample, the left weight matrix can be obtained. A value of the first sample 2610 can be determined based on a value of at least one of a first left element 2631 and a second left element 2632 of the left weight matrix which are located in the same row as the first sample 2610. For example, the prediction decoder 2030 can determine a value of the first left element 2631 or a value of the second left element 2632 as a value of the samples in the weight filter 2600 which are located in the same row as the first sample 2610. A value of the samples in the weight filter 2600 which are located in the same row as the first sample 2610 can be determined as an average of the first left element 2631 and the second left element 2632. A value of the samples in the weight filter 2600 which are located in the same row as the first sample 2610 can be determined by applying different ratios to the first left element 2631 and the second left element 2632.

[0389] In an embodiment, the prediction decoder 2030 can determine a value of at least one sample included in the weight filter 2600 by using the same or different method as that of determining a value of the first sample 2610.

[0390] In an embodiment, the prediction decoder 2030 can determine values of some samples included in the weight filter 2600 by using the same or different method as that of determining the value of the first sample 2610, and then can determine values of samples whose values are not determined in the weight filter 2600 based on the determined values of the some samples.

[0391] Figure 25 is a diagram for describing an operation of generating a prediction block by using weight information according to an embodiment.

[0392] In an embodiment, when the first reference block 2710 and the second reference block 2730 are determined and the weight information is determined, the prediction decoder 2030 can generate a prediction block 2750 for the current block by combining the first reference block 2710 and the second reference block 2730 using the weight information. For example, the prediction decoder 2030 can apply the first weight determined to the first reference block 2710, can apply the second weight determined based on the first weight to the second reference block 2730, and thus, can generate the prediction block 2750. The operation of applying the second weight to the second reference block 2730 can correspond to the operation of applying the first weight to the first reference block 2710, and thus, can be skipped. Figure 26 and Figure 25 The first weight determined is applied to the first reference block 2710, the second weight determined based on the first weight is applied to the second reference block 2730, and thus, the prediction block 2750 can be generated. The operation of applying the second weight to the second reference block 2730 can correspond to the operation of applying the first weight to the first reference block 2710, and thus, can be skipped.

[0393] In an embodiment, the combining of the first reference block 2710 and the second reference block 2730 by using the weight information can be performed by using Equation 5 below.

[0394] [Equation 5]

[0395] pbSamples[x][y] = w predSamplesL0[x][y] + (M - w) predSamplesL1[x][y]

[0396] In Equation 5 above, pbSamples[x][y], predSamplesL0[x][y], and predSamplesL1[x][y] can correspond to pbSamples[x][y], predSamplesL0[x][y], and predSamplesL1[x][y] of Equation 1. Also, w and M - w can correspond to the first weight and the second weight, respectively, and can be reference to the first weight and the second weight determined in Equation 1, respectively. Figure 26 and Figure 27determined information. The M-w as the second weight can be a pair value of the w as the first weight. The pair value indicates a value obtained by applying the determined weight value to a pre-designed calculation formula. In other embodiments, the second weight can be determined as w, and the first weight as the pair value of the second weight can be determined as M-w.

[0397] In an embodiment, the prediction decoder 2030 can generate the prediction block 2750 for the current block by combining the first reference block 2710 and the second reference block 2730 using a single weight. For example, the prediction decoder 2030 can identify the single weight as a first weight to be applied to the first reference block 2710. The prediction decoder 2030 can obtain a pair single weight as a second weight to be applied to the second reference block 2730 by using the single weight. The prediction decoder 2030 can apply the first weight to the first reference block 2710 and the second weight 2740 to the second reference block 2730, and thus, can generate the prediction block 2750 for the current block.

[0398] For example, when w as the single weight is determined, the prediction decoder 2030 can identify the first weight as w, and can determine the second weight as M-w. The prediction decoder 2030 can generate the prediction block 2750 for the current block by adding a value obtained by multiplying each sample of the first reference block 2710 by w to a value obtained by multiplying each sample of the second reference block 2730 by M-w.

[0399] In an embodiment, the prediction decoder 2030 can generate the prediction block 2750 for the current block by combining the first reference block 2710 and the second reference block 2730 using the left weight 2722 and the above weight 2724. For example, the prediction decoder 2030 can identify the left weight 2722 and the above weight 2724 as first weights to be applied to the first reference block 2710. The prediction decoder 2030 can obtain a pair left weight 2742 and a pair above weight 2744 as second weights to be applied to the second reference block 2730 by using the left weight 2722 and the above weight 2724 as the first weights. The prediction decoder 2030 can apply the left weight 2722 and the above weight 2724 to the first reference block 2710, and apply the pair left weight 2742 and the pair above weight 2744 to the second reference block 2730, and thus, can generate the prediction block 2750 for the current block.

[0400] For example, when and are determined as the left weight 2722 and the above weight 2724, respectively, the prediction decoder 2030 can identify the first weights as and and the second weight can be determined as and as The prediction decoder 2030 can add values obtained by multiplying each sample of the lower-left sample region 2712 of the first reference block 2710 by and values obtained by multiplying each sample of the lower-left sample region 2732 of the second reference block 2730 by to determine values of respective samples of a lower-left sample region 2752 of the prediction block 2750 for the current block. Further, the prediction decoder 2030 can add values obtained by multiplying each sample of the upper-right sample region 2714 of the first reference block 2710 by and values obtained by multiplying each sample of the upper-right sample region 2734 of the second reference block 2730 by to determine values of respective samples of an upper-right sample region 2754 of the prediction block 2750 for the current block. The prediction decoder 2030 can generate the prediction block 2750 by using the lower-left sample region 2752 of the prediction block 2750 and the upper-right sample region 2754 of the prediction block 2750.

[0401] In the first reference block 2710, a sample 2716 on a diagonal line connecting a top-left vertex and a bottom-right vertex of the first reference block 2710 can be identified. may be applied to the sample 2716, or may be applied to the sample 2716, or and an average of and can be applied to the sample 2716. Further, for a sample 2736 on a diagonal line in the second reference block 2730, values of and and may be applied as weights, and since this corresponds to the description of the sample 2716 on the diagonal line in the first reference block 2710, a detailed description thereof is not provided here. Referring to Figure 28 , although the left weight and the upper weight are illustrated as a part of a matrix, for convenience of description, this is illustrated as a matrix corresponding to the lower-left sample region 2712 and the upper-right sample region 2714 of the first reference block 2710, and the left weight 2722 and the upper weight 2724 can be constants. In other embodiments, when the left weight 2722 and the upper weight 2724 are determined, the left weight 2722 and the upper weight 2724 can be implemented as a weight filter to be applied to the first reference block 2710.

[0402] Another embodiment using the left weight 2722 and the upper weight 2724 will be described below with reference to Figure 28 .

[0403] In an embodiment, the prediction decoder 2030 can generate the prediction block 2750 for the current block by combining the first reference block 2710 and the second reference block 2730 using the weight filter 2726. For example, the prediction decoder 2030 can identify the weight filter 2726 as a first weight to be applied to the first reference block 2710. The prediction decoder 2030 can obtain a second weight by using the weight filter 2726, where the second weight is to be applied to a pair of values of the second reference block. The prediction decoder 2030 can apply the first weight to the first reference block 2710 and the second weight to the second reference block 2730, and thus, can generate the prediction block 2750 for the current block.

[0404] For example, when the weight filter 2726 is determined as , the prediction decoder 2030 can identify the first weight as and can determine the second weight as . The prediction decoder 2030 can generate the prediction block of the current block 2750 by adding a value obtained by multiplying each sample value of the first reference block 2710 by an element value of the weight filter 2726 at a corresponding position to a value obtained by multiplying each sample value of the second reference block 2730 by an element value of the weight filter 2726 at a corresponding position. That is, the calculation of applying the weight filter 2726 as the first weight to the first reference block and the calculation of applying the weight filter 2726 as the second weight to the second reference block can be element-wise multiplication calculations.

[0405] In an embodiment, the prediction decoder 2030 can generate the prediction block 2750 for the current block by combining the first reference block 2710 and the second reference block 2730 using the weight information. The prediction decoder 2030 can determine the prediction block as the reconstructed current block. Alternatively, the prediction decoder 2030 can generate the reconstructed current block by combining the residual data and the prediction block obtained by the obtainer 2010 from the bitstream.

[0406] Figure 25 is a diagram for describing a detailed process of determining the weight information according to an embodiment.

[0407] In an embodiment, the prediction decoder 2030 can determine the weight information for the combination of the first reference block and the second reference block based on the current reference template, the first reference template, and the second reference template. The operation of determining the weight information can include an operation of determining the first weight, and the operation of determining the first weight can include an operation of determining the left weight and the upper weight or an operation of determining the weight filter. Figure 25

[0408] ​​​​​In an embodiment, the prediction decoder 2030 can determine weights to be respectively applied to samples included in the first reference block based on the left weight and the upper weight. For convenience of description, the weights to be respectively applied to samples included in the first reference block based on the left weight and the upper weight can be denoted as a weight filter, and can be referred to as a weight filter 2800. The prediction decoder 2030 can differently apply a ratio of the left weight and a ratio of the upper weight to each sample included in the weight filter 2800 based on a distance to a diagonal line connecting a left upper vertex and a right lower vertex.

[0409] For example, the prediction decoder 2030 can determine a value of at least one sample included in a first sample region 2810 as a left weight value , where the first sample region 2810 has the longest distance to a diagonal line (hereinafter, a diagonal line) connecting a left upper vertex and a right lower vertex and is located left below the diagonal line. The prediction decoder 2030 can determine a value of at least one sample included in a ninth sample region 2890 having the longest distance to the diagonal line and being located right above the diagonal line as an upper weight value . The prediction decoder 2030 can identify a second sample region 2820, a third sample region 2830, and a fourth sample region 2840 for samples located left below the diagonal line based on the distance to the diagonal line. The prediction decoder 2030 can determine a value of at least one sample included in the second sample region 2820 as . The prediction decoder 2030 can determine a value of at least one sample included in the third sample region 2830 as . The prediction decoder 2030 can determine a value of at least one sample included in the fourth sample region 2840 as . The prediction decoder 2030 can determine a value of at least one sample included in a fifth sample region 2850 including a sample located on the diagonal line as . The prediction decoder 2030 can identify a sixth sample region 2860, a seventh sample region 2870, and an eighth sample region 2880 for samples located right above the diagonal line based on the distance to the diagonal line. The prediction decoder 2030 can determine a value of at least one sample included in the sixth sample region 2860 as . The prediction decoder 2030 can determine a value of at least one sample included in the seventh sample region 2870 as . The prediction decoder 2030 can determine a value of at least one sample included in the eighth sample region 2880 as .

[0410] In an embodiment, the prediction decoder 2030 can determine weights to be respectively applied to samples included in the first reference block based on values of some samples in the weight filter. For example, a value of one sample included in the first to ninth sample regions 2810, 2820, 2830, 2840, 2850, 2860, 2870, 2880, and 2890 can be determined as a value of at least one sample included in each sample region. In a description of the second sample region as an example, the prediction decoder 2030 can refer to Figure 29 The value of the representative sample 2825, which is a sample included in the second sample region 2820, is determined by using a method of determining a value of a first sample in the weight filter, wherein the representative sample 2825 is determined as a value of a sample included in the second sample region 2820. The prediction decoder 2030 can determine a value of at least one sample included in the first to ninth sample regions 2810, 2820, 2830, 2840, 2850, 2860, 2870, 2880, and 2890 by using the same method as a method of determining a value of at least one sample included in the second sample region 2820 based on the value of the representative sample 2825.

[0411] Figure 25 is a diagram for describing a detailed process of determining a weight filter according to an embodiment.

[0412] In an embodiment, the prediction decoder 2030 can determine weight information for a combination of the first reference block and the second reference block based on the current reference template 2990, the first reference template 2910, and the second reference template 2950. The current reference template 2990, the first reference template 2910, and the second reference template 2950 can correspond to the current reference template 2590, the first reference template 2510, and the second reference template 2550 of FIG. 2, respectively. Figure 25 The current reference template 2990, the first reference template 2910, and the second reference template 2950 can correspond to the current reference template 2590, the first reference template 2510, and the second reference template 2550 of FIG. 2, respectively. Figure 25 The operation of determining the weight information can include an operation of determining the first weight. In addition, the operation of determining the first weight can include an operation of determining the left weight and the upper weight or an operation of determining the weight filter.

[0413] In an embodiment, the prediction decoder 2030 can determine a single weight based on values of samples included in the first reference template 2910 and samples included in the second reference template 2950 at positions corresponding to samples included in the current reference template 2990.

[0414] For example, the prediction decoder 2030 can determine a single weight based on values of a first above reference sample 2914-1 included in the first reference template 2910 and a second above reference sample 2954-1 included in the second reference template 2950 at a position corresponding to a first current sample 2994-1 included in the current reference template 2990. Based on Figure 25 Equation 2 of Equation 1 when it is assumed that values of are 8 (hereinafter, it is assumed that values of are all 8), a value of may be determined as 2 according to Equation 2 of Equation 1.

[0415] For example, the prediction decoder 2030 can determine a single weight by performing a linear regression analysis using values of all samples included in the first reference template 2910 and all samples included in the second reference template 2950 at positions corresponding to all samples included in the current reference template 2990. The prediction decoder 2030 can input values of the respective samples to Figure 29 Equation 3 of Equation 1, and thus, a value of may be determined as 3.94.

[0416] For example, the prediction decoder 2030 can determine a single weight by performing a linear regression analysis using values of a plurality of samples included in the first reference template 2910 (a dotted area in 2910 of Equation 1) and a plurality of samples included in the second reference template 2950 (a dotted area in 2950 of Equation 1) at left samples included in the current reference template 2990 and a plurality of samples included in odd rows (a dotted area in 2990 of Equation 1) and above samples and a plurality of samples included in odd columns (a dotted area in 2990 of Equation 1). The prediction decoder 2030 can input values of the respective samples to Figure 29 Equation 3 of Equation 1, and thus, a value of Figure 29 may be determined as 3.7. Figure 25 Figure 25 In an embodiment, the prediction decoder 2030 can determine a left weight based on at least one sample of left samples of the first reference template 2910, at least one sample of left samples of the second reference template 2950, and at least one sample of left samples of the current reference template 2990. The prediction decoder 2030 can determine an above weight based on at least one sample of above samples of the first reference template 2910, at least one sample of above samples of the second reference template 2950, and at least one sample of above samples of the current reference template 2990. In an embodiment, the prediction decoder 2030 can determine a left weight based on at least one sample of left samples of the first reference template 2910, at least one sample of left samples of the second reference template 2950, and at least one sample of left samples of the current reference template 2990. The prediction decoder 2030 can determine an above weight based on at least one sample of above samples of the first reference template 2910, at least one sample of above samples of the second reference template 2950, and at least one sample of above samples of the current reference template 2990.

[0417] In an embodiment, the prediction decoder 2030 can determine a left weight based on at least one sample of left samples of the first reference template 2910, at least one sample of left samples of the second reference template 2950, and at least one sample of left samples of the current reference template 2990. The prediction decoder 2030 can determine an above weight based on at least one sample of above samples of the first reference template 2910, at least one sample of above samples of the second reference template 2950, and at least one sample of above samples of the current reference template 2990.

[0418] ​In an embodiment, the prediction decoder 2030 can determine an upper weight based on a value of a first upper reference sample 2914-1 included in an upper sample of the first reference template 2910 and a value of a second upper reference sample 2954-1 included in an upper sample of the second reference template 2950 at a position corresponding to a first current sample 2994-1 included in the current reference template 2990. Referring to Equation 2 of Figure 25 , the value of may be determined as 2 according to the formula The prediction decoder 2030 can determine a left weight based on a value of a first left reference sample 2912-1 included in a left sample of the first reference template 2910 and a value of a second left reference sample 2952-1 included in a left sample of the second reference template 2950 at a position corresponding to a second current sample 2992-1 included in a left sample of the current reference template 2990. Referring to Equation 2 of Figure 30 , the value of may be determined as 4 according to the formula .

[0419] In an embodiment, a single weight can be determined based on the left weight and the upper weight. For example, when the value of the left weight is 4 and the value of the upper weight is 2, the value of the single weight may be determined as 3 which is an average value between the left weight and the upper weight.

[0420] In an embodiment, the prediction decoder 2030 can determine a weight filter by using at least one of the left weight matrix or the upper weight matrix, wherein the weight filter indicates weights corresponding to samples included in the first reference block and to be respectively applied to the samples included in the first reference block.

[0421] For convenience of description, an example in which a template of a current block or a reference block includes all upper samples and left samples is shown, however, the template is not limited thereto and can include only upper samples or left samples.

[0422] Hereinafter, referring to Figure 30 a detailed operation of determining the weight filter will be described.

[0423] Figure 29 is a diagram for describing a detailed process of determining weight information according to an embodiment.

[0424] In an embodiment, the prediction decoder 2030 can obtain a left weight matrix or an upper weight matrix based on a plurality of samples of the first reference template, a plurality of samples of the second reference template, and a plurality of samples of the current reference template.

[0425] ​For example, the prediction decoder 2030 can obtain a left weight matrix 3010 corresponding to the left side samples of the first reference template and an above weight matrix 3030 corresponding to the above samples of the first reference template by using the current reference template 2990, the first reference template 2910, and the second reference template 2950 of the particular example. Figure 30

[0426] In an embodiment, the prediction decoder 2030 can determine a weight filter 3050 to be applied to each sample included in the first reference block by using at least one of the left weight matrix 3010 or the above weight matrix 3030. The weight filter 3050 can be determined based on at least one column included in the left weight matrix and / or at least one row included in the above matrix column. One sample value included in the weight filter 3050 can be determined based on at least one element included in a column of the above weight matrix corresponding to the position of the sample value and at least one element included in a row of the left weight matrix corresponding to the position of the sample value.

[0427] In an embodiment, the weight filter 3050 can be determined based on a left weight matrix column 3015 of the left weight matrix 3010 most adjacent to the weight filter 3050 and an above weight matrix row 3035 of the above weight matrix 3030 most adjacent to the weight filter 3050. For example, a first sample 3051 of one sample included in the weight filter 3050 can be determined as an average of a first left element 3016 corresponding to the position of the first sample 3051 and included in the left weight matrix column 3015 of the left weight matrix 3010 most adjacent to the weight filter 3050 and a first above element 3036 corresponding to the position of the first sample 3051 and included in the above weight matrix row 3035 of the above weight matrix 3030 most adjacent to the weight filter 3050. In this case, the value of the first sample 3051 can be determined as 5.2. The values of all samples included in the weight filter 3050 can be determined by using the same method as that of determining the value of the first sample 3051. The prediction decoder 2030 can determine the values of all samples, and thus, the weight filter 3050 indicating the weights corresponding to the samples included in the first reference block, respectively, can be determined.

[0428] Figure 31 The illustrated weight filter is an example of the weight filter, and is not limited to the example.

[0429] Figure 32 is a flowchart of an image decoding method according to an embodiment.

[0430] ​The image decoding apparatus 2000 can determine a first reference block in a first reference picture and a second reference block in a second reference picture for bi-prediction of the current block, by using the decoded motion information.

[0431] In an embodiment, the image decoding apparatus 2000 can determine a first reference block in a first reference picture and a second reference block in a second reference picture for bi-prediction of the current block, by using the decoded motion information.

[0432] In an embodiment, the image decoding apparatus 2000 can determine at least one of the first reference block or the second reference block, by using the motion information or the reference block of the neighboring block. For example, the image decoding apparatus 2000 can determine a motion vector by using the motion information or the reference block of the neighboring block, and can determine a block indicated by the determined motion vector as at least one of the first reference block or the second reference block.

[0433] In an embodiment, the image decoding apparatus 2000 can correct the motion vector by comparing a current reference template with a template in a preset area including the block indicated by the determined motion vector, and thus, can determine at least one of the first reference block or the second reference block of the current block. For example, the image decoding apparatus 2000 can determine at least one of the first reference block in the first reference picture or the second reference block in the second reference picture, based on template matching. The first reference picture and the second reference picture can be pictures reconstructed before the current picture. In the template matching, a neighboring sample set reconstructed before the current block in the current picture can be used as a template, and a template of the current picture can be referred to as a current reference template.

[0434] In an embodiment, a template most similar to the current reference template can be searched for in the first reference picture and the second reference picture. A search range can be a preset range around a point indicated by a first motion vector and a second motion vector, which are basic motion vectors used to determine the reference blocks in the first reference picture and the second reference picture, respectively. In the search range of the first reference picture, a template most similar to the current reference template can be determined as the first reference block, and in the search range of the second reference picture, a template most similar to the current reference template can be determined as the second reference block.

[0435] In operation S3120, the image decoding apparatus 2000 can determine weight information for a combination of the first reference block and the second reference block, based on a current reference template of the current block, a first reference template of the first reference block, and a second reference template of the second reference block.

[0436] In an embodiment, the weight information can be information for a combination of the first reference block 2510 and the second reference block 2550 based on the current reference template 2590, the first reference template 2510, and the second reference template 2550.

[0437] In an embodiment, the weight information can include information on a weight for a linear combination of the first reference block and the second reference block. For example, the weight information can include information on a first weight to be applied to the first reference block and a second weight to be applied to the second reference block. When the first weight is determined, the second weight can be determined according to the determined first weight. In other embodiments, when the second weight is determined, the first weight can be determined according to the determined second weight.

[0438] In an embodiment, the first weight can be a single weight to be commonly applied to samples included in the first reference block. The first weight can be determined based on at least one sample of the first reference template, at least one sample of the second reference template, and at least one sample of the current reference template. For example, the single weight can be determined by performing a linear regression analysis on the at least one sample of the first reference template, the at least one sample of the second reference template, and the at least one sample of the current reference template to minimize a difference between a value obtained by combining the first reference template and the second reference template and a value of the current reference template. The single weight can be determined based on a left weight and an above weight.

[0439] In an embodiment, the first weight can correspond to a left weight and an above weight, the left weight being obtained based on at least one left sample of each template and the above weight being obtained based on at least one above sample of each template. The left weight can be a weight to be applied to a lower left sample region of the first reference block, and the above weight can be a weight to be applied to an upper right sample region of the first reference block. The left weight and the above weight can be determined based on at least one sample of the first reference template, at least one sample of the second reference template, and at least one sample of the current reference template.

[0440] For example, the left-side weight can be determined based on one of the left-side samples of the first reference template, one of the left-side samples of the second reference template, and one of the left-side samples of the current reference template. The left-side weight can be determined by performing a linear regression analysis with respect to a plurality of samples among the left-side samples of the first reference template, a plurality of samples among the left-side samples of the second reference template, and a plurality of samples among the left-side samples of the current reference template to minimize a difference between a value obtained by combining the left-side samples of the first reference template and the left-side samples of the second reference template and a value of the left-side samples of the current reference template. The left-side weight can be determined as an average of elements of a left-side weight matrix obtained based on the left-side samples of the first reference template, the left-side samples of the second reference template, and the left-side samples of the current reference template and corresponding to the left-side samples of the first reference template.

[0441] For example, the upper-side weight can be determined based on one of the upper-side samples of the first reference template, one of the upper-side samples of the second reference template, and one of the upper-side samples of the current reference template. The upper-side weight can be determined by performing a linear regression analysis with respect to a plurality of samples among the upper-side samples of the first reference template, a plurality of samples among the upper-side samples of the second reference template, and a plurality of samples among the upper-side samples of the current reference template to minimize a difference between a value obtained by combining the upper-side samples of the first reference template and the upper-side samples of the second reference template and a value of the upper-side samples of the current reference template. The upper-side weight can be determined as an average of elements of an upper-side weight matrix obtained based on the upper-side samples of the first reference template, the upper-side samples of the second reference template, and the upper-side samples of the current reference template and corresponding to the upper-side samples of the first reference template.

[0442] In an embodiment, the first weight can be a weight filter which can be expressed as a matrix which is applied differently according to samples included in the first reference block. The weight filter can indicate weights corresponding to the samples included in the first reference block and to be applied to the samples included in the first reference block, respectively, by using at least one of the left-side weight matrix or the upper-side weight matrix.

[0443] For example, the weight filter can be determined based on a column included in the left-side weight matrix and a row included in the upper-side weight matrix. The weight filter can be determined based on an average of at least one element included in each row of the left-side weight matrix and an average of at least one element included in each column of the upper-side weight matrix. The weight filter can be determined by applying a preset ratio to at least one element included in each row of the left-side weight matrix and at least one element included in each column of the upper-side weight matrix.

[0444] The image decoding apparatus 2000 can generate a prediction block of the current block by combining the first reference block and the second reference block using the weight information, in operation S3130.

[0445] In an embodiment, the weight information can be used in the combination of the first reference block and the second reference block. For example, the first reference block and the second reference block can be combined in a manner that the first weight determined in operation S3120 is applied to the first reference block and the second weight determined based on the first weight is applied to the second reference block.

[0446] In an embodiment, when a single weight is identified as the first weight, the first reference block and the second reference block can be combined by adding a value obtained by multiplying each sample of the first reference block by a value of the determined single weight to a value obtained by multiplying each sample of the second reference block by a value of the paired single weight.

[0447] In an embodiment, a left weight and an above weight can be identified as the first weight. In this case, the first reference block and the second reference block can be combined by adding a value obtained by multiplying samples included in a lower left sample region of the first reference block by the left weight to a value obtained by multiplying samples included in a lower left sample region of the second reference block by the paired left weight, and by adding a value obtained by multiplying samples included in an upper right sample region of the first reference block by the above weight to a value obtained by multiplying samples included in an upper right sample region of the second reference block by the paired above weight.

[0448] In an embodiment, when a weight filter is identified as the first weight, the first reference block and the second reference block can be combined by adding a value obtained by multiplying each sample value of the first reference block by an element of the weight filter at a corresponding position to a value obtained by multiplying each sample value of the second reference block by an element of the paired weight filter at the corresponding position.

[0449] In an embodiment, a prediction block of the current block can be generated by combining the first reference block and the second reference block using the weight information.

[0450] In operation S3140, the image decoding apparatus 2000 can reconstruct the current block by using the prediction block.

[0451] In an embodiment, the image decoding apparatus 2000 can determine the prediction block as the reconstructed current block.

[0452] In an embodiment, the image decoding apparatus 2000 can generate the reconstructed current block by combining residual data obtained from the bitstream with the prediction block.

[0453] Figure 32 FIG. 1 is a block diagram illustrating a configuration of an image encoding apparatus according to an embodiment.

[0454] Referring to Figure 19 , the image encoding apparatus 3200 can include a prediction encoder 3210 and a generator 3230.

[0455] According to an embodiment, the prediction encoder 3210 and the generator 3230 can be implemented as at least one processor. In an embodiment, the prediction encoder 3210 and the generator 3230 can operate according to at least one instruction stored in at least one memory.

[0456] The image encoding apparatus 3200 can include at least one memory that stores input / output data of the prediction encoder 3210 and the generator 3230. Furthermore, the image encoding apparatus 3200 can include a memory controller configured to control data input / output of the memory.

[0457] In an embodiment, the prediction encoder 3210 can correspond to the prediction encoder 1915 shown in FIG. 19, and the generator 3230 can correspond to the entropy encoder 1925 shown in FIG. 19. Figure 19 Figure 21 In an embodiment, the prediction encoder 3210 can determine a prediction mode of a current block in a current picture. The prediction mode of the current block can include an inter mode. The inter mode refers to a mode in which the current block is predicted or reconstructed based on a reference picture in order to reduce temporal redundancy between pictures. The current block can be a largest coding unit, a coding unit, a transform unit, or a prediction unit, which is divided from the current picture to be encoded.

[0458] In an embodiment, the prediction mode of the current block can be determined as the inter mode. The prediction encoder 3210 can perform inter prediction with respect to the current block according to the prediction mode of the current block, and can encode the current block by using a prediction block generated as a result of performing the inter prediction.

[0459] In an embodiment, the prediction encoder 3210 can use one reference picture (e.g., uni-prediction) or two reference pictures (e.g., bi-prediction) in order to encode the current block based on the reference picture. Whether the current block is to be uni-predicted or bi-predicted can be included in a bitstream as a flag or an index. When it is implicitly determined that the current block is to be uni-predicted or bi-predicted according to a prediction mode of a neighboring block related to the current block, information about the neighboring block can be included in the bitstream as the flag or the index.

[0460] In an embodiment, the prediction encoder 3210 can use one reference picture (e.g., uni-prediction) or two reference pictures (e.g., bi-prediction) in order to encode the current block based on the reference picture. Whether the current block is to be uni-predicted or bi-predicted can be included in a bitstream as a flag or an index. When it is implicitly determined that the current block is to be uni-predicted or bi-predicted according to a prediction mode of a neighboring block related to the current block, information about the neighboring block can be included in the bitstream as the flag or the index.

[0461] ​In an embodiment, when the current block is bi-predicted, the prediction encoder 3210 can identify motion information for determining a reference block of the current block. The motion information of the current block can be included in the bitstream. The motion information of the current block can include at least one of a reference picture index, a motion vector, a differential motion vector, or a reference direction, and can include all information for a motion vector used for predicting the current block.

[0462] In an embodiment, the prediction encoder 3210 can use a motion vector of a block included in a previously decoded block adjacent to the current block or a block included in a previously decoded picture as a motion vector predictor of the current block, or can use a motion vector difference that is a difference between a motion vector of the current block and the motion vector predictor, and thus can determine a first motion vector and a second motion vector. In an embodiment, when the current block is bi-predicted, the prediction encoder 3210 can determine a first reference block in a first reference picture and a second reference block in a second reference picture for bi-prediction of the current block. For example, the prediction encoder 3210 can determine the first reference block and the second reference block for bi-prediction of the current block by using the motion information. Alternatively, the prediction encoder 3210 can determine the reference block for the current block based on template matching, wherein the reference block for the current block is the first reference block and the second reference block for bi-prediction of the current block.

[0463] In an embodiment, the prediction encoder 3210 can determine a motion vector by using the motion information or a reference block encoded before the current block, and can determine a block indicated by the determined motion vector as at least one of the first reference block or the second reference block.

[0464] In an embodiment, the prediction encoder 3210 can correct a motion vector by comparing a current reference template with a template in a preset area including a block indicated by the determined motion vector, and thus can determine at least one of the first reference block or the second reference block of the current block. The current reference template can indicate a template of the current block. That is, the reference block can be determined based on only the motion information, or the reference block can be determined based on the motion information and template matching. When the prediction encoder 3210 is capable of defining templates of the first reference block and the second reference block, the prediction decoder 2030 can determine the reference block regardless of whether the motion vector is corrected by performing the template matching.

[0465] In an embodiment, the prediction encoder 3210 can determine weight information for combining the reference blocks to generate a prediction block of the current block based on a template of the reference block and a current reference template.

[0466] In an embodiment, the weight candidate list usage information is information indicating a method of determining weight information. The weight information can be obtained based on a weight candidate value determined according to a weight index in the weight candidate list, or the weight information can be determined by using motion information or a reference block of a neighboring block, independently of the weight candidate list or the weight index.

[0467] In an embodiment, the information indicating the weight information representation method can be included in a sequence parameter set, a picture parameter set, a slice header, or slice data of a bitstream.

[0468] In an embodiment, the prediction encoder 3210 can obtain a weight candidate list and a weight index according to a predetermined method, or can determine weight information by using motion information or a reference block of a neighboring block, without obtaining the weight candidate list and the weight index. In this case, the weight candidate list usage information can not be included in the bitstream.

[0469] In an embodiment, the prediction encoder 3210 can determine weight candidate list usage information indicating whether to use a weight candidate list. The weight candidate list usage information can be included in a bitstream. The weight candidate list usage information can include a flag or an index indicating whether to use the weight candidate list or not to use the weight candidate list.

[0470] In an embodiment, the prediction encoder 3210 can determine whether to use a weight candidate list by using weight candidate list usage information derived from one of reference blocks. The prediction encoder can determine a motion vector indicating the reference block used to derive the weight candidate list usage information, in order to determine whether to use the weight candidate list.

[0471] In an embodiment, the information indicating the weight information representation method can be included in a sequence parameter set, a picture parameter set, a slice header, or slice data of a bitstream.

[0472] In an embodiment, the prediction encoder 3210 can determine weight information of a current block according to a predetermined method. In this case, the information indicating the weight information representation method can not be included in a bitstream.

[0473] In an embodiment, when a weight information representation method is selected to determine weight information, the prediction encoder 3210 can select a weight information representation method that results in a minimum cost when encoding a current block, among a plurality of weight information representation methods. Information indicating the selected weight information representation method can include a flag or an index indicating the selected weight information representation method among the plurality of weight information representation methods.

[0474] In an embodiment, when the weight information representation method is a method to be derived from a reference block, the prediction encoder 3210 can determine at least one of a reference block that is coded before the current block in the current picture or a reference block in a previous reference picture that is coded before the current block. The prediction encoder 3210 can determine the weight information representation method derived from the at least one of the reference blocks as the weight information representation method of the current block. The prediction encoder 3210 can determine motion information indicating the reference block.

[0475] In an embodiment, the encoding of the current block can indicate a process of generating information for allowing the picture decoding apparatus to reconstruct the current block. The information generated via the encoding can be included in the bitstream.

[0476] In an embodiment, when the prediction block of the current block is generated via bi-prediction, the prediction encoder 3210 can encode the current block by using the prediction block.

[0477] In an embodiment, the prediction encoder 3210 can generate residual data corresponding to a difference between the prediction block and the current block. When the prediction block is determined as the current block, the residual data can not be generated.

[0478] The generator 3230 can generate a bitstream including a picture encoding result. The bitstream can include an encoding result regarding the current block.

[0479] In an embodiment, the generator 3230 can transmit the bitstream to the picture decoding apparatus 2000 via a network.

[0480] In an embodiment, the generator 3230 can store the bitstream in a data storage medium including a magnetic medium such as a hard disk, a floppy disk, and a magnetic tape, an optical recording medium such as a CD-ROM and a DVD, a magneto-optical medium such as a floptical disk, or the like.

[0481] The generator 3230 can generate a bitstream including syntax elements generated by encoding a picture. According to a hierarchical structure of the picture, values corresponding to the syntax elements can be included in the bitstream.

[0482] A bit string generated when the generator 3230 entropy-encodes the syntax elements can be included in the bitstream.

[0483] In an embodiment, the bitstream can include motion information of the current block in the current picture.

[0484] In an embodiment, the bitstream can include at least one of information indicating whether the current block is to be bi-predicted, motion information for determining a reference block, weight candidate list usage information, or information indicating a weight information representation method.

[0485] In the following, the process in which the prediction encoder 3210 determines the first reference block in the first reference picture and the second reference block in the second reference picture will now be described.

[0486] In an embodiment, the prediction encoder 3210 can determine at least one of the first reference block of the first reference picture or the second reference block of the second reference picture based on the motion information or the motion information and the template matching. In an embodiment, the first reference picture 2420 and the second reference picture 2440 can be pictures that are encoded before the current picture.

[0487] In an embodiment, the prediction encoder 3210 can determine the first reference block and the second reference block of the current block based on the motion information. Referring to FIG. 24, the prediction encoder 3210 can determine the first reference block 2425 and the second reference block 2435 of the current block 2410 based on the first motion vector 2422 and the second motion vector 2432. Figure 21 The prediction encoder 3210 can determine the first reference picture 2120 and the second reference picture 2130 to be referred to by the current block for bi-prediction of the current block, and can determine the first motion vector 2122 indicating the first reference block 2124 in the first reference picture 2120 and the second motion vector 2132 indicating the second reference block 2134 in the second reference picture 2130. Accordingly, the prediction encoder 3210 can determine the first reference block 2124 based on the first motion vector 2122, and can determine the second reference block 2134 based on the second motion vector 2124.

[0488] In the template matching, a neighboring sample set that is encoded before the current block can be used as a template.

[0489] In an embodiment, the prediction encoder 3210 can determine at least one of the first reference block or the second reference block for the current block based on the motion information and the template matching. In an embodiment, the prediction encoder 3210 can search for a template most similar to the current reference template 2415 in the first reference picture 2420 by using the current reference template 2415, and can determine a block neighboring the first reference template 2430 as the first reference block 2435 as a result of the search. When the current reference template 2415 is located at the upper left of the current block 2410, the prediction encoder 3210 can determine a block located at the lower right of the first reference template 2430 as the first reference block 2435 as the most similar template in the first reference picture 2420 as a result of the search.

[0490] In an embodiment, the prediction encoder 3210 can search for a template most similar to the current reference template 2415 in the second reference image 2440 by using the current reference template 2415, and can determine a block adjacent to a second reference template 2450 that is the most similar template as a second reference block 2455 according to a result of the search. When the current reference template 2415 is located at the upper left of the current block 2410, the prediction encoder 3210 can determine a block located at the lower right of the second reference template 2450 that is the most similar template in the second reference image 2440 as the second reference block 2455 according to a result of the search.

[0491] In an embodiment, in order to search for a template similar to the current reference template 2415, a difference between sample values can be used. For example, a similar template including a sample value most similar to a sample value included in the current reference template 2415 can be determined within the first reference image 2420 and / or the second reference image 2440.

[0492] In an embodiment, a template having a minimum sum of absolute difference (SAD) value can be determined as the most similar template by using a difference between sample values, however, this is merely an example, and thus, a sum of absolute transformed difference (SATD) or a histogram of oriented gradients (HoG) can be used, and this can vary within a range apparent to those skilled in the art.

[0493] In an embodiment, the prediction encoder 3210 can determine a base motion vector for determining each of the reference blocks in order to perform a search for a similar template. For example, a predetermined motion vector (e.g., a zero vector) or a motion vector of a block at a predetermined position can be determined as the base motion vector. For example, a base motion vector for determining the first reference block 2435 can be the first motion vector 2122 of Figure 21 , and a base motion vector for determining the second reference block 2455 can be the second motion vector 2132 of Figure 33 .

[0494] In an embodiment, the prediction encoder 3210 can search for a template most similar to the current reference template 2415 within a preset range around a point indicated by the base motion vector. Further, a block adjacent to the first reference template 2430 in the first reference image 2420 can be determined as the first reference block 2435, in which the first reference template 2430 is a template most similar to the current reference template 2415. The prediction encoder 3210 can determine a block adjacent to the second reference template 2450 in the second reference image 2440 as the second reference block 2455, in which the second reference template 2450 is a template most similar to the current reference template 2415.

[0495] When at least one of the first reference block 2435 or the second reference block 2455 is determined by the prediction encoder 3210 via template matching, the prediction decoder 2030 described above can also determine at least one of the first reference block 2435 or the second reference block 2455 via template matching.

[0496] In an embodiment, the prediction encoder 3210 can determine a first motion vector 2122 indicating the first reference block 2435 and a second motion vector 2132 indicating the second reference block 2455. Information indicating the first motion vector 2122 and the second motion vector 2132 can be included in the bitstream.

[0497] In an embodiment, the information indicating the first motion vector 2122 can include information indicating any one of candidates of the first motion vector 2122. The information indicating the first motion vector 2122 can include a motion vector difference between a candidate of the first motion vector 2122 selected from the candidates of the first motion vector 2122 and the first motion vector 2122 indicating the first reference block 2435.

[0498] In an embodiment, the information indicating the second motion vector 2132 can include information indicating any one of candidates of the second motion vector 2132. The information indicating the second motion vector 2132 can include a motion vector difference between a candidate of the second motion vector 2132 selected from the candidates of the second motion vector 2132 and the second motion vector 2132 indicating the second reference block 2455.

[0499] In an embodiment, the prediction encoder 3210 can determine the first reference block 2435 and / or the second reference block 2455 indicated by a predetermined motion vector in a previous picture. Here, the predetermined first motion vector 2122 and the predetermined second motion vector 2132 can each indicate a motion vector agreed upon in advance between the image encoding apparatus 3200 and the image decoding apparatus 2000.

[0500] Hereinafter, a process in which the prediction encoder 3210 determines weight information based on a current reference template, a first reference template, and a second reference template will now be described.

[0501] In an embodiment, the prediction encoder 3210 can determine weight information for a combination of the first reference template 2510 and the second reference template 2550 based on the current reference template 2590, the first reference template 2510, and the second reference template 2550. The prediction encoder 3210 can determine the weight information by using the current reference template 2590, the first reference template 2510, and the second reference template 2550 without using a weight candidate list and a weight index.

[0502] In an embodiment, the weight information can include information on weights for a linear combination of the first reference block and the second reference block. For example, the weight information can include a weight to be applied to the first reference block and a weight to be applied to the second reference block.

[0503] In an embodiment, the operation of determining the weight information can include an operation of determining a first weight. Also, the first weight can be determined as one of a single weight 2530, a left weight 2532 and an above weight 2534, or a weight filter. The information indicating the weight information representation method can be information indicating that the operation of determining the first weight is an operation of determining the single weight 2530, an operation of determining the left weight 2532 and the above weight 2534, or an operation of determining the weight filter.

[0504] In an embodiment, the first weight, which is a weight to be applied to the first reference block, can be at least one of a constant or a matrix. For example, the first weight can be one weight to be commonly applied to the first reference block 2510, and in this case, the first weight can be a single weight 2530 having a value of The first weight can include a left weight 2532 having a value obtained based on at least one left sample of each template and an above weight 2534 having a value obtained based on at least one above sample of each template. The first weight can include a left weight 2532 having a value obtained based on at least one left sample of each template and an above weight 2534 having a value obtained based on at least one above sample of each template. The first weight can include a left weight 2532 having a value obtained based on at least one left sample of each template and an above weight 2534 having a value obtained based on at least one above sample of each template. The first weight can include a left weight 2532 having a value obtained based on at least one left sample of each template and an above weight 2534 having a value obtained based on at least one above sample of each template.

[0505] In an embodiment, the prediction encoder 3210 can determine a single weight 2530 to be applied to the first reference block based on at least one sample of the first reference template 2510, at least one sample of the second reference template 2550, and at least one sample of the current reference template 2590.

[0506] In an embodiment, the prediction decoder 2030 can determine a weight, which is a left weight 2532, to be applied to the first reference block based on at least one sample of the first reference template 2510, at least one sample of the second reference template 2550, and at least one sample of the current reference template 2590 and a weight, which is an above weight 2534, to be applied to the first reference block The left weight 2532 can be a weight to be applied to a left lower sample region of the first reference block. The above weight 2534 can be a weight to be applied to a right upper sample region of the first reference block.

[0507] In an embodiment, the lower-left sample region of the first reference block can indicate a lower-left triangular region obtained by dividing the first reference block according to a diagonal line connecting a top-left vertex and a bottom-right vertex of the first reference block. The upper-right sample region of the first reference block can indicate an upper-right triangular region obtained by dividing the first reference block according to a diagonal line connecting a top-left vertex and a bottom-right vertex of the first reference block.

[0508] In an embodiment, the left-side weight 2532 can be determined based on at least one of the left-side samples of the first reference template 2510, at least one of the left-side samples of the second reference template 2550, and at least one of the left-side samples of the current reference template 2590. The upper-side weight 2534 can be determined based on at least one of the upper-side samples of the first reference template 2510, at least one of the upper-side samples of the second reference template 2550, and at least one of the upper-side samples of the current reference template 2590.

[0509] In an embodiment, the prediction decoder 2030 can obtain a left-side weight matrix corresponding to the left-side samples of the first reference template 2510 based on the left-side samples of the first reference template 2510, the left-side samples of the second reference template 2550, and the left-side samples of the current reference template 2590. The prediction encoder 3210 can obtain an upper-side weight matrix corresponding to the upper-side samples of the first reference template 2510 based on the upper-side samples of the first reference template 2510, the upper-side samples of the second reference template 2550, and the upper-side samples of the current reference template 2590. For example, the prediction encoder 3210 can determine values of all elements of the left-side weight matrix by applying respective values of the left-side samples of the current reference template 2590, the first reference template 2510, and the second reference template 2550 to Equation 2. The prediction encoder 3210 can determine values of all elements of the upper-side weight matrix by applying respective values of the upper-side samples of the current reference template 2590, the first reference template 2510, and the second reference template 2550 to Equation 2.

[0510] In an embodiment, the prediction encoder 3210 can determine a weight filter 2600 to be applied to each sample included in the first reference block by using at least one of the left-side weight matrix or the upper-side weight matrix. The weight filter 2600 can indicate weights corresponding to the samples included in the first reference block, respectively.

[0511] In an embodiment, the weight filter 2600 can be determined based on at least one column included in the left weight matrix and / or at least one row included in the top weight matrix. One sample value included in the weight filter 2600 can be determined based on at least one element included in a column of the top weight matrix corresponding to a position of the sample value and at least one element included in a row of the left weight matrix corresponding to the position of the sample value.

[0512] In an embodiment, values of respective samples included in the weight filter 2600 can be determined based on at least one element included in each column of the top weight matrix and at least one element included in each row of the left weight matrix. That is, the weight filter 2600 can be determined based on at least one element included in each column of the top weight matrix and at least one element included in each row of the left weight matrix.

[0513] In an embodiment, the operation of the prediction encoder 3210 of the image encoding apparatus 3200 can be the same as that of the prediction decoder 2030 of the image decoding apparatus 2000, and thus the description provided for the operation of the prediction decoder 2030 can be equally applied to the prediction encoder 3210.

[0514] ​ is a flowchart of an image encoding method according to an embodiment.

[0515] In operation S3310, the image encoding apparatus 3200 can determine a first reference block in a first reference picture and a second reference block in a second reference picture for bi-prediction of a current block.

[0516] In an embodiment, the image encoding apparatus 3200 can determine the first reference block in the first reference picture and the second reference block in the second reference picture for bi-prediction of the current block by using decoded motion information. For example, the image encoding apparatus 3200 can determine the first reference block and the second reference block for bi-prediction of the current block based on motion estimation or template matching.

[0517] In an embodiment, the image encoding apparatus 3200 can determine at least one of the first reference block or the second reference block by using motion information or a reference block of a neighboring block. For example, the image encoding apparatus 3200 can determine a motion vector by using motion information or a reference block of a neighboring block, and can determine a block indicated by the determined motion vector as at least one of the first reference block or the second reference block.

[0518] In an embodiment, the image encoding apparatus 3200 can correct the motion vectors by comparing the current reference template with templates in a preset area including the blocks indicated by the determined motion vectors, and thus, can determine at least one of the first reference block or the second reference block of the current block. For example, the image encoding apparatus 3200 can determine at least one of the first reference block in the first reference image or the second reference block in the second reference image based on template matching. The first reference image and the second reference image can be images reconstructed before the current image. In the template matching, a neighboring sample set reconstructed before the current block in the current image can be used as a template, and the template of the current image can be referred to as a current reference template.

[0519] In an embodiment, a template most similar to the current reference template can be searched for in the first reference image and the second reference image. A search range can be a preset range around a point indicated by the first motion vector and the second motion vector, wherein the first motion vector and the second motion vector are base motion vectors used to determine the reference blocks in the first reference image and the second reference image, respectively. In the search range of the first reference image, the template most similar to the current reference template can be determined as the first reference block, and in the search range of the second reference image, the template most similar to the current reference template can be determined as the second reference block.

[0520] In operation S3320, the image encoding apparatus 3200 can determine weight information for a combination of the first reference block and the second reference block based on the current reference template of the current block, the first reference template of the first reference block, and the second reference template of the second reference block.

[0521] In an embodiment, the weight information can be information for a combination of the first reference block 2510 and the second reference block 2550 based on the current reference template 2590, the first reference template 2510, and the second reference template 2550.

[0522] In an embodiment, the weight information can include information on a weight for a linear combination of the first reference block and the second reference block. For example, the weight information can include information on a first weight to be applied to the first reference block and a second weight to be applied to the second reference block. When the first weight is determined, the second weight can be determined according to the determined first weight. In other embodiments, when the second weight is determined, the first weight can be determined according to the determined first weight.

[0523] In an embodiment, the first weight can be a single weight to be commonly applied to samples included in the first reference block. The first weight can be determined based on at least one sample of the first reference template, at least one sample of the second reference template, and at least one sample of the current reference template. For example, the single weight can be determined by performing a linear regression analysis to minimize a difference between a value obtained by combining the first reference template and the second reference template and a value of the current reference template based on at least one sample of the first reference template, at least one sample of the second reference template, and at least one sample of the current reference template.

[0524] In an embodiment, the first weight can correspond to a left weight and an above weight, the left weight being obtained based on at least one left sample of each template and the above weight being obtained based on at least one above sample of each template. The left weight can be a weight to be applied to a lower left sample region of the first reference block, and the above weight can be a weight to be applied to an upper right sample region of the first reference block. The left weight and the above weight can be determined based on at least one sample of the first reference template, at least one sample of the second reference template, and at least one sample of the current reference template.

[0525] For example, the left weight can be determined based on one sample among left samples of the first reference template, one sample among left samples of the second reference template, and one sample among left samples of the current reference template. The left weight can be determined by performing a linear regression analysis to minimize a difference between a value obtained by combining the left samples of the first reference template and the left samples of the second reference template and a value of the left samples of the current reference template based on a plurality of samples among the left samples of the first reference template, a plurality of samples among the left samples of the second reference template, and a plurality of samples among the left samples of the current reference template. The left weight can be determined as an average of elements of a left weight matrix obtained based on the left samples of the first reference template, the left samples of the second reference template, and the left samples of the current reference template and corresponding to the left samples of the first reference template.

[0526] For example, the top weight can be determined based on one of the top samples of the first reference template, one of the top samples of the second reference template, and one of the top samples of the current reference template. The top weight can be determined by performing a linear regression analysis on a plurality of samples among the top samples of the first reference template, a plurality of samples among the top samples of the second reference template, and a plurality of samples among the top samples of the current reference template to minimize a difference between a value obtained by combining the top samples of the first reference template and the top samples of the second reference template and a value of the top sample of the current reference template. The top weight can be determined as an average of elements of a top weight matrix obtained based on the top samples of the first reference template, the top samples of the second reference template, and the top samples of the current reference template and corresponding to the top sample of the first reference template.

[0527] In an embodiment, the first weight can be a weight filter which can be expressed as a matrix which is differently applied according to samples included in the first reference block. The weight filter can indicate weights corresponding to the samples included in the first reference block and to be applied to the samples included in the first reference block, respectively, by using at least one of the left weight matrix or the top weight matrix.

[0528] For example, the weight filter can be determined based on a column included in the left weight matrix and a row included in the top weight matrix. The weight filter can be determined based on an average of at least one element included in each row of the left weight matrix and an average of at least one element included in each column of the top weight matrix. The weight filter can be determined by applying a preset ratio to at least one element included in each row of the left weight matrix and at least one element included in each column of the top weight matrix.

[0529] In operation S3330, the image encoding apparatus 3200 can generate a prediction block of the current block by combining the first reference block and the second reference block using the weight information.

[0530] In an embodiment, the weight information can be used in the combination of the first reference block and the second reference block. For example, the first reference block and the second reference block can be combined in a manner that the first weight determined in operation S3120 is applied to the first reference block and the second weight determined based on the first weight is applied to the second reference block.

[0531] In an embodiment, when a single weight is identified as the first weight, the first reference block and the second reference block can be combined by adding a value obtained by multiplying each sample of the first reference block by a value of the determined single weight to a value obtained by multiplying each sample of the second reference block by a value of the determined pair of single weights.

[0532] In an embodiment, the left weight and the above weight can be identified as the first weight. In this case, the first reference block and the second reference block can be combined by adding a value obtained by multiplying samples included in a lower-left sample region of the first reference block by the left weight and a value obtained by multiplying samples included in a lower-left sample region of the second reference block by the paired left weight, and by adding a value obtained by multiplying samples included in an upper-right sample region of the first reference block by the above weight and a value obtained by multiplying samples included in an upper-right sample region of the second reference block by the paired above weight.

[0533] In an embodiment, when the weight filter is identified as the first weight, the first reference block and the second reference block can be combined by adding a value obtained by multiplying each sample value of the first reference block by an element of the weight filter at a corresponding position and a value obtained by multiplying each sample value of the second reference block by an element of the paired weight filter at the corresponding position.

[0534] The image encoding apparatus 3200 can encode the current block by using the prediction block, in operation S3340.

[0535] As a result of encoding the current block, a bitstream can be generated.

[0536] In an embodiment, the image encoding apparatus 3200 can obtain residual data corresponding to a difference between the prediction block and the current block, and information about the residual data can be included in the bitstream.

[0537] According to an embodiment, the image encoding method and apparatus 3200 and the image decoding method and apparatus 2000 are provided to improve performance of prediction encoding and prediction decoding for the current block 2250.

[0538] According to an embodiment, the image encoding method and apparatus 3200 and the image decoding method and apparatus 2000 are provided to reduce an amount of data required to signal an inter prediction mode.

[0539] According to an embodiment, the image encoding method and apparatus 3200 and the image decoding method and apparatus 2000 are provided to reduce a bit rate of a bitstream.

[0540] Technical features intended to be implemented in the present disclosure are not limited to the above-described technical features, and other un-stated technical features will be clearly understood by those of ordinary skill in the art in view of the following description.

[0541] In an embodiment of the present disclosure, the image decoding method can include determining a first reference block in a first reference picture 2420 and a second reference block in a second reference picture 2440 for bi-prediction of a current block.

[0542] In an embodiment of the disclosure, the image decoding method can include determining weight information for a combination of the first reference block and the second reference block based on the current reference template of the current block 2410, a first reference template of the first reference block, and a second reference template of the second reference block.

[0543] In an embodiment of the disclosure, the image decoding method can include generating a prediction block of the current block 2410 by combining the first reference block and the second reference block using the weight information.

[0544] In an embodiment of the disclosure, the image decoding method can include reconstructing the current block 2410 by using the prediction block.

[0545] According to an embodiment, since the current block is determined without using a weight candidate list or a weight index, an amount of data for signaling can be reduced.

[0546] In an embodiment of the disclosure, the image decoding method can include determining a single weight to be applied to the first reference block based on at least one sample of the first reference template, at least one sample of the second reference template, and at least one sample of the current reference template.

[0547] In an embodiment of the disclosure, the single weight can be determined by performing linear regression analysis on the at least one sample of the first reference template, the at least one sample of the second reference template, and the at least one sample of the current reference template to minimize a difference between a value obtained by combining the first reference template and the second reference template and a value of the current reference template.

[0548] In an embodiment of the disclosure, the single weight can be determined based on a left weight determined based on at least one sample among left samples of the first reference template, at least one sample among left samples of the second reference template, and at least one sample among left samples of the current reference template, or an upper weight determined based on at least one sample among upper samples of the first reference template, at least one sample among upper samples of the second reference template, and at least one sample among upper samples of the current reference template.

[0549] In an embodiment of the disclosure, the image decoding method can include determining a left weight to be applied to a left lower sample region of the first reference block and an upper weight to be applied to a right upper sample region of the first reference block based on at least one sample of the first reference template, at least one sample of the second reference template, and at least one sample of the current reference template.

[0550] In an embodiment of the disclosure, the left-side weight can be determined based on one of the left-side samples of the first reference template, one of the left-side samples of the second reference template, and one of the left-side samples of the current reference template.

[0551] In an embodiment of the disclosure, the upper-side weight can be determined based on one of the upper-side samples of the first reference template, one of the upper-side samples of the second reference template, and one of the upper-side samples of the current reference template.

[0552] In an embodiment of the disclosure, the left-side weight can be determined by performing linear regression analysis on a plurality of samples among the left-side samples of the first reference template, a plurality of samples among the left-side samples of the second reference template, and a plurality of samples among the left-side samples of the current reference template to minimize a difference between a value obtained by combining the left-side samples of the first reference template and the left-side samples of the second reference template and a value of the left-side samples of the current reference template.

[0553] In an embodiment of the disclosure, the upper-side weight can be determined by performing linear regression analysis on a plurality of samples among the upper-side samples of the first reference template, a plurality of samples among the upper-side samples of the second reference template, and a plurality of samples among the upper-side samples of the current reference template to minimize a difference between a value obtained by combining the upper-side samples of the first reference template and the upper-side samples of the second reference template and a value of the upper-side samples of the current reference template.

[0554] In an embodiment of the disclosure, the left-side weight can be determined as an average of elements of a left-side weight matrix corresponding to the left-side samples of the first reference template, which is obtained based on the left-side samples of the first reference template, the left-side samples of the second reference template, and the left-side samples of the current reference template, and the upper-side weight can be determined as an average of elements of an upper-side weight matrix corresponding to the upper-side samples of the first reference template, which is obtained based on the upper-side samples of the first reference template, the upper-side samples of the second reference template, and the upper-side samples of the current reference template.

[0555] In an embodiment of the disclosure, the image decoding method can include obtaining a left weight matrix corresponding to the left samples of the first reference template based on the left samples of the first reference template, the left samples of the second reference template, and the left samples of the current reference template. The image decoding method can include obtaining an above weight matrix corresponding to the above samples of the first reference template based on the above samples of the first reference template, the above samples of the second reference template, and the above samples of the current reference template. The image decoding method can include determining a weight filter by using at least one of the left weight matrix or the above weight matrix, wherein the weight filter indicates weights corresponding to and to be applied to the samples included in the first reference block, respectively.

[0556] In an embodiment of the disclosure, the weight filter can be determined based on a column included in the left weight matrix and a row included in the above weight matrix.

[0557] In an embodiment of the disclosure, the weight filter can be determined based on an average of at least one element included in each row of the left weight matrix and an average of at least one element included in each column of the above weight matrix.

[0558] In an embodiment of the disclosure, the weight filter can be determined by applying a predetermined ratio to at least one element included in each row of the left weight matrix and at least one element included in each column of the above weight matrix.

[0559] In an embodiment of the disclosure, the image decoding method can include identifying a single weight as a first weight to be applied to the first reference block. The image decoding method can include obtaining a second weight to be applied to the second reference block by using the single weight. The image decoding method can include applying the first weight to the first reference block and applying the second weight to the second reference block.

[0560] In an embodiment of the disclosure, the image decoding method can include identifying a left weight as a first weight to be applied to a left-bottom sample region of the first reference block and identifying an above weight as a first weight to be applied to a right-top sample region of the first reference block. The image decoding method can include obtaining a second weight to be applied to the left-bottom sample region of the second reference block by using the left weight and obtaining a second weight to be applied to the right-top sample region of the second reference block by using the above weight. The image decoding method can include applying the first weight to the first reference block and applying the second weight to the second reference block.

[0561] In an embodiment of the disclosure, the image decoding method can include identifying a weight filter as a first weight to be applied to a first reference block. The image decoding method can include obtaining a second weight to be applied to a second reference block by using the weight filter. The image decoding method can include applying the first weight to the first reference block and applying the second weight to the second reference block.

[0562] In an embodiment of the disclosure, the image decoding apparatus 2000 can include at least one memory storing at least one instruction, and at least one processor configured to operate according to the at least one instruction.

[0563] In an embodiment of the disclosure, the at least one processor can be configured to determine a first reference block in a first reference picture 2420 and a second reference block in a second reference picture 2440 for bi-prediction of a current block.

[0564] In an embodiment of the disclosure, the at least one processor can be configured to determine weight information for a combination of the first reference block and the second reference block based on a current reference template of the current block 2410, a first reference template of the first reference block, and a second reference template of the second reference block.

[0565] In an embodiment of the disclosure, the at least one processor can be configured to generate a prediction block of the current block 2410 by combining the first reference block and the second reference block using the weight information.

[0566] In an embodiment of the disclosure, the at least one processor can be configured to reconstruct the current block 2410 by using the prediction block.

[0567] In an embodiment of the disclosure, the image encoding method can include determining a first reference block in a first reference picture and a second reference block in a second reference picture for bi-prediction of a current block 2410 (S3310).

[0568] In an embodiment of the disclosure, the image encoding method can include determining weight information for a combination of the first reference block and the second reference block based on a current reference template of the current block 2410, a first reference template of the first reference block, and a second reference template of the second reference block (S3320).

[0569] In an embodiment of the disclosure, the image encoding method can include generating a prediction block of the current block 2410 by combining the first reference block and the second reference block using the weight information (S3330).

[0570] In an embodiment of the disclosure, the image encoding method can include encoding the current block 2410 by using the prediction block (S3340).

[0571] In an embodiment of the disclosure, the image encoding apparatus 3200 can include at least one memory storing at least one instruction, and at least one processor configured to operate according to the at least one instruction.

[0572] In an embodiment of the disclosure, the at least one processor can be configured to determine a first reference block in a first reference picture and a second reference block in a second reference picture for bi-prediction of the current block 2410.

[0573] In an embodiment of the disclosure, the at least one processor can be configured to determine weight information for a combination of t...

Claims

1. An image decoding method, comprising: Determine the first reference block in the first reference image (2420) and the second reference block in the second reference image (2440) for bidirectional prediction of the current block (2410) (S3110). Based on the current reference template of the current block (2410), the first reference template of the first reference block and the second reference template of the second reference block, weight information for the combination of the first reference block and the second reference block is determined (S3120). The prediction block of the current block (2410) is generated by combining the first reference block and the second reference block using the weight information (S3130); and The current block (2410) is reconstructed using the predicted block (S3140).

2. The image decoding method as described in claim 1, wherein, Determining the weight information includes: determining a single weight to be applied to the first reference block based on at least one sample of the first reference template, at least one sample of the second reference template, and at least one sample of the current reference template.

3. The image decoding method as described in claim 2, wherein, The individual weight is determined by performing linear regression analysis on at least one sample of the first reference template, at least one sample of the second reference template, and at least one sample of the current reference template to minimize the difference between the value obtained by combining the first reference template and the second reference template and the value of the current reference template.

4. The image decoding method as described in claim 2, wherein, The individual weight is determined based on either the left-side weight or the top-side weight. The left-side weight is determined based on at least one sample from the left side of the first reference template, at least one sample from the left side of the second reference template, and at least one sample from the left side of the current reference template. The top-side weight is determined based on at least one sample from the top side of the first reference template, at least one sample from the top side of the second reference template, and at least one sample from the top side of the current reference template.

5. The image decoding method as described in claim 1, wherein, Determining the weight information includes: based on at least one sample of the first reference template, at least one sample of the second reference template, and at least one sample of the current reference template, determining the left weight to be applied to the lower left sample region of the first reference block and the upper weight to be applied to the upper right sample region of the first reference block.

6. The image decoding method according to at least one of claims 4 and 5, wherein, The left-side weight is determined based on one sample from the left-side samples of the first reference template, one sample from the left-side samples of the second reference template, and one sample from the left-side samples of the current reference template. The upper weight is determined based on a sample from the upper sample of the first reference template, a sample from the upper sample of the second reference template, and a sample from the upper sample of the current reference template.

7. The image decoding method according to at least one of claims 4 and 5, wherein, The left-side weights are determined by performing linear regression analysis on multiple samples from the left-side samples of the first reference template, multiple samples from the left-side samples of the second reference template, and multiple samples from the left-side samples of the current reference template to minimize the difference between the value obtained by combining the left-side samples of the first reference template and the left-side samples of the second reference template and the value of the left-side sample of the current reference template. The upper weight is determined by performing linear regression analysis on multiple samples in the upper samples of the first reference template, multiple samples in the upper samples of the second reference template, and multiple samples in the upper samples of the current reference template to minimize the difference between the value obtained by combining the upper samples of the first reference template and the upper samples of the second reference template and the value of the upper sample of the current reference template.

8. The image decoding method according to at least one of claims 4 and 5, wherein, The left-side weights are determined as the average of the elements of a left-side weight matrix obtained based on the left-side samples of the first reference template, the left-side samples of the second reference template, and the left-side samples of the current reference template, and corresponding to the left-side samples of the first reference template. The upper weight is determined as the average of the elements of the upper weight matrix obtained based on the upper samples of the first reference template, the upper samples of the second reference template, and the upper samples of the current reference template, and corresponding to the upper samples of the first reference template.

9. The image decoding method as described in claim 1, wherein, Determining the weight information includes: Based on the left samples of the first reference template, the left samples of the second reference template, and the left samples of the current reference template, a left weight matrix corresponding to the left samples of the first reference template is obtained. Based on the upper samples of the first reference template, the upper samples of the second reference template, and the upper samples of the current reference template, an upper weight matrix corresponding to the upper samples of the first reference template is obtained; and A weight filter is determined by using at least one of the left-hand weight matrix or the top-hand weight matrix, the weight filter indicating the weights corresponding to and to be applied to the samples included in the first reference block.

10. The image decoding method as described in claim 9, wherein, The weighted filter is determined based on a column included in the left-hand weighted matrix and a row included in the upper-hand weighted matrix.

11. The image decoding method as described in claim 9, wherein, The weighted filter is determined based on the average of at least one element in each row of the left-hand weighted matrix and the average of at least one element in each column of the upper-hand weighted matrix.

12. The image decoding method as described in claim 9, wherein, The weight filter is determined by applying a preset ratio to at least one element in each row of the weight matrix on the left and at least one element in each column of the weight matrix above.

13. The image decoding method according to any one of claims 2 to 4, wherein, The prediction block that generates the current block (2410) includes: The individual weight is identified as a first weight that will be applied to the first reference block; A second weight, to be applied to the second reference block, is obtained by using the single weight; and The first weight is applied to the first reference block, and the second weight is applied to the second reference block.

14. An image encoding method, comprising: Determine a first reference block in a first reference image and a second reference block in a second reference image for bidirectional prediction of the current block (2410) (S3310). Based on the current reference template of the current block (2410), the first reference template of the first reference block and the second reference template of the second reference block, weight information for the combination of the first reference block and the second reference block is determined (S3320). The prediction block of the current block (2410) is generated by combining the first reference block and the second reference block using the weight information (S3330); and The current block (2410) is encoded using the prediction block (S3340).

15. A computer-readable recording medium storing a bitstream generated by an image encoding method. in, The bitstream includes motion information for the current block (2410). The motion information of the current block (2410) is generated through the following operations: Determine a first reference block in a first reference image and a second reference block in a second reference image for bidirectional prediction of the current block (2410). Obtain the current reference template that indicates the reference template of the current block (2410), the first reference template that indicates the reference template of the first reference block, and the second reference template that indicates the reference template of the second reference block. Based on the current reference template, the first reference template, and the second reference template, weight information for the combination of the first reference block and the second reference block is determined. The prediction block of the current block (2410) is generated by combining the first reference block and the second reference block using the weight information, and The current block (2410) is encoded using the prediction block.

Images