Apparatus and method for encoding and decoding images using a reference block

Abstract

An image decoding method according to one embodiment includes the steps of: determining a first reference block in a first reference image and a second reference block in a second reference image for dual prediction of the current block; determining weighting information for combining the first reference block and the second reference block based on the current reference template of the current block, the first reference template of the first reference block, and the second reference template of the second reference block; generating a prediction block for the current block by combining the first reference block and the second reference block using the weighting information; and decoding the current block using the prediction block.

Description

[Technical Field]

[0001] The present invention relates to the field of image encoding and decoding, and more specifically to an apparatus and method for encoding and decoding an image using a reference block contained in at least one of a current image or a previous image. [Background technology]

[0002] In image encoding and decoding, the image is divided into blocks, and each block is predictively encoded and predictively decoded through interpretation.

[0003] Interpretation is a technique for compressing images by removing temporal redundancy between images. Interpretation uses a reference image to predict blocks in the current image. The reference block most similar to the current block can be searched within a predetermined search range in the reference image. The current block is predicted based on the reference block, and the predicted block generated as a result is subtracted from the current block to produce a residual block.

[0004] In codecs such as H.264 AVC (Advanced Video Coding) and HEVC (High Efficiency Video Coding), the motion vector of a previously encoded block adjacent to the current block, or a block contained in a previously encoded image, is used as the motion vector predictor for the current block in order to predict the motion vector of the current block. The motion vector difference, which is the difference between the motion vector of the current block and the motion vector predictor, is signaled to the decoder through a predetermined method.

[0005] The residual block generated through interpretation is transmitted to the decoder after transformation and quantization. The decoder dequantizes and detransforms the residual block, and decodes the current block by combining the predicted block of the current block with the residual block. In certain cases, the decoder can filter the decoded current block to remove artifacts within it. [Overview of the project] [Problems that the invention aims to solve]

[0006] One embodiment of an image encoding method and apparatus, and an image decoding method and apparatus, aims to improve the performance of predictive coding and predictive decoding for current blocks.

[0007] One embodiment of an image encoding method and apparatus, and an image decoding method and apparatus, aims to reduce the amount of data required for signaling in interprediction mode.

[0008] One embodiment of an image encoding method and apparatus, and an image decoding method and apparatus, aims to reduce the bitrate of the bitstream.

[0009] The technical problems that the present invention aims to solve are not limited to those mentioned above, and other technical problems not mentioned can be clearly understood by those skilled in the art from the following description. [Means for solving the problem]

[0010] In one embodiment of the present invention, the image decoding method includes the step of determining a first reference block in a first reference image and a second reference block in a second reference image for bi-prediction of the current block.

[0011] In one embodiment of the present invention, an image decoding method includes a step of determining weight value information for combining a first reference block and a second reference block based on a current reference template of a current block, a first reference template of a first reference block, and a second reference template of a second reference block.

[0012] In one embodiment of the present invention, an image decoding method includes a step of generating a prediction block for a current block by combining a first reference block and a second reference block using the weight value information.

[0013] In one embodiment of the present invention, an image decoding method includes a step of decoding a current block using the prediction block.

[0014] In one embodiment of the present invention, an image decoding apparatus 2000 includes at least one memory storing at least one instruction, and at least one processor operating according to the at least one instruction.

[0015] In one embodiment of the present invention, the at least one processor determines a first reference block in a first reference image and a second reference block in a second reference image for bi-prediction of a current block.

[0016] In one embodiment of the present invention, the at least one processor determines weight value information for combining a first reference block and a second reference block based on a current reference template of a current block, a first reference template of the first reference block, and a second reference template of the second reference block.

[0017] In one embodiment of the present invention, the at least one processor generates a prediction block for a current block by combining a first reference block and a second reference block using the weight value information.

[0018] In one embodiment of the present invention, the at least one processor decodes the current block using a prediction block.

[0019] In one embodiment of the present invention, an image encoding method includes the step of determining a first reference block in a first reference image and a second reference block in a second reference image for biprediction of the current block.

[0020] In one embodiment of the present invention, the image encoding method includes the step of determining weighting information for combining the first reference block and the second reference block based on the current reference template of the current block, the first reference template of the first reference block, and the second reference template of the second reference block.

[0021] In one embodiment of the present invention, the image encoding method includes the step of generating a predicted block for the current block by combining a first reference block and a second reference block using weighted value information.

[0022] In one embodiment of the present invention, the image encoding method includes the step of encoding the current block using a prediction block.

[0023] In one embodiment of the present invention, the image encoding device 3200 includes at least one memory for storing at least one instruction, and at least one processor that operates in accordance with at least one instruction.

[0024] In one embodiment of the present invention, the at least one processor determines a second reference block in a first reference image and a second reference block in a second reference image for biprediction of the current block.

[0025] In one embodiment of the present invention, the at least one processor determines weighting information for joining the first reference block and the second reference block based on the current reference template of the current block, the first reference template of the first reference block, and the second reference template of the second reference block.

[0026] In one embodiment of the present invention, the at least one processor generates a predicted block for the current block by combining a first reference block and a second reference block using weighted value information.

[0027] In one embodiment of the present invention, the at least one processor encodes the current block using a prediction block.

[0028] In one embodiment of the present invention, a computer-readable recording medium records a bitstream generated by an image encoding method, wherein the bitstream includes motion information of the current block.

[0029] In one embodiment of the present invention, motion information is generated by determining a first reference block in a first reference image and a second reference block in a second reference image for bi-prediction of the current block, obtaining a current reference template indicating the reference template of the current block, a first reference template indicating the reference template of the first reference block, and a second reference template indicating the reference template of the second reference block, determining weighting information for combining the first reference block and the second reference block based on the current reference template, the first reference template, and the second reference template, generating a predicted block for the current block by combining the first reference block and the second reference block using the weighting information, and encoding the current block using the predicted block. [Effects of the Invention]

[0030] An image encoding method and apparatus, and an image decoding method and apparatus according to one embodiment, can improve the performance of predictive coding and predictive decoding for current blocks.

[0031] An image encoding method and apparatus, and an image decoding method and apparatus according to one embodiment, can reduce the amount of data required for signaling in interprediction mode.

[0032] An image encoding method and apparatus, and an image decoding method and apparatus according to one embodiment, can reduce the bitrate of the bitstream.

[0033] The technical problems that the present invention aims to solve are not limited to those mentioned above, and other technical problems not mentioned can be clearly understood by those skilled in the art from the following description. [Brief explanation of the drawing]

[0034] [Figure 1] This is a block diagram of an image decoding device according to one embodiment. [Figure 2] This is a block diagram of an image encoding device according to one embodiment. [Figure 3] This figure shows the process of dividing the current coding unit and determining at least one coding unit according to one embodiment. [Figure 4] This figure shows the process of dividing a non-square coding unit and determining at least one coding unit according to one embodiment. [Figure 5] This figure shows the process of dividing an encoded unit based on at least one of block shape information and divided shape mode information, according to one embodiment. [Figure 6] This figure shows a method for determining a predetermined coding unit from an odd number of coding units according to one embodiment. [Figure 7] This diagram shows the order in which multiple coding units are processed when a coding unit is currently divided and multiple coding units are determined according to one embodiment. [Figure 8] This figure shows the process by which, according to one embodiment, when the coding units cannot be processed in a predetermined order, it is determined that the current coding unit will be divided into an odd number of coding units. [Figure 9]This figure shows the process of dividing a first coding unit to determine at least one coding unit, according to one embodiment. [Figure 10] This figure shows that, according to one embodiment, if the non-square second coding unit determined by dividing the first coding unit satisfies predetermined conditions, the divisible shape is limited. [Figure 11] This figure shows the process of dividing a square coding unit when, according to one embodiment, the division shape mode information cannot indicate division into four square coding units. [Figure 12] This figure shows that, in one embodiment, the processing order between multiple coding units can change depending on the coding unit partitioning process. [Figure 13] This figure shows the process by which the depth of a coding unit is determined in accordance with changes in the shape and size of the coding unit, when a coding unit is recursively divided to determine multiple coding units according to one embodiment. [Figure 14] This figure shows the depth and part index (PID) of the coding unit, which can be determined based on the shape and size of the coding unit according to one embodiment. [Figure 15] This figure shows that, according to one embodiment, multiple encoding units are determined based on multiple predetermined data units contained in a picture. [Figure 16] This figure shows an embodiment in which, when the combination of shapes that can be divided into an encoding unit differs for each picture, the encoding unit can be determined for each picture. [Figure 17] This figure shows various forms of coding units that can be determined based on segmentation shape mode information represented by binary code, according to one embodiment. [Figure 18] This figure shows another form of coding unit that can be determined based on segmentation shape mode information represented by binary code, according to one embodiment. [Figure 19] This is a block diagram of an image coding and decoding system that performs loop filtering according to one embodiment. [Figure 20] This block diagram shows the configuration of an image decoding device according to one embodiment. [Figure 21] This figure illustrates a dual prediction of the current image according to one embodiment. [Figure 22] This diagram shows the current block and the blocks that are related to it in time and / or space. [Figure 23] This is a diagram illustrating a template for a current block or reference block according to one embodiment. [Figure 24] This figure illustrates the operation of determining a reference block based on template matching according to one embodiment. [Figure 25] This figure illustrates the steps for determining weighted value information according to one embodiment. [Figure 26] This diagram illustrates the operation for determining a weighted filter according to one embodiment. [Figure 27] This figure illustrates the operation of generating prediction blocks using weighted value information according to one embodiment. [Figure 28] This is a diagram illustrating the specific process for determining weighted value information according to one embodiment. [Figure 29] This figure illustrates the specific process for determining a weighted filter according to one embodiment. [Figure 30] This is a diagram illustrating the specific process for determining weighted value information according to one embodiment. [Figure 31] This is a flowchart of an image decoding method according to one embodiment. [Figure 32] This block diagram shows the configuration of an image encoding device according to one embodiment. [Figure 33] This is a flowchart of an image coding method according to one embodiment. [Modes for carrying out the invention]

[0035] The present invention is subject to various modifications and can have a variety of embodiments, which are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to embodiments, and the present invention can include all modifications, equivalents or substitutes that fall within the spirit and technical scope of various embodiments.

[0036] When describing embodiments, if a detailed explanation of related prior art is deemed to unnecessarily obscure the spirit of the present invention, such detailed explanation will be omitted. Furthermore, the numbers used in the description of embodiments (for example, 1st, 2nd, etc.) are merely identification symbols to distinguish one component from other components.

[0037] In the present invention, the expression "at least one of a, b, or c" refers to "a," "b," "c," "a and b," "a and c," "b and c," "any of a, b, and c," or variations thereof.

[0038] In this invention, when one component is referred to as being "connected" or "linked" to another component, it may mean that one component is directly connected to or linked to another component, but unless otherwise specified, it may also mean that one component is connected or linked through an intermediate component.

[0039] In this invention, components represented as "~part (unit)" or "module" may include cases where two or more components are combined into one component, or where one component is divided into two or more more subdivided components. Furthermore, each of the components described below may perform some or all of the functions performed by other components, in addition to its own primary function, and some of the primary functions performed by each component may be performed by other components.

[0040] In this invention, "image" refers to a picture, a still image, a frame, or a video composed of multiple consecutive still images.

[0041] In this invention, "sample" refers to data assigned to a sampling position in an image, which is the data to be processed. For example, pixels within a frame in a spatial domain correspond to samples. A unit containing multiple samples is defined as a block.

[0042] In the following, with reference to Figures 1 to 19, an image encoding method and apparatus, an image decoding method and apparatus, based on a tree structure encoding unit and transformation unit according to one embodiment are disclosed.

[0043] Figure 1 is a block diagram of an image decoding device 100 according to one embodiment.

[0044] The image decoding device 100 includes a bitstream acquisition unit 110 and a decoding unit 120. The bitstream acquisition unit 110 and the decoding unit 120 each include at least one processor. The bitstream acquisition unit 110 and the decoding unit 120 also include a memory for storing instructions executed by at least one processor.

[0045] The bitstream acquisition unit 110 receives a bitstream. The bitstream contains information encoded by the image encoding device 200, which will be described later. The bitstream can also be transmitted from the image encoding device 200. The image encoding device 200 and the image decoding device 100 are connected by wire or wireless, and the bitstream acquisition unit 110 receives the bitstream via wire or wireless. The bitstream acquisition unit 110 receives the bitstream from a recording medium such as optical media or a hard disk. The decoding unit 120 can decode the image based on the information acquired from the received bitstream. The decoding unit 120 acquires syntax elements for decoding the image from the bitstream. The decoding unit 120 can decode the image based on the syntax elements.

[0046] To explain the operation of the image decoding device 100 in detail, the bitstream acquisition unit 110 receives the bitstream.

[0047] The image decoding device 100 performs the operation of obtaining a bin string corresponding to the partition shape mode of the coding unit from the bitstream. Then, the image decoding device 100 performs the operation of determining the partition rule of the coding unit. Furthermore, the image decoding device 100 can perform the operation of partitioning the coding unit into a plurality of coding units based on the bin string corresponding to the partition shape mode and at least one of the partition rule. In order to determine the partition rule, the image decoding device 100 can determine a first allowable range of the size of the coding unit based on the ratio of the width and height of the coding unit. In order to determine the partition rule, the image decoding device 100 can determine a second allowable range of the size of the coding unit based on the partition shape mode of the coding unit.

[0048] The following describes in detail the division of coding units using one embodiment of the present invention.

[0049] First, a picture can be divided into one or more slices or one or more tiles. A slice or tile can be a sequence of one or more Coding Tree Units (CTUs). Depending on the embodiment, a slice may contain one or more tiles, and a slice may contain one or more CTUs. The slices containing one or more tiles can be determined within a picture.

[0050] In contrast to the Maximum Coding Unit (CTU), there is the concept of a Maximum Coding Tree Block (CTB). A Maximum Coding Tree Block (CTB) represents an N×N block containing N×N samples (where N is an integer). Each color component can be divided into one or more Maximum Coding Tree Blocks.

[0051] If a picture has three sample sequences (sample sequences for each of the Y, Cr, and Cb components), the Maximum Encoded Unit (CTU) is a unit that includes the maximum encoded block of the luminal sample and the two corresponding maximum encoded blocks of the chroma sample, as well as the syntax structure used to encode the luminal and chroma samples. If the picture is a monochrome picture, the Maximum Encoded Unit is a unit that includes the maximum encoded block of the monochrome sample and the syntax structure used to encode the monochrome sample. If the picture is encoded with color planes separated by color components, the Maximum Encoded Unit is a unit that includes the picture and the syntax structure used to encode the samples of the picture.

[0052] A single maximum coding block (CTB) can be divided into M×N coding blocks containing M×N samples (where M and N are integers).

[0053] If a picture has sample sequences for each of the Y, Cr, and Cb components, the coding unit (Cu) is a unit that includes the coding block of the luminous sample and the two corresponding coding blocks of the chroma sample, as well as the syntax structure used to code the luminous and chroma samples. If the picture is a monochrome picture, the coding unit is a unit that includes the coding block of the monochrome sample and the syntax structure used to code the monochrome sample. If the picture is coded with color planes separated for each color component, the coding unit is a unit that includes the picture and the syntax structure used to code the samples of the picture.

[0054] As stated above, the maximum coding block and the maximum coding unit are distinct concepts, and the coding block and the coding unit are also distinct concepts. That is, a (maximum) coding unit means a data structure that includes the (maximum) coding block containing the sample and the corresponding syntax structure. However, a person skilled in the art will understand that a (maximum) coding unit or a (maximum) coding block refers to a block of a predetermined size containing a predetermined number of samples. Therefore, in the following specification, unless there are special circumstances, the maximum coding block and the maximum coding unit, or the coding block and the coding unit, will not be distinguished.

[0055] An image can be divided into Coding Tree Units (CTUs). The size of a CTU can be determined based on information obtained from the bitstream. The shape of a CTU is, but is not limited to, a square of the same size.

[0056] For example, information about the maximum size of a Luma-coded block can be obtained from a bitstream. For instance, the maximum size of a Luma-coded block indicated by this information is one of the following: 4x4, 8x8, 16x16, 32x32, 64x64, 128x128, or 256x256.

[0057] For example, information about the maximum size of a bipartite Luma-encoded block and the Luma block size difference can be obtained from the bitstream. The information about the Luma block size difference indicates the size difference between the largest Luma-encoded unit and the largest bipartite Luma-encoded block. Therefore, by combining the information about the maximum size of a bipartite Luma-encoded block and the information about the Luma block size difference obtained from the bitstream, the size of the largest Luma-encoded unit can be determined. Using the size of the largest Luma-encoded unit, the size of the largest chroma-encoded unit can also be determined. For example, if the color format is Y:Cb:Cr=4:2:0, the size of the chroma block is half the size of the Luma block, and similarly, the size of the largest chroma-encoded unit may also be half the size of the largest Luma-encoded unit.

[0058] According to one embodiment, since information regarding the maximum size of a binary-splittable Luma-coded block is obtained from the bitstream, the maximum size of a binary-splittable Luma-coded block can be determined variably. In contrast, the maximum size of a ternary-splittable Luma-coded block can be fixed. For example, the maximum size of a ternary-splittable Luma-coded block in an I-picture is 32×32, and the maximum size of a ternary-splittable Luma-coded block in a P-picture or B-picture is 64×64.

[0059] Furthermore, the maximum encoding unit can be hierarchically divided into encoding units based on the split shape mode information obtained from the bitstream. The split shape mode information can include at least one of the following, obtained from the bitstream: information indicating the presence or absence of a quad split, information indicating the presence or absence of multiple splits, split direction information, and split type information.

[0060] For example, the information indicating whether or not a quad split is performed indicates whether or not the current encoding unit is being split into quads (QUAD_SPLIT).

[0061] If the current encoding unit is not quad-split, the information indicating whether or not it is multi-split can indicate whether the current encoding unit will not be split further (NO_SPLIT) or whether it will be binary / ternary-split.

[0062] If the current encoding unit is to be split in binary or terminally, the split direction information indicates whether the current encoding unit will be split horizontally or vertically.

[0063] If the current coding unit is to be split horizontally or vertically, the splitting type information indicates whether the current coding unit will be split binaryly or ternarily.

[0064] Based on the division direction information and division type information, the division mode of the current encoded unit can be determined. If the current encoded unit is divided horizontally in binary, the division mode can be determined as binary horizontal division (SPLIT_BT_HOR); if it is divided horizontally in a ternary manner, the division mode can be determined as ternary horizontal division (SPLIT_TT_HOR); if it is divided vertically in binary, the division mode can be determined as binary vertical division (SPLIT_BT_VER); and if it is divided vertically in a ternary manner, the division mode can be determined as ternary vertical division (SPLIT_TT_VER).

[0065] The image decoder 100 obtains segmentation mode information from a bitstream from a single binstring. The bitstream received by the image decoder 100 can take the form of fixed-length binary code, unary code, truncated unary code, or a predetermined binary code. A binstring represents information as a sequence of binary numbers. A binstring can consist of at least one bit. Based on the segmentation rule, the image decoder 100 obtains segmentation mode information corresponding to the binstring. Based on a single binstring, the image decoder 100 determines whether or not to segment the encoding unit into quads, and also determines the segmentation direction and segmentation type.

[0066] An encoding unit is less than or equal to the maximum encoding unit. For example, the maximum encoding unit is also an encoding unit because it has the largest size. If the division shape mode information for the maximum encoding unit indicates "not divided," the encoding unit determined in the maximum encoding unit will have the same size as the maximum encoding unit. If the division shape mode information for the maximum encoding unit indicates "divided," the maximum encoding unit may be divided into encoding units. Also, if the division shape mode information for an encoding unit indicates division, the encoding unit may be divided into smaller encoding units. However, image division is not limited to this, and the maximum encoding unit and encoding units may not be distinguished. The division of encoding units will be explained in more detail in Figures 3 to 16.

[0067] Furthermore, one or more prediction blocks may be determined from the coding unit. The prediction blocks are either the same as or smaller than the coding unit. Also, one or more transformation blocks may be determined from the coding unit. The transformation blocks are either the same as or smaller than the coding unit.

[0068] The shapes and sizes of the transformation block and the prediction block may not be related to each other.

[0069] In other embodiments, prediction may be performed using coding units as prediction blocks. Furthermore, transformation may be performed using coding units as transformation blocks.

[0070] The partitioning of the coding unit will be explained in more detail in Figures 3 to 16. The current block and surrounding block of the present invention can represent any of the maximum coding unit, coding unit, prediction block, and transformation block. The current block or current coding unit is a block that is currently being decoded or coded, or a block that is currently being partitioned. The surrounding block may be a block that was decoded before the current block. The surrounding block is spatially or temporally adjacent to the current block. The surrounding block is located to the lower left, left, upper left, top, upper right, right, or lower right of the current block.

[0071] Figure 3 shows the process by which the image decoding device 100 divides the current encoding unit and determines at least one encoding unit, according to one embodiment.

[0072] The block shapes include 4N×4N, 4N×2N, 2N×4N, 4N×N, N×4N, 32N×N, N×32N, 16N×N, N×16N, 8N×N, or N×8N, where N is a positive integer. The block shape information is information that indicates at least one of the shape, orientation, width, and height ratios or size of the coding unit.

[0073] The shape of the encoded unit includes both square and non-square shapes. If the width and height of the encoded unit are the same (i.e., the block shape of the encoded unit is 4N × 4N), the image decoding device 100 determines the block shape information of the encoded unit to be square. The image decoding device 100 also determines the shape of the encoded unit to be non-square.

[0074] If the width and height of the coding unit are different (i.e., the block shape of the coding unit is 4N×2N, 2N×4N, 4N×N, N×4N, 32N×N, N×32N, 16N×N, N×16N, 8N×N, or N×8N), the image decoding device 100 determines the block shape information of the coding unit as non-square. If the shape of the coding unit is non-square, the image decoding device 100 determines the ratio of width to height in the block shape information of the coding unit to be at least one of 1:2, 2:1, 1:4, 4:1, 1:8, 8:1, 1:16, 16:1, 1:32, or 32:1. Furthermore, based on the width and height of the coding unit, the image decoding device 100 determines whether the coding unit is horizontal or vertical. Furthermore, based on at least one of the width, height, or area of ​​the coding unit, the image decoding device 100 determines the size of the coding unit.

[0075] According to one embodiment, the image decoding device 100 determines the shape of the coding unit using block shape information and determines how the coding unit will be divided using division shape mode information. That is, the method of dividing the coding unit indicated by the division shape mode information can be determined depending on which block shape the block shape information used by the image decoding device 100 indicates.

[0076] The image decoding device 100 acquires splitting shape mode information from the bitstream. However, it is not limited to this; the image decoding device 100 and the image encoding device 200 can determine predetermined splitting shape mode information based on block shape information. The image decoding device 100 determines predetermined splitting shape mode information for the largest encoding unit or the smallest encoding unit. For example, the image decoding device 100 determines the splitting shape mode information for the largest encoding unit as a quad split. The image decoding device 100 also determines the splitting shape mode information for the smallest encoding unit as "no split". Specifically, the image decoding device 100 determines the size of the largest encoding unit as 256 × 256. The image decoding device 100 determines predetermined splitting shape mode information as a quad split. A quad split is a splitting shape mode that divides both the width and height of the encoding unit into two equal parts. The image decoding device 100 obtains a coding unit having a size of 128 × 128 from the largest coding unit having a size of 256 × 256 based on the division shape mode information. The image decoding device 100 also determines the size of the smallest coding unit to be 4 × 4. The image decoding device 100 obtains division shape mode information indicating "do not divide" for the smallest coding unit.

[0077] According to one embodiment, the image decoding device 100 utilizes block shape information indicating that the current encoding unit is a square. For example, based on the division shape mode information, the image decoding device 100 decides whether to not divide the square encoding unit, to divide it vertically, to divide it horizontally, or to divide it into four encoding units. Referring to Figure 3, if the block shape information of the current encoding unit 300 indicates a square, the decoding unit 120 decides whether not to divide the encoding unit 310a, which is the same size as the current encoding unit 300, based on the division shape mode information indicating that it will not be divided, or determines the divided encoding units 310b, 310c, 310d, 310e, 310f, etc., based on the division shape mode information indicating a predetermined division method.

[0078] Referring to Figure 3, in one embodiment, the image decoding device 100 determines two coding units 310b obtained by vertically dividing the current coding unit 300 based on division shape mode information indicating that it is divided vertically. The image decoding device 100 determines two coding units 310c obtained by horizontally dividing the current coding unit 300 based on division shape mode information indicating that it is divided horizontally. The image decoding device 100 determines four coding units 310d obtained by vertically and horizontally dividing the current coding unit 300 based on division shape mode information indicating that it is divided vertically and horizontally. In one embodiment, the image decoding device 100 determines three coding units 310e obtained by vertically dividing the current coding unit 300 based on division shape mode information indicating that it is ternarily divided vertically. The image decoding device 100 determines three coding units 310f obtained by horizontally dividing the current coding unit 300 based on division shape mode information indicating that it is ternarily divided horizontally. However, the division shapes into which a square coding unit can be divided should not be interpreted as being limited to the forms described above, but may include a variety of forms that the division shape mode information can indicate. A predetermined division shape into which a square coding unit is divided will be specifically described below through various embodiments.

[0079] Figure 4 shows the process by which the image decoding device 100 divides a non-square coding unit and determines at least one coding unit, according to one embodiment.

[0080] According to one embodiment, the image decoding device 100 utilizes block shape information indicating that the current encoding unit is non-square. Based on the division shape mode information, the image decoding device 100 decides whether to divide the non-square current encoding unit or to divide it in a predetermined way. Referring to Figure 4, if the block shape information of the current encoding unit 400 or 450 indicates that it is non-square, the image decoding device 100 determines an encoding unit 410 or 460 having the same size as the current encoding unit 400 or 450 based on the division shape mode information indicating not to divide it, or determines divided encoding units 420a, 420b, 430a, 430b, 430c, 470a, 470b, 480a, 480b, 480c based on the division shape mode information indicating a predetermined division method. The predetermined division method for dividing the non-square encoding unit will be specifically described below through various embodiments.

[0081] According to one embodiment, the image decoding device 100 determines the form in which the coding unit is divided using division shape mode information, in which case the division shape mode information indicates the number of at least one coding unit generated by dividing the coding unit. Referring to Figure 4, if the division shape mode information indicates that the current coding unit 400 or 450 is divided into two coding units, the image decoding device 100 divides the current coding unit 400 or 450 based on the division shape mode information and determines the two coding units 420a, 420b or 470a, 470b that are included in the current coding unit.

[0082] According to one embodiment, when the image decoding device 100 divides a non-square current coding unit 400 or 450 based on the division shape mode information, the image decoding device 100 divides the current coding unit considering the position of the longer side of the non-square current coding unit 400 or 450. For example, the image decoding device 100 considers the shape of the current coding unit 400 or 450 and divides the current coding unit 400 or 450 in a direction that divides the longer side of the current coding unit 400 or 450 to determine a plurality of coding units.

[0083] According to one embodiment, if the division shape mode information indicates that the coding unit is divided into an odd number of blocks (ternary division), the image decoding device 100 determines the odd number of coding units currently included in the coding unit 400 or 450. For example, if the division shape mode information indicates that the current coding unit 400 or 450 is divided into three coding units, the image decoding device 100 divides the current coding unit 400 or 450 into three coding units 430a, 430b, 430c or 480a, 480b, 480c.

[0084] According to one embodiment, the width-to-height ratio of the current coding unit 400 or 450 is 4:1 or 1:4. When the width-to-height ratio is 4:1, the length of the width is longer than the length of the height, so the block shape information can be horizontal. When the width-to-height ratio is 1:4, the length of the width is shorter than the length of the height, so the block shape information can be vertical. The image decoding device 100 determines whether to divide the current coding unit into an odd number of blocks based on the division shape mode information. The image decoding device 100 also determines the division direction of the current coding unit 400 or 450 based on the block shape information of the current coding unit 400 or 450. For example, if the current coding unit 400 is vertical, the image decoding device 100 divides the current coding unit 400 horizontally to determine coding units 430a, 430b, and 430c. Furthermore, if the current coding unit 450 is in the horizontal direction, the image decoding device 100 divides the current coding unit 450 vertically to determine coding units 480a, 480b, and 480c.

[0085] According to one embodiment, the image decoding device 100 determines an odd number of coding units currently included in coding unit 400 or 450, and the determined coding units are not necessarily all the same size. For example, among the determined odd number of coding units 430a, 430b, 430c, 480a, 480b, and 480c, a predetermined coding unit 430b or 480b may have a different size from the other coding units 430a, 430c, 480a, and 480c. That is, the coding units determined by dividing the current coding unit 400 or 450 may have multiple sizes, and in some cases, the odd number of coding units 430a, 430b, 430c, 480a, 480b, and 480c may each have different sizes.

[0086] According to one embodiment, if the division shape mode information indicates that the coding unit is divided into an odd number of blocks, the image decoding device 100 determines the odd number of coding units currently included in the coding unit 400 or 450, and further, the image decoding device 100 imposes a predetermined restriction on at least one of the odd number of coding units generated by the division. Referring to Figure 4, the image decoding device 100 makes the decoding process for the central coding unit 430b, 480b, and 480c, which are the three coding units 430a, 430b, 430c, 480a, 480b, and 480c generated by the division of the current coding unit 400 or 450, different from that for the other coding units 430a, 430c, 480a, and 480c. For example, the image decoding device 100 restricts the centrally located coding units 430b and 480b from being further divided, or restricts them to being divided only a predetermined number of times, unlike the other coding units 430a, 430c, 480a, and 480c.

[0087] Figure 5 shows, in one embodiment, the process by which the image decoding device 100 divides an encoded unit based on at least one of block shape information and divided shape mode information.

[0088] In one embodiment, the image decoding device 100 determines whether to divide a square first coding unit 500 into coding units or not, based on at least one of block shape information and division shape mode information. According to one embodiment, if the division shape mode information indicates that the first coding unit 500 should be divided horizontally, the image decoding device 100 divides the first coding unit 500 horizontally and determines a second coding unit 510. In one embodiment, the terms first coding unit, second coding unit, and third coding unit used are used to understand the relationship between coding units before and after division. For example, when the first coding unit is divided, the second coding unit is determined, and when the second coding unit is divided, the third coding unit is determined. The relationship between the first coding unit, second coding unit, and third coding unit used below is understood to follow the characteristics described above.

[0089] In one embodiment, the image decoding device 100 decides whether to divide the determined second coding unit 510 into coding units based on the division shape mode information, or not. Referring to Figure 5, the image decoding device 100 divides the non-square second coding unit 510, determined by dividing the first coding unit 500 based on the division shape mode information, into at least one third coding unit 520a, 520b, 520c, 520d, etc., or does not divide it. The image decoding device 100 acquires the division shape mode information, divides the first coding unit 500 based on the acquired division shape mode information, determines a plurality of second coding units (e.g., 510) of various forms, and the second coding unit 510 is divided according to the method in which the first coding unit 500 was divided based on the division shape mode information. According to one embodiment, if a first coding unit 500 is divided into a second coding unit 510 based on the division shape mode information for the first coding unit 500, the second coding unit 510 may also be divided into a third coding unit (e.g., 520a, 520b, 520c, 520d, etc.) based on the division shape mode information for the second coding unit 510. That is, coding units are recursively divided based on the division shape mode information associated with each coding unit. Therefore, a square coding unit may be determined from a non-square coding unit, and such a square coding unit may be recursively divided to determine a non-square coding unit.

[0090] Referring to Figure 5, among the odd number of third coding units 520b, 520c, and 520d determined by the division of the non-square second coding unit 510, a predetermined coding unit (e.g., a centrally located coding unit or a square coding unit) can be recursively divided. According to one embodiment, a non-square third coding unit 520b, which is one of the odd number of third coding units 520b, 520c, and 520d, is divided horizontally into a plurality of fourth coding units. A non-square fourth coding unit 530b or 530d, which is one of the plurality of fourth coding units 530a, 530b, 530c, and 530d, can be further divided into a plurality of coding units. For example, a non-square fourth coding unit 530b or 530d may be divided again into an odd number of coding units. Methods that can be used for the recursive division of coding units will be described later through various embodiments.

[0091] In one embodiment, the image decoding device 100 divides each of the third coding units 520a, 520b, 520c, 520d, etc., into coding units based on the division shape mode information. The image decoding device 100 also decides not to divide the second coding unit 510 based on the division shape mode information. In one embodiment, the image decoding device 100 divides the non-square second coding unit 510 into an odd number of third coding units 520b, 520c, and 520d. The image decoding device 100 sets predetermined restrictions on a predetermined third coding unit among the odd number of third coding units 520b, 520c, and 520d. For example, the image decoding device 100 restricts the coding unit 520c located in the center among the odd number of third coding units 520b, 520c, and 520d so that it cannot be divided any further, or so that it can only be divided a set number of times.

[0092] Referring to Figure 5, the image decoding device 100 restricts the central coding unit 520c, among the odd number of third coding units 520b, 520c, and 520d contained in the non-square second coding unit 510, so that it is not further divided, or is divided in a predetermined division shape (for example, divided into only four coding units, or divided in a shape corresponding to the divided form of the second coding unit 510), or is restricted to being divided only a predetermined number of times (for example, n times, n>0). However, the above restriction regarding the central coding unit 520c is merely an embodiment and should not be interpreted as being limited to the above embodiment, but rather as including various restrictions that allow the central coding unit 520c to be decoded in a different way than the other coding units 520b and 520d.

[0093] According to one embodiment, the image decoding device 100 can acquire division shape mode information, which is currently used to divide the coding unit, at a predetermined position within the coding unit.

[0094] Figure 6 shows a method by which an image decoding device 100 determines a predetermined coding unit from an odd number of coding units, according to one embodiment.

[0095] Referring to Figure 6, the division shape mode information of the current coding units 600 and 650 can be obtained from a sample at a predetermined position among multiple samples contained in the current coding units 600 and 650 (for example, samples 640 and 690 located in the center). However, the predetermined position within the current coding unit 600 from which at least one of such division shape mode information can be obtained should not be interpreted as being limited to the central position shown in Figure 6, but rather as being able to include various positions that may be contained within the current coding unit 600 (for example, the top edge, bottom edge, left side, right side, upper left edge, lower left edge, upper right edge, lower right edge, etc.). The image decoding device 100 obtains the division shape mode information obtained from the predetermined position and decides whether to divide the current coding unit into coding units of various shapes and sizes, or not.

[0096] According to one embodiment, the image decoding device 100 selects one of the coding units when the coding unit is currently divided into a predetermined number of coding units. There are various methods for selecting one of multiple coding units, and these methods will be described later through the various embodiments below.

[0097] According to one embodiment, the image decoding device 100 divides the current encoding unit into a plurality of encoding units and determines the encoding unit at a predetermined position.

[0098] According to one embodiment, the image decoding device 100 utilizes information indicating the position of each of the odd-numbered coding units in order to determine the central coding unit from among the odd-numbered coding units. Referring to Figure 6, the image decoding device 100 divides the current coding unit 600 or the current coding unit 650 to determine the odd-numbered coding units 620a, 620b, 620c or the odd-numbered coding units 660a, 660b, 660c. The image decoding device 100 uses information regarding the positions of the odd-numbered coding units 620a, 620b, 620c or the odd-numbered coding units 660a, 660b, 660c to determine the central coding unit 620b or the central coding unit 660b. For example, the image decoding device 100 determines the central coding unit 620b by determining the positions of the coding units 620a, 620b, 620c based on information indicating the positions of predetermined samples contained in the coding units 620a, 620b, 620c. Specifically, the image decoding device 100 determines the centrally located coding unit 620b by determining the positions of coding units 620a, 620b, and 620c based on information indicating the positions of samples 630a, 630b, and 630c at the upper left corner of coding units 620a, 620b, and 620c.

[0099] According to one embodiment, the information indicating the position of the top-left sample 630a, 630b, 630c included in the coding units 620a, 620b, 620c, respectively, includes information regarding the position or coordinates of the coding units 620a, 620b, 620c within the picture. According to one embodiment, the information indicating the position of the top-left sample 630a, 630b, 630c included in the coding units 620a, 620b, 620c, respectively, includes information indicating the width or height of the coding units 620a, 620b, 620c currently included in coding unit 600, and these widths or heights correspond to information indicating the difference between coordinates within the picture of the coding units 620a, 620b, 620c. In other words, the image decoding device 100 can determine the centrally located coding unit 620b by directly using information about the position or coordinates of coding units 620a, 620b, and 620c within the picture, or by using information about the width or height of coding units corresponding to the difference values ​​between coordinates.

[0100] According to one embodiment, information indicating the position of the upper-left sample 630a of the upper coding unit 620a can be expressed as coordinates (xa, ya), information indicating the position of the upper-left sample 630b of the central coding unit 620b can be expressed as coordinates (xb, yb), and information indicating the position of the upper-left sample 630c of the lower coding unit 620c can be expressed as coordinates (xc, yc). The image decoding device 100 determines the central coding unit 620b using the coordinates of the upper-left samples 630a, 630b, and 630c contained in the coding units 620a, 620b, and 620c, respectively. For example, when the coordinates of the top-left samples 630a, 630b, and 630c are sorted in ascending or descending order, the coding unit 620b containing the coordinates (xb,yb) of the centrally located sample 630b is determined as the central coding unit among the coding units 620a, 620b, and 620c determined by the division of the currently selected coding unit 600. However, the coordinates indicating the positions of the top-left samples 630a, 630b, and 630c represent their absolute positions within the picture. Furthermore, the (dxb,dyb) coordinates, which indicate the relative position of the top-left sample 630b of the central coding unit 620b, and the (dxc,dyc) coordinates, which indicate the relative position of the top-left sample 630c of the bottom-end coding unit 620c, may also be used, based on the position of the top-left sample 630a of the top-left coding unit 620a. Furthermore, the method of determining the coding unit at a predetermined location by using the coordinates of a sample as information indicating the location of the sample included in the coding unit should not be interpreted as being limited to the method described above, but rather as a variety of arithmetic methods that can utilize the coordinates of the sample.

[0101] In one embodiment, the image decoding device 100 divides the currently encoded unit 600 into a plurality of encoded units 620a, 620b, and 620c, and selects an encoded unit from among the encoded units 620a, 620b, and 620c according to a predetermined criterion. For example, the image decoding device 100 selects an encoded unit 620b from among the encoded units 620a, 620b, and 620c that is of a different size.

[0102] According to one embodiment, the image decoding device 100 uses the (xa,ya) coordinates, which indicate the position of the upper left sample 630a of the upper end coding unit 620a; the (xb,yb) coordinates, which indicate the position of the upper left sample 630b of the central coding unit 620b; and the (xc,yc) coordinates, which indicate the position of the upper left sample 630c of the lower end coding unit 620c, to determine the width or height of each coding unit 620a, 620b, and 620c. The image decoding device 100 uses the coordinates (xa,ya), (xb,yb), and (xc,yc), which indicate the positions of the coding units 620a, 620b, and 620c, to determine the size of each coding unit 620a, 620b, and 620c. According to one embodiment, the image decoding device 100 determines the width of the upper end coding unit 620a as the width of the current coding unit 600. The image decoding device 100 determines the height of the upper coding unit 620a as yb-ya. According to one embodiment, the image decoding device 100 determines the width of the central coding unit 620b as the width of the current coding unit 600. The image decoding device 100 determines the height of the central coding unit 620b as yc-yb. According to one embodiment, the image decoding device 100 determines the width or height of the lower coding unit using the width or height of the current coding unit and the widths and heights of the upper coding unit 620a and the central coding unit 620b. Based on the determined widths and heights of the coding units 620a, 620b, and 620c, the image decoding device 100 determines a coding unit having a different size from the other coding units. Referring to Figure 6, the image decoding device 100 determines the central coding unit 620b, which has a different size from the upper coding unit 620a and the lower coding unit 620c, as the coding unit at a predetermined position. However, the process by which the aforementioned image decoding device 100 determines an encoding unit having a different size from other encoding units is merely one embodiment of determining an encoding unit at a predetermined position using the size of an encoding unit determined based on sample coordinates. Therefore, various processes can be used to determine an encoding unit at a predetermined position by comparing the sizes of encoding units determined according to predetermined sample coordinates.

[0103] The image decoding device 100 uses the (xd,yd) coordinates, which indicate the position of sample 670a at the upper left corner of the left coding unit 660a, the (xe,ye) coordinates, which indicate the position of sample 670b at the upper left corner of the central coding unit 660b, and the (xf,yf) coordinates, which indicate the position of sample 670c at the upper left corner of the right coding unit 660c, to determine the width or height of each coding unit 660a, 660b, and 660c. The image decoding device 100 uses the coordinates (xd,yd), (xe,ye), and (xf,yf), which indicate the positions of each coding unit 660a, 660b, and 660c, to determine the size of each coding unit 660a, 660b, and 660c.

[0104] According to one embodiment, the image decoding device 100 determines the width of the left coding unit 660a as xe-xd. The image decoding device 100 determines the height of the left coding unit 660a as the height of the current coding unit 650. According to one embodiment, the image decoding device 100 determines the width of the central coding unit 660b as xf-xe. The image decoding device 100 determines the height of the central coding unit 660b as the height of the current coding unit 650. According to one embodiment, the image decoding device 100 determines the width or height of the right coding unit 660c using the width or height of the current coding unit 650 and the widths and heights of the left coding unit 660a and the central coding unit 660b. Based on the determined widths and heights of the coding units 660a, 660b, and 660c, the image decoding device 100 determines a coding unit having a different size from the other coding units. Referring to Figure 6, the image decoding device 100 determines a central coding unit 660b, which has a different size from the left coding unit 660a and the right coding unit 660c, as the coding unit at a predetermined position. However, the process by which the image decoding device 100 determines a coding unit having a different size from other coding units is merely one embodiment of determining a coding unit at a predetermined position using the size of the coding unit determined based on the sample coordinates. Therefore, various processes can be used to determine a coding unit at a predetermined position by comparing the sizes of the coding units determined according to predetermined sample coordinates.

[0105] However, the sample positions considered in determining the location of the coding unit should not be interpreted as being limited to the upper left corner as described above, but rather as being able to utilize information about the location of any sample included in the coding unit.

[0106] According to one embodiment, the image decoding device 100 selects a coding unit at a predetermined position from an odd number of coding units determined by dividing the current coding unit, taking into consideration the shape of the current coding unit. For example, if the current coding unit is a non-square shape with a width longer than its height, the image decoding device 100 determines a coding unit at a predetermined position along the horizontal direction. That is, the image decoding device 100 determines one coding unit from among coding units with different positions in the horizontal direction and imposes restrictions on that coding unit. If the current coding unit is a non-square shape with a height longer than its width, the image decoding device 100 determines a coding unit at a predetermined position along the vertical direction. That is, the image decoding device 100 determines one coding unit from among coding units with different positions in the vertical direction and imposes restrictions on that coding unit.

[0107] According to one embodiment, the image decoding device 100 uses information indicating the position of each of the even-numbered coding units to determine the coding unit at a predetermined position from among the even-numbered coding units. The image decoding device 100 currently divides the coding unit (binary division) to determine the even-numbered coding units, and uses the information regarding the positions of the even-numbered coding units to determine the coding unit at the predetermined position. This specific process corresponds to the process of determining the coding unit at a predetermined position (e.g., the central position) from among the odd-numbered coding units described in Figure 6, and is therefore omitted here.

[0108] According to one embodiment, when a non-square current coding unit is divided into multiple coding units, predetermined information relating to the coding unit at a predetermined position during the division process can be used to determine the coding unit at a predetermined position among the multiple coding units. For example, the image decoding device 100 uses at least one of the block shape information and division shape mode information stored in the sample included in the central coding unit during the division process to determine the coding unit located in the center among the multiple coding units into which the current coding unit has been divided.

[0109] Referring to Figure 6, the image decoding device 100 divides the current coding unit 600 into multiple coding units 620a, 620b, and 620c based on the division shape mode information, and determines the coding unit 620b located in the center among the multiple coding units 620a, 620b, and 620c. Furthermore, the image decoding device 100 determines the coding unit 620b located in the center by considering the position where the division shape mode information is acquired. That is, the division shape mode information of the current coding unit 600 may be acquired at a sample 640 located in the center of the current coding unit 600, and if the current coding unit 600 is divided into multiple coding units 620a, 620b, and 620c based on the division shape mode information, the coding unit 620b including the sample 640 is determined as the coding unit located in the center. However, the information used to determine the coding unit located in the center is not limited to the division shape mode information, and various types of information may be used in the process of determining the coding unit located in the center.

[0110] According to one embodiment, predetermined information for identifying an encoding unit at a predetermined location is obtained from a predetermined sample included in the encoding unit to be determined. Referring to Figure 6, the image decoding device 100 can use segmentation shape mode information obtained from a sample at a predetermined location within the current encoding unit 600 (for example, a sample located in the center of the current encoding unit 600) in order to determine an encoding unit at a predetermined location (for example, an encoding unit located in the center of the multiple divided encoding units) among a plurality of encoding units 620a, 620b, and 620c determined by dividing the current encoding unit 600. That is, the image decoding device 100 can determine a sample at a predetermined location considering the block shape of the current encoding unit 600, and can also determine an encoding unit 620b among a plurality of encoding units 620a, 620b, and 620c determined by dividing the current encoding unit 600 that includes a sample from which predetermined information (for example, segmentation shape mode information) can be obtained, and impose a predetermined restriction. Referring to Figure 6, according to one embodiment, the image decoding device 100 determines a sample 640 currently located in the center of the coding unit 600 as a sample from which predetermined information can be obtained, and the image decoding device 100 can impose predetermined restrictions on the coding unit 620b containing this sample 640 during the decoding process. However, the position of the sample from which predetermined information can be obtained is not limited to the aforementioned position, but can be interpreted as any sample at any position included in the coding unit 620b determined for the purpose of imposing restrictions.

[0111] According to one embodiment, the location of a sample from which predetermined information can be obtained is determined according to the shape of the currently encoded unit 600. According to one embodiment, block shape information determines whether the shape of the currently encoded unit is square or non-square, and determines the location of a sample from which predetermined information can be obtained according to the shape. An image decoding device 100 according to one embodiment of the present invention uses at least one of the information regarding the width and the information regarding the height of the currently encoded unit to determine a sample located on a boundary that divides at least one of the width and height of the currently encoded unit in half as a sample from which predetermined information can be obtained. As another example, if the block shape information associated with the currently encoded unit indicates that it is non-square, the image decoding device 100 determines one of the samples adjacent to the boundary that divides the long side of the currently encoded unit in half as a sample from which predetermined information can be obtained.

[0112] In one embodiment, the image decoding device 100, when the current coding unit is divided into multiple coding units, utilizes division shape mode information to determine the coding unit at a predetermined position among the multiple coding units. The image decoding device 100 in one embodiment obtains the division shape mode information from a sample at a predetermined position included in the coding unit, and the image decoding device 100 divides the multiple coding units generated by the division of the current coding unit using the division shape mode information obtained from the sample at a predetermined position included in each of the multiple coding units. That is, the coding unit is recursively divided using the division shape mode information obtained from the sample at a predetermined position included in each coding unit. The recursive division process of the coding unit has been described above through Figure 5, so a detailed explanation is omitted.

[0113] An image decoding device 100 according to one embodiment divides the current encoding unit to determine at least one encoding unit, and determines the order in which the at least one encoding unit is decoded according to a predetermined block (e.g., the current encoding unit).

[0114] Figure 7 shows the order in which multiple coding units are processed when an image decoding device 100 according to one embodiment divides the current coding unit and determines multiple coding units.

[0115] In one embodiment, the image decoding device 100 determines second coding units 710a and 710b by vertically dividing the first coding unit 700 according to the division shape mode information, determines second coding units 730a and 730b by horizontally dividing the first coding unit 700, or determines second coding units 750a, 750b, 750c, and 750d by vertically and horizontally dividing the first coding unit 700.

[0116] Referring to Figure 7, the image decoding device 100 can determine the order in which the second coding units 710a and 710b, determined by dividing the first coding unit 700 vertically, are processed horizontally 710c. The image decoding device 100 determines the processing order of the second coding units 730a and 730b, determined by dividing the first coding unit 700 horizontally, in the vertical direction 730c. The image decoding device 100 processes the second coding units 750a, 750b, 750c, and 750d, determined by dividing the first coding unit 700 vertically and horizontally, according to a predetermined order (for example, a raster scan order or z scan order 750e) in which the coding units located in one row are processed before the coding units located in the next row are processed.

[0117] In one embodiment, the image decoding device 100 recursively divides the coding unit. Referring to Figure 7, the image decoding device 100 according to one embodiment divides the first coding unit 700 to determine a plurality of coding units 710a, 710b, 730a, 730b, 750a, 750b, 750c, and 750d, and recursively divides each of the determined plurality of coding units 710a, 710b, 730a, 730b, 750a, 750b, 750c, and 750d. The method for dividing the plurality of coding units 710a, 710b, 730a, 730b, 750a, 750b, 750c, and 750d may correspond to the method for dividing the first coding unit 700. As a result, the multiple coding units 710a, 710b, 730a, 730b, 750a, 750b, 750c, and 750d can each be independently divided into multiple coding units. Referring to Figure 7, the image decoding device 100 divides the first coding unit 700 vertically to determine the second coding units 710a and 710b, and then decides whether or not to independently divide each of the second coding units 710a and 710b.

[0118] According to one embodiment, the image decoding device 100 divides the left second coding unit 710a horizontally into third coding units 720a and 720b, while not dividing the right second coding unit 710b.

[0119] According to one embodiment, the processing order of the coding units is determined based on the coding unit division process. In other words, the processing order of the divided coding units is determined based on the processing order of the coding unit immediately before division. The image decoding device 100 determines the processing order of the third coding units 720a and 720b, which are determined when the left second coding unit 710a is divided, independently of the processing order of the right second coding unit 710b. Since the left second coding unit 710a is divided horizontally to determine the third coding units 720a and 720b, the third coding units 720a and 720b can be processed vertically 720c. Also, since the processing order of the left second coding unit 710a and the right second coding unit 710b corresponds to the horizontal direction 710c, the right coding unit 710b can be processed after the third coding units 720a and 720b included in the left second coding unit 710a are processed vertically 720c. The above description is intended to explain the process by which the processing order of each coding unit is determined based on the coding unit before division, and should not be interpreted as being limited to the embodiments described above. Rather, it should be interpreted as being applicable to various methods in which coding units determined by division into various forms can be processed independently according to a predetermined order.

[0120] Figure 8 shows the process by which an image decoding device 100 according to one embodiment determines that the current coding unit will be divided into an odd number of coding units when the coding units cannot be processed in a predetermined order.

[0121] In one embodiment, the image decoding device 100 determines that the current coding unit is divided into an odd number of coding units based on the acquired division shape mode information. Referring to Figure 8, the square first coding unit 800 may be divided into non-square second coding units 810a and 810b, while the second coding units 810a and 810b are each independently divided into third coding units 820a, 820b, 820c, 820d, and 820e. In one embodiment, the image decoding device 100 divides the left coding unit 810a of the second coding unit horizontally to determine a plurality of third coding units 820a and 820b, and divides the right coding unit 810b into an odd number of third coding units 820c, 820d, and 820e.

[0122] In one embodiment, the image decoding device 100 determines whether the third coding units 820a, 820b, 820c, 820d, and 820e can be processed in a predetermined order, and determines whether there are coding units that have been divided into an odd number of parts. Referring to Figure 8, the image decoding device 100 recursively divides the first coding unit 800 to determine the third coding units 820a, 820b, 820c, 820d, and 820e. Based on at least one of the block shape information and the division shape mode information, the image decoding device 100 determines whether the first coding unit 800, the second coding units 810a, 810b, or the third coding units 820a, 820b, 820c, 820d, and 820e are divided into an odd number of coding units. For example, the coding unit located on the right side of the second coding units 810a and 810b is divided into an odd number of third coding units 820c, 820d, and 820e. The order in which the multiple coding units contained in the first coding unit 800 are processed becomes a predetermined order (for example, a z-scan order 830), and the image decoding device 100 determines whether the conditions are met for the third coding units 820c, 820d, and 820e, which were determined by dividing the right-side second coding unit 810b into an odd number, to be processed according to the predetermined order.

[0123] An image decoding device 100 according to one embodiment determines whether the third coding units 820a, 820b, 820c, 820d, and 820e included in the first coding unit 800 satisfy a condition that allows them to be processed in a predetermined order, and the condition relates to whether at least one of the width or height of the second coding units 810a and 810b is divided in half along the boundary of the third coding units 820a, 820b, 820c, 820d, and 820e. For example, the third coding units 820a and 820b determined by dividing the height of the non-square left second coding unit 810a in half satisfy the condition. The boundary of the third coding units 820c, 820d, and 820e determined by dividing the right second coding unit 810b into three coding units does not divide the width or height of the right second coding unit 810b in half, so the third coding units 820c, 820d, and 820e are determined not to satisfy the condition. The image decoding device 100 determines that a scan sequence disconnection has occurred if these conditions are not met, and based on the determination result, it decides that the second coding unit 810b on the right side will be divided into an odd number of coding units. In one embodiment, when the coding unit is divided into an odd number of units, the image decoding device 100 can impose a predetermined restriction on the coding unit at a predetermined position among the divided coding units. The content of this restriction or the predetermined position has been described above through various embodiments, so a detailed explanation is omitted.

[0124] Figure 9 shows the process by which an image decoding device 100 according to one embodiment divides the first coding unit 900 and determines at least one coding unit.

[0125] In one embodiment, the image decoding device 100 divides the first coding unit 900 based on the division shape mode information acquired by the bitstream acquisition unit 110. The square first coding unit 900 is divided into four square coding units, or into multiple non-square coding units. For example, referring to Figure 9, if the first coding unit 900 is square and the division shape mode information indicates that it is divided into non-square coding units, the image decoding device 100 divides the first coding unit 900 into multiple non-square coding units. Specifically, if the division shape mode information indicates that the first coding unit 900 is divided horizontally or vertically to determine an odd number of coding units, the image decoding device 100 can divide the square first coding unit 900 into second coding units 910a, 910b, 910c determined by vertical division as an odd number of coding units, or into second coding units 920a, 920b, 920c determined by horizontal division.

[0126] An image decoding device 100 according to one embodiment determines whether the second coding units 910a, 910b, 910c, 920a, 920b, and 920c included in the first coding unit 900 satisfy the condition that they can be processed in a predetermined order, and the condition relates to whether at least one of the width and height of the first coding unit 900 is divided in half along the boundaries of the second coding units 910a, 910b, 910c, 920a, 920b, and 920c. Referring to Figure 9, the boundaries of the second coding units 910a, 910b, and 910c determined by vertically dividing the square first coding unit 900 do not divide the width of the first coding unit 900 in half, and therefore it is determined that the first coding unit 900 does not satisfy the condition that it can be processed in a predetermined order. Furthermore, the boundaries of the second coding units 920a, 920b, and 920c, which are determined by horizontally dividing the square first coding unit 900, do not divide the height of the first coding unit 900 in half. Therefore, it is determined that the first coding unit 900 does not satisfy the condition that it can be processed in a predetermined order. The image decoding device 100 determines that the failure to satisfy this condition is a disconnection in the scan order, and based on the determination result, it decides that the first coding unit 900 will be divided into an odd number of coding units. In one embodiment, when the coding unit is divided into an odd number of units, the image decoding device 100 can impose a predetermined restriction on the coding unit at a predetermined position among the divided coding units. The content of this restriction or the predetermined position has been described above through various embodiments, so a detailed explanation is omitted.

[0127] In one embodiment, the image decoding device 100 divides the first coding unit to determine coding units of various forms.

[0128] Referring to Figure 9, the image decoding device 100 divides the square first coding unit 900, the non-square first coding unit 930, or 950 into coding units of various shapes.

[0129] Figure 10 shows that in an image decoding device 100 according to one embodiment, if the non-square second coding unit determined by dividing the first coding unit 1000 satisfies predetermined conditions, the shape in which the second coding unit can be divided is limited.

[0130] In one embodiment, the image decoding device 100 decides to divide the square first coding unit 1000 into non-square second coding units 1010a, 1010b, 1020a, and 1020b based on the division shape mode information acquired by the bitstream acquisition unit 110. The second coding units 1010a, 1010b, 1020a, and 1020b are divided independently. Based on the division shape mode information associated with each of the second coding units 1010a, 1010b, 1020a, and 1020b, the image decoding device 100 decides whether to divide them into multiple coding units or not. In one embodiment, the image decoding device 100 divides the non-square left second coding unit 1010a, which was determined by dividing the first coding unit 1000 vertically, horizontally to determine the third coding units 1012a and 1012b. However, the image decoding device 100 restricts the right-side second coding unit 1010b from being divided horizontally in the same direction as the left-side second coding unit 1010a when the left-side second coding unit 1010a is divided horizontally. If the right-side second coding unit 1010b is divided in the same direction and the third coding units 1014a and 1014b are determined, the left-side second coding unit 1010a and the right-side second coding unit 1010b can be divided horizontally independently, thereby determining the third coding units 1012a, 1012b, 1014a, and 1014b. However, this is the same result as when the image decoding device 100 divides the first coding unit 1000 into four square second coding units 1030a, 1030b, 1030c, and 1030d based on the division shape mode information, which can be inefficient from the standpoint of image decoding.

[0131] In one embodiment, the image decoding device 100 divides the first coding unit 1000 horizontally to determine a non-square second coding unit 1020a or 1020b, which is then divided vertically to determine third coding units 1022a, 1022b, 1024a, and 1024b. However, if one of the second coding units (for example, the upper second coding unit 1020a) is divided vertically, the image decoding device 100 restricts the division of other second coding units (for example, the lower second coding unit 1020b) from being divided vertically in the same direction as the upper second coding unit 1020a, for the reasons mentioned above.

[0132] Figure 11 shows the process by which the image decoding device 100 divides the square coding units when, according to one embodiment, the divided shape mode information cannot indicate division into four square coding units.

[0133] In one embodiment, the image decoding device 100 divides the first coding unit 1100 based on the division shape mode information and determines the second coding units 1110a, 1110b, 1120a, 1120b, etc. The division shape mode information may include information about various forms in which the coding unit can be divided, but the information about various forms may not include information for dividing it into four square coding units. According to such division shape mode information, the image decoding device 100 cannot divide the square first coding unit 1100 into four square second coding units 1130a, 1130b, 1130c, 1130d. Based on the division shape mode information, the image decoding device 100 determines the non-square second coding units 1110a, 1110b, 1120a, 1120b, etc.

[0134] An image decoding device 100 according to one embodiment independently divides non-square second coding units 1110a, 1110b, 1120a, 1120b, etc. Each of the second coding units 1110a, 1110b, 1120a, 1120b, etc. can be divided in a predetermined order through a recursive method. This is a division method that corresponds to a method in which the first coding unit 1100 is divided based on division shape mode information.

[0135] For example, the image decoding device 100 divides the left second coding unit 1110a horizontally to determine square third coding units 1112a and 1112b, and divides the right second coding unit 1110b horizontally to determine square third coding units 1114a and 1114b. Furthermore, the image decoding device 100 may also divide both the left second coding unit 1110a and the right second coding unit 1110b horizontally to determine square third coding units 1116a, 1116b, 1116c, and 1116d. In this case, the coding units of the first coding unit 1100 can be determined in the same way as when it is divided into four square second coding units 1130a, 1130b, 1130c, and 1130d.

[0136] As another example, the image decoding device 100 vertically divides the upper second coding unit 1120a to determine square third coding units 1122a and 1122b, and vertically divides the lower second coding unit 1120b to determine square third coding units 1124a and 1124b. Furthermore, the image decoding device 100 may vertically divide both the upper second coding unit 1120a and the lower second coding unit 1120b to determine square third coding units 1126a, 1126b, 1126c, and 1126d. In this case, the coding units of the first coding unit 1100 can be determined in a similar manner to when it is divided into four square second coding units 1130a, 1130b, 1130c, and 1130d.

[0137] Figure 12 shows that, in one embodiment, the processing order between multiple coding units can change depending on the coding unit partitioning process.

[0138] In one embodiment, the image decoding device 100 divides the first coding unit 1200 based on the division shape mode information. If the block shape is square and the division shape mode information indicates that the first coding unit 1200 is divided in at least one of the horizontal and vertical directions, the image decoding device 100 divides the first coding unit 1200 to determine second coding units (e.g., 1210a, 1210b, 1220a, 1220b, etc.). Referring to Figure 12, the non-square second coding units 1210a, 1210b, 1220a, 1220b determined by dividing the first coding unit 1200 only in the horizontal or vertical direction can be divided independently based on the division shape mode information for each. For example, the image decoding device 100 divides the second coding units 1210a and 1210b, which are generated by vertically dividing the first coding unit 1200, horizontally to determine the third coding units 1216a, 1216b, 1216c, and 1216d. Similarly, it divides the second coding units 1220a and 1220b, which are generated by horizontally dividing the first coding unit 1200, vertically to determine the third coding units 1226a, 1226b, 1226c, and 1226d. The division process of these second coding units 1210a, 1210b, 1220a, and 1220b is as described above in relation to Figure 11, so a detailed explanation is omitted.

[0139] According to one embodiment, the image decoding device 100 processes coding units in a predetermined order. The features of processing coding units in a predetermined order have been described above in relation to Figure 7, so a detailed explanation is omitted. Referring to Figure 12, the image decoding device 100 divides the square first coding unit 1200 to determine four square third coding units 1216a, 1216b, 1216c, 1216d, 1226a, 1226b, 1226c, and 1226d. The image decoding device 100 according to one embodiment determines the processing order of the third coding units 1216a, 1216b, 1216c, 1216d, 1226a, 1226b, 1226c, and 1226d according to the form in which the first coding unit 1200 is divided.

[0140] According to one embodiment, the image decoding device 100 divides the second coding units 1210a and 1210b, which are generated by dividing them vertically, horizontally to determine the third coding units 1216a, 1216b, 1216c, and 1216d. The image decoding device 100 processes the third coding units 1216a and 1216c, which are contained in the left second coding unit 1210a, vertically first, and then processes the third coding units 1216b and 1216d, which are contained in the right second coding unit 1210b, vertically according to a sequence 1217.

[0141] According to one embodiment, the image decoding device 100 divides the second coding units 1220a and 1220b, which are generated by dividing them horizontally, vertically to determine the third coding units 1226a, 1226b, 1226c, and 1226d. The image decoding device 100 processes the third coding units 1226a and 1226b, which are contained in the upper second coding unit 1220a, horizontally first, and then processes the third coding units 1226c and 1226d, which are contained in the lower second coding unit 1220b, horizontally according to a sequence 1227.

[0142] Referring to Figure 12, the second coding units 1210a, 1210b, 1220a, and 1220b are each divided, determining the square third coding units 1216a, 1216b, 1216c, 1216d, 1226a, 1226b, 1226c, and 1226d. The second coding units 1210a and 1210b determined by vertical division and the second coding units 1220a and 1220b determined by horizontal division are divided in different ways, but according to the third coding units 1216a, 1216b, 1216c, 1216d, 1226a, 1226b, 1226c, and 1226d determined thereafter, the first coding unit 1200 is ultimately divided into coding units of the same shape. As a result, the image decoding device 100 can process multiple coding units determined to be the same shape in different orders, even when it recursively divides coding units through different processes based on the divided shape mode information and consequently determines coding units of the same shape.

[0143] Figure 13 shows the process by which the depth of a coding unit is determined in accordance with changes in the shape and size of the coding unit, when a coding unit is recursively divided into multiple coding units according to one embodiment.

[0144] In one embodiment, the image decoding device 100 determines the depth of the coding unit according to a predetermined criterion. For example, the predetermined criterion is the length of the long side of the coding unit. If the length of the long side of the current coding unit is divided into 2n (n>0) times the length of the long side of the coding unit before division, the image decoding device 100 determines the depth of the current coding unit as having increased by n from the depth of the coding unit before division. Hereinafter, the coding unit with increased depth will be referred to as the lower-depth coding unit.

[0145] Referring to Figure 13, according to one embodiment, based on block shape information indicating that it is a square (for example, when the block shape information indicates "0:SQUARE"), the image decoding device 100 can divide the square first coding unit 1300 to determine the lower depth second coding unit 1302, third coding unit 1304, etc. If the size of the square first coding unit 1300 is 2^N × 2^N, the second coding unit 1302, which is determined by dividing the width and height of the first coding unit 1300 by half, can have a size of N × N. Furthermore, the third coding unit 1304, which is determined by dividing the width and height of the second coding unit 1302 by half, can have a size of N / 2 × N / 2. In this case, the width and height of the third coding unit 1304 correspond to 1 / 4 of the first coding unit 1300. If the depth of the first coding unit 1300 is D, then the depth of the second coding unit 1302, which is half the width and height of the first coding unit 1300, will be D+1, and the depth of the third coding unit 1304, which is quarter the width and height of the first coding unit 1300, will be D+2.

[0146] According to one embodiment, based on block shape information indicating a non-square shape (for example, the block shape information indicates "1:NS_VER" which indicates a non-square shape where the height is longer than the width, or "2:NS_HOR" which indicates a non-square shape where the width is longer than the height), the image decoding device 100 can divide the non-square first coding unit 1310 or 1320 to determine the lower depth second coding unit 1312 or 1322, the third coding unit 1314 or 1324, etc.

[0147] The image decoding device 100 can determine a second coding unit (e.g., 1302, 1312, 1322, etc.) by dividing at least one of the width and height of the first coding unit 1310, which is N×2N in size. That is, the image decoding device 100 can divide the first coding unit 1310 horizontally to determine an N×N second coding unit 1302 or an N×N / 2 second coding unit 1322, and may further divide it horizontally and vertically to determine an N / 2×N second coding unit 1312.

[0148] In one embodiment, the image decoding device 100 may divide at least one of the width and height of the 2N×N size first coding unit 1320 to determine a second coding unit (e.g., 1302, 1312, 1322, etc.). That is, the image decoding device 100 may divide the first coding unit 1320 vertically to determine an N×N size second coding unit 1302 or an N / 2×N size second coding unit 1312, and further divide it horizontally and vertically to determine an N×N / 2 size second coding unit 1322.

[0149] In one embodiment, the image decoding device 100 may divide at least one of the width and height of the N×N size second coding unit 1302 to determine a third coding unit (e.g., 1304, 1314, 1324, etc.). That is, the image decoding device 100 divides the second coding unit 1302 vertically and horizontally to determine a third coding unit 1304 of N2×N / 2 size, a third coding unit 1314 of N / 4×N / 2 size, or a third coding unit 1324 of N / 2×N / 4 size.

[0150] In one embodiment, the image decoding device 100 may divide at least one of the width and height of the N / 2 × N size second coding unit 1312 to determine a third coding unit (e.g., 1304, 1314, 1324, etc.). That is, the image decoding device 100 may divide the second coding unit 1312 horizontally to determine an N / 2 × N / 2 size third coding unit 1304, or an N / 2 × N / 4 size third coding unit 1324, or divide it vertically and horizontally to determine an N / 4 × N / 2 size third coding unit 1314.

[0151] In one embodiment, the image decoding device 100 may determine a third coding unit (e.g., 1304, 1314, 1324, etc.) by dividing at least one of the width and height of the second coding unit 1322, which is of N × N / 2 size. That is, the image decoding device 100 may determine a third coding unit 1304 of N / 2 × N / 2 size by dividing the second coding unit 1322 vertically, or a third coding unit 1314 of N / 4 × N / 2 size by dividing it vertically and horizontally, or a third coding unit 1324 of N / 2 × N / 4 size by dividing it vertically and horizontally.

[0152] In one embodiment, the image decoding device 100 divides a square coding unit (e.g., 1300, 1302, 1304) horizontally or vertically. For example, a 2N×2N first coding unit 1300 is divided vertically to determine an N×2N first coding unit 1310, or divided horizontally to determine a 2N×N first coding unit 1320. According to one embodiment, if the depth is determined based on the length of the longest side of the coding unit, the depth of the coding unit determined by dividing the 2N×2N first coding unit 1300 horizontally or vertically may be the same as the depth of the first coding unit 1300.

[0153] In one embodiment, the width and height of the third coding unit 1314 or 1324 correspond to 1 / 4 times that of the first coding unit 1310 or 1320. If the depth of the first coding unit 1320 is D, then the depth of the second coding unit 1312 or 1322, which is 1 / 2 the width and height of the first coding unit 1310 or 1320, becomes D+1, and the depth of the third coding unit 1314 or 1324, which is 1 / 4 the width and height of the first coding unit 1310 or 1320, becomes D+2.

[0154] Figure 14 shows the depth and part index (PID) of the coding unit that can be determined based on the shape and size of the coding unit according to one embodiment.

[0155] In one embodiment, the image decoding device 100 divides a square first coding unit 1400 to determine second coding units of various shapes. Referring to Figure 14, the image decoding device 100 divides the first coding unit 1400 in at least one of the vertical and horizontal directions according to the division shape mode information to determine second coding units 1402a, 1402b, 1404a, 1404b, 1406a, 1406b, 1406c, and 1406d. That is, the image decoding device 100 determines second coding units 1402a, 1402b, 1404a, 1404b, 1406a, 1406b, 1406c, and 1406d based on the division shape mode information for the first coding unit 1400.

[0156] According to one embodiment, the depth of the second coding units 1402a, 1402b, 1404a, 1404b, 1406a, 1406b, 1406c, and 1406d, which are determined according to the division shape mode information for the square first coding unit 1400, can be determined based on the length of the longer side. For example, since the length of one side of the square first coding unit 1400 is the same as the length of the longer side of the non-square second coding units 1402a, 1402b, 1404a, and 1404b, the depth of the first coding unit 1400 and the non-square second coding units 1402a, 1402b, 1404a, and 1404b can be considered to be the same at D. In contrast, when the image decoding device 100 divides the first coding unit 1400 into four square second coding units 1406a, 1406b, 1406c, and 1406d based on the division shape mode information, the length of one side of the square second coding units 1406a, 1406b, 1406c, and 1406d is half the length of one side of the first coding unit 1400. Therefore, the depth of the second coding units 1406a, 1406b, 1406c, and 1406d is D+1, which is one level lower than the depth D of the first coding unit 1400.

[0157] In one embodiment, the image decoding device 100 divides a first coding unit 1410, which has a height longer than its width, horizontally according to the division shape mode information, and divides it into a plurality of second coding units 1412a, 1412b, 1414a, 1414b, and 1414c. In another embodiment, the image decoding device 100 divides a first coding unit 1420, which has a width longer than its height, vertically according to the division shape mode information, and divides it into a plurality of second coding units 1422a, 1422b, 1424a, 1424b, and 1424c.

[0158] According to one embodiment, the depth of the second coding units 1412a, 1412b, 1414a, 1414b, 1414c, 1422a, 1422b, 1424a, 1424b, and 1424c, which are determined according to the division shape mode information for the non-square first coding unit 1410 or 1420, is determined based on the length of the longer side. For example, since the length of one side of the square second coding units 1412a and 1412b is half the length of one side of the non-square first coding unit 1410, whose height is longer than its width, the depth of the square second coding units 1412a and 1412b is D+1, which is one level lower than the depth D of the non-square first coding unit 1410.

[0159] Furthermore, the image decoding device 100 divides the non-square first coding unit 1410 into an odd number of second coding units 1414a, 1414b, and 1414c based on the division shape mode information. The odd number of second coding units 1414a, 1414b, and 1414c include the non-square second coding units 1414a and 1414c and the square second coding unit 1414b. In this case, the length of the longer side of the non-square second coding units 1414a and 1414c and the length of one side of the square second coding unit 1414b are both half the length of one side of the first coding unit 1410. Therefore, the depth of the second coding units 1414a, 1414b, and 1414c is D+1, which is one level lower than the depth D of the first coding unit 1410. The image decoding device 100 determines the depth of the coding unit associated with the non-square first coding unit 1420, in a manner corresponding to the method for determining the depth of the coding unit associated with the first coding unit 1410.

[0160] In one embodiment, the image decoding device 100, when determining the index PID for the division of a divided coding unit, can determine the index based on the size ratio between coding units if the odd number of divided coding units are not the same size. Referring to Figure 14, among the odd number of divided coding units 1414a, 1414b, and 1414c, the central coding unit 1414b has the same width as the other coding units 1414a and 1414c, but its height may be twice that of the other coding units 1414a and 1414c. That is, in this case, the central coding unit 1414b contains two of the other coding units 1414a and 1414c. Therefore, if the index PID of the central coding unit 1414b is 1 according to the scan order, the index of the next coding unit 1414c will be 3, which is 2 higher. In other words, there is a discontinuity in the index values. In one embodiment, the image decoding device 100 determines whether the odd number of divided coding units are of the same size as each other, based on the presence or absence of discontinuities in the index for the divisions between the thus divided coding units.

[0161] In one embodiment, the image decoding device 100 determines whether a plurality of coding units, which have been divided and determined from the current coding unit, have been divided according to a specific division shape, based on the index value used to distinguish them. Referring to Figure 14, the image decoding device 100 divides a rectangular first coding unit 1410, whose height is longer than its width, to determine an even number of coding units 1412a and 1412b, or an odd number of coding units 1414a, 1414b, and 1414c. The image decoding device 100 uses an index PID that represents each coding unit to distinguish each of the plurality of coding units. According to one embodiment, the PID can be obtained from a sample at a predetermined position in each coding unit (for example, the upper left corner sample).

[0162] In one embodiment, the image decoding device 100 uses an index for the division of coding units to determine a coding unit at a predetermined position among the divided and determined coding units. According to one embodiment, if the division shape mode information for a rectangular first coding unit 1410, whose height is longer than its width, indicates that it is divided into three coding units, the image decoding device 100 divides the first coding unit 1410 into three coding units 1414a, 1414b, and 1414c. The image decoding device 100 assigns an index to each of the three coding units 1414a, 1414b, and 1414c. The image decoding device 100 compares the indices for each coding unit to determine the central coding unit among the odd number of divided coding units. Based on the coding unit indices, the image decoding device 100 can determine the coding unit 1414b, which has an index corresponding to the middle value among the indices, as the coding unit at the central position among the coding units determined by the division of the first coding unit 1410. In one embodiment, the image decoding device 100 can determine the index for the division of the divided coding units based on the size ratio between the coding units if the coding units are not the same size as each other. Referring to Figure 14, the coding unit 1414b generated by dividing the first coding unit 1410 has the same width as the other coding units 1414a and 1414c, but its height may be twice that of the other coding units 1414a and 1414c. In this case, if the index PID of the coding unit 1414b located in the center is 1, then the index of the coding unit 1414c located in the next order will be 3, which is an increase of 2. When the index increases uniformly in this way and then the amount of increase changes, the image decoding device 100 can determine that the image has been divided into multiple coding units, including coding units that have different sizes from the other coding units. According to one embodiment, if the divided shape mode information indicates that the image is divided into an odd number of coding units, the image decoding device 100 can divide the current coding unit in such a way that the coding unit at a predetermined position among the odd number of coding units (for example, the central coding unit) is of a different size from the other coding units.In this case, the image decoding device 100 uses an index PID for the coding unit to determine the central coding unit having a different size. However, the aforementioned index, the size of the coding unit at the predetermined position to be determined, or the position are specified to illustrate one embodiment and should not be interpreted as limiting to this, and it should be interpreted that various indices, coding unit positions, and sizes may be used.

[0163] An image decoding device 100 according to one embodiment utilizes a predetermined data unit from which the recursive division of the coding unit is initiated.

[0164] Figure 15 shows that, according to one embodiment, multiple encoding units are determined based on multiple predetermined data units contained in a picture.

[0165] According to one embodiment, a predetermined data unit can be defined as a data unit from which an encoded unit begins to recursively divide using the division shape mode information. That is, it can correspond to the highest-depth encoded unit used in the process of determining the multiple encoded units that currently divide the picture. Hereinafter, for the sake of explanatory convenience, such a predetermined data unit will be referred to as a reference data unit.

[0166] According to one embodiment, the reference data unit exhibits a predetermined size and shape. According to one embodiment, the reference data unit contains M × N samples, where M and N may be the same as each other, or they may be integers expressed as powers of 2. That is, the reference data unit exhibits a square or non-square shape and can subsequently be divided into an integer number of coding units.

[0167] In one embodiment, the image decoding device 100 divides the current picture into multiple reference data units. The image decoding device 100 divides the current picture into multiple reference data units using division shape mode information for each reference data unit. This division process of reference data units corresponds to a division process using a quad-tree structure.

[0168] In one embodiment, the image decoding device 100 predetermines the minimum size that a reference data unit currently contained in a picture can have. Based on this, the image decoding device 100 determines reference data units of various sizes that are larger than or equal to the minimum size, and uses the determined reference data units as a reference to determine at least one encoding unit using the segmented shape mode information.

[0169] Referring to Figure 15, the image decoding device 100 may utilize a square reference coding unit 1500 or a non-square reference coding unit 1502. According to one embodiment, the shape and size of the reference coding unit are determined according to various data units (e.g., sequence, picture, slice, slice segment, tile, tile group, maximum coding unit, etc.) that include at least one reference coding unit.

[0170] In one embodiment, the bitstream acquisition unit 110 of the image decoding device 100 acquires at least one of the following from the bitstream for each of the various data units: information regarding the shape of the reference coding unit and information regarding the size of the reference coding unit. The process for determining at least one coding unit included in the square reference coding unit 1500 has been described above through the process of dividing the current coding unit 300 in Figure 3, and the process for determining at least one coding unit included in the non-square reference coding unit 1502 has been described above through the process of dividing the current coding unit 400 or 450 in Figure 4, so a detailed explanation is omitted.

[0171] According to one embodiment, the image decoding device 100 uses an index for identifying the size and shape of a reference coding unit in order to determine the size and shape of the reference coding unit according to a subset of data units predetermined based on predetermined conditions. That is, the bitstream acquisition unit 110 acquires only the index for identifying the size and shape of the reference coding unit from the bitstream, for each slice, slice segment, tile, tile group, and maximum coding unit, as data units that satisfy predetermined conditions (for example, data units having a size of less than or equal to a slice) from among the various data units (e.g., sequence, picture, slice, slice segment, tile, tile group, maximum coding unit, etc.). By using the index, the image decoding device 100 can determine the size and shape of the reference data unit for each data unit that satisfies the predetermined conditions. When information regarding the shape and size of the reference coding unit is acquired and used from the bitstream for each relatively small data unit, the bitstream utilization efficiency may be poor. Therefore, instead of directly acquiring information regarding the shape and size of the reference coding unit, only the index can be acquired and used. In this case, at least one of the sizes and shapes of reference coding units corresponding to an index indicating the size and shape of a reference coding unit may be predetermined. That is, the image decoding device 100 can determine at least one of the sizes and shapes of reference coding units included in the data unit that serves as the basis for index acquisition by selecting at least one of the predetermined sizes and shapes of reference coding units according to the index.

[0172] In one embodiment, the image decoding device 100 utilizes at least one reference coding unit contained within a single maximum coding unit 1510. That is, the maximum coding unit that divides the image contains at least one reference coding unit, and the coding unit is determined through a recursive division process of each reference coding unit. In one embodiment, at least one of the width and height of the maximum coding unit corresponds to at least one integer multiple of the width and height of the reference coding unit. According to one embodiment, the size of the reference coding unit may be the size obtained by dividing the maximum coding unit n times according to a quad-tree structure. That is, the image decoding device 100 determines the reference coding unit by dividing the maximum coding unit n times according to a quad-tree structure, and according to various embodiments, the reference coding unit can be divided based on at least one of block shape information and division shape mode information.

[0173] An image decoding device 100 according to one embodiment acquires and utilizes block shape information indicating the current encoding unit's configuration, or division shape mode information indicating a method for dividing the current encoding unit, from a bitstream. The division shape mode information may be included in bitstreams associated with various data units. For example, the image decoding device 100 utilizes division shape mode information included in sequence parameter sets, picture parameter sets, video parameter sets, slice headers, slice segment headers, tile headers, and tile group headers. Furthermore, for each maximum encoding unit and reference encoding unit, the image decoding device 100 acquires and utilizes syntax elements corresponding to the block shape information or division shape mode information from the bitstream.

[0174] The following describes in detail a method for determining partition rules according to one embodiment of the present invention.

[0175] The image decoding device 100 determines the image segmentation rules. These segmentation rules may be predetermined between the image decoding device 100 and the image encoding device 200. The image decoding device 100 determines the image segmentation rules based on information obtained from the bitstream. The image decoding device 100 determines the segmentation rules based on information obtained from at least one of the following: sequence parameter set, picture parameter set, video parameter set, slice header, slice segment header, tile header, and tile group header. The image decoding device 100 may determine different segmentation rules depending on the frame, slice, tile, temporal layer, maximum encoding unit, or encoding unit.

[0176] The image decoding device 100 determines a division rule based on the block shape of the coding unit. The block shape includes the size, shape, width and height ratios, and orientation of the coding unit. The image coding device 200 and the image decoding device 100 predetermine, but are not limited to, determining a division rule based on the block shape of the coding unit. The image decoding device 100 determines a division rule based on information obtained from the bitstream received from the image coding device 200.

[0177] The shape of the encoded unit includes both square and non-square shapes. If the width and height of the encoded unit are the same, the image decoding device 100 determines the shape of the encoded unit to be square. If the width and height of the encoded unit are different, the image decoding device 100 determines the shape of the encoded unit to be non-square.

[0178] The size of the coding unit includes various sizes such as 4×4, 8×4, 4×8, 8×8, 16×4, 16×8, ..., 256×256, etc. The size of the coding unit can be classified based on the length of the long side, the length of the short side, or the width of the coding unit. The image decoding device 100 applies the same division rule to coding units classified into the same group. For example, the image decoding device 100 classifies coding units having the same long side length as having the same size. Also, the image decoding device 100 applies the same division rule to coding units having the same long side length.

[0179] The width-to-height ratio of the coded unit includes 1:2, 2:1, 1:4, 4:1, 1:8, 8:1, 1:16, 16:1, 32:1, or 1:32, etc. Furthermore, the orientation of the coded unit includes horizontal and vertical directions. The horizontal direction indicates when the width of the coded unit is longer than its height. The vertical direction indicates when the width of the coded unit is shorter than its height.

[0180] The image decoding device 100 can adaptively determine the division rule based on the size of the coding unit. The image decoding device 100 can determine different permitted division shape modes based on the size of the coding unit. For example, the image decoding device 100 determines whether division is permitted or not based on the size of the coding unit. The image decoding device 100 determines the division direction according to the size of the coding unit. The image decoding device 100 determines the permitted division direction according to the size of the coding unit.

[0181] Determining the division rule based on the size of the coding unit may be a division rule predetermined between the image coding device 200 and the image decoding device 100. Alternatively, the image decoding device 100 may determine the division rule based on information obtained from the bitstream.

[0182] The image decoding device 100 adaptively determines the division rule based on the position of the coding unit. The image decoding device 100 adaptively determines the division rule based on the position of the coding unit within the image.

[0183] Furthermore, the image decoding device 100 may determine a division rule such that coded units generated by different division paths do not have the same block shape. However, it is not limited to this, and coded units generated by different division paths may have the same block shape. Coded units generated by different division paths may have different decoding processing orders. The decoding processing order has been explained with reference to Figure 12, so a detailed explanation is omitted here.

[0184] Figure 16 shows an encoding unit that can be determined for each picture when the combination of shapes that can be divided into encoding units differs for each picture, according to one embodiment.

[0185] Referring to Figure 16, the image decoding device 100 determines different combinations of division shapes in which coding units can be divided for each picture. For example, the image decoding device 100 decodes an image using at least one picture contained in the image, which can be divided into four coding units (picture 1600), two or four coding units (picture 1610), and two, three or four coding units (picture 1620). To divide picture 1600 into multiple coding units, the image decoding device 100 uses only division shape information indicating that it can be divided into four square coding units. To divide picture 1610, the image decoding device 100 may use only division shape information indicating that it can be divided into two or four coding units. To divide picture 1620, the image decoding device 100 may use only division shape information indicating that it can be divided into two, three or four coding units. The aforementioned combinations of division shapes are merely embodiments for explaining the operation of the image decoding device 100, and the aforementioned combinations of division shapes should not be interpreted as being limited to the above embodiments. Rather, it should be interpreted that various combinations of division shapes can be used for each predetermined data unit.

[0186] In one embodiment, the bitstream acquisition unit 110 of the image decoding device 100 acquires a bitstream containing an index indicating a combination of division shape information for each predetermined data unit (e.g., sequence, picture, slice, slice segment, tile, or tile group). For example, the bitstream acquisition unit 110 acquires an index indicating a combination of division shape information from a Sequence Parameter Set, Picture Parameter Set, Slice Header, tile header, or tile group header. The image decoding device 100 uses the acquired index to determine combinations of division shapes that can divide the encoded unit for each predetermined data unit, thereby enabling the use of different division shape combinations for each predetermined data unit.

[0187] Figure 17 shows various forms of coding units that can be determined based on segmentation shape mode information represented by binary code according to one embodiment.

[0188] An image decoding device 100 according to one embodiment can divide an encoded unit into various forms by utilizing block shape information and division shape mode information acquired through the bitstream acquisition unit 110. The forms of the encoded unit that can be divided may include a variety of forms, including those described through the embodiments described above.

[0189] Referring to Figure 17, the image decoding device 100 divides a square coding unit in at least one of the horizontal and vertical directions based on the division shape mode information, and divides a non-square coding unit in either the horizontal or vertical direction.

[0190] In one embodiment, when the image decoding device 100 divides a square encoding unit into four square encoding units by dividing it horizontally and vertically, there can be four types of division shapes indicated by the division shape mode information for the square encoding unit. According to one embodiment, the division shape mode information is represented as a two-digit binary code, and a binary code can be assigned to each division shape. For example, when the encoding unit is not divided, the division shape mode information is represented as (00)b; when the encoding unit is divided horizontally and vertically, the division shape mode information is represented as (01)b; when the encoding unit is divided horizontally, the division shape mode information is represented as (10)b; and when the encoding unit is divided vertically, the division shape mode information is represented as (11)b.

[0191] When an image decoding device 100 according to one embodiment divides a non-square coding unit horizontally or vertically, the types of division shapes that the division shape mode information can indicate are determined according to how many coding units it divides into. Referring to Figure 17, the image decoding device 100 can divide a non-square coding unit into up to three according to one embodiment. The image decoding device 100 divides the coding unit into two coding units, in which case the division shape mode information is represented as (10)b. The image decoding device 100 divides the coding unit into three coding units, in which case the division shape mode information is represented as (11)b. The image decoding device 100 decides not to divide the coding unit, in which case the division shape mode information is represented as (0)b. That is, the image decoding device 100 can use variable length coding (VLC) instead of fixed length coding (FLC) in order to use the binary code that indicates the division shape mode information.

[0192] According to one embodiment, referring to Figure 17, the binary code of the division shape mode information indicating that the coding unit is not divided is represented as (0)b. If the binary code of the division shape mode information indicating that the coding unit is not divided is set to (00)b, then all 2 bits of the binary code of the division shape mode information must be used, even though there is no division shape mode information set to (01)b. However, as shown in Figure 17, if three types of division shapes are used for a non-square coding unit, the image decoding device 100 can determine that the coding unit is not divided even if it uses a 1-bit binary code (0)b as the division shape mode information, and thus can utilize the bitstream efficiently. However, the division shape of a non-square coding unit indicated by the division shape mode information should not be interpreted as being limited to the three types of forms shown in Figure 17, but should be interpreted as being a variety of forms, including the embodiments described above.

[0193] Figure 18 shows another form of coding unit that can be determined based on segmentation shape mode information represented by binary code according to one embodiment.

[0194] Referring to Figure 18, the image decoding device 100 divides square coding units horizontally or vertically, and non-square coding units horizontally or vertically, based on the division shape mode information. That is, the division shape mode information indicates that a square coding unit is divided in one direction. In such cases, the binary code of the division shape mode information indicating that a square coding unit is not divided is represented by (0)b. If the binary code of the division shape mode information indicating that the coding unit is not divided is set to (00)b, then all 2 bits of the binary code of the division shape mode information must be used even though there is no division shape mode information set to (01)b. However, as shown in Figure 18, if three types of division shapes are used for a square coding unit, the image decoding device 100 can determine that the coding unit is not divided even if it uses a 1-bit binary code (0)b as the division shape mode information, thus efficiently utilizing the bitstream. However, the division shape of a square coding unit indicated by the division shape mode information should not be interpreted as being limited to the three types of forms shown in Figure 18, but should be interpreted as being a variety of forms, including the embodiments described above.

[0195] According to one embodiment, block shape information or segmented shape mode information can be represented using binary code, and this information can be directly generated as a bitstream. Furthermore, block shape information or segmented shape mode information that can be represented as binary code may not be immediately generated as a bitstream, but rather used as binary code input to CABAC (context adaptive binary arithmetic coding).

[0196] According to one embodiment, the image decoding device 100 describes the process of acquiring syntax relating to block shape information or segmented shape mode information via CABAC. A bitstream containing binary code for the syntax is acquired through a bitstream acquisition unit 110. The image decoding device 100 performs inverse binary evolution on the bin strings included in the acquired bitstream to detect syntax elements that represent block shape information or segmented shape mode information. The image decoding device 100 according to one embodiment obtains a set of binary bin strings corresponding to the syntax element to be decoded, decodes each bin using probability information, and repeats this process until the bin string composed of the thus decoded bins matches one of the previously obtained bin strings. The image decoding device 100 determines the syntax element by performing inverse binary evolution on the bin strings.

[0197] According to one embodiment, the image decoding device 100 performs an adaptive binary arithmetic coding decoding process to determine the syntax for a bin string and updates the probability model for the bins acquired through the bitstream acquisition unit 110. Referring to Figure 17, the bitstream acquisition unit 110 of the image decoding device 100 acquires a bitstream showing a binary code representing segmented shape mode information according to one embodiment. Using the acquired binary code, which has a size of 1 or 2 bits, the image decoding device 100 determines the syntax for the segmented shape mode information. In order to determine the syntax for the segmented shape mode information, the image decoding device 100 updates the probability for each bit of the 2-bit binary code. That is, depending on whether the value of the first bin in the 2-bit binary code is 0 or 1, the image decoding device 100 may update the probability of having a value of 0 or 1 when decoding the next bin.

[0198] According to one embodiment, the image decoding device 100 updates the probability for a bin used in the process of decoding the bins of the binstring for the syntax during the process of determining the syntax, and determines that a specific bit of the binstring has the same probability without updating the probability.

[0199] Referring to Figure 17, in the process of determining the syntax using a binstring indicating the division shape mode information for a non-square coding unit, the image decoding device 100 determines the syntax for the division shape mode information using one bin that has a value of 0 if the non-square coding unit is not divided. That is, if the block shape information indicates that the coding unit is non-square, the first bin of the binstring for the division shape mode information is 0 if the non-square coding unit is not divided, and 1 if it is divided into two or three coding units. Thus, the probability that the first bin of the binstring for the division shape mode information for a non-square coding unit is 0 is 1 / 3, and the probability that it is 1 is 2 / 3. As mentioned above, since the division shape mode information indicating that a non-square coding unit is not divided can only be represented by a 1-bit binstring with a value of 0, the image decoding device 100 can determine the syntax for the division shape mode information by determining whether the second bin is 0 or 1 only when the first bin of the division shape mode information is 1. In one embodiment, the image decoding device 100 decodes the bins by assuming that when the first bin for the segmented shape mode information is 1, the probability of the second bin being 0 or 1 is the same for both.

[0200] In one embodiment, the image decoding device 100 utilizes a variety of probabilities for each bin in the process of determining the bins of the bin string for the segmented shape mode information. In one embodiment, the image decoding device 100 may determine different bin probabilities for the segmented shape mode information depending on the orientation of the non-square block. In one embodiment, the image decoding device 100 may determine different bin probabilities for the segmented shape mode information depending on the width or length of the longer side of the currently encoded unit. In one embodiment, the image decoding device 100 may determine different bin probabilities for the segmented shape mode information depending on at least one of the shape and length of the longer side of the currently encoded unit.

[0201] In one embodiment, the image decoding device 100 can determine the bin probabilities for the segmented shape mode information as being the same for coding units of a predetermined size or larger. For example, for coding units having a size of 64 samples or more based on the length of the long side of the coding unit, the bin probabilities for the segmented shape mode information can be determined as being the same.

[0202] In one embodiment, the image decoding device 100 determines the initial probability for the bins constituting the bin string of segmented shape mode information based on the slice type (e.g., I-slice, P-slice, or B-slice).

[0203] Figure 19 is a block diagram of an image coding and decoding system that performs loop filtering.

[0204] The encoding stage 1910 of the image encoding and decoding system 1900 transmits an encoded bitstream of an image, and the decoding stage 1950 receives the bitstream and decodes it to output a decoded image. Here, the encoding stage 1910 has a configuration similar to the image encoding device 200 described later, and the decoding stage 1950 has a configuration similar to the image decoding device 100.

[0205] In the coding stage 1910, the prediction coding unit 1915 outputs prediction data through interprediction and intraprediction, and the transformation and quantization unit 1920 outputs quantized transformation coefficients of residual data between the prediction data and the current input image. The entropy coding unit 1925 encodes and transforms the quantized transformation coefficients and outputs them as a bitstream. The quantized transformation coefficients are decoded as spatial domain data via the inverse quantization and inverse transformation unit 1930, and the decoded spatial domain data is output as a decoded image via the deblocking filtering unit 1935 and the loop filtering unit 1940. The decoded image can be used as a reference image for the next input image via the prediction coding unit 1915.

[0206] The encoded image data from the bitstream received by the decoding stage 1950 is decoded as spatial domain residual data via the entropy decoding unit 1955 and the inverse quantization and inverse transform unit 1960. The predicted data output from the prediction decoding unit 1975 and the residual data are combined to form spatial domain image data, and the deblocking filtering unit 1965 and the loop filtering unit 1970 perform filtering on the spatial domain image data and output the decoded image for the current original image. The decoded image can be used by the prediction decoding unit 1975 as a reference image for the next original image.

[0207] The loop filtering unit 1940 of the encoding stage 1910 performs loop filtering using filter information input according to user input or system settings. The filter information used by the loop filtering unit 1940 is output to the entropy encoding unit 1925 and transmitted to the decoding stage 1950 together with the encoded image data. The loop filtering unit 1970 of the decoding stage 1950 performs loop filtering based on the filter information input from the decoding stage 1950.

[0208] The various embodiments described above illustrate the operation related to the image decoding method performed by the image decoding device 100. Below, the operation of the image encoding device 200, which performs an image encoding method corresponding to the reverse process of these image decoding methods, will be described through various embodiments.

[0209] Figure 2 shows a block diagram of an image encoding device 200 that can encode an image based on at least one of block shape information and segmented shape mode information according to one embodiment.

[0210] The image encoding device 200 includes an encoding unit 220 and a bitstream generation unit 210. The encoding unit 220 receives an input image and encodes the input image. The encoding unit 220 encodes the input image and obtains at least one syntax element. The syntax element includes at least one of the following: skip flag, prediction mode, motion vector difference, motion vector prediction method (or index), transform quantized coefficient, coded block pattern, coded block flag, intra prediction mode, direct flag, merge flag, delta QP, reference index, prediction direction, and transform index. The encoding unit 220 determines a context model based on block shape information, which includes at least one of the shape, direction, width, and height ratio or size of the encoded unit.

[0211] The bitstream generation unit 210 generates a bitstream based on the encoded input image. For example, the bitstream generation unit 210 generates the bitstream by entropy encoding syntax elements based on a context model. The image encoding device 200 then transmits the bitstream to the image decoding device 100.

[0212] According to one embodiment, the encoding unit 220 of the image encoding device 200 can determine the shape of the encoding unit. For example, the encoding unit may have a square or non-square shape, and information indicating such a shape may be included in the block shape information.

[0213] According to one embodiment, the encoding unit 220 can determine in what form the encoding unit is divided. The encoding unit 220 determines the form of at least one encoding unit included in the encoding unit, and the bitstream generation unit 210 generates a bitstream that includes division shape mode information which includes information about the form of such encoding unit.

[0214] According to one embodiment, the encoding unit 220 can determine whether or not the encoding unit is divided. If the encoding unit 220 determines that the encoding unit contains only one encoding unit, or that the encoding unit is not divided, the bitstream generation unit 210 generates a bitstream that includes division shape mode information indicating that the encoding unit is not divided. Alternatively, the encoding unit 220 can divide the encoding unit into multiple encoding units, and the bitstream generation unit 210 generates a bitstream that includes division shape mode information indicating that the encoding unit is divided into multiple encoding units.

[0215] According to one embodiment, the division shape mode information may include information indicating how many coding units to divide an encoding unit into, or in which direction to divide it. For example, the division shape mode information may indicate that it is divided in at least one of the vertical and horizontal directions, or that it is not divided at all.

[0216] The image encoding device 200 determines information regarding the division shape mode based on the division shape mode of the encoding unit. The image encoding device 200 determines a context model based on at least one of the shape, orientation, width, and height ratio or size of the encoding unit. Then, based on the context model, the image encoding device 200 generates information regarding the division shape mode for dividing the encoding unit as a bitstream.

[0217] The image encoding device 200 obtains an array for associating at least one of the ratios or dimensions of the shape, orientation, width, and height of the encoding unit with an index for the context model in order to determine the context model. The image encoding device 200 obtains an index for the context model based on at least one of the ratios or dimensions of the shape, orientation, width, and height of the encoding unit in the array. The image encoding device 200 determines the context model based on the index for the context model.

[0218] The image encoding device 200 determines the context model based on block shape information, which includes at least one of the shape, orientation, width, and height ratio or size of a peripheral encoding unit adjacent to the encoding unit. The peripheral encoding unit also includes at least one encoding unit located to the lower left, left, upper left, top, upper right, right, or lower right of the encoding unit.

[0219] Furthermore, the image encoding device 200 compares the width of the upper peripheral encoding unit with the width of the encoding unit in order to determine the context model. The image encoding device 200 also compares the height of the left and right peripheral encoding units with the height of the encoding unit. The image encoding device 200 then determines the context model based on the comparison results.

[0220] Since the operation of the image encoding device 200 is similar to that of the image decoding device 100 described in Figures 3 to 19, a detailed explanation will be omitted.

[0221] Figure 20 is a block diagram showing the configuration of an image decoding device 2000 according to one embodiment.

[0222] Referring to Figure 20, the image decoding device 2000 includes an acquisition unit 2010 and a predictive decoding unit 2030. The acquisition unit 2010 shown in Figure 20 corresponds to the bitstream acquisition unit 110 shown in Figure 1, and the predictive decoding unit 2030 corresponds to the decoding unit 120 shown in Figure 1. Furthermore, the acquisition unit 2010 corresponds to the entropy decoding unit 1955 shown in Figure 19, and the predictive decoding unit 2030 corresponds to the predictive decoding unit 1975 shown in Figure 19.

[0223] In one embodiment, the acquisition unit 2010 and the predictive decoding unit 2030 may be implemented by at least one processor. In one embodiment, the acquisition unit 2010 and the predictive decoding unit 2030 operate according to instructions stored in memory.

[0224] In one embodiment, the image decoding device 2000 includes a memory for storing input / output data from the acquisition unit 2010 and the prediction decoding unit 2030. The image decoding device 2000 also includes a memory control unit for controlling the input / output of data from the memory.

[0225] In one embodiment, the acquisition unit 2010 acquires the bitstream generated as a result of image encoding.

[0226] In one embodiment, the bitstream includes the encoding result for the current block. The bitstream may include information used to decode the current block. The current block may be the largest encoding unit, encoding unit, transformation unit, or prediction unit divided from the current image to be decoded.

[0227] In one embodiment, the image decoding unit 2150 determines the current block based on block shape information and / or segmentation shape mode information contained in the bitstream corresponding to at least one level of sequence parameter set, picture parameter set, video parameter set, slice header, and slice segment header.

[0228] In one embodiment, the acquisition unit 2010 receives the bitstream from the image encoding device via the network.

[0229] In one embodiment, the acquisition unit 2010 acquires a bitstream from a data recording medium, including magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks.

[0230] In one embodiment, the acquisition unit 2010 acquires syntax elements for decoding the image from the bitstream. The values ​​corresponding to the syntax elements are included in the bitstream according to the hierarchical structure of the image.

[0231] In one embodiment, the acquisition unit 2010 acquires syntax elements by entropy decoding the bins contained in the bitstream.

[0232] In one embodiment, the bitstream includes information indicating the prediction mode of the current block in the current image. The prediction mode of the current block includes an intermode, which is a mode that predicts or decodes the current block based on a reference image in order to reduce temporal overlap between images.

[0233] In one embodiment, the prediction decoding unit 2030 performs interpretation on the current block according to the prediction mode of the current block to generate a predicted block of the current block, and decodes the current block using the predicted block.

[0234] In one embodiment, the prediction / decoding unit 2030 uses one reference image (e.g., unidirectional prediction) or two reference images (e.g., bidirectional prediction) when decoding the current block based on the reference image. Whether the current block is predicted unidirectionally or bidirectionally may be determined according to explicit information contained in the bitstream, or it may be implicitly determined from the prediction modes of the surrounding blocks related to the current block.

[0235] In one embodiment, when the current block is bipredicted, the prediction decoding unit 2030 may use motion information contained in the bitstream to determine the reference block of the current block, or it may implicitly determine it from the reference blocks of surrounding blocks associated with the current block. The motion information of the current block includes at least one of a reference image index, a motion vector, a differential motion vector, and a reference direction, and includes all information for predicting the motion vector of the current block.

[0236] In one embodiment, the prediction decoding unit 2030 determines the first and second motion vectors by using the motion vector of a previously decoded block adjacent to the current block, or a block included in a previously decoded image, as a motion vector predictor for the current block, or by using the motion vector difference, which is the difference between the motion vector of the current block and the motion vector predictor.

[0237] In one embodiment, the prediction decoding unit 2030 determines a first reference block in the first reference image and a second reference block in the second reference image for the biprediction of the current block when the current block is bipredicted. For example, the prediction decoding unit 2030 determines the first and second reference blocks for the biprediction of the current block using motion information. The prediction decoding unit 2030 may also determine the first and second reference blocks for the biprediction of the current block based on template matching.

[0238] In one embodiment, the prediction decoding unit 2030 determines a motion vector using motion information or a reference block of a surrounding block, and determines the block pointed to by the determined motion vector as at least one of the first reference block and the second reference block.

[0239] In one embodiment, the prediction decoding unit 2030 may determine at least one of the first and second reference blocks of the current block by improving the motion vector by comparing a template in a predetermined region containing the block pointed to by the determined motion vector with the current reference template. The current reference template represents the template of the current block. That is, the reference block may be determined based on motion information alone, or based on motion information and template matching. If the prediction decoding unit 2030 can define templates for the current block, the first reference block, and the second reference block, it may determine the reference block regardless of whether template matching has been performed to improve the motion vector. In one embodiment, the prediction decoding unit 2030 determines weighting information for combining the reference blocks to generate a prediction block for the current block, based on the template of the reference block and the current reference template.

[0240] In one embodiment, the weighted value candidate list usage information indicates whether or not the weighted value candidate list is used to determine the weighted value. If the weighted value candidate list usage information indicates the use of the weighted value candidate list, the prediction decoding unit 2030 may obtain weighted value information for joining reference blocks based on the determination of one weighted value candidate value from the weighted value candidate list using a weighted value index obtained from the bitstream or from a surrounding block. Alternatively, if the weighted value candidate list usage information indicates that the weighted value candidate list is not used, it may indicate whether or not the prediction decoding unit 2030 determines the weighted value information using motion information or reference blocks of the surrounding block without using the weighted value candidate list and weighted value index.

[0241] In one embodiment, the weighted value candidate list utilization information is included in the bitstream's sequence parameter set, picture parameter set, slice header, or slice data.

[0242] In one embodiment, the weighted value candidate list utilization information may not be included in the bitstream. If the weighted value candidate list utilization information is not included in the bitstream, the prediction decoding unit 2030 may obtain the weighted value candidate list and weighted value index according to a predetermined method, or it may determine the weighted value information using motion information or reference blocks of surrounding blocks without obtaining the weighted value candidate list and weighted value index.

[0243] In one embodiment, the prediction decoding unit 2030 determines whether or not to use the weighted value candidate list based on the information indicated by the information obtained from the bitstream. For example, the weighted value candidate list usage information includes a flag or index indicating that the weighted value candidate list and / or weighted value index will be used, or that the weighted value candidate list and / or weighted value index will not be used.

[0244] In one embodiment, the prediction decoding unit 2030 may obtain weighted value candidate list usage information using at least one surrounding block and the reference block of the surrounding block. The prediction decoding unit 2030 uses at least one of the weighted value candidate lists and / or weighted value indices for the reference block of the surrounding block to determine the weighted value candidate list and / or weighted value index for the reference block of the current block. For example, a pre-configured weighted value candidate list is a list with elements -2, 3, 4, 5, and 10. Alternatively, a pre-configured weighted value candidate list is a list with elements 1, 3, 4, 5, 7, 2, and 6. The weighted value candidate lists are not limited to the disclosed examples. The prediction decoding unit determines the weighted value index for the reference block of the current block as the same value as the weighted value index for the reference block of the surrounding block. The prediction and decoding unit may determine the weighted index of the current block relative to the reference block as a different value from the weighted index of the surrounding blocks by additionally obtaining the delta value for the weighted index from the bitstream or by performing template matching.

[0245] In one embodiment, there are several ways to represent weighted value information. For example, weighted value information may be determined using a single weight (e.g., using a constant w), or a left weight and an upper weight (using a constant w corresponding to the left weight). left and the constant w corresponding to the upper weight value above Weighted value information may be determined using (using) or using a weighted value filter (using pixel-wise weighted values ​​w, each with a corresponding weighted value for the samples within the block).

[0246] In one embodiment, information indicating the representation method of weighted value information is included in the bitstream sequence parameter set, image parameter set, slice header, or slice data.

[0247] In one embodiment, information indicating the representation method of weighted value information may not be included in the bitstream. In this case, the prediction decoding unit 2030 determines the weighted value information of the current block according to a predetermined method.

[0248] In one embodiment, if the prediction decoding unit 2030 uses information obtained from the bitstream to indicate the representation method of weighted value information, it can determine the weighted value information from among multiple weighted value information representation methods by the weighted value information representation method indicated by the information obtained from the bitstream. For example, the information indicating the representation method of weighted value information includes a flag or index indicating one of the multiple weighted value information representation methods.

[0249] In one embodiment, the prediction decoding unit 2030 acquires information indicating the representation method of weighted value information using at least one peripheral block. In this case, even if information indicating the representation method of weighted value information is not acquired from the bitstream, the representation method of weighted value information can be determined using at least one peripheral block. For example, if a peripheral block determines weighted value information using a single weight, the weighted value information representation method of the current block is determined to use a single weight, similar to the peripheral block.

[0250] The process for determining the weighting information for joining reference blocks will be described later with reference to Figures 24 to 29.

[0251] In one embodiment, the prediction decoding unit 2030 generates a prediction block for the current block by combining reference blocks using the determined weight value information.

[0252] In one embodiment, the prediction / decoding unit 2030 determines the predicted block as the currently decoded block.

[0253] In one embodiment, the prediction decoding unit 2030 combines the residual data acquired from the bitstream by the acquisition unit 2010 with the prediction block to generate a decoded current block.

[0254] Standards such as HEVC (High Efficiency Video Coding) and VVC (Versatile Video Coding) generate prediction blocks based on weighted information obtained from the bitstream, which can increase the amount of data required for signaling in interprediction mode. Therefore, in one embodiment, the prediction decoding unit 2030 can reduce the amount of data required for signaling by determining a reference block and using the template of the reference block to determine the weighted information.

[0255] The following describes the biprediction performed on the current block, with reference to Figures 21 and 22.

[0256] Figure 21 is a diagram illustrating the biprediction of the current block according to one embodiment.

[0257] In one embodiment, the current block 2114 included in the current image 2110 is predicted unidirectionally using the first reference image 2120 included in List 0 or the second reference image 2130 included in List 1, or it is predicted bidirectionally using the first reference image 2120 included in List 0 and the second reference image 2130 included in List 1.

[0258] In one embodiment, the prediction decoding unit 2030 determines the first reference image 2120 and the second reference image 2130 that the current block references for double prediction of the current block, and determines a first motion vector 2122 indicating a first reference block 2124 in the first reference image 2120 and a second motion vector 2132 indicating a second reference block 2134 in the second reference image 2130. Therefore, the prediction decoding unit 2030 determines the first reference block 2124 based on the first motion vector 2122, and determines the second reference block 2134 based on the second motion vector 2132.

[0259] For example, the prediction decoding unit 2030 determines the first reference image 2120 and the second reference image 2130 as the images that the current block references based on the information included in the bitstream, or determines the first reference image 2120 and the second reference image 2130 as the images that the current block 2114 references by considering the images that the peripheral blocks related to the current block 2114 reference.

[0260] In one embodiment, the prediction decoding unit 2030 may generate a motion vector candidate list by using the motion vectors of the temporal blocks that are temporally related to the current block and the spatial blocks that are spatially related to the current block in order to determine the first motion vector 2122 and the second motion vector 2132. The prediction decoding unit 2030 determines the first motion vector 2122 and the second motion vector 2132 by using the motion vector candidates indicated by the information included in the bitstream among the motion vector candidates included in the motion vector candidate list.

[0261] FIG. 22 is a diagram showing blocks that are temporally and / or spatially related to the current block 2114.

[0262] Referring to FIG. 22, a temporal block includes a collocated block (hereinafter referred to as "col block") Col that is located at a position corresponding to the current block 2114 within a reference picture having a POC (Picture Order Count) different from that of the current block 2114 of the current block 2114, and at least one block Br that is spatially adjacent to the col block Col. The block Br is located at the lower right of the current block 2114 and the col block Col. The collocated block is determined as any block within a block having a position and size corresponding to the current block within the collocated picture, or a block adjacent to a block having a position and size corresponding to the current block.

[0263] Spatial blocks that are spatially related to the current block 2114 include at least one of the outer lower left block A0, the lower left block A1, the outer upper right block B0, the upper right block B1, and the outer upper left block B2.

[0264] The positions of the temporal blocks and spatial blocks shown in FIG. (0000908) 22 are examples, and depending on the implementation example, the positions and numbers of the temporal blocks and spatial blocks can be variously changed.

[0265] In one embodiment, the prediction decoding unit 2030 directly searches for the first reference block 2124 and the second reference block 2134 used to decode the current block 2114 from the first reference picture 2120 and the second reference picture 2130, or identifies or determines them by other methods. In this case, the prediction decoding unit 2030 can identify or determine the first reference block 2124 and the second reference block 2134 in the same way as the method executed by the image encoding device 3100.

[0266] Referring again to Figure 21, in one embodiment, once the first reference block 2124 in the first reference image 2120 and the second reference block 2134 in the second reference image 2130 are determined, the prediction decoding unit 2030 combines the first reference block 2124 and the second reference block 2134 and decodes the current block 2114 based on the combined result. Here, combining the first reference block 2124 and the second reference block 2134 means linearly combining the samples contained in the first reference block 2124 and the samples contained in the second reference block 2134.

[0267] In one embodiment, the joining of the first reference block 2124 and the second reference block 2134 may be performed in different ways depending on the weighted value candidate list usage information.

[0268] In one embodiment, the weighted value candidate list includes weighted value candidates used for the biprediction of the current block. A weighted value index may be assigned to each weighted value candidate used for the biprediction of the current block. The weighted value candidates are five or three values ​​as defined in the VVC standard. For example, the weighted value candidate list may include weighted value candidate values ​​having the values ​​-2, 3, 4, 5, and 10, or weighted value candidate values ​​having the values ​​3, 4, and 5. A weighted value index from 0 to 4 may also be assigned to each weighted value candidate value. Note that the types / number of weighted value candidate values ​​and the weighted value index assigned to each candidate value are examples and can be varied in various ways within the scope obvious to those skilled in the art.

[0269] For example, the list of weighted candidate values ​​may include the weighted candidate values ​​-4, -3, -2, -1, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, or the weighted candidate values ​​1, 2, 3, 4, 5, 6, 7.

[0270] In one embodiment, the prediction decoding unit 2030 determines weight value information using a weight value candidate list and a weight value index. The weight value index is obtained from the bitstream or from the surrounding block.

[0271] In one embodiment, when weighted value information is determined using a weighted value candidate list, a linear combination of the samples included in the first reference block 2124 and the samples included in the second reference block 2134 can be performed as shown in Equation 1 below.

[0272]

number

[0273] In formula 1, w0 represents a candidate weight value corresponding to the weight value index, and w1 represents the pair value of w0. A pair value means a value obtained by applying the candidate weight value corresponding to the weight value index to a predetermined calculation formula. For example, w1 has the value of 8-w0. In some concrete examples, w1 may be the candidate weight value corresponding to the weight value index, and w0 may be the pair value of w1.

[0274] As mentioned above, when a weighted value candidate value indicated by a weighted value index is identified from among the weighted value candidate values ​​included in the weighted value candidate list, that weighted value candidate value is used as a weight for joining the first reference block and the second reference block. In other words, in one embodiment, when joining the first reference block and the second reference block for the dual prediction of the current block, the size of residual data included in the bitstream can be reduced by selecting a weighted value candidate value that can derive the joining result most similar to the current block.

[0275] Furthermore, the specific operation of determining the weight value information using motion information or reference blocks of surrounding blocks without using the weight value candidate list and weight value index, and then combining the first reference block 2124 and the second reference block 2134, will be described later with reference to Figure 26.

[0276] The templates for current blocks or referenced blocks are described below with reference to Figure 23.

[0277] Figure 23 is a diagram illustrating a template for a current block or reference block according to one embodiment.

[0278] In one embodiment, the prediction decoding unit 2030 identifies a peripheral sample adjacent to the current block 2310 in order to perform a dual prediction for the current block 2310. The peripheral sample adjacent to the current block 2310 is used as a template for generating a prediction block for the current block 2310 and may be referred to as the current reference template 2320. The first and second reference templates are templates for the first and second reference blocks, which are determined based on motion information or based on template matching, and the method for determining the reference blocks is not limited to the examples disclosed herein.

[0279] In one embodiment, the predictive decoding unit 2030 identifies or determines the current reference template 2320 of the current block 2310. The current reference template 2320 includes at least one sample from the upper sample 2330 and the left sample 2340 of the current block 2310.

[0280] Referring to FIG. 23, when the size of the current block 2310 is 8×8 currently, 16 upper samples 2330 and 20 left samples 2340 are shown as the current reference template 2320. However, the number of upper samples 2330 and left samples 2340 used for the dual prediction of the current block 2310 can be set in various ways. The left samples 2340 may or may not include the samples in the upper left outer region 2350 where the vertex touches the current block 2310.

[0281] For example, the prediction decoding unit 2030 identifies p×n (where p is a constant and n is the width of the current block 2310) upper samples 2330, and identifies m×q (where q is a constant and m is the height of the current block 2310) or (m + p)×q left samples 2340.

[0282] In one embodiment, the upper samples 2330 are shown in FIG. 23 as being arranged in two rows. However, the upper samples 2330 may be 0, or arranged in one row, or arranged in three or more rows. In one embodiment, the left samples 2340 are shown as being arranged in two columns each. However, they may be 0, or arranged in one column, or arranged in three or more columns. Also, the left samples 2340 may exclude the samples in the upper left outer region 2350, or may show only the samples in the upper left outer region 2350.

[0283] In one embodiment, the prediction decoding unit 2030 obtains a template of the same or similar shape in the same or similar manner as the current reference template 2320 for the reference block of the current block 2310, which can be referred to as the template of the reference block or the reference template. When the current block 2310 is dual predicted, the first reference template of the first reference block and the second reference template of the second reference block are identified or determined.

[0284] In one embodiment, when the current block 2310 is predicted, once the current reference template 2320, the first reference template, and the second reference template are identified or determined, the predicted block of the current block 2310 can be generated using the current reference template 2320, the first reference template, and the second reference template.

[0285] In one embodiment, the prediction decoding unit 2030 determines weighting information for combining the first reference block and the second reference block based on the current reference template 2320, the first reference template, and the second reference template. The prediction decoding unit 2030 can generate a prediction block for the current block by combining the first reference block and the second reference block using the determined weighting information.

[0286] Figure 24 illustrates the operation of determining a reference block based on template matching according to one embodiment.

[0287] In one embodiment, the prediction decoding unit 2030 determines at least one of the first reference block 2435 of the first reference image 2420 and the second reference block 2455 of the second reference image 2440 in order to perform a dual prediction for the current block 2410. For example, the prediction decoding unit 2030 determines a motion vector using motion information or reference blocks of surrounding blocks, and determines the block pointed to by the determined motion vector as at least one of the first and second reference blocks. Alternatively, the prediction decoding unit 2030 may determine at least one of the first and second reference blocks of the current block by improving the motion vector by comparing a template in a predetermined region containing the block pointed to by the determined motion vector with the current reference template.

[0288] The operation by which the prediction decoding unit 2030 determines the motion vector using motion information or reference blocks of surrounding blocks and determines the reference blocks has been described in more detail in Figures 21 and 22. Therefore, the operation by which the prediction decoding unit 2030 determines at least one of the first and second reference blocks based on template matching will be described below. Specifically, the operation by which the prediction decoding unit 2030 improves the motion vector by comparing the template in a predetermined region including the block pointed to by the determined motion vector with the current reference template and thereby determines at least one of the first and second reference blocks of the current block will be described in detail.

[0289] In one embodiment, the first reference image 2420 and the second reference image 2440 may be images that were decoded before the current image.

[0290] In template matching, a set of peripheral samples decoded prior to the current block is used as a template. In one embodiment, the current reference template 2415, which is the template of the current block, includes at least a portion of the peripheral samples shown in Figure 23.

[0291] In one embodiment, the prediction decoding unit 2030 uses the current reference template 2415 to search for the template most similar to the current reference template 2415 within the first reference image 2420, and determines the block adjacent to the first reference template 2430, which is the most similar template obtained as a result of the search, as the first reference block 2435. In Figure 24, since the current reference template 2415 is located to the left and above the current block 2410, the block located to the right and below the most similar first reference template 2430, as a result of the search within the first reference image 2420, is determined to be the first reference block 2435.

[0292] In one embodiment, the prediction decoding unit 2030 uses the current reference template 2415 to search for the template most similar to the current reference template 2415 within the second reference image 2440, and determines the block adjacent to the second reference template 2450, which is the most similar template obtained as a result of the search, as the second reference block 2455. In Figure 24, since the current reference template 2415 is located to the left and above the current block 2410, the block located to the right and below the most similar second reference template 2450, as a result of the search within the second reference image 2440, is determined to be the second reference block 2455.

[0293] In one embodiment, the difference in sample values ​​may be used to search for a template similar to the currently referenced template 2415. For example, a similar template containing the sample values ​​most similar to those contained in the currently referenced template 2415 may be determined in the first reference image 2420 and / or the second reference image 2440. In one embodiment, the template having the minimum SAD (Sum of Absolute Difference) value may be determined as the most similar template using the difference in sample values, but this is just one example, and SATD (Sum of Absolute Transformed Difference) or HoG (Histogram of Oriented Gradient) may be used, and can be varied in various ways within the bounds that are easily understood by those skilled in the art.

[0294] In one embodiment, the prediction decoding unit 2030 determines a base motion vector for determining each reference block in order to search for similar templates. For example, a predetermined motion vector (e.g., a zero vector, a vector based on information obtained from the bitstream), or the motion vector of a block at a predetermined position is determined as the base motion vector. For example, the base motion vector for determining the first reference block 2435 is the first motion vector 2122 shown in Figure 21, and the base motion vector for determining the second reference block 2455 is the second motion vector 2132 shown in Figure 21.

[0295] In one embodiment, the prediction decoding unit 2030 searches for the template most similar to the current reference template 2415 within a predetermined range centered on the point indicated by the basic motion vector. It also determines the block adjacent to the first reference template 2430, which is the template most similar to the current reference template 2415 in the first reference image 2420, as the first reference block 2435. The prediction decoding unit 2030 determines the block adjacent to the second reference template 2450, which is the template most similar to the current reference template 2415 in the second reference image 2440, as the second reference block 2455.

[0296] Figure 25 is a diagram illustrating the steps for determining weighted value information according to one embodiment.

[0297] In one embodiment, the prediction decoding unit 2030 determines weighting information for joining the first reference block and the second reference block based on the current reference template 2590, the first reference template, and the second reference template. The prediction decoding unit 2030 can determine the weighting information without using a weighting candidate list and a weighting index by utilizing the current reference template 2590, the first reference template 2510, and the second reference template 2550.

[0298] In one embodiment, the weighting information includes information about weighting values ​​for performing a linear combination of the first and second reference blocks. For example, the weighting information includes weighting values ​​applied to the first reference block and weighting values ​​applied to the second reference block.

[0299] In one embodiment, the operation to determine weighted value information includes the operation to determine a first weighted value. Furthermore, the first weighted value may be determined by a single weighted value 2530, a left-side weighted value 2532 and an upper-side weighted value 2534, or a weighted value filter.

[0300] In one embodiment, the first weighting value, which is a weighting value applied to the first reference block, may be at least one constant or matrix. For example, the first weighting value is a single weighting value applied to all first reference blocks, in which case the first weighting value is a single weighting value 2530 having a w value. The first weighting value is obtained based on at least one left-hand sample of each template w left Left-weighted value 2532 with a value, and obtained based on at least one upper sample of each template w above It includes an upper weighted value 2534 which has a value. When the first weighted value is represented as a matrix, it may be a weighted value filter with a w value. Each element of the weighted value filter is determined independently and represents the weighted value corresponding to each pixel or sample. In this invention, the matrix is ​​used to represent each sample value contained in a block, but other configurations may be used to represent each sample value contained in a block, and is not limited to a matrix.

[0301] In one embodiment, when a weight value for application to a first reference block (hereinafter referred to as the first weight value) is determined, a weight value for application to a second reference block (hereinafter referred to as the second weight value) may be determined according to the determined first weight value. For example, if the first weight value is determined as w, the second weight value is determined as Mw (where M is a constant, or a constant matrix with element values ​​M).

[0302] In some specific cases, if a second weighting value is determined, the first weighting value may be determined according to the determined second weighting value.

[0303] In one embodiment, the information indicating the representation method of the weighted value information may be information regarding whether the first weighted value is represented as a single weighted value, a left-weighted value, an upper-weighted value, or a weighted value filter. Note that the representation or determination method of the weighted value representing the first weighted value is merely illustrative and can be modified in various ways to the extent obvious to those skilled in the art.

[0304] In one embodiment, the second weighted value is determined based on the first weighted value. For example, if the first weighted value is determined as a single weighted value 2530, the second weighted value may be determined as a paired single weighted value 2570, which is one weighted value corresponding to the single weighted value 2530. If the first weighted value is determined as a left weighted value 2532 and an upper weighted value 2534, the second weighted value may be determined as a paired left weighted value 2572 and a paired upper weighted value 2574, which correspond to the left weighted value 2532 and the upper weighted value 2534, respectively. If the first weighted value is determined as a weighted value filter, the second weighted value may be determined as a paired weighted value filter.

[0305] In one embodiment, the first weighting value included in the weighting value information is determined according to formula 2.

number

[0306] In one embodiment, the prediction decoding unit 2030 determines a single weighted value 2530 to apply to the first reference block based on at least one sample of the first reference template 2510, at least one sample of the second reference template 2550, and at least one sample of the current reference template 2590.

[0307] For example, the prediction decoding unit 2030 determines a single weighted value 2530 based on the values ​​of the samples in the first reference template 2510 and the second reference template 2550 at the corresponding positions of the samples currently included in the reference template 2590. The prediction decoding unit 2030 can determine the w value by applying the values ​​of the central upper sample 2514-1 from the upper samples of the first reference template 2510, the central upper sample 2554-1 from the upper samples of the second reference template 2550, and the central upper sample 2594-1 from the upper samples of the current reference template 2590 to the formula 2. Note that the positions of the samples used to determine the w value are not limited to the disclosed examples.

[0308] For example, the prediction decoding unit 2030 determines a single weighted value 2530 based on the values ​​of two or more samples currently included in the reference template 2590, and the corresponding values ​​of the samples included in the first reference template 2510 and the second reference template 2550. When determining the first weighted value using multiple samples included in each template, the single weighted value 2530 can be determined according to formula 3.

number

[0309]

number

[0310] According to Equation 3, the prediction decoding unit 2030 applies the values ​​of the first samples 2512-1 and 2514-1 included in the first reference template 2510, the second samples 2552-1 and 2554-1 included in the second reference template 2550, and the current samples 2592-1 and 2594-1 included in the current reference template 2590 to Equation 3 to determine the w value. Note that the positions of the samples used to determine the w value are not limited to the corresponding positions in each template; two or more samples may be used in each template, or all samples in the template may be used to determine the w value.

[0311] For example, if the single weighted value 2530 has been determined, then the single weighted value 2532 and the upper weighted value 2534 are used to determine the single weighted value 2532 and the upper weighted value 2534. The single weighted value 2530 is determined as the average of the left weighted value 2532 and the upper weighted value 2534, or as a weighted average proportional to the number of samples for the left weighted value 2532 and the number of samples for the upper weighted value 2534. In one example, if each template contains only left-side samples of a block or only upper-side samples, the single weighted value 2530 may be the same as the left weighted value 2532 or the upper weighted value 2534, respectively.

[0312] The following describes the process for determining the left-weighted value 2532 and the upper-weighted value 2534.

[0313] In one embodiment, the predictive decoding unit 2030 is a left-weighted value 2532 to apply to the first reference block based on at least one sample of the first reference template 2510, at least one sample of the second reference template 2550, and at least one sample of the current reference template 2590. left , and the upper weighted value is 2534 w above This determines the weighting. The left-weighted value of 2532 is the weighting applied to the lower-left sample area of ​​the first reference block. The upper-weighted value of 2534 is the weighting applied to the upper-right sample area of ​​the first reference block. Note that applying weightings to a specific area, block, or sample indicates that the weighting values ​​can be used to generate predicted blocks for that specific area.

[0314] In one embodiment, the lower left sample region of the first reference block represents the lower left triangular region obtained by dividing the first reference block by a diagonal line connecting the upper left vertex and the lower right vertex of the first reference block. The upper right sample region of the first reference block represents the upper right triangular region obtained by dividing the first reference block by a diagonal line connecting the upper left vertex and the lower right vertex of the first reference block.

[0315] In one embodiment, the left-weighted value 2532 is determined based on at least one sample from the left-side samples of the first reference template 2510, at least one sample from the left-side samples of the second reference template 2550, and at least one sample from the left-side samples of the current reference template 2590. The upper-weighted value 2534 is determined based on at least one sample from the upper-side samples of the first reference template 2510, at least one sample from the upper-side samples of the second reference template 2550, and at least one sample from the upper-side samples of the current reference template 2590.

[0316] For example, the left-weighted value 2532 is determined based on one sample from the left-side samples of the first reference template 2510, one sample from the left-side samples of the second reference template 2550, and one sample from the left-side samples of the current reference template 2590. The upper-weighted value 2534 is determined based on one sample from the upper-side samples of the first reference template 2510, one sample from the upper-side samples of the second reference template 2550, and one sample from the upper-side samples of the current reference template 2590. For example, the left-weighted value 2532 is determined in the same way as the method used to determine w by applying the upper-left sample 2512-1 from the left-side samples of the first reference template 2510, the upper-left sample 2552-1 from the left-side samples of the second reference template 2550, and the upper-left sample 2592-1 from the left-side samples of the current reference template 2590 to Formula 2. The upper weighted value 2534 is determined in the same manner as the method used to determine w by applying Formula 2 to the upper center sample 2514-1 from the upper samples of the first reference template 2510, the upper center sample 2554-1 from the upper samples of the second reference template 2550, and the upper center sample 2594-1 from the upper samples of the current reference template 2590. The sample positions used to determine the left weighted value 2532 and the sample positions used to determine the upper weighted value 2534 are not limited to the disclosed examples.

[0317] For example, the prediction decoding unit 2030 determines the left-weighted value 2532 and the upper-weighted value 2534 based on the values ​​of the left-weighted and upper-weighted values ​​of the first reference template 2510 at positions corresponding to two or more left-weighted samples of the current reference template 2590 and two or more upper-weighted samples of the second reference template 2550.

[0318] For example, the prediction decoding unit 2030 determines the left weight value 2532 based on the values ​​of multiple samples contained in one row every 2, 4, 8, or K rows of the left-hand samples of the current reference template 2590, multiple samples of the first reference template corresponding to the multiple samples contained in one row every 2, 4, 8, or K rows of the left-hand samples, and multiple samples of the second reference template. The prediction decoding unit 2030 determines the upper weight value 2534 based on the values ​​of multiple samples contained in one column every 2, 4, 8, or K columns of the upper-hand samples of the current reference template 2590, multiple samples of the first reference template corresponding to the multiple samples contained in one column every 2, 4, 8, or K columns of the upper-hand samples, and multiple samples of the second reference template. Note that K is a pre-set value and is determined according to the size of the current block.

[0319] For example, the prediction decoding unit 2030 performs linear regression analysis on multiple left-hand samples from the first reference template 2510, multiple left-hand samples from the second reference template 2550, and multiple left-hand samples from the current reference template 2590 to determine K. The linear regression analysis determines a value that minimizes the difference between the combined value of the left-hand samples from the first reference template 2510 and the second reference template 2550 and the left-hand sample value of the current reference template 2590. In this case, w is a left-weighted value of 2532. left This can be determined using the same method as the method used to determine w in equation 3.

[0320] For example, the prediction decoding unit 2030 determines the value by performing linear regression analysis on multiple samples from the upper samples of the first reference template 2510, multiple samples from the upper samples of the second reference template 2550, and multiple samples from the upper samples of the current reference template 2590. The value determined by the linear regression analysis is the value obtained by minimizing the difference between the combined value of the upper samples of the first reference template 2510 and the upper samples of the second reference template 2550 and the upper sample value of the current reference template 2590. In this case, the upper weighted value is 2534. above This can be determined using the same method as the method used to determine w in equation 3.

[0321] For example, the prediction / decoding unit 2030 performs linear regression analysis on the odd-numbered rows of the left-hand samples of the first reference template 2510, the odd-numbered rows of the left-hand samples of the second reference template 2550, and the odd-numbered rows of the left-hand samples of the current reference template 2590 to determine the left-weighted value 2532. The positions of the samples used to determine the left-weighted value 2532 may be determined based on the size of the template, and the positions or number of samples used to determine the left-weighted value 2532 are not limited to the disclosed examples. In one example, the positions or number of samples used to determine the upper-weighted value 2534 are also determined based on the size of the template.

[0322] In one embodiment, the prediction decoding unit 2030 obtains a left-weighted matrix corresponding to the left sample of the first reference template 2510 based on the left sample of the first reference template 2510, the left sample of the second reference template 2550, and the left sample of the current reference template 2590. The prediction decoding unit 2030 may also obtain an upper-weighted matrix corresponding to the upper sample of the first reference template 2510 based on the upper sample of the first reference template 2510, the upper sample of the second reference template 2550, and the upper sample of the current reference template 2590. For example, the prediction decoding unit 2030 applies the values ​​of the left sample of the current reference template 2590, the left sample of the first reference template 2510, and the left sample of the second reference template 2550 to Equation 2 to determine the values ​​of all elements of the left-weighted matrix. The prediction decoding unit 2030 applies the values ​​of the upper samples of the current reference template 2590, the first reference template 2510, and the second reference template 2550 to Equation 2 to determine the values ​​of all elements in the upper weighted matrix.

[0323] The following describes the process of determining weighting information using the left-weighted matrix and the upper-weighted matrix.

[0324] For example, the prediction decoding unit 2030 determines the average of all or some of the elements of the left-weighted matrix as the left-weighted value 2532. The prediction decoding unit 2030 determines the average of all or some of the elements of the upper-weighted matrix as the upper-weighted value 2534. In addition, the prediction decoding unit 2030 determines the average of all or some of the elements of the left-weighted matrix and the upper-weighted matrix as the single-weighted value.

[0325] In one embodiment, the prediction decoding unit 2030 uses at least one of the left weight matrix and the upper weight matrix to determine a weight filter that indicates the weight corresponding to each sample included in the first reference block, to be applied to each sample in the first reference block.

[0326] The process of determining the weighted filter based on the left-weighted matrix or the upper-weighted matrix will be explained below with reference to Figure 26.

[0327] Figure 26 is a diagram illustrating the operation of determining a weighted filter according to one embodiment.

[0328] In one embodiment, the prediction decoding unit 2030 determines weighting information for combining the first reference block and the second reference block based on the current reference template, the first reference template, and the second reference template. Referring to Figure 25, the operation of determining the weighting information includes the operation of determining a first weight, the first weight may be a weighting filter 2600.

[0329] In one embodiment, the prediction decoding unit 2030 obtains a left-weighted matrix or an upper-weighted matrix based on multiple samples of the first reference template, multiple samples of the second reference template, and multiple samples of the current reference template. For example, an upper-weighted matrix is ​​obtained based on the upper samples of the first reference template, the upper samples of the second reference template, and the upper samples of the current reference template. The upper-weighted matrix may be a matrix determined in correspondence with the upper samples of the first reference template. For example, if the prediction decoding unit 2030 identifies p × n upper samples of the first reference template (where p is a constant and n is the width of the weighted matrix 2600), the prediction decoding unit 2030 obtains an upper-weighted matrix that is an r × n matrix (where r is a natural number less than or equal to p).

[0330] In one embodiment, the prediction decoding unit 2030 obtains a left-weighted matrix based on the left-side samples of the first reference template, the left-side samples of the second reference template, and the left-side samples of the current reference template. The left-weighted matrix may be a matrix determined in correspondence with the left-side samples of the first reference template. For example, if the prediction decoding unit 2030 identifies m × q left-side samples of the first reference template (where q is a constant and m is the height of the weighted filter 2600), the prediction decoding unit 2030 obtains a left-weighted matrix that is an m × s matrix (where s is a natural number less than or equal to q).

[0331] Note that the sizes of each template, left-weighted matrix, upper-weighted matrix, and weighted filter are not limited to the examples disclosed.

[0332] In one embodiment, the prediction decoding unit 2030 uses at least one of the left weight matrix and the upper weight matrix to determine a weight filter 2600 to apply to each sample included in the first reference block. The weight filter 2600 provides information about the weights corresponding to each sample included in the first reference block.

[0333] In one embodiment, the weighted value filter 2600 is determined based on at least one column in the left weighted value matrix and / or at least one row in the upper weighted value matrix. A single sample value included in the weighted value filter 2600 is determined based on at least one element in one column of the upper weighted value matrix corresponding to the position of the sample value and at least one element in one row of the left weighted value matrix corresponding to the position of the sample value.

[0334] In one embodiment, the weighted value filter 2600 is determined based on one column in the left-weighted value matrix and at least one row in the upper-weighted value matrix. For example, one column in the left-weighted value matrix is ​​column 2630 of the left-weighted value matrix that is closest to the weighted value filter 2600. One row in the upper-weighted value matrix is ​​row 2620 of the upper-weighted value matrix that is closest to the weighted value filter 2600. For example, the value of the first sample 2610 is determined based on the first left element 2631 and the first upper element 2621. The first left element 2631 is the element of the left-weighted value matrix that is closest to the weighted value filter 2600 and is included in column 2630 of the left-weighted value matrix, and corresponds to the position of the first sample 2610. The first upper element 2621 is included in row 2620 of the upper weighted matrix that is closest to the weighted filter 2600, and is an element of the upper weighted matrix corresponding to the position of the first sample 2610. The value of the first sample 2610 is determined as the average of the first left element 2631 and the first upper element 2621, or it may be determined by considering the ratio of the distance between the first sample 2610 and the first left element 2631 to the distance between the first sample 2610 and the first upper element 2621. For example, if the distance between the first sample 2610 and the first left element 2631 is twice the distance between the first sample 2610 and the first upper element 2621, the prediction decoding unit 2030 determines the value of the first sample 2610 as a value obtained by multiplying the value of the first left element 2631 by 1 / 3 and the value obtained by multiplying the value of the first upper element 2621 by 2 / 3. The method for determining the value of the first sample 2610 based on the first left element 2631 and the first upper element 2621 is not limited to the disclosed example. Furthermore, the values ​​of the samples in the weighted value filter 2600 may be determined using the same or a different method as the method used to determine the value of the first sample 2610.

[0335] In one embodiment, each sample value included in the weighted value filter 2600 is determined based on at least one element in each column of the upper weighted value matrix and at least one element in each row of the left weighted value matrix. That is, the weighted value filter 2600 is determined based on at least one element in each column of the upper weighted value matrix and at least one element in each row of the left weighted value matrix.

[0336] For example, the value of the first sample 2610 included in the weighted value filter 2600 is determined as the average of the representative value of the first upper column corresponding to the column in the upper weighted value matrix corresponding to the position of the first sample 2610, and the representative value of the first left row corresponding to the row in the left weighted value matrix. The representative value of the first upper column is determined based on at least one element included in the first upper column. The representative value of the first left row is determined based on at least one element included in the first left row. The representative value of the first upper column is the average of at least one element included in the first upper column, and the representative value of the first left row may be the average of at least one element included in the first left row. If the values ​​of all samples included in the weighted value filter 2600 are determined in the same manner as the value of the first sample 2610, the weighted value filter 2600 may be determined based on the average of at least one element included in each row of the left weighted value matrix and the average of at least one element included in each column of the upper weighted value matrix. Furthermore, the values ​​of the samples within the weighted filter 2600 may be determined using the same or a different method as the method used to determine the value of the first sample 2610.

[0337] For example, the value of the first sample 2610 included in the weighted value filter 2600 is determined by multiplying the representative value of the first upper column and the representative value of the first left row by a predetermined ratio, taking into account the distance between the first sample 2610 and the upper weighted value matrix and the distance between the first sample 2610 and the left weighted value matrix. If the ratio of the distance between the first sample 2610 and the upper weighted value matrix and the distance between the first sample 2610 and the left weighted value matrix is ​​2:3, the value of the first sample 2610 can be determined by multiplying the representative value of the first upper column by 0.4 and the representative value of the first left row by 0.6. Note that the method for determining the value of the first sample 2610 based on the representative value of the first upper column and the representative value of the first left row is not limited to the disclosed example, and the values ​​of samples in the weighted value filter 2600 may be determined by the same or different method as the method used to determine the value of the first sample 2610.

[0338] In one embodiment, the weighted value filter 2600 may be determined based on applying a predetermined ratio to at least one element in each row of the left-weighted value matrix and to the elements in each column of the upper-weighted value matrix. For example, the value of the first sample 2610 is determined by the representative value of the first upper column and the representative value of the first left-hand row.

[0339] The representative value of the first upper column is determined by the value of at least one element of the first upper element 2621 and the second upper element 2622, which are elements included in the first upper column of the upper weighted matrix. The representative value of the first upper column may be determined as the value of the first upper element 2621 or the value of the second upper element 2622, or as the average value of the first upper element 2621 and the second upper element 2622. The representative value of the first upper column may be determined by applying different ratios to the first upper element 2621 and the second upper element 2622. The said different ratios may be predetermined. Samples that are geographically closer to the first sample 2610 may be determined by applying higher ratios. For example, the representative value of the first upper column may be the sum of the values ​​obtained by multiplying the first upper element 2621 and the second upper element 2622 by 0.5, or it may be the sum of the values ​​obtained by multiplying the first upper element 2621, which is positionally close to the first sample 2610, by 0.7 and the second upper element 2622 by 0.3. The method for determining the representative value of the first upper column is not limited to the disclosed example.

[0340] The representative value of the first left-hand row is determined based on the value of at least one of the first left-hand element 2631 and the second left-hand element 2632, which are elements included in the first left-hand row of the left-weighted matrix. The representative value of the first left-hand row may be determined by the value of the first left-hand element 2631 or the value of the second left-hand element 2632, or by the mean value of the first left-hand row. The representative value of the first left-hand row may also be determined by applying different ratios to the first left-hand element 2631 and the second left-hand element 2632. The said different ratios may be predetermined. For example, samples that are geographically closer to the first sample 2610 may be determined by applying a higher ratio. For example, the representative value of the first left row may be the sum of the values ​​obtained by multiplying the first left element 2631 and the second left element 2632 by 0.5, or it may be the sum of the values ​​obtained by multiplying the first left element 2631 that is positionally close to the first sample 2610 by 0.7 and the second left element 2632 by 0.3. Note that the method for determining the representative value of the first upper column is not limited to the disclosed examples.

[0341] In one embodiment, if each template contains only upper samples, the upper weighted matrix is ​​obtained, and based on this, the sample values ​​for the columns in the weighted filter 2600 that are in the same position as the upper weighted matrix can be determined based on at least one element in one of the columns of the upper weighted matrix. For example, if each template contains only upper samples, the upper weighted matrix is ​​obtained. The value of the first sample 2610 can be determined based on the value of at least one element of the first upper element 2621 and the second upper element 2622 in the upper weighted matrix located in the same column as the first sample 2610. For example, the prediction decoding unit 2030 determines the value of the first upper element 2621 or the value of the second upper element 2622 as the value of all samples located in the same column as the first sample 2610 in the weighted filter 2600. Within the weighted filter 2600, the value of a sample located in the same column as the first sample 2610 is determined as the average of the first upper element 2621 and the second upper element 2622. Within the weighted filter 2600, the value of a sample located in the same column as the first sample 2610 may be determined by applying different ratios to the first upper element 2621 and the second upper element 2622.

[0342] For example, if each template contains only left-hand samples, a left-hand weighted matrix is ​​obtained, and based on at least one element in one row of the left-hand weighted matrix, the sample value for the row corresponding to the left-hand weighted matrix is ​​determined in the weighted filter 2600. For example, if each template contains only left-hand samples, a left-hand weighted matrix is ​​obtained. The value of the first sample 2610 may be determined based on the value of at least one element of the first left-hand element 2631 and the second left-hand element 2632 in the left-hand weighted matrix located in the same row as the first sample 2610. For example, the prediction decoding unit 2030 determines the value of the first left-hand element 2631 or the value of the second left-hand element 2632 as the sample value for the row corresponding to the first sample 2610 in the weighted filter 2600. Within the weighted filter 2600, the sample value for the row corresponding to the first sample 2610 is determined as the average value of the first left-hand element 2631 and the second left-hand element 2632. Within the weighted value filter 2600, the values ​​of samples located in the same row as the first sample 2610 can be determined by applying different ratios to the first left element 2631 and the second upper element 2632.

[0343] In one embodiment, the prediction decoding unit 2030 determines the value of at least one sample included in the weighted value filter 2600 using the same or a different method as the method used to determine the value of the first sample 2610.

[0344] In one embodiment, the prediction decoding unit 2030 can determine the values ​​of some of the samples included in the weighted value filter 2600 using the same or a different method as the method used to determine the value of the first sample 2610, and then determine the values ​​of the samples in the weighted value filter 2600 whose values ​​have not yet been determined, based on the values ​​of the determined samples.

[0345] Figure 27 is a diagram illustrating the operation of generating prediction blocks using weighted value information according to one embodiment.

[0346] In one embodiment, once the first reference block 2710 and the second reference block 2730 are determined and weighted value information is determined, the prediction decoding unit 2030 can generate a prediction block 2750 for the current block by combining the first reference block 2710 and the second reference block 2730 using the weighted value information. For example, the prediction decoding unit 2030 generates the prediction block 2750 by applying the first weighted value determined with reference to Figures 25 and 26 to the first reference block 2710 and applying the second weighted value determined based on the first weighted value to the second reference block 2730. Note that the operation of applying the second weighted value to the second reference block 2730 corresponds to the operation of applying the first weighted value to the first reference block 2710 and can therefore be omitted.

[0347] In one embodiment, combining the first reference block 2710 and the second reference block 2730 using weighted value information can be performed according to the following formula 5.

number

[0348] In one embodiment, the prediction decoding unit 2030 can generate a prediction block 2750 for the current block by combining the first reference block 2710 and the second reference block 2730 using a single weight value. For example, the prediction decoding unit 2030 identifies a single weight value as a first weight value to be applied to the first reference block 2710. The prediction decoding unit 2030 can use the single weight value to obtain a pair of single weight values, which are second weight values ​​to be applied to the second reference block 2730. The prediction decoding unit 2030 can generate a prediction block 2750 for the current block by applying the first weight value to the first reference block 2710 and the second weight value 2746 to the second reference block 2730.

[0349] For example, if a single weighted value w is determined, the prediction decoding unit 2030 identifies the first weighted value as w and determines the second weighted value as Mw. The prediction decoding unit 2030 generates a prediction block 2750 for the current block by adding the value obtained by multiplying each sample of the first reference block 2710 by w and the value obtained by multiplying each sample of the second reference block 2730 by Mw.

[0350] In one embodiment, the prediction decoding unit 2030 can generate a prediction block 2750 for the current block by combining the first reference block 2710 and the second reference block 2730 using the left weight value 2722 and the upper weight value 2724. For example, the prediction decoding unit 2030 identifies the left weight value 2722 and the upper weight value 2724 as first weight values ​​to apply to the first reference block 2710. The prediction decoding unit 2030 uses the first weight values, the left weight value 2722 and the upper weight value 2724, to obtain a pair of left weight values ​​2742 and a pair of upper weight values ​​2744 as second weight values ​​to apply to the second reference block 2730. The prediction decoding unit 2030 generates a prediction block 2750 for the current block by applying the left-weighted value 2722 and the upper-weighted value 2724 to the first reference block 2710, and the paired left-weighted value 2742 and the paired upper-weighted value 2744 to the second reference block 2730.

[0351] For example, when the prediction decoding unit 2030 determines that the left weighted value is w which is 2722 left and the upper weighted value is w which is 2724 above it identifies the first weighted values as w left and w above and determines the second weighted values as the pair left weighted value M - w which is 2742 left and the pair upper weighted value M - w which is 2744 above The prediction decoding unit 2030 determines the values of the respective samples in the lower left sample region 2752 of the prediction block 2750 for the current block by adding the value obtained by multiplying each sample in the lower left sample region 2712 of the first reference block 2710 by w left and the value obtained by multiplying each sample in the lower left sample region 2732 of the second reference block 2730 by M - w left Also, the prediction decoding unit 2030 determines the values of the respective samples in the upper right sample region 2754 of the prediction block 2750 for the current block by adding the value obtained by multiplying each sample in the upper right sample region 2714 of the first reference block 2710 by w above and the value obtained by multiplying each sample in the upper right sample region 2734 of the second reference block 2730 by M - w above The prediction decoding unit 2030 generates the prediction block 2750 using the determined lower left sample region 2752 and upper right sample region 2754 of the prediction block 2750

[0352] Note that in the first reference block 2710, a sample 2716 located on the diagonal line connecting the upper left vertex and the lower right vertex of the first reference block 2710 can be identified. For the sample 2716 on the diagonal line, either w left is applied or w above is applied, and in some cases, the average value of w left and w above is applied. Also, for the sample 2736 on the diagonal line in the second reference block 2730, M - w left and M - w aboveThe values ​​obtained using are applied as weighted values, and a detailed explanation is omitted as it corresponds to the explanation regarding sample 2716 on the diagonal within the first reference block 2710. Note that, referring to Figure 27, the left-weighted and upper-weighted values ​​are shown as part of a matrix, but for the sake of explanation, they are shown as matrices corresponding to the lower-left sample area 2712 and the upper-right sample area 2714 of the first reference block 2710, respectively, and the left-weighted value 2722 and upper-weighted value 2724 may be constants. In some implementations, once the left-weighted value 2722 and upper-weighted value 2724 are determined, they can be implemented in a similar form to a weighted value filter and applied to the first reference block 2710.

[0353] Other embodiments using left-side weighting 2722 and upper-side weighting 2724 will be described later with reference to Figure 28.

[0354] In one embodiment, the prediction decoding unit 2030 can generate a prediction block 2750 for the current block by combining the first reference block 2710 and the second reference block 2730 using a weighted value filter 2726. For example, the prediction decoding unit 2030 identifies the weighted value filter 2726 as a first weight to be applied to the first reference block 2710. The prediction decoding unit 2030 can use the weighted value filter 2726 to obtain a second weight to be applied to the second reference block, which is a paired value. The prediction decoding unit 2030 can generate a prediction block 2750 for the current block by applying the first weight to the first reference block 2710 and the second weight to the second reference block 2730.

[0355] For example, when the weighted value filter 2726 W is determined, the prediction decoding unit 2030 identifies the first weighted value as W and determines the second weighted value as MW. The prediction decoding unit 2030 can generate a prediction block 2750 for the current block by adding the value obtained by multiplying the element value of W at the position corresponding to each sample value in the first reference block 2710 by the value obtained by multiplying the element value of MW at the position corresponding to each sample value in the second reference block 2730. In other words, the operation of applying the first weighted value W to the first reference block and the operation of applying the second weighted value MW to the second reference block can be an element-wise product operation.

[0356] In one embodiment, the prediction decoding unit 2030 can generate a prediction block 2750 for the current block by combining the first reference block 2710 and the second reference block 2730 using weighted value information. The prediction decoding unit 2030 determines the prediction block as the decoded current block. Alternatively, the prediction decoding unit 2030 generates a decoded current block by combining the prediction block with residual data acquired from the bitstream by the acquisition unit 2010.

[0357] Figure 28 is a diagram illustrating the specific process for determining weighted value information according to one embodiment.

[0358] In one embodiment, the prediction decoding unit 2030 determines weighting information for combining the first reference block and the second reference block based on the current reference template, the first reference template, and the second reference template. Referring to Figure 25, the operation of determining the weighting information includes the operation of determining the first weight, and the operation of determining the first weight may include the operation of determining the left weight and the upper weight, or the operation of determining the weight filter.

[0359] In one embodiment, the prediction decoding unit 2030 determines each weight to be applied to the samples included in the first reference block based on the left weight and the upper weight. For convenience of explanation, each weight to be applied to the samples included in the first reference block based on the left weight and the upper weight is shown as a weight filter and referred to as the weight filter 2800. The prediction decoding unit 2030 can apply different ratios of the left weight and the upper weight to each sample included in the weight filter 2800 based on the distance from the diagonal line connecting the upper left vertex and the lower right vertex.

[0360] For example, the prediction decoding unit 2030 determines the value of at least one sample included in the first sample region 2810, which is furthest from the diagonal line connecting the upper left vertex and the lower right vertex (hereinafter referred to as the diagonal) and located to the lower left of the diagonal, as a left-weighted value w left The predictive decoding unit 2030 determines the value of at least one sample included in the 9th sample region 2890, which is furthest from the diagonal and located to the upper right of the diagonal, as the upper weighted value w. above The predictive decoding unit 2030 identifies the second sample region 2820, the third sample region 2830, and the fourth sample region 2840 based on the distance from the diagonal for the sample located to the lower left of the diagonal. The predictive decoding unit 2030 determines the value of at least one sample contained in the second sample region 2820.

number

number

number

number

number

number

number

[0361] In one embodiment, the prediction decoding unit 2030 determines the respective weight values ​​to apply to the samples included in the first reference block based on the values ​​of some of the samples in the weighted filter. For example, the value of one sample included in the first to ninth sample regions 2810, 2820, 2830, 2840, 2850, 2860, 2870, 2880, and 2890 is determined as the value of at least one sample included in each sample region. Taking the second sample region as an example, the prediction decoding unit 2030 determines the value of a representative sample 2825, which is a sample included in the second sample region 2820, according to the method for determining the value of the first sample in the weighted filter shown in Figure 25. The prediction decoding unit 2030 determines the determined value of the representative sample 2825 as the value of the sample included in the second sample region 2820. The prediction decoding unit 2030 can determine the value of at least one sample contained in the first to ninth sample regions 2810, 2820, 2830, 2840, 2850, 2860, 2870, 2880, and 2890 in a similar manner to how it determined the value of at least one sample in the second sample region 2820 based on the value of the representative sample 2825.

[0362] Figure 29 is a diagram illustrating the specific process for determining a weighted filter according to one embodiment.

[0363] In one embodiment, the prediction decoding unit 2030 determines weighting information for combining the first reference block and the second reference block based on the current reference template 2990, the first reference template 2910, and the second reference template 2950. The current reference template 2990, the first reference template 2910, and the second reference template 2950 correspond to the current reference template 2590, the first reference template 2510, and the second reference template 2550 in Figure 25, respectively. Referring to Figure 25, the operation of determining the weighting information includes the operation of determining the first weighting value. Furthermore, the operation of determining the first weighting value may include the operation of determining a single weighting value, the operation of determining the left weighting value and the upper weighting value, or the operation of determining a weighting value filter.

[0364] In one embodiment, the prediction decoding unit 2030 determines a single weighted value based on the values ​​of the sample currently included in the reference template 2990, the sample included in the first reference template 2910, and the sample included in the second reference template 2950 at the corresponding position.

[0365] For example, the prediction decoding unit 2030 determines a single weighted value based on the values ​​of the first current sample 2994-1, which is a sample contained in the current reference template 2990, the first upper reference sample 2914-1, which is a sample contained in the first reference template 2910 at the corresponding position, and the second upper reference sample 2954-1, which is a sample contained in the second reference template 2950. Referring to Equation 2 in Figure 25, if we assume that the value of M is 8 (hereinafter, we will assume that the value of M is always 8),

number

[0366] For example, the prediction / decoding unit 2030 determines a single weighted value by performing a linear regression analysis using the values ​​of all samples currently included in the reference template 2990, all samples included in the first reference template 2910 at the corresponding location, and all samples included in the second reference template 2950. The prediction / decoding unit 2030 can then determine the value of w as 3.94 by substituting each sample value into equation 3 in Figure 25.

[0367] For example, the prediction / decoding unit 2030 determines a single weighted value by performing a linear regression analysis using the values ​​of multiple samples contained in odd-numbered rows of the left-hand samples currently included in the reference template 2990 (the area indicated by dots in 2990 in Figure 29), multiple samples contained in the first reference template 2910 at positions corresponding to multiple samples contained in odd-numbered columns of the upper samples (the area indicated by dots in 2910 in Figure 29), and multiple samples contained in the second reference template 2950 (the area indicated by dots in 2950 in Figure 29). The prediction / decoding unit 2030 can determine the value of w as 3.7 by substituting each sample value into equation 3 in Figure 25.

[0368] In one embodiment, the prediction decoding unit 2030 determines the left weight based on at least one sample from the left-side samples of the first reference template 2910, at least one sample from the left-side samples of the second reference template 2950, ​​and at least one sample from the left-side samples of the current reference template 2990. The prediction decoding unit 2030 determines the upper weight based on at least one sample from the upper-side samples of the first reference template 2910, at least one sample from the upper-side samples of the second reference template 2950, ​​and at least one sample from the upper-side samples of the current reference template 2990.

[0369] For example, the prediction decoding unit 2030 determines the upper weight value based on the values ​​of the first upper reference sample 2914-1, which is included in the upper sample of the first reference template 2910 at the position corresponding to the first current sample 2994-1 included in the upper sample of the current reference template 2990, and the second upper reference sample 2954-1, which is included in the upper sample of the second reference template 2950. Referring to Equation 2 in Figure 25,

number

number

[0370] In one embodiment, the single weighted value is determined based on the left weighted value and the upper weighted value. For example, the left weighted value is w left The value is 4, and the upper weighted value is w above If the value of is 2, the single-weighted value of w is determined to be 3, which is the average of the left-weighted value and the upper-weighted value.

[0371] In one embodiment, the prediction decoding unit 2030 uses at least one of the left weight matrix and the upper weight matrix to determine a weight filter that indicates the weight corresponding to each sample included in the first reference block, to be applied to each sample in the first reference block.

[0372] For the sake of explanation, an example has been shown where the template of the current block or reference block includes both the upper and left-hand samples. However, the template is not limited to this example; it may include only the upper sample or only the left-hand sample.

[0373] The specific process for determining the weighted filter will be explained below with reference to Figure 30.

[0374] Figure 30 is a diagram illustrating the specific process for determining weighted value information according to one embodiment.

[0375] In one embodiment, the prediction decoding unit 2030 obtains a left-weighted matrix or an upper-weighted matrix based on multiple samples of a first reference template, multiple samples of a second reference template, and multiple samples of the current reference template.

[0376] For example, the prediction decoding unit 2030 uses the specific examples of the current reference template 2990, the first reference template 2910, and the second reference template 2950 shown in Figure 29 to obtain the left-weighted matrix 3010 corresponding to the left-side samples of the first reference template and the upper-weighted matrix 3030 corresponding to the upper-side samples of the first reference template.

[0377] In one embodiment, the prediction decoding unit 2030 uses at least one of the left-weighted matrix 3010 and the upper-weighted matrix 3030 to determine a weighted filter 3050 to apply to each sample included in the first reference block. The weighted filter 3050 is determined based on at least one column in the left-weighted matrix and / or at least one row in the upper-weighted matrix. A single sample value included in the weighted filter 3050 is determined based on at least one element in one column of the upper-weighted matrix corresponding to the position of the sample value and at least one element in one row of the left-weighted matrix corresponding to the position of the sample value.

[0378] In one embodiment, the weighted value filter 3050 may be determined based on the column 3015 of the left-weighted value matrix 3010 that is closest to the weighted value filter 3050, and the row 3035 of the upper-weighted value matrix 3030 that is closest to the weighted value filter 3050. For example, a first sample 3051, which is one sample included in the weighted value filter 3050, is determined by the average of the values ​​of a first left element 3016, which corresponds to the position of the first sample 3051, among the elements in the column 3015 of the left-weighted value matrix 3010 that is closest to the weighted value filter 3050, and a first upper element 3036, which corresponds to the position of the first sample 3051, among the elements in the row 3035 of the upper-weighted value matrix 3030 that is closest to the weighted value filter 3050. In this case, the value of the first sample 3051 may be determined as 5.2. The values ​​of all samples included in the weighted value filter 3050 can be determined by the same method used to determine the value of the first sample 3051. By determining the values ​​of all samples, the prediction decoding unit 2030 can determine the weighted value filter 3050 that shows the weighted value corresponding to each sample included in the first reference block.

[0379] Note that the weighted filter shown in Figure 30 is just one example of a weighted filter and is not limited to the disclosed example.

[0380] Figure 31 is a flowchart of an image decoding method according to one embodiment.

[0381] In step S3110, the image decoding device 2000 determines the second reference block in the first reference image and the second reference block in the second reference image for bi-prediction of the current block.

[0382] In one embodiment, the image decoding device 2000 determines the first reference block in the first reference image and the second reference block in the second reference image for dual prediction of the current block, using the decoded motion information.

[0383] In one embodiment, the image decoding device 2000 determines at least one of the first and second reference blocks using motion information or reference blocks of the surrounding blocks. For example, the image decoding device 2000 determines a motion vector using motion information or reference blocks of the surrounding blocks, and determines the block pointed to by the determined motion vector as at least one of the first and second reference blocks.

[0384] In one embodiment, the image decoding device 2000 can determine at least one of the first and second reference blocks of the current block by comparing a template in a predetermined region including the block pointed to by the determined motion vector with the current reference template and improving the motion vector. For example, the image decoding device 2000 determines at least one of the first reference block in the first reference image and the second reference block in the second reference image based on template matching. The first and second reference images may be images that were decoded before the current image. In template matching, a set of surrounding samples decoded before the current block in the current image is used as a template, and the template of the current image may be called the "current reference template".

[0385] In one embodiment, the template most similar to the current reference template is searched for within the first and second reference images. The search range is a predetermined range centered on the points indicated by the first and second motion vectors, which are the basic motion vectors for determining the respective reference blocks within the first and second reference images. Within the search range of the first reference image, the template most similar to the current reference template is determined as the first reference block, and within the search range of the second reference image, the template most similar to the current reference template is determined as the second reference block.

[0386] In step S3120, the image decoding device 2000 determines weighting information for joining the first reference block and the second reference block based on the current reference template of the current block, the first reference template of the first reference block, and the second reference template of the second reference block.

[0387] In one embodiment, the weighted value information is information for joining the first reference block and the second reference block based on the current reference template 2590, the first reference template 2510, and the second reference template 2550.

[0388] In one embodiment, the weighting information includes information about weighting values ​​for performing a linear combination of a first reference block and a second reference block. For example, the weighting information includes information about a first weighting value applied to the first reference block and a second weighting value applied to the second reference block. If the first weighting value is determined, the second weighting value may be determined according to the determined first weighting value. In some embodiments, if the second weighting value is determined, the first weighting value may be determined according to the determined second weighting value.

[0389] In one embodiment, the first weighted value may be a single weighted value applied in common to the samples contained in the first reference block. The first weighted value is determined based on at least one sample from the first reference template, at least one sample from the second reference template, and at least one sample from the current reference template. For example, the single weighted value is determined by performing a linear regression analysis on at least one sample from the first reference template, at least one sample from the second reference template, and at least one sample from the current reference template, minimizing the difference between the combined value of the first and second reference templates and the value of the current reference template. The single weighted value may be determined based on left-weighted and upper-weighted values.

[0390] In one embodiment, the first weighted value includes a left-weighted value obtained based on at least one left-side sample of each template and an upper-weighted value obtained based on at least one upper-side sample of each template. The left-weighted value is for application to the lower-left sample area of ​​the first reference block, and the upper-weighted value is for application to the upper-right sample area of ​​the first reference block. The left-weighted value and the upper-weighted value are determined based on at least one sample of the first reference template, at least one sample of the second reference template, and at least one sample of the current reference template.

[0391] For example, the left-weighted value is determined based on one sample from the left-side samples of the first reference template, one sample from the left-side samples of the second reference template, and one sample from the left-side samples of the current reference template. The left-weighted value is determined by performing a linear regression analysis on multiple samples from the left-side samples of the first reference template, multiple samples from the left-side samples of the second reference template, and multiple samples from the left-side samples of the current reference template, minimizing the difference between the combined value of the left-side samples of the first and second reference templates and the value of the left-side sample of the current reference template. The left-weighted value is determined as the mean of the elements of the left-weighted matrix corresponding to the left-side sample of the first reference template, obtained based on the left-side samples of the first reference template, the left-side samples of the second reference template, and the left-side samples of the current reference template.

[0392] For example, the upper weighted value is determined based on one sample from the upper samples of the first reference template, one sample from the upper samples of the second reference template, and one sample from the upper samples of the current reference template. The upper weighted value is determined by performing a linear regression that minimizes the difference between the combined value of the upper samples of the first and second reference templates and the value of the upper sample of the current reference template, for multiple samples from the upper samples of the first reference template, multiple samples from the upper samples of the second reference template, and multiple samples from the upper samples of the current reference template. The upper weighted value may be determined as the average of the elements of the upper weighted value matrix corresponding to the upper sample of the first reference template, obtained based on the upper samples of the first reference template, the second reference template, and the current reference template.

[0393] In one embodiment, the first weighted value is a weighted value filter represented as a matrix that can be applied differently depending on the samples contained in the first reference block. The weighted value filter uses at least one of the left weighted value matrix and the upper weighted value matrix to indicate the weights corresponding to each sample contained in the first reference block for application to each sample contained in the first reference block.

[0394] For example, a weighted filter may be determined based on one column in the left-weighted matrix and one row in the upper-weighted matrix. A weighted filter may be determined based on the mean of at least one element in each row of the left-weighted matrix and the mean of at least one element in each column of the upper-weighted matrix. A weighted filter may be determined by applying a predetermined ratio to at least one element in each row of the left-weighted matrix and the elements in each column of the upper-weighted matrix.

[0395] In step S3130, the image decoding device 2000 can generate a predicted block for the current block by combining the first reference block and the second reference block using weighted value information.

[0396] In one embodiment, weighting information can be used to join a first reference block and a second reference block. For example, the first and second reference blocks can be joined by applying the first weighting determined in step S3120 to the first reference block and the second weighting determined based on the first weighting to the second reference block.

[0397] In one embodiment, if a single weighted value is identified as the first weighted value, the first and second reference blocks can be joined by adding the value obtained by multiplying the single weighted value determined for each sample of the first reference block by the value obtained by multiplying the paired single weighted value determined for each sample of the second reference block.

[0398] In one embodiment, a left-weighted value and an upper-weighted value are identified as the first weighted value. In this case, the first and second reference blocks can be joined by adding the value obtained by multiplying the sample contained in the lower-left sample area of ​​the first reference block by the left-weighted value and the value obtained by multiplying the sample contained in the lower-left sample area of ​​the second reference block by the paired left-weighted value, and then adding the value obtained by multiplying the sample contained in the upper-right sample area of ​​the first reference block by the upper-right sample area of ​​the second reference block by the paired upper-weighted value.

[0399] In one embodiment, when a weighted value filter is identified as the first weighted value, the first and second reference blocks can be joined by adding the value obtained by multiplying each sample value of the first reference block by the element of the weighted value filter at the corresponding position, and the value obtained by multiplying each sample value of the second reference block by the element of the paired weighted value filter at the corresponding position.

[0400] In one embodiment, a predicted block for the current block is generated by combining a first reference block and a second reference block using weighted value information.

[0401] In step S3140, the image decoding device 2000 decodes the current block using the predicted block.

[0402] In one embodiment, the image decoding device 2000 determines the predicted block as the currently decoded block.

[0403] In one embodiment, the image decoding device 2000 combines residual data acquired from the bitstream with the predicted block to generate a decoded current block.

[0404] Figure 32 is a block diagram showing the configuration of an image encoding device according to one embodiment.

[0405] Referring to Figure 32, the image coding device 3200 includes a predictive coding unit 3210 and a generation unit 3230.

[0406] In one embodiment, the predictive coding unit 3210 and the generation unit 3230 can be implemented by at least one processor. In one embodiment, the predictive coding unit 3210 and the generation unit 3230 operate according to at least one instruction stored in at least one memory.

[0407] The image coding device 3200 includes at least one memory for storing input and output data from the predictive coding unit 3210 and the generation unit 3230. The image coding device 3200 also includes a memory control unit for controlling the input and output of data from the memory.

[0408] In one embodiment, the predictive coding unit 3210 corresponds to the predictive coding unit 1915 shown in Figure 19, and the generation unit 3230 corresponds to the entropy coding unit 1925 shown in Figure 19.

[0409] In one embodiment, the predictive coding unit 3210 determines the prediction mode for the current block in the current image. The prediction mode for the current block includes an intermode. An intermode is a mode that predicts or decodes the current block based on a reference image in order to reduce temporal overlap between images. The current block may be the largest coding unit, coding unit, transformation unit, or prediction unit divided from the current image to be coded.

[0410] In one embodiment, the prediction mode of the current block is determined as the intermode. The prediction coding unit 3210 performs interpretation on the current block according to the prediction mode of the current block, and encodes the current block using the prediction block generated as a result of the interpretation.

[0411] In one embodiment, the predictive coding unit 3210 uses either one reference image (e.g., unidirectional prediction) or two reference images (e.g., bidirectional prediction) when coding the current block based on the reference image. Whether the current block is unidirectionally predicted or bidirectionally may be included in the bitstream as a flag or index. If whether the current block is unidirectionally predicted or bidirectionally is implicitly determined from the prediction mode of the surrounding blocks associated with the current block, information about the surrounding blocks may be included in the bitstream as a flag or index.

[0412] In one embodiment, the predictive coding unit 3210 may identify motion information to be used to determine the reference block of the current block when the current block is bipredicted. The motion information of the current block is included in the bitstream. The motion information of the current block includes at least one of a reference image index, a motion vector, a differential motion vector, and a reference direction, and includes all information for predicting the motion vector of the current block.

[0413] In one embodiment, the predictive coding unit 3210 determines the first and second motion vectors by using the motion vectors of previously coded blocks adjacent to the current block, or blocks included in previously coded images, as motion vector predictors for the current block, or by using the motion vector difference, which is the difference between the motion vector of the current block and the motion vector predictor. In one embodiment, if the current block is bipredicted, the predictive coding unit 3210 determines the first reference block in the first reference image and the second reference block in the second reference image for the biprediction of the current block. For example, the predictive coding unit 3210 determines the first and second reference blocks for the biprediction of the current block using motion information. Alternatively, the predictive coding unit 3210 may determine the first and second reference blocks for the biprediction of the current block based on template matching.

[0414] In one embodiment, the predictive coding unit 3210 determines a motion vector using motion information or a reference block coded prior to the current block, and determines the block pointed to by the determined motion vector as at least one of the first reference block and the second reference block.

[0415] In one embodiment, the predictive coding unit 3210 can determine at least one of the first and second reference blocks of the current block by improving the motion vector by comparing a template in a predetermined region including the block pointed to by the determined motion vector with the current reference template. The current reference template represents the template of the current block. That is, the reference block may be determined based on motion information alone, or based on motion information and template matching. If the predictive coding unit 3210 can define the templates of the current block, the first reference block, and the second reference block, the predictive coding unit 3210 can perform template matching to determine the reference block regardless of whether motion vector improvement has been performed.

[0416] In one embodiment, the predictive coding unit 3210 determines weighting information for combining reference blocks in order to generate a predictive block for the current block, based on the reference block template and the current reference template.

[0417] In one embodiment, the weighted value candidate list utilization information may be information indicating a method for determining the weighted value information. The weighted value information may be obtained based on the determination of a single weighted value candidate value using the weighted value index in the weighted value candidate list, or it may be determined independently of the weighted value candidate list or the weighted value index using motion information or reference blocks of surrounding blocks.

[0418] In one embodiment, information indicating the use of the weighted value candidate list is included in the bitstream's sequence parameter set, picture parameter set, slice header, or slice data.

[0419] In one embodiment, the predictive coding unit 3210 may obtain a weighted value candidate list and a weighted value index according to a predetermined scheme, or it may determine the weighted value information using motion information or reference blocks of surrounding blocks without obtaining a weighted value candidate list and a weighted value index. In this case, the information using the weighted value candidate list may not be included in the bitstream.

[0420] In one embodiment, the predictive coding unit 3210 determines weighted value candidate list usage information indicating whether or not to use the weighted value candidate list. The weighted value candidate list usage information is included in the bitstream. The weighted value candidate list usage information includes a flag or index indicating whether or not to use the weighted value candidate list.

[0421] In one embodiment, the predictive coding unit 3210 determines whether or not to use the weighted value candidate list by using the weighted value candidate list usage information derived from one of the reference blocks. The predictive decoding unit determines a motion vector indicating the reference block used to derive the weighted value candidate list usage information in order to determine whether or not to use the weighted value candidate list.

[0422] In one embodiment, information indicating the representation method of weighted value information is included in the bitstream sequence parameter set, image parameter set, slice header, or slice data.

[0423] In one embodiment, the predictive coding unit 3210 determines the weighting information of the current block according to a previously determined scheme. In this case, information indicating the representation method of the weighting information may not be included in the bitstream.

[0424] In one embodiment, the predictive coding unit 3210, when selecting a representation method for weighted information to determine weighted information, can select the weighted information representation method from among multiple weighted information representation methods that causes the least cost to encode the current block. The information indicating the selected weighted information representation method includes a flag or index indicating one of the weighted information representation methods from among the multiple weighted information representation methods.

[0425] In one embodiment, the predictive coding unit 3210 can determine at least one of the reference blocks in the current screen that were coded before the current block, or a reference block in a previous reference screen that was coded before the current block, when the representation method for weighted value information is derived from a reference block. The predictive coding unit 3210 determines the representation method for weighted value information derived from at least one of the reference blocks as the representation method for weighted value information of the current block. The predictive decoding unit 2030 determines a motion vector indicating the reference block.

[0426] In one embodiment, encoding the current block means the process by which an image decoding device generates information that makes the current block decodeable. The information generated by encoding is included in the bitstream.

[0427] In one embodiment, when a predicted block is generated by a double prediction for the current block, the predictive coding unit 3210 can use the predicted block to encode the current block.

[0428] In one embodiment, the predictive coding unit 3210 generates residual data corresponding to the difference between the predicted block and the current block. If the predicted block is determined to be the current block, residual data may not be generated.

[0429] The generation unit 3230 generates a bitstream containing the encoding result of the image. The bitstream contains the encoding result for the current block.

[0430] In one embodiment, the generation unit 3230 transmits the bitstream to the image decoding device 2000 via a network.

[0431] In one embodiment, the generation unit 3230 records a bitstream on a data recording medium, which includes magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks.

[0432] The generation unit 3230 generates a bitstream containing syntax elements generated through image encoding. The values ​​corresponding to the syntax elements are included in the bitstream according to the hierarchical structure of the image.

[0433] The bit string generated by the generation unit 3230 by entropy encoding the syntax elements may be included in the bitstream.

[0434] In one embodiment, the bitstream includes motion information of the current block in the current image.

[0435] In one embodiment, the bitstream includes at least one of the following: motion information for determining whether the current block is bipredicted, information for using a weighted value candidate list, or information indicating a representation of the weighted value information.

[0436] The following describes the steps by which the predictive coding unit 3210 determines the first reference block in the first reference screen and the second reference block in the second reference screen.

[0437] In one embodiment, the predictive coding unit 3210 can determine at least one of the first reference block of the first reference screen and the second reference block of the second reference screen based on motion information, or based on motion information and template matching, in order to perform a dual prediction for the current block. In one embodiment, the first reference image 2420 and the second reference image 2440 may be images encoded before the current image.

[0438] In one embodiment, the predictive coding unit 3210 determines at least one of the first and second reference blocks for the current block based on motion information. Referring to Figure 21, the predictive coding unit 3210 determines the first reference image 2120 and the second reference image 2130 that the current block refers to for dual prediction of the current block, and determines a first motion vector 2122 that points to the first reference block 2124 in the first reference image 2120 and a second motion vector 2132 that points to the second reference block 2134 in the second reference image 2130. Thus, the predictive coding unit 3210 determines the first reference block 2124 based on the first motion vector 2122 and the second reference block 2134 based on the second motion vector 2132.

[0439] In template matching, the surrounding sample set encoded before the current block is used as the template.

[0440] In one embodiment, the predictive coding unit 3210 determines at least one of the first and second reference blocks for the current block based on motion information and template matching. Referring to Figure 21, the predictive coding unit 3210 uses the current reference template 2415 to search for the template most similar to the current reference template 2415 in the first reference image 2420, and determines the block adjacent to the first reference template 2430, which is the most similar template obtained as a result of the search, as the first reference block 2435. If the current reference template 2415 is located to the left and above the current block 2410, the predictive coding unit 3210 determines the block located to the right and below the most similar first reference template 2430 in the first reference image 2420 as the first reference block 2435.

[0441] In one embodiment, the predictive coding unit 3210 uses the current reference template 2415 to search for the template most similar to the current reference template 2415 within the second reference image 2440, and determines the block adjacent to the second reference template 2450, which is the most similar template obtained as a result of the search, as the second reference block 2455. If the current reference template 2415 is located to the left and above the current block 2410, the predictive coding unit 3210 determines the block located to the right and below the most similar second reference template 2450, based on the search results within the second reference image 2440, as the second reference block 2455.

[0442] In one embodiment, the difference in sample values ​​may be used to search for a template similar to the currently referenced template 2415. For example, a similar template containing the sample values ​​most similar to those contained in the currently referenced template 2415 may be determined in the first reference image 2420 and / or the second reference image 2440.

[0443] In one embodiment, the template having the minimum SAD (Sum of Absolute Difference) value can be determined as the most similar template by utilizing the difference value of the sample values. However, this is just one example, and SATD (Sum of Absolute Transformed Difference) or HoG (Histogram of Oriented Gradient) may also be used, and the method can be modified in various ways within a range easily understood by those skilled in the art.

[0444] In one embodiment, the predictive coding unit 3210 determines a base motion vector for determining each reference block in order to search for similar templates. For example, a predetermined motion vector (e.g., the zero vector) or the motion vector of a block at a predetermined position is determined as the base motion vector. For example, the base motion vector for determining the first reference block 2435 is the first motion vector 2122 shown in Figure 21, and the base motion vector for determining the second reference block 2455 is the second motion vector 2132 shown in Figure 21.

[0445] In one embodiment, the predictive coding unit 3210 searches for the template most similar to the current reference template 2415 within a predetermined range centered on the point pointed to by the basic motion vector. It also determines the block adjacent to the first reference template 2430, which is the template most similar to the current reference template 2415 in the first reference image 2420, as the first reference block 2435. The predictive coding unit 3210 determines the block adjacent to the second reference template 2450, which is the template most similar to the current reference template 2415 in the second reference image 2440, as the second reference block 2455.

[0446] If the predictive coding unit 3210 determines at least one of the first reference block 2435 or the second reference block 2455 by template matching, the predictive decoding unit 2030 can also determine at least one of the first reference block 2435 or the second reference block 2455 by template matching.

[0447] In one embodiment, the predictive coding unit 3210 determines a first motion vector 2122 representing a first reference block 2435 and a second motion vector 2132 representing a second reference block 2455. Information representing the first motion vector 2122 and the second motion vector 2132 may be included in the bitstream.

[0448] In one embodiment, the information indicating the first motion vector 2122 includes information indicating one of the candidates for the first motion vector 2122. The information indicating the first motion vector 2122 includes the motion vector difference between the selected candidate for the first motion vector 2122 and the first motion vector 2122 that points to the first reference block 2435.

[0449] In one embodiment, the information indicating the second motion vector 2132 includes information indicating one of the candidates for the second motion vector 2132. The information indicating the second motion vector 2132 includes the motion vector difference between the selected candidate for the second motion vector 2132 and the second motion vector 2132 that points to the second reference block 2455.

[0450] In one embodiment, the predictive coding unit 3210 may determine a first reference block 2435 and / or a second reference block 2455 pointed to by a motion vector previously determined in the picture. Here, the predetermined first motion vector 2122 and the predetermined second motion vector 2132 represent motion vectors predetermined between the image coding device 3200 and the image decoding device 2000.

[0451] The following describes the steps by which the predictive coding unit 3210 determines weighted value information based on the current reference template, the first reference template, and the second reference template.

[0452] In one embodiment, the predictive coding unit 3210 determines weighting information for combining the first reference block and the second reference block based on the current reference template 2590, the first reference template, and the second reference template. The predictive coding unit 3210 can determine the weighting information without using a weighting candidate list and a weighting index by utilizing the current reference template 2590, the first reference template 2510, and the second reference template 2550.

[0453] In one embodiment, the weighting information includes information about weighting values ​​for performing a linear combination of the first and second reference blocks. For example, the weighting information includes weighting values ​​applied to the first reference block and weighting values ​​applied to the second reference block.

[0454] In one embodiment, the operation for determining weighted value information includes the operation for determining a first weighted value. Furthermore, the first weighted value may be determined by a single weighted value 2530, a left weighted value 2532 and an upper weighted value 2534, or a weighted value filter. Information indicating the representation method of weighted value information is information regarding whether the operation for determining the first weighted value represents an operation for determining a single weighted value 2530, an operation for determining a left weighted value 2532 and an upper weighted value 2534, or an operation for determining a weighted value filter.

[0455] In one embodiment, the first weighting value, which is a weighting value applied to the first reference block, may be at least one constant or matrix. For example, the first weighting value is a single weighting value applied to all first reference blocks, in which case the first weighting value is a single weighting value 2530 having a w value. The first weighting value is obtained based on at least one left-hand sample of each template w left Left-weighted value 2532 with a value, and obtained based on at least one upper sample of each template w above It includes an upper weighted value of 2534, which has a value. If the first weighted value is represented as a matrix, it can be a weighted filter with a w value. Each element of the weighted filter is determined independently and represents the weighted value corresponding to each pixel or sample.

[0456] In one embodiment, the predictive coding unit 3210 determines a single weighted value 2530 to apply to the first reference block based on at least one sample of the first reference template 2510, at least one sample of the second reference template 2550, and at least one sample of the current reference template 2590.

[0457] In one embodiment, the predictive coding unit 3210 is a left-weighted value 2532 to apply to the first reference block based on at least one sample of the first reference template 2510, at least one sample of the second reference template 2550, and at least one sample of the current reference template 2590. left , and the upper weighted value is 2534 w above Determine the following. The left-side weighted value of 2532 is the weight to be applied to the lower left sample area of ​​the first reference block. The upper-side weighted value of 2534 is the weight to be applied to the upper right sample area of ​​the first reference block.

[0458] In one embodiment, the lower left sample region of the first reference block represents the lower left triangular region obtained by dividing the first reference block by a diagonal line connecting the upper left vertex and the lower right vertex of the first reference block. The upper right sample region of the first reference block represents the upper right triangular region obtained by dividing the first reference block by a diagonal line connecting the upper left vertex and the lower right vertex of the first reference block.

[0459] In one embodiment, the left-weighted value 2532 is determined based on at least one sample from the left-side samples of the first reference template 2510, at least one sample from the left-side samples of the second reference template 2550, and at least one sample from the left-side samples of the current reference template 2590. The upper-weighted value 2534 is determined based on at least one sample from the upper-side samples of the first reference template 2510, at least one sample from the upper-side samples of the second reference template 2550, and at least one sample from the upper-side samples of the current reference template 2590.

[0460] In one embodiment, the predictive coding unit 3210 obtains a left-weighted matrix corresponding to the left sample of the first reference template 2510 based on the left sample of the first reference template 2510, the left sample of the second reference template 2550, and the left sample of the current reference template 2590. The predictive coding unit 3210 may obtain an upper-weighted matrix corresponding to the upper sample of the first reference template 2510 based on the upper sample of the first reference template 2510, the upper sample of the second reference template 2550, and the upper sample of the current reference template 2590. For example, the predictive coding unit 3210 applies the values ​​of the left sample of the current reference template 2590, the left sample of the first reference template 2510, and the left sample of the second reference template 2550 to Equation 2 to determine the values ​​of all elements of the left-weighted matrix. The predictive coding unit 3210 applies the values ​​of the upper samples of the current reference template 2590, the first reference template 2510, and the second reference template 2550 to Equation 2 to determine the values ​​of all elements of the upper weighted value matrix.

[0461] In one embodiment, the predictive coding unit 3210 uses at least one of the left-weighted matrix and the upper-weighted matrix to determine a weighted filter 2600 to be applied to each sample in the first reference block. The weighted filter 2600 represents the weights corresponding to each sample in the first reference block.

[0462] In one embodiment, the weighted value filter 2600 is determined based on at least one column in the left weighted value matrix and / or at least one row in the upper weighted value matrix. A single sample value included in the weighted value filter 2600 is determined based on at least one element in one column of the upper weighted value matrix corresponding to the position of the sample value and at least one element in one row of the left weighted value matrix corresponding to the position of the sample value.

[0463] In one embodiment, each sample value included in the weighted value filter 2600 is determined based on at least one element in each column of the upper weighted value matrix and at least one element in each row of the left weighted value matrix. That is, the weighted value filter 2600 is determined based on at least one element in each column of the upper weighted value matrix and at least one element in each row of the left weighted value matrix.

[0464] In one embodiment, the operation of the predictive coding unit 3210 of the image coding device 3200 may be the same as the operation of the predictive decoding unit 2030 of the image decoding device 2000. Therefore, the above-mentioned explanation regarding the operation of the predictive decoding unit 2030 can also be applied to the predictive coding unit 3210.

[0465] Figure 33 is a flowchart of an image coding method according to one embodiment.

[0466] In step S3310, the image coding device 3200 determines the second reference block in the first reference image and the second reference block in the second reference image for bi-prediction of the current block.

[0467] In one embodiment, the image encoding device 3200 determines the first reference block in the first reference image and the second reference block in the second reference image for biprediction of the current block using encoded motion information. For example, the image encoding device 3200 determines the first and second reference blocks for biprediction of the current block based on motion estimation or template matching.

[0468] In one embodiment, the image encoding device 3200 determines at least one of the first and second reference blocks using motion information or reference blocks of the surrounding blocks. For example, the image encoding device 3200 determines a motion vector using motion information or reference blocks of the surrounding blocks, and determines the block pointed to by the determined motion vector as at least one of the first and second reference blocks.

[0469] In one embodiment, the image encoding device 3200 can determine at least one of the first and second reference blocks of the current block by comparing a template in a predetermined region containing the block pointed to by the determined motion vector with the current reference template and improving the motion vector. For example, the image encoding device 3200 determines at least one of the first reference block in the first reference image and the second reference block in the second reference image based on template matching. The first and second reference images may be images encoded before the current image. In template matching, a set of surrounding samples encoded before the current block in the current image is used as a template, and the template of the current image may be called the "current reference template".

[0470] In one embodiment, the template most similar to the currently referenced template is searched for within the first and second reference images. The search range is a predetermined range centered on the points pointed to by the first and second motion vectors, which are the basic motion vectors for determining the respective reference blocks within the first and second reference images. Within the search range of the first reference image, the template most similar to the currently referenced template is determined as the first reference block, and within the search range of the second reference image, the template most similar to the currently referenced template is determined as the second reference block.

[0471] In step S3320, the image coding device 3200 determines weighting information for combining the first reference block and the second reference block based on the current reference template of the current block, the first reference template of the first reference block, and the second reference template of the second reference block.

[0472] In one embodiment, the weighted value information is information for joining the first reference block and the second reference block based on the current reference template 2590, the first reference template 2510, and the second reference template 2550.

[0473] In one embodiment, the weighting information includes information about weighting values ​​for performing a linear combination of a first reference block and a second reference block. For example, the weighting information includes information about a first weighting value applied to the first reference block and a second weighting value applied to the second reference block. If the first weighting value is determined, the second weighting value may be determined according to the determined first weighting value. In some embodiments, if the second weighting value is determined, the first weighting value may be determined according to the determined second weighting value.

[0474] In one embodiment, the first weighted value may be a single weighted value applied in common to the samples contained in the first reference block. The first weighted value is determined based on at least one sample from the first reference template, at least one sample from the second reference template, and at least one sample from the current reference template. For example, the single weighted value is determined by performing a linear regression analysis on at least one sample from the first reference template, at least one sample from the second reference template, and at least one sample from the current reference template, minimizing the difference between the combined value of the first and second reference templates and the value of the current reference template. The single weighted value may be determined based on left-weighted and upper-weighted values.

[0475] In one embodiment, the first weighted value includes a left-weighted value obtained based on at least one left-side sample of each template and an upper-weighted value obtained based on at least one upper-side sample of each template. The left-weighted value is for application to the lower-left sample area of ​​the first reference block, and the upper-weighted value is for application to the upper-right sample area of ​​the first reference block. The left-weighted value and the upper-weighted value are determined based on at least one sample of the first reference template, at least one sample of the second reference template, and at least one sample of the current reference template.

[0476] For example, the left-weighted value is determined based on one sample from the left-side samples of the first reference template, one sample from the left-side samples of the second reference template, and one sample from the left-side samples of the current reference template. The left-weighted value is determined by performing a linear regression analysis on multiple samples from the left-side samples of the first reference template, multiple samples from the left-side samples of the second reference template, and multiple samples from the left-side samples of the current reference template, minimizing the difference between the combined value of the left-side samples of the first and second reference templates and the value of the left-side sample of the current reference template. The left-weighted value is determined as the mean of the elements of the left-weighted matrix corresponding to the left-side sample of the first reference template, obtained based on the left-side samples of the first reference template, the left-side samples of the second reference template, and the left-side samples of the current reference template.

[0477] For example, the upper weighted value is determined based on one sample from the upper samples of the first reference template, one sample from the upper samples of the second reference template, and one sample from the upper samples of the current reference template. The upper weighted value is determined by performing a linear regression that minimizes the difference between the combined value of the upper samples of the first and second reference templates and the value of the upper sample of the current reference template, for multiple samples from the upper samples of the first reference template, multiple samples from the upper samples of the second reference template, and multiple samples from the upper samples of the current reference template. The upper weighted value may be determined as the average of the elements of the upper weighted value matrix corresponding to the upper sample of the first reference template, obtained based on the upper samples of the first reference template, the second reference template, and the current reference template.

[0478] In one embodiment, the first weighted value is a weighted value filter represented as a matrix that can be applied differently depending on the samples contained in the first reference block. The weighted value filter uses at least one of the left weighted value matrix and the upper weighted value matrix to indicate the weights corresponding to each sample contained in the first reference block for application to each sample contained in the first reference block.

[0479] For example, a weighted filter may be determined based on one column in the left-weighted matrix and one row in the upper-weighted matrix. A weighted filter may be determined based on the mean of at least one element in each row of the left-weighted matrix and the mean of at least one element in each column of the upper-weighted matrix. A weighted filter may be determined by applying a predetermined ratio to at least one element in each row of the left-weighted matrix and the elements in each column of the upper-weighted matrix.

[0480] In step S3330, the image coding device 3200 can generate a predicted block for the current block by combining the first reference block and the second reference block using weighted value information.

[0481] In one embodiment, weighting information can be used to join a first reference block and a second reference block. For example, the first and second reference blocks can be joined by applying the first weighting determined in step S3120 to the first reference block and the second weighting determined based on the first weighting to the second reference block.

[0482] In one embodiment, if a single weighted value is identified as the first weighted value, the first and second reference blocks can be joined by adding the value obtained by multiplying the single weighted value determined for each sample of the first reference block by the value obtained by multiplying the paired single weighted value determined for each sample of the second reference block.

[0483] In one embodiment, a left-weighted value and an upper-weighted value are identified as the first weighted value. In this case, the first and second reference blocks can be joined by adding the value obtained by multiplying the sample contained in the lower-left sample area of ​​the first reference block by the left-weighted value and the value obtained by multiplying the sample contained in the lower-left sample area of ​​the second reference block by the paired left-weighted value, and then adding the value obtained by multiplying the sample contained in the upper-right sample area of ​​the first reference block by the upper-right sample area of ​​the second reference block by the paired upper-weighted value.

[0484] In one embodiment, when a weighted value filter is identified as the first weighted value, the first and second reference blocks can be joined by adding the value obtained by multiplying each sample value of the first reference block by the element of the weighted value filter at the corresponding position, and the value obtained by multiplying each sample value of the second reference block by the element of the paired weighted value filter at the corresponding position.

[0485] In step S3340, the image encoding device 3200 encodes the current block using the prediction block.

[0486] A bitstream may be generated as a result of encoding the current block.

[0487] In one embodiment, the image encoding device 3200 acquires residual data corresponding to the difference between the predicted block and the current block, and information regarding the residual data may be included in the bitstream.

[0488] The image encoding method and apparatus 3200 and the image decoding method and apparatus 2000 according to one embodiment currently aim to improve the performance of predictive coding and predictive decoding for block 2250.

[0489] An image encoding method and apparatus 3200 and an image decoding method and apparatus 2000 according to one embodiment aim to reduce the amount of data required for signaling in interprediction mode.

[0490] An image encoding method and apparatus 3200 and an image decoding method and apparatus 2000 according to one embodiment aim to reduce the bitrate of the bitstream.

[0491] The technical problems that the present invention aims to solve are not limited to those mentioned above, and other technical problems not mentioned can be clearly understood by those skilled in the art from the following description.

[0492] In one embodiment of the present invention, the image decoding method includes the step of determining a first reference block in a first reference image 2420 and a second reference block in a second reference image 2440 for bi-prediction of block 2410.

[0493] In one embodiment of the present invention, the image decoding method includes the step of determining weighting information for joining the first reference block and the second reference block based on the current reference template of the current block 2410, the first reference template of the first reference block, and the second reference template of the second reference block.

[0494] In one embodiment of the present invention, the image decoding method includes the step of generating a predicted block for the current block 2410 by combining a first reference block and a second reference block using weighted value information.

[0495] In one embodiment of the present invention, the image decoding method includes the step of decoding the current block 2410 using a predicted block.

[0496] According to one embodiment, since a list of weighted value candidates or a weighted value index is not used to determine the current block, the amount of data required for signaling can be reduced.

[0497] In one embodiment of the present invention, the image decoding method includes the step of determining a single weighted value to apply to a first reference block based on at least one sample of a first reference template, at least one sample of a second reference template, and at least one sample of the current reference template.

[0498] In one embodiment of the present invention, the single weighted value is determined by performing a linear regression analysis on at least one sample of the first reference template, at least one sample of the second reference template, and at least one sample of the current reference template, which minimizes the difference between the combined value of the first and second reference templates and the value of the current reference template.

[0499] In one embodiment of the present invention, the single weighted value is determined based on a left-weighted value determined on at least one sample from the left-side samples of the first reference template, at least one sample from the left-side samples of the second reference template, and at least one sample from the left-side samples of the current reference template, or on an upper-weighted value determined on at least one sample from the upper-side samples of the first reference template, at least one sample from the upper-side samples of the second reference template, and at least one sample from the upper-side samples of the current reference template.

[0500] In one embodiment of the present invention, the image decoding method includes the step of determining a left-weighted value to be applied to the lower-left sample area of ​​the first reference block and an upper-weighted value to be applied to the upper-right sample area of ​​the first reference block, based on at least one sample of the first reference template, at least one sample of the second reference template, and at least one sample of the current reference template.

[0501] In one embodiment of the present invention, the left-weighted value is determined based on one sample from the left-side samples of the first reference template, one sample from the left-side samples of the second reference template, and one sample from the left-side samples of the current reference template.

[0502] In one embodiment of the present invention, the upper weighting value is determined based on one sample from the upper samples of the first reference template, one sample from the upper samples of the second reference template, and one sample from the upper samples of the current reference template.

[0503] In one embodiment of the present invention, the left-weighted value is determined by performing a linear regression analysis on multiple samples from the left-side samples of the first reference template, multiple samples from the left-side samples of the second reference template, and multiple samples from the left-side samples of the current reference template, minimizing the difference between the combined value of the left-side samples of the first and second reference templates and the value of the left-side sample of the current reference template.

[0504] In one embodiment of the present invention, the upper weighted value is determined by performing a linear regression on multiple samples from the upper samples of the first reference template, multiple samples from the upper samples of the second reference template, and multiple samples from the upper samples of the current reference template, minimizing the difference between the combined value of the upper samples of the upper samples of the first reference template and the upper samples of the second reference template and the value of the upper sample of the current reference template.

[0505] In one embodiment of the present invention, the left-weighted value is determined as the average of the elements of a left-weighted matrix corresponding to the left sample of the first reference template, obtained based on the left sample of the first reference template, the left sample of the second reference template, and the left sample of the current reference template, and the upper-weighted value may be determined as the average of the elements of an upper-weighted matrix corresponding to the upper sample of the first reference template, obtained based on the upper sample of the first reference template, the upper sample of the second reference template, and the upper sample of the current reference template.

[0506] In one embodiment of the present invention, the image decoding method includes the step of obtaining a left-weighted matrix corresponding to the left sample of the first reference template, based on the left sample of the first reference template, the left sample of the second reference template, and the left sample of the current reference template. The image decoding method includes the step of obtaining an upper-weighted matrix corresponding to the upper sample of the first reference template, based on the upper sample of the first reference template, the upper sample of the second reference template, and the upper sample of the current reference template. The image decoding method includes the step of determining a weighted filter that shows the weights corresponding to each sample contained in the first reference block, to be applied to each sample contained in the first reference block, using at least one of the left-weighted matrix and the upper-weighted matrix.

[0507] In one embodiment of the present invention, the weighted filter is determined based on one column in the left-side weighted matrix and one row in the upper-side weighted matrix.

[0508] In one embodiment of the present invention, the weighted filter may be determined based on the mean of at least one element in each row of the left-side weighted matrix and the mean of at least one element in each column of the upper-side weighted matrix.

[0509] In one embodiment of the present invention, the weighted value filter may be determined based on applying a predetermined ratio to at least one element in each row of the left weighted value matrix and to each element in each column of the upper weighted value matrix.

[0510] In one embodiment of the present invention, the image decoding method includes the step of identifying a single weighted value as a first weighted value to apply to a first reference block. The image decoding method includes the step of obtaining a second weighted value to apply to a second reference block using the single weighted value. The image decoding method includes the steps of applying the first weighted value to the first reference block and applying the second weighted value to the second reference block.

[0511] In one embodiment of the present invention, the image decoding method includes the step of applying a left-weighted value to the lower-left sample area of ​​a first reference block and identifying the upper-weighted value as a first weighted value to be applied to the upper-right sample area of ​​the first reference block. The image decoding method includes the step of obtaining a second weighted value to be applied to the lower-left sample area of ​​a second reference block using the left-weighted value and to the upper-right sample area of ​​the second reference block using the upper-weighted value. The image decoding method includes the step of applying the first weighted value to the first reference block and applying the second weighted value to the second reference block.

[0512] In one embodiment of the present invention, the image decoding method includes the step of identifying a weighted value filter as a first weighted value for application to a first reference block. The image decoding method includes the step of obtaining a second weighted value for application to a second reference block using the weighted value filter. The image decoding method includes the steps of applying the first weighted value to the first reference block and applying the second weighted value to the second reference block.

[0513] In one embodiment of the present invention, the image decoding device 2000 includes at least one memory for storing at least one instruction, and at least one processor that operates in accordance with at least one instruction.

[0514] In one embodiment of the present invention, at least one processor determines a first reference block in a first reference image 2420 and a second reference block in a second reference image 2440 for bi-prediction of the current block.

[0515] In one embodiment of the present invention, at least one processor determines weighted information for joining the first and second reference blocks based on the current reference template of the current block 2410, the first reference template of the first reference block, and the second reference template of the second reference block.

[0516] In one embodiment of the present invention, at least one processor generates a predicted block for the current block 2410 by combining a first reference block and a second reference block using weighted value information.

[0517] In one embodiment of the present invention, at least one processor decodes the current block 2410 using a prediction block.

[0518] In one embodiment of the present invention, the image encoding method includes the step (S3310) of determining a first reference block in a first reference image and a second reference block in a second reference image for the biprediction of block 2410.

[0519] In one embodiment of the present invention, the image encoding method includes the step (S3320) of determining weighted value information for combining the first reference block and the second reference block based on the current reference template of the current block 2410, the first reference template of the first reference block, and the second reference template of the second reference block.

[0520] In one embodiment of the present invention, the image encoding method includes the step (S3330) of generating a predicted block for the current block 2410 by combining a first reference block and a second reference block using weighted value information.

[0521] In one embodiment of the present invention, the image encoding method includes the step (S3340) of encoding the current block 2410 using a prediction block.

[0522] In one embodiment of the present invention, the image encoding device 3200 includes at least one memory for storing at least one instruction, and at least one processor that operates in accordance with at least one instruction.

[0523] In one embodiment of the present invention, the at least one processor currently determines a first reference block in a first reference image and a second reference block in a second reference image for biprediction of block 2410.

[0524] In one embodiment of the present invention, at least one processor determines weighted information for joining the first and second reference blocks based on the current reference template of the current block 2410, the first reference template of the first reference block, and the second reference template of the second reference block.

[0525] In one embodiment of the present invention, at least one processor generates a predicted block for the current block 2410 by combining a first reference block and a second reference block using weighted value information.

[0526] In one embodiment of the present invention, at least one processor encodes the current block 2410 using a prediction block.

[0527] In one embodiment of the present invention, a computer-readable recording medium records a bitstream generated by an image encoding method, wherein the bitstream currently includes motion information of block 2410.

[0528] In one embodiment of the present invention, motion information is generated by determining the first reference block in the first reference image and the second reference block in the second reference image for bi-prediction of the current block 2410, obtaining the current reference template showing the reference template of the current block 2410, the first reference template showing the reference template of the first reference block, and the second reference template showing the reference template of the second reference block, determining weighting information for combining the first reference block and the second reference block based on the current reference template, the first reference template, and the second reference template, generating a prediction block for the current block 2410 by combining the first reference block and the second reference block using the weighting information, and encoding the current block 2410 using the prediction block.

[0529] Furthermore, the embodiments of the present invention described above can be created as a program executable by a computer, and the created program can be recorded on a recording medium readable by a device.

[0530] A recording medium readable by a device is provided in the form of a non-transitory recording medium. Here, "non-transitory recording medium" simply means a tangible device that does not contain signals (e.g., electromagnetic waves), and this term does not distinguish between cases where data is recorded semi-permanently and cases where it is recorded temporarily. For example, a "non-transitory recording medium" includes a buffer on which data is recorded temporarily.

[0531] According to one embodiment, the methods according to the various embodiments disclosed in the present invention may be provided in a computer program product. The computer program product can be traded as a commodity between a seller and a buyer. The computer program product is distributed in the form of a device-readable recording medium (e.g., compact disc read-only memory (CD-ROM)), or through an application store, or directly online between two user devices (e.g., smartphones) (e.g., by download or upload). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) is at least temporarily recorded or generated on a device-readable recording medium, such as the memory of a manufacturer's server, an application store server, or an intermediary server.

Claims

1. In image decoding methods, The steps include determining the first reference block in the first reference image and the second reference block in the second reference image for bi-prediction of the current block, A step of determining weighted value information for joining the first reference block and the second reference block based on the current reference template of the current block, the first reference template of the first reference block, and the second reference template of the second reference block, The steps include generating a predicted block for the current block by combining the first reference block and the second reference block using the weighted value information, A method for decoding an image, comprising the step of decoding the current block using the prediction block.

2. The step of determining the weighted value information is: A method for decoding an image according to claim 1, comprising the step of determining a single weighted value to apply to the first reference block based on at least one sample of the first reference template, at least one sample of the second reference template, and at least one sample of the current reference template.

3. The aforementioned single weighted value is, The method for decoding an image according to claim 2, characterized in that the result is determined by performing a linear regression analysis on at least one sample of the first reference template, at least one sample of the second reference template, and at least one sample of the current reference template, which minimizes the difference between the combined value of the first and second reference templates and the value of the current reference template.

4. The aforementioned single weighted value is, The image decoding method according to claim 2, characterized in that it is determined based on a left-weighted value determined on at least one sample of the left-side samples of the first reference template, at least one sample of the left-side samples of the second reference template, and at least one sample of the left-side samples of the current reference template, or on an upper-weighted value determined on at least one sample of the upper-side samples of the first reference template, at least one sample of the upper-side samples of the second reference template, and at least one sample of the upper-side samples of the current reference template.

5. The step of determining the weighted value information is: A method for decoding an image according to claim 1, comprising the step of determining a left-weighted value to apply to the lower-left sample area of ​​the first reference block and an upper-weighted value to apply to the upper-right sample area of ​​the first reference block, based on at least one sample of the first reference template, at least one sample of the second reference template, and at least one sample of the current reference template.

6. The aforementioned left-side weighted value is, Determined based on one sample from the left-hand samples of the first reference template, one sample from the left-hand samples of the second reference template, and one sample from the left-hand samples of the current reference template, The aforementioned upper weight value is, The image decoding method according to claim 4, characterized in that it is determined based on one sample from the upper samples of the first reference template, one sample from the upper samples of the second reference template, and one sample from the upper samples of the current reference template.

7. The aforementioned left-side weighted value is, For multiple samples from the left-hand side of the first reference template, multiple samples from the left-hand side of the second reference template, and multiple samples from the left-hand side of the current reference template, a linear regression analysis is performed to minimize the difference between the combined value of the left-hand samples from the first and second reference templates and the value of the left-hand sample of the current reference template. The aforementioned upper weight value is, The image decoding method according to claim 4, characterized in that a linear regression analysis is performed on a plurality of upper samples from the first reference template, a plurality of upper samples from the second reference template, and a plurality of upper samples from the current reference template to minimize the difference between the combined value of the upper samples from the first reference template and the second reference template and the value of the upper sample from the current reference template.

8. The aforementioned left-side weighted value is, Determined as the mean of the elements of the left-weighted matrix corresponding to the left-side sample of the first reference template, obtained based on the left-side sample of the first reference template, the left-side sample of the second reference template, and the left-side sample of the current reference template, The aforementioned upper weight value is, The image decoding method according to claim 4, characterized in that it is determined as the average of the elements of an upper weighted matrix corresponding to the upper sample of the first reference template, obtained based on the upper sample of the first reference template, the upper sample of the second reference template, and the upper sample of the current reference template.

9. The step of determining the weighted value information is: The steps include obtaining a left-weighted matrix corresponding to the left-side sample of the first reference template based on the left-side sample of the first reference template, the left-side sample of the second reference template, and the left-side sample of the current reference template, The steps include obtaining an upper weighted matrix corresponding to the upper sample of the first reference template based on the upper sample of the first reference template, the upper sample of the second reference template, and the upper sample of the current reference template, A method for decoding an image according to claim 1, comprising the step of determining a weighted filter that shows a weight corresponding to each sample included in the first reference block, for application to each sample included in the first reference block, using at least one of the left weighted matrix and the upper weighted matrix.

10. The aforementioned weighted filter is, The image decoding method according to claim 9, characterized in that it is determined based on one column included in the left weight matrix and one row included in the upper weight matrix.

11. The aforementioned weighted filter is, The image decoding method according to claim 9, characterized in that it is determined based on the mean of at least one element in each row of the left-weighted matrix and the mean of at least one element in each column of the upper-weighted matrix.

12. The aforementioned weighted filter is, The image decoding method according to claim 9, characterized in that it is determined based on applying a predetermined ratio to at least one element in each row of the left-side weighted matrix and to each element in each column of the upper-side weighted matrix.

13. The step of generating a predicted block for the current block is: The steps include identifying the single weighted value as the first weighted value for applying to the first reference block, The steps include obtaining a second weighted value to apply to a second reference block using the aforementioned single weighted value, The image decoding method according to claim 2, comprising the steps of applying the first weight value to the first reference block and applying the second weight value to the second reference block.

14. In image encoding methods, The steps include determining the first reference block in the first reference image and the second reference block in the second reference image for biprediction of the current block, A step of determining weighted value information for joining the first reference block and the second reference block based on the current reference template of the current block, the first reference template of the first reference block, and the second reference template of the second reference block, The steps include generating a predicted block for the current block by combining the first reference block and the second reference block using the weighted value information, A method for encoding an image, comprising the step of encoding the current block using the prediction block.

15. In a computer-readable recording medium that records a bitstream generated by an image encoding method, The bitstream currently contains block motion information, The motion information of the current block is as follows: Determine the first reference block in the first reference image and the second reference block in the second reference image for bi-prediction of the current block. Obtain the current reference template indicating the reference template of the current block, the first reference template indicating the reference template of the first reference block, and the second reference template indicating the reference template of the second reference block. Based on the current reference template, the first reference template, and the second reference template, weighting information for joining the first reference block and the second reference block is determined. By combining the first reference block and the second reference block using the weighted value information, a predicted block is generated for the current block. A recording medium generated by encoding the current block using the prediction block.

Images