Image encoding method and device

Abstract

The present disclosure relates to an image decoding method and, more specifically, to a method and a device therefor, the method comprising the steps of: acquiring a residual difference signal of a current block by processing a bitstream; determining a reference block corresponding to the current block in a current picture including the current block; determining a residual signal of the reference block based on the reconstructed signal of the reference block and the prediction signal of the reference block; and determining a reconstruction signal of the current block based on the residual difference signal of the current block, the residual signal of the reference block, and a prediction signal of the current block, the prediction signal of the reference block being determined based on at least one reference pixel included in a neighboring block of the reference block in a current picture, and the prediction signal of the current block is determined based on at least one reference pixel included in a neighboring block of the current block in the current picture.

Description

TECHNICAL FIELD

[0001] The disclosure relates to image coding, and more particularly, to an image coding method, an apparatus for performing the image coding method, an image decoding method, an apparatus for performing the image decoding method, a method of storing or obtaining a bitstream, an apparatus for performing the method of storing or obtaining a bitstream, and a non-transitory storage medium storing a bitstream. BACKGROUND

[0002] Coding refers to a series of signal processing techniques for transmitting digital information through a communication line or storing digital information in a form suitable for a storage medium. The object of coding is an object such as audio, image, and text, and a technique for performing coding specifically for an image is referred to as image coding. Coding of a video signal, which is a sequence of images, is accomplished by removing redundant information while considering spatial correlation, temporal correlation, probabilistic correlation, and the like. However, due to recent development of various media and data transmission media, more efficient image coding methods and apparatuses are required. SUMMARY

[0003] TECHNICAL PROBLEM The technical problem to be solved by the disclosure provides an image coding method capable of reducing the number of bits required to encode a residual signal generated due to intra prediction and improving coding efficiency by applying residual prediction to intra prediction, and an apparatus for performing the image coding method.

[0004] Another technical problem to be solved by the disclosure provides an image coding method capable of reducing the number of bits required to signal intra prediction modes by determining the intra prediction modes in a manner corresponding to each other at an encoder side and a decoder side without signaling the intra prediction modes, and an apparatus for performing the image coding method.

[0005] The technical problems to be solved in the disclosure are not limited to the above-described technical problems, and also include other technical problems that can be seen from the detailed description of the disclosure.

[0006] SOLUTION According to an aspect of the disclosure, a method of decoding a bitstream performed by a device includes obtaining a residual difference signal of a current block by processing the bitstream; determining a reference block corresponding to the current block in a current picture including the current block; determining a residual signal of the reference block based on a reconstructed signal of the reference block and a prediction signal of the reference block; and determining a reconstructed signal of the current block based on the residual difference signal of the current block, the residual signal of the reference block, and a prediction signal of the current block. The prediction signal of the reference block can be determined based on at least one reference pixel included in neighboring blocks of the reference block in the current picture, and the prediction signal of the current block can be determined based on at least one reference pixel included in neighboring blocks of the current block in the current picture.

[0007] According to another aspect of the disclosure, a device configured to decode a bitstream includes a memory including executable instructions; and at least one processor configured to execute the instructions to implement a decoding method including obtaining a residual difference signal of a current block by processing the bitstream; determining a reference block corresponding to the current block in a current picture including the current block; determining a residual signal of the reference block based on a reconstructed signal of the reference block and a prediction signal of the reference block; and determining a reconstructed signal of the current block based on the residual difference signal of the current block, the residual signal of the reference block, and a prediction signal of the current block. The prediction signal of the reference block can be determined based on at least one reference pixel included in neighboring blocks of the reference block in the current picture, and the prediction signal of the current block can be determined based on at least one reference pixel included in neighboring blocks of the current block in the current picture.

[0008] The method can further include determining an intra prediction mode of the reference block from a plurality of intra prediction modes based on the reconstructed signal of the reference block and at least one reference pixel included in neighboring blocks of the reference block. The prediction signal of the reference block can be determined based on the intra prediction mode of the reference block and at least one reference pixel included in neighboring blocks of the reference block in the current picture, and the prediction signal of the current block can be determined based on the intra prediction mode of the reference block and at least one reference pixel included in neighboring blocks of the current block in the current picture.

[0009] Determining the intra prediction mode of the reference block from the plurality of intra prediction modes can include calculating a function value for each of the plurality of intra prediction modes based on the reconstructed signal of the reference block and at least one reference pixel included in neighboring blocks of the reference block; and determining an intra prediction mode corresponding to a lowest function value or a highest function value among the calculated function values as the intra prediction mode of the current block.

[0010] The function value can include at least one of a sum of absolute differences (SAD), a sum of squared differences (SSD), or a number of pixels having the same pixel value.

[0011] The method can further include identifying an intra prediction mode of the reference block based on the reference block being encoded in the intra prediction mode. The prediction signal of the reference block can be determined based on the intra prediction mode of the reference block and at least one reference pixel included in a neighboring block of the reference block in the current picture, and the prediction signal of the current block can be determined based on the intra prediction mode of the current block and at least one reference pixel included in a neighboring block of the current block in the current picture.

[0012] The method can further include obtaining an intra prediction mode of the current block from a bitstream. The prediction signal of the reference block can be determined based on the intra prediction mode of the current block and at least one reference pixel included in a neighboring block of the reference block in the current picture, and the prediction signal of the current block can be determined based on the intra prediction mode of the current block and at least one reference pixel included in a neighboring block of the current block in the current picture.

[0013] The method can further include determining an intra prediction mode of the current block from a plurality of intra prediction modes based on a pixel region neighboring the current block. The prediction signal of the reference block can be determined based on the intra prediction mode of the current block and at least one reference pixel included in a neighboring block of the reference block in the current picture, and the prediction signal of the current block can be determined based on the intra prediction mode of the current block and at least one reference pixel included in a neighboring block of the current block in the current picture.

[0014] The determining of the reference block corresponding to the current block can include determining a first pixel region neighboring the current block in the current picture, determining a second pixel region corresponding to the first pixel region neighboring the current block in the current picture, and determining the reference block corresponding to the second pixel region in the current picture.

[0015] The first pixel region neighboring the current block can include at least one of a pixel region neighboring a top of the current block, a pixel region neighboring an upper left of the current block, or a pixel region neighboring a left side of the current block.

[0016] The first pixel region neighboring the current block can have a fixed size, or have a size determined based on a size of the current block, or have a same width or a same height as a coding unit including the current block, or have a width or a height that is twice a width or a height of the coding unit including the current block.

[0017] The determining of the second pixel region corresponding to the first pixel region neighboring the current block in the current picture can include determining a pixel region corresponding to a lowest function value or a highest function value among the calculated function values as the second pixel region based on pixels of the first pixel region and pixels of a pixel region within a reconstructed region of the current picture.

[0018] The function value can include at least one of a sum of absolute difference (SAD), a sum of squared difference (SSD), or a number of pixels having the same pixel value.

[0019] The determining of the reference block corresponding to the current block can include obtaining a block vector of the current block from a bitstream, and determining the reference block in the current picture based on the block vector of the current block.

[0020] The method can further include obtaining flag information indicating whether residual prediction is performed for the current block from the bitstream. Based on the flag information indicating that the residual prediction is performed for the current block, the operation of obtaining the residual difference signal of the current block and the operation of determining the reconstructed signal of the current block based on the residual difference signal of the current block, the residual signal of the reference block, and the prediction signal of the current block can be performed.

[0021] According to another aspect of the disclosure, a method of encoding a bitstream performed by a device includes determining a reference block corresponding to a current block in a current picture including the current block, determining a residual signal of the reference block based on a reconstructed signal of the reference block and a prediction signal of the reference block, determining a residual difference signal of the current block based on the residual signal of the reference block, an original signal of the current block, and a prediction signal of the current block, and encoding the residual difference signal of the current block into the bitstream by processing the residual difference signal of the current block. The prediction signal of the reference block can be determined based on at least one reference pixel included in a neighboring block of the reference block in the current picture, and the prediction signal of the current block can be determined based on at least one reference pixel included in a neighboring block of the current block in the current picture.

[0022] According to another aspect of the present disclosure, an apparatus configured to encode a bitstream includes a memory including executable instructions and at least one processor configured to execute the instructions to implement an encoding method including determining a reference block corresponding to a current block in a current picture including the current block, determining a residual signal of the reference block based on a reconstructed signal of the reference block and a prediction signal of the reference block, determining a residual difference signal of the current block based on the residual signal of the reference block, an original signal of the current block, and a prediction signal of the current block, and encoding the residual difference signal of the current block into the bitstream by processing the residual difference signal of the current block. The prediction signal of the reference block can be determined based on at least one reference pixel included in neighboring blocks of the reference block in the current picture, and the prediction signal of the current block can be determined based on at least one reference pixel included in neighboring blocks of the current block in the current picture.

[0023] According to another aspect of the present disclosure, a non-transitory storage medium for storing a bitstream generated by an encoding method, the method including determining a reference block corresponding to a current block in a current picture including the current block, determining a residual signal of the reference block based on a reconstructed signal of the reference block and a prediction signal of the reference block, determining a residual difference signal of the current block based on the residual signal of the reference block, an original signal of the current block, and a prediction signal of the current block, and encoding the residual difference signal of the current block into the bitstream by processing the residual difference signal of the current block. The prediction signal of the reference block can be determined based on at least one reference pixel included in neighboring blocks of the reference block in the current picture, and the prediction signal of the current block can be determined based on at least one reference pixel included in neighboring blocks of the current block in the current picture.

[0024] According to another aspect of the present disclosure, a method of storing a bitstream generated by an encoding method, the method including determining a reference block corresponding to a current block in a current picture including the current block, determining a residual signal of the reference block based on a reconstructed signal of the reference block and a prediction signal of the reference block, determining a residual difference signal of the current block based on the residual signal of the reference block, an original signal of the current block, and a prediction signal of the current block, and encoding the residual difference signal of the current block into the bitstream by processing the residual difference signal of the current block. The prediction signal of the reference block can be determined based on at least one reference pixel included in neighboring blocks of the reference block in the current picture, and the prediction signal of the current block can be determined based on at least one reference pixel included in neighboring blocks of the current block in the current picture.

[0025] According to another aspect of the disclosure, a non-transitory storage medium for storing a bitstream generated by an encoding method, the encoding method including determining a reference block corresponding to a current block in a current picture including the current block, determining a residual signal of the reference block based on a reconstructed signal of the reference block and a prediction signal of the reference block, determining a residual difference signal of the current block based on the residual signal of the reference block, an original signal of the current block, and a prediction signal of the current block, and encoding the residual difference signal of the current block into the bitstream by processing the residual difference signal of the current block. The prediction signal of the reference block can be determined based on at least one reference pixel included in neighboring blocks of the reference block in the current picture, and the prediction signal of the current block can be determined based on at least one reference pixel included in neighboring blocks of the current block in the current picture.

[0026] Advantageous Effects According to the proposed method of the disclosure, residual prediction can be applied to intra prediction, so that the number of bits required to encode a residual signal generated due to intra prediction can be reduced, and encoding efficiency can be improved.

[0027] According to the proposed method of the disclosure, the number of bits required to signal an intra prediction mode can be reduced by determining the intra prediction mode according to a predetermined method at the encoder side and the decoder side without signaling the intra prediction mode.

[0028] Technical effects of the proposed method according to the disclosure are not limited to the above-mentioned technical effects, and also include other technical effects apparent from the detailed description of the disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0029] Brief descriptions of each drawing are provided to better understand the drawings cited herein.

[0030] FIG. 1 is a schematic block diagram of an image decoding apparatus according to an embodiment.

[0031] FIG. 2 is a flowchart of an image decoding method according to an embodiment.

[0032] FIG. 3 shows a process in which an image decoding apparatus according to an embodiment determines at least one coding unit by partitioning a current coding unit.

[0033] FIG. 4 shows a process in which an image decoding apparatus according to an embodiment determines at least one coding unit by partitioning a non-square coding unit.

[0034] FIG. 5A process in which the image decoding apparatus according to an embodiment divides a coding unit based on at least one of block shape information and partition shape mode information is shown.

[0035] FIG. 6 A method in which the image decoding apparatus according to an embodiment determines a particular coding unit from among an odd number of coding units is shown.

[0036] FIG. 7 An order in which the image decoding apparatus according to an embodiment processes a plurality of coding units when the plurality of coding units is determined by dividing a current coding unit is shown.

[0037] FIG. 8 A process in which the image decoding apparatus according to an embodiment determines that a current coding unit is to be divided into an odd number of coding units when the coding unit cannot be processed in a particular order is shown.

[0038] FIG. 9 A process in which the image decoding apparatus according to an embodiment determines at least one coding unit by dividing a first coding unit is shown.

[0039] FIG. 10 A shape into which a second coding unit according to an embodiment can be divided is limited when the second coding unit determined by dividing a first coding unit satisfies a particular condition is shown.

[0040] FIG. 11 A process in which the image decoding apparatus according to an embodiment divides a square coding unit when partition shape mode information indicates that the square coding unit is not divided into four square coding units is shown.

[0041] FIG. 12 A processing order among a plurality of coding units according to an embodiment can vary depending on a process of dividing a coding unit is shown.

[0042] FIG. 13 A process in which a depth of a coding unit is determined when the coding unit is recursively divided to determine a plurality of coding units is shown when the shape and size of the coding unit change.

[0043] FIG. 14 A depth that can be determined based on the shape and size of a coding unit and a partial index (PID) used to distinguish the coding unit are shown according to an embodiment.

[0044] FIG. 15 A plurality of coding units is determined based on a plurality of particular data units included in a picture according to an embodiment.

[0045] FIG. 16 is a block diagram of an image encoding and decoding system.

[0046] FIG. 17 is a flowchart of an image encoding method according to the proposed method of the present disclosure.

[0047] FIG. 18 is a flowchart of an image decoding method according to the proposed method of the present disclosure.

[0048] FIG. 19 is a flowchart of a bitstream storage method according to the proposed method of the present disclosure.

[0049] FIG. 20 and FIG. 21 shows intra prediction modes suitable for the proposed method of the present disclosure.

[0050] FIG. 22 shows inter template matching suitable for the proposed method of the present disclosure.

[0051] FIG. 23 shows intra template matching suitable for the proposed method of the present disclosure.

[0052] FIG. 24 shows intra block copy suitable for the proposed method of the present disclosure.

[0053] FIG. 25 shows the proposed method 1 of the present disclosure.

[0054] FIG. 26 shows a template suitable for the proposed method of the present disclosure.

[0055] FIG. 27 shows reference pixels for intra prediction suitable for the proposed method of the present disclosure.

[0056] FIG. 28 shows the proposed method 2 of the present disclosure.

[0057] FIG. 29 shows the proposed method 3 of the present disclosure.

[0058] FIG. 30 shows the proposed method 4 of the present disclosure.

[0059] FIG. 31 and FIG. 32 shows an apparatus configured to perform the proposed method of the present disclosure. DETAILED DESCRIPTION

[0060] Since the present disclosure allows various changes and numerous embodiments, exemplary embodiments will be illustrated in the accompanying drawings and described in detail in the written description. However, this is not intended to limit the present disclosure to a particular mode of practice, and it is to be appreciated that all changes, equivalents, and substitutes included in the present disclosure fall within the scope of the present disclosure.

[0061] In the description of the embodiments, when it is considered that a specific detailed explanation of related art can unnecessarily obscure the essence of the present disclosure, a specific detailed explanation of related art is omitted. Also, numbers (e.g., first and second) in the description of the specification are merely identifier codes to distinguish one component from another component.

[0062] Also, in the specification, it will be understood that when an element is "connected to" or "joined to" another element, unless otherwise stated, the element can be directly connected to or joined to the other element, but the element can also be indirectly connected to or joined to the other element with an intervening element therebetween.

[0063] In the present specification, regarding components expressed as "part (unit)" or "module", two or more components can be combined into one component, or one component can be divided into two or more components according to subdivided functions. Also, each of the components described below can additionally perform some or all functions performed by another component, in addition to its own main function, and some main functions of each component can be completely performed by another component.

[0064] Also, the term "image" or "picture" used herein can refer to a still image, a still image of a video, or a moving image, i.e., a video itself. The image or picture can be referred to as a frame.

[0065] Also, the term "sample" used herein refers to a sampling position assigned to an image and data to be processed. For example, the sample can be used interchangeably with a pixel of an image in a spatial domain, and can be used interchangeably with a transform coefficient in a transform domain. A unit including one or more samples can be defined as a block.

[0066] Also, in the specification, the term "current block" can refer to a block corresponding to a largest coding unit, a coding unit, a prediction unit, or a transform unit within a current picture to be encoded or decoded.

[0067] Furthermore, in the specification, a motion vector in the direction of List 0 can mean that the motion vector is used to point to a block in a reference picture included in List 0 (also referred to as reference list 0 or reference picture list 0), and a motion vector in the direction of List 1 can mean that the motion vector is used to point to a block in a reference picture included in List 1 (also referred to as reference list 1 or reference picture list 1). Furthermore, a unidirectional motion vector can mean that it points to a block in a reference picture included in List 0 or List 1, and a bidirectional motion vector can mean that it includes both a motion vector in the direction of List 0 and a motion vector in the direction of List 1. List 0 can be briefly denoted as L0, and List 1 can be briefly denoted as L1.

[0068] Furthermore, in the specification, the term "binary split" of a block refers to a split that divides the block into two sub-blocks, where each sub-block has one half of the width or one half of the height of the block. Specifically, if a "binary vertical split" is performed for a current block, the current block is split vertically (in the vertical direction) at the midpoint of its width, resulting in two sub-blocks, each of which has one half of the width of the current block and the same height as the current block. If a "binary horizontal split" is performed for a current block, the current block is split horizontally (in the horizontal direction) at the midpoint of its height, resulting in two sub-blocks, each of which has one half of the height of the current block and the same width as the current block.

[0069] Furthermore, in the specification, the term "ternary split" of a block refers to a split that divides the block into three sub-blocks with a 1:2:1 ratio of width or height. Specifically, if a "ternary vertical split" is performed for a current block, the current block is split vertically (in the vertical direction) at positions corresponding to a 1:2:1 ratio of its width, resulting in two sub-blocks each of which has one quarter of the width of the current block and the same height as the current block, and one sub-block which has one half of the width of the current block and the same height as the current block. If a "ternary horizontal split" is performed for a current block, the current block is split horizontally at positions corresponding to a 1:2:1 ratio of its height, resulting in two sub-blocks each of which has one quarter of the height of the current block and the same width as the current block, and one sub-block which has one half of the height of the current block and the same width as the current block.

[0070] Furthermore, in the specification, the term "quad split" of a block refers to a split that divides both the width and the height of the block with a 1:1 ratio, resulting in four sub-blocks. Specifically, when a "quad split" is performed for a current block, the current block is split vertically at the midpoint of its width and split horizontally at the midpoint of its height, resulting in four sub-blocks, each of which has one half of the width and one half of the height of the current block.

[0071] Hereinafter, a description will be made FIGS. 1-16 An image encoding method and apparatus and an image decoding method and apparatus according to embodiments are disclosed.

[0072] FIG. 1 is a schematic block diagram of an image decoding apparatus according to an embodiment.

[0073] The image decoding apparatus 100 can include a receiver 110 and a decoder 120. The receiver 110 and the decoder 120 can include at least one processor. Further, the receiver 110 and the decoder 120 can include a memory storing instructions to be executed by the at least one processor.

[0074] The receiver 110 can receive a bitstream. The bitstream can include information generated by an image encoding apparatus 2200 to be described below to encode an image. Further, the bitstream can be transmitted from the image encoding apparatus 2200. The image encoding apparatus 2200 can be connected to the image decoding apparatus 100 in a wired or wireless manner, and the receiver 110 can receive the bitstream in a wired or wireless manner. The receiver 110 can receive the bitstream from a storage medium such as an optical medium or a hard disk. The decoder 120 can reconstruct an image based on information obtained from the received bitstream. The decoder 120 can obtain a syntax element for reconstructing an image from the bitstream. The decoder 120 can reconstruct an image based on the syntax element.

[0075] A description will be made with reference to FIG. 2 The operation of the image decoding apparatus 100 will be described in more detail.

[0076] FIG. 2 A flowchart of an image decoding method according to an embodiment is shown.

[0077] According to an embodiment of the disclosure, the receiver 110 can receive a bitstream.

[0078] The image decoding apparatus 100 can perform an operation 210 of obtaining a bin string corresponding to a division shape mode of a coding unit from a bitstream. The image decoding apparatus 100 can perform an operation 220 of determining a division rule of the coding unit. Further, the image decoding apparatus 100 performs an operation of dividing the coding unit into a plurality of coding units based on at least one of the bin string corresponding to the division shape mode and the division rule. The image decoding apparatus 100 can determine a first range of a permissible size range of a coding unit according to a ratio of a height to a width of the coding unit in order to determine the division rule. The image decoding apparatus 100 can determine a second range of the permissible size range of the coding unit according to a division shape mode of the coding unit in order to determine the division rule.

[0079] Hereinafter, the division of a coding unit will be described in detail according to an embodiment of the disclosure.

[0080] First, a picture can be divided into one or more slices or one or more tiles. A slice or a tile can be a sequence of one or more largest coding units (coding tree units (CTUs)). There is a largest coding block (coding tree block (CTB)) as a concept corresponding to a largest coding unit (CTU).

[0081] A largest coding block (CTB) indicates an NxN block including NxN samples (where N is an integer). Each color component can be divided into one or more largest coding blocks.

[0082] When a picture has three arrays of samples (arrays of samples for Y, Cr, and Cb components), a largest coding unit (CTU) includes a largest coding block of luma samples, two corresponding largest coding blocks of chroma samples, and a syntax structure for coding the luma and chroma samples. When a picture is a monochrome picture, a largest coding unit includes a largest coding block of monochrome samples and a syntax structure for coding the monochrome samples. When a picture is a picture coded in color planes separated according to color components, a largest coding unit includes a syntax structure for coding the picture and samples of the picture.

[0083] A largest coding block (CTB) can be divided into an MxN coding block including MxN samples (where M and N are integers).

[0084] When a picture has three arrays of samples (arrays of samples for Y, Cr, and Cb components), a coding unit (CU) includes a coding block of luma samples, two corresponding coding blocks of chroma samples, and a syntax structure for coding the luma and chroma samples. When a picture is a monochrome picture, a coding unit includes a coding block of monochrome samples and a syntax structure for coding the monochrome samples. When a picture is a picture coded in color planes separated according to color components, a coding unit includes a syntax structure for coding the picture and samples of the picture.

[0085] As described above, a largest coding block and a largest coding unit are conceptually distinguished from each other, and a coding block and a coding unit are conceptually distinguished from each other. That is, a (largest) coding unit refers to a data structure including a (largest) coding block containing corresponding samples and a syntax structure corresponding to the (largest) coding block. However, because a person of ordinary skill in the art understands that a (largest) coding unit or a (largest) coding block refers to a block of a specific size including a specific number of samples, unless otherwise described, a largest coding block and a largest coding unit or a coding block and a coding unit are referred to without distinction in the following specification.

[0086] An image can be divided into maximum coding units (CTU). The size of each maximum coding unit can be determined based on information obtained from a bitstream. The shape of each maximum coding unit can be a square shape of the same size. However, embodiments are not limited thereto.

[0087] For example, information about the maximum size of a luma coding block can be obtained from a bitstream. For example, the maximum size of a luma coding block indicated by the information about the maximum size of a luma coding block can be one of 4x4, 8x8, 16x16, 32x32, 64x64, 128x128, and 256x256.

[0088] For example, information about a luma block size difference and the maximum size of a luma coding block that can be divided into two blocks can be obtained from a bitstream. The information about the luma block size difference can indicate a size difference between a luma maximum coding unit and a maximum luma coding block that can be divided into two blocks. Accordingly, when the information about the maximum size of a luma coding block that can be divided into two blocks and the information about the luma block size difference obtained from a bitstream are combined with each other, the size of the luma maximum coding unit can be determined. The size of a chroma maximum coding unit can be determined by using the size of the luma maximum coding unit. For example, when a Y:Cb:Cr ratio is 4:2:0 according to a color format, the size of a chroma block can be half of the size of a luma block, and the size of a chroma maximum coding unit can be half of the size of a luma maximum coding unit.

[0089] According to embodiments, because the information about the maximum size of a binary-divisible luma coding block is obtained from a bitstream, the maximum size of the binary-divisible luma coding block can be variably determined. In contrast, the maximum size of a ternary-divisible luma coding block can be fixed. For example, the maximum size of a ternary-divisible luma coding block in an I picture can be 32x32, and the maximum size of a ternary-divisible luma coding block in a P picture or a B picture can be 64x64.

[0090] In addition, a maximum coding unit can be hierarchically divided into coding units based on division shape mode information obtained from a bitstream. At least one of information indicating whether quad division is performed, information indicating whether multi-division is performed, division direction information, and division type information can be obtained from a bitstream as the division shape mode information.

[0091] For example, the information indicating whether quad division is performed can indicate whether a current coding unit is to be quad divided (QUAD_SPLIT).

[0092] When the current coding unit is not quad divided, the information indicating whether multi-division is performed can indicate whether the current coding unit is no longer divided (NO_SPLIT) or is binary divided / ternary divided.

[0093] When the current coding unit is bi-partitioned or tri-partitioned, the partition direction information indicates that the current coding unit is partitioned in one of a horizontal direction and a vertical direction.

[0094] When the current coding unit is partitioned in a horizontal direction or a vertical direction, the partition type information indicates that the current coding unit is bi-partitioned or tri-partitioned.

[0095] A partition mode of the current coding unit can be determined according to the partition direction information and the partition type information. The partition mode when the current coding unit is bi-partitioned in a horizontal direction can be determined as a bi-horizontal partition mode (SPLIT_BT_HOR), the partition mode when the current coding unit is tri-partitioned in a horizontal direction can be determined as a tri-horizontal partition mode (SPLIT_TT_HOR), the partition mode when the current coding unit is bi-partitioned in a vertical direction can be determined as a bi-vertical partition mode (SPLIT_BT_VER), and the partition mode when the current coding unit is tri-partitioned in a vertical direction can be determined as a tri-vertical partition mode SPLIT_TT_VER.

[0096] The image decoding apparatus 100 can obtain the partition shape mode information from a bin string according to one bin. The form of the bitstream received by the image decoding apparatus 100 can include a fixed length bin code, a unary code, a truncated unary code, a predetermined bin code, etc. The bin string is information of a binary number. The bin string can include at least one bit. The image decoding apparatus 100 can obtain the partition shape mode information corresponding to the bin string based on a partition rule. The image decoding apparatus 100 can determine whether to perform quad-partition on a coding unit, whether to not perform partition on a coding unit, a partition direction, and a partition type based on one bin.

[0097] The coding unit can be smaller than or the same as a maximum coding unit. For example, because the maximum coding unit is a coding unit having the largest size, the maximum coding unit is one of the coding units. When the partition shape mode information about the maximum coding unit indicates that no partition is performed, the coding unit determined in the maximum coding unit has the same size as that of the maximum coding unit. When the partition shape mode information about the maximum coding unit indicates that partition is performed, the maximum coding unit can be partitioned into coding units. Also, when the partition shape mode information about a coding unit indicates that partition is performed, the coding unit can be partitioned into smaller coding units. However, the partition of the image is not limited thereto, and the maximum coding unit and the coding unit can not be distinguished. It will be described in more detail with reference to FIGS. 2 to 5. FIGS. 3-16 The partition of the coding unit will be described in more detail.

[0098] Further, one or more prediction blocks for prediction can be determined from the coding unit. The prediction block can be the same as or smaller than the coding unit. Further, one or more transform blocks for transform can be determined from the coding unit. The transform block can be equal to or smaller than the coding unit.

[0099] The shape and size of the transform block and the prediction block can be irrelevant to each other.

[0100] In another embodiment, prediction can be performed by using the coding unit as a prediction unit. Further, transform can be performed by using the coding unit as a transform block.

[0101] The current block and the neighboring block of the present disclosure can refer to one of a maximum coding unit, a coding unit, a prediction block, and a transform block. Further, the current coding unit or the current block is a block that is currently being decoded or encoded or a block that is currently being divided. The neighboring block can be a block that is reconstructed before the current block. The neighboring block can be spatially or temporally adjacent to the current block. The neighboring block can be located at one of a lower left, a left, an upper left, a top, an upper right, a right, a lower right of the current block.

[0102] FIG. 3 A process in which an image decoding apparatus determines at least one coding unit by dividing a current coding unit according to an embodiment is illustrated.

[0103] The block shape can include 4N×4N, 4N×2N, 2N×4N, 4N×N, N×4N, 32N×N, N×32N, 16N×N, N×16N, 8N×N, or N×8N. Here, N can be a positive integer. The block shape information is information indicating at least one of a shape, a direction, a ratio of a width to a height, or a size of the coding unit.

[0104] The shape of the coding unit can include a square and a non-square. When the width and the height of the coding unit are the same (i.e., when the block shape of the coding unit is 4N×4N), the image decoding apparatus 100 can determine the block shape information of the coding unit as a square. The image decoding apparatus 100 can determine the shape of the coding unit as a non-square.

[0105] When the width and height of the coding unit are different from each other (i.e., when the block shape of the coding unit is 4N×2N, 2N×4N, 4N×N, N×4N, 32N×N, N×32N, 16N×N, N×16N, 8N×N, or N×8N), the image decoding apparatus 100 can determine the block shape information of the coding unit as non-square. When the shape of the coding unit is non-square, the image decoding apparatus 100 can determine the ratio of the width to the height in the block shape information of the coding unit as at least one of 1:2, 2:1, 1:4, 4:1, 1:8, 8:1, 1:16, 16:1, 1:32, or 32:1. Also, the image decoding apparatus 100 can determine whether the coding unit is along the horizontal direction or the vertical direction based on at least one of the width length and the height length of the coding unit. Also, the image decoding apparatus 100 can determine the size of the coding unit based on at least one of the width length, the height length, and the area of the coding unit.

[0106] According to an embodiment, the image decoding apparatus 100 can determine the shape of the coding unit by using the block shape information, and can determine the partitioning method of the coding unit by using the partition shape mode information. That is, the coding unit partitioning method indicated by the partition shape mode information can be determined based on the block shape indicated by the block shape information used by the image decoding apparatus 100.

[0107] The image decoding apparatus 100 can obtain the partition shape mode information from the bitstream. However, embodiments are not limited thereto, and the image decoding apparatus 100 and the image encoding apparatus 2200 can determine the pre-agreed partition shape mode information based on the block shape information. The image decoding apparatus 100 can determine the pre-agreed partition shape mode information for the maximum coding unit or the minimum coding unit. For example, the image decoding apparatus 100 can determine the partition shape mode information for the maximum coding unit as quad partition. Also, the image decoding apparatus 100 can determine the partition shape mode information about the minimum coding unit as "no partitioning is performed". Specifically, the image decoding apparatus 100 can determine the size of the maximum coding unit as 256×256. The image decoding apparatus 100 can determine the pre-agreed partition shape mode information as quad partition. The quad partition is a partition shape mode in which the width and the height of the coding unit are both bisected. The image decoding apparatus 100 can obtain a coding unit of 128×128 size from the maximum coding unit of 256×256 size based on the partition shape mode information. Also, the image decoding apparatus 100 can determine the size of the minimum coding unit as 4×4. The image decoding apparatus 100 can obtain the partition shape mode information indicating "no partitioning is performed" for the minimum coding unit.

[0108] According to an embodiment, the image decoding apparatus 100 can use the block shape information indicating that the current coding unit has a square shape. For example, the image decoding apparatus 100 can determine whether to not partition the square coding unit, whether to partition the square coding unit vertically, whether to partition the square coding unit horizontally, or whether to partition the square coding unit into four coding units, based on the partition shape mode information. Referring to FIG. 3 When the block shape information of the current coding unit 300 indicates a square shape, the decoder 120 can not partition the coding unit 310a having the same size as the current coding unit 300 based on the partition shape mode information indicating that no partitioning is performed, or can determine the coding units 310b, 310c, 310d, 310e, or 310f partitioned based on the partition shape mode information indicating a specific partitioning method.

[0109] Referring to FIG. 3 According to an embodiment, the image decoding apparatus 100 can determine two coding units 310b obtained by partitioning the current coding unit 300 in a vertical direction based on the partition shape mode information indicating that partitioning is performed in the vertical direction. The image decoding apparatus 100 can determine two coding units 310c obtained by partitioning the current coding unit 300 in a horizontal direction based on the partition shape mode information indicating that partitioning is performed in the horizontal direction. The image decoding apparatus 100 can determine four coding units 310d obtained by partitioning the current coding unit 300 in the vertical and horizontal directions based on the partition shape mode information indicating that partitioning is performed in the vertical and horizontal directions. According to an embodiment, the image decoding apparatus 100 can determine three coding units 310e obtained by partitioning the current coding unit 300 in the vertical direction based on the partition shape mode information indicating that ternary partitioning is performed in the vertical direction. The image decoding apparatus 100 can determine three coding units 310f obtained by partitioning the current coding unit 300 in the horizontal direction based on the partition shape mode information indicating that ternary partitioning is performed in the horizontal direction. However, the partitioning method of the square coding unit is not limited to the above-described methods, and the partition shape mode information can indicate various methods. Specific partitioning methods of partitioning the square coding unit will be described in detail below through various embodiments.

[0110] FIG. 4 A process in which the image decoding apparatus according to an embodiment determines at least one coding unit by partitioning a non-square coding unit is shown.

[0111] According to an embodiment, the image decoding apparatus 100 can use the block shape information indicating that the current coding unit has a non-square shape. The image decoding apparatus 100 can determine whether to not partition the non-square current coding unit or to partition the non-square current coding unit by using a specific partitioning method based on the partition shape mode information. Referring to FIG. 4 When the block shape information of the current coding unit 400 or 450 indicates a non-square shape, the image decoding apparatus 100 can determine the coding unit 410 or 460 having the same size as the current coding unit 400 or 450 based on the partition shape mode information indicating that no partitioning is performed, or can determine the coding units 420a and 420b, 430a to 430c, 470a and 470b, or 480a to 480c partitioned based on the partition shape mode information indicating a specific partitioning method. The specific partitioning method of partitioning a non-square coding unit will be described in detail below through various embodiments.

[0112] According to an embodiment, the image decoding apparatus 100 can determine the partitioning method of a coding unit by using the partition shape mode information, and in this case, the partition shape mode information can indicate the number of one or more coding units generated by partitioning the coding unit. Referring to FIG. 4 When the partition shape mode information indicates that the current coding unit 400 or 450 is partitioned into two coding units, the image decoding apparatus 100 can determine the two coding units 420a and 420b or 470a and 470b included in the current coding unit 400 or 450 by partitioning the current coding unit 400 or 450 based on the partition shape mode information.

[0113] According to an embodiment, when the image decoding apparatus 100 partitions the non-square current coding unit 400 or 450 based on the partition shape mode information, the image decoding apparatus 100 can consider the position of the long side of the non-square current coding unit 400 or 450 to partition the current coding unit. For example, the image decoding apparatus 100 can consider the shape of the current coding unit 400 or 450, determine a plurality of coding units by partitioning the current coding unit 400 or 450 in the direction of the long side of the current coding unit 400 or 450.

[0114] According to an embodiment, when the partition shape mode information indicates that the coding unit is partitioned (ternary partitioning) into an odd number of blocks, the image decoding apparatus 100 can determine the odd number of coding units included in the current coding unit 400 or 450. For example, when the partition shape mode information indicates that the current coding unit 400 or 450 is partitioned into three coding units, the image decoding apparatus 100 can partition the current coding unit 400 or 450 into three coding units 430a, 430b, and 430c or 480a, 480b, and 480c.

[0115] According to an embodiment, a ratio of a width to a height of the current coding unit 400 or 450 can be 4:1 or 1:4. When the ratio of the width to the height is 4:1, because the width length is longer than the height length, the block shape information can indicate a horizontal direction. When the ratio of the width to the height is 1:4, because the width length is shorter than the height length, the block shape information can indicate a vertical direction. The image decoding apparatus 100 can determine to divide the current coding unit into an odd number of blocks based on the division shape mode information. In addition, the image decoding apparatus 100 can determine a division direction of the current coding unit 400 or 450 based on the block shape information of the current coding unit 400 or 450. For example, when the current coding unit 400 is along the vertical direction, the image decoding apparatus 100 can determine the coding units 430a, 430b, and 430c by dividing the current coding unit 400 in the horizontal direction. In addition, when the current coding unit 450 is along the horizontal direction, the image decoding apparatus 100 can determine the coding units 480a, 480b, and 480c by dividing the current coding unit 450 in the vertical direction.

[0116] According to an embodiment, the image decoding apparatus 100 can determine an odd number of coding units included in the current coding unit 400 or 450, and not all of the determined coding units can have the same size. For example, a specific coding unit 430b or 480b among the determined odd number of coding units 430a, 430b, and 430c or 480a, 480b, and 480c can have a size different from those of the other coding units 430a and 430c or 480a and 480c. That is, the coding units that can be determined by dividing the current coding unit 400 or 450 can have a plurality of sizes, and in some cases, all of the odd number of coding units 430a, 430b, and 430c or 480a, 480b, and 480c can have different sizes.

[0117] According to an embodiment, when the division shape mode information indicates to divide a coding unit into an odd number of blocks, the image decoding apparatus 100 can determine an odd number of coding units included in the current coding unit 400 or 450, and in addition, can impose a specific restriction on at least one coding unit among the odd number of coding units generated by dividing the current coding unit 400 or 450. Referring to FIG. 4, the image decoding apparatus 100 can set a decoding process on the coding unit 430b or 480b, which is located at the center of the three coding units 430a, 430b, and 430c or 480a, 480b, and 480c generated when the current coding unit 400 or 450 is divided, to be different from a decoding process on the other coding units 430a and 430c or 480a and 480c. For example, unlike the other coding units 430a and 430c or 480a and 480c, the image decoding apparatus 100 can limit the coding unit 430b or 480b at the center position not to be divided any more or to be divided only a certain number of times.

[0118] FIG. 5 A process in which the image decoding apparatus divides a coding unit based on at least one of block shape information and division shape mode information according to an embodiment is illustrated.

[0119] According to an embodiment, the image decoding apparatus 100 can determine to divide a first coding unit 500 that is a square into coding units or not to divide the first coding unit 500 that is a square based on at least one of block shape information and division shape mode information. According to an embodiment, when the division shape mode information indicates to divide the first coding unit 500 in a horizontal direction, the image decoding apparatus 100 can determine a second coding unit 510 by dividing the first coding unit 500 in the horizontal direction. The first, second, and third coding units used according to an embodiment are terms for understanding a relationship before and after a coding unit is divided. For example, a second coding unit can be determined by dividing a first coding unit, and a third coding unit can be determined by dividing the second coding unit. It will be understood that the relationship of the first, second, and third coding units follows the above description.

[0120] According to an embodiment, the image decoding apparatus 100 can determine to divide the determined second coding unit 510 into coding units or not to divide the determined second coding unit 510 based on the division shape mode information. Referring to FIG. 5, the image decoding apparatus 100 can divide the second coding unit 510 determined by dividing the first coding unit 500 into one or more third coding units 520a, 520b, 520c, and 520d based on the division shape mode information, or can not divide the second coding unit 510 that is not a square. The image decoding apparatus 100 can obtain the division shape mode information, and can obtain a plurality of second coding units (for example, 510) of various shapes by dividing the first coding unit 500 based on the obtained division shape mode information, and can divide the second coding unit 510 by using the division method of the first coding unit 500 based on the division shape mode information. According to an embodiment, when the first coding unit 500 is divided into the second coding unit 510 based on the division shape mode information of the first coding unit 500, the second coding unit 510 can also be divided into third coding units (for example, 520a, 520b, 520c, and 520d) based on the division shape mode information of the second coding unit 510. That is, the coding units can be recursively divided based on the division shape mode information of each coding unit. Accordingly, a square coding unit can be determined by dividing a non-square coding unit, and a non-square coding unit can be determined by recursively dividing a square coding unit.

[0121] Referring to FIG. 5 A particular coding unit (for example, a coding unit located at a center position or a square coding unit) among the odd number of third coding units 520b, 520c, and 520d determined by dividing the second coding unit 510 that is not a square can be recursively divided. According to an embodiment, the square third coding unit 520b among the odd number of third coding units 520b, 520c, and 520d can be divided into a plurality of fourth coding units in a horizontal direction. A non-square fourth coding unit 530b or 530d among the plurality of fourth coding units 530a, 530b, 530c, and 530d can be re-divided into a plurality of coding units. For example, the non-square fourth coding unit 530b or 530d can be re-divided into an odd number of coding units. Methods that can be used to recursively divide coding units will be described below through various embodiments.

[0122] According to an embodiment, the image decoding apparatus 100 can divide each of the third coding units 520a, 520b, 520c, and 520d into coding units based on the division shape mode information. Also, the image decoding apparatus 100 can determine not to divide the second coding unit 510 based on the division shape mode information. According to an embodiment, the image decoding apparatus 100 can divide the second coding unit 510, which is not a square, into an odd number of third coding units 520b, 520c, and 520d. The image decoding apparatus 100 can impose a specific restriction on a specific third coding unit among the odd number of third coding units 520b, 520c, and 520d. For example, the image decoding apparatus 100 can restrict the third coding unit 520c at a center position among the odd number of third coding units 520b, 520c, and 520d not to be divided any more or to be divided a settable number of times.

[0123] Referring to FIG. 5 , the image decoding apparatus 100 can restrict the third coding unit 520c at a center position among the odd number of third coding units 520b, 520c, and 520d included in the second coding unit 510, which is not a square, to not be divided any more, to be divided by using a specific division method (for example, to be divided into only four coding units or to be divided by using the division method of the second coding unit 510), or to be divided only a specific number of times (for example, to be divided only n times (where n > 0)). However, the restriction on the third coding unit 520c at the center position is not limited to the above-described example and can include various restrictions for decoding the third coding unit 520c at the center position differently from the other third coding units 520b and 520d.

[0124] According to an embodiment, the image decoding apparatus 100 can obtain the division shape mode information for dividing the current coding unit from a specific position in the current coding unit.

[0125] FIG. 6 A method in which the image decoding apparatus according to an embodiment determines a specific coding unit from among odd number of coding units is illustrated.

[0126] Referring to FIG. 6 , the division shape mode information of the current coding unit 600 or 650 can be obtained from a sample at a specific position (for example, a sample 640 or 690 at a center position) among a plurality of samples included in the current coding unit 600 or 650. However, the specific position in the current coding unit 600 from which at least one piece of division shape mode information can be obtained is not limited to FIG. 6The center position in the image can be specified, and may include various positions included in the current encoding unit 600 (e.g., top, bottom, left, right, upper left, lower left, upper right, and lower right positions, etc.). The image decoding device 100 can obtain the division shape pattern information from the specific position and can determine whether to divide the current encoding unit into encoding units of various shapes and sizes or not to divide the current encoding unit.

[0127] According to an embodiment, when the current coding unit is divided into a specific number of coding units, the image decoding device 100 can select one coding unit from the coding units. Various methods can be used to select one coding unit from a plurality of coding units, which will be described below through various embodiments.

[0128] According to an embodiment, the image decoding device 100 can divide the current coding unit into multiple coding units and determine the coding unit at a specific location.

[0129] According to an embodiment, the image decoding device 100 can use information indicating the positions of an odd number of coding units to determine the coding unit at the center position among the odd number of coding units. (See also...) FIG. 6 The image decoding device 100 can determine an odd number of coding units 620a, 620b, and 620c or an odd number of coding units 660a, 660b, and 660c by dividing the current coding unit 600 or the current coding unit 650. The image decoding device 100 can determine an intermediate coding unit 620b or an intermediate coding unit 660b by using information about the positions of the odd number of coding units 620a, 620b, and 620c or the odd number of coding units 660a, 660b, and 660c. For example, the image decoding device 100 can determine the coding unit 620b at the center position by determining the positions of coding units 620a, 620b, and 620c based on information indicating the positions of specific samples included in coding units 620a, 620b, and 620c. In detail, the image decoding device 100 can determine the position of the encoding units 620a, 620b and 620c based on the information indicating the positions of the upper left samples 630a, 630b and 630c of the encoding units 620a, 620b and 620c, and determine the encoding unit 620b at the center position.

[0130] According to an embodiment, the information indicating the positions of the top-left samples 630a, 630b, and 630c included in the coding units 620a, 620b, and 620c, respectively, can include information on positions or coordinates of the coding units 620a, 620b, and 620c in a picture. According to an embodiment, the information indicating the positions of the top-left samples 630a, 630b, and 630c included in the coding units 620a, 620b, and 620c, respectively, can include information indicating widths or heights of the coding units 620a, 620b, and 620c included in the current coding unit 600, and the widths or heights can correspond to information indicating differences between the coordinates of the coding units 620a, 620b, and 620c in the picture. That is, the image decoding apparatus 100 can determine the coding unit 620b at the center position by directly using the information on the positions or coordinates of the coding units 620a, 620b, and 620c in the picture or by using the information on the widths or heights of the coding units corresponding to the differences between the coordinates.

[0131] According to an embodiment, the information indicating the position of the top-left sample 630a of the upper coding unit 620a can include a coordinate (xa, ya), the information indicating the position of the top-left sample 630b of the middle coding unit 620b can include a coordinate (xb, yb), and the information indicating the position of the top-left sample 630c of the lower coding unit 620c can include a coordinate (xc, yc). The image decoding apparatus 100 can determine the middle coding unit 620b by using the coordinates of the top-left samples 630a, 630b, and 630c included in the coding units 620a, 620b, and 620c, respectively. For example, when the coordinates of the top-left samples 630a, 630b, and 630c are ordered in ascending order or descending order, the coding unit 620b including the coordinate (xb, yb) of the sample 630b at the center position can be determined as the coding unit at the center position among the coding units 620a, 620b, and 620c determined by dividing the current coding unit 600. However, the coordinates indicating the positions of the top-left samples 630a, 630b, and 630c can include coordinates indicating absolute positions in a picture, or can use coordinates (dxb, dyb) indicating a relative position of the top-left sample 630b of the middle coding unit 620b with respect to the position of the top-left sample 630a of the upper coding unit 620a and coordinates (dxc, dyc) indicating a relative position of the top-left sample 630c of the lower coding unit 620c with respect to the position of the top-left sample 630a of the upper coding unit 620a. The method of determining a coding unit at a certain position by using the coordinates of samples included in a coding unit as information indicating the positions of the samples is not limited to the above-described method, and can include various arithmetic methods capable of using the coordinates of the samples.

[0132] According to an embodiment, the image decoding apparatus 100 can divide the current coding unit 600 into a plurality of coding units 620a, 620b, and 620c, and can select one of the coding units 620a, 620b, and 620c based on a certain criterion. For example, the image decoding apparatus 100 can select the coding unit 620b having a size different from those of the other coding units from among the coding units 620a, 620b, and 620c.

[0133] According to an embodiment, the image decoding apparatus 100 can determine the width or height of each of the coding units 620a, 620b, and 620c by using a coordinate (xa, ya) that is information indicating the position of the top-left sample 630a of the upper coding unit 620a, a coordinate (xb, yb) that is information indicating the position of the top-left sample 630b of the middle coding unit 620b, and a coordinate (xc, yc) that is information indicating the position of the top-left sample 630c of the lower coding unit 620c. The image decoding apparatus 100 can determine the respective sizes of the coding units 620a, 620b, and 620c by using the coordinates (xa, ya), (xb, yb), and (xc, yc) indicating the positions of the coding units 620a, 620b, and 620c. According to an embodiment, the image decoding apparatus 100 can determine the width of the upper coding unit 620a as the width of the current coding unit 600. The image decoding apparatus 100 can determine the height of the upper coding unit 620a as yb-ya. According to an embodiment, the image decoding apparatus 100 can determine the width of the middle coding unit 620b as the width of the current coding unit 600. The image decoding apparatus 100 can determine the height of the middle coding unit 620b as yc-yb. According to an embodiment, the image decoding apparatus 100 can determine the width or height of the lower coding unit 620c by using the width or height of the current coding unit 600 and the width or height of the upper coding unit 620a and the middle coding unit 620b. The image decoding apparatus 100 can determine a coding unit having a size different from those of the other coding units based on the determined widths and heights of the coding units 620a, 620b, and 620c. Referring to FIG. 6 , the image decoding apparatus 100 can determine the middle coding unit 620b having a size different from those of the upper coding unit 620a and the lower coding unit 620c as the coding unit at the certain position. However, the above-described process in which the image decoding apparatus 100 determines a coding unit having a size different from those of the other coding units corresponds only to an example in which the coding unit at the certain position is determined by using the sizes of the coding units determined based on the coordinates of the samples, and thus various processes in which the coding unit at the certain position is determined by comparing the sizes of the coding units determined based on the coordinates of the certain samples can be used.

[0134] The image decoding apparatus 100 can determine the width or height of each of the coding units 660a, 660b, and 660c by using the coordinates (xd, yd) that are information indicating the position of the top-left sample 670a of the left coding unit 660a, the coordinates (xe, ye) that are information indicating the position of the top-left sample 670b of the middle coding unit 660b, and the coordinates (xf, yf) that are information indicating the position of the top-left sample 670c of the right coding unit 660c. The image decoding apparatus 100 can determine the respective sizes of the coding units 660a, 660b, and 660c by using the coordinates (xd, yd), (xe, ye), and (xf, yf) indicating the positions of the coding units 660a, 660b, and 660c.

[0135] According to an embodiment, the image decoding apparatus 100 can determine the width of the left coding unit 660a as xe-xd. The image decoding apparatus 100 can determine the height of the left coding unit 660a as the height of the current coding unit 650. According to an embodiment, the image decoding apparatus 100 can determine the width of the middle coding unit 660b as xf-xe. The image decoding apparatus 100 can determine the height of the middle coding unit 660b as the height of the current coding unit 650. According to an embodiment, the image decoding apparatus 100 can determine the width or height of the right coding unit 660c by using the width or height of the current coding unit 650 and the width or height of the left coding unit 660a and the middle coding unit 660b. The image decoding apparatus 100 can determine a coding unit having a size different from those of other coding units based on the determined widths and heights of the coding units 660a, 660b, and 660c. Referring to FIG. 6 , the image decoding apparatus 100 can determine the middle coding unit 660b having a size different from those of the left coding unit 660a and the right coding unit 660c as a coding unit at a specific position. However, the above-described process in which the image decoding apparatus 100 determines a coding unit having a size different from those of other coding units corresponds to only an example of determining a coding unit at a specific position by using the sizes of coding units determined based on the coordinates of samples, and thus various processes of determining a coding unit at a specific position by comparing the sizes of coding units determined based on the coordinates of specific samples can be used.

[0136] However, the positions of samples considered in determining the positions of coding units are not limited to the above-described top-left positions, and information on arbitrary positions of samples included in a coding unit can be used.

[0137] According to an embodiment, the image decoding apparatus 100 can select a coding unit at a specific position from among the odd number of coding units determined by partitioning the current coding unit, by considering a shape of the current coding unit. For example, when the current coding unit has a non-square shape with a width longer than a height, the image decoding apparatus 100 can determine a coding unit at a specific position in a horizontal direction. That is, the image decoding apparatus 100 can determine one of the coding units at different positions in the horizontal direction and can impose a restriction on the coding unit. When the current coding unit has a non-square shape with a height longer than a width, the image decoding apparatus 100 can determine a coding unit at a specific position in a vertical direction. That is, the image decoding apparatus 100 can determine one of the coding units at different positions in the vertical direction and can impose a restriction on the coding unit.

[0138] According to an embodiment, the image decoding apparatus 100 can use information indicating respective positions of the even number of coding units to determine a coding unit at a specific position among the even number of coding units. The image decoding apparatus 100 can determine the even number of coding units by partitioning (binary partitioning) the current coding unit, and can determine a coding unit at a specific position by using information about positions of the even number of coding units. Operations related thereto can correspond to those already described above with reference to FIGS. 1 to 6, and thus detailed descriptions thereof will be omitted. FIG. 6 The operation of determining a coding unit at a specific position (e.g., a center position) among the odd number of coding units described in detail above corresponds, and thus detailed descriptions thereof will be omitted.

[0139] According to an embodiment, when a non-square current coding unit is partitioned into a plurality of coding units, a coding unit at a specific position among the plurality of coding units can be determined using specific information about the coding unit at the specific position in a partitioning operation. For example, the image decoding apparatus 100 can determine a coding unit at a center position among the plurality of coding units determined by partitioning the current coding unit using at least one of block shape information or partition shape mode information stored in samples included in the middle coding unit in the partitioning operation.

[0140] Referring to FIG. 6, the image decoding apparatus 100 can divide the current coding unit 600 into the plurality of coding units 620a, 620b, and 620c based on the division shape mode information, and can determine the coding unit 620b at the center position among the plurality of coding units 620a, 620b, and 620c. Also, the image decoding apparatus 100 can determine the coding unit 620b at the center position in consideration of a position at which the division shape mode information is obtained. That is, the division shape mode information of the current coding unit 600 can be obtained from the sample 640 at the center position of the current coding unit 600, and when the current coding unit 600 is divided into the plurality of coding units 620a, 620b, and 620c based on the division shape mode information, the coding unit 620b including the sample 640 can be determined as the coding unit at the center position. However, information for determining the coding unit at the center position is not limited to the division shape mode information, and the coding unit at the center position can be determined using various types of information.

[0141] According to an embodiment, the specific information for identifying the coding unit at the specific position can be obtained from a specific sample included in the coding unit to be determined. Referring to FIG. 6 , the image decoding apparatus 100 can determine the coding unit at the specific position (for example, the coding unit at the center position among the divided plurality of coding units) among the plurality of coding units 620a, 620b, and 620c determined by dividing the current coding unit 600 using the division shape mode information obtained from the sample at the specific position (for example, the sample at the center position of the current coding unit 600) in the current coding unit 600. That is, the image decoding apparatus 100 can determine the sample at the specific position by considering the block shape of the current coding unit 600, can determine the coding unit 620b including the sample from which the specific information (for example, the division shape mode information) can be obtained from among the plurality of coding units 620a, 620b, and 620c determined by dividing the current coding unit 600, and can apply the specific restriction to the coding unit 620b. Referring to FIG. 5 , according to an embodiment, in the decoding process, the image decoding apparatus 100 can determine the sample 640 at the center position of the current coding unit 600 as the sample from which the specific information can be obtained, and can apply the specific restriction to the coding unit 620b including the sample 640. However, the position of the sample from which the specific information can be obtained is not limited to the above-described position, and can include any position of the sample included in the coding unit 620b to be determined to be restricted.

[0142] According to an embodiment, the position of the sample from which the specific information is obtainable can be determined based on a shape of the current coding unit 600. According to an embodiment, the block shape information can indicate whether the current coding unit has a square shape or a non-square shape, and the position of the sample from which the specific information is obtainable can be determined based on the shape. For example, the image decoding apparatus 100 can determine a sample located at a boundary for halving at least one of a width and a height of the current coding unit as the sample from which the specific information is obtainable, by using at least one of information about the width of the current coding unit and information about the height of the current coding unit. As another example, when the block shape information of the current coding unit indicates a non-square shape, the image decoding apparatus 100 can determine one of samples including a boundary for halving a long side of the current coding unit as the sample from which the specific information is obtainable.

[0143] According to an embodiment, when the current coding unit is divided into a plurality of coding units, the image decoding apparatus 100 can use the division shape mode information to determine a coding unit at a specific position among the plurality of coding units. According to an embodiment, the image decoding apparatus 100 can obtain the division shape mode information from a sample at a specific position in a coding unit, and can divide the plurality of coding units generated by dividing the current coding unit by using the division shape mode information obtained from a sample at a specific position in each of the plurality of coding units. That is, the coding units can be recursively divided based on the division shape mode information obtained from a sample at a specific position in each of the coding units. The above has been described with reference to FIGS. 6A and 6B. FIG. 7 The process of recursively dividing the coding units is described, and thus a detailed description thereof will be omitted.

[0144] According to an embodiment, the image decoding apparatus 100 can determine one or more coding units by dividing the current coding unit, and can determine an order in which the one or more coding units are decoded based on a specific block (e.g., the current coding unit).

[0145] FIG. 7 An order in which the image decoding apparatus processes a plurality of coding units when the image decoding apparatus determines the plurality of coding units by dividing a current coding unit is shown according to an embodiment.

[0146] According to an embodiment, based on the division shape mode information, the image decoding apparatus 100 can determine the second coding units 710a and 710b by dividing the first coding unit 700 in a vertical direction, can determine the second coding units 730a and 730b by dividing the first coding unit 700 in a horizontal direction, or can determine the second coding units 750a, 750b, 750c, and 750d by dividing the first coding unit 700 in the vertical direction and the horizontal direction.

[0147] Referring to FIG. 7 , the image decoding apparatus 100 can determine to process the second coding units 710a and 710b determined by dividing the first coding unit 700 in the vertical direction in the horizontal direction order 710c. The image decoding apparatus 100 can determine to process the second coding units 730a and 730b determined by dividing the first coding unit 700 in the horizontal direction in the vertical direction order 730c. The image decoding apparatus 100 can determine to process the second coding units 750a, 750b, 750c, and 750d determined by dividing the first coding unit 700 in the vertical direction and the horizontal direction according to a certain order of processing coding units in a row and then processing coding units in the next row (e.g., a raster scan order or a zigzag scan order 750e).

[0148] According to an embodiment, the image decoding apparatus 100 can recursively divide coding units. Referring to FIG. 7 , the image decoding apparatus 100 can determine the plurality of coding units 710a and 710b, 730a and 730b, or 750a, 750b, 750c, and 750d by dividing the first coding unit 700, and can recursively divide each of the determined plurality of coding units 710a, 710b, 730a, 730b, 750a, 750b, 750c, and 750d. The division method of the plurality of coding units 710a and 710b, 730a and 730b, or 750a, 750b, 750c, and 750d can correspond to the division method of the first coding unit 700. Accordingly, each of the plurality of coding units 710a and 710b, 730a and 730b, or 750a, 750b, 750c, and 750d can be independently divided into a plurality of coding units. Referring to FIG. 8 , the image decoding apparatus 100 can determine the second coding units 710a and 710b by dividing the first coding unit 700 in the vertical direction, and can determine to independently divide or not to divide each of the second coding units 710a and 710b.

[0149] According to an embodiment, the image decoding apparatus 100 can determine the third coding units 720a and 720b by dividing the left second coding unit 710a in the horizontal direction, and can not divide the right second coding unit 710b.

[0150] According to an embodiment, the processing order of a coding unit can be determined based on the process of dividing the coding unit. In other words, the processing order of a divided coding unit can be determined based on the processing order of a coding unit immediately before being divided. The image decoding apparatus 100 can determine the processing order of the third coding units 720a and 720b determined by dividing the left second coding unit 710a independently of the right second coding unit 710b. Since the third coding units 720a and 720b are determined by dividing the left second coding unit 710a in the horizontal direction, the third coding units 720a and 720b can be processed in the vertical direction order 720c. Since the left second coding unit 710a and the right second coding unit 710b are processed in the horizontal direction order 710c, the right second coding unit 710b can be processed after the third coding units 720a and 720b included in the left second coding unit 710a are processed in the vertical direction order 720c. The process of determining the processing order of a coding unit based on a coding unit before being divided is not limited to the above-described example, and various methods can be used to independently process coding units divided and determined to be various shapes in a specific order.

[0151] FIG. 8 A process in which the image decoding apparatus determines that a current coding unit is to be divided into an odd number of coding units when the coding units cannot be processed in a specific order is shown according to an embodiment.

[0152] According to an embodiment, the image decoding apparatus 100 can determine whether a current coding unit is to be divided into an odd number of coding units based on the obtained division shape mode information. Referring to FIG. 8 , the square first coding unit 800 can be divided into the non-square second coding units 810a and 810b, and the second coding units 810a and 810b can be independently divided into the third coding units 820a and 820b and 820c, 820d, and 820e. According to an embodiment, the image decoding apparatus 100 can determine the plurality of third coding units 820a and 820b by dividing the left second coding unit 810a in the horizontal direction, and can divide the right second coding unit 810b into an odd number of third coding units 820c, 820d, and 820e.

[0153] According to an embodiment, the image decoding apparatus 100 can determine whether an arbitrary coding unit is divided into an odd number of coding units by determining whether the third coding units 820a and 820b and 820c, 820d, and 820e can be processed in a specific order. Referring to FIG. 9, the image decoding apparatus 100 can determine the third coding units 820a and 820b and 820c, 820d, and 820e by recursively dividing the first coding unit 800. The image decoding apparatus 100 can determine whether any of the following coding units is divided into an odd number of coding units based on at least one of the block shape information and the division shape mode information: the first coding unit 800, the second coding units 810a and 810b, or the third coding units 820a and 820b and 820c, 820d, and 820e. For example, the right side coding unit among the second coding units 810a and 810b can be divided into an odd number of third coding units 820c, 820d, and 820e. The processing order of the plurality of coding units included in the first coding unit 800 can be a specific order (e.g., a zigzag scan order 830), and the image decoding apparatus 100 can determine whether the third coding units 820c, 820d, and 820e determined by dividing the right side second coding unit 810b into an odd number of coding units satisfy a condition for being processed in the specific order.

[0154] According to an embodiment, the image decoding apparatus 100 can determine whether the third coding units 820a and 820b and 820c, 820d, and 820e included in the first coding unit 800 satisfy a condition for being processed in a specific order, and the condition is related to whether at least one of the width and the height of the second coding units 810a and 810b is to be halved along the boundaries of the third coding units 820a and 820b and 820c, 820d, and 820e. For example, the third coding units 820a and 820b determined when the height of the left side second coding unit 810a of a non-square shape is halved can satisfy the condition. Since the boundaries of the third coding units 820c, 820d, and 820e determined when the right side second coding unit 810b is divided into three coding units fail to halve the width or the height of the right side second coding unit 810b, it can be determined that the third coding units 820c, 820d, and 820e do not satisfy the condition. When the condition is not satisfied as described above, the image decoding apparatus 100 can determine that the scan order is discontinuous, and can determine based on the determination result that the right side second coding unit 810b is to be divided into an odd number of coding units. According to an embodiment, when a coding unit is divided into an odd number of coding units, the image decoding apparatus 100 can impose a specific restriction on a coding unit at a specific position among the divided coding units. The restriction or the specific position has been described above through various embodiments, and thus a detailed description thereof will be omitted.

[0155] FIG. 9 A process in which an image decoding apparatus determines at least one coding unit by dividing a first coding unit is illustrated according to an embodiment.

[0156] According to an embodiment, the image decoding apparatus 100 can divide the first coding unit 900 based on the division shape mode information obtained through the receiver 110. The square first coding unit 900 can be divided into four square coding units, or can be divided into a plurality of non-square coding units. For example, referring to FIG. 9 When the first coding unit 900 has a square shape and the division shape mode information indicates that the first coding unit 900 is divided into non-square coding units, the image decoding apparatus 100 can divide the first coding unit 900 into a plurality of non-square coding units. In detail, when the division shape mode information indicates that an odd number of coding units are determined by dividing the first coding unit 900 in a horizontal direction or a vertical direction, the image decoding apparatus 100 can divide the square first coding unit 900 into the odd number of coding units, for example, the second coding units 910a, 910b, and 910c determined by dividing the square first coding unit 900 in the vertical direction, or the second coding units 920a, 920b, and 920c determined by dividing the square first coding unit 900 in the horizontal direction.

[0157] According to an embodiment, the image decoding apparatus 100 can determine whether the second coding units 910a, 910b, 910c, 920a, 920b, and 920c included in the first coding unit 900 satisfy a condition for processing in a certain order, and the condition is related to whether at least one of the width and the height of the first coding unit 900 will be bisected along the boundaries of the second coding units 910a, 910b, 910c, 920a, 920b, and 920c. Referring to FIG. 9 Since the boundaries of the second coding units 910a, 910b, and 910c determined by dividing the square first coding unit 900 in the vertical direction do not bisect the width of the first coding unit 900, it can be determined that the first coding unit 900 does not satisfy the condition for processing in a certain order. Also, since the boundaries of the second coding units 920a, 920b, and 920c determined by dividing the square first coding unit 900 in the horizontal direction do not bisect the width of the first coding unit 900, it can be determined that the first coding unit 900 does not satisfy the condition for processing in a certain order. When the condition is not satisfied as described above, the image decoding apparatus 100 can determine that the scan order is not continuous, and can determine, based on the determination result, that the first coding unit 900 will be divided into an odd number of coding units. According to an embodiment, when a coding unit is divided into an odd number of coding units, the image decoding apparatus 100 can impose a certain restriction on a coding unit at a certain position among the divided coding units. The restriction or the certain position has been described above through various embodiments, and thus a detailed description thereof will be omitted.

[0158] According to an embodiment, the image decoding apparatus 100 can determine various shapes of coding units by partitioning the first coding unit.

[0159] Referring to FIG. 10 , the image decoding apparatus 100 can partition the square first coding unit 900 or the non-square first coding unit 930 or 950 into various shapes of coding units.

[0160] FIG. 11 It is shown that, according to an embodiment, when the second coding unit having a non-square shape determined by the image decoding apparatus partitioning the first coding unit satisfies a certain condition, the shape into which the second coding unit can be partitioned is limited.

[0161] According to an embodiment, the image decoding apparatus 100 can determine to partition the square first coding unit 1000 into non-square second coding units 1010a and 1010b, or 1020a and 1020b, based on the partition shape mode information obtained through the receiver 110. The second coding units 1010a and 1010b, or 1020a and 1020b, can be independently partitioned. Accordingly, the image decoding apparatus 100 can determine to partition each of the second coding units 1010a and 1010b, or 1020a and 1020b, into a plurality of coding units or not to partition each of the second coding units 1010a and 1010b, or 1020a and 1020b, based on the partition shape mode information of each of the second coding units 1010a and 1010b, or 1020a and 1020b. According to an embodiment, the image decoding apparatus 100 can determine third coding units 1012a and 1012b by partitioning the left second coding unit 1010a determined by partitioning the first coding unit 1000 in the vertical direction in the horizontal direction. However, when the left second coding unit 1010a is partitioned in the horizontal direction, the image decoding apparatus 100 can limit the right second coding unit 1010b from being partitioned in the horizontal direction in which the left second coding unit 1010a is partitioned. When third coding units 1014a and 1014b are determined by partitioning the right second coding unit 1010b in the same direction, because the left second coding unit 1010a and the right second coding unit 1010b are independently partitioned in the horizontal direction, the third coding units 1012a, 1012b, 1014a, and 1014b can be determined. However, this case acts the same as the case in which the image decoding apparatus 100 partitions the first coding unit 1000 into four square second coding units 1030a, 1030b, 1030c, and 1030d based on the partition shape mode information, and can be inefficient in terms of image decoding.

[0162] According to an embodiment, the image decoding apparatus 100 can determine third coding units 1022a and 1022b, or 1024a and 1024b by dividing a non-square second coding unit 1020a or 1020b determined by dividing the first coding unit 1000 in the horizontal direction in the vertical direction. However, when a second coding unit (e.g., the upper second coding unit 1020a) is divided in the vertical direction, the image decoding apparatus 100 can restrict another second coding unit (e.g., the lower second coding unit 1020b) from being divided in the vertical direction in which the upper second coding unit 1020a is divided, for the above-described reason.

[0163] FIG. 12 A process in which the image decoding apparatus divides a square coding unit when division shape mode information indicates that the square coding unit is not to be divided into four square coding units according to an embodiment is illustrated.

[0164] According to an embodiment, the image decoding apparatus 100 can determine second coding units 1110a and 1110b, or 1120a and 1120b, etc. by dividing the first coding unit 1100 based on division shape mode information. The division shape mode information can include information on various methods of dividing coding units, but the information on the various division methods can not include information for dividing a coding unit into four square coding units. According to such division shape mode information, the image decoding apparatus 100 can not divide a square first coding unit 1100 into four square coding units 1130a, 1130b, 1130c, and 1130d. Based on the division shape mode information, the image decoding apparatus 100 can determine non-square second coding units 1110a and 1110b, or 1120a and 1120b, etc.

[0165] According to an embodiment, the image decoding apparatus 100 can independently divide non-square second coding units 1110a and 1110b, or 1120a and 1120b, etc. Each of the second coding units 1110a and 1110b, or 1120a and 1120b, etc. can be recursively divided in a certain order, and the division method can correspond to a method of dividing the first coding unit 1100 based on the division shape mode information.

[0166] For example, the image decoding apparatus 100 can determine the square third coding units 1112a and 1112b by dividing the left second coding unit 1110a in the horizontal direction, and can determine the square third coding units 1114a and 1114b by dividing the right second coding unit 1110b in the horizontal direction. Also, the image decoding apparatus 100 can determine the square third coding units 1116a, 1116b, 1116c, and 1116d by dividing both the left second coding unit 1110a and the right second coding unit 1110b in the horizontal direction. In this case, coding units having the same shape as the four square second coding units 1130a, 1130b, 1130c, and 1130d divided from the first coding unit 1100 can be determined.

[0167] As another example, the image decoding apparatus 100 can determine the square third coding units 1122a and 1122b by dividing the upper second coding unit 1120a in the vertical direction, and can determine the square third coding units 1124a and 1124b by dividing the lower second coding unit 1120b in the vertical direction. Also, the image decoding apparatus 100 can determine the square third coding units 1126a, 1126b, 1126c, and 1126d by dividing both the upper second coding unit 1120a and the lower second coding unit 1120b in the vertical direction. In this case, coding units having the same shape as the four square second coding units 1130a, 1130b, 1130c, and 1130d divided from the first coding unit 1100 can be determined.

[0168] FIG. 12 The order of processing between a plurality of coding units according to an embodiment is shown, and can vary according to processing of dividing a coding unit.

[0169] According to an embodiment, the image decoding apparatus 100 can divide the first coding unit 1200 based on the division shape mode information. When the block shape indicates a square shape and the division shape mode information indicates to divide the first coding unit 1200 in at least one of the horizontal direction and the vertical direction, the image decoding apparatus 100 can determine second coding units (e.g., the second coding units 1210a and 1210b, or 1220a and 1220b, etc.) by dividing the first coding unit 1200. Referring to FIG. 12, the image decoding apparatus 100 can determine the second coding units 1210a and 1210b by dividing the first coding unit 1200 in the horizontal direction, and can determine the second coding units 1220a and 1220b by dividing the first coding unit 1200 in the vertical direction. FIG. 11, the second coding units 1210a and 1210b, or 1220a and 1220b, which are non-square determined by dividing the first coding unit 1200 in only the horizontal direction or the vertical direction, can be independently divided based on the division shape mode information of each coding unit. For example, the image decoding apparatus 100 can determine the third coding units 1216a, 1216b, 1216c, and 1216d by dividing the second coding units 1210a and 1210b generated by dividing the first coding unit 1200 in the vertical direction in the horizontal direction, and can determine the third coding units 1226a, 1226b, 1226c, and 1226d by dividing the second coding units 1220a and 1220b generated by dividing the first coding unit 1200 in the horizontal direction in the horizontal direction. The above has been described with reference to FIG. 7 The process of dividing the second coding units 1210a and 1210b, or 1220a and 1220b is described, and thus a detailed description thereof will be omitted.

[0170] According to an embodiment, the image decoding apparatus 100 can process the coding units in a certain order. The above has been described with reference to FIG. 12 The operation of processing the coding units in a predetermined order is described, and thus a detailed description thereof will be omitted. Reference is made to FIG. 12 , the image decoding apparatus 100 can determine the four square third coding units 1216a, 1216b, 1216c, and 1216d and 1226a, 1226b, 1226c, and 1226d by dividing the square first coding unit 1200. According to an embodiment, the image decoding apparatus 100 can determine the processing order of the third coding units 1216a, 1216b, 1216c, and 1216d and 1226a, 1226b, 1226c, and 1226d based on the division shape in which the first coding unit 1200 is divided.

[0171] According to an embodiment, the image decoding apparatus 100 can determine the third coding units 1216a, 1216b, 1216c, and 1216d by dividing the second coding units 1210a and 1210b generated by dividing the first coding unit 1200 in the vertical direction in the horizontal direction, and can process the third coding units 1216a, 1216b, 1216c, and 1216d in the following processing order 1217: first processing the third coding units 1216a and 1216c included in the left second coding unit 1210a in the vertical direction, and then processing the third coding units 1216b and 1216d included in the right second coding unit 1210b in the vertical direction.

[0172] According to an embodiment, the image decoding apparatus 100 can determine the third coding units 1226a, 1226b, 1226c, and 1226d by dividing the second coding units 1220a and 1220b generated by dividing the first coding unit 1200 in the horizontal direction in the vertical direction, and can process the third coding units 1226a, 1226b, 1226c, and 1226d in the following processing order 1227: first, process the third coding units 1226a and 1226b included in the upper second coding unit 1220a in the horizontal direction, and then process the third coding units 1226c and 1226d included in the lower second coding unit 1220b in the horizontal direction.

[0173] Referring to FIG. 13 The square third coding units 1216a, 1216b, 1216c, and 1216d and 1226a, 1226b, 1226c, and 1226d can be determined by dividing the second coding units 1210a and 1210b and 1220a and 1220b, respectively. Although the second coding units 1210a and 1210b are determined by dividing the first coding unit 1200 in the vertical direction differently from the second coding units 1220a and 1220b determined by dividing the first coding unit 1200 in the horizontal direction, the third coding units 1216a, 1216b, 1216c, and 1216d and 1226a, 1226b, 1226c, and 1226d divided from the second coding units 1210a and 1210b and the second coding units 1220a and 1220b finally show the same shape of coding units divided from the first coding unit 1200. As such, by recursively dividing coding units differently based on the division shape mode information, even if the coding units are finally determined to have the same shape, the image decoding apparatus 100 can process a plurality of coding units in different orders.

[0174] FIG. 13 A process of determining a depth of a coding unit when a shape and a size of the coding unit are changed when the coding unit is recursively divided to determine a plurality of coding units according to an embodiment is shown.

[0175] According to an embodiment, the image decoding apparatus 100 can determine a depth of a coding unit based on a certain criterion. For example, the certain criterion can be a length of a long side of the coding unit. When a length of a long side of a coding unit before being divided is 2n (n>0) times a length of a long side of a current coding unit after being divided, the image decoding apparatus 100 can determine that the depth of the current coding unit is increased by n from the depth of the coding unit before being divided. Hereinafter, a coding unit having an increased depth is denoted as a coding unit of a lower depth.

[0176] Referring toFIG. 14 According to an embodiment, the image decoding apparatus 100 can determine the second coding unit 1302 and the third coding unit 1304 of a lower depth by dividing the square first coding unit 1300 based on the block shape information indicating a square shape (e.g., the block shape information can be represented as "0:SQUARE"). Assuming that the size of the square first coding unit 1300 is 2Nx2N, the second coding unit 1302 determined by dividing the width and height of the first coding unit 1300 by 1 / 2 can have a size of NxN. Also, the third coding unit 1304 determined by dividing the width and height of the second coding unit 1302 by 1 / 2 can have a size of N / 2xN / 2. In this case, the width and height of the third coding unit 1304 are 1 / 4 times the width and height of the first coding unit 1300. When the depth of the first coding unit 1300 is D, the depth of the second coding unit 1302 having a width and height that are 1 / 2 times the width and height of the first coding unit 1300 can be D+1, and the depth of the third coding unit 1304 having a width and height that are 1 / 4 times the width and height of the first coding unit 1300 can be D+2.

[0177] According to an embodiment, the image decoding apparatus 100 can determine the second coding unit 1312 or 1322 and the third coding unit 1314 or 1324 of a lower depth by dividing the non-square first coding unit 1310 or 1320 based on the block shape information indicating a non-square shape (e.g., the block shape information can be represented as "1:NS_VER" indicating a non-square shape in which the height is longer than the width, or can be represented as "2:NS_HOR" indicating a non-square shape in which the width is longer than the height).

[0178] The image decoding apparatus 100 can determine the second coding unit 1302, 1312, or 1322 by dividing at least one of the width and height of the first coding unit 1310 having a size of Nx2N. That is, the image decoding apparatus 100 can determine the second coding unit 1302 having a size of NxN or the second coding unit 1322 having a size of NxN / 2 by dividing the first coding unit 1310 in the horizontal direction, or can determine the second coding unit 1312 having a size of N / 2xN by dividing the first coding unit 1310 in the horizontal and vertical directions.

[0179] According to an embodiment, the image decoding apparatus 100 can determine a second coding unit (e.g., 1302, 1312, or 1322) by dividing at least one of the width and the height of the first coding unit 1320 having a size of 2N×N. That is, the image decoding apparatus 100 can determine a second coding unit 1302 having a size of N×N or a second coding unit 1312 having a size of N / 2×N by dividing the first coding unit 1320 in a vertical direction, or can determine a second coding unit 1322 having a size of N×N / 2 by dividing the first coding unit 1320 in a horizontal direction and a vertical direction.

[0180] According to an embodiment, the image decoding apparatus 100 can determine a third coding unit (e.g., 1304, 1314, or 1324) by dividing at least one of the width and the height of the second coding unit 1302 having a size of N×N. That is, the image decoding apparatus 100 can determine a third coding unit 1304 having a size of N / 2×N / 2, a third coding unit 1314 having a size of N / 4×N / 2, or a third coding unit 1324 having a size of N / 2×N / 4 by dividing the second coding unit 1302 in a vertical direction and a horizontal direction.

[0181] According to an embodiment, the image decoding apparatus 100 can determine a third coding unit (e.g., 1304, 1314, or 1324) by dividing at least one of the width and the height of the second coding unit 1312 having a size of N / 2×N. That is, the image decoding apparatus 100 can determine a third coding unit 1304 having a size of N / 2×N / 2 or a third coding unit 1324 having a size of N / 2×N / 4 by dividing the second coding unit 1312 in a horizontal direction, or can determine a third coding unit 1314 having a size of N / 4×N / 2 by dividing the second coding unit 1312 in a vertical direction and a horizontal direction.

[0182] According to an embodiment, the image decoding apparatus 100 can determine a third coding unit (e.g., 1304, 1314, or 1324, etc.) by dividing at least one of the width and the height of the second coding unit 1322 having a size of N×N / 2. That is, the image decoding apparatus 100 can determine a third coding unit 1304 having a size of N / 2×N / 2 or a third coding unit 1314 having a size of N / 4×N / 2 by dividing the second coding unit 1322 in a vertical direction, or can determine a third coding unit 1324 having a size of N / 2×N / 4 by dividing the second coding unit 1322 in a vertical direction and a horizontal direction.

[0183] According to an embodiment, the image decoding apparatus 100 can divide a square coding unit (e.g., 1300, 1302, or 1304) in a horizontal direction or a vertical direction. For example, the image decoding apparatus 100 can determine a first coding unit 1310 having a size of N×2N by dividing the first coding unit 1300 having a size of 2N×2N in a vertical direction, or can determine a first coding unit 1320 having a size of 2N×N by dividing the first coding unit 1300 in a horizontal direction. According to an embodiment, when a depth is determined based on a length of a longest side of a coding unit, the depth of a coding unit determined by dividing the first coding unit 1300 having a size of 2N×2N in a horizontal direction or a vertical direction can be the same as the depth of the first coding unit 1300.

[0184] According to an embodiment, the width and height of the third coding unit 1314 or 1324 can be 1 / 4 times the width and height of the first coding unit 1310 or 1320. When the depth of the first coding unit 1310 or 1320 is D, the depth of the second coding unit 1312 or 1322 having a width and height that are 1 / 2 times the width and height of the first coding unit 1310 or 1320 can be D+1, and the depth of the third coding unit 1314 or 1324 having a width and height that are 1 / 4 times the width and height of the first coding unit 1310 or 1320 can be D+2.

[0185] FIG. 14 A depth that can be determined based on a shape and size of a coding unit and a partial index (PID) for distinguishing coding units are shown according to an embodiment.

[0186] According to an embodiment, the image decoding apparatus 100 can determine second coding units having various shapes by dividing a square first coding unit 1400. Referring to FIG. 14 , the image decoding apparatus 100 can determine the second coding units 1402a and 1402b, 1404a and 1404b, and 1406a, 1406b, 1406c, and 1406d by dividing the first coding unit 1400 in at least one of a vertical direction and a horizontal direction based on the division shape mode information. That is, the image decoding apparatus 100 can determine the second coding units 1402a and 1402b, 1404a and 1404b, and 1406a, 1406b, 1406c, and 1406d based on the division shape mode information of the first coding unit 1400.

[0187] According to an embodiment, depths of the second coding units 1402a and 1402b, 1404a and 1404b, and 1406a, 1406b, 1406c, and 1406d determined based on the partition shape mode information of the square-shaped first coding unit 1400 can be determined based on lengths of their long sides. For example, because the length of the side of the square-shaped first coding unit 1400 is equal to the length of the long side of the non-square-shaped second coding units 1402a and 1402b and 1404a and 1404b, the first coding unit 1400 and the non-square-shaped second coding units 1402a and 1402b and 1404a and 1404b can have the same depth, e.g., D. However, when the image decoding apparatus 100 partitions the first coding unit 1400 into four square-shaped second coding units 1406a, 1406b, 1406c, and 1406d based on the partition shape mode information, because the length of the side of the square-shaped second coding units 1406a, 1406b, 1406c, and 1406d is 1 / 2 times the length of the side of the first coding unit 1400, the depths of the second coding units 1406a, 1406b, 1406c, and 1406d can be D+1, which is 1 deeper than the depth D of the first coding unit 1400.

[0188] According to an embodiment, the image decoding apparatus 100 can determine a plurality of second coding units 1412a and 1412b and 1414a, 1414b, and 1414c by partitioning a first coding unit 1410 having a height longer than a width in a horizontal direction based on the partition shape mode information. According to an embodiment, the image decoding apparatus 100 can determine a plurality of second coding units 1422a and 1422b and 1424a, 1424b, and 1424c by partitioning a first coding unit 1420 having a width longer than a height in a vertical direction based on the partition shape mode information.

[0189] According to an embodiment, depths of the second coding units 1412a and 1412b and 1414a, 1414b, and 1414c, or 1422a and 1422b and 1424a, 1424b, and 1424c determined based on the partition shape mode information of the non-square-shaped first coding unit 1410 or 1420 can be determined based on lengths of their long sides. For example, because the length of the side of the square-shaped second coding units 1412a and 1412b is 1 / 2 times the length of the long side of the first coding unit 1410 having a non-square shape with a height longer than a width, the depth of the square-shaped second coding units 1412a and 1412b is D+1, which is 1 deeper than the depth D of the non-square-shaped first coding unit 1410.

[0190] Also, the image decoding apparatus 100 can divide the first coding unit 1410, which is not a square, into an odd number of second coding units 1414a, 1414b, and 1414c based on the division shape mode information. The odd number of second coding units 1414a, 1414b, and 1414c can include the second coding units 1414a and 1414c, which are not squares, and the second coding unit 1414b, which is a square. In this case, because the length of the long side of the second coding units 1414a and 1414c, which are not squares, and the length of the side of the second coding unit 1414b, which is a square, are 1 / 2 times the length of the long side of the first coding unit 1410, the depth of the second coding units 1414a, 1414b, and 1414c can be D+1, which is 1 deeper than the depth D of the first coding unit 1410, which is not a square. The image decoding apparatus 100 can determine the depth of the coding units divided from the first coding unit 1420 having a non-square shape in which the width is longer than the height by using the above-described method of determining the depth of the coding units divided from the first coding unit 1410.

[0191] According to an embodiment, when the odd number of divided coding units do not have the same size, the image decoding apparatus 100 can determine the PIDs for identifying the divided coding units based on a size ratio between the coding units. Referring to FIG. 14 , the coding unit 1414b at the center position among the odd number of divided coding units 1414a, 1414b, and 1414c can have a width equal to the widths of the other coding units 1414a and 1414c and a height twice the heights of the other coding units 1414a and 1414c. That is, in this case, the coding unit 1414b at the center position can include two other coding units 1414a or 1414c. Accordingly, when the PID of the coding unit 1414b at the center position is 1 based on the scanning order, the PID of the coding unit 1414c adjacent to the coding unit 1414b can increase by 2 and thus can be 3. That is, there can be a discontinuity in the PID values. According to an embodiment, the image decoding apparatus 100 can determine whether the odd number of divided coding units do not have the same size based on whether there is a discontinuity in the PIDs for identifying the divided coding units.

[0192] According to an embodiment, the image decoding apparatus 100 can determine whether to use a specific division method based on the PID values for identifying a plurality of coding units determined by dividing a current coding unit. Referring to FIG. 14, the image decoding apparatus 100 can determine an even number of coding units 1412a and 1412b or an odd number of coding units 1414a, 1414b, and 1414c by dividing the first coding unit 1410 having a rectangular shape with a height longer than a width. The image decoding apparatus 100 can use a PID indicating each coding unit in order to identify each coding unit. According to an embodiment, the PID can be obtained from a sample at a specific position (e.g., a top-left sample) of each coding unit.

[0193] According to an embodiment, the image decoding apparatus 100 can determine a coding unit at a specific position among divided coding units by using a PID for distinguishing the coding units. According to an embodiment, when the division shape mode information of the first coding unit 1410 having a rectangular shape with a height longer than a width indicates that the coding unit is divided into three coding units, the image decoding apparatus 100 can divide the first coding unit 1410 into three coding units 1414a, 1414b, and 1414c. The image decoding apparatus 100 can assign a PID to each of the three coding units 1414a, 1414b, and 1414c. The image decoding apparatus 100 can compare the PIDs of the odd number of divided coding units in order to determine a coding unit at a center position among the coding units. The image decoding apparatus 100 can determine the coding unit 1414b having a PID corresponding to a middle value among the PIDs of the coding units as a coding unit at a specific position among the coding units determined by dividing the first coding unit 1410. According to an embodiment, when the divided coding units do not have the same size, the image decoding apparatus 100 can determine a PID for distinguishing the divided coding units based on a size ratio between the coding units. Referring to FIG. 15The width of the coding unit 1414b generated by dividing the first coding unit 1410 can be equal to the widths of the other coding units 1414a and 1414c, and the height thereof can be twice the heights of the other coding units 1414a and 1414c. In this case, when the PID of the coding unit 1414b at the center position is 1, the PID of the coding unit 1414c adjacent to the coding unit 1414b can increase by 2 and thus can be 3. When the PID is not uniformly increased as described above, the image decoding apparatus 100 can determine that the coding unit is divided into a plurality of coding units including a coding unit having a size different from those of the other coding units. According to an embodiment, when the division shape mode information indicates that the coding unit is divided into an odd number of coding units, the image decoding apparatus 100 can divide the current coding unit in such a manner that a coding unit at a specific position (e.g., a coding unit at the center position) among the odd number of coding units has a size different from those of the other coding units. In this case, the image decoding apparatus 100 can determine the coding unit at the center position having the different size by using the PID of the coding unit. However, the PID and the size or position of the coding unit at the specific position to be determined are not limited to the above-described example, and various PIDs of the coding units and various positions and sizes can be used.

[0194] According to an embodiment, the image decoding apparatus 100 can use a specific data unit in which the coding unit is recursively divided starts.

[0195] FIG. 15 A determination of a plurality of coding units based on a plurality of specific data units included in a picture according to an embodiment is illustrated.

[0196] According to an embodiment, the specific data unit can be defined as a data unit in which the coding unit is recursively divided starts by using the division shape mode information. That is, the specific data unit can correspond to a coding unit of a highest depth that is used to determine a plurality of coding units divided from a current picture. In the following description, the specific data unit is referred to as a reference data unit for convenience of explanation.

[0197] According to an embodiment, the reference data unit can have a specific size and a specific shape. According to an embodiment, the reference data unit can include MxN samples. Here, M and N can be equal to each other and can be an integer expressed as a power of 2. That is, the reference data unit can have a square shape or a non-square shape and can be divided into an integer number of coding units.

[0198] According to an embodiment, the image decoding apparatus 100 can divide the current picture into a plurality of reference data units. According to an embodiment, the image decoding apparatus 100 can divide the plurality of reference data units divided from the current picture by using the division shape mode information of each of the reference data units. The process of dividing the reference data units can correspond to the division process using the quad-tree structure.

[0199] According to an embodiment, the image decoding apparatus 100 can determine a minimum size allowed for the reference data units included in the current picture in advance. Accordingly, the image decoding apparatus 100 can determine the reference data units having various sizes equal to or greater than the minimum size, and can determine one or more coding units by using the division shape mode information with reference to the determined reference data units.

[0200] Referring to FIG. 3 , the image decoding apparatus 100 can use the square reference coding unit 1500 or the non-square reference coding unit 1502. According to an embodiment, the shape and size of the reference coding unit can be determined based on various data units (e.g., sequence, picture, slice, slice segment, parallel block, parallel block group, largest coding unit, etc.) capable of including one or more reference coding units.

[0201] According to an embodiment, the receiver 110 of the image decoding apparatus 100 can obtain at least one of the reference coding unit shape information and the reference coding unit size information for each of the various data units from the bitstream. The process of dividing the square reference coding unit 1500 into one or more coding units has been described above with respect to the process of dividing the current coding unit 300 of FIG. 4 , and the process of dividing the non-square reference coding unit 1502 into one or more coding units has been described above with respect to the process of dividing the current coding unit 400 or 450 of FIG. 12 , and thus a detailed description thereof will be omitted.

[0202] According to an embodiment, the image decoding apparatus 100 can use an index for identifying a size and a shape of a reference coding unit to determine a size and a shape of a reference coding unit from some data units determined based on a certain condition. That is, the receiver 110 can obtain only an index for identifying a size and a shape of a reference coding unit for each slice, slice segment, parallel block, parallel block group, or largest coding unit, which is a data unit satisfying a certain condition (e.g., a data unit having a size equal to or smaller than a slice) among various data units (e.g., sequence, picture, slice, slice segment, parallel block, parallel block group, largest coding unit, etc.) from a bitstream. The image decoding apparatus 100 can determine a size and a shape of a reference data unit for each data unit satisfying a certain condition by using the index. When a reference coding unit shape information and a reference coding unit size information are obtained from a bitstream according to each data unit having a relatively small size and used, efficiency of using a bitstream can not be high, and thus, the index can be obtained and used only, instead of directly obtaining the reference coding unit shape information and the reference coding unit size information. In this case, at least one of a size and a shape of a reference coding unit corresponding to the index for identifying a size and a shape of a reference coding unit can be determined in advance. That is, the image decoding apparatus 100 can determine at least one of a size and a shape of a reference coding unit included in a data unit used as a unit for obtaining an index by selecting at least one of a size and a shape of a reference coding unit determined based on the index in advance.

[0203] According to an embodiment, the image decoding apparatus 100 can use one or more reference coding units included in a largest coding unit. That is, a largest coding unit divided from a picture can include one or more reference coding units, and a coding unit can be determined by recursively dividing each reference coding unit. According to an embodiment, at least one of a width and a height of a largest coding unit can be an integer multiple of at least one of a width and a height of a reference coding unit. According to an embodiment, a size of a reference coding unit can be obtained by dividing a largest coding unit n times based on a quad tree structure. That is, according to various embodiments, the image decoding apparatus 100 can determine a reference coding unit by dividing a largest coding unit n times based on a quad tree structure, and can divide the reference coding unit based on at least one of block shape information and division shape mode information.

[0204] According to an embodiment, the image decoding apparatus 100 can obtain block shape information indicating a shape of a current coding unit or partition shape mode information indicating a partitioning method of the current coding unit from a bitstream, and can use the obtained information. The partition shape mode information can be included in the bitstream in relation to various data units. For example, the image decoding apparatus 100 can use the partition shape mode information included in a sequence parameter set, a picture parameter set, a video parameter set, a slice header, a slice segment header, a tile header, or a tile group header. Also, the image decoding apparatus 100 can obtain syntax information (e.g., syntax elements) corresponding to the block shape information or the partition shape mode information from the bitstream according to each largest coding unit, each reference coding unit, or each processing block, and can use the obtained syntax elements.

[0205] Hereinafter, a method of determining a partitioning rule according to an embodiment of the disclosure will be described in detail.

[0206] The image decoding apparatus 100 can determine a partitioning rule of an image. The partitioning rule can be determined in advance between the image decoding apparatus 100 and the image encoding apparatus 200. The image decoding apparatus 100 can determine the partitioning rule of the image based on information obtained from a bitstream. The image decoding apparatus 100 can determine the partitioning rule based on information obtained from at least one of a sequence parameter set, a picture parameter set, a video parameter set, a slice header, a slice segment header, a tile header, and a tile group header. The image decoding apparatus 100 can determine the partitioning rule differently according to a frame, a slice, a tile, a temporal layer, a largest coding unit, or a coding unit.

[0207] The image decoding apparatus 100 can determine a partitioning rule based on a block shape of a coding unit. The block shape can include a size, a shape, a ratio of a width to a height, and a direction of the coding unit. The image decoding apparatus 100 can determine the partitioning rule in advance based on block shape information of the coding unit. However, embodiments are not limited thereto. The image decoding apparatus 100 can determine the partitioning rule based on information obtained from a received bitstream.

[0208] A shape of a coding unit can include a square and a non-square. When a width length and a height length of the coding unit are the same, the image decoding apparatus 100 can determine the shape of the coding unit as a square. Also, when the width length and the height length of the coding unit are not the same, the image decoding apparatus 100 can determine the shape of the coding unit as a non-square.

[0209] The size of the coding unit can include various sizes such as 4x4, 8x4, 4x8, 8x8, 16x4, 16x8, …, and 256x256. The size of the coding unit can be classified based on a long side length, a short side length, or an area of the coding unit. The image decoding apparatus 100 can apply the same division rule to the coding units classified into the same group. For example, the image decoding apparatus 100 can classify the coding units having the same long side length as having the same size. Also, the image decoding apparatus 100 can apply the same division rule to the coding units having the same long side length.

[0210] The ratio of the width and the height of the coding unit can include 1:2, 2:1, 1:4, 4:1, 1:8, 8:1, 1:16, 16:1, 32:1, 1:32, etc. Also, the direction of the coding unit can include a horizontal direction and a vertical direction. The horizontal direction can indicate a case where the width length of the coding unit is longer than the height length of the coding unit. The vertical direction can indicate a case where the width length of the coding unit is shorter than the height length of the coding unit.

[0211] The image decoding apparatus 100 can adaptively determine the division rule based on the size of the coding unit. The image decoding apparatus 100 can differently determine the allowable division shape mode based on the size of the coding unit. For example, the image decoding apparatus 100 can determine whether to allow division based on the size of the coding unit. The image decoding apparatus 100 can determine the division direction according to the size of the coding unit. The image decoding apparatus 100 can determine the allowable division type according to the size of the coding unit.

[0212] The division rule determined based on the size of the coding unit can be a division rule determined in advance at the image decoding apparatus 100. Also, the image decoding apparatus 100 can determine the division rule based on information obtained from the bitstream.

[0213] The image decoding apparatus 100 can adaptively determine the division rule based on the position of the coding unit. The image decoding apparatus 100 can adaptively determine the division rule based on the position of the coding unit in the image.

[0214] Also, the image decoding apparatus 100 can determine the division rule such that the coding units generated via different division paths do not have the same block shape. However, embodiments are not limited thereto, and the coding units generated via different division paths have the same block shape. The coding units generated via different division paths can have different decoding processing orders. Since the decoding processing order has been described above with reference to FIG. 16 A detailed description thereof will be omitted.

[0215] FIG. 5 is a block diagram of an image encoding and decoding system.

[0216] The encoding end 1610 (e.g., the image encoding apparatus 200) of the image encoding and decoding system 1600 transmits an encoded bitstream of an image, and the decoding end 1650 (e.g., the image decoding apparatus 100) of the image encoding and decoding system 1600 receives and decodes the bitstream to output a reconstructed image. The decoding end 1650 can be a configuration similar to the image decoding apparatus 100.

[0217] In the encoding end 1610, the prediction encoder 1615 outputs a reference image through inter prediction and intra prediction, and the transformer and quantizer 1616 quantizes residual data between the reference image and a current input image into quantized transform coefficients and outputs the quantized transform coefficients. The entropy encoder 1625 encodes the quantized transform coefficients and outputs the encoded quantized transform coefficients as a bitstream. The quantized transform coefficients are reconstructed into data in a spatial domain via the inverse quantizer and inverse transformer 1630, and the reconstructed data in the spatial domain is output as a reconstructed image via the deblocking filter 1635 and the loop filter 1640. The reconstructed image can be used as a reference image for a next input image in the prediction encoder 1615.

[0218] The encoded image data among the bitstream received by the decoding end 1650 is reconstructed into residual data in a spatial domain via the entropy decoder 1655 and the inverse quantizer and inverse transformer 1660. When the reference image output by the prediction decoder 1675 and the residual data are combined, the image data in the spatial domain can be constructed, and the deblocking filter 1665 and the loop filter 1670 can output a reconstructed image for a current original image by performing filtering on the image data in the spatial domain. The reconstructed image can be used as a reference image for a next original image by the prediction decoder 1675.

[0219] The loop filter 1640 of the encoding end 1610 performs loop filtering by using filter information input according to a user input or a system setting. The filter information used by the loop filter 1640 is output to the entropy encoder 1610 and transmitted to the decoding end 1650 together with the encoded image data. The loop filter 1670 of the decoding end 1650 can perform loop filtering based on filter information input from the decoding end 1650.

[0220] In the present disclosure, a "tree structure" can refer to a hierarchical structure of one or more coding units formed according to whether a split mode of a coding unit is quad split, binary split, ternary split, or no split. For example, a hierarchical structure of blocks generated from a current coding unit according to a split process of The method according to the present disclosure The hierarchical structure of blocks generated from a current coding unit according to a split process of

[0221] In the disclosure, "availability of a block" means whether a block has been encoded or decoded and thus information of the block can be obtained. In detail, in an encoding process, when a current block has been encoded, a neighboring block can be encoded using the encoded information of the current block, and thus the current block can be shown as available. When the current block has not been encoded, the current block can be shown as unavailable. Likewise, in a decoding process, when a current block has been decoded, a neighboring block can be decoded using the encoded information of the current block, and thus the current block can be shown as available. When the current block has not been decoded, the current block can be shown as unavailable.

[0222] In the disclosure, "availability of motion information of a block" means whether motion prediction for a block (prediction other than prediction according to an intra mode or an intra block copy mode) can be performed and thus motion information (a motion vector, a prediction direction (L0-pred, L1-pred, or Bi-pred), and a reference picture index) of the block can be obtained. In detail, in an encoding process, when motion prediction has been performed for a current block and motion information of the current block exists, motion prediction of a neighboring block can be performed using the motion information of the current block, and thus the motion information of the current block can be shown as available. In the encoding process, when motion prediction is not performed for the current block, the motion information of the current block can be shown as unavailable. Likewise, in a decoding process, when motion prediction has been performed for a current block and motion information of the current block exists, motion prediction of a neighboring block can be performed using the motion information of the current block, and thus the motion information of the current block can be shown as available. In the decoding process, when motion prediction is not performed for the current block, the motion information of the current block can be shown as unavailable.

[0223] In the disclosure, "merge candidate" can correspond to a motion vector corresponding to a neighboring block of a current block. Because a prediction motion vector of the current block is determined from a motion vector of the neighboring block, each prediction motion vector can correspond to the neighboring block. Therefore, in the disclosure, for convenience of explanation, "merge candidate" is described as corresponding to a motion vector of a neighboring block or corresponding to a neighboring block, and there is no difference in meaning between the two expressions.

[0224] In the disclosure, "affine merge candidate" can correspond to a control point vector corresponding to a neighboring block or a block group of a current block. Because the control point vector is determined from a motion vector of the neighboring block, or the control point vector is determined based on motion vectors of the neighboring blocks belonging to the block group, each control point vector can correspond to the corresponding neighboring block or the corresponding block group. Therefore, in the disclosure, for convenience of explanation, "affine merge candidate" is described as corresponding to a control point vector determined from a neighboring block or a block group or corresponding to a neighboring block or a block group, and there is no difference in meaning between the two expressions.

[0225] In the disclosure, a "motion vector prediction (MVP) candidate" can correspond to a motion vector of a neighboring block corresponding to a current block. Because a prediction motion vector of the current block is determined from motion vectors of the neighboring blocks, each prediction motion vector can correspond to a neighboring block. Therefore, in the disclosure, for convenience of explanation, the "MVP candidate" is described as corresponding to a motion vector of a neighboring block or corresponding to a neighboring block, and there is no difference in meaning between the two expressions.

[0226] A "merge candidate" is a neighboring block (or a motion vector of a neighboring block) used in a merge mode in an inter prediction method, and an "MVP candidate" corresponds to a neighboring block (or a motion vector of a neighboring block) used in an AMVP mode in the inter prediction method. In the merge mode, not only a motion vector of the merge candidate but also a prediction direction (L0-pred, L1-pred, or Bi-pred) and a reference picture index of the merge candidate can be used to determine a prediction direction and a reference picture index of the current block. On the other hand, in the AMVP mode, a motion vector of the MVP candidate can be used to determine a motion vector of the current block, but a prediction direction and a reference picture index of the current block can be determined separately from a prediction direction and a reference picture index of the MVP candidate.

[0227] FIG. 20 The disclosure proposes an image encoding method based on residual prediction, an apparatus for performing the image encoding method, an image decoding method, an apparatus for performing the image decoding method, a method of storing or obtaining a bitstream, an apparatus for performing the method of storing or obtaining a bitstream, and a non-transitory storage medium storing a bitstream.

[0228] Generally, a picture sequence (or a video) has a continuity characteristic, and thus there is a high similarity between pictures. Therefore, inter prediction that utilizes the similarity between different pictures provides a relatively high prediction performance, requires a relatively small amount of bits to encode a residual signal representing a difference between an original image and a predicted image, and provides a relatively high encoding efficiency. In contrast, in the case of intra prediction, a prediction value of a pixel of a current block in a current picture is determined using a decoded neighboring sample (or a sample of a neighboring block) of the current block as a reference sample. Since the decoded neighboring sample (or the sample of the neighboring block) of the current block generally has a feature similar to a feature of the current block, but cannot accurately represent the feature of the current block, the prediction performance of the intra prediction is relatively lower than that of the inter prediction, the number of bits required to encode the residual signal representing a difference between the original image and the predicted image is relatively large, and the encoding efficiency is relatively low.

[0229] The disclosure proposes a method of applying residual prediction to intra prediction as a method of reducing the number of bits required to encode a residual signal determined by intra prediction and improving coding efficiency. Although the disclosure describes the proposed method by focusing on intra prediction based on residual prediction, the proposed method according to the disclosure can also be applied equally / similarly to inter prediction. That is, the application of the proposed method of the disclosure is not limited to intra prediction.

[0230] In addition, image encoding and image decoding need to be performed in a corresponding manner so that the decoder can correctly reconstruct the encoded image. Therefore, the intra prediction mode used in image encoding is generally signaled to the decoder through a bitstream, and the intra prediction mode is signaled for each block of a specific unit, and thus a considerable number of bits can be required to signal the intra prediction mode.

[0231] As a method for reducing the number of bits required to signal the intra prediction mode, the disclosure proposes a method of determining the intra prediction mode at the decoder side without signaling the intra prediction mode through a bitstream.

[0232] In this specification, the terms are expressed as follows for convenience of explanation.

[0233] -C: current block -R: reference block -C org : original signal of current block -C pred : predicted signal of current block -C res : residual signal of current block -R rec : reconstructed signal of reference block -R pred : (simulated) predicted signal of reference block -R res : (simulated) residual signal of reference block -T C : template of current block -T R : template of reference block The simulated predicted signal refers to a signal (or a prediction factor or a prediction sample) predicted based on the intra prediction mode having the best prediction performance by predicting a block (e.g., a current block or a reference block) based on at least one of available intra prediction modes (e.g., see FIG. 21 , FIG. 20 and related descriptions), and in this specification, can be simply referred to as a predicted signal or a virtual predicted signal. In an example, the predicted signal R predmay be a real prediction signal used during reconstruction of the reference block. pred is not limited to the real prediction signal or the virtual prediction signal of the reference block. In an example, the prediction signal R pred may be a real prediction signal used during reconstruction of the reference block.

[0234] The real residual signal refers to a residual signal (or residual samples) determined using a signal or a prediction factor predicted based on an intra prediction mode having the best prediction performance among available intra prediction modes (for example, see FIG. 21 , FIG. 20 and related descriptions), and in this specification, can be simply referred to as a residual signal or a virtual residual signal. In an example, the residual signal R res may be a real residual signal or a virtual residual signal of the reference block. However, in the proposed method according to the disclosure, the residual signal R res is not limited to the real residual signal or the virtual residual signal of the reference block. In an example, the residual signal R res may be a real residual signal used during reconstruction of the reference block.

[0235] In this specification, that two blocks (for example, a current block and a reference block) correspond to each other can mean that the two blocks are (best) matched to each other. Similarly, that two pixel regions (for example, a first pixel region and a second pixel region) or two templates (for example, a template T C of the current block and a template T R of the reference block) correspond to each other can mean that the two pixel regions or the two templates are (best) matched to each other.

[0236] In this specification, a prediction value or a predicted value can be used interchangeably with a prediction factor.

[0237] In this specification, a block can include, but is not limited to, a coding unit (CU), a prediction unit (PU), a transform unit (TU), or a sub-block included in the block. The block can have any one of various forms. For example, the block can have a square or non-square shape.

[0238] In this specification, a spatial domain can refer to the same picture domain as a picture including a corresponding block, and a temporal domain can refer to a picture domain different from the picture including the corresponding block. Accordingly, a spatial domain of a block can refer to an inside of a picture including the block, and a temporal domain of the block can refer to an inside of a picture different from the picture including the block.

[0239] In this specification, for ease of explanation, as an example, the number of intra prediction modes (for example, see FIG. 20and related descriptions) are described as a particular number, and various other numbers of intra prediction modes can be provided. The range of reference samples (e.g., see FIG. 27 , FIG. 17 and related descriptions) can have a size equal to the size of the current block, or a size that is a predetermined multiple of the size of the current block. In addition, the reference sample line (or reference line) can be one line in contact with the current block, or can be at least one line among a plurality of lines including a line in contact with the current block and a line at a predetermined distance from the current block.

[0240] FIG. 17 is a flowchart of an image encoding method according to the proposed method of the disclosure. FIG. 17 The method shown in FIG. 17 is an example only, and the proposed method of the disclosure is not limited to FIG. 17 Examples. For example, the proposed method of the disclosure can exclude at least one operation shown in FIG. 17 , or can include an operation not shown in FIG. 17 .

[0241] Referring to FIG. 23 , the device can determine a reference block R corresponding to the current block C in a current picture including the current block C (operation 1702). For example, the reference block can be determined from a decoded region or a reconstructed region of the current picture. For example, the reference block can be determined from a predetermined portion of the decoded region or the reconstructed region of the current picture.

[0242] In an example, at operation 1702, the device can determine a reference block corresponding to the current block in the current picture based on template matching (e.g., see FIG. 25 , FIG. 26 , FIG. 29 and FIG. 23 and related descriptions). In this example, the operation of determining a reference block corresponding to the current block in the current picture can include determining a first pixel region adjacent to the current block in the current picture, determining a second pixel region corresponding (or matching) to the first pixel region adjacent to the current block in the current picture, and determining a reference block corresponding (or matching) to the second pixel region in the current picture (e.g., see FIG. 25 , FIG. 26 , FIG. 29 and FIG. 26 and related descriptions). In this example, the first pixel region represents a template T C for the current block, and can be referred to as a first template. In this example, the second pixel region represents a template T R for the reference block, and can be referred to as a second template.

[0243] In this example (e.g., the template matching based example), the first pixel region and the second pixel region can have various forms. In detail, the first pixel region can include at least one of a pixel region adjacent to a top of the current block, a pixel region adjacent to an upper left of the current block, or a pixel region adjacent to a left side of the current block, and the second pixel region can include at least one of a pixel region adjacent to a top of the reference block, a pixel region adjacent to an upper left of the reference block, or a pixel region adjacent to a left side of the reference block (e.g., see FIG. 26 and related descriptions thereof). In this example, the first pixel region adjacent to the current block can have a fixed size (e.g., 4 or 4xH_4x4_Wx4), or a size determined based on a size of the current block (e.g., W / 2 and H / 2, where W and H are a width and a height of the current block), or a width or a height same as a width or a height of a coding unit including the current block (e.g., 4xH and Wx4, where W and H are a width and a height of a coding unit including the current block), or a width or a height twice a width or a height of a coding unit including the current block (e.g., 4x2H and 2Wx4, where W and H are a width and a height of a coding unit including the current block) (e.g., see FIG. 26 and related descriptions thereof).

[0244] In this example (e.g., the template matching based example), the operation of determining the second pixel region corresponding to the first pixel region adjacent to the current block in the current picture can include calculating values of a cost function based on pixels of the first pixel region and pixels of a pixel region in a given search region (e.g., a reconstructed region of the current picture or a specific region within the reconstructed region of the current picture), and determining a pixel region corresponding to a lowest value or a highest value among the calculated values of the cost function as the second pixel region (e.g., see FIG. 26 and related descriptions thereof). As a more detailed example, the cost function can include at least one of a sum of absolute differences (SAD), a sum of squared differences (SSD), or a number of pixels having a same pixel value (e.g., see FIG. 24 and related descriptions thereof).

[0245] As another example, at operation 1702, the device can determine a reference block corresponding to the current block in the current picture based on a block vector (BV) (e.g., see FIG. 28 , FIG. 30 and FIG. 24 and related descriptions thereof). In this example, the operation of determining the reference block corresponding to the current block can include searching for a reference block matching the current block from a given search region of the current picture to obtain a BV of the current block, and determining the reference block in the current picture based on the BV of the current block (e.g., see FIG. 28 , FIG. 30 and FIG. 25and related descriptions).

[0246] The device can determine the residual signal R rec of the reference block based on the reconstructed signal R pred of the reference block and the prediction signal R res of the reference block (operation 1704) (e.g., see FIG. 28 , FIG. 29 , FIG. 30 and FIG. 25 and related descriptions). The prediction signal R pred of the reference block can be determined based on at least one reference pixel included in a neighboring block of the reference block in the current picture.

[0247] The device can determine the residual difference signal R org of the current block based on the original signal C pred of the current block, the prediction signal C res of the current block, and the residual signal R of the reference block (operation 1706) (e.g., see FIG. 28 , FIG. 29 , FIG. 30 and FIG. 25 and related descriptions). The prediction signal of the current block can be determined based on at least one reference pixel included in a neighboring block of the current block in the current picture.

[0248] In an example, at operation 1704, the device can determine an intra prediction mode of the reference block from a plurality of intra prediction modes based on the reconstructed signal R FIG. 28 of the reference block and at least one reference pixel included in a neighboring block of the reference block in the current picture, and can determine the prediction signal R FIG. 25 of the reference block based on the intra prediction mode of the reference block and at least one reference pixel included in a neighboring block of the reference block in the current picture (e.g., see FIG. 28 and FIG. 27 and related descriptions). In this example, the operation of determining the intra prediction mode of the reference block from the plurality of intra prediction modes can include calculating a function value of each of the plurality of intra prediction modes based on the reconstructed signal R FIG. 27and related descriptions). As a more detailed example, the function value can be a cost function and can include at least one of a sum of absolute differences (SAD), a sum of squared differences (SSD), or a number of pixels having the same pixel value (e.g., see FIG. 25 and related descriptions).

[0249] In an example, at operation 1704, when the reference block is encoded in the intra prediction mode, the device can identify the intra prediction mode of the reference block and can determine the prediction signal for the reference block based on the intra prediction mode of the reference block and the at least one reference pixel included in the neighboring blocks of the reference block in the current picture (e.g., see FIG. 28 and FIG. 25 and related descriptions). In this example, at operation 1706, the device can determine the intra prediction mode of the current block based on the pixel region adjacent to the current block from the plurality of intra prediction modes and can encode the intra prediction mode of the current block into the bitstream (or can signal the intra prediction mode of the current block via the bitstream). At operation 1706, the device can determine the intra prediction mode of the current block as the intra prediction mode of the reference block and can determine the prediction signal for the current block based on the intra prediction mode of the current block and the at least one reference pixel included in the neighboring blocks of the reference block in the current picture (e.g., see FIG. 28 and FIG. 29 and related descriptions).

[0250] In an example, at operation 1704, the device can determine the intra prediction mode of the current block based on a pixel region adjacent to the current block from the plurality of intra prediction modes and can encode the intra prediction mode of the current block into the bitstream (or can signal the intra prediction mode of the current block via the bitstream). At operation 1704, the device can determine the intra prediction mode of the current block as the intra prediction mode of the reference block and can determine the prediction signal for the reference block based on the intra prediction mode of the current block and the at least one reference pixel included in the neighboring blocks of the reference block in the current picture (e.g., see FIG. 30 and FIG. 29 and related descriptions). In this example, at operation 1706, the device can determine the prediction signal for the current block based on the intra prediction mode of the current block and the at least one reference pixel included in the neighboring blocks of the current block in the current picture (e.g., see FIG. 30 and FIG. 29 and related descriptions).

[0251] In an example, at operation 1704, the device can determine an intra prediction mode of the current block from the plurality of intra prediction modes based on the pixel region neighboring the current block, and can not encode the intra prediction mode of the current block into the bitstream (or can omit the operation of encoding the intra prediction mode of the current block into the bitstream). In this case, the decoder can determine the intra prediction mode of the current block by performing an operation corresponding to the operation performed at the encoder side without obtaining the intra prediction mode of the current block from the bitstream. At operation 1704, the device can determine the intra prediction mode of the current block as the intra prediction mode of the reference block, and can determine a prediction signal of the reference block based on the intra prediction mode of the current block and at least one reference pixel included in the neighboring block of the reference block in the current picture (e.g., see FIG. 30 and FIG. 29 and related descriptions thereof). In this example, at operation 1706, the device can determine a prediction signal of the current block based on the intra prediction mode of the current block and at least one reference pixel included in the neighboring block of the current block in the current picture (e.g., see FIG. 30 and FIG. 17 and related descriptions thereof).

[0252] The device can process the residual difference signal of the current block determined in operation 1706 to encode it into the bitstream (operation 1708). In an example, the device can obtain transform coefficient information of the current block by performing a transform and quantization (or scaling) based on the residual signal of the current block, and can encode the transform coefficient information into the bitstream. In an example, the device can omit the transform and quantization (or scaling) and encode the residual signal of the current block as the bitstream. In the proposed method of the disclosure, the residual difference signal of the current block can be encoded into the bitstream not only by these examples, but also by various methods.

[0253] The image encoding method according to the proposed method of the disclosure is not limited to include only the configuration shown in FIG. 17 , and can not include some of the configurations shown in FIG. 18 , or can include other configurations described in the disclosure.

[0254] FIG. 18 is a flowchart of an image decoding method according to the proposed method of the disclosure. FIG. 18 The example of may be performed by a device (e.g., the image decoding device 100). FIG. 18 is only an example, and the proposed method of the disclosure is not limited to the example of FIG. 18 . For example, the proposed method of the disclosure can exclude at least one operation shown in FIG. 18 , or can include an operation not shown in FIG. 18 .

[0255] Referring to FIG. 23 , the device can obtain the residual difference signal of the current block by processing the bitstream (operation 1802). In an example, the device can obtain the transform coefficient information of the current block from the bitstream, and perform inverse quantization (or scaling) and inverse transform based on the transform coefficient information to obtain the residual difference signal of the current block. In an example, the device can omit the inverse transform and inverse quantization (or scaling), and obtain the residual difference signal of the current block from the bitstream. In the proposed method of the disclosure, the residual difference signal of the current block can be obtained not only by these examples, but also by various methods.

[0256] The device can determine the reference block R corresponding to the current block C in the current picture including the current block C (operation 1804), for example, a decoded region or a reconstructed region in the current picture. For example, the reference block can be determined from the decoded region or the reconstructed region of the current picture. For example, the reference block can be determined from a predetermined part of the decoded region or the reconstructed region of the current picture.

[0257] In an example, at operation 1804, the device can determine the reference block corresponding to the current block in the current picture based on template matching (for example, see FIG. 25 、 FIG. 26 、 FIG. 29 and FIG. 23 and related descriptions thereof). In this example, the operation of determining the reference block corresponding to the current block in the current picture can include determining a first pixel region adjacent to the current block in the current picture, determining a second pixel region corresponding to the first pixel region adjacent to the current block in the current picture, and determining the reference block corresponding to the second pixel region in the current picture (for example, see FIG. 25 、 FIG. 26 、 FIG. 29 and FIG. 26 and related descriptions thereof). In this example, the first pixel region represents a template for the current block, and can be referred to as a first template. In this example, the second pixel region represents a template for the reference block, and can be referred to as a second template.

[0258] In this example (for example, the example based on template matching), the first pixel region and the second pixel region can have various forms. In detail, the first pixel region can include at least one of a pixel region adjacent to the top of the current block, a pixel region adjacent to the top-left of the current block, or a pixel region adjacent to the left side of the current block, and the second pixel region can include at least one of a pixel region adjacent to the top of the reference block, a pixel region adjacent to the top-left of the reference block, or a pixel region adjacent to the left side of the reference block (for example, see FIG. 26and related descriptions). In this example, the first pixel region neighboring the current block can have a fixed size (e.g., 4 or 4xH_4x4_Wx4), or have a size determined based on a size of the current block (e.g., W / 2 and H / 2, where W and H are a width and a height of the current block), or have a width or a height that is the same as a width or a height of a coding unit including the current block (e.g., 4xH and Wx4, where W and H are a width and a height of the coding unit including the current block), or have a width or a height that is twice a width or a height of the coding unit including the current block (e.g., 4x2H and 2Wx4, where W and H are a width and a height of the coding unit including the current block) (e.g., see FIG. 26 and related descriptions).

[0259] In this example (e.g., the example based on template matching), the operation of determining, in the current picture, a second pixel region corresponding to the first pixel region neighboring the current block can include calculating values of a cost function based on pixels of the first pixel region and pixels of a pixel region in a given search region (e.g., a reconstructed region of the current picture or a particular region within the reconstructed region of the current picture), and determining, as the second pixel region, a pixel region corresponding to a lowest value or a highest value among the calculated values of the cost function (e.g., see FIG. 26 and related descriptions). As a more detailed example, the cost function can include at least one of a sum of absolute differences (SAD), a sum of squared differences (SSD), or a number of pixels having the same pixel value (e.g., see FIG. 24 and related descriptions).

[0260] As another example, at operation 1804, the device can determine, in the current picture, a reference block corresponding to the current block based on the block vector (e.g., see FIG. 28 , FIG. 30 and FIG. 24 and related descriptions). In this example, the operation of determining the reference block corresponding to the current block can include obtaining, from the bitstream, a block vector of the current block, and determining, based on the block vector of the current block, the reference block in the current picture (e.g., see FIG. 28 , FIG. 30 and FIG. 25 and related descriptions).

[0261] The device can determine a residual signal R rec of the reference block based on a reconstructed signal R pred of the reference block and a prediction signal R res of the reference block (operation 1806) (e.g., see FIG. 28 , FIG. 29 , FIG. 30 and FIG. 25and related description). The prediction signal of the reference block can be determined based on at least one reference pixel included in neighboring blocks of the reference block in the current picture.

[0262] The device can determine the reconstructed signal of the current block C based on the residual difference signal of the current block res and the prediction signal of the current block C pred rec (operations 1808) (e.g., see FIG. 28 , FIG. 29 , FIG. 30 and FIG. 25 and related description). The prediction signal of the current block can be determined based on at least one reference pixel included in neighboring blocks of the current block in the current picture.

[0263] In an example, at operation 1806, the device can determine an intra prediction mode of the reference block from a plurality of intra prediction modes based on the reconstructed signal of the reference block and at least one reference pixel included in neighboring blocks of the reference block, and can determine the prediction signal of the reference block based on the intra prediction mode of the reference block and at least one reference pixel included in neighboring blocks of the reference block in the current picture (e.g., see FIG. 28 and FIG. 25 and related description). In this example, at operation 1808, the device can determine the intra prediction mode of the reference block as the intra prediction mode of the current block, and can determine the prediction signal of the current block based on the intra prediction mode of the reference block and at least one reference pixel included in neighboring blocks of the current block in the current picture (e.g., see FIG. 28 and FIG. 27 and related description). In this example, the operation of determining the intra prediction mode of the reference block from a plurality of intra prediction modes can include calculating a function value of each of the plurality of intra prediction modes based on the reconstructed signal of the reference block and at least one reference pixel included in neighboring blocks of the reference block, and determining the intra prediction mode corresponding to the lowest function value or the highest function value among the calculated function values as the intra prediction mode of the current block (e.g., see FIG. 27 and related description). As a more detailed example, the function value can be a cost function, and can include at least one of a sum of absolute differences (SAD), a sum of squared differences (SSD), or a number of pixels having the same pixel value (e.g., see FIG. 25 and related description).

[0264] ​In an example, at operation 1806, when the reference block is coded in the intra prediction mode, the device can identify the intra prediction mode of the reference block and can determine the prediction signal of the reference block based on the intra prediction mode of the reference block and the at least one reference pixel included in the neighboring blocks of the reference block in the current picture (e.g., see FIG. 28 and FIG. 25 and related descriptions). In this example, at operation 1808, the device can determine the intra prediction mode of the current block as the intra prediction mode of the reference block and can determine the prediction signal of the current block based on the intra prediction mode of the current block and the at least one reference pixel included in the neighboring blocks of the current block in the current picture (e.g., see FIG. 28 and FIG. 20 and related descriptions).

[0265] In an example, at operation 1806, the device can obtain the intra prediction mode of the current block from the bitstream (e.g., see FIG. 29 and related descriptions), determine the intra prediction mode of the current block as the intra prediction mode of the reference block, and determine the prediction signal of the reference block based on the intra prediction mode of the current block and the at least one reference pixel included in the neighboring blocks of the reference block in the current picture (e.g., see FIG. 30 and FIG. 29 and related descriptions). In this example, at operation 1808, the device can determine the prediction signal of the current block based on the intra prediction mode of the current block and the at least one reference pixel included in the neighboring blocks of the current block in the current picture (e.g., see FIG. 30 and FIG. 29 and related descriptions).

[0266] In an example, at operation 1806, the device can determine the intra prediction mode of the current block from the plurality of intra prediction modes based on a region of pixels neighboring the current block without obtaining the intra prediction mode of the current block from the bitstream. At operation 1806, the device can determine the intra prediction mode of the current block as the intra prediction mode of the reference block and can determine the prediction signal of the reference block based on the intra prediction mode of the current block and the at least one reference pixel included in the neighboring blocks of the reference block in the current picture (e.g., see FIG. 30 and FIG. 29 and related descriptions). In this example, at operation 1808, the device can determine the prediction signal of the current block based on the intra prediction mode of the current block and the at least one reference pixel included in the neighboring blocks of the current block in the current picture (e.g., see FIG. 30 and FIG. 18 and related descriptions).

[0267] The image decoding method according to the proposed method of the present disclosure is not limited to include only FIG. 18configurations shown in the above description and can not include FIG. 19 some of the configurations shown in the above description, or can include other configurations described in the present disclosure.

[0268] FIG. 19 is a flowchart of a bitstream storage method according to the proposed method of the present disclosure. FIG. 19 The method shown in the above description can be performed by a device (e.g., the image encoding device 200). FIG. 19 are examples, and the proposed method of the present disclosure is not limited to FIG. 19 Examples. For example, the proposed method of the present disclosure can exclude FIG. 19 at least one operation shown in the above description, or can include operations not shown in the above description. FIG. 19

[0269] Referring to FIG. 17 , the device can generate a bitstream according to an encoding method (operation 1902). For example, the encoding method can include an image encoding method according to the proposed method of the present disclosure. In detail, the encoding method can include the image encoding method described above with reference to FIG. 18 .

[0270] Optionally, at operation 1902, the device can generate a bitstream that is decoded according to a decoding method. For example, the decoding method can include an image decoding method according to the proposed method of the present disclosure. In detail, the decoding method can include the image decoding method described above with reference to FIG. 20 .

[0271] The device can store the generated bitstream in a non-transitory computer readable storage medium, or transmit the generated bitstream to another device by using wireless or wired communication (operation 1904).

[0272] FIG. 21 and FIG. 20 Intra prediction modes suitable for the proposed method of the present disclosure are shown. Although the Intra prediction modes of ITU-T H.266 / Versatile Video Coding (VVC) are mainly explained in the examples of FIG. 21 and FIG. 20 , the Intra prediction modes suitable for the proposed method of the present disclosure are not limited to the examples of FIG. 21 and FIG. 20 . For example, the Intra prediction modes suitable for the proposed method of the present disclosure can include other Intra prediction modes of ITU-T H.266 / VVC (e.g., Cross Component Linear Model (CCLM)) and / or Intra prediction modes of advanced video coding standards after ITU-T H.266 / VVC.

[0273] Referring to FIG. 20 ​In the case of the intra prediction, a specific sample (value) among the reference samples 2004 located in a direction corresponding to an intra prediction mode for the current block 2002 (e.g., including a plurality of directions within a range from a left end to an upper right end) can be determined as a prediction value or a prediction factor of a current sample belonging to the current block 2002. For example, the H.264 / Advanced Video Coding (AVC) standard supports 8 directions, the H.265 / High Efficiency Video Coding (HEVC) standard supports 33 directions, and the H.266 / VVC standard supports 65 directions.

[0274] The intra prediction can include non-directional prediction and directional prediction. In the case of the non-directional prediction, there are the following modes: (1) an intra direct current (DC) (prediction) mode (or INTRA_DC mode) which determines an average (of values) of a specific sample among the reference samples 2004 as a prediction value of a current sample of the current block 2002; and (2) an intra planar (prediction) mode (or INTRA_PLANAR mode) which determines a value obtained by applying a weight according to a distance from a current sample to be decoded to a value of a left / top reference sample of the current block as a prediction value or a prediction factor of the current sample.

[0275] In the case of the directional prediction, a sample (value) located in a direction corresponding to a directional prediction mode for the current block 2002 (e.g., including a plurality of directions within a range from a left end to an upper right end) among the reference samples 2004 can be determined as a prediction value or a prediction factor of a current sample belonging to the current block 2002. For example, the H.264 / Advanced Video Coding (AVC) standard supports 8 directions, the H.265 / High Efficiency Video Coding (HEVC) standard supports 33 directions, and the H.266 / VVC standard supports 65 directions.

[0276] In the directional prediction, a sample value corresponding to a fractional position of a reference sample located in a direction of an intra prediction mode for the current block can be calculated, and the calculated sample value can be determined as a prediction value of a current sample of the current block. For example, in the case of the H.266 / VVC, filter coefficients for obtaining a sample value corresponding to a fractional position in a unit of 1 / 32 sample, which is to be used as a prediction value of a current sample of the current block, can be defined as in Table 1. Table 1 shows an example of filter coefficient values used when a sample value to be used as a prediction value of a current sample is obtained for each fractional position (from p=0 to p=31 / 32) of the reference sample 2004. In this case, one of a plurality of interpolation filters (A or B in Table 1) can be selected according to a size of the current block 2002 or the intra prediction mode.

[0277] [Table 1]

[0278] FIG. 21 (b) shows the intra prediction modes of the H.266 / VCC standard and the directions for each intra prediction mode. In general, the intra DC mode and the intra planar mode can be supported, there can be directional prediction modes from number 2 to number 66, and according to the ratio between the width and the height of the block, the directional prediction modes with numbers less than number 2 or greater than number 66 can be used (for example, see FIG. 20 and the related description thereof).

[0279] Referring to FIG. 20 (b), the intra prediction modes applicable to the proposed method of the present disclosure can include an intra planar (prediction) mode (or INTRA PLANAR mode), an intra DC (prediction) mode (or INTRA DC mode), and intra angular (prediction) modes (or INTRA ANGULAR2 to INTRA ANGULAR66). Each intra prediction mode can be represented as a specific value. In an example, when the intra prediction mode has a value of 0, the intra prediction mode can indicate the intra planar (prediction) mode (or INTRA PLANAR mode). In an example, when the intra prediction mode has a value of 1, the intra prediction mode can indicate the intra DC (prediction) mode (or INTRA DC mode). In an example, when the intra prediction mode has a value of one of numbers 2 to 66, the intra prediction mode can indicate a corresponding intra angular (prediction) mode among the plurality of intra angular (prediction) modes (or INTRA ANGULAR2 to INTRA ANGULAR66). As described above, the intra prediction modes applicable to the proposed method of the present disclosure are not limited to the intra prediction modes shown in FIG. 20 (b). In the present specification, for ease of description, the intra planar (prediction) mode (or INTRA PLANAR mode) can be simply referred to as the intra planar mode, the intra DC (prediction) mode (or INTRA DC mode) can be simply referred to as the intra DC mode, and the intra angular (prediction) modes (or INTRA ANGULAR2 to INTRA ANGULAR66) can be simply referred to as the intra angular modes.

[0280] Intra prediction modes applicable to the proposed method of the disclosure can be determined based on most probable modes (MPMs). For example, six MPMs can be used, but this is merely an example, and other number of MPMs can be used. In an example, a device (e.g., the image decoding device 100) can obtain (syntax) information related to an intra prediction mode of a current block from a bitstream, and determine the intra prediction mode of the current block based on an intra prediction mode candidate list including MPMs and the (syntax) information related to the intra prediction mode. In an example, a device (e.g., the image encoding device 200) can obtain (syntax) information related to an intra prediction mode of a current block based on an intra prediction mode candidate list including MPMs and the intra prediction mode of the current block, and can encode the obtained information into a bitstream. In this specification, for convenience of explanation, the intra prediction mode candidate list including MPMs can be simply referred to as an MPM list, and can be used interchangeably with an intra candidate mode list, a candidate mode list, etc.

[0281] As an example not limiting the proposed method of the disclosure, when the neighboring blocks are not available, the intra prediction modes of the neighboring blocks can be set to the intra planar mode by default. In an example, when the intra prediction modes of the left neighboring block and the above neighboring block are both intra non-angular modes (e.g., the intra planar mode or the intra DC mode), the MPM list can include {Planar, DC, V, H, V-4, V+4}. In this specification, for convenience of explanation, Planar denotes the intra planar mode, DC denotes the intra DC mode, V denotes a vertical intra prediction mode (e.g., INTRA_ANGULAR50 in HEVC), FIG. 20 H denotes a horizontal intra prediction mode (e.g., INTRA_ANGULAR18 in HEVC), FIG. 20 V-4 denotes an intra prediction mode having a value obtained by subtracting 4 from the vertical intra prediction mode (e.g., INTRA_ANGULAR46 in HEVC), FIG. 20 and V+4 denotes an intra prediction mode having a value obtained by adding 4 to the vertical intra prediction mode (e.g., INTRA_ANGULAR54 in HEVC). FIG. 21

[0282] ​In an example, when one of the intra prediction modes of the left neighboring block and the above neighboring block is an intra angular mode and the other is an intra non-angular mode, the MPM list can include {Planar, Max, Max-1, Max+1, Max-2, Max+2}. In this specification, for ease of explanation, Max denotes an intra prediction mode corresponding to the maximum of the intra prediction mode of the left neighboring block and the intra prediction mode of the above neighboring block, Max-1 denotes an intra prediction mode corresponding to a value obtained by subtracting 1 from Max, Max+1 denotes an intra prediction mode corresponding to a value obtained by adding 1 to Max, Max-2 denotes an intra prediction mode corresponding to a value obtained by subtracting 2 from Max, and Max+2 denotes an intra prediction mode corresponding to a value obtained by adding 2 to Max.

[0283] In an example, when the intra prediction modes of the left neighboring block and the above neighboring block are both intra angular modes and different from each other, the MPM list can be determined as follows. In this specification, for ease of explanation, Min denotes an intra prediction mode corresponding to the minimum of the intra prediction mode of the left neighboring block and the intra prediction mode of the above neighboring block, Left denotes the intra prediction mode of the left neighboring block, Above denotes the intra prediction mode of the above neighboring block, Min-1 denotes an intra prediction mode corresponding to a value obtained by subtracting 1 from Min, Min-2 denotes an intra prediction mode corresponding to a value obtained by subtracting 2 from Min, Min+1 denotes an intra prediction mode corresponding to a value obtained by adding 1 to Min, and Min+2 denotes an intra prediction mode corresponding to a value obtained by adding 2 to Min.

[0284] - When (Max-Min) is 1, the MPM list includes {Planar, Left, Above, Min-1, Max+1, Min-2}.

[0285] - Otherwise, when (Max-Min) is equal to or greater than 62, the MPM list includes {Planar, Left, Above, Min+1, Max-1, Min+2}.

[0286] - Otherwise, when (Max-Min) is 2, the MPM list includes {Planar, Left, Above, Min+1, Min-1, Max+1}.

[0287] - Otherwise, the MPM list includes {Planar, Left, Above, Min-1, -Min+1, Max-1}.

[0288] In an example, when the intra prediction modes of the left neighboring block and the above neighboring block are both intra angular modes, the MPM list can include {Planar, Left, Left - 1, Left + 1, Left - 2, Left + 2}. In this specification, for ease of explanation, Left - 1 denotes an intra prediction mode corresponding to a value obtained by subtracting 1 from Left (e.g., the intra prediction mode of the left neighboring block), Left + 1 denotes an intra prediction mode corresponding to a value obtained by adding 1 to Left, Left - 2 denotes an intra prediction mode corresponding to a value obtained by subtracting 2 from Left, and Left + 2 denotes an intra prediction mode corresponding to a value obtained by adding 2 to Left.

[0289] FIG. 21 Wide-angle intra prediction suitable for the proposed method of the present disclosure is shown. FIG. 21 (a) of FIG. 10 shows an example of applying wide-angle intra prediction to pixels 1004a of a block 1002a when a width W of the block 1002a is greater than its height H (W > H), and FIG. 21 (b) of FIG. 10 shows an example of applying wide-angle intra prediction to pixels 1004b of a block 1002b when a height H of the block 1002b is greater than its width W (H > W). As described above, the proposed method of the present disclosure is not limited to FIG. 21 an example of FIG. 10.

[0290] Referring to FIG. 20 , the directions of typical intra angular modes are defined within an angle from 45 degrees 1006a or 1006b (e.g., intra angular mode 2 or INTRA_ANGULAR2) to 135 degrees 1008a or 1008b (e.g., intra angular mode 66 or INTRA_ANGULAR66) in a clockwise direction (e.g., see FIG. 20 and related descriptions thereof). When the block 1002a or 1002b is a non-square block, 2W + 1 neighboring pixels adjacent to the top of the block 1002a or 1002b and / or 2H + 1 neighboring pixels adjacent to the left of the block 1002a or 1002b according to a width W and a height H of the block 1002a or 1002b can be used as reference pixels for intra prediction, and some of the intra angular modes (e.g., see FIG. 22and its related description) can be adaptively replaced with wide-angle intra prediction modes. For example, the left-bottom diagonal intra angular mode 2106a or 2106b (e.g., intra angular mode 2 or INTRA_ANGULAR2) can be replaced with intra angular mode 2116a or 2116b, the right-top diagonal intra angular mode 2108a or 2108b (e.g., intra angular mode 66 or INTRA_ANGULAR66) can be replaced with intra angular mode 2118a or 2118b, and the left-top diagonal intra angular mode (e.g., intra angular mode 34 or INTRA_ANGULAR34) can be replaced with diagonal mode 2110a or 2110b.

[0291] FIG. 23 and FIG. 22 A template matching method suitable for the proposed method of the present disclosure is illustrated.

[0292] In the present specification, a template refers to a neighboring pixel region or a neighboring block adjacent to a block. The neighboring pixel region or the neighboring block corresponding to the template can include at least one neighboring pixel (or a reference pixel) adjacent to the block. In an example, the template can refer to a neighboring pixel region or a neighboring block adjacent to the left side of the block. In an example, the template can refer to a neighboring pixel region or a neighboring block adjacent to the top of the block. In an example, the template can refer to a neighboring pixel region or a neighboring block adjacent to the left side and the top of the block. In an example, the template can refer to a neighboring pixel region or a neighboring block adjacent to at least one of the left side, the top, or the top-left of the block.

[0293] In the present specification, template matching (TM) refers to a technique of searching for a template (or a pixel region) that is best matched to a template of a block (or a neighboring pixel region adjacent to the block). For example, template matching can be used as a prediction method of performing prediction on a block based on a template that is best matched to a template of the block (e.g., a coding unit (CU)) in a reference picture or a current picture, or can be used as a motion information derivation method of correcting motion information of a block at a decoder side. In the present specification, inter-template matching can refer to a technique of searching for a template that is best matched to a template of a block in a reference picture, and intra-template matching can refer to a technique of searching for a template that is best matched to a template of a block in a picture (or a current picture) including the block. In the present specification, template matching prediction (TMP) can refer to a technique of performing prediction based on template matching.

[0294] FIG. 22 An inter-template matching suitable for the proposed method of the present disclosure is illustrated. FIG. 22 Examples of the inter-template matching are merely examples, and the inter-template matching suitable for the proposed method of the present disclosure is not limited to FIG. 22 Examples of the inter-template matching. For example, in the case of the inter-template matching, FIG. 23In the example, the search range is shown as a 16x16 pixel region, but the proposed inter-template matching method applicable to the present disclosure can also be similarly applied to other sizes of search ranges.

[0295] Referring to FIG. 23 , the inter-template matching can be used in a prediction method of predicting a current block 2202 or a decoder-side motion vector (MV) derivation method for correcting motion information of the current block 2202 by searching for a closest match between a template 2204 (e.g., an upper and / or left and / or upper-left neighboring pixel region or neighboring block of the current block 2202 (e.g., a current coding unit (CU)) in a current picture 2206 and a template 2214 (e.g., a neighboring pixel region or neighboring block of the same size as the template 2204 in the current picture 2206) in a reference picture 2216. For example, the (inter-) template matching can be used in an advanced motion vector prediction (AMVP) mode or a merge mode.

[0296] In the present specification, the AMVP mode refers to a method of generating a motion vector predictor (MVP) list by using motion information (e.g., a motion vector (MV)) of at least one neighboring block (e.g., at least one of a left neighboring block, an upper neighboring block, or an upper-left neighboring block) of a current block, and signaling, through a bitstream, a motion vector difference (MVD) indicating a difference between a motion vector of the current block and a motion vector predictor and information (e.g., an index indicating an MVP candidate in the MVP list) indicating a motion vector predictor of the current block in the MVP list.

[0297] In an example, when the (inter-) template matching (TM) is used in the AMVP mode, an MVP candidate can be determined based on a template matching error that makes a minimum difference between a current block template (cur.template) 2204 and a reference block template (ref.template) 2214. In addition or alternative to this example, the (inter-) template matching (TM) can be performed only for this specific MVP candidate 2208 to achieve motion vector (MV) refinement.

[0298] In the present specification, the merge mode refers to a method of generating a merge candidate list by using motion information (e.g., a motion vector (MV), a reference picture list index, etc.) of at least one neighboring block (e.g., at least one of a left neighboring block, an upper neighboring block, a lower-left neighboring block, an upper-left neighboring block, an upper-right neighboring block, or a temporal neighboring block) of a current block, and signaling, through a bitstream, information (e.g., an index indicating a merge candidate in the merge candidate list) indicating motion information of the current block in the merge candidate list.

[0299] In an example, when using (inter) template matching (TM) in merge mode, (inter) template matching (TM) can be performed for a merge candidate 2208 indicated by a merge index (or an index indicating a merge candidate in a merge candidate list) in a merge candidate list. The merge candidate can be refined by performing template matching for the merge candidate.

[0300] Template matching (TM) can be run as an additional refinement process for a block-based or sub-block-based bilateral matching method, or can be run independently (as a kind of prediction method). Furthermore, template matching (TM) can be used not only in AMVP mode or merge mode, but also in other methods or can be used independently.

[0301] FIG. 23 Intra template matching suitable for the proposed method of the present disclosure is shown. FIG. 23 The example of FIG. 23 is merely an example, and the intra template matching suitable for the proposed method of the present disclosure is not limited to FIG. 26 The example of FIG. 26 In an example, an L-shaped template (e.g., a neighboring pixel region or a neighboring block of the left side, the top, or the upper left of a block) is shown. However, the intra template matching suitable for the proposed method of the present disclosure can also be applied to templates of other shapes (e.g., a neighboring pixel region or a neighboring block of at least one of the left side, the top, or the upper left of a block) in the same / similar manner in addition to the L-shaped template.

[0302] Referring to FIG. 24 , intra template matching prediction (intra-TMP) can refer to determining a surrounding sample of a current block 2302 in a current picture 2306 as a template 2304, searching for a template most similar to the template 2304 of the current block 2302 in a search area (or a search region) 2310 already decoded or reconstructed in the current picture 2306, and using a value of a reference area (or a reference block or a matching block) 2312 of a template 2314 found as a search result having the best matching rate as a prediction value of the current block 2302. As an example not limiting the proposed method of the present disclosure, the template 2304 of the current block 2302 can have an L shape, and as a result of measuring a distortion between the template 2304 of the current block 2302 and a search template (e.g., using SAD), the smaller the distortion cost, the higher the matching rate can be considered. For example, the size of the search area 2310 can be determined based on the size of the current block 2302 (e.g., in proportion to the size of the current block). This can be advantageous in the case of setting a fixed number of SAD comparisons for each pixel, but the cost function suitable for the proposed method of the present disclosure is not limited to SAD (e.g., see Equation 3), and other cost functions (e.g., SSD of Equation 4, Num of Equation 5, etc., see FIG. 24and related descriptions). In this specification, the intra-TMP can be referred to as intra-template matching or intra-TMP mode or intra-prediction mode based on intra-template matching.

[0303] In summary, the intra-template matching can be used as a special intra-prediction mode, which determines a block 2312 corresponding to a template 2314 (e.g., L-shaped template) that matches the current template 2304 (or the template 2304 corresponding to the current block 2302) in the reconstructed part of the current picture or frame 2306 as the best prediction block for the current block 2302, and copies the best prediction block 2312 (as the predictor of the current block 2302).

[0304] At the encoder side, a device (e.g., the image encoding device 200) can search for a template 2314 that is most similar to the current template 2304 (or the template 2304 corresponding to the current block 2302) from the reconstructed part of the current picture or frame 2306 in the predefined search region 2310, and can use the block 2312 corresponding to the found template 2314 as the prediction block. The device can determine a residual signal of the current block based on a difference between the original image of the prediction block and the original image of the current block, and process the residual signal to encode the residual signal into the bitstream. The device can signal whether to use this mode (e.g., the intra-TMP mode) (via the bitstream), and the decoder can perform the same prediction operation based on the signaling performed by the device.

[0305] For example, at the decoder side, a device (e.g., the image decoding device 100) can perform operations corresponding to the operations performed at the encoder side to search for a template 2314 that is most similar to the current template 2304, and can use the block 2312 corresponding to the found template 2314 as the prediction block. The device can reconstruct the current block based on the residual signal of the current block obtained by processing the bitstream and the prediction block (e.g., based on the sum of the residual signal and the prediction block).

[0306] Compared with the intra block copy mode described with reference to FIG. 24 In the intra-TMP, there can be no signaling related to the block vector. For example, in the case of the intra block copy mode, the (syntax) information indicating the block vector can be signaled through the bitstream, but in the case of the intra-TMP mode, because the decoder can determine the prediction block by performing operations corresponding to the operations performed at the encoder side, the vector information indicating the template 2314 that is most similar to the current template 2304 or the vector information indicating the block 2312 corresponding to the template 2314 can not be signaled through the bitstream.

[0307] FIG. 24 Intra block copy suitable for the proposed method of the present disclosure is shown. Although FIG. 24Examples of the IBC mode applicable to the proposed method of the present disclosure are focused on the IBC mode of ITU-T H.266 / Versatile Video Coding (VVC) and the IBC mode of the Enhanced Compression Model (ECM), but the IBC mode applicable to the proposed method of the present disclosure is not limited to FIGS. 25-30 Examples of the IBC mode applicable to the proposed method of the present disclosure. For example, the IBC mode applicable to the proposed method of the present disclosure can include the IBC mode of the advanced video coding standard after ITU-T H.266 / VVC.

[0308] Referring to FIGS. 17-19 , the IBC mode refers to searching for a block 2412 most similar to the current block 2402 from the reconstructed part of the current picture 2406 by block matching at the encoder side, and signaling a block vector (BV) 2410 indicating the block 2412 in the current picture 2406 through a bitstream, and determining the reference block 2412 based on the signaled block vector 2410 and reconstructing the current block 2402 using the reference block 2412 at the decoder side. The IBC mode can refer to determining a prediction factor for the block 2402 as a block vector 2410 in a spatial domain, similar to performing inter prediction with a motion vector in a temporal domain.

[0309] In an example, the IBC mode applicable to the proposed method of the present disclosure can include the IBC merge mode of ITU-T H.266 / VVC. The IBC merge mode can refer to constructing a merge candidate list having at least one block vector from left neighboring blocks, above neighboring blocks, and top-left neighboring blocks of the current block 2402, and signaling information (e.g., a merge index) indicating a block vector for the current block in the merge candidate list. When the IBC merge mode is applied, the encoder can determine a block vector for the current block 2402, and can signal information (e.g., a merge index) indicating a merge candidate corresponding to the determined block vector in the merge candidate list through a bitstream. When the IBC merge mode is applied, the decoder can obtain information (e.g., a merge index) indicating a merge candidate from the bitstream, construct a merge candidate list in the same manner as the encoder, and then determine a block vector of a merge candidate indicated by the information (e.g., a merge index) in the merge candidate list as the block vector 2410 for the current block 2402.

[0310] In an example, the IBC mode suitable for the proposed method of the present disclosure can include the IBC AMVP mode of ITU-T H.266 / VVC. The IBC AMVP mode can refer to constructing a block vector predictor (BVP) candidate list with at least one block vector predictor from among the left neighboring block and the above neighboring block of the current block 2402, and signaling information (e.g., a BVP index) indicating a BVP candidate in the BVP candidate list corresponding to the BVP for the current block 2402 and a block vector difference (BVD) indicating a difference between the block vector for the current block 2402 and the BVP. When the IBC AMVP mode is applied, an encoder can determine the block vector 2410 for the current block 2402, can determine a BVP candidate from the BVP candidate list based on the block vector 2410, and can signal information (e.g., a BVP index) indicating the determined BVP candidate through a bitstream. When the IBC AMVP mode is applied, a decoder can obtain the BVD and the information (e.g., a BVP index) indicating the BVP candidate from the bitstream, construct the BVP candidate list in the same way as the encoder, and then determine the block vector 2410 for the current block 2402 based on the BVP candidate in the BVP candidate list indicated by the information (e.g., a BVP index) and the BVD (e.g., based on the sum of the BVP and the BVD). Optionally, in the IBC AMVP mode, adaptive motion vector resolution (AMVR) using variable vector precision can be signaled via the bitstream. In the present specification, the IBC BVP candidate and the IBC BVP candidate list can be referred to as the IBC AMVP candidate and the IBC AMVP candidate list, respectively.

[0311] In an example, the IBC mode suitable for the proposed method of the present disclosure can include the IBC mode of an enhanced compression model (ECM). The IBC mode of the ECM can include, for example, the IBC merge mode and / or the IBC AMVP mode. Compared to the IBC merge / AMVP mode of ITU-T H.266 / VVC, the IBC mode of the ECM can improve the IBC merge / AMVP candidate list construction as follows. An IBC merge / AMVP candidate can be inserted into or added to the IBC merge / AMVP candidate list only when the IBC merge / AMVP candidate is valid. At least one of the left neighboring block, the above neighboring block, the top-right neighboring block, the bottom-left neighboring block, or the top-left neighboring block and at least one pair-wise average candidate can be added to the spatial merge / AMVP candidate list. Template-based adaptive reordering can be applied to the IBC merge list (merge candidate adaptive reordering with template matching, ARMC-TM).

[0312] In an example, the IBC mode applicable to the proposed method of the disclosure can include an IBC with template matching (IBC-TM) mode used in combination with template matching (TM). The IBC-TM mode of the ECM can be combined with the IBC merge mode (referred to as IBC-TM merge mode) or can be combined with the IBC AMVP mode (referred to as IBC-TM AMVP mode). Thus, the IBC-TM mode can refer to the combined use of IBC with template matching (TM) for the IBC merge mode or the IBC AMVP mode. The IBC-TM mode can include, for example, the IBC-TM merge mode and / or the IBC-TM AMVP mode.

[0313] For example, the IBC-TM merge mode can include amending at least one BVP candidate in at least one neighboring block (e.g., the left neighboring block, the above neighboring block, and the top-left neighboring block) of the block 2402 according to the template matching method and competing with (or adding the candidate of) the regular IBC merge mode. The BVP candidate list can be arranged based on the template matching cost. For example, the BVP candidate list can be arranged such that a lower template matching cost has a higher priority in the BVP candidate list (or such that a higher template matching cost has a lower priority), or the BVP candidate list can be arranged such that a higher template matching cost has a higher priority in the BVP candidate list (or such that a lower template matching cost has a lower priority).

[0314] For example, the IBC-TM AMVP mode can include amending and ordering the BVP based on the template matching without signaling the block vector difference (BVD) via the bitstream. As a non-limiting example, the BVP can be amended based on integer pixel precision or 4-pixel (or 4-pel) precision.

[0315] The IBC-TM mode (e.g., the IBC-TM merge mode and / or the IBC-TM AMVP mode) can include adding the intra TMP block vector as a spatial candidate to the IBC block vector candidate list (or the IBC AMVP candidate list). In the IBC-TM mode, the template can include the left and top neighboring reference pixels (or the left and top neighboring pixel region or the left and top neighboring block) of the block 2402, or the left neighboring reference pixels (or the left neighboring pixel region or the left neighboring block), or the top neighboring reference pixels (or the top neighboring pixel region or the top neighboring block).

[0316] FIG. 25 An embodiment according to the proposed method of the disclosure (e.g., see FIG. 28 and the related description thereof) is shown. For ease of explanation, FIG. 29 , FIG. 30 , FIG. 20 and FIG. 21Embodiments of the proposed methods 1, 2, 3 and 4 are respectively referred to as “the proposed method 1”, “the proposed method 2”, “the proposed method 3” and “the proposed method 4”. In this specification, for the convenience of explanation, the terms are expressed as follows.

[0317] C: the current block R: the reference block C org : the original signal of the current block C pred : the predicted signal of the current block C res : the residual signal of the current block R rec : the reconstructed signal of the reference block R pred : the (simulated) predicted signal of the reference block R res : the (simulated) residual signal of the reference block T C : the template of the current block T R : the template of the reference block The simulated predicted signal refers to a signal (or a prediction factor or a prediction sample) predicted based on an intra prediction mode having the best prediction performance by predicting a block (e.g., the current block or the reference block) based on at least one of the available intra prediction modes (e.g., see FIG. 20 , FIG. 21 and the related descriptions). In this specification, the simulated predicted signal can be simply referred to as a predicted signal or a virtual predicted signal. In an example, the predicted signal R pred of the reference block can be the simulated predicted signal or the virtual predicted signal of the reference block. However, in the proposed methods according to the present disclosure, the predicted signal R pred of the reference block is not limited to the simulated predicted signal or the virtual predicted signal of the reference block. In an example, the predicted signal R pred of the reference block can be an actual predicted signal used during the reconstruction of the reference block.

[0318] The simulated residual signal can refer to a residual signal (or a residual sample) determined using a signal or a prediction factor predicted based on an intra prediction mode having the best prediction performance among the available intra prediction modes (e.g., see FIGS. 25-30 , FIGS. 17-19 and the related descriptions), and in this specification, can be simply referred to as a residual signal or a virtual residual signal. In an example, the residual signal R res of the reference block can be the simulated residual signal or the virtual residual signal of the reference block. However, in the proposed methods according to the present disclosure, the residual signal R resThis is not limited to the analog or virtual residual signal of the reference block. In the example, the residual signal R of the reference block... res It can be the actual residual signal used during the reconstruction of the reference block.

[0319] Reference FIGS. 25-30 The examples described are intended to aid in understanding the methods proposed in this disclosure (e.g., see [reference]). FIGS. 25-30 (and its related descriptions), and is not intended to limit the methods proposed in this disclosure. Therefore, references can be made to... FIG. 25 The example described excludes some configurations; other configurations, not shown, can be added, and the execution order can be changed. Furthermore, FIG. 25 Examples can be implemented independently, or at least one example can be implemented in combination with each other.

[0320] FIG. 23 The proposed method 1 of this disclosure is illustrated. The proposed method 1 of this disclosure can be performed by a device (e.g., image decoding device 100 or image encoding device 200). The proposed method 1 of this disclosure relates to performing residual prediction based on template matching, and is an example of using the intra-prediction mode of a reference block as the intra-prediction mode of the current block.

[0321] Reference FIG. 17 In operation 1, the device (e.g., image encoding device 200) can use the template T of the current block. C Search for the template T that matches the current block from a given search region in the spatial domain (e.g., search region 2310). C The best-matching part (or template). As an example that does not limit the proposed method of this disclosure, the device can search for the template T of the reference block based on template matching prediction (TMP). R The device can also determine the template T relative to the reference block. R The corresponding reference block. As a more detailed example, the device can be configured by executing the above reference block. FIG. 25 The described operation is used to search for the template T of the reference block. R .

[0322] For example, it can be FIG. 17 Operation 1702 execution FIG. 17 Operation 1, the template T of the current block C Can be with FIG. 25 The first template or the first pixel region corresponds to, and the template T of the reference block. R Can be with FIG. 20 The second template or the second pixel region corresponds to.

[0323] exist FIG. 21 In operation 2, the device (e.g., image encoding device 200) can use the found template TR Reference pixels (or neighboring pixels) of the corresponding reference block determine a prediction signal of the reference block for the set (or multiple sets) of intra prediction modes 2512, and can be compared to the reconstructed signal R rec of the reference block to find a best matching intra prediction mode from the set (or multiple sets) of intra prediction modes. The device can determine the intra prediction mode found in this way as the intra prediction mode of the reference block, and determine a prediction signal R pred of the reference block based on the determined intra prediction mode. As an example of the proposed method without limiting the present disclosure, the set (or multiple sets) of intra prediction modes can include the intra prediction modes described above with reference to FIG. 17 and FIG. 25 .

[0324] For example, operation 1704 of FIG. 25 may perform operation 2 of FIG. 17 , and the intra prediction mode of the reference block determined from the set (or from the multiple sets) of intra prediction modes 2512 in FIG. 25 may correspond to the intra prediction mode of the reference block in FIG. 25 .

[0325] In operation 3 of FIG. 25 , the device (e.g., the image encoding device 200) can determine a residual block (e.g., the residual signal R res of the reference block) corresponding to the intra prediction mode determined in operation 2 as a residual prediction factor of the current block. For example, the device can determine the residual signal R rec of the reference block based on the reconstructed signal R pred of the reference block (e.g., the difference R rec -R pred between the reconstructed signal of the reference block and the prediction signal of the reference block) of the reference block. res .

[0326] In operation 4 of FIG. 17 , the device (e.g., the image encoding device 200) can predict the current block by using the reference pixels (or neighboring pixels) of the current block with the intra prediction mode (e.g., the best matching intra prediction mode) of the reference block determined in operation 2. For example, the device can determine a prediction signal (or predictor) C pred of the current block by applying the intra prediction mode of the reference block to the reference pixels (or neighboring pixels) of the current block.

[0327] In operation 5 of FIG. 25 , the device (e.g., the image encoding device 200) can determine a residual signal R RThe corresponding residual predictor (e.g., the residual signal R of the reference block) res To further predict the residual signal C of the current block. res In the example, as shown above... FIG. 17 The device can be based on the original signal C of the current block. org The prediction signal C of the current block pred The residual signal R of the reference block res To determine the residual difference signal of the current block. In the example, the device can be based on the original signal C of the current block. org and the prediction factor C of the current block pred (For example, the difference C between the original signal of the current block and the predictor of the current block) org -C pred To determine the residual signal C of the current block. res And it can be based on the residual signal C of the current block. res The residual signal R of the reference block res (For example, the difference C between the residual signal of the current block and the residual signal of the reference block) res -R res To determine the residual difference signal of the current block For example, it can be performed based on Equation 1. FIG. 25 The operation of the device is shown in the figure.

[0328] [Equation 1] R res =R rec -R pred C res =C org -C pred =C res -R res For example, it can be FIG. 17 Operation 1706 execution FIG. 25 Operations 3 to 5.

[0329] The device (e.g., image encoding device 200) can process the residual difference signal of the current block. To encode it into a bitstream (e.g., see...) FIG. 25(Operation 1708 and its related description). In the example, the device can obtain the transform coefficient information of the current block by performing transform and quantization (or scaling) based on the residual difference signal of the current block, and can encode the transform coefficient information into a bitstream. In the example, the device can omit the transform and quantization (or scaling) and encode the residual difference signal of the current block into a bitstream. In the proposed method of this disclosure, not only through these examples, but also through various other methods, the residual difference signal of the current block can be encoded into a bitstream.

[0330] Optionally, FIG. 25 Operation 5 can be optional. When not executed... FIG. 25 In operation 5, the residual prediction is not applied to the current block (or the residual signal C of the current block). res Furthermore, the intra-prediction mode of the current block can be derived solely from a template-based reference region. In this case, although the device (e.g., image coding device 200) can determine the residual signal C of the current block by using the intra-prediction mode of the reference block as the intra-prediction mode of the current block, res However, this can be achieved by processing the residual signal C of the current block. res The residual signal C of the current block res Encoded into a bitstream, rather than based on the residual signal R of the reference block. res Predict the residual signal C of the current block. res .

[0331] For example, when FIG. 25 When operation 5 is optional, the device can determine whether to target the current block (or the residual signal C for the current block). res Perform residual prediction. When the device determines not to perform residual prediction, it can base it on the original signal C of the current block. org and the prediction signal C of the current block pred (For example, the difference C between the original signal of the current block and the predicted signal of the current block) org -C pred To determine the residual signal C of the current block. res It can also process the residual signal C of the current block. res To use the residual signal C of the current block res Encoded into a bitstream. In this example, when the device determines the current block (or the residual signal C for the current block)... res When performing residual prediction, the device can perform the above-mentioned reference. FIG. 25 Operation 5 describes the operation to determine the residual difference signal of the current block. It can also process the residual difference signal of the current block. To encode it into a bitstream.

[0332] In this example, whether to apply can be indicated by flag information (sent via a signal through a bitstream). FIG. 25 Operation 5 (or whether to perform residual prediction for the current block). In the example, when the flag value is 0, the flag indicates that residual prediction should be performed for the current block; when the flag value is 1, the flag indicates that residual prediction should not be performed for the current block. The flag value can be any value other than 0 or 1. FIG. 18 When operation 5 is optional, the device (e.g., image encoding device 200) can encode flag information indicating whether residual prediction is performed for the current block in the bitstream.

[0333] For example, a flag indicating whether residual prediction should be performed on the current block can be signaled in a predetermined data unit. This predetermined data unit could be a block, a maximum coding block (or coding tree unit (CTU)), a parallel block, a stripe, a frame, a sequence, etc. When the bitstream does not include the flag information, this information can be inferred as a value indicating whether residual prediction should be applied to the current block or not. An index can be used to signal the flag information. Even when signaling the application of residual prediction to the current block is used in a higher-level data unit (e.g., a frame (or frame parameter set (PPS)), a sequence (or sequence parameter set (SPS)), etc.), the flag indicating whether residual prediction should be performed on the current block can be sent in a lower-level data unit (e.g., a block).

[0334] The above reference can be executed on the decoder side. FIG. 25 The operation described corresponds to the operation on the encoder side.

[0335] For example, the device (e.g., image decoding device 100) can obtain the residual difference signal of the current block by processing the bitstream. (For example, see) FIG. 18 (See Operation 1802 and its related description). Then, the device can perform the above-mentioned steps. FIG. 18 Operations 1 through 4 describe the operations used to determine the residual signal R of the reference block. res and the prediction signal C of the current block pred (For example, see) FIG. 25 (Operations 1804 and 1806 and their related descriptions). The device can then, based on, for example, the residual difference signal of the current block... The residual signal R of the reference block res and the prediction signal C of the current block pred To determine the reconstruction signal C of the current block. rec(For example, see) FIG. 25 (Operation 1808 and its related description). In the example, the device can be based on the residual difference signal of the current block. The residual signal R of the reference block res (For example, the sum of the residual difference signal of the current block and the residual signal of the reference block) +R res To determine the residual signal C of the current block. res And it can be based on the residual signal C of the current block. res And the predicted value C of the current block pred (For example, the sum C of the residual signal of the current block and the predicted value of the current block) res +C pred To determine the reconstruction signal C of the current block. rec .

[0336] For example, it can be performed based on Equation 2. FIG. 18 The operation of related devices (e.g., image decoding device 100). For example, in conjunction with... FIG. 18 In the operation of the relevant device (e.g., image decoding device 100), the template T of the current block C Can correspond to FIG. 18 The first template or first pixel region, and the template T of the reference block. R Can correspond to FIG. 20 The second template or the second pixel region.

[0337] [Equation 2] R res =R rec -R pred C res = +R res C rec =C pred +C res Optionally, similar to the operation on the encoder side, residual prediction for the current block can be optional. In this case, the device (e.g., image decoding device 100) can obtain an indication from the bitstream whether to apply residual prediction to the current block (or the residual signal C of the current block). res The device can perform the aforementioned operations (e.g., based on the residual difference signal of the current block) when the flag information indicates that residual prediction is to be applied to the current block. The residual signal R of the reference block res and the prediction signal C of the current block pred To determine the reconstruction signal C of the current block. rec (For example, see) FIG. 26of the current block based on the residual signal C res of the current block and the prediction signal C pred of the current block (e.g., the sum of the residual signal of the current block and the prediction signal of the current block C res +C pred ) to determine the reconstructed signal C rec of the current block.

[0338] In the proposed method 1 of the disclosure, the encoder and the decoder determine the best intra prediction mode for the reference block, and determine the prediction signal C pred of the current block and the prediction signal R pred of the reference block by using the determined intra prediction mode, so that the intra prediction mode (e.g., the intra prediction mode described above with reference to FIG. 26 ) can not be signaled through the bitstream. The proposed method of the disclosure can be defined or used as a new intra mode, and in this specification, the new intra mode can be referred to as a residual prediction intra mode.

[0339] FIG. 25 Templates suitable for the proposed method of the disclosure are illustrated. FIG. 17 Examples of FIG. 18 Operation 1 or FIG. 29 Operation 1702 or FIG. 26 Operation 1804 or FIG. 26 Operation 3 can be used. In the example of FIG. 26 , it is assumed that the width and height of the block 2602 are W and H, respectively. FIG. 26 Examples of FIG. 26 are merely examples, and the shape of the template and the cost function suitable for the proposed method of the disclosure are not limited to FIG. 26 Examples of the template 2604 of (a) of FIG. 26 , the template 2614 of (b) of FIG. 26 , and the template 2624 of (c) of are shown as 4, W / 2, and H / 2, but the proposed method of the disclosure can be equally applied even when the templates 2604, 2614, and 2624 have different numbers of lines.

[0340] With reference to (a) of FIG. 26 , (b) of FIG. 23 , and (c) of FIG. 26 , the templates 2604, 2614, and 2624 corresponding to the block 2602 can have an L shape, or can include only a top neighboring pixel region (or a top neighboring block), or can include only a left neighboring pixel region (or a left neighboring block). As described above with reference to FIG. 26The L-shaped template can include, for example, a left neighboring pixel region, an above neighboring pixel region, and a top-left neighboring pixel region (or a left neighboring block, an above neighboring block, and a top-left neighboring block) of the block 2602.

[0341] In an example, the shapes of the templates 2604, 2614, and 2624 can be explicitly signaled (via a bitstream). In an example, the shapes of the templates 2604, 2614, and 2624 can be implicitly signaled (without being signaled via a bitstream). In this example, the shapes of the templates 2604, 2614, and 2624 can be determined based on the size of the current block 2602. In a particular example, when the width W of the current block 2602 is greater than its height H, the templates 2604, 2614, and 2624 can include a top neighboring pixel region (or a top neighboring block). In a particular example, when the height H of the current block 2602 is greater than its width W, the templates 2604, 2614, and 2624 can include a left neighboring pixel region (or a left neighboring block).

[0342] The size of the templates 2604, 2614, and 2624 can be fixed or can be variable. In an example, the width and height of the templates 2604, 2614, and 2624 can be 4 and 4xH_4x4_Wx4 (e.g., see (a) of FIG. 26). FIG. 26 In an example, the width and height of the templates 2604, 2614, and 2624 can be W / 2 and H / 2 (e.g., see (b) of FIG. 26). FIG. 26 In an example, the length of the templates 2604, 2614, and 2624 can be equal to the size of the current block (e.g., the current coding unit (CU)), or can be twice the size of the current block (e.g., the current CU) (e.g., see (c) of FIG. 26). FIG. 25

[0343] Referring back to FIG. 17 , in operation 1 of FIG. 26, to search a template T C that best matches the template T C of the current block from a given search region, a device (e.g., the image decoding device 100 or the image encoding device 200) can evaluate the pixel difference between the template T R of the current block and the template being currently searched. The device can determine a pixel region in the given search region that corresponds to the lowest or highest value of the cost function as the template T cur of the reference block. As described above with reference to FIG. 27 and FIG. 27 ​The cost function can include, for example, at least one of a sum of absolute differences (SAD), a sum of squared differences (SSD), or a number of pixels having the same pixel value. For example, the SAD can be determined based on Equation 3, the SSD can be determined based on Equation 4, and the number of pixels having the same pixel value Num, Tmp can be determined based on Equation 5 cur (i) a template T of the current block C pixels of the template T ref (i) a template T of the reference block R and (A? B : C) indicates that B is determined when condition A is satisfied, and C is determined when condition A is not satisfied. Thus, Equation 5 can represent the number of pixels having the same pixel value Num between the template T C of the current block and the template T R of the reference block.

[0344] [Equation 3]

[0345] [Equation 4]

[0346] [Equation 5] Num= FIG. 27 Reference pixels for intra prediction suitable for the proposed method of the present disclosure are shown. FIG. 27 Examples of the reference pixels 2704 for intra prediction suitable for the proposed method of the present disclosure are not limited to FIG. 27 Examples of the reference pixels 2704. For example, in ​ Examples of the reference pixels 2704, the number of lines of the reference pixels 2704 is shown as 4. However, the number of lines of the reference pixels 2704 can be an integer greater than or equal to 1. For example, in ​ Examples of the reference pixels 2704, the line length of the reference pixels 2704 on the left and top is shown as twice the height H and width W of the block 2702 (2H and 2W). However, the line length of the reference pixels 2704 can also be other values. For example, although FIG. 27 focused on intra prediction of the reference block 2702, the reference pixels described above with reference to FIG. 27 can be equally used even in intra prediction of the current block.

[0347] As described above with reference to FIG. 20 and FIG. 21The intra prediction modes applicable to the proposed method of the disclosure can include INTRA PLANAR, INTRA DC, INTRA ANGULAR2 to INTRA ANGULAR66, and / or other intra prediction modes of ITU-T H.266 / VVC (e.g., CCLM mode) and / or intra prediction modes of advanced video coding standards after ITU-T H.266 / VVC. All or at least some of the available intra prediction modes can be considered for predicting the reference block 2702.

[0348] The number (or size) of reference pixels for intra prediction of the reference block 2702 can be different from the template size used for searching the reference block (e.g., see FIG. 26 and related descriptions thereof). In an example, as shown in FIG. 27 , when the length of the template is equal to the block size (e.g., see FIG. 26 and related descriptions of (b) and FIG. 26 (c)), the reference pixels (or length of reference pixels) for prediction can be two times (2W) of the width W of the reference block 2702 at the top and two times (2H) of the height H of the reference block 2702 at the left. As a non-limiting example, 2Wx4 reference pixels can be used at the top-left, top, and top-right of the block 2702, and 2Hx4 reference pixels can be used at the top-left, top, and bottom-left of the block 2702.

[0349] Similar to the multi-reference line intra prediction, multiple lines of (adjacent) reference pixels of the reference block 2702 can be used for prediction. In an example, in FIG. 27 , 4 lines of reference pixels 2704 of the (adjacent) reference pixels of the reference block 2702 can be used for the intra prediction mode determination and / or intra prediction of the reference block 2702. In an example, a number of reference lines other than 4 (e.g., an integer greater than or equal to 1) can be used for the intra prediction mode determination and / or intra prediction of the reference block 2702. Among the multiple lines of reference lines, the (predicted) intra mode determined by the cost function (e.g., minimum distortion cost or minimum cost) and / or the best line (or line index or line number) used for prediction can be used for the intra prediction of the current block. In this case, the intra prediction mode and the reference line index for the current block can not be signaled.

[0350] For example, the cost function used for determining the intra prediction mode of the reference block 2702 and / or for comparing the prediction signals can include at least one of sum of absolute difference (SAD), sum of squared difference (SSD), or number of pixels with the same pixel value (e.g., see FIG. 17 , FIG. 18 and FIG. 26and its related description). For example, SAD can be determined based on Equation 3, SSD can be determined based on Equation 4, and the number of pixels with the same pixel value Num can be determined based on Equation 5. In Equations 3 to 5, Tmp cur (i) represents a pixel of the reference block 2702, and Tmp ref (i) represents a reference pixel 2704 of the reference block 2702.

[0351] FIG. 28 The proposed method 2 of the disclosure is shown.

[0352] The proposed method 2 of the disclosure can be performed by a device (e.g., the image decoding device 100 or the image encoding device 200). The proposed method 2 of the disclosure involves performing residual prediction based on a block vector, and is an example of using an intra prediction mode of a reference block as an intra prediction mode of a current block.

[0353] Referring to FIG. 28 , in operation 1, the device (e.g., the image encoding device 200) can search for a block (or a reference block) R matching the current block in a given search region of a spatial domain by using a block vector BV. For example, the search region can be the same as or different from the search region 2310 for template matching. As an example not limiting the proposed method of the disclosure, the device can obtain the block vector BV based on block matching in the given search region, and search for the block (or the reference block) R matching the current block. As a more detailed example, the device can search for the block vector BV and search for the reference block R by performing the operations described with reference to FIG. 24 , for example, in operations 1702 of FIG. 17 , operation 1 of FIG. 25 .

[0354] In operation 2 of FIG. 28 , the device (e.g., the image encoding device 200) can determine a prediction signal of the reference block for an intra prediction mode set (or multiple intra prediction modes) 2812 by using reference pixels (or neighboring pixels) of the reference block, and can find a best matching intra prediction mode from the intra prediction mode set (or multiple intra prediction modes) by comparing a distortion between the prediction signal of the reference block and a reconstructed signal R rec of the reference block. The device can determine the intra prediction mode found in this way as the intra prediction mode of the reference block, and determine the prediction signal R pred of the reference block based on the determined intra prediction mode. As an example not limiting the proposed method of the disclosure, the intra prediction mode set (or multiple intra prediction modes) can include the intra prediction modes described with reference to FIG. 20 and FIG. 21 , for example, in operations 1702 ofFIG. 17 Operation 1704 execution FIG. 28 Operation 2, and FIG. 28 The intra-prediction mode of the reference block determined from the intra-prediction mode set (or from multiple intra-prediction modes) 2812 can be compared with... FIG. 17 The intra-prediction mode of the reference block corresponds to that of the reference block.

[0355] exist FIG. 28 In operation 3, the device (e.g., image encoding device 200) can output the residual signal of the residual block (e.g., the reference block) corresponding to the intra-frame prediction mode determined in operation 2. Rres This is determined as the residual predictor for the current block. For example, the device can base it on the reconstructed signal R of the reference block. rec The prediction signal R of the reference block pred (For example, the difference R between the reconstructed signal of the reference block and the predicted signal of the reference block) rec -R pred To determine the residual signal R of the reference block res .For example, FIG. 28 Operation 3 can correspond to FIG. 25 Operation 3.

[0356] exist FIG. 28 In operation 4, the device (e.g., image encoding device 200) can predict the current block by using the reference pixel (or neighboring pixel) 2804 of the current block with the intra-prediction mode (e.g., best-match intra-prediction mode) of the reference block determined in operation 2. For example, the device can determine the prediction signal (or prediction factor) C of the current block by applying the intra-prediction mode of the reference block to the reference pixel (or neighboring pixel) 2804 of the current block. pred .

[0357] exist FIG. 28 In operation 5, the device (e.g., image encoding device 200) can base its prediction on the residual predictor of the current block (e.g., the residual signal R of the reference block). res To further predict the residual signal C of the current block. res In the example, as shown above... FIG. 17 The device can be based on the original signal C of the current block. org The prediction signal C of the current block pred The residual signal R of the reference block res To determine the residual difference signal of the current block. In the example, the device can be based on the original signal C of the current block. org and the prediction factor C of the current block pred (For example, the difference C between the original signal of the current block and the predictor of the current block) org -C predto determine a residual signal C res of the current block based on the residual signal C res of the current block and the residual signal R res of the reference block (e.g., a difference C res -R res between the residual signal of the current block and the residual signal of the reference block). For example, the operations of the device shown in FIG. 28 may be performed based on Equation 1 (see FIG. 20 and related description).

[0358] For example, operations 3 to 5 of FIG. 17 may be performed at operation 1706 of FIG. 28 .

[0359] The device (e.g., the image encoding device 200) can process the residual difference signal C of the current block to encode it into the bitstream (e.g., see operation 1708 of FIG. 17 and related description). In an example, the device can obtain transform coefficient information of the current block by performing a transform and quantization (or scaling) based on the residual difference signal of the current block, and can encode the transform coefficient information into the bitstream. In an example, the device can omit the transform and quantization (or scaling), and encode the residual signal of the current block into the bitstream. In the proposed method of the present disclosure, the residual difference signal of the current block can be encoded into the bitstream not only by these examples, but also by various methods.

[0360] Optionally, FIG. 28 operation 5 of may be optional. The residual prediction is not applied to the current block (or not applied to the residual signal C res of the current block), and the intra prediction mode of the current block can be derived only from the template-based reference region. In this case, although the device (e.g., the image encoding device 200) can determine the residual signal C res of the current block by using the intra prediction mode of the reference block as the intra prediction mode of the current block, the residual signal C res of the current block can be encoded into the bitstream by processing the residual signal C res of the current block without predicting the residual signal C res of the current block based on the residual signal R res of the reference block.

[0361] For example, when operation 5 of FIG. 28 is optional, the device can determine whether to apply the residual prediction to the current block (or to the residual signal C resPerform residual prediction. When the device determines not to perform residual prediction, it can base it on the original signal C of the current block. org and the prediction signal C of the current block pred (For example, the difference C between the original signal of the current block and the predicted signal of the current block) org -Cp red To determine the residual signal C of the current block. res It can also process the residual signal C of the current block. res To use the residual signal C of the current block res Encoded into a bitstream. In this example, when the device determines the current block (or the residual signal C for the current block)... res When performing residual prediction, the device can perform the above-mentioned reference. FIG. 28 Operation 5 describes the operation to determine the residual difference signal of the current block. It can also process the residual difference signal of the current block. To encode it into a bitstream.

[0362] In this example, whether to apply can be indicated by flag information (sent via a signal through a bitstream). FIG. 28 Operation 5 (or whether to perform residual prediction for the current block). In the example, when the flag value is 0, the flag indicates that residual prediction should be performed for the current block; when the flag value is 1, the flag indicates that residual prediction should not be performed for the current block. The flag value can be any value other than 0 or 1. FIG. 28 When operation 5 is optional, the device (e.g., image encoding device 200) can encode flag information in the bitstream indicating whether residual prediction is performed for the current block.

[0363] For example, a flag indicating whether residual prediction should be performed on the current block can be signaled in a predetermined data unit. This predetermined data unit could be a block, a maximum coding block (or coding tree unit (CTU)), a parallel block, a stripe, a frame, a sequence, etc. When the bitstream does not include the flag information, this information can be inferred as a value indicating whether residual prediction should be applied to the current block or not. An index can be used to signal the flag information. Even when signaling the application of residual prediction to the current block is used in a higher-level data unit (e.g., a frame (or frame parameter set (PPS)), a sequence (or sequence parameter set (SPS)), etc.), the flag indicating whether residual prediction should be performed on the current block can be sent in a lower-level data unit (e.g., a block).

[0364] The above reference can be executed on the decoder side. FIG. 18 The operation described corresponds to the operation on the encoder side.

[0365] For example, the device (e.g., image decoding device 100) can obtain the residual difference signal of the current block by processing the bitstream. (For example, see) FIG. 24 Operation 1802 and its related description). This device can process bitstreams to obtain the block vector (BV) of the current block (see, for example, Operation 1802 and its related description). FIG. 28 (and its related description), and can search for a block (or reference block) R that matches the current block by using BV. Then, the device can perform the above reference... FIG. 18 Operations 2 through 4 describe the operations used to determine the residual signal R of the reference block. res and the prediction signal C of the current block pred (For example, see) FIG. 18 (Operations 1804 and 1806 and their related descriptions). The device can then, based on, for example, the residual difference signal of the current block... The residual signal R of the reference block res and the prediction signal C of the current block pred To determine the reconstruction signal C of the current block. rec (For example, see) FIG. 28 (Operation 1808 and its related description). In the example, the device can be based on the residual difference signal of the current block. The residual signal R of the reference block res (For example, the residual difference signal of the current block) The sum of the residual signals of the reference block +R res To determine the residual signal C of the current block. res And it can be based on the residual signal C of the current block. res and the prediction factor C of the current block pred (For example, the sum C of the residual signal of the current block and the predictor of the current block) res +C pred To determine the reconstruction signal C of the current block. rec For example, it can be performed based on Equation 2. FIG. 18 Operation of related equipment (e.g., image decoding device 100).

[0366] Optionally, similar to the operation on the encoder side, residual prediction for the current block can be optional. In this case, the device (e.g., image decoding device 100) can obtain an indication from the bitstream whether to apply residual prediction to the current block (or the residual signal C of the current block). resThe device can perform the aforementioned operations (e.g., based on the residual difference signal of the current block) when the flag information indicates that residual prediction is to be applied to the current block. The residual signal R of the reference block res and the prediction signal C of the current block pred To determine the reconstruction signal C of the current block. rec (For example, see) FIG. 20 (Operation 1808 and its related description). When the flag information indicates that residual prediction should not be applied to the current block, the device can base its operation on the residual signal C of the current block. res and the prediction signal C of the current block pred (For example, the sum C of the residual signal of the current block and the prediction signal of the current block) res +C pred To determine the reconstruction signal C of the current block. rec .

[0367] The difference between Method 2 and Method 1 of this disclosure is that Method 2 uses a block vector (BV) instead of template matching to search for the reference block R. Therefore, in Method 2, the block vector BV can be signaled via a bitstream. Similar to Method 1, in Method 2, the encoder and decoder determine the optimal intra-prediction mode for the reference block, and the prediction signal C for the current block is determined using the determined intra-prediction mode. pred The prediction signal R of the reference block pred This allows intra-frame prediction modes to be transmitted via signals without using a bitstream (e.g., see above reference). FIG. 20 The method proposed in this disclosure can be defined or used as a new intra-frame prediction mode (or residual prediction intra-frame mode).

[0368] For example, the intra-prediction mode used for intra-prediction of the reference block can consider the above reference. FIG. 27 , 21 All or at least some of the intra-prediction modes described in section 27. See above for reference. FIG. 27 The reference pixels used for intra-prediction of the reference block are determined. The cost function used to determine the intra-prediction mode of the reference block can be the above-mentioned reference. FIG. 17 At least one of the described cost functions (e.g., see FIG. 18 , FIG. 26 and FIG. 27 (and related descriptions).

[0369] And / or, for the cost function, the sum of absolute transformation differences (SATD) can be used. FIG. 29 Its entire related description is incorporated here by reference.

[0370] FIG. 29 Method 3 proposed in this disclosure is shown.

[0371] The proposed method 3 of this disclosure can be performed by a device (e.g., image decoding device 100 or image encoding device 200). The proposed method 3 of this disclosure relates to performing residual prediction based on template matching (or residual prediction intra-mode), and is an example of signaling the intra-prediction mode of the current block via a bitstream and using the intra-prediction mode of the current block as the intra-prediction mode of a reference block.

[0372] Reference FIG. 20 In operation 1, the device (e.g., image encoding device 200) can determine the prediction signal of the current block by using reference pixels (or neighboring pixels) of the current block for a set of intra prediction modes (or multiple intra prediction modes) 2902, and can determine the prediction signal of the current block by using the original signal C of the current block. org The distortion is compared with the predicted signal of the current block to find the best-matching intra-prediction mode from the set of intra-prediction modes (or multiple intra-prediction modes). The device can determine the intra-prediction mode found in this way as the intra-prediction mode of the current block, and determine the predicted signal C of the current block based on the determined intra-prediction mode. pred As an example that does not limit the methods proposed in this disclosure, the set of intra-prediction modes (or multiple intra-prediction modes) may include those referenced above. FIG. 21 and FIG. 29 The intra-prediction mode is described. The intra-prediction mode of the current block, determined in this way, can be used as the intra-prediction mode of the reference block and can be signaled to the decoder via a bitstream.

[0373] Optionally, in FIG. 20 Operation 1 can determine the intra-prediction mode of the current block according to the Template-Based Intra-Mode Derivation (TIMD) method. Because the decoder can also determine the intra-prediction mode of the current block in the same way, the device (e.g., image encoding device 200) can signal the intra-prediction mode of the current block without transmitting it via a bitstream. Specifically, the device can do so by targeting a set of intra-prediction modes (or multiple intra-prediction modes) (e.g., see...). FIG. 21 and FIG. 17Each of the proposed methods in this disclosure can be applied to the HEVC, VVC, and other video coding standards. For example, each of the proposed methods in this disclosure can be applied to the HEVC, VVC, and other video coding standards to determine one or two intra prediction modes for a template of a current block using reference pixels of the template (or a neighboring pixel region or a neighboring block of the current block), to determine a cost function (e.g., SAD, SSD, SATD) by using a reconstructed signal and a prediction signal of the template of the current block, and to derive one or two intra prediction modes corresponding to the cost function having a lowest value or a highest value among the determined cost functions. As an example of the proposed methods without limiting this disclosure, the template of the current block can include at least one of an above neighboring pixel region, a left neighboring pixel region, or a top-left neighboring pixel region of the current block. The DIMD can be used instead of the TIMD.

[0374] For example, the operation 1704 of FIG. 29 may be performed. FIG. 29 The operation 1 of FIG. 17 may be performed, and the intra prediction mode of the current block determined from the intra prediction mode set (or from the multiple intra prediction modes) 2902 in FIG. 29 may correspond to the intra prediction mode of the current block in

[0375] The operation 2 of FIG. 29 may be performed, the device (e.g., the image encoding device 200) can generate or determine a residual signal C res of the current block based on an original signal C org of the current block and a prediction signal C pred of the current block (e.g., a difference C org -C pred between the original signal of the current block and the prediction signal of the current block). res

[0376] The operation 3 of FIG. 23 may be performed, the device (e.g., the image encoding device 200) can search for a portion (or a template) best matching to a template T C of the current block from a given search region in a spatial domain by using the template T C of the current block. As an example of the proposed methods without limiting this disclosure, the device can search for a template T R of a reference block based on the template matching prediction (TMP). The device can also determine the reference block corresponding to the template T R of the reference block. As a more detailed example, the device can search for the template T R of the reference block by performing the operations described above with respect to FIG. 17 .

[0377] For example, the operation 1704 of FIG. 29 ​Operation 1702 execution FIG. 17 Operation 3, the template T of the current block C Can be with FIG. 17 The first template or the first pixel region corresponds to, and the template T of the reference block. R Can be with FIG. 29 The second template or the second pixel region corresponds to. FIG. 25 Operation 3 can correspond to FIG. 29 Operation 1.

[0378] exist FIG. 29 In operation 4, the device (e.g., image encoding device 200) can predict a reference block by using the reference pixels (or neighboring pixels) of the reference block with the intra-prediction mode of the current block determined in operation 1 (e.g., the best-matching intra-prediction mode). For example, the device can determine the prediction signal (or prediction factor) R of the reference block by applying the intra-prediction mode of the current block to the reference pixels (or neighboring pixels) of the reference block. pred .

[0379] exist FIG. 29 In operation 5, the device (e.g., image encoding device 200) can generate a residual signal R corresponding to the intra-frame prediction mode determined in operation 1 (e.g., the residual signal R of the reference block). res This is determined as the residual predictor for the current block. For example, the device can base it on the reconstructed signal R of the reference block. rec The prediction signal R of the reference block pred (For example, the difference R between the reconstructed signal of the reference block and the predicted signal of the reference block) rec -R pred To determine the residual signal R of the reference block res .

[0380] exist FIG. 17 In operation 6, the device (e.g., image encoding device 200) can be based on a template T of the reference block. R The corresponding residual predictor (e.g., the residual signal R of the reference block) res Additionally, predict the residual signal C of the current block. res In the example, as shown in the reference FIG. 29 As described, the device can be based on the original signal C of the current block. org The prediction signal C of the current block pred The residual signal R of the reference block res To determine the residual difference signal of the current block. In the example, the device can be based on the original signal C of the current block. org and the prediction factor C of the current block pred (For example, the difference C between the original signal of the current block and the predictor of the current block) org -Cpred to determine a residual signal C res of the current block, and can determine a residual difference signal of the current block based on the residual signal C res of the current block and a residual signal R res of the reference block (e.g., a difference C res -R res between the residual signal of the current block and the residual signal of the reference block). For example, FIG. 20 The operations of the device shown in FIG. 16 can be performed based on Equation 1 (see FIG. 17 and related descriptions thereof).

[0381] For example, the operations 1706 of FIG. 29 may perform the operations 4 to 6 of FIG. 17

[0382] The device (e.g., the image encoding device 200) can process the residual difference signal of the current block to encode it into the bitstream (e.g., see FIG. 29 the operation 1708 and related descriptions thereof). In an example, the device can obtain transform coefficient information of the current block by performing a transform and quantization (or scaling) based on the residual difference signal of the current block, and can encode the transform coefficient information into the bitstream. In an example, the device can omit the transform and quantization (or scaling), and encode the residual difference signal of the current block into the bitstream. In the proposed method of the present disclosure, the residual difference signal of the current block can be encoded into the bitstream not only by these examples, but also by various methods.

[0383] Optionally, FIG. 29 the operation 6 of FIG. 29 may be optional. When the operation 6 of is not performed, the residual prediction can not be applied to the current block (or the residual signal C res of the current block). In this case, although the device (e.g., the image encoding device 200) can determine the residual signal C res of the current block using the intra prediction mode of the current block, which is signaled via the bitstream, the residual signal C res of the current block can be encoded into the bitstream without predicting the residual signal C res of the current block based on the residual signal R res of the reference block.

[0384] FIG. 29 For example, when the operation 6 of FIG. 29 is optional, the device can determine whether to predict the residual signal C res ​Perform residual prediction. When the device determines not to perform residual prediction, it can base it on the original signal C of the current block. org and the prediction signal C of the current block pred (For example, the difference C between the original signal of the current block and the predicted signal of the current block) org -C pred To determine the residual signal C of the current block. res It can also process the residual signal C of the current block. res To use the residual signal C of the current block res Encoded into a bitstream. In this example, when the device determines the current block (or the residual signal C of the current block)... res When performing residual prediction, the device can perform the above-mentioned reference. FIG. 29 Operation 6 describes the operation to determine the residual difference signal of the current block. It can also process the residual difference signal of the current block. To encode it into a bitstream.

[0385] In this example, whether to apply can be indicated by flag information (sent via a signal through a bitstream). FIG. 29 Operation 6 (or whether to perform residual prediction for the current block). In the example, when the flag information value is 0, the flag information can indicate that residual prediction should be performed for the current block, and when the flag information value is 1, the flag information can indicate that residual prediction should not be performed for the current block. The flag information value can be any value other than 0 and 1. FIG. 18 When operation 6 is optional, the device (e.g., image encoding device 200) may encode flag information indicating whether residual prediction is performed for the current block in the bit stream.

[0386] For example, a flag indicating whether residual prediction should be performed on the current block can be signaled in a predetermined data unit. This predetermined data unit could be a block, a maximum coding block (or coding tree unit (CTU)), a parallel block, a stripe, a frame, a sequence, etc. When the bitstream does not include the flag information, this information can be inferred as a value indicating whether residual prediction should be applied to the current block or not. An index can be used to signal the flag information. Even when the signaling of residual prediction application to the current block is indicated in a higher-level data unit (e.g., a frame (or frame parameter set (PPS)), a sequence (or sequence parameter set (SPS)), etc.), the flag indicating whether residual prediction should be performed on the current block can be sent in a lower-level data unit (e.g., a block).

[0387] The above reference can be executed on the decoder side. FIG. 29 The operation described corresponds to the operation on the encoder side.

[0388] For example, the device (e.g., image decoding device 100) can obtain the residual difference signal of the current block by processing the bitstream. (For example, see) FIG. 18 (Operation 1802 and its related description). The device can also obtain the intra-prediction mode of the current block by processing the bitstream. Optionally, the device can determine the intra-prediction mode of the current block based on the template (or neighboring pixel region) of the current block (e.g., the TIMD method) without obtaining the intra-prediction mode of the current block from the bitstream. The device can then perform the above-mentioned operation. FIG. 18 Operations 2 through 5 describe the operations used to determine the residual signal R of the reference block. res and the prediction signal C of the current block pred (For example, see) FIG. 29 (Operations 1804 and 1806 and their related descriptions). The device can then, based on, for example, the residual difference signal of the current block... The residual signal R of the reference block res and the prediction signal C of the current block pred To determine the reconstruction signal C of the current block. rec (For example, see) FIG. 29 (Operation 1808 and its related description). In the example, the device can be based on the residual difference signal of the current block. The residual signal R of the reference block res (For example, the residual difference signal of the current block) The sum of the residual signals of the reference block and the reference block +R res To determine the residual signal C of the current block. res And it can be based on the residual signal C of the current block. res and the prediction factor C of the current block pred (For example, the sum C of the residual signal of the current block and the predictor of the current block) res +C pred To determine the reconstruction signal C of the current block. rec .

[0389] For example, it can be performed based on Equation 2. FIG. 18 The operation of related devices (e.g., image decoding device 100). For example, in conjunction with... FIG. 18 In the operation of the relevant device (e.g., image decoding device 100), the template T of the current block C Can be with FIG. 18 The first template or the first pixel region corresponds to, and the template T of the reference block.R Can be with FIG. 29 The second template or the second pixel region corresponds to.

[0390] Alternatively, similar to the operation on the encoder side, residual prediction for the current block can be optional. In this case, the device (e.g., image decoding device 100) can obtain an indication from the bitstream whether to apply residual prediction to the current block (or the residual signal C of the current block). res The device can perform the aforementioned operations (e.g., based on the residual difference signal of the current block) when the flag information indicates that residual prediction is to be applied to the current block. The residual signal R of the reference block res and the prediction signal C of the current block pred To determine the reconstruction signal C of the current block. rec (For example, see) FIG. 26 (Operation 1808 and its related description). When the flag information indicates that residual prediction should not be applied to the current block, the device can base its operation on the residual signal C of the current block. res and the prediction signal C of the current block pred (For example, the sum C of the residual signal of the current block and the prediction signal of the current block) res +C pred To determine the reconstruction signal C of the current block. rec .

[0391] As described above, in method 3 of this disclosure, information indicating whether residual prediction is applied to the current block (or during intra-prediction of the current block) (e.g., flag information) can be signaled (via bitstream). In method 3 of this disclosure, the encoder can determine the optimal intra-prediction mode for the current block and signal it (via bitstream), and the decoder can determine the prediction signal C for the current block using the signaled intra-prediction mode. pred The prediction signal R of the reference block pred The method proposed in this disclosure can be defined or used as a new intra-frame mode (or residual prediction intra-frame mode).

[0392] For example, in FIG. 26 Operation 3 can be performed using the above reference. FIG. 30 Describe the template shape and / or cost function to search for the template T with respect to the current block. C The best matching part (or template). FIG. 30 Its entire related description is incorporated here by reference.

[0393] FIG. 20 Method 4 of the present disclosure is shown.

[0394] The proposed method 4 of the disclosure can be performed by a device (e.g., the image decoding device 100 or the image encoding device 200). The proposed method 4 of the disclosure involves performing residual prediction based on BV (or residual prediction intra mode) and is an example of signaling the intra prediction mode of the current block through a bitstream and using the intra prediction mode of the current block as the intra prediction mode of the reference block.

[0395] Referring to FIG. 21 , at operation 1, the device (e.g., the image encoding device 200) can determine a prediction signal of the current block for the intra prediction mode set (or multiple intra prediction modes) 3002 by using the reference pixels (or the neighboring pixels) of the current block, and can determine a prediction signal of the current block for the intra prediction mode set (or multiple intra prediction modes) 3002 by using the reconstructed signal of the current block and the prediction signal of the current block. org The distortion between the prediction signal of the current block and the original signal of the current block is compared to find the best matching intra prediction mode from the intra prediction mode set (or multiple intra prediction modes). The device can determine the intra prediction mode found in this way as the intra prediction mode of the current block, and determine the prediction signal of the current block based on the determined intra prediction mode. pred As an example not limiting the proposed method of the disclosure, the intra prediction mode set (or multiple intra prediction modes) can include the intra prediction modes described above with reference to FIG. 30 and FIG. 20 The intra prediction mode of the current block determined in this way can be used as the intra prediction mode of the reference block, and can be signaled to the decoder through a bitstream.

[0396] Alternatively, at operation 1 of FIG. 21 , the intra prediction mode of the current block can be determined based on the TIMD method. Since the decoder can also determine the intra prediction mode of the current block in the same way, the device (e.g., the image encoding device 200) can not signal the intra prediction mode of the current block through a bitstream. Specifically, the device can determine the prediction signal of the template of the current block by performing intra prediction on the template of the current block (or the neighboring pixel region of the current block or the neighboring block) using the reference pixels of the template of the current block for each of the intra prediction mode set (or multiple intra prediction modes) (e.g., see FIG. 17 and FIG. 30 and the description thereof), can determine the cost function (e.g., SAD, SSD, and SATD) by using the reconstructed signal of the template of the current block and the prediction signal, and can find one or two intra prediction modes corresponding to the cost function having the lowest value or the highest value among the determined cost functions. As an example not limiting the proposed method of the disclosure, the template of the current block can include at least one of the upper neighboring pixel region, the left neighboring pixel region, or the upper-left neighboring pixel region of the current block. The TIMD can be replaced with the decoder-side intra mode derivation (DIMD) method.

[0397] For example, operation 1704 of FIG. 30 may be performed FIG. 17 operation 1 of FIG. 30 and the intra prediction mode of the current block determined from the intra prediction mode set 3002 (or from the plurality of intra prediction modes) in FIG. 30 may correspond to the intra prediction mode of the current block of

[0398] In operation 2 of FIG. 24 , the device (e.g., image encoding device 200) can generate or determine a residual signal C res of the current block. For example, the device can generate or determine the residual signal C org of the current block based on the original signal C pred of the current block (e.g., the difference C org -C pred between the original signal of the current block and the prediction signal of the current block). res

[0399] In operation 3 of FIG. 17 , the device (e.g., image encoding device 200) can search for a block (or reference block) R matching the current block in a given search region of the spatial domain by using the block vector BV. For example, the search region can be the same as or different from the search region 2310 used for template matching. As an example not limiting the proposed methods of the present disclosure, the device can obtain the block vector BV based on the block matching in the given search region, and search for the block (or reference block) R matching the current block. As a more detailed example, the device can search for the block vector BV of the current block and search for the reference block R by performing the operations described with reference to FIG. 30 . For example, operation 3 of FIG. 30 may be performed FIG. 30 operation 2 of

[0400] In operation 4 of FIG. 30 , the device (e.g., image encoding device 200) can predict the reference block with reference pixels (or neighboring pixels) of the reference block by using the intra prediction mode (e.g., the best matching intra prediction mode) of the current block determined in operation 1. For example, the device can determine a prediction signal (or predictor) R pred of the reference block by applying the intra prediction mode of the current block to the reference pixels (or neighboring pixels) of the reference block.

[0401] In operation 5 of FIG. 17 , the device (e.g., image encoding device 200) can generate or determine a residual block (e.g., a residual signal R res ​This is determined as the residual predictor for the current block. For example, the device can base it on the reconstructed signal R of the reference block. rec The prediction signal R of the reference block pred (For example, the difference R between the reconstructed signal of the reference block and the predicted signal of the reference block) rec -R pred To determine the residual signal R of the reference block res .

[0402] exist FIG. 30 In operation 6, the device (e.g., image encoding device 200) can be based on a template T of the reference block. R The corresponding residual predictor (e.g., the residual signal R of the reference block) res To further predict the residual signal C of the current block. res In the example, as shown above... FIG. 20 The device can be based on the original signal C of the current block. org The prediction signal C of the current block pred The residual signal R of the reference block res To determine the residual difference signal of the current block. In the example, the device can be based on the original signal C of the current block. org and the prediction factor C of the current block pred (For example, the difference C between the original signal of the current block and the predictor of the current block) org -C pred To determine the residual signal C of the current block. res And it can be based on the residual signal C of the current block. res The residual signal R of the reference block res (For example, the difference C between the residual signal of the current block and the residual signal of the reference block) res -R res To determine the residual difference signal of the current block For example, it can be performed based on Equation 1. FIG. 17 The operation of the device shown in the figure (see FIG. 30 (and related descriptions).

[0403] For example, it can be FIG. 17 Operation 1706 execution FIG. 30 Operations 4 to 6.

[0404] The device (e.g., image encoding device 200) can process the residual difference signal of the current block. To encode it into a bitstream (e.g., see...) FIG. 30(Operation 1708 and its related description). In the example, the device can obtain the transform coefficient information of the current block by performing transform and quantization (or scaling) based on the residual difference signal of the current block, and can encode the transform coefficient information into a bitstream. In the example, the device can omit the transform and quantization (or scaling) and encode the residual difference signal of the current block into a bitstream. In the proposed method of this disclosure, not only through these examples, but also through various other methods, the residual difference signal of the current block can be encoded into a bitstream.

[0405] Optionally, FIG. 30 Operation 6 can be optional. When not executed... FIG. 30 In operation 6, residual prediction may not be applied to the current block (or the residual signal C of the current block). res In this case, although the device (e.g., image encoding device 200) can determine the residual signal C of the current block by using the intra-prediction mode of the current block. res The intra-prediction mode of the current block is transmitted via a bitstream signal, but can be processed by handling the residual signal C of the current block. res The residual signal C of the current block res Encoded into a bitstream, rather than based on the residual signal R of the reference block. res Predict the residual signal C of the current block. res .

[0406] For example, when FIG. 30 When operation 6 is optional, the device can determine whether to target the current block (or the residual signal C for the current block). res Perform residual prediction. When the device determines not to perform residual prediction, it can base it on the original signal C of the current block. org and the prediction signal C of the current block pred (For example, the difference C between the original signal of the current block and the predicted signal of the current block) org -C pred To determine the residual signal C of the current block. res It can also process the residual signal C of the current block. res To use the residual signal C of the current block res Encode into a bitstream. In this example, when the device determines the current block (or the residual signal C of the current block)... res When performing residual prediction, the device can perform the above-mentioned reference. FIG. 18 Operation 6 describes the operation to determine the residual difference signal of the current block. It can also process the residual difference signal of the current block. To encode it into a bitstream.

[0407] In this example, whether to apply can be indicated by flag information (sent via a signal through a bitstream). FIG. 30 Operation 6 (or whether to perform residual prediction for the current block). In the example, when the flag value is 0, the flag indicates that residual prediction should be performed for the current block; when the flag value is 1, the flag indicates that residual prediction should not be performed for the current block. The flag value can be any value other than 0 or 1. FIG. 18 When operation 6 is optional, the device (e.g., image encoding device 200) may encode flag information indicating whether residual prediction is performed for the current block in the bit stream.

[0408] For example, a flag indicating whether residual prediction should be performed on the current block can be signaled in a predetermined data unit. This predetermined data unit could be a block, a maximum coding block (or coding tree unit (CTU)), a parallel block, a stripe, a frame, a sequence, etc. When the bitstream does not include the flag information, this information can be inferred as a value indicating whether residual prediction should be applied to the current block or not. An index can be used to signal the flag information. Even when signaling the application of residual prediction to the current block is used in a higher-level data unit (e.g., a frame (or frame parameter set (PPS)), a sequence (or sequence parameter set (SPS)), etc.), the flag indicating whether residual prediction should be performed on the current block can be sent in a lower-level data unit (e.g., a block).

[0409] The above reference can be executed on the decoder side. FIG. 18 The operation described corresponds to the operation on the encoder side.

[0410] For example, the device (e.g., image decoding device 100) can obtain the residual difference signal of the current block by processing the bitstream. (For example, see) FIG. 30 (Operation 1802 and its related description). The device can also obtain the intra-prediction mode of the current block by processing the bitstream. Optionally, the device can determine the intra-prediction mode of the current block based on the template (or neighboring pixel region) of the current block (e.g., the TIMD method) without obtaining the intra-prediction mode of the current block from the bitstream. The device can then perform the above-mentioned operation. FIG. 18 Operations 2 through 5 describe the operations used to determine the residual signal R of the reference block. res and the prediction signal C of the current block pred (For example, see) FIG. 20(Operations 1804 and 1806 and their related descriptions). The device can then, based on, for example, the residual difference signal of the current block... The residual signal R of the reference block res and the prediction signal C of the current block pred To determine the reconstruction signal C of the current block. rec (For example, see) FIG. 21 (Operation 1808 and its related description). In the example, the device can be based on the residual difference signal of the current block. The residual signal R of the reference block res (For example, the sum of the residual difference signal of the current block and the residual signal of the reference block) +R res To determine the residual signal C of the current block. res And it can be based on the residual signal C of the current block. res and the prediction factor C of the current block pred (For example, the sum C of the residual signal of the current block and the predictor of the current block) res +C pred To determine the reconstruction signal C of the current block. rec In the example, it can be performed based on Equation 2. FIG. 20 Operation of related equipment (e.g., image decoding device 100).

[0411] Alternatively, similar to the operation on the encoder side, residual prediction for the current block can be optional. In this case, the device (e.g., image decoding device 100) can obtain an indication from the bitstream whether to apply residual prediction to the current block (or the residual signal C of the current block). res The device can perform the aforementioned operations (e.g., based on the residual difference signal of the current block) when the flag information indicates that residual prediction is to be applied to the current block. The residual signal R of the reference block res and the prediction signal C of the current block pred To determine the reconstruction signal C of the current block. rec (For example, see) FIG. 21 (Operation 1808 and its related description). When the flag information indicates that residual prediction should not be applied to the current block, the device can base its operation on the residual signal C of the current block. res and the prediction signal C of the current block pred (For example, the sum C of the residual signal of the current block and the prediction signal of the current block) res +C pred To determine the reconstruction signal C of the current block. rec .

[0412] As described above, in the proposed method 4 of the disclosure, information (e.g., flag information) indicating whether or not to apply residual prediction to the current block (or during intra prediction of the current block) can be signaled (via a bitstream). In the proposed method 4 of the disclosure, an encoder can determine the best intra prediction mode for the current block and signal the best intra prediction mode (via a bitstream), and a decoder can determine the prediction signal C pred of the current block by using the signaled intra prediction mode. pred In the proposed method 4 of the disclosure, an encoder can signal the block vector BV of the current block (via a bitstream), and a decoder can determine the reference block corresponding to the current block by using the signaled block vector BV of the current block. The proposed method of the disclosure can be defined or used as a new intra mode (or residual prediction intra mode).

[0413] The proposed method of the disclosure can be used as an independent intra prediction mode. For example, the proposed method of the disclosure can be referred to as a residual prediction intra mode. As described above with reference to the proposed methods 1 to 4 of the disclosure, flag information indicating whether or not to apply residual prediction to the current block can be signaled through a bitstream in units of a predetermined block (e.g., a coding unit (CU) and a coding tree unit (CTU)). As described above with reference to the proposed methods 1 to 4 of the disclosure, when the flag information is signaled through a bitstream, residual prediction for the current block can be optional.

[0414] In an example, when the residual prediction intra mode is applied in the proposed method of the disclosure, a residual reorder operation can be applied to the current block. Whether or not to apply the residual reorder operation to the current block can be signaled by flag information separate from the flag information indicating whether or not to apply residual prediction to the current block.

[0415] In an example, when an intra block copy (IBC) mode is applied, the proposed method of the disclosure can be used as an additional option. For example, when the IBC mode is applied to the current block, flag information indicating whether to use a reference block as a prediction factor for the current block in the IBC mode or to generate a residual signal R res of the reference block by using the proposed method of the disclosure as a prediction factor for the current block and perform residual prediction for the current block.

[0416] In an example, the proposed method of the disclosure can be used as an additional option in the intra-TMP mode. For example, when the intra-TMP mode is applied to the current block, a flag information can be signaled through the bitstream, which indicates whether the reference block is used as the predictor of the current block in the intra-TMP mode or the residual signal R res is generated as the predictor of the current block and residual prediction is performed for the current block.

[0417] In an example, the prediction mode (or residual prediction intra mode) according to the proposed method of the disclosure can be weighted combined (e.g., weighted average or weighted sum) with another intra prediction mode (e.g., see FIG. 20 and FIG. 21 and the related description thereof) to determine the predictor of the current block. In an example, the prediction signal C pred of the current block determined based on the proposed method of the disclosure and the prediction signal of the current block determined based on the intra prediction mode (e.g., see FIG. 31 and FIG. 32 and the related description thereof) can be weighted combined (e.g., weighted average or weighted sum) with each other to determine the prediction signal C pred of the current block. In an example, the residual difference signal of the current block determined based on the proposed method of the disclosure and the residual signal of the current block determined based on the intra prediction mode (e.g., see FIG. 31 and FIG. 32 and the related description thereof) can be weighted combined (e.g., weighted average or weighted sum) with each other to determine the residual signal of the current block.

[0418] In an example, in the proposed method of the disclosure, the block (e.g., the current block or the reference block) can be a coding unit (CU) (or coding block), a prediction unit (PU) (or prediction block), a transform unit (TU) (or transform block), or any other unit block.

[0419] In an example, in the proposed method of the disclosure, while the residual signal C res of the current block can be predicted in the spatial domain, the residual signal C res of the current block can also be predicted in the transform domain. When the residual signal is predicted in the transform domain, the apparatus (e.g., the image coding apparatus 200) can generate the residual signal C resThe transform coefficient information is determined by performing a transform (using a transform kernel such as a discrete cosine transform (DCT)), and the residual subtraction is performed in the transform domain by discarding at least some high frequency coefficients from the transform coefficient information (e.g., by setting the high frequency coefficients to 0). The device can then determine a final residual signal of the current block in the spatial domain by inverse transforming the result of the residual subtraction, and process the final residual signal to encode the final residual signal into the bitstream. Based on the proposed method of the disclosure, a decoder can process the bitstream to obtain the residual signal C res of the current block, and can determine a reconstructed signal C rec of the current block.

[0420] FIG. 31 and FIG. 32 An example of a device configured to perform the proposed method of the disclosure is shown. FIG. 31 An example of an image decoding device 100 configured to implement the proposed method of the disclosure is shown, and FIG. 32 An example of an image encoding device 200 configured to implement the proposed method of the disclosure is shown. FIG. 31 and FIG. 18 are merely examples and are not intended to limit the device for implementing the proposed method of the disclosure. For example, in addition to the components shown in FIG. 32 and FIG. 17 , the devices 100 and 200 can include other components (e.g., a transceiver), and some components can be omitted or can be provided outside the devices 100 and 200.

[0421] Referring to ​ , the device 100 can include at least one memory 110 and at least one processor 120. The at least one memory 110 can include instructions to implement the proposed method of the disclosure. The at least one processor 120 can be operatively connected to the at least one memory 110 and configured to execute the instructions included in the at least one memory 110 to implement the proposed method of the disclosure (e.g., see ​ and related descriptions thereof).

[0422] Referring to ​ , the device 200 can include at least one memory 210 and at least one processor 220. The at least one memory 210 can include instructions to implement the proposed method of the disclosure. The at least one processor 220 can be operatively connected to the at least one memory 210 and configured to execute the instructions included in the at least one memory 210 to implement the proposed method of the disclosure (e.g., see ​ and related descriptions thereof).

[0423] The proposed method of the disclosure can be implemented as hardware, software, or a combination of hardware and software. When implemented as software, the proposed method of the disclosure can be implemented as a program executable on a computer, and the implemented program can be stored in a non-transitory computer-readable storage medium. The program stored in the non-transitory computer-readable storage medium can be configured to be executable by at least one processor 120 or 220 of the device 100 or 200, and can be configured with instructions configured to implement the proposed method of the disclosure when executed by the at least one processor 120 or 220.

[0424] The non-transitory computer-readable storage medium can continuously store a computer executable program, or temporarily store a computer executable program for execution or download. In addition, the medium can be any one of various recording media or storage media in which a single or a plurality of hardware is combined, and the medium is not limited to a medium directly connected to a computer system, but can be distributed over a network. Examples of the medium include a magnetic medium (e.g., a hard disk, a floppy disk, or a magnetic tape), an optical medium (e.g., a compact disc read only memory (CD-ROM) or a digital versatile disc (DVD)), a magneto-optical medium (e.g., a floptical disk), and a ROM, a random access memory (RAM), and a flash memory configured to store program instructions. The machine-readable storage medium can be provided as a non-transitory storage medium. The "non-transitory storage medium" is a tangible device, and only means that it does not include a signal (e.g., an electromagnetic wave). The term does not distinguish between cases where data is stored semi-permanently in the storage medium and cases where data is temporarily stored. For example, the "non-transitory storage medium" can include a buffer that temporarily stores data.

[0425] Other examples of the medium include recording media and storage media managed by an application store that distributes applications, or websites, servers, etc. that supply or distribute other various types of software.

[0426] While one or more embodiments of the disclosure have been described with reference to the accompanying drawings, it will be understood by those of ordinary skill in the art that various changes in form and details can be made therein without departing from the spirit and scope defined by the following claims.

Claims

1. A method for decoding a bitstream performed by a device, the method comprising: The residual difference signal of the current block is obtained by processing the bit stream; Determine the reference block corresponding to the current block in the current frame, including the current block; Based on the reconstructed signal and the predicted signal of the reference block, the residual signal of the reference block is determined; and The reconstructed signal of the current block is determined based on the residual difference signal of the current block, the residual signal of the reference block, and the prediction signal of the current block. The prediction signal of the reference block is determined based on at least one reference pixel included in the neighboring blocks of the reference block in the current frame, and The prediction signal for the current block is determined based on at least one reference pixel included in the neighboring blocks of the current block in the current frame.

2. The method according to claim 1, further comprising: Based on the reconstructed signal of the reference block and at least one reference pixel included in the neighboring blocks of the reference block, the intra-prediction mode of the reference block is determined from multiple intra-prediction modes. The prediction signal of the reference block is determined based on the intra-frame prediction mode of the reference block and at least one reference pixel included in the neighboring blocks of the reference block in the current frame. The prediction signal for the current block is determined based on the intra-frame prediction mode of the reference block and at least one reference pixel included in the neighboring blocks of the current block in the current frame.

3. The method according to claim 2, wherein, The step of determining the intra prediction mode of the reference block from the plurality of intra prediction modes includes: Based on the reconstructed signal of the reference block and at least one reference pixel included in the neighboring blocks of the reference block, a function value is calculated for each of the plurality of intra-prediction modes; and The intra-prediction mode corresponding to the lowest or highest function value among the calculated function values ​​is determined as the intra-prediction mode for the current block.

4. The method according to claim 3, wherein, The function value includes at least one of the following: sum of absolute differences (SAD), sum of squared differences (SSD), or the number of pixels with the same pixel value.

5. The method according to claim 1, further comprising: Based on the reference block being encoded in an intra-prediction mode, the intra-prediction mode of the reference block is identified. The prediction signal of the reference block is determined based on the intra-frame prediction mode of the reference block and at least one reference pixel included in the neighboring blocks of the reference block in the current frame. The prediction signal for the current block is determined based on the intra-frame prediction mode of the reference block and at least one reference pixel included in the neighboring blocks of the current block in the current frame.

6. The method according to claim 1, further comprising: Obtain the intra-prediction mode of the current block from the bitstream. The prediction signal of the reference block is determined based on the intra-frame prediction mode of the current block and at least one reference pixel included in the neighboring blocks of the reference block in the current frame. The prediction signal for the current block is determined based on the intra-frame prediction mode of the current block and at least one reference pixel included in the neighboring blocks of the current block in the current frame.

7. The method according to claim 1, further comprising: Based on the pixel regions adjacent to the current block, the intra-prediction mode of the current block is determined from multiple intra-prediction modes. The prediction signal of the reference block is determined based on the intra-frame prediction mode of the current block and at least one reference pixel included in the neighboring blocks of the reference block in the current frame. The prediction signal for the current block is determined based on the intra-frame prediction mode of the current block and at least one reference pixel included in the neighboring blocks of the current block in the current frame.

8. The method according to claim 1, wherein, The step of determining the reference block corresponding to the current block includes: Determine the first pixel region adjacent to the current block in the current frame; In the current frame, determine the second pixel region corresponding to the first pixel region adjacent to the current block; and In the current frame, determine the reference block corresponding to the second pixel region.

9. The method according to claim 8, wherein, The first pixel region adjacent to the current block includes at least one of the pixel region adjacent to the top of the current block, the pixel region adjacent to the upper left of the current block, or the pixel region adjacent to the left side of the current block.

10. The method of claim 8, wherein, The first pixel region adjacent to the current block has a fixed size, a size determined based on the size of the current block, and the same width or height as the coding unit that includes the current block, or a width or height twice that of the coding unit that includes the current block.

11. The method according to claim 8, wherein, The step of determining the second pixel region corresponding to the first pixel region adjacent to the current block in the current image includes: determining the pixel region corresponding to the lowest or highest function value among the function values ​​calculated based on the pixels of the first pixel region and the pixels of the pixel region in the reconstructed area of ​​the current image as the second pixel region.

12. The method according to claim 1, wherein, The step of determining the reference block corresponding to the current block includes: Obtain the block vector of the current block from the bitstream; and The reference block is determined in the current frame based on the block vector of the current block.

13. The method according to claim 1, further comprising: Obtain flag information from the bitstream indicating whether to perform residual prediction for the current block. Specifically, the operation of performing residual prediction for the current block based on the flag information, the operation of obtaining the residual difference signal of the current block, and the operation of determining the reconstruction signal of the current block based on the residual difference signal of the current block, the residual signal of the reference block, and the prediction signal of the current block.

14. A method for encoding a bitstream performed by a device, the method comprising: Determine the reference block corresponding to the current block in the current frame, including the current block; Based on the reconstructed signal and the predicted signal of the reference block, the residual signal of the reference block is determined; The residual difference signal of the current block is determined based on the residual signal of the reference block, the original signal of the current block, and the predicted signal of the current block; and By processing the residual difference signal of the current block, the residual difference signal of the current block is encoded into a bit stream. The prediction signal of the reference block is determined based on at least one reference pixel included in the neighboring blocks of the reference block in the current frame, and The prediction signal for the current block is determined based on at least one reference pixel included in the neighboring blocks of the current block in the current frame.

15. A non-transitory storage medium for storing a bit stream generated by a method, wherein, The method includes: Determine the reference block corresponding to the current block in the current frame, including the current block; Based on the reconstructed signal and the predicted signal of the reference block, the residual signal of the reference block is determined; The residual difference signal of the current block is determined based on the residual signal of the reference block, the original signal of the current block, and the predicted signal of the current block; and By processing the residual difference signal of the current block, the residual difference signal of the current block is encoded into a bit stream. The prediction signal of the reference block is determined based on at least one reference pixel included in the neighboring blocks of the reference block in the current frame, and The prediction signal for the current block is determined based on at least one reference pixel included in the neighboring blocks of the current block in the current frame.

Images