Method and apparatus for encoding or decoding an image by block mapping

Abstract

A method and apparatus for encoding or decoding an image by block mapping are disclosed. According to an embodiment, a method and apparatus for decoding an image are characterized by receiving a bitstream of an encoded image, determining one or more blocks in a picture by partitioning information obtained from the bitstream, determining a block map indicating whether a previous block among the one or more blocks has been decoded, determining, based on the block map, a neighboring area that a current block among the one or more blocks can refer to according to a predetermined scan order, and decoding the current block based on the neighboring area that can be referred to.

Description

[0001] This application is a divisional application of the invention patent application with the application number "201680091331.8" and the title "Method and apparatus for encoding or decoding an image by using block mapping" filed on October 10, 2016. TECHNICAL FIELD

[0002] The present disclosure relates to a method and apparatus for encoding or decoding an image, and more particularly, to a method and apparatus for encoding or decoding an image by using block mapping. BACKGROUND

[0003] Image data is encoded using a codec according to a data compression standard, for example, a Moving Picture Experts Group (MPEG) standard, and is recorded in the form of a bitstream on a recording medium or transmitted via a communication channel.

[0004] As development and supply of hardware for reproducing and storing high definition or high quality image content, the demand for a codec for efficiently encoding or decoding high definition or high quality image content is increasing. The encoded image content can be reproduced by decoding the encoded image content. Recently, a method of efficiently compressing such high resolution or high quality image content has been implemented. For example, a method of efficiently compressing an image by processing an image to be encoded in an arbitrary manner has been implemented.

[0005] Various data units can be used to compress an image, and there can be a containment relationship between such data units. In order to determine the size of a data unit to be used for image compression, various methods can be used to divide the data unit. An image can be encoded or decoded by determining an optimal data unit according to the characteristics of the image. SUMMARY

[0006] TECHNICAL PROBLEM

[0007] A method and apparatus for encoding or decoding an image by using block mapping are provided.

[0008] SOLUTION TO PROBLEM

[0009] According to an aspect of the present disclosure, a method of decoding an image by using block mapping includes receiving a bitstream of an encoded image, determining one or more blocks in a picture by using division information obtained from the bitstream, determining a block map indicating a decoded block among the one or more blocks, determining a neighboring area that can be used for a current block to be decoded among the one or more blocks in a certain scan order based on the block map, and decoding the current block based on the neighboring area that can be used for the reference.

[0010] In one embodiment, in a method of decoding an image by using a block map, the block map can include regions corresponding to the one or more blocks, wherein a region of the block map corresponding to a decoded block among the one or more blocks can be set as an available region, and a region of the block map corresponding to an undecoded block among the one or more blocks can be set as an unavailable region.

[0011] In one embodiment, in a method of decoding an image by using a block map, the step of determining the block map can include updating the block map with a value indicating that each block among the one or more blocks is decoded, whenever each block among the one or more blocks is decoded.

[0012] In one embodiment, in a method of decoding an image by using a block map, the one or more blocks can include one or more coding units, or one or more transform units.

[0013] In one embodiment, in a method of decoding an image by using a block map, the block map can be created for a maximum coding unit.

[0014] In one embodiment, in a method of decoding an image by using a block map, the one or more blocks can have a square shape or a non-square shape.

[0015] In one embodiment, in a method of decoding an image by using a block map, the neighboring region available for being referenced can include at least one of a region temporally neighboring a current block or a region spatially neighboring the current block.

[0016] In one embodiment, in a method of decoding an image by using a block map, the step of determining the neighboring region available for being referenced can include configuring a plurality of merge candidates for deriving motion information of a current block, wherein the plurality of merge candidates include at least one of a region temporally neighboring the current block or a region spatially neighboring the current block, and when there are two or more merge candidates included in a same coding unit among the plurality of merge candidates, replacing at least one of the two or more merge candidates with a merge candidate included in a different coding unit, and the step of decoding the current block can include deriving motion information of the current block based on the plurality of merge candidates, and predicting the current block by using the derived motion information.

[0017] In one embodiment, in a method for decoding an image using block mapping, the step of determining the neighboring regions that can be used for reference may include: configuring a plurality of merging candidates for deriving motion information of the current block, wherein the plurality of merging candidates includes at least one region that is spatially adjacent to the current block or temporally adjacent to the current block; when there are two or more merging candidates with the same motion information among the plurality of merging candidates, replacing at least one of the two or more merging candidates with merging candidates that have different motion information; and the step of decoding the current block may include: deriving motion information of the current block based on the plurality of merging candidates, and predicting the current block using the derived motion information.

[0018] In one embodiment, in a method for decoding an image using a block map, the step of determining the neighboring regions that can be used for reference may include: configuring a plurality of merging candidates for deriving motion information of the current block, wherein the plurality of merging candidates includes at least one region that is spatially adjacent to the current block or temporally adjacent to the current block; when there is a first neighboring region among the plurality of merging candidates corresponding to an unavailable region in the block map, replacing the first neighboring region with a second neighboring region corresponding to an available region in the block map; and the step of decoding the current block may include: deriving motion information of the current block based on the plurality of merging candidates; and predicting the current block using the derived motion information.

[0019] According to another aspect of this disclosure, a method for encoding an image using a block map includes: determining one or more blocks in a picture; determining a block map representing a decoded block among the one or more blocks; determining, based on the block map, a neighboring region that can be referenced by a current block among the one or more blocks to be encoded in a specific scanning order; and encoding the current block based on the neighboring region that can be referenced.

[0020] In one embodiment, in a method of encoding an image using a block map, the block map may include regions corresponding to the one or more blocks, wherein regions in the block map corresponding to decoded blocks among the one or more blocks may be set as available regions, and regions in the block map corresponding to undecoded blocks among the one or more blocks may be set as unavailable regions.

[0021] In one embodiment, in a method of encoding an image using a block map, the step of determining the block map may include updating the block map with a value indicating the decoding of each of the one or more blocks whenever each of the blocks is decoded.

[0022] In one embodiment, in a method of encoding an image by using block mapping, the one or more blocks can have a square shape or a non-square shape.

[0023] In one embodiment, in a method of encoding an image by using block mapping, the neighboring region that can be used for being referenced can include at least one of a region that is temporally neighboring a current block or a region that is spatially neighboring the current block.

[0024] According to another aspect of the disclosure, an apparatus of decoding an image by using block mapping includes a receiver configured to receive a bitstream of an encoded image, a decoder configured to determine one or more blocks in a picture by using partition information obtained from the bitstream, determine a block map indicating a decoded block among the one or more blocks, determine a neighboring region that can be used for being referenced for a current block that is to be decoded in a certain scan order among the one or more blocks based on the block map, and decode the current block based on the neighboring region that can be used for being referenced. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 is a schematic block diagram of an image encoding apparatus 100 according to an embodiment.

[0026] Figure 2 is a schematic block diagram of an image decoding apparatus 200 according to an embodiment.

[0027] Figure 3 is a diagram illustrating a method of determining a region that can be used for being referenced according to an embodiment.

[0028] Figure 4 is a diagram illustrating a block map according to an embodiment.

[0029] Figure 5 is a diagram illustrating a method of configuring a plurality of merge candidates to derive motion information of a current block according to an embodiment.

[0030] Figure 6 illustrates a method of changing a merge candidate having the same motion information according to various embodiments.

[0031] Figure 7 illustrates a method of changing a merge candidate of a region that cannot be used for being referenced according to various embodiments.

[0032] Figure 8 is a flowchart of a method of encoding an image according to an embodiment.

[0033] Figure 9 is a flowchart of a method of decoding an image according to an embodiment.

[0034] Figure 10 A process of determining at least one coding unit as a current coding unit is divided is shown according to an embodiment.

[0035] Figure 11 A process of determining at least one coding unit when a coding unit having a non-square shape is divided is shown according to an embodiment.

[0036] Figure 12 A process of dividing a coding unit based on at least one of block shape information and division shape information is shown according to an embodiment.

[0037] Figure 13 A method of determining a particular coding unit among an odd number of coding units is shown according to an embodiment.

[0038] Figure 14 An order of processing a plurality of coding units when the plurality of coding units is determined by dividing a current coding unit is shown according to an embodiment.

[0039] Figure 15 A process of determining to divide a current coding unit into an odd number of coding units when the coding units cannot be processed in a particular order is shown according to an embodiment.

[0040] Figure 16 A process of determining at least one coding unit by dividing a first coding unit is shown according to an embodiment.

[0041] Figure 17 A shape into which a second coding unit having a non-square shape determined when the first coding unit is divided is constrained when the second coding unit satisfies a particular condition is shown according to an embodiment.

[0042] Figure 18 A process of dividing a coding unit having a square shape when division shape information cannot be used to indicate that the coding unit is divided into four square shapes is shown according to an embodiment.

[0043] Figure 19 An order of processing a plurality of coding units can be changed according to a process of dividing the coding units according to an embodiment.

[0044] Figure 20 A process of determining a depth of a coding unit as a shape and a size of the coding unit are changed when a plurality of coding units is determined when the coding unit is recursively divided is shown according to an embodiment.

[0045] Figure 21A portion index (PID) for distinguishing a depth and a coding unit, which can be determined according to a shape and a size of a coding unit, according to an embodiment, is shown.

[0046] Figure 22 Determining a plurality of coding units according to a plurality of specific data units included in a picture, according to an embodiment, is shown.

[0047] Figure 23 A processing block used as a criterion for determining a determination order of at least one reference coding unit included in a picture, according to an embodiment, is shown. DETAILED DESCRIPTION

[0048] Advantages and features of the embodiments set forth herein and the methods for achieving them will be clear from the following description of the embodiments taken together with the accompanying drawings. However, the present disclosure is not limited to the embodiments and can be implemented in various different forms. The embodiments are merely provided so that the present disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art.

[0049] The terms used herein will be briefly described and various embodiments will be described in detail later.

[0050] In the present disclosure, if possible, general terms widely used now are selected considering the functions of the present disclosure, but not general terms can be selected according to the intention of those skilled in the art, precedents, or new technology, etc. Some terms can be arbitrarily selected by the applicant. In this case, the meanings of the terms will be explained in correspondence with the parts of the present disclosure to be explained in detail. Therefore, the terms used herein should not be defined based on its name but based on its meaning and the entire context of the present disclosure.

[0051] As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise.

[0052] It will be understood that when an element is referred to as being "connected to" another element, it can be directly connected to the other element or electrically connected to the other element through another element, unless otherwise noted. It will be further understood that the terms "comprises" and / or "comprising," or "includes" and / or "including" when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. It will be understood that when an element is referred to as being "connected to" another element, it can be directly connected to the other element or electrically connected to the other element through another element, unless otherwise noted. It will be further understood that the terms "comprises" and / or "comprising," or "includes" and / or "including" when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise.

[0053] Hereinafter, the term "image" should be understood to include a still image such as a still picture of a video and a moving picture (i.e., a dynamic image) as a video.

[0054] The term "sample" should be understood as data assigned to a video sampling position (i.e., data to be processed). For example, pixel values of an image in a spatial domain and transform coefficients in a transform domain can be samples. A unit including at least one such sample can be defined as a block.

[0055] Embodiments will be described in detail below so that those of ordinary skill in the art can easily implement the embodiments. In the drawings, parts irrelevant to clearly describing the present disclosure are omitted.

[0056] Hereinafter, a detailed description will be given of an image encoding apparatus, an image decoding apparatus, a method of encoding an image, and a method of decoding an image according to embodiments. Figures 1 to 23 A detailed description will be given of an image encoding apparatus, an image decoding apparatus, a method of encoding an image, and a method of decoding an image according to embodiments. Hereinafter, a detailed description will be given of a method and an apparatus for encoding or decoding an image by using block mapping. Figures 1 to 9 A detailed description will be given of a method and an apparatus for encoding or decoding an image by using block mapping. Figures 10 to 23 A method of determining a data unit of an image is described.

[0057] Hereinafter, a detailed description will be given of an image encoding apparatus, an image decoding apparatus, a method of encoding an image, and a method of decoding an image according to embodiments. Figures 1 to 9 A detailed description will be given of a method and an apparatus for encoding or decoding an image by using block mapping.

[0058] Figure 1 is a schematic block diagram of an image encoding apparatus 100 according to an embodiment. The image encoding apparatus 100 according to an embodiment includes an encoder 110 and a transmitter 120.

[0059] In one embodiment, the encoder 110 can divide image data of a current picture into maximum coding units according to a maximum size of a coding unit. Each of the maximum coding units can include coding units divided according to a block shape and a division type. In one embodiment, image data of a spatial domain included in the maximum coding units can be hierarchically classified according to a block shape and a division type. The block shape of a coding unit can be a square, a rectangle, or an arbitrary geometric shape, and thus is not limited to a data unit of a specific size.

[0060] As the size of a picture to be encoded increases, by encoding an image in maximum units, the image can be encoded at a greater image compression rate. However, when the coding units increase and the size of the coding units is fixed, the image cannot be effectively encoded by reflecting the characteristics of a continuously changing image.

[0061] For example, when a flat area of the sea or the sky in an image is encoded, the compression rate can be better as the size of the coding unit is larger, but when an image of a crowded area of people or a crowded area of buildings is encoded, the compression rate can be better as the size of the coding unit is smaller.

[0062] To this end, the encoder 110 according to an embodiment sets a maximum coding unit of a different size for each picture or slice, or sets a block shape and a partition type of one or more coding units to be partitioned from the maximum coding unit. The size of the coding unit included in the maximum coding unit can be differently set according to the block shape and the partition type.

[0063] The block shape and the partition type of the one or more coding units can be determined based on a calculated rate-distortion cost. The block shape and the partition type can be differently determined for each picture or slice, or can be differently determined for each maximum coding unit. The determined block shape and partition type are used to encode the image data in coding units.

[0064] In one embodiment, the coding units partitioned from the maximum coding unit can be characterized according to the block shape and the partition type. Hereinafter, a method of determining the coding units according to the block shape and the partition type will be described in more detail. Figures 10 to 23 A method of determining the coding units according to the block shape and the partition type will be described in more detail.

[0065] In one embodiment, the coding units included in the maximum coding unit can be predicted or transformed (e.g., by converting values of a pixel domain to values of a frequency domain) based on processing units of different sizes. In other words, the image encoding apparatus 100 can perform a plurality of processing operations to encode an image based on processing units of various sizes and shapes. To encode the image data, processing operations such as prediction, transformation, and entropy encoding are performed. The same size of processing unit can be used in all operations, or different sizes of processing unit can be used in each operation.

[0066] In one embodiment, the prediction mode of the coding unit can include at least one of an intra mode, an inter mode, or a skip mode, and a specific prediction mode can be performed only for a coding unit of a specific size or type. In one embodiment, the prediction mode having the smallest encoding error can be selected by performing prediction for each coding unit.

[0067] In intra prediction, the encoder 110 can obtain the prediction samples for the current block based on the pixels of the neighboring blocks in the current picture to which the current block belongs. In this case, the encoder 110 can (i) obtain the prediction samples based on the average of the values of the neighboring reference samples of the current block, or based on the result of performing interpolation on the neighboring reference samples, or (ii) obtain the prediction samples based on the reference samples that are located in a specific direction with respect to the pixel to be predicted among the neighboring blocks of the current block. The encoder 110 can determine the prediction mode to be applied to the current block based on the prediction mode applied to the neighboring blocks.

[0068] In one embodiment, the encoder 110 searches for the neighboring blocks adjacent to the current block among the neighboring blocks that are reconstructed before the current block. The encoder 110 can refer to the neighboring blocks that are reconstructed before the current block for intra prediction of the current block.

[0069] In inter prediction, the encoder 110 can obtain the prediction samples for the current block based on the samples specified by the motion vectors in the reference pictures. The encoder 110 can apply the skip mode, the merge mode, or the MVP mode to obtain the prediction samples for the current block. In the skip mode and the merge mode, the encoder 110 can use the motion information of the neighboring blocks as the motion information of the current block. In the skip mode, unlike in the merge mode, the residual between the prediction samples and the reconstructed samples is not transmitted. In the MVP mode, the difference information between the motion vector of the current block and the motion vectors of the neighboring blocks can be encoded using the motion vectors of the neighboring blocks as the motion vector predictor (MVP).

[0070] In inter prediction, the neighboring blocks include the spatial neighboring blocks included in the current picture and the temporal neighboring blocks included in the reference picture (collocated picture). The motion information includes the motion vector and the reference picture. When the motion information of the temporal neighboring blocks is used in the skip mode and the merge mode, the most significant picture included in the reference picture list can be used as the reference picture.

[0071] In one embodiment, as described above, the encoder 110 can search for the blocks reconstructed before the current block to perform inter prediction. The blocks reconstructed before the current block can be the blocks spatially adjacent to the current block or the blocks temporally adjacent to the current block. The encoder 110 can refer to the blocks reconstructed before the current block to derive the motion information of the current block.

[0072] As described above, in intra-frame prediction and inter-frame prediction, blocks reconstructed before the current block are referenced. Here, the blocks can be reconstructed sequentially according to a specific scan order. The scan order can be (but is not limited to) one of various orders (such as raster scan order, zigzag scan order, N-shaped scan order, upper right diagonal scan order, horizontal scan order, vertical scan order, etc.). In the above embodiments, the scan order is not limited to the order of reconstructed blocks, and can be for data storage, data loading, data access, etc.

[0073] As will be referred to below Figures 10 to 23 As described, the block shape of the coding unit can be square, rectangular, or any other geometric shape. Therefore, the maximum coding unit can include both square coding units and non-square coding units. In one embodiment, encoder 110 can perform data access based on a specific scan index included in each coding unit included in the maximum coding unit.

[0074] In one embodiment, when the reference coding unit does not include non-square coding units, the adjacent boundaries between coding units included in the reference coding unit can be continuous. In this case, data access can be performed on each coding unit according to a specific scan order.

[0075] In one embodiment, when the reference coding unit includes at least one non-square coding unit, the boundaries between coding units included in the reference coding unit may be discontinuous. Here, the order of data access between coding units contacting discontinuous boundaries may be interrupted. Therefore, the scanning order can be adaptively controlled between coding units contacting discontinuous boundaries. For example, data access can be performed vertically on rectangular coding units according to an N-shaped scanning order or a vertical scanning order, and horizontally on rectangular coding units according to a Z-shaped scanning order or a horizontal scanning order. The following will refer to... Figure 14 Please describe the scanning order in detail.

[0076] As described above, one or more scan orders can be combined within a maximum coding unit or a reference coding unit. Thus, the order in which blocks are referred to during intra prediction and inter prediction can also be determined according to one or more scan orders. In one embodiment, the encoder 110 can search a block map to find blocks that are decoded before a current block. In one embodiment, the encoder 110 can determine, based on the block map, a neighboring block that the current block will refer to. Furthermore, the encoder 110 can update the block map with a value indicating whether each block is decoded each time the block is decoded. In one embodiment, the block map includes regions corresponding to one or more blocks included in a picture, where a region of the block map corresponding to a decoded block among the one or more blocks can be set as an available region, and a region of the block map corresponding to a non-decoded block among the one or more blocks can be set as a non-available region.

[0077] In one embodiment, the encoder 110 can transform image data of a coding unit based not only on a coding unit used to encode the image data, but also on a data unit different from the coding unit.

[0078] A coding unit can be transformed based on a data unit having a size smaller than or equal to the coding unit. Examples of the data unit used for transformation can include a data unit for an intra mode and a data unit for an inter mode. Hereinafter, a data unit based on which transformation is performed can be referred to as a transform unit.

[0079] As information used to perform encoding, not only information about a block shape and a partition type, but also prediction-related information and transform-related information are required. Thus, the encoder 110 can determine a block shape and a partition type causing a minimum encoding error, a prediction mode for each coding unit, a size of a transform unit used for transformation, and the like.

[0080] The encoder 110 can measure an encoding error of a coding unit by using rate-distortion optimization based on a Lagrange multiplier.

[0081] The transmitter 120 outputs and transmits, in a bitstream, image data of a maximum coding unit encoded based on at least one coding unit determined by the encoder 110 and information about an encoding mode for each coding unit to a decoding apparatus.

[0082] The encoded image data can be a result of encoding residual data of an image.

[0083] The information about the encoding mode for each coding unit can include a block shape, a partition type, a prediction mode for each coding unit, size information of a transform unit, and the like.

[0084] Figure 2is a schematic block diagram of an image decoding apparatus 200 according to an embodiment.

[0085] In one embodiment, the image decoding apparatus 200 includes a receiver 210 and a decoder 220. In one embodiment, the definitions of various terms (such as a coding unit, block shape information, partition type information, information on a transform unit, and information on a prediction mode) used for various processes of the image decoding apparatus 200 are the same as those described above with reference to the image encoding apparatus 100. Figure 1

[0086] The receiver 210 receives a bitstream of an encoded image and parses the bitstream. The receiver 210 extracts image data from the parsed bitstream in units of a largest coding unit and outputs the image data to the decoder 220. The receiver 210 can extract information on a current picture or slice from a parameter set raw byte sequence payload (RBSP) for the current picture or slice.

[0087] In addition, the receiver 210 extracts block shape information and partition type information on a coding unit partitioned from each largest coding unit from the parsed bitstream. The extracted block shape information and partition type information are output to the decoder 220. That is, image data of the bitstream can be partitioned into largest coding units so that the decoder 220 can decode the image data in units of the largest coding units.

[0088] In addition, the receiver 210 can extract information on an encoding mode for each coding unit from the parsed bitstream. The information on the encoding mode can be set for one or more coding units, and can include information on a prediction mode for each coding unit and size information of a transform unit.

[0089] The information on the encoding mode extracted by the receiver 210 is information on an encoding mode in which a minimum encoding error is caused when encoding of a coding unit is repeatedly performed at an encoding side (such as the image encoding apparatus 100 according to an embodiment). Accordingly, the image decoding apparatus 200 can reconstruct an image by decoding data in an encoding method that causes a minimum encoding error.

[0090] The receiver 210 can extract information on an encoding mode for each minimum coding unit. When the information on the encoding mode for each minimum coding unit is recorded, a minimum coding unit having information on the same encoding mode can be inferred as a data unit included in the same largest coding unit. That is, when minimum coding units having the same information are collected and decoded, decoding can be performed based on a coding unit having a minimum encoding error.

[0091] ​The decoder 220 can reconstruct a current picture by decoding image data of each coding unit based on information about a coding mode of each coding unit. The decoding process can include an inverse quantization process, an inverse transform process, and a prediction process including intra prediction and motion compensation.

[0092] In detail, the decoder 220 can rearrange an entropy-decoded bitstream based on a method by which the image encoding apparatus 100 rearranges the entropy-decoded bitstream. The decoder 220 can rearrange coefficients represented in the form of a one-dimensional (1D) vector by reconstructing the coefficients into two-dimensional (2D) block-type coefficients. The decoder 220 can rearrange coefficients by receiving information related to a scanning of the coefficients by the image encoding apparatus 100 and inversely scanning the coefficients based on an order in which the image encoding apparatus 100 scans the coefficients. In addition, the decoder 220 can perform inverse quantization based on a quantization parameter provided from the image encoding apparatus 100 and data of coefficients of a rearranged block. The decoder 220 can perform a discrete cosine inverse transform (inverse DCT) or a discrete sine inverse transform (inverse DST) on a discrete cosine transform (DCT) or a discrete sine transform (DST) performed on a result of quantization by the image encoding apparatus 100. The inverse transform can be performed based on a transform unit or a unit resulting from image partitioning, which is determined by the image encoding apparatus 100.

[0093] The decoder 220 can create prediction data of a coding unit based on prediction-related information provided from the receiver 210 and information about previously decoded blocks and / or pictures. The prediction data of the coding unit and residual data can be used to create reconstructed data. The prediction method performed by the decoder 220 is the same as the prediction method performed by the encoder 110 of the image encoding apparatus 100.

[0094] In detail, when a prediction mode of a current block is an intra prediction mode, the decoder 220 can create a prediction block by performing intra prediction based on information of pixels in a current picture. Here, the current block can be a current coding unit.

[0095] When the prediction mode of the current block is an inter prediction mode, the decoder 220 can perform inter prediction on the current block based on information included in a reference picture by using at least one of a previous picture or a subsequent picture of the current picture as the reference picture. In detail, in the inter prediction, a prediction block of the current block can be created by selecting a reference picture of the current block and selecting a reference block having the same size as the current block. For example, in the inter prediction, the prediction block can be created such that a residual signal for the current block is minimized and a magnitude of a motion vector is minimized. In this case, information of a neighboring block of the current picture can be used to use information of the reference picture. For example, the prediction block of the current block can be created based on information of the neighboring block by a skip mode, a merge mode, an MVP mode, etc.

[0096] Similar to the image encoding apparatus 100 according to the embodiment, the decoder 220 can search for blocks decoded before the current block using the block mapping and determine the neighboring block that can be used for reference.

[0097] In one embodiment, the image decoding apparatus 200 can obtain information about a coding unit that minimizes coding error when each of the coding units is recursively coded during the encoding process, and use the information to decode the current picture. That is, the image data can be decoded based on the optimal coding unit in units of the coding unit.

[0098] Accordingly, based on the information about the optimal coding mode transmitted from the encoding side, the image data of the image is effectively decoded by the coding unit size and the coding mode adaptively determined based on characteristics of the image, and an image having a high resolution or a large amount of data can be reconstructed.

[0099] Figure 3 is a diagram illustrating a method of determining a region that can be used for reference according to an embodiment.

[0100] In one embodiment, the blocks 310, 320, and 330 can be scanned in a certain scan order to store, load, or access data. In one embodiment, the blocks 310, 320, and 330 can be coding units or transform units included in the maximum coding unit 300. In one embodiment, the blocks 310, 320, and 330 can have a square shape or a non-square shape.

[0101] In one embodiment, the image decoding apparatus 200 (or the image encoding apparatus 100) can refer to a neighboring block decoded before the current block 330 to process data of the current block 330. For example, the processing of the data can include intra prediction, inter prediction, or various techniques for referring to a neighboring region. For example, when the neighboring region referred to by the current block 330 is the block (region) assigned with index 26 included in the block 320, it can be determined whether the block with index 26 will be referred to by the current block 330 according to the order of scanning the blocks 310, 320, and 330. When the order of scanning the blocks 310, 320, and 330 is the zigzag order 302, the block 320 with index 26 is decoded before processing data of the current block 330, and thus can be referred to by the current block 330. However, when the order of scanning the blocks 310, 320, and 330 is the N-zigzag scanning order 304, the block 320 with index 26 is decoded after processing data of the current block 330, and thus cannot be referred to by the current block 330. That is, in order to refer to a neighboring block of the current block 330, it should be determined whether the neighboring block is decoded before the current block 330. However, when the blocks 310, 320, and 330 have a non-square shape, the order of scanning the blocks 310, 320, and 330 can be adaptively adjusted or changed. Accordingly, the block mapping can be used to accurately determine the order in which the blocks 310, 320, and 330 are decoded, and to search for a neighboring block decoded before the current block 330.

[0102] Figure 4 is a diagram illustrating a block mapping according to an embodiment.

[0103] In one embodiment, the block mappings 410 and 420 can be set for a current maximum coding unit. A maximum coding unit decoded before the current maximum coding unit can be used as a reference for the current maximum coding unit, and thus the block mapping of the current maximum coding unit can not be needed to be stored any more.

[0104] In one embodiment, the block mappings 410 and 420 can include regions corresponding to one or more blocks. In one embodiment, the block mapping 410 can include regions 411, 412, 413, 414, and 415 corresponding to one or more coding units included in a maximum coding unit. In one embodiment, the block mapping 420 can include regions 421, 422, 423, 424, 425, 426, 427, and 428 corresponding to one or more transform units included in a maximum coding unit.

[0105] In one embodiment, in the block mapping 410, the regions 411, 412, and 413 corresponding to decoded coding units among the one or more coding units can be set as available regions, and the region 415 corresponding to an undecoded coding unit among the one or more coding units can be set as an unavailable region.

[0106] In one embodiment, the block map 410 can be updated when each of the coding units is decoded. For example, each time a coding unit is decoded in a particular scan order, the regions 411, 412, and 413 in the block map 410 corresponding to the decoded coding units can be updated in turn with the available region value (e.g., the value "TRUE"), and the remaining regions 415 can remain with the unavailable region value (e.g., the value "FALSE"). The available regions 411, 412, and 413 in the block map 410 can be used as a reference for the current coding unit 414.

[0107] In one embodiment, in the block map 420, the regions 421, 422, 423, 424, 425, 426 corresponding to the decoded transform units of the one or more transform units can be set as available regions, and the region 428 corresponding to the non-decoded transform unit of the one or more transform units can be set as an unavailable region.

[0108] In one embodiment, the block map 420 can be updated when each of the transform units is decoded. For example, each time a transform unit is decoded in a particular scan order, the regions 421, 422, 423, 424, 425, 426 in the block map 420 corresponding to the decoded transform units can be updated in turn with the available region value (e.g., the value "TRUE"), and the remaining region 428 can remain with the unavailable region value (e.g., the value "FALSE"). The available regions 421, 422, 423, 424, 425, 426 of the block map 420 can be used as a reference for the current transform unit 427.

[0109] In one embodiment, it can be determined whether the value of each of the regions of the block maps 410 and 420 is an available region value or an unavailable region value to determine whether the region will be referenced by the current coding unit 414 or the current transform unit 427.

[0110] In one embodiment, in the block maps 410 and 420, not only the regions that are spatially adjacent to the current block, but also the regions that are temporally adjacent to the current block can be determined as regions that can be used for reference.

[0111] In one embodiment, similar to the shape of the coding units described above, the regions in the block maps 410 and 420 corresponding to the one or more blocks can have a square shape or a non-square shape.

[0112] In one embodiment, the block maps 410 and 420 can be created for each maximum coding unit, and each time each coding unit or each transform unit included in the maximum coding unit is decoded, the previously created block map can be updated.

[0113] In one embodiment, the block maps 410 and 420 can be created for the largest coding unit, and thus, the availability of the area other than the current largest coding unit can be determined by determining the order of processing the largest coding units. That is, the area belonging to the largest coding unit decoded before the current largest coding unit can be determined as an available area, and the area belonging to the largest coding unit not decoded can be determined as an unavailable area.

[0114] Various embodiments of determining the availability of the neighboring block to be referred to by the current block or adaptively changing the neighboring block based on the block maps 410 and 420 will be described below.

[0115] In one embodiment, the image decoding apparatus 200 (or the image encoding apparatus 100) can perform intra prediction by using the neighboring blocks of the picture neighboring the current block to predict the current block. In one embodiment, the current block can be predicted by adding a weight value to one or more pixel values of one or more neighboring blocks to calculate a predicted pixel value of each pixel of the current block.

[0116] Here, the neighboring block is a block reconstructed before the current block, and its availability can be determined using the block map described above. In one embodiment, the neighboring block for intra prediction can be one or more neighboring blocks around the current block, and can include, for example, at least one of the blocks located at the left and upper sides of the current block.

[0117] In one embodiment, the image decoding apparatus 200 (or the image encoding apparatus 100) can derive the motion information of the current block, and perform inter prediction on the current block based on the derived motion information.

[0118] The image used for prediction of the current block is referred to as a reference picture or a reference frame. The area of the reference picture can be indicated using a reference picture index refIdx, a motion vector, etc. indicating the reference picture.

[0119] For the current picture, a reference picture list can be composed of pictures used for prediction, and a reference picture index can indicate a specific reference picture included in the reference picture list. For a P picture, one reference picture list (e.g., reference picture list 0) is needed. For a B picture, two reference picture lists (e.g., reference picture list 0 and reference picture list 1) are needed.

[0120] In detail, an I picture is a picture that can be encoded / decoded by intra prediction. A P picture is a picture that can be encoded / decoded by inter prediction or intra prediction using at least one motion vector and at least one reference picture index to predict the sample value of each block. A B picture is a picture that can be encoded / decoded by inter prediction or intra prediction using at least two motion vectors and at least two reference picture indexes to predict the sample value of each block.

[0121] For a P picture, one reference picture list is needed, and the reference picture list is referred to as reference picture list 0 (hereinafter referred to as L0).

[0122] A B picture is a picture that can be coded using one or more reference pictures (e.g., two reference pictures) through forward inter prediction, backward inter prediction, or bi-directional inter prediction. For a B picture, two reference picture lists are needed, and the two reference picture lists are referred to as reference picture list 0 (hereinafter referred to as L0) and reference picture list 1 (hereinafter referred to as L1).

[0123] Inter prediction using a reference picture selected from L0 will be referred to as L0 prediction, and inter prediction using a reference picture selected from L1 will be referred to as L1 prediction. L0 prediction can be used for forward prediction, and L1 prediction can be used for backward prediction, but the directions of L0 prediction and L1 prediction are not limited to these in the above-described embodiments. Inter prediction using two reference pictures selected from L0 and L1, respectively, is referred to as bi-directional prediction.

[0124] The characteristics of I pictures, P pictures, and B pictures can be defined in units of slices rather than pictures. For example, an I slice having the characteristics of an I picture, a P slice having the characteristics of a P picture, and a B slice having the characteristics of a B picture as a slice unit can be defined.

[0125] For example, when the slice type of the current block is B and colPic is selected from L0, or when the slice type of the current block is P, colPic can be selected from L0.

[0126] In order to prevent low delay, the two directions of prediction of the reference picture lists 0 and 1 can be limited to the forward direction.

[0127] For inter prediction, in a skip mode, information about a neighboring block can be directly used for a current block, and thus, in the skip mode, syntax information such as a residual except for information indicating a block whose motion information is to be used as motion information of the current block is not transmitted.

[0128] In a merge mode, a prediction block of a current block can be created by directly using motion information of a neighboring block. The image encoding apparatus 100 can transmit information indicating whether the merge mode is to be applied, information about a block whose motion information is to be used, residual information, etc., to the image decoding apparatus 200. The image decoding apparatus 200 can reconstruct the current block by adding the prediction block to the residual transmitted from the image encoding apparatus 100.

[0129] The method of indicating the block whose motion information will be used in the merge mode can be applied to the skip mode and the general inter prediction mode. For example, in the merge mode and the other inter prediction mode, the candidate blocks having the information to be used as the motion information of the current block can be commonly used, and the same method can be used to indicate the neighboring block whose motion information will be used.

[0130] In this case, in the skip mode and the general inter prediction mode, only when the reference picture index or the prediction direction (reference picture list) of the neighboring block matches the reference picture index or the prediction direction (reference picture list) of the current block, it can be determined that there is available motion information, and thus the motion vector of the neighboring block can be used as the motion vector predictor candidate of the current block.

[0131] Figure 5 is a diagram illustrating a method of configuring a plurality of merge candidates to derive motion information of a current block according to an embodiment.

[0132] Referring to Figure 5 , the merge candidates of the current block 500 can include a first bottom-left block 502 having an index of 45, a second top-right block 504 having an index of 66, a third bottom-left block 506 having an index of 39, a fourth top-right block 508 having an index of 23, and a fifth top-left block 510 having an index of 7. In addition, there can be a sixth temporal candidate prediction block (not shown).

[0133] The number of the merge candidates can be fixed. For example, the number of the merge candidates can be fixed to five. Among the merge candidates, the number of the spatial candidate prediction blocks can be limited to a certain value. For example, in the case where the merge candidates can include four spatial candidate prediction blocks, when there is an unavailable block among the first bottom-left block 502 having an index of 45, the second top-right block 504 having an index of 66, the third bottom-left block 506 having an index of 39, and the fourth top-right block 508 having an index of 23, the fifth top-left block 510 having an index of 7 can be used to replace the unavailable block in the merge candidate list according to the order of the priority.

[0134] In the case where the motion vector predictor (MVP) is used, when the block availability is determined in the order from the first bottom-left block 502 having an index of 45 to the third bottom-left block 506 having an index of 39, the motion vector of the first available block can be used as the candidate prediction motion vector. Next, when the block availability is determined in the order of the second top-right block 504 having an index of 66, the fourth top-right block 508 having an index of 23, and the fifth top-left block 510 having an index of 7, the motion vector of the first available block can be selected as the candidate prediction motion vector, and the motion vector of the last temporal candidate prediction block can be used as the candidate prediction motion vector.

[0135] In one embodiment, the image decoding apparatus 200 (or the image encoding apparatus 100) can determine whether a block available as a merge candidate based on a block map.

[0136] In one embodiment, motion information of the current block can be derived based on the merge candidate, and the current block can be predicted using the derived motion information. In one embodiment, the motion information can include information on a motion vector, a prediction direction, an index of a reference picture, etc.

[0137] Figure 6 A method of changing a merge candidate having the same motion information according to various embodiments is illustrated.

[0138] Referring to Figure 6 , the merge candidates of the current block 600 can include a first bottom-left block A0 610, a second top-right block B0 620, a third bottom-left block A1 630, a fourth top-right block B1 640, and a fifth top-left block B2 650. In addition, there can be a sixth temporal candidate prediction block (not shown).

[0139] In one embodiment, there can be a case where two or more merge candidates among the spatial merge candidates 610, 620, 630, 640, and 650 of the current block 600 are included in the same coding unit. In this case, the two or more merge candidates belonging to the same coding unit can have the same motion vector. Accordingly, at least one of the two or more merge candidates belonging to the same coding unit can be replaced with a new merge candidate. To replace the merge candidate, a useable merge candidate can be searched by using a block map. Accordingly, in one embodiment, a neighboring area to be a target to be referred to by the current block 600 can be selected using the block map. In one embodiment, the bottom-left block A0 610 and the bottom-left block A1 630 among the spatial merge candidates 610, 620, 630, 640, and 650 of the current block 600 can be included in the same coding unit. In this case, the bottom-left block A0 610 and the bottom-left block A1 630 belonging to the same coding unit have the same motion vector, and at least one of the bottom-left block A0 610 and the bottom-left block A1 630 belonging to the same coding unit can be replaced with a merge candidate having different motion information and located in a different position.

[0140] In one embodiment, when the lower-left block A0 610 and the lower-left block A1 630 are included in the same coding unit and have the same motion vector, the lower-left block A0 610 among the merge candidates can be replaced with the block A0' 612 adjacent to the lower-left block A0 610. In one embodiment, when the lower-left block A0 610 and the lower-left block A1 630 are included in the same coding unit and have the same motion vector, the lower-left block A1 630 among the merge candidates can be replaced with the block A1' 632 adjacent to the lower-left block A1 630. That is, at least one of the lower-left block A0 610 and the lower-left block A1 630 can be replaced with a block (e.g., the block A0' 612 or the block A1' 632) included in another coding unit adjacent to a coding unit to which the lower-left block A0 610 and the lower-left block A1 630 belonging to the same coding unit and having the same motion vector belong.

[0141] However, even when at least one of the lower-left block A0 610 or the lower-left block A1 630 is replaced with a block (e.g., the block A0' 612 or the block A1' 632) included in a coding unit different from a coding unit to which the lower-left block A0 610 and the lower-left block A1 630 belong, the motion vector of the block can be the same as that of the lower-left block A0 610 or the lower-left block A1 630. In this case, at least one of the lower-left block A0 610 or the lower-left block A1 630 can be replaced with a block (e.g., the block A1" 634) located most adjacent to a coding unit to which the lower-left block A0 610 and the lower-left block A1 630 belong and having a different motion vector.

[0142] In the above-described embodiments, the block A0' 612, the block A1' 632, or the block A1" 634 replacing the merge candidate can be a block set as an available region in the block map. That is, the merge candidates can be blocks decoded before the current block 600 is predicted.

[0143] In another embodiment, two or more merge candidates among the spatial merge candidates 610, 620, 630, 640, and 650 and the temporal merge candidates of the current block 600 not included in the same coding unit can have the same motion vector. In this case, at least one of the two or more merge candidates not included in the same coding unit and having the same motion vector can be replaced with a new merge candidate. To replace the merge candidate, an available merge candidate can be searched for by using the block map. Accordingly, in one embodiment, the block map can be used to select a neighboring region as a target to be referred to by the current block 600.

[0144] In one embodiment, among the merge candidates, the lower-left block A0 610 and the lower-left block A1' 632 belong to different coding units, but can have the same motion vector. In this case, at least one of the lower-left block A0 610 and the lower-left block A1' 632 can be replaced with a block having a different motion vector from the motion vectors of the block A0 610 and the block A1' 632 (e.g., a block A1" 634). In one embodiment, the block having the different motion vector can be selected from among the blocks of the coding units adjacent to the block A0 610 or the block A1' 632. Embodiments for replacing the merge candidates are not limited to the above-described embodiment, and such a neighboring region of the current block 600 can be changed to a merge candidate that is among the blocks set as the available region of the block map. For example, the merge candidate of the current block 600 changed to a merge candidate can be selected from among the neighboring blocks adjacent to the current block 600, can be selected from among the blocks included in the coding unit different from the merge candidate of the current block 600, or can be selected from among the blocks adjacent to the current block 600 and most similar to the merge candidate of the current block 600.

[0145] According to the above-described embodiment, the picture decoding apparatus 200 (or the picture encoding apparatus 100) can derive the motion information of the current block 600 based on the merge candidate of the current block 600 changed to a merge candidate, and predict the current block 600 by using the derived motion information.

[0146] Figure 7 A method of changing a merge candidate of a region incapable of being referred to is shown according to various embodiments.

[0147] Referring to Figure 7 , the merge candidates of the current block 700 can include a first lower-left block A0 710, a second upper-right block B0 720, a third lower-left block A1 730, a fourth upper-right block B1 740, and a fifth upper-left block B2 750. In addition, there can be a sixth temporal candidate prediction block (not shown).

[0148] In one embodiment, when there is a merge candidate corresponding to an unavailable region of the block map among the spatial merge candidates 710, 720, 730, 740, and 750 and the temporal merge candidate of the current block 700, the merge candidate corresponding to the unavailable region of the block map can be replaced with a merge candidate corresponding to an available region of the block map.

[0149] For example, among the spatial merge candidates 710, 720, 730, 740, and 750 of the current block 700, the block A0 710, the block A1 730, the block B1 740, and the block B2 750 can correspond to the available regions of the block map, and the block B0 720 can correspond to the unavailable region of the block map. In this case, among the merge candidates, the block B0 720 cannot be referenced by the current block 700, and thus can be changed to the block B0' 722 corresponding to the available region of the block map.

[0150] For example, when the block B0' 722 as a merge candidate and the block B1 740 as another merge candidate are included in the same coding unit, the block B1 740 as another merge candidate corresponding to the available region of the block map can be changed to the block B1' 742 adjacent thereto to prevent the merge candidates from overlapping. In this case, similarly, the block B1' 742 can be selected from among the blocks corresponding to the available region of the block map.

[0151] According to the above-described embodiment, the image decoding apparatus 200 (or the image encoding apparatus 100) can predict the motion information of the current block 700 based on the merge candidate changed by the current block 700, and reconstruct the motion information of the current block 700 by using the predicted motion information.

[0152] When the MVP mode is applied, the motion information of the current block with respect to the reference picture can be obtained based on the sum of the motion vector difference (MVD) between the current block and the neighboring block with respect to the reference picture and the motion vector predictor (MVP) of the neighboring block with respect to the reference picture.

[0153] In detail, the image encoding apparatus 100 can predict the motion vector of the current block by using the motion information of the neighboring block, and transmit the difference between the motion vector of the current block and the predicted motion vector together with the reference picture index indicating the reference picture. The image decoding apparatus 200 can predict the motion vector of the current block by using the motion information of the neighboring block, and derive the motion vector of the current block by using the residual received from the image encoding apparatus 100. The image decoding apparatus 200 can create a prediction block of the current block based on the derived motion vector and the reference picture index information received from the image encoding apparatus 100.

[0154] In one embodiment, when the MVP mode is applied to inter prediction, a method of creating a motion vector predictor (MVP) candidate list of a current block can be the same as the above-described method of creating a merge candidate list. Figures 5 to 7

[0155] ​In one embodiment, the image encoding apparatus 100 or the image decoding apparatus 200 can create an MVP candidate list of a current block. The MVP can represent a predicted value of a motion vector of the current block. In this case, the MVP candidate list can be created using motion vectors of available neighboring blocks adjacent to the current block and / or available blocks among blocks co-located with the current block in a reference picture.

[0156] That is, availability of blocks constituting the MVP candidate list can be determined based on the block map. The co-located blocks of a picture reconstructed before the current picture can be determined as an area available without using the block map.

[0157] The image encoding apparatus 100 can select a best MVP for the current block by applying a motion vector competition (MVC) to the MVP candidates included in the MVP candidate list. The image encoding apparatus 100 can transmit an index of the selected MVP, a motion vector difference (MVD), and a reference picture index to the image decoding apparatus 200 in a bitstream. The MVP index is an index indicating the MVP of the current block, and can indicate one of the MVP candidates included in the MVP candidate list. The reference picture index is an index indicating a reference picture of the current block, and can indicate one of the reference picture candidates included in the reference picture list. The MVD is a difference between a motion vector of the current block and the selected MVP. As the MVD becomes smaller, an amount of information to be transmitted can be reduced.

[0158] The image decoding apparatus 200 can receive the MVP index and the reference picture index from the image encoding apparatus 100. By using the received MVP index, the image decoding apparatus 200 can select the MVP for the current block among the MVP candidates included in the MVP candidate list. By using the received reference picture index, the image decoding apparatus 200 can select the reference picture for the current block among the reference picture candidates included in the reference picture list.

[0159] The image decoding apparatus 200 can derive a motion vector of the current block by using the selected MVP and the reference picture together with the received MVD. For example, the image decoding apparatus 200 can reconstruct the motion vector of the current block by adding the received MVD to the MVP.

[0160] As described above, the block map can be used to determine whether a neighboring block is an area to be referred to for configuring a merge candidate or an MVP candidate. The position of the neighboring block to be referred to can be adaptively changed based on whether the neighboring block is available.

[0161] The block map can be used to create a merge candidate or MVP candidate list, and is also applicable to various techniques of referring to neighboring regions in an encoding process and a decoding process. For example, in intra prediction, as described above, the block map can be used to determine whether a neighboring block included in a picture and neighboring the current block can be used to be referred to.

[0162] Figure 8 is a flowchart of a method of encoding an image according to an embodiment.

[0163] In operation S810, one or more blocks included in a picture are determined. The one or more blocks can be configured as one or more coding units or one or more transform units. The one or more blocks can have a square shape or a non-square shape.

[0164] In operation S820, a block map indicating decoded blocks among the one or more blocks is determined. The block map can be updated whenever each of the blocks is decoded. The block map can be created for each largest coding unit. The block map can include regions corresponding to the one or more blocks, wherein a region corresponding to a decoded block can be set as an available region, and a region corresponding to an undecoded block can be set as an unavailable region.

[0165] In operation S830, based on the block map, neighboring regions available to be referred to by a current block among the one or more blocks to be encoded in a certain scan order are determined. The scan order can be one of various scan orders such as a raster scan order, a zigzag scan order, an N zigzag scan order, a right-up diagonal scan order, a horizontal scan order, a vertical scan order, etc., but is not limited thereto.

[0166] In operation S840, the current block is encoded based on the neighboring regions available to be referred to. Encoded image data of at least one coding unit of each largest coding unit can be transmitted to a decoding side. Information on a block shape and a partition type used to partition each coding unit, information on a prediction mode of each coding unit, information on a size, a shape, and a partition type of each transform unit can be transmitted to the decoding side together with the encoded image data.

[0167] Figure 9 is a flowchart of a method of decoding an image according to an embodiment.

[0168] In operation S910, a bitstream of an encoded image is received. The bitstream can include, in addition to encoded image data, information on a block shape and a block partition type used to partition each coding unit, information on a prediction mode of each coding unit, information on a size, a shape, and a partition type of each transform unit.

[0169] In operation S920, partitioning information obtained from the bitstream is used to determine one or more blocks to be included in the frame. The partitioning information may include information about the block shape and the type of block partitioning used to partition each coding unit. The one or more blocks may have a square shape or a non-square shape.

[0170] In operation S930, a block map indicating the decoded blocks among the one or more blocks is determined. The block map can be updated whenever each block in the block is decoded. A block map can be created for each maximum coding unit. The block map may include regions corresponding to the one or more blocks, wherein regions corresponding to decoded blocks can be set to available regions, and regions corresponding to undecoded blocks can be set to unavailable regions.

[0171] In operation S940, based on the block mapping, a neighboring region that can be used as a reference for the current block that will be decoded in a specific scan order within one or more blocks is determined. The scan order can be one of various scan orders (such as raster scan order, zigzag scan order, N-shaped scan order, upper right diagonal scan order, horizontal scan order, vertical scan order, etc.), but is not limited to this.

[0172] In operation S950, the current block is decoded based on the neighboring regions that can be used for reference.

[0173] In the following text, reference will now be made to Figures 10 to 23 This describes a method for determining data units of an image according to an embodiment.

[0174] Figure 10 The illustration shows the process of determining at least one coding unit by dividing the current coding unit according to an embodiment of the image decoding apparatus 200.

[0175] According to an embodiment, the image decoding apparatus 200 can determine the shape of the coding unit by using block shape information, and determine which shape the coding unit is divided into by using partition shape information. That is, the coding unit partitioning method indicated by the partition shape information can be determined based on the block shape indicated by the block shape information used by the image decoding apparatus 200.

[0176] According to an embodiment, the image decoding device 200 can use block shape information indicating that the current encoding unit has a square shape. For example, the image decoding device 200 can determine, based on the division shape information, whether to divide a square encoding unit, to divide the square encoding unit vertically, to divide the square encoding unit horizontally, or to divide the square encoding unit into four encoding units. (Refer to...) Figure 10When the block shape information of the current coding unit 1000 indicates a square shape, the decoder 220 can not divide the coding unit 1010a having the same size as the current coding unit 1000 according to the division shape information indicating no division, or can determine the coding units 1010b, 1010c, or 1010d based on the division shape information indicating a specific division method.

[0177] Referring to Figure 10 According to an embodiment, the image decoding apparatus 200 can determine two coding units 1010b by dividing the current coding unit 1000 in a vertical direction based on the division shape information indicating division in the vertical direction. The image decoding apparatus 200 can determine two coding units 1010c by dividing the current coding unit 1000 in a horizontal direction based on the division shape information indicating division in the horizontal direction. The image decoding apparatus 200 can determine four coding units 1010d by dividing the current coding unit 1000 in the vertical and horizontal directions based on the division shape information indicating division in the vertical and horizontal directions. However, the division shape into which a square coding unit can be divided is not limited to the above-described shapes, and can include any shape that can be indicated by the division shape information. Specific division shapes into which a square coding unit is divided will now be described in detail through various embodiments.

[0178] Figure 11 A process of determining at least one coding unit as the image decoding apparatus 200 divides a coding unit having a non-square shape according to an embodiment is shown.

[0179] According to an embodiment, the image decoding apparatus 200 can use the block shape information indicating that the current coding unit has a non-square shape. The image decoding apparatus 200 can determine whether to divide the non-square current coding unit according to the division shape information, or whether to divide the non-square current coding unit via a specific method. Referring to Figure 11 When the block shape information of the current coding unit 1100 or 1150 indicates a non-square shape, the image decoding apparatus 200 can not divide the coding unit 1110 or 1160 having the same size as the current coding unit 1100 or 1150 according to the division shape information indicating no division, or determine the coding units 1120a, 1120b, 1130a, 1130b, 1130c, 1170a, 1170b, 1180a, 1180b, and 1180c based on the division shape information indicating a specific division method. Specific division methods of dividing a coding unit having a non-square shape will now be described in detail through various embodiments.

[0180] According to an embodiment, the image decoding apparatus 200 can determine a shape in which a coding unit is divided by using the division shape information, and in this case, the division shape information can indicate a number of at least one coding unit generated as the coding unit is divided. Referring to Figure 11 When the division shape information indicates that the current coding unit 1110 or 1150 is divided into two coding units, the image decoding apparatus 200 can determine two coding units 1120a and 1120b or 1170a and 1170b included in the current coding unit 1110 or 1150 by dividing the current coding unit 1110 or 1150 based on the division shape information.

[0181] According to an embodiment, when the image decoding apparatus 200 divides the current coding unit 1100 or 1150 having a non-square shape based on the division shape information, the image decoding apparatus 200 can divide the current coding unit 1100 or 1150 while considering a position of a long side of the current coding unit 1100 or 1150 having a non-square shape. For example, the image decoding apparatus 200 can determine a plurality of coding units by dividing the current coding unit 1100 or 1150 in a direction in which a long side of the current coding unit 1100 or 1150 is divided, by considering a shape of the current coding unit 1100 or 1150.

[0182] According to an embodiment, when the division shape information indicates that a coding unit is divided into an odd number of blocks, the image decoding apparatus 200 can determine an odd number of blocks included in the current coding unit 1100 or 1150. For example, when the division shape information indicates that the current coding unit 1100 or 1150 is divided into three coding units, the image decoding apparatus 200 can divide the current coding unit 1100 or 1150 into three coding units 1130a, 1130b, and 1130c, or three coding units 1180a, 1180b, and 1180c. According to an embodiment, the image decoding apparatus 200 can determine an odd number of coding units included in the current coding unit 1100 or 1150, and sizes of the determined coding units can not all be the same. For example, a size of the coding unit 1130b or 1180b among the determined odd number of coding units 1130a, 1130b, and 1130c or coding units 1180a, 1180b, and 1180c can be different from sizes of the coding units 1130a and 1130c or the coding units 1180a and 1180c. That is, the coding units that can be determined as the current coding unit 1100 or 1150 is divided can have various sizes, and in some cases, the coding units 1130a, 1130b, and 1130c or the coding units 1180a, 1180b, and 1180c can have different sizes.

[0183] According to an embodiment, when the split shape information indicates that the coding unit is split into an odd number of blocks, the image decoding apparatus 200 can determine an odd number of coding units included in the current coding unit 1100 or 1150, and in addition, the image decoding apparatus 200 can set a specific restriction on at least one coding unit among the odd number of coding units generated via the split. Referring to Figure 11 , the image decoding apparatus 200 can distinguish a decoding process performed on a center coding unit 1130b or 1180b among three coding units 1130a, 1130b, and 1130c or three coding units 1180a, 1180b, and 1180c generated as the current coding unit 1100 or 1150 is split from other coding units 1130a and 1130c or other coding units 1180a or 1180c. For example, the image decoding apparatus 200 can restrict the center coding unit 1130b or 1180b from being split again like the other coding units 1130a and 1130c or the other coding units 1180a or 1180c, or can be split only a certain number of times.

[0184] Figure 12 A process in which the image decoding apparatus 200 splits a coding unit based on at least one of block shape information and split shape information according to an embodiment is illustrated.

[0185] According to an embodiment, the image decoding apparatus 200 can determine, based on at least one of the block shape information and the split shape information, that a first coding unit 1200 having a square shape is to be split or not split into coding units. According to an embodiment, when the split shape information indicates that the first coding unit 1200 is split in a horizontal direction, the image decoding apparatus 200 can determine a second coding unit 1210 by splitting the first coding unit 1200 in the horizontal direction. The first coding unit, the second coding unit, and the third coding unit used according to an embodiment are terms used to indicate a relationship between before and after a coding unit is split. For example, the second coding unit can be determined by splitting the first coding unit, and the third coding unit can be determined by splitting the second coding unit. Hereinafter, it will be understood that the relationship between the first coding unit to the third coding unit depends on the above-described features.

[0186] According to an embodiment, the image decoding apparatus 200 can determine, based on at least one of the block shape information and the split shape information, that the determined second coding unit 1210 is to be split or not split into coding units. Referring to Figure 12, the image decoding apparatus 200 can divide the second coding unit 1210 having a non-square shape determined by dividing the first coding unit 1200 into at least one third coding unit 1220a, 1220b, 1220c, or 1220d based on at least one of the block shape information and the division shape information, or can not divide the second coding unit 1210. The image decoding apparatus 200 can obtain at least one of the block shape information and the division shape information, and obtain a plurality of second coding units (e.g., the second coding unit 1210) having various shapes by dividing the first coding unit 1200 based on at least one of the obtained block shape information and division shape information, wherein the second coding unit 1210 can be divided according to a method of dividing the first coding unit 1200 based on at least one of the block shape information and the division shape information.

[0187] According to an embodiment, when the first coding unit 1200 is divided into the second coding unit 1210 based on at least one of the block shape information and the division shape information for the first coding unit 1200, the second coding unit 1210 can also be divided into a third coding unit (e.g., the third coding units 1220a to 1220d) based on at least one of the block shape information and the division shape information for the second coding unit 1210. That is, a coding unit can be recursively divided based on at least one of the division shape information and the block shape information related to each coding unit. Accordingly, a square coding unit can be determined from a non-square coding unit, and such a square coding unit can be recursively divided so that a non-square coding unit is determined. Referring to Figure 12 When the second coding unit 1210 having a non-square shape is divided, a particular coding unit (e.g., a coding unit located at the center or a square coding unit) among the odd number of third coding units 1220b, 1220c, and 1220d determined can be recursively divided. According to an embodiment, the third coding unit 1220c having a square shape among the third coding units 1220b to 1220d can be divided into a plurality of fourth coding units in a horizontal direction. A fourth coding unit 1240 having a non-square shape among the plurality of fourth coding units can be again divided into a plurality of coding units. For example, the fourth coding unit 1240 having a non-square shape can be divided into odd number of coding units 1250a, 1250b, and 1250c.

[0188] Methods that can be used to recursively divide coding units will be described below through various embodiments.

[0189] According to an embodiment, the image decoding apparatus 200 can determine that each of the third coding units 1220a, 1220b, 1220c, and 1220d is to be divided into coding units, or determine that the second coding unit 1210 is not to be divided, based on at least one of the block shape information and the division shape information. According to an embodiment, the image decoding apparatus 200 can divide the second coding unit 1210 having a non-square shape into an odd number of third coding units 1220b, 1220c, and 1220d. The image decoding apparatus 200 can set a specific restriction on a specific third coding unit among the third coding units 1220b, 1220c, and 1220d. For example, the image decoding apparatus 200 can restrict the third coding unit 1220c located at the center among the third coding units 1220b, 1220c, and 1220d in the second coding unit 1210 having a non-square shape not to be divided any more, or set a number of times of division. Referring to Figure 12 , the image decoding apparatus 200 can restrict the third coding unit 1220c located at the center among the third coding units 1220b, 1220c, and 1220d included in the second coding unit 1210 having a non-square shape not to be divided any more, be divided into a specific division shape (for example, be divided into four coding units or be divided into a shape corresponding to a shape into which the second coding unit is divided), or be divided only a specific number of times (for example, be divided only n times, where n > 0). However, such a restriction on the third coding unit 1220c located at the center is merely an example, and should not be construed as being limited by these examples, but should be construed as including various restrictions as long as the third coding unit 1220c located at the center is decoded differently from the other third coding units 1220b and 1220d.

[0190] According to an embodiment, the image decoding apparatus 200 can obtain at least one of the block shape information and the division shape information for dividing the current coding unit from a specific position in the current coding unit.

[0191] Figure 13 A method of determining a specific coding unit among an odd number of coding units by the image decoding apparatus 200 according to an embodiment is illustrated. Referring to Figure 13 At least one of the block shape information and the division shape information of the current coding unit 1300 can be obtained from a sample located at a specific position (for example, a sample 1340 located at the center) among a plurality of samples included in the current coding unit 1300. However, the specific position in the current coding unit 1300 at which at least one of the block shape information and the division shape information is obtained is not limited to Figure 13The center position illustrated in FIG. 13A is not necessarily a center position of the current coding unit 1300, but can be an arbitrary position (e.g., an uppermost position, a lowermost position, a left side position, a right side position, a left upper position, a left lower position, a right upper position, or a right lower position) included in the current coding unit 1300. The image decoding apparatus 200 can determine that the current coding unit is divided into coding units having various shapes and sizes, or is not divided, by obtaining at least one of block shape information and division shape information from a specific position.

[0192] According to an embodiment, the image decoding apparatus 200 can select one coding unit when the current coding unit is divided into a specific number of coding units. The method of selecting one of a plurality of coding units can be different, and details thereof will be described below through various embodiments.

[0193] According to an embodiment, the image decoding apparatus 200 can divide the current coding unit into a plurality of coding units, and can determine a coding unit located at a specific position.

[0194] Figure 13 A method of determining, by the image decoding apparatus 200, a coding unit located at a specific position among an odd number of coding units according to an embodiment is illustrated.

[0195] According to an embodiment, the image decoding apparatus 200 can use information indicating a position of each of the odd number of coding units to determine a coding unit located at a center among the odd number of coding units. Referring to Figure 13 , the image decoding apparatus 200 can determine the odd number of coding units 1320a, 1320b, and 1320c by dividing the current coding unit 1300. The image decoding apparatus 200 can determine the center coding unit 1320b by using information about positions of the odd number of coding units 1320a, 1320b, and 1320c. For example, the image decoding apparatus 200 can determine the center coding unit 1320b by determining positions of the coding units 1320a, 1320b, and 1320c based on information indicating positions of specific samples included in the coding units 1320a, 1320b, and 1320c. In detail, the image decoding apparatus 200 can determine the center coding unit 1320b by determining positions of the coding units 1320a, 1320b, and 1320c based on information indicating positions of top-left samples 1330a to 1330c in the coding units 1320a, 1320b, and 1320c.

[0196] According to an embodiment, the information indicating the positions of the top-left samples 1330a, 1330b, and 1330c included in the coding units 1320a, 1320b, and 1320c, respectively, can include information on positions or coordinates of the coding units 1320a, 1320b, and 1320c in a picture. According to an embodiment, the information indicating the positions of the top-left samples 1330a, 1330b, and 1330c included in the coding units 1320a, 1320b, and 1320c, respectively, can include information indicating widths or heights of the coding units 1320a, 1320b, and 1320c included in the current coding unit 1300, and such widths or heights can correspond to information indicating differences between coordinates of the coding units 1320a, 1320b, and 1320c in the picture. That is, the image decoding apparatus 200 can determine the center coding unit 1320b by directly using information on positions or coordinates of the coding units 1320a, 1320b, and 1320c in the picture or by using information on widths or heights of the coding units corresponding to differences between coordinates.

[0197] According to an embodiment, the information indicating the position of the top-left sample 1330a of the upper coding unit 1320a can indicate (xa, ya) coordinates, the information indicating the position of the top-left sample 1330b of the center coding unit 1320b can indicate (xb, yb) coordinates, and the information indicating the position of the top-left sample 1330c of the lower coding unit 1320c can indicate (xc, yc) coordinates. The image decoding apparatus 200 can determine the center coding unit 1320b by using the coordinates of the top-left samples 1330a, 1330b, and 1330c included in the coding units 1320a, 1320b, and 1320c, respectively. For example, when the coordinates of the top-left samples 1330a, 1330b, and 1330c are arranged in ascending order or descending order, among the coding units 1320a, 1320b, and 1320c determined when the current coding unit 1300 is divided, the coding unit 1320b including the coordinates (xb, yb) of the center sample 1330b can be determined as the center coding unit. However, the coordinates indicating the positions of the top-left samples 1330a to 1330c can be coordinates indicating absolute positions in a picture, and furthermore, (dxb, dyb) coordinates, i.e., information indicating a relative position of the top-left sample 1330b of the center coding unit 1320b, and (dxc, dyc) coordinates, i.e., information indicating a relative position of the top-left sample 1330c of the lower coding unit 1320c, can be used based on the position of the top-left sample 1330a of the upper coding unit 1320a. Furthermore, a method of determining a coding unit located at a specific position by using coordinates of samples as information indicating positions of samples included in a coding unit is not limited to the above method, and various arithmetic methods capable of using coordinates of samples can be used.

[0198] According to an embodiment, the image decoding apparatus 200 can divide the current coding unit 1300 into a plurality of coding units 1320a, 1320b, and 1320c, and select a coding unit from among the coding units 1320a to 1320c according to a certain criterion. For example, the image decoding apparatus 200 can select the coding unit 1320b having a different size among the coding units 1320a, 1320b, and 1320c.

[0199] According to an embodiment, the image decoding apparatus 200 can determine the width or height of the coding units 1320a, 1320b, and 1320c by respectively using (xa, ya) coordinates (i.e., information indicating the position of the top-left sample 1330a of the upper coding unit 1320a), (xb, yb) coordinates (i.e., information indicating the position of the top-left sample 1330b of the center coding unit 1320b), and (xc, yc) coordinates (i.e., information indicating the position of the top-left sample 1330c of the lower coding unit 1320c). The image decoding apparatus 200 can determine the size of the coding units 1320a, 1320b, and 1320c by respectively using the coordinates (xa, ya), (xb, yb), and (xc, yc) indicating the positions of the coding units 1320a, 1320b, and 1320c.

[0200] According to an embodiment, the image decoding apparatus 200 can determine the width of the upper coding unit 1320a as xb-xa and the height as yb-ya. According to an embodiment, the image decoding apparatus 200 can determine the width of the center coding unit 1320b as xc-xb and the height as yc-yb. According to an embodiment, the image decoding apparatus 200 can determine the width or height of the lower coding unit by using the width and height of the current coding unit and the width and height of the upper coding unit 1320a and the center coding unit 1320b. The image decoding apparatus 200 can determine a coding unit having a different size from other coding units based on the determined width and height of the coding units 1320a, 1320b, and 1320c. Referring to Figure 13 , the image decoding apparatus 200 can determine the center coding unit 1320b having a different size from the sizes of the upper coding unit 1320a and the lower coding unit 1320c as a coding unit located at a certain position. However, the process in which the image decoding apparatus 200 determines a coding unit having a different size from other coding units is merely an example of determining a coding unit located at a certain position by using the sizes of the coding units determined based on sample coordinates, and thus various processes of determining a coding unit located at a certain position by comparing the sizes of the coding units determined according to certain sample coordinates can be used.

[0201] However, the position of the sample considered for determining the position of the coding unit is not limited to the above-described top-left, and information on the position of any sample included in the coding unit can be used.

[0202] According to an embodiment, the image decoding apparatus 200 can select a coding unit located at a specific position among the odd number of coding units determined when the current coding unit is partitioned, while considering the shape of the current coding unit. For example, when the current coding unit has a non-square shape with a longer width than height, the image decoding apparatus 200 can determine a coding unit located at a specific position in a horizontal direction. In other words, the image decoding apparatus 200 can determine one of the coding units having different positions in the horizontal direction, and set a restriction on the one coding unit. When the current coding unit has a non-square shape with a longer height than width, the image decoding apparatus 200 can determine a coding unit located at a specific position in a vertical direction. That is, the image decoding apparatus 200 can determine one of the coding units having different positions in the vertical direction, and set a restriction on the one coding unit.

[0203] According to an embodiment, the image decoding apparatus 200 can use information indicating the position of each of the even number of coding units in order to determine a coding unit located at a specific position among the even number of coding units. The image decoding apparatus 200 can determine the even number of coding units by partitioning the current coding unit, and determine a coding unit located at a specific position by using information on the positions of the even number of coding units. The detailed process thereof can correspond to the process of determining a coding unit located at a specific position (e.g., a center position) among the odd number of coding units described in Figure 13

[0204] According to an embodiment, when the current coding unit having a non-square shape is partitioned into a plurality of coding units, specific information on a coding unit located at a specific position during the partitioning process can be used to determine a coding unit located at a specific position among the plurality of coding units. For example, the image decoding apparatus 200 can use at least one of block shape information and partition shape information stored in a sample included in a center coding unit during the partitioning process in order to determine a coding unit located at the center among the plurality of coding units obtained by partitioning the current coding unit.

[0205] Referring to Figure 13 ​, the image decoding apparatus 200 can determine the coding unit 1320b located at the center among the plurality of coding units 1320a, 1320b, and 1320c determined when the current coding unit 1300 is divided based on at least one of the block shape information and the division shape information. Also, the image decoding apparatus 200 can determine the coding unit 1320b located at the center in consideration of a position at which at least one of the block shape information and the division shape information is obtained. That is, at least one of the block shape information and the division shape information of the current coding unit 1300 can be obtained from a sample 1340 located at the center of the current coding unit 1300, and the coding unit 1320b including the sample 1340 can be determined as the coding unit located at the center when the current coding unit 1300 is divided into the plurality of coding units 1320a, 1320b, and 1320c based on at least one of the block shape information and the division shape information. However, information used to determine the coding unit located at the center is not limited to at least one of the block shape information and the division shape information, and various types of information can be used in determining the coding unit located at the center.

[0206] According to an embodiment, specific information used to identify a coding unit located at a specific position can be obtained from a specific sample included in the coding unit to be determined. Referring to Figure 13 , the image decoding apparatus 200 can use at least one of the block shape information and the division shape information obtained from a sample located at a specific position in the current coding unit 1300 (e.g., a sample located at the center of the current coding unit 1300) in order to determine a coding unit located at a specific position (e.g., a coding unit located at the center among the plurality of coding units) among the plurality of coding units 1320a, 1320b, and 1320c determined when the current coding unit 1300 is divided. That is, the image decoding apparatus 200 can determine a sample located at a specific position in consideration of a block shape of the current coding unit 1300, and determine a coding unit 1320b including the sample from which specific information (e.g., at least one of the block shape information and the division shape information) can be obtained among the plurality of coding units 1320a, 1320b, and 1320c determined when the current coding unit 1300 is divided, and set a specific restriction on the coding unit 1320b. Referring to Figure 13 , according to an embodiment, the image decoding apparatus 200 can determine a sample 1340 located at the center of the current coding unit 1300 as a sample from which specific information can be obtained, and set a specific restriction on a coding unit 1320b including such a sample 1340 during a decoding process. However, a position of a sample from which specific information can be obtained is not limited to the above position, and can be a sample located at any position included in a coding unit 1320b determined to set a restriction.

[0207] According to an embodiment, the position of the sample from which the particular information is obtainable can be determined according to the shape of the current coding unit 1300. According to an embodiment, the block shape information can determine whether the shape of the current coding unit is a square or a non-square, and the position of the sample from which the particular information is obtainable can be determined according to the shape. For example, the image decoding apparatus 200 can determine a sample located at a boundary that bisects at least one of the width and the height of the current coding unit by using at least one of information about the width of the current coding unit and information about the height of the current coding unit as the sample from which the particular information is obtainable. As another example, when the block shape information related to the current coding unit indicates a square shape, the image decoding apparatus 200 can determine one of the samples adjacent to a boundary that bisects the long side of the current coding unit as the sample from which the particular information is obtainable.

[0208] According to an embodiment, when the current coding unit is divided into a plurality of coding units, the image decoding apparatus 200 can use at least one of the block shape information and the division shape information in order to determine a coding unit located at a particular position among the plurality of coding units. According to an embodiment, the image decoding apparatus 200 can obtain at least one of the block shape information and the division shape information from a sample located at a particular position included in a coding unit, and can divide the plurality of coding units generated as the current coding unit is divided by using at least one of the block shape information and the division shape information obtained from the sample located at a particular position included in each of the plurality of coding units. That is, the coding units can be recursively divided by using at least one of the block shape information and the division shape information obtained from the sample located at a particular position included in each of the coding units. Since the process of recursively dividing the coding units has been described above with reference to FIGS. 1 to 6, details thereof will not be provided. Figure 12 Since the process of recursively dividing the coding units has been described above with reference to FIGS. 1 to 6, details thereof will not be provided.

[0209] According to an embodiment, the image decoding apparatus 200 can determine at least one coding unit by dividing the current coding unit, and determine an order in which the at least one coding unit is decoded according to a particular block (e.g., the current coding unit).

[0210] Figure 14 An order in which the plurality of coding units is processed when the plurality of coding units is determined as the current coding unit is divided in the image decoding apparatus 200 is shown according to an embodiment.

[0211] According to an embodiment, the image decoding apparatus 200 can determine the second coding units 1410a and 1410b by dividing the first coding unit 1400 in the vertical direction, determine the second coding units 1430a and 1430b by dividing the first coding unit 1410 in the horizontal direction, or determine the second coding units 1450a, 1450b, 1450c, and 1450d by dividing the first coding unit 1400 in the horizontal and vertical directions according to the block shape information and the division shape information.

[0212] Referring to Figure 14 , the image decoding apparatus 200 can determine the second coding units 1410a and 1410b determined by dividing the first coding unit 1400 in the vertical direction to be processed in the horizontal direction 1410c. The image decoding apparatus 200 can determine the second coding units 1430a and 1430b determined by dividing the first coding unit 1400 in the horizontal direction to be processed in the vertical direction 1430c. The image decoding apparatus 200 can determine the second coding units 1450a, 1450b, 1450c, and 1450d determined by dividing the first coding unit 1400 in the vertical and horizontal directions to be processed according to a certain order in which coding units located in one row are processed and then coding units located in a next row are processed (e.g., a raster scan order or a zigzag order 1450e).

[0213] According to an embodiment, the image decoding apparatus 200 can recursively divide coding units. Referring to Figure 14, the image decoding apparatus 200 can determine a plurality of second coding units 1410a and 1410b, a plurality of second coding units 1430a and 1430b, or a plurality of second coding units 1450a, 1450b, 1450c, and 1450d by dividing the first coding unit 1400, and recursively divide each of the plurality of second coding units 1410a and 1410b, each of the plurality of second coding units 1430a and 1430b, or each of the plurality of second coding units 1450a, 1450b, 1450c, and 1450d. The method of dividing the plurality of second coding units 1410a and 1410b, the plurality of second coding units 1430a and 1430b, or the plurality of second coding units 1450a, 1450b, 1450c, and 1450d can correspond to the method of dividing the first coding unit 1400. Accordingly, each of the plurality of second coding units 1410a and 1410b, each of the plurality of second coding units 1430a and 1430b, or each of the plurality of second coding units 1450a, 1450b, 1450c, and 1450d can be independently divided into a plurality of coding units. For example, the image decoding apparatus 200 can determine a second coding unit 1410a and 1410b by dividing the first coding unit 1400 in the vertical direction, and further determine that each of the second coding units 1410a and 1410b is independently divided or not divided. Figure 14 , the image decoding apparatus 200 can determine a plurality of second coding units 1410a and 1410b, a plurality of second coding units 1430a and 1430b, or a plurality of second coding units 1450a, 1450b, 1450c, and 1450d by dividing the first coding unit 1400, and recursively divide each of the plurality of second coding units 1410a and 1410b, each of the plurality of second coding units 1430a and 1430b, or each of the plurality of second coding units 1450a, 1450b, 1450c, and 1450d. The method of dividing the plurality of second coding units 1410a and 1410b, the plurality of second coding units 1430a and 1430b, or the plurality of second coding units 1450a, 1450b, 1450c, and 1450d can correspond to the method of dividing the first coding unit 1400. Accordingly, each of the plurality of second coding units 1410a and 1410b, each of the plurality of second coding units 1430a and 1430b, or each of the plurality of second coding units 1450a, 1450b, 1450c, and 1450d can be independently divided into a plurality of coding units. For example, the image decoding apparatus 200 can determine a second coding unit 1410a and 1410b by dividing the first coding unit 1400 in the vertical direction, and further determine that each of the second coding units 1410a and 1410b is independently divided or not divided.

[0214] According to an embodiment, the image decoding apparatus 200 can divide the second coding unit 1410a located at the left side into third coding units 1420a and 1420b in the horizontal direction, and can not divide the second coding unit 1410b located at the right side.

[0215] According to an embodiment, the order in which the coding units are processed can be determined based on the partitioning of the coding units. In other words, the order in which the partitioned coding units are processed can be determined based on the order in which the coding units are processed before being partitioned. The image decoding apparatus 200 can determine the order in which the third coding units 1420a and 1420b are processed, which are determined when the second coding unit 1410a located on the left is partitioned independently of the second coding unit 1410b located on the right. Since the third coding units 1420a and 1420b are determined when the second coding unit 1410a located on the left is partitioned in the horizontal direction, the third coding units 1420a and 1420b can be processed in the vertical direction 1420c. Also, since the order in which the second coding unit 1410a located on the left and the second coding unit 1410b located on the right are processed corresponds to the horizontal direction 1410c, the second coding unit 1410b located on the right can be processed after the third coding units 1420a and 1420b included in the second coding unit 1410a located on the left are processed in the vertical direction 1420c. The above description is related to a process of determining the order in which the coding units are processed according to the coding units before the coding units are partitioned, but such a process is not limited to the above embodiment, and any method of independently processing coding units partitioned in various shapes in a specific order can be used.

[0216] Figure 15 A process of determining to partition a current coding unit into an odd number of coding units when the coding unit cannot be processed by the image decoding apparatus 200 in a specific order according to an embodiment is illustrated.

[0217] According to an embodiment, the image decoding apparatus 200 can determine that the current coding unit is partitioned into an odd number of coding units based on the obtained block shape information and the partition shape information. Referring to Figure 15 , the first coding unit 1500 having a square shape can be partitioned into the second coding units 1510a and 1510b having non-square shapes, and the second coding units 1510a and 1510b can be recursively partitioned into the third coding units 1520a and 1520b and the third coding units 1520c, 1520d, and 1520e. According to an embodiment, the image decoding apparatus 200 can partition the second coding unit 1510a located on the left among the second coding units 1510a and 1510b in the horizontal direction to determine a plurality of third coding units 1520a and 1520b, and partition the second coding unit 1510b located on the right into an odd number of third coding units 1520c, 1520d, and 1520e.

[0218] According to an embodiment, the image decoding apparatus 200 can determine whether there is a third coding unit 1520a, 1520b, 1520c, 1520d, and 1520e that is divided into an odd number by determining whether the third coding unit 1520a, 1520b, 1520c, 1520d, and 1520e can be processed in a certain order. Referring to Figure 15 , the image decoding apparatus 200 can determine the third coding units 1520a to 1520e by recursively dividing the first coding unit 1500. The image decoding apparatus 200 can determine whether any one of the first coding unit 1500, the second coding units 1510a and 1510b, and the third coding units 1520a and 1520b and the third coding units 1520c to 1520e is divided into an odd number of coding units based on at least one of block shape information and division shape information. For example, the second coding unit 1510b located on the right among the second coding units 1510a and 1510b can be divided into an odd number of third coding units 1520c to 1520e. The order in which the plurality of coding units included in the first coding unit 1500 are processed can be a certain order (e.g., a zigzag scan order 1530), and the image decoding apparatus 200 can determine whether the third coding units 1520c, 1520d, and 1520e determined when the second coding unit 1520b located on the right is divided into an odd number satisfy a condition that can be processed according to the certain order.

[0219] According to an embodiment, the image decoding apparatus 200 can determine whether the third coding units 1520a, 1520b, 1520c, 1520d, and 1520e included in the first coding unit 1500 satisfy a condition that can be processed according to the certain order, in which the condition relates to whether at least one of a width and a height of each of the second coding units 1510a and 1510b is bisected according to a boundary of the third coding units 1520a, 1520b, 1520c, 1520d, and 1520e. For example, the third coding units 1520a and 1520b determined when the height of the second coding unit 1510a located at the left side and having a non-square shape is bisected satisfy the condition, but it can be determined that the third coding units 1520c, 1520d, and 1520e do not satisfy the condition because the boundaries of the third coding units 1520c, 1520d, and 1520e determined when the second coding unit 1510b located at the right side is divided into three coding units do not bisect the width or the height of the second coding unit 1510b located at the right side. The image decoding apparatus 200 can determine that the scan order is not continuous when the condition is not satisfied, and determine that the second coding unit 1510b located at the right side is divided into an odd number of coding units based on the result of the determination. According to an embodiment, the image decoding apparatus 200 can set a certain restriction on a coding unit located at a certain position among the odd number of coding units obtained by dividing the coding unit, and since such a restriction or the certain position has been described above through various embodiments, details thereof are not provided any more.

[0220] Figure 16 A process of determining at least one coding unit when the image decoding apparatus 200 divides the first coding unit 1600 is shown according to an embodiment. According to an embodiment, the image decoding apparatus 200 can divide the first coding unit 1600 based on at least one of the block shape information and the division shape information obtained through the receiver 210. The first coding unit 1600 having a square shape can be divided into four coding units having a square shape or a plurality of coding units having a non-square shape. For example, referring to FIG. 16, the first coding unit 1600 having a square shape can be divided into four coding units 1610a, 1610b, 1610c, and 1610d having a square shape or a plurality of coding units 1610a, 1610b, 1610c, 1610d, 1610e, 1610f, 1610g, and 1610h having a non-square shape. Figure 16When the block shape information indicates that the first coding unit 1600 is a square and the division shape information indicates division into non-square coding units, the image decoding apparatus 200 can divide the first coding unit 1600 into a plurality of non-square coding units. In detail, when the division shape information indicates that an odd number of coding units is determined by dividing the first coding unit 1600 in a horizontal direction or a vertical direction, the image decoding apparatus 200 can determine the second coding units 1610a, 1610b, and 1610c obtained by dividing the first coding unit 1600 having a square shape in a vertical direction or the second coding units 1620a, 1620b, and 1620c obtained by dividing the first coding unit 1600 in a horizontal direction as the odd number of coding units.

[0221] According to an embodiment, the image decoding apparatus 200 can determine whether the second coding units 1610a, 1610b, and 1610c included in the first coding unit 1600 and the second coding units 1620a, 1620b, and 1620c satisfy a condition that can be processed in a specific order, in which the condition relates to whether at least one of a width and a height of the first coding unit 1600 is bisected according to boundaries of the second coding units 1610a, 1610b, and 1610c and the second coding units 1620a, 1620b, and 1620c. Referring to Figure 16 Since the boundaries of the second coding units 1610a, 1610b, and 1610c determined when the first coding unit 1600 having a square shape is divided in a vertical direction do not bisect the width of the first coding unit 1600, it can be determined that the first coding unit 1600 does not satisfy the condition that can be processed in a specific order. Also, since the boundaries of the second coding units 1620a, 1620b, and 1620c determined when the first coding unit 1600 having a square shape is divided in a horizontal direction do not bisect the height of the first coding unit 1600, it can be determined that the first coding unit 1600 does not satisfy the condition that can be processed in a specific order. The image decoding apparatus 200 can determine that the scan order is discontinuous when the condition is not satisfied, and determine that the first coding unit 1600 is divided into an odd number of coding units based on the determined result. According to an embodiment, the image decoding apparatus 200 can set a specific restriction on a coding unit located at a specific position among the odd number of coding units obtained by dividing the coding unit, and since such a restriction or specific position has been described above through various embodiments, details thereof are not provided again.

[0222] According to an embodiment, the image decoding apparatus 200 can determine coding units having various shapes by dividing a first coding unit.

[0223] Referring to Figure 16The image decoding apparatus can divide the first coding unit 1600 having a square shape and the first coding units 1630 or 1650 having a non-square shape into coding units having various shapes.

[0224] Figure 17 It is shown that, when the second coding units having a non-square shape determined when the first coding unit 1700 is divided satisfy a certain condition, the shapes into which the second coding units can be divided by the image decoding apparatus 200 are restricted according to embodiments.

[0225] According to embodiments, the image decoding apparatus 200 can determine that the first coding unit 1700 having a square shape is divided into the second coding units 1710a and 1710b or the second coding units 1720a and 1720b having a non-square shape based on at least one of the block shape information and the division shape information obtained through the receiver 210. The second coding units 1710a and 1710b or the second coding units 1720a and 1720b can be independently divided. Thus, the image decoding apparatus 200 can determine that the second coding units 1710a and 1710b or the second coding units 1720a and 1720b are divided into a plurality of coding units or not based on at least one of the block shape information and the division shape information related to each of the coding units 1710a and 1710b or each of the coding units 1720a and 1720b. According to embodiments, the image decoding apparatus 200 can determine the third coding units 1712a and 1712b by dividing the second coding unit 1710a located on the left having a non-square shape in a horizontal direction, which is determined when the first coding unit 1700 is divided in a vertical direction. However, when the second coding unit 1710a located on the left is divided in a horizontal direction, the image decoding apparatus 200 can set a restriction that the second coding unit 1710a located on the right is not divided in a horizontal direction like the second coding unit 1710b located on the left. When the third coding units 1714a and 1714b are determined by dividing the second coding unit 1710b located on the right in the same direction (i.e., a horizontal direction), the third coding units 1712a, 1712b, 1714a, and 1714b are determined when the second coding unit 1710a located on the left and the second coding unit 1710b located on the right are independently divided in a horizontal direction. However, this is the same as the result when the first coding unit 1700 is divided into four second coding units 1730a, 1730b, 1730c, and 1730d based on at least one of the block shape information and the division shape information, and thus can be inefficient in terms of image decoding.

[0226] According to an embodiment, the image decoding apparatus 200 can determine the third coding units 1722a and 1722b or the third coding units 1724a and 1724b by dividing the second coding units 1720a or 1720b having a non-square shape determined when the first coding unit 1700 is divided in the horizontal direction, in the vertical direction. However, when one of the second coding units (e.g., the second coding unit 1720a located at the top) is divided in the vertical direction, the image decoding apparatus 200 can set a restriction that the other second coding units (e.g., the second coding units located at the bottom) are not divided in the vertical direction like the second coding unit 1720a located at the top, for the reasons described above.

[0227] Figure 18 A process in which the image decoding apparatus 200 divides a coding unit having a square shape when division shape information cannot be used to indicate that the coding unit is divided into four square shapes, according to an embodiment, is illustrated.

[0228] According to an embodiment, the image decoding apparatus 200 can determine the second coding units 1810a and 1810b or the second coding units 1820a and 1820b by dividing the first coding unit 1800 based on at least one of the block shape information and the division shape information. The division shape information can include information on various shapes into which a coding unit can be divided, but such information on various shapes can not include information for dividing a coding unit into four square coding units. According to such division shape information, the image decoding apparatus 200 cannot divide the first coding unit 1800 having a square shape into four second coding units 1830a, 1830b, 1830c, and 1830d having a square shape. The image decoding apparatus 200 can determine the second coding units 1810a and 1810b or the second coding units 1820a and 1820b having a non-square shape based on the division shape information.

[0229] According to an embodiment, the image decoding apparatus 200 can independently divide each of the second coding units 1810a and 1810b or each of the second coding units 1820a and 1820b having a non-square shape. Each of the second coding units 1810a and 1810b or each of the second coding units 1820a and 1820b can be divided in a certain order via a recursive method, which can be a division method corresponding to a method of dividing the first coding unit 1800 based on at least one of the block shape information and the division shape information.

[0230] For example, the image decoding apparatus 200 can determine third coding units 1812a and 1812b having a square shape by dividing the second coding unit 1810a located at the left side in the horizontal direction, or can determine third coding units 1814a and 1814b having a square shape by dividing the second coding unit 1810b located at the right side in the horizontal direction. Also, the image decoding apparatus 200 can determine third coding units 1816a to 1816d having a square shape by dividing both the second coding unit 1810a located at the left side and the second coding unit 1810b located at the right side in the horizontal direction. In this case, the coding units can be determined in the same manner as when the first coding unit 1800 is divided into four second coding units 1830a to 1830d having a square shape.

[0231] As another example, the image decoding apparatus 200 can determine third coding units 1822a and 1822b having a square shape by dividing the second coding unit 1820a located at the top in the vertical direction, and can determine third coding units 1824a and 1824b having a square shape by dividing the second coding unit 1820b located at the bottom in the vertical direction. Also, the image decoding apparatus 200 can determine third coding units 1826a, 1826b, 1826c, and 1816d having a square shape by dividing both the second coding unit 1820a located at the top and the second coding unit 1820b located at the bottom in the vertical direction. In this case, the coding units can be determined in the same manner as when the first coding unit 1800 is divided into four second coding units 1830a, 1830b, 1830c, and 1830d having a square shape.

[0232] Figure 19 The order in which a plurality of coding units are processed according to an embodiment can be changed according to the processing of dividing the coding units.

[0233] According to an embodiment, the image decoding apparatus 200 can divide the first coding unit 1900 based on the block shape information and the division shape information. When the block shape information indicates a square shape and the division shape information indicates that the first coding unit 1900 is divided in at least one of the horizontal direction and the vertical direction, the image decoding apparatus 200 can divide the first coding unit 1900 to determine the second coding units 1910a and 1910b, the second coding units 1920a and 1920b, or the second coding units 1930b, 1930c, and 1930d. Referring to Figure 19, the second coding units 1910a and 1910b or the second coding units 1920a and 1920b determined when the first coding unit 1900 is divided in a horizontal direction or a vertical direction can be independently divided based on the block shape information and the division shape information. For example, the image decoding apparatus 200 can determine third coding units 1916a, 1916b, 1916c, and 1916d by dividing each of the second coding units 1910a and 1910b generated as the first coding unit 1900 is divided in the vertical direction in the horizontal direction, or can determine third coding units 1926a, 1926b, 1926c, and 1926d by dividing the second coding units 1920a and 1920b generated as the first coding unit 1900 is divided in the vertical direction in the vertical direction. The above has been described with reference to Figure 17 The process of dividing the second coding units 1910a and 1910b or the second coding units 1920a and 1920b is described, and thus details thereof are not provided again.

[0234] According to an embodiment, the image decoding apparatus 200 can process the coding units according to a certain order. The above has been described with reference to Figure 14 The feature regarding processing the coding units according to a certain order is described, and thus details thereof are not provided again. Reference is made to Figure 19 , the image decoding apparatus 200 can determine four third coding units 1916a, 1916b, 1916c, and 1916d or third coding units 1926a, 1926b, 1926c, and 1926d having a square shape by dividing the first coding unit 1900 having a square shape. According to an embodiment, the image decoding apparatus 200 can determine an order of processing the third coding units 1916a, 1916b, 1916c, and 1916d or the third coding units 1926a, 1926b, 1926c, and 1926d based on how the first coding unit 1900 is divided.

[0235] According to an embodiment, the image decoding apparatus 200 can determine third coding units 1916a, 1916b, 1916c, and 1916d by dividing the second coding units 1910a and 1910b generated as the first coding unit 1900 is divided in the vertical direction in the horizontal direction, and can process the third coding units 1916a, 1916b, 1916c, and 1916d according to the following order 1917: first processing the third coding units 1916a and 1916c included in the second coding unit 1910a located at the left side in the vertical direction, and then processing the third coding units 1916b and 1916d included in the second coding unit 1910b located at the right side in the vertical direction.

[0236] According to an embodiment, the image decoding apparatus 200 can determine the third coding units 1926a to 1926d by dividing the second coding units 1920a and 1920b generated as the first coding unit 1900 is divided along the horizontal direction in the vertical direction, and can process the third coding units 1926a to 1926d according to the following order 1927: first, the third coding units 1926a and 1926b included in the second coding unit 1920a located at the top are processed along the horizontal direction, and then the third coding units 1926c and 1926d included in the second coding unit 1920b located at the bottom are processed along the horizontal direction.

[0237] Referring to Figure 19 The third coding units 1916a to 1916d or the third coding units 1926a to 1926d having a square shape can be determined when both the second coding units 1910a and 1910b or the second coding units 1920a and 1920b are divided. The second coding units 1910a and 1910b determined when the first coding unit 1900 is divided in the vertical direction and the second coding units 1920a and 1920b determined when the first coding unit 1900 is divided in the horizontal direction are divided in different shapes, but according to the third coding units 1916a to 1916d and the third coding units 1926a to 1926d determined later, the first coding unit 1900 is divided in coding units having the same shape. Thus, even when coding units having the same shape are determined while the coding units are recursively divided by different processes based on at least one of the block shape information and the division shape information, the image decoding apparatus 200 can process the plurality of coding units determined in the same shape in different orders.

[0238] Figure 20 A process of determining a depth of a coding unit as a shape and a size of the coding unit are changed when a plurality of coding units are determined as the coding unit is recursively divided according to an embodiment is shown.

[0239] According to an embodiment, the image decoding apparatus 200 can determine a depth of a coding unit according to a certain criterion. For example, the certain criterion can be a length of a long side of the coding unit. When a length of a long side of a current coding unit is divided into 1 / 2n of a length of a long side of the coding unit before the coding unit is divided, a depth of the current coding unit can be determined to be increased to n times of a depth of the coding unit before the coding unit is divided, where n > 0. Hereinafter, a coding unit having an increased depth is referred to as a coding unit of a lower depth.

[0240] Referring to Figure 20, according to an embodiment, the image decoding apparatus 200 can determine a second coding unit 2002 and a third coding unit 2004 of a lower depth by dividing the first coding unit 2000 having a square shape based on the block shape information indicating the square shape (e.g., the block shape information can indicate '0:SQUARE'). When the size of the first coding unit 2000 having the square shape is 2N×2N, the second coding unit 2002 determined by dividing the width and the height of the first coding unit 2000 by 1 / 2 can have a size of N×N, and the third coding unit 2004 determined by dividing the width and the height of the second coding unit 2002 by 1 / 2 can have a size of N / 2×N / 2. In this case, the width and the height of the third coding unit 2004 are 1 / 2 of the first coding unit 2000. 1 The size of the second coding unit 2002 determined by dividing the width and the height of the first coding unit 2000 can be N×N. Also, the size of the third coding unit 2004 determined by dividing the width and the height of the second coding unit 2002 by 1 / 2 can be N / 2×N / 2. In this case, the width and the height of the third coding unit 2004 are 1 / 2 of the first coding unit 2000. 2 Accordingly, when the depth of the first coding unit 2000 is D, the depth of the second coding unit 2002 having 1 / 2 of the width and the height of the first coding unit 2000 can be D+1, and the depth of the third coding unit 2004 having 1 / 2 of the width and the height of the first coding unit 2000 can be D+2. 1 2 Accordingly, when the depth of the first coding unit 2000 is D, the depth of the second coding unit 2002 having 1 / 2 of the width and the height of the first coding unit 2000 can be D+1, and the depth of the third coding unit 2004 having 1 / 2 of the width and the height of the first coding unit 2000 can be D+2.

[0241] According to an embodiment, the image decoding apparatus 200 can determine a second coding unit 2012 or 2022 and a third coding unit 2014 or 2024 by dividing the first coding unit 2010 or 2020 having a non-square shape based on the block shape information indicating the non-square shape according to an embodiment (e.g., the block shape information can indicate '1:NS_VER' or '2:NS_HOR', where '1:NS_VER' indicates a non-square shape in which the height is longer than the width, and '2:NS_HOR' indicates a non-square shape in which the width is longer than the height).

[0242] The image decoding apparatus 200 can determine a second coding unit (e.g., the second coding unit 2002, 2012, or 2022) by dividing at least one of the width and the height of the first coding unit 2010 having a size of N×2N. That is, the image decoding apparatus 200 can determine a second coding unit 2002 having a size of N×N or a second coding unit 2022 having a size of N×N / 2 by dividing the first coding unit 2010 in a horizontal direction, or can determine a second coding unit 2012 having a size of N / 2×N by dividing the first coding unit 2010 in horizontal and vertical directions.

[0243] ​According to an embodiment, the image decoding device 200 can determine a second encoding unit (e.g., a second encoding unit 2002, 2012, or 2022) by dividing at least one of the width and height of a first encoding unit 2020 with a size of 2N×N. That is, the image decoding device 200 can determine a second encoding unit 2002 with a size of N×N or a second encoding unit 2012 with a size of N / 2×N by dividing the first encoding unit 2020 along the vertical direction, or it can determine a second encoding unit 2022 with a size of N×N / 2 by dividing the first encoding unit 2020 along both the horizontal and vertical directions.

[0244] According to an embodiment, the image decoding device 200 can determine a third encoding unit (e.g., a third encoding unit 2004, 2014, or 2024) by dividing at least one of the width and height of a second encoding unit 2002 with a size of N×N. That is, the image decoding device 200 can determine a third encoding unit 2004 with a size of N / 2×N / 2 by dividing the second encoding unit 2002 along the vertical and horizontal directions. 2 The third coding unit 2014 of size N / 2 × N / 2, or size N / 2 × N / 2 2 The third coding unit 2024.

[0245] According to an embodiment, the image decoding device 200 can determine a third encoding unit (e.g., a third encoding unit 2004, 2014, or 2024) by dividing at least one of the width and height of a second encoding unit 2012 with a size of N / 2 × N. That is, the image decoding device 200 can determine a third encoding unit 2004 with a size of N / 2 × N / 2 by dividing the second encoding unit 2012 along the horizontal direction. 2 The third coding unit 2024, or the second coding unit 2012, can be divided into units of size N / 2 along the vertical and horizontal directions. 2 The third coding unit 2014 is ×N / 2.

[0246] According to an embodiment, the image decoding device 200 can determine a third encoding unit (e.g., a third encoding unit 2004, 2014, or 2024) by dividing at least one of the width and height of a second encoding unit 2022 with a size of N×N / 2. That is, the image decoding device 200 can determine a third encoding unit 2004 with a size of N / 2×N / 2 by dividing the second encoding unit 2022 along a vertical direction. 2a third coding unit 2014 of size N / 2 x N / 2, or can be determined by dividing the second coding unit 2022 in the vertical direction and the horizontal direction 2 a third coding unit 2024 of size N / 2 x N / 2.

[0247] According to embodiments, the image decoding apparatus 200 can divide a coding unit having a square shape (e.g., the first coding unit 2000, the second coding unit 2002, or the third coding unit 2004) in the horizontal direction or the vertical direction. For example, a first coding unit 2010 of size N x 2N can be determined by dividing the first coding unit 2000 of size 2N x 2N in the vertical direction, or a first coding unit 2020 of size 2N x N can be determined by dividing the first coding unit 2000 in the horizontal direction. According to embodiments, when the depth is determined based on the length of the longest side of a coding unit, the depth of a coding unit determined as the first coding unit 2000 of size 2N x 2N is divided in the horizontal direction or the vertical direction can be the same as the depth of the first coding unit 2000.

[0248] According to embodiments, the width and the height of the third coding unit 2014 or 2024 can be 1 / 2 of the width and the height of the first coding unit 2010 or 2020 2 When the depth of the first coding unit 2010 or 2020 is D, the depth of the second coding unit 2012 or 2022 can be D+1, and the depth of the third coding unit 2014 or 2024 can be D+2, where the width and the height of the second coding unit 2012 or 2022 are 1 / 2 of the width and the height of the first coding unit 2010 or 2020, and the width and the height of the third coding unit 2014 or 2024 are 1 / 2 of the width and the height of the first coding unit 2010 or 2020 2 .

[0249] Figure 21 A part index (PID) for distinguishing the depth and the coding unit that can be determined according to the shape and the size of a coding unit according to embodiments is shown.

[0250] According to embodiments, the image decoding apparatus 200 can determine second coding units having various shapes by dividing the first coding unit 2100 having a square shape. Referring to Figure 21, the image decoding apparatus 200 can determine the second coding units 2102a and 2102b, the second coding units 2104a and 2104b, or the second coding units 2106a, 2106b, 2106c, and 2106d by dividing the first coding unit 2100 in at least one of a vertical direction and a horizontal direction according to the division shape information. That is, the image decoding apparatus 200 can determine the second coding units 2102a and 2102b, the second coding units 2104a and 2104b, or the second coding units 2106a, 2106b, 2106c, and 2106d based on the division shape information of the first coding unit 2100.

[0251] According to an embodiment, a depth of the second coding units 2102a and 2102b, the second coding units 2104a and 2104b, or the second coding units 2106a, 2106b, 2106c, and 2106d determined according to the division shape information of the first coding unit 2100 having a square shape can be determined based on a length of a long side. For example, because a length of one side of the first coding unit 2100 having a square shape is the same as a length of a long side of the second coding units 2102a and 2102b or the second coding units 2104a and 2104b having a non-square shape, a depth of the first coding unit 2100 can be the same as a depth of the second coding units 2102a and 2102b or the second coding units 2104a and 2104b having a non-square shape, i.e., D. In other words, when the image decoding apparatus 200 divides the first coding unit 2100 into four second coding units 2106a, 2106b, 2106c, and 2106d having a square shape based on the division shape information, a length of one side of the second coding units 2106a, 2106b, 2106c, and 2106d having a square shape is 1 / 2 of a length of one side of the first coding unit 2100, and a depth of the second coding units 2106a, 2106b, 2106c, and 2106d can be D+1, i.e., a lower depth than the depth D of the first coding unit 2100.

[0252] According to an embodiment, the image decoding apparatus 200 can divide the first coding unit 2110 (in which a height is longer than a width) into a plurality of second coding units 2112a and 2112b or a plurality of second coding units 2114a, 2114b, and 2114c in a horizontal direction according to the division shape information. According to an embodiment, the image decoding apparatus 200 can divide the first coding unit 2120 (in which a width is longer than a height) into a plurality of second coding units 2122a and 2122b or a plurality of second coding units 2114a, 2114b, and 2114c in a vertical direction according to the division shape information.

[0253] According to an embodiment, the depth of the second coding units 2112a and 2112b, the second coding units 2114a, 2114b, and 2114c, the second coding units 2122a and 2122b, or the second coding units 2124a, 2124b, and 2124c determined according to the division shape information of the first coding unit 2110 or 2120 having a non-square shape can be determined based on the length of the long side. For example, because the length of one side of the second coding units 2112a and 2112b having a square shape is 1 / 2 of the length of the long side of the first coding unit 2110 having a non-square shape (in which the height is longer than the width in the first coding unit 2110), the depth of the second coding units 2112a and 2112b is D+1, i.e., a depth lower than the depth D of the first coding unit 2110 having a non-square shape.

[0254] In addition, the image decoding apparatus 200 can divide the first coding unit 2110 having a non-square shape into an odd number of second coding units 2114a, 2114b, and 2114b based on the division shape information. The odd number of second coding units 2114a, 2114b, and 2114c can include the second coding units 2114a and 2114c having a non-square shape and the second coding unit 2114b having a square shape. In this case, because the length of the long side of the second coding units 2114a and 2114c having a non-square shape and the length of one side of the second coding unit 2114b having a square shape is 1 / 2 of the length of one side of the first coding unit 2110, the depth of the second coding units 2114a, 2114b, and 2114c can be D+1, i.e., a depth lower than the depth D of the first coding unit 2110. The image decoding apparatus 200 can determine the depth of coding units related to the first coding unit 2120 having a non-square shape (in which the width is longer than the height in the first coding unit 2120) in the same manner as determining the depth of coding units related to the first coding unit 2110.

[0255] According to an embodiment, for the operation of determining a PID for distinguishing coding units, when an odd number of coding units do not have the same size, the image decoding apparatus 200 can determine the PID based on a size ratio of the coding units. Referring to Figure 21, the second coding unit 2114b located at the center among the odd number of second coding units 2114a, 2114b, and 2114c can have the same width as the second coding units 2114a and 2114c, but the height is twice the height of the second coding units 2114a and 2114c. That is, the second coding unit 2114b located at the center can include two of the second coding units 2114a and 2114c. Accordingly, when the PID of the second coding unit 2114b located at the center is 1 according to the scanning order, the PID of the second coding unit 2114c next in sequence can be 3, and the PID is increased by 2. That is, the value of the PID can not be continuous. According to an embodiment, the image decoding apparatus 200 can determine whether the odd number of coding units have the same size based on the discontinuity of the PID used to distinguish the coding units.

[0256] According to an embodiment, the image decoding apparatus 200 can determine whether a plurality of coding units determined as the current coding unit is divided have a certain division shape based on the value of the PID. Referring to Figure 21 , the image decoding apparatus 200 can determine the even number of second coding units 2112a and 2112b, or the odd number of second coding units 2114a, 2114b, and 2114c by dividing the first coding unit 2110 having a rectangular shape in which the height is longer than the width. The image decoding apparatus 200 can use the PID indicating each coding unit in order to distinguish the plurality of coding units. According to an embodiment, the PID can be obtained from a sample located at a certain position (e.g., the upper left sample) of each coding unit.

[0257] According to an embodiment, a coding unit located at a certain position can be determined among the coding units determined by using the PID used to distinguish the coding units. According to an embodiment, when the division shape information of the first coding unit 2110 having a rectangular shape in which the height is longer than the width indicates that the first coding unit 2110 is divided into three coding units, the image decoding apparatus 200 can divide the first coding unit 2110 into three second coding units 2114a, 2114b, and 2114c. The image decoding apparatus 200 can assign a PID to each of the three second coding units 2114a, 2114b, and 2114c. The image decoding apparatus 200 can compare the PIDs of the odd number of coding units to determine a center coding unit among the coding units. The image decoding apparatus 200 can determine the second coding unit 2114b having a PID corresponding to a center value among the PIDs of the coding units determined as the first coding unit 2110 is divided as a coding unit located at the center among the coding units based on the PIDs of the coding units. According to an embodiment, when the coding units do not have the same size in determining the PID used to distinguish the coding units, the image decoding apparatus 200 can determine the PID based on the size ratio of the coding units. Referring toFigure 21 The second coding unit 2114b generated as the first coding unit 2110 is divided can have the same width as the second coding units 2114a and 2114c, but the height can be twice the height of the second coding units 2114a and 2114c. In this case, when the PID of the second coding unit 2114b located at the center is 1, the PID of the second coding unit 2114c in the next order can be 3, with the PID increasing by 2. In this way, when the range of increase of the PID is different while increasing uniformly, the image decoding apparatus 200 can determine that the current coding unit is divided into a plurality of coding units including a coding unit having a different size from other coding units. According to an embodiment, when the division shape information indicates division into an odd number of coding units, the image decoding apparatus 200 can divide the current coding unit into a plurality of coding units in which a coding unit located at a specific position (e.g., a center coding unit) has a different size from other coding units. In this case, the image decoding apparatus 200 can determine the center coding unit having a different size by using the PID of the coding unit. However, the PID of the coding unit located at the specific position and the size or position described above are designated to describe an embodiment, and thus should not be interpreted restrictively, and various PIDs of coding units and various positions and sizes can be used.

[0258] According to an embodiment, the image decoding apparatus 200 can use a specific data unit from which recursive division of a coding unit starts.

[0259] Figure 22 A method of determining a plurality of coding units according to a plurality of specific data units included in a picture according to an embodiment is illustrated.

[0260] According to an embodiment, a specific data unit can be defined as a data unit from which a coding unit is recursively divided by using at least one of block shape information and division shape information. That is, the specific data unit can correspond to a coding unit of the highest depth used when a plurality of coding units are determined by dividing a current picture. Hereinafter, the specific data unit is referred to as a reference data unit for convenience of description.

[0261] According to an embodiment, the reference data unit can indicate a specific size and shape. According to an embodiment, the reference coding unit can include M×N samples. Here, M and N can be the same and can be an integer expressed as a multiple of 2. That is, the reference data unit can indicate a square shape or a non-square shape and can be divided into an integer number of coding units later.

[0262] According to an embodiment, the image decoding apparatus 200 can divide the current picture into a plurality of reference data units. According to an embodiment, the image decoding apparatus 200 can divide the plurality of reference data units obtained by dividing the current picture by using the division information about each of the reference data units. Such a division process of the reference data units can correspond to a division process using a quad-tree structure.

[0263] According to an embodiment, the image decoding apparatus 200 can determine in advance a minimum size of a reference data unit that can be used in the current picture. Accordingly, the image decoding apparatus 200 can determine reference data units having various sizes equal to or greater than the minimum size, and determine at least one coding unit based on the determined reference data units by using the block shape information and the division shape information.

[0264] Referring to Figure 22 , the image decoding apparatus 200 can use a reference coding unit 2200 having a square shape, or can use a reference coding unit 2202 having a non-square shape. According to an embodiment, the shape and size of the reference coding unit can be determined according to various data units (e.g., sequence, picture, slice, slice segment, and largest coding unit) in which at least one reference coding unit can be included.

[0265] According to an embodiment, the receiver 210 of the image decoding apparatus 200 can obtain at least one of information about the shape of the reference coding unit and information about the size of the reference coding unit from the bitstream according to various data units. The process of determining at least one coding unit included in the reference coding unit 2200 having a square shape has been described above through the process of dividing the current coding unit 1000, and the process of determining at least one coding unit included in the reference coding unit 2202 having a non-square shape has been described above through the process of dividing the current coding unit 1100 or 1150, and thus the details thereof will not be provided. Figure 10 Figure 11

[0266] ​​According to an embodiment, in order to determine the size and shape of the reference coding unit according to some data units predetermined based on a predetermined condition, the image decoding apparatus 200 can use a PID for distinguishing the size and shape of the reference coding unit. That is, the receiver 210 can obtain only the PID for distinguishing the size and shape of the reference coding unit, which is a data unit satisfying the predetermined condition (e.g., a data unit having a size equal to or smaller than that of a slice) among various data units (e.g., sequence, picture, slice, slice segment, and largest coding unit), from a bitstream according to a slice, a slice segment, and a largest coding unit. The image decoding apparatus 200 can determine the size and shape of the reference coding unit according to the data unit satisfying the predetermined condition by using the PID. When information on the shape of the reference coding unit and information on the size of the reference coding unit are obtained from a bitstream and used according to a data unit having a relatively small size, the use efficiency of the bitstream can be insufficient, and thus, instead of directly obtaining the information on the shape of the reference coding unit and the information on the size of the reference coding unit, only the PID can be obtained and used. In this case, at least one of the size and shape of the reference coding unit corresponding to the PID indicating the size and shape of the reference coding unit can be predetermined. That is, the image decoding apparatus 200 can select at least one of the predetermined size and shape of the reference coding unit according to the PID to determine at least one of the size and shape of the reference coding unit included in a data unit that is a criterion for obtaining the PID.

[0267] According to an embodiment, the image decoding apparatus 200 can use at least one reference coding unit included in one largest coding unit. That is, the largest coding unit in which an image is divided can include at least one reference coding unit, and a coding unit can be determined when each of the reference coding units is recursively divided. According to an embodiment, at least one of the width and height of the largest coding unit can be an integer multiple of at least one of the width and height of the reference coding unit. According to an embodiment, the size of the reference coding unit can be equal to the size of the largest coding unit, which is divided n times according to a quad-tree structure. In other words, according to various embodiments, the image decoding apparatus 200 can determine a reference coding unit by dividing a largest coding unit n times according to a quad-tree structure, and divide the reference coding unit based on at least one of block shape information and division shape information.

[0268] Figure 23 A processing block used as a criterion for determining a determination order of a reference coding unit included in the picture 2300 according to an embodiment is shown.

[0269] According to an embodiment, the image decoding apparatus 200 can determine at least one processing block into which a picture is divided. The processing block is a data unit including at least one reference coding unit into which the picture is divided, and the at least one reference coding unit included in the processing block can be determined in a certain order. In other words, the determination order of the at least one reference coding unit determined in each processing block can correspond to one of various orders for determining the reference coding unit, and can vary according to the processing block. The determination order of the reference coding unit determined per processing block can be one of various orders such as a raster scan order, a zigzag scan order, an N zigzag scan order, a right-up diagonal scan order, a horizontal scan order, and a vertical scan order, but should not be construed limitatively by the scan orders.

[0270] According to an embodiment, the image decoding apparatus 200 can determine the size of at least one processing block included in a picture by obtaining information on the size of the processing block. The image decoding apparatus 200 can obtain the information on the size of the processing block from a bitstream to determine the size of the at least one processing block included in the picture. The size of the processing block can be a certain size of a data unit indicated by the information on the size of the processing block.

[0271] According to an embodiment, the receiver 210 of the image decoding apparatus 200 can obtain the information on the size of the processing block from a bitstream according to a certain data unit. For example, the information on the size of the processing block can be obtained from the bitstream in units of a picture, a sequence, a picture, a slice, and a slice segment. That is, the receiver 210 can obtain the information on the size of the processing block from the bitstream according to such several data units, and the image decoding apparatus 200 can determine the size of at least one processing block into which a picture is divided by using the obtained information on the size of the processing block, wherein the size of the processing block can be an integer multiple of the size of a reference coding unit.

[0272] According to an embodiment, the image decoding apparatus 200 can determine the sizes of the processing blocks 2302 and 2312 included in the picture 2300. For example, the image decoding apparatus 200 can determine the size of the processing block based on the information on the size of the processing block obtained from a bitstream. Referring to Figure 23 , according to an embodiment, the image decoding apparatus 200 can determine the horizontal size of the processing blocks 2302 and 2312 to be four times the horizontal size of a reference coding unit, and the vertical size thereof to be four times the vertical size of the reference coding unit. The image decoding apparatus 200 can determine the determination order of at least one reference coding unit in at least one processing block.

[0273] According to an embodiment, the image decoding apparatus 200 can determine each of the processing blocks 2302 and 2312 included in the picture 2300 based on the size of the processing block, and can determine the determined order of at least one reference coding unit included in each of the processing blocks 2302 and 2312. According to an embodiment, the operation of determining the reference coding unit can include determining the size of the reference coding unit.

[0274] According to an embodiment, the image decoding apparatus 200 can obtain information about the determined order of at least one reference coding unit included in at least one processing block from a bitstream, and can determine the determined order of the at least one reference coding unit based on the obtained information. The information about the determined order can be defined as an order or a direction of determining the reference coding unit in the processing block. That is, the order of determining the reference coding unit can be independently determined per processing block.

[0275] According to an embodiment, the image decoding apparatus 200 can obtain information about the determined order of the reference coding unit from a bitstream according to a certain data unit. For example, the receiver 210 can obtain information about the determined order of the reference coding unit from a bitstream according to a data unit such as a picture, a sequence, a picture, a slice, a slice segment, and a processing block. Since the information about the determined order of the reference coding unit indicates the determined order of the reference coding unit in the processing block, the information about the determined order can be obtained per certain data unit including an integer number of processing blocks.

[0276] According to an embodiment, the image decoding apparatus 200 can determine at least one reference coding unit based on the determined order.

[0277] According to an embodiment, the receiver 210 can obtain information about the determined order of the reference coding unit from a bitstream as information related to the processing blocks 2302 and 2312, and the image decoding apparatus 200 can determine the order for determining at least one reference coding unit included in the processing blocks 2302 and 2312, and can determine at least one reference coding unit included in the picture 2300 according to the determined order of the coding unit. Referring to Figure 23The image decoding apparatus 200 can determine determination orders 2304 and 2314 of at least one reference coding unit associated with processing blocks 2302 and 2312, respectively. For example, when information about the determination order of reference coding units is obtained according to the processing blocks, the determination orders of the reference coding units associated with processing blocks 2302 and 2312 may be different from each other. When the determination order 2304 associated with processing block 2302 is a raster scan order, the reference coding units included in processing block 2302 can be determined according to the raster scan order. On the other hand, when the determination order 2314 associated with processing block 2312 is the reverse of the raster scan order, the reference coding units included in processing block 2312 can be determined according to the reverse of the raster scan order.

[0278] According to an embodiment, the image decoding apparatus 200 can decode at least one determined reference coding unit. The image decoding apparatus 200 can decode an image based on the reference coding unit determined through the above embodiments. Examples of methods for decoding the reference coding unit may include various methods for decoding an image.

[0279] According to an embodiment, the image decoding apparatus 200 can obtain block shape information indicating the shape of the current coding unit or partition shape information indicating the method of partitioning the current coding unit from the bitstream, and can use the block shape information or partition shape information. The block shape information or partition shape information can be included in the bitstream associated with various data units. For example, the image decoding apparatus 200 can use the block shape information or partition shape information included in sequence parameter sets, picture parameter sets, video parameter sets, strip headers, and strip segment headers. Furthermore, the image decoding apparatus 200 can obtain an index corresponding to the block shape information or partition shape information from the bitstream based on the maximum coding unit, reference coding unit, and processing block, and use the index.

[0280] Although this disclosure has been specifically shown and described with reference to embodiments thereof, those skilled in the art will understand that various changes in form and detail may be made therein without departing from the spirit and scope of this disclosure as defined by the appended claims. The embodiments should be understood in a descriptive sense only and not for limiting purposes. Therefore, the scope of this disclosure is not defined by its detailed description but by the appended claims, and all differences within that scope are to be construed as included in this disclosure.

[0281] Embodiments of this disclosure can be programmed into a computer and implemented in a general-purpose digital computer that runs the program using a computer-readable recording medium. Examples of computer-readable recording media include magnetic storage media (e.g., ROM, floppy disk, hard disk, etc.), optical recording media (e.g., CD-ROM or DVD), etc.

Claims

1. A method for decoding an image using block mapping, the method comprising: Based on the partitioning information of the upper-level block, the upper-level block is divided into multiple blocks along the vertical direction. The multiple blocks include a left block, a current block, and a right block, and the width of the current block is greater than the width of either the left block or the right block. Based on the current block obtained by dividing the upper block along the vertical direction, it is determined that the current block is not divided into two lower blocks along the vertical direction; Obtain partitioning information indicating whether the current block is divided into two or three lower-level blocks along the horizontal direction; When the partitioning information indicates that the current block is divided into two lower-level blocks along the horizontal direction, the current block is divided into a first lower-level block and a second lower-level block along the horizontal direction. When the first lower-level block is decoded, the block mapping is updated with an indication that the first lower-level block can be used as a value for neighboring lower-level blocks; The block mapping is used to decode the second lower-level block. Among them, one of the left block, the current block and the right block has a shape that is either a square shape or a non-square shape.

2. A method for encoding an image using block mapping, the method comprising: Based on the partitioning information of the upper-level block, the upper-level block is divided into multiple blocks along the vertical direction. The multiple blocks include a left block, a current block, and a right block, and the width of the current block is greater than the width of either the left block or the right block. Based on the current block obtained by dividing the upper block along the vertical direction, it is determined that the current block is not divided into two lower blocks along the vertical direction; Generate partitioning information indicating whether the current block is divided into two or three lower-level blocks along the horizontal direction; When the partitioning information indicates that the current block is divided into two lower-level blocks along the horizontal direction, the current block is divided into a first lower-level block and a second lower-level block along the horizontal direction. When the first lower-level block is encoded, the block mapping is updated with an indication that the first lower-level block can be used as a value for neighboring lower-level blocks; The block mapping is used to encode the second lower-level block. Among them, one of the left block, the current block and the right block has a shape that is either a square shape or a non-square shape.

3. A non-transitory computer-readable medium for recording a bit stream generated by an encoding device, the bit stream comprising: The current block partitioning information, The partitioning information of the current block is obtained through the following operations: Based on the partitioning information of the upper-level block, the upper-level block is divided into multiple blocks along the vertical direction. The multiple blocks include a left block, a current block, and a right block, and the width of the current block is greater than the width of either the left block or the right block. Based on the current block obtained by dividing the upper block along the vertical direction, it is determined that the current block is not divided into two lower blocks along the vertical direction; Generate partitioning information for the current block, wherein the partitioning information for the current block indicates whether the current block is divided into two or three lower-level blocks along the horizontal direction; When the partitioning information indicates that the current block is divided into two lower-level blocks along the horizontal direction, the current block is divided into a first lower-level block and a second lower-level block along the horizontal direction. When the first lower-level block is encoded, the block mapping is updated with an indication that the first lower-level block can be used as a value for neighboring lower-level blocks; The block mapping is used to encode the second lower-level block. Among them, one of the left block, the current block and the right block has a shape that is either a square shape or a non-square shape.

Images