Method and apparatus for encoding or decoding an image by block mapping

Abstract

A method and apparatus for encoding or decoding an image by block mapping are disclosed. According to an embodiment, a method and apparatus for decoding an image are characterized by receiving a bitstream of an encoded image, determining one or more blocks in a picture by partitioning information obtained from the bitstream, determining a block map indicating whether a previous block among the one or more blocks has been decoded, determining, based on the block map, a neighboring area that a current block among the one or more blocks can refer to according to a predetermined scan order, and decoding the current block based on the neighboring area that can be referred to.

Description

[0001] This application is a divisional application of the invention patent application filed on October 10, 2016, with application number "201680091331.8" and titled "Method and apparatus for encoding or decoding an image by block mapping". Technical Field

[0002] This disclosure relates to a method and apparatus for encoding or decoding images, and more specifically, to a method and apparatus for encoding or decoding images by using block mapping. Background Technology

[0003] Image data is encoded using a codec based on a data compression standard (e.g., the Moving Picture Experts Group (MPEG) standard) and recorded as a bitstream on a recording medium or transmitted via a communication channel.

[0004] With the development and availability of hardware for reproducing and storing high-definition or high-quality image content, the demand for codecs for efficiently encoding or decoding high-definition or high-quality image content is constantly growing. Encoded image content can be reproduced by decoding it. Recently, methods for efficiently compressing such high-resolution or high-quality image content have been implemented. For example, methods for efficiently compressing images by processing the image to be encoded in any manner have been implemented.

[0005] Various data units can be used to compress images, and there may be inclusion relationships between such data units. Various methods can be used to divide the data units to determine their size. Images can be encoded or decoded by determining the optimal data units based on their characteristics. Summary of the Invention

[0006] Technical issues

[0007] A method and apparatus for encoding or decoding images by using block mapping are provided.

[0008] Solution to the problem

[0009] According to one aspect of this disclosure, a method for decoding an image using a block map includes: receiving a bitstream of an encoded image; determining one or more blocks of the image using partitioning information obtained from the bitstream; determining a block map indicating a decoded block among the one or more blocks; determining, based on the block map, a neighboring region that can be used to reference a current block among the one or more blocks that will be decoded in a specific scanning order; and decoding the current block based on the neighboring region that can be used to reference it.

[0010] In one embodiment, in a method of decoding an image using a block map, the block map may include regions corresponding to the one or more blocks, wherein regions in the block map corresponding to decoded blocks among the one or more blocks may be set as available regions, and regions in the block map corresponding to undecoded blocks among the one or more blocks may be set as unavailable regions.

[0011] In one embodiment, in a method of decoding an image using a block map, the step of determining the block map may include updating the block map with a value indicating that each of the one or more blocks has been decoded.

[0012] In one embodiment, in a method of decoding an image using block mapping, the one or more blocks may include: one or more encoding units, or one or more transformation units.

[0013] In one embodiment, in a method of decoding an image using a block map, the block map may be created for the largest coding unit.

[0014] In one embodiment, in a method of decoding an image using block mapping, the one or more blocks may have a square shape or a non-square shape.

[0015] In one embodiment, in a method of decoding an image using block mapping, the neighboring regions that can be referenced may include at least one of: a region adjacent to the current block time or a region adjacent to the current block space.

[0016] In one embodiment, in a method for decoding an image using block mapping, the step of determining the neighboring regions that can be used for reference may include: configuring a plurality of merging candidates for deriving motion information of the current block, wherein the plurality of merging candidates includes at least one region that is temporally adjacent to the current block or spatially adjacent to the current block; when there are two or more merging candidates included in the same coding unit among the plurality of merging candidates, replacing at least one of the two or more merging candidates with a merging candidate included in different coding units; and the step of decoding the current block may include: deriving motion information of the current block based on the plurality of merging candidates, and predicting the current block using the derived motion information.

[0017] In one embodiment, in a method for decoding an image using block mapping, the step of determining the neighboring regions that can be used for reference may include: configuring a plurality of merging candidates for deriving motion information of the current block, wherein the plurality of merging candidates includes at least one region that is spatially adjacent to the current block or temporally adjacent to the current block; when there are two or more merging candidates with the same motion information among the plurality of merging candidates, replacing at least one of the two or more merging candidates with merging candidates that have different motion information; and the step of decoding the current block may include: deriving motion information of the current block based on the plurality of merging candidates, and predicting the current block using the derived motion information.

[0018] In one embodiment, in a method for decoding an image using a block map, the step of determining the neighboring regions that can be used for reference may include: configuring a plurality of merging candidates for deriving motion information of the current block, wherein the plurality of merging candidates includes at least one region that is spatially adjacent to the current block or temporally adjacent to the current block; when there is a first neighboring region among the plurality of merging candidates corresponding to an unavailable region in the block map, replacing the first neighboring region with a second neighboring region corresponding to an available region in the block map; and the step of decoding the current block may include: deriving motion information of the current block based on the plurality of merging candidates; and predicting the current block using the derived motion information.

[0019] According to another aspect of this disclosure, a method for encoding an image using a block map includes: determining one or more blocks in a picture; determining a block map representing a decoded block among the one or more blocks; determining, based on the block map, a neighboring region that can be referenced by a current block among the one or more blocks to be encoded in a specific scanning order; and encoding the current block based on the neighboring region that can be referenced.

[0020] In one embodiment, in a method of encoding an image using a block map, the block map may include regions corresponding to the one or more blocks, wherein regions in the block map corresponding to decoded blocks among the one or more blocks may be set as available regions, and regions in the block map corresponding to undecoded blocks among the one or more blocks may be set as unavailable regions.

[0021] In one embodiment, in a method of encoding an image using a block map, the step of determining the block map may include updating the block map with a value indicating the decoding of each of the one or more blocks whenever each of the blocks is decoded.

[0022] In one embodiment, in a method of encoding an image using block mapping, the one or more blocks may have a square shape or a non-square shape.

[0023] In one embodiment, in a method of encoding an image using block mapping, the neighboring regions that can be referenced may include at least one of a region adjacent to the current block time or a region adjacent to the current block space.

[0024] According to another aspect of this disclosure, an apparatus for decoding an image using a block mapping includes: a receiver configured to receive a bitstream of an encoded image; and a decoder configured to: determine one or more blocks of the image using partitioning information obtained from the bitstream; determine a block mapping indicating a decoded block among the one or more blocks; determine, based on the block mapping, a neighboring region that can be referenced by a current block among the one or more blocks to be decoded in a specific scanning order; and decode the current block based on the neighboring region that can be referenced. Attached Figure Description

[0025] Figure 1 This is a schematic block diagram of the image encoding apparatus 100 according to an embodiment.

[0026] Figure 2 This is a schematic block diagram of an image decoding apparatus 200 according to an embodiment.

[0027] Figure 3 This is a diagram illustrating a method for determining a region that can be used for reference according to an embodiment.

[0028] Figure 4 This is a diagram illustrating the block mapping according to an embodiment.

[0029] Figure 5 This is a diagram illustrating a method for configuring multiple merging candidates to derive motion information for the current block according to an embodiment.

[0030] Figure 6 The methods for modifying merging candidates with the same motion information according to various embodiments are illustrated.

[0031] Figure 7 This illustrates methods for merging candidates that, according to various embodiments, cannot be used for referenced regions.

[0032] Figure 8 This is a flowchart of a method for encoding an image according to an embodiment.

[0033] Figure 9 This is a flowchart of a method for decoding an image according to an embodiment.

[0034] Figure 10 The illustration shows the process of determining at least one coding unit as the current coding unit is divided, according to an embodiment.

[0035] Figure 11 The illustration shows the process of determining at least one coding unit when coding units with non-square shapes are divided, according to an embodiment.

[0036] Figure 12 The process of dividing the coding unit based on at least one of block shape information and partition shape information according to an embodiment is illustrated.

[0037] Figure 13 A method for determining a specific coding unit among an odd number of coding units is shown according to an embodiment.

[0038] Figure 14 This illustrates the order in which multiple coding units are processed when multiple coding units are determined by dividing the current coding unit, according to an embodiment.

[0039] Figure 15 The illustration shows a process for determining to divide the current coding unit into an odd number of coding units when coding units cannot be processed in a specific order, according to an embodiment.

[0040] Figure 16 The process of determining at least one coding unit by dividing a first coding unit, according to an embodiment, is illustrated.

[0041] Figure 17 The embodiment shows that when a second coding unit with a non-square shape, determined when the first coding unit is divided, satisfies certain conditions, the shape into which the second coding unit can be divided is constrained.

[0042] Figure 18 The illustration shows the process of dividing a coding unit with a square shape when the division shape information cannot be used to indicate that the coding unit is divided into four square shapes, according to an embodiment.

[0043] Figure 19 The order in which multiple coding units are processed according to an embodiment can be changed depending on the process of dividing the coding units.

[0044] Figure 20 The illustration shows the process of determining the depth of a coding unit as the shape and size of the coding unit are changed when multiple coding units are determined during recursive partitioning, according to an embodiment.

[0045] Figure 21The diagram illustrates a partial index (PID) for distinguishing depth and coding units, which can be determined based on the shape and size of the coding unit, according to an embodiment.

[0046] Figure 22 The illustration shows how multiple encoding units are determined based on multiple specific data units included in the image, according to an embodiment.

[0047] Figure 23 The diagram illustrates a processing block, according to an embodiment, used as a criterion for determining the order of at least one reference coding unit included in a frame. Detailed Implementation

[0048] The advantages and features of the embodiments set forth herein, as well as the methods of implementing them, will become clear from the following description of the embodiments taken in conjunction with the accompanying drawings. However, this disclosure is not limited to these embodiments and can be implemented in many different forms. The embodiments have only been provided so that this disclosure will be thorough and complete, and will fully convey the scope of this disclosure to those skilled in the art.

[0049] The terminology used herein will be briefly described, followed by a detailed description of various embodiments.

[0050] In this disclosure, where possible, commonly used terms are selected in consideration of the function of this disclosure, but non-common terms may be selected based on the intent of those skilled in the art, precedent, or new technologies. Some terms may be arbitrarily chosen by the applicant. In such cases, the meaning of these terms will be explained in detail in the various parts of this disclosure. Therefore, the terms used herein should not be based on their names but rather on their meanings and the entire context of this disclosure.

[0051] As used herein, the singular form is intended to include the plural form unless the context clearly indicates otherwise.

[0052] It will be understood that when an element is referred to as including another element, unless otherwise stated, the element may also include other elements. The term "unit" as used herein should be understood as a software or hardware component that performs a specific function, such as an FPGA or ASIC. However, the term "unit" is not limited to software or hardware. The term "unit" may be configured to be stored in an addressable storage medium or used to reproduce one or more processors. Thus, for example, the term "unit" may include components (such as software components, object-oriented software components, class components, and task components), processes, functions, attributes, programs, subroutines, fragments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and parameters. The functionality provided in components and "units" may be combined to obtain a smaller number of components and "units," or may be divided into sub-components and "sub-units."

[0053] In the following text, the term "image" should be understood to include both still images (such as still images in video) and moving images as part of video (i.e., dynamic images).

[0054] The term "sample" should be understood as data assigned to a video sampling location (i.e., the data to be processed). For example, pixel values ​​of an image in the spatial domain and transform coefficients in the transform domain can be samples. A unit comprising at least one such sample can be defined as a block.

[0055] The embodiments will now be described in detail so that those skilled in the art can readily implement them. In the accompanying drawings, portions not relevant to clearly describing this disclosure have been omitted.

[0056] The following will refer to Figures 1 to 23 The image encoding apparatus, image decoding apparatus, method for encoding an image, and method for decoding an image according to embodiments are described in detail below. Figures 1 to 9 This describes methods and apparatus for encoding or decoding images using block mapping, and will refer to the following... Figures 10 to 23 Describe a method for determining data units in an image.

[0057] The following will refer to Figures 1 to 9 This describes a method and apparatus for encoding or decoding an image using block mapping.

[0058] Figure 1 This is a schematic block diagram of an image encoding apparatus 100 according to an embodiment. The image encoding apparatus 100 according to the embodiment includes an encoder 110 and a transmitter 120.

[0059] In one embodiment, encoder 110 can divide the image data of the current frame into maximum coding units based on the maximum size of the coding unit. Each of the maximum coding units may include coding units divided according to block shape and partition type. In one embodiment, the image data of the spatial domain included in the maximum coding unit may be hierarchically classified according to block shape and partition type. The block shape of the coding unit may be square, rectangular, or any geometric shape, and is therefore not limited to data units of a specific size.

[0060] As the size of the image to be encoded increases, the image can be encoded at a greater compression rate by encoding it according to the largest unit. However, when the number of coding units increases and the size of the coding unit is fixed, it is not possible to effectively encode the image by reflecting the continuously changing characteristics of the image.

[0061] For example, when flat areas of the ocean or sky in an image are encoded, the compression ratio can be better as the size of the coding unit increases, but when images of crowded areas of people or buildings are encoded, the compression ratio can be better as the size of the coding unit decreases.

[0062] Therefore, according to the embodiment, the encoder 110 sets a maximum coding unit of different size for each frame or strip, or sets the block shape and division type of one or more coding units divided from the maximum coding unit. The size of the coding unit included in the maximum coding unit can be set differently according to the block shape and division type.

[0063] The block shape and partition type of one or more coding units can be determined based on a calculated rate-distortion cost. The block shape and partition type can be determined differently for each frame or strip, or for each maximum coding unit. The determined block shape and partition type are used to encode image data according to coding units.

[0064] In one embodiment, the coding units partitioned from the largest coding unit can be characterized based on block shape and partition type. The following will refer to... Figures 10 to 23 The method for determining coding units based on block shape and partitioning type is described in more detail.

[0065] In one embodiment, the coding unit included in the largest coding unit can be predicted or transformed (e.g., by converting pixel-domain values ​​to frequency-domain values) based on processing units of different sizes. In other words, the image coding apparatus 100 can perform multiple processing operations to encode an image based on processing units of various sizes and shapes. To encode image data, processing operations such as prediction, transformation, and entropy coding are performed. Processing units of the same size can be used in all operations, or processing units of different sizes can be used in each operation.

[0066] In one embodiment, the prediction mode of a coding unit may include at least one of an intra-frame mode, an inter-frame mode, or a skip mode, and a specific prediction mode may be performed only for coding units of a specific size or type. In one embodiment, the prediction mode with the minimum coding error can be selected by performing prediction for each coding unit.

[0067] In intra-frame prediction, encoder 110 may obtain prediction samples for the current block based on pixels from neighboring blocks in the current frame to which the current block belongs. In this case, encoder 110 may (i) obtain prediction samples based on the mean value of neighboring reference samples of the current block, or based on the result of interpolation of the neighboring reference samples, or (ii) obtain prediction samples based on reference samples from neighboring blocks located in a specific direction relative to the pixel to be predicted. Encoder 110 may determine the prediction mode to be applied to the current block based on prediction modes applied to neighboring blocks.

[0068] In one embodiment, encoder 110 searches for neighboring blocks adjacent to the current block among neighboring blocks reconstructed prior to the current block. Encoder 110 may refer to neighboring blocks reconstructed prior to the current block for intra-frame prediction of the current block.

[0069] In inter-frame prediction, encoder 110 can obtain predicted samples for the current block based on samples specified by motion vectors in a reference frame. Encoder 110 can apply skip mode, merge mode, or MVP mode to obtain predicted samples for the current block. In skip mode and merge mode, encoder 110 can use motion information of neighboring blocks as motion information for the current block. In skip mode, unlike in merge mode, the residual between predicted samples and reconstructed samples is not sent. In MVP mode, motion vectors of neighboring blocks can be used as motion vector predictors (MVPs) to encode the difference between the motion vectors of the current block and the motion vectors of neighboring blocks.

[0070] In inter-frame prediction, neighboring blocks include spatially neighboring blocks included in the current frame and temporally neighboring blocks included in the reference frame (co-located frame). Motion information includes motion vectors and reference frames. When the motion information of temporally neighboring blocks is used in skip mode and merge mode, the most significant picture included in the reference frame list can be used as the reference frame.

[0071] In one embodiment, as described above, encoder 110 may search for blocks reconstructed prior to the current block to perform inter-frame prediction. The blocks reconstructed prior to the current block may be spatially adjacent to the current block or temporally adjacent to the current block. Encoder 110 may refer to the blocks reconstructed prior to the current block to derive motion information for the current block.

[0072] As described above, in intra-frame prediction and inter-frame prediction, blocks reconstructed before the current block are referenced. Here, the blocks can be reconstructed sequentially according to a specific scan order. The scan order can be (but is not limited to) one of various orders (such as raster scan order, zigzag scan order, N-shaped scan order, upper right diagonal scan order, horizontal scan order, vertical scan order, etc.). In the above embodiments, the scan order is not limited to the order of reconstructed blocks, and can be for data storage, data loading, data access, etc.

[0073] As will be referred to below Figures 10 to 23 As described, the block shape of the coding unit can be square, rectangular, or any other geometric shape. Therefore, the maximum coding unit can include both square coding units and non-square coding units. In one embodiment, encoder 110 can perform data access based on a specific scan index included in each coding unit included in the maximum coding unit.

[0074] In one embodiment, when the reference coding unit does not include non-square coding units, the adjacent boundaries between coding units included in the reference coding unit can be continuous. In this case, data access can be performed on each coding unit according to a specific scan order.

[0075] In one embodiment, when the reference coding unit includes at least one non-square coding unit, the boundaries between coding units included in the reference coding unit may be discontinuous. Here, the order of data access between coding units contacting discontinuous boundaries may be interrupted. Therefore, the scanning order can be adaptively controlled between coding units contacting discontinuous boundaries. For example, data access can be performed vertically on rectangular coding units according to an N-shaped scanning order or a vertical scanning order, and horizontally on rectangular coding units according to a Z-shaped scanning order or a horizontal scanning order. The following will refer to... Figure 14 Please describe the scanning order in detail.

[0076] As described above, one or more scan sequences can be combined within a maximum coding unit or a reference coding unit. Therefore, the order in which referenced blocks are reconstructed during intra-frame prediction and inter-frame prediction can also be determined based on one or more scan sequences. In one embodiment, encoder 110 can use a block map to search for blocks decoded before the current block. In one embodiment, encoder 110 can determine neighboring blocks that the current block will reference based on the block map. Furthermore, whenever each block is decoded, encoder 110 can update the block map with a value indicating whether each block has been decoded. In one embodiment, the block map includes regions corresponding to one or more blocks included in the frame, wherein regions in the block map corresponding to decoded blocks among the one or more blocks can be set to available regions, and regions in the block map corresponding to undecoded blocks among the one or more blocks can be set to unavailable regions.

[0077] In one embodiment, the encoder 110 can transform the image data of the encoding unit not only based on the encoding unit used to encode the image data, but also based on a data unit different from the encoding unit.

[0078] A coding unit can be transformed based on a data unit with a size smaller than or equal to that of the coding unit. Examples of data units used for transformation include data units for intra-frame modes and data units for inter-frame modes. In the following text, the data unit on which the transformation is based may be referred to as a transform unit.

[0079] The information used to perform encoding requires not only information about the block shape and partition type, but also prediction-related information and transformation-related information. Therefore, encoder 110 can determine the block shape and partition type that minimizes encoding error, the prediction mode for each coding unit, the size of the transformation unit for transformation, and so on.

[0080] The encoder 110 can measure the coding error of the coding unit by using rate-distortion optimization based on Lagrange multipliers.

[0081] The transmitter 120 outputs and sends the image data of the largest coding unit, which is encoded based on at least one coding unit determined by the encoder 110, along with information about the coding mode for each coding unit, in the bitstream to the decoding device.

[0082] Encoded image data can be the result of encoding the residual data of an image.

[0083] Information about the coding mode used for each coding unit may include block shape, partition type, prediction mode used for each coding unit, and size information of the transform unit.

[0084] Figure 2This is a schematic block diagram of an image decoding apparatus 200 according to an embodiment.

[0085] In one embodiment, the image decoding apparatus 200 includes a receiver 210 and a decoder 220. In one embodiment, the definitions of various terms (such as coding unit, block shape information, partitioning type information, information about transform units, and information about prediction modes) used for various processes of the image decoding apparatus 200 are consistent with those referenced above. Figure 1 The same as those described in the image encoding device 100.

[0086] Receiver 210 receives and parses the bitstream of the encoded image. Receiver 210 extracts image data from the parsed bitstream according to the maximum coding unit and outputs the image data to decoder 220. Receiver 210 can extract information about the current frame or stripe from the raw byte sequence payload (RBSP) of the parameter set for the current frame or stripe.

[0087] Furthermore, receiver 210 extracts block shape information and partitioning type information about the coding units divided from each maximum coding unit from the parsed bitstream. The extracted block shape information and partitioning type information are output to decoder 220. That is, the image data of the bitstream can be divided into maximum coding units, so that decoder 220 can decode the image data in units of maximum coding units.

[0088] Furthermore, receiver 210 can extract information about the coding mode used for each coding unit from the parsed bitstream. Information about the coding mode can be set for one or more coding units, and this information may include information about the prediction mode used for each coding unit and the size information of the transform unit.

[0089] The information about the encoding mode extracted by receiver 210 is information about an encoding mode that causes minimal encoding error when encoding of encoding units is repeatedly performed on the encoding side (such as the image encoding apparatus 100 according to the embodiment). Therefore, image decoding apparatus 200 can reconstruct the image by decoding the data using an encoding method that causes minimal encoding error.

[0090] Receiver 210 can extract information about the encoding pattern used for each minimum coding unit. When information about the encoding pattern used for each minimum coding unit is recorded, minimum coding units with information about the same encoding pattern can be inferred to be data units included in the same maximum coding unit. That is, when minimum coding units with the same information are acquired and decoded, decoding can be performed based on the coding unit with the smallest encoding error.

[0091] Decoder 220 can reconstruct the current frame by decoding the image data of each coding unit based on information about the coding mode of each coding unit. The decoding process may include inverse quantization, inverse transform, and prediction processing including intra-frame prediction and motion compensation.

[0092] In detail, decoder 220 can rearrange the bitstream of entropy decoding based on a method in which image encoding device 100 rearranges the bitstream of entropy decoding. Decoder 220 can rearrange the coefficients represented in one-dimensional (1D) vector form by reconstructing the coefficients, which are represented in one-dimensional (1D) vector form, into two-dimensional (2D) block-type coefficients. Decoder 220 can rearrange the coefficients by receiving information related to the scanning of coefficients by image encoding device 100 and performing an inverse scan of the coefficients based on the order in which the image encoding device 100 scans the coefficients. Furthermore, decoder 220 can perform inverse quantization based on quantization parameters provided from image encoding device 100 and data of the coefficients of the rearranged blocks. Decoder 220 can perform inverse discrete cosine transform (inverse DCT) or inverse discrete sine transform (inverse DST) on the discrete cosine transform (DCT) or discrete sine transform (DST) performed by image encoding device 100 on the result of quantization, wherein the quantization is performed by image encoding device 100. The inverse transformation can be performed based on the transformation unit or the unit obtained by image segmentation, wherein the transformation unit or the unit obtained by image segmentation is determined by the image encoding device 100.

[0093] Decoder 220 can create prediction data for coding units based on prediction-related information provided from receiver 210 and information about previously decoded blocks and / or frames. Reconstruction data can be created using the prediction data and residual data of the coding units. The prediction method performed by decoder 220 is the same as the prediction method performed by encoder 110 of image encoding device 100.

[0094] In detail, when the prediction mode of the current block is intra-prediction mode, the decoder 220 can perform intra-prediction based on the information of the pixels in the current frame to create a prediction block. Here, the current block can be the current coding unit.

[0095] When the prediction mode for the current block is inter-frame prediction mode, the decoder 220 can perform inter-frame prediction on the current block based on information included in the reference frame, using at least one frame from the previous or subsequent frames of the current frame as a reference frame. Specifically, in inter-frame prediction, a prediction block for the current block can be created by selecting a reference frame for the current block and selecting a reference block of the same size as the current block. For example, in inter-frame prediction, a prediction block can be created such that the residual signal for the current block is minimized, and the magnitude of the motion vector is minimized. In this case, information from neighboring blocks of the current frame can be used with the information from the reference frame. For example, a prediction block for the current block can be created based on information from neighboring blocks using skip mode, merge mode, MVP mode, etc.

[0096] Similar to the image encoding apparatus 100 according to the embodiment, the decoder 220 can use a block map to search for blocks decoded before the current block and determine neighboring blocks that can be used for reference.

[0097] In one embodiment, the image decoding apparatus 200 can obtain information about the coding unit that minimizes the coding error when each coding unit in the coding unit is recursively encoded during the encoding process, and use this information to decode the current frame. That is, image data can be decoded on a unit-by-unit basis, based on the optimal coding unit.

[0098] Therefore, based on information about the optimal coding mode sent from the coding side, image data of an image can be effectively decoded by adaptively determining the coding unit size and coding mode according to the characteristics of the image, so that even images with high resolution or extremely large data volume can be reconstructed.

[0099] Figure 3 This is a diagram illustrating a method for determining a region that can be used for reference according to an embodiment.

[0100] In one embodiment, blocks 310, 320, and 330 may be scanned in a specific scanning order to store, load, or access data. In one embodiment, blocks 310, 320, and 330 may be encoding units or transformation units included in the maximum encoding unit 300. In one embodiment, blocks 310, 320, and 330 may have a square shape or a non-square shape.

[0101] In one embodiment, image decoding device 200 (or image encoding device 100) may process the data of current block 330 by referring to neighboring blocks decoded before current block 330. For example, data processing may include intra-frame prediction, inter-frame prediction, or various techniques for referencing neighboring regions. For instance, when the neighboring region referenced by current block 330 is a block (region) included in block 320 and assigned index 26, whether the block with index 26 will be referenced by current block 330 can be determined based on the scanning order of blocks 310, 320, and 330. When the scanning order of blocks 310, 320, and 330 is zigzag 302, block 320 with index 26 is decoded before processing the data of current block 330 and is therefore referable by current block 330. However, when the scanning order of blocks 310, 320, and 330 is N-shaped 304, block 320 with index 26 is decoded after processing the data of current block 330 and is therefore not referable by current block 330. In other words, to reference neighboring blocks of the current block 330, it should be determined whether those neighboring blocks were decoded before the current block 330. However, when blocks 310, 320, and 330 have non-square shapes, the order in which blocks 310, 320, and 330 are scanned can be adaptively adjusted or changed. Therefore, block mapping can be used to accurately determine the decoding order of blocks 310, 320, and 330 and to search for neighboring blocks decoded before the current block 330.

[0102] Figure 4 This is a diagram illustrating the block mapping according to an embodiment.

[0103] In one embodiment, block mappings 410 and 420 can be set for the current maximum coding unit. The maximum coding unit decoded before the current maximum coding unit can be used as a reference for the current maximum coding unit, and therefore it is no longer necessary to store the block mapping for the current maximum coding unit.

[0104] In one embodiment, block maps 410 and 420 may include regions corresponding to one or more blocks. In one embodiment, block map 410 may include regions 411, 412, 413, 414, and 415 corresponding to one or more coding units included in a maximum coding unit. In one embodiment, block map 420 may include regions 421, 422, 423, 424, 425, 426, 427, and 428 corresponding to one or more transform units included in a maximum coding unit.

[0105] In one embodiment, in block mapping 410, regions 411, 412, and 413 corresponding to decoded coding units among one or more coding units may be set as available regions, and regions 415 corresponding to undecoded coding units among one or more coding units may be set as unavailable regions.

[0106] In one embodiment, the block map 410 may be updated as each coding unit in the coding unit is decoded. For example, whenever a coding unit is decoded in a specific scan order, regions 411, 412, and 413 in the block map 410 corresponding to the decoded coding units may be updated sequentially with available region values ​​(e.g., the value "TRUE"), while the remaining regions 415 may retain unavailable region values ​​(e.g., the value "FALSE"). The available regions 411, 412, and 413 in the block map 410 may be used as references for the current coding unit 414.

[0107] In one embodiment, in block mapping 420, regions 421, 422, 423, 424, 425, 426 corresponding to decoded transform units in one or more transform units may be set as available regions, and region 428 corresponding to undecoded transform units in one or more transform units may be set as unavailable regions.

[0108] In one embodiment, the block map 420 may be updated as each transform unit in the transform unit is decoded. For example, whenever a transform unit is decoded in a specific scan order, regions 421, 422, 423, 424, 425, and 426 in the block map 420 corresponding to the decoded transform units may be updated sequentially with available region values ​​(e.g., the value "TRUE"), while the remaining regions 428 may retain unavailable region values ​​(e.g., the value "FALSE"). The available regions 421, 422, 423, 424, 425, and 426 of the block map 420 may be used as a reference for the current transform unit 427.

[0109] In one embodiment, the value of each region in the regions of block maps 410 and 420 can be determined as an available region value or an unavailable region value to determine whether the region will be referenced by the current encoding unit 414 or the current transformation unit 427.

[0110] In one embodiment, in block maps 410 and 420, not only regions adjacent to the current block space, but also regions adjacent to the current block time can be identified as regions that can be used for reference.

[0111] In one embodiment, similar to the shape of the coding unit described above, the regions in block maps 410 and 420 corresponding to one or more blocks may have a square shape or a non-square shape.

[0112] In one embodiment, block maps 410 and 420 may be created for each maximum coding unit, and the previously created block maps may be updated whenever each coding unit or each transform unit included in the maximum coding unit is decoded.

[0113] In one embodiment, block maps 410 and 420 may be created for the maximum coding unit, and thus the availability of regions outside the current maximum coding unit can be determined by determining the order in which the maximum coding units are processed. That is, regions belonging to the maximum coding units decoded before the current maximum coding unit can be determined as available regions, and regions belonging to the undecoded maximum coding units can be determined as unavailable regions.

[0114] The following describes various embodiments of determining the availability of neighboring blocks that the current block will reference, or adaptively changing neighboring blocks, based on block mappings 410 and 420.

[0115] In one embodiment, the image decoding device 200 (or the image encoding device 100) can predict the current block by performing intra-frame prediction using neighboring blocks of the frame adjacent to the current block. In one embodiment, the current block can be predicted by adding weight values ​​to one or more pixel values ​​of one or more neighboring blocks to calculate the predicted pixel value for each pixel of the current block.

[0116] Here, neighboring blocks are blocks reconstructed before the current block, and their availability can be determined using the block mapping described above. In one embodiment, neighboring blocks used for intra-frame prediction can be one or more neighboring blocks around the current block, and may include at least one block, for example, located to the left and above the current block.

[0117] In one embodiment, the image decoding device 200 (or the image encoding device 100) can deduce motion information of the current block and perform inter-frame prediction on the current block based on the deduced motion information.

[0118] The image used for prediction of the current block is called a reference picture or reference frame. The region of a reference picture can be represented using a reference picture index (refIdx), motion vectors, etc.

[0119] For the current frame, the reference frame list may consist of frames used for prediction, and the reference frame index may indicate a specific reference frame included in the reference frame list. For a P frame, one reference frame list is required (e.g., reference frame list 0). For a B frame, two reference frame lists are required (e.g., reference frame list 0 and reference frame list 1).

[0120] In detail, an I-frame is a frame that can be encoded / decoded via intra-frame prediction. A P-frame is a frame that can be encoded / decoded via inter-frame or intra-frame prediction by using at least one motion vector and at least one reference frame index to predict the sample values ​​of each block. A B-frame is a frame that can be encoded / decoded via inter-frame or intra-frame prediction by using at least two motion vectors and at least two reference frame indices to predict the sample values ​​of each block.

[0121] For P-frames, a list of reference frames is required, and this list of reference frames is referred to as reference frame list 0 (hereinafter referred to as L0).

[0122] A B-frame is a frame that can be encoded using one or more reference frames (e.g., two reference frames) via forward inter-frame prediction, backward inter-frame prediction, or bidirectional inter-frame prediction. For a B-frame, two reference frame lists are required, and these two reference frame lists are referred to as reference frame list 0 (hereinafter referred to as L0) and reference frame list 1 (hereinafter referred to as L1).

[0123] Inter-frame prediction using a reference frame selected from L0 is referred to as L0 prediction, and inter-frame prediction using a reference frame selected from L1 is referred to as L1 prediction. L0 prediction can be used for forward prediction, and L1 prediction can be used for backward prediction, but the directions of L0 prediction and L1 prediction are not limited to those in the above embodiments. Inter-frame prediction using two reference frames selected from L0 and L1 respectively is referred to as bidirectional prediction.

[0124] The characteristics of I-frames, P-frames, and B-frames can be defined in strips rather than frames. For example, an I-strip with the characteristics of an I-frame, a P-strip with the characteristics of a P-frame, and a B-strip with the characteristics of a B-frame can be defined as strip units.

[0125] For example, when the stripe type of the current block is B and colPic is selected from L0, or when the stripe type of the current block is P, colPic can be selected from L0.

[0126] To prevent low latency, the two directions predicted for reference screen lists 0 and 1 can be restricted to the forward direction.

[0127] For inter-frame prediction, in skip mode, information about neighboring blocks can be directly used for the current block. Therefore, in skip mode, syntax information (such as residuals) other than information about the block that indicates the motion information will be used as the motion information for the current block is not sent.

[0128] In merge mode, a predicted block for the current block can be created by directly using motion information from neighboring blocks. Image encoding device 100 can send information indicating whether to apply merge mode, information about the blocks from which motion information will be used, residual information, etc., to image decoding device 200. Image decoding device 200 can reconstruct the current block by adding the predicted block to the residual sent from image encoding device 100.

[0129] Methods for indicating which blocks will be used in merge mode can be applied to skip mode and general inter-frame prediction mode. For example, in merge mode and another inter-frame prediction mode, candidate blocks with motion information that will be used as the current block can be used together, and the same method can be used to indicate neighboring blocks whose motion information will be used.

[0130] In this case, in skip mode and general inter-frame prediction mode, usable motion information can be determined only when the reference frame index or prediction direction (reference frame list) of the neighboring block matches the reference frame index or prediction direction (reference frame list) of the current block. Therefore, the motion vector of the neighboring block can be used as a candidate motion vector prediction factor for the current block.

[0131] Figure 5 This is a diagram illustrating a method for configuring multiple merging candidates to derive motion information for the current block according to an embodiment.

[0132] Reference Figure 5 The merge candidates for the current block 500 may include a first bottom-left block 502 with index 45, a second top-right block 504 with index 66, a third bottom-left block 506 with index 39, a fourth top-right block 508 with index 23, and a fifth top-left block 510 with index 7. Additionally, a sixth time-candidate prediction block (not shown) may exist.

[0133] The number of merge candidates can be fixed. For example, the number of merge candidates can be fixed at five. Among the merge candidates, the number of spatial candidate prediction blocks can be limited to a specific value. For example, if the merge candidates may include four spatial candidate prediction blocks, when there are unusable blocks among the first lower-left block 502 with index 45, the second upper-right block 504 with index 66, the third lower-left block 506 with index 39, and the fourth upper-right block 508 with index 23, the fifth upper-left block 510 with index 7 can be used to replace the unusable blocks in the merge candidate list according to priority order.

[0134] When the Motion Vector Predictor (MVP) is used, when block availability is determined in the order of the first lower-left block 502 with index 45 to the third lower-left block 506 with index 39, the motion vector of the first available block can be used as a candidate predicted motion vector. Next, when block availability is determined in the order of the second upper-right block 504 with index 66, the fourth upper-right block 508 with index 23, and the fifth upper-left block 510 with index 7, the motion vector of the first available block can be selected as a candidate predicted motion vector, and the motion vector of the last time candidate predicted block can be used as a candidate predicted motion vector.

[0135] In one embodiment, the image decoding device 200 (or the image encoding device 100) may determine whether a block is available as a merging candidate based on a block mapping.

[0136] In one embodiment, motion information for the current block can be derived based on merging candidates, and the derived motion information can be used to predict the current block. In one embodiment, the motion information may include information about motion vectors, prediction directions, indexes of reference frames, etc.

[0137] Figure 6 The methods for modifying merged candidates with the same motion information according to various embodiments are illustrated.

[0138] Reference Figure 6 The merge candidates for the current block 600 may include the first lower left block A0 610, the second upper right block B0 620, the third lower left block A1 630, the fourth upper right block B1 640, and the fifth upper left block B2 650. In addition, a sixth time-prospect prediction block (not shown) may exist.

[0139] In one embodiment, there may be a situation where two or more of the spatial merge candidates 610, 620, 630, 640, and 650 of the current block 600 are included in the same coding unit. In this case, the two or more merge candidates belonging to the same coding unit may have the same motion vector. Therefore, at least one of the two or more merge candidates belonging to the same coding unit can be replaced with a new merge candidate. To replace the merge candidate, available merge candidates can be searched using a block map. Thus, in one embodiment, a block map can be used to select a neighboring region as the target to be referenced by the current block 600. In one embodiment, the lower left block A0 610 and the lower left block A1 630 among the spatial merge candidates 610, 620, 630, 640, and 650 of the current block 600 may be included in the same coding unit. In this case, the lower left blocks A0 610 and A1 630, which belong to the same coding unit, have the same motion vector, and at least one of the lower left blocks A0 610 and A1 630, which belong to the same coding unit, can be replaced by a merge candidate with different motion information and located at different positions.

[0140] In one embodiment, when lower left block A0 610 and lower left block A1 630 are included in the same coding unit and have the same motion vector, lower left block A0 610 in the merge candidate can be replaced by block A0'612, which is adjacent to lower left block A0 610. In one embodiment, when lower left block A0 610 and lower left block A1 630 are included in the same coding unit and have the same motion vector, lower left block A1 630 in the merge candidate can be replaced by block A1'632, which is adjacent to lower left block A1 630. That is, at least one of lower left block A0 610 and lower left block A1 630 can be replaced by a block (e.g., block A0'612 or block A1'632) included in another coding unit adjacent to the coding unit to which lower left blocks A0 610 and A1 630 belong and have the same motion vector.

[0141] However, even when replacing at least one of the lower left blocks A0610 or A1630 with a block included in a coding unit different from the coding units to which the lower left blocks A0610 and A1630 belong (e.g., block A0'612 or block A1'632), the motion vector of that block may be the same as that of the lower left blocks A0610 or A1630. In this case, at least one of the lower left blocks A0610 or A1630 may be replaced with a block located most adjacent to the coding units to which the lower left blocks A0610 and A1630 belong and having a different motion vector (e.g., block A1'634).

[0142] In the above embodiments, the replacement merge candidates, block A0'612, block A1'632, or block A1'634, can be blocks set as available regions in the block map. That is, these merge candidates can be blocks that are decoded before the current block 600 is predicted.

[0143] In another embodiment, two or more merge candidates not included in the same coding unit among the spatial merge candidates 610, 620, 630, 640, and 650 and the temporal merge candidates of the current block 600 may have the same motion vector. In this case, at least one of the two or more merge candidates not included in the same coding unit and having the same motion vector can be replaced with a new merge candidate. To replace a merge candidate, available merge candidates can be searched using a block map. Thus, in one embodiment, a block map can be used to select a neighboring region of the target to be referenced by the current block 600.

[0144] In one embodiment, among the merging candidates, the lower left block A0 610 and the lower left block A1 632 belong to different coding units, but may have the same motion vector. In this case, at least one of the lower left blocks A0 610 and A1 632 can be replaced with a block having a motion vector different from that of blocks A0 610 and A1 632 (e.g., block A1 634). In one embodiment, a block with a different motion vector can be selected from blocks of coding units adjacent to either block A0 610 or block A1 632. Embodiments for replacing merge candidates are not limited to those described above, and such neighboring regions of the current block 600 can be changed to merge candidates: these neighboring regions are among blocks in the available areas set as block maps. For example, the merge candidate of the current block 600 can be changed to a merge candidate selected from neighboring blocks adjacent to the current block 600, from blocks including coding units different from the merge candidate of the current block 600, or from blocks adjacent to the current block 600 and most similar to the merge candidate of the current block 600.

[0145] According to the above embodiments, the image decoding device 200 (or the image encoding device 100) can deduce the motion information of the current block 600 based on the merging candidate of the current block 600 changed to the merging candidate, and predict the current block 600 by using the deduced motion information.

[0146] Figure 7 This illustrates methods for merging candidates that, according to various embodiments, cannot be used for referenced regions.

[0147] Reference Figure 7 The merge candidates for the current block 700 may include the first lower left block A0 710, the second upper right block B0 720, the third lower left block A1 730, the fourth upper right block B1 740, and the fifth upper left block B2 750. In addition, a sixth time-prospect prediction block (not shown) may exist.

[0148] In one embodiment, when there is a merge candidate corresponding to an unavailable region of the block mapping among the spatial merge candidates 710, 720, 730, 740 and 750 and the temporal merge candidates of the current block 700, the merge candidate corresponding to the available region of the block mapping can be used to replace the merge candidate corresponding to the unavailable region of the block mapping.

[0149] For example, among the spatial merge candidates 710, 720, 730, 740, and 750 of the current block 700, blocks A0 710, A1 730, B1 740, and B2 750 can correspond to the available areas of the block mapping, while block B0 720 can correspond to the unavailable areas of the block mapping. In this case, among the merge candidates, block B0 720 cannot be referenced by the current block 700 and can therefore be changed to block B0' 722, which corresponds to the available area of ​​the block mapping.

[0150] For example, when block B0'722, a candidate for merging, and block B1'740, another candidate for merging, are included in the same coding unit, block B1'740, which corresponds to the available area of ​​the block map, can be changed to its neighboring block B1'742 to prevent overlap of merging candidates. In this case, similarly, block B1'742 can be selected from the blocks corresponding to the available area of ​​the block map.

[0151] According to the above embodiments, the image decoding device 200 (or the image encoding device 100) can predict the motion information of the current block 700 based on the merging candidate of the current block 700 changed to the merging candidate, and reconstruct the motion information of the current block 700 by using the predicted motion information.

[0152] When the MVP mode is applied, the motion information of the current block relative to the reference frame can be obtained based on the sum of the motion vector difference (MVD) between the current block and the neighboring blocks relative to the reference frame and the motion vector prediction factor (MVP) of the neighboring blocks relative to the reference frame.

[0153] In detail, the image encoding device 100 can predict the motion vector of the current block using motion information of neighboring blocks, and send the difference between the motion vector of the current block and the predicted motion vector together with a reference frame index indicating a reference frame. The image decoding device 200 can predict the motion vector of the current block using motion information of neighboring blocks, and derive the motion vector of the current block using the residual received from the image encoding device 100. The image decoding device 200 can create a predicted block of the current block based on the derived motion vector and the reference frame index information received from the image encoding device 100.

[0154] In one embodiment, when the MVP mode is applied to inter-frame prediction, the method for creating a candidate list of motion vector predictors (MVPs) for the current block can be the same as described above. Figures 5 to 7 The method for creating the merge candidate list is the same as described.

[0155] In one embodiment, the image encoding device 100 or the image decoding device 200 may create an MVP candidate list for the current block. MVP may represent the predicted value of the motion vector of the current block. In this case, the MVP candidate list can be created using the motion vectors of available neighboring blocks adjacent to the current block and / or available blocks in the reference frame that are in the same position as the current block.

[0156] In other words, the availability of blocks that make up the MVP candidate list can be determined based on block mapping. Corresponding blocks of frames reconstructed before the current frame can be identified as regions available without using block mapping.

[0157] Image encoding device 100 selects the best MVP for the current block by applying motion vector competition (MVC) to MVP candidates included in the MVP candidate list. Image encoding device 100 transmits the index of the selected MVP, the motion vector difference (MVD), and the reference frame index to image decoding device 200 in the bitstream. The MVP index indicates the MVP of the current block and can indicate one of the MVP candidates included in the MVP candidate list. The reference frame index indicates the index of the reference frame of the current block and can indicate one of the reference frame candidates included in the reference frame list. MVD is the difference between the motion vector of the current block and the selected MVP. As the MVD becomes smaller, the amount of information to be transmitted can be reduced.

[0158] The image decoding device 200 can receive an MVP index and a reference frame index from the image encoding device 100. Using the received MVP index, the image decoding device 200 can select an MVP for the current block from among the MVP candidates included in the MVP candidate list. Using the received reference frame index, the image decoding device 200 can select a reference frame for the current block from among the reference frame candidates included in the reference frame list.

[0159] The image decoding device 200 can deduce the motion vector of the current block using the selected MVP and a reference frame along with the received MVD. For example, the image decoding device 200 can reconstruct the motion vector of the current block by adding the received MVD to the MVP.

[0160] As mentioned above, block mapping can be used to determine whether neighboring blocks are regions to be referenced for configuring merge candidates or MVP candidates. The location of the neighboring blocks to be referenced can be adaptively changed based on whether neighboring blocks are available.

[0161] Block mapping can be used to create lists of merge candidates or MVP candidates, and is also applicable to various techniques for referencing neighboring regions during encoding and decoding processes. For example, in intra-frame prediction, as described above, block mapping can be used to determine whether neighboring blocks included in the frame and adjacent to the current block can be used for reference.

[0162] Figure 8 This is a flowchart of a method for encoding an image according to an embodiment.

[0163] In operation S810, one or more blocks are determined to be included in the frame. These blocks may be configured as one or more encoding units or one or more transformation units. The blocks may have a square or non-square shape.

[0164] In operation S820, a block map indicating the decoded blocks among the one or more blocks is determined. The block map can be updated whenever each block in the block is decoded. A block map can be created for each maximum coding unit. The block map may include regions corresponding to the one or more blocks, wherein regions corresponding to decoded blocks can be set to available regions, and regions corresponding to undecoded blocks can be set to unavailable regions.

[0165] In operation S830, based on the block mapping, a neighboring region is determined that can be used as a reference for the current block to be encoded in a specific scan order among the one or more blocks. The scan order can be one of various scan orders (such as raster scan order, zigzag scan order, N-shaped scan order, upper right diagonal scan order, horizontal scan order, vertical scan order, etc.), but is not limited to this.

[0166] In operation S840, the current block is encoded based on the neighboring regions that can be used for reference. Image data encoded by at least one coding unit of each maximum coding unit can be sent to the decoding side. Information regarding the block shape and the partitioning type used to partition each coding unit, information regarding the prediction mode of each coding unit, and information regarding the size, shape, and partitioning type of each transform unit can be sent to the decoding side along with the encoded image data.

[0167] Figure 9 This is a flowchart of a method for decoding an image according to an embodiment.

[0168] In operation S910, a bitstream of encoded image is received. In addition to the encoded image data, the bitstream may also include information about the block shape and the block partitioning type used to partition each coding unit, information about the prediction mode of each coding unit, and information about the size, shape, and partitioning type of each transform unit.

[0169] In operation S920, partitioning information obtained from the bitstream is used to determine one or more blocks to be included in the frame. The partitioning information may include information about the block shape and the type of block partitioning used to partition each coding unit. The one or more blocks may have a square shape or a non-square shape.

[0170] In operation S930, a block map indicating the decoded blocks among the one or more blocks is determined. The block map can be updated whenever each block in the block is decoded. A block map can be created for each maximum coding unit. The block map may include regions corresponding to the one or more blocks, wherein regions corresponding to decoded blocks can be set to available regions, and regions corresponding to undecoded blocks can be set to unavailable regions.

[0171] In operation S940, based on the block mapping, a neighboring region that can be used as a reference for the current block that will be decoded in a specific scan order within one or more blocks is determined. The scan order can be one of various scan orders (such as raster scan order, zigzag scan order, N-shaped scan order, upper right diagonal scan order, horizontal scan order, vertical scan order, etc.), but is not limited to this.

[0172] In operation S950, the current block is decoded based on the neighboring regions that can be used for reference.

[0173] In the following text, reference will now be made to Figures 10 to 23 This describes a method for determining data units of an image according to an embodiment.

[0174] Figure 10 The illustration shows the process of determining at least one coding unit by dividing the current coding unit according to an embodiment of the image decoding apparatus 200.

[0175] According to an embodiment, the image decoding apparatus 200 can determine the shape of the coding unit by using block shape information, and determine which shape the coding unit is divided into by using partition shape information. That is, the coding unit partitioning method indicated by the partition shape information can be determined based on the block shape indicated by the block shape information used by the image decoding apparatus 200.

[0176] According to an embodiment, the image decoding device 200 can use block shape information indicating that the current encoding unit has a square shape. For example, the image decoding device 200 can determine, based on the division shape information, whether to divide a square encoding unit, to divide the square encoding unit vertically, to divide the square encoding unit horizontally, or to divide the square encoding unit into four encoding units. (Refer to...) Figure 10When the block shape information of the current coding unit 1000 indicates a square shape, the decoder 220 may, based on the division shape information indicating that no division is to be performed, not divide the coding unit 1010a with the same size as the current coding unit 1000, or may determine the coding units 1010b, 1010c or 1010d based on the division shape information indicating a specific division method.

[0177] Reference Figure 10 According to an embodiment, the image decoding apparatus 200 can determine two coding units 1010b by dividing the current coding unit 1000 vertically based on division shape information indicating division along the vertical direction. The image decoding apparatus 200 can determine two coding units 1010c by dividing the current coding unit 1000 horizontally based on division shape information indicating division along the horizontal direction. The image decoding apparatus 200 can determine four coding units 1010d by dividing the current coding unit 1000 vertically and horizontally based on division shape information indicating division along both the vertical and horizontal directions. However, the division shape into which a square coding unit can be divided is not limited to the shapes described above, and can include any shape indicated by the division shape information. Specific division shapes into which square coding units are divided will now be described in detail through various embodiments.

[0178] Figure 11 The illustration shows the process of determining at least one coding unit by dividing coding units having a non-square shape according to an embodiment of the image decoding apparatus 200.

[0179] According to an embodiment, the image decoding apparatus 200 can use block shape information indicating that the current coding unit has a non-square shape. The image decoding apparatus 200 can determine whether to divide the non-square current coding unit, or whether to divide the non-square current coding unit via a specific method, based on the division shape information. (See also...) Figure 11 When the block shape information of the current encoding unit 1100 or 1150 indicates a non-square shape, the image decoding device 200 may, based on the division shape information indicating that no division is to be performed, not divide the encoding unit 1110 or 1160 with the same size as the current encoding unit 1100 or 1150, or determine the encoding units 1120a, 1120b, 1130a, 1130b, 1130c, 1170a, 1170b, 1180a, 1180b, and 1180c based on the division shape information indicating a specific division method. Specific division methods for dividing non-square-shaped encoding units will now be described in detail through various embodiments.

[0180] According to an embodiment, the image decoding apparatus 200 can determine the shape into which the coding unit is divided by using division shape information, and in this case, the division shape information can indicate the number of at least one coding unit generated as the coding unit is divided. (See also...) Figure 11 When the division shape information indicates that the current coding unit 1110 or 1150 is divided into two coding units, the image decoding device 200 can determine the two coding units 1120a and 1120b or 1170a and 1170b included in the current coding unit 1110 or 1150 by dividing the current coding unit 1110 or 1150 based on the division shape information.

[0181] According to an embodiment, when the image decoding device 200 divides a current coding unit 1100 or 1150 with a non-square shape based on the division shape information, the image decoding device 200 can divide the current coding unit 1100 or 1150 while considering the position of the long side of the current coding unit 1100 or 1150 with a non-square shape. For example, the image decoding device 200 can determine multiple coding units by dividing the current coding unit 1100 or 1150 along the direction of dividing the long side of the current coding unit 1100 or 1150, considering the shape of the current coding unit 1100 or 1150.

[0182] According to an embodiment, when the partitioning shape information indicates that the encoding unit is divided into an odd number of blocks, the image decoding device 200 can determine the odd number of blocks included in the current encoding unit 1100 or 1150. For example, when the partitioning shape information indicates that the current encoding unit 1100 or 1150 is divided into three encoding units, the image decoding device 200 can divide the current encoding unit 1100 or 1150 into three encoding units 1130a, 1130b, and 1130c, or three encoding units 1180a, 1180b, and 1180c. According to an embodiment, the image decoding device 200 can determine the odd number of encoding units included in the current encoding unit 1100 or 1150, and the determined encoding units may not all be the same size. For example, the size of coding unit 1130b or 1180b among the determined odd number of coding units 1130a, 1130b, and 1130c, or coding units 1180a, 1180b, and 1180c, may be different from the size of coding units 1130a and 1130c, or coding units 1180a and 1180c. That is, the coding units that can be determined as the current coding units 1100 or 1150 are divided may have multiple sizes, and in some cases, coding units 1130a, 1130b, and 1130c, or coding units 1180a, 1180b, and 1180c, may have different sizes.

[0183] According to an embodiment, when the shape information indicates that the coding unit is divided into an odd number of blocks, the image decoding device 200 can determine the odd number of coding units included in the current coding unit 1100 or 1150, and furthermore, the image decoding device 200 can set specific restrictions on at least one coding unit among the odd number of coding units generated by the division. (Refer to...) Figure 11 The image decoding apparatus 200 can distinguish the decoding process performed on the central encoding unit 1130b or 1180b among the three encoding units 1130a, 1130b, and 1130c or the three encoding units 1180a, 1180b, and 1180c generated as the current encoding unit 1100 or 1150 is divided, from the other encoding units 1130a and 1130c or other encoding units 1180a or 1180c. For example, the image decoding apparatus 200 can restrict the central encoding unit 1130b or 1180b to no longer be divided like the other encoding units 1130a and 1130c or other encoding units 1180a or 1180c, or to be divided only a specific number of times.

[0184] Figure 12 The image decoding apparatus 200 according to an embodiment is shown to perform a process of dividing encoding units based on at least one of block shape information and segmentation shape information.

[0185] According to an embodiment, the image decoding device 200 can determine, based on at least one of block shape information and partition shape information, whether a first coding unit 1200 having a square shape will be partitioned or not partitioned into a coding unit. According to an embodiment, when the partition shape information indicates that the first coding unit 1200 is partitioned along a horizontal direction, the image decoding device 200 can determine a second coding unit 1210 by partitioning the first coding unit 1200 along a horizontal direction. The terms first coding unit, second coding unit, and third coding unit as used in the embodiment are terms used to indicate the relationship between coding units before and after partitioning. For example, a second coding unit can be determined by partitioning the first coding unit, and a third coding unit can be determined by partitioning the second coding unit. Hereinafter, it will be understood that the relationship between the first and third coding units depends on the aforementioned features.

[0186] According to an embodiment, the image decoding device 200 can determine, based on at least one of block shape information and partition shape information, whether the determined second coding unit 1210 will be divided into or not divided into coding units. (Refer to...) Figure 12The image decoding apparatus 200 can divide a second encoding unit 1210 with a non-square shape, determined by dividing the first encoding unit 1200, into at least one third encoding unit 1220a, 1220b, 1220c, or 1220d based on at least one of block shape information and division shape information, or it can choose not to divide the second encoding unit 1210. The image decoding apparatus 200 can obtain at least one of block shape information and division shape information, and obtain a plurality of second encoding units (e.g., second encoding units 1210) with various shapes by dividing the first encoding unit 1200 based on at least one of the obtained block shape information and division shape information, wherein the second encoding unit 1210 can be divided according to a method of dividing the first encoding unit 1200 based on at least one of block shape information and division shape information.

[0187] According to an embodiment, when the first coding unit 1200 is divided into a second coding unit 1210 based on at least one of block shape information and partition shape information for the first coding unit 1200, the second coding unit 1210 can also be divided into a third coding unit (e.g., third coding units 1220a to 1220d) based on at least one of block shape information and partition shape information for the second coding unit 1210. That is, coding units can be recursively divided based on at least one of partition shape information and block shape information associated with each coding unit. Therefore, square coding units can be determined from non-square coding units, and such square coding units can be recursively divided so that non-square coding units are determined. (Refer to...) Figure 12 When the second coding unit 1210 with a non-square shape is divided, a specific coding unit (e.g., a central coding unit or a square coding unit) among the odd number of third coding units 1220b, 1220c, and 1220d can be recursively divided. According to an embodiment, the third coding unit 1220c with a square shape among the third coding units 1220b to 1220d can be divided horizontally into a plurality of fourth coding units. The fourth coding unit 1240 with a non-square shape among the plurality of fourth coding units can be further divided into a plurality of coding units. For example, the fourth coding unit 1240 with a non-square shape can be divided into an odd number of coding units 1250a, 1250b, and 1250c.

[0188] The following describes methods that can be used to recursively divide coding units through various embodiments.

[0189] According to an embodiment, the image decoding device 200 can determine, based on at least one of block shape information and partition shape information, that each of the third coding units 1220a, 1220b, 1220c, and 1220d will be divided into a coding unit, or that the second coding unit 1210 will not be divided. According to an embodiment, the image decoding device 200 can divide the second coding unit 1210, which has a non-square shape, into an odd number of third coding units 1220b, 1220c, and 1220d. The image decoding device 200 can set specific restrictions on certain third coding units among the third coding units 1220b, 1220c, and 1220d. For example, the image decoding device 200 can restrict the third coding unit 1220c located at the center of the third coding units 1220b, 1220c, and 1220d from being divided again, or from being divided a set number of times. (See also...) Figure 12 The image decoding apparatus 200 may restrict the centrally located third coding unit 1220c, which is included in the third coding units 1220b, 1220c, and 1220d of the second coding unit 1210 having a non-square shape, from being divided into a specific division shape (e.g., divided into four coding units or divided into a shape corresponding to the shape into which the second coding unit is divided), or divided only a specific number of times (e.g., divided only n times, where n > 0). However, such a restriction on the centrally located third coding unit 1220c is merely an example and should not be construed as being limited by these examples, but rather as including various restrictions as long as the centrally located third coding unit 1220c is decoded differently from the other third coding units 1220b and 1220d.

[0190] According to an embodiment, the image decoding device 200 can obtain at least one of block shape information and partition shape information for dividing the current coding unit from a specific position in the current coding unit.

[0191] Figure 13 This illustrates a method by which an image decoding apparatus 200 determines a specific coding unit among an odd number of coding units, according to an embodiment. (Refer to...) Figure 13 At least one of block shape information and partition shape information of the current encoding unit 1300 can be obtained from a sample located at a specific position (e.g., sample 1340 located at the center) among a plurality of samples included in the current encoding unit 1300. However, the specific position in the current encoding unit 1300 for obtaining at least one of the block shape information and partition shape information is not limited to... Figure 13The center position shown in the image can be any position included in the current encoding unit 1300 (e.g., topmost position, bottommost position, leftmost position, rightmost position, top leftmost position, bottom leftmost position, top rightmost position, or bottom rightmost position). The image decoding device 200 can determine whether the current encoding unit is divided into encoding units with various shapes and sizes, or not divided, by obtaining at least one of block shape information and partition shape information from a specific position.

[0192] According to an embodiment, the image decoding apparatus 200 can select one coding unit when the current coding unit is divided into a specific number of coding units. The methods for selecting one of multiple coding units can differ, and details thereof will be described below through various embodiments.

[0193] According to an embodiment, the image decoding device 200 can divide the current encoding unit into multiple encoding units and determine the encoding unit located at a specific position.

[0194] Figure 13 This illustrates a method by which an image decoding apparatus 200 determines a coding unit located at a specific position among an odd number of coding units, according to an embodiment.

[0195] According to an embodiment, the image decoding apparatus 200 can use information indicating the position of each of the odd-numbered coding units to determine the coding unit located at the center among the odd-numbered coding units. (See also...) Figure 13 The image decoding device 200 can determine an odd number of coding units 1320a, 1320b, and 1320c by dividing the current coding unit 1300. The image decoding device 200 can determine the central coding unit 1320b by using information about the positions of the odd number of coding units 1320a, 1320b, and 1320c. For example, the image decoding device 200 can determine the central coding unit 1320b by determining the positions of coding units 1320a, 1320b, and 1320c based on information indicating the positions of specific samples included in coding units 1320a, 1320b, and 1320c. Specifically, the image decoding device 200 can determine the central coding unit 1320b by determining the positions of coding units 1320a, 1320b, and 1320c based on information indicating the positions of the upper left samples 1330a to 1330c in coding units 1320a, 1320b, and 1320c.

[0196] According to an embodiment, the information indicating the positions of the top-left sample points 1330a, 1330b, and 1330c included in encoding units 1320a, 1320b, and 1320c may include information about the positions or coordinates of the encoding units 1320a, 1320b, and 1320c in the frame. According to an embodiment, the information indicating the positions of the top-left sample points 1330a, 1330b, and 1330c included in encoding units 1320a, 1320b, and 1320c may include information indicating the width or height of the encoding units 1320a, 1320b, and 1320c included in the current encoding unit 1300, and such width or height may correspond to information indicating the difference between the coordinates of the encoding units 1320a, 1320b, and 1320c in the frame. In other words, the image decoding device 200 can determine the centered encoding unit 1320b by directly using information about the position or coordinates of the encoding units 1320a, 1320b and 1320c in the image, or by using information about the width or height corresponding to the difference between the encoding unit and its coordinates.

[0197] According to an embodiment, the information indicating the position of the upper left sample point 1330a of the upper encoding unit 1320a can indicate (xa, ya) coordinates, the information indicating the position of the upper left sample point 1330b of the center encoding unit 1320b can indicate (xb, yb) coordinates, and the information indicating the position of the upper left sample point 1330c of the lower encoding unit 1320c can indicate (xc, yc) coordinates. The image decoding apparatus 200 can determine the center encoding unit 1320b by using the coordinates of the upper left samples 1330a, 1330b, and 1330c respectively included in the encoding units 1320a, 1320b, and 1320c. For example, when the coordinates of the top-left samples 1330a, 1330b, and 1330c are arranged in ascending or descending order, among the coding units 1320a, 1320b, and 1320c determined when the current coding unit 1300 is divided, coding unit 1320b with coordinates (xb, yb) of the center sample 1330b can be determined as the center coding unit. However, the coordinates indicating the positions of the top-left samples 1330a to 1330c can be coordinates indicating the absolute positions in the screen, and furthermore, (dxb, dyb) coordinates (i.e., information indicating the relative position of the top-left sample 1330b of the center coding unit 1320b) and (dxc, dyc) coordinates (i.e., information indicating the relative position of the top-left sample 1330c of the bottom coding unit 1320c) can be used based on the position of the top-left sample 1330a of the upper coding unit 1320a. Furthermore, the method of determining a coding unit at a specific location by using the coordinates of the sample points as information indicating the position of the sample points included in the coding unit is not limited to the above methods, and various arithmetic methods that can use the coordinates of the sample points can be used.

[0198] According to an embodiment, the image decoding device 200 can divide the current encoding unit 1300 into a plurality of encoding units 1320a, 1320b, and 1320c, and select encoding units from encoding units 1320a to 1320c according to a specific criterion. For example, the image decoding device 200 can select encoding unit 1320b with a different size among encoding units 1320a, 1320b, and 1320c.

[0199] According to an embodiment, the image decoding device 200 can determine the width or height of encoding units 1320a, 1320b, and 1320c by using (xa, ya) coordinates (i.e., information indicating the position of the upper left sample point 1330a of the upper encoding unit 1320a), (xb, yb) coordinates (i.e., information indicating the position of the upper left sample point 1330b of the center encoding unit 1320b), and (xc, yc) coordinates (i.e., information indicating the position of the upper left sample point 1330c of the lower encoding unit 1320c), respectively. The image decoding device 200 can also determine the dimensions of encoding units 1320a, 1320b, and 1320c by using the coordinates (xa, ya), (xb, yb), and (xc, yc) indicating the positions of the encoding units 1320a, 1320b, and 1320c, respectively.

[0200] According to an embodiment, the image decoding apparatus 200 can determine the width of the upper encoding unit 1320a as xb-xa and the height as yb-ya. According to an embodiment, the image decoding apparatus 200 can determine the width of the center encoding unit 1320b as xc-xb and the height as yc-yb. According to an embodiment, the image decoding apparatus 200 can determine the width or height of the lower encoding unit by using the width and height of the current encoding unit and the width and height of the upper encoding unit 1320a and the center encoding unit 1320b. The image decoding apparatus 200 can determine encoding units with different dimensions from other encoding units based on the determined widths and heights of the encoding units 1320a, 1320b, and 1320c. (Refer to...) Figure 13 The image decoding apparatus 200 can determine a central encoding unit 1320b, which has a size different from that of the upper encoding unit 1320a and the lower encoding unit 1320c, as an encoding unit located at a specific position. However, the process by which the image decoding apparatus 200 determines an encoding unit with a size different from that of other encoding units is merely an example of determining an encoding unit located at a specific position by using the size of the encoding unit determined based on sample point coordinates. Therefore, various processes can be used to determine an encoding unit located at a specific position by comparing the size of the encoding unit determined according to specific sample point coordinates.

[0201] However, the position of the sample points considered for determining the position of the coding unit is not limited to the upper left as described above, and information about the position of any sample points included in the coding unit can be used.

[0202] According to an embodiment, the image decoding device 200 can select a coding unit located at a specific position from an odd number of coding units determined when the current coding unit is divided, while considering the shape of the current coding unit. For example, when the current coding unit has a non-square shape with a width longer than its height, the image decoding device 200 can determine a coding unit located at a specific position along the horizontal direction. In other words, the image decoding device 200 can determine one coding unit among coding units with different positions along the horizontal direction and set a restriction on said coding unit. When the current coding unit has a non-square shape with a height longer than its width, the image decoding device 200 can determine a coding unit located at a specific position along the vertical direction. That is, the image decoding device 200 can determine one coding unit among coding units with different positions along the vertical direction and set a restriction on said coding unit.

[0203] According to an embodiment, the image decoding apparatus 200 can use information indicating the position of each of an even number of coding units to determine the coding unit located at a specific position among the even number of coding units. The image decoding apparatus 200 can determine the even number of coding units by dividing the current coding unit, and determine the coding unit located at the specific position by using information about the positions of the even number of coding units. Detailed processing can be described in... Figure 13 The process of determining the coding unit located at a specific position (e.g., the center position) among an odd number of coding units, as described in the paper, is not provided in detail hereafter.

[0204] According to an embodiment, when a current coding unit having a non-square shape is divided into multiple coding units, specific information about the coding unit located at a specific position during the division process can be used to determine the coding unit located at the specific position among the multiple coding units. For example, the image decoding apparatus 200 can use at least one of block shape information and division shape information stored in the samples included in the central coding unit during the division process to determine the coding unit located at the center among the multiple coding units obtained by dividing the current coding unit.

[0205] Reference Figure 13The image decoding apparatus 200 can divide the current encoding unit 1300 into a plurality of encoding units 1320a, 1320b, and 1320c based on at least one of block shape information and partition shape information, and determine the encoding unit 1320b located at the center among the plurality of encoding units 1320a, 1320b, and 1320c. Furthermore, the image decoding apparatus 200 can determine the encoding unit 1320b located at the center by considering the position of at least one of the obtained block shape information and partition shape information. That is, at least one of the block shape information and partition shape information of the current encoding unit 1300 can be obtained from the sample point 1340 located at the center of the current encoding unit 1300, and when the current encoding unit 1300 is divided into a plurality of encoding units 1320a, 1320b, and 1320c based on at least one of the block shape information and partition shape information, the encoding unit 1320b including the sample point 1340 can be determined as the encoding unit located at the center. However, the information used to determine the centered coding unit is not limited to at least one of block shape information and partition shape information; various types of information can be used when determining the centered coding unit.

[0206] According to an embodiment, specific information for identifying a coding unit located at a specific position can be obtained from specific samples included in the coding unit to be determined. (Refer to...) Figure 13 The image decoding apparatus 200 can use at least one of block shape information and partition shape information obtained from a sample point located at a specific position in the current encoding unit 1300 (e.g., a sample point located at the center of the current encoding unit 1300) to determine the encoding unit located at the specific position (e.g., the encoding unit located at the center of the plurality of encoding units) among the plurality of encoding units 1320a, 1320b, and 1320c determined when the current encoding unit 1300 is partitioned. That is, the image decoding apparatus 200 can determine the sample point located at the specific position considering the block shape of the current encoding unit 1300, and determine the encoding unit 1320b that includes a sample point from the plurality of encoding units 1320a, 1320b, and 1320c determined when the current encoding unit 1300 is partitioned, and set specific constraints on the encoding unit 1320b. (Refer to...) Figure 13 According to an embodiment, the image decoding apparatus 200 may determine a sample 1340 located at the center of the current encoding unit 1300 as a sample from which specific information can be obtained, and set specific restrictions on the encoding unit 1320b including such a sample 1340 during the decoding process. However, the location of the sample from which specific information can be obtained is not limited to the above-mentioned location, and may be a sample located at any location included in the encoding unit 1320b that is determined to have restrictions set.

[0207] According to an embodiment, the location of sample points from which specific information can be obtained can be determined based on the shape of the current encoding unit 1300. According to an embodiment, block shape information can determine whether the shape of the current encoding unit is square or non-square, and the location of sample points from which specific information can be obtained can be determined based on the shape. For example, the image decoding device 200 can determine sample points located at the boundary that bisects at least one of the width and height of the current encoding unit using at least one of information about the width of the current encoding unit and information about the height of the current encoding unit as sample points from which specific information can be obtained. As another example, when the block shape information associated with the current encoding unit indicates a square shape, the image decoding device 200 can determine one of the sample points adjacent to the boundary that bisects the long side of the current encoding unit as a sample point from which specific information can be obtained.

[0208] According to an embodiment, when the current coding unit is divided into multiple coding units, the image decoding apparatus 200 can use at least one of block shape information and partition shape information to determine the coding unit located at a specific position among the multiple coding units. According to an embodiment, the image decoding apparatus 200 can obtain at least one of block shape information and partition shape information from samples located at specific positions included in the coding unit, and can divide the multiple coding units generated as the current coding unit is divided by using at least one of the block shape information and partition shape information obtained from samples located at specific positions in each of the multiple coding units. That is, the coding unit can be recursively divided by using at least one of the block shape information and partition shape information obtained from samples located at specific positions in each coding unit. Since reference has been made above... Figure 12 The process of recursively partitioning the coding unit is described, so its details are not provided here.

[0209] According to an embodiment, the image decoding apparatus 200 can determine at least one coding unit by dividing the current coding unit, and determine the order in which the at least one coding unit is decoded according to a specific block (e.g., the current coding unit).

[0210] Figure 14 This illustrates the order in which multiple coding units are processed when multiple coding units are determined in the image decoding apparatus 200 when dividing the current coding unit, according to an embodiment.

[0211] According to an embodiment, the image decoding device 200 can determine the second encoding units 1410a and 1410b by dividing the first encoding unit 1400 along the vertical direction, determine the second encoding units 1430a and 1430b by dividing the first encoding unit 1410 along the horizontal direction, or determine the second encoding units 1450a, 1450b, 1450c and 1450d by dividing the first encoding unit 1400 along the horizontal and vertical directions according to block shape information and division shape information.

[0212] Reference Figure 14 The image decoding apparatus 200 can determine second encoding units 1410a and 1410b, which are determined by dividing the first encoding unit 1400 along the vertical direction, to be processed along the horizontal direction 1410c. The image decoding apparatus 200 can determine second encoding units 1430a and 1430b, which are determined by dividing the first encoding unit 1400 along the horizontal direction, to be processed along the vertical direction 1430c. The image decoding apparatus 200 can determine second encoding units 1450a, 1450b, 1450c, and 1450d, which are determined by dividing the first encoding unit 1400 along both the vertical and horizontal directions, to be processed in a specific order, wherein, according to the specific order, encoding units in one row are processed, and subsequently, encoding units in the next row are processed (e.g., raster scan order or zigzag order 1450e).

[0213] According to an embodiment, the image decoding device 200 can recursively divide encoding units. (See also...) Figure 14The image decoding device 200 can determine a plurality of second coding units 1410a and 1410b, a plurality of second coding units 1430a and 1430b, or a plurality of second coding units 1450a, 1450b, 1450c, and 1450d by dividing the first coding unit 1400, and recursively divide each of the plurality of second coding units 1410a and 1410b, each of the plurality of second coding units 1430a and 1430b, or each of the plurality of second coding units 1450a, 1450b, 1450c, and 1450d. The method of dividing the plurality of second coding units 1410a and 1410b, the plurality of second coding units 1430a and 1430b, or the plurality of second coding units 1450a, 1450b, 1450c, and 1450d can correspond to the method of dividing the first coding unit 1400. Therefore, each of the plurality of second coding units 1410a and 1410b, each of the plurality of second coding units 1430a and 1430b, or each of the plurality of second coding units 1450a, 1450b, 1450c, and 1450d can be independently divided into a plurality of coding units. (Refer to...) Figure 14 The image decoding device 200 can determine the second encoding units 1410a and 1410b by dividing the first encoding unit 1400 along the vertical direction, and further determine whether each of the second encoding units 1410a and 1410b is divided independently or not divided.

[0214] According to an embodiment, the image decoding device 200 may divide the second encoding unit 1410a located on the left side into third encoding units 1420a and 1420b along the horizontal direction, and may not divide the second encoding unit 1410b located on the right side.

[0215] According to an embodiment, the order in which the coding units are processed can be determined based on the division process of the coding units. In other words, the order in which the divided coding units are processed can be determined based on the order in which the coding units are processed before being divided. The image decoding unit 200 can determine the order in which the third coding units 1420a and 1420b are processed, wherein the third coding units 1420a and 1420b are determined when the second coding unit 1410a located on the left is divided independently of the second coding unit 1410b located on the right. Since the third coding units 1420a and 1420b are determined when the second coding unit 1410a located on the left is divided along the horizontal direction, the third coding units 1420a and 1420b can be processed along the vertical direction 1420c. Furthermore, since the processing order of the second coding unit 1410a located on the left and the second coding unit 1410b located on the right corresponds to the horizontal direction 1410c, the second coding unit 1410b located on the right can be processed after the third coding units 1420a and 1420b included in the second coding unit 1410a located on the left are processed along the vertical direction 1420c. The above description describes the relevant processing for determining the order in which coding units are processed according to the coding units before they are divided. However, such processing is not limited to the above embodiment, and any method can be used to independently process coding units divided into various shapes along a specific order.

[0216] Figure 15 The illustration shows the process of determining to divide the current encoding unit into an odd number of encoding units when the encoding unit cannot be processed by the image decoding device 200 in a specific order, according to an embodiment.

[0217] According to an embodiment, the image decoding apparatus 200 can determine that the current coding unit is divided into an odd number of coding units based on the obtained block shape information and partition shape information. (See also...) Figure 15 A first encoding unit 1500 having a square shape can be divided into second encoding units 1510a and 1510b having non-square shapes, and the second encoding units 1510a and 1510b can be recursively divided into third encoding units 1520a and 1520b, as well as third encoding units 1520c, 1520d, and 1520e. According to an embodiment, the image decoding device 200 can divide the second encoding unit 1510a located on the left side of the second encoding units 1510a and 1510b in a horizontal direction to determine a plurality of third encoding units 1520a and 1520b, and divide the second encoding unit 1510b located on the right side into an odd number of third encoding units 1520c, 1520d, and 1520e.

[0218] According to an embodiment, the image decoding device 200 can determine whether there are encoding units divided into an odd number by determining whether the third encoding units 1520a, 1520b, 1520c, 1520d, and 1520e can be processed in a specific order. (See also...) Figure 15 The image decoding device 200 can determine the third coding units 1520a to 1520e by recursively dividing the first coding unit 1500. The image decoding device 200 can determine whether any one of the first coding unit 1500, the second coding units 1510a and 1510b, and the third coding units 1520a and 1520b and the third coding units 1520c to 1520e is divided into an odd number of coding units based on at least one of block shape information and division shape information. For example, the rightmost second coding unit 1510b among the second coding units 1510a and 1510b can be divided into an odd number of third coding units 1520c to 1520e. The order in which the plurality of coding units included in the first coding unit 1500 are processed can be a specific order (e.g., a zigzag scanning order 1530), and the image decoding device 200 can determine whether the third coding units 1520c, 1520d, and 1520e, which are determined when the second coding unit 1520b located on the right is divided into an odd number, meet the conditions that can be processed according to the specific order.

[0219] According to an embodiment, the image decoding apparatus 200 can determine whether third encoding units 1520a, 1520b, 1520c, 1520d, and 1520e included in the first encoding unit 1500 satisfy a condition that allows them to be processed in the specific order, wherein the condition is related to whether at least one of the width and height of each of the second encoding units 1510a and 1510b is bisected according to the boundaries of the third encoding units 1520a, 1520b, 1520c, 1520d, and 1520e. For example, when the height of the second coding unit 1510a, located on the left and having a non-square shape, is bisected, the third coding units 1520a and 1520b satisfy the condition. However, it can be determined that the third coding units 1520c, 1520d, and 1520e do not satisfy the condition because the boundaries of the third coding units 1520c, 1520d, and 1520e, determined when the second coding unit 1510b on the right is divided into three coding units, do not bisecte the width or height of the second coding unit 1510b on the right. The image decoding apparatus 200 can determine that the scanning order is discontinuous when the condition is not met, and based on the determination, determine that the second coding unit 1510b on the right is divided into an odd number of coding units. According to an embodiment, the image decoding apparatus 200 can set specific restrictions on coding units located at specific positions among the odd number of coding units obtained by dividing the coding units, and since such restrictions or specific positions have been described above through various embodiments, their details are not provided hereafter.

[0220] Figure 16 This illustration shows the process of determining at least one encoding unit when the image decoding apparatus 200 divides the first encoding unit 1600 according to an embodiment. According to the embodiment, the image decoding apparatus 200 may divide the first encoding unit 1600 based on at least one of block shape information and division shape information obtained by the receiver 210. The first encoding unit 1600 having a square shape may be divided into four encoding units having square shapes or multiple encoding units having non-square shapes. For example, referring to… Figure 16When the block shape information indicates that the first coding unit 1600 is a square, and the division shape information indicates that it is divided into non-square coding units, the image decoding device 200 can divide the first coding unit 1600 into multiple non-square coding units. Specifically, when the division shape information indicates that an odd number of coding units have been determined by dividing the first coding unit 1600 along a horizontal or vertical direction, the image decoding device 200 can determine the second coding units 1610a, 1610b, and 1610c obtained by dividing the first coding unit 1600, which has a square shape, along a vertical direction, or the second coding units 1620a, 1620b, and 1620c obtained by dividing the first coding unit 1600 along a horizontal direction, as the odd number of coding units.

[0221] According to an embodiment, the image decoding apparatus 200 can determine whether second encoding units 1610a, 1610b, and 1610c, and second encoding units 1620a, 1620b, and 1620c, included in the first encoding unit 1600, satisfy conditions that allow them to be processed in a specific order. These conditions are related to whether at least one of the width and height of the first encoding unit 1600 is bisected according to the boundaries of the second encoding units 1610a, 1610b, and 1610c, and second encoding units 1620a, 1620b, and 1620c. (Refer to...) Figure 16 Since the boundaries of the second coding units 1610a, 1610b, and 1610c, defined by dividing the first coding unit 1600, which has a square shape, along the vertical direction, do not bisect the width of the first coding unit 1600, it can be determined that the first coding unit 1600 does not meet the condition that it can be processed in a specific order. Furthermore, since the boundaries of the second coding units 1620a, 1620b, and 1620c, defined by dividing the first coding unit 1600, which has a square shape, along the horizontal direction, do not bisect the height of the first coding unit 1600, it can be determined that the first coding unit 1600 does not meet the condition that it can be processed in a specific order. The image decoding apparatus 200 can determine that the scanning order is discontinuous when the conditions are not met, and based on the determination, determine that the first coding unit 1600 is divided into an odd number of coding units. According to an embodiment, the image decoding apparatus 200 may set specific restrictions on the coding units located at specific positions among the odd number of coding units obtained by dividing the coding units, and since such restrictions or specific positions have been described above by various embodiments, their details are not provided hereafter.

[0222] According to an embodiment, the image decoding device 200 can determine encoding units with various shapes by dividing the first encoding unit.

[0223] Reference Figure 16The image decoding device can divide the first encoding unit 1600 with a square shape and the first encoding unit 1630 or 1650 with a non-square shape into encoding units with various shapes.

[0224] Figure 17 The image decoding device 200 can divide a second encoding unit into a shape that is constrained when the second encoding unit, which has a non-square shape and is determined when the first encoding unit 1700 is divided, satisfies certain conditions.

[0225] According to an embodiment, the image decoding device 200 can determine, based on at least one of block shape information and partition shape information obtained through the receiver 210, that a first coding unit 1700 having a square shape is divided into second coding units 1710a and 1710b or second coding units 1720a and 1720b having non-square shapes. The second coding units 1710a and 1710b or second coding units 1720a and 1720b can be divided independently. Therefore, the image decoding device 200 can determine, based on at least one of the block shape information and partition shape information associated with each of the coding units 1710a and 1710b or each of the coding units 1720a and 1720b, that the second coding units 1710a and 1710b are divided into multiple coding units or are not divided. According to an embodiment, the image decoding apparatus 200 can determine third coding units 1712a and 1712b by dividing a second coding unit 1710a, which has a non-square shape on the left side, along a horizontal direction. The second coding unit 1710a is determined when the first coding unit 1700 is divided along a vertical direction. However, when the second coding unit 1710a on the left side is divided along a horizontal direction, the image decoding apparatus 200 can impose the following limitation: the second coding unit 1710a on the right side is not divided along a horizontal direction like the second coding unit 1710b on the left side. When the third coding units 1714a and 1714b are determined when the second coding unit 1710b on the right side is divided along the same direction (i.e., horizontal direction), the third coding units 1712a, 1712b, 1714a, and 1714b are determined when both the second coding unit 1710a on the left side and the second coding unit 1710b on the right side are independently divided along a horizontal direction. However, this is the same result as when the first coding unit 1700 is divided into four second coding units 1730a, 1730b, 1730c and 1730d based on at least one of block shape information and partition shape information, and therefore may be inefficient in image decoding.

[0226] According to an embodiment, the image decoding apparatus 200 can determine third encoding units 1722a and 1722b or third encoding units 1724a and 1724b by dividing second encoding units 1720a or 1720b, which have non-square shapes and are determined when the first encoding unit 1700 is divided in the horizontal direction, along the vertical direction. However, when one of the second encoding units (e.g., the second encoding unit 1720a located at the top) is divided in the vertical direction, for the reasons described above, the image decoding apparatus 200 may set the following limitation: other second encoding units (e.g., the second encoding unit located at the bottom) are not divided in the vertical direction like the second encoding unit 1720a located at the top.

[0227] Figure 18 The image decoding apparatus 200, according to an embodiment, performs a process of dividing encoding units with square shapes when the division shape information cannot be used to indicate that the encoding unit is divided into four square shapes.

[0228] According to an embodiment, the image decoding apparatus 200 can determine second coding units 1810a and 1810b or second coding units 1820a and 1820b by dividing the first coding unit 1800 based on at least one of block shape information and partition shape information. The partition shape information may include information about various shapes into which the coding unit can be divided, but such information about various shapes may not include information for dividing the coding unit into four square coding units. Based on such partition shape information, the image decoding apparatus 200 cannot divide the first coding unit 1800, which has a square shape, into four second coding units 1830a, 1830b, 1830c, and 1830d, which also have square shapes. The image decoding apparatus 200 can determine second coding units 1810a and 1810b or second coding units 1820a and 1820b, which have non-square shapes, based on the partition shape information.

[0229] According to an embodiment, the image decoding device 200 can independently divide each of the second encoding units 1810a and 1810b or each of the second encoding units 1820a and 1820b having non-square shapes. Each of the second encoding units 1810a and 1810b or each of the second encoding units 1820a and 1820b can be divided in a specific order via a recursive method, wherein the recursive method can be a division method corresponding to a method for dividing the first encoding unit 1800 based on at least one of block shape information and partition shape information.

[0230] For example, the image decoding device 200 can determine the third coding units 1812a and 1812b with square shapes by dividing the second coding unit 1810a located on the left side along the horizontal direction, or it can determine the third coding units 1814a and 1814b with square shapes by dividing the second coding unit 1810b located on the right side along the horizontal direction. Furthermore, the image decoding device 200 can determine the third coding units 1816a to 1816d with square shapes by dividing both the second coding unit 1810a located on the left side and the second coding unit 1810b located on the right side along the horizontal direction. In this case, the coding units can be determined in the same manner as when the first coding unit 1800 is divided into four second coding units 1830a to 1830d with square shapes.

[0231] As another example, the image decoding device 200 can determine third coding units 1822a and 1822b with square shapes by dividing the top second coding unit 1820a along the vertical direction, and can determine third coding units 1824a and 1824b with square shapes by dividing the bottom second coding unit 1820b along the vertical direction. Furthermore, the image decoding device 200 can determine third coding units 1826a, 1826b, 1826c, and 1816d with square shapes by dividing both the top second coding unit 1820a and the bottom second coding unit 1820b along the vertical direction. In this case, the coding units can be determined in the same manner as when the first coding unit 1800 is divided into four second coding units 1830a, 1830b, 1830c, and 1830d with square shapes.

[0232] Figure 19 The order in which multiple coding units are processed according to an embodiment can be changed depending on the process of dividing the coding units.

[0233] According to an embodiment, the image decoding apparatus 200 can divide the first encoding unit 1900 based on block shape information and partition shape information. When the block shape information indicates a square shape and the partition shape information indicates that the first encoding unit 1900 is divided along at least one of the horizontal and vertical directions, the image decoding apparatus 200 can divide the first encoding unit 1900 to determine second encoding units 1910a and 1910b, second encoding units 1920a and 1920b, or second encoding units 1930b, 1930c, and 1930d. (See also...) Figure 19The second coding units 1910a and 1910b, or second coding units 1920a and 1920b, which have non-square shapes and are determined when the first coding unit 1900 is divided in a horizontal or vertical direction, can be independently divided based on block shape information and division shape information. For example, the image decoding device 200 can determine the third coding units 1916a, 1916b, 1916c, and 1916d by dividing each of the second coding units 1910a and 1910b generated as the first coding unit 1900 is divided in a vertical direction along a horizontal direction, or by dividing the second coding units 1920a and 1920b generated as the first coding unit 1900 is divided in a vertical direction along a vertical direction. (See above.) Figure 17 The process of dividing the second coding units 1910a and 1910b or the second coding units 1920a and 1920b is described, and therefore its details are not provided.

[0234] According to an embodiment, the image decoding apparatus 200 can process the encoding units in a specific order. (See above for reference.) Figure 14 The characteristics of processing coding units according to a specific order are described, therefore their details are not provided further. (See reference...) Figure 19 The image decoding device 200 can determine four third encoding units 1916a, 1916b, 1916c, and 1916d, or third encoding units 1926a, 1926b, 1926c, and 1926d, each having a square shape, by dividing a first encoding unit 1900 having a square shape. According to an embodiment, the image decoding device 200 can determine the order in which the third encoding units 1916a, 1916b, 1916c, and 1916d, or third encoding units 1926a, 1926b, 1926c, and 1926d, are processed based on how the first encoding unit 1900 is divided.

[0235] According to an embodiment, the image decoding apparatus 200 can determine third coding units 1916a, 1916b, 1916c, and 1916d by dividing second coding units 1910a and 1910b, which are generated as the first coding unit 1900 is divided vertically, along the horizontal direction. The third coding units 1916a, 1916b, 1916c, and 1916d can be processed in the following order 1917: first, the third coding units 1916a and 1916c, which are included in the second coding unit 1910a located on the left side, are processed vertically; then, the third coding units 1916b and 1916d, which are included in the second coding unit 1910b located on the right side, are processed vertically.

[0236] According to an embodiment, the image decoding apparatus 200 can determine the third coding units 1926a to 1926d by dividing the second coding units 1920a and 1920b, which are generated as the first coding unit 1900 is divided horizontally, along the vertical direction. The third coding units 1926a to 1926d can be processed in the following order 1927: first, the third coding units 1926a and 1926b, which are included in the second coding unit 1920a located at the top, are processed horizontally, and then the third coding units 1926c and 1926d, which are included in the second coding unit 1920b located at the bottom, are processed horizontally.

[0237] Reference Figure 19 The third coding units 1916a to 1916d or 1926a to 1926d, which have square shapes, can be determined when both second coding units 1910a and 1910b or both second coding units 1920a and 1920b are divided. The second coding units 1910a and 1910b, determined when the first coding unit 1900 is divided vertically, and the second coding units 1920a and 1920b, determined when the first coding unit 1900 is divided horizontally, are divided with different shapes. However, according to the subsequently determined third coding units 1916a to 1916d and third coding units 1926a to 1926d, the first coding unit 1900 is divided into coding units with the same shape. Therefore, even when coding units with the same shape are determined correspondingly when they are recursively divided based on at least one of block shape information and division shape information through different processes, the image decoding device 200 can process multiple coding units determined with the same shape in different orders.

[0238] Figure 20 The illustration shows the process of determining the depth of a coding unit as the shape and size of the coding unit are changed when multiple coding units are determined as the coding units are recursively divided, according to an embodiment.

[0239] According to an embodiment, the image decoding apparatus 200 can determine the depth of the coding unit based on a specific criterion. For example, the specific criterion may be the length of the long side of the coding unit. When the length of the long side of the current coding unit is divided into 1 / 2n of the length of the long side of the coding unit before the division, it can be determined that the depth of the current coding unit has increased to n times the depth before the division, where n > 0. In the following, the coding unit with the increased depth is referred to as a coding unit with a lower depth.

[0240] Reference Figure 20According to an embodiment, the image decoding device 200 can determine a lower-depth second coding unit 2002 and a third coding unit 2004 by dividing a first coding unit 2000 with a square shape based on block shape information indicating a square shape (e.g., the block shape information may indicate '0:SQUARE'). When the size of the first coding unit 2000 with a square shape is 2N×2N, by dividing it according to 1 / 2 1 The size of the second coding unit 2002, determined by dividing the width and height of the first coding unit 2000, can be N×N. Furthermore, the size of the third coding unit 2004, determined by dividing the width and height of the second coding unit 2002 by half, can be N / 2×N / 2. In this case, the width and height of the third coding unit 2004 are half the size of the first coding unit 2000. 2 Correspondingly, when the depth of the first coding unit 2000 is D, it has 1 / 2 the width and height of the first coding unit 2000. 1 The depth of the second coding unit 2002 can be D+1, and it has half the width and height of the first coding unit 2000. 2 The depth of the third coding unit 2004 can be D+2.

[0241] According to an embodiment, the image decoding device 200 can determine the second encoding unit 2012 or 2022 and the third encoding unit 2014 or 2024 by dividing the first encoding unit 2010 or 2020 having a non-square shape based on block shape information indicating a non-square shape according to the embodiment (e.g., the block shape information may indicate '1:NS_VER' or '2:NS_HOR', where '1:NS_VER' indicates a non-square shape with a height longer than its width, and '2:NS_HOR' indicates a non-square shape with a width longer than its height).

[0242] The image decoding device 200 can determine a second encoding unit (e.g., a second encoding unit 2002, 2012, or 2022) by dividing at least one of the width and height of a first encoding unit 2010 with a size of N×2N. That is, the image decoding device 200 can determine a second encoding unit 2002 with a size of N×N or a second encoding unit 2022 with a size of N×N / 2 by dividing the first encoding unit 2010 along the horizontal direction, or it can determine a second encoding unit 2012 with a size of N / 2×N by dividing the first encoding unit 2010 along both the horizontal and vertical directions.

[0243] According to an embodiment, the image decoding device 200 can determine a second encoding unit (e.g., a second encoding unit 2002, 2012, or 2022) by dividing at least one of the width and height of a first encoding unit 2020 with a size of 2N×N. That is, the image decoding device 200 can determine a second encoding unit 2002 with a size of N×N or a second encoding unit 2012 with a size of N / 2×N by dividing the first encoding unit 2020 along the vertical direction, or it can determine a second encoding unit 2022 with a size of N×N / 2 by dividing the first encoding unit 2020 along both the horizontal and vertical directions.

[0244] According to an embodiment, the image decoding device 200 can determine a third encoding unit (e.g., a third encoding unit 2004, 2014, or 2024) by dividing at least one of the width and height of a second encoding unit 2002 with a size of N×N. That is, the image decoding device 200 can determine a third encoding unit 2004 with a size of N / 2×N / 2 by dividing the second encoding unit 2002 along the vertical and horizontal directions. 2 The third coding unit 2014 of size N / 2 × N / 2, or size N / 2 × N / 2 2 The third coding unit 2024.

[0245] According to an embodiment, the image decoding device 200 can determine a third encoding unit (e.g., a third encoding unit 2004, 2014, or 2024) by dividing at least one of the width and height of a second encoding unit 2012 with a size of N / 2 × N. That is, the image decoding device 200 can determine a third encoding unit 2004 with a size of N / 2 × N / 2 by dividing the second encoding unit 2012 along the horizontal direction. 2 The third coding unit 2024, or the second coding unit 2012, can be divided into units of size N / 2 along the vertical and horizontal directions. 2 The third coding unit 2014 is ×N / 2.

[0246] According to an embodiment, the image decoding device 200 can determine a third encoding unit (e.g., a third encoding unit 2004, 2014, or 2024) by dividing at least one of the width and height of a second encoding unit 2022 with a size of N×N / 2. That is, the image decoding device 200 can determine a third encoding unit 2004 with a size of N / 2×N / 2 by dividing the second encoding unit 2022 along a vertical direction. 2The third coding unit 2014 of size N / 2 can be determined either by dividing the second coding unit 2022 along the vertical and horizontal directions. 2 The third coding unit 2024.

[0247] According to an embodiment, the image decoding apparatus 200 can divide a square-shaped coding unit (e.g., a first coding unit 2000, a second coding unit 2002, or a third coding unit 2004) along a horizontal or vertical direction. For example, a first coding unit 2010 of size N×2N can be determined by dividing the first coding unit 2000 of size 2N×2N along a vertical direction, or a first coding unit 2020 of size 2N×N can be determined by dividing the first coding unit 2000 of size 2N along a horizontal direction. According to an embodiment, when the depth is determined based on the length of the longest side of the coding unit, the depth of the coding unit determined by dividing the first coding unit 2000 of size 2N×2N along a horizontal or vertical direction can be the same as the depth of the first coding unit 2000.

[0248] According to an embodiment, the width and height of the third encoding unit 2014 or 2024 can be half the width and height of the first encoding unit 2010 or 2020. 2 When the depth of the first coding unit 2010 or 2020 is D, the depth of the second coding unit 2012 or 2022 can be D+1, and the depth of the third coding unit 2014 or 2024 can be D+2. The width and height of the second coding unit 2012 or 2022 are half the width and height of the first coding unit 2010 or 2020, and the width and height of the third coding unit 2014 or 2024 are half the width and height of the first coding unit 2010 or 2020. 2 .

[0249] Figure 21 The diagram illustrates a partial index (PID) for distinguishing depth and coding units, which can be determined based on the shape and size of the coding unit, according to an embodiment.

[0250] According to an embodiment, the image decoding device 200 can determine second encoding units of various shapes by dividing a first encoding unit 2100 having a square shape. (Refer to...) Figure 21The image decoding device 200 can determine second encoding units 2102a and 2102b, second encoding units 2104a and 2104b, or second encoding units 2106a, 2106b, 2106c and 2106d by dividing the first encoding unit 2100 along at least one of the vertical and horizontal directions based on the division shape information. In other words, the image decoding device 200 can determine the second encoding units 2102a and 2102b, second encoding units 2104a and 2104b, or second encoding units 2106a, 2106b, 2106c and 2106d based on the division shape information of the first encoding unit 2100.

[0251] According to an embodiment, the depth of the second coding units 2102a and 2102b, second coding units 2104a and 2104b, or second coding units 2106a, 2106b, 2106c and 2106d, determined based on the division shape information of the first coding unit 2100 having a square shape, can be determined based on the length of the long side. For example, since the length of one side of the first coding unit 2100 having a square shape is the same as the length of the long side of the second coding units 2102a and 2102b or second coding units 2104a and 2104b having a non-square shape, the depth of the first coding unit 2100 can be the same as the depth of the second coding units 2102a and 2102b or second coding units 2104a and 2104b having a non-square shape, i.e., D. In other words, when the image decoding device 200 divides the first coding unit 2100 into four second coding units 2106a, 2106b, 2106c and 2106d with square shapes based on the division shape information, the length of one side of the second coding units 2106a, 2106b, 2106c and 2106d with square shapes is 1 / 2 the length of one side of the first coding unit 2100, and the depth of the second coding units 2106a, 2106b, 2106c and 2106d can be D+1, that is, a depth lower than the depth D of the first coding unit 2100.

[0252] According to an embodiment, the image decoding device 200 can divide the first encoding unit 2110 (in the first encoding unit 2010, the height is longer than the width) into a plurality of second encoding units 2112a and 2112b or a plurality of second encoding units 2114a, 2114b and 2114c in the horizontal direction according to the division shape information. According to an embodiment, the image decoding device 200 can also divide the first encoding unit 2120 (in the first encoding unit 2120, the width is longer than the height) into a plurality of second encoding units 2122a and 2122b or a plurality of second encoding units 2114a, 2114b and 2114c in the vertical direction according to the division shape information.

[0253] According to an embodiment, the depth of the second coding units 2112a and 2112b, second coding units 2114a, 2114b and 2114c, second coding units 2122a and 2122b, or second coding units 2124a, 2124b and 2124c, determined based on the division shape information of the first coding unit 2110 or 2120 with a non-square shape, can be determined based on the length of the long side. For example, since the length of one side of the second coding units 2112a and 2112b with a square shape is half the length of the long side of the first coding unit 2110 with a non-square shape (wherein, in the first coding unit 2110, the height is longer than the width), the depth of the second coding units 2112a and 2112b is D+1, that is, a depth lower than the depth D of the first coding unit 2110 with a non-square shape.

[0254] Furthermore, the image decoding device 200 can divide the first encoding unit 2110, which has a non-square shape, into an odd number of second encoding units 2114a, 2114b, and 2114c based on the division shape information. The odd number of second encoding units 2114a, 2114b, and 2114c may include second encoding units 2114a and 2114c with non-square shapes and second encoding unit 2114b with a square shape. In this case, because the length of the longer side of the second encoding units 2114a and 2114c with non-square shapes and the length of one side of the second encoding unit 2114b with a square shape are half the length of one side of the first encoding unit 2110, the depth of the second encoding units 2114a, 2114b, and 2114c can be D+1, that is, a depth lower than the depth D of the first encoding unit 2110. The image decoding device 200 can determine the depth of the encoding unit associated with the first encoding unit 2120, which has a non-square shape (wherein the width of the first encoding unit 2120 is longer than its height), in the same manner as determining the depth of the encoding unit associated with the first encoding unit 2110.

[0255] According to an embodiment, for the operation of determining the PID for distinguishing coding units, when an odd number of coding units do not have the same size, the image decoding device 200 may determine the PID based on the size ratio of the coding units. (Refer to...) Figure 21The central second coding unit 2114b among the odd-numbered second coding units 2114a, 2114b, and 2114c may have the same width as the second coding units 2114a and 2114c, but its height is twice that of the second coding units 2114a and 2114c. That is, the central second coding unit 2114b may include two such units as second coding units 2114a and 2114c. Therefore, when the PID of the central second coding unit 2114b is 1 according to the scanning order, the PID of the next-order second coding unit 2114c may be 3, an increase of 2. In other words, the PID values ​​may be discontinuous. According to an embodiment, the image decoding apparatus 200 may determine whether the odd-numbered coding units have the same size based on the discontinuity of the PID used to distinguish the coding units.

[0256] According to an embodiment, the image decoding apparatus 200 can determine, based on the value of the PID, whether a plurality of coding units, determined as the current coding unit is divided, have a specific division shape. (Refer to...) Figure 21 The image decoding device 200 can determine an even number of second coding units 2112a and 2112b, or an odd number of second coding units 2114a, 2114b, and 2114c, by dividing a first coding unit 2110, which has a rectangular shape and a height greater than its width. The image decoding device 200 can use a PID indicating each coding unit to distinguish multiple coding units. According to an embodiment, the PID can be obtained from a sample point located at a specific position (e.g., the upper left sample point) of each coding unit.

[0257] According to an embodiment, a coding unit located at a specific position can be determined among coding units identified by using a PID used to distinguish coding units. According to an embodiment, when the division shape information of a first coding unit 2110 with a rectangular shape and a height greater than its width indicates that the first coding unit 2110 is divided into three coding units, the image decoding device 200 can divide the first coding unit 2110 into three second coding units 2114a, 2114b, and 2114c. The image decoding device 200 can assign a PID to each of the three second coding units 2114a, 2114b, and 2114c. The image decoding device 200 can compare the PIDs of an odd number of coding units to determine the center coding unit among the coding units. The image decoding device 200 can determine the second coding unit 2114b, which has a PID corresponding to the center value among the coding units, as the center coding unit among these coding units, based on the PIDs of the coding units determined as the first coding unit 2110 is divided. According to an embodiment, when determining the PID for distinguishing encoding units, when the encoding units do not have the same size, the image decoding device 200 may determine the PID based on the size ratio of the encoding units. (Refer to...) Figure 21 The second coding unit 2114b, generated as a result of the division of the first coding unit 2110, may have the same width as the second coding units 2114a and 2114c, but its height may be twice that of the second coding units 2114a and 2114c. In this case, when the PID of the second coding unit 2114b located at the center is 1, the PID of the second coding unit 2114c in the next sequence may be 3, an increase of 2. In this way, when the growth range of the PID increases uniformly but varies, the image decoding device 200 can determine that the current coding unit is divided into multiple coding units (including coding units with different sizes than other coding units). According to an embodiment, when the division shape information indicates division into an odd number of coding units, the image decoding device 200 may divide the current coding unit into multiple coding units, wherein, among the multiple coding units, the coding unit located at a specific position (e.g., the center coding unit) has a different size than other coding units. In this case, the image decoding device 200 can determine the center coding unit with a different size by using the PID of the coding unit. However, the PID and size or position of the encoding unit located at a specific location described above are specified for the purpose of describing the embodiment and should not be interpreted in a limited way. Various PIDs, positions and sizes of the encoding unit can be used.

[0258] According to an embodiment, the image decoding device 200 may use a specific data unit starting from the recursive division of the encoding unit.

[0259] Figure 22 The illustration shows how multiple encoding units are determined based on multiple specific data units included in the image, according to an embodiment.

[0260] According to an embodiment, a specific data unit can be defined as a data unit that begins to be recursively divided into coding units by using at least one of block shape information and partition shape information. That is, the specific data unit may correspond to the highest-depth coding unit used when determining multiple coding units by partitioning the current frame. Hereinafter, for ease of description, the specific data unit will be referred to as a reference data unit.

[0261] According to an embodiment, the reference data unit may indicate a specific size and shape. According to an embodiment, the reference coding unit may include M×N sample points. Here, M and N may be the same and may be integers represented as multiples of 2. That is, the reference data unit may indicate a square shape or a non-square shape, and may later be divided into an integer number of coding units.

[0262] According to an embodiment, the image decoding device 200 can divide the current frame into multiple reference data units. According to an embodiment, the image decoding device 200 can divide the multiple reference data units obtained by dividing the current frame using division information about each reference data unit. Such reference data unit division processing can correspond to division processing using a quadtree structure.

[0263] According to an embodiment, the image decoding device 200 can predetermine the minimum size of reference data units that can be included in the current frame. Therefore, the image decoding device 200 can determine reference data units of various sizes that are equal to or greater than the minimum size, and determine at least one encoding unit based on the determined reference data units by using block shape information and partition shape information.

[0264] Reference Figure 22 The image decoding apparatus 200 may use a reference coding unit 2200 having a square shape, or it may use a reference coding unit 2202 having a non-square shape. According to an embodiment, the shape and size of the reference coding unit may be determined based on various data units (e.g., sequences, frames, stripes, strip segments, and maximum coding units) that may include at least one reference coding unit.

[0265] According to an embodiment, the receiver 210 of the image decoding apparatus 200 can obtain at least one of information about the shape of a reference coding unit and information about the size of a reference coding unit from the bitstream, based on various data units. (The above has been...) Figure 10 The process of dividing the current coding unit 1000 describes the process of determining at least one coding unit included in the reference coding unit 2200 having a square shape, and the above has been described through Figure 11 The process of dividing the current coding unit 1100 or 1150 describes the process of determining at least one coding unit included in the reference coding unit 2202 which has a non-square shape, and therefore its details are not provided hereafter.

[0266] According to an embodiment, in order to determine the size and shape of a reference coding unit based on some data units predetermined based on predetermined conditions, the image decoding apparatus 200 can use a PID for distinguishing the size and shape of the reference coding unit. That is, the receiver 210 can obtain from the bitstream only the PID used to distinguish the size and shape of the reference coding unit from various data units (e.g., sequences, frames, stripes, strip segments, and maximum coding units) that satisfy predetermined conditions (e.g., data units with a size equal to or smaller than the stripe). The image decoding apparatus 200 can determine the size and shape of the reference coding unit based on the data units that satisfy the predetermined conditions by using the PID. When information about the shape and size of the reference coding unit is obtained from the bitstream and used based on data units with relatively small sizes, the bitstream utilization efficiency may be insufficient, and therefore, instead of directly obtaining information about the shape and size of the reference coding unit, only the PID can be obtained and used. In this case, at least one of the size and shape of the reference coding unit corresponding to the PID indicating the size and shape of the reference coding unit can be predetermined. In other words, the image decoding device 200 can select at least one of the dimensions and shapes of the predetermined reference coding units according to the PID, so as to determine at least one of the dimensions and shapes of the reference coding units included in the data units as criteria for obtaining the PID.

[0267] According to embodiments, the image decoding apparatus 200 may use at least one reference coding unit included in a maximum coding unit. That is, the maximum coding unit for dividing the image may include at least one reference coding unit, and the coding unit may be determined when each of the reference coding units is recursively divided. According to embodiments, at least one of the width and height of the maximum coding unit may be an integer multiple of at least one of the width and height of the reference coding unit. According to embodiments, the size of the reference coding unit may be equal to the size of the maximum coding unit, wherein the maximum coding unit is divided n times according to a quadtree structure. In other words, according to various embodiments, the image decoding apparatus 200 may determine the reference coding unit by dividing the maximum coding unit n times according to a quadtree structure, and divide the reference coding unit based on at least one of block shape information and division shape information.

[0268] Figure 23 The processing block shown is used as a criterion for determining the order of reference coding units included in screen 2300, according to an embodiment.

[0269] According to an embodiment, the image decoding apparatus 200 can determine at least one processing block for dividing a screen. A processing block is a data unit that includes at least one reference coding unit for dividing the screen, and the at least one reference coding unit included in the processing block can be determined in a specific order. In other words, the determination order of the at least one reference coding unit determined in each processing block can correspond to one of various orders used to determine the reference coding units, and can vary depending on the processing block. The determination order of the reference coding units determined according to the processing block can be one of various orders (such as raster scan order, zigzag scan order, N-shaped scan order, upper right diagonal scan order, horizontal scan order, and vertical scan order), but should not be interpreted restrictively by the scan order.

[0270] According to an embodiment, the image decoding apparatus 200 can determine the size of at least one processing block included in an image by obtaining information about the size of the processing block. The image decoding apparatus 200 can obtain information about the size of the processing block from a bitstream to determine the size of the at least one processing block included in the image. The size of the processing block can be a specific size of a data unit indicated by the information about the size of the processing block.

[0271] According to an embodiment, the receiver 210 of the image decoding apparatus 200 can obtain information about the size of a processing block from the bitstream based on specific data units. For example, information about the size of a processing block can be obtained from the bitstream in data units of images, sequences, frames, stripes, and strip segments. That is, the receiver 210 can obtain information about the size of a processing block from the bitstream based on such a number of data units, and the image decoding apparatus 200 can determine the size of at least one processing block for dividing a frame by using the obtained information about the size of the processing block, wherein the size of the processing block can be an integer multiple of the size of a reference coding unit.

[0272] According to an embodiment, the image decoding apparatus 200 can determine the sizes of processing blocks 2302 and 2312 included in the frame 2300. For example, the image decoding apparatus 200 can determine the size of the processing block based on information about its size (obtained from the bitstream). See also... Figure 23 According to an embodiment, the image decoding apparatus 200 may determine the horizontal dimensions of processing blocks 2302 and 2312 to be four times the horizontal dimensions of the reference coding unit, and their vertical dimensions to be four times the vertical dimensions of the reference coding unit. The image decoding apparatus 200 may determine the determination order of at least one reference coding unit in at least one processing block.

[0273] According to an embodiment, the image decoding apparatus 200 may determine each of the processing blocks 2302 and 2312 included in the frame 2300 based on the size of the processing blocks, and may determine the determination order of at least one reference coding unit included in each of the processing blocks 2302 and 2312. According to an embodiment, the operation of determining the reference coding unit may include determining the size of the reference coding unit.

[0274] According to an embodiment, the image decoding apparatus 200 can obtain information from a bitstream regarding the determined order of at least one reference coding unit included in at least one processing block, and can determine the determined order of the at least one reference coding unit based on the obtained information. The information regarding the determined order can be defined as the order or direction of the reference coding units within the processing block. That is, the determined order of the reference coding units can be determined independently per processing block.

[0275] According to an embodiment, the image decoding apparatus 200 can obtain information about the determined order of reference coding units from the bitstream based on specific data units. For example, the receiver 210 can obtain information about the determined order of reference coding units from the bitstream based on data units (such as images, sequences, frames, stripes, strip segments, and processing blocks). Since the information about the determined order of reference coding units indicates the determined order of reference coding units in a processing block, the information about the determined order can be obtained according to specific data units comprising an integer number of processing blocks.

[0276] According to an embodiment, the image decoding device 200 may determine at least one reference coding unit based on a determined order.

[0277] According to an embodiment, receiver 210 can obtain information about the determined order of reference coding units from the bitstream as information related to processing blocks 2302 and 2312, and image decoding device 200 can determine the order for determining at least one reference coding unit included in processing blocks 2302 and 2312, and can determine at least one reference coding unit included in frame 2300 according to the determined order of coding units. (Refer to...) Figure 23The image decoding apparatus 200 can determine determination orders 2304 and 2314 of at least one reference coding unit associated with processing blocks 2302 and 2312, respectively. For example, when information about the determination order of reference coding units is obtained according to the processing blocks, the determination orders of the reference coding units associated with processing blocks 2302 and 2312 may be different from each other. When the determination order 2304 associated with processing block 2302 is a raster scan order, the reference coding units included in processing block 2302 can be determined according to the raster scan order. On the other hand, when the determination order 2314 associated with processing block 2312 is the reverse of the raster scan order, the reference coding units included in processing block 2312 can be determined according to the reverse of the raster scan order.

[0278] According to an embodiment, the image decoding apparatus 200 can decode at least one determined reference coding unit. The image decoding apparatus 200 can decode an image based on the reference coding unit determined through the above embodiments. Examples of methods for decoding the reference coding unit may include various methods for decoding an image.

[0279] According to an embodiment, the image decoding apparatus 200 can obtain block shape information indicating the shape of the current coding unit or partition shape information indicating the method of partitioning the current coding unit from the bitstream, and can use the block shape information or partition shape information. The block shape information or partition shape information can be included in the bitstream associated with various data units. For example, the image decoding apparatus 200 can use the block shape information or partition shape information included in sequence parameter sets, picture parameter sets, video parameter sets, strip headers, and strip segment headers. Furthermore, the image decoding apparatus 200 can obtain an index corresponding to the block shape information or partition shape information from the bitstream based on the maximum coding unit, reference coding unit, and processing block, and use the index.

[0280] Although this disclosure has been specifically shown and described with reference to embodiments thereof, those skilled in the art will understand that various changes in form and detail may be made therein without departing from the spirit and scope of this disclosure as defined by the appended claims. The embodiments should be understood in a descriptive sense only and not for limiting purposes. Therefore, the scope of this disclosure is not defined by its detailed description but by the appended claims, and all differences within that scope are to be construed as included in this disclosure.

[0281] Embodiments of this disclosure can be programmed into a computer and implemented in a general-purpose digital computer that runs the program using a computer-readable recording medium. Examples of computer-readable recording media include magnetic storage media (e.g., ROM, floppy disk, hard disk, etc.), optical recording media (e.g., CD-ROM or DVD), etc.

Claims

1. A method for decoding an image using block mapping, the method comprising: Based on the partitioning information of the upper-level block, the upper-level block is divided into multiple blocks along the vertical direction. The multiple blocks include a left block, a current block, and a right block, and the width of the current block is greater than the width of either the left block or the right block. Based on the current block obtained by dividing the upper block along the vertical direction, it is determined that the current block is not divided into two lower blocks along the vertical direction; Obtain partitioning information indicating whether the current block is divided into two or three lower-level blocks along the horizontal direction; When the partitioning information indicates that the current block is divided into two lower-level blocks along the horizontal direction, the current block is divided into a first lower-level block and a second lower-level block along the horizontal direction. When the first lower-level block is decoded, the block mapping is updated with an indication that the first lower-level block can be used as a value for neighboring lower-level blocks; The block mapping is used to decode the second lower-level block. The width of the left block is the same as the width of the right block.

2. A method for encoding an image using block mapping, the method comprising: Based on the partitioning information of the upper-level block, the upper-level block is divided into multiple blocks along the vertical direction. The multiple blocks include a left block, a current block, and a right block, and the width of the current block is greater than the width of either the left block or the right block. Based on the current block obtained by dividing the upper block along the vertical direction, it is determined that the current block is not divided into two lower blocks along the vertical direction; Generate partitioning information indicating whether the current block is divided into two or three lower-level blocks along the horizontal direction; When the partitioning information indicates that the current block is divided into two lower-level blocks along the horizontal direction, the current block is divided into a first lower-level block and a second lower-level block along the horizontal direction. When the first lower-level block is encoded, the block mapping is updated with an indication that the first lower-level block can be used as a value for neighboring lower-level blocks; The block mapping is used to encode the second lower-level block. The width of the left block is the same as the width of the right block.

3. A non-transitory computer-readable medium for recording a bit stream generated by an encoding device, the bit stream comprising: The current block partitioning information, The partitioning information of the current block is obtained through the following operations: Based on the partitioning information of the upper-level block, the upper-level block is divided into multiple blocks along the vertical direction. The multiple blocks include a left block, a current block, and a right block, and the width of the current block is greater than the width of either the left block or the right block. Based on the current block obtained by dividing the upper block along the vertical direction, it is determined that the current block is not divided into two lower blocks along the vertical direction; Generate partitioning information for the current block, wherein the partitioning information indicates whether the current block is divided into two or three lower-level blocks along the horizontal direction; When the partitioning information indicates that the current block is divided into two lower-level blocks along the horizontal direction, the current block is divided into a first lower-level block and a second lower-level block along the horizontal direction. When the first lower-level block is encoded, the block mapping is updated with an indication that the first lower-level block can be used as a value for neighboring lower-level blocks; The block mapping is used to encode the second lower-level block. The width of the left block is the same as the width of the right block.

Images