Method and device for encoding / decoding an image

Abstract

A method for encoding / decoding an image and a device thereof are provided. According to one embodiment, a method for decoding an image is provided, comprising the steps of: obtaining block type information and / or partition type information about a first coding unit included in the image from a bit stream; determining at least one second coding unit included in the first coding unit based on the obtained block type information and / or partition type information; and decoding the image based on the at least one second coding unit, wherein the block type information indicates the first coding unit type, and the partition type information indicates whether the first coding unit is divided into the second coding unit. A coding method corresponding to the decoding method may be provided. In addition, a coding device capable of executing the coding method or a decoding device capable of executing the decoding method may be provided.

Description

[0001] This application is a divisional application of a patent application with application date of October 10, 2016, application number 201680085605.2, and titled “Method and device for encoding / decoding an image”. Technical Field

[0002] The method and apparatus according to the embodiment may encode or decode an image by using various data units included in the image. Background Art

[0003] The image data is encoded using a codec according to a predetermined data compression standard such as Moving Picture Experts Group (MPEG), and then stored in a recording medium or transmitted through a communication channel in the form of a bit stream.

[0004] As hardware capable of reproducing and storing high-resolution or high-quality image content is being developed and provided, the demand for codecs that effectively encode or decode high-resolution or high-definition image content is increasing. The encoded image content can be reproduced by decoding. Recently, methods for effectively compressing such as high-resolution or high-definition image content have been implemented. For example, methods for effectively compressing images are performed by processing images that will be encoded via any method.

[0005] Various data units may be used to compress an image, and a containment relationship may exist between such data units. The data unit may be divided via various methods to determine the size of the data unit used in such image compression, and as the data unit optimized according to the characteristics of the image is determined, the image may be encoded or decoded. Summary of the invention

[0006] Technical issues

[0007] In a conventional compression method, a square coding unit is determined by determining whether to split a coding unit while determining the size of the coding unit included in a picture, and then by performing a recursive splitting process of uniformly splitting the coding unit into four coding units having the same size.

[0008] However, in recent circumstances where the demand for high-resolution images is growing rapidly and the amount of data required for image reproduction is increasing, there is a need to perform efficient image encoding and decoding processing, and at the same time, it has become apparent that the quality of the reconstructed image deteriorates due to the use of coding units having a uniform shape (i.e., a square).

[0009] Technical Solution

[0010] According to an aspect of the embodiment, an image decoding method for decoding an image, the image decoding method comprising: obtaining at least one of block shape information and division shape information about a first coding unit included in the image from a bit stream; determining at least one second coding unit included in the first coding unit based on at least one of the block shape information and the division shape information; and decoding the image based on the at least one second coding unit, wherein the block shape information indicates a shape of the first coding unit, and the division shape information indicates whether the first coding unit is divided into the at least one second coding unit.

[0011] According to one aspect of the embodiment, an image decoding device for decoding an image, the image decoding device comprising: a bitstream acquirer, configured to obtain at least one of block shape information and division shape information about a first coding unit included in the image from a bitstream; a decoder, configured to determine at least one second coding unit included in the first coding unit based on at least one of the block shape information and the division shape information, and decode the image based on the at least one second coding unit, wherein the block shape information indicates the shape of the first coding unit, and the division shape information indicates whether the first coding unit is divided into the at least one second coding unit.

[0012] According to an aspect of the embodiment, an image encoding method for encoding an image, the image encoding method comprising: generating a bit stream including at least one of block shape information and division shape information about a first coding unit included in the image; determining at least one second coding unit included in the first coding unit based on at least one of the block shape information and the division shape information; and encoding the image based on the at least one second coding unit, wherein the block shape information indicates a shape of the first coding unit, and the division shape information indicates whether the first coding unit is divided into the at least one second coding unit.

[0013] According to an aspect of the embodiment, an image encoding device for encoding an image, the image encoding device comprising: a bitstream generator, configured to generate a bitstream including at least one of block shape information and division shape information about a first coding unit included in the image; an encoder, configured to determine at least one second coding unit included in the first coding unit based on at least one of the block shape information and the division shape information, and encode the image based on the at least one second coding unit, wherein the block shape information indicates a shape of the first coding unit, and the division shape information indicates whether the first coding unit is divided into the at least one second coding unit.

[0014] Beneficial Effects

[0015] By using coding units having various shapes while performing encoding and decoding processes on an image, coding units adapted to characteristics of the image can be used, so that efficient image encoding and decoding can be performed and the quality of a reconstructed image can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a block diagram illustrating an image decoding apparatus capable of decoding an image based on at least one of block shape information and division shape information according to an embodiment.

[0017] Figure 2 is a block diagram illustrating an image encoding apparatus capable of encoding an image based on at least one of block shape information and division shape information according to an embodiment.

[0018] Figure 3 A process of determining at least one coding unit when a current coding unit is split according to an embodiment is illustrated.

[0019] Figure 4 A process of determining at least one coding unit when a coding unit having a non-square shape is split according to an embodiment is illustrated.

[0020] Figure 5 A process of splitting a coding unit based on at least one of block shape information and partition shape information according to an embodiment is illustrated.

[0021] Figure 6 A method of determining a predetermined coding unit from among an odd number of coding units according to an embodiment is illustrated.

[0022] Figure 7 Illustrated is an order in which a plurality of coding units are processed when a plurality of coding units are determined when a current coding unit is split, according to an embodiment.

[0023] Figure 8 A process of determining that a current coding unit is split into an odd number of coding units when coding units cannot be processed in a predetermined order according to an embodiment is illustrated.

[0024] Fig. 9 A process of determining at least one coding unit when a first coding unit is split according to an embodiment is illustrated.

[0025] Fig.10 It is shown that according to an embodiment, when a second coding unit having a non-square shape determined when a first coding unit is split satisfies a predetermined condition, the shape of the second coding unit that can be split is limited.

[0026] Fig.11A process of splitting a coding unit having a square shape when division shape information does not indicate that the coding unit having a square shape is split into four coding units having square shapes according to an embodiment is illustrated.

[0027] Fig.12 It is shown that a processing order among a plurality of coding units according to an embodiment may be changed according to a split process of the coding units.

[0028] Fig.13 A process of determining a depth of a coding unit when a shape and a size of a coding unit are changed when a plurality of coding units are determined when the coding unit is recursively split according to an embodiment is illustrated.

[0029] Fig.14 Depth that may be determined according to a shape and a size of a coding unit and a partial index (PID) for distinguishing coding units according to an embodiment are illustrated.

[0030] Fig.15 It is shown that a plurality of coding units according to an embodiment are determined according to a plurality of predetermined data units included in a picture.

[0031] Fig.16 A processing block as a standard in a step of determining an order of determining reference coding units included in a picture according to an embodiment is shown.

[0032] Fig.17 Illustrations of coding units that can be determined for each picture when a combination of shapes for dividing coding units is different for each picture according to an embodiment.

[0033] Fig.18 Various shapes of coding units that may be determined based on division shape information that may be represented in a binary code, according to an embodiment are illustrated.

[0034] Fig.19 Other shapes of coding units that may be determined based on division shape information that may be represented in a binary code according to an embodiment are shown.

[0035] Fig. 20 is a block diagram of an image encoding and decoding system that performs loop filtering.

[0036] Fig.21 An example of filtering units included in a maximum coding unit and filtering performance information of the filtering units according to an embodiment is illustrated.

[0037] Fig. 22 A process of merging or dividing coding units determined according to a predetermined encoding method according to an embodiment is illustrated.

[0038] Fig.23An index according to a Z scan order of coding units according to an embodiment is shown.

[0039] Fig.24 is a diagram illustrating reference samples used for intra prediction of a coding unit according to an embodiment. DETAILED DESCRIPTION

[0040] Optimal Mode

[0041] According to an aspect of the embodiment, an image decoding method for decoding an image, the image decoding method comprising: obtaining at least one of block shape information and division shape information about a first coding unit included in the image from a bit stream; determining at least one second coding unit included in the first coding unit based on at least one of the block shape information and the division shape information; and decoding the image based on the at least one second coding unit, wherein the block shape information indicates a shape of the first coding unit, and the division shape information indicates whether the first coding unit is divided into the at least one second coding unit.

[0042] The determining of the at least one second coding unit may include: determining a shape of the first coding unit from among a square shape and a non-square shape based on block shape information; and determining the at least one second coding unit based on the shape of the first coding unit.

[0043] The determining of the at least one second coding unit may include determining a plurality of second coding units having a plurality of types of sizes based on the division shape information.

[0044] The obtaining of at least one of the block shape information and the partition shape information may include obtaining at least one of the block shape information and the partition shape information from a bitstream associated with a sample at a predetermined position in the first coding unit.

[0045] The image decoding method may further include decoding the image by determining a predetermined second coding unit from among the at least one second coding unit and limiting the number of times the predetermined second coding unit is divided.

[0046] The decoding of the image may include decoding the image by determining a second coding unit at a predetermined position from among the at least one second coding unit and limiting the number of times the second coding unit at the predetermined position is split.

[0047] The decoding of the image may include: determining a second coding unit including a sample at a predetermined position from among the at least one second coding unit; and decoding the image by limiting the number of times the determined second coding unit is divided.

[0048] The image decoding method may further include: determining a reference coding unit by dividing a width and a height of the maximum coding unit; and determining the reference coding unit as the first coding unit.

[0049] The image decoding method may further include: dividing the image according to at least one processing block including at least one maximum coding unit, wherein an order of processing the at least one maximum coding unit included in the at least one processing block may be changed according to the at least one processing block.

[0050] The step of determining at least one second coding unit may include: when the division shape information about the first coding unit indicates that the first coding unit is divided in a vertical direction or a horizontal direction, determining multiple second coding units by dividing the first coding unit in a vertical direction or a horizontal direction, wherein not all of the multiple second coding units are divided in a direction perpendicular to the direction in which the first coding unit is divided.

[0051] The image decoding method may further include determining a depth of each coding unit based on a length of a longer side of the first coding unit and the at least one second coding unit.

[0052] An order of processing at least one third coding unit determined when one of the at least one second coding unit is split may be determined based on a shape in which a second coding unit related to the at least one third coding unit is split.

[0053] According to one aspect of the embodiment, an image decoding device for decoding an image, the image decoding device comprising: a bitstream acquirer, configured to obtain at least one of block shape information and division shape information about a first coding unit included in the image from a bitstream; a decoder, configured to determine at least one second coding unit included in the first coding unit based on at least one of the block shape information and the division shape information, and decode the image based on the at least one second coding unit, wherein the block shape information indicates the shape of the first coding unit, and the division shape information indicates whether the first coding unit is divided into the at least one second coding unit.

[0054] According to an aspect of the embodiment, an image encoding method for encoding an image, the image encoding method comprising: generating a bit stream including at least one of block shape information and division shape information about a first coding unit included in the image; determining at least one second coding unit included in the first coding unit based on at least one of the block shape information and the division shape information; and encoding the image based on the at least one second coding unit, wherein the block shape information indicates a shape of the first coding unit, and the division shape information indicates whether the first coding unit is divided into the at least one second coding unit.

[0055] According to an aspect of the embodiment, an image encoding device for encoding an image, the image encoding device comprising: a bit stream generator, configured to generate a bit stream including at least one of block shape information and division shape information about a first coding unit included in the image; an encoder, configured to determine at least one second coding unit included in the first coding unit based on at least one of the block shape information and the division shape information, and encode the image based on the at least one second coding unit, wherein the block shape information indicates a shape of the first coding unit, and the division shape information indicates whether the first coding unit is divided into the at least one second coding unit.

[0056] Invention Mode

[0057] By referring to the following detailed description of the embodiments and the accompanying drawings, it is easier to understand the advantages and features of one or more embodiments of the present invention and the methods for achieving them. In this regard, the present embodiment may have different forms and should not be interpreted as being limited to the description set forth herein. On the contrary, these embodiments are provided so that this disclosure will be thorough and complete and the concept of the present embodiment will be fully conveyed to those of ordinary skill in the art.

[0058] Hereinafter, terms used in the specification will be briefly defined, and embodiments will be described in detail.

[0059] All terms including descriptive or technical terms used herein should be interpreted as having meanings that are obvious to those of ordinary skill in the art. However, these terms may have different meanings according to the intentions of those of ordinary skill in the art, precedents, or the emergence of new technologies. In addition, some terms may be arbitrarily selected by the applicant, and in this case, the meaning of the selected terms will be described in detail in the detailed description of the present invention. Therefore, the terms used herein must be defined based on the meaning of the terms and the descriptions throughout the specification.

[0060] An expression used in the singular includes the plural expression unless the expression used in the singular has an apparently different meaning in the context.

[0061] When a component "includes" or "contains" an element, unless there is a specific description to the contrary, the component may further include other elements without excluding other elements. In addition, the term "unit" in the embodiment of the present invention represents a software component or a hardware component such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC), and performs a specific function. However, the term "unit" is not limited to software or hardware. "Unit" may be formed to be located in an addressable storage medium, or may be formed to operate one or more processors. Therefore, for example, the term "unit" may refer to a component such as a software component, an object-oriented software component, a class component, and a task component, and may include a process, a function, an attribute, a step, a subroutine, a program code segment, a driver, a firmware, a microcode, a circuit, data, a database, a data structure, a table, an array, or a variable. The function provided by a component and a "unit" may be associated with a smaller number of components and a "unit", or may be divided into additional components and a "unit".

[0062] Hereinafter, the term "image" may refer to a static image such as a still image or a dynamic image such as a moving image (ie, a video itself).

[0063] Hereinafter, a "sample" is data assigned to a sampling position of an image and may represent data as a processing target. For example, a pixel value in an image in a spatial domain or a transform coefficient on a transform domain may be a sample. A unit including at least one sample may be defined as a block.

[0064] Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings. In the following description, well-known functions or constructions are not described in detail to avoid obscuring the embodiments in unnecessary detail.

[0065] Figure 1 is a block diagram of an image decoding apparatus 100 capable of decoding an image based on at least one of block shape information and division shape information according to an embodiment.

[0066] Reference Figure 1 According to an embodiment, the image decoding apparatus 100 may include: a bitstream acquirer 110 for acquiring predetermined information such as division shape information and block shape information from a bitstream; and a decoder 120 for decoding an image by using the acquired information. According to an embodiment, when the bitstream acquirer 110 of the image decoding apparatus 100 acquires at least one of the block shape information and the division shape information, the decoder 120 of the image decoding apparatus 100 may determine at least one coding unit for dividing the image based on at least one of the block shape information and the division shape information.

[0067] According to an embodiment, the decoder 120 of the image decoding apparatus 100 may determine the shape of the coding unit based on the block shape information. For example, the block shape information may include information indicating whether the coding unit is square or non-square. The decoder 120 may determine the shape of the coding unit by using the block shape information.

[0068] According to an embodiment, the decoder 120 may determine a shape according to which the coding unit is divided based on the division shape information. For example, the division shape information may indicate information about a shape of at least one coding unit included in the coding unit.

[0069] According to an embodiment, the decoder 120 may determine whether the coding unit is split according to the division shape information. The division shape information may include information about at least one coding unit included in the coding unit, and when the division shape information indicates that only one coding unit is included in the coding unit or the coding unit is not split, the decoder 120 may determine that the coding unit including the division shape information is not split. When the division shape information indicates that the coding unit is divided into a plurality of coding units, the decoder 120 may divide the coding unit into a plurality of coding units based on the division shape information.

[0070] According to an embodiment, the division shape information may indicate how many coding units the coding unit is to be divided into or the direction in which the coding unit is to be divided. For example, the division shape information may indicate that the coding unit is divided in at least one of a vertical direction and a horizontal direction or that the coding unit is not divided.

[0071] Figure 3 A process of determining at least one coding unit when the image decoding apparatus 100 splits a current coding unit according to an embodiment is illustrated.

[0072] According to an embodiment, the decoder 120 may determine the shape of the coding unit by using the block shape information, or may determine the shape according to which the coding unit is divided by using the partition shape information. In other words, the method of dividing the coding unit indicated by the partition shape information may be determined based on which block shape is indicated by the block shape information used by the decoder 120.

[0073] According to an embodiment, the decoder 120 may use block shape information indicating that the current coding unit has a square shape. For example, the decoder 120 may determine whether to divide the square coding unit, whether to divide the square coding unit vertically, whether to divide the square coding unit horizontally, or whether to divide the square coding unit into four coding units according to the division shape information. Figure 3, when the block shape information of the current coding unit 300 indicates a square shape, the decoder 120 may not divide the coding unit 310a having the same size as the current coding unit 300 according to the division shape information indicating no division, or may determine the coding units 310b to 310d based on the division shape information indicating a predetermined division method.

[0074] Reference Figure 3 According to an embodiment, the decoder 120 may determine two coding units 310b obtained by dividing the current coding unit 300 in the vertical direction based on the division shape information indicating division in the vertical direction. The decoder 120 may determine two coding units 310c obtained by dividing the current coding unit 300 in the horizontal direction based on the division shape information indicating division in the horizontal direction. The decoder 120 may determine four coding units 310d obtained by dividing the current coding unit 300 in the vertical and horizontal directions based on the division shape information indicating division in the vertical and horizontal directions. However, the division shape for dividing the square coding unit may not be limited to the above-mentioned shape and may include various shapes that may be indicated by the division shape information. The predetermined division shape for dividing the square coding unit will be described in detail below through various embodiments.

[0075] Figure 4 A process of determining at least one coding unit when the image decoding apparatus 100 splits a coding unit having a non-square shape according to an embodiment is illustrated.

[0076] According to an embodiment, the decoder 120 may use block shape information indicating that the current coding unit has a non-square shape. The decoder 120 may determine whether to split the current coding unit having a non-square shape, or whether to split the current coding unit having a non-square shape using a predetermined method. Figure 4 , when the block shape information of the current coding unit 400 or 450 indicates a non-square shape, the decoder 120 may not divide the coding unit 410 or 460 having the same size as the current coding unit 400 or 450 according to the division shape information indicating no division, or may determine the coding units 420a, 420b, 430a, 430b, 430c, 470a, 470b, 480a, 480b, and 480c based on the division shape information indicating a predetermined division method. The predetermined division method for dividing the non-square coding unit will be described in detail later through various embodiments.

[0077] According to an embodiment, the decoder 120 may determine the shape in which the coding unit is divided by using block shape information. In this case, the division shape information may indicate the number of at least one coding unit generated when the coding unit is divided. Figure 4 , when the division shape information indicates that the current coding unit 400 or 450 is divided into two coding units, the decoder 120 can determine the two coding units 420a and 420b or 470a and 470b respectively included in the current coding unit 400 or 450 by dividing the current coding unit 400 or 450 based on the division shape information.

[0078] According to an embodiment, when the decoder 120 splits the current coding unit 400 or 450 having a non-square shape based on the division shape information, the decoder 120 may split the current coding unit 400 or 450 having the non-square shape in consideration of the position of the longer side. For example, the decoder 120 may determine a plurality of coding units by splitting the current coding unit 400 or 450 in a direction in which the longer side of the current coding unit 400 or 450 is split in consideration of the shape of the current coding unit 400 or 450.

[0079] According to an embodiment, when the division shape information indicates that the coding unit is divided into an odd number of blocks, the decoder 120 may determine an odd number of coding units included in the current coding unit 400 or 450. For example, when the division shape information indicates that the current coding unit 400 or 450 is divided into three coding units, the decoder 120 may divide the current coding unit 400 or 450 into three coding units 430a to 430c or 480a to 480c. According to an embodiment, the decoder 120 may determine an odd number of coding units included in the current coding unit 400 or 450, wherein the sizes of the determined coding units are different. For example, the size of the coding unit 430b or 480b among the odd number of coding units 430a to 430c or 480a to 480c may be different from the sizes of the coding units 430a and 430c or 480a and 480c. In other words, the coding units that may be determined when the current coding unit 400 or 450 is split may have different types of sizes, and in some cases, an odd number of coding units 430a to 430c or 480a to 480c may have different sizes.

[0080] According to an embodiment, when the division shape information indicates that the coding unit is divided into an odd number of blocks, the decoder 120 may determine an odd number of coding units included in the current coding unit 400 or 450, and further, set a predetermined restriction on at least one of the odd number of coding units generated by dividing the current coding unit 400 or 450. Figure 4, the decoder 120 may decode a coding unit 430b or 480b located in the middle of three coding units 430a to 430c or 480a to 480c generated when the current coding unit 400 or 450 is divided in a different manner from the coding units 430a and 430c or 480a and 480c. For example, the decoder 120 may define that the coding unit 430b or 480b located in the middle is not further divided like the coding units 430a and 430c or 480a and 480c, or is divided only a specific number of times.

[0081] Figure 5 A process of splitting a coding unit based on at least one of block shape information and division shape information by the image decoding apparatus 100 according to an embodiment is illustrated.

[0082] According to an embodiment, the decoder 120 may determine whether to divide the first coding unit 500 having a square shape into coding units based on at least one of the block shape information and the division shape information. According to an embodiment, when the division shape information indicates that the first coding unit 500 is divided in the horizontal direction, the decoder 120 may determine the second coding unit 510 by dividing the first coding unit 500 in the horizontal direction. The terms "first coding unit", "second coding unit" and "third coding unit" according to an embodiment are used in the context of dividing the coding unit. For example, the second coding unit may be determined when the first coding unit is divided, and the third coding unit may be determined when the second coding unit is divided. The relationship between the first to third coding units used later may be understood to follow the above-mentioned sequential characteristics.

[0083] According to an embodiment, the image decoding apparatus 100 may determine whether to split the determined second coding unit 510 into coding units based on at least one of block shape information and division shape information. Figure 5 , the decoder 120 may divide the coding unit 510 having a non-square shape determined by dividing the first coding unit 500 into at least one third coding unit (for example, third coding units 520a to 520d) based on at least one of the block shape information and the division shape information, or may not divide the second coding unit 520. The bitstream acquirer 110 of the image decoding apparatus 100 may obtain at least one of the block shape information and the division shape information, and the decoder 120 may divide the first coding unit 500 to obtain a plurality of second coding units (for example, second coding units 510) having various shapes based on at least one of the block shape information and the division shape information, and the second coding unit 510 may be divided according to the manner in which the first coding unit 500 is divided based on at least one of the block shape information and the division shape information.

[0084] According to an embodiment, when the first coding unit 500 is divided into the second coding unit 510 based on at least one of the block shape information and the division shape information about the first coding unit 500, the second coding unit 510 may also be divided into the third coding unit, for example, the third coding units 520a to 520d, based on at least one of the block shape information and the division shape information about the second coding unit 510. In other words, the coding unit may be recursively divided based on at least one of the block shape information and the division shape information related to the coding unit. Therefore, a square coding unit may be determined from a non-square coding unit, and such a square coding unit may be recursively divided to determine the non-square coding unit. Referring to Figure 5 , a predetermined coding unit (e.g., a coding unit located in the middle or a coding unit having a square shape) among the odd number of third coding units 520b to 520d determined when the second coding unit 510 having a non-square shape is divided may be recursively divided. According to an embodiment, a square third coding unit 520c, which is one of the odd number of third coding units 520b to 520d, may be divided in a horizontal direction to be divided into a plurality of fourth coding units. A non-square fourth coding unit 540, which is one of the plurality of fourth coding units, may be further divided into a plurality of coding units. For example, a non-square fourth coding unit 540 may be further divided into an odd number of coding units 550a to 550c.

[0085] A method for recursively splitting a coding unit will be described below through various embodiments.

[0086] According to an embodiment, the decoder 120 may determine whether to divide each of the third coding units (for example, the third coding units 520a to 520d) into coding units or not to divide the second coding unit 510 based on at least one of the block shape information and the division shape information. According to an embodiment, the decoder 120 may divide the second coding unit 510 having a non-square shape into an odd number of third coding units 520b to 520d. The image decoding device 100 may set a predetermined limitation on a predetermined third coding unit among the odd number of third coding units 520b to 520d. For example, the image decoding device 100 may limit the middle coding unit 520c among the odd number of third coding units 520b to 520d to no longer be divided or to be divided a settable number of times. Reference Figure 5, the image decoding apparatus 100 may define the middle coding unit 520c among the odd number of third coding units 520b to 520d included in the second coding unit 510 having a non-square shape as no longer being divided, being divided into a predetermined division method (e.g., only into four coding units or into a shape corresponding to the shape in which the second coding unit 510 is divided), or being divided only a predetermined number of times (e.g., only n times, where n>0). However, the definition of the middle coding unit 520c is a simple embodiment, and thus the present invention should not be restrictively interpreted as the above-mentioned embodiment, and should be interpreted as the definition including various definitions of decoding the middle coding unit 520c differently from the coding units 520b and 520d.

[0087] According to an embodiment, the image decoding apparatus 100 may obtain at least one of block shape information and partition shape information for partitioning the current coding unit from a predetermined position in the current coding unit.

[0088] Figure 6 A method for determining a predetermined coding unit from an odd number of coding units by the decoder 120 according to an embodiment is shown. Figure 6 , at least one of the block shape information and the partition shape information of the current coding unit 600 may be obtained from a sample located at a predetermined position (e.g., a sample 640 located in the middle) among a plurality of samples included in the current coding unit 600. However, the predetermined position in the current coding unit 600 for obtaining at least one of the block shape information and the partition shape information should not be restrictively interpreted as Figure 6 , but should be interpreted as including various positions (e.g., uppermost position, lowermost position, left position, right position, upper left position, lower left position, upper right position, and lower right position) in the current coding unit 600. The image decoding apparatus 100 may obtain at least one of the block shape information and the division shape information from a predetermined position to determine whether to divide the current coding unit into coding units having various shapes and sizes.

[0089] According to an embodiment, when the current coding unit is divided into a predetermined number of coding units, the image decoding apparatus 100 may select one of the plurality of coding units. The method of selecting one of the plurality of coding units may vary, and a description of such a method will be described below through various embodiments.

[0090] According to an embodiment, the decoder 120 of the image decoding apparatus 100 may divide a current coding unit into a plurality of coding units and determine a coding unit at a predetermined position.

[0091] Figure 6A method of determining, by the image decoding apparatus 100 , a coding unit at a predetermined position from among an odd number of coding units according to an embodiment is illustrated.

[0092] According to an embodiment, the decoder 120 may use information indicating the position of each of the odd-numbered coding units to determine a coding unit located in the middle of the odd-numbered coding units. Figure 6 , the decoder 120 may determine an odd number of coding units 620a to 620c by dividing the current coding unit 600. The decoder 120 may determine the coding unit 620b in the middle by using information about the positions of the odd number of coding units 620a to 620c. For example, the decoder 120 may determine the coding unit 620b in the middle by determining the positions of the coding units 620a to 620c based on information indicating the positions of predetermined samples included in the coding units 620a to 620c. In detail, the decoder 120 may determine the coding unit 620b in the middle by determining the positions of the coding units 620a to 620c based on information indicating the positions of the upper left samples 630a to 630c of the coding units 620a to 620c.

[0093] According to an embodiment, the information indicating the positions of the upper left samples 630a to 630c respectively included in the coding units 620a to 620c may include information about the positions or coordinates of the coding units 620a to 620c in the picture. According to an embodiment, the information indicating the positions of the upper left samples 630a to 630c respectively included in the coding units 620a to 620c may include information indicating the width or height of the coding units 620a to 620c included in the current coding unit 600, wherein the width or height may correspond to the information indicating the difference between the coordinates of the coding units 620a to 620c in the picture. In other words, the image decoding apparatus 100 may determine the coding unit 620b located in the middle by directly using the information about the positions or coordinates of the coding units 620a to 620c in the picture or by using the information about the width or height of the coding units indicating the difference between the coordinates.

[0094] According to an embodiment, the information indicating the position of the upper left sample 630a of the top coding unit 620a may indicate (xa, ya) coordinates, the information indicating the position of the upper left sample 630b of the middle coding unit 620b may indicate (xb, yb) coordinates, and the information indicating the position of the upper left sample 630c of the bottom coding unit 620c may indicate (xc, yc) coordinates. The image decoding apparatus 100 may determine the middle coding unit 620b by using the coordinates of the upper left samples 630a to 630c respectively included in the coding units 620a to 620c. For example, when the coordinates of the upper left samples 630a to 630c are arranged in ascending or descending order, the middle coding unit 620b including (xb, yb) as the coordinate of the upper left sample 630b may be determined as a coding unit located in the middle among the coding units 620a to 620c determined when the current coding unit 600 is split. Here, the coordinates indicating the positions of the upper left samples 630a to 630c may represent coordinates indicating absolute positions in the picture, and (dxb, dyb) coordinates as information indicating the relative position of the upper left sample 630b of the middle coding unit 620b and (dxc, dyc) coordinates as information indicating the relative position of the upper left sample 630c of the bottom coding unit 620c may also be used based on the position of the upper left sample 630a of the top coding unit 620a. In addition, the method of determining the coding unit at a predetermined position by using the coordinates of the samples included in the coding unit as the information indicating the position of the samples should not be restrictively interpreted as the above method, and may be interpreted as various arithmetic methods capable of using the coordinates of the samples.

[0095] According to an embodiment, the image decoding apparatus 100 may divide the current coding unit 600 into a plurality of coding units 620a to 620c and select coding units from the coding units 620a to 620c according to a predetermined criterion. For example, the decoder 120 may select a coding unit 620b having a different size from the coding units 620a to 620c.

[0096] According to an embodiment, the image decoding apparatus 100 may determine the width or height of each of the coding units 620a to 620c by using the (xa, ya) coordinates as information indicating the position of the upper left sample 630a of the top coding unit 620a, the (xb, yb) coordinates as information indicating the position of the upper left sample 630b of the middle coding unit 620b, and the (xc, yc) coordinates as information indicating the position of the upper left sample 630c of the bottom coding unit 620c. The image decoding apparatus 100 may determine the size of each of the coding units 620a to 620c by using the coordinates (xa, ya), (xb, yb), and (xc, yc) indicating the positions of the coding units 620a to 620c.

[0097] According to an embodiment, the image decoding apparatus 100 may determine the width of the top coding unit 620a as xb-xa, and the height of the top coding unit 620a as yb-ya. According to an embodiment, the decoder 120 may determine the width of the middle coding unit 620b as xc-xb, and the height of the middle coding unit 620b as yc-yb. According to an embodiment, the decoder 120 may determine the width or height of the bottom coding unit by using the width or height of the current coding unit, and the width and height of the top coding unit 620a and the middle coding unit 620b. The decoder 120 may determine a coding unit having a size different from other coding units based on the determined width and height of the coding units 620a to 620c. Referring to Figure 6 , the image decoding apparatus 100 may determine the middle coding unit 620b having a size different from the sizes of the top coding unit 620a and the bottom coding unit 620c as the coding unit at the predetermined position. However, since the process of determining the coding unit having a size different from other coding units by the image decoding apparatus 100 is only an embodiment of determining the coding unit at the predetermined position by using the size of the coding unit determined based on the sample coordinates, various processes of determining the coding unit at the predetermined position by comparing the sizes of the coding units determined according to the predetermined sample coordinates may be used.

[0098] However, the position of the sample considered to determine the position of the coding unit should not be limitedly interpreted as the upper left, but may be interpreted as information on the position of any sample included in the coding unit may be used.

[0099] According to an embodiment, considering the shape of the current coding unit, the image decoding device 100 may select a coding unit at a predetermined position from an odd number of coding units determined when the current coding unit is divided. For example, when the current coding unit has a non-square shape in which the width is longer than the height, the decoder 120 may determine the coding unit at a predetermined position in the horizontal direction. In other words, the decoder 120 may determine the coding unit from the coding units having different positions in the horizontal direction, and set a limit on the coding unit. When the current coding unit has a non-square shape in which the height is longer than the width, the decoder 120 may determine the coding unit at a predetermined position in the vertical direction. In other words, the decoder 120 may determine the coding unit from the coding units having different positions in the vertical direction, and set a limit on the coding unit.

[0100] According to an embodiment, the image decoding apparatus 100 may use information indicating the position of each of the even-numbered coding units to determine the coding unit at a predetermined position from the even-numbered coding units. The decoder 120 may determine the even-numbered coding units by dividing the current coding unit, and determine the coding unit at the predetermined position by using the information about the position of the even-numbered coding units. The detailed process may be the same as that described above with reference to Figure 6 The described process of determining a coding unit at a predetermined position (eg, a middle position) from among an odd number of coding units is similar, and thus a description thereof will not be provided.

[0101] According to an embodiment, when a current coding unit having a non-square shape is divided into a plurality of coding units, predetermined information about a coding unit at a predetermined position may be used during a division process to determine the coding unit at the predetermined position from among the plurality of coding units. For example, the decoder 120 of the image decoding apparatus 100 may use at least one of block shape information and division shape information stored in a sample included in an intermediate coding unit during a division process to determine a coding unit located in the middle from among a plurality of coding units obtained by dividing the current coding unit.

[0102] Reference Figure 6 , the decoder 120 of the image decoding apparatus 100 may divide the current coding unit 600 into a plurality of coding units 620a to 620c based on at least one of the block shape information and the division shape information, and determine a coding unit 620b located in the middle from the plurality of coding units 620a to 620c. In addition, the decoder 120 may determine the coding unit 620b located in the middle considering a position where at least one of the block shape information and the division shape information is obtained. In other words, at least one of the block shape information and the division shape information of the current coding unit 600 may be obtained from a sample 640 located in the middle of the current coding unit 600, and when the current coding unit 600 is divided into a plurality of coding units 620a to 620c based on at least one of the block shape information and the division shape information, the coding unit 620b including the sample 640 may be determined as a coding unit located in the middle. However, information for determining a coding unit located in the middle should not be restrictively interpreted as at least one of the block shape information and the division shape information, and various types of information may be used during a process of determining a coding unit located in the middle.

[0103] According to an embodiment, predetermined information for identifying a coding unit at a predetermined position may be obtained from a predetermined sample included in a coding unit to be determined. Figure 6, the decoder 120 may use at least one of the block shape information and the partition shape information obtained from the sample located at a predetermined position in the current coding unit 600 (for example, the sample located in the middle of the current coding unit 600) to determine the coding unit at the predetermined position (for example, the coding unit located in the middle of the multiple coding units) from the multiple coding units 620a to 620c determined when the current coding unit 600 is divided. In other words, the decoder 120 may determine the sample at the predetermined position considering the block shape of the current coding unit 600, and the decoder 120 may determine the coding unit 620b from the multiple coding units 620a to 620c determined when the current coding unit 600 is divided and set a predetermined restriction on it, wherein the coding unit 620b includes such a sample point: the predetermined information (for example, at least one of the block shape information and the partition shape information) is obtained from the sample point. Referring to Figure 6 According to an embodiment, the decoder 120 may determine a sample 640 located in the middle of the current coding unit 600 as a sample from which predetermined information is obtained, and the decoder 120 may set a predetermined position for the coding unit 620b including the sample 640 during the decoding process. However, the position of the sample from which the predetermined information is obtained should not be restrictively interpreted as the above position, and the sample may be interpreted as a sample included in an arbitrary position in the coding unit 620b determined to be limited.

[0104] According to an embodiment, the position of the sample from which the predetermined information is obtained may be determined based on the shape of the current coding unit 600. According to an embodiment, the block shape information may be used to determine whether the shape of the current coding unit is a square or a non-square, and the position of the sample from which the predetermined information is obtained may be determined based on the shape. For example, the decoder 120 may determine a sample located on a boundary that divides the width and height of the current coding unit into two equal parts by using at least one of the information about the width of the current coding unit and the information about the height of the current coding unit as the sample from which the predetermined information is obtained. As another example, when the block shape information about the current coding unit indicates a non-square shape, the decoder 120 may determine one of the samples adjacent to the boundary that divides the longer side of the current coding unit into two equal parts as the sample from which the predetermined information is obtained.

[0105] According to an embodiment, when a current coding unit is divided into a plurality of coding units, the image decoding apparatus 100 may use at least one of block shape information and division shape information to determine a coding unit at a predetermined position from among the plurality of coding units. According to an embodiment, the bitstream acquirer 110 may obtain at least one of block shape information and division shape information from a sample at a predetermined position included in the coding unit, and the decoder 120 may divide the plurality of coding units by using at least one of block shape information and division shape information obtained from a sample at a predetermined position included in each of a plurality of coding units generated when the current coding unit is divided. In other words, the coding unit may be recursively divided by using at least one of block shape information and division shape information obtained from a sample at a predetermined position in each coding unit. Since reference has been made to Figure 5 The recursive partitioning process of the coding unit has been described, so its details are not provided again.

[0106] According to an embodiment, the image decoding apparatus 100 may determine at least one coding unit by splitting a current coding unit, and determine an order in which to decode the at least one coding unit according to a predetermined block (for example, the current coding unit).

[0107] Figure 7 FIG. 1 illustrates an order in which a plurality of coding units are processed when the image decoding apparatus 100 determines a plurality of coding units by splitting a current coding unit according to an embodiment.

[0108] According to an embodiment, the decoder 120 may determine the second coding units 710a and 710b by dividing the first coding unit 700 in a vertical direction, determine the second coding units 730a and 730b by dividing the first coding unit 700 in a horizontal direction, or determine the second coding units 750a to 750d by dividing the first coding unit 700 in vertical and horizontal directions according to the block shape information and the division shape information.

[0109] Reference Figure 7 , the decoder 120 may determine an order such that the second coding units 710a and 710b determined by dividing the first coding unit 700 in the vertical direction are processed in the horizontal direction 710c. The image decoding apparatus 100 may determine the processing order of the second coding units 730a and 730b determined by dividing the first coding unit 700 in the horizontal direction to be the vertical direction 730c. The image decoding apparatus 100 may determine the second coding units 750a to 750d determined by dividing the first coding unit 700 in the vertical and horizontal directions to be processed according to a predetermined order (e.g., a raster scan order or a z scan order 750e), wherein the coding units in one row are processed and then the coding units in the next row are processed.

[0110] According to an embodiment, the image decoding apparatus 100 may recursively divide the coding unit. Figure 7 , the decoder 120 may determine a plurality of coding units 710a, 710b, 730a, 730b, 750a, 750b, 750c, and 750d by dividing the first coding unit 700, and recursively divide each of the determined plurality of coding units 710a, 710b, 730a, 730b, 750a, 750b, 750c, and 750d. The method of dividing the plurality of coding units 710a, 710b, 730a, 730b, 750a, 750b, 750c, and 750d may be similar to the method of dividing the first coding unit 700. Therefore, the plurality of coding units 710a, 710b, 730a, 730b, 750a, 750b, 750c, and 750d may each be independently divided into a plurality of coding units. Referring to Figure 7 , the decoder 120 may determine the second coding units 710a and 710b by dividing the first coding unit 700 in a vertical direction, and further, may independently determine whether to divide each of the second coding units 710a and 710b.

[0111] According to an embodiment, the decoder 120 may split the second coding unit 710a on the left side in a horizontal direction to obtain third coding units 720a and 720b, and may not split the second coding unit 710b on the right side.

[0112] According to an embodiment, the processing order of the coding units may be determined based on the division process of the coding units. In other words, the processing order of dividing the coding units may be determined based on the processing order of the coding units before being divided. The decoder 120 may determine the order in which the third coding units 720a and 720b determined when the second coding unit 710a on the left is divided independently of the second coding unit 710b on the right are processed. Since the third coding units 720a and 720b are determined when the second coding unit 710a on the left is divided in the horizontal direction, the third coding units 720a and 720b may be processed in the vertical direction 720c. In addition, since the order in which the second coding unit 710a on the left and the second coding unit 710b on the right are processed is in the horizontal direction 710c, the third coding units 720a and 720b included in the second coding unit 710a on the left may be processed in the vertical direction 720c, and then the second coding unit 710b on the right may be processed. Since the above description is used to describe the process of determining the processing order according to the coding units before being divided, the process should not be restrictively interpreted as the above embodiment, and various methods of independently processing the coding units divided and determined in various shapes according to a predetermined order may be used.

[0113] Figure 8 A process of determining, by the image decoding apparatus 100 , that a current coding unit is split into an odd number of coding units when coding units cannot be processed in a predetermined order according to an embodiment is illustrated.

[0114] According to an embodiment, the image decoding apparatus 100 may determine that the current coding unit is split into an odd number of coding units based on the block shape information and the division shape information obtained by the bitstream acquirer 110. Figure 8 , the first coding unit 800 having a square shape may be divided into second coding units 810a and 810b having non-square shapes, and the second coding units 810a and 810b may be independently divided into third coding units 820a to 820e. According to an embodiment, the decoder 120 may determine a plurality of third coding units 820a and 820b by dividing the left coding unit 810a in the second coding unit in a horizontal direction, and the right coding unit 810b may be divided into an odd number of third coding units 820c to 820e.

[0115] According to an embodiment, the decoder 120 may determine whether the coding units divided into an odd number exist by determining whether the third coding units 820a to 820e can be processed in a predetermined order. Figure 8 , the decoder 120 may determine the third coding units 820a to 820e by recursively dividing the first coding unit 800. The decoder 120 may determine whether there are coding units divided into an odd number among the first coding unit 800, the second coding units 810a and 810b, and the third coding units 820a to 820e based on at least one of the block shape information and the division shape information. For example, the coding unit located on the right side of the second coding units 810a and 810b may be divided into an odd number of third coding units 820c to 820e. The order in which the plurality of coding units included in the first coding unit 800 are processed may be a predetermined order 830 (e.g., a z-scan order), and the decoder 120 may determine whether the third coding units 820c to 820e determined when the second coding unit 810b on the right side is divided into an odd number satisfy the condition that they can be processed according to the predetermined order.

[0116] According to an embodiment, the image decoding apparatus 100 may determine whether the third coding units 820a to 820e included in the first coding unit 800 satisfy a condition that can be processed according to a predetermined order, wherein the condition is related to whether at least one of the width and height of the second coding units 810a and 810b is divided into two equal parts along the boundary of the third coding units 820a to 820e. For example, the third coding units 820a and 820b determined when the second coding unit 810a on the left side having a non-square shape is divided into two equal parts satisfy the condition, but since the boundary of the third coding units 820c to 820e determined when the second coding unit 810b on the right side is divided into three coding units cannot divide the width or height of the second coding unit 810b on the right side into two equal parts, the third coding units 820c to 820e do not satisfy the condition. In addition, when the condition is not satisfied, the image decoding apparatus 100 may determine that the scanning order is broken, and determine that the second coding unit 810b on the right side is divided into an odd number of coding units based on the determination result. According to an embodiment, when the coding unit is divided into an odd number of coding units, the image decoding device 100 may set a predetermined limitation for a coding unit at a predetermined position among multiple coding units. Since the details about the limitation or predetermined position have been described above through various embodiments, the details are not provided again.

[0117] Fig. 9 1 shows a process of determining at least one coding unit by the image decoding apparatus 100 when the first coding unit 900 is divided according to an embodiment. According to an embodiment, the decoder 120 may divide the first coding unit 900 based on at least one of the block shape information and the division shape information obtained by the bitstream acquirer 110. The first coding unit 900 having a square shape may be divided into four coding units having a square shape or a non-square shape. For example, referring to Fig. 9 , when the block shape information indicates that the first coding unit 900 is a square and the division shape information indicates that the first coding unit 900 is divided into non-square coding units, the decoder 120 may divide the first coding unit 900 into a plurality of non-square coding units. In detail, when the division shape information indicates that the first coding unit 900 is divided in a horizontal or vertical direction to determine an odd number of coding units, the decoder 120 may divide the first coding unit 900 having a square shape into second coding units 910a to 910c determined when the first coding unit 900 is divided in a vertical direction, or second coding units 920a to 920c determined when the first coding unit 900 is divided in a horizontal direction, as an odd number of coding units.

[0118] According to an embodiment, the decoder 120 may determine whether the second coding units 910a to 910c and 920a to 920c included in the first coding unit 900 satisfy a condition that can be processed according to a predetermined order, wherein the condition is related to whether at least one of the width and height of the first coding unit 900 is divided into two equal parts along the boundary of the second coding units 910a to 910c and 920a to 920c. Fig. 9 , since the boundaries of the second coding units 910a to 910c determined when the first coding unit 900 having a square shape is divided in the vertical direction cannot divide the width of the first coding unit 900 into two equal parts, it can be determined that the first coding unit 900 does not meet the condition that can be processed according to the predetermined order. In addition, since the boundaries of the second coding units 920a to 920c determined when the first coding unit 900 having a square shape is divided in the horizontal direction cannot divide the width of the first coding unit 900 into two equal parts, it can be determined that the first coding unit 900 does not meet the condition that can be processed according to the predetermined order. When the condition is not satisfied, the image decoding apparatus 100 determines that the scanning order is broken, and may determine that the first coding unit 900 is divided into an odd number of coding units based on the determination result. According to an embodiment, when the coding unit is divided into an odd number of coding units, the image decoding apparatus 100 may set a predetermined limitation on the coding unit at a predetermined position among the plurality of coding units, and since the details about the limitation or the predetermined position have been described above through various embodiments, the details thereof are not provided.

[0119] According to an embodiment, the image decoding apparatus 100 may determine coding units having various shapes by splitting the first coding unit.

[0120] Reference Fig. 9 , the image decoding apparatus 100 may divide the first coding unit 900 having a square shape and the first coding unit 930 or 950 having a non-square shape into coding units having various shapes.

[0121] Fig.10 It is shown that when a second coding unit having a non-square shape determined when a first coding unit is split satisfies a predetermined condition, a shape of the second coding unit that can be split is limited by the image decoding apparatus 100 according to an embodiment.

[0122] According to an embodiment, the decoder 120 may determine to divide the first coding unit 1000 having a square shape into the second coding units 1010a, 1010b, 1020a, and 1020b having a non-square shape based on at least one of the block shape information and the division shape information obtained by the bitstream acquirer 110. The second coding units 1010a, 1010b, 1020a, and 1020b may be independently divided. Therefore, the decoder 120 may determine to divide or not divide the second coding units 1010a, 1010b, 1020a, and 1020b based on at least one of the block shape information and the division shape information related to each of the second coding units 1010a, 1010b, 1020a, and 1020b. According to an embodiment, the decoder 120 may determine the third coding units 1012a and 1012b by dividing the second coding unit 1010a having a non-square shape and determined on the left side when the first coding unit 1000 is divided in the vertical direction. However, when the second coding unit 1010a on the left is divided in the horizontal direction, the decoder 120 may define the second coding unit 1010b on the right to be not divided in the same horizontal direction as the direction in which the second coding unit 1010a on the left is divided. When the second coding unit 1010b on the right is divided in the same direction and the third coding units 1014a and 1014b are determined, the third coding units 1012a, 1012b, 1014a, and 1014b may be determined when the second coding unit 1010a on the left and the second coding unit 1010b on the right are independently divided in the horizontal direction. However, this is the same as the result that the decoder 120 divides the first coding unit 1000 into four second coding units 1030a to 1030d having a square shape based on at least one of the block shape information and the division shape information, and therefore the efficiency in image decoding may be low.

[0123] According to an embodiment, the decoder 120 may determine the third coding units 1022a, 1022b, 1024a, and 1024b by dividing the second coding unit 1020a or 1020b having a non-square shape and determined when the first coding unit 1000 is divided in the horizontal direction. However, when one of the second coding units (e.g., the top second coding unit 1020a) is divided in the vertical direction, for the above reasons, the decoder 120 may define the other second coding units (e.g., the bottom second coding unit 1020b) not to be divided in the vertical direction like the direction in which the top second coding unit 1020a is divided.

[0124] Fig.11A process of splitting a coding unit by the image decoding apparatus 100 when the division shape information does not indicate that a coding unit having a square shape is split into four coding units having square shapes according to an embodiment is illustrated.

[0125] According to an embodiment, the decoder 120 may determine the second coding units 1110a, 1110b, 1120a, 1120b, etc. by dividing the first coding unit 1100 based on at least one of the block shape information and the division shape information. The division shape information may include information about various shapes into which the coding unit may be divided, but sometimes, the information about the various shapes may not include information for dividing the coding unit into four square coding units. According to such division shape information, the decoder 120 cannot divide the first coding unit 1100 having a square shape into four square second coding units 1130a to 1130d. Based on the division shape information, the decoder 120 may determine the second coding units 1110a, 1110b, 1120a, 1120b, etc. having non-square shapes.

[0126] According to an embodiment, the decoder 120 may independently divide the second coding units 1110a, 1110b, 1120a, 1120b, etc. having a non-square shape. Each of the second coding units 1110a, 1110b, 1120a, 1120b, etc. may be divided in a predetermined order by a recursive method similar to the method of dividing the first coding unit 1100 based on at least one of the block shape information and the division shape information.

[0127] For example, the decoder 120 may determine the third coding units 1112a and 1112b having a square shape by dividing the second coding unit 1110a on the left side in the horizontal direction and determine the third coding units 1114a and 1114b having a square shape by dividing the second coding unit 1110b on the right side in the horizontal direction. In addition, the decoder 120 may determine the third coding units 1116a to 1116d having a square shape by dividing both the second coding unit 1110a on the left side and the second coding unit 1110b on the right side in the horizontal direction. In this case, the coding unit may be determined in the same manner that the first coding unit 1110 is divided into four square second coding units 1130a to 1130d.

[0128] As another example, the decoder 120 may determine the third coding units 1122a and 1122b having a square shape by dividing the top second coding unit 1120a in the vertical direction and determine the third coding units 1124a and 1124b having a square shape by dividing the bottom second coding unit 1120b in the vertical direction. In addition, the decoder 120 may determine the third coding units 1122a, 1122b, 1124a, and 1124b having a square shape by dividing both the top second coding unit 1120a and the bottom second coding unit 1120b in the vertical direction. In this case, the coding unit may be determined in the same manner that the first coding unit 1110 is divided into four square second coding units 1130a to 1130d.

[0129] Fig.12 It is shown that a processing order among a plurality of coding units according to an embodiment may be changed according to a split process of the coding units.

[0130] According to an embodiment, the decoder 120 may divide the first coding unit 1200 based on the block shape information and the division shape information. When the block shape information indicates a square shape and the division shape information indicates that the first coding unit 1200 is divided in at least one of a horizontal direction and a vertical direction, the decoder 120 may divide the first coding unit 1200 to determine a second coding unit (e.g., second coding units 1210a, 1210b, 1220a, 1220b, etc.). Fig.12 , the second coding units 1210a, 1210b, 1220a and 1220b having a non-square shape and determined when the first coding unit 1200 is divided only in the horizontal or vertical direction can be independently divided based on the block shape information and the division shape information about each of the second coding units 1210a, 1210b, 1220a and 1220b. For example, the decoder 120 may determine the third coding units 1216a to 1216d by dividing the second coding units 1210a and 1210b in the horizontal direction, wherein the second coding units 1210a and 1210b are generated when the first coding unit 1200 is divided in the vertical direction, and the decoder 120 may determine the third coding units 1226a to 1226d by dividing the second coding units 1220a and 1220b in the vertical direction, wherein the second coding units 1220a and 1220b are generated when the first coding unit 1200 is divided in the horizontal direction. Fig.10 The division process of the second coding units 1210a, 1210b, 1220a and 1220b has been described, so the details thereof are not provided.

[0131] According to an embodiment, the decoder 120 may process the coding units according to a predetermined order. Figure 7 The characteristics of processing the encoding units according to a predetermined order are described above, and the details thereof are not provided. Fig.12 , the decoder 120 may determine four square third coding units 1216a to 1216d or 1226a to 1226d by dividing the first coding unit 1200 having a square shape. According to an embodiment, the decoder 120 may determine the processing order of the third coding units 1216a to 1216d or 1226a to 1226d according to the shape into which the first coding unit 1200 is divided.

[0132] According to an embodiment, the decoder 120 may determine the third coding units 1216a to 1216d by dividing each of the second coding units 1210a to 1210b in a horizontal direction, wherein the second coding units 1210a and 1210b are generated when the first coding unit 1200 is divided in a vertical direction, and the decoder 120 may process the third coding units 1216a to 1216d according to an order 1217 in which the third coding units 1216a and 1216b included in the second coding unit 1210a on the left are first processed in a vertical direction and then the third coding units 1216c and 1216d included in the second coding unit 1210b on the right are processed in a vertical direction.

[0133] According to an embodiment, the decoder 120 may determine the third coding units 1226a to 1226d by dividing each of the second coding units 1220a to 1220b in a vertical direction, wherein the second coding units 1220a and 1220b are generated when the first coding unit 1200 is divided in a horizontal direction, and the decoder 120 may process the third coding units 1226a to 1226d in an order of first processing the third coding units 1226a and 1226b included in the top second coding unit 1220a in a horizontal direction and then processing the third coding units 1226c and 1226d included in the bottom second coding unit 1220b in a horizontal direction.

[0134] Reference Fig.12, the third coding units 1216a to 1216d and 1226a to 1226d having a square shape may be determined when each of the second coding units 1210a, 1210b, 1220a, and 1220b is divided. The second coding units 1210a and 1210b determined when the first coding unit 1200 is divided in the vertical direction and the second coding units 1220a and 1220b determined when the first coding unit 1200 is divided in the horizontal direction have different shapes, but the first coding unit 1200 is divided into coding units having the same shape according to the third coding units 1216a to 1216d and 1226a to 1226d determined later. Therefore, even when the coding units having the same shape are determined as a result of recursively dividing the coding units through different processes based on at least one of the block shape information and the division shape information, the decoder 120 may process the coding units having the same shape in different orders.

[0135] Fig.13 A process of determining a depth of a coding unit when a shape and a size of a coding unit are changed when a plurality of coding units are determined when the coding unit is recursively split according to an embodiment is illustrated.

[0136] According to an embodiment, the decoder 120 may determine the depth of the coding unit according to a predetermined criterion. For example, the predetermined criterion may be the length of the longer side of the coding unit. When the length of the longer side of the coding unit before being divided is 2n (where n>0) times the length of the longer side of the current coding unit, the decoder 120 may determine that the depth of the current coding unit is n higher than the depth of the coding unit before being divided. Hereinafter, the coding unit with a higher depth will be referred to as a coding unit of a lower depth.

[0137] Reference Fig.13According to an embodiment, the decoder 120 may determine the second coding unit 1302 and the third coding unit 1304 of the lower depth by dividing the first coding unit 1300 having a square shape based on block shape information indicating a square shape (for example, the block shape information may indicate "0: SQUARE"). When the size of the first coding unit 1300 having a square shape is 2N×2N, the second coding unit 1302 determined by dividing the width and height of the first coding unit 1300 into 1 / 21 may have a size of N×N. In addition, the third coding unit 1304 determined by dividing the width and height of the second coding unit 1302 into 1 / 2 may have a size of N / 2×N / 2. In this case, the width and height of the third coding unit 1304 correspond to 1 / 22 times the width and height of the first coding unit 1300. When the depth of the first coding unit 1300 is D, the depth of the second coding unit 1302 can be D+1, where the width and height of the second coding unit 1302 are 1 / 21 times the width and height of the first coding unit 1300, and the depth of the third coding unit 1304 can be D+2, where the width and height of the third coding unit 1304 are 1 / 22 times the width and height of the first coding unit 1300.

[0138] According to an embodiment, the decoder 120 may determine a second coding unit 1312 or 1322 and a third coding unit 1314 or 1324 of a lower depth by dividing a first coding unit 1310 or 1320 having a non-square shape based on block shape information indicating a non-square shape (for example, the block shape information may indicate "1: NS_VER" indicating that the height is longer than the width, or indicate "2: NS_HOR" indicating that the width is longer than the height).

[0139] The decoder 120 may determine the second coding unit (e.g., the second coding units 1302, 1312, 1322, etc.) by dividing at least one of the height and the width of the first coding unit 1310 having a size of N×2N. In other words, the decoder 120 may determine the second coding unit 1302 having a size of N×N or the second coding unit 1322 having a size of N×N / 2 by dividing the first coding unit 1310 in the horizontal direction, or determine the second coding unit 1312 having a size of N / 2×N by dividing the first coding unit 1310 in the horizontal and vertical directions.

[0140] According to an embodiment, the decoder 120 may determine the second coding unit (e.g., the second coding unit 1302, 1312, 1322, etc.) by dividing at least one of the height and the width of the first coding unit 1320 having a size of 2N×N. In other words, the decoder 120 may determine the second coding unit 1302 having a size of N×N or the second coding unit 1312 having a size of N / 2×N by dividing the first coding unit 1320 in the vertical direction, or determine the second coding unit 1322 having a size of N×N / 2 by dividing the first coding unit 1320 in the horizontal and vertical directions.

[0141] According to an embodiment, the decoder 120 may determine a third coding unit (e.g., third coding units 1304, 1314, 1324, etc.) by dividing at least one of the height and the width of the second coding unit 1302 having a size of N×N. In other words, the decoder 120 may determine a third coding unit 1304 having a size of N / 2×N / 2, a third coding unit 1314 having a size of N / 22×N / 2, or a third coding unit 1324 having a size of N / 2×N / 22 by dividing the second coding unit 1302 in horizontal and vertical directions.

[0142] According to an embodiment, the decoder 120 may determine a third coding unit (e.g., third coding units 1304, 1314, 1324, etc.) by dividing at least one of the height and the width of the second coding unit 1312 having a size of N / 2×N. In other words, the decoder 120 may determine the third coding unit 1304 having a size of N / 2×N / 2 or the third coding unit 1324 having a size of N / 2×N / 22 by dividing the second coding unit 1312 in the horizontal direction, or determine the third coding unit 1314 having a size of N / 22×N / 2 by dividing the second coding unit 1312 in the horizontal and vertical directions.

[0143] According to an embodiment, the decoder 120 may determine a third coding unit (e.g., third coding units 1304, 1314, 1324, etc.) by dividing at least one of the height and the width of the second coding unit 1322 having a size of N×N / 2. In other words, the decoder 120 may determine the third coding unit 1304 having a size of N / 2×N / 2 or the third coding unit 1314 having a size of N / 22×N / 2 by dividing the second coding unit 1322 in a vertical direction, or determine the third coding unit 1324 having a size of N / 2×N / 22 by dividing the second coding unit 1322 in horizontal and vertical directions.

[0144] According to an embodiment, the decoder 120 may split the coding units (e.g., the first coding unit 1300, the second coding unit 1302, the third coding unit 1304, etc.) having a square shape in a horizontal or vertical direction. For example, the first coding unit 1300 having a size of 2N×2N may be split in a vertical direction to determine a first coding unit 1310 having a size of N×2N, or in a horizontal direction to determine a first coding unit 1320 having a size of 2N×N. According to an embodiment, when the depth is determined based on the length of the longest side of the coding unit, the depth of the coding unit determined when the first coding unit 1300, the second coding unit 1302, or the third coding unit 1304 is split in a horizontal or vertical direction may be the same as the depth of the first coding unit 1300, the second coding unit 1302, or the third coding unit 1304.

[0145] According to an embodiment, the width and height of the third coding unit 1314 or 1324 may be 1 / 22 times that of the first coding unit 1310 or 1320. When the depth of the first coding unit 1310 or 1320 is D, the depth of the second coding unit 1312 or 1314 may be D+1, wherein the width and height of the second coding unit 1312 or 1314 are 1 / 2 times the width and height of the first coding unit 1310 or 1320, and the depth of the third coding unit 1314 or 1324 may be D+2, wherein the width and height of the third coding unit 1314 or 1324 are 1 / 22 times the width and height of the first coding unit 1310 or 1320.

[0146] Fig.14 Depth that may be determined according to a shape and a size of a coding unit and a partial index (PID) for distinguishing coding units according to an embodiment are illustrated.

[0147] According to an embodiment, the decoder 120 may determine second coding units having various shapes by dividing the first coding unit 1400 having a square shape. Fig.14 , the decoder 120 may determine the second coding units 1402a, 1402b, 1404a, 1404b, 1406a, 1406b, 1406c, and 1406d by dividing the first coding unit 1400 in at least one of the horizontal and vertical directions according to the division shape information. In other words, the decoder 120 may determine the second coding units 1402a, 1402b, 1404a, 1404b, 1406a, 1406b, 1406c, and 1406d based on the division shape information about the first coding unit 1400.

[0148] According to an embodiment, the depths of the second coding units 1402a, 1402b, 1404a, 1404b, 1406a, 1406b, 1406c, and 1406d determined according to the division shape information about the first coding unit 1400 having a square shape may be determined based on the length of the longer side. For example, since the lengths of the longer sides of the second coding units 1402a, 1402b, 1404a, and 1404b having a non-square shape are the same as the length of one side of the first coding unit 1400 having a square shape, the depths of the first coding unit 1400 and the second coding units 1402a, 1402b, 1404a, and 1404b having a non-square shape may be D, that is, the same. On the other hand, when the decoder 120 divides the first coding unit 1400 into four second coding units 1406a to 1406d having a square shape based on the division shape information, since the length of a side of each of the four second coding units 1406a to 1406d having a square shape is 1 / 2 of the length of a side of the first coding unit 1400, the depth of the second coding units 1406a to 1406d can be D+1, that is, one depth lower than the depth D of the first coding unit 1400.

[0149] According to an embodiment, the decoder 120 may divide the first coding unit 1410 into a plurality of second coding units 1412a and 1412b or 1414a, 1414b, and 1414c by dividing the first coding unit 1410 having a height longer than a width in a horizontal direction according to the division shape information. According to an embodiment, the decoder 120 may divide the first coding unit 1420 into a plurality of second coding units by dividing the first coding unit 1420 having a width longer than a height in a vertical direction according to the division shape information.

[0150] According to an embodiment, the depth of the second coding units 1412a, 1412b, 1414a, 1414b, and 1414c determined according to the division shape information about the first coding unit 1410 or 1420 having a non-square shape may be determined based on the length of the longer side. For example, since the length of one side of each of the second coding units 1412a and 1412b having a square shape is 1 / 2 of the length of one side of the first coding unit 1410 having a non-square shape (whose height is longer than its width), the depth of the second coding units 1402a, 1402b, 1404a, and 1404b having a square shape is D+1, that is, one depth lower than the depth D of the first coding unit 1410 having a non-square shape.

[0151] In addition, the decoder 120 may divide the first coding unit 1410 having a non-square shape into an odd number of second coding units 1414a to 1414c based on the division shape information. The odd number of second coding units 1414a to 1414c may include second coding units 1414a and 1414c having a non-square shape and a second coding unit 1414b having a square shape. Here, since the length of the longer sides of the second coding units 1414a and 1414c having non-square shapes and the length of one side of the second coding unit 1414b having a square shape are 1 / 2 of the length of one side of the first coding unit 1410, the depths of the second coding units 1414a to 1414c may be D+1, that is, one depth lower than the depth D of the first coding unit 1410. The decoder 120 may determine the depth of the coding unit associated with the first coding unit 1420 having a non-square shape (whose width is longer than its height) in a manner similar to determining the depth of the coding unit associated with the first coding unit 1410.

[0152] According to an embodiment, while determining the PID for distinguishing the coding units, the decoder 120 may determine the PID based on a size ratio between the coding units when an odd number of coding units do not have the same size. Fig.14 , the coding unit 1414b located in the middle of the odd number of coding units 1414a to 1414c has the same width as the coding units 1414a and 1414c, but the height is twice the height of the coding units 1414a and 1414c. In this case, the coding unit 1414b located in the middle may include two coding units 1414a or two coding units 1414c. Therefore, when the PID of the coding unit 1414b located in the middle according to the scanning order is 1, the PID of the coding unit 1414c located in the next order may increase by 2, that is, 3. In other words, the value of the PID may be discontinuous. According to an embodiment, the decoder 120 may determine whether the odd number of coding units have the same size based on the discontinuity of the PID used to distinguish the coding units.

[0153] (Determination using three divisions of PID)

[0154] According to an embodiment, the image decoding apparatus 100 may determine whether a plurality of coding units determined when a current coding unit is split has a specific split shape based on a value of a PID for distinguishing coding units. Fig.14, the image decoding apparatus 100 may determine an even number of coding units 1412a and 1412b or an odd number of coding units 1414a to 1414c by dividing the first coding unit 1410 having a rectangular shape whose height is longer than its width. The image decoding apparatus 100 may use an ID indicating each coding unit to distinguish a plurality of coding units. According to an embodiment, the PID may be obtained from a sample at a predetermined position (e.g., an upper left sample) of each coding unit.

[0155] According to an embodiment, the image decoding apparatus 100 may determine a coding unit at a predetermined position from among coding units determined through division by using a PID for distinguishing coding units. According to an embodiment, when division shape information about a first coding unit 1410 having a rectangular shape (whose height is longer than its width) indicates division into three coding units, the image decoding apparatus 100 may divide the first coding unit 1410 into three coding units 1414a to 1414c. The image decoding apparatus 100 may assign a PID to each of the three coding units 1414a to 1414c. The image decoding apparatus 100 may compare the PIDs of the coding units to determine an intermediate coding unit from an odd number of coding units. The image decoding apparatus 100 may determine, based on the PIDs of the coding units, a coding unit 1414b having a PID corresponding to an intermediate value among a plurality of PIDs as a coding unit located in the middle among a plurality of coding units determined when the first coding unit 1410 is divided. According to an embodiment, while determining the PID for distinguishing coding units, the image decoding apparatus 100 may determine the PID based on a size ratio between coding units when the coding units do not have the same size. Referring to Fig.14, the coding unit 1414b generated when the first coding unit 1410 is divided may have the same width as the coding units 1414a and 1414c, but the height may be twice the height of the coding units 1414a and 1414c. In this case, when the PID of the coding unit 1414b located in the middle is 1, the PID of the coding unit 1414c located in the next order may be increased by 2, that is, 3. In this way, when the range change is increased while the PID is uniformly increased, the image decoding apparatus 100 may determine that the coding unit is divided into a plurality of coding units including coding units having sizes different from other coding units. According to an embodiment, when the division shape information indicates division into an odd number of coding units, the image decoding apparatus 100 may divide the current coding unit into an odd number of coding units, wherein the coding unit (e.g., the middle coding unit) at a predetermined position in the odd number of coding units has a size different from other coding units. In this case, the image decoding apparatus 100 may determine the middle coding units having different sizes by using the PIDs of the coding units. However, since the PID and the size or position of the encoding unit at a predetermined position are designated to describe the embodiment, the embodiment is not limited thereto, and various PIDs and various positions and sizes of the encoding unit may be used.

[0156] According to an embodiment, the decoder 120 may use a predetermined data unit from which the coding unit is recursively divided.

[0157] Fig.15 It is shown that a plurality of coding units according to an embodiment are determined according to a plurality of predetermined data units included in a picture.

[0158] According to an embodiment, a predetermined data unit may be defined as a data unit from which a coding unit is recursively divided by using at least one of block shape information and division shape information. In other words, the predetermined data unit may correspond to a coding unit of the highest depth used while determining a plurality of coding units for dividing a current picture. Hereinafter, for convenience of description, such a predetermined data unit is referred to as a reference data unit.

[0159] According to an embodiment, the reference data unit may indicate a predetermined size and shape. According to an embodiment, the reference data unit may include M×N samples. Here, M and N may be the same and may be integers expressed as multiples of 2. In other words, the reference data unit may indicate a square shape or a non-square shape and may then be divided into an integer number of coding units.

[0160] According to an embodiment, the decoder 120 of the image decoding apparatus 100 may divide the current picture into a plurality of reference data units. According to an embodiment, the decoder 120 may divide the plurality of reference data units by using division information about each of the plurality of reference data units obtained by dividing the current picture. Such a division process of the reference data unit may correspond to a division process using a quadtree structure.

[0161] According to an embodiment, the decoder 120 may predetermine a minimum size of a reference data unit that can be included in the current picture. Therefore, the decoder 120 may determine reference data units of various sizes equal to or larger than the minimum size, and determine at least one coding unit based on the determined reference data unit by using the block shape information and the partition shape information.

[0162] refer to Fig.15 , the image decoding apparatus 100 may use a reference coding unit 1500 having a square shape, or may use a reference coding unit 1502 having a non-square shape. According to an embodiment, the shape and size of the reference coding unit may be determined according to various data units (e.g., a sequence, a picture, a slice, a slice segment, and a maximum coding unit) that may include at least one reference coding unit.

[0163] According to an embodiment, the bitstream acquirer 110 of the image decoding apparatus 100 may obtain at least one of information about the shape of the reference coding unit and information about the size of the reference coding unit from the bitstream according to various data units. Figure 3 The process of dividing the current coding unit 300 of the reference coding unit 1500 having a square shape describes the process of determining at least one coding unit included in the reference coding unit 1500 having a square shape, and the above has been described by Figure 4 The process of splitting the current coding unit 400 or 450 describes a process of determining at least one coding unit included in the reference coding unit 1500 having a non-square shape, and thus the details thereof will not be provided.

[0164] According to an embodiment, in order to determine the size and shape of the reference coding unit according to some data units predetermined based on a predetermined condition, the decoder 120 may use a PID for distinguishing the reference coding unit. In other words, the bitstream acquirer 110 may obtain only a PID for distinguishing the size and shape of the reference coding unit from the bitstream according to a slice, a slice segment, and a maximum coding unit, wherein the reference coding unit is a data unit (e.g., a data unit having a size equal to or smaller than a slice) among various data units (e.g., a sequence, a picture, a slice, a slice segment, and a maximum coding unit) that satisfies a predetermined condition. The decoder 120 may determine the size and shape of the reference data unit according to the data unit that satisfies the predetermined condition by using the PID. When information about the shape of the reference coding unit and information about the size of the reference coding unit are obtained from the bitstream and used according to a data unit having a relatively small size, the use efficiency of the bitstream may be insufficient, and therefore, the information about the shape of the reference coding unit and the information about the size of the reference coding unit are not directly obtained, but only the PID may be obtained and used. In this case, at least one of the size and shape of the reference coding unit related to the PID indicating the size and shape of the reference coding unit may be predetermined. In other words, the decoder 120 may select at least one of the size and shape of the predetermined reference coding unit according to the PID to determine at least one of the size and shape of the reference coding unit included in the data unit as a criterion for obtaining the PID.

[0165] According to an embodiment, the decoder 120 may use at least one reference coding unit included in one maximum coding unit. In other words, the maximum coding unit into which the image is divided may include at least one reference coding unit, and the coding unit may be determined when each of the reference coding units is recursively divided. According to an embodiment, at least one of the width and height of the maximum coding unit may be an integer multiple of at least one of the width and height of the reference coding unit. According to an embodiment, the size of the reference coding unit may be equal to the size of the maximum coding unit, wherein the maximum coding unit is divided n times according to a quadtree structure. In other words, according to various embodiments, the decoder 120 may determine the reference coding unit by dividing the maximum coding unit n times according to the quadtree structure, and divide the reference coding unit based on at least one of the block shape information and the division shape information.

[0166] Fig.16 A processing block as a standard in a step of determining an order of determining reference coding units included in the picture 1600 according to an embodiment is shown.

[0167] According to an embodiment, the decoder 120 may determine at least one processing block for dividing a picture. A processing block is a data unit including at least one reference coding unit for dividing an image, and at least one reference coding unit included in the processing block may be determined in a specific order. In other words, the determination order of at least one reference coding unit determined in each processing block may correspond to one of a plurality of orders for determining the reference coding unit, and may be changed according to the processing block. The determination order of the reference coding unit determined by each processing block may be one of various orders (such as a raster scan order, a Z scan order, an N scan order, an upper right diagonal scan order, a horizontal scan order, and a vertical scan order), but should not be restrictively interpreted by the scan order.

[0168] According to an embodiment, the decoder 120 may determine the size of at least one processing block included in the image by obtaining information about the size of the processing block. The decoder 120 may obtain information about the size of the processing block from the bitstream to determine the size of the at least one processing block included in the image. The size of the processing block may be a predetermined size of a data unit indicated by the information about the size of the processing block.

[0169] According to an embodiment, the bitstream acquirer 110 of the image decoding device 100 may obtain information about the size of the processing block from the bitstream according to a specific data unit. For example, the information about the size of the processing block may be obtained from the bitstream according to the data units of the image, sequence, picture, slice, and slice segment. In other words, the bitstream acquirer 110 may obtain information about the size of the processing block from the bitstream according to such several data units, and the decoder 120 may determine the size of at least one processing block for dividing the picture by using the obtained information about the size of the processing block, wherein the size of the processing block may be an integer multiple of the size of the reference coding unit.

[0170] According to an embodiment, the decoder 120 may determine the sizes of the processing blocks 1602 and 1612 included in the picture 1600. For example, the decoder 120 may determine the sizes of the processing blocks based on information about the sizes of the processing blocks obtained from the bitstream. Fig.16 According to an embodiment, the decoder 120 may determine the horizontal size of the processing blocks 1602 and 1612 to be four times the horizontal size of the reference coding unit, and determine the vertical size of the processing blocks 1602 and 1612 to be four times the vertical size of the reference coding unit. The decoder 120 may determine the determination order of at least one reference coding unit in at least one processing block.

[0171] According to an embodiment, the decoder 120 may determine each of the processing blocks 1602 and 1612 included in the picture 1600 based on the size of the processing block, and the reference coding unit determiner 120 may determine the determination order of at least one reference coding unit included in each of the processing blocks 1602 and 1612. According to an embodiment, the determination of the reference coding unit may include determination of the size of the reference coding unit.

[0172] According to an embodiment, the decoder 120 may obtain information about the determination order of at least one reference coding unit included in at least one processing block from the bitstream, and determine the determination order of at least one reference coding unit based on the obtained information. The information about the determination order may be defined as the order or direction of determining the reference coding units in the processing block. In other words, the order of determining the reference coding units may be determined independently for each processing block.

[0173] According to an embodiment, the image decoding apparatus 100 may obtain information about the determination order of the reference coding units from the bitstream according to a specific data unit. For example, the bitstream acquirer 110 may obtain information about the determination order of the reference coding units from the bitstream according to data units such as an image, a sequence, a picture, a slice, a slice segment, and a processing block. Since the information about the determination order of the reference coding units indicates the determination order of the reference coding units in the processing block, the information about the determination order may be obtained for each specific data unit (including an integer number of processing blocks).

[0174] According to an embodiment, the image decoding apparatus 100 may determine at least one reference coding unit based on the determined order.

[0175] According to an embodiment, the bitstream acquirer 110 may obtain information about the determined order of the reference coding units from the bitstream as information related to the processing blocks 1602 and 1612, and the decoder 120 may determine the order of determining at least one reference coding unit included in the processing blocks 1602 and 1612, and determine at least one reference coding unit included in the picture 1600 according to the determined order of the coding units. Fig.16, the decoder 120 may determine determination orders 1604 and 1614 of at least one reference coding unit associated with the processing blocks 1602 and 1612, respectively. For example, when information about the determination order of the reference coding unit is obtained for each processing block, the determination orders of the reference coding units associated with the processing blocks 1602 and 1612 may be different from each other. When the determination order 1604 associated with the processing block 1602 is a raster scan order, the reference coding units included in the processing block 1602 may be determined according to the raster scan order. On the other hand, when the determination order 1614 associated with the processing block 1612 is a reverse order of the raster scan order, the reference coding units included in the processing block 1612 may be determined according to the reverse order of the raster scan order.

[0176] According to an embodiment, the decoder 120 may decode the determined at least one reference coding unit. The decoder 120 may decode the image based on the reference coding unit determined by the above embodiment. Examples of a method of decoding a reference coding unit may include various methods of decoding an image.

[0177] According to an embodiment, the image decoding device 100 may obtain and use block shape information indicating the shape of the current coding unit or partition shape information indicating a method of partitioning the current coding unit from a bitstream. The block shape information or the partition shape information may be included in a bitstream associated with various data units. For example, the image decoding device 100 may use the block shape information or the partition shape information included in a sequence parameter set, a picture parameter set, a video parameter set, a slice header, and a slice segment header. In addition, the image decoding device 100 may obtain and use syntax related to the block shape information or the partition shape information from a bitstream according to a maximum coding unit, a reference coding unit, and a processing block.

[0178] According to an embodiment, the decoder 120 may determine a type for dividing a coding unit into different division shapes according to a predetermined data unit. According to an embodiment, the decoder 120 of the image decoding apparatus 100 may differently determine a combination of shapes for dividing a coding unit according to a predetermined data unit (e.g., a sequence, a picture, and a slice).

[0179] Fig.17 Illustrations of coding units that may be determined for each picture when a combination of shapes for splitting coding units differs according to each picture, according to an embodiment.

[0180] Reference Fig.17, the decoder 120 may determine a combination of division shapes for dividing the coding unit differently for each picture. For example, the decoder 120 may decode the image by using a picture 1700 that can be divided into 4 coding units, a picture 1710 that can be divided into 2 or 4 coding units, and a picture 1720 that can be divided into 2, 3, or 4 coding units, included in at least one picture in the image. In order to divide the picture 1700 into a plurality of coding units, the decoder 120 may use only the division shape information indicating that the picture 1700 is divided into 4 square coding units. In order to divide the picture 1710, the decoder 120 may use only the division shape information indicating that the picture 1710 is divided into 2 or 4 coding units. In order to divide the picture 1720, the decoder 120 may use only the division shape information indicating that the picture 1720 is divided into 2, 3, or 4 coding units. Such a combination of division shapes is merely an embodiment for describing the operation of the image decoding device 100, and thus the combination of division shapes should not be restrictively interpreted as the above embodiment, and should be interpreted as a combination of various division shapes that can be used according to a predetermined data unit.

[0181] According to an embodiment, the bitstream acquirer 110 of the image decoding apparatus 100 may obtain a bitstream including an index indicating a combination of division shape information according to a predetermined data unit (e.g., a sequence, a picture, and a slice). For example, the bitstream acquirer 110 may obtain an index indicating the combined division shape information from a sequence parameter set, a picture parameter set, or a slice header. The decoder 120 of the image decoding apparatus 100 may determine a combination of division shapes for dividing a coding unit according to a predetermined data unit by using the obtained index, thereby using a combination of different division shapes according to a predetermined data unit.

[0182] Fig.18 Various shapes of coding units that may be determined based on division shape information that may be represented in a binary code, according to an embodiment are illustrated.

[0183] According to an embodiment, the image decoding apparatus 100 may divide the coding unit into various shapes by using block shape information and division shape information obtained through the bitstream acquirer 110. The shape for dividing the coding unit may correspond to various shapes including the shapes described above through the above embodiments.

[0184] Reference Fig.18 , the decoder 120 may split the coding unit having a square shape in at least one of a horizontal direction and a vertical direction based on the division shape information, and split the coding unit having a non-square shape in a horizontal direction or a vertical direction.

[0185] According to an embodiment, when the decoder 120 is capable of dividing a coding unit having a square shape in the horizontal direction and the vertical direction to obtain four square coding units, the division shapes that may be indicated by the division shape information about the coding unit having a square shape may be four. According to an embodiment, the division shape information may be represented as two binary code maps, and a binary code may be assigned to each division shape. For example, when the coding unit is not divided, the division shape information may be represented as (00)b, when the coding unit is divided in the horizontal and vertical directions, the division shape information may be represented as (01)b, when the coding unit is divided in the horizontal direction, the division shape information may be represented as (10)b, and when the coding unit is divided in the vertical direction, the division shape information may be represented as (11)b.

[0186] According to an embodiment, when the decoder 120 splits a coding unit having a non-square shape in a horizontal or vertical direction, the type of the split shape that may be indicated by the split shape information may be determined based on how many coding units the coding unit is split into. Fig.18 , according to an embodiment, the decoder 120 may divide a coding unit having a non-square shape into at most three coding units. The decoder 120 may divide the coding unit into two coding units, in which case the division shape information may be represented as (10)b. The decoder 120 may divide the coding unit into three coding units, in which case the division shape information may be represented as (11)b. The decoder 120 may determine not to divide the coding unit, in which case the division shape information may be represented as (0)b. In other words, the decoder 120 may use variable length coding (VLC) instead of fixed length coding (FLC) to indicate the division shape information using binary code.

[0187] According to the embodiment, referring to Fig.18 , the binary code of the division shape information indicating that the coding unit is not divided may be represented as (0)b. When the division shape information indicating that the coding unit is not divided is set to (00)b, the binary code of the division shape information of 2 bits may be used in its entirety even though there is no division shape information set to (01)b. However, Fig.18 As shown in FIG. 1 , when three division shapes are used for a coding unit having a non-square shape, even by using a 1-bit binary code (0)b as the division shape information, the decoder 120 may determine that the coding unit is not to be divided, so that the bitstream can be effectively used. However, the division shape of the coding unit having a non-square shape indicated by the division shape should not be restrictively interpreted as Fig.18 The three shapes shown in FIG. 1 should be interpreted as including various shapes of the above-mentioned embodiments.

[0188] Fig.19 Other shapes of coding units that may be determined based on division shape information that may be represented in a binary code according to an embodiment are shown.

[0189] Reference Fig.19 , the decoder 120 may divide the coding unit having a square shape in a horizontal direction or a vertical direction, and divide the coding unit having a non-square shape in a horizontal direction or a vertical direction based on the division shape information. In other words, the division shape information may indicate that the coding unit having a square shape is divided only in one direction. In this case, the binary code of the division shape information indicating that the coding unit having a square shape is not divided may be represented as (0)b. When the division shape information indicating that the coding unit is not divided is set to (00)b, the binary code of the 2-bit division shape information may be used in its entirety, even though there is no division shape information set to (01)b. However, as Fig.19 As shown in FIG. 1 , when three division shapes are used for a coding unit having a non-square shape, even by using a 1-bit binary code (0)b as the division shape information, the decoder 120 may determine that the coding unit is not to be divided, so that the bitstream can be effectively used. However, the division shape of the coding unit having a non-square shape indicated by the division shape should not be restrictively interpreted as Fig.19 The three shapes shown in FIG. 1 should be interpreted as including various shapes of the above-mentioned embodiments.

[0190] According to an embodiment, block shape information or partition shape information may be represented by using a binary code, and such information may be immediately generated in a bitstream. Alternatively, block shape information or partition shape information represented by a binary code may not be immediately generated in a bitstream, and may be used as a binary code input during context adaptive binary arithmetic coding (CABAC).

[0191] Now, a process of obtaining syntax about block shape information or partition shape information by CABAC by the image decoding device 100 according to an embodiment will be described. A bitstream including a binary code about the syntax may be obtained by the bitstream acquirer 110. The decoder 120 may detect a syntax element indicating block shape information or partition shape information by performing inverse binarization on a bin string included in the obtained bitstream. According to an embodiment, the decoder 120 may obtain a group of binary bin strings corresponding to the syntax elements to be decoded, and decode each bin by using probability information, and the decoder 120 may repeat such an operation until the bin string including the bin decoded in this way is the same as one of the bin strings obtained in advance. The decoder 120 may determine the syntax element by performing inverse binarization on the bin string.

[0192] According to an embodiment, the decoder 120 may determine the syntax of the bin string by performing a decoding process of adaptive binary arithmetic coding, and may update the probability model of the bin obtained by the bitstream acquirer 110. Fig.18 According to an embodiment, the bitstream acquirer 110 of the image decoding device 100 may obtain a bitstream indicating a binary code indicating the division shape information. The decoder 120 may determine the syntax of the division shape information by using the obtained binary code having 1 bit or 2 bits. The decoder 120 may update the probability of each of the 2 bits of the binary code to determine the syntax of the division shape information. In other words, the decoder 120 may determine the probability that the next bin may have a value of 0 or 1 when decoding the next bin based on whether the first bin in the 2-bit binary code is 0 or 1.

[0193] According to an embodiment, while determining the syntax, the decoder 120 may update the probability about the bin used while decoding the bin of the bin string about the syntax, and the decoder 120 may determine that specific bits in the bin string have the same probability without updating the probability.

[0194] Reference Fig.18 , while determining the syntax by using a bin string indicating the division shape information about the coding unit having a non-square shape, the decoder 120 may determine the syntax about the division shape information by using one bin having a value of 0 when the coding unit having a non-square shape is not divided. In other words, when the block shape information indicates that the current coding unit has a non-square shape, the first bin of the bin string about the division shape information may be 0 when the current coding unit is not divided, and may be 1 when the current coding unit is divided into 2 or 3 coding units. Therefore, the probability that the first bin of the bin string about the division shape information of the coding unit having a non-square shape is 0 may be 1 / 3, and the probability that it is 1 may be 2 / 3. As described above, since the division shape information indicating that the coding unit having a non-square shape is not divided may represent only a bin string of 1 bit having a value of 1, the decoder 120 may determine the syntax about the division shape information by determining whether the second bin is 0 or 1 only when the first bin of the division shape information is 1. According to an embodiment, when the first bin about the division shape information is 1, the decoder 120 may decode the bins for which the probability of determining whether the second bin is 0 or 1 is the same.

[0195] According to an embodiment, while determining the bins of the bin string for dividing the shape information, the image decoding apparatus 100 may use various probabilities for each bin. According to an embodiment, the decoder 120 may determine the probability of dividing the bins of the shape information differently based on the direction of the non-square block. According to an embodiment, the decoder 120 may determine the probability of dividing the bins of the shape information differently based on the area of ​​the current coding unit or the length of the longer side. According to an embodiment, the decoder 120 may determine the probability of dividing the bins of the shape information differently based on at least one of the shape of the current coding unit or the length of the longer side.

[0196] According to an embodiment, the decoder 120 may determine that the probability of dividing the bin of the shape information is the same for coding units having at least a predetermined size. For example, the decoder 120 may determine that the probability of dividing the bin of the shape information is the same for coding units having a size of at least 64 samples based on the length of the longer side of each coding unit.

[0197] According to an embodiment, the decoder 120 may determine initial probabilities of bins forming a bin string dividing shape information based on a slice type (eg, I slice, P slice, B slice, etc.).

[0198] Fig. 20 is a block diagram illustrating an image encoding and decoding system that performs loop filtering.

[0199] The encoder 2010 of the image encoding and decoding system 2000 transmits a bit stream encoded by the image, and the decoder 2050 outputs a reconstructed image by receiving the bit stream and decoding it. Here, the encoder 2010 may have a structure similar to the image encoding apparatus 200 described later, and the decoder 2050 may have a structure similar to the image decoding apparatus 100.

[0200] In the encoder 2010, the prediction encoder 2015 outputs a reference image through inter-frame prediction and intra-frame prediction, and the transformer and quantizer 2020 outputs residual data by quantizing the residual data between the reference image and the current input image into quantized transform coefficients. The entropy encoder 2025 outputs a bit stream by encoding and transforming the quantized transform coefficients. The quantized transform coefficients are reconstructed into data in the spatial domain through the inverse quantizer and inverse transformer 2030, and the data in the spatial domain is output as a reconstructed image through the deblocking filter unit 2035 and the loop filter unit 2040. The reconstructed image can be used as a reference image of the next input image through the prediction encoder 2015.

[0201] The encoded image data from the bit stream received by the decoder 2050 is reconstructed as residual data in the spatial domain through the entropy decoder 2055 and the inverse quantizer and inverse transformer 2060. When the reference image output from the prediction decoder 2075 and the residual data are combined, the image data in the spatial domain is formed, and the deblocking filter unit 2065 and the loop filter unit 2070 may perform filtering on the image data in the spatial domain to output a reconstructed image with respect to the current original image. The reconstructed image may be used as a reference image of the next original image through the prediction decoder 2075.

[0202] The loop filtering unit 2040 of the encoder 2010 performs loop filtering by using filter information input according to user input or system setting. The filter information used by the loop filtering unit 2040 is output to the entropy encoder 2025 to be sent to the decoder 2050 together with the encoded image data. The loop filtering unit 2070 of the decoder 2050 may perform loop filtering based on the filter information input from the entropy decoder 2055.

[0203] Fig.21 An example of filtering units included in a maximum coding unit and filtering performance information of the filtering units according to an embodiment is illustrated.

[0204] When the filtering units of the loop filtering unit 2040 of the encoder 2010 and the loop filtering unit 2070 of the decoder 2050 are configured as described above with reference to Figures 3 to 5 When the data unit is similar to the coding unit according to the described embodiment, the filter information may include block shape information and division shape information of the data unit for indicating the filtering unit, and the filtering unit is processed to filter the loop filtering performance information indicating whether the loop filtering is performed.

[0205] The filtering units included in the maximum coding unit 2100 according to an embodiment may have the same block shape and division shape as the coding units included in the maximum coding unit 2100. In addition, the filtering units included in the maximum coding unit 2100 according to an embodiment may be divided based on the sizes of the coding units included in the maximum coding unit 2100. Fig.21 For example, the filtering unit may include a filtering unit 2140 having a square shape and a depth D, filtering units 2132 and 2134 having a non-square shape and a depth D, filtering units 2112, 2114, 2116, 2152, 2154 and 2164 having a square shape and a depth D+1, filtering units 2162 and 2166 having a non-square shape and a depth D+1, and filtering units 2122, 2124, 2126 and 2128 having a square shape and a depth D+2.

[0206] Block shape information, split shape information (depth), and loop filtering performance information of a filtering unit included in the maximum coding unit 2100 may be encoded as shown in Table 1.

[0207] Table 1

[0208]

[0209] Already passed above Fig.13 A process of determining a plurality of coding units when a coding unit is recursively divided according to block shape information and division shape information according to an embodiment is described. The loop filtering performance information of a filtering unit according to an embodiment indicates that loop filtering is performed on the filtering unit when a flag value is 1, and loop filtering is not performed on the filtering unit when the flag value is 0. Referring to Table 1, information of a data unit for determining a filtering unit to be filtered by the loop filtering unit 2040 and the loop filtering unit 2070 may all be encoded and transmitted as filter information.

[0210] Since the coding unit configured according to the embodiment is a coding unit configured to minimize the error with the original image, it is expected to have high spatial correlation in the coding unit. Therefore, by determining the filtering unit based on the coding unit according to the embodiment, the operation of determining the filtering unit separately from the operation of determining the coding unit can be omitted. In addition, therefore, by determining the filtering unit based on the coding unit according to the embodiment, information for determining the division shape of the filtering unit can be omitted, so that the transmission bit rate of the filter information can be saved.

[0211] In the above embodiment, it is described that the filtering unit is determined based on the coding unit according to the embodiment, but the filtering unit may be split up to an arbitrary depth based on the coding unit, and thus the shape of the filtering unit may be determined only up to the arbitrary depth.

[0212] The determination of the filtering unit described in the above embodiments may be applied not only to loop filtering but also to various embodiments such as deblocking filtering and adaptive loop filtering.

[0213] According to an embodiment, the image decoding apparatus 100 may divide the current coding unit by using at least one of block shape information and division shape information, wherein the block shape information may be predetermined to indicate that only a square shape is used, and the division shape information may be predetermined to indicate that the current coding unit is not divided or is divided into four square coding units. In other words, the coding unit of the current coding unit may always have a square shape according to the block shape information, and the current coding unit may not be divided or is divided into four square coding units based on the division shape information. The image decoding apparatus 100 may obtain a bitstream generated by using a predetermined coding unit (which is predetermined to use only such a block shape and division shape) through the bitstream acquirer 110, and the decoder 120 may use only the previously predetermined block shape and division shape. In this case, the image decoding apparatus 100 may solve the compatibility problem with the predetermined encoding method by using a predetermined decoding method similar to the predetermined encoding method. According to an embodiment, when the image decoding apparatus 100 uses a predetermined decoding method that uses only a predetermined block shape and a division shape that can be indicated by the block shape information and the division shape information among various shapes, the block shape information indicates only a square shape, so that the image decoding apparatus 100 may not perform a process of obtaining the block shape information from the bitstream. A syntax indicating whether a predetermined decoding method is used may be used, and such a syntax may be obtained from a bitstream according to a data unit including a plurality of coding units having various shapes, such as a sequence, a picture, a slice unit, and a maximum coding unit. In other words, the bitstream acquirer 110 may determine whether a syntax indicating block shape information will be obtained from a bitstream based on the syntax indicating whether a predetermined decoding method is used.

[0214] Fig.23 An index according to a Z scan order of coding units according to an embodiment is shown.

[0215] The image decoding apparatus 100 according to an embodiment may sequentially scan the lower layer data units included in the upper layer data units according to a Z scan order. In addition, the image decoding apparatus 100 according to an embodiment may sequentially access data according to Z scan indexes in the coding units included in the maximum coding unit or the processing block.

[0216] As above Figure 3 and Figure 4 As described above, the image decoding apparatus 100 according to the embodiment may divide the reference coding unit into at least one coding unit. Here, the reference coding unit may include a coding unit having a square shape and a coding unit having a non-square shape. The image decoding apparatus 100 according to the embodiment may perform data access according to the Z-scan index in each coding unit included in the reference coding unit. Here, the method of applying the Z-scan index may change based on whether a coding unit having a non-square shape exists in the reference coding unit.

[0217] According to an embodiment, when a coding unit having a non-square shape does not exist in a reference coding unit, the coding units of the lower depth in the reference coding unit may have continuous Z scan indexes. For example, according to an embodiment, the coding unit of the upper depth may include four coding units of the lower depth. Here, the boundaries of the four coding units of the lower depth adjacent to each other may be continuous, and the coding units of the lower depth may be scanned in a Z scan order according to an index indicating a Z scan order. The index indicating a Z scan order according to an embodiment may be set for the coding unit in increasing numbers according to the Z scan order. In this case, deeper coding units of the same depth may be scanned according to the Z scan order.

[0218] According to an embodiment, when at least one coding unit having a non-square shape exists in a reference coding unit, the image decoding device 100 may divide each of the coding units in the reference coding unit into sub-blocks and scan the sub-blocks according to a Z scan order. For example, when a coding unit having a non-square shape in a vertical or horizontal direction exists in a reference coding unit, a Z scan may be performed by using a sub-block divided from the coding unit. In addition, for example, when the reference coding unit is divided into an odd number of coding units, a Z scan may be performed by using a sub-block. A sub-block is a coding unit that is no longer divided or a coding unit obtained by dividing an arbitrary coding unit, and may have a square shape. For example, four sub-blocks having a square shape may be divided from a coding unit having a square shape. In addition, for example, two sub-blocks having a square shape may be divided from a coding unit having a non-square shape.

[0219] Reference Fig.23, for example, the image decoding apparatus 100 according to an embodiment may scan the coding units 2302, 2304, 2306, 2308, and 2310 of the lower depth in the coding unit 2300 according to the Z scanning order. The coding unit 2300 and the coding units 2302 to 2310 are relatively upper-layer coding units and lower-layer coding units. The coding unit 2300 includes coding units 2306 and 2310 having a non-square shape in a horizontal direction. The coding units 2306 and 2310 having a non-square shape have non-continuous boundaries with the coding units 2302 and 2304 having a square shape. In addition, when the coding unit having a non-square shape is divided into an odd number of coding units, the coding unit 2308 has a square shape and is a coding unit located in the middle. Similar to the coding units 2306 and 2310 having a non-square shape, the coding unit 2308 has a non-continuous boundary with the adjacent coding units 2302 and 2304 having a square shape. When a coding unit having a non-square shape is divided into an odd number of coding units, when the coding unit 2300 includes coding units 2306 and 2310 having a non-square shape or includes a coding unit 2308 located in the middle, the adjacent boundaries between the coding units are discontinuous, so that continuous Z scan indexes cannot be set. Therefore, the image decoding device 100 can continuously set the Z scan index by dividing the coding unit into sub-blocks. In addition, the image decoding device 100 may perform continuous Z scans on the coding units 2306 and 2310 having a non-square shape or the coding unit 2308 located in the middle of the odd number of coding units having a non-square shape.

[0220] The coding units 2302 to 2310 in the coding unit 2300 are divided into sub-blocks to obtain Fig.23 . A Z scan index may be set for each of the sub-blocks, and since adjacent boundaries between the sub-blocks are continuous, the sub-blocks may be scanned according to a Z scan order. For example, in a decoding device according to an embodiment, the coding unit 2308 may be divided into sub-blocks 2322, 2324, 2326, and 2328. Here, sub-blocks 2322 and 2324 may be scanned after data processing is performed on sub-block 2330, and sub-blocks 2326 and 2328 may be scanned after data processing is performed on sub-block 2332. In addition, the sub-blocks may be scanned according to a Z scan order.

[0221] In the above embodiments, the data cells may be scanned according to a Z-scan order for data storage, data loading, data access, etc.

[0222] In addition, in the above embodiments, the data units are scanned according to the Z scanning order, but the scanning order of the data units can be changed, for example, a raster scanning order, an N scanning order, an upper right diagonal scanning order, a horizontal scanning order, and a vertical scanning order, so as not to be limited to the Z scanning order.

[0223] In addition, in the above embodiment, the coding units in the reference coding unit are scanned, but the embodiment is not limited thereto, and the target to be scanned may be any block in the maximum coding unit or the processing block.

[0224] In addition, in the above embodiment, scanning is performed according to the Z scan order by dividing the block into sub-blocks only when at least one block having a non-square shape exists, but for a simplified embodiment, scanning may be performed according to the Z scan order by dividing the block into sub-blocks even when the block having a non-square shape does not exist.

[0225] The image decoding apparatus 100 according to an embodiment may generate prediction data by performing inter prediction or intra prediction on a coding unit, generate residual data by performing inverse transformation on a transformation unit included in a current coding unit, and reconstruct the current coding unit by using the generated prediction data and the residual data.

[0226] The prediction mode of the coding unit according to an embodiment may be at least one of an intra mode, an inter mode, and a skip mode. According to an embodiment, the prediction mode may be independently selected according to the coding unit.

[0227] When the coding unit having a 2N×2N shape according to an embodiment is split into two coding units having a 2N×N shape or an N×2N shape, inter mode prediction and intra mode prediction may be performed on each coding unit, respectively. In addition, the skip mode may be applied to the coding unit having a 2N×N shape or an N×2N shape according to an embodiment.

[0228] Meanwhile, the image decoding apparatus 100 according to an embodiment may allow dual prediction in skip mode for a coding unit having an 8×4 shape or a 4×8 shape. In skip mode, since only skip mode information is received for a coding unit, the use of residual data for the coding unit is omitted. Therefore, in this case, the overhead of inverse quantization and inverse transformation can be saved. In contrast, the image decoding apparatus 100 according to an embodiment may allow dual prediction for a coding unit to which a skip mode is applied to increase decoding efficiency. In addition, while setting the number of interpolation taps to a relatively low value during motion compensation, the image decoding apparatus 100 according to an embodiment may allow dual prediction for a coding unit having an 8×4 or 4×8 shape, thereby efficiently using memory bandwidth. For example, an interpolation filter having a tap number lower than 8 (e.g., a 2-tap interpolation filter) may be used instead of an 8-tap interpolation filter.

[0229] In addition, the image decoding apparatus 100 according to an embodiment may signal intra or inter prediction information for each region included in a current coding unit by splitting the region in a preset shape (eg, diagonal-based splitting).

[0230] The image decoding apparatus 100 according to an embodiment may obtain a prediction sample of a current coding unit by using neighboring samples of the current coding unit using an intra mode. Here, intra prediction is performed by using pre-reconstructed neighboring samples, which are referred to as reference samples.

[0231] Fig.24 2 is a diagram illustrating reference samples used for intra prediction of a coding unit according to an embodiment. Fig.24 , for a coding unit 2300 having a non-square block shape, a horizontal length w, and a vertical length h, w+h upper reference samples 2302, w+h left reference samples 2304, and one upper left reference sample 2306 are required, that is, a total number of reference samples of 2(w+h)+1 is required. In order to prepare the reference samples, padding is performed on an area where no reference samples exist, and a reference sample filtering process may be performed according to a prediction mode to reduce a quantization error included in the reconstructed reference samples.

[0232] In the above embodiments, the number of reference samples when the current coding unit has a non-square shape has been described, and the number of reference samples is also applicable when the current coding unit has a square shape.

[0233] The operations related to the image decoding method performed by the image decoding device 100 have been described above in the above-mentioned various embodiments. Hereinafter, the operations of the image encoding device 200 that performs the image encoding method will be described through various embodiments, wherein the operations of the image encoding device 200 are the inverse process of the image decoding method.

[0234] Figure 2 is a block diagram illustrating an image encoding apparatus 200 capable of encoding an image based on at least one of block shape information and division shape information according to an embodiment.

[0235] Reference Figure 2According to an embodiment, the image encoding apparatus 200 may include a bitstream generator 210 for generating a bitstream including predetermined information such as division shape information and block shape information, and an encoder 220 for encoding an image by using the predetermined information. According to an embodiment, the encoder 220 of the image encoding apparatus 200 may determine at least one coding unit for dividing the image based on at least one of the block shape information and the division shape information, and the bitstream generator 210 of the image encoding apparatus 200 may generate a bitstream including at least one of the block shape information and the division shape information. Here, the block shape information may refer to information or syntax indicating a shape of a coding unit, and the division shape information may refer to information or syntax indicating a shape into which the coding unit is to be divided.

[0236] According to an embodiment, the encoder 220 of the image encoding apparatus 200 may determine the shape of the coding unit. For example, the coding unit may have a square shape or a non-square shape, and information indicating such a shape may be included in the block shape information.

[0237] According to an embodiment, the encoder 220 may determine a shape for dividing the coding unit. The encoder 220 may determine the shape of at least one coding unit included in the coding unit, and the bitstream generator 210 may generate a bitstream including the division shape information, wherein the division shape information includes information about such a shape of the coding unit.

[0238] According to an embodiment, the encoder 220 may determine whether the coding unit is divided. When the encoder 220 determines that the coding unit includes only one coding unit or the coding unit is not divided, the bitstream generator 210 may generate a bitstream including division shape information indicating that the coding unit is not divided. In addition, the encoder 220 may divide the coding unit into a plurality of coding units, and the bitstream generator 210 may generate a bitstream including division shape information indicating that the coding unit is divided into a plurality of coding units.

[0239] According to an embodiment, information indicating the number of coding units obtained by dividing the coding unit or indicating the direction in which the coding unit is divided may be included in the division shape information. For example, the division shape information may indicate at least one of the horizontal direction or the vertical direction for dividing the coding unit or indicate that the coding unit is not divided.

[0240] Figure 3 A process of determining at least one coding unit by the image encoding unit 200 when a current coding unit is split according to an embodiment is illustrated.

[0241] According to an embodiment, the encoder 220 may determine the shape of the coding unit. For example, the encoder 220 may determine the shape of the coding unit having an optimal rate-distortion (RD) cost in consideration of the RD cost.

[0242] According to an embodiment, the encoder 220 may determine that the current coding unit has a square shape, and accordingly may determine the shape into which the coding unit having the square shape is divided. For example, the encoder 220 may determine whether to divide the coding unit having the square shape, whether to divide the coding unit in a vertical direction, whether to divide the coding unit in a horizontal direction, or divide the coding unit into four coding units. Figure 3 , the encoder 220 may determine not to split the coding unit 310a having the same size as the current coding unit 300, or to split the current coding unit 300 into coding units 310b to 310d, based on the block shape information indicating the predetermined splitting method.

[0243] Reference Figure 3 According to an embodiment, the encoder 220 may determine two coding units 310b by dividing the current coding unit 300 in the vertical direction. The encoder 220 may determine two coding units 310c by dividing the current coding unit 300 in the horizontal direction. The encoder 220 may determine four coding units 310d by dividing the current coding unit 300 in the vertical and horizontal directions. However, the division shapes into which the coding unit having a square shape may be divided are not restrictively interpreted as the above shapes, and may include various shapes that may be indicated by the division shape information. The predetermined division shapes into which the square coding unit may be divided will be described in detail below through various embodiments.

[0244] According to an embodiment, the bitstream generator 210 of the image encoding apparatus 200 may generate a bitstream including division shape information indicating a shape of the current coding unit 300 divided by the encoder 220 .

[0245] Figure 4 A process of determining at least one coding unit by the image encoding apparatus 200 when a coding unit having a non-square shape is split according to an embodiment is illustrated.

[0246] According to an embodiment, the encoder 220 may determine whether to split the current coding unit having a non-square shape, or whether to split the current coding unit by using a predetermined method. Figure 4, the encoder 220 may determine not to divide the coding unit 410 or 460 having the same size as the current coding unit 400 or 450, or may determine the coding units 420a, 420b, 430a, 430b, 430c, 470a, 470b, 480a, 480b, and 480c according to a predetermined division method. The bitstream generator 210 of the image encoding apparatus 200 may generate a bitstream including division shape information indicating such a division shape. A predetermined division method for dividing a coding unit having a non-square shape will now be described in detail through various embodiments.

[0247] According to an embodiment, the encoder 220 may determine the shape into which the coding unit is divided. Figure 4 , the encoder 220 may divide the current coding unit 400 or 450 to determine two coding units 420a and 420b or 470a and 470b, and the bitstream generator 210 may generate a bitstream including division shape information indicating such a division shape.

[0248] According to an embodiment, when the encoder 220 splits the current coding unit 400 or 450 having a non-square shape, the current coding unit may be split considering the position of the longer side of the current coding unit 400 or 450 having the non-square shape. For example, the encoder 220 may determine a plurality of coding units by splitting the longer side of the current coding unit 400 or 450 considering the shape of the current coding unit 400 or 450, and the bitstream generator 210 may generate a bitstream including split shape information indicating such a split shape.

[0249] According to an embodiment, the encoder 220 may determine an odd number of coding units included in the current coding unit 400 or 450. For example, the encoder 220 may divide the current coding unit 400 or 450 into three coding units 430a to 430c or 480a to 480c. According to an embodiment, the encoder 220 may determine an odd number of coding units included in the current coding unit 400 or 450, wherein the sizes of the coding units may be different. For example, the size of a predetermined coding unit 430b or 480b among the odd number of coding units 430a to 430c or 480a to 480c may be different from the size of the coding units 430a and 430c or 480a and 480c. In other words, the coding units determined by dividing the current coding unit 400 or 450 may have a variety of types of sizes, and in some cases, the odd number of coding units 430a to 430c or 480a to 480c may have different sizes.

[0250] According to an embodiment, the encoder 220 may determine an odd-numbered coding unit included in the current coding unit 400 or 450, and further, may set a predetermined limitation on at least one of the odd-numbered coding units. Figure 4 , the encoder 220 sets the decoding process of the coding unit 430b or 480b located in the middle of the three coding units 430a to 430c or 480a to 480c generated by dividing the current coding unit 400 or 450 to be different from the decoding process of the coding units 430a and 430c or 480a and 480c. For example, the encoder 220 may define that the coding unit 430b or 480b located in the middle is not further divided like the coding units 430a and 430c or 480a and 480c, or is defined to be divided only a predetermined number of times.

[0251] Figure 5 A process of splitting a coding unit by the image encoding apparatus 200 according to an embodiment is illustrated.

[0252] According to an embodiment, the encoder 220 may determine whether to divide the first coding unit 500 having a square shape into coding units. According to an embodiment, the encoder 220 may determine the second coding unit 510 by dividing the first coding unit 500 in the horizontal direction, and the first coding unit, the second coding unit, and the third coding unit used according to the embodiment are terms for understanding the division relationship between the coding units. For example, the second coding unit may be determined when the first coding unit is divided, and the third coding unit may be determined when the second coding unit is divided. The relationship between the first coding unit to the third coding unit used below may be understood as following the above-mentioned features.

[0253] According to an embodiment, the image encoding apparatus 200 may determine whether to split the second coding unit 510 into coding units based on at least one of block shape information and division shape information. Figure 5, the encoder 220 may divide the second coding unit 510 having a non-square shape and determined by dividing the first coding unit 500 into at least one third coding unit 520a, 520b to 520d, etc. based on at least one of the block shape information and the division shape information, or may not divide the second coding unit 510. The bitstream generator 210 of the image encoding apparatus 200 may generate a bitstream including at least one of the block shape information and the division shape information, and the encoder 220 may determine a plurality of second coding units (for example, the second coding unit 510) having various shapes by dividing the first coding unit 500 based on at least one of the block shape information and the division shape information, wherein the second coding unit 510 may be divided according to a method of dividing the first coding unit 500 based on at least one of the block shape information and the division shape information. According to an embodiment, when the first coding unit 500 is split into the second coding unit 510 based on at least one of the block shape information and the division shape information about the first coding unit 500, the second coding unit 510 may also be split into at least one third coding unit (e.g., third coding units 520a, 520b, 520c, or 520d) based on at least one of the block shape information and the division shape information about the second coding unit 510. Therefore, a square coding unit may be determined from a coding unit having a non-square shape, and a coding unit having a non-square shape may be determined when such a coding unit having a square shape is recursively split. Referring to Figure 5 , a predetermined coding unit (e.g., a coding unit located in the middle or a coding unit having a square shape) among the odd number of third coding units 520b to 520d determined when the second coding unit 510 having a non-square shape is divided may be recursively divided. According to an embodiment, a third coding unit 520c having a square shape, which is one of the odd number of third coding units 520b to 520d, may be divided into a plurality of fourth coding units in a horizontal direction. A fourth coding unit 540 having a non-square shape, which is one of the plurality of fourth coding units, may be divided into a plurality of coding units again. For example, a fourth coding unit 540 having a non-square shape may be divided into an odd number of coding units 550a to 550c again.

[0254] The coding unit may be recursively split based on at least one of block shape information and partition shape information associated with each coding unit. A method used for recursively splitting the coding unit may be described below through various embodiments.

[0255] According to an embodiment, the encoder 220 may determine whether to divide each of the third coding units 520a to 520d, etc. into coding units or not to divide the second coding unit 510 based on at least one of the block shape information and the division shape information. According to an embodiment, the encoder 220 may divide the non-square-shaped second coding unit 510 into an odd number of third coding units 520b to 520d. The image encoding device 200 may set a predetermined limitation on a predetermined third coding unit among the odd number of third coding units 520b to 520d. For example, the image encoding device 200 may limit the coding unit 520c located in the middle of the odd number of third coding units 520b to 520d to no longer be divided or to be divided a set number of times. Refer to Figure 5 , the image encoding apparatus 200 may define the middle coding unit 520c of the odd number of third coding units 520b to 520d included in the second coding unit 510 having a non-square shape to be no longer divided, divided according to a predetermined division shape (for example, divided only into four coding units or divided into a shape corresponding to the shape in which the second coding unit 510 is divided), or divided only a predetermined number of times (for example, divided only n times, where n>0). However, the definition of the middle coding unit 520c is only a simple embodiment, and thus should not be restrictively interpreted as the above embodiment, and may include various definitions of decoding the middle coding unit 520c differently from the coding units 520b and 520d.

[0256] According to an embodiment, the bitstream generator 210 of the image encoding apparatus 200 may generate a bitstream including at least one of block shape information and partition shape information for partitioning the current coding unit, together with a bitstream of a sample at a predetermined position in the current coding unit.

[0257] Figure 6 A method for determining a predetermined coding unit from an odd number of coding units by the encoder 220 according to an embodiment is shown. The encoder 220 of the image encoding apparatus 200 may determine to split a current coding unit into coding units having various shapes and sizes, or not to split the current coding unit. Figure 6 , the bitstream generator 210 may generate a bitstream including at least one of block shape information and partition shape information of the current coding unit 600 together with a bitstream related to a sample located at a predetermined position among a plurality of samples included in the current coding unit 600 (e.g., a sample 640 located in the middle). However, the predetermined position in the current coding unit 600 related to at least one of the block shape information and the partition shape information should not be restrictively interpreted as Figure 6The predetermined position may be interpreted as including various positions (e.g., the topmost position, the bottommost position, the left position, the right position, the upper left position, the lower left position, the upper right position, and the lower right position) included in the current coding unit 600.

[0258] According to an embodiment, when the current coding unit is split into a predetermined number of coding units, the image encoding apparatus 200 may select one coding unit. A method of selecting one of the coding units may vary, and such a method will be described below through various embodiments.

[0259] According to an embodiment, the encoder 220 of the image encoding apparatus 200 may divide a current coding unit into a plurality of coding units and determine a coding unit at a predetermined position.

[0260] Figure 6 A method of determining, by the image encoding apparatus 200 , a coding unit at a predetermined position from among an odd number of coding units according to an embodiment is illustrated.

[0261] According to an embodiment, the encoder 220 may use information indicating the position of each of the odd-numbered coding units to determine a coding unit located in the middle of the odd-numbered coding units. Figure 6 , the encoder 220 may determine an odd number of coding units 620a to 620c by dividing the current coding unit 600. The encoder 220 may determine the coding unit 620b in the middle by using information about the positions of the odd number of coding units 620a to 620c. For example, the encoder 220 may determine the coding unit 620b in the middle by determining the positions of the coding units 620a to 620c based on information indicating the positions of predetermined samples included in the coding units 620a to 620c. In detail, the encoder 220 may determine the coding unit 620b in the middle by determining the positions of the coding units 620a to 620c based on information indicating the positions of the upper left samples 630a to 640c in the coding units 620a to 620c.

[0262] According to an embodiment, the information indicating the positions of the upper left samples 630a to 630c respectively included in the coding units 620a to 620c may include information about the positions or coordinates of the coding units 620a to 620c in the picture. According to an embodiment, the information indicating the positions of the upper left samples 630a to 630c respectively included in the coding units 620a to 620c may include information indicating the width or height of the coding units 620a to 620c included in the current coding unit 600, wherein the width or height may correspond to the information indicating the difference between the coordinates of the coding units 620a to 620c in the picture. In other words, the image encoding apparatus 200 may directly use the information about the positions or coordinates of the coding units 620a to 620c in the picture or may use the information about the width or height of the coding units indicating the difference between the coordinates to determine the coding unit 620b located in the middle.

[0263] According to an embodiment, the information indicating the position of the upper left sample 630a of the top coding unit 620a may indicate (xa, ya) coordinates, the information indicating the position of the upper left sample 630b of the middle coding unit 620b may indicate (xb, yb) coordinates, and the information indicating the position of the upper left sample 630c of the bottom coding unit 620c may indicate (xc, yc) coordinates. The image encoding apparatus 200 may determine the middle coding unit 620b by using the coordinates of the upper left samples 630a to 630c respectively included in the coding units 620a to 620c. For example, when the coordinates of the upper left samples 630a to 630c are arranged in ascending or descending order, the middle coding unit 620b including (xb, yb) as the coordinate of the upper left sample 630b may be determined as a coding unit located in the middle among the coding units 620a to 620c determined when the current coding unit 600 is split. Here, the coordinates indicating the positions of the upper left samples 630a to 630c may represent coordinates indicating absolute positions in the picture, and (dxb, dyb) coordinates as information indicating the relative position of the upper left sample 630b of the middle coding unit 620b and (dxc, dyc) coordinates as information indicating the relative position of the upper left sample 630c of the bottom coding unit 620c may also be used based on the position of the upper left sample 630a of the top coding unit 620a. In addition, the method of determining the coding unit at a predetermined position by using the coordinates of the samples included in the coding unit as the information indicating the position of the samples should not be restrictively interpreted as the above method, and may be interpreted as various arithmetic methods capable of using the coordinates of the samples.

[0264] According to an embodiment, the image encoding apparatus 200 may divide the current coding unit 600 into a plurality of coding units 620a to 620c and select coding units from the coding units 620a to 620c according to a predetermined criterion. For example, the encoder 220 may select a coding unit 620b having a different size from the coding units 620a to 620c.

[0265] According to an embodiment, the image encoding apparatus 200 may determine the width or height of each of the coding units 620a to 620c by using the (xa, ya) coordinates as information indicating the position of the upper left sample 630a of the top coding unit 620a, the (xb, yb) coordinates as information indicating the position of the upper left sample 630b of the middle coding unit 620b, and the (xc, yc) coordinates as information indicating the position of the upper left sample 630c of the bottom coding unit 620c. The image encoding apparatus 200 may determine the size of each of the coding units 620a to 620c by using the coordinates (xa, ya), (xb, yb), and (xc, yc) indicating the positions of the coding units 620a to 620c.

[0266] According to an embodiment, the image encoding apparatus 200 may determine the width of the top coding unit 620a as xb-xa, and the height of the top coding unit 620a as yb-ya. According to an embodiment, the encoder 220 may determine the width of the middle coding unit 620b as xc-xb, and the height of the middle coding unit 620b as yc-yb. According to an embodiment, the encoder 220 may determine the width or height of the bottom coding unit by using the width or height of the current coding unit, and the widths and heights of the top coding unit 620a and the middle coding unit 620b. The encoder 220 may determine a coding unit having a size different from other coding units based on the determined widths and heights of the coding units 620a to 620c. Referring to Figure 6 , the image encoding apparatus 200 may determine the middle coding unit 620b having a size different from the sizes of the top coding unit 620a and the bottom coding unit 620c as the coding unit at the predetermined position. However, since the process of determining the coding unit having a size different from other coding units by the image encoding apparatus 200 is only an embodiment of determining the coding unit at the predetermined position by using the size of the coding unit determined based on the sample coordinates, various processes of determining the coding unit at the predetermined position by comparing the sizes of the coding units determined according to the predetermined sample coordinates may be used.

[0267] However, the position of the sample considered to determine the position of the coding unit should not be restrictively interpreted as the upper left, but may be interpreted that information on the position of any sample included in the coding unit may be used.

[0268] According to an embodiment, considering the shape of the current coding unit, the image encoding device 200 may select a coding unit at a predetermined position from an odd number of coding units determined when the current coding unit is divided. For example, when the current coding unit has a non-square shape in which the width is longer than the height, the decoder 120 may determine a coding unit at a predetermined position in the horizontal direction. In other words, the encoder 220 may determine a coding unit from coding units having different positions in the horizontal direction and set a limit on the coding unit. When the current coding unit has a non-square shape in which the height is longer than the width, the encoder 220 may determine a coding unit at a predetermined position in the vertical direction. In other words, the encoder 220 may determine a coding unit from coding units having different positions in the vertical direction and set a limit on the coding unit.

[0269] According to an embodiment, the image encoding apparatus 200 may use information indicating the position of each of the even-numbered coding units to determine the coding unit at a predetermined position from the even-numbered coding units. The encoder 220 may determine the even-numbered coding units by dividing the current coding unit, and determine the coding unit at the predetermined position by using the information about the position of the even-numbered coding units. The detailed process may be the same as that described above with reference to Figure 6 The described process of determining a coding unit at a predetermined position (eg, a middle position) from among an odd number of coding units is similar, and thus a description thereof will not be provided.

[0270] According to an embodiment, when a current coding unit having a non-square shape is divided into a plurality of coding units, predetermined information used while dividing the current coding unit may be used to determine a coding unit at a predetermined position from among the plurality of coding units. For example, the encoder 220 of the image encoding apparatus 200 may use at least one of the block shape information and the division shape information as predetermined information used while dividing the current coding unit to determine a coding unit located in the middle from among the plurality of coding units obtained by dividing the current coding unit.

[0271] Reference Figure 6, the encoder 220 of the image encoding apparatus 200 may divide the current coding unit 600 into a plurality of coding units 620a to 620c, and determine a coding unit 620b located in the middle from the plurality of coding units 620a to 620c, and the bitstream generator 210 may generate a bitstream including at least one of block shape information and division shape information used when dividing the current coding unit 600. The encoder 220 may determine the coding unit 620b located in the middle by considering the position of a sample related to the bitstream including at least one of the block shape information and division shape information used when dividing the current coding unit 600. In other words, a bitstream including at least one of the block shape information and division shape information of the current coding unit 600 may be generated together with a bitstream related to a sample 640 located in the middle of the current coding unit 600, and at this time, the encoder 220 may determine the coding unit 620b including the sample 640 as a coding unit located in the middle among the plurality of coding units 620a to 620c. However, information for determining a coding unit located in the middle from among a plurality of coding units obtained by dividing the current coding unit should not be restrictively interpreted as at least one of block shape information and division shape information used while dividing the current coding unit, and various types of information may be used. In this regard, the process of determining a coding unit at a predetermined position by the image encoding device 200 may be an inverse process of determining a coding unit at a predetermined position from among a plurality of coding units determined from the current coding unit by the image decoding device 100, and thus a detailed description thereof is not provided.

[0272] According to an embodiment, the image encoding apparatus 200 may determine at least one coding unit by splitting a current coding unit, and determine an order in which to decode the at least one coding unit according to a predetermined block (for example, the current coding unit).

[0273] Figure 7 2 shows an order in which a plurality of coding units are processed when the image encoding apparatus 200 determines a plurality of coding units by splitting a current coding unit according to an embodiment. Figure 7 The processing of the plurality of coding units by the image coding device 200 is similar to that described above with reference to Figure 7 The operation of the described image decoding device 100 is similar, so a detailed description thereof will not be provided.

[0274] Figure 8 A process of determining, by the image encoding apparatus 200 , that a current coding unit is split into an odd number of coding units when coding units cannot be processed in a predetermined order according to an embodiment is illustrated.

[0275] According to an embodiment, the encoder 220 of the image encoding apparatus 200 may determine to divide the current coding unit into an odd number, and the bitstream generator 210 may generate a bitstream including block division information indicating the shape of the current coding unit and division shape information indicating the division shape of the current coding unit (divided into an odd number). Figure 8 , the first coding unit 800 having a square shape may be divided into second coding units 810a and 810b having a non-square shape, and the second coding units 810a and 810b may be independently divided into third coding units 820a to 820e. According to an embodiment, the encoder 220 may determine a plurality of third coding units 820a and 820b by dividing the left coding unit 810a in the second coding unit in the horizontal direction, and divide the right coding unit 810b into an odd number of third coding units 820c to 820e. Figure 8 The process of determining that the current coding unit is divided into an odd number of coding units by the image encoding apparatus 200 may be the same as that described above with reference to Figure 8 This is an inverse process of the operation of the described image decoding device 100, so its detailed description is not provided again.

[0276] Fig. 9 The process of determining at least one coding unit when the image encoding apparatus 200 divides the first coding unit 900 according to an embodiment is shown. According to an embodiment, the encoder 220 may divide the first coding unit 900, and the bitstream generator 210 may generate a bitstream including at least one of block shape information indicating the shape of the first coding unit and division shape information indicating the shape into which the first coding unit 900 is divided. The first coding unit 900 having a square shape may be divided into four coding units having a square shape or a plurality of coding units having a non-square shape. For example, referring to Fig. 9 , the encoder 220 may divide the first coding unit 900 into a plurality of coding units having a non-square shape, in which case the bitstream generator 210 may generate a bitstream including block shape information indicating that the first coding unit 900 is a square and division shape information indicating that the first coding unit 900 is divided into coding units having a non-square shape. Specifically, the encoder 220 may divide the first coding unit 900 having a square shape into an odd number of coding units, i.e., second coding units 910a to 910c determined in a vertical direction or second coding units 920a to 920c determined in a horizontal direction, in which case the bitstream generator 210 may generate a bitstream including division shape information indicating that the first coding unit 900 is divided in a horizontal or vertical direction to determine an odd number of coding units. Due Fig. 9The process of determining at least one coding unit by dividing the first coding unit 900 by the image coding device 200 is the above reference Fig. 9 This is an inverse process of the operation of the described image decoding device 100, and thus a detailed description thereof will not be provided.

[0277] Fig.10 It is shown that when a second coding unit having a non-square shape determined when the first coding unit 1000 is split satisfies a predetermined condition according to an embodiment, the shape of the second coding unit that can be split is defined by the image encoding apparatus 200.

[0278] According to an embodiment, the encoder 220 may determine to divide the first coding unit 1000 having a square shape into second coding units 1010a and 1010b or 1020a and 1020b having a non-square shape. The second coding units 1010a and 1010b or 1020a and 1020b may be independently divided. Therefore, the encoder 220 may determine to divide the second coding units 1010a and 1010b or 1020a and 1020b into a plurality of coding units or not to divide the second coding units 1010a and 1010b or 1020a and 1020b. The operation of defining the shape of the second coding unit having a non-square shape that can be divided by the image encoding device 200 when the second coding unit satisfies a predetermined condition is the same as that described above with reference to Fig.10 The operation of the described image decoding device 100 is reversed, and a detailed description thereof is not provided.

[0279] Fig.11 200 according to an embodiment shows a process of splitting a coding unit by the image encoding apparatus 200 when the split shape information does not indicate that a coding unit having a square shape is split into four coding units having a square shape. In this regard, the operation of the image encoding apparatus 200 may be the same as that described above with reference to FIG. Fig.11 The operation of the described image decoding device 100 is reversed, and thus a detailed description thereof is not provided.

[0280] Fig.12 It is shown that a processing order among a plurality of coding units according to an embodiment may be changed according to a split process of the coding units.

[0281] According to an embodiment, the encoder 220 may split the first coding unit 1200 having a square shape in at least one of a horizontal direction and a vertical direction. According to an embodiment, the bitstream generator 210 may generate a bitstream including block shape information indicating that the first coding unit 1200 has a square shape and division shape information indicating that the first coding unit 1200 is split in at least one of a horizontal direction and a vertical direction.

[0282] According to an embodiment, the encoder 220 may determine a second coding unit (eg, second coding units 1210a, 1210b, 1220a, 1220b, 1230a, 1230b, 1230c, 1230d, etc.) by dividing the first coding unit 1200. Fig.12 , the second coding units 1210a and 1210b or 1220a and 1220b having a square shape and determined when the first coding unit 1200 is divided only in the horizontal or vertical direction may be independently divided. For example, the encoder 220 may determine the third coding units 1216a to 1216d by dividing each of the second coding units 1210a and 1210b (which are generated by dividing the first coding unit 1200 in the vertical direction) in the horizontal direction, or may determine the third coding units 1226a to 1226d by dividing each of the second coding units 1220a and 1220b (which are generated by dividing the first coding unit 1200 in the horizontal direction) in the horizontal direction. Fig.12 The operation of the related image encoding device 200 is the same as that of the above reference Fig.12 The operation of the described image decoding device 100 is reversed, and a detailed description thereof is not provided.

[0283] Fig.13 1 and 10. In the example embodiment, when a plurality of coding units are determined when a coding unit is recursively split, a process of determining the depth of a coding unit when the shape and size of the coding unit are changed is shown. Fig.13 In contrast to the process in which the decoder 120 of the described image decoding apparatus 100 determines the depth of a coding unit, a detailed description thereof is not provided.

[0284] According to an embodiment, the image encoding apparatus 200 may determine whether a plurality of coding units determined when a current coding unit is split have a specific split shape based on a value of a PID for distinguishing coding units. Fig.14 , the image encoding apparatus 200 may determine an even number of encoding units 1412a and 1412b or an odd number of encoding units 1414a to 1414c by dividing the first encoding unit 1410 having a rectangular shape having a height longer than a width. The image encoding apparatus 200 may use a PID indicating each encoding unit to distinguish each of the plurality of encoding units. According to an embodiment, the PID may be obtained from a sample at a predetermined position in each encoding unit (e.g., an upper left sample). Fig.14 The operation of the related image encoding device 200 can be referred to above. Fig.14 The operation of the described image decoding device 100 is reversed, and thus a detailed description thereof is not provided.

[0285] Fig.15 It is shown that a plurality of coding units according to an embodiment are determined according to a plurality of predetermined data units included in a picture. According to an embodiment, the encoder 220 may use the above-mentioned reference coding unit as a predetermined data unit from which the coding unit is recursively divided. Fig.15 The operation of the image encoding device 200 using the reference encoding unit can be the same as that of the above reference Fig.15 The described image decoding apparatus 100 uses the operation of the reference encoding unit in contrast, and a detailed description thereof is not provided.

[0286] According to an embodiment, the bitstream generator 210 of the image encoding apparatus 200 may generate a bitstream including at least one of information about the shape of the reference coding unit and information about the size of the reference coding unit according to various data units. Figure 3 The process of dividing the current coding unit 300 describes the process of determining at least one coding unit included in the reference coding unit 1500 having a square shape, and the above has been described by Figure 4 The process of splitting the current coding unit 400 or 450 describes a process of determining at least one coding unit included in the reference coding unit 1500 having a non-square shape, and thus a detailed description thereof is not provided again.

[0287] According to an embodiment, the encoder 220 may use an index for identifying the size and shape of the reference coding unit to determine the size and shape of the reference coding unit according to some data units predetermined based on a predetermined condition. In other words, the bitstream generator 210 may generate a bitstream including an index for identifying the size and shape of the reference coding unit according to a data unit (e.g., a data unit having a size equal to or smaller than a slice) that satisfies a predetermined condition among various data units (e.g., a sequence, a picture, a slice, a slice segment, and a maximum coding unit). The encoder 220 may use the index to determine the size and shape of the reference coding unit according to the data unit that satisfies the predetermined condition. According to an embodiment, at least one of the sizes and shapes of the reference coding unit related to the index indicating the size and shape of the reference coding unit may be predetermined. In other words, the encoder 220 may select at least one of the sizes and shapes of the predetermined reference coding unit according to the index to determine at least one of the sizes and shapes of the reference coding unit included in the data unit as a standard for determining the index. Since the operation of the encoder 220 using the operation for identifying the size and shape of the reference coding unit may be similar to the operation of the decoder described above, a detailed description thereof is not provided.

[0288] Fig.16A processing block as a standard in a step of determining an order of determining reference coding units included in the picture 1600 according to an embodiment is shown.

[0289] According to an embodiment, the encoder 220 may determine the size of at least one processing block included in the image by obtaining information about the size of the processing block. The encoder 220 may determine the size of at least one processing block included in the image, and the bitstream generator 210 may generate a bitstream including the information about the size of the processing block. The size of such a processing block may be a predetermined size of a data unit indicated by the information about the size of the processing block.

[0290] According to an embodiment, the bitstream generator 210 of the image encoding device 200 may generate a bitstream including information about the size of a processing block according to a specific data unit. For example, the bitstream including information about the size of a processing block may be generated according to data units such as an image, a sequence, a picture, a slice, and a slice segment. In other words, the bitstream generator 210 may generate a bitstream including information about the size of a processing block according to several data units, and the encoder 220 may determine the size of at least one processing block for dividing a picture by using the information about the size of the processing block, wherein the size of the processing block may be an integer multiple of the size of the reference coding unit.

[0291] According to an embodiment, the encoder 220 may determine the sizes of the processing blocks 1602 and 1612 included in the picture 1600. For example, the encoder 220 may determine the sizes of the processing blocks based on information about the sizes of the processing blocks. Fig.16 According to an embodiment, the encoder 220 may determine the horizontal size of the processing blocks 1602 and 1612 to be four times the horizontal size of the reference coding unit, and determine the vertical size of the processing blocks 1602 and 1612 to be four times the vertical size of the reference coding unit. The encoder 220 may determine the determination order of at least one reference coding unit in at least one processing block. Since the operation of the encoder 220 related to the processing block may be the same as that described above with reference to Fig.16 The operation of the decoder 120 described is similar and a detailed description thereof will not be provided.

[0292] According to an embodiment, the bitstream generator 210 of the image encoding device 200 may generate a bitstream including block shape information indicating the shape of the current coding unit or partition shape information indicating a method of dividing the current coding unit. The block shape information or the partition shape information may be included in a bitstream associated with various data units. For example, the bitstream generator 210 of the image encoding device 200 may use the block shape information or the partition shape information included in a sequence parameter set, a picture parameter set, a video parameter set, a slice header, and a slice segment header. In addition, the bitstream generator of the image encoding device 200 may generate a bitstream including a syntax indicating the block shape information or the partition shape information according to the maximum coding unit, the reference coding unit, and the processing block.

[0293] According to an embodiment, the encoder 220 may differently determine the type of division shape into which the coding unit may be divided according to a predetermined data unit. According to an embodiment, the encoder 220 of the image encoding apparatus 200 may differently determine a combination of shapes into which the coding unit may be divided according to a predetermined data unit (e.g., a sequence, a picture, and a slice).

[0294] Fig.17 Illustrations of coding units that may be determined for each picture when a combination of shapes for splitting coding units differs according to each picture, according to an embodiment.

[0295] Reference Fig.17 , the encoder 220 may determine the combination of division shapes into which the coding unit may be divided differently for each picture. For example, the encoder 220 may encode the image by using the picture 1700 that may be divided into 4 coding units, the picture 1710 that may be divided into 2 or 4 coding units, and the picture 1720 that may be divided into 2, 3, or 4 coding units included in at least one picture in the image. The encoder 220 may divide the picture 1700 into 4 square coding units. The encoder 220 may divide the picture 1710 into 2 or 4 coding units. The encoder 220 may divide the picture 1720 into 2, 3, or 4 coding units. Since the above-mentioned combination of division shapes is only an embodiment for describing the operation of the image encoding device 200, the embodiment is not applied to restrictively interpret the combination of division shapes, and various combinations of division shapes may be used for each predetermined data unit.

[0296] According to an embodiment, the encoder 220 of the image encoding apparatus 200 may determine a combination of division shapes into which a coding unit may be divided according to a predetermined data unit by using an index indicating a combination of division shape information, and accordingly, a combination of different division shapes may be used according to a predetermined data unit. In addition, the bitstream generator 210 of the image encoding apparatus 200 may generate a bitstream including an index indicating a combination of division shape information according to a predetermined data unit (e.g., a sequence, a picture, and a slice). For example, the bitstream generator 210 may generate a bitstream including an index indicating a combination of division shape information according to a sequence parameter set, a picture parameter set, a slice header, or a slice segment header.

[0297] Fig.18 and Fig.19 Various shapes of coding units that may be determined based on division shape information that may be represented in a binary code, according to an embodiment are illustrated.

[0298] According to an embodiment, the encoder 220 of the image encoding apparatus 200 may divide the coding unit into various shapes, and the bitstream generator 210 may generate a bitstream including block shape information and division shape information. The shape of the coding unit that may be divided may correspond to various shapes including the shapes described above through the above embodiments. Fig.18 , the encoder 220 may divide the coding unit having a square shape in at least one of a horizontal direction and a vertical direction based on the division shape information, and divide the coding unit having a non-square shape in a horizontal direction or a vertical direction. Since the characteristics of the binary code regarding the division shape information that can be used by the image encoding apparatus 200 can be the same as those described above by Fig.18 and Fig.19 The characteristics of the described image decoding device 100 are accordingly such that a detailed description thereof will not be provided.

[0299] The image encoding apparatus 200 according to an embodiment may generate prediction data by performing inter prediction or intra prediction on a coding unit, generate residual data by performing inverse transformation on a transformation unit included in a current coding unit, and encode the current coding unit by using the generated prediction data and the residual data.

[0300] The prediction mode of the coding unit according to an embodiment may be at least one of an intra mode, an inter mode, and a skip mode. According to an embodiment, a prediction mode having a minimum error may be selected by independently performing prediction according to coding units.

[0301] When a coding unit having a 2N×2N shape according to an embodiment is divided into two coding units having a 2N×N shape or an N×2N shape, inter-mode prediction and intra-mode prediction may be performed separately on each of the coding units. In addition, according to an embodiment, the encoder 220 of the image encoding device 200 may encode the coding unit by using a CU skip mode not only when the coding unit has a square shape but also when the coding unit has a non-square shape. Since an image can be decoded by using a CU skip mode even when a coding unit determined based on at least one of block shape information and division shape information has not only a square shape but also a non-square shape, a relatively more adaptive skip mode may be used, so that image encoding / decoding efficiency may be improved. The characteristics of the image encoding device 200 using the skip mode in such a coding unit having a non-square shape may be similar to the characteristics described above in relation to the use of the skip mode of the image encoding device 200, and a detailed description thereof is not provided.

[0302] Fig. 22 A process of merging or dividing coding units determined according to a predetermined encoding method according to an embodiment is illustrated.

[0303] According to an embodiment, the image encoding apparatus 200 may determine a coding unit for dividing a picture by using a predetermined coding unit. For example, the image encoding apparatus 200 may divide a coding unit of a current depth or four coding units of a lower depth based on division information of the coding unit. As described above, according to an embodiment, the image encoding apparatus 200 may determine a coding unit by using block division information indicating that a current coding unit always has a square shape and division shape information indicating that the current coding unit is or is not divided into four square coding units. Fig. 22 , the picture 2200 or 2220 may be divided into square coding units determined according to predetermined coding units.

[0304] However, when the above predetermined coding unit is used, whether the current coding unit is divided is determined based on whether a relatively small object included in the current coding unit is suitable for presentation, so that large objects and small objects in the picture may not be encoded by one coding unit. Here, an object is a group of samples included in a picture and may represent a region of samples that is distinguished from other regions because the samples have similar sample values. Fig. 22, the image encoding device 200 can determine the coding unit for encoding the small object 2221 by dividing the first coding unit 2222 into four coding units of the lower depth to reconstruct the small object 2221. However, since the large object 2223 is not included in the current coding unit 2222, it is not suitable to encode the large object 2223 by using the current coding unit 2222. In addition, since the current coding unit 2222 is divided to encode the small object 2221, as a result, unnecessary division processing of the coding unit is performed to encode the large object 2223, which is inefficient. In other words, when the image encoding device 200 is able to use one coding unit to encode part of the large object 2223, image encoding can be efficiently performed.

[0305] According to an embodiment, the encoder 220 of the image encoding device 200 may divide the current coding unit by using at least one of block shape information and division shape information, wherein the block shape information may be predetermined so that only a square shape is used, and the division shape information may be predetermined so that the current coding unit is not divided or is divided into four square coding units. Such processing may correspond to the coding unit determination processing used in the above predetermined encoding method. In this case, the encoder may use sample values ​​included in the picture to merge coding units determined by using a predetermined encoding method or to divide the determined coding units. For example, the encoder 220 detects various objects included in the picture by checking areas with similar sample values, and performs a merging / division process of the coding unit based on the areas of the detected objects.

[0306] Reference Fig. 22 According to an embodiment, the encoder 220 may determine a plurality of coding units for dividing the picture 2200 by using a predetermined encoding method. However, although a region 2201 having similar sample values ​​exists in the picture, a process of dividing the similar region into a plurality of coding units instead of one coding unit may be performed. In this case, even when the coding unit is determined by a predetermined encoding method, the encoder 220 may merge the coding units into one coding unit 2202 and encode the coding unit 2202. Referring to Fig. 22 According to another embodiment, the encoder 220 may divide a coding unit 222 for encoding a small object 2221 into four coding units by using a predetermined encoding method. Since the coding units thus obtained are not all included in the large object 2223, the encoder 220 may perform a merging process 2225 on coding units having similar sample values ​​to obtain one coding unit.

[0307] According to an embodiment, the encoder 220 may determine the coding unit by using the split information of the coding unit, not splitting the coding unit, or by splitting the coding unit into four coding units using a predetermined encoding method, and then splitting the coding unit again in consideration of the sample values ​​of the samples included in the picture. In other words, in order to determine the coding unit according to the object, the decoder 120 may not only merge the coding units, but also split the predetermined coding units. Fig. 22 , the encoder 220 may merge the coding units for the object 2223 and perform a splitting process 2226 on the merged coding units for the object 2223 to determine an optimal coding unit for the object 2223. In other words, the encoder 220 may determine a region that does not include the object 2223 as a coding unit 2227 separated from the object 2223 through the splitting process 2226.

[0308] When a bitstream about an image is generated after merging or dividing coding units determined according to a predetermined encoding method through the operation of the image encoding device 200, the image decoding device 100 can decode the image by performing an image decoding method corresponding to an inverse operation of the image encoding method after obtaining the bitstream.

[0309] Fig.23 An index according to a Z scan order of coding units according to an embodiment is shown.

[0310] The encoder 220 of the image encoding apparatus 200 according to the embodiment may scan the lower layer data units included in the upper layer data units according to the Z scan order. In addition, the image encoding apparatus 200 according to the embodiment may sequentially access data according to the Z scan index in the coding units included in the maximum coding unit or the processing block. Figure 3 or Figure 4 As described, the encoder 220 of the image encoding apparatus 200 according to the embodiment may divide the reference coding unit into at least one coding unit. Here, coding units having a square shape and coding units having a non-square shape may be mixed in the reference coding unit. Since the characteristics of the index according to the Z scanning order of the coding unit for the image encoding apparatus 200 may be similar to the characteristics of the image decoding apparatus 100, a detailed description thereof is not provided.

[0311] Although the present disclosure has been specifically shown and described with reference to the embodiments of the present disclosure, it will be understood by those skilled in the art that various changes in form and detail may be made without departing from the spirit and scope of the present disclosure as defined by the appended claims. The embodiments should be understood to be for descriptive purposes only, not for limiting purposes. Therefore, the scope of the present disclosure is not limited by the detailed description of the present disclosure, but by the appended claims, and all differences within the scope will be interpreted as included in the present disclosure.

[0312] The embodiments of the present disclosure may be written as a computer program and may be implemented in a general-purpose digital computer that executes the program using a computer-readable recording medium. Examples of computer-readable recording media include magnetic storage media (e.g., ROM, floppy disks, hard disks, etc.), optical recording media (e.g., CD-ROM or DVD), etc.

Claims

1. An image decoding method for decoding an image, the image decoding method comprising: obtaining, from a bitstream, information indicating a size of a maximum coding unit and information related to a size of a reference coding unit; obtaining a plurality of maximum coding units by dividing the image according to information indicating a size of the maximum coding unit; determining a plurality of reference coding units by dividing a width and a height of a maximum coding unit among the plurality of maximum coding units according to information related to a size of the reference coding unit; determining a reference coding unit as a first coding unit among the plurality of reference coding units; obtaining, from the bitstream, division shape information about a first coding unit included in the image, wherein the division shape information indicates whether the first coding unit is vertically or horizontally divided into sub coding units and whether the first coding unit is divided into two sub coding units or three sub coding units; determining sub coding units split from the first coding unit based on the division shape information; and When the sub-coding unit is not divided into smaller sub-coding units, the sub-coding unit is decoded to reconstruct the image, Among them, when the first coding unit is vertically divided into three sub-coding units according to the division shape information, the middle sub-coding unit in the middle of the three sub-coding units is allowed to be further vertically divided into three sub-coding units, is not allowed to be further vertically divided into two sub-coding units and is not allowed to be further divided into four sub-coding units by dividing the width and height of the middle sub-coding unit.

2. An image decoding device for decoding an image, the image decoding device comprising: a bitstream acquirer configured to acquire information indicating a size of a maximum coding unit and information related to a size of a reference coding unit from a bitstream, and acquire division shape information about a first coding unit included in an image from the bitstream, wherein the division shape information indicates whether the first coding unit is vertically or horizontally divided into sub-coding units and whether the first coding unit is divided into two sub-coding units or three sub-coding units; A decoder configured to obtain a plurality of maximum coding units by dividing an image according to information indicating a size of a maximum coding unit, determine a plurality of reference coding units by dividing a width and a height of a maximum coding unit among the plurality of maximum coding units according to information related to the size of the reference coding unit, determine a reference coding unit as a first coding unit among the plurality of reference coding units, determine a sub coding unit divided from the first coding unit based on division shape information, and when the sub coding unit is not divided into smaller sub coding units, decode the sub coding unit to reconstruct the image, Among them, when the first coding unit is vertically divided into three sub-coding units according to the division shape information, the middle sub-coding unit in the middle of the three sub-coding units is allowed to be further vertically divided into three sub-coding units, is not allowed to be further vertically divided into two sub-coding units and is not allowed to be further divided into four sub-coding units by dividing the width and height of the middle sub-coding unit.

3. An image encoding method for encoding an image, the image encoding method comprising: Obtaining a plurality of maximum coding units by dividing the image according to the size of the maximum coding unit; determining a plurality of reference coding units by dividing a width and a height of a maximum coding unit among the plurality of maximum coding units according to a size of the reference coding unit; determining a reference coding unit as a first coding unit among the plurality of reference coding units; Vertically or horizontally dividing the first coding unit into two sub-coding units or three sub-coding units; When the sub-coding unit is not divided into smaller sub-coding units, encoding the sub-coding unit in the image; generating a bitstream including information indicating a size of a maximum coding unit, information about a size of a reference coding unit, and division shape information about a first coding unit included in an image, wherein the division shape information indicates whether the first coding unit is vertically or horizontally divided into sub coding units and whether the first coding unit is divided into two sub coding units or three sub coding units, Among them, when the first coding unit is vertically divided into three sub-coding units according to the division shape information, the middle sub-coding unit in the middle of the three sub-coding units is allowed to be further vertically divided into three sub-coding units, is not allowed to be further vertically divided into two sub-coding units and is not allowed to be further divided into four sub-coding units by dividing the width and height of the middle sub-coding unit.

4. An image encoding device for encoding an image, the image encoding device comprising: a bitstream generator configured to generate a bitstream including information indicating a size of a maximum coding unit, information about a size of a reference coding unit, and division shape information about a first coding unit included in an image, wherein the division shape information indicates whether the first coding unit is vertically or horizontally divided into sub-coding units and whether the first coding unit is divided into two sub-coding units or three sub-coding units; An encoder configured to obtain a plurality of maximum coding units by dividing an image according to a size of a maximum coding unit, determine a plurality of reference coding units by dividing a width and a height of a maximum coding unit among the plurality of maximum coding units according to a size of the reference coding unit, determine a reference coding unit as a first coding unit among the plurality of reference coding units, vertically or horizontally divide the first coding unit into two sub-coding units or three sub-coding units, and when a sub-coding unit is not divided into smaller sub-coding units, encode the sub-coding units in the image, Among them, when the first coding unit is vertically divided into three sub-coding units according to the division shape information, the middle sub-coding unit in the middle of the three sub-coding units is allowed to be further vertically divided into three sub-coding units, is not allowed to be further vertically divided into two sub-coding units and is not allowed to be further divided into four sub-coding units by dividing the width and height of the middle sub-coding unit.

Images