Method and apparatus for encoding and decoding a video according to an encoding order

Abstract

A method and apparatus for encoding and decoding a video according to an encoding order are provided. A video decoding method includes obtaining split information indicating whether a current block is split, splitting the current block into at least two lower blocks when the split information indicates that the current block is split, obtaining encoding order information indicating an encoding order of the at least two lower blocks of the current block, determining a decoding order of the at least two lower blocks according to the encoding order information, and decoding the at least two lower blocks according to the decoding order.

Description

[0001] This application is a divisional application of the invention patent application filed on April 19, 2017, with application number "201780032518.5" and titled "Method and apparatus for encoding and decoding video according to encoding order". Technical Field

[0002] Methods and apparatus consistent with exemplary embodiments of this application relate to encoding and decoding video, and more specifically, to intra-frame or inter-frame prediction techniques for determining the encoding and decoding order of images. Background Technology

[0003] When high-quality video is encoded, a large amount of data is generated. However, since the bandwidth allowed for transmitting video data may be limited, the data rate applied to transmitting video data may be correspondingly limited. Therefore, in order to efficiently transmit video data, there is a need to provide video data encoding and decoding methods with the highest image quality and higher compression.

[0004] Video data can be compressed by utilizing spatial and temporal redundancy between pixels. Because spatially adjacent pixels and temporally adjacent pixels often share common characteristics, encoded information is sent in data units composed of pixels to remove redundancy between adjacent pixels.

[0005] Instead of directly sending the pixel values ​​of the pixels included in the data unit, information about the method used to obtain these pixel values ​​is sent. For each data unit, a prediction method is determined for predicting pixel values ​​similar to the original value, and encoded information about the prediction method is sent from the encoder to the decoder. Because the predicted value is not exactly equal to the original value, residual data representing the difference between the original and predicted values ​​is also sent from the encoder to the decoder.

[0006] When predictions are accurate, the amount of data used to specify the prediction method increases, but the size of the residual data decreases accordingly. Therefore, the prediction method is determined by considering both the size of the encoded information and the size of the residual data. Specifically, data units divided from the frame can have one of various sizes. In this respect, as the size of the data unit increases, the prediction accuracy decreases, and the amount of encoded information also decreases. Therefore, the size of the blocks is determined based on the characteristics of the frame.

[0007] Prediction methods include intra-frame prediction and inter-frame prediction. Intra-frame prediction involves predicting pixels in a block from its neighboring pixels. Inter-frame prediction involves predicting pixels by referencing pixels in different frames that include the block. Therefore, spatial redundancy is removed by intra-frame prediction, and temporal redundancy is removed by inter-frame prediction.

[0008] As the number of prediction methods increases, the size of the coding information used to indicate the prediction methods also increases. Therefore, when predicting coding information from different blocks that will be applied to the blocks, the size of the coding information can be reduced.

[0009] Ultimately, since the loss of video data is allowed to reach a level that is imperceptible to the human eye, the residual data can be lossily compressed according to the transform and quantization processes, and by doing so, the amount of residual data can be reduced. Summary of the Invention

[0010] Technical issues

[0011] To improve coding efficiency, it is necessary to determine the optimal coding order from multiple coding order candidates.

[0012] Solution

[0013] Multiple aspects of exemplary embodiments provide a video encoding method, including: determining whether to divide a current block and determining the encoding order of a plurality of lower-level blocks. Multiple aspects of exemplary embodiments also provide a video decoding method, including: dividing a current block and determining the encoding order of the divided plurality of lower-level blocks. Multiple aspects of exemplary embodiments also provide a non-transitory computer-readable recording medium having programs recorded on it for performing the video encoding method and the video decoding method.

[0014] Further aspects of the exemplary embodiments will be set forth in part in the description which follows, and will also be apparent from the description, or may be learned by practice of the embodiments.

[0015] According to one aspect of an exemplary embodiment, a video decoding method is provided, comprising: obtaining partitioning information indicating whether to partition a current block to be decoded; when the partitioning information indicates partitioning the current block, partitioning the current block into at least two lower-level blocks; obtaining encoding order information indicating an encoding order, wherein the at least two lower-level blocks of the current block are encoded in the encoding order; determining a decoding order of the at least two lower-level blocks according to the encoding order indicated by the encoding order information; and decoding the at least two lower-level blocks according to the decoding order.

[0016] According to one aspect of an exemplary embodiment, a video coding method is provided, comprising: determining a coding order of a plurality of lower-level blocks of a current block, wherein the current block is divided into the plurality of lower-level blocks; determining whether to divide the current block by comparing the coding efficiency of the current block with the coding efficiency of the plurality of lower-level blocks according to the coding order; and generating a bitstream including partitioning information indicating whether the current block is divided into the plurality of lower-level blocks.

[0017] According to one aspect of an exemplary embodiment, a video decoding apparatus is provided, comprising: a block divider configured to: obtain partitioning information indicating whether to divide a current block to be decoded, and when the partitioning information indicates to divide the current block, divide the current block into at least two lower-level blocks; an encoding order determiner configured to: obtain encoding order information indicating an encoding order, wherein the at least two lower-level blocks of the current block are encoded according to the encoding order, and determine a decoding order of the at least two lower-level blocks according to the encoding order indicated by the encoding order information; and a decoder configured to decode the at least two lower-level blocks according to the decoding order.

[0018] According to one aspect of an exemplary embodiment, a video encoding apparatus is provided, comprising: an encoding order determiner configured to determine the encoding order of a plurality of lower-level blocks of a current block, wherein the current block is divided into the plurality of lower-level blocks; a block partitioner configured to determine whether to partition the current block by comparing the encoding efficiency of the current block with the encoding efficiency of the plurality of lower-level blocks according to the encoding order; and an output unit configured to generate a bitstream including partitioning information indicating whether the current block is partitioned into the plurality of lower-level blocks.

[0019] Beneficial effects

[0020] It is expected that encoding efficiency can be improved by encoding or decoding images according to the best encoding order selected from multiple encoding order candidates. Attached Figure Description

[0021] The above and other aspects will become clearer and more readily understood from the following description of exemplary embodiments in conjunction with the accompanying drawings, wherein:

[0022] Figure 1A A block diagram of an image encoding apparatus based on a tree-structured encoding unit according to an exemplary embodiment of the present disclosure is shown.

[0023] Figure 1B A block diagram of an image decoding device based on a tree-structured coding unit according to an exemplary embodiment is shown.

[0024] Figure 2 This illustrates a process for determining at least one coding unit when the current coding unit is divided, according to an exemplary embodiment.

[0025] Figure 3 This illustrates a process for determining at least one coding unit when coding units having a non-square shape are divided, according to an exemplary embodiment.

[0026] Figure 4 This illustrates a process for dividing coding units based on at least one of block shape information and partition shape information, according to an exemplary embodiment.

[0027] Figure 5 A method for determining a predetermined coding unit from an odd number of coding units is illustrated according to an exemplary embodiment;

[0028] Figure 6 This illustrates the order in which multiple coding units are processed when multiple coding units are determined in the case of a current coding unit being divided, according to an exemplary embodiment.

[0029] Figure 7 This illustrates a process, according to an exemplary embodiment, for determining that the current coding unit is divided into an odd number of coding units when multiple coding units cannot be processed in a predetermined order;

[0030] Figure 8 This illustrates a process for determining at least one coding unit when the first coding unit is divided, according to an exemplary embodiment.

[0031] Figure 9 The shape of the second coding unit that can be divided when the second coding unit, which has a non-square shape, is determined to be divided when the first coding unit is divided according to an exemplary embodiment, satisfies a predetermined condition is limited;

[0032] Figure 10 This illustrates a process for dividing a coding unit with a square shape into four coding units with square shapes, according to an exemplary embodiment.

[0033] Figure 11 This illustrates that the processing order among multiple coding units according to an exemplary embodiment can be changed based on the division of the coding units into processing steps;

[0034] Figure 12 This illustrates a process for determining the depth of a coding unit when the shape and size of the coding unit change, in the case where multiple coding units are determined when the coding unit is recursively divided, according to an exemplary embodiment.

[0035] Figure 13 The illustration shows a depth that can be determined based on the shape and size of the encoding unit, and a partial index (PID) for distinguishing between encoding units, according to an exemplary embodiment.

[0036] Figure 14 This illustrates how, according to an exemplary embodiment, multiple encoding units are determined based on multiple predetermined data units included in the screen;

[0037] Figure 15 This illustrates a standard processing block in the process of determining the order of reference coding units included in a frame, according to an exemplary embodiment.

[0038] Figure 16This illustrates a video decoding apparatus, according to an exemplary embodiment, which involves dividing a current block and determining the encoding order of the resulting lower-level blocks;

[0039] Figure 17 This illustrates a video encoding apparatus, according to an exemplary embodiment, which involves dividing a current block and determining the encoding order of the resulting lower-level blocks;

[0040] Figure 18A , Figure 18B and Figure 18C The default encoding order is shown according to an exemplary embodiment;

[0041] Figure 19A and Figure 19B This shows the cases where the encoding unit is encoded in the forward direction and the cases where the encoding unit is encoded in the reverse direction;

[0042] Figure 20 The tree structure of the maximum coding unit is shown, wherein the tree structure of the maximum coding unit is used to describe the maximum coding unit and the coding order of multiple coding units included in the maximum coding unit;

[0043] Figure 21A and Figure 21B This demonstrates a method for determining the coding order when the coding unit is divided into three coding units;

[0044] Figure 22 and Figure 23 A method for predicting the current block is shown, wherein the method differs depending on whether the right block and the left block are reconstructed;

[0045] Figure 24 This illustrates a video decoding method performed by a video decoding device according to an exemplary embodiment;

[0046] Figure 25 This illustrates a video encoding method performed by a video encoding device according to an exemplary embodiment.

[0047] Best practice

[0048] Multiple aspects of exemplary embodiments provide a video encoding method, the video encoding method including determining whether to divide a current block and determining the encoding order of a plurality of lower-level blocks. Multiple aspects of exemplary embodiments also provide a video decoding method, the video decoding method including dividing a current block and determining the encoding order of the divided plurality of lower-level blocks. Multiple aspects of exemplary embodiments also provide a non-transitory computer-readable recording medium having a program for performing the video encoding method and the video decoding method.

[0049] According to one aspect of an exemplary embodiment, a video decoding method is provided, comprising: obtaining partitioning information indicating whether to partition a current block to be decoded; when the partitioning information indicates partitioning the current block, partitioning the current block into at least two lower-level blocks; obtaining encoding order information indicating an encoding order, wherein the at least two lower-level blocks of the current block are encoded in the encoding order; determining a decoding order of the at least two lower-level blocks according to the encoding order indicated by the encoding order information; and decoding the at least two lower-level blocks according to the decoding order.

[0050] The mode of the present invention

[0051] Various aspects of this disclosure will be more readily understood from the following detailed description of the embodiments and the accompanying drawings. In this regard, the exemplary embodiments may take different forms and should not be construed as limited to the description set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concepts of the exemplary embodiments to those skilled in the art.

[0052] In the following, the terminology used in the specification will be briefly defined, and the embodiments will be described in detail.

[0053] All terms, including descriptive or technical terms as used herein, should be interpreted as having a meaning that would be understood by one of ordinary skill in the art. However, terms may have different meanings depending on the purpose, precedent, or emergence of new technologies for those of ordinary skill in the art. Furthermore, some terms may be arbitrarily chosen, in which case the meaning of the chosen terms will be described in detail in the detailed description of this disclosure. Therefore, the terms used herein must be defined based on their meanings and the description throughout the specification.

[0054] Unless they have a distinctly different meaning in the context, expressions used in the singular form include those used in the plural form.

[0055] When a component "comprises" or "contains" an element, the component may also include other elements, without excluding them, unless a specific description to the contrary exists. Furthermore, the term "unit" in embodiments of this disclosure refers to a software component or hardware component (such as a field-programmable gate array (FPGA) or application-specific integrated circuit (ASIC)) that performs a specific function. However, the term "unit" is not limited to software or hardware. A "unit" may be formed in an addressable storage medium or may be configured to operate one or more processors. Thus, for example, the term "unit" may refer to a component, such as a software component, an object-oriented software component, a class component, and a task component, and may include processes, functions, attributes, procedures, subroutines, program code segments, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, or variables. The functionality provided by components and "units" may be associated with a smaller number of components and "units," or may be divided into additional components and "units."

[0056] The term "current block" refers to one of the coding units, prediction units, and transform units that will be encoded or decoded. The term "lower block" refers to a data unit partitioned from the "current block." The term "upper block" refers to a data unit that includes the "current block."

[0057] In the following text, a "sample" is data assigned to a sampling location in an image and can be data that is the target of processing. For example, pixel values ​​in the spatial domain of an image or transform coefficients in the transform domain can be samples. A unit comprising at least one sample can be defined as a block.

[0058] Reference will now be made in detail to the embodiments, examples of which are illustrated in the accompanying drawings. In the following description, well-known functions or structures will not be described in detail to avoid obscuring the embodiments with unnecessary detail.

[0059] As used herein, the term “and / or” includes any and all combinations of one or more of the items in the relevant list. When a term such as “at least one of…” follows a column of elements, the term modifies the entire column of elements, not any individual element in the list.

[0060] Figure 1A A block diagram of an image encoding device 100 based on a tree-structured encoding unit according to an exemplary embodiment of the present disclosure is shown.

[0061] The image encoding device 100 includes a maximum encoding unit determiner 110, an encoding unit determiner 120, and an output unit 130.

[0062] The maximum coding unit determiner 110 divides the frame or the stripes included in the frame into multiple maximum coding units based on the size of the maximum coding unit. The maximum coding unit can be a data unit with dimensions such as 32×32, 64×64, 128×128, 256×256, etc., and the shape of the data unit can be a width and a length of 2... N The maximum coding unit determiner 110 can provide maximum coding unit size information, which indicates the size of the maximum coding unit, to the output unit 130. The output unit 130 can include the maximum coding unit size information in the bitstream.

[0063] The coding unit determiner 120 determines coding units by partitioning the largest coding unit. A coding unit can be determined based on its maximum size and depth. Depth can be defined as the number of times a coding unit is spatially partitioned from the largest coding unit. When the depth increases by 1, the coding unit is partitioned into at least two coding units. Therefore, as the depth increases, the size of the coding unit decreases according to the depth. Based on rate-distortion optimization, whether to partition the coding unit is determined based on whether partitioning the coding unit is effective. Partition information indicating whether a coding unit has been partitioned can then be generated. The partition information can be represented in the form of one or more flags.

[0064] The coding unit can be divided using various methods. For example, a square coding unit can be divided into four square coding units, each with a width and height half the height and width of the square coding unit. The square coding unit can be divided into two rectangular coding units with half the width. The square coding unit can be divided into two rectangular coding units with half the height. The square coding unit can be divided into three coding units according to its width or height in a 1:2:1 or other ratio.

[0065] A rectangular coding unit whose width is twice its height can be divided into two square coding units. A rectangular coding unit whose width is twice its height can be divided into two rectangular coding units whose width is four times its height. A rectangular coding unit whose width is twice its height can be divided into two rectangular coding units and one square coding unit according to a width ratio of 1:2:1 or other ratios.

[0066] Similarly, a rectangular coding unit whose height is twice its width can be divided into two square coding units. A rectangular coding unit whose height is twice its width can be divided into two rectangular coding units whose height is four times its width. Likewise, a rectangular coding unit whose height is twice its width can be divided into two rectangular coding units and one square coding unit according to a height ratio of 1:2:1 or other ratios.

[0067] When the image encoding device 100 is configured to implement two or more partitioning methods, information regarding the partitioning methods applicable to the encoding unit can be determined for each frame, wherein the partitioning methods are derived from a variety of partitioning methods available to the image encoding device 100. Therefore, only a specific partitioning method can be used for each frame. When the image encoding device 100 is configured to implement only one partitioning method, it is not necessary to separately determine information regarding the partitioning methods applicable to the encoding unit.

[0068] When the partitioning information of a coding unit indicates that a coding unit is to be partitioned, partitioning shape information indicating the partitioning method for the coding unit can be generated. If only one partitioning method can be used in the frame including the coding unit, partitioning shape information may not be generated. If it is determined that the partitioning method is suitable for the coding information adjacent to the coding unit, partitioning shape information may not be generated.

[0069] The largest coding unit can be divided into multiple smallest coding units based on the size information of the smallest coding unit. The depth of the largest coding unit can be defined as the highest depth, and the depth of the smallest coding unit can be defined as the lowest depth. Therefore, a coding unit with a higher depth can include multiple coding units with lower depths.

[0070] Based on the maximum size of the coding unit as described above, the image data of the current frame is divided into maximum coding units. A maximum coding unit may include multiple coding units divided according to depth. Since the maximum coding unit is divided according to depth, the image data in the spatial domain included within the maximum coding unit can be layered according to depth.

[0071] The maximum depth or minimum size of the coding unit can be preset to limit the maximum number of times the maximum coding unit can be divided into layers.

[0072] The coding unit determiner 120 compares the coding efficiency of hierarchically partitioning the current coding unit with the coding efficiency of the current coding unit. Then, based on the comparison result, the coding unit determiner 120 determines whether to partition the current coding unit. When the coding unit determiner 120 determines that partitioning the current coding unit is more efficient, the coding unit determiner 120 partitions the current coding unit hierarchically. However, based on the comparison result, when the coding unit determiner 120 determines that maintaining the current coding unit is more efficient for coding, the coding unit determiner 120 does not partition the current coding unit. The determination of whether to partition the current coding unit can be independent of whether neighboring coding units have been partitioned.

[0073] According to an exemplary embodiment, during the encoding process, it can be determined whether to partition coding units from coding units with larger depths. For example, the encoding efficiency of the coding unit with the largest depth is compared with the encoding efficiency of the coding unit with a depth one less than the largest depth, and it is determined which of the following is effectively encoded in each region of the largest coding unit: the coding unit with the largest depth and the coding unit with a depth one less than the largest depth. Based on the determination result, it is determined whether to partition coding units with a depth one less than the largest depth in each region of the largest coding unit. Then, it is determined which of the following two items is further effectively encoded in each region of the largest coding unit: the coding unit with the largest depth and the coding unit with a depth one less than the largest depth selected according to the determination result, and the coding unit with a depth two less than the largest depth. The same determination process is performed for each of the coding units with smaller depths, and finally, it is determined whether to partition the largest coding unit based on which of the largest coding unit and the hierarchical structure generated by partitioning the largest coding unit hierarchically is further effectively encoded.

[0074] During the encoding process, the decision to partition a coding unit can be made based on coding units with smaller depths. For example, the coding efficiency of the largest coding unit is compared to the coding efficiency of a coding unit with a depth one greater than the largest coding unit, and it is determined which of these two units is effectively encoded. If the largest coding unit is more efficient, it is not partitioned. If the coding efficiency of a coding unit with a depth one greater than the largest coding unit is higher, the largest coding unit is partitioned, and the comparison process is iteratively applied to partition the coding units.

[0075] When checking coding efficiency from coding units with greater depth, a computationally intensive but highly efficient tree structure is obtained. Conversely, when checking coding efficiency from coding units with smaller depth, a less computationally intensive but less efficient tree structure is obtained. Therefore, considering both coding efficiency and computational cost, various methods can be used to design algorithms for obtaining hierarchical tree structures with the largest coding units.

[0076] To determine the coding efficiency of a coding unit based on each depth, the coding unit determiner 120 determines the most efficient prediction and transformation method for coding the coding unit. To determine the most efficient prediction and transformation method, the coding unit can be divided into predetermined data units. Depending on the method of dividing the coding unit, the data units can have one of various shapes. The method performed to determine the data units for coding units is defined as a partitioning pattern. For example, when a 2N×2N coding unit (where N is a positive integer) is no longer divided, the size of the prediction units included in that coding unit is 2N×2N. When a 2N×2N coding unit is divided, the size of the prediction units included in that coding unit can be 2N×N, N×2N, or N×N, depending on the partitioning pattern. The partitioning pattern according to this exemplary embodiment can produce symmetrical data units obtained by symmetrically dividing the height or width of the coding unit, data units obtained by asymmetrically dividing the height or width of the coding unit (such as according to a ratio of 1:n or n:1), data units obtained by diagonally dividing the coding unit, data units obtained by geometrically dividing the coding unit, partitions with arbitrary shapes, etc.

[0077] The coding unit can be predicted and transformed based on the data units included in the coding unit. However, according to this exemplary embodiment, the data units for prediction and the data units for transformation can be determined separately. The data units for prediction can be defined as prediction units, and the data units for transformation can be defined as transformation units. The partitioning patterns applied to the prediction units and the partitioning patterns applied to the transformation units can be different from each other, and the prediction of the prediction units and the transformation of the transformation units can be performed in parallel and independently within the coding unit.

[0078] To determine an effective prediction method, a coding unit can be divided into at least one prediction unit. Similarly, to determine an effective transform method, a coding unit can be divided into at least one transform unit. The division to prediction units and the division to transform units can be performed independently of each other. However, when reconstructed samples from a coding unit are used in intra-frame prediction, a dependency relationship is formed between the prediction units or transform units included in the coding unit, such that the division to prediction units and transform units can influence each other.

[0079] Prediction units included in a coding unit can be predicted using either intra-frame prediction or inter-frame prediction. Intra-frame prediction involves predicting prediction unit samples using reference samples adjacent to the prediction unit. Inter-frame prediction involves predicting prediction unit samples by obtaining reference samples from a reference frame referenced by the current frame.

[0080] For intra-frame prediction, the coding unit determiner 120 can apply various intra-frame prediction methods to the prediction unit to select the most effective intra-frame prediction method. Intra-frame prediction methods include various orientation modes, such as DC mode, planar mode, vertical mode, and horizontal mode.

[0081] Intra-frame prediction can be performed for each prediction unit when reconstructed samples adjacent to the coding unit are used as reference samples. However, when reconstructed samples in the coding unit are used as reference samples, reconstruction for the reference samples in the coding unit precedes prediction for the reference samples in the coding unit, making the prediction order of prediction units dependent on the transformation order of transform units. Therefore, when reconstructed samples in the coding unit are used as reference samples, only the intra-frame prediction method for the transform unit corresponding to the prediction unit and the actual intra-frame prediction can be performed for each transform unit.

[0082] The coding unit determiner 120 determines the optimal motion vector and reference frame to select the most effective inter-frame prediction method. For inter-frame prediction, the coding unit determiner 120 determines multiple motion vector candidates from coding units spatially and temporally adjacent to the current coding unit, and determines the most effective motion vector among the multiple motion vector candidates as the motion vector. Similarly, the coding unit determiner 120 determines multiple reference frame candidates from coding units spatially and temporally adjacent to the current coding unit, and determines the most effective reference frame among the multiple reference frame candidates. In an exemplary embodiment, the reference frame can be determined from a list of reference frames pre-determined for the current frame. In an exemplary embodiment, for prediction accuracy, the most effective motion vector among the multiple motion vector candidates can be determined as the predicted motion vector, and the motion vector can be determined by compensating the predicted motion vector. Inter-frame prediction can be performed in parallel for each prediction unit in the coding unit.

[0083] The coding unit determiner 120 can reconstruct the coding unit by obtaining only information indicating the motion vector and the reference frame, according to a skip mode. According to the skip mode, all coding information, including the residual signal, except for the information indicating the motion vector and the reference frame, is skipped. Because the residual signal is skipped, the skip mode can be used when the prediction accuracy is very high.

[0084] The partitioning mode to be used can be restricted according to the prediction method used for the prediction unit. For example, only the partitioning mode for prediction units of size 2N×2N or N×N can be applied to intra-frame prediction, while the partitioning mode for prediction units of size 2N×2N, 2N×N, N×2N, or N×N can be applied to inter-frame prediction. Additionally, only the partitioning mode for prediction units of size 2N×2N can be applied to the skip mode of inter-frame prediction. The image coding device 100 can change the partitioning mode used for each prediction method according to coding efficiency.

[0085] Image coding apparatus 100 can perform transformations based on coding units or transform units included in coding units. Image coding apparatus 100 can transform residual data, which is the difference between original and predicted values, for pixel pairs included in the coding unit. For example, image coding apparatus 100 can perform lossy compression on the residual data using quantization and Discrete Cosine Transform (DCT) / Discrete Sine Transform (DST). Optionally, image coding apparatus 100 can perform lossless compression on the residual data without quantization.

[0086] Image encoding device 100 can determine the most efficient transform units for quantization and transform. According to an exemplary embodiment, transform units within an encoding unit can be recursively divided into smaller regions in a manner similar to how encoding units are partitioned according to a tree structure. Therefore, residual data within an encoding unit can be partitioned based on transform units having a tree structure based on transform depth. Image encoding device 100 can generate transform partitioning information regarding the partitioning of encoding units and transform units according to the determined tree structure of transform units.

[0087] The transform depth can also be set in the image encoding device 100, where the transform depth indicates the number of times the transform unit is divided by the height and width of the encoding unit. For example, in the current encoding unit of 2N×2N, the transform depth can be 0; when the size of the transform unit is N×N, the transform depth can be 1; and when the size of the transform unit is N / 2×N / 2, the transform depth can be 2. That is to say, the transform unit based on the tree structure can be set according to the transform depth.

[0088] In summary, the coding unit determiner 120 determines the most efficient prediction method for the current prediction unit from among various intra-frame and inter-frame prediction methods. Then, the coding unit determiner 120 determines a prediction unit determination scheme based on the coding efficiency of the prediction results. Similarly, the coding unit determiner 120 determines a transform unit determination scheme based on the coding efficiency of the transform results. Based on the most efficient prediction unit and transform unit determination schemes, the coding efficiency of the coding unit is finally determined. Based on the coding efficiency of the coding units at each depth, the coding unit determiner 120 finally determines the hierarchical structure of the largest coding unit.

[0089] The coding unit determiner 120 can measure coding efficiency of coding units based on depth, prediction efficiency of prediction methods, etc., by using rate-distortion optimization based on Lagrange multipliers.

[0090] The coding unit determiner 120 can generate partitioning information indicating whether to partition coding units according to each depth based on the hierarchical structure of the determined maximum coding unit. Then, the coding unit determiner 120 can generate partitioning pattern information to be used when determining prediction units and transformation unit partitioning information to be used when determining transformation units for the partitioned coding units. Additionally, when coding units can be partitioned using at least two partitioning methods, the coding unit determiner 120 can generate both partitioning information and partitioning shape information indicating the partitioning methods. The coding unit determiner 120 can generate information about the prediction and transformation methods used in the prediction and transformation units.

[0091] Output unit 130 can output multiple pieces of information generated by maximum coding unit determiner 110 and coding unit determiner 120 according to the hierarchical structure of maximum coding units in the bit stream.

[0092] The following will refer to Figures 3 to 12 Describe a method for determining coding units, prediction units, and transform units based on the tree structure of the largest coding unit.

[0093] Figure 1B A block diagram of an image decoding device 150 based on a tree-structured encoding unit is shown according to an exemplary embodiment.

[0094] The image decoding device 150 includes a receiver 160, an encoded information extractor 170, and a decoder 180.

[0095] The definitions of terms used for decoding operations performed by the image decoding device 150 (including coding units, depth, prediction units, transform units, various partitioning information, etc.) are the same as those described above with reference to FIG1 and the image encoding device 100. Since the image decoding device 150 is designed to reconstruct image data, various encoding methods used by the image encoding device 100 can also be applied to the image decoding device 150.

[0096] Receiver 160 receives and parses the bitstream of the encoded video. Encoding information extractor 170 extracts multiple pieces of information from the parsed bitstream that will be used when decoding the maximum coding unit and provides this information to decoder 180. Encoding information extractor 170 can extract information about the maximum size of the coding unit of the current frame from the header, sequence parameter set, or frame parameter set of the current frame.

[0097] Encoding information extractor 170 extracts the final depth and partitioning information of multiple coding units regarding the tree structure from the parsed bitstream, based on each maximum coding unit. The extracted final depth and partitioning information is output to decoder 180. Decoder 180 can partition the maximum coding units based on the extracted final depth and partitioning information, thereby determining the tree structure of the maximum coding units.

[0098] The partitioning information extracted by the encoding information extractor 170 is partitioning information about the tree structure determined to produce the minimum encoding error, wherein the determination is performed by the image encoding device 100. Therefore, the image decoding device 150 can reconstruct the image by decoding the data according to the decoding scheme that produces the minimum encoding error.

[0099] The encoding information extractor 170 can extract partitioning information about data units (such as prediction units and transformation units included in the encoding unit). For example, the encoding information extractor 170 can extract partitioning pattern information about the most efficient partitioning pattern for the prediction unit. The encoding information extractor 170 can extract transformation partitioning information about the most efficient tree structure for the transformation unit.

[0100] The encoding information extractor 170 can obtain information about the most effective prediction method for the prediction units divided from the encoding units. Then, the encoding information extractor 170 can obtain information about the most effective transformation method for the transform units divided from the encoding units.

[0101] Encoded information extractor 170 extracts information from bitstream according to a method for configuring bitstream, wherein the method is performed by output unit 130 of image encoding device 100.

[0102] Decoder 180 can partition the largest coding unit into coding units with the most efficient tree structure based on partitioning information. Then, decoder 180 can partition the coding unit into prediction units according to partitioning pattern information. Decoder 180 can partition the coding unit into transform units according to transform partitioning information.

[0103] Decoder 180 can predict the prediction unit based on information about the prediction method. Decoder 180 can perform inverse quantization and inverse transformation on the residual data, which is the difference between the original value and the predicted value of the pixel, based on information about the method of transforming the transformation unit. Decoder 180 can reconstruct the pixels of the coding unit based on the prediction result of the prediction unit and the transformation result of the transformation unit.

[0104] Figure 2 This illustrates a process for determining at least one coding unit when the image decoding device 150 divides the current coding unit, according to an exemplary embodiment.

[0105] According to this exemplary embodiment, the image decoding device 150 can determine the shape of the coding unit by using block shape information, or it can determine the shape on which the coding unit is divided by using partition shape information. That is, the method of determining the partitioned coding unit indicated by the partition shape information can be based on which block shape is indicated by the block partition information used by the image decoding device 150.

[0106] According to this exemplary embodiment, the image decoding device 150 can use block shape information indicating that the current coding unit has a square shape. For example, the image decoding device 150 can determine whether to divide the coding unit into square units based on the division shape information, that is: whether to divide the square coding unit vertically, whether to divide the square coding unit horizontally, or whether to divide the square coding unit into four coding units. (Refer to...) Figure 2 When the block shape information of the current coding unit 200 indicates a square shape, the decoder 180 may not divide the coding unit 210a with the same size as the current coding unit 200 according to the division shape information indicating that it should not be divided, or it may determine the coding units 210b, 201c and 201d divided based on the division shape information indicating a predetermined division method.

[0107] Reference Figure 2 According to an exemplary embodiment, the image decoding device 150 can determine two encoding units 210b obtained by dividing the current encoding unit 200 vertically based on division shape information indicating division along the vertical direction. The image decoding device 150 can determine two encoding units 210c obtained by dividing the current encoding unit 200 horizontally based on division shape information indicating division along the horizontal direction. The image decoding device 150 can determine four encoding units 210d obtained by dividing the current encoding unit 200 vertically and horizontally based on division shape information indicating division along both the vertical and horizontal directions. However, the division shape used to divide the square encoding units is not limited to the above shapes and can include various shapes that can be indicated by the division shape information. Predetermined division shapes for dividing square encoding units will be described in detail below through various embodiments.

[0108] Figure 3 The illustration shows a process for determining at least one encoding unit when the image decoding device 150 divides the encoding units having a non-square shape, according to an exemplary embodiment.

[0109] According to this exemplary embodiment, the image decoding device 150 can use block shape information indicating that the current coding unit has a non-square shape. The image decoding device 150 can determine whether to divide the current coding unit with a non-square shape, or determine whether to divide the current coding unit with a non-square shape using a predetermined method. (Refer to...) Figure 3 When the block shape information of the current encoding unit 300 or 350 indicates a non-square shape, the image decoding device 150 may, based on the division shape information indicating no division, not divide the encoding unit 310 or 360 with the same size as the current encoding unit 300 or 350, or may determine the encoding units 320a, 320b, 330a, 330b, 330c, 370a, 370b, 380a, 380b, and 380c divided according to the division shape information indicating a predetermined division method. The predetermined division method for dividing non-square encoding units will be described in detail below through various embodiments.

[0110] According to this exemplary embodiment, the image decoding device 150 can determine the shape of the coding unit being divided by using division shape information, and in this case, the division shape information can indicate the number of at least one coding unit generated when the coding unit is divided. (Refer to...) Figure 3 When the division shape information indicates that the current coding unit 300 or 350 is divided into two coding units, the image decoding device 150 can divide the current coding unit 300 or 350 based on the division shape information to determine two coding units 320a and 320b or two coding units 370a and 370b respectively included in the current coding unit 300 or 350.

[0111] According to this exemplary embodiment, when the image decoding device 150 divides a current coding unit 300 or 350 with a non-square shape based on the division shape information, the image decoding device 150 may consider the position of the longer side when dividing the current coding unit 300 or 350 with a non-square shape. For example, the image decoding device 150 may determine multiple coding units by dividing the current coding unit 300 or 350 along the direction of the longer side of the current coding unit 300 or 350, taking into account the shape of the current coding unit 300 or 350.

[0112] According to this exemplary embodiment, when the partitioning shape information indicates that the encoding unit is divided into an odd number of blocks, the image decoding device 150 can determine the odd number of encoding units included in the current encoding unit 300 or 350. For example, when the partitioning shape information indicates that the current encoding unit 300 or 350 is divided into three encoding units, the image decoding device 150 can divide the current encoding unit 300 or 350 into three encoding units 330a, 330b, and 330c, or 380a, 380b, and 380c. According to this exemplary embodiment, the image decoding device 150 can determine the odd number of encoding units included in the current encoding unit 300 or 350, wherein the determined encoding units have different sizes. For example, the size of encoding unit 330b or 380b among the odd number of encoding units 330a, 330b, and 330c, or 380a or 380c, is different from the size of encoding units 330a and 330c, or 380a or 380c. In other words, when the current coding unit 300 or 350 is divided, the coding units that can be determined can have different types of sizes.

[0113] According to this exemplary embodiment, when the shape information indicating that the encoding unit is divided into an odd number of blocks, the image decoding device 150 can determine the odd number of encoding units included in the current encoding unit 300 or 350. Furthermore, a predetermined restriction can be set on at least one encoding unit among the odd number of encoding units generated by dividing the current encoding unit 300 or 350. (Refer to...) Figure 3 The image decoding device 150 can decode the encoding unit 330b or encoding unit 380b at the center of the three encoding units 330a, 330b, and 330c or 380a, 380b, and 380c generated when the current encoding unit 300 or 350 is divided, in a manner different from that of encoding units 330a and 330c or 380a and 380c. For example, unlike encoding units 330a and 330c or 380a and 380c, the image decoding device 150 can restrict the encoding unit 330b or 380b located at the center to not be further divided, or restrict the encoding unit 330b or 380b located at the center to be divided only a specific number of times.

[0114] Figure 4 This illustrates a process by which an image decoding device 150 divides coding units based on at least one of block shape information and segmentation shape information, according to an exemplary embodiment.

[0115] According to this exemplary embodiment, the image decoding device 150 can determine whether to divide a first coding unit 400 having a square shape into a plurality of coding units based on at least one of block shape information and partition shape information. According to this exemplary embodiment, when the partition shape information indicates that the first coding unit 400 is divided along a horizontal direction, the image decoding device 150 can determine a second coding unit 410 by dividing the first coding unit 400 along a horizontal direction. The terms "first coding unit," "second coding unit," and "third coding unit" according to the exemplary embodiment are used in the context of dividing coding units. For example, the second coding unit may be determined when the first coding unit is divided, and the third coding unit may be determined when the second coding unit is divided. The relationship between the first coding unit and the third coding unit as used below can be understood as following the above-described sequential characteristics.

[0116] According to this exemplary embodiment, the image decoding device 150 can determine whether to divide the determined second coding unit 410 into multiple coding units based on at least one of block shape information and partition shape information. (Refer to...) Figure 4 The image decoding device 150 may divide a second coding unit 410, which has a non-square (i.e., rectangular shape) determined by dividing the first coding unit 400, into at least one third coding unit (e.g., third coding units 420a, 420b, 420c, and 420d) based on at least one of block shape information and division shape information, or it may not divide the second coding unit 410. The image decoding device 150 may obtain at least one of block shape information and division shape information, and the image decoding device 150 may divide the first coding unit 400 based on at least one of block shape information and division shape information to obtain a plurality of second coding units (e.g., second coding units 410) with various shapes, and the second coding unit 410 may be divided according to the manner in which the first coding unit 410 is divided based on at least one of block shape information and division shape information. According to this exemplary embodiment, when the first coding unit 400 is divided into a second coding unit 410 based on at least one of block shape information and partition shape information regarding the first coding unit 400, the second coding unit 410 can also be divided into a third coding unit, for example, third coding units 420a, 420b and 420c, 420d, based on at least one of partition shape information and partition shape information regarding the second coding unit. That is, coding units can be recursively divided based on at least one of partition shape information and block shape information associated with the coding unit. The method for recursively dividing coding units will now be described through various embodiments.

[0117] According to this exemplary embodiment, the image decoding device 150 may determine, based on at least one of block shape information and partition shape information, whether to divide each of the third coding units (e.g., third coding units 420a, 420b, 420c, and 420d) into multiple coding units or to not divide the second coding unit 410. The image decoding device 150 may divide the second coding unit 410, which has a non-square shape, into an odd number of third coding units 420b, 420c, and 420d. The image decoding device 150 may set predefined restrictions on predetermined third coding units among the odd number of third coding units 420b, 420c, and 420d. For example, the image decoding device 150 may restrict the coding unit 420c located at the center of the odd number of third coding units 420b, 420c, and 420d to no longer be divided or to be divided a predetermined number of times. (Refer to...) Figure 4 The image decoding device 150 may restrict the encoding unit 420c located at the center of an odd number of third encoding units 420b, 420c, and 420d included in the second encoding unit 410 having a non-square shape, to no longer be divided, to be divided in a predetermined manner (e.g., divided into only four encoding units, or divided into a shape corresponding to the shape into which the second encoding unit 410 is divided), or to be divided only a predetermined number of times (e.g., divided only n times, where n > 0). However, the restriction on the encoding unit 420c located at the center is merely exemplary, and therefore this disclosure should not be construed as limitingly applicable to the above embodiments, but should be interpreted as including various restrictions on decoding the encoding unit 420c located at the center, different from those on the encoding units 420b and 420d.

[0118] According to this exemplary embodiment, the image decoding device 150 can obtain at least one of block shape information and partition shape information used to partition the current coding unit from a predetermined position in the current coding unit.

[0119] Figure 5 This illustrates a method by which an image decoding device 150 determines a predetermined coding unit from an odd number of coding units, according to an exemplary embodiment. (Refer to...) Figure 5 At least one of the block shape information and the partition shape information of the current encoding unit 500 can be obtained from a sample at a predetermined position among a plurality of samples included in the current encoding unit 500 (e.g., sample 540 located at the center). However, the predetermined position in the current encoding unit 500 used to obtain at least one of the block shape information and the partition shape information should not be interpreted restrictively as... Figure 5The center of the image should be interpreted as including various positions within the current coding unit 500 (e.g., topmost position, bottommost position, leftmost position, rightmost position, upper leftmost position, lower leftmost position, upper rightmost position, and lower rightmost position). The image decoding device 150 can obtain at least one of block shape information and division shape information from the predetermined positions to determine whether to divide the current coding unit into coding units with various shapes and sizes.

[0120] According to this exemplary embodiment, when the current coding unit is divided into a predetermined number of coding units, the image decoding device 150 may select one of a plurality of coding units. The method for selecting one of the plurality of coding units may vary, and a description of such a method will be given below through various embodiments.

[0121] According to this exemplary embodiment, the image decoding device 150 can divide the current encoding unit into multiple encoding units and determine the encoding unit located at the predetermined position.

[0122] According to this exemplary embodiment, the image decoding device 150 can use information indicating the position of each of the odd number of coding units to determine the coding unit located at the center of the odd number of coding units. (Refer to...) Figure 5 The image decoding device 150 can determine an odd number of encoded units 520a, 520b, and 520c by dividing the current encoding unit 500. The image decoding device 150 can determine the central encoding unit 520b by using information about the positions of the odd number of encoding units 520a, 520b, and 520c. For example, the image decoding device 150 can determine the central encoding unit 520b by determining the positions of encoding units 520a, 520b, and 520c based on information indicating the positions of predetermined sample points included in encoding units 520a, 520b, and 520c. Specifically, the image decoding device 150 can determine the central encoding unit 520b by determining the positions of encoding units 520a, 520b, and 520c based on information indicating the positions of sample points 530a, 530b, and 530c located to the upper left of encoding units 520a, 520b, and 520c.

[0123] According to this exemplary embodiment, the information indicating the positions of the upper left samples 530a, 530b, and 530c respectively included in the encoding units 520a, 520b, and 520c may include information about the positions or coordinates of the encoding units 520a, 520b, and 520c in the image. According to this exemplary embodiment, the information indicating the positions of the upper left samples 530a, 530b, and 530c respectively included in the encoding units 520a, 520b, and 520c may include information indicating the width or height of the encoding units 520a, 520b, and 520c included in the current encoding unit 500, wherein the width or height may correspond to information indicating the difference between the coordinates of the encoding units 520a, 520b, and 520c in the image. That is, the image decoding device 150 can determine the encoding unit 520b located at the center by directly using information about the positions or coordinates of the encoding units 520a, 520b, and 520c in the image or by using information about the width or height of the encoding units (indicating the difference between coordinates).

[0124] According to this exemplary embodiment, the information indicating the position of the upper left sample 530a of the top encoding unit 520a can indicate (xa, ya) coordinates, the information indicating the position of the upper left sample 530b of the center encoding unit 520b can indicate (xb, yb) coordinates, and the information indicating the position of the upper left sample 530c of the bottom encoding unit 520c can indicate (xc, yc) coordinates. The image decoding device 150 can determine the center encoding unit 520b by using the coordinates of the upper left samples 530a, 530b, and 530c respectively included in the encoding units 520a, 520b, and 520c. For example, when the coordinates of the upper left samples 530a, 530b, and 530c are arranged in ascending or descending order, the coding unit 520b, which includes the coordinates (xb, yb) of the upper left sample 530b, can be determined as the coding unit located at the center of the coding units 520a, 520b, and 520c determined when the current coding unit is divided. Here, the coordinates indicating the positions of the upper left samples 530a, 530b, and 530c can represent the coordinates indicating the absolute position in the screen. Furthermore, based on the position of the upper left sample 530c of the top coding unit 520a, the coordinates (dxb, dyb) of the upper left sample 530b, which is the information of the relative position of the coding unit 520b indicating the center, and the coordinates (dxc, dyc) of the upper left sample 530c, which is the information of the relative position of the coding unit 520b indicating the bottom, can be used. Furthermore, the method of determining the coding unit at a predetermined position by using the coordinates of the sample points included in the coding unit as information indicating the position of the sample points should not be limited to the above method, and can be interpreted as various arithmetic methods that can use the coordinates of the sample points.

[0125] According to this exemplary embodiment, the image decoding device 150 may divide the current encoding unit 500 into a plurality of encoding units 520a, 520b, and 520c, and may select an encoding unit from the encoding units 520a, 520b, and 520c according to a predetermined criterion. For example, the image decoding device 150 may select an encoding unit 520b with a different size from the encoding units 520a, 520b, and 520c.

[0126] According to this exemplary embodiment, the image decoding device 150 can determine the width or height of each of the encoding units 520a, 520b, and 520c by using (xa, ya) coordinates, (xb, yb) coordinates, and (xc, yc) coordinates, wherein the (xa, ya) coordinates indicate the position of the upper left sample 530a of the top encoding unit 520a, the (xb, yb) coordinates indicate the position of the upper left sample 530b of the center encoding unit 520b, and the (xc, yc) coordinates indicate the position of the upper left sample 530c of the bottom encoding unit 520c. The image decoding device 150 can determine the size of each of the encoding units 520a, 520b, and 520c by using the coordinates (xa, ya), (xb, yb), and (xc, yc) indicating the position of the encoding units 520a, 520b, and 520c.

[0127] According to an exemplary embodiment, the image decoding device 150 may determine the width of the top encoding unit 520a as xb-xa and the height of the top encoding unit 520a as yb-ya. According to an embodiment, the image decoding device 150 may determine the width of the center encoding unit 520b as xc-xb and the height of the center encoding unit 520b as yc-yb. According to this exemplary embodiment, the image decoding device 150 may determine the width or height of the bottom encoding unit by using the width or height of the current encoding unit and the width and height of the top encoding unit 520a and the center encoding unit 520b. The image decoding device 150 may determine an encoding unit with a different size from the other encoding units based on the determined width and height of the encoding units 520a, 520b, and 520c. (Refer to...) Figure 5The image decoding device 150 can determine the encoding unit 520b at a predetermined position as having a size different from that of the top encoding unit 520a and the bottom encoding unit 520c. However, since the process by which the image decoding device 150 determines the encoding unit with a size different from that of the other encoding units is merely an exemplary embodiment of determining the encoding unit at a predetermined position by using the size of the encoding unit determined based on the sample point coordinates, various processes can be used to determine the encoding unit at the predetermined position by comparing the size of the encoding unit determined according to the predetermined sample point coordinates.

[0128] However, the position of the sample points considered for determining the position of the coding unit should not be limited to the upper left, but can be interpreted as information about the position of any sample points included in the coding unit.

[0129] According to this exemplary embodiment, the image decoding device 150 may consider the shape of the current coding unit and select a coding unit at a predetermined position from an odd number of coding units determined when the current coding unit was divided. For example, when the current coding unit has a non-square shape with a width greater than its height, the image decoding device 150 may determine the coding unit at the predetermined position in the horizontal direction. In other words, the image decoding device 150 may determine the coding unit from coding units with different positions in the horizontal direction and may set a constraint on the coding unit. When the current coding unit has a non-square shape with a height greater than its width, the image decoding device 150 may determine the coding unit at the predetermined position in the vertical direction. In other words, the image decoding device 150 may determine the coding unit from coding units with different positions in the vertical direction and set a constraint on the coding unit.

[0130] According to this exemplary embodiment, the image decoding device 150 can use information indicating the position of each of an even number of coding units to determine the coding unit at a predetermined position from the even number of coding units. The image decoding device 150 can determine the even number of coding units by dividing the current coding unit, and determine the coding unit at the predetermined position by using information about the positions of the even number of coding units. Detailed procedures may correspond to those already referred to above. Figure 5 The process of determining the coding unit at a predetermined position (e.g., the center position) from an odd number of coding units is described, and therefore its description is no longer provided.

[0131] According to this exemplary embodiment, when a current coding unit having a non-square shape is divided into multiple coding units, predetermined information about coding units at predetermined positions can be used during the division process to determine the coding unit at the predetermined position from the multiple coding units. For example, the image decoding device 150 can use at least one of block shape information and division shape information stored in samples included in the coding unit at the center during the division process to determine the coding unit located at the center from the multiple coding units obtained by dividing the current coding unit.

[0132] Reference Figure 5 The image decoding device 150 can divide the current encoding unit 500 into a plurality of encoding units 520a, 520b, and 520c based on at least one of block shape information and partition shape information, and determine the encoding unit 520b located at the center from the plurality of encoding units 520a, 520b, and 520c. Alternatively, the image decoding device 150 can determine the encoding unit 520b located at the center by considering the position obtained from at least one of the block shape information and partition shape information. That is, at least one of the block shape information and partition shape information of the current encoding unit can be obtained from a sample point 540 located at the center of the current encoding unit 500, and when the current encoding unit 500 is divided into a plurality of encoding units 520a, 520b, and 520c based on at least one of the block shape information and partition shape information, the encoding unit 520b including the sample point 540 can be determined as the encoding unit located at the center. However, the information used to determine the coding unit located at the center should not be limited to at least one of block shape information and partition shape information; various types of information can be used during the process of determining the coding unit located at the center.

[0133] According to this exemplary embodiment, predetermined information for identifying the coding unit at a predetermined location can be obtained from predetermined samples included in the coding unit to be determined. (Refer to...) Figure 5The image decoding device 150 can use at least one of block shape information and partition shape information obtained from a sample point located at a predetermined position in the current coding unit 500 (e.g., a sample point located at the center of the current coding unit 500) to determine a coding unit at a predetermined position (e.g., a coding unit located at the center of the plurality of coding units) 520a, 520b, and 520c determined when the current coding unit 500 was partitioned. That is, the image decoding device 150 can determine the sample point at the predetermined position considering the block shape of the current coding unit 500, and the image decoding device 150 can determine and set predetermined constraints for coding unit 520b among the plurality of coding units 520a, 520b, and 520c determined when the current coding unit 500 was partitioned, wherein coding unit 520b includes a sample point from which predetermined information (e.g., at least one of block shape information and partition shape information) is obtained. (Refer to...) Figure 5 The image decoding device 150 can determine a sample 540 located at the center of the current encoding unit 500 as the sample from which predetermined information is obtained, and the image decoding device 150 can set a predetermined position of the encoding unit 520b including the sample 540 during the decoding process. However, the position of the sample from which predetermined information is obtained should not be limited to the above position, and the sample can be interpreted as a sample at any position included in the encoding unit 520b that is determined to be restricted.

[0134] According to this exemplary embodiment, the location of the sample point from which predetermined information is obtained can be determined based on the shape of the current encoding unit 500. According to this exemplary embodiment, block shape information can be used to determine whether the shape of the current encoding unit is square or non-square, and the location of the sample point from which predetermined information is obtained can be determined based on the shape of the current encoding unit. For example, the image decoding device 150 can determine a sample point located on a boundary that bisects at least one of the width and height of the current encoding unit using at least one of information about the width of the current encoding unit and information about the height of the current encoding unit as a sample point from which predetermined information is obtained. As another example, when the block shape information about the current encoding unit indicates a non-square shape, the image decoding device 150 can determine one of the sample points adjacent to the boundary that bisects the longer side of the current encoding unit as a sample point from which predetermined information is obtained.

[0135] According to this exemplary embodiment, when a current coding unit is divided into multiple coding units, the image decoding device 150 can use at least one of block shape information and partition shape information to determine the coding unit at a predetermined position among the multiple coding units. According to the exemplary embodiment, the image decoding device 150 can obtain at least one of block shape information and partition shape information from samples at predetermined positions included in the coding unit, and the image decoding device 150 can divide the multiple coding units by using at least one of the block shape information and partition shape information obtained from samples at predetermined positions included in each of the multiple coding units generated when the current coding unit is divided. In other words, the coding units can be recursively divided by using at least one of the block shape information and partition shape information obtained from samples at predetermined positions in each coding unit. Since reference has been made above... Figure 4 The recursive partitioning process of the coding unit is described, so its details are not provided here.

[0136] According to this exemplary embodiment, the image decoding device 150 can determine at least one coding unit by dividing the current coding unit, and can determine the order in which the at least one coding unit is decoded according to a predetermined block (e.g., the current coding unit).

[0137] Figure 6 This illustrates the order in which an image decoding device 150 processes multiple coding units when it determines multiple coding units by dividing the current coding unit, according to an exemplary implementation.

[0138] According to this exemplary embodiment, the image decoding device 150 can determine second coding units 610a and 610b by dividing the first coding unit 600 along the vertical direction, and determine second coding units 630a and 630b by dividing the first coding unit 600 along the horizontal direction, or determine second coding units 650a, 650b, 650c and 650d by dividing the first coding unit 600 along both the vertical and horizontal directions, based on block shape information and division shape information.

[0139] Reference Figure 6The image decoding device 150 can determine the order in which the second encoding units 610a and 610b, determined by dividing the first encoding unit 600 along the vertical direction, are processed along the horizontal direction 610c. The image decoding device 150 can determine the processing order of the second encoding units 630a and 630b, determined by dividing the first encoding unit 600 along the horizontal direction, as along the vertical direction 630c. The image decoding device 150 can determine that the second encoding units 650a, 650b, 650c, and 650d, determined by dividing the first encoding unit 600 along both the horizontal and vertical directions, are processed according to a predetermined order (e.g., raster scan order or zigzag scan order 650e), wherein, in this predetermined order, encoding units on one row are processed and then encoding units on the next row are processed.

[0140] According to this exemplary embodiment, the image decoding device 150 can recursively divide encoding units. (Refer to...) Figure 6 The image decoding device 150 can determine a plurality of coding units 610a, 610b, 630a, 630b, 650a, 650b, 650c, and 650d by dividing a first coding unit 600, and can recursively divide each of the determined plurality of coding units 610a, 610b, 630a, 630b, 650a, 650b, 650c, and 650d. The method of dividing the plurality of coding units 610a, 610b, 630a, 630b, 650a, 650b, 650c, and 650d can be similar to the method of dividing the first coding unit 600. Therefore, the plurality of coding units 610a, 610b, 630a, 630b, 650a, 650b, 650c, and 650d can all be independently divided into multiple coding units. (Refer to...) Figure 6 The image decoding device 150 can determine the second coding units 610a and 610b by dividing the first coding unit 600 along the vertical direction. In addition, the image decoding device 150 can determine whether to divide or not divide each of the second coding units 610a and 610b independently.

[0141] According to this exemplary embodiment, the image decoding device 150 may divide the second encoding unit 610a on the left side in the horizontal direction to obtain the third encoding units 620a and 620b, and may not divide the second encoding unit 610b on the right side.

[0142] According to this exemplary embodiment, the processing order of coding units can be determined based on the partitioning process of coding units. In other words, the processing order of the partitioned coding units can be determined based on the processing order of the coding units just before they were partitioned. The image decoding device 150 can determine the processing order of the third coding units 620a and 620b, which were determined when the second coding unit 610a on the left side was partitioned independently of the second coding unit 610b on the right side. Since the third coding units 620a and 620b were determined when the second coding unit 610a on the left side was partitioned along the horizontal direction, the third coding units 620a and 620b can be processed along the vertical direction 620c. Furthermore, since the processing order of the second coding unit 610a on the left side and the second coding unit 610b on the right side is along the horizontal direction 610c, the third coding units 620a and 620b included in the second coding unit 610a on the left side can be processed along the vertical direction 620c, and then the second coding unit 610b can be processed. Since the above description is intended to describe a process that determines the processing order based on the coding units before they are divided, the process should not be limited to the above embodiments. Various methods can be used to independently process coding units that are divided and determined according to various shapes in a predetermined order.

[0143] Figure 7 This illustrates a process, according to an exemplary embodiment, in which the image decoding device 150 determines that when encoding units cannot be processed in a predetermined order, the current encoding unit is divided into an odd number of encoding units.

[0144] According to this exemplary embodiment, the image decoding device 150 can determine that the current coding unit is divided into an odd number of coding units based on the partition shape information and the obtained block shape information obtained by the bitstream acquirer 110. (Refer to...) Figure 7 The first coding unit 700, which has a square shape, can be divided into second coding units 710a and 710b, which have non-square shapes. The second coding units 710a and 710b can be independently divided into third coding units 720a, 720b, 720c, 720d, and 720e. According to this exemplary embodiment, the image decoding device 150 can determine a plurality of third coding units 720a and 720b by dividing the left-hand coding unit 710a among the plurality of second coding units along a horizontal direction, and the right-hand coding unit 710b can be divided into an odd number of third coding units 720c, 720d, and 720e.

[0145] According to this exemplary embodiment, the image decoding device 150 can determine whether there are any coding units that are divided into an odd number by determining whether the third coding units 720a, 720b, 720c, 720d, and 720e can be processed in a predetermined order. (Refer to...) Figure 7The image decoding device 150 can determine the third coding units 720a, 720b, 720c, 720d, and 720e by recursively dividing the first coding unit 700. The image decoding device 150 can determine whether there are any coding units that are divided into an odd number of units among the first coding unit 700, the second coding units 710a and 710b, and the third coding units 720a, 720b, 720c, 720d, and 720e, based on at least one of block shape information and division shape information. For example, the coding unit located on the right side of the second coding units 710a and 710b can be divided into an odd number of third coding units 720c, 720d, and 720e. The order in which the multiple coding units included in the first coding unit 700 are processed can be a predetermined order 730 (e.g., a zigzag scanning order). The image decoding device 150 can determine whether the third coding units 720c, 720d, and 720e, which are determined when the second coding unit 710b on the right is divided into an odd number, meet the conditions that can be processed according to the predetermined order.

[0146] According to this exemplary embodiment, the image decoding device 150 can determine whether the third encoding units 720a, 720b, 720c, 720d, and 720e included in the first encoding unit 700 satisfy a condition that allows them to be processed in a predetermined order. This condition is related to whether at least one of the width and height of the second encoding units 710a and 710b is bisected along the boundary of the third encoding units 720a, 720b, 720c, 720d, and 720e. For example, the third encoding units 720a and 720b determined when the second encoding unit 710a on the left side, which has a non-square shape, is bisected satisfies the condition. However, since the boundary of the third encoding units 720c, 720d, and 720e determined when the second encoding unit 710 on the right side is divided into three encoding units cannot bisecte the width or height of the second encoding unit 710b on the right side, the third encoding units 720c, 720d, and 720e do not satisfy the condition. Furthermore, the image decoding device 150 can determine a scan sequence break when the condition is not met, and can determine, based on the determined result, that the second coding unit 710b on the right is divided into an odd number of coding units. According to this exemplary embodiment, when the coding units are divided into an odd number of coding units, the image decoding device 150 can set a predetermined restriction on the coding units at predetermined positions among the plurality of coding units. Since details regarding the restriction or predetermined position have been described above through various embodiments, these details will not be provided again.

[0147] Figure 8This illustration shows the process by which an image decoding device 150 determines at least one encoding unit when a first encoding unit 800 is divided, according to an exemplary embodiment. According to this exemplary embodiment, the image decoding device 150 may divide the first encoding unit 800 based on at least one of block shape information and division shape information obtained by the receiver 160. A first encoding unit 800 having a square shape may be divided into four encoding units having a square shape or a non-square shape. For example, referring to… Figure 8 When the block shape information indicates that the first coding unit 800 is a square, and the division shape information indicates that the first coding unit 800 is divided into non-square coding units, the image decoding device 150 can divide the first coding unit 800 into multiple non-square coding units. Specifically, when the division shape information indicates that the first coding unit 800 is divided along a horizontal or vertical direction to determine an odd number of coding units, the image decoding device 150 can divide the first coding unit 800, which has a square shape, into: second coding units 810a, 810b, and 810c determined when the first coding unit 800 is divided along a vertical direction, or second coding units 820a, 820b, and 820c determined when the first coding unit 800 is divided along a horizontal direction, as an odd number of coding units.

[0148] According to this exemplary embodiment, the image decoding device 150 can determine that the second encoding units 810a, 810b, and 810c, and 820a, 820b, and 820c included in the first encoding unit 800 satisfy conditions that allow them to be processed in a predetermined order, wherein the conditions relate to whether at least one of the width and height of the first encoding unit 800 is bisected along the boundaries of the second encoding units 810a, 810b, and 810c, and 820a, 820b, and 820c. (Refer to...) Figure 8Since the boundaries of the second coding units 810a, 810b, and 810c, determined when the first coding unit 800, having a square shape, is divided vertically, cannot bisect the width of the first coding unit 800, it can be determined that the first coding unit 800 does not meet the condition for being processed according to a predetermined order. Furthermore, since the boundaries of the second coding units 820a, 820b, and 820c, determined when the first coding unit 800, having a square shape, is divided horizontally, cannot bisect the width of the first coding unit 800, it can be determined that the first coding unit 800 does not meet the condition for being processed according to a predetermined order. When this condition is not met, the image decoding device 150 determines that the scanning order is broken, and can determine, based on the determined result, that the first coding unit 800 is divided into an odd number of coding units. According to this exemplary embodiment, when the encoding unit is divided into an odd number of encoding units, the image decoding device 150 may set a predetermined restriction on the encoding unit at a predetermined position among the multiple encoding units. Since details regarding the restriction or predetermined position have been described above through various embodiments, such details will not be provided again.

[0149] According to this exemplary embodiment, the image decoding device 150 can determine encoding units with various shapes by dividing the first encoding unit.

[0150] Reference Figure 8 The image decoding device 150 can divide the first encoding unit 800 with a square shape and the first encoding unit 830 or 850 with a non-square shape into encoding units with various shapes.

[0151] Figure 9 The image decoding device 150 shows that, according to an exemplary embodiment, when a second encoding unit with a non-square shape, determined when the first encoding unit 900 is divided, satisfies a predetermined condition, the shape of the second encoding unit that can be divided is limited by the second encoding unit that can be divided.

[0152] According to this exemplary embodiment, the image decoding device 150 can determine, based on at least one of block shape information and partition shape information obtained by the receiver 160, that a first encoding unit 900 having a square shape is divided into second encoding units 910a, 910b, 920a, and 920b having non-square shapes. The second encoding units 910a, 910b, 920a, and 920b can be divided independently. Therefore, the image decoding device 150 can determine whether to divide the second encoding units 910a, 910b, 920a, and 920b based on at least one of the block shape information and partition shape information associated with each of the second encoding units 910a, 910b, 920a, and 920b. According to this exemplary embodiment, the image decoding device 150 can determine third encoding units 912a and 912b by dividing the second encoding unit 910a, which has a non-square shape and is determined when the first encoding unit 900 is divided in a vertical direction. However, when the second coding unit 910a on the left is divided along a horizontal direction, the image decoding device 150 can restrict the second coding unit 910b on the right to be divided without a horizontal direction (as in the direction along which the second coding unit 910a on the left is divided). When the second coding unit 910b on the right is divided along the same direction and the third coding units 914a and 914b are determined, the third coding units 912a, 912b, 914a, and 914b can be determined even when the second coding units 910a on the left and 910b on the right are divided independently along a horizontal direction. However, this is the same result as when the image decoding device 150 divides the first coding unit 900 into four second coding units 930a, 930b, 930c, and 930d with square shapes based on at least one of block shape information and division shape information, and therefore may be inefficient in terms of image decoding.

[0153] According to this exemplary embodiment, the image decoding device 150 can determine the third encoding units 922a, 922b, 924a, and 924b by dividing the second encoding units 920a or 920b, which have non-square shapes and are determined when the first encoding unit 900 is divided in a horizontal direction. However, when one of the second encoding units (e.g., the top second encoding unit 920a) is divided in a vertical direction, the image decoding device 150 may restrict another second encoding unit (e.g., the bottom second encoding unit 920b) to be divided in a direction other than vertical (such as the direction along which the top second encoding unit 920a is divided) for the reasons described above.

[0154] Figure 10The illustration shows the process by which the image decoding device 150 divides a coding unit with a square shape into four coding units with a square shape when the division shape information does not indicate that the coding unit with a square shape is divided into four coding units with a square shape, according to an exemplary embodiment.

[0155] According to this exemplary embodiment, the image decoding device 150 can determine second coding units 1010a, 1010b, 1020a, 1020b, etc., by dividing the first coding unit 1000 based on at least one of block shape information and partition shape information. The partition shape information may include information about various shapes into which the coding unit can be divided; however, sometimes, the information about various shapes may not include information for dividing the coding unit into four square coding units. Based on this partition shape information, the image decoding device 150 cannot divide the first coding unit 1000, which has a square shape, into four square second coding units 1030a, 1030b, 1030c, and 1030d. Based on the partition shape information, the image decoding device 150 can determine second coding units 1010a, 1010b, 1020a, 1020b, etc., which have non-square shapes.

[0156] According to this exemplary embodiment, the image decoding device 150 can independently divide second coding units 1010a, 1010b, 1020a, 1020b, etc., having non-square shapes. Each of the second coding units 1010a, 1010b, 1020a, 1020b, etc., can be divided in a predetermined order using a recursive method, wherein the recursive method can correspond to a method of dividing the first coding unit 1000 based on at least one of block shape information and division shape information.

[0157] For example, the image decoding device 150 can determine the third coding units 1012a and 1012b, which have square shapes, by dividing the second coding unit 1010a on the left side horizontally, and can determine the third coding units 1014a and 1014b, which have square shapes, by dividing the second coding unit 1010b on the right side horizontally. Alternatively, the image decoding device 150 can determine the third coding units 1016a, 1016b, 1016c, and 1016d, which have square shapes, by dividing both the second coding unit 1010a on the left side and the second coding unit 1010b on the right side horizontally. In this case, the coding units can be determined in the same way as the second coding units 1030a, 1030b, 1030c, and 1030d, which are divided into four squares in the first coding unit 1000.

[0158] As another example, the image decoding device 150 can determine the third coding units 1022a and 1022b with square shapes by dividing the top second coding unit 1020a vertically, and can determine the third coding units 1024a and 1024b with square shapes by dividing the bottom second coding unit 1020b vertically. Alternatively, the image decoding device 150 can determine the third coding units 1022a, 1022b, 1024a, and 1024b with square shapes by dividing both the top second coding unit 1020a and the bottom second coding unit 1020b vertically. In this case, the coding units can be determined in the same way as the second coding units 1030a, 1030b, 1030c, and 1030d, which are divided into four squares in the first coding unit 1000.

[0159] Figure 11 This illustrates, according to an exemplary embodiment, how the processing order among multiple coding units can be changed based on the division of coding units.

[0160] According to this exemplary embodiment, the image decoding device 150 may divide the first coding unit 1100 based on block shape information and partition shape information. When the block shape information indicates a square shape and the partition shape information indicates that the first coding unit 1100 is divided along at least one of the horizontal and vertical directions, the image decoding device 150 may divide the first coding unit 1100 to determine a second coding unit (e.g., second coding units 1110a, 1110b, 1120a, 1120b, etc.). (Refer to...) Figure 11 The second coding units 1110a, 1110b, 1120a and 1120b with non-square shapes, determined when the first coding unit 1100 is divided only along the horizontal or vertical direction, can be independently divided based on the block shape information and division shape information of each of the second coding units 1110a, 1110b, 1120a and 1120b. For example, the image decoding device 150 can determine the third coding units 1116a, 1116b, 1116c, and 1116d by dividing the second coding units 1110a and 1110b along the horizontal direction, wherein the second coding units 1110a and 1110b are generated when the first coding unit 1100 is divided along the vertical direction. Similarly, the image decoding device 150 can determine the third coding units 1126a, 1126b, 1126c, and 1126d by dividing the second coding units 1120a and 1120b along the vertical direction, wherein the second coding units 1120a and 1120b are generated when the first coding unit 1100 is divided along the horizontal direction. Since the division process of the second coding units 1110a, 1110b, 1120a, and 1120b has been described above, its details will not be provided further.

[0161] According to this exemplary embodiment, the image decoding device 150 can process encoding units in a predetermined order. Since the above has been referred to... Figure 6 The characteristics regarding the processing of encoded units according to a predetermined order are described, therefore further details are not provided.

[0162] Reference Figure 11 The image decoding device 150 can determine the processing order of four square third coding units 1116a, 1116b, 1116c, and 1116d or 1126a, 1126b, 1126c, and 1126d by dividing a first coding unit 1100 having a square shape. According to this exemplary embodiment, the image decoding device 150 can determine the processing order of the third coding units 1116a, 1116b, 1116c, and 1116d or 1126a, 1126b, 1126c, and 1126d based on the shape of the first coding unit 1100 being divided.

[0163] According to this exemplary embodiment, the image decoding device 150 can determine the third encoding units 1116a, 1116b, 1116c, and 1116d by dividing each of the second encoding units 1110a and 1110b in the horizontal direction, wherein the second encoding units 1110a and 1110b are generated when the first encoding unit 1100 is divided in the vertical direction, and the image decoding device 150 can process the third encoding units 1116a, 1116b, 1116c, and 1116d in the vertical direction by first processing the third encoding units 1116a and 1116b included in the second encoding unit 1110a on the left side in the vertical direction and then processing the third encoding units 1116c and 1116d included in the second encoding unit 1110b on the right side in the vertical direction.

[0164] According to this exemplary embodiment, the image decoding device 150 can determine the third encoding units 1126a, 1126b, 1126c, and 1126d by dividing each of the second encoding units 1120a and 1120b in a vertical direction, wherein the second encoding units 1120a and 1120b are generated when the first encoding unit 1100 is divided in a horizontal direction, and the image decoding device 150 can process the third encoding units 1126a, 1126b, 1126c, and 1126d in a horizontal direction according to the following order 1127: first processing the third encoding units 1126a and 1126b included in the top second encoding unit 1120a in a horizontal direction, and then processing the third encoding units 1126c and 1126d included in the bottom second encoding unit 1120b in a horizontal direction.

[0165] Reference Figure 11When each of the second coding units 1110a, 1110b, 1120a, and 1120b is divided, third coding units 1116a, 1116b, 1116c, 1116d, 1126a, 1126b, 1126c, and 1126d with square shapes can be determined. The second coding units 1110a and 1110b determined when the first coding unit 1100 is divided in a vertical direction and the second coding units 1120a and 1120b determined when the first coding unit 1100 is divided in a horizontal direction have different shapes, but according to the subsequently determined third coding units 1116a, 1116b, 1116c, 1116d, 1126a, 1126b, 1126c, and 1126d, the first coding unit 1100 is divided into coding units with the same shape. Therefore, even when coding units with the same shape are determined as a result of recursively dividing coding units based on at least one of block shape information and partition shape information via different processes, the image decoding device 150 can process coding units with the same shape in different orders.

[0166] Figure 12 This illustrates a process for determining the depth of a coding unit when the shape and size of the coding unit change, in the case where multiple coding units are determined during recursive partitioning, according to an exemplary embodiment.

[0167] According to this exemplary embodiment, the image decoding device 150 can determine the depth of the coding unit based on a predetermined criterion. For example, the predetermined criterion may be the length of the longer side of the coding unit. When the length of the longer side of the coding unit before it was divided is 2n times the length of the longer side of the current coding unit (where n > 0), the image decoding device 150 can determine that the depth of the current coding unit is n times higher than the depth of the coding unit before it was divided. In the following, the coding unit with the higher depth will be referred to as the coding unit with the lower depth.

[0168] Reference Figure 12According to this exemplary embodiment, the image decoding device 150 can determine a second encoding unit 1202 and a third encoding unit 1204 of a lower layer depth by dividing a first encoding unit 1200 having a square shape based on block shape information indicating a square shape (e.g., the block shape information may indicate "0:SQUARE"). When the size of the first encoding unit 1200 having a square shape is 2N×2N, the second encoding unit 1202, determined by dividing the width and height of the first encoding unit 1200 by 1 / 2, may have a size of N×N. Furthermore, the third encoding unit 1204, determined by dividing the width and height of the second encoding unit 1202 by 1 / 2, may have a size of N / 2×N / 2. In this case, the width and height of the third encoding unit 1204 correspond to 1 / 4 times the width and height of the first encoding unit 1200. When the depth of the first coding unit 1200 is D, the depth of the second coding unit 1202 can be D+1, and the depth of the third coding unit 1204 can be D+2. The width and height of the second coding unit 1202 are 1 / 2 times the width and height of the first coding unit 1200, and the width and height of the third coding unit 1204 are 1 / 4 times the width and height of the first coding unit 1200.

[0169] According to this exemplary embodiment, the image decoding device 150 can determine the lower layer depth by dividing the first encoding unit 1210 or 1220 having a non-square shape based on block shape information indicating a non-square shape (e.g., the block shape information may indicate "1:NS_VER" or "2:NS_HOR", where "1:NS_VER" indicates that the height is longer than the width, and "2:NS_HOR" indicates that the width is longer than the height).

[0170] The image decoding device 150 can determine a second coding unit (e.g., second coding unit 1020, 1212, 1222, etc.) by dividing the width and height of the first coding unit 1210, which has a size of N×2N. In other words, the image decoding device 150 can determine a second coding unit 1202 of size N×N or a second coding unit 1222 of size N×N / 2 by dividing the first coding unit 1210 along the horizontal direction, or it can determine a second coding unit 1212 of size N / 2×N by dividing the first coding unit 1210 along both the horizontal and vertical directions.

[0171] According to this exemplary embodiment, the image decoding device 150 can determine a second encoding unit (e.g., second encoding units 1202, 1212, 1222, etc.) by dividing the width and height of the first encoding unit 1220, which has a size of 2N×N. That is, the image decoding device 150 can determine a second encoding unit 1202 with a size of N×N or a second encoding unit 1212 with a size of N / 2×N by dividing the first encoding unit 1220 along the vertical direction, or it can determine a second encoding unit 1222 with a size of N×N / 2 by dividing the first encoding unit 1220 along both the horizontal and vertical directions.

[0172] According to this exemplary embodiment, the image decoding device 150 can determine a third encoding unit (e.g., third encoding unit 1204, 1214, 1224, etc.) by dividing the width and height of the second encoding unit 1202 with a size of N×N. That is, the image decoding device 150 can determine a third encoding unit 1204 with a size of N / 2×N / 2, a third encoding unit 1214 with a size of N / 4×N / 2, or a third encoding unit 1224 with a size of N / 2×N / 4 by dividing the second encoding unit 1202 along the vertical and horizontal directions.

[0173] According to this exemplary embodiment, the image decoding device 150 can determine a third encoding unit (e.g., third encoding units 1204, 1214, 1224, etc.) by dividing the width and height of the second encoding unit 1212, which has a size of N / 2 × N. That is, the image decoding device 150 can determine a third encoding unit 1204 with a size of N / 2 × N / 2 or a third encoding unit 1224 with a size of N / 2 × N / 4 by dividing the second encoding unit 1212 along the horizontal direction, or it can determine a third encoding unit 1214 with a size of N / 4 × N / 2 by dividing the second encoding unit 1212 along both the vertical and horizontal directions.

[0174] According to this exemplary embodiment, the image decoding device 150 can determine a third encoding unit (e.g., third encoding units 1204, 1214, 1224, etc.) by dividing the width and height of the second encoding unit 1214 with a size of N×N / 2. That is, the image decoding device 150 can determine a third encoding unit 1204 with a size of N / 2×N / 2 or a third encoding unit 1214 with a size of N / 4×N / 2 by dividing the second encoding unit 1212 along the vertical direction, or it can determine a third encoding unit 1224 with a size of N / 2×N / 4 by dividing the second encoding unit 1212 along both the vertical and horizontal directions.

[0175] According to this exemplary embodiment, the image decoding device 150 may divide the image into square-shaped encoding units (e.g., first encoding units 1200, 1202, and 1204) along a horizontal or vertical direction. For example, the first encoding unit 1200 with a size of 2N×2N may be divided vertically to determine a first encoding unit 1210 with a size of N×2N, or it may be divided horizontally to determine a first encoding unit 1220 with a size of 2N×N. According to this exemplary embodiment, when the depth is determined based on the length of the longer side of the encoding unit, the depth of the encoding unit determined when the first encoding units 1200, 1202, or 1204 are divided horizontally or vertically may be the same as the depth of the first encoding units 1200, 1202, or 1204.

[0176] According to this exemplary embodiment, the width and height of the third encoding unit 1214 or 1224 can be half the size of the first encoding unit 1210 or 1220. When the depth of the first encoding unit 1210 or 1220 is D, the depth of the second encoding unit 1212 or 1222 can be D+1, and the depth of the third encoding unit 1214 or 1224 can be D+2, wherein the width and height of the second encoding unit 1212 or 1214 are half the size of the width and height of the first encoding unit 1210 or 1220, and the width and height of the third encoding unit 1214 or 1224 are half the size of the width and height of the first encoding unit 1210 or 1220.

[0177] Figure 13 The diagram illustrates a depth that can be determined based on the shape and size of the encoding unit, according to an exemplary embodiment, and a partial index (PID) used to distinguish between encoding units.

[0178] According to this exemplary embodiment, the image decoding device 150 can determine second coding units with various shapes by dividing a first coding unit 1300 having a square shape. (Refer to...) Figure 13 Based on the division shape information, the image decoding device 150 can determine the second coding units 1302a, 1302b, 1304a, 1304b, 1306a, 1306b, 1306c, and 1306d by dividing the first coding unit 1300 along at least one of the horizontal and vertical directions. In other words, the image decoding device 150 can determine the second coding units 1302a, 1302b, 1304a, 1304b, 1306a, 1306b, 1306c, and 1306d based on the division shape information regarding the first coding unit 1300.

[0179] According to this exemplary embodiment, the depths of the second coding units 1302a, 1302b, 1304a, 1304b, 1306a, 1306b, 1306c, and 1306d, determined based on the division shape information regarding the first coding unit 1300 having a square shape, can be determined based on the length of the longer side. For example, since the length of the longer side of the second coding units 1302a, 1302b, 1304a, and 1304b having a non-square shape is the same as the length of one side of the first coding unit 1300 having a square shape, the depths of the first coding unit 1300 and the second coding units 1302a, 1302b, 1304a, and 1304b having a non-square shape can be D, i.e., the same. On the other hand, when the image decoding device 150 divides the first encoding unit 1300 into four second encoding units 1306a, 1306b, 1306c and 1306d with square shapes based on the division shape information, since the length of one side of each of the second encoding units 1306a, 1306b, 1306c and 1306d with square shapes is 1 / 2 the length of one side of the first encoding unit 1300, the depth of the second encoding units 1306a, 1306b, 1306c and 1306d can be D+1, that is, one depth lower than the depth D of the first encoding unit 1300.

[0180] According to this exemplary embodiment, the image decoding device 150 can divide the first encoding unit 1310 into a plurality of second encoding units 1312a, 1312b, 1314a, 1314b, and 1314c by dividing the first encoding unit 1310 with a height greater than its width along the horizontal direction according to the division shape information. According to this exemplary embodiment, the image decoding device 150 can divide the first encoding unit 1320 into a plurality of second encoding units 1322a and 1322b, or 1324a, 1324b, and 1324c, by dividing the first encoding unit 1320 with a width greater than its height along the vertical direction according to the division shape information.

[0181] According to this exemplary embodiment, the depths of the second coding units 1312a, 1312b, 1314a, 1314b, 1314c, 1322a, 1322b, 1324a, 1324b, and 1324c, determined based on the division shape information of the first coding unit 1310 or 1320 having a non-square shape, can be determined based on the length of the longer side. For example, since the length of one side of each of the second coding units 1312a and 1312b having a square shape is half the length of one side of the first coding unit 1310 having a non-square shape with a height longer than its width, the depths of the second coding units 1302a, 1302b, 1304a, and 1304b having a square shape are D+1, that is, one depth lower than the depth D of the first coding unit 1310 having a non-square shape.

[0182] Furthermore, the image decoding device 150 can divide the first encoding unit 1310, which has a non-square shape, into an odd number of second encoding units 1314a, 1314b, and 1314c based on the division shape information. The odd number of second encoding units 1314a, 1314b, and 1314c may include second encoding units 1314a and 1314c with non-square shapes and second encoding unit 1314b with a square shape. Here, since the length of the longer side of the second encoding units 1314a and 1314c with non-square shapes and the length of one side of the second encoding unit 1314b with a square shape are half the length of one side of the first encoding unit 1310, the depth of the second encoding units 1314a, 1314b, and 1314c can be D+1, that is, one depth lower than the depth D of the first encoding unit 1310. The image decoding device 150 can determine the depth of the coding unit associated with the first coding unit 1320, which has a non-square shape with a width longer than its height, in a manner similar to determining the depth of the coding unit associated with the first coding unit 1310.

[0183] According to this exemplary embodiment, when determining a PID for distinguishing between coding units, the image decoding device 150 can determine the PID based on the size ratio between coding units when an odd number of coding units do not have the same size. (Refer to...) Figure 13The width of encoding unit 1314b located at the center of the odd-numbered encoding units 1314a to 1314c is the same as the width of encoding units 1314a and 1314c, but its height is twice the height of encoding units 1314a and 1314c. In this case, encoding unit 1314b located at the center of the odd-numbered encoding units 1314a to 1314c may include two encoding units 1314a or two encoding units 1314c. Therefore, when the PID of the encoding unit 1314b located at the center according to the scanning order is 1, the PID of the encoding unit 1314c located in the next order can be increased by 2, that is, 3. That is, the value of the PID can be discontinuous. According to this exemplary embodiment, the image decoding device 150 can determine whether the odd-numbered encoding units have the same size based on the discontinuity of the PID used to distinguish between encoding units.

[0184] According to this exemplary embodiment, the image decoding device 150 can determine whether a plurality of coding units determined when the current coding unit is divided have a specific division shape based on the value of a PID used to distinguish between coding units. (Refer to...) Figure 13 The image decoding device 150 can determine an even number of coding units 1312a and 1312b or an odd number of coding units 1314a to 1314c by dividing a first coding unit 1310 having a rectangular shape with a height longer than its width. The image decoding device 150 can use an ID indicating each coding unit to distinguish between multiple coding units. According to this exemplary embodiment, a PID can be obtained from a sample located at a predetermined position (e.g., the upper left sample) of each coding unit.

[0185] According to this exemplary embodiment, the image decoding device 150 can determine the coding unit located at a predetermined position among a plurality of coding units determined by division by using a PID for distinguishing between coding units. According to this exemplary embodiment, when the division shape information of a first coding unit 1310 having a rectangular shape with a height longer than its width indicates division into three coding units, the image decoding device 150 can divide the first coding unit 1310 into three coding units 1314a, 1314b, and 1314c. The image decoding device 150 can assign a PID to each of the three coding units 1314a, 1314b, and 1314c. The image decoding device 150 can compare the PIDs of the coding units to determine the coding unit located at the center among an odd number of coding units. The image decoding device 150 can determine the coding unit 1314b, having a PID corresponding to the center value among the plurality of PIDs, as the coding unit located at the center among the plurality of coding units based on the plurality of PIDs of the plurality of coding units determined when the first coding unit 1310 is divided. According to this exemplary embodiment, when determining the PID for distinguishing between encoding units, the image decoding device 150 may determine the PID based on the size ratio between encoding units when the encoding units do not have the same size. (Refer to...) Figure 13 When the first coding unit 1310 is divided, the width of the resulting coding unit 1314b can be the same as the width of coding units 1314a and 1314c, but the height can be twice the height of coding units 1314a and 1314c. In this case, when the PID of the coding unit 1314b located at the center is 1, the PID of the next sequential coding unit 1314c can be increased by 2, i.e., 3. Therefore, when the range of increase changes and the PID increases uniformly, the image decoding device 150 can determine that the coding unit is divided into multiple coding units, wherein the multiple coding units include coding units with different dimensions than other coding units. According to this exemplary embodiment, when the division shape information indicates division into an odd number of coding units, the image decoding device 150 can divide the current coding unit into an odd number of coding units, wherein, among the odd number of coding units, the coding unit at a predetermined position (e.g., the center coding unit) has a different dimension than other coding units. In this case, the image decoding device 150 can determine the center coding unit with a different dimension by using the PID of the coding unit. However, since the PID and size or position of the encoding unit at the predetermined position are specified to describe this exemplary embodiment, this disclosure is not limited thereto, and various PIDs and various shapes and sizes of the encoding unit may be used.

[0186] According to this exemplary embodiment, the image decoding device 150 may use the following predetermined data unit: recursively dividing the encoding unit starting from the predetermined data unit.

[0187] Figure 14 This illustrates how, according to an exemplary embodiment, multiple encoding units are determined based on multiple predetermined data units included in the image.

[0188] According to this exemplary embodiment, a predetermined data unit can be defined as a data unit that is recursively divided into coding units using at least one of block shape information and division shape information, starting from that data unit. That is, the predetermined data unit may correspond to the highest-depth coding unit used when determining multiple coding units in the current frame division. Hereinafter, for ease of description, such a predetermined data unit is referred to as a reference data unit.

[0189] According to this exemplary embodiment, the reference data unit may indicate a predetermined size and shape. According to the exemplary embodiment, the reference data unit may include M×N sample points. Here, M and N may be equal to each other and may be integers representing multiples of 2. That is, the reference data unit may indicate a square shape or a non-square shape, and may subsequently be divided into an integer number of coded units.

[0190] According to this exemplary embodiment, the image decoding device 150 can divide the current frame into a plurality of reference data units. According to this exemplary embodiment, the image decoding device 150 can divide the plurality of reference data units by using division information about each of the plurality of reference data units obtained by dividing the current frame. This reference data unit division process can correspond to a division process using a quadtree structure.

[0191] According to this exemplary embodiment, the image decoding device 150 may predetermine the minimum size of the reference data units that can be used in the current frame. Therefore, the image decoding device 150 may determine reference data units of various sizes that are equal to or greater than the minimum size, and may determine at least one encoding unit based on the determined reference data units by using block shape information and partition shape information.

[0192] Reference Figure 14 The image decoding device 150 may use a reference coding unit 1400 having a square shape, or it may use a reference coding unit 1402 having a non-square shape. According to this exemplary embodiment, the shape and size of the reference coding unit may be determined based on various data units (e.g., sequences, frames, stripes, strip segments, and maximum coding units) that may include at least one reference coding unit.

[0193] According to this exemplary embodiment, the receiver 160 of the image decoding device 150 can obtain at least one of information regarding the shape of a reference coding unit and information regarding the size of a reference coding unit from the bitstream based on various data units. This has already been demonstrated above. Figure 10 The process of dividing the current coding unit 1000 describes the process of determining at least one coding unit included in the reference coding unit 1400 having a square shape, and the above has been passed through Figure 11 The process of dividing the current coding unit 1100 or 1150 describes the process of determining at least one coding unit included in the reference coding unit having a non-square shape, so its description is not provided here.

[0194] According to this exemplary embodiment, in order to determine the size and shape of a reference coding unit based on some data units predetermined based on predetermined conditions, the image decoding device 150 can use a PID to check the size and shape of the reference coding unit. That is, the receiver 160 can obtain only the PID for checking the size and shape of the reference coding unit from the bitstream based on the strip, strip segment, and maximum coding unit, wherein the reference coding unit is a data unit that satisfies predetermined conditions among various data units (e.g., sequence, frame, strip, strip segment, and maximum coding unit) (e.g., a data unit with a size equal to or smaller than the strip). The image decoding device 150 can determine the size and shape of the reference data unit based on the data unit that satisfies the predetermined conditions by using the PID. When information about the shape and size of the reference coding unit is obtained from the bitstream, and data units with relatively small sizes are used, the bitstream utilization efficiency may be insufficient; therefore, information about the shape and size of the reference coding unit is not directly obtained, but only the PID is obtained and used. In this case, at least one of the size and shape of the reference coding unit corresponding to the PID indicating the size and shape of the reference coding unit can be predetermined. In other words, the image decoding device 150 can select at least one of the dimensions and shapes of a predetermined reference coding unit based on the PID, so as to determine at least one of the dimensions and shapes of the reference coding unit included in the data unit, which serves as a standard for obtaining the PID.

[0195] According to this exemplary embodiment, the image decoding device 150 may use at least one reference coding unit included in a maximum coding unit. That is, the maximum coding unit for dividing the image may include at least one reference coding unit, and the coding unit may be determined when each reference coding unit is recursively divided. According to this exemplary embodiment, at least one of the width and height of the maximum coding unit may be an integer multiple of at least one of the width and height of the reference coding unit. According to this exemplary embodiment, the size of the reference coding unit may be equal to the size of the maximum coding unit, wherein the maximum coding unit is divided n times according to a quadtree structure. That is, according to various embodiments, the image decoding device 150 may determine the reference coding unit by dividing the maximum coding unit n times according to a quadtree structure, and may divide the reference coding unit based on at least one of block shape information and division shape information.

[0196] Figure 15 This illustrates a standard processing block, according to an exemplary embodiment, used in the process of determining the order of reference coding units included in screen 1500.

[0197] According to this exemplary embodiment, the image decoding device 150 can determine at least one processing block for dividing a screen. A processing block is a data unit that includes at least one reference coding unit for dividing the image, and the at least one reference coding unit included in the processing block can be determined in a specific order. That is, the order of the at least one reference coding unit determined in each processing block can correspond to one of various orders used to determine the reference coding units, and can vary depending on the processing block. The order of the reference coding units determined for each processing block can be one of various orders (such as raster scan order, zigzag scan order, N-shaped scan order, upper right diagonal scan order, horizontal scan order, and vertical scan order), but should not be construed as being limited to these scan orders.

[0198] According to this exemplary embodiment, the image decoding device 150 can determine the size of at least one processing block included in an image by obtaining information about the size of the processing block. The image decoding device 150 can obtain information about the size of the processing block from a bitstream to determine the size of at least one processing block included in the image. The size of the processing block can be a predetermined size of a data unit indicated by information about the size of the processing block.

[0199] According to this exemplary embodiment, the receiver 160 of the image decoding device 150 can obtain information about the size of a processing block from the bitstream based on specific data units. For example, information about the size of a processing block can be obtained from the bitstream using images, sequences, frames, stripes, and strip segments as data units. That is, the receiver 160 can obtain information about the size of a processing block from the bitstream based on such a number of data units, and the image decoding device 150 can determine the size of at least one processing block for dividing a frame by using the obtained information about the size of the processing block, wherein the size of the processing block can be an integer multiple of the size of a reference coding unit.

[0200] According to this exemplary embodiment, the image decoding device 150 can determine the sizes of processing blocks 1502 and 1512 included in the frame 1500. For example, the image decoding device 150 can determine the size of the processing block based on information about the size of the processing block, wherein this information is obtained from the bitstream. (Refer to...) Figure 15 According to an exemplary embodiment, the image decoding device 150 may determine that the horizontal dimension of processing blocks 1502 and 1512 is four times the horizontal dimension of the reference coding unit, and that the vertical dimension of processing blocks 1502 and 1512 is four times the vertical dimension of the reference coding unit. The image decoding device 150 may determine the order of at least one reference coding unit in at least one processing block.

[0201] According to this exemplary embodiment, the image decoding device 150 may determine each of the processing blocks 1502 and 1512 included in the image 150 based on the size of the processing blocks, and may determine the order of at least one reference coding unit included in each of the processing blocks 1502 and 1512. According to this exemplary embodiment, the process of determining the reference coding unit may include determining the size of the reference coding unit.

[0202] According to this exemplary embodiment, the image decoding device 150 can obtain information from a bitstream regarding the order of at least one reference coding unit included in at least one processing block, and can determine the order of the at least one reference coding unit based on the obtained information. The information regarding the order can be defined as determining the order or direction of the reference coding units in the processing block. That is, the order of the reference coding units can be determined independently for each processing block.

[0203] According to this exemplary embodiment, the image decoding device 150 can obtain information about the order of reference coding units from the bitstream based on specific data units. For example, the receiver 160 can obtain information about the order of reference coding units from the bitstream based on data units such as images, sequences, frames, stripes, strip segments, and processing blocks. Since the information about the order of reference coding units indicates the order of reference coding units in a processing block, the information about the order can be obtained for each specific data unit comprising an integer number of processing blocks.

[0204] According to this exemplary embodiment, the image decoding device 150 may determine at least one reference coding unit based on a determined order.

[0205] According to this exemplary embodiment, receiver 160 can receive information about the order of reference coding units from the bitstream as information related to processing blocks 1502 and 1512, and image decoding device 150 can determine the order of at least one reference coding unit included in processing blocks 1502 and 1512, and determine at least one reference coding unit included in frame 1500 based on the order of the coding units. (Refer to...) Figure 15 The image decoding device 150 can determine the order 1504 and 1514 of at least one reference coding unit associated with processing blocks 1502 and 1512, respectively. For example, when information about the order of reference coding units is obtained for each processing block, the order of reference coding units associated with processing block 1502 and the order of reference coding units associated with processing block 1512 may be different from each other. When the order 1504 associated with processing block 1502 is a raster scan order, the reference coding units included in processing block 1502 can be determined according to the raster scan order. On the other hand, when the order 1514 associated with processing block 1512 is the reverse of the raster scan order, the reference coding units included in processing block 1512 can be determined according to the reverse of the raster scan order.

[0206] Refer to Figure 1 to Figure 15 This describes a method for dividing an image into maximum coding units and then further dividing each maximum coding unit into coding units with a hierarchical tree structure. References will now be made to... Figures 16 to 25 Describe how to encode or decode code units of the same depth according to which encoding order.

[0207] Figure 16 This illustrates a video decoding device 1600, according to an exemplary embodiment, which involves dividing the current block and determining the encoding order of the divided lower-level blocks.

[0208] The video encoding device 1600 includes a block divider 1610, an encoding order determiner 1620, and a block decoder 1630. (See reference...) Figure 16The block divider 1610, the encoding order determiner 1620, and the block decoder 1630 are formed as separate elements, but in an exemplary embodiment, the block divider 1610, the encoding order determiner 1620, and the block decoder 1630 may be integrated.

[0209] Reference Figure 16 The block divider 1610, the encoding order determiner 1620, and the block decoder 1630 are considered to be components located within a single device, but they do not need to be combined into one device. Therefore, in an exemplary embodiment, the block divider 1610, the encoding order determiner 1620, and the block decoder 1630 can be distributed.

[0210] In some embodiments, the block partitioner 1610, the encoding order determiner 1620, and the block decoder 1630 may be implemented using a single processor. In some embodiments, the block partitioner 1610, the encoding order determiner 1620, and the block decoder 1630 may be implemented using multiple processors.

[0211] can be Figure 1B The decoder 180 performs the functions performed by the block divider 1610, the encoding order determiner 1620, and the block decoder 1630.

[0212] The block partitioner 1610 can obtain partitioning information indicating whether to partition the current block. The partitioning information indicates whether to partition the current block into at least two smaller blocks. When the partitioning information indicates that the current block should be partitioned, the block partitioner 1610 partitions the current block into at least two lower-level blocks.

[0213] The current block can be divided into various shapes based on its shape. For example, when the current block is a square, it can be divided into four lower-level blocks, each of which is a square.

[0214] When at least two partitioning methods are permitted for the shape of the current block, the block partitioner 1610 can select a partitioning method based on the partitioning shape information. Therefore, when the partitioning information indicates that the current block is to be partitioned, the block partitioner 1610 can obtain partitioning shape information indicating the partitioning method for the current block. The block partitioner 1610 can then partition the current block according to the partitioning method indicated by the partitioning shape information.

[0215] For example, when the current block is a square of size 2N×2N, the shape information can indicate the partitioning method, whichever is applied to the current block: N×N partitioning, 2N×2N partitioning, N×2N partitioning, vertical non-uniform trisection partitioning, and horizontal non-uniform trisection partitioning. The N×N partitioning involves dividing the current block into four blocks, each with a size of N×N. The 2N×N partitioning involves dividing the current block into blocks of size 2N×N. The N×2N partitioning involves dividing the current block into blocks of size N×2N. The vertical non-uniform trisection partitioning involves dividing the 2N×2N block into three blocks with equal widths and a height ratio of 1:2:1. The horizontal non-uniform trisection partitioning involves dividing the 2N×2N block into three blocks with equal heights and a width ratio of 1:2:1. In addition to these partitioning methods, the current block can also be partitioned according to one of the various horizontal and vertical partitioning methods.

[0216] When the current block is a vertical rectangle of size 2N×N, the partitioning shape information indicates the partitioning method to be applied to the current block, either an N×N partition or a vertical non-uniform trisection. The N×N partition involves dividing the current block into two blocks, each of size N×N. The vertical non-uniform trisection involves dividing the 2N×N block into three blocks with equal widths and a height ratio of 1:2:1. In addition to these partitioning methods, the current block can be partitioned according to one of various horizontal and vertical partitioning methods.

[0217] When the current block is a horizontal rectangle of size N×2N, the partitioning shape information indicates the partitioning method to be applied to the current block, either an N×N partition or a horizontal non-uniform trisection. The N×N partition involves dividing the current block into two blocks, each of size N×N. The horizontal non-uniform trisection involves dividing the N×2N block into three blocks with equal heights and width ratios of 1:2:1 or other ratios. In addition to these partitioning methods, the current block can be partitioned according to one of various horizontal and vertical partitioning methods.

[0218] In addition to the above divisions, methods such as dividing the current block according to asymmetric ratios, dividing the current block into triangular blocks, and dividing the current block into geometric blocks can also be used to divide the current block into square or rectangular shapes.

[0219] When the partitioning information indicates that the current block should not be partitioned, the block partitioner 1610 does not partition the current block. Then, the block decoder 1630 decodes the current block.

[0220] When the current block is a coding unit, the block divider 1610 determines that the current block is the final coding unit. The final coding unit is no longer divided into coding units with a higher depth. According to this exemplary embodiment, when the current block, which is the final coding unit, is divided into non-coding unit data units, the block decoder 1630 may allow the block divider 1610 to divide the current block.

[0221] According to this exemplary embodiment, block partitioner 1610 can divide the current block into one or more prediction units according to a hierarchical tree structure. Similarly, block partitioner 1610 can divide the current block into one or more transform units according to a hierarchical tree structure. Block decoder 1630 can reconstruct the current block based on the prediction results for one or more prediction units and the transform results for one or more transform units.

[0222] When the current block is a prediction unit, the block decoder 1630 can perform prediction on the current block. When the current block is a transform unit, the block decoder 1630 can perform inverse quantization and inverse transform on the quantized transform coefficients of the current block to obtain residual data.

[0223] The encoding order determiner 1620 obtains encoding order information indicating the encoding order of lower-level blocks. The encoding order determiner 1620 can determine the encoding order of lower-level blocks based on the obtained encoding order information.

[0224] Encoding order information indicates the encoding order of two or more lower-level blocks included in the current block. The amount of data for encoding order information is determined based on the number of lower-level blocks and the method used to determine the encoding order.

[0225] For example, when two lower-level blocks exist, the encoding order information can be determined as an indication of the lower-level block that was previously encoded. Therefore, the encoding order information can be a flag with a data size of 1 bit.

[0226] However, when there are four lower-level blocks, the number of possible encoding orders is 4! = 24. Therefore, 5 bits of data are needed to indicate these 24 encoding orders. In other words, the number of possible encoding orders increases as the number of lower-level blocks increases. Therefore, to reduce the amount of data required for encoding order information, encoding order determination techniques can be used by determining whether the encoding orders of some lower-level block pairs have been swapped according to a predetermined default encoding order. The encoding order information indicating whether the encoding orders of lower-level block pairs have been swapped indicates whether the swapping is in the forward or reverse direction relative to the default encoding order.

[0227] The current frame, including the current block, is encoded and decoded according to the default encoding order. All blocks and pixels encoded and decoded within the current frame are encoded and decoded at the same level according to the default encoding order. Therefore, lower-level blocks at the same level derived from the current block are also encoded and decoded according to the default encoding order. This will be described below. Figures 18A to 18C An exemplary embodiment of the default encoding order is shown in the figure.

[0228] Therefore, when a lower-level block pair is encoded according to the default encoding order, the lower-level block pair is encoded in the forward direction. Conversely, when a lower-level block pair is encoded according to the reverse of the default encoding order, the lower-level block is encoded in the reverse direction.

[0229] For example, when two lower-level blocks are horizontally adjacent and encoded in the positive direction, the encoding order information can be determined by encoding the lower-level block on the left first. On the other hand, when two horizontally adjacent lower-level blocks are encoded in the opposite direction, the encoding order information can be determined by encoding the lower-level block on the right first.

[0230] Similarly, when two lower-level blocks are vertically adjacent and encoded in the positive direction, the encoding order information can be determined by encoding the top lower-level block first. On the other hand, when two lower-level blocks are vertically adjacent and encoded in opposite directions, the encoding order information can be determined by encoding the bottom lower-level block first.

[0231] When the encoding order information only indicates the encoding order of lower-level block pairs, the encoding order information has a data size of 1 bit. Encoding order information with a data size of 1 bit can be defined as an encoding order flag.

[0232] Reference Figure 20 Describes how the encoding order flag works when the current block of a square is divided into four lower-level blocks.

[0233] Reference Figure 21A and Figure 21B Describe how the encoding order of at least three blocks arranged vertically or horizontally is changed according to the encoding order flag.

[0234] The encoding order determiner 1620 can obtain encoding order information from the bitstream. The encoding order information may be located after the partitioning information in the bitstream.

[0235] The encoding order determiner 1620 can internally determine encoding order information based on the environment surrounding the current block. The encoding order information can be determined based on whether neighboring blocks adjacent to the current block have been encoded. For example, the encoding order determiner 1620 can determine, among multiple lower-level blocks, the lower-level block that will be encoded first, having more neighboring blocks.

[0236] The encoding order information can be set to be equal to the encoding order information of the upper-level blocks applied to the current block. For example, when the current block is a prediction unit or a transform unit, the encoding order determiner 1620 can apply the following encoding order information to the current block: this encoding order information has been applied to encoding units that include the current block. As another example, when the current block is an encoding unit, the encoding order determiner 1620 can apply the following encoding order information to the current block: this encoding order information has been applied to encoding units with a depth lower than the current block's depth.

[0237] When there are at least two encoding orders for the current block, the encoding order determiner 1620 can obtain only one encoding order flag from the bit stream and can determine another encoding order flag to interoperate with the encoding order flag obtained from the bit stream.

[0238] The encoding order determiner 1620 can check the encoding order change permission information for the upper-level data unit of the current block. The encoding order change permission information indicates whether changing the encoding order of the blocks included in the upper-level data unit of the current block is allowed. When the encoding order change permission information indicates that changing the encoding order is not allowed, all blocks of the upper-level data unit are decoded according to the default encoding order. When the encoding order change permission information indicates that the encoding order information for the current block has been encoded, the encoding order determiner 1620 can obtain that encoding order information.

[0239] Encoding order change permission information can be included in video parameter sets, sequence parameter sets, picture parameter sets, stripe headers, and maximum coding unit headers. When at least two types of encoding order information exist, the two encoding order change permission messages for at least two types of encoding order information can be stored separately in different headers.

[0240] The encoding order determiner 1620 can determine whether changing the encoding order of the current block is permissible based on the depth or block size of the current block. For example, the encoding order determiner 1620 can obtain encoding order information only if the depth of the current block is included in the depth where the encoding order is permissible to change. As another example, the encoding order determiner 1620 can obtain encoding order information only if the block size of the current block is a block size where the encoding order is permissible to change.

[0241] The encoding order determiner 1620 can determine whether changing the encoding order of the current block is permitted based on the stripe type of the current block. For example, when the current block corresponds to type I, the encoding order determiner 1620 can obtain encoding order information if the depth of the current block is included within the depth range where the encoding depth is allowed to be changed for type I. As another example, when the current block corresponds to type P or type B, the encoding order determiner 1620 can obtain encoding order information if the depth of the current block is included within the depth range where the encoding order is allowed to be changed for type P or type B. Whether changing the encoding order applied to each stripe type based on each depth can be determined based on encoding efficiency.

[0242] The encoding order determiner 1620 can determine whether changing the encoding order of the current block is permitted based on the color components of the current block. For example, when the current block is a luma block, the encoding order determiner 1620 can obtain encoding order information if the depth of the current block is included within the depth range where changing the encoding order for luma blocks is permitted. Similarly, when the current block is a chroma block, the encoding order determiner 1620 can obtain encoding order information if the depth of the current block is included within the depth range where changing the encoding order for chroma blocks is permitted. Whether changing the encoding order applied to each color component based on each depth can be determined based on encoding efficiency.

[0243] In addition to the aforementioned current block depth, block size, stripe type, and color components, the encoding order determiner 1620 can also determine whether the encoding order of the current block is allowed to be changed based on other characteristics of the current block.

[0244] The encoding order change permission information can indicate the depth, block size, stripe type, and color components of the current block whose encoding order is allowed to be changed. When the encoding order change permission information does not include the depth, block size, stripe type, and color components of the current block whose encoding order is allowed to be changed, the default values ​​can be used to determine whether to allow changing the encoding order based on the depth, block size, stripe type, and color components of the current block.

[0245] Block decoder 1630 decodes lower-level blocks according to the decoding order. The decoding method performed by block decoder 1630 may include, as described above... Figure 2 and Figure 5 The described decoding method.

[0246] Figure 17 A video encoding apparatus 1700, according to an exemplary embodiment, is shown that involves dividing a current block and determining the encoding order of the divided lower-level blocks.

[0247] The video encoding device 1700 includes an encoding order determiner 1710, a block divider 1720, and an output unit 1730. (See reference...) Figure 17 The encoding order determiner 1710, block divider 1720, and output unit 1730 are formed as separate elements, but in an exemplary embodiment, the encoding order determiner 1710, block divider 1720, and output unit 1730 may be integrated.

[0248] Reference Figure 17 The encoding order determiner 1710, block divider 1720, and output unit 1730 are considered as components located within a single device, but they do not need to be combined into a single device. Therefore, in an exemplary embodiment, the encoding order determiner 1710, block divider 1720, and output unit 1730 can be distributed.

[0249] In some embodiments, the encoding order determiner 1710, the block divider 1720, and the output unit 1730 may be implemented using a single processor. In some embodiments, the encoding order determiner 1710, the block divider 1720, and the output unit 1730 may be implemented using multiple processors.

[0250] can be Figure 1A The encoding unit determiner 120 performs the functions performed by the encoding order determiner 1710, the block divider 1720, and the output unit 1730.

[0251] The encoding order determiner 1710 determines the encoding order of the lower-level blocks of the current block. The encoding order determiner 1710 can determine the encoding order of the blocks to be encoded by... Figure 16 The encoding order determiner 1620 uses an encoding order determination method to determine the encoding order candidates applicable to the next-level block. The encoding order determiner 1710 can compare the encoding efficiency of the encoding order candidates and then determine the best encoding order among them.

[0252] The encoding order applicable to lower-level blocks can be restricted based on preset conditions. If the encoding order is determined internally based on the environment surrounding the current block, the process of determining the candidate encoding order can be omitted.

[0253] When the information allowing changes to the encoding order of the current block indicates that changing the encoding order of the current block is permitted, the encoding order determiner 1710 may determine the encoding order of the current block to be different from the default encoding order. On the other hand, when the information allowing changes to the encoding order of the current block indicates that changing the encoding order of the current block is not permitted, the encoding order determiner 1710 may determine the encoding order of the current block based on the default encoding order.

[0254] In an exemplary embodiment, the encoding order determiner 1710 can determine the encoding order by analyzing the neighboring blocks of the current block and the texture of the current block. Since the encoding order is determined based on texture similarity, the encoding efficiency of the encoding order candidates is not always calculated. Therefore, the computational efficiency of the encoding process can be improved.

[0255] The block partitioner 1720 compares the coding efficiency of the current block with the coding efficiency of the lower-level blocks according to the coding order to determine whether to partition the current block. When the efficiency of the current block is high, the block partitioner 1720 determines the partition information to indicate that the current block should not be partitioned. On the other hand, when the coding efficiency of the lower-level blocks according to the coding order is high, the block partitioner 1720 determines the partition information to indicate that the current block should be partitioned.

[0256] When the partitioning information is determined to indicate that the current block is partitioned and the encoding order is determined from the encoding order candidates, the encoding order determiner 1710 generates encoding order information for the lower-level block, which indicates the encoding order among the encoding order candidates.

[0257] When at least two partitioning methods can be used for the current block, the block partitioner 1720 can select the optimal partitioning method based on coding efficiency, which is derived from the multiple partitioning methods that can be used for the current block. The block partitioner 1720 can compare the coding efficiency of multiple optimal coding orders for the multiple partitioning methods to select the optimal partitioning method. The block partitioner 1720 can compare the coding efficiency of the lower-level blocks based on the optimal partitioning method and optimal coding order with the coding efficiency of the current block to determine the coding efficiency of the current block. Figure 17 The block divider 1720 can be used by... Figure 16 The partitioning method used by the block partitioner 1610.

[0258] Ideally, the encoding order determiner 1710 and the block partitioner 1720 would search for the optimal partitioning method and encoding order by performing encoding based on all partitioning methods and encoding orders. However, this would significantly increase encoding computation. Therefore, the encoding order determiner 1710 and the block partitioner 1720 could be configured to restrict the partitioning methods and encoding orders allowed for the current block, or they could be configured to first search for the optimal partitioning method and then search for the optimal encoding order based on the optimal partitioning method.

[0259] Output unit 1730 includes the partitioning information of the current block in the bitstream, indicating whether to partition it. When the encoding order of the current block is determined among at least two encoding order candidates, output unit 1730 may also include the encoding order information in the bitstream.

[0260] Figure 17The encoding order determiner 1710 and block divider 1720 can execute Figure 1A The function of the encoding unit determiner 120. Figure 17 The output unit 1730 can execute Figure 1A The function of output unit 130.

[0261] Figures 18A to 18C The default encoding order according to an exemplary embodiment is shown. Figures 18A to 18C The default encoding order is a zigzag encoding order. According to the zigzag encoding order, data units are encoded from left to right, and when all data units in the current row are encoded, the data units in the rows below the current row are encoded from left to right. This zigzag encoding order is called the raster scan order.

[0262] Figure 18A The encoding order is shown according to the zigzag encoding order of the largest coding units included in the current frame 1800. According to the zigzag encoding order, indices 0 to 15 are assigned to the largest coding units. According to the zigzag encoding order, the largest coding units with indices 0 to 3 and included in the first row are encoded first, and the largest coding units with indices 4 to 7 and included in the second row are encoded from left to right. The largest coding units are internally encoded according to the zigzag scan order.

[0263] Figure 18B The encoding order of the largest encoding unit 1810 with index 6 among the multiple largest encoding units included in the current frame 1800 is shown. Indices 0 to 15 are assigned to encoding units with a final depth and divided according to a zigzag encoding order. The zigzag encoding order is applied to data units with the same depth. The last encoding unit with depth n is not encoded until all the lower-level encoding units of the encoding unit with depth n are encoded. For example, the encoding unit with index 15 is not encoded until all the encoding units with indices 5 to 14 are encoded. The encoding units are also internally encoded according to the zigzag encoding order.

[0264] Figure 18C The reference point is shown for coding unit 1824 with index 6, which is included in the largest coding unit 1810. Around the coding unit 1824 with index 6 being encoded, only coding units 1812 with index 0 and 1822 with index 5 are reconstructed. Therefore, only pixel 1850 of coding unit 1812 and pixel 1860 of coding unit 1822 are used as reference pixels for coding unit 1824.

[0265] It can be applied in different directions according to the data unit. Figures 18A to 18CThe zigzag encoding order can be varied. For example, the zigzag encoding order can be changed to allow data cells in the same row to be encoded from right to left. Alternatively, the zigzag encoding order can be changed such that after all data cells in the current row are encoded, data cells in the rows above the current row are encoded. Another possible zigzag encoding order is to encode data cells in the same column from top to bottom, and then after all data cells in the current column are encoded, data cells in the columns to the right of the current column are encoded.

[0266] Figure 19A and Figure 19B The diagram shows the case 1900 where encoding unit 1910 is encoded in the forward direction and the case 1920 where encoding unit 1920 is encoded in the reverse direction. Now, referring to... Figure 19A and Figure 19B Describe the effect of changing the encoding order.

[0267] Based on the intra-frame mode along the upper right direction Figure 19A and Figure 19B The coding units 1910 and 1920 are used for prediction. Figure 19A and Figure 19B The solid line 1930 corresponds to pixels in the original image that are arranged in a straight line and have constant values. Therefore, when the current coding unit is predicted along the direction of the solid line 1930, the prediction accuracy for coding units 1910 and 1920 can be improved.

[0268] When encoding is performed in the positive direction, the coding units to the left, above, and to the upper right of the current block 1910 are reconstructed before the current block coding unit 1910. Therefore, the current coding unit 1910 references the pixel or coding information of the left, above, and upper right coding units. For example, pixel 1916 at the bottom corner of the upper right coding unit is used to predict the current coding unit 1910. Since pixel 1916 is spatially distant from the current coding unit 1910, the prediction accuracy for portion 1914 of the current coding unit 1910 may be low.

[0269] However, when the encoding is performed in the reverse direction, the coding units to the right, above, and to the upper left of the current coding unit 1910 are reconstructed before the current coding unit 1920. Therefore, during intra-frame prediction, pixel 1926 at the left corner of the right-hand coding unit can be used to predict the current coding unit 1920. Since pixel 1926 is adjacent to the current coding unit 1920, the prediction accuracy for the portion 1924 of the current coding unit 1920 can be higher than the prediction accuracy for the portion 1914 of the current coding unit 1910.

[0270] For example, in reference Figure 19A and Figure 19B As described in the embodiments concerning intra-frame prediction, for inter-frame prediction, there are many cases where prediction accuracy can be improved by obtaining coding information from blocks located in the opposite direction. If the current coding unit and the coding unit to its right are coding units about the same object, the motion information of the current coding unit can be similar to the motion information of the coding unit to its right. Therefore, coding efficiency can be improved by deriving the motion information of the current coding unit from the motion information of the coding unit to its right.

[0271] Therefore, the coding efficiency of an image can be improved by comparing the coding efficiency of the current coding unit being encoded in the positive direction with the coding efficiency of the current coding unit being encoded in the negative direction to determine the coding order.

[0272] Figure 20 The tree structure of the maximum coding unit 2050 is shown, wherein the tree structure of the maximum coding unit 2050 is used to describe the maximum coding unit 2050 and the coding order of the coding units included in the maximum coding unit 2050.

[0273] The largest coding unit 2050 is divided into multiple coding units 2056, 2058, 2060, 2062, 2068, 2070, 2072, 2074, 2080, 2082, 2084, and 2086. The largest coding unit 2050 corresponds to the topmost node 2000 of the tree structure. Coding units 2056, 2058, 2060, 2062, 2068, 2070, 2072, 2074, 2080, 2082, 2084, and 2086 correspond to multiple nodes 2006, 2008, 2010, 2012, 2018, 2020, 2022, 2024, 2030, 2032, 2034, and 2036, respectively. The upper coding order markers 2002, 2014, and 2026 in the indicator tree structure correspond to arrows 2052, 2064, and 2076, while the lower coding order markers 2004, 2016, and 2028 correspond to arrows 2054, 2066, and 2078.

[0274] The top encoding order flag indicates the encoding order of two encoding units out of four encoding units of the same depth, where these two units are located at the top. If the top encoding order flag is 0, encoding is performed in the positive direction. Conversely, if the top encoding order flag is 1, encoding is performed in the negative direction.

[0275] Similarly, the lower encoding order flag indicates the encoding order of two encoding units out of four encoding units at the same depth, where these two encoding units are located at the bottom. If the lower encoding order flag is 0, encoding is performed in the positive direction. On the other hand, if the lower encoding order flag is 1, encoding is performed in the negative direction.

[0276] For example, since the upper encoding sequence flag 2014 is 0, the encoding order of encoding units 2068 and 2070 is determined to be in the positive direction from left to right. Conversely, since the lower encoding sequence flag 2016 is 1, the encoding order of encoding units 2072 and 2074 is determined to be in the reverse direction from right to left.

[0277] In an exemplary embodiment, the upper encoding order flag and the lower encoding order flag may be set to have the same value. For example, when the upper encoding order flag 2002 is determined to be 1, the lower encoding order flag 2004 corresponding to the upper encoding order flag 2002 may be determined to be 1. Since the values ​​of the upper encoding order flag and the lower encoding order flag are determined to be 1 bit, the amount of information in the encoding order information is reduced.

[0278] In an exemplary embodiment, the upper coding order representation and lower coding order flag of the current coding unit can be determined by considering at least one of the upper coding order flag and the lower coding order flag applied to coding units having a lower depth than the current coding unit. For example, the upper coding order flag 2026 and the lower coding order flag 2028 applied to coding units 2080, 2082, 2084, and 2086 can be determined based on the lower coding order flag 2016 applied to coding units 2072 and 2074. Therefore, the upper coding order flag 2026 and the lower coding order flag 2028 can be determined to have values ​​equal to the value of the lower coding order flag 2016. Since the values ​​of the upper coding order flag and the lower coding order flag are determined from the upper coding unit of the current coding unit, coding order information is not obtained from the bitstream. Therefore, the amount of information in the coding order information is reduced.

[0279] Figure 21A and Figure 21B A method for determining the coding order when the coding unit is divided into three coding units is shown.

[0280] Figure 21A An exemplary embodiment is shown relating to a method for swapping the coding order based on a coding order flag only when the coding orders of spatially adjacent coding units are adjacent to each other.

[0281] Encoding unit 2100 is divided into three encoding units 2110, 2120, and 2130. When the default encoding order is from left to right, encoding is performed in the order of encoding units 2110, 2120, and 2130. However, the encoding order can be changed according to encoding order flags 2140 and 2150.

[0282] The encoding order flag 2140 indicates the encoding order of encoding units 2110 and 2120. When the encoding order flag 2140 is 0, the encoding order of encoding units 2110 and 2120 is determined to be in the forward direction. Therefore, encoding unit 2110 is encoded before encoding unit 2120. However, when the encoding order flag 2140 is 1, the encoding order of encoding units 2110 and 2120 is determined to be in the reverse direction. Therefore, encoding unit 2120 is encoded before encoding unit 2110.

[0283] Encoding sequence flag 2150 indicates the encoding order of encoding units 2120 and 2130. Encoding sequence flag 2150 is obtained when encoding sequence flag 2140 indicates a positive direction. When encoding sequence flag 2140 indicates a negative direction, the encoding order of encoding units 2120 and 2130 is not adjacent, therefore, encoding sequence flag 2150 is not obtained. When encoding sequence flag 2150 is 0, the encoding order of encoding units 2120 and 2130 is determined to be positive. Therefore, encoding unit 2120 is encoded before encoding unit 2130. However, when encoding sequence flag 2150 is 1, the encoding order of encoding units 2120 and 2130 is determined to be negative, therefore, encoding unit 2130 is encoded before encoding unit 2120.

[0284] according to Figure 21A In this embodiment, the encoding order of the three encoding units includes three possibilities. Therefore, one or two encoding order flags are used to determine the encoding order.

[0285] Figure 21B An exemplary embodiment related to a method for determining the encoding order based on an encoding order flag 2160 indicating the direction in which the encoding order will be applied to three encoding units is shown. The encoding order flag 2160 indicates whether the encoding order is forward or reverse. For example, when the encoding order flag 2160 is 0, the encoding order of encoding units 2110, 2120, and 2130 can be determined to be forward. Therefore, when the encoding order flag 2160 is 0, encoding can be performed in the order of encoding unit 2110, encoding unit 2120, and encoding unit 2130.

[0286] When the encoding order flag 2160 is 1, the encoding order of encoding units 2110, 2120, and 2130 can be determined to be reversed. Therefore, when the encoding order flag 2160 is 1, encoding can be performed in the order of encoding unit 2130, encoding unit 2120, and encoding unit 2110.

[0287] according to Figure 21B In this embodiment, the encoding order of the three encoding units includes two cases. Therefore, an encoding order flag is used to determine the encoding order.

[0288] exist Figure 21A and Figure 21B The method for determining the coding order used in the embodiments can be applied to four or more coding units.

[0289] Figure 22 and Figure 23 The method for predicting the current block is shown, which differs depending on whether the right and left blocks are reconstructed. Figure 22 This shows the prediction of the current block 2220 based on the intra-frame mode. Figure 23 This shows the prediction status of the current block 2300 according to the inter-frame mode. The current blocks 2200 and 2300 are coding units or prediction units being predicted.

[0290] Reference Figure 22 The first embodiment 2220 illustrates reference pixels 2202, 2206, 2208, and 2210 used for intra-frame prediction when the blocks in the upper row and the left row are reconstructed. In the first embodiment 2220, reference pixels 2202 and 2206 in the reconstructed upper block and reference pixel 2208 in the reconstructed left block can be used for intra-frame prediction. Reference pixel 2210 in the lower left block can be used only after the lower left block is reconstructed. In order to use reference pixels 2202, 2206, 2208, and 2210, prediction directions included in the first intra-frame prediction direction group 2225 can be used for intra-frame prediction of the current block 2200.

[0291] The second embodiment 2230 illustrates reference pixels 2202, 2204, 2212, and 2214 used for intra-frame prediction when the blocks in the upper row and the right-hand block are reconstructed. In the second embodiment 2230, reference pixels 2202 and 2204 in the reconstructed upper block and reference pixel 2212 in the reconstructed right-hand block can be used for intra-frame prediction. Reference pixel 2214 in the lower-right block can be used only when the lower-right block is reconstructed. To use reference pixels 2202, 2204, 2212, and 2214, prediction directions included in the second intra-frame prediction direction group 2235 can be used for intra-frame prediction of the current block 2200.

[0292] The third embodiment 2240 illustrates reference pixels 2202, 2208, and 2212 used for intra-frame prediction when the upper block, right block, and left block are reconstructed. In the third embodiment 2240, reference pixel 2202 in the upper block, reference pixel 2208 in the left block, and reference pixel 2212 in the right block can be used for intra-frame prediction. The prediction directions included in the third intra-frame prediction direction group 2245 can be used for intra-frame prediction of the current block 2200.

[0293] According to the first embodiment 2220 and the second embodiment 2230, the prediction accuracy deteriorates when the reference pixel 2210 in the lower left block and the reference pixel 2214 in the lower right block cannot be used. However, in the third embodiment 2240, the used reference pixels 2202, 2208 and 2212 are all adjacent to the current block, so the prediction accuracy is relatively high compared to other embodiments.

[0294] Fourth embodiment 2250 illustrates reference pixels 2202, 2204, and 2206 used for intra-prediction when only blocks in the upper row are reconstructed. In fourth embodiment 2250, only reference pixels 2202, 2204, and 2206 in the reconstructed upper blocks can be used for intra-prediction. Prediction directions included in the fourth intra-prediction direction group 2255 can be used for intra-prediction of the current block 2200.

[0295] Unlike the third embodiment 2240, in the fourth embodiment 2250, only the reference pixel 2202 in the upper block is a pixel adjacent to the current block 2200. Since the reference pixels 2204 and 2206 are spatially distant from the current block 2200, the prediction accuracy deteriorates compared to the first embodiment 2220, the second embodiment 2230, and the third embodiment 2240. Therefore, the intra-frame prediction used in the fourth embodiment 2250 can be a vertical prediction mode using the reference pixel 2202 in the upper block adjacent to the current block 2200, or a directional prediction mode in a direction adjacent to the vertical mode.

[0296] In the zigzag coding order, intra-frame prediction according to the first embodiment 2220 is used. However, when the coding order of two horizontally adjacent blocks is swapped, the right block can be predicted first according to the intra-frame prediction of the fourth embodiment 2250. After the right block is reconstructed, the left block can be reconstructed by predicting according to the intra-frame prediction of the third embodiment 2240.

[0297] In predictions based on planar or DC modes, the reference pixels to be used can be determined differently depending on whether the left and right blocks are reconstructed.

[0298] exist Figure 23In this process, only after the left block of the current block 2300 is reconstructed, the blocks including reference pixels 2302, 2304, 2306, 2308, and 2310 are determined as motion vector candidates in the first candidate list. A motion vector candidate can be selected from the motion vector candidates in the first candidate list, and multiple encoded information (such as motion vectors, reference frame indexes, etc.) required for inter-frame prediction can be obtained from the selected motion vector candidate.

[0299] Once only the right-hand block of the current block 2300 is reconstructed, multiple blocks, including reference pixels 2302, 2310, 2312, 2314, and 2316, are identified as motion vector candidates in the second candidate list. A motion vector candidate can be selected from these multiple candidates in the second candidate list, and multiple encoded information (such as motion vectors, reference frame indices, etc.) required for inter-frame prediction can be obtained from the selected motion vector candidate.

[0300] Once the left and right blocks of the current block 2300 have been fully reconstructed, a valid candidate list can be selected from the first and second candidate lists. Subsequently, motion vector candidates can be determined from the selected candidate list. In an exemplary embodiment, when the left and right blocks of the current block 2300 have been fully reconstructed, a third candidate list different from the first and second candidate lists can be generated. For example, multiple blocks including reference pixels 2302, 2304, 2310, 2312, and 2314 can be included as motion vector candidates in the third candidate list.

[0301] When the left and right blocks of the current block 2300 have not been reconstructed, encoding information cannot be obtained from the left and right blocks. Therefore, a fourth candidate list including multiple blocks as motion vector candidates can be used, wherein the multiple blocks are located in the row above the current block 2300. For example, multiple blocks including reference pixels 2302, 2308, 2310, and 2312 can be included in the fourth candidate list as motion vector candidates.

[0302] In a zigzag coding order, inter-frame prediction based on the first candidate list can be used. However, when the coding order of two horizontally adjacent blocks is swapped, inter-frame prediction for the right block can be performed first based on either the second or fourth candidate list. After the right block is reconstructed, the left block can be reconstructed by inter-frame prediction based on either the second or third candidate list.

[0303] Figure 24 A video decoding method performed by a video decoding device 1600 according to an exemplary embodiment is shown.

[0304] In operation 2410, partitioning information indicating whether to partition the current block is obtained. When at least two partitioning methods are allowed for the shape of the current block, partitioning shape information indicating the partitioning method for the current block can be obtained.

[0305] In operation 2420, when the partitioning information indicates that the current block is to be partitioned, the current block is divided into at least two lower-level blocks. The current block can be partitioned according to the partitioning method indicated by the partitioning shape information.

[0306] In operation 2430, encoding order information indicating the encoding order of multiple lower-level blocks of the current block is obtained. This encoding order information can be obtained from the bitstream or determined internally based on the environment surrounding the current block. The encoding order information can be determined when encoding order change permission information obtained for the upper-level data units of the current block indicates that the encoding order of the current block can be changed.

[0307] In operation 2440, the decoding order of the plurality of lower-level blocks is determined based on the encoding order information.

[0308] In operation 2450, the plurality of lower-level blocks are decoded according to the decoding order.

[0309] Figure 25 This illustrates a video encoding method performed by a video encoding device 1700 according to an exemplary embodiment.

[0310] In operation 2510, the encoding order of multiple lower-level blocks of the current block is determined. The encoding order of the multiple lower-level blocks can be selected from available encoding order candidates, or it can be determined internally based on the environment surrounding the current block. Operation 2510 can be omitted when the encoding order change permission information set for the upper-level data units of the current block indicates that the encoding order of the current block cannot be changed.

[0311] In operation 2520, it is determined whether to partition the current block by comparing its coding efficiency with that of the plurality of lower-level blocks according to their coding order. When at least two partitioning methods can be used for the current block, the optimal partitioning method can be selected. Based on the optimal partitioning method, it is determined whether to partition the current block by comparing its coding efficiency with that of the plurality of lower-level blocks according to their coding order.

[0312] In operation 2530, partitioning information indicating whether to partition the current block is included in the bitstream. When the encoding order of the plurality of lower-level blocks is selected from a plurality of encoding order candidates, encoding order information indicating the encoding order selected from the plurality of encoding order candidates can be included in the bitstream.

[0313] According to Figure 1 to Figure 25The described video coding technique, based on multiple coding units with a tree structure, encodes spatial domain image data in each of the multiple coding units in the tree structure, and performs decoding on each maximum coding unit according to a video decoding technique based on multiple coding units with a tree structure, so that the spatial domain image data is reconstructed, and in doing so, the picture and the video as a sequence of pictures can be reconstructed. The reconstructed video can be reproduced by a playback device, stored in a storage medium, or transmitted via a network.

[0314] According to embodiments of the present disclosure, the program can be written into a computer program and can be implemented in a general-purpose digital computer that executes the program using a non-transitory computer-readable recording medium.

[0315] While preferred embodiments of this disclosure have been described, those skilled in the art will understand that various substitutions, modifications, or alterations can be made to this disclosure without departing from the spirit and scope defined by the claims. That is, the claims will be construed as including all such substitutions, modifications, or alterations to this disclosure. Therefore, the descriptions provided in the specification and drawings should be considered descriptive only and not for limiting purposes.

Claims

1. A video decoding method, comprising: The largest coding unit is divided into multiple coding units, the multiple coding units including a first coding unit and a second coding unit; Obtain first partitioning information, which indicates whether to divide the first coding unit into three lower coding units, the three lower coding units including a first left lower coding unit, a center lower coding unit and a first right lower coding unit; When the first partitioning information indicates that the first coding unit is divided into the three lower-level coding units, the first coding unit is divided into the three lower-level coding units by partitioning one side of the width and height of the first coding unit, wherein the one side is equal to or greater than the other side of the width and height of the first coding unit; The first decoding order of the three lower-level coding units is determined by using first coding order information indicating one of the positive and negative directions; The three lower-level coding units of the first coding unit are decoded according to the first decoding order, wherein the first decoding order is one of the following orders: the order from the first left lower-level coding unit to the center lower-level coding unit to the first right lower-level coding unit, and the order from the first right lower-level coding unit to the center lower-level coding unit to the first left lower-level coding unit. Obtain second partitioning information, which indicates whether to divide the second coding unit into two lower-level coding units, the two lower-level coding units including a second left lower-level coding unit and a second right lower-level coding unit; When the second partitioning information indicates that the second coding unit is divided into the two lower-level coding units, the second coding unit is divided into the two lower-level coding units; The second decoding order of the two lower-level coding units is determined by using second coding order information that indicates one of the positive and negative directions; The two lower-level coding units of the second coding unit are decoded according to the second decoding order, wherein the second decoding order is one of the following: the order from the second left lower-level coding unit to the second right lower-level coding unit, and the order from the second right lower-level coding unit to the second left lower-level coding unit.

2. A video encoding method, comprising: The largest coding unit is divided into multiple coding units, the multiple coding units including a first coding unit and a second coding unit; The first coding unit is divided into three lower coding units by dividing one of the width and height of the first coding unit, wherein the one side is equal to or greater than the other side of the width and height of the first coding unit, and the three lower coding units include a first left lower coding unit, a center lower coding unit, and a first right lower coding unit. Generate first partitioning information, which indicates whether to divide the first coding unit into the three lower-level coding units; The three lower-level coding units of the first coding unit are encoded according to a first coding order, wherein the first coding order is one of the following: the order from the first left lower-level coding unit to the center lower-level coding unit to the first right lower-level coding unit, and the order from the first right lower-level coding unit to the center lower-level coding unit to the first left lower-level coding unit. Generate first encoding order information that indicates the first encoding order of the three lower-level encoding units, used to indicate one of the positive and negative directions; The second coding unit is divided into two lower coding units, including a second left lower coding unit and a second right lower coding unit; A second partitioning information is generated, which indicates whether the second coding unit is divided into the two lower-level coding units. The two lower-level coding units of the second coding unit are encoded according to the second coding order, wherein the second coding order is one of the following: the order from the second left lower-level coding unit to the second right lower-level coding unit, and the order from the second right lower-level coding unit to the second left lower-level coding unit; Second coding order information is generated that indicates the second coding order of the two lower-level coding units, and is used to indicate one of the positive and negative directions.

3. A method for storing a bitstream generated by a video encoding method, the bitstream being generated as follows: The first partitioning information indicates whether to divide the first coding unit into three lower-level coding units; The first encoding order information, which indicates the first encoding order of the three lower-level encoding units, is used to indicate one of the positive and negative directions; The second partitioning information indicates whether to divide the second coding unit into two lower-level coding units; as well as The second encoding order information, which indicates the second encoding order of the two lower-level encoding units, is used to indicate one of the positive and negative directions. in, The video encoding method includes: The largest coding unit is divided into multiple coding units, the multiple coding units including a first coding unit and a second coding unit; The first coding unit is divided into three lower coding units by dividing one of the width and height of the first coding unit, wherein the one side is equal to or greater than the other side of the width and height of the first coding unit, and the three lower coding units include a first left lower coding unit, a center lower coding unit, and a first right lower coding unit. Generate first partitioning information, which indicates whether to divide the first coding unit into the three lower-level coding units; The three lower-level coding units of the first coding unit are encoded according to a first coding order, wherein the first coding order is one of the following: the order from the first left lower-level coding unit to the center lower-level coding unit to the first right lower-level coding unit, and the order from the first right lower-level coding unit to the center lower-level coding unit to the first left lower-level coding unit. Generate first encoding order information that indicates the first encoding order of the three lower-level encoding units, used to indicate one of the positive and negative directions; The second coding unit is divided into two lower coding units, including a second left lower coding unit and a second right lower coding unit; A second partitioning information is generated, which indicates whether the second coding unit is divided into the two lower-level coding units. The two lower-level coding units of the second coding unit are encoded according to the second coding order, wherein the second coding order is one of the following: the order from the second left lower-level coding unit to the second right lower-level coding unit, and the order from the second right lower-level coding unit to the second left lower-level coding unit; Generate second encoding order information indicating a second encoding order of the two lower-level encoding units, used to indicate one of the positive and negative directions; and The bit stream, including first partitioning information, first encoding order information, second partitioning information, and second encoding order information, is stored in a computer-readable storage medium.

Images